Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

AnimePahe started providing fake kwik links in the api response #10

Open
newaccforhc opened this issue Nov 20, 2020 · 11 comments
Open

AnimePahe started providing fake kwik links in the api response #10

newaccforhc opened this issue Nov 20, 2020 · 11 comments
Labels
help wanted Extra attention is needed

Comments

@newaccforhc
Copy link


name: "馃悶 Bug report"
about: Report a bug
title: "[Bug] AnimePahe started providing fake kwik links in the api response"
labels: "bug"


Describe the bug

AnimePahe started providing fake kwik links in the api response (those links go to 404). So, downloads from Animepahe aren't working. Now you need to go through adfly link before getting the real kwik link.

Command You Entered

anigrab -u https://animepahe.com/anime/1e1d8732-a053-24d1-72cb-c8c6a895e3cb

Expected behavior

Should give direct download links.

Actual behavior

Threw an error.

Other details

Maybe some adfly bypassing method can help to resolve this.

On a positive note, they removed captcha to open kwik links now. So, cheers!

@ngomile ngomile added the help wanted Extra attention is needed label Nov 20, 2020
@ngomile
Copy link
Owner

ngomile commented Nov 20, 2020

I'm not sure how to handle this one but it might be that some header is required or something. I've checked but I honestly don't know, I believe the extractor is still fine but as you've stated, it's being given a fake link.

@newaccforhc
Copy link
Author

newaccforhc commented Nov 20, 2020

@ngomile
Oh.. No.. it has nothing to do with headers.. It's just you get a totally different link if you go through adfly or shst link..

Maybe doing something to bypass adfly or shst links would work..
There are many packages/modules to do that on github but, most of them are not maintained now.. So, idk if they still work or not.. But, you can check.

@uali6981
Copy link

uali6981 commented Dec 1, 2020

I got this error on "animePahe"

Bypassing captcha please wait... (node:6004) UnhandledPromiseRejectionWarning: TypeError: response.body.match is not a function or its return value is not iterable at module.exports.extract (C:\Users\------\AppData\Roaming\npm\node_modules\anigrab\src\extractors\kwik.js:51:44) (node:6004) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:6004) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

is this issue on my system or with code?

@newaccforhc
Copy link
Author

@ngomile looks like you can refer here:
KevCui/animepahe-dl#8

@uali6981
Copy link

uali6981 commented Dec 3, 2020

@newaccforhc
my bad
that tool in m3u8. So I did not get enough downloading speed on that software

@newaccforhc
Copy link
Author

Oh no.. I'm not talking about m3u8.
Sorry for not mentioning before,
What I wanted to say is that, you can get the real kwik links by referring to that tool.
That tool is using stream links which have /e/ in them. If you replace it with /f/ then you get the kwik download links.

@ngomile
Copy link
Owner

ngomile commented Dec 6, 2020

So I was doing a little experimenting trying to get this working again like it was, I haven't looked deeply into the link @newaccforhc provided. I'll do that soon though seeing as how what I came up with is inefficient and uses way too much data and CPU. Essentially, I tried to use puppeteer-core to get the link from adfly. Starts too many Chrome processes and is too slow. For the curious souls the code looked like this:

async function getRealKwikLink(url) {
    const BLOCKED_RESOURCES_REG = /ads|inpagepush|css|facebook|analytics|cloud|png|hcaptcha|gif/;
    let kwikURL = '';
    let browser;

    try {
        browser = await puppeteer.launch({
            executablePath: browserExecutablePath,
        });
        const page = await browser.newPage();
        page.setRequestInterception(true);
        page.on('request', request => {
            if (BLOCKED_RESOURCES_REG.test(request.url())) {
                request.abort();
            } else {
                request.continue();
            }
        });

        await page.goto(url, { waitUntil: 'networkidle2' });
        await page.waitForSelector('#skip_bu2tton', { visible: true });
        kwikURL = await page.evaluate(
            '(() => document.querySelector("#skip_bu2tton").getAttribute("href"))()'
        );
    } catch (error) {
        console.error(error);
    } finally {
        await browser.close();
    }

    return kwikURL;
}

But it actually worked as shown below after entering this command anigrab beastars -u -s animepahe:

Capture

Just way too slow, this will be sorted out eventually... maybe.

@IshigamiYuu
Copy link

Screenshot (14)
help it keeps going like this

@ngomile
Copy link
Owner

ngomile commented Dec 9, 2020

@IshigamiYuu animepahe isn't working at the moment.

@justfoolingaround
Copy link

The "kwik" links aren't fake. Kwik has a "DRM" system (really not a DRM but since you're having issues with it, we're going to act like it is one).

I'm not familiar with Node so I'll just drop you the method to get valid stream url(s) from AnimePahe.

First, make an API call to get the "kwik" URL. Then, make a GET request to that URL with "https://animepahe.com/" as the referer (referer in headers). This now allows you to access the site for further extraction. Now, you need to extract the m3u8 URL. There should be some tools in Node that allows JS evaluation but for a faster approach, use regex, here's the algorithm to extract "kwik" m3u8.

To stream/download the m3u8 is also quite troublesome due to the above mentioned "DRM" system, hence, you need to use the "kwik" url (not https://kwik.cx but the url from which you extracted the m3u8) as the referer in the download header.

Then, you should be able to stream/download from AnimePahe.

This is also the reason why web browsers are failing to stream even from the official AnimePahe site.

Good luck!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

6 participants
@ngomile @newaccforhc @uali6981 @IshigamiYuu @justfoolingaround and others