Adorama failing to connect with a 403 error

[TK-SpartanGolf6]

I'm scraping eight Adorama RTX 3080's and they're all failing to scrape with the same error below:

Caught except during request: got response with status code 403 for (Adorama link)

I believe Adorama has great scalping/botting protection as when I try to manually go to Adorama, it asks me to do a ReCaptcha. I am suspecting that Adorama is a lost cause, but does anyone know how to get around this or if it will be patched in later updates to scrape Adorama?

[MRizkBV]

I think the script would need to support rotating user agents to avoid captcha? Not something I can do myself but hopefully someone else can.

[EricJMarti]

Challenge accepted!

Adorama does have some serious protections in place, so hopefully these changes will help with that: a7ccd4a

I also added first-class support for Adorama: 6504842

These changes are building and should be available on Docker Hub in about 20 minutes.

@MRizkBV I think rotating user agents would cause more harm since some detection mechanisms validate if the user agent makes sense. See: https://piprogramming.org/articles/How-to-make-Selenium-undetectable-and-stealth--7-Ways-to-hide-your-Bot-Automation-from-Detection-0000000017.html

CAPTCHA is a bit more difficult but not impossible to defeat. That said, for Adorama, I have yet to encounter a CAPTCHA using this bot. However, I did encounter one when I searched for "rtx" in Chrome on my Mac.

Edit: Available now on Docker Hub!