Best Buy Connection Closed

[tn2389]

I've recently been receiving a connection aborted request from Best Buy. It's likely that Best Buy is blocking the scrape requests even though I have a fairly lenient timeout. Is there a way to work around this to get Best Buy active again in my rotation? Example error:

[bstby_1] caught exception during request: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

[MRizkBV]

I found the only fix to this is to go to src/scraper/BestBuy.py and edit the file. At the very bottom, change 'requests' to 'selenium'. I hope others have a way to make it work on requests again as selenium is slow and a resource hog.

Of course that means you will have to build your own Docker image till an update is pushed to fix it.

[EricJMarti]

I found the only fix to this is to go to src/scraper/BestBuy.py and edit the file. At the very bottom, change 'requests' to 'selenium'. I hope others have a way to make it work on requests again as selenium is slow and a resource hog.

Of course that means you will have to build your own Docker image till an update is pushed to fix it.

This change was committed in: cbd1b17

Best Buy must have changed their site since the requests-based approach was developed.