"access denied, got HTTP status code 403 (forbidden)" and in-stock GPUs show as not in stock

[julius1504]

Hello,
I tried to set up this bot for some shops in my area (Germany). For some vendors, it says "access denied" (see log).
Also, i put in a link with a 3090 that I know is in stock just for testing, but the bot says it is not. When I run the container, it gives me warnings that it's only using generic scrapers.

Does this bot only work with US vendors? What do I need to do to make it work in my region? Or am I just missing a configuration step?

Thanks guys!

[LOG]

W2021-01-11 21:18:05,484 [root] warning: using generic scraper for url: https://www.computeruniverse.net/de/pny-geforce-rtx3080-xlr8-gaming-epic-x-rgb-m-10-gb-enthusiast-grafikkarte
I2021-01-11 21:18:05,485 [cmptrnvrs_nt_32] scraper initialized for https://www.computeruniverse.net/de/pny-geforce-rtx3080-xlr8-gaming-epic-x-rgb-m-10-gb-enthusiast-grafikkarte
...
I2021-01-11 21:24:10,540 [ltrnt_d_1] not in stock
I2021-01-11 21:24:12,588 [ltrnt_d_2] not in stock
I2021-01-11 21:24:14,555 [ltrnt_d_3] not in stock
I2021-01-11 21:24:17,413 [ltrnt_d_4] not in stock
I2021-01-11 21:24:19,400 [ltrnt_d_5] not in stock
I2021-01-11 21:24:21,141 [cskng_d_1] not in stock
I2021-01-11 21:24:23,381 [cskng_d_2] not in stock
I2021-01-11 21:24:25,687 [cskng_d_3] not in stock
I2021-01-11 21:24:27,841 [cskng_d_4] not in stock
W2021-01-11 21:20:29,411 [cmptrnvrs_nt_1] access denied, got HTTP status code 403 (forbidden)
W2021-01-11 21:20:31,593 [cmptrnvrs_nt_2] access denied, got HTTP status code 403 (forbidden)
W2021-01-11 21:20:33,859 [cmptrnvrs_nt_3] access denied, got HTTP status code 403 (forbidden)

[lonicade]

Might want to try slowing down your refresh rate... some site admins set thresholds, and block when you refresh too much. Also, you could try scrambling/chunking your list of urls in your .yaml, if you're using a combined one (verses by domain), so that you aren't doing all the pulls at one site back-to-back. It might be valuable to have an argument to randomize the scrapes a bit, to make it look less structured/bot-like..