[fpc-devel] wiki slow

[Jonas Maebe]

On 23/02/2025 14:22, Jonas Maebe via fpc-devel wrote:
> On 23/02/2025 13:56, wkitty42--- via fpc-devel wrote:
>> FWIW: instead of rejecting, i would drop... that way their networking 
>> stack has to wait on timeouts and adds a delay load to their stuff... 
>> dropping also makes you "invisible" to them... with your rejects, your 
>> (icmp) responses tell them the system still exists, so they keep 
>> hammering...
> 
> I know, but this rejecting is not causing issues for now, and I prefer 
> to behave as a good net citizen as long as possible.

Maybe I should reconsider for Alibaba Cloud in particular: they actually 
posted a howto on building a web scraper using their cloud 
infrastructure in December 2024: 
https://www.alibabacloud.com/blog/building-a-custom-chatbot-with-web-scraping-and-alibaba-cloud-model-studio_601863

You just pass it the address of a website, and they scrape it for you. 
Since the scraping happens using faked browser agents and without any 
regard for robots.txt, it's clearly abusive.

I would recommend anyone administrating a web server to permanently 
blacklist the following ranges:

47.235.0.0 - 47.246.255.255
47.250.0.0 - 47.254.255.255
47.74.0.0 - 47.87.255.255 (*)
43.135.128.0 - 43.135.191.255
43.153.0.0 - 43.153.127.255
47.88.0.0 - 47.91.255.255


Jonas

(*) this is the worst offender for us, currently at 1.5 million rejected 
packets totaling 91MB