Reported that more than 99.9% of users of open source search engines were spambots



With the major search engines swept by SEO spam, ' searchmysite.net ' is an open source search engine for searching personal and independent websites. The official searchmysite.net blog reports that search engine optimization (SEO) spam bots account for 100% of all searchmysite.net users.

Almost all searches on my independent search engine are now from SEO spam bots
https://blog.searchmysite.net/posts/almost-all-searches-on-my-independent-search-engine-are-now-from-seo-spam-bots/

searchmysite.net is a search engine launched around 2020. On major search engines, SEO spam pushes your personal site out of the top search, making it very difficult to reach. So, searchmysite.net seems to have adopted various methods, such as introducing community-driven curation and moderation, and significantly lowering the search rank of pages containing advertisements to reduce incentives for SEO spam. ..

But the problem is spambots. This is a spambot that allows search engines to access specific sites and raise their search rankings, accessing search engines as programmed.

Below is a summary of the number of users of searchmysite.net during the 11 days from May 1st to May 11th, 2022. 'Real Users' is the number of human users, 'Spam bots' is the number of spambot users, and '% age spam' is the spambot user rate. The spambot rate has already exceeded 99.9% and finally reached 100% on May 11th.



According to searchmysite.net, the number of users has increased dramatically since mid-April 2022. Although searchmysite.net has introduced theoriginal analytics system , this method only shows human users, and abnormal activity could not be recognized without checking the server log.

According to searchmysite.net, most of the few human users visit searchmysite.net from links such as the social news site Hacker News, with little traffic from other search engines. In other words, the sudden increase in users is likely to be a bot.



Looking at the server log, it seems that the IP address, user agent, and search query were all different, and at first I thought that another search engine was scraping the results or was under DDoS attack. that's right. However, further investigation revealed that most search queries followed a certain pattern, which was the work of spambots for black hat SEO.

'I don't agree with the conspiracy theory, but I have to think that the'Internet of Death'theory has some truth. What is the'Internet of Death'theory? The Internet is now. The idea is that it's empty and there are no real people, the bot talks to the bot to generate content and clicks, everything is shared by the bot, and the bot gains a share of the ever-growing advertising revenue. The idea is, but the reality hasn't reached that point yet. Whether or not others know your existence, there are still real people doing great things on the internet. I know, but I'm really worried that my project to connect these people might have had unintended negative consequences. '

in Web Service, Posted by log1i_yk