According to access to some sites, the embarrassment of the AI search startup are alleged restrictions aimed at stopping his call on the Internet AI from access to some websites Report with cloudflare. In the report, Cloudflare claims that when the block encounters embarrassment, the startup will hide his creeping identity “in order to circumvent the preferences of the page.”
The report only adds concerns about the embarrassment of vacuuming content without permission, because the company was caught blocking Paywalls and ignoring robots.txt files last year. Aravind Srinivas Aravind Srinivas at that time blamed his activity On third -party indexes used by the website.
Now Cloudflare, one of the world’s largest internet architecture providers, claims that he has received complaints from customers who claimed that bots of embarrassment still have access to their websites even after placing their preferences in the robot file.
To test this, Cloudflare claims that he has created modern domains with similar restrictions in relation to scrapers and embarrassment. It turned out that the startup first tries to access the pages, identifying as the names of his sticks: “Prestlexitybot” or “challenging the user”.
But if the website has restrictions on scratching artificial intelligence, Cloudflare claims that embarrassment will change its user agent – a fragment of information that informs about the website you operate the browser and device, or if the visitor is a bot – to “lick in Google Chrome on MacOS.” Cloudflare says that this “unknown crawler” uses “rotary” IP addresses that The company does not include On the IP address list used by its bots.
In addition, Cloudflare claims that embarrassment changes its autonomous system networks (ASN), the number used to identify IP networks controlled by one operator to bypass the blocks. “This activity was observed in tens of thousands of domains and millions of demands a day,” writes Cloudflare.
In a statement to The VergeThe spokesman for embarrassment, Jesse Dwyer, called the Cloudflare report “an advertising feat”, adding that “there are many misunderstandings in the blog post.” Since then, Cloudflare has been embarrassed as a verified bot and introduced “secretiveness” blocking methods.