omgilibot
by Webz.io
Other AI bots · AI crawler
Collects web data that Webz.io sells as datasets, including to AI companies.
Data-extraction and dataset crawlers whose content often ends up in AI products.
Control it with robots.txt
Well-behaved crawlers read robots.txt before crawling. To block omgilibot on your whole site:
robots.txt
User-agent: omgilibot Disallow: /
Blocking AI search and answer crawlers can also remove your pages from AI answers and citations. Weigh traffic from AI assistants before blocking them.