ConnuBot
ConnuBot is the crawler Connu uses to check websites: how easily search engines and AI assistants can find, read and trust a site. If you saw it in your logs, this page explains what it did.
When it visits
- Someone checked your site with our free website score at connu.ai/score.
- You, or a business you work with, asked Connu to review the site.
- Your site is part of a study of public business websites. We publish only totals and never name a site in a study.
What it does
- It identifies itself as ConnuBot in the user agent.
- It reads your robots.txt first and follows it. A page robots.txt disallows for ConnuBot is not fetched.
- It stays on your site: your domain and its www version.
- For a free score or a study it reads up to 20 pages; for a client review, up to a few hundred.
- It sends at most two requests at a time, with a pause between them. If your server answers 429 or with a server error, it waits and retries at most twice, honouring Retry-After.
- It reads the HTML your server sends and doesn't run JavaScript, submit forms or log in.
- Speed is measured with Google PageSpeed Insights, so that visit comes from Google's Lighthouse, not from ConnuBot.
How to block it
Add these lines to your robots.txt. ConnuBot reads the file at the start of every visit, so the change applies to the next one.
User-agent: ConnuBot Disallow: /
Questions
Write to info@connu.ai with your domain, and we'll tell you what was checked and remove your site from any study on request.