ArnorBot
The web crawler of Dealright. Updated 2026-10-14.
What it is
ArnorBot reads the public pages of business websites. Dealright uses the
pages to keep records about US businesses correct: the business name, address
and the contact details that the business publishes itself.
How to identify it
- User agent:
ArnorBot/0.1 (+https://bot.dealright.ai)
- Some requests still send the earlier form
ArnorBot/0.1 (+https://dealright.ai/bot). It is the same crawler.
- robots.txt token:
ArnorBot
- It sends requests only from the addresses in
ips.json.
How it behaves
- It obeys robots.txt (RFC 9309) and
Crawl-delay.
- It sends one request at a time to a host, with at least one second between requests.
- It slows down on 429 and 503 responses and obeys
Retry-After.
- It reads only public pages. It does not log in, submit forms or solve CAPTCHAs.
- If your site refuses ArnorBot (for example with a 403), it stops. It does not
try again with a different identity or address.
How to block it
Add this to your robots.txt:
User-agent: ArnorBot
Disallow: /
To slow it down instead:
User-agent: ArnorBot
Crawl-delay: 10
Contact and opt-out
To ask us to stop crawling a site, to delete data from your site, or to
report a problem, send an email to
bot@dealright.ai. Include the domain.