DrifterBot
DrifterBot is the fetcher behind Drifter's technical audit. It visits a website when someone who markets that place asks Drifter to check how the site reads to AI systems.
Exact identity
Every DrifterBot request carries this user agent:
User agent
DrifterBot/1.0 (+https://drifter.travel)What we fetch
One audit is a single visit. It requests:
- Your homepage
robots.txtsitemap.xml, plus up to two sitemap locations your robots.txt declares if the standard one is unusablellms.txt
Page performance is measured by Google Lighthouse through Google's PageSpeed service. Those requests come from Google, not from DrifterBot.
Each fetch has a 10 second timeout. Temporary errors are retried once, honoring Retry-After. Firewall and challenge responses are never retried.
When we fetch
- Every audit is customer-initiated. It runs because a person asked Drifter to check this site, or asked us to keep their existing report current.
- Automated re-audits run at most once per URL in any 24 hour window.
- A person re-running checks from their dashboard can trigger one additional audit.
Our crawl policy, stated plainly
- We read robots.txt first. When it sets a Crawl-delay for DrifterBot, or for all crawlers, the rest of the audit's fetches are spaced by that value, up to 10 seconds.
- Customer-initiated audits are treated as exempt from Disallow. This matches the user-requested fetcher class that major AI providers document for their own bots. We log every Disallow we encounter, so the policy runs with eyes open.
- DrifterBot never crawls beyond the files listed above, never executes JavaScript, and never identifies as anything other than DrifterBot.
IP ranges
We do not publish IP ranges yet. Our hosting egress is not stable enough to publish a list that stays true. When that changes, the ranges will appear here.
Contact and opt-out
Questions, or want DrifterBot to stop fetching your site? Write to hello@drifter.travel and we will stop auditing it.