InstantIndexingProAI-readiness audit
Free. One page, eleven checks, a score from 0 to 100: can Googlebot, Bingbot and the crawlers behind ChatGPT, Claude, Perplexity and Gemini read it — or does a robots.txt rule, a firewall, a nosnippet tag or JavaScript-only text hide it?
Your score and every check appear here, with the fixes in plain words.
Why it matters
Blocked crawlers can’t cite you
AI assistants answer from what their crawlers could read. If your robots.txt blocks OAI-SearchBot, PerplexityBot or Claude-SearchBot, those assistants can’t read your pages, so they can’t cite or recommend them.
Search crawlers versus training crawlers
GPTBot, ClaudeBot, Google-Extended and CCBot collect pages for model training. Blocking them is a legitimate choice and doesn’t stop the search crawlers. The check shows both, so you decide on purpose.
Why test the firewall?
robots.txt can allow a crawler while a firewall or CDN bot shield still turns it away. We request the page once as a normal browser and once each named as OAI-SearchBot, PerplexityBot and Claude-SearchBot. If only the crawler requests are refused, something is blocking AI crawlers.
Why does text without JavaScript matter?
Most AI crawlers read the HTML your server sends and do not run JavaScript. If your content only appears after scripts run, they see an almost empty page.
What is llms.txt?
A short plain-text file at the root of your site that points AI assistants to your most important pages. It is a young convention, cheap to add, and some assistants already read it. Signed-in users can generate one automatically.