AI Crawler View: See Your Page the Way ChatGPT Does

More Tools

ChatGPT, Claude and Perplexity do not see your website the way you do. They do not run JavaScript, they do not wait for animations, and they do not scroll. They request your HTML once and read whatever comes back.

That gap is where AI visibility fails. A page can look complete in a browser and arrive almost empty to a crawler. A firewall rule can refuse AI bots months before anyone notices. A robots.txt line written for scrapers can lock out the exact crawlers that feed AI answers.

This tool closes the gap by showing you the real thing. We fetch your page identifying as GPTBot, exactly as OpenAI's crawler does, and show you the title, headings, structured data and readable text that came back. Then we check which AI crawlers your robots.txt actually allows.

More Tools

Let's find your next lever for growth .

Book a Discovery Call.

Audit current organic performance.

Map out exactly where your opportunity is.

"Our work with Growthack was about much more than just SEO. It was about truly understanding our buyers and bringing our brand to life in a way that resonated with them. The results speak for themselves: a significant increase in qualified leads, stronger brand authority and a website that actively supports our growth ambitions."

Joseph Alexander - Official Framer Partner

Victoria Beaven

Head of Marketing, elementsuite

No pitch decks, no pressure. Tell us where you are and we'll share how we can help.

60+

Partnerships over 5 years

98%

Satisfaction Rate.

Let's find your next lever for growth .

Book a Discovery Call.

Audit current organic performance.

Map out exactly where your opportunity is.

Victoria Beaven

Head of Marketing, elementsuite

No pitch decks, no pressure. Tell us where you are and we'll share how we can help.

60+

Partnerships over 5 years

98%

Satisfaction Rate.

Let's find your next lever for growth .

Book a Discovery Call.

Audit current organic performance.

Map out exactly where your opportunity is.

Victoria Beaven

Head of Marketing, elementsuite

No pitch decks, no pressure. Tell us where you are and we'll share how we can help.

60+

Partnerships over 5 years

98%

Satisfaction Rate.

Testimonials

"Repton School has worked with Growthack for the past year and the impact has been outstanding. We have seen a significant increase in organic traffic to our website, much stronger Google rankings across key search terms, and a clear rise in enquiries to the school as a direct result of their work."

The team are knowledgeable, proactive and an absolute pleasure to work with. We would highly recommend Growthack to any organisation looking to strengthen its digital presence.

Repton School

Marketing Team

The team are knowledgeable, proactive and an absolute pleasure to work with. We would highly recommend Growthack to any organisation looking to strengthen its digital presence.

Repton School

Marketing Team

5

/ 5

Based on 30+ Google reviews.

"Working with Growthack over the past 12 months has been transformative for PassMeFast's organic performance. From the outset, the team took the time to understand our business model, customer intent, and technical challenges — especially following a complex CMS migration that had a significant impact on traffic and leads."

Albena Dimitrova

Head of Digital, PassMeFast

"Kevin and his team provide not only expertise in organic search optimisation, but great communication and care for our business. We recently relaunched our website, and their advice and attention to detail played a significant contribution to a seamless migration with immediate positive results in organic visit activity."

Matt Ward

Head of Retail, Manage At Home

"I've found Growthack easy to work with and communication has been consistently good. They explain things clearly and make SEO straightforward for us to understand and implement. The team brings a broad range of expertise, with everyone contributing their own perspective and ideas. We'd happily recommend them to other companies looking for a practical and knowledgeable SEO partner."

Adam Dickens

Marketing Executive, SIS Pitches

Albena Dimitrova

Head of Digital, PassMeFast

Matt Ward

Head of Retail, Manage At Home

Adam Dickens

Marketing Executive, SIS Pitches

Albena Dimitrova

Head of Digital, PassMeFast

Matt Ward

Head of Retail, Manage At Home

Adam Dickens

Marketing Executive, SIS Pitches

Average organic growth within 12 months post-migration

Annual revenue attributed to organic each year

Client retention rate after year one

Brands helped with our organic growth systems

We're in the business of making things actually happen for our clients.

I've personally led 60+ growth engagements. Let me show you what's possible for yours.

Kevin Kapezi

Founder & Director

Book a call

About the AI Crawler View

Most SEO tools render your page like a browser. That is useful for Google, which executes JavaScript, but it is misleading for AI search, which largely does not.

We do the opposite. We make two requests to the same URL: one identifying as GPTBot, one as a normal browser. Comparing the two answers a question no rendered crawl can: is the problem your content, or is it your server?

You get one of these verdicts:

AI crawlers can read this page. Your content is in the HTML. Crawlers can read and cite it.
AI crawlers get only part of this page. Some of what a visitor sees is missing or restricted. The listed fixes close the gap.
This page is effectively invisible to AI. Your content is rendered by JavaScript. Crawlers receive a near-empty shell, so there is nothing for them to quote.
Your server blocks AI crawlers. A browser loads the page normally, but a request identifying as an AI crawler is refused before it ever reaches your content. This is a firewall, CDN or bot-protection rule, not robots.txt, and no amount of on-page work will fix it.
Your site refuses automated requests. Neither agent could reach the page. We deliberately do not claim AI crawlers are being singled out here, because from a single vantage point that is genuinely unknowable.

Alongside the verdict you get the crawler's actual view: title, meta description, H1, every heading it found, structured data types, the framework detected, and a preview of the readable text with a word count. Plus an access check for eight AI crawlers and a look for an llms.txt file.

What it checks

The AI crawler's view of your page

Title, meta description, H1, all H2 and H3 headings, structured data types, canonical, robots meta, detected framework, HTML size, and the readable text with its word count.

AI crawler access, per bot

GPTBot (ChatGPT training), OAI-SearchBot (ChatGPT search), ChatGPT-User (ChatGPT browsing), ClaudeBot (Claude training), Claude-User (Claude browsing), PerplexityBot, Google-Extended (Gemini and AI Overviews) and CCBot (Common Crawl, which feeds many models).

Server-level blocking

Whether your infrastructure refuses AI user agents outright, which robots.txt checkers cannot detect.

llms.txt

Whether you publish the emerging convention for telling AI systems what your site covers.

Each issue comes with its severity and a specific fix, and you can copy the whole crawler view to share with a developer.

Use cases

Before you blame your content

If a page is not being cited, check that it is readable at all. Rewriting copy that a crawler never receives is wasted work.

After a site rebuild

Framer, Next.js, React, Vue, Nuxt and similar frameworks vary enormously in how much they render server-side. A redesign can silently move your content out of the HTML.

After a security change

Bot-protection rules and WAF settings are routinely tightened without anyone considering AI crawlers. This is the fastest way to catch it.

Auditing a new client site

A minute per key page tells you whether an AI search programme is even viable yet, and gives you a concrete technical finding for the pitch.

Settling the "should we block AI crawlers" question

See exactly which bots you currently allow, then decide deliberately rather than by inherited default.

Checking a competitor

The tool works on any public URL, so you can see whether rivals are readable, blocked, or publishing an llms.txt.

How to use it

Paste the full URL of the page you want to check, including https://.
Select Check what AI sees.
Read the verdict at the top. It tells you whether the page is readable, partly readable, invisible, or blocked.
Open What the crawler received and compare it with your live page. Anything missing here is missing from AI answers.
Check AI crawler access for any red dots, which show crawlers your robots.txt disallows.
Work through What to fix, which is ordered by severity, with high-severity items first.
Use Copy the AI view to send the findings to whoever maintains the site.

Check your highest-value pages first: your home page, core service or product pages, and any page you want quoted in an AI answer.

FAQs

Why does this show something different to my browser?

Your browser runs JavaScript, so it builds the finished page. AI crawlers largely do not. We show you the raw HTML response, which is what they actually work from.

Does it run JavaScript?

No, deliberately. Running it would hide the exact problem this tool exists to find.

My word count looks lower than what is on the page. Is that a bug?

Usually not. It is the point. If your visible copy is injected by JavaScript after load, it is not in the HTML, so a crawler never sees it. Compare the text preview against your live page to see what is missing.

What does "your server blocks AI crawlers" actually mean?

A normal browser request succeeds but a request identifying as GPTBot is refused. That is a firewall, CDN or bot-protection rule rejecting the user agent before your content is reached. Robots.txt checkers will not detect this, because the block sits in front of robots.txt.

How do I fix a server-level block?

Allow the AI crawler user agents in your WAF or bot-protection settings. On Cloudflare that is Security then Bots, or a WAF skip rule. Then re-run the check.

Should I be allowing AI crawlers at all?

That is a commercial decision, not a technical one. If you want to appear in AI answers, they need access. If you would rather not be used for model training, you can allow the search and browsing bots while disallowing the training ones. The tool separates them so you can tell which is which.

Does this tell me whether ChatGPT will cite me?

No, and be wary of anything that claims otherwise. This checks whether your page is reachable and readable, which is the prerequisite. Whether a model chooses to cite you depends on authority, relevance and competition. We would rather be accurate about the limit than sell you a number we cannot stand behind.

What is llms.txt?

An emerging convention, similar in spirit to robots.txt, for telling AI systems what your site covers and pointing them at your key pages. Adoption is early and support is not guaranteed, so we report it as a low-severity item rather than a must-have.

Does it check my whole site?

No, one page per check. Run it on the pages that matter most. If you need site-wide analysis, that is what our other tools and our audits are for.

Is it really free?

Yes, and there is no signup. We only fetch the public page, the same as any crawler would.

Want to be the answer, not a missing source?

Being readable is the floor, not the finish line. We help brands get found, read and cited by ChatGPT, Perplexity and Google AI, starting with making sure the machines can read the page at all. Questions about your results: [email protected]

Key Benefits

This tool can help you with analysing:

AI Readiness Score – a 0–100 composite score with a letter-grade verdict
Reach, Read & Recommend performance per AI engine
Which competitors AI names instead of you
AI crawler accessibility and technical readiness

The tool combines live checks – crawler accessibility, robots.txt, structured data – with live queries to ChatGPT, Perplexity and Google AI Overviews. By showing both a free preview score and a full gated breakdown, it gives you a clear, evidence-based view of where AI search is helping or hurting your brand, and a ranked list of fixes to close the gap.

Use Cases

Client Audits

Diagnose exactly why a brand isn’t appearing in ChatGPT or Google AI Overviews, with a ranked fix list to act on.

Competitive Analysis

See which competitors AI names in your category’s buying questions, and where they’re getting cited from.

GEO / AI-SEO Technical Audit

Check whether AI crawlers are blocked in robots.txt and whether structured data (schema) is in place.

Content Gap Analysis

Identify categories or questions AI doesn’t currently associate with your brand.

How to Check AI Visibility

Enter your Website URL and Company Name (both required).
Optionally add your Brand Name (if different from your company name), Industry, and Target Market to sharpen the buying questions tested.
Click “Run free AI check” to map your brand across AI search, analyse ChatGPT and Google AI Overview visibility, and check crawler accessibility.
Review your free AI Readiness Score and the Reach / Read / Recommends matrix, showing whether each AI engine can access, parse and cite your site.
Enter your email at the gate to unlock the full report.
Review the unlocked sections: Executive Summary, Scoring Framework, Strengths & Weaknesses, Recommendations, ChatGPT Visibility, Google AI Overview Readiness, and Full Readiness Breakdown.
Check the Lost-Deal Map to see which buying questions AI answers with a competitor’s name instead of yours.
Work through the ranked fix list, tagged Reach (crawler/access issues) or Read (machine-readability issues).
Click “Download PDF” for a branded report, or “Book a call” to go straight to a consultation.

Additional Notes

The tool checks visibility across ChatGPT, Perplexity and Google AI Overviews, and cross-references crawler accessibility against a wider set of AI bots (GPTBot, ClaudeBot, Google-Extended and others).
The free stage never reveals the “Recommends you” result for ChatGPT or Google AI – only the Perplexity result is shown pre-gate, to keep the full report as the incentive to unlock.
Some figures are estimated where an engine has limited data for your specific queries; these are labelled as estimated rather than observed.
The scan runs against a backend service to query live AI engines, so your domain and query results pass through Growthack’s backend during the scan.

Tool FAQs

For additional questions or support, please contact [email protected]

Which AI engines does it check?

ChatGPT, Perplexity, and Google AI Overviews. Crawler accessibility is also checked against a wider set of AI bots, including GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot and Google-Extended.

What’s the difference between the free and full report?

The free report shows your overall score, grade, and whether each engine can reach and read your site, plus your Perplexity result. The full report (unlocked with an email) adds live ChatGPT and Google AI results, your complete score breakdown, competitor names, source citations, and a ranked fix list.

What is the “Lost-Deal Map”?

It’s a set of real buying-intent questions in your category, tested across AI engines, showing whether your brand is named in the answer or a competitor is recommended instead.

Why are some results locked behind an email?

The full breakdown and live engine results are gated to generate qualified leads – this is a lead-generation tool as much as a diagnostic one.

How is the AI Readiness Score calculated?

It’s a weighted composite across categories including ChatGPT Visibility, Google AI Overview Visibility, AI Accessibility, Technical Readiness, and AI-Cited Authority.

Is my data saved or shared?

The scan runs against a backend service to query live AI engines, so – unlike a purely client-side tool – your domain and query results pass through Growthack’s backend during the scan.

FAQs

AI Search

Migrations

General

Beyond SEO®

Pricing

Process

How do you actually measure AI visibility?

We start by defining a representative set of commercially relevant prompts and buyer queries around your products, services and market. These become the benchmark for ongoing AI visibility tracking, reported in a live dashboard alongside your traditional organic performance instead of a one-off audit.

The dashboard tracks AI Overview Presence (how often Google's AI Overview cites you across that keyword set), AI Brand Mentions (how often ChatGPT names you directly), and AI Search Opportunity (keywords you already rank for but aren't yet cited on –– the fastest-moving target).

Underneath that sits your actual citation footprint: which of your pages AI systems are pulling from, broken down by content type, with real example answers showing exactly what was asked and what got cited.

Every keyword is checked head-to-head against named competitors, so it's clear not just whether you're showing up, but whether a rival is showing up instead. Because generated answers can change considerably over time, we focus on trends across a consistent query set rather than individual responses.

You can see a live example, with sample data, at reports.growthack.io/demo/

What does an AI Search Visibility engagement actually include?

We begin by establishing a baseline of your existing AI visibility and identifying the topics, prompts and competitors that matter commercially.

From there, we assess and audit the signals influencing that visibility across technical architecture, entities, structured data, content and third-party authority. Recommendations are prioritised according to the gaps identified rather than applying a generic GEO checklist.

For ongoing engagements, we then monitor the same query set over time, compare your visibility against competitors and report AI performance alongside organic search and commercial KPIs.

This creates a repeatable process:

Benchmark → diagnose → prioritise → implement → monitor → improve.

Can I check my own AI visibility before talking to you?

Yes. Our free AI Visibility Checker provides an initial benchmark of how your brand appears within AI-powered search experiences.

You can also use our AI Overviews Share of Voice tool to assess visibility within Google's AI-generated results and compare your presence against competitors.

These tools provide a useful starting point. Client reporting goes further by tracking a defined set of commercially relevant queries consistently over time.

Is AI search reducing organic traffic, and how do you account for that?

AI-generated answers can reduce clicks for some searches because users increasingly receive information without visiting a website. That makes click volume alone an increasingly incomplete measure of organic performance.

Our reporting therefore considers search visibility, AI visibility, qualified traffic and commercial outcomes together. The objective isn't simply to maximise clicks. It's to understand whether your brand is being discovered by the right audiences and whether organic and AI search are contributing to demand, conversions and revenue.

This also means we don't interpret declining informational clicks in isolation if visibility and commercial performance tell a different story.

Do you guarantee AI visibility or citations?

No. No credible agency can guarantee rankings, AI citations or inclusion within a particular generated response.

AI results are dynamic and ultimately controlled by the platforms producing them.

What we do guarantee is clear strategy, honest advice and execution based on evidence. We establish a measurable baseline, identify the areas where your visibility can realistically be improved and monitor performance over time.

What makes a brand more likely to appear in AI search results?

There isn't a single GEO ranking factor.

AI visibility is influenced by a combination of signals including technically accessible content, clear brand and entity information, topical relevance, structured information and credible third-party references.

We therefore look for gaps across your website and wider digital footprint rather than simply optimising individual pages for ChatGPT. Our Technical Architecture and Authority Building systems address two important parts of that wider picture.

Which AI platforms do you track?

We monitor visibility across major AI-powered discovery platforms including ChatGPT, Google AI Overviews, Gemini and Perplexity.

Rather than treating every platform equally, reporting focuses on the platforms and query types most relevant to your audience. The monitoring set can also evolve as AI search behaviour changes and new platforms gain meaningful adoption.

How is AI Search Visibility different from traditional SEO?

Traditional SEO primarily measures how webpages perform within search engines through metrics such as rankings, impressions, clicks and organic conversions.

AI Search Visibility adds another layer: whether your brand appears within generated answers, how frequently it appears compared with competitors and which sources AI systems use when forming those answers.

The disciplines overlap heavily. Strong technical SEO, useful content and established brand authority remain important foundations for AI visibility. This is why we treat AI search as part of our Beyond SEO® approach rather than as a replacement for SEO.

What is AI Search Visibility (or GEO)?

AI Search Visibility measures how visible a brand is within AI-powered search experiences such as ChatGPT, Google AI Overviews, Gemini and Perplexity.

Generative Engine Optimisation (GEO) describes the work involved in improving that visibility. Rather than focusing solely on where an individual webpage ranks, we look at whether AI systems understand, mention, cite or recommend your brand for commercially relevant topics and buyer queries.

FAQs

AI Search

Migrations

General

Beyond SEO®

Pricing

Process

How do you actually measure AI visibility?

You can see a live example, with sample data, at reports.growthack.io/demo/

What does an AI Search Visibility engagement actually include?

We begin by establishing a baseline of your existing AI visibility and identifying the topics, prompts and competitors that matter commercially.

For ongoing engagements, we then monitor the same query set over time, compare your visibility against competitors and report AI performance alongside organic search and commercial KPIs.

This creates a repeatable process:

Benchmark → diagnose → prioritise → implement → monitor → improve.

Can I check my own AI visibility before talking to you?

Yes. Our free AI Visibility Checker provides an initial benchmark of how your brand appears within AI-powered search experiences.

You can also use our AI Overviews Share of Voice tool to assess visibility within Google's AI-generated results and compare your presence against competitors.

These tools provide a useful starting point. Client reporting goes further by tracking a defined set of commercially relevant queries consistently over time.

Is AI search reducing organic traffic, and how do you account for that?

This also means we don't interpret declining informational clicks in isolation if visibility and commercial performance tell a different story.

Do you guarantee AI visibility or citations?

No. No credible agency can guarantee rankings, AI citations or inclusion within a particular generated response.

AI results are dynamic and ultimately controlled by the platforms producing them.

What makes a brand more likely to appear in AI search results?

There isn't a single GEO ranking factor.

Which AI platforms do you track?

We monitor visibility across major AI-powered discovery platforms including ChatGPT, Google AI Overviews, Gemini and Perplexity.

How is AI Search Visibility different from traditional SEO?

Traditional SEO primarily measures how webpages perform within search engines through metrics such as rankings, impressions, clicks and organic conversions.

What is AI Search Visibility (or GEO)?

AI Search Visibility measures how visible a brand is within AI-powered search experiences such as ChatGPT, Google AI Overviews, Gemini and Perplexity.

FAQs

AI Search

Migrations

General

Beyond SEO®

Pricing

Process

How do you actually measure AI visibility?

You can see a live example, with sample data, at reports.growthack.io/demo/

What does an AI Search Visibility engagement actually include?

We begin by establishing a baseline of your existing AI visibility and identifying the topics, prompts and competitors that matter commercially.

For ongoing engagements, we then monitor the same query set over time, compare your visibility against competitors and report AI performance alongside organic search and commercial KPIs.

This creates a repeatable process:

Benchmark → diagnose → prioritise → implement → monitor → improve.

Can I check my own AI visibility before talking to you?

Yes. Our free AI Visibility Checker provides an initial benchmark of how your brand appears within AI-powered search experiences.

You can also use our AI Overviews Share of Voice tool to assess visibility within Google's AI-generated results and compare your presence against competitors.

These tools provide a useful starting point. Client reporting goes further by tracking a defined set of commercially relevant queries consistently over time.

Is AI search reducing organic traffic, and how do you account for that?

This also means we don't interpret declining informational clicks in isolation if visibility and commercial performance tell a different story.

Do you guarantee AI visibility or citations?

No. No credible agency can guarantee rankings, AI citations or inclusion within a particular generated response.

AI results are dynamic and ultimately controlled by the platforms producing them.

What makes a brand more likely to appear in AI search results?

There isn't a single GEO ranking factor.

Which AI platforms do you track?

We monitor visibility across major AI-powered discovery platforms including ChatGPT, Google AI Overviews, Gemini and Perplexity.

How is AI Search Visibility different from traditional SEO?

Traditional SEO primarily measures how webpages perform within search engines through metrics such as rankings, impressions, clicks and organic conversions.

What is AI Search Visibility (or GEO)?

AI Search Visibility measures how visible a brand is within AI-powered search experiences such as ChatGPT, Google AI Overviews, Gemini and Perplexity.