What does AIO stand for?

AIO stands for AI Optimisation. It is the infrastructure discipline of making an organisation machine-readable: crawler access, server-side rendering, structured data, published indexes such as llms.txt, page performance and consistent identity across sources.

Do AI crawlers execute JavaScript?

Most do not, or do so selectively and with lower priority. If your main content is inserted into the page by client-side JavaScript, a crawler that fetches the raw HTML receives a near-empty document. The reliable approach is server-side rendering or static generation, so the complete content is in the initial response.

What is llms.txt and do we need one?

llms.txt is a plain-text file at the root of a site giving language models a curated map of the most authoritative pages, in the spirit of robots.txt for crawlers. It is a convention rather than a standard, and support varies. It costs little to publish, makes your own priorities explicit, and is worth generating from live content so it stays accurate.

How do we know if AI crawlers are being blocked?

Check server logs for the named agents rather than relying on robots.txt alone. Blocks most often come from CDN bot-management rules, WAF configurations or rate limiting, none of which appear in robots.txt. If GPTBot, ClaudeBot and PerplexityBot appear in your logs and receive 200 responses, access is working.

Does Core Web Vitals affect AI visibility?

Indirectly but measurably. Crawlers operate under time and resource budgets; slow, heavy pages get crawled less often and less deeply. Performance also affects conventional rankings, and AI systems disproportionately retrieve pages that already rank well, so the effect compounds through two routes.

AI Optimisation (AIO) Services | Machine-Readable Websites | OMAV Technology

AI Optimisation (AIO)

AI Optimisation is the discipline of making an organisation machine-readable across every surface: crawler access policy, server-side rendering, schema.org entity graphs, published indexes such as llms.txt, page performance, and identity consistency across third-party sources. OMAV delivers AIO as engineering work with implemented changes and verification rather than recommendation documents.

AI Optimisation

The infrastructure layer under every other AI visibility service

AI Optimisation is the discipline of making an organisation machine-readable across every surface: crawler access policy, rendering strategy, structured data, published indexes, page performance, and consistency of identity across third-party sources. It is the layer that determines whether the rest of your AI visibility work can have any effect at all.

The most common finding in an AIO audit is unglamorous. A site renders its main content through JavaScript, so a crawler that does not execute scripts receives an almost empty document. Or robots.txt silently blocks the crawlers that feed generative answers. Or the page takes eleven seconds to become useful and gets abandoned mid-fetch. None of these are content problems, and no amount of writing fixes them.

OMAV treats AIO as an engineering discipline because that is what it is. The deliverables are technical specifications, implemented changes and verification, not recommendations documents.

Audit scope

What an AI Optimisation audit examines

Every item below is checked directly against the served response rather than inferred from a tool score.

Server-rendered content completeness

We fetch each template with scripts disabled and compare the text present against the rendered page. Any content that only appears after JavaScript execution is flagged, because most AI crawlers do not execute scripts and those that do deprioritise sites that require it.

Crawler access policy

robots.txt reviewed line by line for the named AI agents: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot and others. Plus firewall, CDN and bot-management rules, which block these agents far more often than robots.txt does.

Structured data graph

Whether schema.org markup exists, whether it validates, whether it describes the organisation as a connected entity graph rather than isolated fragments, and critically whether it matches what the visible page says.

Published indexes

llms.txt and ai.txt presence and quality, XML sitemap accuracy and freshness, RSS availability, and whether any structured feed or API exposes the same information in a parseable form.

Core Web Vitals and fetch cost

Largest Contentful Paint, Interaction to Next Paint and Cumulative Layout Shift, plus raw metrics that matter to crawlers specifically: time to first byte, HTML document weight, and how much of the page is usable from the initial response alone.

Cross-source entity consistency

Comparing how the organisation is described on its own site against directories, professional networks, review platforms, partner sites and press coverage. Contradictions here directly reduce model confidence.

Implementation

What gets built

Rendering correction. Moving content generation from client-side to server-side, or adding server-side rendering where a framework requires it, so the initial HTML response contains the complete page.
Crawler permissions. Explicit named directives for each AI agent in robots.txt, plus allow rules at CDN and WAF level where bot management was silently blocking them.
Entity schema graph. A connected Organization, WebSite, WebPage, Service, Article and FAQPage graph with stable identifiers, so nodes reference each other rather than repeating themselves.
llms.txt and ai.txt. A curated index of authoritative pages with descriptions, plus a stated usage and attribution policy, generated from live content so it cannot go stale.
Performance work. Critical CSS inlining, font subsetting and self-hosting, image format and sizing correction, deferred non-critical scripts, and elimination of layout shift.
Identity reconciliation. Correcting the organisation’s description on third-party sources so every place a model might read about you says the same thing.

This website is itself built to the specification above. Disable JavaScript and reload: the content is unchanged. View source and the schema graph, llms.txt link and complete copy are all present in the initial response.

Reference

The AI crawlers that matter and what they feed

Named AI crawlers and their purpose
User agent	Operator	What it feeds
GPTBot	OpenAI	Model training corpus
OAI-SearchBot	OpenAI	ChatGPT search index
ChatGPT-User	OpenAI	Live browsing during a conversation
ClaudeBot	Anthropic	Model training and retrieval
Claude-User	Anthropic	Live retrieval during a conversation
PerplexityBot	Perplexity	Perplexity answer index
Google-Extended	Google	Gemini grounding and AI Overviews
Applebot-Extended	Apple	Apple Intelligence
CCBot	Common Crawl	Open corpus used by many model builders
Meta-ExternalAgent	Meta	Meta AI training and retrieval

Blocking any of these is a legitimate commercial decision. Blocking them accidentally, through a CDN bot rule nobody reviewed, is the situation AIO audits most often uncover.