November 15, 2026 • By AICompatible Team • 7 min read

Answer Engine Optimization: Your Essential 90-Day Starter Guide

# AEO Fundamentals: Minimum Viable Checklist

In brief: For your first 90 days of Answer Engine Optimization (AEO), prioritize three actions: add a clear, concise answer paragraph to every page, structure H2/H3 headings as questions, and implement Schema.org markup (Article, FAQPage, HowTo). For bot access, simply allow crawlers like GPTBot, ClaudeBot, and Google-Extended in robots.txt; experimental files like llms.txt remain optional at this stage.

1. Answer-First Structure: Summary Paragraph at Page Top

When ChatGPT, Perplexity, or Google AI Overviews search for an answer, they scan the first 200–300 words of a page. Place a 2–4 sentence paragraph at the start of every piece of content that directly answers the user's question:

<p><strong>In brief:</strong> [Core answer] to [question]. 
[Additional context or condition]. [Next step or reference].</p>

Example (B2B SaaS context):

<p><strong>In brief:</strong> API integration typically takes 2–5 business days, 
depending on your existing infrastructure and number of endpoints. 
Standard REST integrations are faster; custom webhook architectures 
require additional testing. Detailed steps are outlined below.</p>

This paragraph should be independent of your meta description; the meta description serves search engine snippets, while the answer paragraph is designed for direct citation by AI models.

2. Question-Format H2/H3 Hierarchy

Answer engines use headings for question-answer matching. Write every H2 or H3 as a natural question:

Correct: "How do I generate an API key?"
Incorrect: "API Key Generation"

The first paragraph under the heading should contain the answer. Subsequent paragraphs can add detail, examples, or caveats. Example structure:

<h2>Which schema type is appropriate for e-commerce product pages?</h2>
<p>E-commerce product pages require <code>Product</code> schema, 
which must include <code>name</code>, <code>image</code>, 
<code>offers</code> (price, stock status), and 
<code>aggregateRating</code> fields.</p>

<h3>What additional schemas add value?</h3>
<p>If the product page includes an FAQ section, add <code>FAQPage</code> schema; 
if user reviews are present, add <code>Review</code> schema.</p>

Four to six question headings per page is sufficient; more than that dilutes content and loses focus.

3. Schema.org Markup: Three Core Types

Answer engines prioritize structured data. In the initial phase, implement these three schema types:

Schema Type	Use Case	Required Fields
`Article`	Blog posts, guides	headline, datePublished, author, image
`FAQPage`	FAQ sections	mainEntity (Question → acceptedAnswer)
`HowTo`	Step-by-step instructions	name, step (name, text)

Add schema in JSON-LD format within a <script type="application/ld+json"> tag in the page <head> or at the end of <body>. Example FAQPage schema:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [{
    "@type": "Question",
    "name": "Which bots can robots.txt block?",
    "acceptedAnswer": {
      "@type": "Answer",
      "text": "robots.txt can block crawlers that respect the courtesy protocol, 
      such as GPTBot, ClaudeBot, and CCBot. However, this is not a security 
      mechanism; non-compliant bots require server- or WAF-level blocking."
    }
  }]
}
</script>

Validate schema errors using Google's Rich Results Test. Broken schema is worse than no schema; answer engines ignore invalid JSON-LD.

4. Bot Access: robots.txt and User-Agent Management

For answer engines to crawl your content, you must allow the relevant crawlers in your robots.txt file. The following User-agents are used for model training or search indexing:

# OpenAI (ChatGPT training and search)
User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

# Anthropic (Claude training)
User-agent: ClaudeBot
Allow: /

# Google (Gemini/Vertex AI training; does NOT affect Search ranking)
User-agent: Google-Extended
Allow: /

# Perplexity (search indexing)
User-agent: PerplexityBot
Allow: /

# Common Crawl (general dataset)
User-agent: CCBot
Allow: /

Important: Google-Extended controls only training/grounding use for Gemini apps and Vertex AI; it does not affect Google Search ranking. Google AI Overviews are managed via standard Googlebot and snippet controls (nosnippet, max-snippet, data-nosnippet).

For user-initiated fetches (ChatGPT-User, Claude-User, Perplexity-User), OpenAI and other providers note that robots.txt rules may not apply uniformly. If you wish to block access from these agents, implement User-agent-based filtering at the server or WAF level; robots.txt alone is insufficient.

5. Unnecessary or Premature Implementations to Skip

In your first 90 days, do not spend time on the following:

llms.txt file: An experimental format proposed in 2024; it is not confirmed that major search or AI providers use it for ranking or attribution. You may add it as an optional experiment, but it is not a priority.
Excessive schema types: Specialized schemas like Product, Event, or LocalBusiness are only necessary for relevant page types. Adding Organization or WebSite schema to every page does not directly contribute to AEO.
Excessively long content: Articles of 3,000+ words provide no advantage for answer engines; 800–1,200 words with clear headings and structured data are more effective.
Custom bot blocks: Blocking crawlers like Bytespider or Amazonbot does not improve AEO performance; only consider this if server load is an issue.
Meta keywords tag: No modern search engine or answer engine uses this tag.

6. First 30-Day Implementation Plan

Proceed in the following order:

Days 1–7: Add answer-first paragraphs to 20% of existing content (highest-traffic pages).
Days 8–14: Convert H2/H3 headings on those same pages to question format.
Days 15–21: Implement Article and FAQPage schema on these pages; validate with Google Rich Results Test.
Days 22–30: Update robots.txt (using the User-agent list above); check for errors in Search Console's 'robots.txt report'.

Once these steps are complete, roll out the same structure to remaining content. Answer engines' crawling and citation process can take 4–8 weeks; do not expect immediate results.

Frequently Asked Questions

Does allowing all AI bots in robots.txt create copyright risk?

robots.txt is a courtesy protocol, not an access security mechanism. If you wish to prevent your content from being used in model training, seek legal counsel; robots.txt alone provides no legal protection. However, blocking GPTBot or ClaudeBot means those providers will skip your content in future training crawls (this does not apply to past crawls).

Will pages without Schema.org markup never appear in answer engines?

No; absence of schema does not completely exclude a page, but it creates a disadvantage compared to competitors with structured data. ChatGPT or Perplexity can still cite pages without schema, but content with schema receives higher confidence scores and citation priority. View schema as a "competitive advantage," not a "requirement."

Where is the robots.txt testing tool in Google Search Console?

The legacy 'robots.txt tester' has been removed. Instead, use the 'robots.txt report' (Settings > robots.txt) and 'URL Inspection' tool in Search Console. The robots.txt report shows your file's validity and accessibility; URL Inspection tests whether a specific page is crawlable by Googlebot.

Is adding an llms.txt file beneficial?

llms.txt is an experimental format proposed in 2024, and it is not confirmed that major search/AI providers use it for ranking or attribution. You may add it as an optional experiment (for example, to describe your site structure or priority content), but at this stage answer-first structure, question headings, and schema markup have far higher priority. Rather than spending time on llms.txt, optimize your existing content according to the checklist above.