LLM.txt & AI Crawler Setup Guide for Outbound sales teams

Create a text file at /sales-intelligence.txt with a brief overview of your outbound sales intelligence stack and data sources.

Include markdown-style links to your most critical data schemas, API documentation for enrichment tools, and CRM sync protocols.

Add a 'Data Dictionary' section to clarify key fields (e.g., ICP attributes, trigger events, technographics) for AI consumption.

User-agent: SalesIntelBot Allow: /accounts/ Allow: /contacts/ Disallow: /internal-crm-notes/

Verify your crawler permissions using the AI bot tester provided by your intelligence platform (e.g., ZoomInfo, Apollo.io's bot simulator).

Monitor data refresh rates and API call logs to ensure AI bots are accessing and processing the correct prospect and company intelligence.

Wrap your primary prospect data points (e.g., job title, company, industry) in JSON-LD structured data to signal their importance to AI scrapers.

Use `schema.org/JobPosting` or `schema.org/Organization` with descriptive properties for firmographic and technographic data extraction.

Ensure all prospect contact information tables use proper `<thead>` and `<tbody>` tags for structured data extraction by intelligence tools.

Keep core prospect attributes (e.g., firmographics, technographics, pain points) within logically grouped data clusters (e.g., < 500 tokens per prospect profile).

Avoid ambiguous data references; ensure each data point is clearly linked to a specific company or contact within your CRM or intelligence platform.

Eliminate vague descriptors and replace them with specific, quantifiable data (e.g., 'Uses Salesforce' instead of 'Uses CRM').