High Priority
Deploy /recruiter.txt Protocol
Establish a machine-readable summary of your entire agency website hierarchy, specifically for AI agents focused on talent acquisition and market data.
Create a text file at /recruiter.txt with a brief introduction of your recruitment agency's specializations (e.g., IT staffing, executive search).
Include markdown-style links to your most important service pages, candidate portals, and employer branding content.
Add a 'Candidate FAQs' and 'Client FAQs' section to address common queries from AI models acting as job seekers or hiring managers.
Configure your Recruitment agencies crawler protocols effortlessly.
Join 2,000+ teams scaling with AI.
High Priority
LLM Selective Indexing for Talent Pools
Fine-tune which sections of your agency's website should be ingested by AI crawlers to identify relevant candidate profiles and industry insights.
User-agent: * (or specific bot like GPTBot, ClaudeBot) Allow: /jobs/ Allow: /talent-glossary/ Allow: /industry-reports/ Disallow: /internal-admin/
Verify your crawler permissions using tools that simulate AI bot access to your site.
Monitor crawl frequency in your server logs to ensure AI bots are accessing your core talent and market data pages, not just boilerplate.
Medium Priority
Semantic Job Posting & Candidate Profile Structure
Utilize semantic HTML5 elements to help LLM scrapers understand the structure and key attributes of your job listings and candidate profiles.
Wrap individual job descriptions in `<article>` tags to denote distinct opportunities.
Use `<section>` with descriptive `aria-label` attributes for key job components like 'Responsibilities', 'Qualifications', and 'Benefits'.
Ensure all skill matrices or salary data tables use proper `<thead>` and `<tbody>` tags for structured data extraction by AI.
High Priority
RAG-Friendly Content for Candidate Matching
Structure your content (job descriptions, candidate personas, market analyses) so it can be easily 'chunked' and utilized by Retrieval-Augmented Generation (RAG) pipelines for advanced candidate matching.
Keep related job requirements or candidate skill sets within distinct content blocks (e.g., 500 words) for easier RAG processing.
Avoid ambiguous phrasing; explicitly state the primary job title or required skill in summaries of each section.
Eliminate vague pronouns ('it', 'they') and replace them with specific job titles, company names, or skill sets to ensure clarity for AI.
