High Priority
Deploy `/courses.txt` Protocol for AI Ingestion
Establish a machine-readable index of your entire course catalog and critical platform pages specifically for AI learning agents and discovery bots.
Create a text file at `/courses.txt` with a brief overview of your primary course offerings and platform focus.
Include markdown-style links to your most important course landing pages, curriculum outlines, and instructor bios.
Add a 'Curriculum FAQ' section within the file to directly answer common queries about course prerequisites, learning outcomes, and platform features that AI bots might use for content summarization or recommendation.
Configure your Course creators crawler protocols effortlessly.
Join 2,000+ teams scaling with AI.
High Priority
Courseware Selective Indexing
Fine-tune which sections of your course platform and content should be ingested by AI crawlers like OpenAI's GPTBot or Google's Gemini.
Implement `User-agent: GPTBot` and `User-agent: Gemini` directives in your `robots.txt` file.
Use `Allow:` directives to grant access to key course content areas (e.g., `/courses/`, `/lessons/`, `/community/`) and `Disallow:` for administrative or user-specific pages (e.g., `/dashboard/`, `/checkout/`).
Verify your crawler permissions and indexing scope using platform-specific tools or by monitoring server logs for AI bot activity on permitted URLs.
Medium Priority
Semantic HTML for Course Structure Ingestion
Utilize HTML5 semantic elements and ARIA attributes to help LLM crawlers understand the hierarchical structure and importance of your course content.
Wrap individual course lessons or modules within `<article>` tags to signal them as distinct, self-contained content units.
Employ `<section>` tags with descriptive `aria-label` attributes (e.g., `aria-label="Module 1: Introduction to Podcasting"`) to delineate major course segments.
Ensure all tables presenting learning objectives, module breakdowns, or pricing tiers use proper `<thead>`, `<tbody>`, and `<th>` tags for structured data extraction by AI.
High Priority
RAG-Friendly Lesson Snippet Optimization
Structure your course content and supporting materials so they can be easily 'chunked' and retrieved by Retrieval-Augmented Generation (RAG) pipelines for AI-powered Q&A and summarization.
Keep related concepts, examples, and exercises within discrete content blocks (e.g., individual lesson pages or well-defined sections) ideally under 700 words.
Avoid 'floating' context by repeating the primary subject or lesson topic in section summaries and introductions; explicitly state what each chunk is about.
Eliminate ambiguous pronouns (e.g., 'this', 'it', 'they') and replace them with the specific course name, module title, or concept being discussed to ensure AI clarity.
