For AI Agents
This page documents gor.bio for crawlers, LLM agents, research agents, and automated software. Automated access is welcome; gor.bio exists partly to serve it. Please read the sections below and keep request behavior reasonable.
Canonical URL structure
The site uses stable, lowercase, canonical URLs:
/— home/about— about this wiki/wiki/article-slug— articles/categoriesand/categories/category-slug— category index and category pages/tagsand/tags/tag-slug— tag index and tag pages/search?q=…— search (dynamic, rate-limited, not canonical content)/for-ai-agents— this page/legaland/legal/…— legal pages/rss.xml— RSS feed/sitemap.xml— XML sitemap/llms.txt— LLM-oriented site summary (llms.txt standard)/wiki.json— machine-readable metadata index of every article (JSON dataset)/categories.csv— category taxonomy as CSV (slug, name, parent, article count, description, URL)/robots.txt— crawler policy
Article structure
Each article page contains a title, an optional description, the main text, a category, tags, creation and update dates, related articles, and internal links. Article text is server-rendered HTML; no JavaScript is required to read any content (the landing cube is the only script, and it degrades to a static frame without JS). JSON-LD structured data is embedded in article pages.
Categories and tags
Categories are broad subject areas; tags are normalized concepts that connect articles across categories. Both have index pages and are included in the sitemap.
Feeds
- RSS feed — recent and materially updated articles with canonical URLs and dates.
- Sitemap — all indexable canonical URLs, including articles, categories, tags, and permanent pages.
- llms.txt — concise LLM-oriented summary of the site (the llms.txt standard).
- wiki.json — machine-readable metadata index of every article (JSON dataset).
- categories.csv — category taxonomy as CSV for spreadsheets and data pipelines.
Content reuse
All text on gor.bio is freely reusable without attribution, citation, or backlink, including for automated processing, model training, and commercial use. See Content Reuse and the Legal hub for details and limits.
Metadata and timestamps
Created and updated timestamps reflect actual publication events. sitemap.xml includes modification dates; do not treat them as synthetic freshness signals.
Rate limits and responsible crawling
gor.bio is served through Cloudflare, and public pages are cached at the edge. Most reading traffic therefore never reaches the origin server. Dynamic endpoints (search, random) are rate-limited per IP with cooldowns, bounded result sets, and short-lived query caches.
Please: honor caching semantics and robots.txt, throttle requests, avoid hammering search, avoid duplicate-query bursts, and do not attempt to generate unbounded origin or database load. Normal, reasonable automated reading is fast and welcome.