User-agent: * Allow: / # Block backend API endpoints — no SEO value, waste of crawl budget Disallow: /api/ # Sitemap Sitemap: https://jobpilot-ai.pages.dev/sitemap.xml