User-agent: * Disallow: /user-agents.php Disallow: /spider-bots.php Disallow: /index.php?*