Description:
Better Robots.txt replaces the default WordPress robots.txt workflow with a structured version you can configure and preview before publishing. Instead of a blank text box, it provides a guided wizard with presets, plain-language explanations, and a final Review & Save step where you inspect the generated robots.txt before it goes live. The plugin helps you control how search engines, AI crawlers, SEO tools, archive bots, bad bots, social preview bots, and other automated agents interact with your site.
Through the workflow you can manage search engine visibility, AI and LLM crawler behavior, AI usage signals such as search, ai-input, and ai-train preferences, SEO tool crawlers, bad bots, archive and Wayback access, feed crawlers and crawl traps, WooCommerce crawl cleanup, CSS, JavaScript, and image crawling rules, social media preview crawlers, ads.txt and app-ads.txt allowance, llms.txt generation, and advanced directives such as crawl-delay and custom rules. Setup starts with four presets: Essential, AI-First, Fortress, and Custom.
Beginners get a guided setup with preset-based configuration and plain-language explanations, while advanced users can edit core WordPress protection rules, apply fine-grained crawler controls by category, and consolidate output. This Premium edition unlocks the most restrictive and advanced protection options, including the Fortress preset and additional high-control modules. The plugin is designed for the AI era and is trusted by thousands of WordPress sites.
Main Features of Better Robots.txt – AI-Ready Crawl Control & Bot Governance (Premium):
- Guided setup wizard with presets, plain-language explanations, and a final Review & Save step to inspect the generated robots.txt before it goes live
- Control how search engines, AI crawlers, SEO tools, archive bots, bad bots, and social preview bots interact with your site
- AI usage signals including search, ai-input, and ai-train preferences, plus optional llms.txt generation
- WooCommerce crawl cleanup and removal of low-value crawl paths that waste crawl budget
- Editable core WordPress protection rules and advanced directives such as crawl-delay and custom rules
- Control over archive and Wayback access, feed crawlers, crawl traps, and ads.txt and app-ads.txt allowance
- Premium edition unlocks the Fortress preset and the most restrictive and advanced protection modules
EffectIO members receive updates through the managed update channel. Install the EffectIO Connect plugin on your site and it handles activation, updates and licensing automatically.
Need a dashboard view of all your EffectIO assets? The EffectIO Dashboard shows what is installed, what is licensed, and what has an update available.
The 3.0.1 version stays on this update channel for the subscription year.
Reviews
There are no reviews yet.