Use this directive to instruct search engines like yahoo not to access information and web pages that slide under a selected route. For example, in case you planned to block all engines like google from accessing your site and all its posts, your robots.txt file may appear like this: By https://rankingseotoolz.com/robots-txt-generator