How to serve robots.txt file for multi-store Magento setup

What is robots.txt file and how does it work? Web site owners use the /robots.txt file to give instructions about their site to web robots (aka search engine spiders, e.g. Google bot); this is called The Robots Exclusion Protocol. It … Read More