+- Web Design Forum HTML CSS JavaScript PHP Graphic Design SEO forum (http://www.webdesignforum.com.au)
+-- Forum: Search Engine Marketing (/forumdisplay.php?fid=18)
+--- Forum: Search Engine Optimisation (/forumdisplay.php?fid=8)
+--- Thread: Why the robots.txt file is important ?? (/showthread.php?tid=34290)
Why the robots.txt file is important ?? - Skymedia - 06-26-2018 07:43 PM
Why the robots.txt file is important ??
RE: Why the robots.txt file is important ?? - sinelogixseo - 06-27-2018 05:17 PM
It is a text file that tells web robots that which pages on your site to crawl and not to crawl.
RE: Why the robots.txt file is important ?? - kanetailor - 06-27-2018 08:46 PM
It indicates some parts of your site you don’t want accessed by search engine crawlers.
RE: Why the robots.txt file is important ?? - rishabjoy - 06-28-2018 06:02 PM
It is a text file that tells web robots that which pages on your site to crawl and not to crawl.
RE: Why the robots.txt file is important ?? - rhcalvin - 07-20-2018 06:18 PM
Robots.txt is a text file that lists webpages which contain instructions for search engines robots.