Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
What is the use of robots.txt?
09-23-2016, 10:43 PM
Post: #1
What is the use of robots.txt?
Hello Guys,


I would like to know that What is the use of robots.txt? tell me ..

Free astrology consultancy services | Free online Vedic astrology in hindi | Free astrology on phone | online horoscope astrology reading
Visit this user's website Find all posts by this user
Quote this message in a reply
09-26-2016, 10:01 PM
Post: #2
RE: What is the use of robots.txt?
Prevention of crawling the confidential information like credit card, email, phone number.

Mobile Application Development Madrid | Mobile Application Development Istanbul
Visit this user's website Find all posts by this user
Quote this message in a reply
10-07-2016, 04:49 PM
Post: #3
RE: What is the use of robots.txt?
Is a text file webmasters create to instruct robots how to crawl & index pages on their website.

Electrical Panel Manufacturers | DG Synchronization Panel Manufacturers
Visit this user's website Find all posts by this user
Quote this message in a reply
10-11-2016, 09:36 PM
Post: #4
RE: What is the use of robots.txt?
It is file that will help in crawling.

Vashikaran guru | Muslim Vashikaran
Visit this user's website Find all posts by this user
Quote this message in a reply
10-12-2016, 09:22 PM
Post: #5
RE: What is the use of robots.txt?
It will use to instruct crawler how to crawl & index pages on their website.

Black magic specialist in Pune Love problem solution in Mumbai | Black Magic Specialist in Australia
Find all posts by this user
Quote this message in a reply
10-12-2016, 10:07 PM
Post: #6
RE: What is the use of robots.txt?
Robots file use to instruct crawler.

Black Magic Specialist | Love Marriage Solution
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)