www.VFXskills.com                                                                                 www.MAXlife.TV

TechsChoice.com
Google Enter your search terms

What is a robots.txt file and how do I make one?


A robots.txt is a file placed in your root directory (where the index.html file is located) and tells search engine spiders not to crawl or index certain sections or pages of your site or to allow indexing of the site.

The file itself is a simple text file, which can be created in Notepad.


There is nothing difficult about creating a basic robots.txt file. It can be created using notepad or whatever is your favorite text editor. Each entry has just two lines:

User-Agent: [Spider or Bot name]
Disallow: [Directory or File Name]

This line can be repeated for each directory or file you want to exclude, or for each spider or bot you want to exclude.

EXAMPLES:

You have a file, privatefile.htm, in a directory called 'private' that you do not wish to be indexed by Google. You know that the spider that Google sends out is called 'Googlebot'. You would add these lines to your robots.txt file:

User-Agent: Googlebot
Disallow: /private/privatefile.htm

 

Once again you can use the wildcard, '*', to let all spiders know they are welcome. The second, disallow, line you just leave empty, that is your disallow from nowhere.

User-agent: *
Disallow: 


 

4. Allow no spiders to index any part of your site

This requires just a tiny change from the command above - be careful!

User-agent: *
Disallow: /

If you use this command while building your site, don't forget to remove it once your site is live!

Site Map         

 

Please add us to your website or blog!

Privacy Policy - Disclaimer
free sound files, free media files, free tutorials, free web hosting, cheapest domain names, inexpensive domain names, screen capture software, Computer Software Tutorials, online training videos, computer basics tutorial, free windows tutorial, free computer tutorial for kids, computer programming tutorials, free computer courses, pc tutorial, beginner computer tutorials, free computer software tutorials, free internet tutorial, free typing tutorial programs, computing tutorials, computer science tutorials, graphic design tutorials, free visual basic tutorials, free Photoshop tutorials, newbie computer tutorials free flash tutorials, access tutorial online, computer course, pc maintenance online tutorials, absolute beginners computer tutorials, free online computer classes, frshirtsbestee online learning, free computer training download, free computer tutorial for seniors, computer parts tutorial, computer tutorials for children, computer hardware tutorials, free excel tutorials, computer fundamental introduction basics tutorial, Microsoft tutorial, free grocery coupons, Free computer software tutorials, tech support, technical support, the tech support forum, game development, 3d game development software tutorials, graphic design tutorials, graphic illustration tutorials, character animation and design tutorials, web development, Clipart and Banner Ads, adobe photoshop tutorials and downloads, software lessons, how to do it yourself, Fierce Layouts learn adobe Photoshop, learn silverlight, learn microsoft expressions, asp.net, video editing, youtube video, digital image editing software, national association of Photoshop professionals (naap) Flash Animation, graphics, computer software, computer books, manhattan flash animation presentations for advertising and banner ads, adobe, photoshop, macromedia, microsoft, software and application development, learn to, how to, tutorials, hardware, software Tech Support - Free Technical Support for Beginners and Advanced Users of Computers, Videographers, photographers, Programmers and Web Designers, Tutorials and How To Start, computer help, video, audio, nonlinear editing, cameras, microsoft excel free online video lessons, learn excel, learn microsoft word

Free software tutorials, downloads and technical support information for computer geeks, photographers and videographers.