When creating your website you will need to communicate certain information to search engines about the nature of your website (pointing the search engine to your sitemap for instance). You may also want to keep some of the pages of your website private so will want to instruct a search engine not to display these when someone searches for your website online. When a search engine robot is looking for websites it will first try to find a /robot.txt file that is associated with that website. This file will contain all of the information you wish to pass on to the search engine robot. This information will mainly consist of what you want the search engine to display (or not to display) about your site should anyone enter a search for your site into the search engine. Its worth noting that if malicious software (malware) tries to access your site it can choose to ignore the /robot.txt file assigned to your website for nefarious means.
A handy guide on robots.txt file can be found at http://www.robotstxt.org/robotstxt.html
If you should have any issues or have any questions regarding this topic please contact us via a support ticket (open a ticket) with our hosting department and we will get back to you as soon as possible