If you aren’t in the know, search engine is not as smart as what you think it is. It can’t tell which pages of your site should be included in the index and which files to ignore. Therefore we need to instruct it on what to do using a robots.txt file. What is robots.txt file? [...]

