limit crawlers and bots

Our websites are crawled from different bots and search engines, sometimes it is beneficial or sometime it is creating the issue. Bots and crawlers are consumed lots of bandwidth. To avoid this issue we can upload robots.txt file in our home directory. In this file we can block bots and for that...