Robots.txt is one of the more accessible areas of SEO since you can access any site’s robots.txt.
http://www.example.com/robots.txt
You can view any site’s robots.txt file by entering the URL above directly into the browser. Replace www.example.com with your chosen website domain.