# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # Exclude the following from being indexed by search engines: User-agent: * Disallow: # Sends requests with improper accept-type set, triggering errors. User-Agent: Sogou Pic Spider Disallow: / Sitemap: https://www.prefinery.com/sitemap.xml