# All robots will spider the domain User-agent: * Disallow: User-agent: * Disallow: /config.asp Disallow: /database/ Disallow: /images/