Crawling

Prevent Google crawling using ZXTM traffic script

If you are running ZXTM Load Balancer Virtual Appliance in your environment and plan to prevent your site or clients websites from been crawled by search engines, then you need to create a traffic script to accomplish this. This post is based on the assumption that: [highlight]- you are already using zxtm and have setup your Virtual Servers – the domain┬áDNS A record is pointing to your zxtm public IP address[/highlight] Step 1: Create text file – Create a file called robots.