Crawler

Welcome to this analysis website. Behind this site a spider crawls the whole web and analyse from every host, what generator is used to show up the website on that host. The generator of a website is specified through the meta tag generator. You can find this tag inside the header section of an HTML-document. In general it looks like this:

Because there are no rules, what is in the content part, every generator is formated on its own way. Some have a company name, other the product, some have just the version of the product, without the name and so on. Some of the found generators are grouped together, so the product or company could be analysed and compared with some better numbers. The generator Wordpress, for example, is found with several thousand different versions.

The data is collected through a crawler named MetaGeneratorCrawler. It visits the entrypage of every linked domain or subdomain regardless, where the link points on the crawled site.

The goal of this site and crawler is, to collect which generators are used through the web. The country level domains give an extra information, how the software is spread around the world.

If you do not whish, that the crawler visit your site again, feel free to add the following line to your robots.txt: