CrUX Dataset

CrUX Dataset Usage Statistics · Download List of All Websites using CrUX Dataset

CrUX is a data collection system that gathers information about how real users interact with websites. This website is included in the user experiences data gathered from Google Chrome and thus considered sufficiently popular on the Internet.

Viewport Meta

Viewport Meta Usage Statistics · Download List of All Websites using Viewport Meta

This page uses the viewport meta tag which means the content may be optimized for mobile content.

IPhone / Mobile Compatible

IPhone / Mobile Compatible Usage Statistics · Download List of All Websites using IPhone / Mobile Compatible

The website contains code that allows the page to support IPhone / Mobile Content.

AmazonBot Disallow

AmazonBot Disallow Usage Statistics · Download List of All Websites using AmazonBot Disallow

AmazonBot is a crawler that web content publishers can refer to for information - website has disallow rules.

AI Bot

Artificial Intelligence

Artificial Intelligence Usage Statistics · Download List of All Websites using Artificial Intelligence

This technology represents us finding some form of mention of AI or ChatGPT on the homepage of the website.

FastMail

FastMail Usage Statistics · Download List of All Websites using FastMail

FastMail provides secure and reliable email environments.

Secure Email

SPF

SPF Usage Statistics · Download List of All Websites using SPF

The Sender Policy Framework is an open standard specifying a technical method to prevent sender address forgery.

Namecheap DNS

Namecheap DNS Usage Statistics · Download List of All Websites using Namecheap DNS

DNS services provided by Namecheap.

SSL by Default

SSL by Default Usage Statistics · Download List of All Websites using SSL by Default

The website redirects traffic to an HTTPS/SSL version by default.

LetsEncrypt

LetsEncrypt Usage Statistics · Download List of All Websites using LetsEncrypt

Let’s Encrypt is a free open Certificate Authority.

Root Authority

nginx

nginx Usage Statistics · Download List of All Websites using nginx

nginx [engine x] is a HTTP server and mail proxy server written by Igor Sysoev.

RSS

RSS Usage Statistics · Download List of All Websites using RSS

A family of web feed formats used to publish frequently updated content such as blog entries, news headlines or podcasts.

GPTBot Disallow

GPTBot Disallow Usage Statistics · Download List of All Websites using GPTBot Disallow

The website has disallow rules for OpenAI GPTBot.

AI Bot

Common Crawl Bot Disallow

Common Crawl Bot Disallow Usage Statistics · Download List of All Websites using Common Crawl Bot Disallow

Block a site from being crawled by Common Crawl Crawler.

AI Bot

Anthropic Claude Bot Disallow

Anthropic Claude Bot Disallow Usage Statistics · Download List of All Websites using Anthropic Claude Bot Disallow

Block Anthropic Claude bot.

AI Bot

ByteDance Bytespider Disallow

ByteDance Bytespider Disallow Usage Statistics · Download List of All Websites using ByteDance Bytespider Disallow

Doubao potentail AI bot disallow rule.

AI Bot

Facebook Bot Disallow

Facebook Bot Disallow Usage Statistics · Download List of All Websites using Facebook Bot Disallow

Blocks Facebook crawling.

AI Bot

Google Bard Disallow

Google Bard Disallow Usage Statistics · Download List of All Websites using Google Bard Disallow

The website blocks Bard AI and Vertex AI scraping requests.

AI Bot

Webz Omgili Disallow

Webz Omgili Disallow Usage Statistics · Download List of All Websites using Webz Omgili Disallow

Webz.io disallow block for big data indexing.

AI Bot

Perplexity

Perplexity Usage Statistics · Download List of All Websites using Perplexity

AI chatbot-powered research and conversational search engine disallow rules.

AI Bot