Log In · Signup for Free
EN  DE  FR  ES  IT  Русский  한국어  日本語  简体中文  عربي
🤖
Live and expired websites.INFO

Datasets .INFO All Robots.txt

Download a current list of Datasets .INFO All Robots.txt with website, company, location and technology detection fields.

One-time dataset price
$8,294.79

One-time delivery in Parquet or TSV format.

Buy Dataset

Compare Team Plan Ultra

Dataset facts
Domains
1,811,596
Detections
75,524,806
Generated
2026-07-18 UTC
Formats
Parquet, TSV
Scope
.INFO
What's included
  • ✓ Domain and technology detection history
  • ✓ Company and website location signals
  • ✓ Spend, revenue and employee estimates
  • ✓ Traffic and authority rankings
  • ✓ Available business contact and social fields
View fields and technical schema →
Find another dataset
Browse all datasets
Dataset sampleParquet sampleTSV sample

Preview representative rows before purchasing. Full downloads contain all available fields and records.

DomainTechnologyFirst detectedLast detectedCompanyLocationSpend
horizonclinic.infoWix2016-11-072026-07-19Sanca Horizon Centre BoksburgBarcelona, Catalonia, ES$204
horizonclinic.infoWix Premium2016-11-072021-04-22Sanca Horizon Centre BoksburgBarcelona, Catalonia, ES$204
horizonclinic.infoRSS2016-11-072026-07-19Sanca Horizon Centre BoksburgBarcelona, Catalonia, ES$204
horizonclinic.infoApple Mobile Web Clips Icon2016-11-072026-07-19Sanca Horizon Centre BoksburgBarcelona, Catalonia, ES$204
md7.infoQuadraNet2013-10-132015-01-27RU$875
md7.infoGoDaddy DNS2013-11-122015-01-30RU$875
md7.infoGoDaddy2013-11-122013-11-12RU$875
md7.infojQuery2013-11-122026-07-24RU$875
techygeekshome.infoBidvertiser Advertising2014-01-112014-05-18TechyGeeksHomeGrandville, MI, US$350
techygeekshome.infoSix Degrees Group2014-01-112016-07-20TechyGeeksHomeGrandville, MI, US$350
techygeekshome.infoApache2014-02-232020-03-03TechyGeeksHomeGrandville, MI, US$350
techygeekshome.infoGeoTrust SSL2014-05-182016-12-27TechyGeeksHomeGrandville, MI, US$350
rumiforum.infoCloudflare DNS2025-08-072026-07-19$443
rumiforum.infoCloudflare Mail2025-08-072026-07-19$443
rumiforum.infoFacebook Domain Verification2025-08-072026-07-19$443
rumiforum.infoGoogle Webmaster2025-08-072026-07-19$443
pharuey.infoGoDaddy Registrar2025-04-302026-04-29$350
pharuey.infoViewport Meta2026-01-062026-02-19$350
pharuey.infoIPhone / Mobile Compatible2026-01-062026-02-19$350
pharuey.infoMobile Non Scaleable Content2026-01-062026-02-19$350
wimborne.infoAdobe ColdFusion2015-01-272016-11-22WimborneWimborne Minster, ENG, GB$670
wimborne.infoTitan Internet2015-01-272016-08-28WimborneWimborne Minster, ENG, GB$670
wimborne.infoSimplyScroll2014-12-182016-11-22WimborneWimborne Minster, ENG, GB$670
wimborne.infojQuery2014-12-182026-05-16WimborneWimborne Minster, ENG, GB$670
cdlimoservices.infoOffice 365 Mail2024-04-062026-07-19cdlimoservices$490
cdlimoservices.infoSPF2024-04-062026-07-19cdlimoservices$490
cdlimoservices.infoGoDaddy Email2024-04-062026-07-19cdlimoservices$490
cdlimoservices.infoMicrosoft Exchange Online2024-04-062026-07-19cdlimoservices$490
autoschluessel.infoSSL by Default2023-02-102026-07-19$866
autoschluessel.infoCloudFront2023-02-102026-02-19$866
autoschluessel.infojQuery2023-02-102026-02-19$866
autoschluessel.infojQuery 2.2.42023-02-102023-02-10$866
Related datasets
Live and Expired websites using Robots.txt127,636,552Live and Expired websites using Robots.txt in .COM TLD77,559,832Live websites using Robots.txt32,823,758Live websites using Robots.txt in .COM TLD21,028,271Live and Expired websites using Robots.txt in United States7,906,043Live and Expired websites using Robots.txt in .NET TLD5,151,971Live and Expired websites using Robots.txt in .ORG TLD5,149,472Live and Expired websites using Robots.txt in .RU TLD3,677,100