July 03, 2026
20 min read
We logged 1,348,706 requests across two production web properties for 174 days and classified every user-agent. AI crawlers now generate 7.2% of all observed traffic — more than Googlebot, Bingbot, DuckDuckBot, Baiduspider, and PetalBot combined. OpenAI alone accounts for 38% of the AI crawl, 69% of AI requests still feed training pipelines, and a robots.txt honeypot shows 99% of AI-crawler violations trace to a single bot. For publishers, infrastructure teams, and anyone deciding what to do about the new tenants of the crawl economy.
June 23, 2026
18 min read
We pointed an NS-typed crawl at 301 million registry domains to ask who actually answers the first question of every web request. 277.6 million are delegated, and the answer is concentrated: registrar-bundled DNS runs nearly half the delegated web, GoDaddy leads at 18.9% as the default nobody chose, and Cloudflare follows at 13.9% as the operator everybody migrates to — more than every hyperscaler cloud combined. Beneath the leaderboard sits a quieter finding: 92.5% of delegated domains depend on a single DNS organization with no independent backup. Built from 151 million-plus apex NS records.
June 22, 2026
20 min read
India has 954 million internet users and the world's second-highest IPv6 capability — 78% of its users can reach the modern Internet, and Reliance Jio's all-IP network runs at ~95%. We then crawled its national namespace: 20.2 million .in hostnames, 10.2 million resolving across forward-DNS, MX, AAAA, NS, DMARC and IP-to-ASN data. The content layer tells the opposite story. Only 20% of .in domains are hosted on Indian soil — the United States alone hosts twice as much; only 20% offer IPv6 despite the world-class access network; Amazon AWS is the single largest operator of the Indian web. India consumes the Internet on world-class rails and publishes it on borrowed ground.
June 21, 2026
20 min read
We pointed an MX-typed crawl at 390 million registered domains to ask who actually handles the Internet's email. Only 38.7% can receive mail at all. Among those that can, Google Workspace leads Microsoft 365 by 1.4 to 1 (13.2% vs 9.4%) — the inverse of the enterprise market — yet both are dwarfed by registrar- and host-bundled email, which serves nearly a third of all mail-capable domains directly and most of the long tail besides. The result is a geographic map: Google owns the Americas and the developer world, Microsoft owns Northern Europe and the Commonwealth, and domestic hosts own their home ccTLDs. Built from 151 million apex MX records.
June 20, 2026
24 min read
A CNAME record is a forwarding address — it names who actually serves a hostname. We pointed a CNAME-typed crawl at 1.86 billion names and found 297 million aliases. Of the 169 million that point outward, 41% land on site builders and managed hosts, 20% on domain-parking lots, and only 6% on CDNs — because Cloudflare, which fronts a quarter of the web, flattens its CNAMEs into A/AAAA and barely appears. The alias map is a rental map: the web is served by a short list of platforms, and the record type you query decides which half you can see.
June 13, 2026
18 min read
On 28 March 2026 IPv6 crossed half of Google's traffic — the eyeball Internet reached the majority. We pointed an AAAA-typed crawl at 315 million registry domains to measure the other half of the question, and the content Internet looks nothing like it: only 20.4% of domains publish any IPv6 address, Cloudflare alone generates 44.7% of it, and 45% of all domain IPv6 is a CDN edge in front of an origin that may still be IPv4-only. Strip the CDNs and origin-native IPv6 falls to 11%. This is server-side IPv6, measured from 1.88 billion DNS answers.
June 01, 2026
19 min read
We resolved the full DNS, MX, web-edge, and email-authentication stack of 500 Fortune 500 corporate domains against a May 2026 typed crawl. No single vendor owns enterprise DNS — 28% of the F500 still run their own. Proofpoint sits in front of 48% of mail-enabled inboxes. Akamai serves a third of the web edge. And the F500 enforce DMARC at 73% versus ~42% globally — yet MTA-STS reaches just 2.3%. This is the corporate Internet's plumbing, measured from the records themselves.
May 07, 2026
34 min read
We analyzed 29.3 million .pl hostnames in our June 2026 snapshot against NASK's 2.61 million registered domains to find that Poland's namespace is dominated by ISP reverse-DNS infrastructure to a degree few European ccTLDs match: Orange Polska's three reverse-DNS zones alone account for 16.0 percent of every observable .pl hostname, and twenty-eight operators together generate over ten million entries — roughly a third of the namespace. Alongside the infrastructure layer sit Polish shared-hosting platforms (home.pl, nazwa.pl, kylos.pl) and a frozen ecosystem of mid-2000s Polish-language platforms — pinger.pl, digart.pl, fora.pl, republika.pl — that still resolve in 2026, plus a regional architecture of twenty-three city and voivodeship second-level domains that NASK launched in the 1990s and that, three decades on, account for under 1.5 percent of the namespace. Refreshed June 2026 with first-party DNS-resolution data.
May 03, 2026
32 min read
We extracted every Kubernetes signal we could find from a 17 April 2026 DNS crawl — heritage=external-dns TXT markers AND CNAME chains terminating in managed-Kubernetes ingress endpoints (AWS ELB k8s-prefixed names, .azmk8s.io, .gke.goog, .openshiftapps.com, .k8s.ondigitalocean.com, etc.). 74,508 unique apex domains carry at least one strict-precision Kubernetes signal (41,565 with TXT markers, 34,219 with strict CNAME pointers, 1,276 in both). 20,420 distinct cluster identities are visible. 13,620 apexes (32.8% of TXT-marker side) use the literal string "default" as their cluster identifier. 815 use the literal example strings from the ExternalDNS README. 6,842 apexes publish a sensitive Kubernetes namespace (argocd, vault, kube-system, istio-system) to public DNS. 1,936 apexes have already migrated to the Gateway API. This is the first combined-signal cluster-identity census of the public Kubernetes footprint.
May 02, 2026
21 min read
We classified every TXT record from a 17 April 2026 DNS crawl — 840 GB of raw JSONL (56 GB after xz compression) — and built a vendor census from the verification tokens domains leak into DNS. 40.2 million unique apexes carry at least one tracked SaaS verification token. Google's 26.0 million-apex footprint is 3.4x Microsoft 365's 7.6 million. Domain marketplaces (AfterNic + dan.com + 4.cn + Aliyun + west.cn + 17ex + Sedo + DomainEasy) collectively touch 5.0 million apexes — more than Atlassian, Stripe, Adobe, Apple, and DocuSign verification tokens combined. Zoho's 1.23 million is the single largest non-Google, non-Microsoft SaaS verification footprint we measure. The TXT layer is the closest thing the public Internet has to a SaaS census.