Methods

What is measured

Substack scans every post published on or after 21 July 2026 (11:30 EDT) with Pangram when it is published, stores the result, and serves it to logged-in readers who choose "Scan for AI text". This study reports those stored results. It therefore measures what Substack's detector says, not ground truth about authorship. That is the policy-relevant quantity, since it is what Substack labels, but it inherits Pangram's error profile, and it says nothing about AI use that Pangram does not flag.

Sampling

Substack post ids are sequential integers assigned when a draft is created. A wave draws ids uniformly at random between a lower bound and the current frontier (the highest id visible in recency-ranked site search), and every drawn id is requested from Substack's by-id endpoint. Ids that belong to drafts, deleted posts and unpublished scheduled posts return 404 and are recorded as such. Posts dated before the cutoff are recorded but fall outside the analysis population. Because ids are assigned at draft time, a small share of posts published in the window carry ids below the lower bound (0.3% in a 15,000-post reference set); they are outside the frame.

Outcomes

The population is public posts: Substack's audience field is everyone. Paid, subscriber-only and founding posts answer "Subscription required" to a non-subscriber, so they could only ever be a gap; they are recorded, counted above, and left out of every figure on this site (a change made on 2026-11-13; before that they were an outcome of their own, which changed no share among scanned posts). For each post in the population the detection endpoint returns one of: a result with three fractions (AI, AI-assisted, human) and Substack's label; "AI detection unavailable" when the writer has disabled it for that post; "not supported" for podcasts and videos; "Not enough text" for posts too short for Pangram to score. Bands on this site are Substack's labels, not thresholds chosen here. The reader-facing badge and the stored boolean on the post object fire only at a fraction of exactly 1.0.

Politeness

One request a second at most across all hosts, a single identified User-Agent with a contact address, exponential backoff honouring Retry-After, no paywall circumvention, and the detection endpoint paced at one request per seconds. Requests logged: ; by status: .

Publication context

Each sampled publication's public profile supplies the author's follower count, bestseller tier, subscriber count (as Substack displays it, rounded to two figures) and, for publications on a category leaderboard, the category. The 33 category leaderboards (525 publications each) are crawled as a second category source. Substack's own category is therefore known only for the ranked tier.

Category classifier

Because Substack publishes a category only for ranked publications, every sampled publication is also classified by into one of the same 33 categories from its name, tagline, author bio, language and the titles, subtitles and descriptions of its newest posts. No post bodies are sent. The classifier is checked blind against publications whose category Substack does publish: ranked publications drawn outside the sample ( per category, seeded) plus sampled publications that happen to be ranked. The leaderboard category is never shown to the model.