AI on Substack: a survey

Every post in the sample falls into one outcome. Writers can turn detection off per post; podcasts and videos are not scanned. Paywalled and subscriber-only posts ( of the posts dated since the cutoff, ) return no result to a non-subscriber and are outside the sample. The sampling design and the exact endpoint behaviour are on the Methods page.

Intervals are 95% Wilson: fully AI –, any AI –, n=. Bands are Substack's own labels on the result ("Fully AI-assisted text", "AI-assisted text", "Partially AI-assisted text", "Mostly Human-Written", "Human-Written", "Fully Human-Written"); they step on the human fraction at 1%, 10%, 50%, 90% and 99%. "Any AI" means any label but "Fully Human-Written". The histogram is the fraction scored as AI, which is why a bar can hold two bands: a post with 5% AI-assisted text and no AI text sits in the leftmost bin as "Human-written", and a post 95% AI sits in the rightmost as "AI-assisted". The badge Substack shows readers fires only at 100%: the stored flag is true for posts here, against with any AI detected.

Category here is assigned by a classifier () from each publication's name, tagline, author bio and recent post titles, using Substack's own 33 leaderboard categories. Checked blind against publications Substack ranks, it agrees with the leaderboard of the time and counting its second choice; the confusion matrix is on the Methods page. Categories with fewer than ten scanned posts are left off. Substack's own leaderboard categories cover only sampled publications; that breakdown is below. Subscriber size is the publication's subscriber count as Substack shows it on the author's profile (rounded to two figures); where that is hidden, Substack's rounded free count or size label is used instead.

Substack's own labels: all category leaderboards, linked. "Ranked pubs" is the leaderboard length we crawled; the other columns count the sample. Country leaderboards ("country: IT") are not treated as categories.

Publications are reached through a random post from the after-window, so a publication with ten posts since the cutoff is ten times as likely to appear as one with a single post. The figures above weight each publication by one over its count of posts after the cutoff, which undoes that and describes publications that posted at least once after the cutoff. Publications that went quiet before the cutoff cannot appear at all, so this is not evidence about whether Substack as a whole sped up. Unweighted medians: posts per week before, after, over publications.

Each sampled publication's archive is paged back days before the cutoff; the after window is the same length or shorter if today comes first. A ratio above zero means the publication posted more often after Substack began scanning. Only publications whose archive crawl reached the window start are counted.