Skip to content

Cloudflare raises its Markdown for Agents limit to 6 MiB

An edge rewrite now streams HTML-to-markdown conversion in process instead of buffering it, and drops the token-count headers some integrations relied on.

Source: Cloudflare

Cloudflare published a changelog update on 9 October reworking how Markdown for Agents converts pages. The feature lets any client fetch a page as markdown instead of HTML by sending Accept: text/markdown on a zone where it is enabled. Previously, Cloudflare buffered the full HTML response and forwarded it to a separate conversion service; it now runs an in-process streaming engine at the edge that converts content as it arrives.

The practical change is the size limit: conversion now handles up to 6 MiB of decompressed HTML, up from 2 MiB, measured after decompression rather than on the compressed response. Two things also disappeared: the x-markdown-tokens and x-original-tokens response headers are no longer sent, and Content-Length is dropped from converted responses rather than recalculated, since the body now streams.

curl https://example.com/ -H "Accept: text/markdown"

Why it matters: if you have an agent or crawler pulling pages this way, larger pages that used to fail now convert, and the streaming path should cut time to first byte. If anything in your pipeline read x-markdown-tokens to budget context before sending a page to a model, that header is gone, and you will need to count tokens yourself.

Share

Related reading

More from The Wire

All briefs