BigQuery can now chunk your documents for RAG without leaving SQL

Google’s BigQuery release notes mark AI.CHUNK_DOCUMENT as generally available as of 21 October. It is a table-valued SQL function that splits unstructured documents into chunks sized for vector embeddings and retrieval-augmented generation, without needing a separate chunking service outside the warehouse.

The function does layout-aware chunking rather than cutting text at a fixed character count. A plain splitter can produce a chunk that has lost the heading or table header that gave it meaning. AI.CHUNK_DOCUMENT carries that ancestral context, such as the section heading or table header a passage sits under, into the chunk itself, so the chunk still makes sense when a retrieval step pulls it out in isolation. It is designed to run alongside AI.PARSE_DOCUMENT, which handles the parsing step before chunking.

SELECT *
FROM AI.CHUNK_DOCUMENT(TABLE my_parsed_docs)

Why it matters: anyone building RAG on top of BigQuery has usually been chunking documents in application code or a separate pipeline before generating embeddings. Doing it as a SQL function means the chunking step lives next to the data and can run as part of a scheduled query or pipeline, and the layout-awareness should mean fewer retrieved chunks that make no sense without the heading that got stripped off them.

The caveat: Google’s release note doesn’t give a maximum document size or a list of supported input formats, so check the function’s own reference docs before relying on it for anything unusual.