Question

In a Retrieval Augmented Generation (RAG) system, what is the primary technical limitation of using a small, fixed-size chunking strategy on highly technical documents that contain cross-references?

Accepted Answer

The primary technical limitation of using a small, fixed-size chunking strategy on highly technical documents with cross-references is the loss of semantic context, which occurs when a chunk is split in a way that separates a reference from the information it describes. In a Retrieval Augmented Generation system, chunking is the process of breaking long documents into smaller segments for processing. When a fixed-size strategy is used, the system ignores the logical structure of the text, such as paragraphs or sections, and instead cuts the document based on a specific number of characters or tokens. In technical writing, a cross-reference often points to a specific definition, formula, or diagram located elsewhere in the document. If a small chunk contains only the reference but not the target content, the retriever cannot provide the relevant context to the language model. Because the language model only receives the information present within the retrieved chunk, it fails to connect the reference to the actual definition, leading to incomplete or inaccurate answers. This isolation of related information prevents the system from maintaining the comprehensive understanding required to process complex technical material.

Home → All Courses → Engineering and Technology Courses → Generative AI Application Development → Flashcard

In a Retrieval Augmented Generation (RAG) system, what is the primary technical limitation of using a small, fixed-size chunking strategy on highly technical documents that contain cross-references?