Tagged “metadata”
-
Deduplicating chunks before you index them
One warning in forty manuals becomes forty chunks. Which duplicate chunks to collapse, which to keep, and why a version difference is not a duplicate.
-
Metadata is half the chunk
A chunk with no source, date or heading path can't be filtered or cited. What to attach, what to embed with the text, and what to keep out of the vector.
-
Figures, captions, and the text that refers to them
An image is not indexable and its caption is. Keeping figure, caption and the paragraph saying see Figure 3 together, or at least connected.
-
Chunking slide decks
A slide is six words and a chart, and the meaning was in the room. What to do with decks that must be indexed anyway, starting with the speaker notes.
-
Rewriting a chunk so it can stand alone
Chunks are full of pronouns, acronyms and see-above references pointing at text you cut away. What to resolve at index time, and what resolving it breaks.