The Brief
In early 2023 a client wanted to know whether GPT-3 could answer questions about their own documents, text the model had never seen, without inventing an answer around them. The engagement was contracted as a proof of concept in February 2023, and that is what it was: enough of a working pipeline to judge whether the approach deserved a real build.
What I Built
- Document indexing and chunking: the client's text split into passages, cleaned and stored with ids so every answer could be traced back to the passage it came from.
- Embeddings: one vector per chunk through the OpenAI API, stored alongside the text.
- Similarity retrieval: each question embedded the same way, chunks ranked by cosine similarity, the top passages selected.
- Prompt assembly and answer generation: the retrieved passages and the question composed into a single prompt so GPT-3 answers from the client's text rather than from memory.
- Plain Python throughout, following the OpenAI cookbook embeddings pattern of the time. No framework, because none of the RAG frameworks existed yet in a form worth depending on.
Why It Holds Up
The retrieval step is what makes the answers checkable. Because every answer is built from named passages, the client could open the source and see whether the model had read it correctly. That traceability is the reason the same pattern went on to become the default way of putting a language model in front of private data.
What I would add on a production build today, and did not in a two-week proof of concept: chunking by document structure instead of fixed length, hybrid keyword-plus-vector search, and a small evaluation set of real questions with known answers so retrieval quality is measured rather than eyeballed.