01 The Idea
In 2023 I went through a podcast phase. Programming, working out, walking the dog, doing yard work — there was always a podcast running. It was informative, but I could never hold onto the details. I'd remember the shape of an intriguing idea and none of the specifics, which made it almost impossible to go back and act on anything.
When GPT-4 shipped that March, I set myself a challenge: build a chatbot grounded in the podcasts I'd actually been listening to. Not a model guessing from training data — a model answering from transcripts of the specific episodes. That's the RAG pattern: retrieve the relevant source material first, then let the model summarize and answer over it.
The stack:
- .NET MVC — the UX/UI front end
- .NET Semantic Kernel — the GenAI orchestrator
- Custom .NET Core CRON jobs — the ingestion pipeline
- MongoDB — the main datastore
- QDrant — the vector store for similarity search
- OpenAI GPT-4 — summarization and final answer
02 The Ingestion Pipeline
Before anything can be retrieved, the source material has to be collected and indexed. I built that as a chain of three CRON jobs — Downloader, Transcriber, Context Vectorizer — each one handing work to the next.
The Downloader checks the YouTube channels I followed for new videos, pulls them down, strips the audio to WAV, and persists the raw audio to MongoDB. Landing a new file kicks off a transcription task.
┌─────────────┐ ┌─────────────┐ ┌──────────────────┐ │ Downloader │────▶│ Transcriber │────▶│ Context │ │ │ │ │ │ Vectorizer │ │ YouTube → │ │ WAV → │ │ Transcription → │ │ WAV audio │ │ Timestamped │ │ Vector chunks │ │ → MongoDB │ │ fragments │ │ → QDrant │ └─────────────┘ └─────────────┘ └──────────────────┘
03 Transcription
The Transcriber takes the raw WAV and runs it through a local build of Whisper. The output is an array of fragments — each one a roughly 10-second window with an approximate start/end timestamp and the text spoken in that span. Those timestamps matter later: they're what let an answer point back to the exact moment in the episode.
var processor = whisperFactory.CreateBuilder()
.WithLanguage("auto")
.WithPrintProgress()
.Build();
var youTubeInfo = await _youtubeInformationRepository
.FindByIdAsync(context.YouTubeVideoInformationId);
var downloadedFile = await _fileRepository
.DownloadToBytesAsync(youTubeInfo.AudioFileId);
using (var fileStream = new MemoryStream(downloadedFile))
{
await foreach (var result in
processor.ProcessAsync(fileStream))
{
resultList.Add(new TranscriptionFragment()
{
StartTime = result.Start,
EndTime = result.End,
TranscribedText = result.Text
});
}
}
04 Sliding-Window Context
The final job, the Context Vectorizer, is where the retrieval quality is won or lost. A naïve approach would embed each 10-second fragment on its own — but a thought rarely fits in 10 seconds, and chopping on hard boundaries throws away the context that spans them.
So instead of isolated chunks, I built overlapping windows that slide across the whole episode. Each chunk carries a little of its neighbors with it, so no idea gets cut in half at a boundary. Those windowed chunks are what get embedded and inserted into QDrant.
Sliding-window context (overlapping chunks → QDrant): ┌──────────────┐ ┌────────────────────────────────────┐ │ Transcribed │ │ (timestamp) Speaker: transcript │ │ Segment 1 ├──▶│ text for this ~10 second window... │ ├──────────────┤ ├────────────────────────────────────┤ │ Transcribed │ │ (timestamp) Speaker: transcript │ │ Segment 2 ├──▶│ text for this ~10 second window... │ ├──────────────┤ ├────────────────────────────────────┤ │ Transcribed │ │ (timestamp) Speaker: transcript │ │ Segment 3 ├──▶│ text for this ~10 second window... │ ├──────────────┤ ├────────────────────────────────────┤ │ Transcribed │ │ (timestamp) Speaker: transcript │ │ Segment 4 ├──▶│ text for this ~10 second window... │ └──────────────┘ └────────────────────────────────────┘ Each chunk overlaps its neighbors to preserve cross-boundary context.
05 Retrieval & Answer
With everything indexed, the front end is simple. The user types a natural-language search — something like "the video where they talked about meditation lowering anxiety" — and Semantic Kernel hands the query to QDrant, which runs a nearest-neighbors search to find the transcribed segments closest to the question.
Those matched segments are then expanded — pulling the windows above and below each hit — to rebuild enough surrounding context, and that becomes the context window for GPT-4. A summarization prompt does the rest: the model reads the retrieved material and answers the user's question grounded in what was actually said.
User query
│
▼
┌───────────────┐ nearest ┌───────────────┐
│ Semantic │ neighbors │ QDrant │
│ Kernel │────────────▶│ Vector DB │
│ │ │ │
└───────────────┘ └───────┬───────┘
│ matched
│ chunks +
│ context
▼
┌───────────────┐
│ GPT-4 │
│ Summarize │
│ & Answer │
└───────┬───────┘
│
▼
Answer to user
06 What I'd Build Next
The POC answered questions well enough to prove the pattern. The obvious places to take it further:
- Timestamped citations — every fragment already carries a start time, so an answer could link straight to the exact second in the episode it came from. That turns "I think they mentioned it somewhere" into a deep link.
- Speaker diarization — attributing each segment to a speaker would let you ask "what did the host think?" versus the guest.
- A managed vector store — QDrant was great for a local POC; a managed service would take the ops burden off scaling the index.
- Re-ranking before the LLM — a cheap re-rank pass over the nearest-neighbor hits would tighten the context window before it ever reaches GPT-4.