Asset Assistant: Building an AI Search Tool That Finds Files Even When You Cannot Remember Their…
Asset Assistant: Building an AI Search Tool That Finds Files Even When You Cannot Remember Their NameA local retrieval system combining OCR, keyword search, semantic embeddings, and image matching without generating answers.Regular search only looks at a file name. It cannot find a file by a phrase…
Asset Assistant: Building an AI Search Tool That Finds Files Even When You Cannot Remember Their NameA local retrieval system combining OCR, keyword search, semantic embeddings, and image matching without generating answers.Regular search only looks at a file name. It cannot find a file by a phrase you remember inside it, or a picture you remember by what it shows. Here is how I built an AI tool that actually reads a file, and why I made sure it only ever points to something real.What this coversWhy regular search does not workWhy I did not use a generative AI modelHow a file gets ready to be searchedHow the search worksWhat it does when it is not sureTesting it before shippingWhat comes nextWhy regular search does not workSay you are looking for a picture. You remember a line of text that was written on it. You do not remember the file name. You do not remember the folder.Open a normal file explorer and try to search for that. You can not. Regular search only looks at file names. It never looks at what is actually inside a file. So you are stuck. You either ask a coworker if they remember it, or you just recreate the whole thing from scratch.This happens more than people admit.You remember a phrase from a slide, but not which deck it is inYou remember what a thumbnail looked like, but not what it is calledYou remember what a video was about, but the file is just named “video_final2”None of that is searchable with regular file search. Over months and years, this adds up to a lot of wasted time, and sometimes an outdated or wrong file gets used by mistake because nobody could find the current one.I set out to fix this with an AI assistant. This post walks through how it is built. More importantly, it explains how I made sure it never makes things up.Why I did not use a generative AI modelMost AI project write-ups reach for a generative model. That means something that reads your question and writes an answer. I did not build it that way.Here is the reasoning. The file someone is looking for already exists somewhere in the library. This is not a job for AI to write something new. It is a job to find the right existing file and prove it is the right one. So I built this as a retrieval system, not a generative one. It only ever returns files that are actually there.Generative Model vs Retrieval OnlyThere is a tradeoff. It can not write you a summary or a new sentence. It can only say “here is the file” or “I could not find one.” For a search tool, that is exactly the right tradeoff. I would rather it say nothing than make something up.How a file gets ready to be searchedBefore anything is searchable, every file has to be turned into text and numbers a computer can compare. That happens once, when a file is added to the library. It does not happen again every time someone searches.OCR, short for optical character recognition, scans images, thumbnails, and scanned PDFs, and pulls out any text that is rendered inside them. That includes the words on a slide, a caption baked into a thumbnail, or text in a screenshot. This is what makes “I remember text written on the image” searchable at all.Text extraction pulls the actual text straight out of native slide decks, PDFs, and scripts. No OCR is needed when the text is already digital.CLIP, a model that understands images and text together, looks at what an image actually shows. It reads the visual content itself, not just any text on it, and turns that into the same kind of numbers used for text. This is what lets a text query like “the bootcamp thumbnail” match a picture, even if the picture has no readable words on it at all.Once that is done, every file has both the text it contains and a description of what it visually looks like, ready to be searched.How the search worksEvery search is checked three different ways at once, and the three scores are combined into a single ranked list.How the search combines three signals into one ranked answerIn plain terms, here is what each one does.Keyword match, using BM25, looks for the exact words you typed. It is the classic keyword scoring method search engines have used for decades. It is fast and simple, and great when you remember specific terms. No model and no training needed.Meaning match, using all-MiniLM-L6-v2, a small sentence embedding model, turns your query into a set of numbers and compares it to the meaning of every file, not just its words. This is what lets a search like “the bootcamp thumbnail” find a file that never uses that exact phrase. It is a small model, about 22 million parameters and roughly 80MB, light enough to run on a laptop CPU.Image match, using CLIP ViT-B/32, compares your query against what is visually inside each image, using the embeddings created in the step before. It is about 151 million parameters, roughly 600MB, and again runs entirely on CPU.The three scores are not just averaged. They are combined on a scale calibrated by margin, meaning a result’s confidence is judged by how far ahead it is of the rest of the field, not just its raw score. This is part of what keeps the confidence labels meaningful instead of random.All of this runs locally, on a regular computer. No GPU, no cloud AI bill. That was a deliberate choice. It keeps the cost fixed and predictable instead of tied to how often people use it.The full stack. Every piece uses open model weights and runs on CPUWhat a real search looks likeThe diagram above is the idea. Here is an actual search from the tool, not a mockup. The query typed in was simply “person driving a car.”A real search for person driving a car, run against the content libraryThe top result is a photo with no caption and no matching filename at all. It was found only by what the picture shows, and it is labeled 79% high. The other two results also show a car, but not a person driving one, so they score lower and are labeled low confidence. Each card shows the file type, the confidence score, the source document, and a button to see why it matched.What it does when it is not sureThis is the part I spent the most time on, and the part I think matters most for anyone building AI tools. It is deciding what the system does when it is not sure.Every result gets checked against two confidence thresholds before it is ever shown.What happens after every searchIf a result clears the high bar, it is shown immediately, along with its source link and the exact text or image region that matched.If it only clears the lower bar, it is clearly labeled low confidence, fewer results are shown, and the system asks you to refine your search or check it yourself.If nothing clears either bar, it says so, plainly. No invented file names, no guessing. Instead, it offers to broaden the search or show the closest matches, clearly marked as unconfirmed.Clicking why this matched on the top result from the search above opens this panel. It shows the source file, the exact path it lives at, and the score from each search channel that led to the 79% figure.The “why this matched” panel for that same top resultA few other guardrails sit alongside this.GaurdRails — RisksTesting it before shippingBefore I called this ready for real use, I ran it against a labeled test set and set a bar it had to clear.Evaluation ResultsThe rule of having zero tolerance for invented results is not up for negotiation. If that number is ever above zero, the system does not ship. That is final.What comes nextThe prototype is done and tested. The next step is putting it in front of a small group of real users, watching how they actually search, and seeing whether they trust it enough to use it instead of digging through folders by hand. If the trust holds up over a couple of weeks of real use, it rolls out further from there.If you are building something similar, or thinking through the same question of retrieval versus generation, I would love to hear how you are approaching it.This story is published on Generative AI. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories.Subscribe to our newsletter and YouTube channel to stay updated with the latest news and updates on generative AI. Let’s shape the future of AI together!Asset Assistant: Building an AI Search Tool That Finds Files Even When You Cannot Remember Their… was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.Source: Generative AI Pub — Published — Category: Image AI