When Search Can See: Building Native Semantic Image Search in ODC

Most of the RAG stack I have been building inside ODC starts with the same assumption. The content is text. Chunk it, embed it, store it, retrieve it.

That worked for chunking strategies, vector storage, and semantic search over text. But image search breaks that assumption.

Tag everything manually. Search by filename. Filter by category. They all require someone to have already described the image before the search can find it. The moment a user types “red sneaker with white sole” and expects a photo back, none of those options help.

So I wanted to find out whether ODC could do semantic image search natively. Where the image itself is the thing being searched, not a label someone attached to it.

Text query “Get me a cat with a hat.” Results ranked by vector similarity.

The Insight: Search Does Not Have to Care About Pixels

Text and images feel like completely different retrieval problems. You can tokenize text. You cannot tokenize a photo.

But embedding models do not treat them as different problems. A multimodal embedding model converts both into vectors in the same shared space. A text query and an image of the thing it describes land near each other in that space. That proximity is what makes search work.

The model I used is Cohere Embed v4. It supports text, images, and combinations of both in one shared embedding space. I chose it for two practical reasons. It is a true joint text-and-image model, so both modalities land in the same space without extra mapping work. And it has a free tier, which makes it easy to prototype before committing to anything. The request and response parsing in the C# layer are Cohere-specific, but the ODC layers above it do not need to know that.

That shared vector space is the point. Everything else in this article is implementation detail around it.

Before We Go Further

This is a native ODC implementation. C# External Logic handles the embedding and scoring math. ODC Libraries provide the wrapper. Standard ODC entities store the vectors.

Search runs as a linear scan, scoring every stored vector against the query. That keeps the design simple and works for small to medium collections. It is not built for very large corpora, and I will be specific about where that boundary sits. If you are an ODC developer who wants semantic image search without standing up external infrastructure, this is for you.

ODC’s native Semantic Search covers vector-based retrieval over text attributes. This is the image side of that story.

Three Layers, One Clear Job Each

The implementation splits into three layers.

Three layers: ODC App, ODC Library, C# External Logic down to Cohere.

MultimodalEmbeddingLibrary is a stateless C# External Logic library. It calls Cohere, parses the response, normalizes vectors, packs the binary, and scores candidates. It holds no state between calls. It reads from and writes to nothing. All storage orchestration belongs to the ODC App layer.

Image Vector Library is an ODC Library wrapping MultimodalEmbeddingLibrary. This is where API credentials flow in from Site Properties and where ODC-typed structures like VectorCandidate and ImageSearchResult live. It is the only layer that speaks both C# and ODC.

Image Semantic Search is the ODC App. Entities, ingestion logic, search orchestration, UI. This is where everything connects.

The C# layer takes its endpoint, API key, and model name as parameters from ODC orchestration. None of those are hardcoded.

The C# Details That Actually Mattered

The tricky parts of this build are not in the UI. They are in the details between ODC, C#, binary data, and the embedding API. A few of those details matter enough to walk through.

1. The MIME type problem

Cohere Embed v4 does not accept a raw base64 image string. It requires a data URI: data:image/jpeg;base64,<data>. Send raw base64 and you get HTTP 422 with no useful explanation.

The C# library detects the MIME type from the first bytes of the image binary before encoding. FF D8 FF is JPEG. 89 50 4E 47 is PNG. 47 49 46 38 is GIF. The WebP signature is longer. Anything unrecognized falls back to image/jpeg. The detection happens before the API call, so the data URI is correct before anything leaves the process.

2. The async problem

In this implementation, the ODC-facing External Logic methods are exposed synchronously. The HTTP call itself is async, so the C# layer bridges that internally.

Task.Run(async () => await CallAsync(...)).GetAwaiter().GetResult();

Not elegant, but it keeps the ODC-facing action synchronous while the HTTP call stays async internally.

3. Normalization

After the float array comes back from Cohere, the library normalizes it to unit length before packing. Both EmbedImage and EmbedText, the two C# embedding actions in MultimodalEmbeddingLibrary, go through the same step.

Because both ingestion and query vectors are unit-normalized identically, dot product at search time equals cosine similarity. No square root. No division. Just multiply-and-sum across the arrays.

If normalization happened on only one path, the scores would be meaningless. Shared normalization is not an optimization. It is a correctness requirement.

4. Packing vectors into binary

ODC stores the vector as Binary Data. The C# layer converts the normalized float[] into a packed byte[] using little-endian float32 encoding. Each float is 4 bytes, so the size is deterministic.

512 dimensions  × 4 = 2048 bytes
1536 dimensions × 4 = 6144 bytes

This is also why model variants cannot be mixed. A 512-dimension vector and a 1536-dimension vector are different shapes. They cannot be compared. The C# layer derives the dimension count at scoring time from queryVector.Length / 4. There is no need to store it as a separate attribute.

5. Memory management

SearchVector scores every candidate in the list. For a small corpus that is trivial. As the candidate count grows, the way memory is handled starts to matter. Allocating a new float[] per candidate and leaving the garbage collector to clean up adds avoidable pressure.

The library rents from ArrayPool<float>.Shared instead. Rent, compute, return. The query vector is unpacked once before the loop, not once per candidate.

Boring search loops are good search loops.

The Two Search Paths

Both paths converge at SearchVector (C# in MultimodalEmbeddingLibrary). That is intentional.

Text-to-image starts with a string. The UI calls SearchByText (ODC App), which calls TextToVector (ODC Library), which calls EmbedText (C#) to produce the query vector. That vector flows into SearchVectors (ODC Library), which calls SearchVector (C#) with the full candidate list. Back comes a ranked list of image IDs and scores.

Text query “Get me a cat with glasses.” One strong match returned.

Image-to-image starts with a photo the user provides. The UI calls SearchByImage (ODC App), which calls ImageToVector (ODC Library), which calls EmbedImage (C#) to produce the query vector. That vector flows into SearchVectors (ODC Library), which calls SearchVector (C#). The excludeId parameter prevents the query image from appearing in its own results if it is already in the corpus. Query images are never persisted. Storing them would pollute the index.

Image match mode. Reference image staged, three visually similar results returned.

SearchVector does not know which path produced the query vector. It just scores vectors.

What Search Looks Like at Runtime

Both paths take different inputs but run the same steps.

Both search paths converge at SearchVector.

Different inputs. Same representation. Same scoring action. Same result shape. The UI and the orchestration do not need two retrieval systems. They only need to know a query vector came in and a ranked list came back.

The Storage Design

Two entities.

Image stores the binary and metadata: Id, FileName, ImageBinary, CreatedDate, UpdatedOn, IsActive. The housekeeping fields handle soft deletes and auditing. This is not a throwaway demo entity.

ImageVector holds the packed embedding linked via ImageId as a foreign key: VectorBytes (Binary Data), ModelVersion (Text, indexed), UpdatedOn, IsActive.

Image and ImageVector entities in ODC Studio.

ModelVersion is indexed because every search query filters by it. It is also the guard against silent breakage when model variants change. A 512-dimension vector and a 1536-dimension vector are different shapes and cannot be scored against each other. Filtering by ModelVersion keeps each search scoped to compatible embeddings.

The SaveAndEmbedImage server action in the ODC App is the single entry point for all image ingestion. It is the only place that writes to both entities. Nothing else in the app writes image binaries or vector bytes directly.

The Ingestion Flow

Ingestion is a one-time operation per image.

Ingestion screen. Image staged, Generate and Store Vector button visible, five images indexed.

The UI calls SaveAndEmbedImage (ODC App), which calls ImageToVector (ODC Library), which calls EmbedImage (C# in MultimodalEmbeddingLibrary). The C# layer detects the MIME type, builds the data URI, calls Cohere, normalizes the float array, and packs it as a byte[]. That binary is returned up through the library layer to ODC orchestration, which stores it in ImageVector.VectorBytes alongside the image record.

Call chain from SaveAndEmbedImage down to EmbedImage and back.

The image and its vector are treated as one indexing operation. If the vector cannot be generated, the image is not indexed. If the image cannot be stored, the vector is not left orphaned.

Honest Limitations

Linear scan does not scale indefinitely. SearchVector (C#) scores every candidate on every query. That is fine for small to medium collections. It will not hold up for very large corpora. I have not benchmarked the exact tipping point. It depends on vector dimension, hardware, and corpus size. At large scale, the right move is a dedicated vector database with approximate nearest neighbor indexing.

One model variant per corpus. Switching variants means re-embedding everything. ModelVersion is the guard, but the re-indexing cost is real.

No retry logic. A transient Cohere API failure throws immediately. Retry logic belongs in ODC orchestration.

Single-image ingestion in Phase 1. Bulk ingestion belongs in a Timer-driven batch pattern, not a single interactive request. That is in the roadmap.

Images stored in ODC entities. Phase 1 stores image binaries directly in the database. That is a deliberate choice for proving the pattern natively. Many ODC apps store images in external storage like S3. Wiring up external image storage is not part of Phase 1.

What This Actually Changes

Every component I built before this assumed text. Chunking, vector storage, semantic retrieval. Text in, text out.

The shared vector space changes that. A text query goes through SearchByText and an image query goes through SearchByImage, but both ultimately call the same SearchVector (C#). The retrieval layer does not know which one it received.

The search interface does not need to know either. A text box and an image upload both produce a query vector and feed the same pipeline. The user’s intent, whether expressed in words or in a photo, gets translated into the same representation.

That is the shift. Not the components. The fact that the representation is shared.

Ingestion gallery on the left, search results on the right. Same corpus, full loop.

What Comes Next

This is Phase 1. Linear scan, native ODC, no external vector storage layer.

The next step is improving candidate selection before scoring. I am looking at IVF-style pruning to reduce the number of vectors scored per query, with the goal of pushing beyond linear scan without immediately reaching for an external vector database. After that, reranking is interesting. So is tiling for images where small visual details matter. Eventually, agentic image search. I do not know yet which ships first. I will figure that out as I build.

The Forge component is available now.

If your current image search story is filenames, manual tags, or category filters, this is the gap I built it for. I would like to hear what kind of image search problem you are trying to solve.