Describe a product in your own words, or show a photo. The model finds the closest matches among 121,549 real Amazon listings. Everything runs inside your browser: no server, nothing you type or upload leaves your device.
Starting…
How it works
- Your words or photo become a vector.A CLIP model turns text and images into 512 numbers that live in the same space, so "red sneaker" and a photo of one land close together.
- The model was fine-tuned on Amazon listings.Trained with a contrastive loss on 109,395 product photo–title pairs from the Amazon Berkeley Objects dataset, so it learned how products are actually described.
- FAISS finds the nearest products.Every catalog photo is pre-embedded into a compressed IVF-PQ index. A query is compared against all 121,549 products in milliseconds.
- It all runs in your browser.The CLIP encoders are exported to ONNX and quantized to INT8, then run with ONNX Runtime Web (WebAssembly). The IVF-PQ index ships as ~6 MB of plain arrays and is scanned exhaustively in JavaScript. The same model is also served as a Dockerised FastAPI API on AWS EC2.
Results
| Text → image, held-out test | Before fine-tuning | After |
|---|---|---|
| Recall@1 | 41.6% | 82.4% |
| Recall@5 | 65.8% | 96.2% |
| Recall@10 | 74.2% | 98.0% |
| API latency, p95 (AWS EC2) | 79 ms on 2 CPU cores | |
| Search in this browser | measured on your first search | |
Recall@1 is how often the exact product described is the first result, measured on 6,077 products the model never saw in training.