I've been running local LLMs for a while but wanted to try something different — local embeddings as a practical daily tool.

Sift is a Chrome extension that loads `EmbeddingGemma-300M` (q4) via `Transformers.js` and scores every item in your HN, Reddit, and X feeds against categories you pick. Low-relevance posts get dimmed, high-relevance ones stay vivid. All inference happens in the browser — nothing leaves your machine.

Technical details:

* Model: `google/embeddinggemma-300m`, exported to ONNX via optimum with the full sentence-transformers pipeline (Transformer + Pooling + Dense + Normalize) as a single graph
* Quantization: int8 (onnxruntime), q4 via MatMulNBits (block\_size=32, symmetric), plus a separate no-GatherElements variant for WebGPU
* Runtime: Transformers.js v4 in a Chrome MV3 service worker. WebGPU when available, WASM fallback
* Scoring: `cosine` similarity against category anchor embeddings, 25 built-in categories

The part I'm most happy with — the fine-tuning loop:

1. Browse normally, thumbs up/down items you like or don't care about
2. Export labels as anchor/positive/negative triplet CSV
3. Fine-tune with the included Python script or a free Colab notebook (MultipleNegativesRankingLoss via sentence-transformers)
4. ONNX export produces 4 variants: fp32, int8, q4 (WASM), q4-no-gather (WebGPU)
5. Push to HuggingFace Hub or serve locally, reload in extension

The fine-tuned model weights contain only numerical parameters — no training data or labels baked in.

What I learned:

* `torch.onnx.export()` doesn't work with Gemma3's sliding window attention (custom autograd + vmap break tracing). Had to use optimum's main\_export with library\_name='sentence\_transformers'
* WebGPU needs the GatherElements-free ONNX variant or it silently fails
* Chrome MV3 service workers only need wasm-unsafe-eval in CSP for WASM — no offscreen documents or sandbox iframes

Open source (Apache-2.0): [https://github.com/shreyaskarnik/Sift](https://github.com/shreyaskarnik/Sift)

Happy to answer questions about the ONNX export pipeline or the browser inference setup.

Built a Chrome extension that runs EmbeddingGemma-300M (q4) in-browser to score HN/Reddit/X feeds — no backend, full fine-tuning loop