The Problem

• Healthcare workers in remote areas have zero reliable Wi-Fi.
• Clinical note-taking steals time from patient care.
• Existing AI scribes send sensitive data to the cloud — a privacy & compliance risk.

What if the entire AI pipeline ran on a single phone, fully offline?

The Solution: Scribend

A medical scribe that runs entirely on a Snapdragon phone:

• 🎙️ Listen — capture messy room audio
• 📝 Transcribe — speech → text (Whisper)
• 🧠 Remember — find relevant past notes (vector search)
• 📋 Write — structured SOAP note as JSON (Llama)

All on-device → private, fast, works with no signal.

On-Device Architecture

All models quantized to INT8, exported with ExecuTorch, and run on the Snapdragon NPU.

Models & Tooling — How We Chose

• 🐝 Hugging Face for the LLMs & embedder — Llama-3.2, Qwen, SmolLM2 & all-MiniLM-L6-v2.
• 🧩 Qualcomm AI Hub to source our Whisper model, pre-optimized for the Snapdragon NPU.
• 🔬 Experimented across many models to test accuracy (model_evaluation/):
  • Speech: Whisper tiny / base / small → small.en won.
  • SOAP notes: Llama 1B/3B, Qwen 1.5B/3B, SmolLM2 1.7B → Llama-3.2-1B best on-device.
• ⚙️ ExecuTorch (PyTorch Edge) to quantize → .pte and run on-device.

My Role — Developer 3: Local Storage

The on-device "memory" of the app.

Mission: compile a vector database for Android, design the schema, and build cosine-similarity search over patient history — fast, and leak-free.

Tech: SQLite + sqlite-vec, compiled with the Android NDK for arm64-v8a.

Why Vector Search?

• An AI model turns a sentence into 384 numbers (an embedding).
• Similar meaning → similar numbers.
• "Find relevant notes" becomes "find the closest vectors" — by meaning, not keywords.

Cosine distance measures the angle between vectors:

0.0 = same meaning … 1.0 = unrelated

The Database Design

CREATE VIRTUAL TABLE patient_vectors USING vec0(
    patient_id   INTEGER PARTITION KEY,   -- search scoped per patient
    encounter_id INTEGER,                 -- links a hit to its visit
    embedding    FLOAT[384] distance_metric=cosine,
    +chunk_text  TEXT                     -- original note, returned with the hit
);

• Plain SQL tables for patients, encounters, vitals
• Partition key → fast and keeps patients isolated (privacy)

It Works — Real Embeddings, Real Clustering

20 real MiniLM vectors — search groups them by meaning, with no shared keywords:

→ Respiratory, cardiac, diabetic & sleep notes self-organize — validated on real embeddings.

Proven on Real Hardware 📱

Ran natively on the Galaxy S25 Ultra (Snapdragon 8 Elite, arm64-v8a, Android 15):

RUNNING ON: samsung SM-S938U1 · arm64-v8a · Android 15
✓ open DB   ✓ apply schema   ✓ insert 384-dim vectors
✓ cosine KNN (nearest = 0.000000)   ✓ patient isolation
ALL PASSED

✅ Verified on-device · ✅ No memory leaks · ✅ Sub-millisecond search

Engineering Highlights

• Self-contained & offline: vendored sqlite-vec + SQLite, statically linked → no runtime downloads.
• Clean C API (open / insert / search / close) → teammates integrate without touching SQLite.
• Built for both laptop (fast tests) and phone (NDK cross-compile) from one CMake config.
• Tested at every level: C unit test, Python suite, 5k-vector stress, on-device run.

What Each Developer Built

"No-Waiting" strategy — everyone coded against fillers (stubs), then integrated at the "Big Swap".

The Big Swap: real vectors, models & JNI replaced the fillers — pre-agreed interfaces meant it just worked.

How We Shipped in 48 Hours 🤖

• 🧠 Claude Opus 4.5 — paired on the hard parts: NDK cross-compile, sqlite-vec C API, and debugging on-device crashes.
• 🐙 GitHub Copilot — in-editor autocomplete and boilerplate across C, Python & Kotlin.
• ⚡ Result: 4 devs moved like 8 — more time for real model evaluation and on-device validation, less on plumbing.

Roadmap / Next Steps

• 🔌 Swap random vectors for real MiniLM embeddings (Dev 2)
• 🔗 Wire the C API into the JNI layer (Dev 1)
• 🖥️ Connect to the Compose UI (Dev 4)
• 📈 Benchmark at clinic scale (tens of thousands of notes)