Local · Private · No keys

Ask anything.

Nippy talks straight to llama-server on your device. Nothing leaves the box.

Termux setup · Pixel

Runs entirely on the phone. Model downloads, weights on-device, no keys, no proxy. Order below is the working path on Pixel 10 Pro XL / Android 15.

1 · Install Termux

Install Termux from F-Droid, not the Play Store version. Then update the base and grab build tools.

pkg update -y && pkg upgrade -y
pkg install -y clang cmake git wget curl python make

2 · Build llama.cpp

llama.cpp’s llama-server is the local OpenAI-compatible endpoint Nippy talks to.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_LLAMAFILE=OFF
cmake --build build --config Release -j

First build takes 10–15 min on Pixel 10. Binary lands at build/bin/llama-server.

3 · Download a model (GGUF)

Pick one alias from the roster. All under 3 GB except Fanar; MedGemma is licence-gated (accept terms on the model page).

mkdir -p ~/models && cd ~/models
wget -O granite.gguf \
  https://huggingface.co/bartowski/ibm-granite_granite-4.0-micro-GGUF/resolve/main/ibm-granite_granite-4.0-micro-Q4_K_M.gguf

Other roster URLs

# Qwen3-4B Instruct 2507
wget -O qwen3-4b.gguf \
  https://huggingface.co/bartowski/Qwen_Qwen3-4B-Instruct-2507-GGUF/resolve/main/Qwen_Qwen3-4B-Instruct-2507-Q4_K_M.gguf

# Phi-4 mini
wget -O phi4-mini.gguf \
  https://huggingface.co/bartowski/microsoft_Phi-4-mini-instruct-GGUF/resolve/main/microsoft_Phi-4-mini-instruct-Q4_K_M.gguf

# Gemma 3 1B
wget -O gemma3-1b.gguf \
  https://huggingface.co/bartowski/google_gemma-3-1b-it-GGUF/resolve/main/google_gemma-3-1b-it-Q4_K_M.gguf

# SmolLM3 3B
wget -O smollm3.gguf \
  https://huggingface.co/bartowski/HuggingFaceTB_SmolLM3-3B-GGUF/resolve/main/HuggingFaceTB_SmolLM3-3B-Q4_K_M.gguf

# MedGemma 4B (accept licence on HF first)
wget -O medgemma.gguf \
  https://huggingface.co/bartowski/google_medgemma-4b-it-GGUF/resolve/main/google_medgemma-4b-it-Q4_K_M.gguf

# BitNet b1.58 2B
wget -O bitnet.gguf \
  https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-gguf/resolve/main/ggml-model-i2_s.gguf

# Fanar-1-9B (heaviest; ~5 GB)
wget -O fanar.gguf \
  https://huggingface.co/QCRI/Fanar-1-9B-Instruct-GGUF/resolve/main/fanar-1-9b-instruct-Q4_K_M.gguf

4 · Serve on `127.0.0.1:8080`

Point llama-server at the model file. Nippy talks to this endpoint.

~/llama.cpp/build/bin/llama-server \
  -m ~/models/granite.gguf \
  --host 127.0.0.1 --port 8080 \
  -c 4096 -t 6

Leave that running. Open a new Termux tab (Ctrl-Alt-N) for anything else.

5 · Wire Nippy

In Settings above:

Endpoint → http://127.0.0.1:8080/v1
Model → tap a roster card, or type the alias by hand
Stream responses → on

Status pill should flip from probing to ok within a couple of seconds. If it stays off, check llama-server is still running and the port matches.

Notes. Weights stay on the device. No telemetry, no proxy. Battery drain on inference is real — expect fan-less phones to warm up under sustained streaming. For long sessions, plug in.