pumTALK

Getting started

Speak and it replies out loud. Or type in the box at the bottom. Two modes (side panel → Mode):

• Listening — hands-free. Just talk; a pause ends your turn.
• Push-to-talk — hold the mic button (or Space) to talk.

The pipeline

Your voice flows through five stages (shown in the bar at the top, and as faces in Voice mode):

Mic → Whisper → LLM → Voice → Speaker

• Mic — captures your speech (energy VAD detects when you talk).
• Whisper — transcribes speech to text (faster-whisper, GPU with CPU fallback).
• LLM — Ollama or LM Studio generates the reply (streaming).
• Voice — Piper, or your cloned F5-TTS/XTTS voice.
• Speaker — plays it back.

Voice cloning (your own voice)

Click + next to Voice, name it, and record 15s of natural sentences. It's cloned with F5-TTS (most accurate) or XTTS (faster).

• Speak naturally — don't count or read lists (that causes "bleed" of the reference words).
• After recording, check/edit the reference text so it matches exactly what you said.
• Voice adherence (Generation panel) — higher = sticks to the target text (stops the bleed), lower = more expressive.

Models & LLM

• Backend — pick Ollama or LM Studio, set the server URL and API key.
• Download model — pull any Ollama model by name, or a GGUF URL for LM Studio (with live % and speed). HF token supports gated models.
• Eject (header) unloads the model from GPU; trash deletes it.
• Generation panel — temperature, top-p/k, penalties, max tokens, context.
• Prompt panel — system prompt + stop sequences.

Built-in tools

Turn on Web search in the Tools panel to activate them. The model calls the right tool automatically — you can also just ask normally:

• search_web — "search the web for X" (SearXNG).
• ddg_answer — quick facts, e.g. "who won the 1998 World Cup?"
• fetch_url — "open and summarise this page: URL" (reads a link).
• get_weather — "what's the weather in Paris?"
• calculator — "what is sqrt(256) + 7*3?" (safe math).
• youtube_transcript — "summarise this video: URL".
• arxiv_search — "find papers on quantum computing".
• query_wikidata — factual SPARQL lookups.
• get_current_time — "what time is it?" / "what's today's date?"
• send_email — enable in Settings (SMTP), then "email Bob the following…".

Microphone & camera

• Pick the right microphone; the meter's yellow line is the speech threshold — if your voice doesn't cross it, raise Sensitivity.
• The mic pauses while the AI talks, so it won't hear itself.
• Camera toggles the webcam preview (and your face in Voice mode).

Controls & shortcuts

• Stop button or Esc — interrupt the AI.
• Chat / Voice toggle — switch between text chat and the face view.
• Barge-in — talk loudly to interrupt (when enabled).
• Space — push-to-talk (in Push-to-talk mode).

Desktop control (PC)

Turn on Settings → Desktop control and pumTALK can use your computer for you:

• Commands — run terminal commands, open apps and URLs, close apps.
• Files — list, read, write, move, rename, delete, search.
• Screen & input — screenshot, read the screen (OCR), type, click, press keys.
• System — CPU/RAM/disk/battery, volume, brightness, clipboard.
• Risky actions always ask first — pick Trust once, Trust always, or Cancel.

Remote access (internet)

By default pumTALK only listens on your own machine. To reach it from anywhere, run it behind a reverse proxy (Nginx Proxy Manager, Caddy, etc.):

• Start with PUMTALK_HOST=0.0.0.0 PUMTALK_TOKEN=your-secret ./start.sh.
• Point your proxy at http://localhost:8000, enable WebSockets (upgrade headers), and force HTTPS.
• Open https://your-domain/?token=your-secret — the token is remembered.
• The mic, camera and speaker run in your browser, so you can talk to your home PC from any device.