THE FIELD GUIDES

How to Build Your Own Jarvis System

Build your own offline Jarvis on a Windows PC: it listens to your voice, thinks with a local AI model, and talks back — no cloud, no API key. Open Terminal or PowerShell from the Start menu and paste one line at a time. More setups are on the guides page.

How it works (4 parts)

A voice assistant is just four pieces wired together: ears (speech-to-text), a brain (a local AI model), a voice (text-to-speech), and a little script that passes your words between them. You install the four pieces, then run the script.

1. The basics: Python and Git

STEP 1
Install Python and Git
winget install --id Python.Python.3.12 -e
winget install --id Git.Git -e
Close and reopen Terminal afterwards so python and pip are found.
STEP 2
Check they work
python --version
pip --version

2. The brain: Ollama + a local model

STEP 1
Install Ollama
winget install --id Ollama.Ollama -e
Open Ollama once from the Start menu so it starts running in the tray.
STEP 2
Download a model
A 3B model is a good balance for a PC. If your PC is older, use llama3.2:1b:
ollama pull llama3.2:3b
Test it talks back:
ollama run llama3.2:3b "Say hello like Jarvis would."

3. The ears: Whisper (speech-to-text)

STEP 1
Install Whisper and mic tools
This turns what you say into text, all on your PC:
pip install faster-whisper sounddevice scipy

The first time the script runs, Whisper downloads its model (a few hundred MB). After that it works offline.

4. The voice: Piper (text-to-speech)

STEP 1
Install Piper
pip install piper-tts
STEP 2
Download a voice
Grab one voice file and its config (this is the Jarvis-style English voice):
curl -L -o en_US-lessac-medium.onnx https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx
curl -L -o en_US-lessac-medium.onnx.json https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.json
Test the voice:
echo At your service. | piper -m en_US-lessac-medium.onnx -f test.wav

5. Wire it together: the Jarvis script

STEP 1
Install the Ollama Python library
pip install ollama
STEP 2
Save this as jarvis.py
Make a file called jarvis.py in the same folder as your voice files and paste this in. It records 5 seconds, writes down what you said, asks the model, and speaks the answer:
import subprocess, sys import sounddevice as sd from scipy.io.wavfile import write from faster_whisper import WhisperModel import ollama # 1. Record 5 seconds from your mic fs = 16000 print("Listening...") rec = sd.rec(int(5 * fs), samplerate=fs, channels=1) sd.wait() write("in.wav", fs, rec) # 2. Speech -> text (Whisper, runs on your PC) stt = WhisperModel("base.en", device="cpu", compute_type="int8") segments, _ = stt.transcribe("in.wav") you = " ".join(s.text for s in segments).strip() print("You:", you) if not you: sys.exit() # 3. Think (local model via Ollama) reply = ollama.chat(model="llama3.2:3b", messages=[ {"role": "system", "content": "You are Jarvis, a concise, helpful assistant."}, {"role": "user", "content": you}, ])["message"]["content"] print("Jarvis:", reply) # 4. Text -> speech (Piper) and play it with open("out.txt", "w", encoding="utf-8") as f: f.write(reply) subprocess.run('type out.txt | piper -m en_US-lessac-medium.onnx -f out.wav', shell=True) subprocess.run('powershell -c (New-Object Media.SoundPlayer "out.wav").PlaySync()', shell=True)
STEP 3
Run Jarvis
Make sure Ollama is open, then:
python jarvis.py
When it prints Listening..., say something. It will print what you said, print Jarvis's reply, and speak it out loud.

Make it better (optional)

Once the basic loop works, these are the usual next steps:

STEP 1
Always listening (wake word)
Add a wake word like "Jarvis" so it only answers when called:
pip install openwakeword
STEP 2
Let it do things on your PC
Have the model return a command and run it (open apps, search files, control music). Start small and always print the command before running it, so you can see what it will do.
STEP 3
Give it a personality
Change the system line in the script to set its tone — formal butler, gamer sidekick, whatever you like. That one line is your Jarvis's character.

Heads up: a model that runs commands on your PC can do real damage if it misreads you. Keep a confirmation step before anything that deletes, installs, or sends — read what it wants to do first.

Stuck on a step? Send a message and say which one.