THE FIELD GUIDES
How to Build Your Own Jarvis System
Build your own offline Jarvis on a Windows PC: it listens to your voice, thinks with a local AI model, and talks back — no cloud, no API key. Open Terminal or PowerShell from the Start menu and paste one line at a time. More setups are on the guides page.
How it works (4 parts)
A voice assistant is just four pieces wired together: ears (speech-to-text), a brain (a local AI model), a voice (text-to-speech), and a little script that passes your words between them. You install the four pieces, then run the script.
1. The basics: Python and Git
STEP 1
Install Python and Git
winget install --id Python.Python.3.12 -ewinget install --id Git.Git -epython and pip are found.STEP 2
Check they work
python --versionpip --version2. The brain: Ollama + a local model
STEP 1
Install Ollama
winget install --id Ollama.Ollama -eSTEP 2
Download a model
A 3B model is a good balance for a PC. If your PC is older, use Test it talks back:
llama3.2:1b:ollama pull llama3.2:3bollama run llama3.2:3b "Say hello like Jarvis would."3. The ears: Whisper (speech-to-text)
STEP 1
Install Whisper and mic tools
This turns what you say into text, all on your PC:
pip install faster-whisper sounddevice scipyThe first time the script runs, Whisper downloads its model (a few hundred MB). After that it works offline.
4. The voice: Piper (text-to-speech)
STEP 1
Install Piper
pip install piper-ttsSTEP 2
Download a voice
Grab one voice file and its config (this is the Jarvis-style English voice):Test the voice:
curl -L -o en_US-lessac-medium.onnx https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnxcurl -L -o en_US-lessac-medium.onnx.json https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/medium/en_US-lessac-medium.onnx.jsonecho At your service. | piper -m en_US-lessac-medium.onnx -f test.wav5. Wire it together: the Jarvis script
STEP 1
Install the Ollama Python library
pip install ollamaSTEP 2
Save this as jarvis.py
Make a file called
jarvis.py in the same folder as your voice files and paste this in. It records 5 seconds, writes down what you said, asks the model, and speaks the answer:import subprocess, sys
import sounddevice as sd
from scipy.io.wavfile import write
from faster_whisper import WhisperModel
import ollama
# 1. Record 5 seconds from your mic
fs = 16000
print("Listening...")
rec = sd.rec(int(5 * fs), samplerate=fs, channels=1)
sd.wait()
write("in.wav", fs, rec)
# 2. Speech -> text (Whisper, runs on your PC)
stt = WhisperModel("base.en", device="cpu", compute_type="int8")
segments, _ = stt.transcribe("in.wav")
you = " ".join(s.text for s in segments).strip()
print("You:", you)
if not you:
sys.exit()
# 3. Think (local model via Ollama)
reply = ollama.chat(model="llama3.2:3b", messages=[
{"role": "system", "content": "You are Jarvis, a concise, helpful assistant."},
{"role": "user", "content": you},
])["message"]["content"]
print("Jarvis:", reply)
# 4. Text -> speech (Piper) and play it
with open("out.txt", "w", encoding="utf-8") as f:
f.write(reply)
subprocess.run('type out.txt | piper -m en_US-lessac-medium.onnx -f out.wav', shell=True)
subprocess.run('powershell -c (New-Object Media.SoundPlayer "out.wav").PlaySync()', shell=True)STEP 3
Run Jarvis
Make sure Ollama is open, then:When it prints
python jarvis.pyListening..., say something. It will print what you said, print Jarvis's reply, and speak it out loud.Make it better (optional)
Once the basic loop works, these are the usual next steps:
STEP 1
Always listening (wake word)
Add a wake word like "Jarvis" so it only answers when called:
pip install openwakewordSTEP 2
Let it do things on your PC
Have the model return a command and run it (open apps, search files, control music). Start small and always print the command before running it, so you can see what it will do.
STEP 3
Give it a personality
Change the
system line in the script to set its tone — formal butler, gamer sidekick, whatever you like. That one line is your Jarvis's character.Heads up: a model that runs commands on your PC can do real damage if it misreads you. Keep a confirmation step before anything that deletes, installs, or sends — read what it wants to do first.
Stuck on a step? Send a message and say which one.