The problem
We're a military family: my wife serves in the Air Force, we have four kids, and we move countries every few years. The day lived in five places at once: a paper calendar, two phones, a group chat and my head. Chores got forgotten, pickups got double-booked, and "what's for dinner?" got asked thirty times a day.
Off-the-shelf apps didn't fit. They charge monthly, keep our family's data in someone else's cloud, and none of them work for a three-year-old and a thirty-six-year-old on the same screen. So I treated my family as the customer and wrote down what they actually needed:
- One shared picture of the day: schedule, chores, meals and trips, the same on every screen.
- Usable by everyone: big tap targets for the little ones, voice for anyone, no logins at home.
- Private by default: audio, video and the AI stay on our own network, with a one-tap guest mode.
- Keeps working when parts fail: if the AI is busy or down, the house still runs and nothing is lost.
- Portable: it has to move with us to the next country.
What I built
Family web app
Calendar, chore boards, meals, groceries and trips on a PHP/MySQL app with a REST API that the other services use too.
Kiosks
Touchscreens in several rooms that stay in sync within seconds. Tap your name for your day; check off chores; guest mode hides everyone's names.
Jake
A voice assistant anyone can talk to, on any screen or page: "Hey Jake, what's for dinner?" He knows the family's real schedule and what each person is allowed to change.
Household engine
A Python service that sends reminders and daily digests, keeps chore lists fresh each morning, and nudges me to study every night.
Home automation
Home Assistant knows who's home and drives guest mode; a separate camera node runs the video analytics.
The lab
A Proxmox host I built (Ryzen 9 7950X3D, RTX 4090), with VMs and Docker containers, reachable privately over Tailscale.
Architecture
Family data, audio, video and the AI never leave the house network. Only the public web apps (like this site) go out through the tunnel, and they carry no family data.
Jake, the AI everyone talks to
Jake is one AI with one personality on every surface: kiosks, the family site, a phone voice page, a desktop companion and a Discord secretary. Before every answer he gets a live briefing of the family's real day, so he answers from facts, not guesses.
- Prompting beat fine-tuning. I fine-tuned two models to "be Jake"; both came out less useful than the stock model. I kept the smartest model the GPU can run and built the personality with prompts. Real intelligence beat perfect character accuracy.
- Permissions by age. Parents can change the calendar and dinner plan (with a spoken "yes" first); kids can check off their own chores; food requests from the little ones go to a parent for approval.
- Privacy you can hear. The wake word is checked in memory and thrown away until someone says "Hey Jake." Guest mode hides family details everywhere at once.
- Graceful when busy. The GPU is shared with other workloads. When Jake is "resting," messages queue up and get answered when he's back, and everything else keeps working.
- Kids get the whole answer. When a child taps their name, the screen stops listening until Jake finishes their morning or evening rundown, so the check-in actually lands.
Running it like production
My family is a demanding customer: if the screen on the wall is wrong at 7 a.m., I hear about it. So it runs like a production system:
- Three layers of backup: nightly images of every VM, a nightly off-site copy of the data to cloud storage, and code pushed daily to private GitHub repos. I've restored from them to prove they work.
- Tested every night: an automated suite of about 50 checks grades Jake's answers against the real family data, and browser tests use a simulated microphone to check "Hey Jake" end to end.
- Written down: a system map, deploy runbooks, rollback steps and an incident log. Every change is backed up first and can be undone.
- A real migration: I moved the whole platform off paid web hosting onto my own servers in one evening with zero data loss, verified table by table. I repointed every dependent service, retired the old host behind a notice page, and scrubbed old secrets out of git history before publishing the code.
What it taught me
Discovery beats features. The best changes came from watching my kids use it, like holding the microphone until Jake finishes their rundown, not from my to-do list.
Every design is a trade-off you can explain. One GPU for three workloads, local versus cloud, fine-tuning versus prompting. I can defend each choice to someone technical, and explain it to someone who isn't.
Reliability is a feature. Backups, tests and runbooks aren't overhead. They're why my family trusts it.
I build much of the code by directing AI coding agents, and I own the architecture, the review, the testing and the change control. It's the way a lot of software will be built from now on, and I'm fluent in it.