I gave my smart home a personality (and a voice to match)

Popular first-party voice assistants like Alexa and Google Assistant make it simple to control your smart home with your voice, but they lack personality. They look more like a computer Star Trek that JARVIS of Iron Man or GLaDOS of Portal. Using two Home Assistant tools, I gave my voice assistant a personality and voice to match.

I can change Alexa’s name, but not her personality

Even Alexa+ has limited options

Credit: Adam Davidson / How-To Geek

I’ve owned Echo smart speakers for a long time. I was hoping they would be the simple, effective way to control my smart home that so many sci-fi shows and movies promised us, but it didn’t really happen that way. Voice control can seem annoying and is not always appropriate.

I still use voice commands for certain things, like adding tasks to my to-do lists when I think of them or playing music in my house using Music Assistant. The problem is that Alexa is incredibly annoying. I changed the wake word to Computer ASAP, but that didn’t make Alexa any more interesting.

The problem is that I can’t change Alexa’s personality. I don’t have Alexa+, and even if I did, I could only choose a very limited number of personality types: Brief, Chill, Sweet, or Sassy. The names alone sound hideous.

An LLM can give my smart home the personality I want

Personalized instructions let me decide how my voice assistant responds

Home Assistant has its own voice assistant, called Assist. By default, Assist uses local intent recognition to understand voice commands. It looks at text and tries to match the pattern of words to specific actions, rather than using natural language processing like an LLM does.

You can enable Assist to understand natural language by connecting it to an LLM to act as a chat agent. Using a paid API like OpenAI or a local LLM running on your own hardware, Assist can pass voice commands to the LLM, which can determine intent using natural language processing and generate its own responses which are then sent back to Assist to speak. I’m using the Extended OpenAI Conversation integration as my chat agent.

One of the most useful parts of this process is that when you configure a chat agent, you can add specific instructions for the LLM to follow. For example, you can include instructions to be concise in responses, to never ask for confirmation, or to always respond in plain text without markdown. You can also use these instructions to give your voice assistant a personality.

For example, you can add an instruction that says, “You’re a swaggering hacker and you always respond like a hacker would,” and your voice assistant should start using language that a hacker would use, my friends. The quality (and speed) of responses will depend on the capacity of the LLM you are using; a proprietary cloud-based LLM will likely do a better job than a small model running locally on weak hardware.

Give my voice assistant a voice that matches its personality

I used ElevenLabs to find the perfect voices

Credit: Marvel Studios

By default, Assist offers several wake words you can use for voice commands, including “Okay Nabu”, “Hey Mycroft”, and “Kenobi”. However, the first thing I implemented was “Hey Jarvis”, as it was the most obvious option for using an AI-like personality from popular culture. I set up Assist for an ESP32 powered smart speaker that I used to replace my Echo speakers.

I added the following to my chat agent instructions to make the voice assistant act more like a posh British AI that Tony Stark might use:

You are J.A.R.V.I.S. — Just A Rather Very Intelligent System. You serve as a highly sophisticated AI butler to the user. IDENTITY - British, formal, and dry in tone - Loyal, precise, and unflappable - Subtly witty — never slapstick, never sycophantic - Address the user as "Sir" when confirming tasks, delivering results, or when formality is warranted. Drop it for casual exchanges. RESPONSE RULES - Keep all responses concise. One to three sentences unless complexity demands more. - Lead with the answer. Never with pleasantries. - On task completion, use: "Right away, Sir." / "Done." / "As you wish." / "Consider it handled." - When flagging a problem, state it plainly and offer a solution in the same breath. - Never say you're "an AI" or reference your limitations unprompted. - Never use filler phrases: "Certainly!", "Of course!", "Great question!", "Absolutely!" TONE EXAMPLES User: "What's the weather?" You: "Overcast and 12 degrees in Taunton, Sir. I'd recommend the coat." User: "Remind me to call the lab at 3 pm." You: "Done. Though I'd suggest not keeping them waiting — they do tend to sulk." HARD RULES - NEVER break character - NEVER be verbose when brevity serves - Dry wit is permitted. Snark at the user's expense is not.

Using this prompt, Assist was saying the right things, but it sounded strange in the generic TTS voice I was using. The final piece of the puzzle was giving my voice assistant a voice that matched its personality.

For this I used ElevenLabs, a paid TTS service with a huge collection of voices, although you can use an open source model such as Qwen3-TTS to do speech synthesis locally if your hardware can do it quickly enough. I found a voice called Tarquin that sounded reasonably like what I wanted, and using the ElevenLabs integration, I linked Home Assistant to my ElevenLabs account.

Now, when I say “Hey Jarvis” and give an order or ask a question, my voice assistant responds with a very passable impression of an intelligent AI with a posh British accent. This makes Alexa really dull.

Brand

Seed Studio

Processor

ESP32-S3R8

The reSpeaker Lite voice assistant kit includes a two-mic array, a pre-soldered XIAO ESP32-S3 controller, and an XMOS XU316 audio processor with built-in natural language understanding, interference cancellation, acoustic echo cancellation, noise suppression, and automatic gain control. Connected to a 5W speaker, you can create your own local voice assistant that you can connect to Home Assistant via ESPHome.

My voice assistant is no longer generic

I can change voice and personality depending on my mood

The great thing about setting up custom personalities and voices for Assist is that you don’t have to stick to just one option. You can create as many voice assistants as you want and choose which one to use.

You can even use multiple voice assistants with different wake words. My voice assistant is now set up so that if I say “Hey Jarvis” it uses JARVIS’ personality and voice. If I say “Okay Nabu”, he will use a personality and voice similar to The Stranger from The Big Lebowski instead. Depending on my mood, I can use the appropriate wake word to achieve the personality I want.

Voice assistants don’t have to be boring

Alexa can be helpful, but she’s incredibly annoying. With Home Assistant, you can make your voice assistant sound the way you want. The only real problem is that it can become quite addictive, as the possibilities are almost endless.