I Asked My Home AI Agent About Amazon Orders

I spent some time wiring a stupidly fun experiment at home.

The question was simple:

Where is my Amazon order?

I wanted to ask this out loud and get a spoken answer back from my home AI agent.

The setup was a bit of a Frankenstein:

• Home Assistant Voice Preview for voice input
• Hermes for routing the request
• a shopping skill for order lookup (Amazon is the test case, the skill started from Shopify experiments)
• ElevenLabs for text-to-speech

It worked.

The funny part: “AI understands shopping” was the boring bit. The useful bit was that I did not have to open anything.

I asked a question in the room. The agent checked the order status. The answer came back in the room.

This is the kind of AI interaction I want more of at home.

For coding, writing, planning, or debugging, I still want text. I want history, copy-paste, logs, diffs, and all the usual nerd comfort.

For tiny household questions, text boxes are annoying. I do not want to unlock the phone, open an app, search through orders, read the tracking status, and close everything again. I want to ask the thing and move on.

The shape of the skill

The skill itself has to be narrow.

This is important. I do not want a voice agent that tries to be clever about everything. I want a small skill that knows how to answer one class of question well.

For an order status question, a good answer is short:

Your order is arriving tomorrow. It was shipped yesterday and is currently in Leipzig.

That is enough.

If the agent starts reading me a full tracking page, the UX is already dead.

The rough edges

This is still very experimental.

The integration stack is absolutely not something I would recommend to a normal person yet. Home Assistant Voice Preview is promising, but it is still preview software. Hermes needs careful wiring. TTS quality matters more than I expected, because bad speech makes the whole thing feel broken even when the answer is correct.

There is also a big difference between “works once” and “I trust this enough to use daily”.

The boring details matter:

• What happens if the order list is empty?
• What happens if there are three active orders?
• How much should the agent say out loud?
• Should it mention prices?
• Should it require confirmation before reading personal details?

Voice makes privacy questions much more concrete. A text answer sits on a screen. A spoken answer is broadcast into the room. That changes the design.

Why I liked it

Because it felt boring in the right way.

It was a small loop across real systems:

voice input, message routing, skill execution, order lookup, spoken response.

That is where personal agents become useful for me: a collection of small skills close to the systems I already use.

The experiment is rough. I will probably break it three more times before it becomes reliable.

Still, asking the room about an order and getting a useful answer back felt like a glimpse of the right interface.