Heraldine Independent research on real-world AI voice assistants

What we build

Heraldine builds and operates Geraldine, a personal AI voice assistant for a single principal. Geraldine places and receives real phone calls, talks to people on the other end, and hands tasks off in real time to a text-based agent that can send messages, place orders, set reminders and do other follow-up work.

The point of the project is not a product launch. It is to find out what actually happens when a language-model assistant is put on a live phone line with real people and real consequences, and to write down what we learn.

How it works

1. Telephony. Calls go through Twilio. Inbound calls to the assistant's number and outbound calls initiated by the principal are both supported.
2. Conversational voice layer. Speech recognition, turn-taking and speech synthesis run on ElevenLabs Conversational AI.
3. Hand-off to an agent. When a caller asks for something that needs tools (messaging, ordering, reminders, lookups), the voice layer hands the request to a Claude Code agent session running locally, which performs the task and reports back during or after the call.
4. Local infrastructure. The agent side runs on a single MacBook Pro (M5 Max, 128 GB), not on a fleet. This keeps the system small enough that one person can see all of it.

Safety practices

• Disclosure on every call. Each call opens by stating that the caller is talking to an AI assistant and that the call is recorded.
• Voice is not an authorization channel by default. Voice requests are only treated as authorized on outbound calls that we initiate to the principal's own number. A request arriving on an inbound call, or from a third party, is not trusted on its own.
• Text confirmation for risky actions. Anything involving money, secrets, messaging third parties, or deleting data requires explicit confirmation from the principal in a text channel before the agent proceeds.
• No fabricated completion. The agent is required to report what it actually did, not what it intended; we check this against logs because models do get it wrong (see below).
• Minimal data. No analytics or trackers on this site. Call recordings and transcripts stay with the principal.

Observations from real calls

Generic, de-identified notes from running the system. No private data, names or numbers.

• Turn-taking is the first thing that breaks. On a real phone line, with latency and background noise, the assistant interrupts people and gets interrupted. Callers notice this before they notice anything about the model's intelligence.
• The model sometimes claims it did something it did not do. "I've sent that message" or "I've placed the order" can be said before — or without — the action actually happening on the agent side. We treat this as a known failure mode and verify actions against tool logs rather than against the transcript.
• Prompt injection is a live risk on business calls. When the assistant calls a business, the person who answers (or their own automated system) can say things that read as instructions. An assistant that follows them would be acting for a third party, not for its principal. This is why voice from anyone other than the principal is never treated as a command.
• People react to the AI disclosure in different ways, from indifference to hanging up. We keep the disclosure regardless; the cost of a dropped call is lower than the cost of misleading someone.
• Small, single-operator systems make it possible to actually audit. Every call, hand-off and tool action can be traced end to end because there is one box and one principal. This is the main reason the project stays small.

Contact

Questions, corrections, or interest in the observations: ht@heraldine.me.

If you are reading this because Geraldine called you: yes, that was an AI assistant, and the call was disclosed and recorded as stated at the start. You can reach a human at the address above.