A shared test mailbox feels simple until your CI suite runs in parallel, a retry sends a second verification email, or an LLM agent reads the wrong magic link. At that point, the mailbox stops being a convenience and becomes global mutable state.
The more reliable pattern is straightforward: create one inbox per attempt. Not one mailbox for the whole test suite. Not one address per environment. One short-lived, isolated inbox for the single signup, password reset, OTP check, or agent step that needs to receive email.
For email automation, this small architectural change removes most of the ambiguity that makes email-based tests flaky.
A shared mailbox is a hidden coordination problem
Email delivery is asynchronous by design. SMTP systems can queue, retry, and deliver messages later than your test expects. The core SMTP specification, RFC 5321, describes a store-and-forward protocol, not an immediate request-response API. That means your test harness should assume messages may arrive late, arrive more than once, or arrive in a different order than your application actions.
A shared mailbox makes that uncertainty worse because every test, retry, developer run, and agent session reads from the same message pool. Your code must then answer questions it should not have to answer:
- Is this email from my current attempt or a previous failed run?
- Is this the first OTP, the resent OTP, or a delayed duplicate?
- Did another parallel worker already consume the same verification link?
- Is the message safe to show to an LLM agent?
Those questions are not business logic. They are artifacts of an inbox model that was designed for humans, not automation.
What “one inbox per attempt” means
An attempt is the smallest unit of work that may independently trigger an email and be retried. It is not always the same as a test run.
A few examples make the boundary clear:
- A signup form submission that expects one verification email
- A password reset request that expects one reset link
- A magic-link login attempt for one browser session
- An LLM tool call that needs to receive and extract a one-time code
- A QA scenario retry after a timeout or application error
If the action can be retried, it deserves a new inbox. The previous attempt may still receive a delayed email, but that email can no longer contaminate the new attempt because it lands in a different inbox.
This is the core advantage of programmable temp inboxes: the inbox is not a shared account you log into. It is a resource you create, use, observe, and discard.
Shared test mailbox vs one inbox per attempt
The difference is not just cleanliness. It changes the reliability properties of your entire email testing toolchain.
| Concern | Shared test mailbox | One inbox per attempt |
|---|---|---|
| Parallel CI | Workers compete for the same message pool | Each worker reads only its own inbox |
| Retries | Old emails can satisfy new attempts by accident | Retried attempts get fresh inboxes |
| Message matching | Requires broad searches across noisy history | Starts with inbox isolation, then narrow matchers |
| Debugging | Hard to prove which run produced which email | Inbox ID and attempt ID form a clear trace |
| LLM agent safety | Agent may see unrelated or hostile content | Agent receives a minimized, scoped view |
| Cleanup | Requires mailbox sweeping and retention rules | Expire or close the attempt inbox |
| Dedupe | Must dedupe across many unrelated flows | Dedupe within a small, known context |
The shared mailbox approach asks your tests to be clever. The per-attempt inbox approach makes the system less ambiguous.
Why shared mailboxes fail under retries
Retries are where shared mailboxes usually break first.
Imagine a signup test that submits a form, waits 30 seconds, and times out because the verification email is delayed. CI retries the test. The second attempt submits the form again and starts polling the same mailbox. A minute later, the first email finally arrives.
If your matcher says “find the latest email with subject Verify your account,” the second attempt may consume the first attempt’s link. Depending on how your application invalidates tokens, the test may pass incorrectly, fail intermittently, or create a user state that breaks later tests.
Teams often respond by adding more filters: timestamp windows, subject rules, sender checks, regexes, and “pick the most recent matching message.” These help, but they do not fix the root problem. The test is still searching a global pile of messages.
With one inbox per attempt, delayed email from attempt A stays in inbox A. Attempt B only waits on inbox B. The matching problem becomes smaller and more deterministic.
The reference flow for a per-attempt inbox
A reliable email verification flow has five stages.
- Create an inbox for the attempt: Before triggering the application action, create a disposable inbox through an API for email. Store the returned descriptor, including the email address and inbox identifier.
- Use that address in the product flow: Submit the signup, password reset, invitation, or login request with the attempt-specific address.
- Wait inside that inbox only: Prefer real-time webhooks for low-latency delivery, with polling as a bounded fallback when webhooks are unavailable or delayed.
- Extract the minimal artifact: Read structured JSON emails and extract only the OTP, verification URL, or magic link required for the next step.
- Mark consumed and clean up: Record an idempotency key for the artifact, finish the attempt, and let the inbox expire or close according to your lifecycle policy.
In provider-neutral pseudocode, the shape looks like this:
const attempt = {
id: newAttemptId(),
purpose: "signup_verification"
};
const inbox = await emailTool.createInbox({
metadata: { attempt_id: attempt.id }
});
await app.signUp({ email: inbox.email });
const message = await emailTool.waitForMessage({
inbox_id: inbox.id,
deadline_ms: 90_000,
match: {
expected_sender: "[email protected]",
subject_contains: "Verify"
}
});
const verificationUrl = extractVerificationUrl(message);
await consumeOnce({
attempt_id: attempt.id,
artifact: verificationUrl
});
The exact API shape depends on your provider, but the contract should stay the same: create an inbox, wait by inbox ID, receive a machine-readable message, extract the artifact, and make consumption idempotent.
Webhook-first does not replace inbox isolation
Real-time webhooks are the right default for most automated email receipt because they reduce latency and avoid wasteful polling. But webhooks alone do not solve the shared mailbox problem.
If a webhook payload says “a new message arrived,” your handler still needs to know which attempt owns that message. In a shared mailbox, ownership is inferred from brittle content matching. In a per-attempt model, ownership is explicit because the message is delivered for a specific inbox.
A production-ready webhook handler should still be defensive:
- Verify signed payloads before parsing or processing the message
- Route by stable identifiers such as inbox ID and delivery ID
- Acknowledge quickly, then process asynchronously when needed
- Dedupe at the delivery, message, and artifact levels
- Keep polling available as a fallback for recovery and debugging
Mailhook supports this style directly with disposable inbox creation via API, structured JSON email output, real-time webhook notifications, a polling API for emails, and signed payloads for security. For exact integration semantics, use the canonical Mailhook llms.txt reference.
Why this matters even more for LLM agents
LLM agents amplify the risks of shared test mailboxes because agents are designed to act on context. If the context contains unrelated emails, stale links, forwarded messages, or malicious instructions inside an email body, the agent may take the wrong action.
An agent should not browse a general-purpose mailbox. It should call a small tool with a narrow contract, such as “wait for the verification email for this attempt” or “extract the OTP from this inbox.” The tool should return a minimized result, not the entire mailbox history.
One inbox per attempt gives agent workflows a safer boundary:
- The agent cannot accidentally select a code from another run
- Prompt-injection content from unrelated messages is excluded
- The tool can hide raw HTML and expose only typed artifacts
- Retry loops can be bounded per attempt
- Logs can reference inbox IDs and delivery IDs without dumping sensitive content
This does not eliminate the need for security controls. Inbound email should still be treated as untrusted input. Links should be validated before use, webhook authenticity should be verified, and agents should receive the smallest useful view of the message. But inbox isolation makes every other control easier to enforce.
The real cost of shared mailboxes
Teams often keep shared mailboxes because they appear cheaper. There is one account, one password, and one place to look. But the cost moves into the test harness.
Shared mailboxes require complex matchers, mailbox cleanup jobs, IMAP or UI login logic, flaky sleeps, manual debugging, and special handling for parallel runs. They also create security problems when credentials are stored in CI or exposed to agent tooling.
A temporary email API flips the model. Instead of centralizing all messages in one human mailbox, it gives automation a disposable resource with a clear lifecycle. The harness becomes simpler because it no longer has to defend against unrelated history.
The operational question is not “how many inboxes are we creating?” It is “how much ambiguity are we forcing every test and agent to resolve?”
How to migrate away from a shared test mailbox
You do not need to rewrite every email-dependent test at once. Start by putting a provider-neutral abstraction in front of mailbox access.
-
Create an inbox descriptor type: Store
email,inbox_id,attempt_id, timestamps, and any metadata your harness needs. Avoid passing around a bare email string when the inbox ID is the real handle. -
Wrap message retrieval: Replace global mailbox search with a function like
waitForMessage(inbox_id, matcher, deadline). Keep the matcher, but make inbox scope mandatory. - Add attempt-level correlation: Generate an attempt ID for every email-triggering action. Use it in logs, test names, metadata, and application-side correlation when possible.
- Move parsing to structured data: Prefer structured JSON emails over scraping rendered HTML. Extract the OTP, link, or token as a typed artifact.
- Add consume-once semantics: Treat verification links and OTPs as artifacts that can be used once per attempt. Store a hash or idempotency key to avoid double processing.
- Roll out flow by flow: Start with the flakiest signup or login test, then migrate password reset, invitations, and agent-driven flows.
The migration usually reveals hidden dependencies. Some tests assume a persistent inbox history. Some debug scripts assume human mailbox login. Some agents may be over-permissioned. Those are exactly the assumptions worth removing.
When a shared mailbox is still acceptable
Shared mailboxes are not always wrong. They can be fine for manual smoke testing, local exploratory debugging, or low-volume flows where a human reviews the message.
They are a poor fit when any of these are true:
- CI runs tests in parallel
- The flow includes OTPs or magic links
- Tests are retried automatically
- LLM agents read or act on email content
- You need auditability across attempts
- You need to avoid leaking unrelated messages into logs or prompts
In other words, shared mailboxes are acceptable for humans. They are fragile for deterministic automation.
Where Mailhook fits
Mailhook is built for the inbox-per-attempt model. It provides programmable temp inboxes that developers, QA automation, and LLM agents can create through a RESTful API. Received emails are delivered as structured JSON, so your code can assert on fields and extract artifacts without logging into a mailbox or scraping a UI.
For delivery, you can use real-time webhooks for event-driven workflows and the polling API as a fallback. For domain strategy, Mailhook supports instant shared domains for fast setup and custom domain support when you need tighter control or allowlisting. Signed payloads help secure webhook processing, and batch email processing supports higher-throughput workflows.
If you are implementing an email verification API, signup test harness, or agent tool that waits for email, treat the inbox as the resource. The address is only how the outside world routes mail to that resource.
FAQ
Is one inbox per attempt the same as one inbox per test run? Not always. A test run may contain multiple email-triggering actions, and each action may be retried. The safest boundary is the attempt, meaning the single action that expects a specific email response.
Can plus-addressing replace per-attempt inboxes? Plus-addressing can help with correlation, but it does not provide true isolation if all messages still land in one shared mailbox. It is useful for lightweight debugging, but inbox IDs are stronger for parallel CI and agent workflows.
Should I use webhooks or polling to receive test emails? Use webhooks first when possible because they are fast and event-driven. Keep polling as a bounded fallback for recovery, local development, or cases where webhook delivery is temporarily unavailable.
How long should a disposable inbox live? Long enough for the attempt and a short drain window for late arrivals, but not longer than necessary. The right TTL depends on your application’s email latency, CI time budget, and retention policy.
Does this pattern help with LLM agent safety? Yes. A per-attempt inbox limits what the agent can see, reduces stale-message selection, and allows your tool to return only the needed artifact, such as an OTP or verification link.
Build email flows without shared mailbox chaos
If your tests or agents still depend on a shared test mailbox, the next reliability improvement is simple: make inbox creation part of the attempt lifecycle.
With Mailhook, you can create disposable email addresses via API, receive emails as structured JSON, consume them through webhooks or polling, and verify signed webhook payloads. Review the Mailhook llms.txt integration reference to wire the pattern into your CI, QA automation, or LLM agent workflow.
