Skip to content
Engineering

How to Fetch Emails Reliably With Webhooks or Polling

| | 12 min read
A cyberpunk night scene in a rain-soaked automation tunnel where a disposable inbox creation console feeds a real-time email event stream into a signed webhook gate and a separate polling reconciliation terminal. Show the inbox, an incoming message card, and two distinct processing paths converging on one durable record store, with luminous data lines, wet reflective surfaces, neon signage, atmospheric fog, visible light rays, drifting particles, subtle holographic interface elements, and a moody noir atmosphere with strong depth. Make the workflow feel deterministic and machine-driven, with edges fading organically into smoke and black through haze and scattered light particles, no hard border.
A cyberpunk night scene in a rain-soaked automation tunnel where a disposable inbox creation console feeds a real-time email event stream into a signed webhook gate and a separate polling reconciliation terminal. Show the inbox, an incoming message card, and two distinct processing paths converging on one durable record store, with luminous data lines, wet reflective surfaces, neon signage, atmospheric fog, visible light rays, drifting particles, subtle holographic interface elements, and a moody noir atmosphere with strong depth. Make the workflow feel deterministic and machine-driven, with edges fading organically into smoke and black through haze and scattered light particles, no hard border.

Fetching an email sounds simple until automation depends on it. A human can refresh a mailbox, scan a subject line, and tolerate a few seconds of uncertainty. An AI agent, QA test, or signup verification flow cannot. It needs a reliable way to know whether the expected email arrived, whether it has already been processed, and when to stop waiting.

The most dependable approach is not to treat email like a traditional inbox. Treat it like an event stream: messages arrive asynchronously, delivery can be retried, and consumers must be safe against duplicates. Webhooks and polling are the two common ways to fetch emails from that stream. Each has different strengths, and the best production systems often use both.

This guide explains how to fetch emails reliably with webhooks, polling, or a hybrid pattern that gives you low latency without sacrificing determinism.

Start with a reliability contract

Before choosing webhooks or polling, define what “reliable” means for your workflow. For most email-driven automation, reliability is not just “we eventually saw an email.” It means the system can make a correct decision every time, even when delivery is delayed, a webhook is retried, or the same message appears more than once.

A practical reliability contract usually includes five guarantees:

Guarantee Why it matters Implementation rule
Inbox isolation Prevents unrelated emails from matching the wrong run Use a unique disposable inbox per test, agent task, or user operation
Bounded waiting Prevents stuck jobs and infinite agent loops Use explicit deadlines and terminal states
Idempotent processing Makes retries safe Store processed message IDs or event IDs before triggering side effects
Replay or fallback Recovers from missed notifications Support polling even if webhooks are the primary path
Structured output Reduces parsing errors for automation Consume normalized JSON fields instead of scraping a visual inbox

For AI agents and LLM workflows, the bounded waiting rule is especially important. Do not ask a model to “keep checking until the email arrives.” Give the tool a timeout, a match condition, and a clear result such as found, not_found, or failed.

Webhooks vs polling: what each is best at

Webhooks are push-based. When an email arrives, the email API calls your endpoint with a payload. This is ideal when you need low latency and want to avoid repeatedly asking the API for updates.

Polling is pull-based. Your worker asks the API for new emails on a schedule or inside a wait loop. This is ideal when you want full control over timing, retries, and local test execution, or when inbound network access is unavailable.

The mistake is framing webhooks and polling as rivals. In reliable systems, they often play different roles. Webhooks are excellent for fast notification. Polling is excellent for reconciliation.

Method Best for Main risk Reliability fix
Webhooks Low-latency automation and event-driven workers Missed, delayed, or retried delivery Verify signatures, acknowledge quickly, and dedupe
Polling Deterministic waits, CI tests, local dev, agent tools Excessive requests or duplicate reads Use cursors, deadlines, backoff, and idempotent storage
Hybrid Production systems that need both speed and certainty More moving parts Use webhook-first delivery with polling fallback

If you are building temporary inbox flows, a webhook-first, polling fallback pattern is usually the most resilient option. Webhooks wake up your system quickly, while polling gives you a controlled way to confirm final state.

A reliable webhook flow

A webhook handler should be small, fast, and boring. Its job is not to do everything. Its job is to verify the request, persist enough information to process safely, and return a successful response quickly.

A solid webhook flow looks like this:

  • Verify the webhook signature before trusting the payload.
  • Parse the JSON payload and validate required fields.
  • Store the event or email using an idempotency key.
  • Enqueue downstream work instead of doing slow processing inline.
  • Return a 2xx response only after the event is safely recorded.

Mailhook supports real-time webhook notifications and signed payloads, which are useful when your system needs both fast delivery and a way to verify that inbound events are authentic. For a machine-readable overview of Mailhook’s current capabilities, including features intended for AI systems, see the official Mailhook llms.txt.

Here is a simplified webhook handler pattern, intentionally provider-neutral:

async function handleInboundEmailWebhook(request) {
  const body = await request.text();
  const signature = request.headers.get("x-webhook-signature");

  if (!verifySignature(body, signature)) {
    return new Response("invalid signature", { status: 401 });
  }

  const event = JSON.parse(body);
  const idempotencyKey = `${event.inbox_id}:${event.email_id}`;

  const inserted = await storeIfNew(idempotencyKey, event);

  if (inserted) {
    await enqueueEmailProcessingJob(idempotencyKey);
  }

  return new Response("ok", { status: 200 });
}

The key detail is storeIfNew. If the provider retries the webhook, or your endpoint receives the same event twice, your system should not click a verification link twice, create two accounts, or trigger duplicate agent actions. The handler can safely accept duplicate delivery because processing is idempotent.

A reliable polling flow

Polling is often the simplest way to fetch emails in tests, scripts, and agent tools. It is also easier to reason about because the caller controls the loop. The challenge is to avoid brittle sleeps and uncontrolled retries.

A reliable polling loop has four parts:

  • A match condition, such as recipient, subject, sender, or a token in the body.
  • A cursor or “since” marker so each request only asks for relevant new messages.
  • A deadline so the workflow ends predictably.
  • Dedupe logic so repeated responses do not trigger repeated actions.

Fixed sleeps are the classic source of flaky email automation. Sleeping for 10 seconds may pass on a good day and fail during a slow delivery window. A better pattern is to poll frequently at first, apply modest backoff, and stop at a clear deadline.

async function waitForEmail({ inboxId, matches, deadlineMs }) {
  const startedAt = Date.now();
  let cursor = null;
  let delayMs = 1000;
  const seen = new Set();

  while (Date.now() - startedAt < deadlineMs) {
    const response = await fetchEmails({ inboxId, cursor });
    cursor = response.next_cursor ?? cursor;

    for (const email of response.emails) {
      const key = `${inboxId}:${email.id}`;

      if (seen.has(key)) continue;
      seen.add(key);

      if (matches(email)) {
        return { status: "found", email };
      }
    }

    await sleep(delayMs);
    delayMs = Math.min(Math.round(delayMs * 1.5), 5000);
  }

  return { status: "not_found" };
}

If your workflow relies on polling in CI or agent execution, it is worth reading more about how to wait for email deterministically. The main principle is simple: wait for a condition, not for an arbitrary amount of time.

A backend automation pipeline showing an inbound email moving into a temporary inbox, then splitting into webhook notification and polling fallback paths that both feed an idempotent processor.

Dedupe is not optional

When developers first implement email fetching, they often assume each message will be delivered exactly once. That assumption breaks quickly. Webhooks can be retried. Polling can return overlapping pages. Email itself can contain forwarded, resent, or provider-normalized fields that are not globally unique enough for your workflow.

The safest approach is to create your own idempotency boundary. For example, store a unique record using a stable email identifier scoped to the inbox. If your provider exposes event IDs separately from email IDs, decide whether you dedupe at the event level, the message level, or both.

In most automation flows, message-level idempotency is the right default. If the same email is observed twice, your system should treat it as the same input. Event-level idempotency can still be useful for logging webhook delivery attempts.

A good processing table might include fields like these:

Field Purpose
Inbox ID Scopes the message to a specific automation run or agent task
Email ID Identifies the message within that inbox
Received timestamp Supports ordering and debugging
Processing status Tracks pending, processed, ignored, or failed states
Match result Records why the message did or did not satisfy the workflow

This is also where structured JSON output helps. Mailhook receives emails as structured JSON, so your automation can match against explicit fields instead of relying on brittle mailbox scraping. That matters when an LLM tool needs to extract a verification code, inspect a sender, or confirm that a transactional email arrived.

For deeper implementation details, especially around duplicate-safe pull flows, see these best practices for polling email APIs without duplicates.

Use webhook-first with polling fallback in production

For many production workflows, the most reliable design is a hybrid:

  1. Create an isolated disposable inbox for the operation.
  2. Register or configure webhook delivery for that inbox or domain.
  3. When a webhook arrives, store the email and wake the waiting job.
  4. If no webhook arrives before a short grace period, start polling.
  5. Stop when the expected email is found or the deadline expires.

This gives you the best of both models. Webhooks keep the happy path fast. Polling protects the system from missed callbacks, network interruptions, local development constraints, and deployment mistakes.

The hybrid pattern is especially useful for LLM agents because agents should not be responsible for infrastructure uncertainty. The agent should call a tool like “wait for verification email,” and the tool should hide the messy details of webhooks, polling, retries, and dedupe.

A clean agent-facing response might look like this:

{
  "status": "found",
  "inbox_id": "inbox_123",
  "email_id": "email_456",
  "from": "[email protected]",
  "subject": "Verify your account",
  "received_at": "2026-07-15T21:11:00Z",
  "extracted_code": "123456"
}

Or, when the email does not arrive:

{
  "status": "not_found",
  "inbox_id": "inbox_123",
  "waited_ms": 60000,
  "reason": "deadline_exceeded"
}

That distinction is important. not_found is not the same as a system error. It means the email was not observed within the contract you defined. Your agent or workflow can then decide whether to retry the upstream action, report failure, or ask for human intervention.

Common failure modes and how to handle them

Reliable email fetching is mostly about planning for boring failures before they happen.

Failure mode Symptom Recommended response
Delayed email delivery Polling times out too early Use realistic deadlines and report not_found clearly
Duplicate webhook Same action runs twice Enforce idempotency before side effects
Missed webhook No event reaches your endpoint Run polling fallback before declaring failure
Shared inbox contamination Wrong email matches Use one disposable inbox per operation
HTML parsing mismatch Agent extracts the wrong value Prefer structured fields and deterministic extraction logic
Endpoint downtime Webhook retries fail repeatedly Store events durably and reconcile with polling

The most dangerous failure is a false positive, not a timeout. A timeout is visible. A false positive can verify the wrong account, use the wrong code, or make an agent believe a task succeeded when it did not. Inbox isolation and strict matching are the best defenses.

Observability makes reliability measurable

You cannot improve what you cannot see. Track a few simple metrics for every email-dependent workflow:

  • Time from inbox creation to email received.
  • Time from email received to processing completed.
  • Number of webhook retries or duplicate events.
  • Polling attempts per successful match.
  • Timeout rate by provider, environment, or test suite.

These metrics help you distinguish between application bugs and upstream delivery delays. They also help tune deadlines. For example, if 99 percent of verification emails arrive within 20 seconds, a 60-second deadline may be reasonable. If a specific upstream service frequently takes two minutes, your contract should reflect that reality or the workflow should fail with a useful explanation.

Where Mailhook fits

Mailhook is built for programmable email receipt rather than human inbox management. You can create disposable inboxes via API, receive emails as structured JSON, use real-time webhooks, poll for emails, work with instant shared domains, and use custom domain support when needed.

That combination is useful for AI agents, LLM tools, QA automation, and signup verification flows because the inbox becomes an automation primitive. Instead of asking a person or model to operate a mailbox, your system creates an inbox, waits for a specific email, processes structured data, and ends with a deterministic result.

You can explore the product at Mailhook, and you can give AI coding tools or agents the concise product context via Mailhook’s llms.txt.

Frequently Asked Questions

Is webhook delivery more reliable than polling? Webhooks are usually faster, but polling is often easier to control. The most reliable systems use webhooks for low-latency notification and polling as a fallback or reconciliation mechanism.

How often should I poll for emails? Start with a short interval, such as one second, then apply modest backoff up to a small maximum. Always use an overall deadline so the workflow cannot run forever.

What is the best way to avoid duplicate email processing? Store each email using an idempotency key, usually based on inbox ID plus email ID. Check that key before running side effects such as clicking a link, completing signup, or notifying an agent.

Should AI agents parse email HTML directly? Prefer structured JSON fields and deterministic extraction logic. If an LLM is used for interpretation, give it already-normalized email content and keep retries, deadlines, and dedupe in application code.

When should I use a disposable inbox? Use a disposable inbox whenever a workflow needs isolation, such as signup tests, verification emails, agent tasks, or client operations. A unique inbox per operation prevents unrelated messages from contaminating results.

Build email fetching as infrastructure, not a side task

If your automation depends on inbound email, reliability should be designed in from the beginning. Use isolated inboxes, structured JSON, idempotent processing, explicit deadlines, and a webhook-first flow with polling fallback where appropriate.

Mailhook gives developers and AI agents the building blocks to fetch emails programmatically without relying on a human mailbox. Create disposable inboxes via API, receive email data as JSON, and choose webhooks, polling, or both depending on your workflow’s reliability needs.

Related Articles