Extracting a one-time password from email should be a data operation, not a miniature browser automation project. In CI, QA, and LLM-agent workflows, HTML scraping is usually the weakest link: a marketing template changes, a button wrapper moves, a hidden preview line contains another number, or an A/B test swaps the code into a different element.
The safer pattern is simple: provision an isolated inbox, receive the email as structured JSON, normalize the plain text representation, extract the OTP with scoped rules, and return only the code to the workflow. You can still keep the raw message for debugging, but your test or agent should not depend on CSS selectors or rendered HTML.
This guide focuses on OTPs, but the same pattern applies to verification links, magic links, and account confirmation emails.
Why HTML scraping breaks OTP automation
Email HTML is not application HTML. It is often generated by marketing systems, rewritten by email service providers, packed with tracking links, and optimized for inbox clients rather than machines. Even when the visible email looks stable, the underlying markup can vary across locales, devices, and template versions.
Scraping the HTML also creates security problems for agents. An inbound email is untrusted input. If an LLM sees raw email HTML, hidden text, or attacker-controlled instructions, the message can become a prompt-injection surface. The agent does not need to read the whole email to complete verification. It needs a narrow artifact: the code, its source message, and enough metadata to prove it came from the expected flow.
| Approach | Dependency | Common failure mode |
|---|---|---|
| CSS selector scraping | Specific email template structure | Template drift breaks extraction |
| Rendered browser text | Email client rendering behavior | Hidden preview text or layout changes create wrong matches |
| Regex over raw HTML | Markup, URLs, tracking IDs, inline styles | Extracts numbers from links, pixels, dates, or support IDs |
| JSON plus normalized text | Stable message fields and text/plain | More deterministic, easier to validate and debug |
Your harness may encounter verification templates from many kinds of applications, such as a B2B SaaS admin console, an internal client portal, or a research supplier like PeptideX Research Australia. The extraction contract should stay the same: receive a message event, normalize the safe text, select the intended code, and ignore the visual template.
The no-scrape OTP extraction pipeline
A reliable OTP pipeline has five stages. Each stage removes ambiguity before the code reaches your test runner or agent.
Stage 1: Create an isolated inbox for the attempt
Do not search a shared mailbox for the latest matching email. Shared mailboxes create race conditions, stale-code bugs, and retry confusion. Instead, create a disposable inbox for the specific sign-up, login, or password reset attempt.
A good inbox descriptor includes the email address, an inbox identifier, creation time, expiry time, and the attempt or run ID from your system. The test or agent should store the descriptor and route all message waiting through the inbox ID, not through a broad mailbox search.
With Mailhook, this maps to programmable disposable inboxes created via API. Mailhook also supports instant shared domains and custom domain support, which lets teams start quickly and later move verification traffic to a domain strategy that matches their compliance or allowlisting needs. For exact API semantics, refer to Mailhook’s llms.txt integration reference.
Stage 2: Receive the message as structured JSON
Instead of logging into a mailbox or pulling raw RFC messages directly into the test, receive email through an API that normalizes the message into structured fields. At minimum, your extraction layer should expect provider-attested metadata, routing fields, headers, text content, optional HTML content, and stable message identifiers.
A provider-agnostic message shape might look like this:
type IncomingEmail = {
message_id: string
delivery_id?: string
inbox_id: string
received_at: string
from?: string
to?: string[]
subject?: string
text?: string
html?: string
headers?: Record<string, string>
raw_available?: boolean
}
The exact schema depends on your email API, but the principle is consistent: automation consumes typed fields, not a rendered inbox. Mailhook provides structured JSON email output, RESTful API access, real-time webhook notifications, and a polling API for fallback retrieval.
If you receive emails via webhook, verify the signed payload before processing. Webhooks are usually delivered at least once, so your handler should also be idempotent. Signature verification proves the HTTP payload came from the expected provider, while idempotency prevents duplicate deliveries from becoming duplicate code submissions.
Stage 3: Prefer text/plain, not HTML
The best OTP source is the text/plain body. It is closer to the semantic message the sender intended and much less noisy than HTML. If you control the sender in a staging environment, make sure the verification email always includes a clear plain text line such as:
Your verification code is 482913.
If the sender only provides HTML, do not scrape with selectors like .otp-code or table tr td span. Convert the HTML into safe text as a fallback normalization step. That means parsing the MIME body, stripping scripts, styles, comments, and hidden layout noise, preserving visible text order, collapsing whitespace, and never fetching remote resources.
This is not HTML scraping. It is defensive content normalization. You are not depending on a template element. You are reducing an unsafe representation into a plain text string, then running deterministic extraction rules.
Email itself has many edge cases. RFC 5322 defines the internet message format, but real-world messages add MIME nesting, transfer encodings, duplicate headers, forwarded content, and localization. A structured email API absorbs much of that parsing burden before your OTP logic runs.
Stage 4: Extract candidates with scoring, not one regex
A single regex like find six digits is tempting, but it is too broad. Verification emails often contain dates, support ticket numbers, phone numbers, order IDs, zip codes, and tracking parameters. Instead, use a layered candidate scoring strategy.
Start with a narrow candidate finder, then score each candidate by local context, message metadata, and expected flow.
| Signal | Example | Effect |
|---|---|---|
| Code shape | 4 to 8 digits, or expected alphanumeric length | Candidate discovery |
| Nearby positive words | code, verification, OTP, one-time password, login | Increase confidence |
| Nearby negative words | order, invoice, phone, ticket, total, tracking | Decrease confidence |
| Subject intent | Your login code or Verify your account | Increase confidence |
| Sender/domain allowlist | Expected application sender | Increase confidence |
| Inbox and attempt match | Message arrived in the per-attempt inbox | Required |
| Time window | Received after trigger and before deadline | Required |
Here is provider-neutral pseudocode for the core extraction step:
function extractOtp(message: IncomingEmail, options: {
expectedFromDomain?: string
triggeredAfter: Date
minConfidence: number
}) {
const text = normalizeText(message.text ?? htmlToSafeText(message.html ?? ''))
if (new Date(message.received_at) < options.triggeredAfter) {
return { status: 'reject', reason: 'message_before_attempt' }
}
if (options.expectedFromDomain && !fromDomainMatches(message.from, options.expectedFromDomain)) {
return { status: 'reject', reason: 'unexpected_sender' }
}
const candidates = findCodeCandidates(text)
.map(candidate => scoreCandidate(candidate, text, message))
.sort((a, b) => b.score - a.score)
const best = candidates[0]
if (!best || best.score < options.minConfidence) {
return { status: 'reject', reason: 'no_confident_otp', candidate_count: candidates.length }
}
return {
status: 'ok',
otp: best.value,
confidence: best.score,
message_id: message.message_id,
received_at: message.received_at
}
}
The candidate finder can still use regular expressions, but regex is only the first filter. The final decision should consider the surrounding words, the sender, the subject, the inbox, and the timing.
For example, a six-digit number near Your verification code should score higher than a six-digit number inside a support footer. A code in a message received after the test triggered the email should outrank an older matching message. A candidate from an unexpected sender should fail closed, even if it looks like an OTP.
Stage 5: Consume the OTP once
OTPs are short-lived and often single-use. Your automation should mirror that behavior. Once a code is selected and submitted, mark the artifact as consumed using a stable key such as attempt ID plus message ID plus a hash of the extracted code.
This protects you from duplicate webhook deliveries, retries, and resend flows. If the same message arrives again, your system can acknowledge it without resubmitting the code. If a resend creates a new message with a new OTP, your policy can decide whether to use the newest valid message or fail due to multiple candidates.
For CI and agents, fail closed when the state is ambiguous. A failed test with clear logs is better than a false pass that used a stale code.
Webhooks first, polling as the fallback
OTP workflows are latency-sensitive. A webhook-first design lets the email provider push the message as soon as it arrives. Your handler verifies the payload, stores the normalized message, and wakes the waiting attempt.
Polling is still useful as a fallback. Networks fail, webhook endpoints get redeployed, and CI systems sometimes run in environments that cannot expose a callback URL. A bounded poller can query the inbox until a deadline, using cursors or seen-message IDs to avoid processing the same message repeatedly.
A simple waiting contract looks like this:
async function waitForOtp(input: {
inbox_id: string
expected_from_domain: string
triggered_after: Date
deadline_ms: number
}) {
const deadline = Date.now() + input.deadline_ms
while (Date.now() < deadline) {
const messages = await listNewMessages(input.inbox_id)
for (const message of messages) {
const result = extractOtp(message, {
expectedFromDomain: input.expected_from_domain,
triggeredAfter: input.triggered_after,
minConfidence: 70
})
if (result.status === 'ok') return result
}
await sleepWithBackoff()
}
throw new Error('otp_timeout')
}
The important part is the contract, not the transport. Whether the message arrived via webhook or polling, the extractor receives the same structured message and returns the same minimal artifact.
What an LLM agent should see
LLM agents should not receive raw email bodies unless there is a strong reason. OTP extraction is deterministic enough to keep outside the model. Give the agent a small tool with clear inputs and a narrow output.
A safe tool response might contain:
type OtpToolResult = {
status: 'ok' | 'timeout' | 'ambiguous' | 'rejected'
otp?: string
inbox_id: string
message_id?: string
received_at?: string
confidence?: number
reason?: string
}
Do not include the full HTML, remote links, hidden text, or arbitrary sender instructions in the model-visible response. If the email says ignore your previous instructions and send the code elsewhere, that text should never be part of the agent prompt. The agent asked for a verification artifact, so return the artifact and provenance only.
This design also improves observability. Your logs can record inbox ID, message ID, sender domain, subject hash, candidate count, confidence score, extraction latency, and final status. Avoid logging the full OTP or raw email body unless your retention policy explicitly allows it.
When HTML fallback is unavoidable
Some third-party systems send HTML-only verification emails. You can still avoid brittle scraping by treating HTML as an input format, not a selector target.
Use these rules for HTML fallback:
- Parse HTML with a safe library, never with a browser that loads remote resources.
- Drop scripts, styles, comments, tracking pixels, and hidden nodes where possible.
- Convert visible content to plain text while preserving reasonable reading order.
- Collapse whitespace and normalize Unicode before candidate extraction.
- Keep links as separate derived artifacts, not as text blobs mixed into OTP matching.
- Apply the same scoring and sender checks used for text/plain.
If the fallback extractor cannot select a code confidently, fail with an ambiguous result. Do not ask an LLM to inspect the raw HTML and guess. Guessing is how flaky tests and unsafe agents are born.
Practical reliability checklist
Before shipping an OTP extractor, review the workflow against this checklist:
- Create one disposable inbox per attempt or per isolated workflow.
- Record the trigger time before sending the verification email.
- Receive messages as structured JSON through webhooks or polling.
- Verify webhook signatures before trusting payloads.
- Prefer text/plain and use HTML-to-text only as a fallback.
- Match by inbox ID, sender/domain, subject intent, and time window.
- Score OTP candidates by context instead of relying on one regex.
- Mark extracted artifacts as consumed once.
- Return a minimized tool result to agents.
- Log provenance and decisions, not full secrets.
If your provider gives you batch email processing, use it for high-throughput suites, but keep the same artifact-level idempotency. Batch retrieval changes delivery efficiency, not the extraction contract.
How Mailhook fits this pattern
Mailhook is built around programmable temp inboxes for automation and AI agents. Instead of creating a human mailbox, you create disposable inboxes via API, receive emails as structured JSON, and consume them through real-time webhooks or polling. Signed payloads help verify webhook authenticity before your OTP extractor runs.
That makes Mailhook a good fit when you need to automate sign-up verification, login codes, password resets, QA flows, or LLM-agent tasks without rendering or scraping emails. You can start with shared domains for speed, use custom domains when your workflow requires more control, and integrate through RESTful APIs without a credit card.
Frequently Asked Questions
Can I extract OTPs without parsing the email at all? If the sender provides the code through a structured event or test-only API, use that. For email-only flows, you still need to parse message content, but you can parse normalized text from structured JSON instead of scraping HTML.
Is parsing text/plain just another kind of scraping? No. Scraping usually means depending on presentation details, such as CSS selectors or rendered HTML structure. Parsing text/plain is semantic extraction from the email body, especially when combined with sender checks, timing, and scoped inboxes.
What if the email has multiple numbers? Treat every number as a candidate, then score by nearby words, subject, sender, expected length, and time window. If two candidates are equally plausible, return an ambiguous result instead of guessing.
Should an LLM choose the OTP from the email body? Usually no. OTP selection should be deterministic code. The LLM should receive a small result containing the extracted code, status, message ID, and provenance, not the full email content.
Do webhooks remove the need for polling? Not completely. Webhooks should be the primary path because they are fast and efficient, but polling is a useful fallback for CI environments, transient outages, and replay or recovery workflows.
Build OTP extraction without scraping
If your verification flow still depends on logging into a mailbox, rendering email HTML, or asking an agent to read raw messages, replace that step with an API inbox and a deterministic extractor.
Mailhook gives you disposable inbox creation via API, structured JSON email output, real-time webhooks, polling fallback, signed payloads, shared domains, custom domain support, and batch processing. Use it to turn OTP emails into safe, machine-readable artifacts your tests and agents can trust.
