Phone as Gateway: How to Measure SMS Deliverability

Deliverability is the metric that decides whether your product promise matches reality. When you operate an android phone as sms gateway, the REST API can return success while messages stall in modem queues, exhaust prepaid balance, or never produce a delivery report your backend records. Measuring deliverability means tracing each message from API accept through carrier final state — not counting HTTP 200 responses. This spoke covers definitions, baseline metrics, DLR wiring, latency buckets, segmentation by SIM and traffic class, geo-specific prepaid behavior, a conceptual logging sample, and the HowTo sequence to stand up a dashboard before you scale. Anchor the full phone-as-gateway program at the hub: How to use an Android phone as an SMS gateway.

HTTP 200 is a receipt that the queue accepted work — not proof a handset showed a code.

Service pricing is based on device count and total SMS sent through the gateway. You need a working Android phone with a SIM and SMS credit from your mobile operator. Operator message costs are yours—we do not sell carrier SMS balance. Platform tiers on device and SMS volume pricing meter connected devices and SMS volume (Priced by devices and SMS send volume. You use your own phone and operator SMS credit.) — free tier includes 300 SMS lifetime on one device; paid plans from $19/month with higher allowances. None of that replaces deliverability instrumentation. Operator SMS credit depletes per transmitted PDU; delivery failure from balance exhaustion looks like an application bug if you only watch API metrics. Pair commercial planning with technical measurement.

E-commerce and SaaS teams often learn webhook-driven observability from platform docs such as the Shopify API documentation — order events, fulfillment callbacks, idempotent consumers. SMS deliverability on a handset gateway follows the same discipline: emit state transitions, consume them reliably, reconcile unknowns. The transport differs; the operational maturity does not.

Why deliverability measurement matters

Teams adopt an Android phone gateway to control cost and sender identity. Cost savings evaporate if login codes fail silently during peak signup. Marketing lists show “sent” in a dashboard while half the handsets never receive the promo. Without deliverability measurement, you optimize the wrong layer — adding API retries when the modem is throttled, or scaling webhook workers when the SIM balance is empty.

Deliverability also anchors compliance conversations. Regulators and carriers care about consent and complaint rates; your internal SLA cares about confirmed delivery to opted-in numbers. You cannot prove either without terminal status records. Pending forever is not delivered; failed with reason code is at least honest.

Finally, deliverability metrics justify fleet decisions. When delivery rate on SIM A stays 96% while SIM B drops to 78% under the same content class, the data supports splitting OTP and bulk, swapping operators, or adding a second phone. Gut feeling and signal bars are not capacity planning.

Deliverability vs delivery rate

Delivery rate is a snapshot formula: delivered messages divided by messages that reached a terminal state within your measurement window. Deliverability is the operational practice: defining SLA, instrumenting the path, segmenting routes, alerting on regression, and reconciling unknowns. Industry blogs use the terms interchangeably; runbooks should not.

Exclude invalid numbers from OTP SLA numerators if your policy treats them as product validation failures, but never hide them entirely — a spike in invalid MSISDN often means a form bug or list import error. For bulk, pre-send scrubbing belongs upstream; deliverability measurement confirms scrub quality.

Time window matters. OTP measured at T+60 seconds differs from OTP measured at T+24 hours. Publish the window next to every dashboard tile. Executives comparing weekly reports with different windows draw wrong conclusions.

Where measurement lives in the stack

Four layers participate in deliverability. Application — your backend accepts a send request and assigns a correlation ID. Gateway service — queues the job, selects device/SIM, returns accept status. Handset modem — submits PDU to carrier; may retry on RF errors. Carrier + recipient — routes SMS, emits delivery or failure back through DLR chain. Measurement joins layer one and four with timestamps from two and three where available.

The DLR (delivery report) page documents status semantics your webhook consumer must parse. The SMS webhook integration page covers endpoint design — HTTPS, fast 2xx, idempotency keys. The SMS API documentation lists live API parameters for send and status query. Deliverability dashboards read from webhook persistence, not from polling alone — polling is backup when webhooks lag.

Control-plane health (device online, last heartbeat) correlates with deliverability but is not substitute. A green device with empty prepaid balance produces failed DLRs, not offline queues. Split alerts: device stale vs delivery rate drop.

Baseline metrics to track

Start with six metrics before adding exotic analytics. The table below lists definitions and starter targets; tune after your canary benchmark on real SIMs.

Store raw events; compute rollups in your metrics store. Raw retention of at least 30 days supports incident replay when a carrier changes behavior mid-month. Aggregate hourly for dashboards; keep message-level detail for support tickets.

DLR webhooks and status codes

Final deliverability numbers come from DLR events. Typical lifecycle: accepted → sent (modem submit) → delivered or failed. Some routes expose intermediate states; your consumer should treat anything non-terminal as pending until SLA timeout converts it to unknown-failed for SLA math.

Map carrier-specific reason codes to internal enums before alerting. “Rejected: spam” and “Rejected: insufficient balance” need different runbooks. Persist the raw code string for operator support tickets.

Webhook reliability is part of deliverability. If your consumer returns 503 under load, the gateway may retry DLR delivery — idempotent handlers prevent duplicate accounting. If retries exhaust, you lose terminal states and unknown ratio climbs even while users received SMS.

Latency buckets and SLAs

Plot time-to-DLR histograms with buckets: 0–15s, 15–30s, 30–60s, 1–5m, 5m+. OTP signup flows care about the first three buckets. Bulk campaigns may flatten across hours if consent and quiet hours allow.

P50 without P95 hides tail pain. A median of 8 seconds with P95 of 4 minutes means one in twenty users waits — unacceptable for OTP. Report both on the same panel.

Clock skew between application server and webhook timestamps distorts latency. Use monotonic correlation: store server accept time once, DLR receive time on webhook ingress, compute delta in UTC with NTP-synced hosts.

Segment by route, SIM, and content

Account-level delivery rate is a vanity metric for fleets. Segment by ICCID, device ID, destination country, traffic class, and template ID. Regression on one SIM during an otherwise healthy account is the common production surprise.

Content class segmentation separates OTP templates from marketing footers. Operators throttle promotional keywords even when sent from the same API key. If your gateway does not tag traffic class at send time, add it before measuring — otherwise OTP inherits bulk’s bad day.

Route segmentation includes dual-SIM slot choice. Slot one and slot two may use different operators with different deliverability profiles. Tag outbound messages with slot identifier when the platform exposes it.

OTP vs bulk deliverability profiles

OTP demands high delivery rate, low latency, and near-zero unknowns. Users retry login; duplicate OTP sends increase support load and carrier suspicion. Measure OTP on dedicated SIMs when bulk runs the same day — shared modem contention shows up as latency tail, not always as hard failures.

Bulk tolerates lower immediate delivery rate if list size and consent are documented, but sustained failure spikes trigger operator blocks that also kill OTP on a shared SIM. Deliverability segmentation is risk isolation, not vanity.

Transactional alerts (shipping, appointment) sit between OTP and bulk — often minute-scale SLA. Give them their own traffic class in metrics so marketing experiments do not blur alert reliability.

Carrier rejection signals

Learn the failure codes your operator returns through DLR. Common categories: invalid destination, spam/content policy, insufficient balance, handset unreachable, network timeout. Each category maps to a different fix — scrub lists, rewrite templates, top up balance, wait and retry, relocate gateway phone.

Sudden shift from delivered to failed with spam codes after months of stability often means template or URL change, not hardware failure. Diff template IDs in metrics when marketing updates copy without telling ops.

Some failures are effective deliverability drops at the recipient OEM — message delivered to network but filtered on device. Canary handsets on major OEMs in your market detect this; pure DLR “delivered” may still miss user perception.

Geo note: Bangladesh prepaid

Prepaid-heavy markets introduce balance-driven deliverability cliffs. A campaign may show 94% delivery until pack depletion, then 40% failure over ten minutes. The SMS Gateway Bangladesh geo guide covers operator context; deliverability dashboards there must include recharge ownership and pack type beside ICCID.

Promotional SMS rates on Bangladesh operators sometimes differ from standard transactional rates — wrong rate class looks like content failure. Document which SIMs are registered for which use case with the operator.

Separate OTP SIMs from bulk SIMs in Dhaka and Chittagong deployments as you would globally — shared throttle during Ramadan campaigns or flash sales affects login if metrics are not segmented.

Vs Twilio-style cloud SMS

Cloud aggregators expose MessageStatus, ErrorCode, and delivery dashboards in one vendor console. You pay per message and rent numbers; deliverability tooling is bundled. Android handset gateways separate platform fee (devices + volume) from operator airtime you recharge. You build DLR storage and dashboards — or integrate your existing observability stack.

Compare economics and operational ownership on Twilio vs Android SMS gateway. Deliverability measurement effort is real on handset routes; so is control over local sender ID and per-message operator cost at moderate scale.

Hybrid architectures sometimes use aggregator for international and handset for domestic OTP. Segment deliverability metrics by route type so hybrid teams do not average away a failing leg.

Code sample: metrics and logging

Below is conceptual Node-style pseudocode for correlating send accept with DLR webhook events. Map field names to live parameters in the SMS API documentation and status codes on SMS delivery reports (DLR). This is not copy-paste production code — it shows the join key and metric increments ops teams need.

// On send accept — persist once
async function onSendAccepted(event) {
 await db.messageMetrics.insert({
 correlationId: event.reference,
 simIccid: event.sim,
 trafficClass: event.tags?.class ?? 'unknown',
 acceptedAt: Date.now(),
 state: 'pending',
  });
}

// Webhook consumer — idempotent by message id + status
async function onDlrWebhook(payload) {
 const terminal = ['delivered', 'failed'].includes(payload.status);
 await db.messageMetrics.update(
    { correlationId: payload.reference },
    {
 state: payload.status,
 reasonCode: payload.errorCode ?? null,
 dlrAt: Date.now(),
 latencyMs: Date.now() - payload.acceptedAt,
    }
  );
 if (terminal) {
 metrics.increment(`sms.${payload.status}`, { sim: payload.sim });
 metrics.histogram('sms.time_to_dlr_ms', payload.latencyMs);
  }
}

// SLA job — pending older than 10m → unknown for OTP class
async function reconcileUnknowns() {
 const stale = await db.messageMetrics.findPendingOlderThan(10 * 60 * 1000);
 for (const row of stale) {
 if (row.trafficClass === 'otp') {
 metrics.increment('sms.unknown_timeout', { sim: row.simIccid });
    }
  }
}

Export the same counters to Grafana, Datadog, or CloudWatch. Alert on sms.failed rate per SIM and on sms.unknown_timeout for OTP class. Wire webhook signing verification before trusting payloads — see SMS webhook integration security notes.

HowTo: build a deliverability dashboard

Follow the HowTo schema steps embedded at the top of this article: define SLA, wire DLR persistence, instrument accept events, build per-SIM panels, automate canaries, publish runbooks. Sequence matters — SLA before thresholds, webhooks before canary promotion gates.

Week one: single SIM, test numbers, 500-message canary at production rate. Week two: add second SIM or traffic class split. Week three: connect alerts to on-call with linked runbooks. Do not skip week one because “volume is low” — low volume hides throttle until launch day.

Benchmark methodology

Canary cohorts use real handsets on target operators, not simulators. Include at least two Android OEMs and one iOS recipient if your user base is mixed — routing differs. Send at planned production rate, not burst-then-idle, unless your workload is genuinely bursty and you measure both patterns.

Record environment: firmware version on gateway phone, signal RSSI if available, Wi-Fi vs mobile data control path, prepaid balance before and after. Benchmark metadata explains outliers when someone asks “why March 12?”

Compare canary results to prior baseline; promote to production only when delivery rate and P95 latency meet SLA. Regression of two points may be noise; regression of ten points is a release blocker.

Failure modes and triage

Triage order: confirm webhook health → check per-SIM failure codes → verify balance → compare traffic class mix → inspect RF and device heartbeat. Skipping straight to phone reboot wastes minutes during OTP incidents.

Operator credit and false negatives

You need a working Android phone with a SIM and SMS credit from your mobile operator. Operator message costs are yours—we do not sell carrier SMS balance. Deliverability dashboards must sit beside airtime monitoring. A graph of delivery rate with a vertical annotation when balance crossed zero teaches teams faster than postmortems.

Postpaid accounts fail differently — credit limit or bill dispute — but still produce balance-like failure codes. Document billing owner per SIM in the same registry ICCID lives in.

Pricing and volume caps

Service plan SMS allowances on device and SMS volume pricing cap platform volume, not carrier deliverability. Hitting platform cap stops accepts; hitting operator throttle stops delivery with API still green. Monitor both counters.

Free tier (300 SMS lifetime) is sufficient to wire DLR, run canaries, and validate dashboard joins before paid scale. Deliverability discipline should exist before you need it at volume.

Dashboard checklist

• ☐Deliverability SLA documented per traffic class (OTP, transactional, bulk)
• ☐DLR webhook endpoint load-tested and monitored for 5xx rate
• ☐Every send logs correlation ID joinable to DLR record
• ☐Per-SIM delivery rate panel with seven-day baseline comparison
• ☐P95 latency chart segmented by traffic class
• ☐Unknown-message reconciliation job with SLA timeout configured
• ☐Failure reason codes grouped and top-N displayed hourly
• ☐Prepaid balance alert linked to same dashboard as delivery rate
• ☐Weekly canary cohort automated after SIM or device changes
• ☐Incident runbook linked from alert notifications
• ☐Bangladesh or geo-specific recharge notes attached if applicable
• ☐Comparison baseline documented vs prior week and vs canary

Reconciliation and unknown recovery

Unknown states poison SLA reporting. A message pending beyond your OTP timeout should transition to unknown_timed_out in metrics — not remain pending forever. Reconciliation jobs query gateway status APIs where available, compare against webhook gaps, and mark final state when carrier eventually reports.

Reconciliation frequency depends on traffic class. OTP: every 5 minutes for pending under 15 minutes old. Bulk: hourly batch for previous day with manual review queue for persistent unknowns above 2% of segment size.

Never reconcile by resending blindly. Duplicate OTP sends frustrate users and increase carrier suspicion. Resend only after confirmed non-delivery terminal state or explicit user request with rate limit.

Store reconciliation audit rows: message ID, action taken, operator response, timestamp. Compliance and support need evidence when users dispute “never received” weeks later.

Multi-device fleet deliverability

Fleets with three or more gateway phones need fleet-level and device-level dashboards. Fleet rollup hides SIM B failure when SIM A and C compensate in volume — OTP routed to SIM B still fails while aggregate looks healthy.

Tag every DLR with device_id and iccid. Failover routing should increment failover_send_count separately — deliverability on primary vs secondary paths differs and should inform stickiness rules.

When one device reboots nightly for maintenance, expect unknown spike during window — annotate dashboard with maintenance events or deliverability alerts false-positive.

Device addition triggers mandatory canary on new ICCID before production traffic shares load. Borrowing throughput from an unmeasured SIM is fleet deliverability debt.

Webhook observability parallel (Shopify-style)

Teams familiar with Shopify API webhooks already understand idempotent consumers, HMAC verification, and retry semantics. SMS DLR webhooks deserve the same infrastructure: dead-letter queue for failed processing, replay tool for incident recovery, and schema versioning when gateway adds new status fields.

Map Shopify’s “at-least-once delivery” mental model to DLR: you may receive duplicate delivered events — handle without double-counting success metrics. Map order-id correlation to SMS correlation_id join pattern in the code sample above.

E-commerce OTP (order pickup codes) sits at intersection: Shopify order webhook triggers send; DLR webhook confirms delivery before marking order ready. End-to-end tracing spans both platforms — use shared correlation ID in order metadata and SMS reference field.

Alert thresholds that work

Static thresholds (“alert if delivery rate < 90%”) noise on small samples. Use relative drop: alert when hourly delivery rate falls 10+ points below same hour seven-day median with minimum volume 50 messages. OTP routes use tighter band (5 points) and lower minimum volume (20).

Alert on unknown ratio crossing 1% for OTP before absolute delivery rate triggers — unknowns lead failures by minutes when webhooks break. Page on-call for unknown spike; Slack-only for gradual delivery drift.

Balance alerts belong in same paging policy as deliverability when prepaid SIMs dominate — failure code “insufficient balance” should bypass aggregation delay and page immediately during business hours.

Deliverability case patterns

Case A — Webhook lag mistaken for send failure: Support resends OTP triple while modem delivered first attempt. Root cause: webhook consumer CPU saturated; DLR delayed 4 minutes. Fix: scale consumer; user sees three codes. Prevention: monitor webhook processing latency separately from time-to-DLR.

Case B — Template change without canary: Marketing adds URL to OTP template Friday deploy; delivery rate drops from 97% to 61% spam failures. Rollback template_version; segment OTP template changes through same canary as bulk.

Case C — Bangladesh prepaid cliff: Campaign delivers 94% until hour three; failures jump as pack empties. Dashboard annotated with balance zero crossing. Fix: auto-pause queue; recharge runbook on Bangladesh geo guide.

Case D — Dual SIM wrong slot: OTP routed to slot with promotional throttle history while slot two clean. Per-ICCID metrics exposed misconfiguration in routing table — not modem failure.

Developer Center and live parameters

Deliverability instrumentation starts at send API accept. The SMS API documentation documents reference fields, device selectors, and callback URLs — map each to log columns before writing dashboards. Changing parameter names without updating join logic breaks latency histograms silently.

Status poll endpoints supplement webhooks during webhook consumer outages — not as primary metrics source. Polling load scales with pending count; prefer webhook recovery over permanent poll architecture.

Weekly deliverability ops rhythm

Monday: review per-SIM delivery rate vs baseline for prior week; flag ICCIDs below threshold. Tuesday: reconcile unknown backlog from weekend OTP traffic; close timed-out rows. Wednesday: canary send on any SIM changed in last seven days. Thursday: webhook consumer error budget review — 5xx rate, p99 processing time. Friday: publish one-page ops summary for product and support with top failure codes and any planned SIM maintenance.

Monthly: full ramp re-test per production SIM; compare to prior month; update safe operating rate in wiki. Quarterly: template and traffic-class audit — confirm OTP still isolated; bulk segments still tagged; suppression list growth reviewed with legal.

This rhythm prevents deliverability from becoming a launch-only checklist. Handset gateways drift as operators change policy, balances deplete, and phones move shelves — metrics without cadence rot quietly.

Document owners per SIM in the same registry as ICCID: who recharges, who receives alerts, who approves canary promotion after maintenance. Unowned SIMs become single points of silent failure.

Support ticket taxonomy should include deliverability fields: correlation ID, SIM slot, template_id, time user reports missing SMS. Support without correlation ID forces guesswork — train macros to collect ID from user app debug screen where available.

SLA worksheet (copy to wiki)

Traffic class: ___. Measurement window: ___ seconds. Minimum delivery rate: ___%. Maximum unknown ratio: ___%. Maximum P95 latency: ___ seconds. Alert relative drop: ___ points vs 7-day median. Minimum volume before alert: ___ messages. Primary ICCID: ___. Failover ICCID: ___. Webhook endpoint owner: ___. Reconciliation job owner: ___. Last canary date: ___. Last safe rate benchmark: ___ MPH.

Fill worksheet before production OTP. Re-sign when SIM, operator plan, or template family changes. Auditors and enterprise customers ask for this document during security review — ad hoc answers erode trust faster than a bad week of metrics.

Enterprise security review packet

Enterprise buyers ask how you measure SMS delivery on handset routes. Deliverability packet includes: SLA worksheet, sample dashboard screenshot per SIM, webhook architecture diagram, reconciliation policy, last canary report, and incident runbook links. Honest metrics beat marketing claims — compare architecture to cloud vendors on Twilio vs Android SMS gateway without pretending handset routes include vendor-managed analytics you have not built.

DLR field glossary from DLR documentation appended to packet. Security reviewers map your statuses to their GRC vocabulary — delivered, failed, unknown timeout must be defined in one page.

Prepaid markets appendix: link Bangladesh geo guide when fleet includes BD SIMs; explain balance monitoring as deliverability control. Reviewers in US HQ often miss prepaid cliff failure mode without explicit appendix.

Common mistakes

• Treating API accept rate as deliverability.
• No SLA timeout — pending messages inflate success forever.
• Single account rollup hiding one bad SIM in a fleet.
• Webhook consumer without idempotency — double-count or miss events.
• Mixing OTP and bulk on one SIM without class tags in metrics.
• Ignoring prepaid balance until failure cliff mid-campaign.
• Benchmarking once at launch, never after SIM swap or plan change.
• Comparing handset deliverability to Twilio marketing SLAs without building equivalent telemetry.
• Alert fatigue — thresholds without runbook links.
• Discarding raw DLR reason codes — operator tickets need them.

Closing the loop: segment → metric → action

Every segment export should produce a post-campaign report within 24 hours: sent, delivered, failed, STOP count, top failure codes, utilization vs pace cap, and recommendation (repeat, pause, or rewrite list query). Reports without action items become dashboard wallpaper — assign owner for each recommendation.

Compare segment reports week over week; seasonal segments (holiday promo) need year-over-year comparison not only wow delta — deliverability shifts when operator policies change annually.

Feed learnings back to hub program: when segment X consistently underperforms, document in android phone as sms gateway runbook appendix — future campaigns inherit institutional memory instead of rediscovering throttle limits.

Pair this deliverability program with throughput planning on the hub spoke for per-SIM safe rates — delivery rate without throughput headroom still fails OTP at peak. Document both metrics in the same wiki space so on-call engineers need not hunt across disconnected runbooks during incidents.

Next steps

Wire DLR webhooks, deploy the logging join, run a 500-message canary, and publish per-SIM delivery rate before the next product launch. Continue the hub: Android phone as SMS gateway cornerstone, compare routes on Twilio vs Android SMS gateway, Bangladesh context on SMS Gateway Bangladesh, status semantics on SMS delivery reports (DLR), endpoint design on SMS webhook integration, and plan limits on device and SMS volume pricing. Deliverability measurement is unglamorous — it is also the difference between “we sent it” and “they got it.” Revisit metrics after every SIM swap, template change, and fleet expansion — the hub program at android phone as sms gateway assumes you can prove delivery, not assume it. Schedule the next canary before you close the incident ticket — deliverability debt compounds when measurement pauses after every firefight. Treat the dashboard as production infrastructure with the same uptime expectations as the API itself.