WebRTC call quality

For products built around a call — telehealth visits, patient monitoring, remote interpreting — the DOM stream cannot explain the failures users actually report. The SDK patches RTCPeerConnection and navigator.mediaDevices and records call state and quality metrics on the same timeline as everything else.

Media frames are never captured

Backstory records metrics and state only. The call’s audio and video never leave the device, in any configuration or privacy mode. That is what makes this safe to enable where the media itself is protected health information.

Enabling it

Backstory.init({
  projectKey: "pk_live_…",
  webrtc: { enabled: true },
});

The collector is a lazy module (backstory-webrtc.js), fetched only when enabled, so the core bundle is unaffected. It can also be switched on centrally from project settings without a deploy.

Configuration

Option	Type	Default	Notes
enabled	boolean	false	Master switch. Loads the lazy module when true
sampleIntervalSec	number	5	Seconds between emitted quality samples at full health. getStats() is polled internally at 1 Hz regardless; this controls what is sent
captureSdp	boolean	false	Capture redacted session descriptions. Never honored in strict privacy mode
emitOnThresholdCross	boolean	true	Emit a sample immediately when a detector threshold is crossed, even while sampling is thinned
maxPeerConnections	number	8	Connections tracked at once. Extras are counted but not sampled

What is captured

Lifecycle and error events

Emitted immediately, never thinned, because they are rare and individually valuable.

Event	When
pc_created	A peer connection was constructed. ICE servers are counted, never recorded verbatim
connection_state, ice_state, ice_gathering, signaling_state	Any state transition, with the time spent in the previous state
ice_candidate_error	A STUN or TURN server was unreachable or refused, with the error code
candidate_pair_change	The media path changed, with local and remote candidate types and whether it is now relayed
track_added, track_muted, track_unmuted, track_ended	Track lifecycle. A remote track muting is the freeze the user sees; unmute carries how long it lasted
codec_change, resolution_change	Encoder or decoder adaptation mid-call
ice_restart	The call is recovering from a network change
media_acquired	Capture succeeded, with resolution and frame rate
media_error	Capture failed. The most actionable event we record; see the table below
device_inventory, device_change	Device counts at start, and any change mid-session (a headset unplugged)
data_channel	Channel label and state. Payloads are never captured
pc_closed	With the call duration

Media errors are classified by DOMException name, which is a fixed and far more useful vocabulary than the message text:

Class	DOMException	Meaning
permission_denied	NotAllowedError	The user or a policy blocked access
device_in_use	NotReadableError	Another application holds the camera or microphone
device_not_found	NotFoundError	No such device present
constraints_unsatisfied	OverconstrainedError	The requested constraints cannot be met
aborted, security, type_error	AbortError, SecurityError, TypeError	Hardware failure, insecure context, empty constraints

Quality metrics

Polled from getStats() at 1 Hz, differenced on the device, and emitted as a compact per-track delta. What arrives is already the computed answer.

Field	Meaning
lossPct	Packet loss over the window, 0 to 100
rttMs	Round trip time from the selected candidate pair
jitterMs, jitterBufferMs	Network instability, and average jitter buffer delay
freezes, freezeMs	Video freeze count and total duration. Derived from frame deltas on browsers that do not report them directly
frameDropPct	Dropped over decoded frames
concealmentPct	Audio samples the decoder invented to cover gaps. Above roughly 5% is audible
audioLevel	0 to 1, from the receiver or the local source
fps, w, h	Frame rate and resolution
kbps, availableOutgoingKbps, availableIncomingKbps	Throughput and the congestion controller’s estimate
qualityLimitation, limitedByCpuPct, limitedByBandwidthPct	Whether the device or the network was the constraint
nackCount, pliCount, keyFrames	Retransmission and recovery churn
codec	Negotiated codec MIME type
stalled	No packets moved on a live track in the window

Each sample also carries the connection state, the candidate types, whether the path is relayed, and the network type.

Sampling ladder

Quality sampling degrades with the recorder’s memory and bandwidth governor rather than switching off abruptly.

Governor level	Interval	Behavior
Healthy	5 s	Full sampling
Constrained	15 s	Lower cadence
Degraded	30 s	Only samples that cross a threshold
Survival, paused	off	Lifecycle events only

Privacy

Data	Behavior
Audio and video frames	Never captured
ICE candidate addresses	Dropped. Only candidate type, protocol, network type, and a per-session salted hash are kept
Session descriptions (SDP)	Off by default; never in strict. When enabled, a=candidate, c=, a=ice-ufrag, and a=ice-pwd lines are stripped and the remainder runs through the PII detectors
deviceId	Always hashed. It is a fingerprinting surface
Device labels	Recorded as the browser supplies them
Data channel payloads	Never captured; label and state only

Detectors

Thirteen deterministic detectors run server-side on every call, cluster across sessions, and route through the normal notification rules. Thresholds are tunable per project.

Detector	Default rule	Severity
webrtc_connect_failure	Never reached connected within 15 s, or reached failed	Critical
webrtc_one_way_media	One direction moved zero packets for 6 s while the other flowed and both were negotiated	Critical
webrtc_media_permission_denied	NotAllowedError	High
webrtc_device_unavailable	NotReadableError or NotFoundError	High
webrtc_freeze	A single freeze over 2 s, or more than 5% of the call frozen	High
webrtc_audio_dropout	Concealment above 5% sustained 10 s	High
webrtc_reconnect_storm	3 or more ICE restarts or path changes within 60 s	High
webrtc_call_abandoned	Ended inside 30 s with a quality score below 60	High
webrtc_packet_loss	Above 5% sustained 10 s	Medium
webrtc_high_latency	Round trip time above 300 ms sustained 15 s	Medium
webrtc_cpu_limited	Encoder CPU-limited for more than 20% of the call	Medium
webrtc_low_resolution	Below 240 px high for 20 s after a better resolution was seen	Low
webrtc_relay_fallback	The selected path is a TURN relay	Info

Every call also gets a composite quality score from 0 to 100, weighted toward loss and freezes. The same function computes it in the SDK and on the server, so a detector and a dashboard can never disagree.

Triggers

Every detector is available as a recording trigger, named webrtc:<kind>. Use them in conditional recording to store only sessions where a call went wrong, and in monitor mode to record a burst around each problem.

Backstory.init({
  projectKey: "pk_live_…",
  webrtc: { enabled: true },
  recording: {
    mode: "conditional",
    // Store the session if the call misbehaves, otherwise discard it.
    triggers: ["webrtc:connect_failure", "webrtc:one_way_media", "webrtc:freeze"],
    // Telehealth monitoring: a dropped feed records the minute before it happened.
    monitorModeAfterMinutes: 15,
    monitorTriggers: ["webrtc:freeze", "webrtc:reconnect_storm", "webrtc:connect_failure"],
  },
});

See recording controls for how conditional recording and monitor mode work.

Connections created before the SDK loads

If your app constructs a peer connection before Backstory.init() runs (a call joined from a deep link, for instance), the constructor patch cannot see it. Hand it over explicitly:

const pc = new RTCPeerConnection(config);
Backstory.webrtc.attach(pc); // safe to call more than once

Backstory.webrtc.list() returns the tracked connections with their current state, relay status, latency, loss, frame rate, and quality score, which is useful in your own diagnostics UI and is what the debug panel renders.

Verify it with the SDK demo

Run pnpm --filter @backstory/sdk demo, open http://localhost:5500, and choose Run complete test in the WebRTC section. The loopback uses real peer connections and records connection setup, audio/video tracks, a data channel, one-second quality samples, a 3.5-second video freeze, an ICE restart, an unsatisfied camera constraint, and a clean hangup.

Open the resulting session and select Media in DevTools. The session header also shows a WebRTC health badge when calls are present; selecting it opens Media directly. Freeze bands align with the replay playhead, while the summary shows total freeze time, the worst call-quality score, failed connections, and device errors. Track rows expose codec, resolution, frame rate, loss, jitter, bitrate, dropped frames, recovery requests, stalls, and audio concealment when the browser reports them.

Mobile

The native SDKs instrument the platform WebRTC stacks and normalize into the same metric shape, so one detector set and one Calls view serve web and mobile.

// iOS
Backstory.start(options)
Backstory.webrtc.attach(peerConnection)   // RTCPeerConnection from the WebRTC framework

// Android
Backstory.start(context, options)
Backstory.webrtc().attach(peerConnection)

// Flutter
Backstory.webrtc.attach(peerConnection);

// React Native
import { Backstory } from "@backstory/react-native";
Backstory.webrtc.attach(pc);

Vendor SDKs that hide the peer connection

The web SDK currently needs access to the underlying RTCPeerConnection. Initialize Backstory before the call vendor so newly constructed connections are patched automatically, or use Backstory.webrtc.attach(peerConnection) when the vendor exposes one. Vendors that expose neither the connection nor a standards-based stats hook cannot provide call metrics to the web SDK yet.

API

Endpoint	Purpose
GET /v1/calls	List calls with quality score, setup time, relay status, and issues. Filterable by network type, browser, device, release, quality range, and whether the call connected
GET /v1/calls/{id}	One call with the full metric series and event markers
GET /v1/sessions/{id}/calls	Calls within a session, used by the Media panel
GET /v1/calls/overview	Connect failure rate, relay rate, setup time percentiles, quality distribution, top failure reasons, and cohort breakdowns

Troubleshooting: what a user reports, and what proves it

They say	Look at	Detector
“It was choppy”	freezes and freezeMs on the inbound video track; frameDropPct	webrtc_freeze
“They could not hear me”	Outbound audio packets against the remote report; stalled; audioLevel at zero while unmuted	webrtc_one_way_media
“The video never appeared”	Connection state never reaching connected; ice_candidate_error; setup time	webrtc_connect_failure
“My camera does not work”	media_error with class device_in_use or permission_denied	webrtc_device_unavailable, webrtc_media_permission_denied
“The audio kept cutting out”	concealmentPct on the inbound audio track	webrtc_audio_dropout
“It was blurry the whole time”	w and h on the outbound track, plus qualityLimitation	webrtc_low_resolution, webrtc_cpu_limited
“We kept talking over each other”	rttMs and jitterBufferMs	webrtc_high_latency
“It dropped and came back repeatedly”	ice_restart and candidate_pair_change counts	webrtc_reconnect_storm
“It only breaks at the office”	localCandidate and remoteCandidate types, and relayed	webrtc_relay_fallback

Browser support

getStats() field coverage varies. Every field is feature-detected and omitted rather than zero-filled when unavailable, and freezes are derived from frame deltas where freezeCount is missing. Chromium browsers provide the fullest set, including the quality limitation reason; Safari and Firefox provide fewer fields, so some detectors are less sensitive there. The detectors themselves degrade gracefully: a missing input means the rule does not fire, never a false positive.