Anurag’s Read

Adopting AV1 for Real-Time Communication (RTC) at Scale.

A detailed look at shipping a new codec on weak phones. It covers power, binary size, device gating, adaptive fallback, rate control and loss recovery with real trade-offs.
Score
8.3
Verdict
Worth reading
Blog and date
Meta,
Original length and level
16 min, advanced
Topics
video codec, mobile, rtc, performance
Read the original

Context

Many Meta calls run on low-end phones with 10 to 400 kbps of bandwidth. H.264 looks blurry below 100 kbps, and AV1 saves at least 20% bitrate offline. Real-time calls need under about 300 ms of end-to-end latency.

AV1 also costs more power, memory and binary size on weak devices.

Architecture

Each device gets an rtc_score from an ML model that uses logged performance metrics. The score decides whether the device uses AV1 at all. During a call the encoder runs a preset from high to ultra-low complexity.

The app watches encode latency and moves the preset up or down. If this is not enough, the sender switches to H.264. Receivers report decode latency, so the sender can switch when the peer is slow.

Rate control and loss recovery run on top of this.

Mechanisms

M1

Encoder and decoder choice

The open-source AV1 encoder used 14% more power than H.264 on a Pixel 8. Meta used an internal encoder and added an ultra-low-complexity preset that matches H.264 cost. A/B tests picked dav1d as decoder because it gave better power use and longer talk time.

M2

Device eligibility model

Memory, release year and Android version were poor predictors. The model takes low-level performance metrics from the logging pipeline. Version 1.1 shipped in August 2025. Version 2 uses two tiers for high-end and low-end devices and trains on AV1-only data.

M3

Complexity adaptation

Encode latency is tracked all the time. High latency lowers the preset, and spare headroom raises it. Preset and codec are chosen together to prevent flapping. Low battery also triggers H.264.

A mid-range phone may encode H.264 but still decode AV1 from a strong peer.

M4

Rate control

Average bitrate hides delay, so the team uses VBV delay. At 100 kbps a 20 kbit frame plus 5 kbit of backlog takes about 250 ms, above the 200 ms target. After an overshoot the encoder lowers the rate of later frames. Keyframe boosting is not used, and reference picture resampling changes resolution without a keyframe.

M5

Loss recovery

A two-layer temporal structure lets the base layer decode alone, and FEC protects only that layer. The encoder pins up to four long-term reference frames and drops the oldest. Flags and frame ids travel in proprietary RTP header extensions. The encoder predicts from the latest acknowledged reference, or sends a keyframe if none exists.

Design decisions

No.ChoseInstead ofWhy
D1Use an internal low-complexity encoderOpen-source AV1 encoderIt used 14% more power on a Pixel 8.
D2Ship the codec in the appDynamic download of the libraryDownload failures would hurt call quality.
D3Score devices with MLRules on memory, year or OS versionThese rules did not predict real AV1 speed.
D4Limit keyframe bitrateKeyframe boostingLarge keyframes cause spikes and freezes.

Key numbers

  • AV1 saves at least 20% bitrate versus H.264.
  • Target bitrates run from 10 to 400 kbps.
  • End-to-end latency target is under about 300 ms.
  • libAOM adds 1.7 MB uncompressed and about 600 kB compressed.
  • Keyframes are about 10 times larger than P-frames.
  • The long-term reference buffer holds 4 frames.

Failures and limits

  • Even a 2023 octa-core phone could not encode 320x180 at 15 fps, likely from CPU throttling.
  • AV1 encoding raised memory use and caused crash regressions.
  • Early conservative rate control caused undershoot, which slowed bandwidth ramp-up.
  • Older long-term references predict less well.
  • Group calls and hardware AV1 coverage remain open.

Lessons you can use

  • Test new codecs on power and memory, not only on quality.
  • Gate features with measured device performance instead of simple specs.
  • Watch runtime signals and keep a fallback path for slow devices.
  • Measure delay with buffer models, not average bitrate.

Glossary

VBVA buffer model that measures how much a stream overshoots its target bitrate.
LTR frameA frame kept as a stable reference for later prediction.
RPRA feature that changes resolution without a keyframe.
dav1dAn open-source AV1 decoder.