Computer Networking › Section B · The application layer › Topic B.8
Real-time voice and video
A phone call or video meeting cannot buffer seconds ahead the way a film can. Its media must cross the network in a fraction of a second, so real-time applications are built around UDP, timing information, small buffers and clever ways of hiding the packets that never arrive in time.
The delay budget of a conversation
Conversation is interactive, so what matters is mouth-to-ear delay: the time from a word being spoken to its being heard. ITU-T guidance treats up to about 150 ms one way as comfortable for most users, and beyond about 400 ms people start talking over each other. That budget covers encoding, packetising, crossing the network, buffering, decoding and playing. Waiting for a lost packet to be retransmitted can take a whole extra round trip, which usually breaks the budget. So real-time media normally travels over UDP, and a packet that misses its moment is simply skipped.
Packetising voice
A voice sender collects a short slice of audio, typically 20 ms, encodes it and sends it as one packet. Shorter slices reduce delay but add more headers per second; longer slices do the reverse. Each packet carries an RTP header of 12 bytes, a UDP header of 8 bytes and an IP header of 20 bytes for IPv4 or 40 bytes for IPv6. For a low-rate codec, these headers can be as large as the audio itself, so the rate on the wire is well above the codec's rate. Video frames are much larger and are split across several packets, all carrying the same timestamp.
Worked example: the real bit rate of a call
A call uses the 64 kbit/s telephone codec with 20 ms packets over IPv4. 1. Audio per packet: \(64{,}000 \times 0.020 = 1{,}280\) bits, which is 160 bytes. 2. Headers: \(12 + 8 + 20 = 40\) bytes, so each packet is 200 bytes. 3. Packets per second: \(1 \div 0.020 = 50\). 4. Rate on the wire: \(200 \times 8 \times 50 = 80{,}000\) bit/s, or 80 kbit/s, 25% more than the codec rate. 5. With Opus at 24 kbit/s instead, each packet carries 60 bytes of audio plus 40 of headers, so the rate is 40 kbit/s, two-thirds above the codec rate. Check: the lower the codec rate, the larger the share taken by headers.
RTP and RTCP
The Real-time Transport Protocol (RTP), specified in RFC 3550, runs in applications on top of UDP and labels each media packet. Its header holds a payload type, naming the codec; a sequence number, which rises by one per packet so the receiver can spot losses and reordering; a timestamp, giving the sampling instant of the media so it can be played at the right moment; and a synchronisation source identifier (SSRC), distinguishing streams, such as one person's audio from another's video. Its companion, the RTP Control Protocol (RTCP), sends periodic reports of loss, jitter and delay, which senders use to adjust their bit rate. RTP itself guarantees nothing about delivery.
Jitter and the playout buffer
Packets leave the sender every 20 ms but arrive with varying delays, because they meet different queues on the way. This variation is jitter. Playing each packet the moment it arrives would make speech speed up and stutter. The receiver therefore holds packets in a jitter buffer and plays each at a fixed time after it was sent, using the RTP timestamps. The playout delay is a trade-off: a longer delay means fewer packets arrive too late, but the conversation lags more. Adaptive jitter buffers estimate the current delay and its variation, and adjust the playout delay, usually during silences, where a change is inaudible.
Worked example: a playout deadline
Packets are sent at 0, 20, 40, 60 and 80 ms and take 45, 70, 50, 105 and 60 ms to arrive. 1. With a playout delay of 80 ms, packet 1 must arrive by 80 ms, packet 2 by 100 ms, and so on. 2. Arrivals are at 45, 90, 90, 165 and 140 ms. 3. Packet 4 arrives at 165 ms, after its playout time of 140 ms, so it is too late and is treated as lost. 4. With a playout delay of 110 ms, packet 4's deadline is 170 ms and all five play. Check: the longer delay saves a packet but adds 30 ms to every word, eating into the 150 ms budget.
Recovering from loss
Late or lost packets cannot wait for retransmission, so other tools hide them. Forward error correction (FEC) sends extra information so a missing packet can be rebuilt, for example a parity packet for every few media packets, at the cost of extra bandwidth. Opus can carry a low-rate copy of the previous frame in each packet, so one lost packet costs only quality, not silence. Packet loss concealment fills a gap by repeating or smoothly stretching nearby audio. Video decoders patch missing blocks from the previous frame, or ask for a fresh key frame.
Setting up calls: SIP and SDP
Before media flows, the two ends must find each other and agree on codecs, addresses and ports. The Session Initiation Protocol (SIP), RFC 3261, does this with text messages much like HTTP's: an INVITE request starts a call, responses such as 180 Ringing and 200 OK follow, an ACK confirms, and BYE ends it. The media details travel inside as a Session Description Protocol (SDP) description, which lists the codecs each side can use. SIP is widely used in internet telephony and in mobile operators' voice over LTE and Wi-Fi calling.
WebRTC and getting through NAT
WebRTC, standardised jointly by the W3C and the IETF and built into all major browsers, gives web pages and apps real-time audio, video and data channels. It deliberately leaves signalling to the application, which exchanges SDP offers and answers over any channel, such as a WebSocket. Most devices sit behind NAT, so peers use ICE to find a working path: STUN servers tell each device its public address, and if no direct path works, a TURN server relays the media. All WebRTC media is encrypted, with keys agreed through DTLS and media carried as Secure RTP (SRTP).
Group calls and network priority
Sending your video separately to every participant in a group call would exhaust an upload link, so most services use a selective forwarding unit (SFU), a server to which each participant uploads once, often in several resolutions at the same time, a technique called simulcast. The SFU forwards to each viewer the version that suits that viewer's screen and connection, without decoding the video. Inside managed networks, such as companies, packets can be marked with Differentiated Services (DiffServ) code points so routers queue voice ahead of bulk traffic; on the public internet, such markings are generally not honoured.
Common mistakes to avoid
First, thinking real-time media uses TCP for reliability; retransmissions arrive too late to play. Second, believing RTP guarantees delivery or timing; it only labels packets so the receiver can cope. Third, confusing the sequence number, which detects loss, with the timestamp, which schedules playout. Fourth, ignoring header overhead: a 24 kbit/s codec can need about 40 kbit/s on the wire. Fifth, assuming a bigger jitter buffer is always better; it adds delay to every word. Sixth, thinking SIP carries the voice; it sets up the call, and RTP carries the media.
Key takeaways
Conversation needs about 150 ms or less one way, so real-time media uses UDP and skips late packets. Voice is sent in packets of about 20 ms, and RTP, UDP and IP headers add 40 bytes or more to each. RTP, RFC 3550, carries payload type, sequence number, timestamp and SSRC; RTCP reports loss and jitter. Jitter buffers play packets at a fixed delay after sending, trading lateness against lag. FEC, redundancy and concealment hide losses. SIP with SDP sets up calls. WebRTC brings encrypted real-time media to browsers, using ICE, STUN and TURN through NAT. SFUs with simulcast scale group calls.
Flashcards
Flashcard
Time from speaking a word to hearing it
Test your understanding
Question 1 of 5
A call uses the 64 kbit/s codec with 20 ms packets over IPv6, so the headers are 12 + 8 + 40 bytes. What is the bit rate on the wire?
Question 2 of 5
Packets are sent every 20 ms and played 80 ms after sending. One packet, sent at 40 ms, arrives at 130 ms. What happens to it?
Question 3 of 5
Which RTP header field lets a receiver notice that a packet has gone missing?
Question 4 of 5
Two WebRTC peers are both behind NAT and no direct path between them works. How does the call still connect?
Question 5 of 5
Why do video-call apps rarely retransmit lost media packets?
