The Application Layer

The Application layer is where network-aware software lives. It defines protocols that user programs — browsers, email clients, SSH terminals — use to communicate. At this layer we deal with names, resources, and services, not packets or frames.

1. Application Architectures and Transport Service Requirements

An application-layer protocol defines how an application's processes, running on different end systems, exchange messages. It specifies message types (request, response), syntax (field structure), semantics (meaning of fields), and the rules for when and how processes transmit and respond. Open protocols are specified in public RFC standards (e.g. HTTP, SMTP, DNS) to enable multi-vendor interoperability, while proprietary protocols (e.g. Zoom, Skype) remain vendor-controlled.

Application processes communicate by writing to and reading from sockets, addressed by the destination host's IP address and the transport port number. Distributed applications typically follow one of two major architectural paradigms:

Exam note: Peer-to-peer (P2P) and overlay networks are covered in a separate course and will NOT be tested on the exam; the exam focuses on Client-Server application protocols (HTTP, DNS, DHCP, SMTP, POP3, IMAP).

What applications require from the transport layer: Different applications impose radically different demands across four service dimensions:

• Reliable data transfer: Financial transactions, web documents, and email cannot tolerate loss; audio and video streaming can often tolerate minor packet loss.
• Throughput: Video streaming requires minimum sustained bandwidth; elastic applications (text chat, email) adapt to whatever bandwidth is available.
• Timing (latency and jitter): Real-time gaming and VoIP require tight upper bounds on delivery delay; web browsing and file transfers tolerate latency variance.
• Security: Confidentiality, integrity, and endpoint authentication.

Crucial internet principle: Standard TCP and UDP provide zero timing guarantees and zero throughput guarantees. Packets are delivered with variable delay, and transmission rate fluctuates dynamically with network congestion. Security is not native to core TCP or UDP; it is layered on top via TLS (Transport Layer Security).

2. HTTP — HyperText Transfer Protocol

HTTP is a request/response application-layer protocol. The client sends a request message; the server returns a response. HTTP is stateless: each request is independent.

HTTP message format

HTTP/1.1 messages are human-readable ASCII text streams structured into well-defined sections:

# ── HTTP Request Format ──
GET /articles/networking.html HTTP/1.1        ← Request line (Method, URI, Protocol version)
Host: example.com                             ← Header lines (Host is MANDATORY in HTTP/1.1)
User-Agent: Mozilla/5.0 (Macintosh; Intel...) ← Client software identifier
Accept: text/html,application/xhtml+xml        ← Preferred media types
Accept-Language: en-GB,en;q=0.9               ← Language preference
Connection: keep-alive                        ← Keep underlying TCP connection open
                                              ← Mandatory blank line (CRLF) separates headers from body
[Optional Request Body, e.g. for POST/PUT]

# ── HTTP Response Format ──
HTTP/1.1 200 OK                               ← Status line (Protocol version, Status code, Status phrase)
Date: Wed, 14 Oct 2026 09:30:00 GMT           ← Timestamp of response generation
Server: Apache/2.4.52 (Ubuntu)                ← Server software
Content-Type: text/html; charset=UTF-8        ← MIME type of the payload
Content-Length: 4218                          ← Exact body length in bytes
Cache-Control: max-age=3600                   ← Caching directive
                                              ← Mandatory blank line (CRLF)
<!DOCTYPE html><html><body>...</body></html>  ← Entity body

Persistent vs. non-persistent HTTP

In non-persistent HTTP (HTTP/1.0 default), each TCP connection is closed immediately after the server transmits a single response. In persistent HTTP (HTTP/1.1 default), the TCP connection remains open across successive requests, allowing multiple objects to be fetched over the same socket.

Worked latency calculation: Consider a web page consisting of 1 base HTML file and 10 small embedded images located on the same server. Let round-trip time between client and server be RTT = 50 ms. Assume packet transmission delays are negligible.

• Non-persistent HTTP (sequential): Each object requires an independent TCP connection. Establishing TCP takes 1 RTT (SYN + SYN-ACK); requesting and receiving the object takes 1 RTT (HTTP request + first data byte). Each object costs 2 RTT = 100 ms.
  • Total time = (1 HTML + 10 images) × 2 RTT = 11 × 2 RTT = 22 RTT = 1,100 ms (1.10 s).
• Persistent HTTP without pipelining: One initial TCP handshake (1 RTT), then base HTML is requested and received (1 RTT). The client parses HTML and requests each of the 10 images sequentially over the existing open TCP socket, taking 1 RTT per image.
  • Total time = 1 RTT (handshake) + 1 RTT (HTML) + 10 RTT (images) = 12 RTT = 600 ms (0.60 s).
• Persistent HTTP with pipelining: After receiving base HTML, the client sends requests for all 10 images back-to-back without waiting for intervening responses. The server replies in a single burst.
  • Total time = 1 RTT (handshake) + 1 RTT (HTML) + 1 RTT (pipelined images) = 3 RTT = 150 ms (0.15 s).
• Non-persistent HTTP with K parallel TCP connections: The browser first downloads the base HTML over 1 connection (2 RTT). After discovering the M embedded images, it fetches them across K concurrent connections in parallel batches of size K, where each batch takes 2 RTT (1 RTT TCP handshake + 1 RTT HTTP request/response).
  • Formula: Total time = 2 RTT (base HTML) + 2 × ceil(M / K) × RTT.
  • For 1 base HTML + 10 images with K = 5 parallel connections (RTT = 50 ms): Total time = 2 RTT + 2 × ceil(10 / 5) RTT = 2 RTT + 4 RTT = 6 RTT = 300 ms (0.30 s).

To reduce non-persistent latency without pipelining, modern browsers open parallel TCP connections (typically up to 6 connections per domain), allowing multiple objects to download concurrently at the expense of higher connection overhead.

HTTP/1.1 (RFC 9112) — the baseline

Common HTTP/1.1 request methods:

HTTP/1.1 added persistent connections (keep-alive by default) to reuse one TCP connection for multiple requests. Without keep-alive, each request required a new TCP+TLS handshake. However, requests are still serialised per connection (head-of-line blocking), so browsers open typically about 6 parallel connections per origin. (HTTP/1.1 also defined pipelining to send multiple requests without waiting, but it is practically unused due to proxy bugs and HOL blocking).

HTTP status codes

Statelessness and Session Management (Cookies & Sessions)

By architectural design, HTTP is a stateless protocol (RFC 9110). Each request from a client is executed as an isolated transaction, without the server retaining memory or context from prior interactions. Statelessness simplifies distributed system design: backend servers can be scaled horizontally behind a load balancer, any node can process any request without shared application memory, and server crashes do not corrupt client conversations.

However, practical web applications require state: remembering user logins, persisting shopping baskets across pages, tracking multi-step wizard forms, and applying authorization policies. Because the transport and application protocols provide no native concept of a continuous "user session", developers bridge the gap using three complementary mechanisms:

1. HTTP Cookies (RFC 6265)

A cookie is a small key-value pair generated by a web server and stored locally by the user's browser. When the server wishes to establish state, it returns the Set-Cookie HTTP response header. The browser stores this pair and automatically attaches it in the Cookie header on every subsequent request matching the cookie's domain and path (setting Domain=example.com also sends the cookie to subdomains).

# Server response establishing an authenticated session cookie:
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Set-Cookie: sid=s%3A9xKz81LmPq4.7A; Path=/; Domain=example.com; Secure; HttpOnly; SameSite=Lax; Max-Age=86400

# Subsequent browser request to any URL on example.com:
GET /dashboard HTTP/1.1
Host: example.com
Cookie: sid=s%3A9xKz81LmPq4.7A

Third-party cookies and privacy regulation: While first-party cookies belong to the domain in the browser address bar, third-party cookies are set by embedded resources (tracking pixels, analytics, ad networks) across disparate websites to build user browsing profiles. Increasing privacy regulations (such as the EU ePrivacy Directive and GDPR) require explicit consent banners prior to setting non-essential cookies, and modern browsers increasingly phase out third-party cookies by default.

Stateless vs. stateful protocol contrast: Protocols differ fundamentally in where state is retained:

Key security attributes defined in RFC 6265 (and its updates) control cookie exposure:

• HttpOnly: Forbids client-side scripts (such as document.cookie) from accessing the cookie. This prevents Cross-Site Scripting (XSS) attacks from stealing authentication tokens.
• Secure: Directs the browser to send the cookie exclusively over encrypted HTTPS connections, preventing cleartext sniffing on local Wi-Fi or transit links.
• SameSite: Governs whether cookies are attached to cross-site requests, mitigating Cross-Site Request Forgery (CSRF):
  • Strict: Never included on cross-site requests, even when following an external hyperlink.
  • Lax (modern browser default): Withheld on cross-site sub-resource requests (images, iframes, AJAX), but sent when the user navigates via a top-level GET link.
  • None: Sent with all cross-site requests; requires the Secure flag.
• Max-Age / Expires: Defines cookie lifetime. Omission creates a session cookie, destroyed when the browser process terminates.

2. Server-side sessions

In a traditional session architecture, the actual state data (user profile, permissions, shopping cart items) is stored on the server — in RAM, a relational database, or an in-memory key-value cache such as Redis. The client receives only an opaque, cryptographically random session ID (e.g. sid=9xKz...) stored in an HttpOnly cookie.

The scaling dilemma Beyond: When scaling to multiple web servers behind a load balancer, subsequent requests from the same user might hit different physical machines. Solutions include:

• Sticky sessions (session affinity): The load balancer hashes the client IP or cookie to route the user consistently to the same backend node. Drawback: Uneven load distribution; if a server crashes, all its active user sessions vanish.
• Centralised distributed cache: All application servers connect to a shared Redis or Memcached cluster. Any server can retrieve or update the user's session state. Drawback: Adds a network hop per request and creates a critical infrastructure dependency.

3. Stateless tokens (JWT — JSON Web Tokens, RFC 7519) Beyond

To eliminate server-side session lookups entirely, modern architectures frequently employ self-contained signed tokens. A JSON Web Token (JWT) encodes all required user claims directly in the token itself and signs it cryptographically.

On the wire, a JWT is transmitted as three Base64URL-encoded strings concatenated by periods: xxxxx.yyyyy.zzzzz (Header.Payload.Signature):

# Wire format: three Base64URL strings
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyXzEwNDIiLCJyb2xlIjoiZWRpdG9yIiwiZXhwIjoxNzI4ODk2NDAwfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c

# Decoded structure (for illustration):
# 1. Header (algorithm & token type)
{"alg": "HS256", "typ": "JWT"}

# 2. Payload (claims — subject, role, expiration)
{"sub": "user_1042", "role": "editor", "exp": 1728896400}

# 3. Signature (verifies integrity and authenticity)
HMACSHA256(base64Url(header) + "." + base64Url(payload), server_secret_key)

Crucial security note: The payload is only Base64URL-encoded, not encrypted. Anyone who intercepts the token can decode and view its claims — never store secrets, passwords, or sensitive keys in a JWT payload! Furthermore, with symmetric HS256, every verifying service requires the shared secret key, enabling any backend service to forge tokens. For distributed microservices, use asymmetric key pairs (RS256 or ES256) where only the authentication issuer holds the private key while resource servers verify with the public key.

The client sends the token in the Authorization: Bearer <token> header. Any backend service holding the public key can verify the signature and authenticate the request without querying a central session database.

Comparison: Cookies, Server Sessions, and Stateless Tokens

Web caching and conditional GET

A web cache (proxy server) satisfies HTTP requests on behalf of an origin server. Caches reside in client browsers, corporate proxies, and Content Delivery Networks (CDNs), drastically reducing bandwidth consumption and server load.

Caching behavior is governed by HTTP response headers:

• Cache-Control: max-age=3600: The cached copy is fresh for 3,600 seconds (1 hour). Subsequent requests within this window are served immediately from local cache without contacting the server.
• Cache-Control: no-cache: The client may cache the copy, but must revalidate it with the origin server before every use.
• Cache-Control: no-store: The response contains sensitive data and must not be written to disk or cache storage anywhere.

Conditional GET validation: When a cached object expires, the cache validates whether the file was modified on the server using conditional validators:

• If-Modified-Since: <HTTP-date>: Paired with server's previous Last-Modified header.
• If-None-Match: "33a64df5": Paired with server's entity tag (ETag), a cryptographic content hash.

# Client validation request:
GET /styles.css HTTP/1.1
Host: example.com
If-None-Match: "v1.4.2"
If-Modified-Since: Wed, 14 Oct 2026 08:00:00 GMT

# Server response if the file has NOT changed:
HTTP/1.1 304 Not Modified
Date: Wed, 14 Oct 2026 09:30:00 GMT
ETag: "v1.4.2"
Cache-Control: max-age=3600
                                              ← Empty body: saves bandwidth!

Because the server sends status 304 Not Modified with an empty body, the client reuses its cached copy, saving transmission time and bandwidth.

3. E-mail protocols — SMTP, POP3, and IMAP

Unlike real-time web traffic, electronic mail was designed from the beginning as an asynchronous, store-and-forward distributed system. When you send an email, the recipient's computer does not need to be turned on or connected to the network. Intermediate servers hold and forward the message until it arrives in the recipient's mailbox.

Email architecture: MUA, MSA, MTA, and MDA

Delivering an email involves four distinct functional agents defined across RFC 5321 and RFC 6409:

SMTP dialogue (RFC 5321)

SMTP (Simple Mail Transfer Protocol) is a text-based, command-response protocol running over TCP. Commands are four-letter ASCII strings followed by parameters; replies are three-digit numbers followed by descriptive text.

# S = Server (receiving MTA), C = Client (sending MTA/MSA)
S: 220 mail.example.com ESMTP Postfix
C: EHLO mail.sender.org
S: 250-mail.example.com
S: 250-PIPELINING
S: 250-SIZE 35882577
S: 250-STARTTLS
S: 250 8BITMIME
C: MAIL FROM:<alice@sender.org>
S: 250 2.1.0 Ok
C: RCPT TO:<bob@example.com>
S: 250 2.1.5 Ok
C: DATA
S: 354 End data with <CR><LF>.<CR><LF>
C: From: "Alice" <alice@sender.org>
C: To: "Bob" <bob@example.com>
C: Date: Wed, 14 Oct 2026 09:30:00 +0000
C: Subject: Project Architecture Review
C: 
C: Hi Bob, the network deployment plan is attached.
C: .
S: 250 2.0.0 Ok: queued as 4YtK9x1pLsz04
C: QUIT
S: 221 2.0.0 Bye

Envelope sender vs. header From: Notice that the sender was declared twice: once as MAIL FROM:<alice@sender.org> (the envelope sender) and once inside the DATA payload as From: "Alice" <alice@sender.org> (the message header). The envelope sender is recorded by the final delivery MTA in the Return-Path header; MTAs route all bounce error reports strictly to this address. SMTP limits line lengths to a maximum of 1,000 octets including the trailing CRLF.

SMTP reply code classes: Server responses use three-digit status codes indicating transmission state:

• 2xx (Positive Completion): Action successfully completed (e.g. 250 OK, 220 Service ready).
• 3xx (Intermediate Positive): Command accepted, awaiting further input (e.g. 354 Start mail input; end with <CRLF>.<CRLF>).
• 4xx (Transient Negative Failure): Temporary issue (e.g. 451 Requested action aborted: local error, mailbox temporarily full, greylisting). The sending MTA queues the message and retries delivery periodically.
• 5xx (Permanent Negative Failure): Permanent rejection (e.g. 550 No such user here, 554 Transaction failed). The sender does not retry and generates a bounce message.

Dot-stuffing (transparency rule): Because SMTP terminates the DATA block with a line containing only a single period (CRLF.CRLF), any message body line that naturally begins with a period would prematurely terminate message delivery. To prevent this, sending MTAs prepend an extra period (..) to any line starting with a period. Receiving MTAs strip the leading period upon reception.

SMTP vs. HTTP comparison: Although both are ASCII command-response protocols running over persistent TCP:

• Push vs. Pull: SMTP is fundamentally a push protocol (the sending machine pushes data to the receiver), whereas HTTP is a pull protocol (the client pulls resources from the server).
• Payload packaging: SMTP packages all textual parts and binary attachments into a single composite multipart MIME message transmitted in one DATA block. In contrast, HTTP encapsulates each distinct object (HTML, image, CSS) in its own separate response message.

Mail retrieval dialogues: POP3 vs. IMAP

Both retrieval protocols establish stateful sessions over TCP:

# ── POP3 Session (Port 110/995): Authorization → Transaction → Update ──
S: +OK POP3 server ready
C: USER bob
S: +OK User name accepted
C: PASS secret123
S: +OK Logged in
C: LIST
S: +OK 2 messages (3200 octets)
S: 1 1200
S: 2 2000
S: .
C: RETR 1
S: +OK 1200 octets
S: [Message 1 content...]
S: .
C: DELE 1
S: +OK Message 1 marked for deletion
C: QUIT
S: +OK Logging out (messages marked for deletion are permanently purged)

# ── IMAP Session (Port 143/993): Not Authenticated → Authenticated → Selected ──
S: * OK IMAP4rev1 Service Ready
C: a001 LOGIN bob secret123
S: a001 OK LOGIN completed
C: a002 SELECT INBOX
S: * 18 EXISTS
S: * 2 RECENT
S: a002 OK [READ-WRITE] SELECT completed
C: a003 FETCH 1 BODY[HEADER.FIELDS (FROM SUBJECT DATE)]
S: * 1 FETCH (BODY[HEADER.FIELDS (FROM SUBJECT DATE)] {112}
S: From: Alice <alice@sender.org>
S: Subject: Architecture Review
S: Date: Wed, 14 Oct 2026 09:30:00 +0000
S: )
S: a003 OK FETCH completed
C: a004 LOGOUT
S: * BYE Server logging out
S: a004 OK LOGOUT completed

MIME — Multipurpose Internet Mail Extensions (RFC 2045–2049)

Original SMTP restricted message bodies to 7-bit US-ASCII lines no longer than 1,000 characters. MIME generalised email to support non-ASCII character sets (UTF-8), formatted HTML, binary attachments (images, PDFs), and multipart messages.

MIME adds headers to the message header and to each part of the body:

MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="frontier_bnd_42"

--frontier_bnd_42
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Hi Bob, please find the quarterly deployment plan attached.

--frontier_bnd_42
Content-Type: application/pdf; name="plan.pdf"
Content-Disposition: attachment; filename="plan.pdf"
Content-Transfer-Encoding: base64

JVBERi0xLjQKJcTl8uXrp/Og0MTGCjQgMCBvYmoKPDwKL0ZpbHRlciAvRmxhdGVEZWNvZGUK...
--frontier_bnd_42--

POP3 vs. IMAP

Once mail is delivered to the recipient's mailbox by the MDA, the recipient retrieves it using one of two protocols:

Webmail (HTTP): Webmail clients (e.g. Gmail, Outlook on the web) interact with mail servers differently: the user accesses the web application over HTTPS; the web application server then acts as an MUA communicating via IMAP or internal RPCs with the mailbox store and via SMTP with the MSA.

STARTTLS vs. Implicit TLS

Email encryption over the transport layer can be negotiated in two fundamentally different ways:

• Implicit TLS (Direct TLS): The client initiates an immediate TLS handshake right upon connecting, before any application protocol text is sent. Dedicated port numbers are assigned: SMTPS on port 465, IMAPS on port 993, POP3S on port 995. This is the modern recommended approach (RFC 8314) because there is no plain-text window.
• Explicit TLS (STARTTLS): The connection begins as unencrypted plain text on the standard port (e.g., port 587 for mail submission, port 25 for inter-MTA relay, or port 143 for IMAP). The server advertises STARTTLS in its capability list; the client sends the STARTTLS command; both sides immediately switch the established socket to TLS.

The STRIPTLS vulnerability: Because STARTTLS begins in cleartext, an on-path attacker (MITM) can intercept the server's initial greeting and delete the STARTTLS capability keyword before passing it to the client. Unsuspecting clients fall back to sending credentials and email bodies in cleartext. Server-to-server SMTP over TCP port 25 usually relies on opportunistic TLS (encryption without strict certificate validation), making it susceptible to tampering. Technologies like MTA-STS (RFC 8461 Beyond) and DANE (RFC 7672 Beyond) enforce strict TLS policies and DNSSEC verification to prevent downgrade attacks.

4. DNS — Domain Name System (RFC 1035)

DNS translates human-readable names like www.example.com into IP addresses. It is a global, distributed, hierarchical, cached database. Without DNS, you'd have to remember 203.0.113.10 instead of example.com.

The DNS hierarchy

Domain names are read right-to-left in terms of hierarchy: www.example.com. (the trailing dot is the root, usually omitted). The zones are: root (.) → TLD (.com, .org, .fi) → second-level domain (example) → subdomain (www). Each zone is managed by authoritative name servers.

There are 13 logical root name server addresses (A through M), operated by 12 organisations including ICANN, Verisign, NASA, and others. Each address is served by many physical machines via anycast — there are well over 1,700 and growing root server instances globally.

Full resolution walkthrough

1. Browser checks its own DNS cache (TTL-based). Hit → done.
2. OS checks its DNS cache and /etc/hosts (Linux/macOS) or C:\Windows\System32\drivers\etc\hosts. Hit → done.
3. OS stub resolver sends a query to the configured recursive resolver (UDP port 53 by default; the TC truncated bit instructs the client to retry over TCP; EDNS(0) per RFC 6891 allows larger UDP messages; TCP is also used for zone transfers and, increasingly, for large answers such as DNSSEC). This is often 192.168.1.1 (your router) which forwards to the ISP's resolver, or a public resolver like 8.8.8.8 (Google), 1.1.1.1 (Cloudflare), or 9.9.9.9 (Quad9).
4. Recursive resolver checks its cache. If uncached, begins iterative resolution:
   1. Queries a root server: "Where is .com?" → root responds with NS records for .com TLD servers.
   2. Queries .com TLD: "Where is example.com?" → TLD responds with NS records for example.com's authoritative servers.
   3. Queries example.com's authoritative server: "What is www.example.com?" → authoritative server returns the A record.
5. Recursive resolver caches the result (honoring TTL), returns it to the stub resolver.
6. OS returns the IP to the application. The application caches it in its own resolver.

Beyond mapping hostnames to IP addresses, DNS provides essential infrastructure services:

• Host aliasing: A host with a complex canonical name (e.g. server14.prod.us-east.example.com) can have mnemonic alias names via CNAME records (e.g. www.example.com).
• Mail server aliasing: MX records allow an organization's email domain (e.g. @example.com) to point to dedicated mail servers (e.g. mail.hostingprovider.com). If a domain has no MX record, mail transfer agents fall back to looking up the domain's A or AAAA record. Setting a null MX (example.com. IN MX 0 .) explicitly announces to the world that this domain does not accept email.
• Load distribution: Busy sites associate multiple IP addresses with a single hostname in round-robin DNS. Resolvers cycle through the list, spreading client traffic across servers.

DNS Caching and TTL: Caching is critical to internet performance. Every DNS record carries a Time-to-Live (TTL). Once a resolver caches an answer, it satisfies subsequent requests instantly without touching authoritative servers. Resolvers also employ negative caching (RFC 2308): caching the non-existence of a name (NXDOMAIN) for the duration of the zone's SOA minimum TTL to prevent repeated flooding of invalid lookups. Note that unauthenticated DNS responses can be poisoned by attackers (see Chapter 7: Network Security for DNS cache poisoning).

# Trace DNS resolution with +trace (illustrative output)
$ dig +trace www.example.com

.               518400  IN  NS  a.root-servers.net.     ← root servers
com.             172800  IN  NS  a.gtld-servers.net.     ← .com TLD
example.com.    172800  IN  NS  a.iana-servers.net.     ← authoritative
www.example.com.  3600  IN  A   203.0.113.10            ← answer

DNS Glue Records: Notice that if the authoritative nameserver for example.com is ns1.example.com (inside the domain being looked up), a circular lookup dependency would occur: resolving example.com requires knowing the IP of ns1.example.com, but finding that IP requires resolving example.com! To break this deadlock, the parent .com TLD server includes a glue record (the A/AAAA IP address of ns1.example.com) directly in the Additional section of its referral response.

Common DNS record types — extended

5. DHCP — Dynamic Host Configuration Protocol (RFC 2131)

DHCP is an application-layer configuration protocol carried over UDP (server port 67, client port 68) that automatically assigns network parameters to joining hosts:

• Assigned IPv4 address and subnet mask
• Default gateway (first-hop router)
• DNS server IP addresses
• Lease time (validity duration), renewal timer T1 (50% of lease, unicast to leasing server), and rebinding timer T2 (87.5% of lease, broadcast to any server)
• Domain name, NTP servers, and vendor options

Additional DHCP message types include DHCPNAK (server rejects request), DHCPDECLINE (client detects address conflict via ARP probe), and DHCPRELEASE (client gracefully relinquishes address). Before binding an offered address, clients typically send an ARP probe to verify no other host is using it. If no DHCP server responds, clients fall back to link-local autoconfiguration (169.254.0.0/16 per RFC 3927).

DORA in detail

Client              Server
  │                    │
  │── DHCPDISCOVER ───►│  UDP src:0.0.0.0:68, dst:255.255.255.255:67
  │   (broadcast)      │  "I need an IP configuration"
  │                    │
  │◄── DHCPOFFER ──────│  UDP src:server_IP:67, dst:client/broadcast:68
  │                    │  "Here is 192.168.1.50, mask /24, GW 192.168.1.1, lease 24h"
  │                    │
  │── DHCPREQUEST ────►│  UDP src:0.0.0.0:68, dst:255.255.255.255:67
  │   (broadcast)      │  "I accept the offer from server 192.168.1.1"
  │                    │
  │◄── DHCPACK ────────│  UDP src:server_IP:67, dst:client:68
  │                    │  "Confirmed. Lease valid for 24h."
  │                    │
  │── (at T1 = 12h) ──►│  DHCPREQUEST unicast to leasing server
  │◄── DHCPACK ────────│  Lease renewed for another 24h
  │                    │
  │── (at T2 = 21h) ──►│  DHCPREQUEST broadcast to ANY server (rebinding)

DHCP relay agents (RFC 1542): Because DHCP discovery relies on local broadcasts that routers do not forward, a DHCP relay agent configured on the router intercepts local broadcasts and forwards them as unicast UDP datagrams to a centralised DHCP server across subnets, inserting the local network interface IP into the giaddr (Gateway IP Address) field.

Security note: Standard DHCP has no authentication. A rogue DHCP server on the local LAN can hand out false default gateways and DNS servers, intercepting all client traffic. Switches prevent this using DHCP snooping (see Chapter 7: Network Security).

6. HTTPS and TLS Basics

HTTPS (HTTP Secure, port 443) is simply standard HTTP operating over an encrypted TLS (Transport Layer Security) channel. TLS provides three indispensable security guarantees:

• Confidentiality: Symmetric encryption (AES-GCM, ChaCha20-Poly1305) prevents eavesdroppers from inspecting plaintext payloads.
• Integrity: Cryptographic authentication (AEAD in TLS 1.3) ensures messages cannot be modified or replayed without detection.
• Authentication: X.509 certificates and the Public Key Infrastructure (PKI) verify that the client is communicating with the genuine server.

Symmetric vs. asymmetric cryptography: Encrypting bulk application data with asymmetric algorithms (RSA or ECC) is computationally too slow. TLS solves this by combining both primitives:

• Asymmetric cryptography (public-key): Used exclusively during the handshake to authenticate the server and securely establish a shared secret between client and server without transmitting the key itself.
• Symmetric cryptography (shared secret): Once the shared secret is established, both parties derive symmetric keys to encrypt application data rapidly.

Certificates and the CA chain of trust: When connecting to https://example.com, the server presents an X.509 certificate signed by a Certificate Authority (CA). The browser validates:

1. Hostname match: The certificate Subject Alternative Name (SAN) matches example.com.
2. Validity window: Current date falls within the certificate's NotBefore and NotAfter timestamps.
3. Signature chain: The certificate signature chains up through intermediate CAs to a pre-installed root CA in the operating system or browser trust store.
4. Revocation status: The certificate has not been revoked (checked via OCSP or CRLs).

What HTTPS does NOT hide: While HTTPS encrypts request paths, headers, query parameters, cookies, and bodies, it does not conceal:

• Destination IP address and destination port (visible to all intermediate routers).
• Server hostname in the TLS SNI (Server Name Indication) extension (visible in plaintext unless Encrypted Client Hello, ECH, is enabled).
• Traffic timing, packet sizes, and transferred data volume (amenable to traffic analysis).

HSTS (HTTP Strict Transport Security, RFC 6797): Web servers declare Strict-Transport-Security: max-age=31536000; includeSubDomains to force browsers to automatically convert all http:// links to https://, eliminating SSL-stripping attacks.

7. FTP — File Transfer Protocol (RFC 959) Nice-to-know

FTP uses two separate parallel TCP connections: an "out-of-band" control connection (TCP port 21) for commands and authentication, and a separate data connection (TCP port 20 in active mode, or a negotiated ephemeral port in passive mode) for each file transfer. Because classic FTP transmits usernames and passwords in plaintext and complicates firewall traversal, it is largely obsolete, replaced by SFTP (SSH File Transfer Protocol) and FTPS (FTP over TLS).

8. Practical Sockets & Video Streaming

Socket Programming: Python TCP and UDP Examples

Network applications interact with the transport layer via system socket APIs. Here are minimal, fully functioning Python examples:

# ── TCP Echo Server (server.py) ──
import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('127.0.0.1', 65432))
s.listen(1)
print("TCP server listening on port 65432...")
conn, addr = s.accept() # accept() creates a new connection socket per client!
print(f"Connected by {addr}")
while True:
    data = conn.recv(1024)
    if not data:
        break
    conn.sendall(data)
conn.close()
s.close()

# ── TCP Client (client.py) ──
import socket

c = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
c.connect(('127.0.0.1', 65432))
c.sendall(b"Hello Tampere University TCP")
response = c.recv(1024) # Note: TCP is a byte stream; recv() may return partial data!
print("Received:", response.decode())
c.close()

# ── UDP Echo Server & Client ──
import socket

# Server:
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(('127.0.0.1', 65433))
data, addr = s.recvfrom(1024) # Preserves datagram boundary
s.sendto(data, addr)
s.close()

# Client:
c = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
c.sendto(b"Hello Tampere University UDP", ('127.0.0.1', 65433))
reply, server_addr = c.recvfrom(1024)
print("Received:", reply.decode())
c.close()

Socket programming principles: In TCP, accept() spawns a dedicated connection socket for each connecting client (demultiplexing by the 4-tuple, as covered in Chapter 4). Because TCP provides an unstructured byte stream with no record boundaries, applications must delimit messages (e.g. using delimiters like CRLF or prefixing length headers). In contrast, UDP operates on atomic datagrams where recvfrom() preserves message boundaries.

Video Streaming and Content Distribution

Modern internet video accounts for over half of global consumer traffic. Video files are encoded at multiple bitrates and divided into discrete chunks (typically 2 to 10 seconds long) served over standard HTTP using DASH (Dynamic Adaptive Streaming over HTTP) or HLS (HTTP Live Streaming). A manifest file lists chunk URLs and bitrates; the client measures observed network throughput and dynamically requests higher or lower quality chunks as network conditions fluctuate. Video chunks are heavily distributed and cached across Content Delivery Networks (CDNs) placed at network edges close to users to minimize latency and transit bottlenecks.

9. Beyond the Course

Advanced protocols and security extensions encountered in modern production infrastructures:

HTTP/2 (RFC 9113) — binary multiplexing Beyond

HTTP/2 uses a binary framing layer over one TCP connection. All HTTP/2 communication is divided into frames and streams:

• Streams: independent bidirectional exchanges (one request + response = one stream). Streams are multiplexed — many streams share one TCP connection concurrently.
• Frames: smallest unit of communication. Types: HEADERS (request/response metadata), DATA (body), PRIORITY, RST_STREAM, SETTINGS, PUSH_PROMISE, PING, GOAWAY, WINDOW_UPDATE, CONTINUATION.
• HPACK header compression: static table of 61 common headers + dynamic table of recently seen headers. Dramatically reduces per-request overhead (headers were ~800 bytes in HTTP/1.1; can be <20 bytes with HPACK).
• Server push: server can send resources proactively before the client requests them (e.g. push CSS and JS when the HTML is requested). Deprecated in practice due to complexity and caching conflicts; Chrome removed support in 2022.

HTTP/2 still suffers from TCP head-of-line blocking: if a TCP segment is lost, all HTTP/2 streams wait until that segment is retransmitted, even though only one stream needed that data.

HTTP/3 (RFC 9114) — QUIC-based Beyond

HTTP/3 runs over QUIC instead of TCP. Each HTTP/3 stream is a QUIC stream; a lost packet only blocks the stream that needed it. Connection setup is 1-RTT (0-RTT for resumed connections). Deployment is widespread: as of 2024 roughly 30% of websites support HTTP/3 according to publicly tracked surveys, though the share changes rapidly.

DNS over HTTPS (DoH) and DNS over TLS (DoT) Beyond

Classic DNS is unencrypted — anyone on-path can see your DNS queries. DoT (RFC 7858) wraps DNS in TLS on TCP port 853. DoH (RFC 8484) sends DNS queries as HTTPS requests (port 443), making them indistinguishable from web traffic. Both encrypt queries and responses, preventing eavesdropping and tampering. Cloudflare (1.1.1.1), Google (8.8.8.8), and Quad9 (9.9.9.9) all support both.

TLS 1.3 handshake (1 RTT) Beyond

Client → Server:  ClientHello
  - TLS version: 1.3
  - Supported cipher suites: TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, …
  - Client random (32 bytes)
  - key_share: client's ECDH public key (e.g. X25519)

Server → Client:  ServerHello + {EncryptedExtensions + Certificate + CertificateVerify + Finished}
  - Selected cipher suite
  - Server random
  - key_share: server's ECDH public key
  - Server derives handshake keys immediately → rest of server's messages are encrypted
  - Certificate: server's X.509 certificate chain
  - CertificateVerify: signature over handshake transcript using server's private key
  - Finished: HMAC of entire handshake (integrity check)

Client → Server:  {Finished}
  - Client derives the same session keys independently (ECDHE)
  - Verifies server certificate chain against trusted CAs
  - Verifies CertificateVerify signature
  - Sends Finished HMAC
  → Both sides now have matching symmetric session keys. Application data flows.

Key derivation (HKDF) Beyond: The session keys are derived from the ECDH shared secret using HKDF (HMAC-based Key Derivation Function). Because the ECDH key pair is ephemeral (generated fresh per connection), compromising the server's long-term private key later cannot decrypt past sessions — this is Perfect Forward Secrecy (PFS).

TLS 1.3 removed all weak cipher suites that TLS 1.2 supported (RC4, 3DES, SHA-1, static RSA key exchange). Only AEAD ciphers with (EC)DHE are supported.

SSH — Secure Shell (RFC 4253) Beyond

SSH provides encrypted, authenticated remote shell access, file transfer (SCP, SFTP), and tunnelling over TCP port 22. It replaced telnet (cleartext) and rsh/rlogin (cleartext + weak authentication).

The SSH protocol has three layers: the transport layer (key exchange, encryption), user authentication, and connection (channels for shell, file transfer, tunnels).

Host key verification: When you connect to an SSH server for the first time, SSH presents the server's public key fingerprint (stored in ~/.ssh/known_hosts) following a Trust On First Use (TOFU) model. If the host key changes later, SSH displays a severe warning and blocks connection to protect against Man-in-the-Middle (MITM) attacks.

SSH authentication methods

• Password: sent inside the encrypted SSH channel (protected from eavesdropping, but vulnerable to password guessing).
• Public key: the client proves possession of its private key by signing data derived from the session transcript; the server verifies the signature against public keys stored in ~/.ssh/authorized_keys. Supported algorithms include Ed25519 (recommended) and RSA (≥3072-bit).
• Certificate: CA-signed SSH certificates enable centralised access control without distributing individual authorized_keys files.

SSH tunnelling

# Local forward (-L): maps local port 8080 to database:3306
# Note: "database" is resolved from the SSH server's perspective, not client's
ssh -L 8080:database:3306 user@ssh.example.com

# Remote forward (-R): binds remote server port 9000 to local port 3000
# Note: binds to server's loopback interface by default (see GatewayPorts)
ssh -R 9000:localhost:3000 user@server.example.com

# Dynamic SOCKS5 proxy (-D): forwards all traffic dynamically
ssh -D 1080 user@server.example.com

Email Authentication: SPF, DKIM, and DMARC Beyond

MTA-STS and DANE for Strict Mail Transport Security Beyond

Because server-to-server SMTP over TCP port 25 historically negotiates opportunistic TLS, on-path attackers can strip the STARTTLS capability and force transmission into plaintext. Two standards enforce mandatory transport encryption:

• MTA-STS (MTA Strict Transport Security, RFC 8461): A receiving domain publishes an HTTPS policy file (e.g. https://mta-sts.example.com/.well-known/mta-sts.txt) and a DNS TXT record declaring that sending MTAs must deliver via TLS with a valid certificate or fail.
• DANE (DNS-based Authentication of Named Entities, RFC 7672): Uses DNSSEC-signed TLSA records to bind the receiving MTA's TLS certificate or public key directly to DNS, guaranteeing cryptographic verification without reliance on commercial CAs.