HTTP and HTTPS Protocol: A Complete Guide to Web Communication
Deep dive into HTTP methods, status codes, headers, keep-alive, and protocol evolution. Understand HTTP/1.1, HTTP/2, and HTTP/3 differences.
Introduction
HTTP (Hypertext Transfer Protocol) is a request-response protocol. The client sends a request, the server sends a response. That is the core model.
HTTP is stateless. Each request stands alone. The server does not remember previous requests from the same client unless you add session management on top.
sequenceDiagram
participant Client
participant Server
Client->>Server: GET /index.html HTTP/1.1
Client->>Server: Host: example.com
Client->>Server: Accept: text/html
Server->>Client: HTTP/1.1 200 OK
Server->>Client: Content-Type: text/html
Server->>Client: Content-Length: 1256
Server->>Client: <html>...</html>
The protocol has evolved through several versions. HTTP/1.1 remains widespread, but HTTP/2 and HTTP/3 offer real performance gains.
Core Concepts
HTTP Methods
HTTP defines several methods (also called verbs) that indicate the desired action:
GET
Retrieves data. GET requests should be safe and idempotent. Safe means they do not modify server state. Idempotent means multiple identical requests have the same effect as one.
GET /api/users HTTP/1.1
Host: example.com
POST
Submits data to be processed. POST requests typically cause state changes on the server.
POST /api/users HTTP/1.1
Host: example.com
Content-Type: application/json
{"name": "Alice", "email": "alice@example.com"}
PUT
Replaces a resource at a specific URL. If the resource exists, it is updated. If it does not exist, it is created.
PUT /api/users/123 HTTP/1.1
Host: example.com
Content-Type: application/json
{"name": "Alice", "email": "alice.new@example.com"}
PATCH
Partially updates a resource. Unlike PUT which replaces the whole resource, PATCH updates only the specified fields.
PATCH /api/users/123 HTTP/1.1
Host: example.com
Content-Type: application/json
{"email": "alice.updated@example.com"}
DELETE
Removes a resource.
DELETE /api/users/123 HTTP/1.1
Host: example.com
Other Methods
| Method | Purpose | Safe | Idempotent |
|---|---|---|---|
| HEAD | Like GET but returns only headers | Yes | Yes |
| OPTIONS | Returns supported methods for a URL | Yes | Yes |
| CONNECT | Converts to a tunnel (used for proxies) | No | No |
| TRACE | Echoes the request (for debugging) | Yes | Yes |
The definitions of “safe” and “idempotent” sound clean in theory but get messy in practice. Here is where it gets interesting.
Safe methods (GET, HEAD, OPTIONS, TRACE) are supposed to not modify server state. But plenty of GET endpoints do logging, track analytics, or update last-accessed timestamps. The server changes; the request just isn’t supposed to request those changes. That is the distinction: safe means the request is not asking for side effects, not that side effects cannot happen.
Idempotent methods (GET, HEAD, PUT, DELETE, OPTIONS, TRACE) mean multiple identical requests should produce the same server state as one. DELETE is idempotent — deleting an already-deleted resource still yields “gone.” PUT is idempotent — overwriting the same data twice does not change anything after the first write.
POST and PATCH are neither safe nor idempotent. Sending POST twice creates two resources. PATCH twice on the same field applies the patch twice (which matters if the patch is relative, like “add 10 to counter”).
The murkier cases:
# GET with side effects — technically violates safe semantics
GET /api/users/123/activate
# A/B testing frameworks often do this. The fix:
POST /api/users/123/activate
Some frameworks let GET accept bodies (technically valid per RFC 9110, but most servers reject it). If you have a GET that modifies state, switch to POST.
HEAD is like GET but never returns a body. Useful for checking if a resource exists or getting headers without downloading anything. Proxies use HEAD to validate cached responses.
OPTIONS returns what the server supports for a given URL. CORS preflight requests use OPTIONS to ask “can I do a DELETE to this URL from a different origin?” The response includes Allow: GET, HEAD, OPTIONS if DELETE is not allowed.
CONNECT is odd — it converts the connection into a TCP tunnel, typically for HTTPS through a proxy. You rarely see it outside of corporate proxy setups.
TRACE echoes the request back so clients can see what proxies modified. It is mostly a debugging tool. Most production servers disable it to prevent information leakage.
HTTP Status Codes
Status codes tell you what happened with the request. They are grouped by range:
1xx: Informational
The request was received, continuing process.
HTTP/1.1 100 Continue
100 Continue means the client should send the request body. This is useful when sending large requests to check if the server will accept them first.
2xx: Success
The request was successfully received, understood, and accepted.
| Code | Meaning |
|---|---|
| 200 | OK - Standard success response |
| 201 | Created - Resource was created |
| 204 | No Content - Success with no response body |
3xx: Redirection
Further action needed to complete the request.
HTTP/1.1 301 Moved Permanently
Location: https://example.com/new-page
| Code | Meaning |
|---|---|
| 301 | Moved Permanently - Resource now at new URL |
| 302 | Found - Temporary redirect |
| 304 | Not Modified - Cached version is still valid |
301 and 302 are the most common redirects. The difference matters for SEO: 301 tells search engines the move is permanent.
4xx: Client Errors
The request has bad syntax or cannot be fulfilled.
HTTP/1.1 404 Not Found
| Code | Meaning |
|---|---|
| 400 | Bad Request - Malformed syntax |
| 401 | Unauthorized - Authentication required |
| 403 | Forbidden - Authenticated but not authorized |
| 404 | Not Found - Resource does not exist |
| 429 | Too Many Requests - Rate limited |
5xx: Server Errors
The server failed to fulfill a valid request.
| Code | Meaning |
|---|---|
| 500 | Internal Server Error - Something broke on the server |
| 502 | Bad Gateway - Upstream server returned error |
| 503 | Service Unavailable - Server is temporarily overloaded |
| 504 | Gateway Timeout - Upstream server took too long |
HTTP Headers
Headers provide metadata about the request or response. They are essential for caching, authentication, content negotiation, and more.
Common Request Headers
GET /api/data HTTP/1.1
Host: example.com
Accept: application/json
Authorization: Bearer eyJhbGciOiJIUzI1NiIs...
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
| Header | Purpose |
|---|---|
| Host | Domain name of the server |
| Accept | Media types the client can handle |
| Authorization | Credentials for authentication |
| Cache-Control | Caching directives |
| User-Agent | Client application information |
| Cookie | Session data previously set by server |
Common Response Headers
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 512
Cache-Control: max-age=3600
Set-Cookie: session=abc123; HttpOnly; Secure
X-Request-ID: req-12345
| Header | Purpose |
|---|---|
| Content-Type | Media type of the response |
| Content-Length | Size of the response body |
| Cache-Control | Caching directives for clients |
| Set-Cookie | Data to store on the client |
| ETag | Version identifier for caching |
| X-Request-ID | Unique request identifier |
Keep-Alive and Connection Management
HTTP/1.0 closed the connection after each request by default. HTTP/1.1 introduced persistent connections.
With Keep-Alive, multiple requests can reuse the same TCP connection:
# Without Keep-Alive: 3 TCP connections for 3 requests
GET /page1.html -> TCP connection 1 -> close
GET /page2.html -> TCP connection 2 -> close
GET /page3.html -> TCP connection 3 -> close
# With Keep-Alive: 1 TCP connection for 3 requests
GET /page1.html -> TCP connection 1
GET /page2.html -> TCP connection 1 (reused)
GET /page3.html -> TCP connection 1 (reused)
Connection: close -> close
Keep-Alive reduces latency by avoiding TCP handshake overhead. It also reduces server load by needing fewer connections.
# Request with Keep-Alive
GET /api/data HTTP/1.1
Host: example.com
Connection: keep-alive
# After response, connection stays open for more requests
Topic-Specific Deep Dives
HTTP/2: Multiplexing
HTTP/1.1 improved with persistent connections, but requests still had to complete in order (head-of-line blocking). HTTP/2 introduced multiplexing.
With multiplexing, multiple requests and responses flow simultaneously over a single connection:
graph LR
A[Stream 1: GET /index.html] --> B[Single TCP Connection]
C[Stream 2: GET /style.css] --> B
D[Stream 3: GET /app.js] --> B
B --> A
B --> C
B --> D
This eliminates head-of-line blocking. The browser can request all assets in parallel without waiting.
HTTP/2 also adds header compression (HPACK), server push, and stream prioritization.
HTTP/3: QUIC Transport
HTTP/3 takes a different approach. It runs over QUIC instead of TCP. QUIC is a transport protocol built on UDP.
Why UDP? TCP requires a handshake before sending data. QUIC combines the handshake with TLS, reducing connection establishment time:
graph LR
A[TCP + TLS 1.2] --> B[3 round trips to ready]
C[QUIC + TLS 1.3] --> D[1 round trip to ready]
HTTP/3 benefits include faster establishment (0-RTT resumption), no head-of-line blocking, better performance on lossy networks, and connection migration when switching networks.
The catch: HTTP/3 requires UDP traffic to be allowed through firewalls. Most modern sites support it alongside HTTP/2.
QUIC vs TCP: Under the Hood
This deserves a closer look. TCP and QUIC solve the same problem — reliable, ordered data delivery — but take very different paths to get there.
TCP’s problem: connection establishment is slow
To send encrypted data over TCP, you need:
- TCP handshake: SYN -> SYN-ACK -> ACK (1 RTT)
- TLS handshake: ClientHello -> ServerHello + certificates -> Finished (1-2 RTTs)
That is 2-3 round trips before the first byte of application data arrives. On a 50ms connection, that is 100-150ms of waiting.
QUIC combines transport and security in one handshake
QUIC is built on UDP. It re-implements everything TCP gives you — reliability, ordering, congestion control — but inside UDP. This lets it fold the TLS handshake directly into the first connection handshake:
- QUIC + TLS 1.3: ClientHello + 0-RTT data -> ServerHello + 0-RTT response (1 RTT for new connections, 0 RTT for resumption)
For resumption, QUIC can send application data immediately on the first packet — no round trip needed.
Head-of-line blocking comparison
HTTP/2 solves application-layer head-of-line blocking with multiplexing, but TCP still enforces ordering at the transport layer. If packet 5 is lost, TCP blocks all Stream 2 data until packet 5 retransmits, even if Stream 3’s data arrived fine.
QUIC tracks streams independently. A lost packet only blocks the stream that packet belongs to — other streams keep flowing. This matters a lot on lossy networks (mobile, coffee shop WiFi).
Connection migration
TCP connections are identified by the 4-tuple (client IP, client port, server IP, server port). Switch from WiFi to cellular and the connection dies.
QUIC uses a Connection ID that survives address changes. The network stack detects the new path and migrates the connection automatically. This is why Google YouTube sees fewer rebuffers when users switch networks.
The UDP firewall problem
Most networks allow TCP and UDP ports 80 and 443. QUIC runs on UDP:443, which some firewalls block or throttle. HTTP/3 falls back to HTTP/2 when QUIC is blocked, but you lose the performance gains on those connections.
graph LR
A[New connection, cold start] --> B[TCP+TLS: ~3 RTTs]
A --> C[QUIC+TLS 1.3: ~1 RTT]
D[Resumption, warm] --> E[TCP+TLS: ~1 RTT]
D --> F[QUIC+TLS 1.3: ~0 RTT]
G[Packet loss] --> H[TCP: all streams blocked on lost packet]
G --> I[QUIC: only that stream blocked, others continue]
QUIC’s implementation in Linux kernel networking path is still maturing compared to TCP. For high-throughput, high-concurrency servers, QUIC adds CPU overhead that TCP does not. This is why Cloudflare and Google use userspace QUIC implementations (like quiche and gQUIC) rather than kernel-space.
HTTP/3 Performance Benchmarks
The numbers vary significantly based on network conditions. Here is what real-world data shows.
Cold connection latency (new server, no cache)
| Protocol | Handshake RTTs | Latency (50ms RTT) |
|---|---|---|
| HTTP/1.1 | 1 (TCP only) | ~50ms |
| HTTP/2 | 2 (TCP + TLS) | ~100ms |
| HTTP/3 | 1 (QUIC + TLS) | ~50ms |
With TLS 1.3 0-RTT, HTTP/3 can be faster than HTTP/1.1 for repeat connections.
Warm connection (resumption)
| Protocol | Resumption RTTs | Latency (50ms RTT) |
|---|---|---|
| HTTP/1.1 | 1 | ~50ms |
| HTTP/2 | 1 (TLS ticket) | ~50ms |
| HTTP/3 | 0 (0-RTT) | ~0ms (data on 1st pkt) |
On a 50ms network, HTTP/3 sends application data immediately on resumption while HTTP/1.1 and HTTP/2 still wait for the TLS ticket exchange.
Packet loss impact
On a 2% packet loss network (lossy WiFi):
- HTTP/1.1: ~5-10% throughput reduction (head-of-line blocking at application)
- HTTP/2: ~15-20% throughput reduction (TCP head-of-line blocking + head-of-line blocking)
- HTTP/3: ~2-5% throughput reduction (QUIC handles loss per-stream)
HTTP/3’s gain on lossy networks is the most compelling argument for adoption. On clean fiber connections, the differences are negligible.
Practical adoption numbers
As of early 2026, approximately:
- 27% of websites support HTTP/3 (Cloudflare radar, Q1 2026)
- ~57% of browser traffic uses HTTP/3 when available
- Major CDNs (Cloudflare, Fastly, AWS CloudFront) all support HTTP/3
The “HTTP/3 adoption is low” argument is losing weight. If you are serving mobile traffic on lossy networks, HTTP/3’s improvements are real and measurable.
HTTPS: HTTP Over TLS
HTTPS adds encryption via TLS (Transport Layer Security). The SSL/TLS and HTTPS post covers the details, but here is the quick version:
sequenceDiagram
participant Client
participant Server
Client->>Server: ClientHello (TLS)
Server->>Client: ServerHello, Certificate, KeyExchange
Client->>Server: KeyExchange, ChangeCipherSpec
Client->>Server: Finished
Server->>Client: ChangeCipherSpec
Server->>Client: Finished
Note over Client,Server: Encrypted HTTP now
The padlock icon in your browser means the connection uses HTTPS and the server certificate has been verified.
Certificate Pinning: Preventing MITM on High-Risk Clients
Most HTTPS implementations trust the CA system. Certificate pinning goes further — you lock your app or client to a specific certificate or public key, preventing trust of any other certificate even if a CA is compromised.
Why bother? If an attacker gets a valid CA and issues a forged certificate for your domain, a normal HTTPS client will trust it. With pinning, only your known certificate is accepted.
Implement with Expect-CT header:
Strict-Transport-Security: max-age=31536000; includeSubDomains
Expect-CT: max-age=86400, report-uri="https://example.com/ct-report"
The Expect-CT header tells browsers to enforce certificate transparency and reject connections to certificates not logged in CT logs.
For mobile apps, pin against the public key hash rather than the full certificate. Certificates rotate; public key hashes survive certificate changes:
# Mobile app: pin the SubjectPublicKeyInfo (SPKI) hash
# Google's TrustManager for Android pinning example:
CertificatePinner.Builder builder = new CertificatePinner.Builder();
builder.add("example.com", "sha256/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=");
Backup pins are essential. If your primary certificate rotates and you do not have a backup pin, your app breaks for all users. Google maintains backup pins for their domains and rotates them before primary expiry.
Dynamic pinning — fetch pins from your server at runtime rather than hardcoding them. Change them when your certificate rotates. This way you do not need app updates for every certificate rotation.
CA pinning — pin against the CA certificate itself for simpler setups. This rejects any certificate issued by a different CA, but does not protect against the pinned CA being compromised.
Revocation problem: If a pinned certificate is compromised, users with the old pin cannot connect. You need a mechanism to push pin updates before the certificate rotates. HPKP (HTTP Public Key Pinning) is deprecated in browsers because of this risk. The modern approach is Expect-CT + CT logs + pinning at the mobile app layer rather than browser layer.
For browsers: rely on Expect-CT and CT log monitoring rather than HPKP. For mobile apps: implement pinning with backup pins and a remote update mechanism.
Caching with HTTP
HTTP caching reduces server load and improves performance. The Cache-Control header controls caching behavior:
# Cache for 1 hour
Cache-Control: max-age=3600
# Do not cache at all
Cache-Control: no-store
# Check freshness but reuse cached copy
Cache-Control: no-cache
# Cache only on CDN, not in browser
Cache-Control: private, max-age=3600
ETags provide another caching mechanism:
# Server response includes ETag
HTTP/1.1 200 OK
ETag: "v1.2.3"
# Client caches and later checks if changed
GET /api/data HTTP/1.1
If-None-Match: "v1.2.3"
# Server responds with 304 if unchanged
HTTP/1.1 304 Not Modified
When to Use HTTP
HTTP is the right choice when:
- You need stateless request-response communication
- Clients and servers have no shared state
- Standard browser clients need to access your API
- You want simple, well-understood semantics
- Caching through standard HTTP mechanisms helps performance
- You are building web applications, REST APIs, or serving content
When to Use HTTPS
HTTPS is required when:
- You transmit any sensitive data (credentials, personal info, payment data)
- Your application requires authentication
- You need to prevent man-in-the-middle attacks
- Browser security warnings would harm your users
- You handle session cookies or tokens
- Your application deals with financial or healthcare data
When Not to Use HTTP/HTTPS
HTTP/HTTPS may not be the right choice when:
- Real-time bidirectional communication is needed (use WebSockets)
- You need server-to-server streaming with minimal overhead (use gRPC)
- You are building low-latency games or trading systems where TCP/UDP directly makes more sense
- You need to push data to clients without polling (consider Server-Sent Events)
HTTP Version Trade-off Analysis
| Factor | HTTP/1.1 | HTTP/2 | HTTP/3 |
|---|---|---|---|
| Transport | TCP | TCP | QUIC (UDP) |
| Handshake | 1 RTT | 2 RTTs | 1 RTT (new) / 0 RTT (resumed) |
| Multiplexing | No (pipelining broken) | Yes (stream-based) | Yes (independent streams) |
| Head-of-line blocking | Yes (application) | Yes (TCP layer) | No (stream-level) |
| Header compression | None | HPACK | QPACK |
| Server push | No | Yes | Yes |
| Connection migration | No | No | Yes (via Connection ID) |
| Firewall issues | None | None | Some UDP restrictions |
| Browser support | Universal | Universal | ~57% of traffic |
| Complexity | Low | Medium | High |
| CPU overhead | Low | Medium | Higher |
| Best for | Simple APIs, legacy | Mixed-content sites | Mobile, lossy networks, repeat visitors |
Production Failure Scenarios
| Failure | Impact | Mitigation |
|---|---|---|
| Server returns 500 errors | Users see errors, no data retrieved | Implement retry logic with exponential backoff; monitor error rates |
| Slow response times | Poor user experience, timeouts | Set appropriate timeouts; use circuit breakers; scale horizontally |
| SSL/TLS certificate expired | Browser warnings, service unavailable | Automate certificate renewal (Let’s Encrypt, certbot); monitor expiration |
| Malformed responses | Client crashes or data corruption | Validate responses with schema; implement graceful degradation |
| Header injection | Security vulnerability | Sanitize all header values; use security headers (HSTS, CSP) |
| HTTP/2 connection drops | Reconnection overhead | Implement connection pooling; use HTTP/3 as fallback |
| Reverse proxy timeout | Gateway 504 errors | Configure appropriate timeouts; implement health checks |
| Content-Type mismatch | Client cannot parse response | Always set correct Content-Type; use content negotiation |
Common Pitfalls / Anti-Patterns
Improper Use of Status Codes
Do not return 200 OK for errors. Clients rely on status codes to determine success or failure.
# Wrong - 200 for error
HTTP/1.1 200 OK
{"error": "User not found"}
# Correct - proper status code
HTTP/1.1 404 Not Found
{"error": "User not found"}
Ignoring Idempotency
GET and HEAD should be safe (no side effects). PUT should be idempotent. POST is neither.
# GET with side effects - BAD
GET /api/users/123/activate # Changes server state!
# Proper REST - use POST or PATCH
POST /api/users/123/activate
PATCH /api/users/123 {"status": "active"}
Not Handling Timeouts
Clients may timeout before server responds. Always implement proper timeout handling and retry logic.
Missing Content-Type
Always specify Content-Type for requests with bodies. Clients may not guess correctly.
# Always specify content type
Content-Type: application/json
Keeping Connections Open Indefinitely
Without Keep-Alive, each request requires a new TCP handshake. With Keep-Alive but no timeout, servers can run out of connections.
Interview Questions
- HTTP/1.1: persistent connections, no multiplexing (head-of-line blocking)
- HTTP/2: multiplexing, HPACK compression, server push, but still TCP head-of-line blocking at transport layer
- HTTP/3: QUIC transport over UDP, 0-RTT resumption, no head-of-line blocking at any layer, connection migration support
- New problems: QUIC needs UDP, CPU overhead higher, firewall issues, more complex server implementation
- Safe methods: GET, HEAD, OPTIONS, TRACE — should not modify server state
- Idempotent methods: GET, HEAD, PUT, DELETE, OPTIONS, TRACE — multiple identical requests have same effect as one
- POST and PATCH are neither safe nor idempotent
- Edge case: GET requests can have side effects (logging, timestamps) which violates safe semantics
- TLS 1.3 reduces handshake to 1 RTT for new connections, 0 RTT for resumption
- Client sends ClientHello with supported cipher suites and key share
- Server responds with ServerHello, certificate, key share, finished
- Client sends finished, data can start flowing immediately
- 0-RTT uses pre-shared keys from previous session to send data immediately
- 100 Continue indicates the server is willing to receive the request body
- Client sends Expect: 100-continue header before sending large body
- Server responds with 100 if it will accept, or 417 if not
- Solves: avoid sending large body to a server that might reject it (auth, Content-Type validation)
- `max-age=3600`: cache for 1 hour, serve from cache without revalidation
- `no-cache`: always revalidate with server (send If-None-Match or If-Modified-Since), use cached copy if 304
- `no-store`: never cache, must fetch fresh every time
- ETag vs Last-Modified: ETags are opaque validators, more precise; Last-Modified is timestamp-based
- `private` vs `public`: private only caches in browser, public can be cached by CDNs/proxies
- Pinning locks your client to only trust a specific certificate or public key
- Prevents MITM even if a CA is compromised and issues a forged certificate
- Implementation: HSTS + Expect-CT for browsers, TrustManager pinning for mobile
- Risks: pinning a certificate that rotates breaks users; HPKP is deprecated because of this
- Mitigation: always have backup pins, use remote pin update mechanisms
- Like GET but returns only headers, no body
- Checking if a resource exists before fetching (for large files)
- Checking cache freshness: `HEAD /file -> 304` means cached version is still valid
- Proxies use HEAD to validate if cached response is still fresh without downloading entire body
- Used in webhook signature verification (HEAD to check payload without processing)
- HTTP/1.1: browsers open 6 connections per origin, but each connection still queues requests (pipelining was never fully implemented)
- HTTP/2: multiplexing allows parallel requests, but TCP enforces ordering — if packet N is lost, all streams wait for retransmission even if their data arrived
- HTTP/3: QUIC tracks streams independently, loss only blocks the affected stream, others continue
- Trade-off: HTTP/3's reliability is at stream level, so a single lost packet in stream 1 does not block stream 2
- `Connection: keep-alive` (default in HTTP/1.1) keeps TCP connection open for multiple requests
- `Connection: close` terminates after response
- Reduces TCP handshake overhead (3-way handshake per request without keep-alive)
- Server must manage connection pool per client — too many idle keep-alive connections can exhaust file descriptors and memory
- Servers set `keep-alive timeout` to close idle connections after N seconds
- QUIC is a reliable, ordered, congestion-controlled transport built on UDP
- TCP requires separate handshakes for TCP and TLS; QUIC combines them into fewer round trips
- QUIC implements its own loss detection and recovery independently of the path (handles reordering natively)
- Allows connection migration (switch IP addresses without dropping connection)
- Runs on UDP:443 so it needs firewall cooperation; drops back to HTTP/2 when blocked
- Trade-off: userspace implementation means more CPU overhead than kernel-space TCP
- `Upgrade: websocket` header tells server the client wants to switch protocols
- Server responds with `101 Switching Protocols` if it accepts
- Used for WebSocket connections, but also for upgrading to TLS (TLS through proxy)
- After 101, the connection switches to the new protocol — same TCP connection, different application protocol
- Most common: WebSocket upgrade from HTTP to ws:// or wss://
- Content-Encoding: gzip, br (Brotli), deflate, zstd — tells client how to decode the response body
- Compression reduces bytes over wire, significant for text-based content (HTML, JSON, CSS, JS)
- Accept-Encoding: client advertises what it supports; server picks one
- Order matters: typically `Accept-Encoding: gzip, br, deflate` — server picks preferred
- For binary formats (images, video), compression is often already applied; additional HTTP compression adds overhead
- Vary tells caches that the response varies based on those request headers
- `Vary: Accept-Encoding`: same URL, different encoding (gzip vs brotli) = different cached response
- `Vary: Accept-Language`: same URL, different language (en vs fr) = different cached response
- Without Vary, a cache might serve gzip-compressed response to a client that expects plain text
- Can combine: `Vary: Accept-Encoding, Accept-Language` — cache key becomes tuple of all three
- Server push lets server send resources to client before client asks for them (push promises)
- Client receives pushed responses alongside requested page — no extra RTT for CSS/JS/fonts
- Implementation: `` in HTML or `PUSH_PROMISE` frames in HTTP/2
- Problem: server pushes resources client already has (already in cache) — wastes bandwidth
- Modern alternative: `` in HTML gives client control — browser only fetches what's needed
- Most sites disable HTTP/2 server push in favor of preload hints
- HTTP/2 allows multiplexing streams with weight (1-256) and dependency flags
- Stream dependency: stream A depends on stream B — B must complete before A gets bandwidth
- Prioritization ensures critical resources (HTML, CSS) get bandwidth before non-critical (images, analytics)
- Without prioritization: large image download could starve CSS, causing render delay
- Problem: browsers implement different prioritization strategies — server-side control is limited
- HTTP/3 dropped stream dependencies in favor of simpler per-stream flow control
- Client sends `Expect: 100-continue` before sending body, waits for 100 before proceeding
- Purpose: check if server will accept the body before sending large data (avoid sending GB to a 401 response)
- Server responds 100 to proceed, or 417 (Expectation Failed) to reject
- If client sends body without waiting for 100: server may read partially, get confused, or timeout
- Some proxies strip 100-continue handling — client can timeout waiting for 100 that never comes
- Used in resumable uploads and large file uploads where auth must be validated first
- CONNECT method creates a tunnel — opens a TCP pipe to the destination through a proxy
- `CONNECT example.com:443 HTTP/1.1` → proxy responds `200 Connection Established` → full encrypted tunnel
- Used for HTTPS through corporate proxies — client establishes tunnel, then does TLS inside it
- Security issue: proxies cannot inspect encrypted traffic (MITM is impossible on CONNECT tunnels)
- Enterprise proxies use this to inspect HTTPS traffic — they terminate TLS at the proxy first
- Malicious use: exfiltrate data through tunnel to bypass DLP controls
- Client requests specific byte range: `Range: bytes=0-1023` — gets only that portion
- Server responds `206 Partial Content` with `Content-Range: bytes 0-1023/4096`
- `Content-Range: bytes 0-1023/4096` — served bytes 0-1023 of 4096 total
- Used by video players (seek to specific position without downloading entire file)
- Without Range header: `200 OK` with full file — wasteful for large media
- Server can also respond `416 Range Not Satisfiable` if range is invalid (beyond file size)
- Content-Length: server tells client exact byte count before sending body
- Chunked: server sends data in chunks with size prefix, no Content-Length needed
- Chunked is required when: response is generated dynamically and total size unknown upfront
- Example: streaming JSON array — you start sending `[` before knowing if it will be 10 or 10 million elements
- HTTP/1.1 requires either Content-Length or Transfer-Encoding: chunked for request/response with body
- Server can fall back to chunked encoding if it does not know size in advance (common with CGI/scripts)
- SSE is one-way: server pushes events over a long-lived HTTP connection
- Uses `Content-Type: text/event-stream` and standard HTTP — works through proxies without special handling
- WebSockets use `Upgrade: websocket` and switch to a different protocol on the same TCP connection
- SSE: simpler, works with HTTP/2 multiplexing, automatic reconnection, no custom client library needed
- WebSockets: bidirectional, lower latency for high-frequency two-way communication
- For server-to-client push (notifications, live updates, streams): SSE is often simpler to implement and debug
Further Reading
Official Specifications and RFCs
- RFC 9110: HTTP Semantics — The definitive HTTP specification
- RFC 9111: HTTP Caching — Caching behavior and directives
- RFC 9113: HTTP/2 — HTTP/2 protocol definition
- RFC 9114: HTTP/3 — HTTP/3 over QUIC
- RFC 8446: TLS 1.3 — The current TLS standard
Browser and Network Resources
- MDN Web Docs: HTTP — Comprehensive HTTP reference
- Chromium Network Dump — HTTP/2 and HTTP/3 implementation details
- HTTP Archive — Historical HTTP usage statistics and trends
Protocol Analysis Tools
- Wireshark — Packet capture for HTTP/2 and HTTP/3 debugging
- curl — Command-line HTTP testing with HTTP/3 support (
--http3flag) - Chrome DevTools Network Panel — Visualize request timing and protocol negotiation
Deep Dives by Topic
HTTP/3 and QUIC Implementation
- IETF QUIC Working Group — Draft specifications and discussions
- Chromium QUIC Implementation — How Google implements QUIC in Chrome
- Cloudflare QUIC blog series — Production insights from one of the largest QUIC deployers
TLS and Certificate Management
- Let’s Encrypt — Free automated certificates
- Certificate Transparency (CT) Logs — Monitor certificate issuance
- SSL Labs SSL Test — Analyze TLS configuration quality
Web Security Headers
- Security Headers Analyzer — Check HSTS, CSP, and other security headers
- Mozilla Observatory — Security header scanner with recommendations
Related Posts on GeekWorkBench
- TCP/IP & UDP — Transport layer protocols underlying HTTP
- SSL/TLS and HTTPS — Encryption layer for secure HTTP
- DNS and DNSSEC — Name resolution that HTTPS depends on
- REST API Design — Best practices for HTTP-based API design
Conclusion
Key Bullets
- HTTP is a stateless request-response protocol at the application layer
- HTTP methods indicate action: GET (read), POST (create), PUT (replace), PATCH (modify), DELETE (remove)
- Status codes group by type: 1xx (info), 2xx (success), 3xx (redirect), 4xx (client error), 5xx (server error)
- Headers carry metadata for caching, authentication, content negotiation, and security
- HTTP/1.1 introduced persistent connections; HTTP/2 added multiplexing; HTTP/3 uses QUIC over UDP
- HTTPS wraps HTTP in TLS encryption, requiring certificates and adding handshake overhead
- Always use HTTPS for sensitive data and production services
Copy/Paste Checklist
# Check HTTP headers with curl
curl -I https://example.com
# Check SSL/TLS configuration
curl -v https://example.com 2>&1 | grep -E "(SSL|TLS|HTTP/)"
# Test specific HTTP method
curl -X POST https://api.example.com/users \
-H "Content-Type: application/json" \
-d '{"name":"Alice"}'
# Check for security headers
curl -I https://example.com | grep -iE "(strict-transport|x-frame|x-content-type)"
# Monitor response time
curl -w "\nTime: %{time_total}s\n" https://example.com
Metrics
- Request rate (requests per second by endpoint, method, status code)
- Response latency (p50, p95, p99 by endpoint)
- Error rate (4xx, 5xx by type)
- Connection pool utilization
- Active connections (concurrent requests)
- SSL/TLS handshake duration
- Certificate expiration tracking
Logs
- All requests with method, path, status, duration
- Error stack traces with request context (request ID)
- Slow requests (> 500ms threshold configurable)
- SSL handshake failures and certificate issues
- Connection establishment and termination events
- Health check results with detailed failures
Alerts
- Error rate exceeds 1% for 5 minutes
- p99 latency exceeds 2 seconds
- SSL certificate expires within 30 days
- Active connections approach server limit
- Health check failures for more than 30 seconds
- Unusual spike in 4xx errors (potential attack)
Security Checklist
- Enable HTTPS on all endpoints (no HTTP fallback for sensitive data)
- Set
Strict-Transport-Securityheader (HSTS) - Implement
Content-Security-Policyheader - Set
X-Content-Type-Options: nosniff - Set
X-Frame-Options: DENYto prevent clickjacking - Remove or obfuscate server version headers
- Implement rate limiting to prevent abuse
- Use secure, HttpOnly, SameSite cookies
- Validate all request headers and body data
- Implement CORS properly for cross-origin requests
- Log and monitor authentication failures
- Use POST, not GET, for sensitive data transmission
Category
Tags
Related Posts
SSL, TLS, and HTTPS: Securing Web Communication
Understand TLS handshake, certificates, cipher suites, and how HTTPS works. Learn the differences between SSL and TLS and why encryption matters.
TCP, IP, and UDP: Understanding Internet Transport Protocols
Compare TCP vs UDP, learn the three-way handshake, flow control, congestion control, when to use each protocol, and how QUIC changes things.
Cloud Security: IAM, Network Isolation, and Encryption
Implement defense-in-depth security for cloud infrastructure—identity and access management, network isolation, encryption, and security monitoring.