HTTP Request Structure Deep Dive

In the previous lesson, we covered the basics of HTTP. Now let us go deeper into the request side of the conversation. An HTTP request is a precisely structured message, and understanding each component will make you a more effective API developer.

---

1. The Four Components of Every Request

Every HTTP request consists of exactly four parts:

graph TD
    A[HTTP Request] --> B["1. Request Line<br/>METHOD /path HTTP/1.1"]
    A --> C["2. Headers<br/>Key: Value pairs"]
    A --> D["3. Empty Line<br/>Separates headers from body"]
    A --> E["4. Body<br/>JSON payload (optional)"]

    style A fill:#4f46e5,color:#fff
    style B fill:#0891b2,color:#fff
    style C fill:#059669,color:#fff
    style D fill:#6b7280,color:#fff
    style E fill:#d97706,color:#fff

Here is a raw HTTP request:

POST /api/v1/users HTTP/1.1        ← Request Line
Host: api.example.com              ← Headers
Content-Type: application/json     ← Headers
Authorization: Bearer abc123       ← Headers
Content-Length: 67                  ← Headers
                                   ← Empty line (CRLF)
{                                  ← Body starts
  "name": "Jane Doe",
  "email": "jane@example.com"
}

---

2. The Request Line

The request line is the first line and contains three pieces of information:

METHOD  /path  HTTP/version
POST    /users HTTP/1.1

The Method

Tells the server what to do:

Method	Meaning	Has Body?	Idempotent?	Safe?
GET	Retrieve	No	Yes	Yes
POST	Create	Yes	No	No
PUT	Replace	Yes	Yes	No
PATCH	Partial update	Yes	Usually	No
DELETE	Remove	Usually no	Yes	No
HEAD	Get headers only	No	Yes	Yes
OPTIONS	Get supported methods	No	Yes	Yes

The Path

The path identifies the resource. It can include:

/users                    ← Collection
/users/42                 ← Single resource
/users/42/posts           ← Nested resource
/users/42/posts?limit=5   ← With query parameters

Less Common Methods

HEAD: Identical to GET but returns only headers, no body. Useful for checking if a resource exists without downloading it:

curl -I https://jsonplaceholder.typicode.com/posts/1

OPTIONS: Returns what methods are supported for a URL. Used by browsers for CORS preflight requests:

curl -X OPTIONS https://jsonplaceholder.typicode.com/posts -i

---

3. Headers: The Metadata Layer

Headers carry metadata about the request. They are key-value pairs separated by a colon.

Essential Request Headers

Header	Purpose	Example
Host	Server hostname	Host: api.example.com
Content-Type	Format of request body	Content-Type: application/json
Accept	Expected response format	Accept: application/json
Authorization	Authentication credentials	Authorization: Bearer token123
User-Agent	Client identification	User-Agent: curl/8.4.0
Content-Length	Size of body in bytes	Content-Length: 128
Accept-Encoding	Supported compression	Accept-Encoding: gzip, deflate
Cache-Control	Caching behavior	Cache-Control: no-cache

Content-Type Values

The Content-Type header tells the server what format your body is in:

Content-Type	Format	Used For
application/json	JSON	REST API requests
application/x-www-form-urlencoded	Form data	HTML forms
multipart/form-data	File uploads	File uploads
text/plain	Plain text	Simple text
application/xml	XML	SOAP APIs

For REST APIs, you will almost always use application/json.

Content Negotiation

The Accept header tells the server what format you want back:

# I want JSON
curl -H "Accept: application/json" https://api.example.com/data

# I want XML
curl -H "Accept: application/xml" https://api.example.com/data

# I will accept anything
curl -H "Accept: */*" https://api.example.com/data

Most REST APIs only support JSON, but some support multiple formats.

---

4. The Request Body

The body contains the actual data you are sending. Not all requests have a body:

Method	Body?	Purpose
GET	No	Nothing to send; you are reading
POST	Yes	Data for the new resource
PUT	Yes	Complete replacement data
PATCH	Yes	Partial update data
DELETE	Usually no	Sometimes includes reason

JSON Body Examples

Creating a user (POST):

{
  "name": "Jane Developer",
  "email": "jane@dev.com",
  "role": "admin",
  "preferences": {
    "theme": "dark",
    "language": "en"
  }
}

Updating a field (PATCH):

{
  "email": "newemail@dev.com"
}

Full replacement (PUT):

{
  "id": 42,
  "name": "Jane Developer",
  "email": "newemail@dev.com",
  "role": "admin",
  "preferences": {
    "theme": "light",
    "language": "en"
  }
}

---

5. URL Structure in Detail

A URL has many parts, and understanding each one helps you construct correct API calls:

https://api.example.com:443/v2/users/42?include=posts&sort=date#section1
└─┬──┘ └───────┬───────┘└┬┘ └────┬───┘ └──────────┬──────────┘ └──┬──┘
scheme      host       port    path          query string       fragment

Component	Description	Example
Scheme	Protocol	https
Host	Server domain	api.example.com
Port	Network port (usually implicit)	:443 for HTTPS, :80 for HTTP
Path	Resource location	/v2/users/42
Query String	Parameters for filtering	?include=posts&sort=date
Fragment	Client-side reference	#section1 (not sent to server)

Path Segments

Each / separates a path segment. In REST APIs:

/v2/users/42/posts/7/comments
 │    │    │   │   │    │
 │    │    │   │   │    └── Resource type
 │    │    │   │   └─────── Resource ID
 │    │    │   └──────────── Resource type
 │    │    └───────────────── Resource ID
 │    └────────────────────── Resource type
 └─────────────────────────── API version

Query String Parameters

Query strings start with ? and use & between parameters:

# Single parameter
curl "https://api.example.com/users?role=admin"

# Multiple parameters
curl "https://api.example.com/users?role=admin&sort=name&order=asc&page=2&limit=20"

Common query parameter conventions:

Parameter	Purpose	Example
page	Pagination page number	?page=3
limit / per_page	Results per page	?limit=25
sort	Sort field	?sort=created_at
order	Sort direction	?order=desc
q / search	Search query	?q=javascript
fields	Specific fields to return	?fields=name,email
include	Related resources to embed	?include=posts,comments
filter	Advanced filtering	?filter[status]=active

---

6. Constructing Requests in cURL

Let us practice constructing different types of requests:

Simple GET (default)

curl https://jsonplaceholder.typicode.com/posts/1

GET with Custom Headers

curl -H "Accept: application/json" \
     -H "X-Custom-Header: my-value" \
     https://jsonplaceholder.typicode.com/posts/1

POST with JSON Body

curl -X POST \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"title": "Test", "body": "Content", "userId": 1}' \
     https://jsonplaceholder.typicode.com/posts

PUT with Complete Data

curl -X PUT \
     -H "Content-Type: application/json" \
     -d '{"id":1,"title":"Full Replace","body":"New body","userId":1}' \
     https://jsonplaceholder.typicode.com/posts/1

Viewing the Raw Request

Use -v to see exactly what cURL sends:

curl -v -X POST \
     -H "Content-Type: application/json" \
     -d '{"title":"Debug Me"}' \
     https://jsonplaceholder.typicode.com/posts

Output shows:

> POST /posts HTTP/2
> Host: jsonplaceholder.typicode.com
> User-Agent: curl/8.4.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 20
>
> {"title":"Debug Me"}

---

7. Request Encoding

When sending special characters in URLs or bodies, encoding matters:

URL Encoding

Special characters in query parameters must be URL-encoded:

Character	Encoded	Example
Space	%20 or +	?q=hello%20world
&	%26	?q=rock%26roll
=	%3D	?formula=1%2B1%3D2
/	%2F	?path=a%2Fb%2Fc

cURL handles basic encoding, but be aware when values contain special characters:

# Search for "hello world" (space encoded as %20)
curl "https://api.example.com/search?q=hello%20world"

# Or use --data-urlencode for automatic encoding
curl -G --data-urlencode "q=hello world" https://api.example.com/search

---

Summary and Key Takeaways

1. Every HTTP request has four parts: request line, headers, empty line, body.
2. The Content-Type header tells the server what format your body is in.
3. The Accept header tells the server what format you want back.
4. URLs have a precise structure: scheme, host, port, path, query string, fragment.
5. Query parameters handle filtering, sorting, pagination, and search.
6. Use curl -v to see the raw request your client sends.
7. Special characters in URLs need URL encoding.

---

Lesson Review Quiz