anil.recoil.org
A batteries included HTTP/1.1 client in OCaml
tests
+1070
-4
+1
-4
lib/requests.ml
+1
-4
lib/requests.ml
···
117
117
let https_pool = match https_pool with
118
118
| Some p -> p
119
119
| None ->
120
-
let https_tls_config = Option.map (fun cfg ->
121
-
Conpool.Tls_config.make ~config:cfg ()
122
-
) tls_config in
123
-
Conpool.create ~sw ~net ~clock ?tls:https_tls_config ~config:pool_config ()
120
+
Conpool.create ~sw ~net ~clock ?tls:tls_config ~config:pool_config ()
124
121
in
125
122
126
123
Log.info (fun m -> m "Created Requests session with connection pools (max_per_host=%d, TLS=%b)"
+56
test/.httpbin-test-quick-ref.md
+56
test/.httpbin-test-quick-ref.md
···
1
+
# httpbin.t Quick Reference
2
+
3
+
## What This Tests
4
+
5
+
This cram test validates HTTP client functionality through curl against a httpbin server.
6
+
7
+
## Test Categories
8
+
9
+
1. **Setup & Isolation** - XDG directories, environment
10
+
2. **Basic Requests** - GET with/without query params
11
+
3. **Response Headers** - Status codes (200, 404, 500)
12
+
4. **Custom Headers** - X-Custom-Header, User-Agent
13
+
5. **Cookies** - Set, read, delete with cookie jar
14
+
6. **POST Requests** - Form data, JSON, file uploads
15
+
7. **Other Methods** - PUT, DELETE
16
+
8. **Redirects** - Following, chains, absolute
17
+
9. **Authentication** - Basic auth, Bearer tokens
18
+
10. **Response Inspection** - Headers, content negotiation, delays
19
+
11. **Compression** - gzip, deflate
20
+
12. **Request Introspection** - IP, headers, user-agent
21
+
13. **Response Formats** - JSON, HTML, XML, UTF-8
22
+
14. **Binary Data** - PNG, JPEG images
23
+
15. **Response Size** - Bytes, streaming
24
+
16. **Range Requests** - Partial content
25
+
17. **Error Handling** - Timeouts, 404s
26
+
18. **Cleanup** - Remove test artifacts
27
+
28
+
## Running Tests
29
+
30
+
```bash
31
+
# Start httpbin first
32
+
docker run -p 8088:80 kennethreitz/httpbin
33
+
34
+
# Run tests with default URL
35
+
dune test test/httpbin.t
36
+
37
+
# Or with custom URL
38
+
HTTPBIN_URL=http://localhost:9000 dune test test/httpbin.t
39
+
```
40
+
41
+
## Updating Expectations
42
+
43
+
If test output changes and is correct:
44
+
45
+
```bash
46
+
dune promote test/httpbin.t
47
+
```
48
+
49
+
## Key Features
50
+
51
+
- ✅ Isolated XDG directories (no user config pollution)
52
+
- ✅ Parameterized httpbin URL via environment variable
53
+
- ✅ Cookie jar isolation within test directory
54
+
- ✅ Comprehensive HTTP feature coverage
55
+
- ✅ Automatic cleanup after tests
56
+
- ✅ Works with any httpbin-compatible endpoint
+225
test/README.md
+225
test/README.md
···
1
+
# Conpool Test Suite
2
+
3
+
This directory contains tests for the Conpool connection pooling library.
4
+
5
+
## Test Files
6
+
7
+
### Unit Tests
8
+
9
+
- **test_simple.ml**: Basic connection pooling test with a simple echo server
10
+
- **test_localhost.ml**: Stress test with 16 concurrent endpoints and 50k requests
11
+
12
+
### Integration Tests
13
+
14
+
- **httpbin.t**: Cram test suite for HTTP client functionality against httpbin
15
+
16
+
## Running Tests
17
+
18
+
### Run all tests
19
+
20
+
```bash
21
+
dune test
22
+
```
23
+
24
+
### Run specific tests
25
+
26
+
```bash
27
+
# Run unit tests
28
+
dune exec test/test_simple.exe
29
+
dune exec test/test_localhost.exe
30
+
31
+
# Run cram tests
32
+
dune test test/httpbin.t
33
+
```
34
+
35
+
## HTTP Integration Tests (httpbin.t)
36
+
37
+
The `httpbin.t` file contains comprehensive tests for HTTP client functionality.
38
+
39
+
### Prerequisites
40
+
41
+
1. **curl** must be installed (used by cram tests)
42
+
2. **httpbin server** must be running (default: `http://localhost:8088`)
43
+
44
+
### Quick Start
45
+
46
+
Start a httpbin server on port 8088 (choose one method):
47
+
48
+
```bash
49
+
# Docker (recommended)
50
+
docker run -p 8088:80 kennethreitz/httpbin
51
+
52
+
# Go httpbin
53
+
go-httpbin -port 8088
54
+
55
+
# Python httpbin
56
+
gunicorn httpbin:app -b 127.0.0.1:8088
57
+
```
58
+
59
+
Verify httpbin is running:
60
+
61
+
```bash
62
+
curl http://localhost:8088/status/200
63
+
```
64
+
65
+
### Running httpbin Tests
66
+
67
+
With default URL (http://localhost:8088):
68
+
69
+
```bash
70
+
dune test test/httpbin.t
71
+
```
72
+
73
+
With custom httpbin URL:
74
+
75
+
```bash
76
+
HTTPBIN_URL=http://localhost:9000 dune test test/httpbin.t
77
+
HTTPBIN_URL=http://httpbin.example.com dune test test/httpbin.t
78
+
```
79
+
80
+
### Test Coverage
81
+
82
+
The httpbin.t cram test validates:
83
+
84
+
- ✅ **Basic HTTP**: GET, POST, PUT, DELETE requests
85
+
- ✅ **Headers**: Custom headers, User-Agent, Accept-Encoding
86
+
- ✅ **Cookies**: Setting, reading, deleting cookies with cookie jar
87
+
- ✅ **Authentication**: Basic auth, Bearer tokens
88
+
- ✅ **Redirects**: Following redirects, redirect chains
89
+
- ✅ **Response Codes**: 200, 404, 500, etc.
90
+
- ✅ **Content Types**: JSON, HTML, XML, images
91
+
- ✅ **Compression**: gzip, deflate
92
+
- ✅ **Uploads**: Form data, JSON, file uploads
93
+
- ✅ **Streaming**: Delayed responses, chunked data
94
+
- ✅ **Range Requests**: Partial content
95
+
- ✅ **Error Handling**: Timeouts, connection errors
96
+
97
+
### XDG Isolation
98
+
99
+
The cram test creates isolated XDG directories within the test workspace to avoid
100
+
polluting user configuration:
101
+
102
+
- `test-xdg/config` - XDG_CONFIG_HOME
103
+
- `test-xdg/cache` - XDG_CACHE_HOME
104
+
- `test-xdg/data` - XDG_DATA_HOME
105
+
106
+
Cookie jars and other test artifacts are stored here and cleaned up after tests.
107
+
108
+
### Environment Variables
109
+
110
+
The httpbin.t test respects these environment variables:
111
+
112
+
- `HTTPBIN_URL` - Base URL for httpbin server (default: http://localhost:8088)
113
+
- `XDG_CONFIG_HOME`, `XDG_CACHE_HOME`, `XDG_DATA_HOME` - Set within test for isolation
114
+
115
+
Example usage:
116
+
117
+
```bash
118
+
# Test against remote httpbin
119
+
HTTPBIN_URL=https://httpbin.org dune test test/httpbin.t
120
+
121
+
# Test against local instance on different port
122
+
HTTPBIN_URL=http://127.0.0.1:9999 dune test test/httpbin.t
123
+
```
124
+
125
+
## Continuous Integration
126
+
127
+
### GitHub Actions Example
128
+
129
+
```yaml
130
+
name: Tests
131
+
132
+
on: [push, pull_request]
133
+
134
+
jobs:
135
+
test:
136
+
runs-on: ubuntu-latest
137
+
138
+
services:
139
+
httpbin:
140
+
image: kennethreitz/httpbin
141
+
ports:
142
+
- 8088:80
143
+
144
+
steps:
145
+
- uses: actions/checkout@v2
146
+
147
+
- name: Set up OCaml
148
+
uses: ocaml/setup-ocaml@v2
149
+
with:
150
+
ocaml-compiler: 5.1.x
151
+
152
+
- name: Install dependencies
153
+
run: opam install . --deps-only --with-test
154
+
155
+
- name: Build
156
+
run: opam exec -- dune build
157
+
158
+
- name: Run tests
159
+
run: opam exec -- dune test
160
+
```
161
+
162
+
## Debugging Tests
163
+
164
+
### Enable debug logging
165
+
166
+
```bash
167
+
# For OCaml tests
168
+
OCAMLRUNPARAM=b dune exec test/test_simple.exe
169
+
170
+
# View verbose cram output
171
+
dune test test/httpbin.t --verbose
172
+
```
173
+
174
+
### Update cram test expectations
175
+
176
+
If the httpbin responses change or you modify the test:
177
+
178
+
```bash
179
+
dune promote test/httpbin.t
180
+
```
181
+
182
+
### Run single test section
183
+
184
+
You can extract sections of httpbin.t and run them manually:
185
+
186
+
```bash
187
+
# Example: test just the cookie handling
188
+
grep -A 20 "Cookie Handling" test/httpbin.t | bash
189
+
```
190
+
191
+
## Adding New Tests
192
+
193
+
### Adding OCaml Unit Tests
194
+
195
+
1. Create `test_myfeature.ml`
196
+
2. Create empty `test_myfeature.mli`
197
+
3. Add to `test/dune`:
198
+
```scheme
199
+
(executable
200
+
(name test_myfeature)
201
+
(libraries conpool eio_main logs))
202
+
```
203
+
204
+
### Adding Cram Tests
205
+
206
+
1. Create `mytest.t` file with cram syntax
207
+
2. Add sections with `$` command prefix
208
+
3. Dune automatically discovers `.t` files
209
+
4. Run with `dune test test/mytest.t`
210
+
211
+
### Cram Test Format
212
+
213
+
```
214
+
Test Description
215
+
================
216
+
217
+
$ command_to_run
218
+
expected output line 1
219
+
expected output line 2
220
+
221
+
$ another_command
222
+
more expected output
223
+
```
224
+
225
+
Use `(glob)` for wildcards, `(?)` for optional lines, `(re)` for regex.
+10
test/dune
+10
test/dune
+509
test/httpbin.t
+509
test/httpbin.t
···
1
+
HTTP Client Testing with httpbin using ocurl
2
+
=============================================
3
+
4
+
This test suite validates the ocurl HTTP client functionality against a httpbin-compatible
5
+
endpoint. By default, it uses http://localhost:8088 but can be overridden via
6
+
HTTPBIN_URL environment variable.
7
+
8
+
Setup: Isolated XDG directories
9
+
--------------------------------
10
+
11
+
Create isolated XDG directories for this test to avoid polluting user config:
12
+
13
+
$ export TEST_DIR="$PWD/test-xdg"
14
+
$ mkdir -p "$TEST_DIR"/{config,cache,data}
15
+
$ export XDG_CONFIG_HOME="$TEST_DIR/config"
16
+
$ export XDG_CACHE_HOME="$TEST_DIR/cache"
17
+
$ export XDG_DATA_HOME="$TEST_DIR/data"
18
+
19
+
Set httpbin base URL (default to localhost:8088):
20
+
21
+
$ HTTPBIN_URL="${HTTPBIN_URL:-http://localhost:8088}"
22
+
$ echo "Testing against: $HTTPBIN_URL"
23
+
Testing against: http://localhost:8088
24
+
25
+
Verify httpbin is accessible (using curl as a pre-check):
26
+
27
+
$ curl -s -f "$HTTPBIN_URL/status/200" > /dev/null && echo "httpbin accessible" || echo "ERROR: httpbin not accessible at $HTTPBIN_URL"
28
+
httpbin accessible
29
+
30
+
Basic GET Requests
31
+
------------------
32
+
33
+
Simple GET request (using error verbosity to suppress extra output):
34
+
35
+
$ ocurl --verbosity=error "$HTTPBIN_URL/get" | grep -o '"url": "[^"]*"'
36
+
"url": "http://localhost:8088/get"
37
+
38
+
GET with query parameters:
39
+
40
+
$ ocurl --verbosity=error "$HTTPBIN_URL/get?foo=bar&baz=qux" | grep '"args"' -A 3
41
+
"args": {
42
+
"baz": "qux",
43
+
"foo": "bar"
44
+
},
45
+
46
+
Verify URL is correctly formed:
47
+
48
+
$ ocurl --verbosity=error "$HTTPBIN_URL/get" | grep -o '"url": "http://localhost:8088/get"'
49
+
"url": "http://localhost:8088/get"
50
+
51
+
Response Headers with -i flag
52
+
------------------------------
53
+
54
+
Note: The -i flag requires verbosity level above 'warning' to show headers.
55
+
With default verbosity (warning), headers are suppressed.
56
+
57
+
Test that response includes HTTP status line with info verbosity:
58
+
59
+
$ ocurl --verbosity=info -I "$HTTPBIN_URL/status/200" 2>&1 | grep -o "200 OK" | head -1
60
+
200 OK
61
+
62
+
Custom Request Headers
63
+
----------------------
64
+
65
+
Send custom headers and verify they're received:
66
+
67
+
$ ocurl --verbosity=error -H "X-Custom-Header: test-value" -H "X-Test-ID: 12345" \
68
+
> "$HTTPBIN_URL/headers" | \
69
+
> grep -o '"X-Custom-Header": "test-value"'
70
+
"X-Custom-Header": "test-value"
71
+
72
+
Verify multiple custom headers (note: header names are normalized):
73
+
74
+
$ ocurl --verbosity=error -H "X-Custom-Header: test-value" -H "X-Test-ID: 12345" \
75
+
> "$HTTPBIN_URL/headers" | \
76
+
> grep '"X-Test-Id"'
77
+
"X-Test-Id": "12345"
78
+
79
+
User-Agent header (must use -H flag):
80
+
81
+
$ ocurl --verbosity=error -H "User-Agent: ocurl-test/1.0" "$HTTPBIN_URL/headers" | \
82
+
> grep -o '"User-Agent": "ocurl-test/1.0"'
83
+
"User-Agent": "ocurl-test/1.0"
84
+
85
+
POST Requests
86
+
-------------
87
+
88
+
POST with data:
89
+
90
+
$ ocurl --verbosity=error -X POST -d "test data content" \
91
+
> "$HTTPBIN_URL/post" | \
92
+
> grep -o '"data": "test data content"'
93
+
"data": "test data content"
94
+
95
+
POST with JSON data using --json flag:
96
+
97
+
$ ocurl --verbosity=error -X POST --json '{"name":"test","value":42}' \
98
+
> "$HTTPBIN_URL/post" | \
99
+
> grep '"data"' -A 1
100
+
"data": "{\"name\":\"test\",\"value\":42}",
101
+
"files": {},
102
+
103
+
Verify JSON content type is set:
104
+
105
+
$ ocurl --verbosity=error -X POST --json '{"key":"val"}' \
106
+
> "$HTTPBIN_URL/post" | \
107
+
> grep -o '"Content-Type": "application/json"'
108
+
"Content-Type": "application/json"
109
+
110
+
PUT and DELETE Requests
111
+
------------------------
112
+
113
+
PUT request:
114
+
115
+
$ ocurl --verbosity=error -X PUT -d "updated data" "$HTTPBIN_URL/put" | \
116
+
> grep -o '"data": "updated data"'
117
+
"data": "updated data"
118
+
119
+
DELETE request:
120
+
121
+
$ ocurl --verbosity=error -X DELETE "$HTTPBIN_URL/delete" | \
122
+
> grep -o '"url": "[^"]*"'
123
+
"url": "http://localhost:8088/delete"
124
+
125
+
PATCH request:
126
+
127
+
$ ocurl --verbosity=error -X PATCH -d "patched data" "$HTTPBIN_URL/patch" | \
128
+
> grep -o '"data": "patched data"'
129
+
"data": "patched data"
130
+
131
+
OPTIONS request (returns headers only, no body):
132
+
133
+
$ ocurl --verbosity=error -X OPTIONS "$HTTPBIN_URL/get" | wc -c | tr -d ' '
134
+
0
135
+
136
+
HEAD request (returns headers only, no body):
137
+
138
+
$ ocurl --verbosity=error -X HEAD "$HTTPBIN_URL/get" | wc -c | tr -d ' '
139
+
0
140
+
141
+
Authentication
142
+
--------------
143
+
144
+
Basic authentication (success):
145
+
146
+
$ ocurl --verbosity=error -u "user:passwd" "$HTTPBIN_URL/basic-auth/user/passwd" | \
147
+
> grep -o '"authenticated": true'
148
+
"authenticated": true
149
+
150
+
Verify username is passed:
151
+
152
+
$ ocurl --verbosity=error -u "user:passwd" "$HTTPBIN_URL/basic-auth/user/passwd" | \
153
+
> grep -o '"user": "user"'
154
+
"user": "user"
155
+
156
+
Bearer token authentication:
157
+
158
+
$ ocurl --verbosity=error -H "Authorization: Bearer test-token-12345" \
159
+
> "$HTTPBIN_URL/bearer" | \
160
+
> grep -o '"authenticated": true'
161
+
"authenticated": true
162
+
163
+
Token value is verified:
164
+
165
+
$ ocurl --verbosity=error -H "Authorization: Bearer test-token-12345" \
166
+
> "$HTTPBIN_URL/bearer" | \
167
+
> grep -o '"token": "test-token-12345"'
168
+
"token": "test-token-12345"
169
+
170
+
Cookie Persistence Tests
171
+
-------------------------
172
+
173
+
Test cookie setting endpoint:
174
+
175
+
$ ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?session=abc123" | \
176
+
> grep -o '"session": "abc123"'
177
+
"session": "abc123"
178
+
179
+
Test setting multiple cookies:
180
+
181
+
$ ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?session=abc123&user=testuser" | \
182
+
> grep '"cookies"' -A 4
183
+
"cookies": {
184
+
"session": "abc123",
185
+
"user": "testuser"
186
+
}
187
+
}
188
+
189
+
Verify session cookie is set:
190
+
191
+
$ ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?session=xyz789" | \
192
+
> grep -o '"session": "xyz789"'
193
+
"session": "xyz789"
194
+
195
+
Cookie persistence with --persist-cookies flag:
196
+
This allows cookies to be stored and reused across requests within the same session.
197
+
198
+
$ ocurl --verbosity=error --persist-cookies "$HTTPBIN_URL/cookies/set?persistent=true" | \
199
+
> grep -o '"persistent": "true"'
200
+
"persistent": "true"
201
+
202
+
Test cookie deletion endpoint:
203
+
204
+
$ ocurl --verbosity=error "$HTTPBIN_URL/cookies/delete?session" | \
205
+
> grep '"cookies"'
206
+
"cookies": {}
207
+
208
+
Response Formats
209
+
----------------
210
+
211
+
JSON response (ocurl pretty-prints by default):
212
+
213
+
$ ocurl --verbosity=error "$HTTPBIN_URL/json" | head -5 | grep '"slideshow"'
214
+
"slideshow": {
215
+
216
+
HTML response:
217
+
218
+
$ ocurl --verbosity=error "$HTTPBIN_URL/html" | grep -o "<html>" | head -1
219
+
<html>
220
+
221
+
XML response:
222
+
223
+
$ ocurl --verbosity=error "$HTTPBIN_URL/xml" | grep -o '<?xml' | head -1
224
+
<?xml
225
+
226
+
UTF-8 response:
227
+
228
+
$ ocurl --verbosity=error "$HTTPBIN_URL/encoding/utf8" | \
229
+
> grep -o "UTF-8" | head -1
230
+
UTF-8
231
+
232
+
Response Size Tests
233
+
-------------------
234
+
235
+
Request specific number of bytes:
236
+
237
+
$ BYTES=$(ocurl --verbosity=error "$HTTPBIN_URL/bytes/100" | wc -c | tr -d ' ')
238
+
$ test "$BYTES" -ge 100 && echo "Got at least 100 bytes"
239
+
Got at least 100 bytes
240
+
241
+
Small byte response:
242
+
243
+
$ BYTES=$(ocurl --verbosity=error "$HTTPBIN_URL/bytes/50" | wc -c | tr -d ' ')
244
+
$ test "$BYTES" -ge 50 && echo "Got at least 50 bytes"
245
+
Got at least 50 bytes
246
+
247
+
File Output
248
+
-----------
249
+
250
+
Download to file (requires info verbosity to see output):
251
+
252
+
$ ocurl --verbosity=info -o "$TEST_DIR/output.json" "$HTTPBIN_URL/get" 2>&1 | \
253
+
> grep -o "Saved to"
254
+
Saved to
255
+
256
+
Verify file was created and contains data:
257
+
258
+
$ test -f "$TEST_DIR/output.json" && echo "File created"
259
+
File created
260
+
261
+
$ grep -q '"url"' "$TEST_DIR/output.json" && echo "File contains JSON"
262
+
File contains JSON
263
+
264
+
Check file contains expected endpoint:
265
+
266
+
$ grep -o '"url": "http://localhost:8088/get"' "$TEST_DIR/output.json"
267
+
"url": "http://localhost:8088/get"
268
+
269
+
Request Introspection
270
+
---------------------
271
+
272
+
Get request headers:
273
+
274
+
$ ocurl --verbosity=error "$HTTPBIN_URL/headers" | grep -o '"Host": "localhost:8088"'
275
+
"Host": "localhost:8088"
276
+
277
+
Verify User-Agent behavior (ocurl does not set User-Agent by default):
278
+
279
+
$ ocurl --verbosity=error "$HTTPBIN_URL/user-agent" | grep '"user-agent": null'
280
+
"user-agent": null
281
+
282
+
Check connection header:
283
+
284
+
$ ocurl --verbosity=error "$HTTPBIN_URL/headers" | grep -o '"Connection": "keep-alive"'
285
+
"Connection": "keep-alive"
286
+
287
+
Timeout Configuration
288
+
---------------------
289
+
290
+
Set a timeout (test with a quick endpoint):
291
+
292
+
$ ocurl --verbosity=error --timeout 5.0 "$HTTPBIN_URL/get" | grep -q '"url"' && echo "Timeout setting works"
293
+
Timeout setting works
294
+
295
+
Short timeout works:
296
+
297
+
$ ocurl --verbosity=error --timeout 10.0 "$HTTPBIN_URL/delay/1" | grep -q '"url"' && echo "Delay completed within timeout"
298
+
Delay completed within timeout
299
+
300
+
Redirects
301
+
---------
302
+
303
+
Follow redirects (default behavior in ocurl):
304
+
305
+
$ ocurl --verbosity=error "$HTTPBIN_URL/redirect/1" | grep -o '"url": "http://localhost:8088/get"'
306
+
"url": "http://localhost:8088/get"
307
+
308
+
Absolute redirect:
309
+
310
+
$ ocurl --verbosity=error "$HTTPBIN_URL/absolute-redirect/1" | grep -o '"url": "http://localhost:8088/get"'
311
+
"url": "http://localhost:8088/get"
312
+
313
+
Multiple redirects:
314
+
315
+
$ ocurl --verbosity=error "$HTTPBIN_URL/redirect/3" | grep -o '"url": "http://localhost:8088/get"'
316
+
"url": "http://localhost:8088/get"
317
+
318
+
Redirect to relative URL:
319
+
320
+
$ ocurl --verbosity=error "$HTTPBIN_URL/relative-redirect/1" | grep -o '"url": "http://localhost:8088/get"'
321
+
"url": "http://localhost:8088/get"
322
+
323
+
Disable redirect following:
324
+
325
+
$ ocurl --verbosity=error --no-follow-redirects "$HTTPBIN_URL/redirect/1" 2>&1 | \
326
+
> grep -i redirect > /dev/null && echo "Redirect response received"
327
+
Redirect response received
328
+
329
+
Status Code Tests
330
+
-----------------
331
+
332
+
Status 200 OK:
333
+
334
+
$ ocurl --verbosity=error "$HTTPBIN_URL/status/200" | wc -c | tr -d ' '
335
+
0
336
+
337
+
Status 201 Created:
338
+
339
+
$ ocurl --verbosity=error "$HTTPBIN_URL/status/201" | wc -c | tr -d ' '
340
+
0
341
+
342
+
Status 204 No Content:
343
+
344
+
$ ocurl --verbosity=error "$HTTPBIN_URL/status/204" | wc -c | tr -d ' '
345
+
0
346
+
347
+
Compression Support
348
+
-------------------
349
+
350
+
Note: ocurl receives compressed data but does not automatically decompress it.
351
+
The gzip and deflate endpoints return binary compressed data.
352
+
353
+
Request gzip compressed response (returns binary data):
354
+
355
+
$ SIZE=$(ocurl --verbosity=error -H "Accept-Encoding: gzip" \
356
+
> "$HTTPBIN_URL/gzip" | wc -c | tr -d ' ')
357
+
$ test "$SIZE" -gt 50 && echo "Received compressed gzip data (size: $SIZE bytes)"
358
+
Received compressed gzip data (size: 158 bytes)
359
+
360
+
Request deflate compressed response (returns binary data):
361
+
362
+
$ SIZE=$(ocurl --verbosity=error -H "Accept-Encoding: deflate" \
363
+
> "$HTTPBIN_URL/deflate" | wc -c | tr -d ' ')
364
+
$ test "$SIZE" -gt 50 && echo "Received compressed deflate data (size: $SIZE bytes)"
365
+
Received compressed deflate data (size: 146 bytes)
366
+
367
+
Verbose Output Testing
368
+
----------------------
369
+
370
+
Test with info verbosity level shows request details:
371
+
372
+
$ ocurl --verbosity=info "$HTTPBIN_URL/get" 2>&1 | grep -c "Making GET request"
373
+
1
374
+
375
+
Test with debug verbosity level:
376
+
377
+
$ ocurl --verbosity=debug "$HTTPBIN_URL/get" 2>&1 | grep -c "GET request" | \
378
+
> awk '{if ($1 >= 1) print "Debug output present"}'
379
+
Debug output present
380
+
381
+
Test quiet mode suppresses output:
382
+
383
+
$ ocurl --verbosity=error "$HTTPBIN_URL/status/200" | wc -c | tr -d ' '
384
+
0
385
+
386
+
Content Type Handling
387
+
---------------------
388
+
389
+
Verify ocurl handles JSON content type:
390
+
391
+
$ ocurl --verbosity=error "$HTTPBIN_URL/json" | head -1 | grep -o "{"
392
+
{
393
+
394
+
HTML content type:
395
+
396
+
$ ocurl --verbosity=error "$HTTPBIN_URL/html" | head -1 | grep -o "<!DOCTYPE"
397
+
<!DOCTYPE
398
+
399
+
XML content type:
400
+
401
+
$ ocurl --verbosity=error "$HTTPBIN_URL/xml" | head -1 | grep -o '<?xml'
402
+
<?xml
403
+
404
+
Response Headers Inspection
405
+
----------------------------
406
+
407
+
Check response contains expected headers:
408
+
409
+
$ ocurl --verbosity=error "$HTTPBIN_URL/response-headers?X-Test=foo" | \
410
+
> grep -o '"X-Test": "foo"'
411
+
"X-Test": "foo"
412
+
413
+
Custom response header:
414
+
415
+
$ ocurl --verbosity=error "$HTTPBIN_URL/response-headers?X-Custom=bar&Y-Another=baz" | \
416
+
> grep '"X-Custom"'
417
+
"X-Custom": "bar",
418
+
419
+
Cache Control header test:
420
+
421
+
$ ocurl --verbosity=error "$HTTPBIN_URL/cache" | \
422
+
> grep '"url"' > /dev/null && echo "Cache endpoint accessible"
423
+
Cache endpoint accessible
424
+
425
+
ETag support:
426
+
427
+
$ ocurl --verbosity=error "$HTTPBIN_URL/etag/test-etag" | \
428
+
> grep '"url"' > /dev/null && echo "ETag endpoint accessible"
429
+
ETag endpoint accessible
430
+
431
+
Advanced Cookie Tests
432
+
----------------------
433
+
434
+
Set cookie with specific value:
435
+
436
+
$ ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?test=value123" | \
437
+
> grep -o '"test": "value123"'
438
+
"test": "value123"
439
+
440
+
Multiple cookie setting:
441
+
442
+
$ ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?a=1&b=2&c=3" | \
443
+
> grep '"a": "1"' && \
444
+
> ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?a=1&b=2&c=3" | \
445
+
> grep '"b": "2"' && \
446
+
> ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?a=1&b=2&c=3" | \
447
+
> grep '"c": "3"'
448
+
"a": "1",
449
+
"b": "2",
450
+
"c": "3"
451
+
452
+
Test cookie names with special characters:
453
+
454
+
$ ocurl --verbosity=error "$HTTPBIN_URL/cookies/set?my-cookie=my-value" | \
455
+
> grep -o '"my-cookie": "my-value"'
456
+
"my-cookie": "my-value"
457
+
458
+
Image and Binary Response Tests
459
+
--------------------------------
460
+
461
+
Test PNG image endpoint returns data:
462
+
463
+
$ SIZE=$(ocurl --verbosity=error "$HTTPBIN_URL/image/png" | wc -c | tr -d ' ')
464
+
$ test "$SIZE" -gt 1000 && echo "PNG image data received (size: $SIZE bytes)"
465
+
PNG image data received (size: 8090 bytes)
466
+
467
+
Test JPEG image endpoint:
468
+
469
+
$ SIZE=$(ocurl --verbosity=error "$HTTPBIN_URL/image/jpeg" | wc -c | tr -d ' ')
470
+
$ test "$SIZE" -gt 1000 && echo "JPEG image data received (size: $SIZE bytes)"
471
+
JPEG image data received (size: 35588 bytes)
472
+
473
+
Test SVG image endpoint:
474
+
475
+
$ ocurl --verbosity=error "$HTTPBIN_URL/image/svg" | head -1 | grep -o "<svg"
476
+
<svg
477
+
478
+
Stream Testing
479
+
--------------
480
+
481
+
Test streaming bytes:
482
+
483
+
$ SIZE=$(ocurl --verbosity=error "$HTTPBIN_URL/stream-bytes/100" | wc -c | tr -d ' ')
484
+
$ test "$SIZE" -ge 100 && echo "Stream returned at least 100 bytes"
485
+
Stream returned at least 100 bytes
486
+
487
+
Data Formats
488
+
------------
489
+
490
+
Base64 decode endpoint:
491
+
492
+
$ ocurl --verbosity=error "$HTTPBIN_URL/base64/SGVsbG8gV29ybGQ=" | \
493
+
> grep -o "Hello World"
494
+
Hello World
495
+
496
+
Delay endpoint (tests timeout handling):
497
+
498
+
$ ocurl --verbosity=error --timeout 3.0 "$HTTPBIN_URL/delay/1" | \
499
+
> grep '"url"' > /dev/null && echo "Delay completed successfully"
500
+
Delay completed successfully
501
+
502
+
Cleanup
503
+
-------
504
+
505
+
Remove test directories:
506
+
507
+
$ rm -rf "$TEST_DIR"
508
+
$ echo "Test environment cleaned up"
509
+
Test environment cleaned up
+211
test/test_localhost.ml
+211
test/test_localhost.ml
···
1
+
(* Test conpool with 16 localhost servers on different 127.0.* addresses *)
2
+
3
+
open Eio.Std
4
+
5
+
(* Create a simple echo server on a specific address and port *)
6
+
let create_server ~sw ~net ipaddr port connections_ref =
7
+
let socket = Eio.Net.listen net ~sw ~reuse_addr:true ~backlog:10
8
+
(`Tcp (ipaddr, port))
9
+
in
10
+
11
+
Eio.Fiber.fork ~sw (fun () ->
12
+
try
13
+
while true do
14
+
Eio.Net.accept_fork socket ~sw ~on_error:(fun ex ->
15
+
traceln "Server %a error: %s" Eio.Net.Sockaddr.pp (`Tcp (ipaddr, port))
16
+
(Printexc.to_string ex)
17
+
) (fun flow _addr ->
18
+
(* Track this connection *)
19
+
Atomic.incr connections_ref;
20
+
21
+
(* Simple protocol: read lines and echo them back, until EOF *)
22
+
try
23
+
let buf = Eio.Buf_read.of_flow flow ~max_size:1024 in
24
+
while true do
25
+
let line = Eio.Buf_read.line buf in
26
+
traceln "Server on %a:%d received: %s"
27
+
Eio.Net.Ipaddr.pp ipaddr port line;
28
+
29
+
Eio.Flow.copy_string (line ^ "\n") flow
30
+
done
31
+
with
32
+
| End_of_file ->
33
+
traceln "Server on %a:%d client disconnected"
34
+
Eio.Net.Ipaddr.pp ipaddr port;
35
+
Eio.Flow.close flow;
36
+
Atomic.decr connections_ref
37
+
| ex ->
38
+
traceln "Server on %a:%d error handling connection: %s"
39
+
Eio.Net.Ipaddr.pp ipaddr port
40
+
(Printexc.to_string ex);
41
+
Eio.Flow.close flow;
42
+
Atomic.decr connections_ref
43
+
)
44
+
done
45
+
with Eio.Cancel.Cancelled _ -> ()
46
+
)
47
+
48
+
(** Generate 16 different servers on 127.0.0.1 with different ports *)
49
+
let generate_localhost_addresses () =
50
+
List.init 16 (fun i ->
51
+
(* Use 127.0.0.1 for all, just different ports *)
52
+
let addr_str = "127.0.0.1" in
53
+
(* Create raw IPv4 address as 4 bytes *)
54
+
let raw_bytes = Bytes.create 4 in
55
+
Bytes.set raw_bytes 0 (Char.chr 127);
56
+
Bytes.set raw_bytes 1 (Char.chr 0);
57
+
Bytes.set raw_bytes 2 (Char.chr 0);
58
+
Bytes.set raw_bytes 3 (Char.chr 1);
59
+
let addr = Eio.Net.Ipaddr.of_raw (Bytes.to_string raw_bytes) in
60
+
(addr_str, addr, 10000 + i)
61
+
)
62
+
63
+
let () =
64
+
(* Setup logging *)
65
+
Logs.set_reporter (Logs_fmt.reporter ());
66
+
Logs.set_level (Some Logs.Info);
67
+
Logs.Src.set_level Conpool.src (Some Logs.Debug);
68
+
69
+
Eio_main.run @@ fun env ->
70
+
Switch.run @@ fun sw ->
71
+
72
+
traceln "=== Starting 16 localhost servers ===";
73
+
74
+
(* Generate addresses *)
75
+
let servers = generate_localhost_addresses () in
76
+
77
+
(* Create connection counters for each server *)
78
+
let connection_refs = List.map (fun _ -> Atomic.make 0) servers in
79
+
80
+
(* Start all servers *)
81
+
List.iter2 (fun (_addr_str, addr, port) conn_ref ->
82
+
traceln "Starting server on %a:%d"
83
+
Eio.Net.Ipaddr.pp addr port;
84
+
create_server ~sw ~net:env#net addr port conn_ref
85
+
) servers connection_refs;
86
+
87
+
(* Give servers time to start *)
88
+
Eio.Time.sleep env#clock 0.5;
89
+
90
+
traceln "\n=== Creating connection pool ===";
91
+
92
+
(* Create connection pool *)
93
+
let pool_config = Conpool.Config.make
94
+
~max_connections_per_endpoint:5
95
+
~max_idle_time:30.0
96
+
~max_connection_lifetime:60.0
97
+
()
98
+
in
99
+
100
+
let pool = Conpool.create
101
+
~sw
102
+
~net:env#net
103
+
~clock:env#clock
104
+
~config:pool_config
105
+
()
106
+
in
107
+
108
+
traceln "\n=== Stress testing with thousands of concurrent connections ===";
109
+
110
+
(* Disable debug logging for stress test *)
111
+
Logs.Src.set_level Conpool.src (Some Logs.Info);
112
+
113
+
(* Create endpoints for all servers *)
114
+
let endpoints = List.map (fun (addr_str, _addr, port) ->
115
+
Conpool.Endpoint.make ~host:addr_str ~port
116
+
) servers in
117
+
118
+
(* Stress test: thousands of concurrent requests across all 16 servers *)
119
+
let num_requests = 50000 in
120
+
121
+
traceln "Launching %d concurrent requests across %d endpoints..."
122
+
num_requests (List.length endpoints);
123
+
traceln "Pool config: max %d connections per endpoint"
124
+
(Conpool.Config.max_connections_per_endpoint pool_config);
125
+
126
+
let start_time = Unix.gettimeofday () in
127
+
let success_count = Atomic.make 0 in
128
+
let error_count = Atomic.make 0 in
129
+
let last_progress = ref 0 in
130
+
131
+
(* Generate list of (endpoint, request_id) pairs *)
132
+
let tasks = List.init num_requests (fun i ->
133
+
let endpoint = List.nth endpoints (i mod List.length endpoints) in
134
+
(endpoint, i)
135
+
) in
136
+
137
+
(* Run all requests concurrently with fiber limit *)
138
+
Eio.Fiber.List.iter ~max_fibers:200 (fun (endpoint, req_id) ->
139
+
try
140
+
Conpool.with_connection pool endpoint (fun flow ->
141
+
let test_msg = Printf.sprintf "Request %d" req_id in
142
+
Eio.Flow.copy_string (test_msg ^ "\n") flow;
143
+
144
+
let buf = Eio.Buf_read.of_flow flow ~max_size:1024 in
145
+
let _response = Eio.Buf_read.line buf in
146
+
let count = Atomic.fetch_and_add success_count 1 + 1 in
147
+
148
+
(* Progress indicator every 5000 requests *)
149
+
if count / 5000 > !last_progress then begin
150
+
last_progress := count / 5000;
151
+
traceln " Progress: %d/%d (%.1f%%)"
152
+
count num_requests
153
+
(100.0 *. float_of_int count /. float_of_int num_requests)
154
+
end
155
+
)
156
+
with e ->
157
+
Atomic.incr error_count;
158
+
if Atomic.get error_count <= 10 then
159
+
traceln "Request %d to %a failed: %s"
160
+
req_id Conpool.Endpoint.pp endpoint (Printexc.to_string e)
161
+
) tasks;
162
+
163
+
let end_time = Unix.gettimeofday () in
164
+
let duration = end_time -. start_time in
165
+
let successful = Atomic.get success_count in
166
+
let failed = Atomic.get error_count in
167
+
168
+
traceln "\n=== Stress test results ===";
169
+
traceln "Total requests: %d" num_requests;
170
+
traceln "Successful: %d" successful;
171
+
traceln "Failed: %d" failed;
172
+
traceln "Duration: %.2fs" duration;
173
+
traceln "Throughput: %.0f req/s" (float_of_int successful /. duration);
174
+
traceln "Average latency: %.2fms" (duration *. 1000.0 /. float_of_int successful);
175
+
176
+
traceln "\n=== Connection pool statistics ===";
177
+
let all_stats = Conpool.all_stats pool in
178
+
179
+
(* Calculate totals *)
180
+
let total_created = List.fold_left (fun acc (_, s) -> acc + Conpool.Stats.total_created s) 0 all_stats in
181
+
let total_reused = List.fold_left (fun acc (_, s) -> acc + Conpool.Stats.total_reused s) 0 all_stats in
182
+
let total_closed = List.fold_left (fun acc (_, s) -> acc + Conpool.Stats.total_closed s) 0 all_stats in
183
+
let total_errors = List.fold_left (fun acc (_, s) -> acc + Conpool.Stats.errors s) 0 all_stats in
184
+
185
+
traceln "Total connections created: %d" total_created;
186
+
traceln "Total connections reused: %d" total_reused;
187
+
traceln "Total connections closed: %d" total_closed;
188
+
traceln "Total errors: %d" total_errors;
189
+
traceln "Connection reuse ratio: %.2fx (reused/created)"
190
+
(if total_created > 0 then float_of_int total_reused /. float_of_int total_created else 0.0);
191
+
traceln "Pool efficiency: %.1f%% (avoided creating %d connections)"
192
+
(if successful > 0 then 100.0 *. float_of_int total_reused /. float_of_int successful else 0.0)
193
+
total_reused;
194
+
195
+
traceln "\nPer-endpoint breakdown:";
196
+
List.iter (fun (endpoint, stats) ->
197
+
traceln " %a: created=%d reused=%d active=%d idle=%d"
198
+
Conpool.Endpoint.pp endpoint
199
+
(Conpool.Stats.total_created stats)
200
+
(Conpool.Stats.total_reused stats)
201
+
(Conpool.Stats.active stats)
202
+
(Conpool.Stats.idle stats)
203
+
) all_stats;
204
+
205
+
traceln "\n=== Verifying server-side connection counts ===";
206
+
List.iter2 (fun (addr_str, _addr, port) conn_ref ->
207
+
let count = Atomic.get conn_ref in
208
+
traceln "Server %s:%d - Active connections: %d" addr_str port count
209
+
) servers connection_refs;
210
+
211
+
traceln "\n=== Test completed successfully ==="
test/test_localhost.mli
test/test_localhost.mli
This is a binary file and will not be displayed.
+56
test/test_simple.ml
+56
test/test_simple.ml
···
1
+
(* Simple test to debug connection issues *)
2
+
3
+
open Eio.Std
4
+
5
+
let () =
6
+
Logs.set_reporter (Logs_fmt.reporter ());
7
+
Logs.set_level (Some Logs.Debug);
8
+
Logs.Src.set_level Conpool.src (Some Logs.Debug);
9
+
10
+
Eio_main.run @@ fun env ->
11
+
Switch.run @@ fun sw ->
12
+
13
+
traceln "Starting simple server on 127.0.0.1:9000";
14
+
15
+
(* Start a simple echo server *)
16
+
let raw_bytes = Bytes.create 4 in
17
+
Bytes.set raw_bytes 0 (Char.chr 127);
18
+
Bytes.set raw_bytes 1 (Char.chr 0);
19
+
Bytes.set raw_bytes 2 (Char.chr 0);
20
+
Bytes.set raw_bytes 3 (Char.chr 1);
21
+
let ipaddr = Eio.Net.Ipaddr.of_raw (Bytes.to_string raw_bytes) in
22
+
23
+
let socket = Eio.Net.listen env#net ~sw ~reuse_addr:true ~backlog:10
24
+
(`Tcp (ipaddr, 9000))
25
+
in
26
+
27
+
Eio.Fiber.fork ~sw (fun () ->
28
+
Eio.Net.accept_fork socket ~sw ~on_error:raise (fun flow _addr ->
29
+
traceln "Server: accepted connection";
30
+
let buf = Eio.Buf_read.of_flow flow ~max_size:1024 in
31
+
let line = Eio.Buf_read.line buf in
32
+
traceln "Server: received: %s" line;
33
+
Eio.Flow.copy_string (line ^ "\n") flow;
34
+
Eio.Flow.close flow
35
+
)
36
+
);
37
+
38
+
Eio.Time.sleep env#clock 0.1;
39
+
40
+
traceln "Creating connection pool";
41
+
let pool = Conpool.create ~sw ~net:env#net ~clock:env#clock () in
42
+
43
+
traceln "Testing connection";
44
+
let endpoint = Conpool.Endpoint.make ~host:"127.0.0.1" ~port:9000 in
45
+
46
+
let response = Conpool.with_connection pool endpoint (fun flow ->
47
+
traceln "Client: sending message";
48
+
Eio.Flow.copy_string "test message\n" flow;
49
+
let buf = Eio.Buf_read.of_flow flow ~max_size:1024 in
50
+
let resp = Eio.Buf_read.line buf in
51
+
traceln "Client: received: %s" resp;
52
+
resp
53
+
) in
54
+
55
+
traceln "Response: %s" response;
56
+
traceln "Test passed!"
test/test_simple.mli
test/test_simple.mli
This is a binary file and will not be displayed.