Benchmark results

Generated by php benchmarks/render-results.php from benchmarks/results/*.json. Re-run everything with benchmarks/run-all.sh; method and limits are in README.md.

Everything below was measured against a local HTTP/2 mock server (Node.js) on loopback, on one shared machine. Nothing here says anything about Apple APNs or Google FCM.

Conditions

item value
first run started (UTC) 2026-11-21T00:48:21+00:00
CPU Intel® Xeon® Processor @ 2.10GHz, 4 cores
memory 16473744 kB
kernel 6.18.44-fc-v726
PHP 8.3.6 (cli, NTS), memory_limit -1
libcurl / TLS 8.5.0 / OpenSSL/3.0.13 (HTTP/2)
mock server Node.js v22.23.5 (tests/Support/H2Mock/server.mjs, HTTP/2 over TLS, ECDSA P-256 certificates generated per run)
package padosoft/laravel-mobile-push at commit d35d0d2 (plus the uncommitted working tree)
pushok edamov/pushok 0.19.1 (commit 34e97101ab)
load average when the first run started (1/5/15 min) 19.38 / 19.32 / 18.52

The machine was not idle. The 1-minute load average at the start of each run was (transport-warm-d0: 19.4, transport-warm-d20: 20.0, transport-cold-d0: 20.4, transport-warm-single: 22.2, transport-cold-single: 22.2, pipeline-apns-d0: 20.8, pipeline-apns-d20: 20.8, pipeline-fcm-d0: 20.3, pipeline-fcm-d20: 20.5, pipeline-mixed-d0: 20.2, window-sweep: 19.9, connections-sweep: 18.1, large-batch-memory: 18.0, many-tenants: 23.2, compare-pushok-w100: 23.0, compare-pushok-w20: 26.3, compare-pushok-d0: 22.2), on 4 cores, so the Node mock server, PHP and other processes on the host competed for CPU during every measurement. Treat absolute figures as indicative and compare numbers only within one table (same run, same conditions). Cells that show spread N% varied by more than 15% between repeated runs; for those the range, not the median, is the finding.

Command lines

php benchmarks/run.php --scenario=transport-warm --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/transport-warm-d0.json
php benchmarks/run.php --scenario=transport-warm --count=20000 --server-delay-ms=20 --repeat=5 --json --out=benchmarks/results/transport-warm-d20.json
php benchmarks/run.php --scenario=transport-cold --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/transport-cold-d0.json
php benchmarks/run.php --scenario=transport-warm --count=1 --server-delay-ms=0 --repeat=30 --json --out=benchmarks/results/transport-warm-single.json
php benchmarks/run.php --scenario=transport-cold --count=1 --server-delay-ms=0 --repeat=30 --json --out=benchmarks/results/transport-cold-single.json
php benchmarks/run.php --scenario=pipeline-apns --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/pipeline-apns-d0.json
php benchmarks/run.php --scenario=pipeline-apns --count=20000 --server-delay-ms=20 --repeat=5 --json --out=benchmarks/results/pipeline-apns-d20.json
php benchmarks/run.php --scenario=pipeline-fcm --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/pipeline-fcm-d0.json
php benchmarks/run.php --scenario=pipeline-fcm --count=20000 --server-delay-ms=20 --repeat=5 --json --out=benchmarks/results/pipeline-fcm-d20.json
php benchmarks/run.php --scenario=pipeline-mixed-outcomes --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/pipeline-mixed-d0.json
php benchmarks/run.php --scenario=window-sweep --count=10000 --server-delay-ms=20 --repeat=3 --json --out=benchmarks/results/window-sweep.json
php benchmarks/run.php --scenario=connections-sweep --count=20000 --server-delay-ms=20 --repeat=3 --json --out=benchmarks/results/connections-sweep.json
php benchmarks/run.php --scenario=large-batch-memory --million --json --out=benchmarks/results/large-batch-memory.json
php benchmarks/run.php --scenario=many-tenants --tenants=200 --per-tenant=5 --repeat=3 --json --out=benchmarks/results/many-tenants.json
php benchmarks/run.php --scenario=compare-pushok --count=5000 --window=100 --server-delay-ms=10 --repeat=7 --json --out=benchmarks/results/compare-pushok-w100.json
php benchmarks/run.php --scenario=compare-pushok --count=5000 --window=20 --server-delay-ms=10 --repeat=7 --json --out=benchmarks/results/compare-pushok-w20.json
php benchmarks/run.php --scenario=compare-pushok --count=5000 --window=100 --server-delay-ms=0 --repeat=7 --json --out=benchmarks/results/compare-pushok-d0.json

How to read the tables: requests per run are completed exchanges seen by the client; req/s is requests divided by wall time of the run (median and range over the repeated runs); latency columns are the median over runs of the per-request duration reported by the transport (hand-over to libcurl until completion, so it contains the mock’s artificial delay and any queueing inside libcurl); peak MiB over start is memory_get_peak_usage() after a peak reset minus usage at the start; growth is usage at the end minus at the start, raw and after gc_collect_cycles(); cpu ms/req is user+system CPU of the PHP process per request (the Node server’s CPU is not included); connections and max concurrent streams are counted by the mock server, not by the client.

Results

Raw transport, warm pool, no server delay

php benchmarks/run.php --scenario=transport-warm --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/transport-warm-d0.json

CPU-bound: the cost of the PHP side per request.

Raw CurlTransport, APNs-shaped requests, pool already connected (one request sent before measuring).

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
warm pool, window 100, 1 conn 20,000 6,284 (5,743-8,078) spread 37% 12.0 34.7 56.3 97.6 0.56 516 / 522 0.042 0 0 / 1 100

Raw transport, warm pool, 20 ms server delay

php benchmarks/run.php --scenario=transport-warm --count=20000 --server-delay-ms=20 --repeat=5 --json --out=benchmarks/results/transport-warm-d20.json

Latency-bound: throughput is about window / delay until the client or server saturates.

Raw CurlTransport, APNs-shaped requests, pool already connected (one request sent before measuring).

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
warm pool, window 100, 1 conn 20,000 3,005 (2,749-3,193) 30.9 51.9 69.8 103.3 0.56 516 / 522 0.054 0 0 / 1 100

Raw transport, new pool for every run

php benchmarks/run.php --scenario=transport-cold --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/transport-cold-d0.json

Each run pays one TCP + TLS + HTTP/2 connection setup.

Raw CurlTransport, APNs-shaped requests, a new registry/pool (new TCP+TLS connection) for every run.

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
fresh pool each run, window 100, 1 conn 20,000 7,088 (5,192-8,493) spread 47% 11.4 25.8 35.7 114.2 1.04 1,003 / 1,016 0.040 0 1 / 5 100

One request on a warm pool (30 runs)

php benchmarks/run.php --scenario=transport-warm --count=1 --server-delay-ms=0 --repeat=30 --json --out=benchmarks/results/transport-warm-single.json

The latency columns are the time of that single request: median over the 30 runs.

Raw CurlTransport, APNs-shaped requests, pool already connected (one request sent before measuring).

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
warm pool, window 100, 1 conn 1 1,278 (503-1,685) spread 93% 0.7 0.7 0.7 0.7 0.01 1 / 6 0.000 0 0 / 1 1

One request on a new pool (30 runs)

php benchmarks/run.php --scenario=transport-cold --count=1 --server-delay-ms=0 --repeat=30 --json --out=benchmarks/results/transport-cold-single.json

Same, but each run first opens a TCP + TLS + HTTP/2 connection inside the measured request: the difference to the warm table is the connection setup. cpu ms/req here is the whole CPU of that one request.

Raw CurlTransport, APNs-shaped requests, a new registry/pool (new TCP+TLS connection) for every run.

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
fresh pool each run, window 100, 1 conn 1 149 (82-303) spread 148% 6.6 6.6 6.6 6.6 0.01 7 / 23 2.000 0 1 / 30 1

Full pipeline, APNs, warm, no server delay

php benchmarks/run.php --scenario=pipeline-apns --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/pipeline-apns-d0.json

Full Sender pipeline (compile, guards, transport, classify, report) over the real transport, mock scripted like APNS (2xx with ids).

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
apns pipeline, warm, window 100, 1 conn 20,000 6,303 (5,680-7,102) spread 23% 8.7 24.2 42.2 65.0 1.93 517 / 523 0.069 0 0 / 1 100

Full pipeline, APNs, warm, 20 ms server delay

php benchmarks/run.php --scenario=pipeline-apns --count=20000 --server-delay-ms=20 --repeat=5 --json --out=benchmarks/results/pipeline-apns-d20.json

Full Sender pipeline (compile, guards, transport, classify, report) over the real transport, mock scripted like APNS (2xx with ids).

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
apns pipeline, warm, window 100, 1 conn 20,000 2,499 (2,297-2,723) spread 17% 33.0 48.7 63.7 85.9 1.93 517 / 523 0.085 0 0 / 1 100

Full pipeline, FCM, warm, no server delay

php benchmarks/run.php --scenario=pipeline-fcm --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/pipeline-fcm-d0.json

Full Sender pipeline (compile, guards, transport, classify, report) over the real transport, mock scripted like FCM (2xx with ids).

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
fcm pipeline, warm, window 100, 1 conn 20,000 4,949 (4,459-7,961) spread 71% 12.6 27.9 46.3 75.6 1.86 517 / 523 0.066 0 0 / 1 100

Full pipeline, FCM, warm, 20 ms server delay

php benchmarks/run.php --scenario=pipeline-fcm --count=20000 --server-delay-ms=20 --repeat=5 --json --out=benchmarks/results/pipeline-fcm-d20.json

Full Sender pipeline (compile, guards, transport, classify, report) over the real transport, mock scripted like FCM (2xx with ids).

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
fcm pipeline, warm, window 100, 1 conn 20,000 3,003 (2,633-3,049) 26.5 39.2 50.2 68.8 1.86 517 / 523 0.073 0 0 / 1 100

Full pipeline, mixed outcomes, no server delay

php benchmarks/run.php --scenario=pipeline-mixed-outcomes --count=20000 --server-delay-ms=0 --repeat=5 --json --out=benchmarks/results/pipeline-mixed-d0.json

Outcome counts per run are listed under the table.

Same pipeline with a realistic outcome mix, next to an all-accepted control with identical settings, to show the cost of classification and retry planning.

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
apns pipeline, all accepted (control), window 100 20,000 6,973 (6,357-7,960) spread 23% 8.4 17.3 31.3 33.3 1.93 517 / 523 0.069 0 0 / 1 100
apns pipeline, mixed outcomes (95% accepted, 3% 410 Unregistered, 1% 400 BadDeviceToken, 1% 500), window 100 20,000 7,113 (5,862-7,572) spread 24% 8.6 19.6 26.2 35.2 1.93 517 / 523 0.073 0 0 / 1 100
  • apns pipeline, all accepted (control), window 100: outcomes in one run: {"accepted\/none":20000}, HTTP statuses seen by the transport: {"200":20000}
  • apns pipeline, mixed outcomes (95% accepted, 3% 410 Unregistered, 1% 400 BadDeviceToken, 1% 500), window 100: outcomes in one run: {"rejected\/destination_unregistered":600,"rejected\/destination_rejected":200,"deferred\/provider_internal":200,"accepted\/none":19000}, HTTP statuses seen by the transport: {"410":600,"400":200,"500":200,"200":19000}

Window sweep (20 ms server delay, one connection)

php benchmarks/run.php --scenario=window-sweep --count=10000 --server-delay-ms=20 --repeat=3 --json --out=benchmarks/results/window-sweep.json

Raw CurlTransport, warm, one connection, fixed server delay, varying window. The mock allows 1000 concurrent streams per connection.

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
window 10 10,000 413 (405-420) 21.7 41.8 47.2 52.2 0.27 260 / 266 0.096 0 0 / 1 10
window 50 10,000 1,938 (1,934-2,061) 23.3 39.9 47.2 53.6 0.29 261 / 266 0.063 0 0 / 1 50
window 100 10,000 3,993 (3,705-4,104) 23.8 31.0 37.8 49.6 0.30 260 / 266 0.064 0 0 / 1 100
window 250 10,000 8,436 (7,958-8,479) 27.7 38.7 60.8 62.3 0.31 260 / 266 0.053 0 0 / 1 250
window 500 10,000 7,433 (6,511-8,999) spread 34% 59.3 104.6 116.8 119.8 2.01 1,958 / 266 0.052 0 0 / 1 500

Connections sweep (20 ms server delay, window 500)

php benchmarks/run.php --scenario=connections-sweep --count=20000 --server-delay-ms=20 --repeat=3 --json --out=benchmarks/results/connections-sweep.json

Raw CurlTransport, warm, fixed server delay, 1/2/4 connections. The mock advertises SETTINGS_MAX_CONCURRENT_STREAMS = peer limit.

case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
1 connection(s), peer stream limit 1000, window 500 20,000 8,066 (6,980-8,079) 54.9 123.8 163.8 204.9 2.48 2,480 / 522 0.056 0 0 / 1 500
2 connection(s), peer stream limit 1000, window 500 20,000 7,447 (6,330-9,386) spread 41% 57.3 106.6 141.8 200.4 3.62 3,632 / 522 0.056 0 0 / 1 500
4 connection(s), peer stream limit 1000, window 500 20,000 8,196 (8,166-10,069) spread 23% 50.5 104.6 137.9 161.7 2.64 2,612 / 522 0.055 0 0 / 1 500
1 connection(s), peer stream limit 100, window 500 20,000 3,826 (3,635-3,915) 99.1 322.0 475.5 986.9 1.69 1,674 / 522 0.110 0 0 / 1 100
2 connection(s), peer stream limit 100, window 500 20,000 6,488 (6,009-6,720) 63.1 159.6 228.4 377.2 1.69 1,674 / 523 0.083 0 0 / 2 100
4 connection(s), peer stream limit 100, window 500 20,000 8,338 (6,484-8,485) spread 24% 52.6 97.2 130.6 169.0 1.71 1,697 / 524 0.056 0 0 / 4 100

Large batches (memory stays flat?)

php benchmarks/run.php --scenario=large-batch-memory --million --json --out=benchmarks/results/large-batch-memory.json

Destinations are produced lazily by a generator and the report is consumed as a cursor; memory_get_usage is sampled during the run (about every 5% of the destinations). Latency is sampled (every k-th request) to keep the recorder itself small.

case requests wall s req/s usage MiB at first checkpoint (5%) max usage MiB usage MiB at end range MiB over 2nd half peak MiB over start growth after gc errors connections outcomes
apns pipeline, 100,000 destinations from a generator, window 100 100,000 14.8 6,735 6.64 7.13 7.03 0.25 2.56 1,175 KiB 0 0 {“accepted/none”:100000}
apns pipeline, 1,000,000 destinations from a generator, window 100 1,000,000 173.2 5,774 6.66 7.15 7.05 0.25 2.51 1,119 KiB 0 0 {“accepted/none”:1000000}

Many tenants (200 pool keys in one process)

php benchmarks/run.php --scenario=many-tenants --tenants=200 --per-tenant=5 --repeat=3 --json --out=benchmarks/results/many-tenants.json

One process, one transport, N different pool keys used one after the other (each key = its own connection). Registry limit 32 is the default; 1000 shows what happens without effective eviction.

Per-run series of the last repeat (every 20 tenants):

200 pool keys in sequence, 5 requests each, registry limit 32 (fds before the run: 6; after close(): 6, server open connections after close: 0; PHP memory after gc at the end: 3.81 MiB, start 3.78 MiB)

tenants done pools in registry evictions open fds PHP memory MiB server: open connections server: connections opened in total
20 20 0 66 4.20 20 20
40 32 8 102 4.61 32 40
60 32 28 102 4.99 32 60
80 32 48 102 5.38 32 80
100 32 68 102 5.76 32 100
120 32 88 102 6.15 32 120
140 32 108 102 6.54 32 140
160 32 128 102 6.92 32 160
180 32 148 102 7.31 32 180
200 32 168 102 7.69 32 200

200 pool keys in sequence, 5 requests each, registry limit 1000 (fds before the run: 6; after close(): 6, server open connections after close: 0; PHP memory after gc at the end: 3.86 MiB, start 3.81 MiB)

tenants done pools in registry evictions open fds PHP memory MiB server: open connections server: connections opened in total
20 20 0 66 4.23 20 20
40 40 0 126 4.65 40 40
60 60 0 186 5.06 60 60
80 80 0 246 5.49 80 80
100 100 0 306 5.89 100 100
120 120 0 366 6.31 120 120
140 140 0 426 6.73 140 140
160 160 0 486 7.14 160 160
180 180 0 546 7.55 180 180
200 200 0 606 7.96 200 200
case requests per run req/s: median (min-max) p50 ms p95 ms p99 ms max ms peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
200 pool keys in sequence, 5 requests each, registry limit 32 1,000 530 (528-542) 1.1 8.5 13.0 20.0 4.04 3,867 / 822 0.494 0 200 / 200 4
200 pool keys in sequence, 5 requests each, registry limit 1000 1,000 554 (503-1,141) spread 115% 1.0 9.0 22.4 43.3 4.28 4,115 / 4,302 0.480 0 200 / 200 4

Comparison with edamov/pushok (compare-pushok-w100)

php benchmarks/run.php --scenario=compare-pushok --count=5000 --window=100 --server-delay-ms=10 --repeat=7 --json --out=benchmarks/results/compare-pushok-w100.json

Same 5000 alert notifications (~256 byte body), same mock server delay (10 ms), window/concurrency 100, 1 connection. One mock server per variant. Round 0 is a discarded warm-up; the order of variants rotates and alternates direction every round. pushok exposes no per-request durations, so latency is not compared.

case requests per run req/s: median (min-max) peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
ours: raw CurlTransport (warm pool) 5,000 5,443 (3,630-6,275) spread 49% 0.06 0 / 6 0.052 0 0 / 1 100
ours: full Sender pipeline (warm pool) 5,000 2,830 (2,253-3,503) spread 44% 1.42 0 / 6 0.085 0 0 / 1 100
ours: full Sender pipeline (new pool per run) 5,000 2,974 (2,510-4,172) spread 56% 2.00 591 / 605 0.080 0 1 / 9 100
pushok (autoCloseConnections=false, warm) 5,000 5,308 (3,176-6,708) spread 67% 7.00 0 / 6 0.055 0 0 / 1 100
pushok (library default: connection closed after each push()) 5,000 5,295 (4,118-6,702) spread 49% 7.01 0 / 9 0.055 0 1 / 9 100

Discarded warm-up round (round 0), req/s: ours: raw CurlTransport (warm pool) = 3,392; ours: full Sender pipeline (warm pool) = 2,630; ours: full Sender pipeline (new pool per run) = 2,251; pushok (autoCloseConnections=false, warm) = 3,175; pushok (library default: connection closed after each push()) = 2,626

Order of variants per round: ours: raw CurlTransport > ours: full Sender pipeline > ours: full Sender pipeline > pushok > pushok / ours: raw CurlTransport > pushok > pushok > ours: full Sender pipeline > ours: full Sender pipeline / ours: full Sender pipeline > pushok > pushok > ours: raw CurlTransport > ours: full Sender pipeline … (rotates, direction alternates)

Body size seen by the server (mean bytes over the recorded requests): ours: raw CurlTransport (warm pool) = 256; ours: full Sender pipeline (warm pool) = 256; ours: full Sender pipeline (new pool per run) = 256; pushok (autoCloseConnections=false, warm) = 256; pushok (library default: connection closed after each push()) = 256

Comparison with edamov/pushok (compare-pushok-w20)

php benchmarks/run.php --scenario=compare-pushok --count=5000 --window=20 --server-delay-ms=10 --repeat=7 --json --out=benchmarks/results/compare-pushok-w20.json

Same 5000 alert notifications (~256 byte body), same mock server delay (10 ms), window/concurrency 20, 1 connection. One mock server per variant. Round 0 is a discarded warm-up; the order of variants rotates and alternates direction every round. pushok exposes no per-request durations, so latency is not compared.

case requests per run req/s: median (min-max) peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
ours: raw CurlTransport (warm pool) 5,000 1,434 (1,286-1,629) spread 24% 0.02 0 / 6 0.070 0 0 / 1 20
ours: full Sender pipeline (warm pool) 5,000 1,270 (1,207-1,485) spread 22% 1.10 0 / 6 0.103 0 0 / 1 20
ours: full Sender pipeline (new pool per run) 5,000 1,282 (1,204-1,394) 1.22 119 / 129 0.107 0 1 / 9 20
pushok (autoCloseConnections=false, warm) 5,000 1,496 (1,278-1,674) spread 27% 6.99 0 / 6 0.069 0 0 / 1 20
pushok (library default: connection closed after each push()) 5,000 1,507 (1,234-1,672) spread 29% 6.99 0 / 9 0.070 0 1 / 9 20

Discarded warm-up round (round 0), req/s: ours: raw CurlTransport (warm pool) = 1,605; ours: full Sender pipeline (warm pool) = 1,307; ours: full Sender pipeline (new pool per run) = 1,372; pushok (autoCloseConnections=false, warm) = 1,517; pushok (library default: connection closed after each push()) = 1,368

Order of variants per round: ours: raw CurlTransport > ours: full Sender pipeline > ours: full Sender pipeline > pushok > pushok / ours: raw CurlTransport > pushok > pushok > ours: full Sender pipeline > ours: full Sender pipeline / ours: full Sender pipeline > pushok > pushok > ours: raw CurlTransport > ours: full Sender pipeline … (rotates, direction alternates)

Body size seen by the server (mean bytes over the recorded requests): ours: raw CurlTransport (warm pool) = 256; ours: full Sender pipeline (warm pool) = 256; ours: full Sender pipeline (new pool per run) = 256; pushok (autoCloseConnections=false, warm) = 256; pushok (library default: connection closed after each push()) = 256

Comparison with edamov/pushok (compare-pushok-d0)

php benchmarks/run.php --scenario=compare-pushok --count=5000 --window=100 --server-delay-ms=0 --repeat=7 --json --out=benchmarks/results/compare-pushok-d0.json

Same 5000 alert notifications (~256 byte body), same mock server delay (0 ms), window/concurrency 100, 1 connection. One mock server per variant. Round 0 is a discarded warm-up; the order of variants rotates and alternates direction every round. pushok exposes no per-request durations, so latency is not compared.

case requests per run req/s: median (min-max) peak MiB over start growth KiB (raw / after gc) cpu ms/req transport errors connections opened (this run / since server reset) max concurrent streams (server)
ours: raw CurlTransport (warm pool) 5,000 7,118 (3,506-8,713) spread 73% 0.06 0 / 6 0.041 0 0 / 1 100
ours: full Sender pipeline (warm pool) 5,000 4,008 (3,541-5,868) spread 58% 1.42 0 / 6 0.073 0 0 / 1 100
ours: full Sender pipeline (new pool per run) 5,000 4,308 (4,074-7,743) spread 85% 2.00 591 / 605 0.072 0 1 / 9 100
pushok (autoCloseConnections=false, warm) 5,000 8,470 (3,475-10,600) spread 84% 7.01 0 / 6 0.043 0 0 / 1 100
pushok (library default: connection closed after each push()) 5,000 7,063 (2,698-9,091) spread 91% 7.01 0 / 9 0.043 0 1 / 9 100

Discarded warm-up round (round 0), req/s: ours: raw CurlTransport (warm pool) = 3,264; ours: full Sender pipeline (warm pool) = 2,145; ours: full Sender pipeline (new pool per run) = 1,974; pushok (autoCloseConnections=false, warm) = 2,726; pushok (library default: connection closed after each push()) = 2,422

Order of variants per round: ours: raw CurlTransport > ours: full Sender pipeline > ours: full Sender pipeline > pushok > pushok / ours: raw CurlTransport > pushok > pushok > ours: full Sender pipeline > ours: full Sender pipeline / ours: full Sender pipeline > pushok > pushok > ours: raw CurlTransport > ours: full Sender pipeline … (rotates, direction alternates)

Body size seen by the server (mean bytes over the recorded requests): ours: raw CurlTransport (warm pool) = 256; ours: full Sender pipeline (warm pool) = 256; ours: full Sender pipeline (new pool per run) = 256; pushok (autoCloseConnections=false, warm) = 256; pushok (library default: connection closed after each push()) = 256

How pushok was configured

  • version: 0.19.1 (34e97101ab)
  • constructor: new Client(Token::useExisting("benchmark.fake.token", [...]), true, $curlOptions)
  • CURLOPT_PORT: mock port (pushok sets 443; user options win: $this->curlOptions + $request->getOptions())
  • CURLOPT_RESOLVE: [“api.push.apple.com:PORT:127.0.0.1”]
  • CURLOPT_CAINFO: the mock’s CA bundle (peer verification stays on)
  • CURLOPT_SSL_VERIFYHOST: 0 because the mock certificate is for localhost/127.0.0.1, not api.push.apple.com. This library run therefore skips the hostname check; this package always checks it. The cost of that check is negligible next to the handshake itself, but it is a difference.
  • CURLOPT_PROXY: “” (the container sets HTTPS_PROXY, pushok does not disable it)
  • library settings: setNbConcurrentRequests(window), setMaxConcurrentConnections(connections), setAutoCloseConnections(false) for the warm variant (library default is true)
  • everything else: everything else is the library default (CURLOPT_TIMEOUT 10, HTTP/2, CURLMOPT_PIPELINING=MULTIPLEX, MAX_HOST_CONNECTIONS=connections; no CURLOPT_PIPEWAIT)
  • queueing: notifications are added with addNotification() in a loop and held in memory until push(); addNotifications() was not used because it does an in_array() scan per notification (quadratic)

Behaviour on the same mock-scripted situations (observed facts)

pushok 0.19.1 (34e97101ab); this package at commit ee540e6; libcurl 8.5.0. Window 20, one connection for both. “server” counts are the mock’s own (received / processed per device token).

processed-then-dropped, once, on a reused connection - Request 1 (200) opens the connection; request 2 is processed by the server, which then destroys the TCP connection without answering (rule drop, times 1; the server answers 200 to the same request if it arrives again).

  • this package: returned [{"token":"e0..002","status":"outcome_unknown","reason":"transport_unknown","http":null,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":false}]; server: {"d0..001":{"received":1,"processed":1},"e0..002":{"received":1,"processed":1}}
  • pushok: returned [{"status":200,"reason":"","apns_id":"00000000-0000-4000-8000-000000000001"}]; server: {"d0..001":{"received":1,"processed":1},"e0..002":{"received":2,"processed":2}}

processed-then-dropped, once, as the first request on a new connection - Same rule, but nothing was sent on the connection before.

  • this package: returned [{"token":"e0..003","status":"outcome_unknown","reason":"transport_unknown","http":null,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":false}]; server: {"e0..003":{"received":1,"processed":1}}
  • pushok: threw Exception: Send failure: Broken pipe; server: {"e0..003":{"received":1,"processed":1}}

a batch of 5 where one request is always processed-then-dropped - Five notifications in one call; the third is processed and its connection destroyed every time it is sent.

  • this package: returned [{"token":"d0..00a","status":"accepted","reason":"none","http":200,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":false},{"token":"d0..00b","status":"outcome_unknown","reason":"transport_unknown","http":null,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":false},{"token":"e0..00e","status":"outcome_unknown","reason":"transport_unknown","http":null,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":false},{"token":"d0..00c","status":"outcome_unknown","reason":"transport_unknown","http":null,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":false},{"token":"d0..00d","status":"outcome_unknown","reason":"transport_unknown","http":null,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":false}]; server: {"d0..00a":{"received":1,"processed":1},"d0..00b":{"received":1,"processed":1},"e0..00e":{"received":1,"processed":1},"d0..00c":{"received":1,"processed":1},"d0..00d":{"received":1,"processed":1}}
  • pushok: threw Exception: Send failure: Broken pipe; server: {"d0..00a":{"received":1,"processed":1},"d0..00b":{"received":1,"processed":1},"e0..00e":{"received":1,"processed":1},"d0..00c":{"received":1,"processed":1},"d0..00d":{"received":1,"processed":1}}

HTTP 503 ServiceUnavailable - The server answers 503 {“reason”:“ServiceUnavailable”} to every request.

  • this package: returned [{"token":"d0..014","status":"deferred","reason":"provider_unavailable","http":503,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":true}]; server: {"d0..014":{"received":1,"processed":1}}
  • pushok: returned [{"token":"d0..014","status":503,"reason":"ServiceUnavailable","apns_id":"","ts410":""}]; server: {"d0..014":{"received":1,"processed":1}}

HTTP 503 once, then 200 - The first request gets 503, any later one gets 200.

  • this package: returned [{"token":"d0..015","status":"deferred","reason":"provider_unavailable","http":503,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":true}]; server: {"d0..015":{"received":1,"processed":1}}
  • pushok: returned [{"token":"d0..015","status":503,"reason":"ServiceUnavailable","apns_id":"","ts410":""}]; server: {"d0..015":{"received":1,"processed":1}}

HTTP 410 Unregistered with timestamp - The server answers 410 {“reason”:“Unregistered”,“timestamp”:1763600000123}.

  • this package: returned [{"token":"d0..016","status":"rejected","reason":"destination_unregistered","http":410,"attempts":1,"destination_action":"invalidate","invalidated_at_ms":1763600000123,"next_attempt":false}]; server: {"d0..016":{"received":1,"processed":1}}
  • pushok: returned [{"token":"d0..016","status":410,"reason":"Unregistered","apns_id":"","ts410":"1763600000123"}]; server: {"d0..016":{"received":1,"processed":1}}

response cut in the middle of the body - The server sends 200 and the first half of a 2 KiB body, then destroys the connection.

  • this package: returned [{"token":"d0..017","status":"outcome_unknown","reason":"transport_unknown","http":null,"attempts":1,"destination_action":"none","invalidated_at_ms":null,"next_attempt":false}]; server: {"d0..017":{"received":1,"processed":1}}
  • pushok: returned [{"token":"d0..017","status":200,"reason":"","apns_id":"","ts410":""}]; server: {"d0..017":{"received":1,"processed":1}}

What these numbers do and do not show

They show, for one machine and one PHP/libcurl/Node combination, how much CPU and memory this package’s PHP code spends per request when the peer is a fast local HTTP/2 server with a configurable fixed delay, how throughput reacts to the window and connection settings, whether memory stays flat for large lazy batches, and how the pool registry bounds connections and file descriptors.

They do not show:

  • anything about Apple APNs or Google FCM: no real provider was contacted, no real credentials were used, and real providers add network latency, rate limiting, TLS session behaviour, per-connection stream limits and server-side throttling that the mock does not model;
  • throughput on a different machine, a different libcurl or a different PHP build; the Node mock server shares the CPU with the PHP process, so at zero delay the server may limit the measured rate as much as the client does;
  • the cost of persisting results, queues, Laravel service resolution, or real token signing beyond the first token (tokens are cached for the run);
  • latency under real network conditions: latency columns contain the mock’s artificial delay plus loopback and scheduling noise;
  • a verdict on any other library beyond the situations listed in the comparison (one library version, one configuration, written down above).

Observations (written by hand for the run above; not regenerated)

All of these are about this machine on the day of the run, which was shared and busy (load average about 19-21 on 4 cores for the whole session). Wall-clock figures swung by 20-90% between repeated runs of the same command, so conclusions below rely on CPU per request, memory, connection counts and recorded behaviour, which were stable, and on throughput only where ranges do not overlap.

Cost of the pipeline. The full Sender pipeline spends more CPU per request than the bare transport: about 0.066-0.069 ms/req (APNs, FCM, no delay) against 0.042 ms/req for the raw transport, and 0.073-0.085 against 0.054 with a 20 ms server delay. The extra is roughly 0.03 ms of PHP CPU per notification for compile, guards, bearer, classification, retry planning and result objects. With a 20 ms server delay the pipeline reached 2,499 req/s (range 2,297-2,723) for APNs and 3,003 (2,633-3,049) for FCM versus 3,005 (2,749-3,193) for the raw transport at window 100: the pipeline was slower in this run, and the APNs pipeline noticeably so; the tables flag where the spread exceeded 15%. At zero delay the Node mock and the PHP process compete for the same cores and the ranges overlap.

Mixed outcomes. 5% non-accepted outcomes (3% 410, 1% 400, 1% 500) cost about 0.073 ms/req against 0.069 for the all-accepted control (same run conditions); the difference is within the noise of the CPU measure. The per-run outcome counts (600 unregistered, 200 rejected, 200 deferred, 19,000 accepted per 20,000) match the script exactly, and the server received each request once: nothing was retried inline.

Warm versus cold. Over 30 runs of a single request, the median request took 0.7 ms (range 0.5-1.9) on a warm pool and 6.6 ms (3.2-11.1) when it had to open the connection first (TCP + TLS 1.3 + HTTP/2 against a local server); client CPU for that request was about 2 ms. Over 20,000 requests that setup is invisible in throughput. Real providers add network round trips to every handshake; this number is a local floor, not an estimate.

Window. With a 20 ms server delay and one connection, throughput followed window / delay at small windows (window 10: 413 req/s; 50: 1,938; 100: 3,993, against theoretical 500, 2,500, 5,000) and stopped scaling between 250 and 500 (8,436 at 250; 7,433 at 500, range 6,511-8,999) while p50 latency doubled from 28 ms to 59 ms: beyond about 250 in flight the client or the mock is the limit on this machine and extra requests only queue. Memory grows with the window (peak 0.3 MiB over start at 10-250, 2.0 MiB at 500).

Connections. When the peer allows 1000 concurrent streams, 1, 2 and 4 connections gave the same throughput within the noise (8,066 / 7,447 / 8,196 req/s). When the peer advertises 100 streams and the window is 500, throughput rose with the connection count (3,826 -> 6,488 -> 8,338 req/s) and p99 latency fell (475 ms -> 228 -> 131). The server counted exactly 1, 2 and 4 connections. This supports the transport’s rule “keep window <= peer limit x connections”, not any particular real peer limit.

Large batches. 100,000 and 1,000,000 destinations from a generator ran with PHP memory between 6.6 and 7.2 MiB for the whole run (range over the second half: 0.25 MiB in both runs; peak 2.5 MiB over the starting usage); the 1,000,000 run did not use more memory than the 100,000 run. 0 transport errors, 1 connection, all accepted, 5,774-6,735 req/s.

Many tenants. 200 pool keys in one process with the default registry limit (32): the registry stayed at 32 pools, the server never saw more than 32 open connections and the process never held more than 102 file descriptors (6 at the start and after close()); 168 pools were evicted. With limit 1000 the process held 606 descriptors and 200 server connections at the end. PHP memory looks like it grows with the number of tenants (4.2 to 7.7 MiB) but returns to the starting level after gc_collect_cycles() (3.81 MiB vs 3.78 MiB at start): closed pools leave cycles that wait for the collector. After the garbage collection the registry limited to 32 holds about 0.8 MiB more than at start, the unbounded one about 4.3 MiB. Each new connection cost about 2.4 ms of client CPU (0.49 ms/req over 5 requests).

Against pushok 0.19.1 (same notifications, mock, delay and concurrency; hostname verification off for pushok only, see above):

  • CPU per request: pushok 0.043 / 0.055 / 0.069-0.070 ms (zero delay / window 100 + 10 ms / window 20 + 10 ms); this package’s raw transport 0.041 / 0.052 / 0.070; this package’s full pipeline 0.072-0.073 / 0.080-0.085 / 0.103-0.107. The pipeline does work pushok does not do (compile, guards, classification, retry planning).
  • Throughput: ranges overlap heavily and spreads were 22-91%, so no ordering of pushok and this package’s raw transport can be claimed. At window 20 (tightest runs) the medians were 1,434 req/s (raw transport), 1,496 and 1,507 (pushok), 1,270 and 1,282 (full pipeline), ranges 1,204-1,674; the full pipeline was lower than the others in that table.
  • Memory: pushok holds every notification and response in memory until push() returns: peak 7.0 MiB over start for 5,000 notifications (0.06 MiB for the raw transport, 1.4 MiB for the pipeline with lazy input and a cursor report). That scales with N for pushok; this package stays flat (see large batches).
  • Connections: both used one connection with window 100 and with 20; pushok’s default (autoCloseConnections=true) opened one new connection per push() call, as designed.
  • Behaviour: when a request is processed and the connection is then destroyed, on a connection that had already carried a request, pushok returned HTTP 200 for it and the server had processed the push twice (libcurl re-sent the in-memory body on a new connection); this package reported outcome_unknown and the server received it once. As the first request on a new connection, pushok threw Exception: Send failure: Broken pipe (and, in a batch, no responses at all were returned for any of the five notifications, although the server processed all five); this package returned one result per destination: the first one accepted and four outcome_unknown (the in-flight siblings of the dropped connection, which the server did process). For a 503 both returned the status without retrying (pushok: status 503, reason ServiceUnavailable; this package: deferred/provider_unavailable with a next attempt planned); a 410 gave pushok status 410 with the timestamp as a string and this package rejected/destination_unregistered with invalidate and the timestamp in milliseconds. When the connection was cut in the middle of the response body, pushok returned status 200 and this package outcome_unknown.

Defaults. On this machine the measurements do not argue against window = 100 and maxConnections = 1: 100 in flight reaches about 80% of the window/delay limit at 20 ms, memory is small, and one connection is as fast as four as long as the window does not exceed the peer’s stream limit. They also show headroom up to a window of about 250 locally (twice the throughput at 20 ms), which is a local, mock-based result: the provider’s real stream limit, latency and throttling decide whether it is useful, and a larger window also puts more requests in Unknown when a connection is lost. No change of defaults is justified by these data alone.

Not measured. Real APNs/FCM latency, throughput, stream limits and throttling; behaviour under packet loss or slow links; FPM/Octane/queue-worker processes; Redis/database-backed guard or token stores; a second PHP/libcurl version; token signing cost per request (tokens are cached for the run); other libraries; what pushok does with a window above its own default of 20 beyond the settings shown (it was run at 20 and 100).


This page is generated from benchmarks/RESULTS.md. To change it, edit that file in the repository, not this site.