Queues, workers, Horizon and the scheduler

queue() puts work on a Laravel queue; this page says on which queue, with what timeouts, which jobs exist, what to run in production (a Horizon block and its plain queue:work equivalent, both printed by mobile-push:install), which entries the package adds to the scheduler, and what has and has not been verified per queue driver.

Related pages: delivery profiles (what the jobs do), retries and limits, commands, troubleshooting.

Two different “connections”

A push connection is an APNs or FCM app binding (mobile-push.connections.*). A queue connection is a Laravel queue backend (config/queue.php). They are unrelated names. On PendingSend, the queue connection is chosen with onQueueConnection(), never connection().

use Padosoft\MobilePush\Core\Message\PushMessage;
use Padosoft\MobilePush\Laravel\Facades\MobilePush;

// Which push connection: ->connection('ios'). Which queue: onQueueConnection() / onQueue().
$effectId = MobilePush::to($user)
    ->bulk()                          // traffic class: uses the bulk queue by default
    ->onQueueConnection('redis')      // the Laravel queue connection
    ->onQueue('push-bulk')            // the queue name (overrides the default for the class)
    ->delay(30)                       // seconds, or a DateTimeInterface
    ->afterCommit()                   // wait for the surrounding database transaction to commit (default)
    ->queue(PushMessage::make('Weekly digest', 'Three new things for you'));

queue() returns the effect id (several ids joined by , when the targets span several push connections).

Queue names

Setting Env Default Used for
queue.connection MOBILE_PUSH_QUEUE_CONNECTION null (the app’s default queue connection) The queue backend for every package job.
queue.queue MOBILE_PUSH_QUEUE push Transactional traffic (one-to-one messages), the sweeper, relay follow-ups.
queue.bulk_queue MOBILE_PUSH_BULK_QUEUE push-bulk Bulk traffic (->bulk(), campaigns).

A job goes to the queue named by onQueue() when given, otherwise to the queue of its traffic class. Keep the two queues on separate worker pools so a large campaign cannot delay a login code. Follow-up jobs created inside a worker (continuations, relay follow-ups) inherit the queue of their traffic class.

Relay follow-ups are folded. On the durable profile, when many effects of one scope are deferred at once (a provider throttle answered to thousands of small effects), at most 20 precise follow-up relay jobs are queued per scope, queue and queue.follow_up_bucket (seconds, default 60, aligned to the clock); the rest share one scope-wide relay at the end of the bucket. A delay is never shorter than the provider asked for and at most one bucket longer. queue.follow_up_bucket = 0 disables the folding. Verified with a fake transport and the queue fake (10,000 deferred effects made 21 relay jobs), not on Redis, SQS or Horizon.

The jobs

Job Profile What it does Retried by the queue?
SendPushJob light Sends one small chunk of destinations (queue.chunk 500, queue.chunk_bytes 98304), then dispatches a new job for the Deferred ones. Stops starting new windows at the chunk deadline and dispatches one new job, at once, for the destinations it never attempted. Encrypted payload (ShouldBeEncrypted). If its connection no longer matches the binding it was queued for, it sends nothing, dispatches BindingMismatchDetected once, records Skipped / binding_mismatch for each destination (to the log sinks when enabled) and ends cleanly, like the durable relay. No: tries = 1, never release()d
RelayDeliveriesJob durable Claims due deliveries of one effect (or of a whole scope), sends them, records outcomes before it finishes. Carries only ids. Yes: tries = 3, backoff = [10, 60] s. Safe: a rerun claims again and never re-sends a delivery with a recorded or unknown outcome.
SweepLedgerJob durable Applies lease expiry, expires overdue deliveries, dispatches a scope-wide relay when work is due. Idempotent. tries = 1 (the next minute’s run covers it)
PlanCampaignJob, SweepCampaignsJob campaigns Write one slice of a campaign’s audience to the ledger; detect completion and stalled planners. See campaigns
SendLiveActivityTransitionJob Live Activities Sends one Live Activity transition. Never retried by the queue; a lost job is replayed by the module. No

Laravel’s own SendQueuedNotifications job is the only owner of queueing when a notification implements ShouldQueue: the push channel then sends inside that job and does not queue a second one. See notifications and fallback.

All package jobs set onConnection / onQueue / delay at dispatch. The light job is marked after commit (held until the surrounding transaction commits, dropped on rollback) when both the afterCommit() call option and queue.after_commit (MOBILE_PUSH_QUEUE_AFTER_COMMIT, default true) are on. The durable relay is dispatched after the commit of the ledger connection’s transaction. A job dispatched from inside a worker (a continuation, a follow-up, a sweep) never waits for a commit.

Tags

Horizon tags (and displayName) never contain a token, a user id or a device id:

Job Tags
SendPushJob mobile-push, `provider:<apns
RelayDeliveriesJob mobile-push, relay, scope:<scope>, effect:<id> (when for one effect)
SweepLedgerJob mobile-push, sweep
PlanCampaignJob mobile-push, campaign, scope:<scope>, campaign:<id>
SendLiveActivityTransitionJob mobile-push, live-activity, <target>:<id> (the activity or channel row id)

Horizon only stores tags when it is installed, so look for them in the Horizon dashboard.

The timeout chain

Retries are the package’s business, so the queue must never redeliver a job that is still working. That requires a strict order of timeouts, each with a margin:

transport timeout  <  chunk deadline  <  job timeout  <  worker timeout  <  retry_after (SQS: visibility timeout)
and, in the durable profile:  chunk deadline  <  ledger lease
Link Where it is set Default
transport timeout transport.timeout_ms (or the largest connections.*.timeout_ms) 30000 ms (30 s)
chunk deadline queue.chunk_deadline (MOBILE_PUSH_QUEUE_CHUNK_DEADLINE): wall time of one light job or one relay run before it stops starting new work and hands over (see below) 40 s
job timeout queue.timeout (MOBILE_PUSH_QUEUE_TIMEOUT): the $timeout of every package job 50 s
worker timeout Horizon supervisor timeout, or queue:work --timeout; declare it in queue.worker_timeout when it cannot be read not set
retry_after queue.connections.<connection>.retry_after; for SQS the queue’s visibility timeout, declared in queue.visibility_timeout the connection’s
ledger lease ledger.lease_seconds 120 s

The margin of each link is max(5 s, 10 % of the lower value). The defaults are chosen to fit a stock Laravel application with no change, whose database and redis queue connections use retry_after 90: 30 s (transport) < 40 s (chunk deadline) < 50 s (job timeout) < 55 s (worker timeout, the value mobile-push:install prints) < 90 s (retry_after), and, in the durable profile, 40 s < 120 s (ledger lease). The worker timeout must be at least 50 + 5 = 55 s and retry_after at least 55 + 5.5, rounded up to 61 s: that is what mobile-push:install prints. A connection that sets no retry_after falls back to the framework’s 60, which still fits. The chain is checked as a whole: if you raise one value (a slower transport, a bigger chunk), raise the ones above it too, and mobile-push:doctor tells you which link breaks.

The chunk deadline

Both profiles enforce queue.chunk_deadline, the same single setting, measured the same way: from the package clock (Psr\Clock\ClockInterface), starting when the job begins to send. Neither ever interrupts a request that is on the wire, and the deadline itself never waits or sleeps.

  • Light profile (SendPushJob). The Sender takes destinations from the job one window at a time (the connection’s window, window 100 by default). Before each new window the job checks the clock: when the deadline has passed, or the time left is shorter than the slowest window this job has had so far, it gives the Sender no further destination. The window in flight is finished as usual: its destinations get the outcomes the transport reports, OutcomeUnknown included, and are never handed over or sent again. Every destination the Sender never received (never compiled, never written) goes, with its retry budget unchanged, to one new SendPushJob dispatched at once (no delay) on the same queue connection and queue, with the next continuation number and a fresh deadline. The binding reference, effect key, traffic class and pinned expiry are carried unchanged, exactly as for a continuation of Deferred destinations, so a connection re-pointed to another app in between is still Skipped / binding_mismatch and a credential rotation on the same app is not. A job always sends at least its first window, so a chain of handovers is never longer than the number of windows in the chunk. The handover writes one info line to the application log (counts only, no destination).
  • Durable profile (RelayDeliveriesJob). The relay stops claiming new chunks on the same rule (deadline passed, or less time left than its slowest chunk) and dispatches a fresh relay for the same effect (or scope) at once; unclaimed deliveries simply stay due in the ledger.

What the deadline does not guarantee: a window (light) or a chunk (relay) that starts before the deadline runs to its end, so a job can run past queue.chunk_deadline by up to one window. The slowest-window rule keeps that overrun short when windows take a similar time; a window much slower than every earlier one in the same job is not predicted. One window is bounded by the transport timeout per pass (plus in-process retries within the retry policy’s inline threshold). Keep queue.chunk_deadline plus one window under the job timeout: with the defaults (40 s and a 30 s transport timeout) a window that hits the transport timeout right before the deadline could still reach the 50 s job timeout. doctor checks the order of the values and reports that sum as an information line (“Worst case: …”) when it can reach the job timeout, with the job timeout that would rule it out (above 70 s with the defaults; the worker timeout and retry_after then have to move after it).

Verified with scenario tests on a fake transport and a fake clock (tests/Scenario/Laravel/Queue/LightChunkDeadlineTest.php): a chunk cut at its deadline is completed by exactly one follow-up with every destination requested once; a chunk that fits gets no follow-up; a window in flight when the deadline passes (one response lost) is not re-sent; a provider slower than the whole deadline produces one job per window; a continuation’s budgets survive the handover; with the default chain and a provider answering in 28 s per window every light job returned after 28 s instead of running for 84 s. Not measured on a real worker under a real slow provider.

How doctor reports it

php artisan mobile-push:doctor prints one line per link in the section “Queue timeouts” (a real run of the defaults, durable profile, with a database queue connection whose retry_after is 90 as in a fresh Laravel application):

Queue timeouts
  ✓ transport_timeout (30 s) < chunk_deadline (40 s), margin 5 s
  ✓ chunk_deadline (40 s) < job_timeout (50 s), margin 5 s
  ! not verifiable: job_timeout (50 s) < worker_timeout (?), margin 5 s: not verifiable
      Remedy: worker_timeout cannot be read here: check it by hand or declare it (the Horizon supervisor "timeout" or `queue:work --timeout` (declare it in mobile-push.queue.worker_timeout)).
  ! not verifiable: worker_timeout (?) < retry_after (90 s), margin 5 s: not verifiable
      Remedy: worker_timeout cannot be read here: check it by hand or declare it (the Horizon supervisor "timeout" or `queue:work --timeout` (declare it in mobile-push.queue.worker_timeout)).
  ✓ job_timeout (50 s) < retry_after (90 s), margin 5 s
  ✓ chunk_deadline (40 s) < lease_seconds (120 s), margin 5 s
  i Worst case: a window that starts right before the chunk deadline (40 s) may run for the transport timeout (30 s), past the job timeout (50 s). The worker then stops the job and the destinations of that window end with an unknown outcome (they are never sent twice). To rule it out, set the job timeout above 70 s and move the worker timeout and retry_after after it.

There is no violation. The last line is information, not a failure (see the chunk deadline). The two “not verifiable” lines are about the one value the application cannot read, the queue:work --timeout flag (see below); the line that follows them proves the rule that matters most, job timeout below retry_after. With a higher queue.timeout (for example 120) and the same retry_after of 90 the last two lines turn into a violation with a remedy such as Raise retry_after to at least 132 s, or lower job_timeout to at most 81.818 s.

Each link is one of:

Status Meaning doctor exit code
ok The upper value exceeds the lower by at least the margin. 0
violation It does not. The remedy names the key to raise and the largest value the lower one may have. 2
not verifiable One side cannot be read from this process. It is never reported as ok. 1 (a warning)
not applicable The sync driver: jobs run in-process with no worker and no redelivery. 0

What can and cannot be read:

  • Horizon: the worker timeout is read from horizon.defaults and horizon.environments.<APP_ENV> for every supervisor that uses the same queue connection and lists push or push-bulk; a supervisor without timeout counts as 60 s. The smallest value is checked against the job timeout and the largest against retry_after.
  • queue:work --timeout: it is a command-line flag of a process the application cannot see. It stays “not verifiable” until you declare the value in queue.worker_timeout (MOBILE_PUSH_QUEUE_WORKER_TIMEOUT). Declaring is trusting yourself: the doctor then verifies the arithmetic, not the process.
  • SQS: the visibility timeout is an attribute of the queue, not part of Laravel’s config. Declare it in queue.visibility_timeout (MOBILE_PUSH_QUEUE_VISIBILITY_TIMEOUT).
  • redis, database, beanstalkd: retry_after is read from the connection (the framework default of 60 when the connection sets none).
  • Any other driver: retry_after is “not verifiable” (“driver … cannot be inspected”).
  • mobile-push:health has its own simpler queue check: it compares retry_after with the job timeout only.

What to run

php artisan mobile-push:install prints the block below with the values of your settings (timeout = ceil(queue.timeout + margin), here 55; memory 256 MB). Take the text from the command, not from this page, when your timeouts differ.

Horizon

// config/horizon.php, inside 'environments' => ['production' => [ ... ]]
'push' => [
    'connection' => 'redis',
    'queue' => ['push'],
    'balance' => false,          // fixed, always-on processes for one-to-one pushes
    'processes' => 3,
    'tries' => 1,                // the package owns retries; the job never re-runs blindly
    'timeout' => 55,
    'memory' => 256,
],
'push-bulk' => [
    'connection' => 'redis',
    'queue' => ['push-bulk'],
    'balance' => 'auto',         // scales with the backlog of campaign / bulk traffic
    'minProcesses' => 1,
    'maxProcesses' => 8,
    'balanceMaxShift' => 1,
    'balanceCooldown' => 3,
    'tries' => 1,
    'timeout' => 55,
    'memory' => 256,
],

The printed connection is redis; change it if your queue connection has another name. tries => 1 is right for SendPushJob; RelayDeliveriesJob declares its own tries = 3 (a job-level value wins over the worker option) and is safe to rerun. Remember retry_after of that Redis connection must be at least 61 s.

Plain queue:work

Without Horizon, two workers under Supervisor or systemd (the connection is queue.connection, else your default queue connection, else redis):

php artisan queue:work redis --queue=push --tries=1 --timeout=55 --memory=256
php artisan queue:work redis --queue=push-bulk --tries=1 --timeout=55 --memory=256

--memory=256 makes the worker exit when it grows past 256 MB so the process manager restarts it. (A test that runs a worker inside a large test process must raise it, for example --memory=2048: a worker that hits the limit stops after its first job.)

The scheduler

The durable profile needs the scheduler: add the standard Laravel cron entry (* * * * * php artisan schedule:run) or run schedule:work.

What the package registers itself (each only when its condition holds). The full list, with the daily maintenance, is in commands; in short:

Name What Registered when Frequency
mobile-push:sweep-ledger Dispatches SweepLedgerJob on the push queue delivery.profile = durable and queue.sweeper.schedule (default true) every minute
mobile-push:sweep-campaigns Dispatches SweepCampaignsJob on the bulk queue campaigns enabled, durable profile, campaigns.sweeper.schedule every minute
mobile-push:alerts Evaluates the operational alerts (withoutOverlapping(): needs a cache lock) alerts.enabled every minute
mobile-push:live-activities-flush Sends the Live Activity updates that are due (withoutOverlapping()) live_activities.enabled every minute
mobile-push:maintenance:<command> The prune commands of the enabled modules and credentials:check maintenance.schedule (default true) daily at maintenance.daily_at (default 03:17)

The daily maintenance runs deliveries:prune (durable profile), devices:prune (devices module), logs:prune (database log sink), live-activity:prune (Live Activities), campaign:prune (campaigns, durable profile) and credentials:check (always; it sends expiry alerts only when alerts.enabled). Each runs withoutOverlapping(), and onOneServer() too when the default cache store is shared between servers. Set maintenance.schedule to false to schedule them yourself, and run php artisan schedule:list to see what is registered on a host.

What the package does not schedule: mobile-push:channels:sync --cleanup --force (broadcast channels: delete the unused ones), which you may add yourself:

// routes/console.php (Laravel 11+) or app/Console/Kernel.php
use Illuminate\Support\Facades\Schedule;

Schedule::command('mobile-push:channels:sync --cleanup --force')->dailyAt('03:30');

Each prune command is safe to run on a profile or module that is off: it prints a notice and exits 0. credentials:check needs a connections source (ManagerCredentialsSource is bound by the package); it never contacts a provider. Command details: commands.

The cache store

The queue and the guards share state through one Laravel cache store: provider tokens (encrypted), circuit breakers, rate-limit buckets, throttle gates, the relay follow-up markers, metrics counters, the fallback idempotency marker.

Setting Env Default
cache.store MOBILE_PUSH_CACHE_STORE null (the application’s default store)
cache.prefix MOBILE_PUSH_CACHE_PREFIX mobile-push

Use a lock-capable shared store: redis, memcached, database or dynamodb. The file store is atomic on one host only; array lives inside one process (fine for tests, wrong for workers). Breakers and the rate limiter update their state with compare-and-set under cache locks; with a store that cannot take locks the update fails and a send can throw Guard state is too contended to update before the network. doctor checks reachability, read-after-write and lock support. An unreachable cache behaves as an empty one (tokens are re-fetched, state is lost) and never throws from the token and guard stores themselves.

What is verified, per queue driver

Driver Status
sync Run in tests: jobs execute in-process; timeout links are reported “not applicable”.
database Run in the scenario tests with a real queue:work loop (light chunks, continuations, the relay and the sweeper).
redis Run in CI (runtimes job, Redis 7 with phpredis, as queue and as cache store) and once locally: durable pushes were queued, relayed by queue:work redis and by Horizon, and the ledger ended consistent. Throughput and behaviour under load were not measured.
sqs Run in CI against an emulator (backends workflow, ElasticMQ, aws/aws-sdk-php 3): durable pushes were queued, relayed by queue:work sqs and the ledger ended consistent, with memcached and then DynamoDB Local as cache store; doctor compared the job timeout with the declared visibility timeout. Never run against AWS: delivery delays, FIFO queues, IAM and redelivery after a real visibility timeout were not observed. The visibility timeout is only compared when you declare it.
beanstalkd Run in CI (backends workflow, beanstalkd 1.13 with pda/pheanstalk 8): durable pushes and a 250-device campaign were queued and relayed by queue:work beanstalkd, and doctor read retry_after (90 s) from the connection. Note that beanstalkd forgets an empty tube, and Laravel’s Queue::size() then throws TubeNotFoundException; the package itself never asks a queue for its size. Load was not measured.
Horizon Run in CI with the block printed by install as its supervisors: jobs processed, tags present, no token in the payloads, and doctor validates the timeout chain from the supervisor config. Auto-balancing under a real backlog was not observed.
Octane RoadRunner: run in CI (sixty requests through two workers, one push each, logs flushed per request; see runtimes). Swoole and FrankenPHP: run in CI the same way (backends workflow), and on OpenSwoole, plus requests through the real transport to a local HTTP/2 mock server on each of them (sequential, then eight at a time), which the server counted on one connection per worker. Swoole’s curl coroutine hooks are not supported for sending from HTTP workers: a shared pool crashes the workers; with transport.reuse = auto the transport holds (one connection per send) but some concurrent requests still fail in Octane’s shared application; doctor warns about that configuration (see runtimes).

Redis, Horizon and SQS (through an emulator) are exercised by CI jobs with simulated sends and a handful of jobs. Treat all three as not yet proven under your load: run mobile-push:doctor and a canary push in a staging environment on your real backend before you rely on it, and check troubleshooting for the symptoms of a bad chain.


This page is generated from docs/queues-and-horizon.md. To change it, edit that file in the repository, not this site.