Laravel Job Batching for Large Background Tasks: A Practical Walkthrough

Laravel job batching is what I reach for when one background task is too big for one worker: a 200,000-row CSV import, a bulk message send to the whole user base, a nightly reindex. Dispatch it as a single queued job and it runs on one worker, takes an hour, and gives you no way to show progress or react when it half-fails. The fix is Bus::batch(): you slice the dataset into hundreds of small jobs, dispatch them as one named batch, and let your workers process them in parallel. The batch object then tracks how many jobs have finished, lets you hook into completion and failure, and lets you cancel mid-flight. This post walks through exactly how to wire that up.

When should I use Bus::batch instead of a single job or a chain?

A single dispatched job is fine for one discrete unit of work. A chain (Bus::chain) is for jobs that must run in strict sequence, one after another. Batching is the third shape, and it is the one people miss: many independent jobs that can run concurrently, but that you still want to treat as one logical unit. You care about the group, not the individual job.

The trigger is almost always one of these:

• A dataset too large to process in one job without hitting the queue worker timeout or exhausting memory — see my notes on PHP memory management in long-running scripts for why a 200k-row loop in one job is a bad idea.
• Work that benefits from parallelism: 400 chunks across 8 workers finish roughly 8x faster than one worker grinding through everything serially.
• A task where you need to show a progress bar to the user, or fire a notification, generate a report, or flip a flag only once every piece is done.
• Bulk operations where some items will legitimately fail (a bad email, a missing record) and you do not want one failure to kill the other 199,999.

How do I make a job batchable and chunk a huge dataset?

Every job that goes into a batch must use the Illuminate\Bus\Batchable trait. That trait gives the job a $this->batch() accessor so it can check whether the batch has been cancelled before doing expensive work. Here is an import job that processes one chunk of IDs. Note the Customer model import — the job runs in its own namespace, so the model has to be imported explicitly.

<?php

namespace App\Jobs;

use App\Models\Customer;
use Illuminate\Bus\Batchable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;

class ImportCustomerChunk implements ShouldQueue
{
    use Batchable, Queueable;

    /** @param  array<int>  $ids */
    public function __construct(public array $ids) {}

    public function handle(): void
    {
        // If someone cancelled the batch, stop quietly.
        if ($this->batch()?->cancelled()) {
            return;
        }

        foreach (Customer::whereIn('id', $this->ids)->cursor() as $customer) {
            // ... do the real per-row work
        }
    }
}

Now the dispatch side. You do not load full models — you pull just the ID list and split it into many small jobs, one per chunk. A list of 200k integers is cheap; the heavy per-row work stays inside each job, processed by cursor(). Bus::batch() takes the array of jobs and exposes a fluent set of lifecycle callbacks before you dispatch.

use App\Jobs\ImportCustomerChunk;
use Illuminate\Bus\Batch;
use Illuminate\Support\Facades\Bus;
use Throwable;

$jobs = Customer::query()
    ->pendingImport()
    ->pluck('id')                      // collection of ids, not models
    ->chunk(500)                       // 500 ids per job
    ->map(fn ($ids) => new ImportCustomerChunk($ids->all()))
    ->all();

$batch = Bus::batch($jobs)
    ->name('Nightly customer import')
    ->then(function (Batch $batch) {
        // All jobs completed successfully.
    })
    ->catch(function (Batch $batch, Throwable $e) {
        // First batch job failure detected.
    })
    ->finally(function (Batch $batch) {
        // Batch has finished executing — runs on success OR failure.
    })
    ->onQueue('imports')
    ->dispatch();

return $batch->id;   // persist this so you can poll progress later

Order matters with the callbacks: then() runs only if every job succeeds, catch() runs on the first failed job, and finally() always runs once the batch stops executing. A critical gotcha — these closures are serialized and run inside a worker, so do not reference $this or anything that cannot be serialized. Capture scalar values (like an ID) by value instead.

How do I report live progress of a running batch?

This is where batching earns its keep. Persist the batch ID from dispatch, then look the batch up at any time with Bus::findBatch($id). The returned object carries everything you need to render a progress bar without touching the database yourself.

use Illuminate\Support\Facades\Bus;

$batch = Bus::findBatch($batchId);

if (is_null($batch)) {
    abort(404);
}

return response()->json([
    'total'     => $batch->totalJobs,
    'pending'   => $batch->pendingJobs,    // not yet processed
    'processed' => $batch->processedJobs(),// total - pending
    'failed'    => $batch->failedJobs,
    'progress'  => $batch->progress(),     // integer 0-100
    'finished'  => $batch->finished(),
    'cancelled' => $batch->cancelled(),
]);

Point a polling fetch or a small Livewire/Vue component at that endpoint and you have a real progress bar backed by the queue itself. progress() is an integer percentage; processedJobs() is totalJobs minus pendingJobs. Note processedJobs() is a method while totalJobs, pendingJobs, and failedJobs are properties — that asymmetry trips people up and throws a confusing error if you call a property as a method.

If a background task can finish without anyone knowing whether it worked, you do not have a feature — you have a liability with a progress bar bolted on as an afterthought.Md Raihan Hasan

How does failure handling actually work in a batch?

By default a batch is fail-fast: the moment one job throws, the batch is marked as cancelled and no further jobs are processed. For a strict all-or-nothing operation that is correct. But for a bulk send or import where individual records will sometimes fail, that default is wrong — one bad row should not abort the other 199,999. Call allowFailures() and the batch keeps processing the rest while still recording the failures.

$batch = Bus::batch($jobs)
    ->name('Bulk newsletter send')
    ->allowFailures()     // failed jobs do NOT cancel the batch
    ->finally(function (Batch $batch) {
        if ($batch->hasFailures()) {
            // some jobs failed — log, alert, queue a retry
        }
    })
    ->dispatch();

A subtle but important point: with allowFailures(), then() will not fire if any job failed — only finally() runs, and you inspect $batch->hasFailures() and $batch->failedJobs there. Failed batch jobs land in the standard failed_jobs table and can be retried by batch ID with php artisan queue:retry-batch {batchId}. If you are sending bulk email this way, get your sending infrastructure right first — my email deliverability guide for developers covers why a batch of 50k sends needs warmup and throttling.

Adding jobs to a running batch and cancelling

You can grow a batch after it starts — useful when one initial job discovers more work to dispatch. From inside a batched job, call $this->batch()->add([...]). This is the standard pattern for a coordinator job that fans out: a single first job counts the work, then adds the real processing jobs to its own batch. To stop everything, call $batch->cancel(); your jobs see this via the $this->batch()?->cancelled() guard shown earlier and bail out instead of doing more work.

// Inside a Batchable job's handle() method:
if ($this->batch()->cancelled()) {
    return;
}

$this->batch()->add([
    new ImportCustomerChunk($nextChunk),
]);

What are the operational gotchas — the table, pruning, and workers?

Batching is backed by a real database table, job_batches, that Laravel uses to track every batch's counts and serialized callbacks. You need the migration in place before any of this works.

php artisan make:queue-batches-table
php artisan migrate

Two operational realities bite people once they ship this:

• The job_batches table grows forever unless you prune it. Schedule php artisan queue:prune-batches in your scheduler — by default it removes finished batches older than 24 hours, and add --unfinished=72 to also clear stale batches that never completed (a worker died mid-batch). Without this the table quietly bloats for months.
• Batching gives you parallelism only if you have workers to run jobs in parallel. One worker processes one job at a time, so a batch of 400 jobs on a single worker is just a slow serial loop with extra bookkeeping. Run multiple worker processes — and run them under Supervisor so they survive reboots and deploys, which I walk through in Laravel queue workers on production with Supervisor.

use Illuminate\Support\Facades\Schedule;

Schedule::command('queue:prune-batches --hours=48 --unfinished=72')
    ->daily();

There is one more trap worth naming: the batch callbacks (then/catch/finally) are themselves queued and run on a worker, not synchronously at dispatch time. If your workers are stopped, the batch never finishes and finally() never fires — your progress bar sits at 99% forever. When a batch looks stuck, check that workers are actually running before you suspect your code.

Get this pattern into your toolbox and large background work stops being a gamble. Chunk the dataset so no single job is doing too much, dispatch it as a named batch with then/catch/finally wired up, persist the batch ID so you can poll progress(), decide deliberately between fail-fast and allowFailures(), and remember the whole thing is only as parallel as the workers you run and only as clean as your pruning schedule. The difference between a batch that quietly rots in job_batches and one you can watch, cancel, and trust is about ten lines of operational discipline.