Cron Jobs on Multiple Instances: Why Your Scheduled Task Runs Twice
A scheduler inside your app is a timer in memory, and every replica has its own. What that does under autoscaling, rolling deploys and sleeping instances, and where the schedule should live instead.
The weekly report went out three times. Same recipients, same numbers, three emails a few milliseconds apart. Nobody had touched the schedule. What had changed, a week earlier, was the instance count on the web service: from one to three.
It is an old problem that keeps arriving in new frameworks. For the Node.js version, Google still shows a Stack Overflow question from thirteen years ago titled "Nodejs cron plugin vs running nodejs script from crontab", with a single answer. The same problem shows up today as the related search "Nestjs/schedule multiple instances", and as a Rails thread that opens with "A doubt that woke me up last night".
This article is about where a schedule lives, not which primitive to use. If you are still choosing between a cron job, a background worker and a queue, the difference between the three and how each one fails is covered separately. Here the work is already a scheduled task, the application already runs on more than one instance or on one that sometimes sleeps, and the task is running the wrong number of times.
Cron running on multiple server instance
The heading is a Stack Overflow question, left exactly as it was typed, and as of writing it is the second result Google returns for this problem. The mechanism behind it fits in one sentence: a scheduler that runs inside your application is a timer held in that process's memory, and every copy of the process holds its own.
Libraries such as node-cron, @nestjs/schedule and APScheduler do not coordinate across processes by default. Each one reads the schedule from your code at startup, works out the next fire time and waits. Start three containers from the same image and you have three timers aimed at the same minute. None of them is misbehaving. Each one does what the single copy on your laptop did.
An answer on r/node, which Google quotes in its snippet for the Node comparison, states the other half plainly: in-process scheduling means "scheduled jobs will only fire as long as your script is running". The number of executions follows the number of running processes in both directions. More processes, more runs. No process, no run.
| Stack | Where the scheduler runs | With three instances, by default | What the project documents |
|---|---|---|---|
Node.js: node-cron | Inside the app process | Three runs | Version 4 adds distributed: true, so one instance executes each fire |
NestJS: @nestjs/schedule | Inside the app process | Three runs | Nothing on multiple instances; intervals use setInterval() under the hood |
| Python: APScheduler 3.x | Inside the app or a worker process | Three runs, or missed runs if they share a job store | Sharing a job store between processes: "You can't." |
| Python: Celery beat | A separate beat process, or inside a worker with -B | One run with a single beat; one per worker started with -B | Only a single scheduler per schedule; -B not recommended for production |
| PHP: Laravel scheduler | schedule:run on every server that has the cron entry | Three runs, unless the task uses onOneServer() | "generate the report three times. Not good!" |
| Ruby: Solid Queue | A scheduler process | One enqueue per task and time, even with several schedulers | A unique index on task_key and run_at |
The Celery documentation, verbatim
"You have to ensure only a single scheduler is running for a schedule at a time, otherwise you'd end up with duplicate tasks." The same page calls running beat inside a worker with -B convenient "if you'll never run more than one worker node", and not recommended for production. Scale that worker service to two and the embedded schedulers become two.
Laravel's documentation makes the point with a worked example: a report scheduled for Friday evening, a scheduler running on three worker servers, and the result, in its own words, "the scheduled task will run on all three servers and generate the report three times. Not good!"
Two different kinds of "running twice"
This article is about one scheduled moment firing on several machines at once. A different problem produces the same symptom: a run that is still going when the next scheduled run starts, on the same machine. That one is handled by an overlap policy (skip the new run, allow both, or replace the old one) and has an article of its own coming. The two need separate fixes. An overlap setting decides what happens when a run meets the previous run; it says nothing about how many machines started one.
Three places a schedule can live
Strip the frameworks away and a schedule can sit in one of three places. They part ways on the events that cause trouble: more instances, no instances, and a deploy.
| Where the schedule lives | Runs per fire with N instances | Instance asleep or scaled to zero | During a deploy | What stays with you |
|---|---|---|---|---|
Inside the web process (node-cron, @nestjs/schedule, APScheduler in the app) | N | Nothing fires | Old and new processes can both fire | Deduplication, every time the instance count changes |
In one dedicated always-on process (a single Celery beat, a standalone scheduler) | One, while there is exactly one | Fires, if that process is kept running | Briefly two, depending on how the process is replaced | Keeping the count at exactly one, and noticing when it is zero |
Outside every app process, as a job started on schedule (a crontab on one host, a Kubernetes CronJob, a managed cron job) | One per schedule, whatever the app's replica count | Fires; it starts its own container | Unaffected by the app's rollout | Idempotency, because "once" is still approximate |
The middle row is what most people mean by an always-on worker, or what people shopping for hosting describe as "persistent background processes". It fixes the multiplication by moving the schedule out of the part that scales. It does not remove the need for a count of exactly one; it concentrates that need in a single place you can watch. Web services that autoscale from one instance to many are the right home for request-handling code and the wrong home for the timer.
If you searched for BackgroundWorker
"Background worker" is a crowded phrase. In Google's German results, as of writing, much of the search around it concerns the .NET BackgroundWorker class, which moves work off a desktop UI thread, or browser Service Workers and background sync. Neither is a server process, and neither has a schedule that multiplies across instances. The always-on worker in this article is a long-running server process deployed next to your application.
The replica count changes when you are not looking
If the instance count were a number you set once, the fix would be to set it to one and move on. In practice it changes for reasons that have nothing to do with the schedule.
Autoscaling
Horizontal autoscaling adds instances when CPU or memory crosses a threshold and removes them when load falls. A schedule inside that service now fires a different number of times depending on traffic at the scheduled minute. The job at 03:00 runs once, because the service has scaled down to one. The job at 13:00 runs four times, because lunchtime traffic scaled it to four. Both pass in testing, and the bug report reads like an intermittent fault that nobody can reproduce.
A rolling deploy runs two of everything
A rolling release starts the new containers before it stops the old ones, so that traffic always has somewhere to go. For a stretch of seconds during a push-to-deploy release, both versions of your application exist, and each has its own scheduler loaded. If the rollout straddles a scheduled minute, a service running a single replica fires twice. This is the version that catches teams who already know about multiple instances and deliberately run one.
The idle environment in blue-green is not idle
Blue-green keeps a complete second copy of the application beside the live one, ready to take traffic. The router decides which copy receives requests. Nothing decides which copy runs its timers, because a scheduler does not wait for traffic. How blue-green and rolling deployments differ in what runs side by side is usually discussed in terms of requests and schema changes; timers belong on the same list. Unless the idle environment starts with its scheduler switched off, both colours send the Friday report.
Set Interval in Node.js vs. Cron Job?
This Stack Overflow question dates from 2013 and still sits in Google's discussions block for the Node comparison. The failure it leads to runs the other way from everything above. Too few runs, often zero.
// Fires 24 hours after this process started, not at a time of day.
// Any restart or deploy before then starts the countdown again.
setInterval(sendDailyDigest, 24 * 60 * 60 * 1000);That line does not mean "once a day". It means "24 hours after this process started, and every 24 hours after that, for as long as the process lives". Every restart resets the countdown, so an application deployed more often than once a day never reaches the first tick and the digest silently never goes out. Cron-expression schedulers such as node-cron avoid the reset, because they compute the next wall-clock time instead of counting from startup. They share the second problem, though: a process that is not running has no timer at all.
Sleeping instances make it worse. A host that spins a container down after a stretch without traffic stops the process, and the timer goes with it. Nothing wakes the container at 03:00, because waking it takes a request, and the schedule was the thing that was supposed to produce one. A schedule that depends on the process being awake inherits every reason the process might not be.
Locks, unique rows or one scheduler: what each fix leaves with you
They are not interchangeable. Each one removes a failure mode and quietly hands you a different one.
A lock skips, it does not catch up
The most common fix keeps the scheduler in every instance and makes them race for a lock in a shared store. Whoever takes the lock runs the job; everyone else stands down. ShedLock, a lock library for Spring's scheduled tasks, describes itself in one line: "ShedLock makes sure that your scheduled tasks are executed at most once at the same time." As of writing, Laravel's version is a single method, and it needs a cache store that every server shares:
Schedule::command('report:generate')
->fridays()
->at('17:00')
->onOneServer();node-cron added the same idea in version 4. As of writing, the option uses an environment-variable flag out of the box to mark which instance runs, and can use a Redis coordinator when that one instance should not be a single point of failure:
cron.schedule('0 17 * * 5', generateWeeklyReport, {
name: 'weekly-report',
distributed: true,
});The sentence to notice in ShedLock's documentation comes further down. If a task is already running on one node, "execution on other nodes does not wait, it is simply skipped." Right for preventing duplicates, and wrong in the one case that hurts: the node holding the lock dies halfway through the job. The others skipped. Nobody retries. A lock turns "ran three times" into "ran at most once", and at most once includes zero.
Where the lock lives matters as well. All three of these can keep it in Redis, and a lock key is normally written with an expiry so that a crashed holder cannot block the job forever. Under a volatile-* eviction policy, keys with an expiry are precisely the keys Redis may remove when memory runs short. When Redis reaches maxmemory and starts evicting, a lock that vanishes early lets a second instance start the job while the first is still running it. If a lock guards anything expensive, keep it on a Redis instance that is not also a cache under memory pressure, or on one set to noeviction.
A unique row per run
A sturdier variant stops treating duplicates as a race and treats them as a constraint. Rails' Solid Queue lets several schedulers run the same recurring configuration, for redundancy, and stops duplicates in the database: each enqueued run writes a row to a table with "a unique index on task_key and run_at, ensuring only one entry per task per time will be created." Two schedulers can both try, and the second insert fails. As of writing, the guarantee holds as long as finished jobs are kept, which is the default.
The difference from a lock is that the record outlives the attempt. A lock expires, and after it expires nothing remembers that the 17:00 run happened. A row keyed by task and scheduled time is still there at 17:01. That is what makes the approach worth copying into any application with a relational database, whatever scheduler it uses, which the section on idempotency below does.
How do I share a single job store among one or more worker processes?
APScheduler's FAQ asks this in so many words, and the documented short answer is "You can't." The long answer explains that sharing a persistent job store between processes "will lead to incorrect scheduler behavior like duplicate execution or the scheduler missing jobs". The workaround the project gives for the 3.x series is to run the scheduler as one dedicated process and have the rest of the application talk to it.
The third fix follows from it: take the schedule out of the replicated service and give it a single home. Celery's standalone beat is the same design. It works, and its price is the middle row of the earlier table. One scheduler is a single point of failure, and keeping the count at exactly one through restarts and deploys becomes your job instead of the framework's. It is also the point where it is fair to ask why the scheduler is a process you run at all, rather than a scheduled job that exists once, however many replicas are running.
Even outside the app, it is "approximately once"
Moving the schedule out of the application removes the multiplication by replica count. It does not buy exactly-once execution, and Kubernetes says as much in its own CronJob documentation: "A CronJob creates a Job object approximately once per execution time of its schedule." It goes on to say that "there are certain circumstances where two Jobs might be created, or no Job might be created", and concludes: "Therefore, the Jobs that you define should be idempotent."
Every fix above reduces the number of duplicates, and none of them guarantees zero. What makes the remaining duplicates harmless is the job refusing to do the same work twice, keyed on its name plus the time it was scheduled for:
CREATE TABLE scheduled_runs (
task text NOT NULL,
scheduled_for timestamptz NOT NULL,
started_at timestamptz NOT NULL DEFAULT now(),
finished_at timestamptz,
PRIMARY KEY (task, scheduled_for)
);
-- First statement of the job. No row returned means another copy already claimed this run.
INSERT INTO scheduled_runs (task, scheduled_for)
VALUES ('weekly-report', '2026-09-11 17:00:00+02')
ON CONFLICT DO NOTHING
RETURNING task;The job works out which scheduled time it belongs to, rounded to the schedule rather than read off the clock, tries to claim it, and exits quietly if the claim returns nothing. Three instances firing at once produce one row and two clean exits. A double fire during a rolling deploy is caught the same way, and so is a platform scheduler's occasional second run. The job sets finished_at when it completes, so a row where that column is still empty an hour later is both the sign that a claimed run died and the condition to alert on.
How Runsite handles it
Both of the safe shapes above are available. Run the schedule as a Runsite cron job instead of inside the app: each job is defined once and starts a fresh container from your image at its scheduled time, as a separate resource from your web service, so scaling the web service from one instance to four does not change how many containers the schedule starts. Every run is recorded with its exit code, output and duration, and a failed or timed-out run can alert you by email or webhook, which covers the zero-runs case a lock never sees.
Web services are where the request-handling code belongs. As of writing, on Pro and Business plans they can autoscale horizontally between a minimum and maximum instance count, which is the setting that makes an in-app scheduler unpredictable, so the working rule is to keep timers out of any service with autoscaling switched on. On sleep, the entry Nano plan suspends a web service after 14 days without a request, and Starter and above do not sleep. A timer left inside a Nano service stops firing once the service has been quiet for two weeks.
If a schedule has to stay inside the application for now, its lock needs a shared store, and managed Redis in the same region is the usual place; just keep lock keys off an instance that evicts. The idempotency table belongs in managed PostgreSQL. All of it runs in Frankfurt, Germany, as of writing, with a signed GDPR data processing agreement included with every account. A complete example of a job that belongs outside the app is the scheduled PostgreSQL backup to S3, and setup details are in the Runsite docs.
The short version
- A scheduler inside your application is a timer in that process's memory. Three instances hold three timers, and each one fires.
- By default,
node-cron,@nestjs/schedule, APScheduler and Laravel's scheduler multiply with the instance count. Celery beat and Solid Queue are built around a separate scheduler process. - The instance count changes without you: autoscaling at the scheduled minute, both versions alive during a rolling deploy, and the idle side of blue-green running its own timers.
- The opposite failure is silence.
setIntervalcounts from process start and resets on every deploy, and a sleeping instance has no timer at all. - A lock prevents duplicates by skipping. If the holder dies mid-run nothing retries, and a lock key with a TTL is exactly what a
volatile-*Redis policy evicts. - A unique row per task and scheduled time outlives the attempt, which a lock does not.
- One dedicated scheduler process fixes the multiplication and makes you responsible for there being exactly one of it.
- Even Kubernetes promises only "approximately once". Moving the schedule out of the app reduces duplicates; an idempotency key on task plus scheduled time makes the rest harmless.