Imagine the /api/login endpoint you made, of a Node.js backend, got hit with lots of login requests. Each one looks up the user by email, verifies the password against a legacy bcrypt hash, re-hashes it with Argon2id, verifies an MFA code, then issues a JWT. If you use a pure JavaScript implementation like bcryptjs, all of that hashing runs on the main (master) thread. The server has, let's say, 4 CPU cores, but Node.js runs your JavaScript code on a single thread, so it can only use one core for all of these requests. When it's under heavy load, the server either drops or delays requests, and clustering fixes this by running one worker per core.

According to the Node.js docs, clusters of Node.js processes can be used to run multiple instances of Node.js that distribute workloads among their application threads. It basically allows you to create many workers that share the same port, using all (or some) of the available CPU cores. The cluster module has been part of Node.js since v0.8, and is stable in current versions.

Cluster

Clusters of Node.js processes can be used to run multiple instances of Node.js that can distribute workloads among their application threads.

nodejs.org

Why Use Clustering

Node.js runs your JavaScript code on a single thread, so one process can only use one CPU core. On an 4 cored server, that leaves 3 cores sitting on the couch eating chips while one is fighting for its life . Clustering can be used to ensure all CPU cores are used to distribute work across all workers.

When To Use Clustering

When you have alot of incoming HTTP requests that require heavy computation. It helps with distributing the load to independant workers, making the response time faster with less errors. Note that it won't help that much if your app mostly waits on databases or APIs, since Node.js already handles those things well on a single thread.

Let's test that out, shall we?

Tests

Without Clustering

index.ts:

import { serve } from "@hono/node-server";
import { Hono } from "hono";

const app = new Hono();

app.get("/", (c) => {
  let num = 0;

  // simulate a heavy computation
  for (let i = 0; i < 50_000_000; i++) {
    num++;
  }

  return c.json({ message: `Iterated ${num} times.` });
});

serve(
  {
    fetch: app.fetch,
    port: 3000,
  },
  () => console.log(`Server is running on http://localhost:3000`),
);

Then run npx tsx ./index.ts. On a new terminal window, run npx loadtest -n 1000 -c 500 -k http://localhost:3000.

Server is running on http://localhost:3000

Loadtest Result:

Target URL:          http://localhost:3000
Max requests:        1000
Concurrent clients:  1000
Running on cores:    2
Agent:               keepalive

Completed requests:  1000
Total errors:        381
Total time:          30.354 s
Mean latency:        20986.8 ms
Effective rps:       33

Percentage of requests served within a certain time
  50%      23800 ms
  90%      30194 ms
  95%      30208 ms
  99%      30219 ms
 100%      30220 ms (longest request)

   -1:   381 errors

With Clustering

index.ts stays the same, cluster.ts:

import cluster from "cluster";
import { cpus } from "os";
import { join } from "path";
import { cwd } from "process";

// how many cpu cores we have
const cores = cpus().length;

cluster.setupPrimary({
  exec: join(cwd(), "index.ts"), // our index.ts
});

for (let i = 0; i < cores; i++) {
  // forking = new worker
  cluster.fork();
}

// if a worker crashes for some reason..... just spawn another one
cluster.on("exit", () => cluster.fork());

Then this time, run npx tsx ./cluster.ts, and do the same loadtest in a new terminal.

Server is running on http://localhost:3000
Server is running on http://localhost:3000
Server is running on http://localhost:3000
Server is running on http://localhost:3000

Loadtest Result:

Target URL:          http://localhost:3000
Max requests:        1000
Concurrent clients:  1000
Running on cores:    2
Agent:               keepalive

Completed requests:  1000
Total errors:        0
Total time:          26.792 s
Mean latency:        13402.4 ms
Effective rps:       37

Percentage of requests served within a certain time
  50%      13356 ms
  90%      24098 ms
  95%      25419 ms
  99%      26391 ms
 100%      26603 ms (longest request)

Summary

Better Clustering With PM2

Instead of manually running clustering, production can better handle using the pm2 npm package. "PM2 is a production process manager for Node.js/Bun applications with a built-in load balancer. It allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks" (PM2). It's as easy as:

pm2 start index.js -i max

Conclusion

Node.js runs your JavaScript on a single thread, so on a multi-core server most of your CPU sits idle while one core handles everything. Clustering fixes that by summoning workers per core, all sharing the same port, with the master process handling the workers and spawning another one on crash.

In our tests, the same machine and the same app code went from timing out on 381 of 1000 requests to timing out on none, with about 36% lower mean latency, just by adding a small cluster.ts file.

For production, you can just skip the cluster.ts file and use PM2 (pm2 start index.js -i max). You get clustering, crash recovery, and some other additional features out of the box.

Sources

• "Cluster." Node.js, nodejs.org/api/cluster.html.
• Loban, Volodymyr. "Clustering in Node.js." Medium, 22 Sep. 2024, medium.com/@vloban/clustering-in-node-js-4e0bf17b7f0b.
• Nahid, Shams. "Clustering in Node.js: Look for the Limitations." Medium, 27 Sep. 2022, shams-nahid.medium.com/clustering-in-node-js-look-for-the-limitations-e6ff7f846a3c.
• "PM2." npm, www.npmjs.com/package/pm2.
• "Scaling Your Node.js App Using the 'Cluster' Module." YouTube, uploaded by Software Developer Diaries, 26 Feb. 2023, youtu.be/6lHvks6R6cI.