Node.js handles a large number of concurrent connections because it never blocks on I/O. The same design has a limit: JavaScript runs on a single thread per process. A CPU-heavy request stalls every other request on that process. Scaling a Node service means working around that limit in a deliberate order.
Measure before you scale
Add request-level metrics first: latency percentiles, error rate, event loop lag and memory. The perf_hooks.monitorEventLoopDelay API reports event loop lag . If lag climbs above roughly 100 milliseconds under load, something is blocking the thread, and adding servers will only hide the problem.
Profile with node --inspect or clinic.js and read the flame graph. Slow JSON parsing, synchronous crypto calls and large regular expressions show up here more often than the database does.
Use every CPU core
A default Node process uses one core. On a four-core machine, three sit idle. Run one process per core with the built-in cluster module or let PM2 manage it:
pm2 start server.js -i maxIn containers, set the replica count in Kubernetes or ECS instead and give each container one CPU. Keep the processes stateless, so any request can reach any instance.
Move heavy work off the request path
Image resizing, PDF generation and report exports belong in worker threads or a job queue. BullMQ with Redis is a common choice: the API accepts the request, writes a job and returns a 202 response, and a separate worker does the work. Retries, delays and rate limits come with the queue, which saves you from writing them.
Cache at three levels
- HTTP: set
Cache-Controlheaders so a CDN serves public responses without touching your server. - Shared cache: store expensive query results in Redis with a time-to-live that matches how stale the data may be.
- In-process: keep small, rarely changing lookups in memory, for example feature flags or a country list.
Decide how each cache entry is invalidated before you write it. A cache without an invalidation rule produces stale data bugs that are hard to reproduce.
Tune the database connection
Most Node slowdowns that look like server problems start at the database. Use a connection pool and size it to the database, not to your traffic. Ten application instances with a pool of 20 each open 200 connections, which can overwhelm a small PostgreSQL server. Add indexes for the queries your metrics show, paginate with keyset pagination on large tables, and use read replicas for reporting queries.
Scale horizontally behind a load balancer
Once a single machine is tuned, add more machines behind a load balancer such as NGINX, an AWS Application Load Balancer or a Kubernetes service. Move session data out of memory into Redis or signed tokens. Add health checks that test the database connection, and shut down gracefully by handling SIGTERM, finishing in-flight requests and then closing connections.
Plan for failure
Set timeouts on every outbound call. Add a circuit breaker around third-party APIs so one slow vendor does not take your service down with it. Log structured JSON with a request ID, and send traces through OpenTelemetry so you can follow one request across services.
A practical order of work
- Add metrics and find the actual bottleneck.
- Fix slow queries and blocking code.
- Add caching for the hottest reads.
- Run a process per core.
- Move slow tasks to a queue.
- Add instances behind a load balancer.
Common questions
Is Node.js a bad choice for CPU-heavy work?
Node handles CPU-heavy work if you move it off the main thread. Worker threads run JavaScript in parallel inside one process, and a job queue sends heavy tasks to separate worker processes. For numeric computing or video encoding, a service written in Go, Rust or Python with native libraries may suit the job better, and your Node API can call it.
How many requests per second can one Node process handle?
The answer depends entirely on what each request does. A process that returns cached JSON can serve thousands of requests per second on one core, while one that runs several database queries and renders a template may manage a few hundred. Load test your own endpoints with k6 or autocannon instead of trusting a generic number.
When should I switch from PM2 to containers?
PM2 works well on a single server or a small group of virtual machines. Move to containers when you need identical environments across development and production, automated scaling, or zero-downtime rollouts across many machines. Containers also make it simpler to run Node next to Redis and PostgreSQL in local development.
What causes memory leaks in Node services?
Common causes include unbounded in-memory caches, event listeners added on every request, global arrays that collect data, and timers that never clear. Watch heap size over several hours, take heap snapshots in Chrome DevTools and compare them to find the growing object type. Set a memory limit and let the orchestrator restart a leaking process while you track down the cause.
Teams often jump to step six and pay for servers that a missing index would have made unnecessary. If your Node application is slowing down and you want an outside review, our backend development team can profile it and give you a prioritized fix list.