76techs / Blog

Pragmatic Scalability: What I Do Before Adding Servers or Splitting into Microservices

|

When systems feel slow or unstable, measure first and fix the highest-ROI bottleneck before adding infrastructure complexity.

Pragmatic Scalability: What I Do Before Adding Servers or Splitting into Microservices

When a system starts feeling slow, flaky, or kind of unstable, two reflexes show up almost immediately: add more servers or split into microservices. Both can work, and both can save products. But both can also become expensive distractions when nobody has actually found the bottleneck. My approach is boring on purpose: measure first, then fix the cheapest constraint with the highest ROI.

The painful truth about scaling

A lot of people think scaling means handling more requests. That is only half the story. Real scaling means growing without turning the system into a stress machine, while keeping:

Low latency (especially p95/p99, not average)

Stability under traffic spikes and failures

Predictable cost (not we scaled and the bill doubled overnight)

Architecture maintainable (so every change does not feel like surgery)

My pragmatic order (the stuff I try before big moves)

Cache hot endpoints (carefully, not blindly)

Move heavy work to queues (stop expensive work inside requests)

Fix the database reality (queries and indexes)

Add minimum viable observability (so we stop guessing)

1) Cache hot endpoints (the right way, not the YOLO cache)

Hot endpoints are the routes everyone hits constantly: dashboards, summaries, profile data, and high-frequency listings. Caching these endpoints is often the best ROI in performance engineering, but it can also ship wrong data if done lazily.

Why caching gives fast ROI

Reduces database pressure

Cuts response latency

Absorbs traffic spikes without multiplying infrastructure

Practical cache key model (examples)

user:123:dashboard:v1 (short TTL like 60-300s)

tenant:45:stats:v2 (multi-tenant summaries)

tenant:45:orders:status=paid:page=2:sort=created_at_desc:v1

Where caching usually fails

One key fits all designs that ignore filters, pagination, and sorting

Global cache when data must be user or tenant scoped

No expiration and no invalidation strategy

Caching because we heard it is good instead of because we can explain it

If I cannot explain key format, TTL, and invalidation in one minute, I do not ship caching just to try.

2) Move heavy work to queues (stop doing expensive things inside requests)

A lot of products feel slow because the API is trying to do too much right now. The request-response path should be fast, predictable, and boring.

What goes to queue immediately

Email and SMS sending

PDF and report generation

Image processing

Slow external integrations

Webhooks with retries

Anything that risks timeouts under peak load

How product UX improves

The API shifts from I will do everything now to accepted and processing, with clearer status tracking. Users feel fewer timeouts and less randomness.

Classic danger: duplicate side effects

Jobs can retry, and retries can duplicate effects like billing, emails, or updates. Idempotency and queue backlog monitoring are not optional.

3) Optimize queries and indexes (DB is guilty until proven innocent)

In many systems, the app layer is not the bottleneck. The database is. Adding app servers often adds more concurrent pressure on the database.

Clear signs the DB is the constraint

Endpoint stays slow even after scaling app instances

High DB CPU or screaming disk IO

Slow query logs repeat the same offenders

N+1 query patterns

Performance degrades as tables grow

Lock waits, deadlocks, or connection pool saturation

What I do in practice (not fancy, just effective)

Identify highest-cost queries via APM or slow logs

Fix N+1 with proper eager loading or joins

Avoid SELECT * when not needed

Create indexes aligned with WHERE, JOIN, and ORDER BY

Validate plans with EXPLAIN and re-check after changes

Common traps

Too many indexes hurting writes and storage

Poor composite index order

Optimizing the wrong query because nobody measured

Tuning without baseline metrics (expensive guesswork)

Optimization without measurement is expensive guesswork.

4) Metrics and alerts (because guessing does not scale)

Without measurement, there is no control. Without alerts, you find out about incidents from customers.

The metrics I care about

p95/p99 latency per endpoint

Error rate (4xx/5xx)

Throughput (req/s)

DB query time and connection usage

Queue size and queue wait time

CPU and memory saturation

Cost anomaly signals (FinOps)

The alerts that actually help

p95/p99 above threshold for sustained windows

5xx spikes

Queue backlog growing

DB connections near limit

Cost anomaly alerts

I avoid CPU-only alerting. CPU is often a symptom, not the cause. A system can look fine on CPU while the DB is locked and everything times out.

Practical example: Microservices will fix it until metrics said otherwise

I have seen teams propose microservices because the system felt slow and unstable. The story was: it is too big, we need to split.

After measuring first, we found:

A dashboard endpoint recalculated expensive stats on every request

Missing indexes on the most-used filters

Report generation happening in the request path

No visibility into p95/p99 or queue health

DB connections spiking under traffic bursts

The pragmatic plan before any rewrite

Cache hot dashboard endpoints with tenant-scoped keys and short TTL

Move report generation to a queue (async with status tracking)

Add the right indexes and fix N+1 patterns

Add p95/p99, queue backlog, and DB connection alerts

Performance improved, stability improved, and cost became predictable without premature operational complexity.

And microservices? When do they become a loss?

I like microservices when context demands them. They become expensive when used too early, especially when:

Domain boundaries are still changing and likely to be split wrong

The team lacks mature CI/CD and observability

Services over-communicate (distributed monolith)

Operational overhead explodes (deployments, tracing, incidents, on-call fatigue)

In many cases, a well-designed modular monolith gives better ROI: clear boundaries, simpler deployment, and gradual evolution without distributed coordination overhead.

Conclusion

Before scaling infrastructure or splitting into services, I look for quick ROI in fundamentals.

I prioritize:

Cache where it clearly helps

Queues for heavy async work

Database optimization

Minimum viable observability

These steps solve a huge chunk of scaling pain and make the next architecture decision clearer, because now you are acting on evidence, not vibes.

Pragmatic next step

If you are dealing with rising cloud costs, slow APIs, unstable traffic spikes, or uncertainty between modular monolith and microservices, the first win is usually the same: measure, find the real bottleneck, and fix the highest-ROI constraint first.

Rising costs

Slow API

Spikes causing instability

Architecture uncertainty

I can deliver

Evidence-based bottleneck map

Quick wins in 1-2 weeks

Phased evolution plan in 1-2 months