From Frameworks to Metal: How to Reach 102,000 RPS with Go and Stop Being Afraid of High Load

There comes a point in a developer's career when you start getting tired of constraints.

Standard frameworks. Ready-made ORMs. Abstraction on top of abstraction. Layers that exist to hide other layers. And somewhere at the bottom of all of it, a simple map that just wanted to find a key.

At first, it's convenient. Then it gets cramped.

Eventually, you start wanting to do the opposite: remove everything unnecessary and see what the machine can actually do when you stop getting in its way.

That's how my current project evolved — a high-load product aggregator written in Go.

No SQL in the hot path. No heavyweight frameworks. No standard JSON marshaler. Just mmap, custom indexes, direct memory access, and a little bit of unsafe.

And the result turned out to be somewhat more interesting than I expected.

---

From Syntax to Hardware

Learning Go usually starts innocently enough.

First:

go func() {
    ...
}()

Then come:

• goroutines;
• channels;
• sync.Mutex;
• WaitGroup;
• sync.Map;
• sync.Pool;
• atomic;
• worker pools;
• queues;
• caching;
• Docker;
• microservices.

And then real-world load arrives.

Suddenly, the question is no longer:

Do you know how to write concurrent code?

The question becomes:

How much work actually needs to be done for a single request?

That's a completely different level of optimization.

Because if a request goes through ten abstractions, creates several objects, performs multiple data conversions, hits a database, retrieves the data, converts it into structs, and then converts those structs back into JSON — no amount of atomic is going to save you.

And that's when the evolution starts going in reverse.

Slices instead of unnecessary maps.

Direct memory access instead of intermediate objects.

Atomic operations where an atomic is actually enough.

mmap instead of constant I/O.

Indexes instead of calculations.

Raw bytes instead of conversions.

And eventually, one question:

What if we simply remove everything from the hot path that isn't absolutely necessary?

---

Makodb: A Database That Doesn't Try to Be SQL

The project is built around a custom mmap-based KV database — Makodb.

Its job is deliberately narrow: serve data as quickly as possible for read-heavy workloads.

There is no SQL query, ORM, or conversion pipeline in the hot path:

storage
   ↓
object
   ↓
ORM
   ↓
struct
   ↓
JSON

Instead:

mmap
  ↓
index
  ↓
docID
  ↓
raw data
  ↓
JSON buffer

The main principles are:

• lock-free reads;
• one write lock per shard;
• mmap;
• sharding;
• sorted uint64 indexes;
• merge-style intersection/union;
• direct range queries;
• sort indexes;
• minimal allocations;
• no objects in the hot read path.

One particularly interesting component is the turbo index.

An index is simply a sorted array of uint64 values:

[docID1][docID2][docID3]...[docIDN]

So intersecting two indexes becomes a standard merge operation:

A:  1  4  7  12  20  35
B:  2  4  8  12  19  35

AND:

    4        12       35

No temporary maps.

No temporary objects.

No O(n*m).

Just two pointers and sequential memory access.

---

700,000+ Landing Pages and 4 Million+ Products

This isn't a synthetic benchmark with a single Hello World endpoint.

The actual project contains:

• 700k+ landing pages;
• 4M+ products;
• categories;
• slugs;
• sorting;
• filters;
• price ranges;
• text indexes;
• pagination;
• different combinations of parameters.

And one principle is particularly important:

There is no special magic for page 1 and page 500,000.

There is no situation where the first page is instant and deep pagination suddenly turns into OFFSET 500000.

There is an index.

The required position can be accessed directly.

Sorting is also not performed from scratch for every request — prebuilt sort indexes handle it.

Filtering becomes a set of operations on indexes.

---

And Then It Turned Out the Database Wasn't the Problem

This was probably the most interesting stage.

Under normal load, data queries were completing in roughly:

0.161–3 ms.

But the browser was showing considerably more.

So I started measuring.

Data retrieval — fast.

Filtering — fast.

Indexes — fast.

Result composition — fast.

And then:

w.Write(...)

Suddenly, it became clear that sending the result could sometimes cost more than computing it.

That's where things got interesting.

---

Goodbye, encoding/json

The standard:

json.NewEncoder(w).Encode(v)

is perfectly fine for most applications.

But if the goal is to squeeze every possible bit of performance out of a single machine, the standard JSON marshaler becomes expensive.

So I built SilentJSON.

The idea is simple:

We already know the response structure. Why should the runtime figure it out again?

Instead of dynamically inspecting types through reflection, the serializer uses precomputed field metadata, offsets, and specialized marshal functions.

Conceptually:

struct
  ↓
known offset
  ↓
direct memory access
  ↓
append bytes
  ↓
JSON

Instead of:

interface{}
   ↓
reflect
   ↓
temporary object
   ↓
conversion
   ↓
JSON

And this leads to another important principle.

---

If the Structure Is Known, Don't Make It Universal

Universality is usually considered a virtue.

But universality has a cost.

For example:

map[string]interface{}

is convenient.

But you pay for that convenience in:

• predictability;
• allocations;
• cache locality;
• type safety;
• serialization speed.

For a narrow, specialized project, you can make a completely different tradeoff.

For example:

[]{
    key,
    weight,
    value
}

Or a specialized raw type.

Or a response structure whose layout is known in advance.

It's less universal.

But extremely transparent.

And fast.

---

The Most Revealing Experiment: GET vs HEAD

At some point, there was a very simple way to test the hypothesis.

If HEAD executes the entire internal request logic but doesn't send the response body, we can compare it directly with a normal GET.

The results were very revealing.

At:

Concurrency: 5800
Duration:    60s

HEAD

Total: 6,167,463 requests
RPS:   102,339
Errors: 0.0%

Average latency:

/shop                 44 ms
/shop/{category}      45 ms
/shop/{slug}          92 ms
/products              44 ms
/products/turbo        45 ms
/categories/tree      46 ms

Maximum latency:

310–483 ms

In other words, the entire internal workload can be processed at more than 100,000 requests per second.

Now the same server, the same workload, the same concurrency — but a real GET.

GET

Total: 3,355,057 requests
RPS:   55,632
Errors: 0.0%

Average latency:

/shop                 84 ms
/shop/{category}      86 ms
/shop/{slug}         164 ms
/products              81 ms
/products/turbo        77 ms
/categories/tree      81 ms

There it is.

Almost a factor of two.

Not search. Not indexes. Not filtering. Not the database.

The result has to be assembled and physically pushed out of the machine.

---

Where the Algorithm Ends and Physics Begins

The architecture now looks roughly like this:

                    102k RPS
                       │
                       ▼
             ┌─────────────────┐
             │    Go / CPU     │
             │                 │
             │     Makodb      │
             │     indexes     │
             │     filters     │
             │     business    │
             └────────┬────────┘
                      │
                  JSON ready
                      │
                      ▼
             ┌─────────────────┐
             │ output pipeline │
             │                 │
             │ memory → kernel │
             │ → socket → NIC  │
             └────────┬────────┘
                      │
                      ▼
                   ~55k RPS

And this is where it gets slightly funny.

Because the application is already saying:

"I can do more."

And the hardware responds:

"Where exactly are you planning to put all those bytes?"

---

5,800 Concurrent Requests

At 5800 concurrent requests, the system reached approximately:

55–56k RPS of full GET requests.

And the interesting part is that it doesn't collapse.

0 errors
0.0%

No avalanche of timeouts.

No gradual degradation into death.

No sudden throughput collapse.

It simply reaches its physical ceiling.

---

What Happens at 105,800 Concurrent?

Naturally, the next question was:

What if we push it harder?

So:

go run main.go -c 105800 -d 60s

Result:

Total: 3,419,927 requests
RPS:   54,801
Errors: 0.0%

Average latency increased:

/shop             ~1.51 s
/category         ~1.52 s
/slug             ~3.03 s
/products         ~1.51 s
/products/turbo   ~1.51 s
/categories       ~1.51 s

But the server didn't die.

That's an important distinction.

The system couldn't process more than roughly 55k RPS — but it held that ceiling under an absurd number of concurrent requests.

So as concurrency increases:

200
  ↓
800
  ↓
2800
  ↓
5800
  ↓
15800
  ↓
55800
  ↓
105800

throughput eventually stops growing.

But it doesn't collapse.

We hit a plateau.

---

Load Test Series

Concurrency	RPS	Avg latency	Errors
200	36,209	2–9 ms	0%
800	53,202	7–23 ms	0%
2,800	55,781	34–78 ms	0%
5,800	55,632	77–164 ms	0%
15,800	55,868	219–448 ms	0%
55,800	55,061	794–1,599 ms	0%
105,800	54,801	1,507–3,025 ms	0%

And that's probably the most interesting graph in the whole experiment.

RPS
60k ┤
    │        ┌─────────────────────────────────────
55k ┤────────┘
    │
50k ┤
    │
40k ┤
    │
30k ┤
    │
20k ┤
    │
10k ┤
    └──────────────────────────────────────────────
      200  800  2.8k  5.8k  15.8k  55.8k  105.8k

After roughly 2–3 thousand concurrent requests:

throughput becomes almost horizontal.

What keeps increasing is latency.

That's classic behavior for a system that has reached a resource ceiling.

---

The Funny Part: This Isn't a Server Cluster

This experiment wasn't performed on some enormous fleet of machines.

There wasn't:

load balancer
    ↓
20 application servers
    ↓
10 database servers
    ↓
Redis cluster
    ↓
Kafka cluster

It's one machine.

Ordinary workstation/home-server hardware.

And that's exactly why the experiment became interesting.

At one point I used to test what Go could do with a simple Hello World.

And I remember a very different picture: after a few hundred requests, things started going south.

Here:

5,800 concurrent
→ ~55k GET RPS

5,800 concurrent
→ ~102k HEAD RPS

105,800 concurrent
→ ~55k GET RPS
→ 0 errors

At some point, you stop asking:

"How do I optimize Go?"

And start asking:

"Where exactly does the program end and the computer begin?"

---

Why Does This Work?

1. Minimal Work Per Request

The main secret isn't some magical function.

It's much more boring:

A request should do as little work as possible.

If the data is already indexed, don't search for it.

If the sort order is already known, don't sort it.

If two conditions can be intersected using a merge operation, don't build another map.

If the JSON structure is known, don't discover it through reflection.

If the document already lives in mmap, don't drag it through several intermediate representations.

---

2. Indexes Instead of Computation

A product may have indexes for:

category
brand
price
rating
date
text
...

A query such as:

category = electronics
AND
brand = x
AND
price = 5000..50000
ORDER BY price DESC

doesn't become:

find all products
→ check category
→ check brand
→ check price
→ sort everything
→ take 60

Instead:

category index
       ↓
      AND
       ↓
brand index
       ↓
      AND
       ↓
price range
       ↓
sort index
       ↓
page

Most of the work is performed on compact numeric indexes.

---

3. mmap

Data lives in memory-mapped storage.

The read path doesn't have to constantly perform traditional filesystem I/O.

Conceptually:

disk file
   │
   ▼
 mmap
   │
   ▼
memory address
   │
   ▼
read

This is particularly well suited to read-heavy workloads.

---

4. Lock-Free Read Path

Writes are a different story.

But reads shouldn't have to wait for other reads.

There is no global:

mu.Lock()
...
mu.Unlock()

on every request.

The hot read path uses atomic reads and direct memory access.

Sharding additionally distributes write contention.

---

5. JSON Became the Last Enemy

And this is probably the most important conclusion of the entire experiment.

After optimizing the database, indexes, and search, it turned out that the next major problem wasn't retrieving the data.

It was sending it.

That's why moving to a specialized JSON marshaler made such a noticeable difference.

And then something even more interesting became apparent:

Even when JSON is already as lean as possible, the bytes still have to physically leave the machine.

At that point, no architectural abstraction can help.

The bytes have to move.

---

Where Is the Real Bottleneck Now?

Very roughly:

request
   │
   ├── routing
   ├── filtering
   ├── index intersection
   ├── sorting
   ├── document lookup
   ├── JSON composition
   │
   └── network output

The first stages have become fast enough that optimizing them further no longer produces the biggest gains.

Now the problem is much more physical:

throughput ≈ available bandwidth / response size

Of course, the real system is more complicated.

But at this scale, that's already the right way to think about it.

If a response is N bytes and the server needs to send tens of thousands of those responses every second, we very quickly stop talking about algorithms.

We start talking about:

how many bytes per second the entire system can move.

---

And That's Where It Gets Really Funny

You optimize the algorithm.

Then memory.

Then indexes.

Then JSON.

Then unnecessary conversions.

And eventually you discover:

CPU:
    "I can still go."

Makodb:
    "I can still go."

Indexes:
    "We're barely working."

JSON:
    "I've already been put on a diet."

Network stack:
    "Guys..."

PCIe:
    "Slow down."

NIC:
    "Where exactly are you taking all those bytes?"

And this is actually a beautiful stage of optimization.

Because it means you have finally reached the hardware.

---

What I Took Away from the Experiment

The most important result wasn't 102,339.

It wasn't even 55,632 RPS.

The real conclusion is much simpler:

High load itself isn't scary. The scary part is how much work you force the system to perform for each request.

If a request requires:

SQL
→ ORM
→ struct
→ reflection
→ map
→ conversion
→ JSON
→ copy
→ socket

then high load quickly turns into a fight against your own architecture.

If the request looks more like:

request
  ↓
index
  ↓
memory
  ↓
bytes
  ↓
socket

then suddenly a single ordinary machine can perform tens of thousands of these operations every second.

And if you remove the response body and leave only the internal processing, you get more than 100,000 requests per second.

---

Final Numbers

On the current hardware, the experiment demonstrated:

Full GET

≈55,000 RPS

with:

• 5,800 concurrent requests;
• real JSON;
• real business logic;
• filtering;
• sorting;
• index operations;
• 0.0% errors.

HEAD

102,339 RPS

with the same internal request processing and 0.0% errors.

Extreme concurrency

105,800 concurrent requests

without the server crashing and without errors.

Load

Millions of requests per minute.

And after a certain level of concurrency, the system doesn't start dying.

It simply says:

"I can't go any faster. But I'm not going to fall over either."

And perhaps that's the best result a load test can give you.

Not the biggest number.

But a predictable plateau.

Because a server that turns into a pumpkin when you go from 5,000 to 100,000 concurrent requests is a problem.

A server that says:

"My ceiling is here. Beyond that, things will simply get slower."

is already a system you can reason about, plan around, and scale.

And that leaves the most interesting question:

If the algorithms are no longer the bottleneck, JSON is already on a diet, and the CPU still has room — how far can we go if the next optimization target is no longer the code, but the path the bytes take from memory all the way to the network interface?