At a glance:
- Auto-scaling closes a capacity gap, not a latency gap, and that difference decides who wins auctions during a traffic spike.
- New instances take 30 to 300 seconds to start serving traffic. The auctions that ran during that window are already decided by the time the new capacity arrives.
- Standard monitoring shows what your platform did, not what the exchange did with a late response, so timed-out bids can disappear from your dashboards entirely.
- Average latency staying in range can hide a p99 problem that's already costing win rate.
- When tuning gains flatten and the latency tail doesn't track your own traffic patterns, the bottleneck is likely the shared host, not the application.
Five auto-scaling assumptions that hold up until bid density peaks
You probably don't start questioning auto-scaling or shared infrastructure because of anything you see at baseline performance. The dashboards look healthy, and the load tests pass.
The questions start when bid density climbs, whether that's during a campaign peak, a sustained load test, or a Monday morning win rate review that starts with something unexpectedly off.
"Auto-scaling handles our traffic peaks."
That assumption holds until the first major campaign push. Going into it, the configuration looks right: auto-scaling is tuned, warm pools are pre-initialized, and the architecture is prepared to handle the volume.
What you don't see in the pre-event review is the provisioning gap. Hyperscaler auto-scaling default warm-up is around 300 seconds. With warm pools pre-initialized, that can come down to around 30 seconds under favorable conditions.
Auction timeouts run at 100 milliseconds or less. The gap between a scaling trigger and a new instance taking live auction traffic is measured in minutes. Thousands of auctions close during that window.
In the meantime, your existing nodes are carrying the full auction load at higher utilization. Every bid being processed on those nodes either makes the exchange timeout window or misses it based on what those nodes can deliver.
Auto-scaling solves a capacity problem, but RTB is often a latency problem. New capacity coming online in 30 to 300 seconds doesn't change the auctions that already happened during the provisioning window.
"Our monitoring would tell us if bids were timing out."
Standard monitoring captures what your bidding platform did, not what the exchange did with the response.
When a bid arrives after the auction window closes, the exchange drops it and moves on. That shows up as a missed auction on the exchange side, not as a latency error in your logs. Your infrastructure dashboard still shows average response time within acceptable ranges.
The problem is discovered unexpectedly: tracking p99 bid response latency against win rate by exchange over time isn’t a comparison most monitoring configurations are designed to surface. By the time someone pulls it, win rates have already dropped.
"The system performs fine at baseline, so peak performance will look the same."
Your baseline load test looks healthy. At 50,000 bids per second, p99 response latency sits at 6 milliseconds.
Then the campaign launches. The platform runs at 90,000 bids per second during a major campaign event, and the physical hardware is now more loaded than it was during the test.
Average response latency stays within spec, but p99 starts drifting on the bids that happen to hit moments of resource contention.
Nothing in the dashboards suggests a problem at first. Then p99 starts moving, and a little later, so do win rates.
"Adding more instances will fix latency during a spike."
When latency starts climbing, the instinct is usually to scale out: more instances, more capacity, and the problem resolves.
More instances increase total throughput capacity, but they don't change what's happening on the nodes already serving live traffic. The bids on those nodes are meeting the exchange timeout window or missing it based on what those nodes can deliver right now, and new capacity coming online in 30 to 300 seconds doesn't change those outcomes.
You can have enough capacity for the traffic and still lose bids because of p99 latency on the nodes already handling live auctions.
"Enough tuning will get us to consistent p99."
The tuning path is real and worth running: profiling the bid pipeline, optimizing cache hit rates, adjusting network settings, and tuning container resource allocation. Each change moves the numbers, and some improvements are significant.
Eventually, though, the gains get smaller.
When p99 variance tracks with time-of-day patterns that don't match the platform's own traffic, or when the tail widens during periods your logs show as quiet, the source is likely shared resource behavior at the physical host level, CPU and memory contention from co-tenant workloads that application-layer tuning can't reach.
At some point, you stop looking for another tuning win and start wondering whether the limitation is in the application or the environment it's running on.
Better to review these assumptions with a spreadsheet open than with a campaign live and win rates dropping.
