At a glance:
- A latency addendum in a demand partner's contract, a 5ms p99 requirement, turned a routine infrastructure refresh into a contractual engineering constraint.
- The initial design held up at baseline load, but p99 climbed into unacceptable territory at the traffic levels a major campaign produces, while average latency stayed clean the whole time.
- Meeting that same p99 target on hyperscaler compute meant paying for reserved capacity and warm pools sized for the worst case, not the auto-scaling buffer the original design assumed.
- Bare metal held p99 nearly flat between baseline and peak, since there's no co-tenant contention to account for, at the cost of longer provisioning lead times that require planning further ahead.
- At the exchange's volumes, inter-AZ transfer charges for redundant clearing traffic added real cost that bare metal's single-facility data movement didn't carry.
When a latency guarantee in a partner contract changes the infrastructure conversation
You've been working through the infrastructure for a new clearing engine for about three weeks.
The brief is straightforward: replace the shared service that's been slowing the platform down with a purpose-built clearing engine. Budget is approved, and the target is Q3.
The first draft points to a hyperscaler. Your team knows the tooling, the service catalog fits the design, and nobody expects much debate. Then the partnership lead comes back with one comment: the exchange's largest demand partner, a trading desk managing a significant book of programmatic spend, has a latency addendum in their master services agreement.
Clearing operations need to stay below a 5-millisecond p99 response time to keep that partner on the platform. What was signed two years ago, when the exchange was smaller, is now an engineering constraint.
What the initial hyperscaler design delivers
The clearing engine design called for a cluster of c6i instances behind an application load balancer, auto-scaling based on CPU and request queue depth. At 40,000 bids per second in testing, p99 clearing latency came in around 3.8 milliseconds.
The problem doesn't show up until the load tests move beyond baseline. At 100,000 bids per second, representing roughly the volume the exchange processes during major campaign events, p99 clearing latency climbed to between 9 and 13 milliseconds depending on the time of day.
The variance tracked with how loaded the surrounding infrastructure in the AZ seemed to be, a pattern that's consistent with shared resource contention from co-tenant workloads.
Up until that point, the numbers looked acceptable because average clearing latency stayed comfortably within spec. The requirement, though, is p99, not the average.
What the evaluation process looked like
From there, the evaluation came down to two options: hyperscaler compute with reserved instances and pre-initialized warm pools configured, or dedicated bare metal, with Servers.com by Nexcess in the comparison matrix.
The hyperscaler path represented what the team knew. The design would need to be more aggressive on reservation coverage and warm pool depth to keep p99 in range under load, which meant paying for capacity that would spend most of its time waiting for campaign peaks. The auto-scaling buffer that normally absorbs capacity variance would need to be pre-provisioned as reserved capacity instead.
The Servers option came down to two questions: would the automation model fit a team that's used to hyperscaler compute, and would p99 stay consistent under sustained load?
The latency results told a different story. Without other tenants' workloads competing for physical CPU and memory, the clearing latency at 100,000 bids per second tracked much closer to the baseline numbers. In the test environment, p99 came in around 4.1 milliseconds at sustained peak load versus 3.8 milliseconds at baseline, a much smaller spread than the hyperscaler tests had produced.
The automation question turned out to be less dramatic than expected. Our platform provides API-driven provisioning and a self-service portal, so the operational model wasn't the fully manual bare metal process the initial evaluation had assumed.
What was different was provisioning new capacity. That timeline is measured in hours, not minutes, which means capacity decisions need more lead time and pre-planning.
For a clearing engine with well-defined load characteristics and a predictable peak model, that was a tradeoff the team could accept.
The cost comparison at scale
At steady-state load, the reserved instance path on hyperscaler and our bare metal path came out within a similar monthly range for the core compute footprint.
The reserved instance cost included the pre-provisioned warm pool capacity required to meet the p99 guarantee: headroom that sits idle during off-peak hours but is necessary to absorb surges without waiting for auto-scale provisioning.
That's where the numbers started to separate. Hyperscaler inter-AZ transfer can run $0.01 per GB in each direction, and clearing engine traffic involves significant cross-AZ movement for redundancy.
At the exchange's modeled bid volumes, that line item added material cost on top of the compute reservation. On dedicated bare metal within a single facility, that traffic pattern didn't carry the same per-GB charge structure.
By the end of the evaluation, the deciding factor was which option could reliably meet a contractual p99 commitment under sustained load without a cost structure that changed every time bid volume did, not which one looked cheaper on paper.
The evaluation is still underway at the time of writing, but the addendum already did its real work: it turned an infrastructure preference into a contract term, one with a specific number attached to a specific partner relationship, and that number stops being theoretical the day the new clearing engine goes live.
