Reducing latency in SaaS with distributed bare metal

When it comes to SaaS application hosting, latency should be just as much a boardroom topic as it is an engineering one. The physical distance between your customers and your compute shows up directly in conversion rates, retention curves, support queues and contract renewals.

Your customers are benchmarking experiences against the very best apps they use, not just your direct competitors. At the same time, data-residency mandates are requiring SaaS teams to serve and store data locally (with those who fail to do so facing penalties and/or lost deals).  

This is where distributed bare metal becomes a strategic advantage.

Placing dedicated servers across multiple global points of presence means data doesn’t have to travel as far to reach end users and increases control over where that data is processed and stored.

What is distributed bare metal?

A distributed bare metal deployment strategy involves installing applications across multiple isolated bare metal servers as opposed to virtual machines. Typically deployed across several vendors, these physical servers can be strategically located close to your customer base to reduce SaaS latency and improve overall platform performance.  

Why low latency in SaaS is a strategic priority

“Today’s applications - including enterprise applica­tions - need to be always on and always available and often must serve a global base of users who expect almost instantaneous response times regardless of where they are located,” writes Phillip Merrick, CEO and co-founder of pgEdge, Inc.

For high-demand SaaS applications, just milliseconds of tail latency (the small percentage of requests that take significantly longer than average to process) equates to user abandonment and lost revenue. Even minor delays in database responsiveness can lead to significant failures. In fact, research suggests that database performance issues lose businesses on average $7900 per minute of downtime. That means reducing latency is an indisputable growth lever.  

In some cases, latency requirements may come explicitly from your end customers. It’s not uncommon for enterprise SaaS buyers to expect service level objectives (SLOs), with provisions for credit payouts if those agreed objectives aren’t realized. Serving clients from nearby points of presence reduces network variance and makes those SLOs more defensible.

Even seemingly minimal timeouts and jitter can conflate into slowdowns that spark ticket escalations and unnecessary troubleshooting work. Beyond digging into time, resource and cash, these headaches compound to cause reputational harm. Studies show that 68% of SaaS customers consider switching after just one major outage.

In short, it means prioritizing latency reduction across the stack is imperative. As a SaaS platform’s footprint grows, server isolation and locality start to matter even more. Dedicated, resource-isolated footprints alongside localized deployments help to ensure stable performance, translating into higher retention and fewer reactive incidents.

How distributed bare metal reduces latency 

For SaaS teams, latency isn’t just a number, but more about how fast your product feels. Distributed bare metal architectures place dedicated compute exactly where your users are. Requests take fewer hops and face less contention than if they were placed in a hyperscale cloud environment, resulting in consistent performance you can control.

This principle is put into practice through proximity, multi-vendor reach, and dedicated server control.

1. Proximity to end users

To keep your SaaS interfaces feeling instant, you should be looking to place compute within 100-150ms of your highest-value user clusters. In terms of the distance between the serving hardware and the end user cluster, that means deploying within a radius of 5000km.

Placing applications close to end users reduces the number of long-haul hops and internet exchange points in the data path. In turn, that reduces latency and jitter. Ultimately, the shortest path always wins, and a distributed bare metal deployment means you can make sure you have bare metal servers deployed in data center locations that meet recommended parameters.

2. Multi-vendor reach

No single infrastructure provider will have the perfect data center map for your specific user base. So, adopting a hybrid SaaS infrastructure by partnering with multiple vendors is a smart move. It means you can increase your chances of securing hardware in optimum locations, thereby ensuring seamless end-user experiences for customers in those regions.

For these reasons, we often encourage customers to consider adopting a multi-vendor approach. B2B real-time video streaming provider nanocosmos, which offers its real-time live streaming solutions as a SaaS platform offering, is a prime example.

For nanocosmos’ customers, uptime is non-negotiable. So, in order to maintain 100% uptime, we worked together to achieve a geographically dispersed multi-vendor infrastructure strategy, combining servers.com hardware with various other compute sources. It means nanocosmos can eliminate single points of failure whilst ensuring infrastructure remains close to global end users. As Oliver Lietz, CEO of nanocosmos shared:

“We collaborate with multiple partners and are continually seeking new partnerships to strengthen our position. We actively seek high-bandwidth connections per server, modern high core counts and high core frequency servers. This, in combination with a wide variety of locations to deploy is a good starting point.”

With multiple vendor’s locations to pick and choose from, it’s much more likely that you’ll be able to deploy in the specific region that you need (often even down to the specific metro). It also means you’ll be able to optimize your SaaS infrastructure portfolio by network blend and control supply capacity during surges.

3. Stable performance

Multi-tenant virtualized environments introduce unpredictable interference. When multiple businesses share underlying hardware, they can be left vying for limited resource resulting in performance degradation: errors, slower response times, failed requests or even system wide failures with significant end user customer impact.

Bare metal solutions eliminate that risk. The server hardware isn’t shared with other businesses, so there’s no performance implications directly from resource contention. Add the additional scope for hardware customization that comes with being the server’s sole user (think custom CPU, RAM and storage as a minimum) and suddenly you’ve got an architecture that’s carefully tailored to your latency and overarching performance goals.

It's a sentiment that’s increasingly being shared within the SaaS community, reflected in forums like the subreddit r/SaaS where one contributor writes that bare metal is “the secret sauce to lots of stable SaaS platforms earning lots of MRR.”

The benefits don’t stop at latency

Distributed bare metal isn’t just a strategy for latency reduction. It brings wider benefits for SaaS hosting. For example, when workloads are sustained and steady, bare metal deployments are a more cost-efficient option than equivalent public cloud solutions with their on-demand pricing models and complex billing structures.

It’s exactly what e-commerce enablement platform, Dukaan, found. After starting a migration away from cloud to bare metal in 2023, Dukaan cut costs dramatically. Subhash Choudhary, Dukaan’s Co-founder and CTO explains:

 “At every layer of our stack, we were paying a ‘convenience tax’. We were paying AWS for the privilege of not having to manage the underlying hardware.”

To ensure peak performance during peak traffic, the team have proxies in 22 regions worldwide over a distributed network allowing traffic to be directed and managed across different server locations.  

It’s worth mentioning too that this additional economy, in combination with the performance stability afforded by dedicated hardware, is a significant advantage for performance-intensive, high-utilization AI workloads – especially those sustained over long periods of time.

Distributed bare metal architectures also offer significantly more strategic control and risk management. Due to the geographic diversity of these deployments, SaaS teams also have the flexibility to comply with changing data residency laws by demonstrating clear isolation and the ability to keep regulated data in-country or approved regions.

Likewise, when deployed across multiple vendors, the risk of a single provider outage or dramatic pricing hike is minimized, aiding long term system redundancy whilst protecting your bottom line.

An architecture for faster, more sovereign, SaaS

Latency in SaaS is a strategic business consideration and distributed bare metal is an effective strategy for the delivery of faster, more predictable and sovereign SaaS platforms. By putting compute close to users, choosing the exact metros and network blends you need, and engineering out noisy-neighbors, SaaS teams can cut tail latency and improve performance.

The upside extends past performance: steadier costs for steady workloads, real control over where data lives and how it moves and, fundamentally, better user experiences.