Server Provisioning

Here is what a server provisioning task looks like in our internal system. Time from start to end is between 25 and 40 minutes. Though the process seems very straightforward, we've put significant effort to deliver the best experience to our customers with it.

Server select

During this process, we choose the server we are going to provide to the client. We have several criteria for this selection:

First of all, the server should match the ordered configuration and be available for installation; A firmware update should have been run on this server as recently as possible. To ensure that, we have an automated firmware update procedure that regularly runs on every server not leased to customers, and we deliver the freshest firmware guaranteeing fewer bugs and more stability; If the customer already has servers in the same location, the new one should be as close to them as possible in order to ensure minimal latency in the private network; All other things being equal, the server should be in the least occupied rack. In other words, we allocate servers the POD using the sparsest strategy. Thus, we provide each customer with the biggest possibility to expand within the rack where their servers are, again, ensuring minimal latency.

Server turning on

It may be hard to believe, but many hosting providers still keep servers turned on regardless of whether they are really leased to the customer. In Servers.com that is not the case. All servers are kept switched off while they are not leased - except for the short periods when the firmware updates occur. Apart from being green, that's how we save costs, and reduce the wear of a server equipment. So, before we provision a server to a customer, we turn it on.

Switch groups

That's a technical step - we determine, to which switches the server is connected - each server should be connected to two switches in the private network, two in the public one, and one in the out-of-band management network.

CPN allocation

When a customer gets the bare metal server for the first time, we have to notify every region of Servers.com Cloud that a new customer has joined, and a new private network should be allocated. Cloud Private Network allocation step in a provisioning algorithm is introduced to make sure, customer gets private network connectivity across all data centers not only between his bare metal servers, but between the bare metal and cloud servers as well.

Private, public, out-of-band management network and alias IP allocation

These three consequent steps are pretty much self-describing and straightforward: provisioning system allocates networks to be used on a server. However, even here, we found room for optimization. By default each customer has a /23 subnet assigned in private network from the general private network pool 10.0.0.0/8. We allocate these /23 networks with the most sparse strategy - that means, that when the customer needs more than his default network, we will most likely be able to expand it by just changing the netmask - and all customer's servers will still remain in a single IP subnet.

Temporary network setup/teardown

In Servers.com we install operating systems automatically, using PXE. in order for PXE to become available we have to temporarily alter the network settings for the server being installed. In order to achieve network redundancy we use link aggregation - which is incompatible with PXE, and thus has to be temporarily disabled while OS is being installed.

OS Installation

Operating systems are installed with official installers. Network settings and disk partitioning chosen by the customer are pushed inside the OS with an automatically generated kickstart file. While all the steps above take several minutes, this stage is the longest one, taking up to 90% of the total server provisioning time. Installation from scratch means the customer always gets latest versions of all packages from official repositories. This way we ensure security from the first second of server lease.

Availability check

When the operating system is installed, we want to make sure it has been installed correctly. Before giving server to the customer, we want to make sure it is up and running - that's why we check it is available via all three networks - public, out-of-band management, and private network.

VNC connection setup

Web-based VNC console provides access to server's monitor and keyboard directly from Self-Service Portal. During this stage, we set up a websocket-to-VNC proxy to connect to the specific server. VNC access to the server itself is available due to iDRAC Enterprise - a KVM-over-IP solution for Dell servers.

Notification

This is the last stage of provisioning. We're sending an email with access credentials to the customer, and only once this email is sent, we start billing of the server.

Suggested Articles