Better 1x10 than 10x1 revisited

After reading “Better 1x10 than 10x1" you're convinced you need a big server. Then you go to a big enterprise computer shop and ask for one system with “10” units of processing capacity. The seller shows you two different models: one called "10x1", equipped with 10 processors of relative “speed” equal to 1. The other one, called "1x10", equipped with only 1 processor but 10 times faster. Both systems have a certificate that indicates they have 10 units of capacity. And both server prices are equal. Which one would you choose?

First of all, you must be aware that the most used measure to express the capacity of any server, and in general of any service center whatever it be, is the maximum units of work (UOW) per unit of time (UOT) it can service (Footnote 1). This is the maximum throughput it supports, also called the bandwidth. In our particular case, both servers have the same value: 10 UOW/UOT. So this is not going to help us in our election of the server. Know the maximum throughput is necessary, but not sufficient.

To find the answer let’s switch to another point of view: we move away from the overall behavior, represented by the benchmarked maximum throughput, to the individual perception of performance, represented by the response time. Does a particular user experience the “same” performance in both servers?

Suppose that 1 customer (requesting 1 UOW) arrives to an empty (100% free) server. In such a circumstance, the customer sees not queue ahead, does not have to wait, and receives immediate service. In the 1x10-server case it spends, let's say, 10 units of time, receiving service (this time is called the service time S). In the 10x1-server case it spends 1 unit of time, just because the processor speed is 10 times faster.

This makes a difference, doesn't it? In the low load zone, where queuing seldom occurs, the 10x1-server shows an obvious advantage over the 1x10-server, due to the increased speed of its processor.

In the high load zone, there are a stream of users (UOWs) arriving and long queues build up. The user has to patiently wait for the queue to advance until it is his turn. So a user request spend much more time waiting to be serviced than being serviced. The time spent in queue, waiting to be serviced, is called the waiting time (W). This waiting time, being much greater than the service time, dominates the user experience.

The waiting time is essentially the same for the two kind of servers. If you arrive to a waiting line and see 1000 customers waiting ahead of you, you’ll have to wait the same if the queue advances 10 customers every 10 units of time, as is the case for the 10x1-server, than if the queue advances 1 customer every 1 unit of time, as is the case of the 1x10-server. This means that in the high load zone both type of servers essentially deliver the same individual user performance.

Summarizing: in the low load zone the 10x1-server is the winner, and moving away into the high load zone the 10x1-server advantage gets blurred. And at the end, where the very high load zone is reached, both servers are equally performers.

You've now a good reason to prefer one kind of server, the 10x1-server, to the other, the 1x10-server: the response time will be better in the low load zone. This may or may not be significant in your particular case of interest. It should be more noticeable in long running not parallelizable tasks than in short running ones, but the fact is there and you should be aware of it.

Server vendors typically put a premium price on 10x1-servers. I would like to think that this increase in price is because of its better performance, but I suppose that the real reason relies on higher research & development, and production costs of faster processors and components.

A final word here: in the previous “Better 1x10 than 10x1" blog entry we exposed a reason to prefer bigger servers: the random nature of user arrivals or demands. And here we've seen that it is better to achieve the big capacity with less and faster processors. The pattern “10x1 better than 1x10” repeats, but from a different point of view.

Think about it.

Footnote:

(1) Units of work (UOW) and units of time (UOT) are generic units. In the computer system world they can express, i.e., transactions per minute or tpm (UOW=transaction, UOT=minute), IOPS (UOW=IO operation, UOW=second), and similar. But it is generally applicable to other areas: public service desks (UOW=certain customer request, UOT=minute), highway tolls (UOW=cars passing & paying, UOT=minute), and so on. The only limit is our imagination.