BlogBest practices for scalability from an economical perspective.

In Xentime, over many years we have accumulated experience that I would like to share in series of articles.First of all, you must be aware of pitfalls in the clouds, which are popular these days. See article:
dangers of the clouds.If you took a right decision to build your own platform, you start from analysis of data storage options, because this is most important.In simplistic terms the issue look trivial: you add HDDs with growth of load and you replicate everything.
But how to measure stability and efficiency of such system and how to predict expenses on scalability ? A question.Turns out that HDDs were not meant for long term data storage. Engineers from Seagate, IBM and other service centers, who are in charge of data recovery know that very well.Also, according to Dirichlet principle, probability  of the absence of failure approaches to zero with growth of HDDs number. By the way, take a look at statistics from Google: «Failure Trends in a Large Disk Drive Population». They say, that after the first scan error, IDE PATA drives are 39 times more likely to fail within 60 days  than drives without scan errors. After the first reallocation, drives are over 14 times  more likely to fail within 60 days than drives without reallocation counts.  I would use IDE PATA drives, because there's no statistics of failure rates on large population of SCSI drives . Also they consume more power: 12 watt ( IDE drive consume 7 watts ).Conclusion: you need cheap IDE PATA hard drives.
CPUs.According to Amdahl's law, theoretical maximum speedup of system, using multiple processors
calculated with the help of this formula:

where S -- the number of processors
and P -- the result of improvement in percents. So, with 4 CPU's we have:
    1/(0.5+((1-0.5)/4)) = 1.6
with 8 CPU's: 2.9
 
That will give you an idea how number of servers will grow.
Exponentialy, because when you add another processor to server you get small improvement of performance.
 

Power ConsumptionCPU consume 50 watt approximately
DDR3 consume about 8 watts per DIMM
IDE PATA drive – ~7 watts. 50 + 8*2 + 7*2 = 80 watt

17 July, 2009