Dont Panic


The dreaded scaling word

13 Nov 2012
As part of UOS online lessons need to be able to upload small sound clips from the client to the server. These clips are aprox 300k in size and can be sent a received reasonably quickly. So some Node.JS later i have a serve which will receive and server sound clips. Happy days.


Anyone would think that the hard part is over as i have got the functionality i need up and running. but this is the internet and Im going to have to think about what happens when several users are pushing data back and forth to the server and how that will impact on storage space on my server and traffic. each of which will have there very real world costs from a hosting company. ImageImage For now I'm not even thinking about traffic costs but purely storage. Each user must complete 150 reading lessons with an average of lets say 3 blocks of 5 words per lesson. so over the entive course the user will upload 300k * 5 * 3 * 150 =  675Mbytes of data. The average course takes a year so roughly 675 / 365 = 1.8 ish Mb a day. (this is spreading out the load over all days even though a large number of days will be zero traffic, holidays etc) although they will upload 675 Mb of data we will only be holding a window of say 20Mb of data per student so:

Traffic -- 1.8Mb/day/user

Storage -- 20Mb/user

with roughly 5000 active users of UOS we could maybe expect a 5th of these to convert to online in the next year, so:

Traffic -- 1.8Gb/day

Storage -- 20Gb

This is starting to look dangerous, what happens when we catch up with users all online and even grow the total users to say 10000?

Traffic -- 540Gb/month

Storage -- 200Gb

this just shows how a relitively small ammount of code pushed over the network and stored on a server ver soon adds up when you start thinking about users in the thousands.