Serving HUGE Files or HUGE Number of Simultaneous Users from a Linux or Windows 2012 Server.

On the weekend of August 28 to 30, 2015 up
here in the Pacific Northwest (in upper
Washington State and Lower British Columbia)
we had a HUGE windstorm and various websites
went down and people were complaining about
one of the big power companies (BC Hydro)
whose website went down and could not service
the 500,000+ website users to get updates on
power restorations and maps of outages.

In fact our biggest telco (TELUS) caused the
BC Hydro website system to completely CRASH
the ENTIRE WEEKEND...they could not handle
the service load.

That got me thinking as to WHY the website crashed
when the rest of Telus and BC Hydro was mostly up?

After a few news reports we found out some BASIC
computer information services flaws were RAMPANT
thorughout the website services system and that
basic, beginners-mistake just FLOORED ME causing
me to shake my head in asking HOW CAN ANYONE
BE THAT STUPID in setting up your website systems ????

The company I work for does all sorts of video technology,
high tech computing/opto-electronics and other more esoteric
UAV/Autonomous Software systems Research and Development.

We are a SMALL COMPANY yet we seem to be able to serve
our customers with flair and aplomb when it comes to
high-end, if secretive, web communications services.

We regularly serve MULTI-STREAMS of Uncompressed
4K Video files that are 100 GIGABYTES to OVER ONE PETABYTE
in size and yet WE don't seem to have the same problem
at the multi-billion dollar TELUS and BC Hydro companies!

We serve WAAAAY MORE DATA PER DAY THAN THEY DO !!!!
Yet we do all right!

---

So as a HELPER for the newbies in web systems services,
here is a primer on what YOU NEED TO KNOW:

You Must KNOW AND CALCULATE your total available
bandwidth in Megabytes or Gigabytes Per Second.

i.e. One Megabyte = 1,000,000 Bytes

i.e. One Gigabyte = 1,000,000,000 Bytes

Then divide MEGS/per sec or GIG/per sec by the number
of Bytes per web page you are serving and then divide
that by your reported PEAK, MEAN and AVERAGE number
of unique users per second or PEAK, MEAN and AVERAGE
unique users per minute during each hour of the day.

To get this number you need to log how many users actually use
or are PREDICTED TO USE your site during each hour of the day!

That final total is how many users you can serve
per second or minute duing each ONE HOUR BLOCK
within a 24 hour day of web site availability!

Example:

These are the available fibre optic
bandwidths you can buy from any major
telco. The total bandwidth Is the
TRUE deliverable payload in Megabytes
Per Second or Gigabytes Per Second you
can expect AFTER any major switcher/router
and servicing overhead is taken into account.
I didn't do megabits or gigabits per second because
to me I need BYTES, MEGABYTES and GIGABYTES
PER SECOND as a real-world unit of data
speed measure for standardization sake.

OC-1 = 6.264 MB/sec
OC-3 = 18.576 MB/Sec
OC-12 = 75.168 MB/sec
OC-24 = 150.276 MB/Sec
OC-48 = 300.672 MB/Sec
OC-192 = 1.1889 GB/sec
OC-768 = 4.8107 GB/sec

So when I do my planning, I ALWAYS MAKE SURE my servers
and my software and TOTAL AVAILABLE BANDWIDTH can
service AT LEAST 500,000 Unique Users Per Minute
so I nned the number of leased lines and servers
that can handle that sort of load.

For REALLY HEAVY-USE WEBSITES for something like
ABC Sports-type of bandwidth needs, I should be
able to service 3,000,000 to 5,000,000 users per minute
(200 kilobytes per web page) with a few OC-768 lines
with 80% text and 20% photos per web page.

If you are serving STREAMING VIDEO you need to calculate
Frames Per Second multiplied by Kilobytes per Frame which
is used for low-latency high quality INTRA-FRAME video
compression used for times when each frame is sent
live without delay (i.e. during a live satellite feed newscast)
or if you can accept a five-second or 10 second delay
(i.e. not a live stream - a streaming movie or advertisment)
then an INTERFRAME compression codec such as H.264
(CODEC = COmpressor/DECompressor)
can be used so then you need to use
the formula of:

Number of Group-of-Frames in One Second
multiplied by Size in Bytes per Group of Frames (GOP)

With those Number of Bytes per second Allowed
Per User per Second or Per Minute, you then compare
if your network cards, your disk arrays, your server
HARDWARE and OS software and your switches/routers
can handle that sort of bandwidth per second or minute
and are MATCHED to the OC-XXX bandwidth level you
want to buy from your telco.

So use this calculation finding out how many
simultaneous users you can have PER SECOND
for text-heavy websites that use 80% text and
20% pictures where each web-page is
200 Kilobytes (200,000 Bytes) in size:

Maximum Possible Simultaneous Users PER SECOND:
To get users per MINUTE multiply each below by 60.

TEXT HEAVY WEBSITES:
------------------------------
OC-1 = 6.264 MB/sec = 31 Users Per Second
OC-3 = 18.576 MB/Sec = 92 Users Per Second
OC-12 = 75.168 MB/sec = 375 Users Per Second
OC-24 = 150.276 MB/Sec = 751 Users Per Second
OC-48 = 300.672 MB/Sec = 1503 Users Per Second
OC-192 = 1.1889 GB/sec = 59,445 Users Per Second
OC-768 = 4.8107 GB/sec = 240,535 Users Per Second

VIDEO STREAMING WEBSITES:

Serving simultaneous viewers of a 10 Minute Video
compressed using H.264 HQ at 960 by 540 pixels
(i.e. Quarter-HD) suitable for laptop, tablet or
smartphone playback: (1800 Kilobits/sec -with stereo audio)
Total bandwidth per user is 225,000 Bytes per second
x 60 seconds x 10 minutes:

--------------------------------
OC-1 = 6.264 MB/sec = 27 Viewer Per 10 Minute Video
OC-3 = 18.576 MB/Sec = 82 Viewers Per 10 Minute Video
OC-12 = 75.168 MB/sec = 334 Viewers Per 10 Minute Video
OC-24 = 150.276 MB/Sec = 667 Viewers Per 10 Minute Video
OC-48 = 300.672 MB/Sec = 1336 Viewers Per 10 Minute Video
OC-192 = 1.1889 GB/sec = 52,840 Viewers Per 10 Minute Video
OC-768 = 4.8107 GB/sec = 213,808 Viewers Per 10 Minute Video


These are the types of number you MUST be aware of.

Good luck on setting up your text-heavy or streaming video website
using your OWN servers or co-locating or using someone else's servers!

I suggest two Windows 2012 Enterprise Servers containing Tyan Thunder
motherboards (using Four AMD Opteron Processors with 16-cores each)
and 1 TERABYTE of RAM Memory and each motherboard has TWO CISCO
40 Gigabit fibre network cards connected to an internal 40 GB firewall
and router combo and then connected out to a leased
fibre-optic telco connection.

Use a 250 Kilobytes per second QOS (Quality-of-Service) bandwidth throttling
setting per user and you are good to go for serving 960x540 pixel video at the
number of viewers per second as described above for the fibre optic
lines allowed to be leased by most major telcos.

Most Windows 2012 Servers the 4-processor 16 core AMD Opteron chips
with one gigabyte or RAM can handle a load of 10,000 Logged in Users
each claiming 250 Kilobytes per second of Bandwidth --- That's a
Windows 2012 Server limitation not a hardware limitation!

P.S. If your Telco and network Hardware manufacturer uses
Kilobits per Second, Megabits per Second or Gigabits per Second
as a measure of bandwidth, then just divide that number by EIGHT
in order to get KiloBytes, MegaBytes or GigaBbytes per Second.
(i.e. there are 8-bits in ONE BYTE)

a reply to: StargateSG7

"Good" government websites start around $1 billion I hear.

a reply to: greencmp

A similar issue arose with the Obamacare
When various STATE agencies were too
STUPID to do some basic math which
Resulted in website crashes that lasted
For days if not weeks or months!!!

If they had made a few calls to their telcos
And server operators/server suppliers, the
State Obamacare agencies would have been
able to CALCULATE The load of N-number
of state residents spending on average
5 minutes per web page times three times
per month Times X-number of bytes per
page which Must fit into N number of available
megabytes or gigabytes per second of telco or cable internet bandwidth available per county!

If THAT above equation had been filled in
With the appropriate numbers, the Obamacare website crashes would have NEVER happened!

for an investment of just ten hours of phone calls
To some telcos and a few hundred thousand
Dollars on 40 gigabit network cards, firewall
and routers and some 64-core Win-2012 servers
with lots of RAM memory, this whole Obamacare
website crash debacle would have NEVER
happened.

---

A few years ago I saved an agency of our
Provincial government from making a
30 million dollars a year bandwidth mistake
Just by outlining some simple math on how
Much it would cost to implement a web based
Streaming Video training system if 1000,
10,000 and 25,000 users used it for
Just 20 minutes a day.

Just 5 phone calls and 30 minutes of math
Showed that a library of DVDs for $300,000
was Faaaar cheaper than live streams of videos
Which would have cost 30 million dollars a year
To provide on a streaming basis over high speed
Lines province wide!

The original developers of such post secondary
Courseware had NO IDEA about bandwidth costs!
So for less than two hours of my time, I helped
save the taxpayer from a BIG financial sinkhole
. ..the lesson in all this...DO YOUR MATH !!!

Be it for fuel costs, bandwidth costs,
Hourly and salary employee costs
And service or product costs!

You just have to take into account
your minimum, maximum, average, mean
And PEAK PERIOD costs for any given
Product and/or service over a given monthly
Or yearly usage and then compare if your
Minimum and average net or gross revenues
Over a month or year Will COVER your average
and maximum possible costs and still give you
A bit of profit!

If costs outweigh revenues then you
DO NOT DO IT !!!

Simple equation: Total Revenue minus
Total Cost Must always be greater than Zero!

Its that simple!

And for ME a good business rule of thumb
To ensure your personal life and business life
Is kept Sane use the following equation:

( Total Revenue - total costs per month) minus
(160 hours per month x $50 per hour) should
Equal a value equal to or greater than $8,000
per month.

That above equation, if followed and achieved,
Means you have a viable personal profit and
It also means you still have a sane personal
Life and time to enjoy it at an upper middle
Class lifestyle.

Final Lesson: DO THE MATH FIRST !!!!

a reply to: StargateSG7

And remember that we are talking about an organization that can't be trusted to fill potholes.

Government bureaucracies have no interest in efficiency because it undercuts expansion. It is also possible that the streaming service solution was an intentional boondoggle. It is likely that someone was fired for recognizing the problem and fixing it either way.

Sad but true.

a reply to: greencmp

Im my case the higher learning agency
Was utterly floored by my calculations
And didn't believe them at first until
I broke it down by each telco supplier's
Bandwidth cost on a hourly, daily, monthly
And yearly basis. I even made a forecast
Implying that telco internet bandwidth costs
Would drop by 30% within five years.

The amount of video stream data they
Would be sending per month per student
Would have been in the tens of millions per year!

My final suggestion was to buy 300,000 training
DVD's at a buck a piece and put them in the
College libraries for students to check out
On a weekly basis.

So far so good that somebody listened...as a
variation on that system seems to be working
Quite well and was a heck of a lot cheaper
To implement!

And you're right...a number of people were
Canned based not only based on my memos
But also because others pushed ahead
Too fast for web streaming and wasted
A few taxpayer millions on expensive
bandwidth...still nice to know i saved
a few bucks for Joe And Jane Taxpayer!

You really suggest 2012 Enterprise? Where do i buy that?
Just two servers huh?
Not load balancing those web servers? What happens when you want to patch?



V

originally posted by: Variable
You really suggest 2012 Enterprise? Where do i buy that?
Just two servers huh?
Not load balancing those web servers? What happens when you want to patch?

V

If you want Windows 2012 Data Center for 4 processor or more goto here:

www.microsoft.com...

it starts around $6500.00 but the beauty of it is that it is a BREEZE to setup
and maintain your cloud and clients from ONE central location. FAAAR EASIER
than a Linux system --- You setup your Active Directory Domain with tens of
thousands users using a naming structure similar to DOMAIN_USER_00001 to
DOMAIN_USER_XXXXX each using the SAME login script and secured access
privileges set from ONE Domain Access Security Policy which then points
to a master PRIVATE web page for use by users that are successfuly logged
in and authorized to access the domain and it's storage systems or you can
can allow totally PUBLIC access to a master web page which everyone gets
a copy of as soon as they are assigned one of the TCP Ports numbered from
10000 to 65000 which are public access ports which any anonymous user or
public user can access. Using ONE TCP port per user prevents cross-linking
and keeps everybody on a "separate line". Since all web browsers use
Port 80 or 8080 for web page access first, once any unique user hits your
first web page you switch that user to a separate assigned port to keep
their communications private and separate from everyone else!

For your server you MUST get a system that has 4 CPU processors
such as the AMD Opteron 16-core server chips! VERY IMPORTANT!
For each core you assign a ONE thread to handle a block of users
usually setup in blocks of 256, 512, 1024 or 2048 users per CPU core.
This divides the web page and video serving workload amongst
all available CPU cores.

I HIGHLY suggest setting the Windows operating system itself SEPARATE
FROM EVERYTHING ELSE up on a one terabyte SSD Flash drive (i.e. Drive_C
- Use SanDisk or Samsung or Kingston 1 terabyte ENTERPRISE-LEVEL drives)
and then setup your internal webserver applications such as Exchange Server,
or SQL Server, Apache or FTP or Email Services on a second SSD drive called
"Drive_D".

Then Setup a 3rd SSD drive called "Drive_E" which contains each user account
from DOMAIN_USER_00001 to DOMAIN_USER_XXXXX and all their personal files.
I highly suggest assigning a proper DISK QUOTA SYSTEM and BANDWIDTH QUOTA
SYSTEM so each user gets a minimum of 10 Megabytes of personal disk space
up to a maximum of 100 to 200 megabytes of disk space. Set the bandwidth
quota to allow the download of one 200,000 Byte (200 kilobyte) webpage
and one stream of 960 by 540 video max size so use 2000 kilobits per Second
for each TCP/UPD port connection as a MAXIMUM BANDWIDTH QUOTA
per each user. If you are using two 40 gigabit network cards which have
some data overhead for each user you can reasonably attain a maximum
of 10,000 users simultaneously on a VIDEO HEAVY website.

Make sure that when you set your folder privileges for each user
whenever they login, they only get to access their personal storage
space (Read/Write access) and have only Read-only access to the
master global web page. They should get NO OTHER ACCESS
to any other part of any other storage disk or user account!

Finally setup a 4th SSD drive called "Drive_F" which is a MIRROR or BACKUP
of all the USER accounts on Drive_E and using a Server-capable backup program
will back up all user accounts from Drive_E to Drive_F starting at 2:00am each
night and should take less than 40 minutes for one terabyte when you set
the SAFE-COPY/SAFE BACKUP FLAGS.

The server itself REQUIRES a minimum of 256 Gigabytes of
ECC RAM memory (i.e. error correcting memory) but in our
line of work we put in one Terabyte of RAM so that each
user gets 4 Megabytes of working web page memory.

Since our web server software has overhead associated
with it it can NEVER MATCH the available network card
bandwidth BUT you DO GET that extra safety margin
which allows ou to add and subtract users without delays.

We do this for safety-margins sake which means on our system
we only let 250 users PER SECOND simultaneous use our VERY
video-heavy website at a reasonable speed for SUPER HQ/HD
video playback.

I have been able to get away with 200 kilobytes
for heavy-text and small video-oriented websites
which would give me 5000 Simultaneous users
per second and I can reduce to 100 kb per page
in a pinch to get me that 10,000 users per server
per second.

Since MOST users will stay only 2 to 10 minutes on your website
you can server about 20,000 to 60,000 unique users per hour
ON AVERAGE taking into account any network overhead.

If you CUSTOMIZE your Apache webserver using GPU-based
stream processing, I can get as much as 20,000 Simultaneous
video users per Windows 2012 server.

So with two servers you could have as low as as 5,000 users
simultaneous for video heavy websites and up to as high as
40,000 users for text heavy/small video sites with just TWO servers.

The key is that ALL your applications, your server bandwidth, your fibre-line
to the telco , your network card and your switches/firewall MUST
all match in terms of bandwidth. SO I suggest TWO 40 Gigabit network
card in each server!

Wish I had your capital or budget.

originally posted by: staple
Wish I had your capital or budget.

Kinda scarey actually what our company has spent on computer gear.
A few million dollars over the years ... At last count I think we have
somewhere over 5000 hard disks (Petabytes+ of storage!), Custom
Linux and Windows Servers multi-screen 4K display systems, etc.
We deal with multi-stream 4K UNCOMPRESSED VIDEO so we need
that sort of omputing system horsepower.

One system I personally have access to at another
related company has 400+ AMD FirePro 9100 GPU
cards for now over 1.1 PETAFLOPS (or 1.1 Quadrillion
64-bit Floating Point Operations Per Second)
of MASSIVE GPU-based number crunching
horsepower (an upgrade from the
560 TeraFlops we used to have!)
We can DO A LOT with that sort
of supercomputing horsepower!

a reply to: StargateSG7

My personal system has four of the
AMD Firepro 9100 suckers for 8+ TeraFLOPS.

It can now play this new very-realistic simulation game
I have called Global Thermo-Nuclear War...

Would You Like to Play?

...

...

...

Input Your Move Here: ___

Press ENTER to Continue...