I've had a need recently to learn how 95th percentile works and found a LOT of information and misinformation on the net about it. I'll refrain from commenting much about the politics of billing on the 95th percentile, that's a whole 'nother can o' worms.
So here's the short description:
- Store the 5minute averages of traffic.
- Sort the samples descending.
- Throw away the top 5% of the samples.
- The highest number remaining is what you bill. (usually in mbps)
That isn't the same as calculating the peak bandwidth and multiplying it by .95. It's also not the same as taking the total bandwidth used and multiplying it by .95. The customer is billed on some arbitrary rate of bandwidth sometimes this is good, sometimes not.
At first it seems that you need 5 minute data for every customer for the entire month to do accurate calculations, but there is a way that you only need to keep about 37 hours of data through the month.
A month with 31 days contains 8928 sample periods. The top 5% or 446 of those will be discarded. The 447th will be what we bill the customer on. We can start out the month filling up the 447 slots. When they are full, all future samples are tested against them. If they are greater then #447 they get inserted in the list and 447 drops off.
This reduces our storage costs from 8928 samples per month per server to just 447. At the end of the month we save the 447th slot clear the buffer and start over.
Now I'm off to see if I can code that in SQL.