Hello, my name is Bil MacLeslie and I'm the CEO of ipHouse.  I want to 
personally apologize for todays outage.  I'm sincerely sorry this happened 
and affected so many of our customers.  I also want to thank all of our 
customers for their patience while we restored power and got things back to 
normal.  Thank you!

This morning at approximately 8:51am a 480v circuit breaker tripped, 
shutting down power to an integrated 480v/208v transformer & PDU (power 
distribution unit).  This caused a power outage for some of our colocation 
customers who are fed power from this PDU.  At 9:32am, power was restored 
to the PDU and by 9:56am all circuits were back on.  I personally staggered 
powering circuits back on to watch for an overload condition.

The NEC (National Electric Code) requires that the maximum circuit load be 
80% of the rated capacity of the circuit breaker, e.g. 16A maximum on a 20A 
breaker.  We regularly take power readings from each circuit and maintain 
power utilization within the requirements of the NEC.  When reading power 
utilization today, we found nothing that would cause the 480v circuit to be 
overloaded.  All indicators show that the circuit is not overloaded, nor is 
the PDU overloaded.  The true cause for the breaker being tripped will 
likely never be known, but does not appear to be from overloading.

Because of this outage, some customers have had equipment failures that are 
indicative of a voltage surge on the system.  The UPS and PDU system is 
protected by a transient voltage surge suppressor (TVSS) and we are 
investigating the aspect of a surge further.  We are also currently 
investigating to see if the UPS contains data that may shed some light on 
the situation.  As we get more data, we will provide it to you.

One last note.  Moments like these are trials that test the mettle of the 
people who make this operation run.  My thanks goes out to my staff, who 
answered the calls, took the heat and made things work under stressful 
conditions.  Thank you.

If you have any questions for me, please send them to [log in to unmask]

Support can be reached Monday thru Friday from 8:00am until 8:00pm,
Saturdays from 11:00am until 4:00pm via phone at 612-337-6340, or via
email at [log in to unmask]

-- 

Bil K. MacLeslie - TGWGTD - ipHouse   http://www.ipHouse.com/