While debugging a problem with a customer, one of our routers that
handles colocation and our staff segment went unresponsive after we
turned on debugging on the unit.
This caused an approximate 15m outage while some physically went to
the system and power cycled it.
This unit had been online for the last 19-23 months without a reboot,
I don't know the exact uptime as we just rebooted it, but it had been
a long time.
Postmortem is that the debugging brought the CPU load on the router to
100% and it no longer was processing any packets, of any kind.
I am sorry that this occurred while I was debugging something for a
client.
This outage affected colocation customers, managed infrastructure
customers, and our staff network.
All other services, including clustered email, clustered web hosting,
DNS, etc, were all online during this outage, as well as dialup and
DSL, T1, and metro optical ethernet connectivity.
Support can be reached Monday thru Friday from 8:00am until 8:00pm,
Saturdays from 11:00am until 4:00pm via phone at 612-337-6340, or via
email at [log in to unmask]
--
Mike Horwath [log in to unmask]
ipHouse - Welcome home!
