Doug McIntyre [log in to unmask]> writes:
>On Sunday, May 5th during 2:00AM-4:00AM we will be upgrading our core
>routing and switching capacity. We will be swapping out older gear
>(Juniper M20s that have worked extremely well for years and years)
>with new current models (Juniper MX80) with much greater capacity.
Unfortunately, this maintenance did not go as planned. The new
switching core stack was put into service, and was passing traffic.
But unlike during its whole test phase in our test network, it was not
behaving well talking with the rest of the network and was randomly
dropping large swaths of packets here and there. Sometimes it would work,
sometimes not. No ports were taking on errors, but yet it was like I had
flakey fiber cables.
At around 2:30AM I cut back onto the old main switching core, which
eventually restored all services back to normal.
I had also planned on cutting onto new our new routers at this time,
but they had been setup to utilize the dual 10G ports on the new
switching stack. I changed my planned migration of these, and
attempted to get things switched over to use some unallocated copper
ports, but I wasn't able to complete the reconfiguration onto 1Gbps
copper ports to match the current switch setup in the allowed maintenance
window time. So this plan was put on hold. I don't want to go beyond the
time allowed.
I did leave myself setup in a better position for further testing though, and
I will have to troubleshoot both the packet loss issues on the switch stack,
(which may negate the new router reconfig), or to finish the router reconfig
bypassing the new switch setup until further testing can be completed.
If you have any problems or questions please let us know at
[log in to unmask], or call us up at 612-337-6340.
--
Doug McIntyre <[log in to unmask]>
-- ipHouse/Goldengate/Bitstream/ProNS --
Network Engineer/Provisioning/Jack of all Trades
