 |
 |
Hi everyone- FYI in case others are impacted. We had many reports inside AS3128 of problems with Google services [AS15169] beginning around 15:47 CST. I won't bore folks with all details but in short, we observed what looked like a hashing issue. For example, our ip w.x.y.z could ping 8.8.8.8, but not w.x.y.z+1. I was able to reproduce easily myself with 8.8.4.4. Our primary outbound path to 8.8.4.0/24 was MICE. Swinging our outbound path didn't fix anything, but I was able to work around our issues at ~17:38CST by sending 0:15169 to the MICE RR. The relief was basically immediate, so the problem appears to have been inbound from Google via MICE. Hard to say where the issue is/was. I am on a remote switch, but no other issues with other MICE ASN were reported inside AS3128, FWIW. I would be interested to hear if anyone else had/has a problem or if anyone has a suggestion on how to make positive contact with Google on this as we are not direct GGC or direct PNI users. For example, is [log in to unmask] my best/only bet? I know some have suggested IRC in the past, I do not run in those circles but I guess I will learn to if that's my only real option. Thanks- -Michael