On a partially related note, I think we should keep all of these little technical details in the back of our mind for when we have nonprofit status, and can collect the money for a larger switching platform.  I think they can really influence the scalability, redundancy, as well as ease of monitoring/troubleshooting.

-----Original Message-----
From: MICE Discuss [mailto:[log in to unmask]IPHOUSE.NET] On Behalf Of Andrew Hoyos
Sent: Tuesday, September 20, 2016 9:31 AM
To: [log in to unmask]
Subject: Re: [MICE-DISCUSS] EX4200 (1G Switch) Packet Loss -> 10G Participants

Option #1 seems to be the most logical (with a 2x10g LAG, split between the two 4500/4550 switches even MCLAG style). That removes that traffic from the VC paths and simplifies that side of the house.

I’d even pony up this EX-UM-2XFP I have sitting here unused.

Are we graphing the VC links?

SNMP support for that came in 14.something, otherwise, there is a SLAX script that can put values in the util mib:
https://kb.juniper.net/InfoCenter/index?page=content&id=KB27711&actp=search
https://github.com/dgarros/juniper-ex-vcp-to-mib/blob/master/ex-vcp-to-mib.slax

I know that the interfaces themselves are clean, as Doug showed us, but that doesn’t rule out microbursts that are maxing them out.

--
Andrew Hoyos
[log in to unmask]



> On Sep 20, 2016, at 9:21 AM, Jason Hanke <[log in to unmask]> wrote:
>
> We've had pretty significant growth (20+ Gig) over the last year or so.
>
> <graph_image.png>
>
> Most of this traffic is CDN->Eyeball. The CDN networks are taking increasingly larger connections creating a large delta between the largest and smallest connections on the switch. A 4x10G LAG can put a pretty big dent in the VC links on its own.
>
> I have two ideas-
>
> 1) Reconfigure the 1G switch to have a LAG group back into the main switch.
> 2) Offer community strings letting the CDN networks send traffic to the eyeballs without traversing the VC links.
>
> Jay
>
>
>
> On Tue, Sep 20, 2016 at 8:29 AM, Doug McIntyre <[log in to unmask]> wrote:
> No, that isn't the case, it is a dual ring, so there are paths
>
> FPC0->FPC1->FPC2->FPC0
> FPC0->FPC2->FPC1->FPC0
>
> There are ways to tell which is the "active" path through the rings, as I
> stated before, I think one path is mainly "active" and one is mainly passive.
> I don't know if just disabling one VCP port will work.
>
> But I guess there is a KB on it.
> https://kb.juniper.net/InfoCenter/index?page=content&id=KB17821&actp=search
>
> So, we could failover the active path to the other ring.
> Or set one of the VCP ports to disable.
>
> Should we do a 'virtual-chassis active path failover' event sometime?
>
>
> On Tue, Sep 20, 2016 at 08:07:00AM -0500, Jeremy Lumby wrote:
> > So if I understand correctly from your "show virtual-chassis active-topology" below, all traffic from the 4500 to the 4200 is going via the 4550?  And if that is the case, is there an easy way in the software that we could tell it to have return traffic from the 4200 to the 4500 also go via the 4550?  This would allow us to easily test to see if it is the specific cable that runs from the 4200 to the 4500.
> >
> > -----Original Message-----
> > From: MICE Discuss [mailto:[log in to unmask]IPHOUSE.NET] On Behalf Of Doug McIntyre
> > Sent: Tuesday, September 20, 2016 12:24 AM
> > To: [log in to unmask]
> > Subject: Re: [MICE-DISCUSS] EX4200 (1G Switch) Packet Loss -> 10G Participants
> >
> > On Mon, Sep 19, 2016 at 11:01:00PM -0500, Jeremy Lumby wrote:
> > > This leads me to wonder if there is possibly an issue with the stacking modules, or they are being saturated.
> > ...
> >
> > Your understanding of the vc port speeds is correct.
> >
> > The VCP ports are built into the EX4200 and EX4500. The EX4550 has addon
> > cards that do those functions. We can devote a port to VCP functionality
> > instead, but it'll be a 10G limit instead of 32G
> >
> > The VCP ports are 32Gbps single direction in each port. So in the dual-ring
> > the cables have are 32Gbps in each direction.
> >
> > You can see which ports are connected to what. (FPC0 is the EX4500,
> > FPC1 is the EX4200, FPC2 is the EX4550).
> >
> > show virtual-chassis active-topology
> > fpc0:
> > --------------------------------------------------------------------------
> >   Destination ID        Next-hop
> >
> >   1                    2(vcp-1.32768)  1(vcp-0.32768)
> >
> >   2                    2(vcp-1.32768)
> >
> > fpc1:
> > --------------------------------------------------------------------------
> >   Destination ID        Next-hop
> >
> >   0                    0(vcp-0.32768)  2(vcp-1.32768)
> >
> >   2                    2(vcp-1.32768)
> >
> > fpc2:
> > --------------------------------------------------------------------------
> >   Destination ID        Next-hop
> >
> >   0                    0(vcp-255/2/0.32768)
> >
> >   1                    1(vcp-255/2/3.32768)
> >
> >
> >
> > You can monitor each of the VCP ports on each of the members. I don't think
> > we are close to saturation.
> >
> > request session member 1
> > (ie. rlogin to the EX4200 switch).
> >
> > monitor int vcp-0
> > Interface: vcp-0, Enabled, Link is Up
> > Encapsulation: Virtual-Chassis-Interface, Speed: 32000mbps
> > Traffic statistics:                                              Current delta
> >   Input bytes:               25685118521                               [12138]
> >   Output bytes:              27856843010                               [13204]
> >   Input packets:                33377693                                  [16]
> >   Output packets:               33371644                                  [16]
> > Error statistics:
> >   Input errors:                        0                                   [0]
> >   Input drops:                         0                                   [0]
> >   Input framing errors:                0                                   [0]
> >   Carrier transitions:                 0                                   [0]
> >   Output errors:                       0                                   [0]
> >   Output drops:                        0                                   [0]
> >
> >
> > vcp-1 on FPC1 is much less usage. Still zero error stats. All switches
> > show zero error stats on any of the respective VCP port status.
> >
> > I suppose further troubleshooting could be to replace the VCP cables
> > one by one coming out of the EX4200.
> >
> > Its too bad that we've got the quad fiber expansion in the EX4200,
> > another test could be to put the dual 10G expansion in it, and make
> > those VCP ports and make the VC fabric over those ports instead. But
> > we've got 3 member ports lit on that quad 1G fiber.
> >
> >
> >
> > --
> > Doug McIntyre                            <[log in to unmask]>
> >                     ~.~ ipHouse ~.~
> >        Network Engineer/Provisioning/Jack of all Trades
>
> --
> Doug McIntyre                            <[log in to unmask]>
>                     ~.~ ipHouse ~.~
>        Network Engineer/Provisioning/Jack of all Trades
>
>
>
> --
> Jay Hanke
> CTO
> Neutral Path Communications
> 3 Civic Center Plaza, Suite 204
> Mankato, MN 56001
> (507) 327-2398 mobile
> [log in to unmask]
> www.neutralpath.net
>
> To unsubscribe from the MICE-DISCUSS list, click the following link:
> http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
>