Network Performance Monitoring

Hey Guys,

I wanted to get some feedback on what you guys are doing for network performance monitoring. Unfortunately for me, and the places I have worked, this has always been one of those "nice to have" things that never made it to the top of the pile.

We're rolling out Cisco UC with the assistance of an external vendor and I don't want to miss this opportunity to get something put in place that will save me a lot of head aches later on. I am absolutely trying to avoid someone coming to me saying "The network is slow" and having absolutely no tools to track down the problem.

Right now we're doing some pretty simple up/down type stuff, and in the past I have setup SNMP to monitor interface utilization but I would really like something that goes above that. Each site will have a netflow enabled router - I don't have any hands on experience with that but I have heard good things.

I guess what I am looking for is the following + any suggestions you might have: Interface utilization & rates Host utilization & rates Trends for host/interface Utilization per protocol

Posted Aug 10, 2011, 3:17 PM UTC

210
jjr798•

You are looking for SolarWinds.

6
puremessagebeep -f 2000 -r 999999•

Our network group uses Solarwinds.

1
telecomando2•

Unless you have no budget then look at ntop with the nflow plug in, cacti, etc. Tons of linux tools to do the job.

3
assangeleakinglol•

I use cacti via SNMP.

2
simulated_conscienceOPIT Manager•

Isn't that just going to give me per interface utilization though?

What sort of granularity does that give me if I'm monitoring a WAN/trunk port?

1
omgwtfbbq69•

I use SolarWinds Orion NPM. It has very good alerting.

2
ehcanada•

I recommend Paessler PRTG Monitor. It is cost effective for an enterprise deployment. Ajax interface is fairly slick and well laid out. Collectors can be distributed in remote data centers to prevent constant snmp read and replies from clogging up the WAN. Collectors are totally set-it-and-forget-it. Everything is presented in one web interface. You can easily adjust snmp intervals down to 10 seconds.

With PRTG you have the option to pull up "per interval" data for periods over 365 days previous. This is important and is often overlooked with other tools. Other tools such as MRTG/Cacti will continually "roll-up" their data into hour and then day and then week. For example, with PRTG if you are preparing some calculations for a DR plan and want to know what the traffic was on your primary Internet pipe was between 8am and 5pm on the last Tuesday before Thanksgiving, you can pull up the actual per interval graph for that period. With MRTG you get to look at a graph of the entire week and try to make some bogus assumptions about peak traffic. That is really useless imo.

PRTG is a pretty decent netflow collector, too. You can see topN stats for 15-minute intervals. Netflow data is difficult to trend. How do you graph who the Top Talker is over a year? You will just see a busy file share as Top Host which isn't really useful. Netflow is something you just go to in realtime during periods of congestion to see who is using the bandwidth. Combined with PRTG per-interval historical data, you can look at bandwidth utilization and take spot samples during peak periods to see what application was using the bandwidth. Unfortunately, a lot of netflow TopN reports will show "TCP other" high in the report. That is due to not defining the tcp service. You can manually define these service ports in PRTG if you know what is being used. Otherwise you are stuck with the well-known ports although some protocols like CIFS/SMB (tcp/445) doesn't seem to be well-known to a lot of monitoring apps.

Orion Solarwinds sux ass. bottom line. The presentation for network devices is that of a server. Emphasis is on cpu and memory instead of interface throughput. It is constant clickey-clickey through that app to display the information important to a network admin. It is slow and doesn't allow you to adjust polling intervals easily. By the time you find the one bit of information your going for, you end up forgetting the big picture and get side tracked. Presentation of netflow data is just as bad.

A couple of things you will want to do is first collect statistics on your QoS queues, primarily of your priority queues. You will want to see if your routers are discarding voice packets due to strict policing of realtime queues. This is difficult since the OID values you want to poll are dynamic based on your QoS configuration. Another thing would be to setup Cisco SLA agents between wan routers and have your Network Monitoring application graph the results. Don't overdue this because you will be introducing simulated voice traffic into your wan which will have priority over actual user data traffic. Also dn't forget your UC server will be collecting all sorts of call statistics which will be station-to-station which is more authoritative for call quality than monitoring individual queues in network devices. You may be able to collect this data from UC using snmp as well. Or, more likely you will be using another software app to collect it.

2
simulated_conscienceOPIT Manager•

This is an awesome response, thanks for taking the time to reply. Definitely going to take a close look at the product. You are right, the pricing is very reasonable.

Edit: Is it easy to correlate network utilization between switches/routers and servers? I am assuming the real time netflow information will provide that. But can I go back 1-month and see a breakdown for wan traffic as x% file sharing, x% backups, etc?

1
ehcanada•

Correlating all of this data is pretty much a manual exercise. To see network utilization from servers you will need to monitor interface counters on your access switches. You have to correlate switch port with the connected host. Nobody cares that Sw2 fa0/5 is avg 110 Mbps all day long. Replace SW2 fa0/5 with "Sharepoint Primary" and it starts to make sense for management.

To correlate netflow data over a longer period than 15 minutes, you probably will have to export the data into csv files and then merge all that data into a single file so you can build some charts in Excel. You can get TopN for applications like Top applications on your WAN are File Sharing, SSH and HTTP.

Some applications that use dynamic port allocations like RTP are not able to be monitored by netflow. If you have Cisco routers, look at monitoring NBAR stats. NBAR is Network Based Application Recognition. It is a Cisco feature in IOS that has some application inteligence that can track applications beyond just service port. For example, for applications like RTP that use a different control protocol (H.225) and then the end points just allocate a dynamic UDP port between 1024 and 65535, NBAR can look at the RTP handshake at the beginning of the flow and determine is an RTP stream whereas netflow will just report UDP/Other. You have to enable "ip nbar protocol-discovery" on the interface connecting to the WAN. But, once your router starts monitoring application you can monitor the nbar stats with snmp. This document from Cisco pretty much give you everything you need to get started with Nbar and Snmp.

2
simulated_conscienceOPIT Manager•

Totally awesome reply, thank you. Wasn't aware of NBAR and I will indeed take a closer look.

1