NetFlow Basics: An Introduction to Monitoring Network Traffic

What is NetFlow?

NetFlow is a network monitoring protocol, developed by Cisco, designed to capture measurements about the volume and types of traffic traversing a network device. Sounds simple, right? Let’s dive a bit deeper.

To fully understand what NetFlow is and why it’s used for network monitoring, we first need to know what a flow is.

When computers need to talk to one another, they establish communication channels, commonly referred to as connections. (Technically speaking, these communication channels can only be called connections when the TCP protocol is involved.) A flow refers to any connection or connection-like communication channel.

In more technical terms, a flow is defined by its 5-tuple, a collection of five data points:

• The source and destination IP addresses exchange information
• The source and destination ports, if any (ICMP, for example, doesn’t use ports)
• The protocol

Flow identifies a communication channel, and all packets sharing the same 5-tuple fields belong to the same flow.

NetFlow is functionality built into network devices that collect measurements for each flow and exports them to another system for analysis. NetFlow captures a number of details, including the timestamp of a flow’s first and last packets (and therefore its duration), the total number of bytes and packets exchanged, and a summary of the flags used in TCP connections.

By collecting and analyzing this flow data in a NetFlow analyzer, we can learn details about how the network is being used. Flow analysis is helpful in troubleshooting network issues, identifying bandwidth hogs, and tracking which external IPs or countries you’re exchanging data with.

The evolution of NetFlow

NetFlow was originally introduced in Cisco routers in 1995 as a software technique to summarize network flow data for packets routed over Cisco equipment. Originally intended for use on LANs, it didn’t scale well for high-bandwidth connections and was eventually replaced by another technique called express forwarding.

Cisco realized, however, that having network flow data was very useful, and moved to implement NetFlow in network hardware instead. Over the years, NetFlow has become the de facto industry standard that other vendors have since imitated, using slightly different names for their flavor of flow analysis to avoid trademark issues. For example, Juniper offers J-Flow, Huawei offers NetStream, and sFlow is a multi-vendor offering. (sFlow is actually quite a bit different from the rest, which we break down here in sFlow vs Netflow)

In fact, flow-based monitoring protocols became so popular that in 2008, the IETF released IPFIX, which is now the official industry standardization of NetFlow.

NetFlow itself has also followed an interesting evolution over the years. Starting as a static protocol with a fixed set of statistics collected for all flows, it’s now extensible. In version 9, the latest version in 2021, you can choose which statistics to enable, and vendors can implement extensions to attach proprietary information to flow entries.

IPFIX is sometimes described as NetFlow v10 because it’s so similar to NetFlow v9.

We’ll continue to refer to NetFlow in this article because it’s still the most widely used standard, but everything we discuss pertains generally to all other flow protocols.

How does NetFlow work?

Using NetFlow requires three pieces:

1. Flow exporter: an appliance or network device (usually a router or firewall) in charge of collecting flow information and exporting it to a flow collector.
2. Flow collector: an appliance or server that receives exported flow information.
3. Flow analyzer: an application that analyzes flow information collected by the flow collector.

Generating NetFlow data starts at the network device when a packet arrives. First, the device checks if the packet’s 5-tuple is present in a table of recently seen flows called the flow cache.

If the 5-tuple is in the table, the cache entry is updated: packet count is incremented by one, the byte count is increased by packet length, and so on. If the flow isn’t in the cache, it means the packet belongs to a previously unseen flow, so a new entry is added to the table.

Of course, the whole point of NetFlow is to export information to a collector for storage or analysis. For this reason, the flow information is periodically exported to the flow collector in a process known as flow expiration.

A specific flow entry expires under two scenarios:

• Inactive timeout. If a flow is inactive—no packets have been observed for it for a while—it’s assumed the flow has finished and the entry expires. Typically, the timeframe for inactivity is configured to 15 seconds.
• Active timeout. If a flow remains active for a certain period of time, it expires. The default for an active timeout on many platforms is as long as 30 minutes.

An active timeout may seem counterintuitive, but the timeout exists so that the flow analyzer can get information for long-running flows sooner rather than later. Long flows tend to be “elephant” flows that carry large amounts of data, and learning about their existence only after they finish is counterproductive. A forever-running flow would never expire and would thus escape detection!

As well, many devices come programmed with active timeout values that are less than ideal. Thirty minutes, even as little as five, is too long. We recommend using an active timeout value of just one minute, so the flow analyzer is frequently receiving information about the network and can present a more accurate representation of what’s happening on it.

Once the flow collector receives a flow entry, it forwards it to the flow analyzer, which extracts and presents relevant insights.

What can you see with NetFlow?

Every analyzer extracts different pieces of information from the incoming flow data. With TrafficInsights from Auvik, you can see:

• All flows across flow-monitored devices
• Traffic by application, protocol, domain, source, and destination IPs and ports
• Top addresses, conversations, and autonomous systems
• Sources and destinations by geolocation

Having this information is important, but what you can do with it is what really matters. This data can help you answer questions like:

• Who’s using banned applications, like BitTorrent?
• Who’s hogging bandwidth and slowing down the network?
• Why your web server is receiving so many connections from North Korea?
• What was a hacked server connected to during an infection?

Why use NetFlow?

Simply stated, you should use NetFlow because it gives you deep network visibility. Network traffic analysis is one of the most commonly deployed network visibility tools in the IT service management world.

It’s also much more advantageous than many of the common alternatives. These might include deep packet inspection, which we discuss below in the limitations of NetFlow, and active monitoring, which introduces an unnecessary load on the network.

The most obvious advantage to using flow analysis is that you probably already have it. You almost certainly have NetFlow or IPFIX support on your network devices already—you simply need to activate it and point its output to a collector.

If not, installing flow analysis is easy and relatively inexpensive. Extra hardware is almost never needed. Configuration is limited to a few nodes on the network and can be completed in a few minutes with zero downtime. Quickly add flow analysis to the network and instantly gain vital insights into all traffic flows.

In the diagram below, configuring NetFlow on the main router connected to the internet (R1) lets you monitor all the traffic in and out of the company, and watch for any attacks on the servers in the DMZ. This is typically referred to as north-south traffic.

If you enable NetFlow on the main switch connecting the different departments (S1), you also gain instant visibility into all internal network traffic as well, typically referred to as east-west traffic.

NetFlow is low overhead (and that’s a good thing)

The bandwidth needed to export NetFlow data is typically less than 0.5% of your total bandwidth consumption. For example, if you’re monitoring a link with 100Mbit/s usage, the router would consume an extra 0.5Mbit/s to export the NetFlow data.

If bandwidth usage is a concern for you, most vendors offer a feature called sampled NetFlow. Sampled NetFlow is the statistical sampling of packets: one packet from a certain range is processed by NetFlow and the rest are skipped. Statistical sampling preserves most of the important flow properties, as long as the analyzer is aware that sampling is enabled.

Certain router models can’t keep up with full NetFlow computing, and implement sampling as a way to reduce CPU load. If your network moves large volumes of traffic and the devices on it support sampled NetFlow, it’s a good idea to enable it.

As a rule of thumb, sampling 1 out of 64 to 1 out of 100 packets should be safe for gigabit-scale bandwidth usage. 1 out of 1,000 to 1 out of 10,000 packets is recommended for multi-gigabit to multi-terabit usage. Your equipment vendor’s documentation or website should have further guidance.

NetFlow vs. SNMP

SNMP, or Simple Network Management Protocol, has been the de facto standard for network monitoring for the past few decades and is now in its 3rd major revision, SNMPv3. Similar to NetFlow, SNMP has a concept of agents (analogous to a flow exporter) and managers (analogous to a flow collector).

NetFlow and SNMP have some similarities, but it’s where they differ that’s most interesting. While they can both be used to monitor the throughput of a network, only NetFlow provides visibility into the what and the where of the traffic.

There is a bit of a difference however with data collection. With NetFlow, there is only one way to get data from the flow exporter to the flow collector. The flow exporter actively sends data to the flow collector. SNMP, on the other hand, has two ways of getting data from an agent to a manager. SNMP can utilize traps, a push notification from the agent to the manager, or SNMP polls, a pull request to the agent, initialized from a manager. SNMP polling is the primary way most network management systems collect performance data.

Another major difference between NetFlow and SNMP is that NetFlow focuses only on traffic passing through a network device. SNMP allows network administrators to collect data on other device performance metrics, such as CPU and memory utilization, and even on hardware health information – like fan speed and temperature.

So does this mean that SNMP is better than NetFlow? In a word, no. NetFlow and SNMP both have their place in a comprehensive network management strategy and in some ways can complement each other nicely. Where SNMP falls short of giving network admins visibility into the what and the where of the traffic, NetFlow is there to fill in that gap.

Limitations of NetFlow

While NetFlow can be used to substantially increase the visibility a network administrator has, it’s not without its shortcomings. I would always recommend network administrators enable NetFlow as part of their network performance monitoring program, but keep these limitations in mind!

NetFlow identifies the device but not necessarily the user

I mentioned that a flow includes the source and destination IP addresses exchanging information, which is a big leap in visibility for a lot of network admins. The natural extension to this, and a question I get all the time speaking to network admins, is, “Does that mean I can identify the user?”. With NetFlow alone, the answer is no. While NetFlow allows network administrators to track down the device sending or receiving the traffic, it’s only looking at the network traffic and not the user logged into that device. There are ways you can get details on the user, however. Some network management systems further integrate into authentication platforms, like Microsoft’s Active Directory, to identify the user logged in to a specific device, integrating the visibility into what user is producing or consuming that traffic along with the device.

NetFlow records the IP address at the time of data collection

When the source and destination IP addresses are collected as part of the flow record, they are done at the point of data collection (the flow exporter). Why is this important? Well, if you use Network Address Translation (NAT) in your network, and do the NetFlow collection after you’ve NAT’d the traffic (on the ‘outside’ interface of a firewall), you won’t see the internal IPs associated with that traffic. I’ve been in this spot once or twice, trying to troubleshoot who is consuming all that bandwidth when every flow says “It’s the firewall!” You’ll want to ensure you’re always collecting flow traffic from an interface where you’ll see your internal IPs before it is NAT’d.

NetFlow doesn’t see inside the flows

The 5-tuple of information collected by NetFlow is actually extracted out of the header of an IP packet. While the header of the packet is useful to tell a router where a packet came from and where it’s going, it actually doesn’t do a great job of telling a network administrator what’s inside the packet. In today’s world where so many applications have moved to the cloud (with most of that traffic flowing through port 443), it can become difficult to accurately pinpoint the source of a network performance issue.

The solution? Many organizations look to Deep Packet Inspection (DPI), including TLS inspection, as a way of peering inside the network traffic to analyze it for security threats and policy compliance. There are challenges with DPI, however, including:

• The inspection appliances need to sit in-line with the traffic and add an additional failure point.
• There are privacy implications to doing DPI on all traffic.
• It can be cost-prohibitive for some organizations.

For these reasons, DPI tends to be overkill on most networks. Fortunately, there are solutions in the market that combine the best of both worlds, like Auvik TrafficInsights. TrafficInsights acts as a flow collector and flow analyzer without needing to sit in-line and decrypt traffic, but rather leverages machine learning and advanced traffic classification to identify what types of applications are traversing your network, without needing to see inside the packets.

NetFlow can be used for monitoring, but not active management

What happens when your Flow Analyzer spots something bad in the NetFlow traffic? Nothing. While this is not really a limitation (NetFlow was never designed to do this), it is something to be aware of. There are other security tools, such as an Intrusion Prevention System (IPS) that are better equipped to spot and block threats in network traffic.

NetFlow doesn’t tell you where a device is connected

The data collected through NetFlow provides you insight into the source and destination IP addresses, but often when troubleshooting network issues, that’s only half of the battle. Once you know what IP address is causing a spike in network traffic, how do you track it down and stop it? If you’re not in control of the endpoint producing the traffic, say it’s on a guest network or an unauthorized device on your production network, it can be difficult to stop. Often the most direct action to take is to shut down the switch port that device is plugged into. But again, how do you figure that out?

While there are some manual ways to track down where the IP address is connected, having an automated network topology mapping tool at your fingertips can track down that device in seconds, getting the rest of your users back up and running quickly.

---

NetFlow is a powerful tool for monitoring any network. With its simplicity and ability to collect a wide array of detailed metrics with little overhead or cost, you can use NetFlow to inspect, assess, plan, troubleshoot, and secure the network.

Are you new to NetFlow, or looking for a new NetFlow monitoring tool? If so, check out Auvik TrafficInsights^TM. Start a free trial now, and use Auvik free for 14 days.