What is a Backplane? A Network Backplane Throughput Primer

Bottlenecks and performance issues are the bane of network engineers everywhere. They can be hard to nail down, have a variety of different potential root causes, and give people an excuse to “blame the network”. Understanding network backplane boards, backplane throughput, and concepts like blocking vs non-blocking switches, can help you better understand network design and troubleshoot bottlenecks when they come up.

Unfortunately, the information on the web around network backplanes and backplane throughput can be a bit confusing, or outright conflict with other sources (try and figure out the difference between backplanes and network fabric with a simple DuckDuckGo or Google search). To help solve that problem, we’ll provide you with a crash course on network backplanes, related terminology, and how to monitor backplane throughput.

Before we get started, if you’re not already comfortable with network throughput and bandwidth, we recommend reading Network Throughput vs Bandwidth and How to Measure It before you go any further. If you’re already familiar with the basics, let’s dive in!

What is a backplane?

A backplane, or backplane board, is the circuitry that provides connectivity between the different modules on a switching or routing device. For example, suppose we have a 24-port, Layer 2 managed switch that has three 8-port modules installed. The data bus those three modules are plugged into is the backplane: it provides connectivity between the modules.

Backplanes come in two basic flavors: “active” backplanes, which include some sort of computing power on the backplane, and “passive” backplanes, generally just the physical connections with no built-in intelligence.

Passive backplane boards tend to offer better performance — higher throughput because there is no processing overhead — and reliability — fewer components means fewer points of failure.

The term backplane is often used interchangeably with the term “switch fabric”. However, that’s a bit of an oversimplification as a switch fabric refers to a matrix-style connection topology a backplane could use (in place of or in conjunction with a bus-style topology).

For network monitoring and management, backplane capacity (or backplane bandwidth) is a very important concept. Backplane capacity acts as a cap on the maximum throughput of a network device. Using our 24 port switch example above, suppose each port offers 1Gbps (Gigabit per second) speed, and the backplane capacity is 10Gbps. While you could use ANY of the ports at its theoretical 1Gbps maximum speed, you couldn’t use all of them at the same time because 24Gbps would exceed the 10Gbps backplane capacity.

Note: While our example uses a Layer 2 switch, backplanes and backplane capacity apply to both Layer 2 and Layer 3 network devices.

Differences between backplane throughput vs interface throughput

The differences between interface throughput and backplane throughput might be obvious to most, but let’s make it clear: Interface throughput is a measurement of throughput for a single port (i.e. an interface) on a network device, while backplane throughput is the actual throughput of the entire network device sending and receiving on all interfaces simultaneously.

Using our 24 port managed switch example, each interface (each port on the switch) has its own throughput metric. Meaning: at any given time, port 1 may have 1Mbps throughput, port 2 25Mbps, port 3 0Mbps, and so on, up to port 24. At the same time, the backplane throughput is a single metric for the entire switch.

Blocking vs. non-blocking backplanes

When discussing backplanes on network devices, blocking vs non-blocking backplanes (or blocking vs non-blocking switches) are extremely important concepts. In ideal settings, a network device is able to provide full bandwidth to all ports simultaneously. This would be called a non-blocking backplane.

A blocking backplane, on the other hand, cannot provide full bandwidth to all ports simultaneously. The full capacity of all interfaces is greater than the full capacity of the switch (or other network devices). Our 24 port switch with 1Gbps capacity each but only a 10Gbps backplane is an example of a blocking switch. To be non-blocking, our switch would need 48Gbps backplane capacity.

We know what you’re thinking, “Why 48Gbps? 24 x 1Gbps = 24Gbps… so why do we need 48Gbps to be non-blocking?”. It comes down to the fact marketing folks tend to spec backplane capacity based on transmit (Tx) and receive (Rx) throughput. To get each 1Gbps, the math counts 1Gbps for Tx and 1Gbps for Rx on the backplane. The backplane needs to be able to support the full max throughput of ALL the ports on the network device to be considered non-blocking.

The language of backplanes

Before we move on, let’s define some other common terms associated with network backplanes:

• Mpps (million packets per second). Mpps is a common metric used to measure throughput on switches and routers. Usually, Mpps is measured using the smallest Ethernet packet size possible (64 bytes), plus some overhead.
• Forwarding rate. The forwarding rate of a networking device is its maximum Mpps.
• Wire speed. The hypothetical maximum speed of a physical link. For example, a 1Gbps link can theoretically send 1,000,000,000 bits of data per second over a given medium.
• Oversubscription. A condition where the theoretical maximum capacity of a network device’s links is greater than what it can actually provide. Oversubscription is common when connecting access switches to distribution switches, but can also occur if all the ports on a blocking switch are used.
• Fabric bandwidth. Often used interchangeably with the term “backplane bandwidth” or “backplane capacity”.

You’ll often see it mentioned that a 1 Gbps port has a rate of 1.488Mpps. The math used to derive that number works like this:

• 1,000,000,000 bits/8 gives equals 125,000,000 bytes.
• A 64-byte Ethernet packet + 8-byte header + 12-byte frame interval equals 84 bytes per frame
• So 125,000,000/84 = ~1,488,095 packets per second, or 1.488Mpps

From there, it’s easy to see how that number scales up (e.g. 14.88Mpps for 10Gbps links) or down (e.g. 0.1488 for 100Mbps)

How to determine the capacity of a device’s backplane

Let’s start drilling down to the specifics. Now that you know what backplane capacity is, how can you determine it for a specific switch or router? While stacking switches and network virtualization can complicate things a bit, the easiest method is to check manufacturer specifications. Often, it’ll be in the specifications as “fabric bandwidth”, “backplane capacity”, or similar.

If you know your device is non-blocking, which many modern network devices are, you can also derive your backplane capacity by working backward from the number of ports and their capacity. Suppose you have 24 1Gbps ports on a non-blocking switch, the math is: 24x1x 2 (for full-duplex) = 48Gbps.

If you have a blocking backplane, should you be worried?

What if you look into your backplane capacity and find that you have blocking backplanes and oversubscription scenarios? Is that inherently a problem? Like many things in networking, it depends.

In some cases, like spine and leaf configurations where access switches connect back to distribution switches, oversubscription is textbook design. In other cases, blocking backplanes can become a major bottleneck.

So, if you’re experiencing network performance issues, confirming whether you have a blocking or a non-blocking backplane would be a good step. Or, if you’re looking at redesigning or expanding your network, you’ll want to know if your current devices can keep up or if you need to replace them.

Symptoms related to backplane issues

Issues related to blocking backplanes can be hard to nail down. You’ll see performance bottlenecks, but you might not be able to immediately tell they are related to a blocking backplane.

Here’s a tip: if you’re seeing performance issues, dropped packets, high switch CPU utilization, and you can’t reconcile the problem to a single interface, there is a chance backplane capacity is the bottleneck.

How to measure your device’s backplane throughput

Given the importance of backplane throughput to network performance, measuring and tracking it is an important part of network monitoring. For managed switches and routers, you can often run commands to determine your current backplane utilization. For example, on select Cisco devices commands like “show controllers utilization” can help.

Additionally, network monitoring protocols like SNMP are also useful for monitoring backplane throughput. Managed network devices will often have OIDs (Object Identifiers) that can report backplane utilization.

For example, in the CISCO-STACK-MIB (Management Information Base), “sysTraffic” (OID .1.3.6.1.4.1.9.5.1.1.8) and “sysTrafficMeterTable” (OID 1.3.6.1.4.1.9.5.1.1.32) are common sources of backplane throughput information. Using network monitoring tools, you can track, report on, and alert against these values in most cases.

While many modern switches are non-blocking, oversubscription is still common, and in many cases intentional, part of network design. With a firm understanding of backplane capacity and throughput, you can be more effective in your network design and networking management efforts.

Get your free 14-day Auvik trial here.