In separate incidents this past month, I’ve helped clients troubleshoot network problems that turned out to be due to misconfigured switches. In all cases, the errors turned out to be things that I don’t think should ever have been changed from their default settings.
So I thought it might be useful to have a brief discussion about how switches work and what features should or should not be used in normal office environments.
MAC learning
The first feature to talk about is called MAC Learning. An Ethernet switch uses Ethernet MAC addresses to decide how to forward frames. At the Ethernet layer, we tend to refer to packets as frames, but they’re approximately the same thing.
A MAC address is just a 48-bit sequence that uniquely identifies the device. Standard IPv4 addresses are 32 bits long, so we can uniquely identify many more devices using MAC address than using IP addresses.
IPv6 addresses are 128 bits long, but the most common IPv6 address structure actually includes the MAC address to help simplify dynamic address generation.
Every frame has a source MAC and a destination MAC. The switch looks up the destination MAC in a dynamically generated internal table and forwards the frame out the port associated with the MAC. The switch learns these MAC addresses by simply listening.
Whenever it receives a frame from a port, it looks at the source MAC. If it doesn’t already have that address in the MAC table, it creates an entry that associates the address with the port.
This process is called MAC learning.
There are a couple of special cases, though. First are broadcasts and multicasts. These are special MAC addresses.
The broadcast address is just 48 bits of all 1s. A multicast MAC address, by convention, begins with three special bytes: 01-00-5e. This is then followed by three more bytes (24 bits) that generally map to the multicast IP address, and they work with both IPv4 and IPv6.
Broadcasts, by definition, go to all devices. Whenever a switch sees all 1s in the destination MAC address, it forwards the frame out all active ports.
Multicasts go to devices that have subscribed to the particular multicast data stream. So, multicast frames are forwarded to some or none, but perhaps not all ports. There are a lot of subtleties about how the switch knows where to forward multicasts, but for now this is sufficient detail.
The other very important exception is what the switch does when it encounters an unknown MAC address, that is, one that’s not in the MAC table. In this case, the switch has no choice but to forward the frame to all ports. This is called strong>flooding. The hope is that the correct device will respond and the switch will be able to correct the missing entry in the MAC table.
On many switches, it’s possible to disable MAC learning. There are legitimate reasons for doing this. You might want to configure a static entry for a particular port in your MAC table so it can’t ever be over-written. You might have a huge number of MAC addresses on your system so it’s not possible to keep them all in the table. These examples might require disabling MAC learning on one or more ports.
However, these are incredibly rare special cases. Most of the time you really want MAC learning enabled. Without MAC learning, your switch will flood every frame. This can cause severe performance problems.
So don’t turn it off unless you really mean to and have thought about the consequences. And if you’re having bad performance throughout what should be a small office network, have a look at whether MAC learning might have accidentally been disabled.
MAC learning should be enabled by default on every switch. Don’t turn it off.
Port security
The second feature is also related to the MAC table, and it’s called port security. This is a feature on most Cisco switches that allows you to specify a maximum number of MAC addresses that can appear in the MAC table for any given interface.
This is often done to prevent people from connecting little workgroup switches under their desks and hooking up a bunch of machines to a single port. It’s also sometimes done to force particular MAC addresses, and consequently specific devices, to be the only allowed devices on particular ports.
The trouble is that it’s not really a very good way of achieving either of these goals. I could hook up a small router or firewall and make a dozen computers look like just one MAC. I could similarly reconfigure my device to have the same MAC address as the one that’s supposed to be there.
But, worse, this feature can cause serious problems. For example, if you enable it on a port that has a wireless access point on it, you can have serious problems when a device roams to a different access point on the other side of the office. And you can find yourself restricting the number of wireless devices connected to that access point.
In short, I think port security causes more problems than it solves. If you really want to control which devices can connect where, it’s probably better to use a Network Access Control system such as 802.1x.
Port security is disabled by default. Don’t turn it on.
Spanning tree
The final feature I think should always be enabled is spanning tree. I’ve previously written an article about spanning tree, so I won’t go into the details again. The point of this protocol is simply to prevent loops.
Ethernet has a critical flaw: It’s impossible for any device to tell whether it has already forwarded a particular frame. So if you have a loop in your network, two switches connected through more than one path, possibly a very circuitous path, or possibly a single loop of cable, every switch will happily forward every frame around that loop an infinite number of times.
The solution is to make sure there are no loops. You can do this manually, but that doesn’t always work.
Sometimes, particularly in meeting rooms for some reason, somebody will get the clever idea of connecting a workgroup switch to two different ports. I’m not sure what their logic is, but it happens a lot. And every time it happens, they knock out the whole office.
Then, because the network is completely congested with forwarding every packet an infinite number of times, it’s extremely difficult to figure out where the loop actually is.
But, if you enable spanning tree, the network proactively detects loops and shuts them down. This gives you the additional benefit that you can deliberately make multiple connections between switches for redundancy in case of a failure.
Spanning tree is generally enabled by default, although it depends on the manufacturer. Keep it turned on.
