Network Basics: Enterprise WAN Router

My daughter is exploring networking as a possible career choice. She’s digging into the early chapters of a certification study guide and has come to a section describing the differences between a local area network (LAN) and wide area network (WAN).

She and I chatted about the material, highlighting the characteristics of LANs and WANs. “Oh, so a LAN is basically a network in a building or connecting other really close buildings, and a WAN connects LANs to other faraway LANs?” she asked. “Yes, that’s pretty much what each does. But operating a LAN and operating a WAN bring up different concerns,” I responded.

The difference one must understand about enterprise WANs is that technically they’re not operated by the enterprise IT team. Service providers operate the WAN. An enterprise IT team merely connects to that service provider’s WAN using a special-purpose WAN router at each LAN site.

The purpose of a WAN router is to act as a gateway between the LAN and the service provider network. When you think about a WAN router in this way, you quickly realize that it’s a different sort of device from an Ethernet switch.

While it’s relatively easy to build redundant paths in an Ethernet LAN, the service provider circuits that WAN routers connect to tend to be very costly. As a result, many enterprises will only have a single WAN circuit for each location. WAN routers uplink to a costly and valuable resource: a circuit that needs to be monitored, protected, and utilized carefully.

Let’s take a look at how a WAN router can help make the most of a WAN circuit.

Management: Keep track of utilization and errors

Network engineers accustomed to monitoring Ethernet interfaces are used to seeing a small number or even no errors. For the most part, Ethernet links inside a data center or campus environment run cleanly. Interfaces that log errors are often ignored unless the performance of a connected host is severely affected.

Compared to Ethernet links, WAN circuits require more careful monitoring to be sure they’re operating at peak ability. To know what sorts of things to check, we’ll review some key fields in the status of a Cisco serial T1 interface, shown below.

WAN-ROUTER#show interfaces s0/0/0
Serial0/0/0 is up, line protocol is up
  Hardware is GT96K with integrated T1 CSU/DSU
  Description: Circuit ID ABC123456 | Vendor SuperCarrier | Support 800-GET-HELP
  Internet address is 11.12.13.1/30
  MTU 1500 bytes, BW 1536 Kbit/sec, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation PPP, LCP Open
  Open: IPCP, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters 1w6d
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/32768 (size/max)
  5 minute input rate 11000 bits/sec, 3 packets/sec
  5 minute output rate 4000 bits/sec, 3 packets/sec
     1609565 packets input, 549503344 bytes, 0 no buffer
     Received 0 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     1537805 packets output, 345646393 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up

WAN-ROUTER#

• Notice that the description field was populated by a human with the circuit ID, carrier name, and support number. This is very convenient when troubleshooting WAN circuit issues. You know all the key information to open a case with the carrier. Also, the description field is polled by network management stations (NMS) where it can show up in alerts, graphs, and reports.
• The input and output rates describe how many bits per second are flowing through the serial interface. Stay on top of this key metric. A circuit that never sees above 20% utilization could be downsized to reduce costs. A circuit operating at a steady 80% + utilization during peak hours should be considered saturated and ready to be upsized. Pay close attention to NMS graphs, and set threshold alarms for WAN circuit utilization that’s outside of the norm.
• Someone deploying a WAN router for the first time should pay special attention to input and output errors. Why? WAN circuits already have low bandwidth compared to LAN links. Errors on a WAN circuit make an already slow link underperform. While occasional bursts of errors on WAN circuits are not uncommon, steadily accumulating errors should be reported to the carrier for resolution. It’s not a bad idea to check these stats whenever you happen to log into the WAN router, but the best way to track them is with an NMS that will alert you when errors exceed a threshold you’ve set.
• Carrier transitions effectively track the number of times the circuit has gone up or down. More precisely, it tracks the number of times the carrier signal has been interrupted. Outside of scheduled carrier maintenance, there should never be carrier transitions. A growing number of these indicate an unstable circuit that should be reviewed by the provider.

Quality of Service: Fast lanes through a traffic jam

Once the WAN circuit is properly monitored, the next consideration is effectively managing the traffic flowing through that circuit, especially big traffic bursts that can overwhelm a small WAN circuit. When these traffic bursts come along, the various conversations traversing the circuit slow down, an effect similar to a highway traffic jam. When there are only a few cars on the road, all vehicles can travel the speed limit. But when the number of cars exceeds the road’s carrying capacity, all traffic tends to slow down because of the heavy volume.

While most network traffic adjusts to congestion and soldiers on, some types of traffic are not tolerant of slow downs, aka excessive latency. For example, voice-over IP traffic must be sent across the wire in a timely fashion, or else the voice packets arrive too late. The result is a choppy or garbled conversation.

The networking industry has quality of service (QoS) schemes to classify traffic and prioritize certain traffic classes over others. WAN circuits are an especially critical place to install a QoS scheme. QoS can make sure that voice, video, or other latency-sensitive traffic has a fast lane always available to them. QoS can also ensure that a minimum amount of bandwidth is reserved for traffic that simply must get through, even if it slows down a bit.

The specifics of QoS configuration are beyond the scope of this post. For more information, Cisco has written a comprehensive set of QoS configuration guides for IOS that explain this complex topic in detail.

Routing: Getting traffic from the LAN to the WAN and back

Implied in the name WAN router is that the device routes. In the role of the gateway, a WAN router is responsible for routing traffic between the LAN and WAN. What’s the method of determining the best way to perform that routine? Let’s consider a few key points.

• Avoid static routing. In many sites, I’ve noticed that WAN routers are deployed with static routes. Static routes are hard-coded configuration statements that direct traffic in a specific direction. While static routes are perhaps the simplest to understand and deploy, they quickly become unmanageable. A network deployed with static routes usually implements them across every device in the network. When the network changes, each of those static routes must be changed by hand. This places a huge administrative burden on the IT team and results in an error-prone process. Missing even one static route means that traffic isn’t delivered where it needs to go. A far better approach is to use a dynamic routing protocol. Dynamic routing protocols are used by network devices to announce to other network devices the routes they can reach. In this way, inputting static routes by hand is eliminated. For LANs, OSPF and EIGRP are by far the most common choices. Open Shortest Path First (OSPF) is an industry-standard protocol and is available via a large number of networking equipment vendors. Enhanced Interior Gateway Routing Protocol (EIGRP) has only recently become an open protocol but is very popular in Cisco networks. For WANs, the service provider will dictate your choice of routing protocol. Most often, Border Gateway Protocol (BGP) or OSPF are offered.

  

• Uplink the WAN router to an isolated LAN segment. To maximize routing efficiency, I recommend connecting a WAN router to an isolated link that’s not used for any other hosts. Assuming your core network is capable of routing between VLANs, the idea is to build a VLAN used solely as the uplink for the WAN router. The LAN will route from host VLANs to the WAN router via the uplink VLAN. While the most common IP numbering scheme for an uplink VLAN like this is a /30 (for example, network 192.168.100.0 with a subnet mask of 255.255.255.252), a /30 results in only two usable hosts on the line: the LAN switch side and the router side. Expanding to a /29 (for example, network 192.168.100.0 with a subnet mask of 255.255.255.248) results in six usable hosts on the line. A /29 is helpful for the following scenarios.
  1. Adding a WAN optimizer appliance. WAN optimizers often share the link between the LAN and WAN router, accelerating traffic. They usually require an IP address. A /29 leaves enough room for this to happen. A /30 does not.
  2. Replacing equipment while minimizing disruption. With a /29, it’s possible to install new equipment like a LAN switch or WAN router in parallel with the old infrastructure, minimizing the disruption to the user community when the time comes to cutover. With a /30, building a parallel link is a more complex task.
  3. Adding redundant equipment. Some networks are leaned on so heavily by their organizations that those organizations are willing to spend money on redundant equipment. In some designs, the redundant equipment will share a link. A /29 allows for flexibility and choice without having to redesign the link. A /30 does not.

  In short, using a /29 on an isolated VLAN between LAN and WAN gives a network engineer flexibility as network needs change over time.

Putting it all together

WAN routers are special devices that perform a critical function in an organization’s network infrastructure. Making the most of them means ensuring they’re running well, the traffic flowing through them is handled well, and their uplink to the main LAN is designed well.

Following these guidelines leads to a well-managed WAN that can be monitored effectively and react automatically to changing network conditions. Invest the effort upfront to create a WAN router template for your organization, and the long-term payoff is predictable WAN behavior that can be counted on.