Electricity Basics for the Network Administrator

The topic of electricity is as fascinating as it is deep, and we’re about to ignore 99.9% of it, and stick with “Electicity 101”. Seriously. If your study of electricity is the slightest bit open-ended, you’ll get lost reading for hours and still not understand what relevant information you need from your networking equipment and data center vendors.

Want to know how transistors work? Won’t help. Superconductors? Pretty cool, but irrelevant to ordering gear. Electron flow manipulation? It’s a beautiful ability, a proud testament. But we’ve got a job to do, and to quote Gold Leader from Star Wars, “Stay on target.”

A network administrator needs to be a smart consumer of electricity. Being a smart consumer means knowing which questions to ask and understanding the answers. Let’s start with some electricity 101: Basic electricity terms.

PDUs and diversity

Mission-critical gear typically packs at least two power supplies, and it’s important to make sure each one plugs into a different Power Distribution Unit (PDU). PDUs are often called power strips, which is only sort of accurate, since PDUs often pack far more advanced features than simply providing outlets (more on that below).

Being a smart consumer means knowing which questions to ask and understanding the answers.

Each PDU should be pulled from a different upstream circuit, preferably with diverse paths back to the core, to protect from localized problems inside the data center. If this sounds familiar, it’s because it’s exactly how we network folks talk about pulling fiber. Any overlap in the “diverse” paths is a risk.

The facilities people may work for a commercial data center leasing you half a rack, or an electrician that your office building’s landlord called to build a server room. Either way, what’s likely going to happen is they’ll provide you with a bunch of outlets to plug your servers, switches, and other gear into. They’ll tell you a particular circuit is 20 amps, but you can’t run above about 16. 80% utilization is pretty standard, with a 20% buffer for spikes—like when a server admin reboots an entire rack of servers simultaneously. (Servers use more power while booting.)

Volts and amperes (amps)

As a network administrator, you should know how to talk about basic electricity terms like volts and amps. Volts are easy—the voltage you’re working with is governed by the country you’re in. Amperage fluctuates according to your needs. In both cases, what you need to figure out is the value of each that you need for each circuit.

In the United States, we traditionally use 120 volts, but modern data centers either prefer 208-240 volts, or are headed that way for efficiency. Many power supplies can automatically switch voltage levels, but if you’re unsure, look it up or ask your vendor representative.

A circuit will probably push 15 amps in a home, and 20 amps in a rack, unless someone states otherwise. Thirty amp circuits are common for larger equipment.

How do you know when you need a non-standard circuit? You need to know what you’re powering off the circuit, and you need to look at the data sheet for that gear. For example, I’m looking at the power utilization reported by a Cisco Nexus 5696 switch with six 1100 watt power supplies, three plugged into circuit A and three plugged into circuit B. Both circuits are 30 amps, and the chassis is reporting 1,400 watts in use.

This is where knowing your voltage comes into play. Assuming you’re running at 120 volts, quickly converting the watts to amps (amps = watts / volts so 1400 / 120) shows me that a single chassis is using 11.7 amps.

Since you need to plan for one of those circuits to fail, that 11.7 amps is considered taken on both of them. If you’re running at 208 volts, the math is 1400 / 208 or 6.7 amps. The reasons behind the greater efficiency are fun to learn, but as a network geek it’s not on the test; what you absolutely should remember is higher voltage is more efficient.

A common mistake when planning network expansions is overlooking the power required for expansions cards within an existing chassis. A good equipment vendor will allow you to check the power budget in software, finding out what’s in use (preferably broken down by component) and what the power supplies provide.

So if you see that a line card uses 250 watts, you need to make sure you have that much available before adding another card. And the 80% utilization rule of thumb applies here as well. Doing routine work like adding more optics to a fiber card will sneak your usage up so you want to pad how much extra power you keep on hand.

Another, related, mistake is not understanding how your chassis is allocating power. Specifically some large chassis will allow you to combine the power supplies to provide more usable power (2 x 1100 watt supplies offering 2200 usable watts). Or you can choose to run in “high availability” (or some equivalent term) mode, where only 50% of the power is used and a failed power supply doesn’t cause an outage. If you combine the power supply output, make sure you understand which slots or interfaces will turn off in the event of a failure.

Cleanliness and reliability

Any server room needs its power conditioned or “cleaned.” This means the peaks and valleys (think jitter in network terms) need to be smoothed out upstream from your equipment, but inside the local site.

Any professional data center will have this covered, but it’s worth verifying that every circuit they pull to your cage or rack is cleaned. There are a few ways to accomplish this but if you’re building a server room you absolutely need to make sure it happens. It’s really not negotiable if you’re running anything more important than a study lab.

One way people clean power is by running it through an uninterrupted power supply (UPS). This is a core UPS function, and you’ll need any important gear on a UPS anyway. Even professional data centers use them to keep power flowing for a few minutes while the big diesel generators that the tour guide showed you fire up.

For a server room you probably won’t have a diesel generator, but a few minutes of UPS time is often enough to both clean the power and compensate for a junky utility company prone to short outages.

A few final tips

Many PDUs are “metered,” such as this one I ran in my home lab for a while. Metered simply means you don’t have to do any math to figure out your power usage: The PDU tells you how many amps you’re using on a little LED display. This is a great feature, letting anyone racking equipment do a final check on what can be plugged in where.

More advanced PDUs, sometimes called smart PDUs, are actively manageable via IP or SNMP, letting you power cycle a specific machine remotely and collect usage statistics.

Since it’s likely that someone will need to work in the data center on occasion, it’s worth creating a work area with its own desk and power outlets. You want to avoid anyone touching production equipment unless they need to, even if it’s plugging in and unplugging laptops. Tripping circuit breakers, loosening adjacent cables, unplugging the wrong cable at the end of an 8-hour maintenance … nothing good can come of sharing a PDU, and plenty of bad can.

Another bad idea is stringing PDUs together instead of dedicating a circuit to each one. This is often referred to as “daisy chaining.” Just like daisy chaining switches together is a bad idea that can technically work, daisy chaining PDUs is a quick and dirty practice that will at best devolve into a rat’s nest over the years. At worst, it will cause a fire as you run a circuit too hot for too long.

Finally, it’s worth the effort to verify your equipment’s power supplies are using the same plug types as your power strips. Wikipedia has a great condensed chart, but you don’t have to dig too deep into this topic. Just ask your vendor reps for the information or look it up on data sheets, then make sure they match, which they probably will without your intervention.

If the power cable is plugged into an overhead receptacle, which sometimes happens for larger power supplies, the plug should have a locking mechanism to keep micro-vibrations from loosening it out of the socket over time.

It’s not your job to become an electrician any more than it’s your electrician’s job to learn BGP. I hammer on this point because the type of person who tends to choose to work with computers is the life-long bookworm who says things like, “Let me do some research and get back to you!” But aside from satisfying the curiosity that drives most geeks, it’s worth learning something of the jobs on either side of yours, if only to avoid embarrassing yourself.