If the market for utility communications technology is any indication, a major transformation is underway in the electric power sector. The transformation could be the result of two years of hellacious storms or the simple realization these new technologies can improve overall system reliability and efficiency. Whatever the reason, and those reasons certainly vary, utilities are embracing the smart grid as more than just a way to modify customer behavior.

First off, an increasing number of utilities are voting with their dollars and moving away from a multiplicity of communications networks to single end-to-end networks that break down communications silos to enhance efficiency and reliability. But there's more. Utilities also are finding these new Internet protocol (IP)-based networks can do more than just enable the smart grid.

Indeed, some utilities, particularly those owned by municipalities, are looking to expand these systems beyond an energy smart grid to include smart water, transportation and emergency services. In some cases, this new “everything grid” may even offer a new profit center. So, while the industry is seeing utilities converge and integrate networks internally, many utilities now see that expanding the reach of their smart grid communications systems also could increase the benefits of that investment.

Of course, some utilities have moved faster than others. The most successful generally have done two things:

  1. Invested in four critical technologies: advanced metering infrastructure (AMI), outage management systems (OMS), geographic information systems (GIS) and distribution management systems (DMS)

  2. Invested in the applications and communications systems necessary to integrate those technologies.

All This Is Possible

Where utilities have deployed those technologies, the benefits start immediately. Before a storm hits, they can use the DMS, which is a collection of applications that monitor and control the increasingly complex distribution network, to simulate the changing load flows on the system that a hurricane or other disaster might cause and plan how to reconfigure the system to continue service to customers. But it is during an actual crisis and all through the restoration process that these technologies really prove their value:

  • The AMI system provides accurate data on failures.

  • The supervisory control and data acquisition (SCADA) system sends system status changes to the OMS.

  • The GIS updates the network model for managers.

  • The DMS identifies a disturbance and its location, isolates the faulted area by opening switches upstream and downstream of the fault, and restores power to all but those on the faulted section of the feeder — all in a minute or two.

In addition, as Gary Rackliffe, ABB's vice president of smart grids for North America, points out, this kind of comprehensive smart grid provides highly valuable “situation awareness” not only for utility managers and crews, but also for government agencies such as the Federal Emergency Management Agency and various state agencies, the media and the public.

Again, all this is possible only if utilities have an adequate communications infrastructure encompassing an enterprise-level IT system that typically includes backbone fiber communications out to the critical substations; a local area network within the substations; a wireless wide area network for data backhaul and distribution automation (DA); a radio frequency mesh AMI network; and possibly even home area networks (HAN).

Moving to IP-Based Systems

As utilities begin to weigh the advantages — and cost — of these applications, they are finding they must make some basic decisions first about the details of any new communications infrastructure, and more and more IP/multiprotocol label switching (MPLS) systems seem to be the preferred choice.

First off, an IP/MPLS network is simply a packet-switched network that uses the transmission control protection (TCP)/IP stack enhanced with the MPLS standard. A packet-switched network is a digital communications network that groups all transmitted data into usable blocks, or packets. The network over which these packets travel is a shared one that routes each packet independently from all others and allocates transmission resources as needed.

This is best illustrated by the seven-layer open system interconnection model (above), which defines a networking framework to implement communications protocols. Control is passed from one layer to the next, starting at the top (the application) layer and proceeding to the bottom (physical) layer. Levels 1 through 3 contain most of the hardware. This protocol architecture allows each layer to operate, more or less, independently, and higher-layer software and applications can function properly regardless of the physical network.

The MPLS standard, which was introduced to guarantee delivery and reduce network delay in the IP environment, is a bit different. It operates above the IPs (in an unofficial Layer 2.5) in the OSI model. It operates on packets (Layer 2) while working with IP addresses (Layer 3), using labels to route packets differently.

Priority Traffic

An IP/MPLS network offers utilities several advantages:

  • While network traffic will generally take a prespecified path on the network, an alternative path also may be specified in advance, making it possible for the network to be self-healing, increasing quality of service and availability.

  • Because traffic is labeled at the source, it can be prioritized over other traffic.

  • Voice and video traffic can be given travel paths with the least delay and the best redundancy.

“IP/MPLS allows us to bring security, segregation and a broadband foundation all the way out to smaller distribution offices,” said Mark Madden, Alcatel-Lucent's regional vice president for North American energy markets. “Instead of building multiple single-purpose networks, we can now safely and reliably put all applications on a single broadband network.”

While, there have been concerns about putting all those applications on a single network, Madden conceded, “IP/MPLS can segregate traffic and provide firewalls to ensure quality of service for critical applications, so less-critical applications, like web browsers, don't interfere with control traffic.”

Todd Gurela, worldwide operations director of connected energy networks at Cisco, has a similar view. IP allows utilities to have a managed secure network with quality of service, he said. “In the IT world that means prioritization of traffic over the network.” With IP, he noted, “You can prioritize so that safety and security of data always takes priority.”

Merging Networks

A number of utilities are already benefiting from IP/MPLS installations. Tearing down communications silos and merging networks is just one of the benefits, and one of the first was Canada's AltaLink transmission utility.

“We've gone through all phases of the project, from proof of concept to technology evaluation, vendor reviews, and lab setup and testing,” said Clinton Struth, principal engineer, network communications at AltaLink. AltaLink built its own lab to conduct “a whole array of interoperability tests” ahead of deployment, Struth added.

The utility started its fourth year of deployment in January 2013 and now has some 300 nodes in service. “We're carrying everything from current differentials, distance teleprotection, SCADA, emergency voice and radio, basically all the traditional services in addition to Internet backhaul, field office and internal corporate network connectivity, VoIP solutions and synchrophasors,” Struth said.

This is a far cry from when he started with the utility 13 years ago. At that time, there were three very rigid and very distinct communications silos. “Now we're the one service provider network for the whole utility,” he said, adding that, “while there are some other technologies coming available, from our perspective, IP/MPLS is the only technology that can do that. Nothing is as mature as IP/MPLS.”

With IP/MPLS, he noted, “You can leverage intelligence into network layer, yet you still have complete control to prioritize traffic through your network. There's an array of different knobs you can turn to make do what you need it to do.”

Struth echoes Alcatel's Madden in citing the importance of the system's “rigid prioritization” feature in AltaLink's decision to go with IP/MPLS. Essentially, the system says, “I will drop everything to keep teleprotection and SCADA alive,” said Struth. Although other technologies, such as carrier Ethernet, are offering similar services now, “when we were doing our review, nothing could match what MPLS was doing,” Struth added. “And the proof is we have it running with only eight of us in the team working on it, not an army.”

National Grid in Massachusetts has just launched a smart grid communications pilot project that encompasses multiple applications — AMI, HAN and DA — operating simultaneously over a Cisco GridBlocks IPv6 network.

The team includes: Itron (AMI and HAN); Cisco (communication technology and network management); GE (WiMAX); Verizon (IP and wireless communications services); IBM (legacy system integration); Wipro (web development services); GridMaven (network monitoring and management); Navigant (evaluation services); S&C Electric, G&W Electric, Beckwith Electric (DA); Schweitzer Engineering Laboratories (protective relays and automation), Lindsey Manufacturing (current and voltage sensors) and Power Delivery Products (capacitor control and monitoring).

The program will involve 15,000 customers in northern Worcester, Massachusetts. While the bulk of the pilot program will use a GE WiMAX transport system, National Grid also is looking at Verizon LTE as a way to get out of the networking business. The utility has wisely built its own lab to verify its design and the interoperability of all the equipment included in the pilot project.

Still, it is a big country and one size does not fit all, nor does one combination of networks fit all when it comes to expanding and leveraging the smart grid. Take PPL Electric Utilities (PPL), for example. With a stimulus pilot program grant from the Department of Energy, the utility installed a DA system that included a full suite of reclosers, motorized air brakes, remote capacitor controllers and a DMS, all tied together by WiMAX radio.

According to Tim Figura, PPL's manager of telecommunications, WiMAX radios were installed on towers at eight substations, with backhaul from the substation by fiber. Why WiMAX? At the time, PPL wanted a broadband-speed radio network to connect all 500 DA devices in the program, said Figura. But experience can be a tough teacher: The test comes first, the lesson later.

No Need for Speed

By 2011, when PPL decided to expand its DA system to the Poconos, an area of wooded hills and valleys in northeastern Pennsylvania, it had decided against WiMAX for two reasons: first, the area's topography worked against the technology, which pretty much requires a line of sight between the towers and the DA devices; second, the existing WiMAX system had been unreliable. Equipment issues and radio interference had caused major headaches. Figura said they could have solved the first problem “with a lot of repeaters,” which would have raised costs substantially, but, more critically, experience with the existing WiMAX system had “left a bad taste in our mouth.”

Further, he added, the utility began to realize it did not need high-bandwidth communications in the Poconos. “Unless we want video on the side of the pole, I'm personally not convinced we need megabytes of data per second going out to a DA device,” he said, adding that “it takes maybe 200 bytes of data, not even kilobytes, to send the status of a recloser or capacitor — not a lot of data.”

In addition, new communications protocols such as DNP 3.0, which is commonly used in SCADA systems, “records all your events. Why don't I just wait for the device to tell me there's a problem instead of always checking in on it? We don't need to pay for huge bandwidth,” Figura said.

After a year of study — and a change in perspective — PPL has decided on a new communications strategy for DA: commercial cell phones. “We're using the same tower, but it's completely isolated from rest of the traffic. It's a pure Ethernet-based system with a private IP address,” Figura said. “We're seeing about 3 MB/sec of capability, which is about what we're getting from WiMAX.” Latency, he added, is about 200 msec.

Additionally, said Figura, with 80% to 90% of PPL's outages occurring during non-storm events, PPL questioned whether its customers would be willing to spend millions of additional dollars for a private radio system to survive a hurricane. On a final note, Figura said, “Our core business is power, not communications. For AT&T, Verizon and Sprint, that's their business. We have 10,000 sq miles to cover, so it didn't make sense to spend the money to build a system that would survive a hurricane.”

Wired and Wireless

Another approach — one not generally thought of for distribution systems — is power line communications (PLC). Until now, issues have included low bandwidth and how to get information past transformers, but Alstom believes it has solved those problems and is looking to expand PLC into DA. The utility has successfully installed 50 PLC nodes on a medium-voltage power line in northern France and is looking to demo the technology in the United States.

“If you look what has been used in PLC, it's mostly narrowband, meaning it hardly achieves speeds above 100 kb/sec,” said Alstom's Smart Grid Director Laurent Schmitt. “That is not consistent with the needs of DA.”

He added, “We now have the architecture to reach speeds of 3 MB/sec to 5 MB/sec over 3 km to 5 km.” This, he said, would allow the integration of a wide range of services through the same backbone such as telephony, video surveillance, remote meter reading, real-time data exchange with grid controls and active demand-response management.

On-Ramp Wireless is offering a new machine-to-machine (M2M) technology that could benefit utilities with larger service areas. According to Jason Wilson, an On-Ramp Wireless senior vice president, the company's new signal-processing techniques allow fast communication with tens of thousands of devices within the hundreds of square mile reach of a single access point. The system, which communicates over the unlicensed industrial, scientific and medical (ISM) band, while not broadband, can be used for AMI and miscellaneous sensor reads, including downed lines and intruder alarms.

“You can communicate with devices that can't be reached with AMI meshed radio,” said Wilson. San Diego Gas & Electric is using the On-Ramp technology to communicate with 2,000 fault detector sensors. The utility expects to have 10,000 of these sensors installed by 2017.

In and Above the Cloud

Silver Spring Networks takes a different approach, offering utilities several information services in the cloud. The cloud is where the vendor has installed a communications network, running multiple applications — and runs it on behalf of its customers. “We capture a lot of data,” says Michelle Rae McLean, the company's director of product marketing. As McLean explains it, Silver Spring's customers can subscribe to various channels of its SilverLink service, which she likens to cable TV, to get network performance reports and alerts that affect any of the smart grid applications running on the network, including AMI, DA and DSM or load management.”

Utilities needing all-weather communications may also look above the cloud for help. Inmarsat, a global satellite communications provider, has developed a 15-sq-inch satellite dish that can be used on top of utility poles for AMI and on top of utility trucks to keep communications flowing, no matter the weather and irrespective of the traffic on cellular systems.

“These are very small and easy to install,” said Chuck Moseley, Inmarsat's director of M2M energy sales. Further, he said, people can be trained to operate them in about an hour, and with the ability to transfer data at 492 kb/sec and to provide WiFi from each truck, “satellite is entering the conversation.”

Eternal Vigilance

In today's world, however, every grid, no matter how smart, is subject to cyber attack. “With the proliferation of portable media and smart devices, and the increased difficulty with corralling critical data,” Ernie Hayden, managing principal for Verizon Business, told DistribuTECH that “it's possible to have a data breach at any time under many different circumstances and [managers] must establish a strong, practiced incident response team to protect important data.”

With that in mind, U.S. President Barack Obama recently issued an executive order setting up a voluntary program of cyber security standards for electric utilities and other companies operating critical infrastructure. The order directs federal agencies to consider incorporating new cyber security standards into existing regulations, and directs the government to share more information about computer threats with the private sector and issue more security clearances to allow industry representatives to receive classified information.

Further, the Electric Power Research Institute recently issued a report on “Intrusion Detection Systems for Advanced Metering Infrastructure.” The report — which covers the AMI collection engine, meter data management system and data collection unit as well as meters — gives utilities and AMI vendors a perspective on the unique monitoring requirements of AMI as well as key research challenges related to intrusion detection technology and large-scale deployment. Following are the key takeaways from the report:

  • The need to monitor the edge as well as the head of the communications network

  • The size of AMI networks can present scalability issues

  • The need to protect IDS sensors from becoming compromised.

The 21st Century Grid

Yes, it has taken multiple storms, federal stimulus funds and technological advances, but, with experience, utilities are starting to see the benefits of investing in comprehensive smart grid communications systems. Of course, the topology, hardware and software of these new networks will be as varied as the areas they serve, but the goals of this growing transformation are universal: shorter outages, increased reliability and efficiency, and, possibly, even new markets. Truly, all things are possible with the converged networks and expanded reach these new systems bring to the electric power industry.

ABB | www.abb.com

Alcatel-Lucent | www.alcatel-lucent.com

Alstom | www.alstom.com/us

AltaLink | www.altalink.ca

Beckwith Electric | www.beckwithelectric.com

Cisco | www.cisco.com

DistribuTECH | www.distributech.com

FEMA | www.fema.gov

G&W Electric | www.gwelec.com

GE | www.ge.com

GridMaven | www.gridmaven.com

IBM | www.ibm.com

Inmarsat | www.inmarsat.com

Itron | www.itron.com

Lindsey Manufacturing | www.lindsey-usa.com

National Grid | www.nationalgridus.com

Navigant | www.navigant.com

On-Ramp Wireless | www.onrampwireless.com

Power Delivery Products

www.powerdeliveryproducts.com

PPL | www.pplweb.com

S&C Electric | www.sandc.com

Schweitzer Engineering Laboratories | www.selinc.com

San Diego Gas & Electric | www.sdge.com

Silver Spring Networks | www.silverspringnet.com

Verizon | www.verizon.com

Wipro | www.wipro.com