Oncor Knows Outage Detection

When an electric disturbance occurs, sometimes customers are not immediately aware of it. And, thanks to Oncor embracing the latest technology, power may be restored without customers even knowing there was a problem.

Oncor constantly strives to improve system reliability through advances in technology and quality of service. Part of this involves working with industry leaders to provide the best leading-edge equipment and software. This includes advanced meters, a radio-frequency network and system head-end software from Landis+Gyr, along with outage management system (OMS) software from Intergraph, distribution management/advanced application systems from Siemens and meter data management systems (MDMS) from Ecologic Analytics.

Additionally, Oncor used IBM as the project manager and developer for the integration of Oncor's OMS with its advanced metering system (AMS), a network that will consist of 3.2 million meters by the end of 2012. The AMS deployment and the OMS system upgrade were both initiated in the 2007-2008 time frame. The two systems were not integrated until the spring of 2012. This permitted both systems to reach a level of both scale and maturity prior to the integration.

Initial AMS Service Restoration Use

Prior to implementing the AMS-OMS integration in 2011, a Web interface to MDMS was implemented that allowed distribution system operators the ability to remotely interrogate, or ping, a meter. This relatively straightforward functionality is an effective tool. On blue-sky days, in a typical month, at least 20% of all single customer-outage events are found to have good meter voltage. This indicates there is some issue on the customer's side of the meter, possible a main breaker tripped, and saves the time and expense of a truck roll. Recently, Oncor enhanced this functionality to provide the actual voltage reading to the operator, not simply an acknowledgment that the meter is energized.

Also prior to implementing the AMS-OMS integration, a report was built to display the results of AMS and OMS data comparisons. Queries are executed against the MDMS database to retrieve all meters that cannot be communicated with and against the OMS database to retrieve all meters showing an open outage. The report flags all meters, produced in the OMS query, shown to have power in the MDMS database. This report has become a valuable tool for the operations personnel during cleanup of large-scale events.

AMS-OMS Design Considerations

The AMS and OMS projects were developed in parallel independent work streams with the intention of ultimately integrating them after both systems were deployed and operating at a highly functional level. The integrated design principles include the following:

Using commercially off-the-shelf software with virtually no custom modification
Designing the integration with full consideration of the inherent characteristics (both good and bad) of all the systems
Ensuring adaptability of the system through settings, not reprogramming.

For example, the system is designed to recognize that, during major outages, the inherent characteristics of the radio-frequency mesh relative to the electrical grid will reduce the number of last-gasp messages the system receives.

Based on feedback Oncor received from other utilities, there was a particular bias to minimize the number of false positives encountered by operators and ensure IT systems would not become overloaded during major events. Here is how Oncor manages the false positives:

The last-gasp time delay is set at 40 seconds, which is 10 seconds longer than the last automatic reclose in most cases
The system holds all outage messages in the MDMS for 2 minutes and 45 seconds to ensure no power restore message is received prior to triggering the OMS
The system blocks signals to the OMS where only one service on a transformer bank sends a last-gasp message.

Following is how Oncor manages major event overloading:

The system sends a message to the MDMS with a list of premises on a feeder anytime supervisory control and data acquisition (SCADA) indicates the entire feeder is out and the MDMS does not report those meters as out to the OMS
The system blocks all but two outages signals from meters associated with a distribution transformer bank
The system contains a queue with a configurable threshold setting that automatically shuts down if the threshold is met
A kill switch was designed to allow human intervention to disable the AMS-OMS connection in situations where significant outage traffic is expected.

To connect the systems, there is an enterprise service bus (ESB) between the head-end system and the MDMS, and a separate ESB between the MDMS and OMS.

Deployment Results

Integration first began in the early hours of March 8, 2012, when Oncor enabled the integrated AMS-OMS in two districts in the Dallas-Fort Worth, Texas, U.S., area. Project managers carefully watched the integration of 750,000 meters, having much anticipated a time when AMS data would be useful for the utility in restoring outages. That night, outages at approximately 30 homes were communicated not by customers but by advanced meters, allowing the utility's crews to restore power before many customers were even aware of a problem. By May, all of the 2.9 million installed advanced meters were fully integrated with the two systems.

With virtually no tuning, the system has exceeded the utility's expectations. On blue-sky days, about 20% of Oncor's outages are resolved without receiving a customer call. Since March, Oncor has resolved nearly 3,400 outage events impacting more than 35,000 customers without customers ever having to call and notify Oncor of the outage. The majority of those outage events occurred at transformers or fuses during the business day and affected an average of 10 customers. Without the AMS-OMS integration, those outages would not have been known until customers came home and reported the issue.

Additionally, on blue-sky days, more than 11,000 outage events impacting more than 600,000 customers were reported by both a customer call and the AMS-OMS integration, and the scope of the outage was almost immediately identified. When relying only on customer calls, determining the scope could take up to an hour to fully analyze because the calls come in over a relatively long period of time.

One unexpected side benefit of this integration was it exposed the ability to detect power-quality issues prior to them being detected and reported by customers. The last-gasp function in the meter is set to trigger any time the voltage dips to 80% or less. When the system was initially deployed, several events involved power-quality issues — usually involving bad electrical connections that fail in a relatively short time frame or, in some cases, intermittent secondary faults — not permanent outage issues. This has allowed for the opportunity to repair facilities prior to significant customer disruption.

This led to Oncor's use of the AMS data on individual premises to detect and correct similar issues on a scheduled basis prior to having a forced outage, which often requires attention outside normal work hours. In just six months, Oncor has avoided hundreds of power outages by analyzing and responding to this data.

Current Initiatives

Any successful system deployment is a combination of the technical deployment and change management with the people involved, as well as the careful coordination of the timing and speed of both. Functionality within the AMS-OMS recently has been enabled to allow operators to ping meters associated with an outage event, meters beyond a network device, or any single or group of meters chosen, and to view the responses within the OMS. This is expected to further increase the number of saved truck rolls for isolated outage events. This also will improve the operator work flow for pinging single meters by allowing all of the tasks to be performed within the OMS application.

Also, the restoration verification function between the OMS and AMS has been enabled. After an outage has been restored, the OMS sends a signal to the AMS indicating which premises are believed to have been restored. The MDMS confirms which premises are restored by analyzing the data and provides an exception report for any premise it cannot confirm as energized. This functionality is expected to reduce the possibility of nested outages (outage area subsets not restored when larger blocks of the system are restored) and improve overall restoration efficiency by reducing truck rolls and customer callbacks.

Obviously, the better the connectivity model represents the electrical system, the better the AMS-OMS system performs, especially in the area of restoration verification. While much connectivity model data improvement was done in conjunction with the OMS deployment, additional data-quality improvement opportunity has been exposed by the integration of the AMS-OMS system. An initiative is underway at Oncor to improve the connectivity model and have the appropriate processes in place to maintain data integrity.

Two major system enhancements are underway. The first involves improving the scalability of all the messaging on the AMS portion of the system from hundreds of transactions per second to between 5,000 and 7,000 transactions per second. This will keep the system enabled during major outage events and not just in the mid-sized events.

The second enhancement is more complex because it involves both business processes with numerous parties and IT system modifications with Oncor's customer information system. This modification enables the ability to bring single-premise outages directly into the OMS. This effort is complex because, although a system can be designed to account for Oncor work orders that involve removing meters, work processes cannot be easily modified in situations where contract electricians pull meters to perform their work. Also, it would be inefficient to have service restoration personnel respond to theft-of-service situations (especially after hours) instead of having revenue security personnel respond on the day after, during normal work hours.

There's More to Come

The value of the individual smart grid systems Oncor has deployed has met expectations and improved customer service and operational efficiency. Furthermore, initial integration of these systems has exposed some real benefits that have quickly exceeded expectations. While much has been accomplished, what has been demonstrated thus far is only the tip of the proverbial iceberg relative to the potential of these integrated systems. As more systems are integrated, data models advanced, communications and IT systems improved, and data analytics employed, the benefits and opportunities are expected to grow exponentially.

Joe Wolf ([email protected]) is the T&D services manager at Oncor. He is responsible for the distribution management system, which includes mobile workforce management, outage management, distribution supervisory control and data acquisition, and distribution network analysis applications. Of his 32 years with Oncor, Wolf spent 20 years associated with operating, supervising and managing Oncor's distribution operating centers. He has served as the Intergraph InService Customer Organization (I2CO) president since 2009.