Transformer Failures Can Disrupt Service for Weeks and Cost Millions of Dollars. In 2004, a catastrophic transformer failure and fire at the Westwing substation of Arizona Public Service Co. (APS; Phoenix, Arizona, U.S.) destroyed five transformers (525/230 kV, 500 MVA), pushed Phoenix-area residents to the brink of rotating outages, and cost APS and its partners in the substation an estimated US$28 million. It took nearly two years to restore full capacity. With today's prices even higher and the lead times longer, preventing transformer failures is more critical than ever.

After an extensive review of the cause of the fire and the utility's maintenance procedures, APS and industry experts developed several recommendations to avoid future substation fires. One of the recommendations was to install on-line dissolved gas analysis (DGA) monitors on all of the utility's T&D fleet of 230-kV and higher transformers. On-line DGA monitors can sample and report the dissolved gases in the transformer insulating oil every four hours. This near-real-time capability would help APS to prevent catastrophic transformer failures in the future.

NEEDLE IN THE HAYSTACK

APS began using Serveron's Tru-Gas on-line DGA monitoring in 2003 and currently uses the company's latest-version eight-gas DGA monitors, the TM-8. The monitors bring laboratory-grade DGA right to the transformer. They continuously sample the transformer oil and report results every four hours. They also have the capability of sampling every hour should fault conditions rapidly change within the transformer.

However, achieving the goal of avoiding future transformer failures would not be simple. The large volume of data the monitors produce creates a heavy burden on maintenance engineers who are required to analyze the data. Ultimately, they must recommend if a transformer should remain in service or be removed from service because it is exhibiting signs of impending failure.

Without data management and analyses tools, maintenance engineers would be looking for the proverbial needle in a haystack around the clock. Such circumstances demanded a unique system that would accurately analyze the mountain of data the on-line DGA monitors generate and pinpoint the samples that truly warrant attention — a very small fraction — as opposed to samples that are essentially the same as their previous samples.

These were the objectives of the APS Transformer Oil Analysis and Notification (TOAN) system. Currently deployed to about 130 APS transmission and generation transformers, the TOAN on-line monitoring system eventually will monitor approximately 170 of APS's most-critical system transformers (230 kV and above). In addition to its on-line monitoring capabilities, TOAN analyzes all manual DGA samples from all APS transformers.

TOAN BASICS

TOAN classifies transformer faults into four types: cellulose degradation, overheating oil, low-energy discharge and high-energy discharge with arcing. Further description of these four general fault types can be found in IEEE Guideline C57.104. Typically, DGA uses various ratios of specific dissolved gases to try to classify the fault type. However, as seen in Fig. 1, these methods are frequently incorrect or can result in a “Not Identifiable” classification because expected gases are sometimes not present. In addition, these methods do not include a way to accurately classify the severity of a fault.

Because of these limitations, an alternative to the traditional DGA techniques that could deliver the very high degree of accuracy required to automatically screen the large volume of on-line DGA data was needed. APS's choice for the analysis-engine portion of TOAN was an innovative system called ANNEPS, developed by Virginia Tech University.

ANNEPS combines the pattern recognition power of artificial neural networks (ANN) with an expert system that together analyzes the DGA data. To test the Virginia Tech ANNs, APS used its industry database of nearly 200 DGA samples. These samples marry pre-failure DGA data with post-failure inspections results, thus delivering an unprecedented degree of accuracy in ANN testing and training. Virginia Tech's ANNs were found to have an overall 93% accuracy rate, which was sufficient to begin building TOAN. Subsequent training of new ANNs by APS has increased the overall accuracy to 96%.

THE CRITICAL FEW

TOAN is designed to be a data-management tool that reduces the workload of DGA analysts through exception-based data processing. This means TOAN sends out a notification to the responsible maintenance engineer when something is different about the current sample compared to previous samples. In TOAN, this means the current sample either has a new fault type or the severity of the currently predicted fault type has changed.

TOAN has six severity levels that roughly equate to manual sampling plans that have annual, semi-annual, quarterly, monthly, weekly or daily sampling frequencies, respectively. The automated and exception-based processing of TOAN also brings consistency to the analysis; this is a marked improvement over multiple engineers individually performing analyses using their preferred methods. It also improves the accuracy of the analysis by using artificial intelligence. A fuzzy logic system that analyzes the signature gases of each fault type provides the output to the multi-tiered severity classification system.

TOAN's notification engine sends e-mail notifications to responsible individuals when the fault type or fault severity changes. This exception-based process filters out more than 99.5% of samples, so engineers can focus on only those samples that report new behavior.

THE “DAY LATE AND A DOLLAR SHORT” PROBLEM

Current gas rate measurement technology for on-line DGA monitors was not able to give accurate and timely gas rate measurements, as the technology uses an averaging algorithm to make its estimation. According to statisticians, averages are potentially problematic.

For example, Bill Gates and 49 unemployed workers are sitting around the table. Their average net worth is about a billion dollars. Unfortunately, this is not useful information. The same holds true for the average gas rate when the transformer is going through a rapid evolution of fault severity. In such cases, as shown in Fig. 2, the gas rate will be underreported and lag behind the actual value by, perhaps, a day. This could be the “Day Late and a Dollar Short” problem.

To solve this problem, APS employed data-mining techniques. Certain data-mining algorithms can quickly detect and accurately report changes in the slope of time-series data, which for the purposes of this article is the current gassing rate. The algorithm of choice implements a process called piecewise linear approximation (PLA). The program breaks up the time-series data into pieces, or segments, and uses linear-approximation techniques to measure the slope of each segment.

As can be seen in Fig. 3, the PLA algorithm connects the dots and measures the actual gassing rate. When the gassing behavior changes, new segments are drawn with new slopes or gassing rates. The PLA algorithm is run for every new sample received.

THE “HOLD STILL, I'M TRYING TO MEASURE YOU” PROBLEM

A second challenge that becomes very noticeable with the use of on-line monitors is the cyclic behavior of carbon monoxide (CO) and carbon dioxide (CO₂) gases dissolved in the oil. The ratio of the CO to CO₂ is an indication of whether or not the paper insulation within the transformer is aging normally or abnormally.

CO and CO₂ are generated primarily by the degradation of the paper insulation due to overheating. This occurs when the transformer is highly loaded or when the temperature within the transformer is high. However, the gases are partially reabsorbed by the paper when the temperature of the oil drops. With normal manual sampling, perhaps six months apart, the cyclic behavior of these gases is not readily apparent. However, with on-line monitors sampling every four hours, year-round, the cyclic behavior becomes evident.

Fig. 4 shows an example of a typical manual sampling frequency of six months overlaid on the on-line monitoring data. It can easily be seen how the up and down nature of the data can send confusing information about the health of the transformer. Not only do the individual gases rise and fall, but the important CO₂-to-CO ratio also rises and falls. When trying to get an accurate measurement, this can be called the “Hold Still, I'm Trying to Measure You” problem.

So, how does one measure a consistent gassing rate, without the influence of the cyclic behavior of CO and CO₂, so the exception-based processing of TOAN will work as intended? For APS, the solution was to use harmonic-regression techniques, so that the harmonic components can be removed from the analysis and the underlying trends can be analyzed.

Fig. 5 shows the same transformer with the unprocessed data and the data with the daily, semi-annual and annual harmonics removed. The linear regression of the deharmonized data has a very high linear correlation and represents the steady-state gassing rate of the transformer. Thus, the influence of the hot summers and cold winters is removed, and the behavior of the transformer can be trended as if it were in a consistent environment and loading pattern year-round.

The harmonic-regression technique also can be used to accurately predict the gas values in the future. In Fig. 6, the harmonic regression of the particular transformer was determined and frozen in early February while additional data was collected for 230 more days (through late September). As can be seen from the graph, the prediction of the peak CO₂ value is very close to the actual data. With de-harmonized CO₂-to-CO ratio data and accurate predictions of future CO₂ and CO values, engineers have a powerful tool to more accurately predict future insulation condition and prepare better replacement plans.

TWO SAVES

TOAN is meeting its goal to prevent catastrophic failures, so far chalking up two saves. One transformer was taken out of service for rewinding, and the other was able to be fixed on-site and returned to service. Obviously, the largest potential for property protection, reliability and productivity gains resides with the large, expensive transmission transformers that provide power to large numbers of customers. The largest of these transformers costs more than $6 million and can require significant lead time (up to three years) to even purchase.

The cost of TOAN is insignificant compared to the potential benefit when installed on multimillion-dollar assets whose health is essential to reliability. Nearly every electric utility employs extra-high-voltage transformers. One study estimates there are 100,000 transmission transformers valued at $200 billion in the U.S. electric grid. The same study found the failure rate of these transformers to be about 1.5% (the failure rate of APS's transmission transformers is about 1% per year).

Within the industry, catastrophic failure rates are much lower — perhaps 30 per year — but despite the low risk, the consequences can be severe. With such large numbers involved, clearly a system like TOAN can avoid huge capital expenses, even if only preventing a few failures. And by extending the useful life of existing transformers, the potential impact could quickly mount into the billions of dollars within a decade.

MEETING GOALS

APS is also meeting its goal of TOAN being an exception-based processing system, achieving an exception rate of better than 99.5%. This turns the mountain of nearly 290,000 samples each year into actionable information. The fault-type classification accuracy of TOAN is at 96% and should continue to improve as more and more inspection-based DGA samples are gathered.

For the first time, APS also has been able to accurately measure the gassing rate of quickly evolving transformer faults using data-mining techniques. This enables the utility to determine accurate, consistent and timely fault severity measurements. The experience of APS clearly shows that the electric utility industry has to be careful not to impose artificial models on data (in this case, averaging the gas rate over a fixed interval), but rather let the data tell its own story using data-mining techniques.

---

Donald Lamontagne (Donald.Lamontagne@aps.com) has served as the section leader of Arizona Public Service Co.'s T&D Reliability Analysis and Management department for the last eight years. He has more than 17 years experience in nuclear and fossil power plant construction and operation. Lamontagne is the inventor of APS's Transformer Oil Analysis and Notification (TOAN) system, winner of the Edison Electric Institute's 2008 Edison Award. He is also the author of two patent applications for the TOAN system, one for its exception-based notification system and one for the development of piecewise linear approximation and harmonic-regression algorithms to accurately calculate gassing rates from on-line dissolved gas monitors.

TOAN WINS EDISON AWARD

The Edison Award is presented annually, usually to one U.S. and one international electric utility, as selected by members of the Edison Electric Institute (EEI). It is EEI's most prestigious award and has been presented since 1922. Arizona Public Service Co. (APS) received the 2008 Edison Award with the following citation:

APS, the operating utility subsidiary of Pinnacle West Capital Corp., serves more than 1 million customers throughout Arizona. APS developed a first-of-its-kind tool for near-real-time monitoring of major transformers serving the company's customers. The on-line Transformer Oil Analysis and Notification (TOAN) system has enabled the company to automatically determine the health of transformers by evaluating dissolved gases inside transformers, thereby improving system reliability, enhancing productivity and preventing catastrophic failure. The patent-pending monitoring and artificial intelligence analysis/notification system doubled the accuracy of existing methods while eliminating time-consuming and repetitive manual efforts.