How Utilities Are Mitigating Infrastructure Risks With AI

The utility industry faces a rising number of risks related to maintaining and inspecting its infrastructure, ranging from climate change to aging assets to labor shortages and more. As these risks grow, so does the need to invest in mitigation strategies, many of which involve harnessing new developments in technology. One of these technological developments is computer vision. As an application of artificial intelligence (AI), computer vision has become an efficient and cost-effective means for managing the growing burdens of asset inspection and asset maintenance.

Deploying Technology to Mitigate Growing Risks

The utility industry is no stranger to the increased risks posed by climate change. The 2017 and 2018 wildfires in California were found to be sparked by downed power lines and faulty electrical equipment during a period of record-breaking drought. The fires killed 104 people, devastated the local environment and communities, and led to PG&E settling more than $14 billion in lawsuits that held the company liable. In the wake of these fires and other catastrophic weather events, such as hurricanes and deep freezes in Texas, utility companies are shoring up their mitigation efforts to better prepare for related risks.

These heightened risks come at a time when many utility companies are already challenged by aging infrastructure. The U.S. Department of Energy reported that 70% of power transformers and transmission lines are more than 25 years old. Companies are faced with tough choices: which assets to maintain, upgrade, or replace, based on their risk profiles. These choices are especially critical given that utility companies have reported skyrocketing costs for liability insurance, along with a decline in the number of insurers and the addition of built-in wildfire exclusions.

In the face of these and other risks, mitigation efforts are vital, but also costly. For example, since 2019, Xcel Energy in Colorado has invested nearly $500 million in mitigation costs alone, and in October 2023 was awarded a $100 million grant from the U.S. Department of Energy for projects to protect grid resiliency and prevent wildfires. These projects include boosting safety features in power lines to reduce sparks that could cause fires as well as relocating some high-risk distribution circuits below the ground. Notably, the grant will also fund projects that seek to build on emerging technology, such as unmanned aerial systems—i.e., drones—that can inspect power lines for weaknesses and identify vegetation that could pose heightened fire risks.

Other segments of the utility industry are also turning to technology solutions using drones. For example, nuclear power utilities have used drones to access and inspect areas where safety may be a risk for a human inspector, or that are otherwise hard to reach. Drones can inspect construction activities, concrete structures, hard-to-reach components, storage systems, radiation mapping, and more.

Drones are only one part of this new technological equation, in fact. The other half is computer vision, which allows for automatic analysis of the images collected. Without computer vision (see sidebar), a human would have to manually review every minute of footage collected. In addition to the labor hours involved in such a task, it also introduces the risk of human error.

As such, in the case of the nuclear power industry, one technology company developed computer vision machine learning models to assist utilities in automatically detecting defects in their concrete containment structures. Using this tool, nuclear utilities can deploy drones to conduct inspections of the structures, upload their image datasets, and then use machine learning models to evaluate and review the results.

Computer Vision in Action: Inspecting Solar Panels

The solar power field has similarly embraced technological developments to mitigate risks and challenges to its infrastructure assets. For example, one large European utility company expanded its renewable energy sources to include a large solar field in Africa. In order to achieve optimal performance at these solar panel assets, the field must be regularly inspected. Many kinds of obstructions (such as sand, dirt, bird droppings, and debris) can prevent optimal performance or even cause lasting damage. However, such inspections are labor- and time-intensive, all of which is complicated by their geographic distance from the parent company.

To aid with challenges such as these, Genium, a Silicon Valley based software and engineering service company, has developed a computer vision-based solution called SolarScan. It combines data collection, processing, and analysis to streamline and improve the solar panel inspection process.

Genium’s solution involves deploying drones, such as Parrot’s Anafi model, to collect RGB (red-green-blue) and radiometric images of solar panels. When combined with information from a solar utility’s existing voltage and amperage sensors, the data collected is aggregated and processed through the cloud.

Then, using computer vision learning models that are able to learn and recognize anomalies, the SolarScan system efficiently detects equipment, system, environmental, and other issues. These range from shorted circuits to soiled panels to vegetation overgrowth and more. Finally, the system generates detailed reports and notifies a technician—who then uses customized maintenance tools and dashboards to determine exactly where the problem lies, and how and when to prioritize it.

The result is significant gains in efficiency and cost savings.

Costs and Benefits of Computer Vision Approaches

Technologies such as computer vision and drones were not widely available in the commercial arena until recently. Roadblocks to their adoption included the computing power needed to store and analyze the vast quantity of data collected from video and images, as well as the need for engineers with the expertise to create, train, and deploy the machine learning models and neural networks involved. The costs for these resources were, for many years, prohibitive to most companies.

However, it is important to note that technology is advancing rapidly, which means the landscape has changed significantly in a short amount of time. For example, compared to just a few years ago, the costs of drones have plummeted while their capabilities (including range, speed, communications, and more) have swiftly improved.

Moreover, thanks to cloud-related cost savings, computer vision technology is now entirely cost-effective. Rather than hosting the supercomputers themselves, companies can partner with cloud-based computing services to store their data, along with taking advantage of an increasing number of pre-built deep-learning vision models and platforms available from providers.

As such, the costs of deploying computer vision solutions are now becoming viable, especially when factoring in the benefits of these approaches in better detecting and reducing issues related to failing or high-risk infrastructure. The analytical capabilities of AI and computer vision technology are increasing exponentially, while the cost of using these technologies decreases. This, in turn, can save millions of dollars in human inspection costs.

Thus, executives see this technology as a way to assess and mitigate some of the utility’s major risks. While major well-established utility companies may not be ready to scale a computer vision-based inspection solution across their entire system, smaller pilot programs help to demonstrate their effectiveness in the shorter term. These programs increasingly show that computer-vision powered inspections can help prolong the lifespan of assets and provide data-driven insights to optimize their performance.

The Next Step for Computer Vision and AI Solutions

Although still in the early stages, computer vision is already greatly improving the efficiency and cost-effectiveness of inspection processes for industry assets, allowing for the quick detection of problems with solar panels, power lines, containment structures, pipelines, and other assets that require maintenance.

Looking ahead, these technologies can be taken even further to strengthen the industry’s management of its infrastructure. The next phase of deployment involves not merely using computer vision and AI to detect the issues, but to predict them.

Specifically, combined with computer vision algorithms, AI machine learning models can be set to identify patterns in detected maintenance issues and anomalies and to learn from historical data regarding when equipment failed and why. Given enough time and data, the models can then use these patterns to predict when and where the next failure will occur, before it occurs. Utility companies are thus poised to better predict when assets might be at higher risk of failure and target their limited inspection and maintenance resources accordingly.

Computer vision and AI are the key to supercharging risk-based decision-making, and are in reach for the utility industry to build upon. and evolve.

Gordon Feller is a leading advisor on energy futures and provides insights to top executives at many of the world’s leading organizations. From his Silicon Valley base, he’s worked during 40 years with large companies, associations, universities, governments, and international organizations, and he’s published more than 400 articles.

What Is Computer Vision?

Computer vision is part of the growing field of artificial intelligence. In short, computer vision enables computers to recognize and interpret information gathered from images, videos, or multi-dimensional data.

For example, computer vision is at the core of today’s driver assistance systems and emerging self-driving technology in automobiles, such as Tesla’s Autopilot. The technology begins with a network of cameras and other sensors around the vehicle that capture a continuous stream of visual data from the surroundings. These cameras pick up details about the road, traffic, pedestrians, road signs, and any potential obstacles. The onboard computer must then be able to recognize and interpret these details, and then make decisions to slow down or move based on what it “sees.”

As humans, our eyes have evolved over millennia to take in patterns of light and darkness and send that information to our brain, allowing us to instantly see and recognize the objects before us. In order for a computer to “see” something, it must be taught through machine learning and neural networks. Machine learning (or deep learning) involves building algorithms and models that allow a computer to teach itself to recognize and see what it needs to see. Through layers of processing and massive amounts of data, these algorithms extract essential features from the images and videos, such as edges, shapes, colors, and textures. Over time, the system’s predictions—that is, the prediction that a certain shape corresponds with an object it wants to identify—become increasingly accurate.

Automated driving systems are just one of many applications of computer vision technology, which is transforming industries from healthcare (by training computers to assist in medical diagnoses by analyzing radiology images, for example) to security (by enabling facial recognition to detect possible known threats), and more.