Aviation and digital technologies have teamed up and taken off in directions we never dreamed of a few years ago. When Category 5 Hurricane Matthew was churning up the ocean in late September and early October last year, the RQ-4 Global Hawk unmanned aircraft system (UAS) was flying through it. The UAS, more commonly known as a drone, dropped expendable sensors to record temperature, pressure, humidity, wind speed and direction. It was also outfitted with special radar to examine the storm’s formation, structure and intensification. The data was delivered in real time to the National Hurricane Center, where sophisticated software turned it into useful information.
The Global Hawk is huge by drone standards, about the size of a private jet. It is capable of long-duration flights (more than 30 hours) and high altitudes (60,000 ft). This particular drone belongs to NASA and was monitoring the hurricane for the National Oceanic and Atmospheric Administration (NOAA).
The NASA/NOAA drone is a blending of the latest state-of-the-art drone airframe with cutting-edge instrumentation packages and sophisticated analytical software to make sense of the huge amounts of data received. It’s beyond-visual-line-of-sight (BVLOS) technology and many other onboard systems are unmatched by anything accessible to commercial drone service providers. This UAS represents what is possible with today’s airborne applications. It allowed NOAA to safely gather vast amounts of storm-related data without placing pilots and other aircrew members in harm’s way.
Trickle-Down Technology
Hurricane Matthew gave commercial UASs the opportunity to aid in the restoration effort. Thanks to the Federal Aviation Administration’s Small UAS Rule (FAA regulation Part 107), filed in August 2016, commercial operators with exemptions under Section 333 were able to fly storm-recovery missions. NextEra Energy Inc.’s Florida Power & Light and Southern Company’s Georgia Power divisions used drones to evaluate damage to power lines and other infrastructure in areas impacted by severe flooding. Drones were flown over power lines to determine where trees and debris needed to be cleared to reach the sources of power outages without endangering workers.
Sophisticated technology such as high-resolution imaging systems, automated analysis of data and of course drones initially had their roots in military applications, but as the technology advanced, much of the declassified technology trickled down to the commercial marketplace. While our industry doesn’t really need a drone with the capability to fly at 60,000 ft, it can use the quantitative remote-sensing apparatus used by the RQ-4. Utilities also can benefit from the RQ-4’s high-tech BVLOS equipment for long-range surveys of transmission lines required by the North American Electric Reliability Corporation (NERC) standards.
Previously, the only way utilities could achieve NERC’s FAC-003 requirements was with airborne inspections using fixed-wing aircraft and helicopters. Since then, aviation technology has provided new applications with new possibilities in the form of the drone. Commercial drones became available several years ago, and utility personnel started tinkering with the ultra-stable flying machines. In 2014, San Diego Gas & Electric became one of the first utilities to receive FAA approval to test-fly drones in conjunction with transmission line inspections. From that point on, drones have become another device in the industry’s collection of aircraft for grid monitoring.
Hardware Advancements
The possibilities appear to be endless as engineers investigate and explore the technology. Interestingly, Goldman Sachs Research reports that today’s drone technology has surpassed manned aircraft in endurance, range, safety and cost efficiency, which is supported by the NASA/NOAA UAS flights into Hurricane Matthew. Goldman Sachs expects the next generation of UASs to widen the gap between manned and unmanned aircraft, and to give users more flexibility in terms of equipment and deliverables. That does not mean UAS airframes will be replacing fixed-wing aircraft and helicopters. It means the grid has another tool.
It is up to the utility problem solvers to define the mission, select the instrumentation payload that will provide the data necessary for the mission, and then decide which airframe is best for that payload. To get the best results, users need to educate themselves about the airframes and instrumentation payloads that are available to capture the data. It is too bad there is not a cheat sheet for all this airborne technology, but given the speed at which the technology is advancing, it would grow to an unmanageable size very quickly. Let’s look at the imaging hardware.
LiDAR has become the workhorse of imaging technology for providing data to build 3-D models of vegetation, transmission lines and other infrastructure. Courtesy of Picket.
LiDAR Advancements
Light detection and ranging (LiDAR) has proven to be the go-to technology for infrastructure monitoring. Its data can be turned into detailed 3-D geometrical images that are staggering in their clarity. In the early days, LiDAR units were huge and required heavy-duty airframes to get them airborne. But as the technology matured, the size decreased, the image resolution increased and the data grew immensely. The shrinking payloads allowed for smaller aircraft, but it didn’t stop there. Miniaturization has continued this process and resulted in payload packages sized for drones.
LiDAR is a sensing technology based on pulsating laser light. Simply put, a laser pulse travels to the target and returns to the sensor with X, Y and Z coordinates for everything in the beam’s path. The LiDAR data points can be accurate to millimeters, depending on the type of equipment used. This collection of points defines what is referred to as a LiDAR point cloud, which is a 3-D geometrical representation of the ground surface and the objects above it. Today’s LiDAR systems are classified as small-footprint imaging or large-footprint imaging. Most of today’s commercial LiDAR use the small-footprint technology. These LiDAR systems can use single or multiple pulse schemes. The multiple pulse schemes fire the next laser pulse before the previous pulse’s reflection is received by the unit. This type of LiDAR allows the aircraft to fly at higher altitudes, capturing data for larger areas.
Traditional linear LiDAR is being challenged by what some providers refer to as the next generation of LiDAR. This new breed of LiDAR is based on avalanche photodiode array technology, and is referred to as continuous wave, flash, single photon or Geiger-mode LiDAR. This next generation of focal plane array LiDAR has been around for many years, but it only recently has been made available to the commercial market.
Like the drone, Geiger-mode LiDAR was developed for the military and has been restricted to government use, but in the past couple of years that changed with declassification. Harris Corp. brought this technology forward from its earlier government work with the technology to address commercial applications. The technology has been adapted from tactical systems to wide-area mapping systems, but there is an issue to be addressed. The array LiDAR generates an enormous amount of data with each mission that has to be dealt with, but more on that in the software discussion.
Geiger-mode LiDAR is a totally different approach than the old-school LiDAR. Geiger-mode LiDAR uses a type of sensor that is based on focal plane array technology. To contrast the two offerings, linear LiDAR uses a single pulse beam that zigzags across the ground as the aircraft flies forward. The array-based system uses a circular projection beam that illuminates the ground with a beam that continually rotates while the aircraft flies forward. This lets the laser hit the target from multiple directions, which eliminates any shadowing effect the linear approach generates. Geiger-mode collects hundreds of thousands of points, making for higher densities, and relies on software to process the output resolution. This allows for high-altitude flights at faster speeds and delivers higher resolution than linear LiDAR can provide.
Photogrammetry
Photogrammetry also has been getting a great deal of attention in the fields of measurement, inspection and surveying. It is a system that uses triangulation, or in this case aerial triangulation, of photographs to make measurements. The process that chiefly interests the electric utility industry is called stereophotogrammetry, which entails estimating 3-D coordinates of points on a surface using two or more photographic images. Common points are identified on each image, which is referred to as multi-ray photogrammetry. The rays are the line of sight from the camera to the point on the object. Typically, this process uses large-format aerial cameras to capture images. There is a large fore and side overlap in each of the stereopair photographs, which are pixel-to-pixel matched using algorithms via computer programs. This multi-ray process allows providers to develop 3-D topographic maps, vegetation management models and detailed models of facilities, much like those generated by LiDAR imaging.
Interestingly, photogrammetry and LiDAR imaging applications are increasing rapidly, which has brought about a discussion as to which one is the best and which one is the most cost effective. The answer to those questions is, it depends. What type of aircraft is going to be used? What is the mission? What are the deliverables? Both technologies have their best applications and limitations, so it is fair to say neither tool is for every task.
Photogrammetric imaging models can be used to accurately determine the amount of material in a tailings pile. Courtesy of Aaron Ames, POWER Engineers.
LiDAR is inherently more accurate than photogrammetry, but photogrammetry’s point cloud has an RGB (red, green, blue color information) value for each point. That means the user has a colorized point cloud, which is easier to analyze and interpret. LiDAR point clouds can be colorized with ortho photos, but many users say it does not look natural. Photogrammetry offers more resolution for applications where visual appeal is needed. To be fair, LiDAR surveys can be done in any lighting condition or after dark, which is a real handicap for photogrammetry. It really is up to the user as to which one is best for their application. The user must ask the right questions and know what they want to get the best results from the provider.
Software and Big Data
Aircraft armed with LiDAR, high-resolution cameras, thermal imaging systems and UV-visible corona cameras, and other sensors have the capacity to generate more point cloud data than the average desktop computer can hold, much less process. Aerial inspection technology has gotten so advanced that utilities can now see individual trees along the right-of-way and identify imminent threats to the line. They also can see extreme details of specific transmission structures supporting the transmission line for maintenance assessments. It’s part of the law of accelerating returns where engineers start improving technologies by adding features and merging applications, taking everything to new levels of functionality.
This is becoming a real problem for end users as they try to turn these massive amounts of data into useful information. The size of data has gotten so large. We have gone beyond terabytes and jumped almost exponentially to amounts such as petabytes, zettabytes and yottabytes. Geeks point out that a petabyte is 1 quadrillion bytes, and that is a lot of data. To get an idea of how large this really is, think about a gigabyte. A gigabyte equals 7 minutes of high-definition TV video. Now think about how much data is generated during a long flight building 3-D maps of a transmission line. Welcome to the era of big data, which has ushered in new approaches to handling data. Recently, Oracle stated that companies that understand how to manage, mine and decipher big data will prosper. Those that don’t understand big data will not survive.
A maintenance crew working from an aerial lift bucket checks out a suspicious area on a pole while a drone guides their work. Courtesy of AAIR.
Don’t Freak Out
While all of this imaging technology was advancing, the geek squad developed a solution to the flood of imaging and sensor data. It is called cloud technology, which is another name for cloud computing via the internet. This technological marvel includes storing and data mining software that is cloud based. This allows the data to be stored and manipulated using computing power/software over the internet. Cloud technology is maintained, operated and managed by cloud computing service providers with remote storage servers. One news release called it an invisible computer. Several aerial inspection and monitoring companies have taken advantage of the cloud computing concept and are providing their clients with detailed reports, action items, 3-D models and graphics.
Austin Energy and the IC2 Institute Global Commercial Group at the University of Texas at Austin are participating in a research project testing image-pattern recognition software that detects worn and damaged equipment. The software uses algorithms that compare images of the installed hardware to images of new transmission equipment. The algorithm automatically flags worn or damaged tower components, hot spots and coronas without human intervention, and notifies Austin Energy of needed action.
Siemens is developing a new automated method to systematically inspect industrial applications using drones. The system uses flying sensors such as high-resolution color and near-infrared cameras. The data is sent to Siemens via a web interface for smart imaging analysis to identify trouble spots. Siemens will use this new technology to inspect wind farms, electrical grids, pipe lines, buildings and transportation systems.
Sterlite Grid, India’s largest developer of private transmission systems, signed a collaboration agreement with Sharper Shape to use Sharper Shape’s UASs and sophisticated software for digital inspections and analysis of its transmission system. The data from the inspections will be examined automatically using Sharper Shape’s cloud-based proprietary software algorithms to review LiDAR, high-resolution photographs, UV corona photos and equipment condition. It also will automatically produce detailed reports classifying the condition of conductors, structures and vegetation. After review by personnel, the cloud-based software can deliver the final inspection reports directly to Sterlite Grid.
Recently, GE stated that it had several pilot projects with utilities in the works to use drones and cloud-based computing to improve, maintain and inspect the utilities’ infrastructure. In addition, GE Ventures has invested in San Francisco-based Airware, which has built a hardware, software and cloud services platform that will enable utilities to develop and safely operate commercial drones. GE sees UASs as flying sensors that could be used to inspect wind turbines, transmission lines, substations and other critical infrastructure. GE’s assessment software and cloud-based computing will allow information sharing across the enterprise.
Access to transmission lines in environmentally sensitive areas and maintenance outages are not a problem when the work takes place from a helicopter. Courtesy of Air2.
Game Changers
Airborne inspection and monitoring technologies have definitely proven to be game-changing tools for utilities. As these technologies are deployed across the electrical grid, the amount of data being produced has increased exponentially. This has become a real problem for end users and providers alike. Oracle made some predictions about big data in 2017. The company said we live in a world being driven by data. Oracle went on to say big data and cloud-based technologies are making dramatic transformations and characterizing how organizations define their data storage. Their choices of cloud technologies will make a critical difference in the organization’s future.
Fortunately, advanced software is coming to the rescue. It is taming the game-changing aerial technology. Software developers have tackled the big data challenge with software designed to automate the data-handling processing. They are using self-learning algorithms to calculate and analyze the collected data to the point where they can automatically identify and prioritize current and potential problems without human input.
The era of big data is here, and the marketplace is creating innovative methods to address the issues of big data post-processing. Utilities must become savvier on airborne technology and analyzing big data. Quickly being able to identify problems along transmission and distribution lines will improve reliability and reduce costs for everyone.
