The June 2022 issue of IEEE Spectrum is here!

IEEE websites place cookies on your device to give you the best user experience. By using our websites, you agree to the placement of these cookies. To learn more, read our Privacy Policy.

Smart image analysis algorithms, fed by cameras carried by drones and ground vehicles, can help power companies prevent forest fires

The 2021 Dixie Fire in northern California is suspected of being caused by Pacific Gas & Electric's equipment. The fire is the second-largest in California history.

The 2020 fire season in the United States was the worst in at least 70 years, with some 4 million hectares burned on the west coast alone. These West Coast fires killed at least 37 people, destroyed hundreds of structures, caused nearly US $20 billion in damage, and filled the air with smoke that threatened the health of millions of people. And this was on top of a 2018 fire season that burned more than 700,000 hectares of land in California, and a 2019-to-2020 wildfire season in Australia that torched nearly 18 million hectares.

While some of these fires started from human carelessness—or arson—far too many were sparked and spread by the electrical power infrastructure and power lines. The California Department of Forestry and Fire Protection (Cal Fire) calculates that nearly 100,000 burned hectares of those 2018 California fires were the fault of the electric power infrastructure, including the devastating Camp Fire, which wiped out most of the town of Paradise. And in July of this year, Pacific Gas & Electric indicated that blown fuses on one of its utility poles may have sparked the Dixie Fire, which burned nearly 400,000 hectares.

Until these recent disasters, most people, even those living in vulnerable areas, didn't give much thought to the fire risk from the electrical infrastructure. Power companies trim trees and inspect lines on a regular—if not particularly frequent—basis.

However, the frequency of these inspections has changed little over the years, even though climate change is causing drier and hotter weather conditions that lead up to more intense wildfires. In addition, many key electrical components are beyond their shelf lives, including insulators, transformers, arrestors, and splices that are more than 40 years old. Many transmission towers, most built for a 40-year lifespan, are entering their final decade.

The way the inspections are done has changed little as well.

Historically, checking the condition of electrical infrastructure has been the responsibility of men walking the line. When they're lucky and there's an access road, line workers use bucket trucks. But when electrical structures are in a backyard easement, on the side of a mountain, or otherwise out of reach for a mechanical lift, line workers still must belt-up their tools and start climbing. In remote areas, helicopters carry inspectors with cameras with optical zooms that let them inspect power lines from a distance. These long-range inspections can cover more ground but can't really replace a closer look.

Recently, power utilities have started using drones to capture more information more frequently about their power lines and infrastructure. In addition to zoom lenses, some are adding thermal sensors and lidar onto the drones.

Thermal sensors pick up excess heat from electrical components like insulators, conductors, and transformers. If ignored, these electrical components can spark or, even worse, explode. Lidar can help with vegetation management, scanning the area around a line and gathering data that software later uses to create a 3-D model of the area. The model allows power system managers to determine the exact distance of vegetation from power lines. That's important because when tree branches come too close to power lines they can cause shorting or catch a spark from other malfunctioning electrical components.

AI-based algorithms can spot areas in which vegetation encroaches on power lines, processing tens of thousands of aerial images in days.Buzz Solutions

Bringing any technology into the mix that allows more frequent and better inspections is good news. And it means that, using state-of-the-art as well as traditional monitoring tools, major utilities are now capturing more than a million images of their grid infrastructure and the environment around it every year.

AI isn't just good for analyzing images. It can predict the future by looking at patterns in data over time.

Now for the bad news. When all this visual data comes back to the utility data centers, field technicians, engineers, and linemen spend months analyzing it—as much as six to eight months per inspection cycle. That takes them away from their jobs of doing maintenance in the field. And it's just too long: By the time it's analyzed, the data is outdated.

It's time for AI to step in. And it has begun to do so. AI and machine learning have begun to be deployed to detect faults and breakages in power lines.

Multiple power utilities, including Xcel Energy and Florida Power and Light, are testing AI to detect problems with electrical components on both high- and low-voltage power lines. These power utilities are ramping up their drone inspection programs to increase the amount of data they collect (optical, thermal, and lidar), with the expectation that AI can make this data more immediately useful.

My organization, Buzz Solutions, is one of the companies providing these kinds of AI tools for the power industry today. But we want to do more than detect problems that have already occurred—we want to predict them before they happen. Imagine what a power company could do if it knew the location of equipment heading towards failure, allowing crews to get in and take preemptive maintenance measures, before a spark creates the next massive wildfire.

It's time to ask if an AI can be the modern version of the old Smokey Bear mascot of the United States Forest Service: preventing wildfires before they happen.

Damage to power line equipment due to overheating, corrosion, or other issues can spark a fire.Buzz Solutions

We started to build our systems using data gathered by government agencies, nonprofits like the Electrical Power Research Institute (EPRI), power utilities, and aerial inspection service providers that offer helicopter and drone surveillance for hire. Put together, this data set comprises thousands of images of electrical components on power lines, including insulators, conductors, connectors, hardware, poles, and towers. It also includes collections of images of damaged components, like broken insulators, corroded connectors, damaged conductors, rusted hardware structures, and cracked poles.

We worked with EPRI and power utilities to create guidelines and a taxonomy for labeling the image data. For instance, what exactly does a broken insulator or corroded connector look like? What does a good insulator look like?

We then had to unify the disparate data, the images taken from the air and from the ground using different kinds of camera sensors operating at different angles and resolutions and taken under a variety of lighting conditions. We increased the contrast and brightness of some images to try to bring them into a cohesive range, we standardized image resolutions, and we created sets of images of the same object taken from different angles. We also had to tune our algorithms to focus on the object of interest in each image, like an insulator, rather than consider the entire image. We used machine learning algorithms running on an artificial neural network for most of these adjustments.

Today, our AI algorithms can recognize damage or faults involving insulators, connectors, dampers, poles, cross-arms, and other structures, and highlight the problem areas for in-person maintenance. For instance, it can detect what we call flashed-over insulators—damage due to overheating caused by excessive electrical discharge. It can also spot the fraying of conductors (something also caused by overheated lines), corroded connectors, damage to wooden poles and crossarms, and many more issues.

Developing algorithms for analyzing power system equipment required determining what exactly damaged components look like from a variety of angles under disparate lighting conditions. Here, the software flags problems with equipment used to reduce vibration caused by winds.Buzz Solutions

But one of the most important issues, especially in California, is for our AI to recognize where and when vegetation is growing too close to high-voltage power lines, particularly in combination with faulty components, a dangerous combination in fire country.

Today, our system can go through tens of thousands of images and spot issues in a matter of hours and days, compared with months for manual analysis. This is a huge help for utilities trying to maintain the power infrastructure.

But AI isn't just good for analyzing images. It can predict the future by looking at patterns in data over time. AI already does that to predict weather conditions, the growth of companies, and the likelihood of onset of diseases, to name just a few examples.

We believe that AI will be able to provide similar predictive tools for power utilities, anticipating faults, and flagging areas where these faults could potentially cause wildfires. We are developing a system to do so in cooperation with industry and utility partners.

We are using historical data from power line inspections combined with historical weather conditions for the relevant region and feeding it to our machine learning systems. We are asking our machine learning systems to find patterns relating to broken or damaged components, healthy components, and overgrown vegetation around lines, along with the weather conditions related to all of these, and to use the patterns to predict the future health of the power line or electrical components and vegetation growth around them.

Buzz Solutions' PowerAI software analyzes images of the power infrastructure to spot current problems and predict future ones

Right now, our algorithms can predict six months into the future that, for example, there is a likelihood of five insulators getting damaged in a specific area, along with a high likelihood of vegetation overgrowth near the line at that time, that combined create a fire risk.

We are now using this predictive fault detection system in pilot programs with several major utilities—one in New York, one in the New England region, and one in Canada. Since we began our pilots in December of 2019, we have analyzed about 3,500 electrical towers. We detected, among some 19,000 healthy electrical components, 5,500 faulty ones that could have led to power outages or sparking. (We do not have data on repairs or replacements made.)

Where do we go from here? To move beyond these pilots and deploy predictive AI more widely, we will need a huge amount of data, collected over time and across various geographies. This requires working with multiple power companies, collaborating with their inspection, maintenance, and vegetation management teams. Major power utilities in the United States have the budgets and the resources to collect data at such a massive scale with drone and aviation-based inspection programs. But smaller utilities are also becoming able to collect more data as the cost of drones drops. Making tools like ours broadly useful will require collaboration between the big and the small utilities, as well as the drone and sensor technology providers.

Fast forward to October 2025. It's not hard to imagine the western U.S facing another hot, dry, and extremely dangerous fire season, during which a small spark could lead to a giant disaster. People who live in fire country are taking care to avoid any activity that could start a fire. But these days, they are far less worried about the risks from their electric grid, because, months ago, utility workers came through, repairing and replacing faulty insulators, transformers, and other electrical components and trimming back trees, even those that had yet to reach power lines. Some asked the workers why all the activity. "Oh," they were told, "our AI systems suggest that this transformer, right next to this tree, might spark in the fall, and we don't want that to happen."

Indeed, we certainly don't.

Dead bugs don’t, giving researchers a new way to assess antibiotic efficacy

Dexter Johnson is a contributing editor at IEEE Spectrum, with a focus on nanotechnology.

In this artist’s impression, a graphene drum detects the nanomotion of a single bacterium.

The two-dimensional material graphene is basically all surface. This makes it highly sensitive to atoms or molecules because its entire volume can serve as a sensor surface. This has led researchers and industry to exploit graphene as both a biosensor and an electronic sensor for detecting the smallest changes to an environment.

Now researchers at Delft University of Technology (TU Delft) in the Netherlands have used graphene sheets to serve as a kind of drum for detecting the movements of a single bacterium. In research published in the journal Nature Nanotechnology, the TU Delft researchers created a device that can sense an object, such as a bacterium, sticking to the surface of the graphene drum and creating oscillations with nanoscale amplitudes.

“When a single bacterium is adhered on the graphene drum, it can transduce a time-dependent deflection on the graphene that is then detected by laser light,” said Farbod Alijani, assistant professor at TU Delft and lead researcher on the project.

The researchers performed experiments in a cuvette containing live E. coli bacteria in a growth medium. In the TU Delft teams’ first measurements, they were immediately able to detect the movements of the bacteria, according to Alijani.

These movements were manifested as a noisy signal within a spectrum that was consistent with biological processes. This observation is the first time that the sound generated by a single bacterium in its aqueous growth environment could be detected, according to Alijani.

The sensor is detecting oscillations that primarily come from the motion of flagella, the tail-like structures that propel the bacterium through its environment. To get a sense of how lightly these flagella beat on the graphene drums, take the force of a boxer punching a bag and divide it by 10 billion, says Alijani.

“When a single bacterium is adhered on the graphene drum, it can transduce a time-dependent deflection on the graphene that is then detected by laser light.” —Farbod Alijani

Alijani believes that the technique’s ability to trace changes in a bacterium’s motion on the nanoscale could help in the administration of antibiotics and could make it a valuable tool in monitoring antibiotic resistance.

Antibiotic resistance is determined by monitoring the oscillations levels. Persistent oscillations indicate that the bacterium is alive and resisting the antibiotic. A decrease in oscillations would indicate that the antibiotics are killing the bacteria.

Alijani points out that while there are many semi-automated antibiotic sensitivity tests currently on the market, the bacteria need to grow, which takes time. This new test provides clear indicators of antibiotic resistance within just 1 or 2 hours as compared to current antibiotic sensitivity tests that require at least 24 to 48 hours, according to Alijani.

“Our technology stands out in terms of sensitivity and speed as it can perform antibiotic susceptibility at the single-cell level using many parallelized sensors,” said Alijani, who notes that one chip can host up to 10,000 sensors.

This graphene sensor falls into the category of a biosensor as opposed to an electronic sensor. The use of graphene in biosensors is gaining broad commercial use, while electronic sensors based on graphene remain at an early stage, with companies like U.K.-based Paragraf beginning to roll out Hall-effect sensors based on graphene.

“We are picking up vibrations optically, not electrically and the actuation of the drums is simply by the bacterium itself (no electrical actuation),” Alijani added. “It is the bacterium that vibrates the sensor.”

The graphene that the TU Delft team used is what’s known as chemical vapor deposition (CVD) bilayer graphene. CVD graphene is markedly cheaper to produce than mechanically exfoliated graphene that needs to be stripped off of graphite one atomic layer at a time in a process known as the “scotch tape” method. This makes the raw material for making the device somewhat more affordable.

While raw-material considerations are always an important factor, “the immediate next step in development is to validate the technology against a variety of bacteria and antibiotics with different modes of action,” said Alijani. “As we bring the technology closer to market, we will very soon launch a startup in collaboration with our vaporization partners.”

In these next steps, Alijani said the team will look to further optimize the readout system, as well as better understand the relation between biophysical processes of a single bacterium and the nanoscale vibrations detected.

They churn out bandages and periscopes for fighters on the frontlines

One month into the Russian invasion of Ukraine, a group of more than 100 makers from all over Ukraine manufactured and supplied a number of 3D-printed products to the Armed Forces of Ukraine, the Territorial Defense Force, and the Air Forces. For security reasons, this group does not disclose most of their work. But they do share common achievements.

According to their data, 3,019 individual parts were 3D printed in the first 16 days of the war, which were used for 930 finished products. This is data from only one group of volunteers, and it is very difficult to track the total amount of help in the form of 3D-printed products. However, it is safe to say that fast, flexible 3D-printing production has shown all its advantages in Ukraine.

This is a startling accomplishment considering that before 24 February 2022, 3D printing was very rarely used in manufacturing components for military equipment in Ukraine.

There are a couple of reasons for this. First, the 3D-printing facilities and services available in Ukraine usually use fused deposition modeling (FDM) 3D-printing technology, which often results in components with poor performance and less than optimal survivability in wartime. Second, the number of 3D printers was very limited in Ukraine and did not allow for the production of certain components evenly throughout the country. And for volunteers living in Ukraine and for those like me who are outside of our home country, there were many problems and questions: what exactly to print, in what quantity, how to provide logistics in the places where the products are needed, and how to get the permits required to modernize military equipment.

Given these constraints, how has 3D printing become one of the most important activities for volunteers trying to help the Ukrainian military? It turns out the COVID-19 epidemic played an important role in resolving many of the issues associated with 3D printing before the war. During COVID-19, companies, volunteers, universities, and concerned citizens (including me) began to create a system for networking. Thanks to these communication systems and volunteer centers, it was possible to supply personal protective equipment (like faceshields) for physicians and social workers. By the beginning of the full-scale war in February, logistics systems for the 3D-printing industry had already been established.

Even so, at the start of the conflict, 3D printers were in short supply, and there was a limited supply of consumables like filament. When volunteers from abroad joined the fight, they dispatched a large number of 3D printers throughout Ukraine in a short time. In addition, citizens who had 3D printers at home began to give their printers to 3D-printing hubs established to supply components to the frontlines. Ukrainian filament companies also began to make supplies directly available, effectively resolving any outstanding questions concerning materials and printers.

But the main issue for the 3D-printing community remained: What could be 3D printed that would most help the military? The Ukrainian company 3D Tech ADDtive was the first to come up with an initiative to defend Ukraine. The company was one of the first to work on 3D printing of components for drones and weapons, but the impact of these components was limited. Therefore, when it received new information that there was a great shortage of combat application tourniquets (CATs) for the military, in just a few days they had developed a tourniquet design that could be 3D printed, and began to modify it for better performance.

The Ukrainian company 3D Tech ADDtive developed a combat application tourniquet [left] containing multiple 3D-printed parts [right].3D Tech ADDtive

Other volunteers also joined the modernization and implementation of computer-aided designs with publicly available 3D models for printing. In particular, the project “3DPrintingforUkraine” improved performance for even industrial tourniquets.

The 3DPrintingforUkraine project also developed tourniquets whose components could be readily manufactured and assembled via a 3D printer. 3DPrintingforUkraine

Printing such tourniquets can be difficult, as nonstandard filaments, including flexible materials such as nylon and others such as polyethylene terephthalate glycol (PETG), are necessary. Meanwhile, the logistics of delivering expensive printing materials are currently more difficult to solve than for more standard 3D-print composites such as acrylonitrile butadiene styrene (ABS) or PETG.

Today, however, the 3D printing of this important materiel continues, thanks to the help of volunteers and the regular donation of caring people, mainly from Eastern Europe.

Spools of 3D-printer filament fill the back seat of a car, providing a supply-chain lifeline for 3D printers across Ukraine being used to supply troops and medics working in the country's defense. 3D Tech ADDtive

As the war continued on, another shortage arose with the Israeli Emergency Bandage—a smartly designed dressing made specifically for use with one hand. Due to the large number of mobilized Ukrainians, there was simply not enough of these bandages to go around. Therefore, together with garment companies, makers have organized the production of a 3D-printed version of the bandages. In fact, after only a few days of producing these substitute Israeli Emergency Bandages, volunteers used them to complete individual first-aid kits, which were then sent to the front.

The Israeli Emergency Bandage [left], a popular staple of military first-aid kits around the world, was so much in demand among Ukrainian forces that a comparable 3D-printed bandage [right] was devised as an alternative.3D Tech ADDtive

In addition to health-care products, the 3D-printing community in Ukraine has been making tactical tools for the military. The most useful for the military are periscopes, which volunteers disguise as needed. This design of the 3D-printed periscope is quite light and consists of a 50-millimeter-diameter tube, two mirrors, and two printed parts. This gives Ukrainian soldiers encountering the enemy in urban areas a safer way to look around corners and over walls.

Both the military periscope [left] and its 3D-printed alternate versions [right] can be crucial tools for troops on the frontlines—especially in tight urban settings, enabling sometimes lifesaving ways of looking around corners and over walls.3D Tech ADDtive

Three-D printing shows amazing flexibility and can respond quickly to the needs of volunteers. The communication that was established in peacetime, through conferences and scientific and technical societies including IEEE, allows for better understanding of the needs and opportunities of each region and hub. Thanks to this volunteer-driven, maker-powered movement, the Ukrainian Army has a better opportunity to offer a worthy resistance to the Russian Army by making it possible to equip military units with necessary equipment quickly.

IEEE member Roman Mykhailyshyn was born in Ukraine and lived in the city of Ternopil in western Ukraine most of his life, becoming an associate professor in the department of automation and technological processes and manufacturing at Ternopil National Technical University in 2019. He is currently a Fulbright visiting scholar at the department of robotics engineering at Worcester Polytechnic Institute, in Massachusetts, working on a project about the manipulation of flexible objects by industrial robots.

“Being in another country when you have a war at home is very motivating,” says Mykhailyshyn. “After the news of the beginning of a full-scale Russian offensive against Ukraine, I felt despair and anxiety, but later it grew into anger at all things Russian. I’m sure a lot of people feel that way. For me, the volunteer activities and constant communication between Fullbrighters from Ukraine have joined us together and helped us to morally come to terms with what we can and cannot do.”

“Constant communication with family, colleagues, and friends who are in Ukraine is incredibly helpful, although such communications can be quite difficult,” he says. “Personal connections are one of my primary sources of information about what is happening in Ukraine. Because some of the volunteer organizations’ organizers studied or lived part of their lives in my city, I know them well.”

Mykhailyshyn notes that he made a significant portion of his connections at scientific and technical conferences, including UKRCON, which is held every two years. “Such events allow attendees to find like-minded people and establish the necessary communication,” he says. “Many of these people I communicate with, and they talk about their volunteer contribution to the victory of Ukraine. The rest of the information I receive through the social networks of official organizations and volunteers.”

Download this free whitepaper to learn how battery energy storage up to 1500 VDC can deliver power efficiencies and cost reductions

The explosive growth of the battery energy storage industry has created a need for higher DC voltages in utility-scale applications.

Download this free whitepaper and learn how you can achieve a smooth transfer of power, efficiencies and cost reductions with battery energy storage system components up to1500 VDC.