Between the Poles

Using a drone to capture high precision underground utility location data up to 15 meters deep

For a long time it has been a dream of underground utility locators to deploy a drone based system for mapping underground utility infrastructure. The advantages of such a system are speed of execution - drone-based system can gather field data much quicker than traditional land-based technology. .The second advantage is safety - no boots on the pavement allows the field operator to remain safe at all times, regardless of terrain conditions. The third advantage is precision providing cm accuracy at a depth of 15 meters.

With respect to spatial positioning we support GNSS and RTK. But we can have either a fixed reference point on the ground upon which we work or have a satellite correction of our positioning. And this is how we're able to have such a high level of accuracy in our mapping.

A most important general application of Skipper NDT technology is, as opposed to traditional methods, soil conditions have no impact on our measurements, meaning that if the soil is humid, or if you have different soil types, it will have no impact on our measurements. And the proof is that we can even see through water. A specialized Skipper NDT application is river crossings, being able to detect utilities under water is a rare capability.

A third application is an out-of-straightness assessment. The Skipper NDT technology has proven to be as precise as an invasive pipeline tool to measure the mechanical deformation of pipelines.

A caveat is that we are more suited for rural areas than urban due to the regulation for drone fight, but also due to the EM noise that can be prevalent in those areas, which is very important. We do require the standard permits to fly drones in different locations which can vary from country to country. We're more of a rural-based technology applicable for both energy and utility segments. Electromagnetic noise is one of the competitive advantages that we've developed through time.  We're able to filter those EM noises out of the signal that is required in order to have the cleanest magnetic environment. We filter interferences that could be created due to high voltage lines or metallic objects in the proximity of the pipeline.

Two main operators have validated this application, one being GRTgaz, the incumbent French gas operator, and the other one being Veolia, the large utility company. We've done operations in Europe, US, Asia and Africa. We have collaborated with prime operators such as TotalEnergies, GRTgaz, Patronus, US-based operators such as Kinder-Morgan, as well as operators in the field of utilities, such as Veolia. It is particularly interesting for markets in Europe, where there is a regulation that obliges all operators, energy or utility, to map their entire infrastructure with the inch precision by 2032.

Hardware payload

The Skipper NDT is a four-kilogram payload enabling a high-precision pipeline mapping. It is composed of several sensors, magnetic, but also centimetric GPS and IMU, sensors that calculate the distance between the drone and the ground to provide the depth of cover, and an electronic card, which interpolates all this data. The result is an acquisition pace which is three times higher than traditional methods. The device can collect data at up to 12 kilometers per hour with a data density that is up to 1,000 times more than traditional tools with a point every centimetre, if required. In addition we perform magnetometer compensation that allow us to increase by 25 times the native resolution of our magnetometers to have very high-precision data. We can detect pipelines that are metallic, so no PVC, but we can also detect active cables, cables in which a current is being transmitted. We have been actually tested on pipes going from a diameter of 80 millimetres to 1.2 meters, meaning we can go from very small to the largest pipeline seamlessly

We are drone agnostic, meaning that the drone we are showing is the DJIM 600. But we can integrate our payload on any tool being able to carry our payload. The way we market this product is either directly with operators which know us now or to service providers that might have already have a drone able to carry out payload.

In terms of software, we apply several algorithms to the data to have the cleanest magnetic information. These include field data level control, which compensate the movement of the acquisition vector that could cause potential false positive, or at least complicate the analysis of the data. But we also we filter EM interferences that could be created due to high voltage lines or metallic objects in the proximity of the pipeline. We also have several algorithms patented, which will process this data in order to provide the operator with the cleanest and most precise information.

Operational process

Here, we describe the operational process of data gathering through the Skipper NDT technology. We've simplified the process so that operator in the field will input two points and will then deploy the drone, which will go back and forth over the area. The objective is to hover over a zone with a width of 10 meters around the pipeline location. to collect data. We will then apply different magnetic acquisition filters in order to provide a magnetic map from which we will derive a number of points that will be provided to the operator within the format that suits best their system.  In this example a seven-inch diameter,gas pipeline was measured over a distance of 207 feet over 32. The number of profiles that were done by the drone was five to cover the entire magnetic anomaly. And the acquisition time for this coverage was of six minutes. The results of this acquisition were compared against an open ditch measurement, and the two tables describe basically the performance that was obtained with the Skipper technology. We can see that laterally, we have an average precision of 5.2 inches and a standard deviation of four inches. And horizontally, meaning depth of cover, we have an average precision of 1.6 inch with a standard deviation of 2.1 inch. We see here that the data is fairly constrained, meaning that there is a high degree of certitude on the data that is provided by the Skipper NDT algorithms. This shows why it is important that we gather the entire magnetic signal. We can see that on one of the pipelines that was measured, two different magnetic interferences were gathered, both in the low frequency map and a high frequency map. And this was highlighted by our algorithms. Those kind of interferences are characteristic of a metallic object. Our analysts that reviewed the data suggested that there might be an abandoned line crossing the existing line. After further investigation, it was confirmed that an abandoned line was actually crossing the new line that was built. This constitute a serious hazard potential, and hence the data was then integrated into the global information system of the client. This is why we insist on gathering a large magnetic map that allows us to gather not only the magnetic signal of the pipeline, but also of its environment.

This post is based on Luigi Kassir's (Skipper NDT) talk at Subsurface Utility Mapping Strategy Forum (SUMSF).

Canada plans to allow BVLOS drone operations in remote areas without a waiver

Canada plans to introduce in 2021 certification that would allow pilots to undertake beyond visual line of site (BVLOS) flights in isolated areas without the need for a waiver. This opens the possibility of more frequent, low cost inspection possibilities for transmission utilities with large stretches of line running through vast open areas or forest corridors.

Background

Transmission line inspections for vegetation management and other purposes are essential in ensuring grid reliability and resilience.  They are generally performed by manned helicopters or by a ground crew.  Data is collected with cameras and analyzed to detect diseased trees that could fall and hit a powerline (fall ins) or trees that were encroaching on power lines (grow ins).  These inspections are mandated by NERC in North America and are not optional.  Beyond visual line of sight (BVLOS) drones (UAVs) and software analytics using machine learning has made it possible to approach completely automating this expensive process.  Electric Power Research Institute (EPRI) has estimated that the reduction in cost with using an automated UAS compared to a helicopter inspection is between 55% and 28%. Automating transmission line inspections for vegetation management using BVLOS drones not only saves money but also could improve the resiliency and reliability of the transmission grid.

There are other important use cases as well including precision agriculture, public safety for search and rescue missions and delivery services to remote and rural communities.

Currently, there are two levels of drone pilot certification in Canada – basic and advanced – which roughly correspond to the FAA’s Part 107 waivers in the US. New rules for commercially operating UAVs in the U.S. were published in 2017 by the FAA as Part 107 of the Federal Aviation Regulations.  For non-standard flights, such as BVLOS missions, Canada require operators to acquire a Special Flight Operations Certificate (SFOC) which requires operators to fly their drones beyond visual line-of-sight in isolated areas outside of concentrated population centres and not near restricted airspace such as airports.  In addition there are special provisions for drones with onboard detect-and-avoid (DAA) solution which allows them to fly safely without human intervention even in populated areas.  For example, in March 2020 first approval for BVLOS drone flights was granted in Canada for Iris Automation’s Onboard Detect-and-Avoid technology.

However, Canada plans to introduce in 2021 a tier of certification that would allow pilots to undertake BVLOS flights in low-risk conditions without need for a waiver. This opens possibilities for transmission utilities with large stretches of line running through vast open areas or forest corridors.

Proposed routine BVLOS flights without a waiver

In January 2019, Transport Canada (TC) published new RPAS regulations for operations within visual line of sight (VLOS), and these regulations permit BVLOS operations under low-risk conditions with the issuance of a Special Flight Operations Certificate (SFOC). In April 2020 Transport Canada posted a new Notice of Proposed Amendment (NPA) to update Canadian regulations to allow for routine beyond visual line of sight (BVLOS) drone operations in “Low Risk” environments. The proposed regulatory amendments would be the first of a series of steps towards allowing more routine BVLOS operations. These amendments will not address more complex and higher-risk BVLOS operations in populated areas. Higher risk BVLOS operations will be subject to a separate regulatory process at a later date.

This proposal aims to begin to unlock the potential for beyond visual line-of-sight operations in Canada. Specifically Transport Canada is proposing the following amendments for isolated areas.

• 250 g to 25 kg BVLOS operations in isolated areas; within 1km of an area with a population of more than 25 people per square km; over an area with a population density of more than 25 people per square km; and, in controlled airspace;
• 25 kg to 150 kg BVLOS operations in Isolated Areas, and within 1km of an area with a population of more than 25 people per square km; and,
• 150 kg to 650 kg BVLOS operations in Isolated Areas with a max altitude of 400 feet above ground level.
  

Faster and safer mapping underground infrastructure using magnetometers mounted on a drone

Technologies such as ground penetrating radar (GPR) or acoustic locating for detecting underground utilities involve directing some type of signal, electromagnetic or acoustic, into the ground and measuring the reflection off underground objects. A major challenge for these techniques is attenuation which limits the depth of penetration of the signal. For example, wet soils such as clays can render ground penetrating radar ineffective.  Technologies for remotely detecting underground objects that are not susceptible to attenuation include gravimetry and magnetometry.  I recently blogged about increased sensitivity of gravimetric equipment using quantum effects, an approach that has shown promising results in academic experiments. However, collecting data is still relatively slow and requires "boots on the pavement" which always raises safety issues. Now a novel approach using magnetometry allows centimetre-accurate locations to be measured for underground metallic pipe networks.  Even more importantly, a device combining magnetometers and accurate GNSS is light enough to be mounted on a UAV or drone, which provides much faster and safer data acquisition.

In 2012 a French government’s decree (DT-DICT) mandated that energy network operators accurately map their energy (oil and gas) pipeline networks by 2026.  This decree has motivated French energy companies to assess different alternatives for locating and mapping their underground assets.   SKIPPER NDT was started in 2016 based on the concept of leveraging the physical principles of magnetism to perform contactless measurements on pipelines with the objective to improve operational safety and rationalizing operational expenditure. Their latest development involved mounting on a drone all the necessary equipment to perform a high precision magnetometric measure to detect underground pipelines. SKIPPER NDT has partnered with Total, GRTgaz, Teréga and other major French energy network operators to demonstrate the performance of their technology in real conditions. The French National Centre for Scientific Research (CNRS) has also provided an objective scientific validation of their approach. Recently I had the opportunity to speak with Luigi Kassir, Co-founder and COO of SKIPPER NDT.

The disadvantage of a remote detecting device such as GPR that has to be wheeled at a walking pace, typically in a grid pattern covering the area of interest is that it requires "boots on the pavement" which is often dangerous especially in active traffic.  Towing a rig at highway speeds as in the case of recent ground penetrating equipment is one way to speed up data acquisition and avoid boots on the pavement.  However, attenuation is still a problem and adverse soil conditions can render GPR ineffective. As an alternative SKIPPER NDT has developed a device that integrates several magnetometers and accurate GNSS geolocating equipment which is light enough to be loaded on a UAV or drone.  The advantage of this approach is that it is unaffected by soil conditions, dramatically speeds up data acquisition and avoids the safety issues associated with ground-based equipment.

Magnetometry involves detecting and measuring the magnetic field emitted by underground conducting objects.  To enhance the magnetic signal emitted by a metallic pipe an electric signal generator can be used to inject an electromagnetic signal. The SKIPPER NDT device consists of a proprietary electronic logic card, five 3-axis fluxgate magnetic sensors, a topographic GNSS+RTK capacity as well as a high-precision inertial unit. Magnetic field strength readings, measured in nanoteslas (nT), are recorded continuously by the SKIPPER NDT device and are used to compute the location of the pipe (XYZ) and its depth of cover. A suite of algorithms has been developed to interpret the magnetic field strength measurements under various conditions, for example, overcoming external electromagnetic interference along the pipeline route, such as power poles.

Most importantly the device weighs only 2.5 kg, making it light enough to be mounted on a UAV or drone. The success of Skipper NDT's drone mounted technology has been scientifically demonstrated and validated under live field conditions. SKIPPER NDT’s performances were validated in the field with the French pipeline operator Teréga. Surveyors contracted by Teréga geolocated two pipelines with diameters of 6’’ and 8’’ in an open ditch. The ditches were then backfilled.  In a blind test  the SKIPPER NDT team was invited to detect and map the pipelines using hteir drone-based device. Using their drone mounted system the SKIPPER NDT team was able to capture magnetic field readings over 482 feet in just 6 minutes. It was found that the average precision of detection of the underground pipeline was 6.1 inches.

Furthermore, because SKIPPER NDT’s records continuous measurement along the pipeline, it can precisely map bends and elbows. It can also detect metallic objects and structures near the pipeline including other metallic utilities. For example, during one of the field tests, SKIPPER NDT was able to identify another pipeline whose presence had not been previously reported but which crossed the target pipeline.

There are limitations to the effectiveness of magnetometry.  External magnetic noise can be a problem.  Sources of magnetic noise that are prevalent in urban environments can drown out the target signal and complicate detection of underground pipelines.  For this reason, the technology is most effective in rural and semi-rural areas.

In summary the advantage of the SKIPPER NDT drone-mounted device is that it provides fast, safe, and centimetre-accurate data acquisition. Its automated data acquisition and processing tools mean results can be delivered faster than conventional methods making the solution also rapid and cost-efficient.

Automated drone systems improve power line inspections and reduce costs

Transmission line inspections are mandatory in North America. Traditionally helicopters are used for these inspections, but with helicopters the risk of accidents is significant and helicopters are costly to maintain and operate.  Drone inspections dramatically reduce the risk of accidents.  Aeriosense Technologies has developed an automated drone inspection system that can reduce the cost of inspections by up to 55%. 

Background

Transmission line inspections are essential in ensuring grid reliability and resilience.  They are generally performed by manned helicopters or by a ground crew.  Data may be collected with cameras and analyzed to detect a variety of conditions including corrosion, evidence of flash over, cracks in cross arms, and right-of-way issues such as vegetation encroachment. Inspections every year or 18 months are mandated by NERC in North America and are not optional. 

With over 200,000 miles of high-voltage transmission lines and 5.5 million miles of distribution lines in the United States, improving the efficiency and reducing the risk of inspections would have a major impact on the reliability of the power grid.  Drone technology, legislation and regulations enabling beyond visual line of sight (BVLOS) flights, software for planning flights and analytics for feature identification are making it conceivable that the expensive process of inspections could be completely automated.  Even now current technologies make it possible to dramatically reduce the cost of power line inspections. According to Navigant Research, globally electric utilities are expected to spend over $13 billion a year on drones and robotics by 2026, a dramatic increase from about $2 billion currently. 

Automated drone inspections

Flight plan for transmission tower inspection

Using automated unmanned aerial systems (UAS) Aeriosense Technologies can reduce the cost and improve the safety of vertical infrastructure inspection.  Unlike many other companies which offer drone services to many different industries, Aeriosense focuses only on the electric power sectors, currently transmission but in the future distribution.  I had a chance to chat with Alex Babakov, a co-founder of Aeriosense, who has many years of experience in the electric power sector, including working for BC Hydro Research/ Powertech’s High Voltage Laboratory (HVL). 

Alex explained to me that Aeriosense provides a complete visual inspection of pylons and poles, cables, and right of way including vegetation encroachment using a UAS typically with a photo camera.  Examples of the types of problems that can be identified include broken strands in lines crossing canyons and rivers, corrosion on insulators and other equipment, evidence of flash over, cracks in cross arms, and problems in the right of way such as human or vegetation encroachment.  Aeriosense also conducts inspections as part of storm response to identify damage resulting from extreme weather conditions.

Flight plan for transmission line inspection

Automation enables Aeriosense's drone systems to provide higher quality inspections at lower cost. In preparation for an inspection flight, the available GIS and LiDAR data is analyzed with the aid of machine learning algorithms to create an optimal flight path.  With a flight path optimized in this way cost reductions of about 36% can be achieved compared to manually operated drone-based inspections.  Furthermore compared to manual drone inspections automated inspections reduce pilot fatigue and safety risks while conducting inspections.

Aeriosense is also active in distribution inspections. The sheer size of the distribution network in terms of total number of kilometers and number of pieces of equipment makes it clear that an automated approach is required. With current technology and regulations, automated inspections can be applied to some but not all distribution circuits. In the future with new technology developments and regulations automated inspections will be able to be applied to more distribution circuits.

Cost reduction

Based on data of inspection efficiencies from our past projects and the data coming from the field trials done by Electric Power Research Institute (EPRI) the reduction in cost with using an automated UAS compared to a helicopter inspection is between 55% and 28%.  The challenge is doing these cost comparisons is that there are intangibles that are difficult to quantify. Safety is one of these. Helicopter accidents are not infrequent in the electric utility industry and more often than not they lead to fatalities.  BC Hydro, for example, has a requirement for helicopters to be dual engine for redundancy and rates for this equipment are much higher.  Other utilities may not be willing to pay for the extra security whichl reduces the rates of the helicopters.

Improved safety

One of the most important benefits of Aeriosense's automated UAS approach is safety. Because of the relatively high risk of helicopter inspections, utilities have been looking for alternatives. Drone-based inspections are much safer and this provides additional motivation for utilities to move toward adopting drone-based inspections.

Development of training datasets for machine learning

Another benefit of the drone-based inspections are that utilities get a historical, unbiased record with the imagery that is captured by the drones. This enables engineers to go back and review the condition of assets not from a written record but from photos.  This provides a more objective view of how the condition of an asset has changed over time.

Collecting historical data has another important benefit in that it can provide a body of imagery that can be used to train machine learning algorithms. Collecting the data now will allow them to apply the AI tools currently being developed in the future.

Future applications of automated drone systems

The impact of more frequent, less costly inspections is improved reliability. Combining the efficiency of automated UAS inspections with defect detection using machine learning will allow utilities to get a very clear, timely situation assessment of their distribution system.  Aeriosense is currently working with the Centre for Energy Advancement through Technological Innovation on a project to help utilities identify current opportunities of automated drone applications and to develop best practices in implementing automated drone inspection programs.  Aeriosense is currently working in the transmission and distribution sector, but is actively exploring applications in the substations and generation verticals.

Another area that Aeriosense is currently piloting is the use of beyond visual line of sight (BVLOS) UAS flights for transmission inspections. FAA rules permit beyond visual line of sight (BVLOS) UAVs through Part 107 waivers.  I have blogged about Xcel Energy which obtained one of the first three waivers to fly UAVs for beyond visual line of sight operations. Xcel Energy estimates that flying drones beyond line of sight will eventually cost between $200 and $300 per mile, compared to helicopter flights that cost an average of $1,200 to $1,600 per mile.  More than 30 of these waivers have been granted and more than 20 companies are operating beyond BVLOS drones, still with some restrictions.  FAA very recently began allowing BVLOS operation using radar to track the location of the UAV.

Summary

Automated drone technology is already impacting transmission line inspections by improving safety, reducing the cost and time required to complete inspections, and by collecting an objective historical record that can provide training data for future machine learning applications. It not only enables more frequent inspections and improved reliability, but it also enables more rapid inspections, which is especially critical after a storm or other emergency situation. Research is underway at Aeriosense to determine the applicability of automated drone technology in other sectors, such as substations and generation, of the electric power industry.

First substation built using full life-cycle BIM construction goes live one month early

Hubei Electric Engineering Corporation (HEEC) was this year's winner of the Bentley Year in Infrastructure Communications and Utilities award.  One of the key objectives of the ground-breaking CNY 120 million (US$16.8 million) Miluo Western 220kV substation project was to reuse the design BIM model and data during construction and to provide a virtual reality BIM model to the owner on completion of the project to improve operations and maintenance procedures. As a result of the application of digital technology including reality modeling using UAVs, full life-cycle building information modeling (BIM), integrated BIM and geospatial, construction monitoring, and construction simulation, construction was completed 30 days ahead of schedule saving CNY 6.3 million (US$ 895,000).

Hubei Electric Engineering Corporation (HEEC) began applying 3D design to projects beginning in 2013. In 2017 the State Grid Corp began to promote 3D design and held a competition to select the best design companies to carry out 3D demonstration projects. HEEC came in first in this competition and was awarded the Miluo Western 220kV substation project. In addition 3D design standards for China's electric power industry had been developed and HEEC was a leading contributor to six of these standards. On this project HEEC strictly implemented these standards during design.

The Miluo Western 220kV substation supplies power to 160,000 customers in a 16 km² area. The total investment in the 10 month project was CNY 120 million ($16.8 million).

Digital technology including reality modeling using UAVs, building information modeling (BIM), integrated BIM+geospatial, construction monitoring, and construction simulation, enabled collaboration between site preparation, mechanical design, electrical design, protection, and communications design teams.  The challenges that HEEC targeted on this project were several.

First of all, the site was constricted because of existing houses and a key objective was to minimize impact on the existing buildings. By scanning the area and using geospatial reality modeling techniques, the area occupied by the substation was reduced by 22% saving CNY 2.5 million and avoiding the demolition of six houses.

Secondly, the project was the first to apply the 3D standards mandated for the entire Chinese utility industry. The Miluo Western 220kV substation is among the first batch of 3D demonstration projects based on 3D standards.  This is the first of the 3D demonstration projects to go into operation, in June 2019.

Third, because the site chosen for the substation was constricted, cables and equipment were closed together than in other substations. HEEC reengineered the design process and developed a series of system processes to ensure the quality of 3D design by collaborative design between mechanical, electrical, protection, communication, and site preparation teams. During design, clash detection between these different elements avoided 30 instances of rework during construction saving CNY 800,000.

Fourthly, from the beginning the objective was to find a way to reuse the design BIM model during construction. Typically a separate BIM model is developed by construction contractors to be used during construction. HEEC investigated ways that 3D pdfs, mobile handhelds, cloud storage, and other technologies could be used to enable the 3D design BIM to be used to schedule construction.

Fifth, HEEC used reality capture techniques including UAVs to monitor construction progress on a daily basis. Due to uncertain factors factors in the construction process, there will necessarily be deviations between planned and actual construction. By monitoring these on a daily basis ensured that these do not hold up the project. At YII2019 HEEC showed an amazing video with actual and planned construction side by side where each frame represented a day.

Finally, an important goal was to use digital technology to improve operations and maintenance procedures. At the end of the project HEEC was able to hand over to the State Grid a virtual reality BIM+geospatial model designed to improve operations and maintenance procedures.

In summary digital technology including reality capture, construction simulation, and full-lifecycle BIM resulted in tangible benefits; avoiding the demolition of 6 houses and reducing the area required for the substation by 22% for a saving of CNY 2.5 million; significant savings in materials saving CNY 210,000; avoided rework saving CNY 800,000; reducing design time by 65 days; and completing construction 30 days ahead of schedule generating economic benefits of CNY 1.5 million. Taken together the total investment was reduced by CNY 6.3 million (US$ 895,000). In addition by providing a virtual reality model to the owner on completion of the project it is expected that substantive savings will be achieved during operations and maintenance.

Mapping underground utilities prior to design helps highway construction project complete on schedule and below budget

The winner of the Bentley Year in Infrastructure 2019 Roads and Highways award is a remarkable project by Foth Infrastructure and Environment LLC for a $38.9 million highway revitalization project in Cedar Falls, Iowa. This project was unique for several reasons. First of all, before design, a 3D model of all underground infrastructure in the right-of-way was created. This was an important factor enabling the project to complete on-time and 3% under budget. Secondly, Foth projected that innovative design including roundabouts would save the community $32 million over 25 years by reducing the number of accidents, decreasing travel time, lowering fuel usage and emissions, increasing property values, and leading to greater economic growth.

University Avenue was an Iowa Department of Transportation (DOT) six-lane, divided highway which the City of Cedar Falls took ownership of in 2014. Built over 60 years ago, the corridor carries 20,000 vehicles per day and was suffering from deteriorated pavement, a crash rate 20% above the state average, a lack of pedestrian and bicycle accommodations, a pedestrian death, and inefficient traffic operations.

Innovative, advanced traffic designs were developed; multi-lane roundabouts, a dogbone interchange, and complete streets – which allowed for a reduction of lanes that still decreased congestion and increased overall traffic flow. Technical innovations included use of drone and mobile scanning utilized for reality modelling, a building information (BIM) for creating 3D modles and fostering collaboration, and 3D visualization and traffic modelling for communicating alternative designs to stakeholders and public.

The project is one of the few of which I am aware in which a 3D model of underground utilities was developed before the start of the design phase. This enabled the detection of 200 utility conflicts during the design phase avoiding utility conflicts during construction and was the primary reason that the project completed on schedule and 3% under bid saving the City $700,000.

The project is also unique in that the design and build contractor estimated the benefits of the proposed innovative design using roundabouts over the 25 year lifetime.  It was projected that the community would recognize $32 million of benefits from fewer accidents, faster travel times, less gas consumption and associated emissions, growing property values, and increasing economic growth.

From a public safety perspective, Foth estimated an 89% reduction in personal injury accidents. As owner of the highway, the City is projected to realize $27 million in benefits from economic growth, corridor improvements, avoided traffic signal maintenance, and reduced lighting costs. Reality modeling including mapping underground utilities prior to design,  innovative highway design, and digital design technology based on BIM saved $521 thousand in project costs by eliminating utility conflicts and the resulting change orders, enhanced public engagement, and design acceleration.

Remarkably, despite facing aggressive schedules, vast information coordination, complex utility coordination, intense public involvement, and a very tight design schedule involving public participation, Foth was able to deliver the project on time and below budget.

Automating transmission line inspections with BVLOS drones and machine learning

Transmission line inspections for vegetation management and other purposes are essential in ensuring grid reliability and resilence.  They are generally performed by manned helicopters or by a ground crew.  Data is collected with cameras and analyzed to detect diseased trees that could fall and hit a powerline (fall ins) or trees that were encroaching on powerlines (grow ins).  These inspections are mandated by NERC in North America and are not optional.  Beyond visual line of sight (BVLOS) drones (UAVs) and software analytics using machine learning has made it possible to approach completely automating this expensive process.  Automating transmission line inspections for vegetation management using BVLOS drones not only saves money but also could improve the resiliency and reliability of the transmission grid.

Mapping transmission lines with BVLOS UAVs

At ILMF 2019 in a very interesting talk Paolo Guzzini of Delair described how BVLOS drones has been deployed to map transmission lines in remote areas of Iceland.  Delair conducted two surveys on behalf of EFLA Consulting Engineers, a general engineering and consulting company. EFLA needed to map existing power lines and survey an area where Landsnet, the main transmission system operator in Iceland, planned to build a new power line between two cities. Iceland's airspace regulations allows BVLOS UAVs to be used in areas that are not densely populated. Delair deployed fixed-wing drones to gather two types of data: photography and LiDAR.  The drones were programmed to fly a predetermined flight plan so the pilot had to only monitor the drone's progress and be available for emergencies.  Delair’s drones were able to capture comprehensive data along the planned route and deliver the processed results to the client the next day — all at a fraction of the cost of hiring a helicopter or ground crew.  UAVs with a range of 110-120 minutes flew for up to 10 km per flight. In one case the UAV flew a total of 70 km with a 21.4 megapixel photo camera to capture a corridor of 200 m width to an accuracy of 0.5 m vertical and 0.2 m horizontal accuracy. The actual vertical accuracy achieved was 0.15 m.  In a second mission a UAV flew at 100 m a corridor 10 km in length with a LiDAR camera to capture a corridor 200 m wide with the objective of vertical accuracy of 0.5 m and horizontal accuracy of 0.2 m. The actual accuracy achieved was 4 cm horizontally and 2 cm vertically. The advantage of using LiDAR is that it can achieve better accuracy in location especially in the vertical dimension. In addition the LiDAR approach was able to detect detect conductors and poles.  In both cases the range of the flights was limited by weather conditions, radio range (15 km), and weight of the camera payloads.

Reality capture for vegetation management of transmission lines with BVLOS UAVs

Periodic transmission line inspection is mandated for North American transmission lines by NERC.  Inspections are currently carried out by manned helicopter or ground crews which are expensive and for that reason not done more often than is required by NERC.  Full automation, which is possible with BVLOS UAVs and automated feature extraction of lines, pylons, vegetation and ground, would dramatically reduce the cost of inspection and enable more frequent flyovers.  The major benefits would be lower costs and a reduction in the number and length of outages resulting from vegetation transmission line interactions.

In the U.S. on July 15, 2016, President Obama signed the FAA Extension, Safety, and Security Act of 2016. The 2016 Act requires the FAA to establish a process to allow certain UAV operations related to utilities, pipelines, and oil and gas production to be conducted beyond the visual line of sight (BVLOS) of the operator and either in the daytime or nighttime.  As a result FAA rules permit beyond visual line of sight (BVLOS) UAVs through Part 107 waivers.  I blogged over a year ago about Xcel Energy which obtained one of the first three waiver to fly UAVs for beyond visual line of sight operations.   Paolo related that 32 of these waivers have been granted and that more than 20 companies are operating beyond BVLOS drones but still with some restrictions.  Paolo related that the FAA very recently began allowing BVLOS operation using radar to track the location of the UAV.  Paolo expects to see many long-distance UAV operations in the near future, making drones fully competitive against helicopters.

Over 20 countries have regulations that permit BVLOS UAV flights. In some areas of the world, Delair has even been able to control flights via a 3G data connection, allowing a drone to inspect up to 30 miles in a single flight.

Post-processing

Paolo told me that Delair's post processing software uses a combination of machine learning and other technologies to classify LiDAR point clouds with 85 to 90 % reliability in order to identify potential vegetation encroachments.  I have blogged about Enview and other companies that also offer software solutions for this problem.

Conclusion

Rapidly evolving regulation to permit BVLOS operations, Delair's successful deployment of BVLOS drones with radio or 3G communications and automated post-processing, often cloud-based, to identify potential vegetation encroachment indicate that we are on the cusp of being able to completely automate this type of transmission line inspection.  Using drones and automated feature extraction software not only dramatically reduces the cost of these inspections but makes possible more frequent inspections including on-demand - for example immediately after storms - which would impact positively the reliability and resilience of the transmission grid.

Automating vegetation management for electric power transmission lines using deep learning with LIDAR

With FAA rules permitting beyond visual line of sight (BVLOS) UAVs available through Part 107 waivers the cost of capturing the data required for vegetation management for power lines, whether transmission or distribution, could drop dramatically.  However, manual post processing of the imagery to identify areas of vegetation encroachment, grow-ins and fall-ins, remains an error prone, tedious and time consuming process.  I have come across several firms (for example EPRI, SiteSee, Trimble RealWorks) over the years that have been working towards automating the process for identifying vegetation encroachments including grow-ins and fall-ins, but I have not encountered a utility that is using this technology operationally.  At Distributech 2019 in New Orleans I had a chance to chat with a startup Enview, based in San Francisco,  who described to me how their software has been implemented  over 7,000 miles of transmission lines last year to automate the identification of vegetation encroachment from LiDAR point clouds.  Furthermore, they expect the total distance covered to double this year.

Vegetation is a significant source of outages for many utilities. In some regions from one quarter to one half of all outages can be ascribed to vegetation.  Vegetation encroachment initiated the massive North American northeastern outage of 2003.  It is so critical for utility operations that utilities even have a vice president responsible for vegetation management.  It can be costly for utilities to identify areas of high risk vegetation encroachment which typically are carried out by manned helicopters.  Scheduling and routing crews to prune the trees that require attention is often sub-optimal adding to the cost of vegetation management.  Optimizing these processes can produce significant benefits in the form of fewer outages and reduced costs. I have blogged about Energisa in Brazil who used satellite stereo imagery to prioritize and optimize tree pruning and ComEd's pilot project in Chicago that uses LiDAR imagery to not only prioritize and optimize but also to change the pricing structure for procuring distribution tree trimming.

Enview applies machine learning, computer vision, and geospatial big data analytics to highly reliable feature extraction from LiDAR point clouds to identify the exact location, height and extent of trees and then to determine proximity to power lines and other utility equipment.  Utility GIS data can be integrated and results visualized using colour-coding to identify the level of risk for each tree.  This enables the utility to prioritize their tree trimming and optimize contractor routing.  In environmentally sensitive areas the visualization can be used to inform discussions with the various stakeholders responsible for different land uses.

At Distributech I had a chance to chat with Krassimir Piperkov, COO at Enview, who described how this technology has been deployed on more than 7,000 miles of infrastructure last year, and is on track to be deployed on over twice that much this year. He gave a very recent example of annual vegetation management analytics performed on 230kV transmission lines in California. These lines run through suburban and environmentally-sensitive, heavily forested areas.

3D visualization of a 230kV span (blue lines) with potential grow-in (yellow) and fall-in (red) vegetation that falls within client-determined clearance boundaries.

LIDAR data was acquired by a third party through a helicopter-mounted Teledyne Optech LiDAR sensor package. Four band imagery was co-collected during the LIDAR survey and registered into an orthomosaic of RGB and NIR bands. The data collection generated almost 2GB per linear mile. The Enview data analytics engine processed this data to (i) first classify the massive amounts of LiDAR data and (ii) then automatically calculate potential grow-in and fall-in vegetation points with high speed and accuracy. Insights and data were delivered to the operator not only via GIS-compatible file format for incorporation into existing workflows but also via an intuitive web portal accessible across silos. As a result of the superior accuracy and speed, the client was able to confidently prioritize vegetation prescription work to the areas that need it most and do so much sooner than otherwise possible with traditional methods.

The results being achieved with Enview's technology suggest that we are on the cusp of the widespread adoption and application of deep learning technology to automate specific tasks. The combination of UAVs for data collection and machine learning for feature extraction dramatically reduces the cost of inspections for utilities. This would permit them to be done more frequently than with manned aircraft and would lead to improvement in the reliability of above ground electric power and communications networks.

Generating real-time 3D reconstructions using a UAV equipped with a digital camera linked to a laptop

Drones are very efficient at capturing a large volume of data that often requires hours of post-processing in the cloud to create 3D reconstructions that can be used for 3D visualization and other applications.  In his talk at SPAR3D 2018 in Anaheim Michael Jones of Lockheed Martin showed how a drone with just a photo camera and access to a GPU (graphical processing unit) on a laptop can use SLAM for real-time 3D reconstruction enabling advanced visualization and 3D modeling for immediate ‘in context’ information. The ability to generate 3D point clouds and 3D imagery in real-time as a vehicle flies enables applications that need to operate real-time, for example, the ability to automate the inspection of large critical infrastructure in real-time, comparing architectural drawings to actual 3D imagery for immediate progress updates.

SLAM is a technology that uses the identification of 3D objects from a 2D or other camera to identify 3D objects which can be used for 3D reconstruction. Applications include anything requiring 3D information such as collision avoidance, inspections, or navigation in GPS-denied locations. I have blogged about a a UAV equipped with LiDAR and using SLAM for navigation can be used in areas where GPS is not available. But LiDAR is heavy and expensive and requires an expensive drone to carry it for a significant amount of time.

Modern GPUs are designed for massively parallel computation and SLAM, which is based on triangulation, can be parallelized to take advantage of this capability.  Laptops equipped for gaming include very fast GPUs. In his talk Michael demonstrated how a drone with just a photo camera connected via a wireless link to a laptop can use SLAM to create a 3D reconstruction of an area in real-time.  The software, developed in Canada, is called Hydra and is commercially available. The pro Michael hinted at work underway that would enable SLAM processing onboard the drone.

Geospatial imagery analytics market expected to grow rapidly

An very interesting report on the geospatial imagery analytics market in 2017 and its growth through 2022 has been released by MarketsandMarkets. The analysis looks at different types of imagery analytics (imagery analytics, video analytics), how the imagery is collected (GIS, satellites, UAVs), and how it is used by different vertical industries (defense & security, insurance, agriculture, healthcare & life sciences). The major players considered in the study include Hexagon AB, Harris Corporation, DigitalGlobe, ESRI, Google, Trimble, RMSI, Planet Labs, UrtheCast Corporation, Fugro N.V., KeyW Corporation, and Satellite Imaging Corporation.

The study estimates that the geospatial imagery analytics market was worth about US$ 3.41 billion in 2017. The video imagery segment is expected to grow at a higher rate than the image analytics segment. The study also sees a high growth rate for UAVs as a way of collecting imagery and video. Surprisingly for me, the healthcare and life sciences segment is projected to grow at the highest rate among different industry segments.  North America, followed by Asia and Europe, are expected to be the largest markets for imagery analytics in 2022.  MarketsandMarkets projects a high growth rate (31.1% CAGR) for geospatial imagery analytics reaching US$ 13.21 billion in 2022. Advances in artificial intelligence and big data are expected to be major contributors to this high growth rate. 

The analysis is based on a bottom-up approach by estimating the revenues of key market players. The data were verified through primary research by conducting extensive interviews with chief executive officers, vice presidents, directors, and other executives.

"Geospatial Imagery Analytics Market"