Between the Poles

Eliminating the as-built backlog by automating field utility data capture

The as-built update backlog is the time it takes for as-builts submitted from the field after completion of construction for newly installed network equipment to get entered into your utility GIS.  It is generally measured in months which is appalling today when we have real-time or near real-time data about so many aspects of the real world from traffic to weather.

Field sourcing

The key to solving this problem is to enable crowdsourcing of updates from the people who actually do the construction, the field workers, so construction crews can make those updates where and when the work happens. In order to do that, we really need data capture tools that are much faster and simpler, in other words, more automated than today's solutions.

There are a number of enabling technologies that can change the way that we approach this problem; computer vision, machine learning, augmented reality, voice interaction, and reality capture can all play a role. A lot of these things are being driven by other industries outside the geospatial technology industry. Part of the challenge in bringing together a solution for the as-builting process is looking broadly at these different types of technologies and where they're applied and bringing them together into a solution for utilities. You need to enable the people doing the construction to capture accurate location and attribute data quickly and efficiently. Anyone should be able to use the system with minimal or no training and it should use inexpensive commodity equipment.

Automating feature recognition

As an example, we're in a new subdivision construction site and we're going to do some as-built data capture of some electric equipment using an iPad. We point our mobile device at a piece of utility equipment and the application automates the process of detecting what the equipment is. The user just taps the screen to add the feature, an electrical pedestal. Next, we point our mobile device at the neighbouring electric meter. And again, it's automatically recognized and we tap to confirm that we want to add it.  We can then scan the meter to add more detailed attributes according to rules that we've set up.

Now we want to add a cable. If the trench was still open so the cable was visible, we could automatically recognize it like we did with the other features, but since it's hidden, we select it from a list and then just tap to add points along the path of the cable. As we walk, you can see the cumulative distance of the cable path being shown at the top. Lastly, we'll add an electric cabinet in the same way and scan its attributes, including the transformer number and manufacturer on the front side. Then the KVA from the yellow 50 on the side. We zoom in more closely to get additional attributes like the serial number, work order number from the smaller label.

Once we have completed scanning the various objects, the data gets pushed automatically to the GIS with no user intervention needed. We can see the features that we just captured on a map with their attributes plus photos. The newly captured data about the equipment is immediately available in any GIS client. For example, we can see the newly captured equipment data on a handheld mobile device or it can be viewed on any web browser. The data can then be pushed on from the GIS to real-time systems like an ADMs outage management system or network management system.

The app is able to automatically recognize features and attributes using machine learning based image recognition, to tell what type of feature it is, read barcodes and QR codes, and use text recognition to capture detailed attributes. The additional attribute information also reinforces the feature recognition in some cases. YYou can also use voice recognition as a backup if there's a scenario where you have to correct something that was misrecognized or something that can't be identified correctly.

Lastly is an example of capturing very detailed attributes quickly. We captured 22 attributes in just over a second from an electric meter.

Location accuracy

Location accuracy is particularly important for underground assets. AR gives much greater relative accuracy than consumer GPS and phones do. It measures accurate distances and angles in 3D for recording of relative asset layout. In a range of tests we found AR is accurate to about 1%. As a simple example. we measured a 10 foot distance next to a tape measure. We've added a virtual pipe and you can see the measured distance in the augmented reality app is 10 inches and it's within an inch of the 10 foot mark or within about 1%. Tests of up to about 100 feet have found the same 1% accuracy. AR provides much better accuracy than you could get with anything except the very highest precision GPS.

Visual positioning system

Apple's GeoAnchors is their implementation of the AR Cloud. The idea is that as you're moving your mobile phone around among 3D objects captured in a street view scene, it uses the camera view to match what the camera is seeing against the highly accurate street view type imagery, which Apple calls look around imagery and that refines the position.

Conclusion

Field sourcing of data is key.. You need to enable the people doing the construction to capture the data, and these new technologies really open up some very exciting new ways that we can provide systems to the construction workers to enable them to capture accurate location data. We foresee that visual positioning systems are very likely going to supersede high precision GPS in a lot of scenarios.

This post is based on Peter Batty's (SSP Innovations) talk at the Subsurface Utility Mapping Strategy Forum (SUMSF).

Keeping utility GISs up to date using automated real-time reality capture

Maintaining accurate, up-to-date, comprehensive data about assets, above ground and below ground, has been a challenge for utility and communications firms for decades. Now new technologies including machine learning and computer vision - based feature extraction and text recognition and high accuracy location capture using GPS and augmented reality clouds are being applied to automate the real-time capture of utility network information in the field during construction, when new utilities are installed and when utilities are exposed and relocated.  When fully implemented this will automate real-time maintenance and update of utility network information and form the foundation for utility digital twins.

Background

The decades-old as-builting process remains marred by manual paper-based processes that is characterized by inefficiency that causes critical processes such as outage restoration to require more time than they should and low data quality which leads to incomplete, out of date and inaccurate underground utility records that are responsible for frequent construction delays, injuries and fatalities for workers and the public, and a drag of billions of dollars on national economies.

One of the persistent themes from the very beginning of the Between The Poles was the challenge of integrating CAD and GIS data in a common workflow (some examples; 2006, 2007, 2008 ).  Critical metrics for utility readiness for the smart grid is the as-built backlog, field update backlog and the location accuracy of the assets in utility GISs.  As-built backlogs typically stretch into months, updates from the field take months to be entered into the records database (if they are entered at all), and the recorded location of assets in the GIS can be meters and even tens of meters from their actual location.   The challenge is to replace conventional paper-based workflows with real-time digital work flows that can provide the basis for utility digital twins.

Real-time reality capture using machine learning and augmented reality clouds

New technologies that have been developed for consumer-based applications include machine learning for image and text recognition and more recently augmented reality clouds for recording accurate relative locations and distance measurements are now being applied for capturing utility network information.  This week at the Mirrored World: Role of Digital Twins in the Utilities Industry virtual event, Skye Perry and Peter Batty of SSP Innovations, discussed and demonstrated how these technologies are being applied to the efficient, real-time reality capture of information about utility network facilities in the field during construction. 

Peter Batty demonstrated functionality being developed for a new product SSP Vision that supports the semi-automated capture of information about newly installed electric network facilities.  At an actual construction site Peter demonstrated automated feature recognition of electric and telecom pedestals, automated capture of attribute information from cabinets, scanning bar and QR codes and text for work order number, serial number, manufacturer and other attribute information, and using augmented reality + GPS to record the location of equipment and linear features such as a cable from the pedestal to the transformer.  All of this information was captured and pushed up in real-time and was available in seconds in the utility GIS.

The key features that are being developed for SSP Vision are automatically recognizing equipment such as pedestals and cabinets, automatically detecting and capturing attributes from bar codes, QR codes and text recognition using computer vision and machine learning. The recording of accurate locations is enabled using augmented reality clouds and GPS.  Augmented reality clouds add much greater relative accuracy including distance measurements and work both outdoors and indoors.  At this point properties like feature type can be specified manually if automated feature recognition is unable to detect the type of feature or if a new feature is being entered as part of a new design being carried out using AR in the field. AR also enables capturing linear features with support for snapping to features such as pedestals and cabinets.

Foundation for keeping a utility digital twin up to date

This is truly ground-breaking and shows how these technologies can be applied now in a semi-automated fashion to dramatically improve the efficiency of capturing location and attribute information about utility facilities during construction. The longer term vision is that the capture of this information could be fully automated by simply mounting a camera on a construction crew member's hard hat and feature recognition and attribute identification and capture fully automated requiring no manual input from the crew. This will automate the real-time maintenance and update of utility network information and form the foundation for utility digital twins.

Identifying areas of high risk vegetation encroachment on power lines from satellite imagery

Vegetation is a significant source of outages for many utilities. In some regions from one quarter to one half of all outages can be ascribed to vegetation. Vegetation encroachment initiated the massive North American Northeastern outage of 2003. With the growing incidence of extreme events managing vegetation near power lines is becoming increasingly crucial in tackling potential fire hazards, securing infrastructure increasing resilience. The traditional approach to identifying areas of high-risk vegetation encroachment involves helicopters and ground crews and is time consuming and costly. Drones have been used to reduce the cost of data collection, but it is still a costly process. Now an innovative startup is using satellite imagery together with deep learning to identify high risk areas of vegetation encroachment. By applying machine learning algorithms to high resolution satellite imagery, it is possible to distinguish vegetation from other objects, differentiate trees from grass and bushes and identify trees at risk of encroachment. Comparing a drone-based approach with the satellite imagery methodology reveals that the latter approach is highly accurate in identifying high risk areas and significantly less costly than a drone-based process.

Background

Power line inspections for vegetation management and other purposes are essential in ensuring grid reliability and resilience. They are generally performed by manned helicopters or by a ground crew. Data is collected with cameras and analyzed to detect diseased trees that could fall and hit a power line (fall ins) or trees that are encroaching on power lines (grow ins). These inspections are mandated by NERC in North America and are not optional. Beyond visual line of sight (BVLOS) drones have reduced the cost of collecting line inspection data. Now a new company is reducing the cost of identifying trees at risk by analyzing high resolution satellite imagery to identify high-risk areas, thus reducing the total length of power line that requires more detailed inspection with drones, helicopters, or ground crews.

Evolving machine learning models at Spacept

A Swedish company Spacept has developed a vegetation management solution that applies machine learning models to multi-band satellite imagery to automatically identify areas of high risk for vegetation encroachment on above-ground transmission and distribution lines anywhere on the planet. Spacept develops its own in-house training data for machine learning models which enables follow-up ground inspections of areas of high risk to develop pruning and other response programs. The training data is constantly enhanced with feedback from new projects for continuous improvement of the models.

Satellite-based vegetation encroachment risk assessment

The key business benefit of Spacept’s satellite imagery-based solution is automating the process for identifying the high-risk areas that require ground-based inspection, whether helicopter, drone-based, or ground crew, thus significantly lowering the cost of vegetation encroachment inspections. Spacept’s satellite-based vegetation encroachment technology makes it possible for power grid operators to monitor vegetation encroachment over their entire infrastructure in a cost-effective manner. Spacept uses the most recent, highest resolution (0.50 – 0.30 cm) satellite imagery commercially available which satisfies the requirements for vegetation encroachment detection. Spacept's approach makes it possible to monitor hundreds of thousands of kilometers of power lines every year anywhere in the world.

Workflow

The workflow of identifying and downloading satellite imagery, applying machine learning for feature identification, and the assignment of risk categories to individual trees is currently a semi-automated process. The longer-term objective is to reduce manual intervention and achieve an end-to-end automated workflow. The current workflow proceeds from a geographical layout of the power line to be inspected in the form of a standard GIS file. The most recent available high resolution satellite images satisfying certain criteria are identified and downloaded from data providers. The criteria typically include panchromatic and multi-spectral bands, resolution of at least 50 cm, vegetation capture with leaf on, and minimal cloud coverage. The downloaded images are fed to the current vegetation model which automatically extracts features and distinguishes trees from other man-made and natural objects. One of the important capabilities of the model is that it is very efficient in differentiating trees from bushes. After the automated processing, the imagery and labelling are reviewed by experienced assessors who review each image and make corrections to the labelling for problem areas, for example, for cases of visually distinguishable bushes or edges of trees. After completion of the labelling step, the distance between each tree and the power line is calculated and a risk category is assigned using risk categories provided by the client. The final deliverable is a GIS file containing the location and the corresponding risk categories assigned to the trees.

In-field case study

To assess the potential business benefits of satellite-based analysis Spacept partnered with infrastructure health management platform Sterblue to apply their satellite-based power line vegetation management solution for thousands of kilometres of a French power grid operator grid. Spacept applied their algorithms to selected satellite imagery to generate a risk file for the targeted segment of the grid. Sterblue assigned emergency (risk) levels based on the distance from the power line using a classification of emergency levels provided by the grid operator. Sterblue then carried out in detail inspections of at-risk areas with its drone inspection fleet. A ground inspection team was sent to verify the locations of the trees physically and collect ground truth. A total of 283 locations were identified. The vegetation at each location was manually validated and was compared with the satellite predictions.

The most important conclusion on the business benefit of the satellite imagery-based analysis is that it was able to identify all the trees that the subsequent ground inspection identified as at risk. In a real-world scenario, applying the satellite imagery analysis as a first step in a power line inspection would dramatically reduce the area and trees that would require ground-based inspection using helicopter or drone-based inspection. As a result of this process, Sterblue reduced its operating costs by 80%.

A more detailed analysis of the satellite data attempted to classify trees into detailed emergency levels U0 to U3. Comparing this analysis with Sterblue's classification was used to create a "confusion matrix" showing instances of agreement and disagreement between the two analyses. It was found that there was good agreement between the satellite and ground-based inspections and analyses.

Conclusion

Several companies have recognized that earth observation imagery is of limited value without the application of analytics to interpret the data. For example, Spacept is applying analytics to the huge amount of earth observation data to provide a valuable high risk vegetation encroachment detection service that is safer, faster, and less costly than ground-based approaches to transmission and distribution line vegetation monitoring.

Canadian Underground Forum begins tomorrow

You can see the program for April 29 and 30 and register here.

Speaker lineup

One of the early innovators in applying LiDAR to mapping underground pipelines, Joseph Hlady, President at Lux Modus in Calgary, Alberta will explain n his keynote the benefits that his technology has brought to the oil and gas pipeline industry. Another early innovator is Otto Ballintijn, CEO of Reduct who has developed inertial or gyro technology for mapping underground pipe networks. In his keynote he will explain the evolution of the technology including the concept, operational method, and data output of gyro mapping technology. A number of jurisdictions around the world are developing digital twins at the urban, regional and even national level. Alex Shalash of UEVO will discuss creating a city subsurface infrastructure map which unifies fragmented information and produces an intelligent digital twin. Danny Petrecca of Locusview, an early leader in capturing and managing underground data in the gas industry, will review why as-builting still remains paper-based and discuss a digital construction solution that stitches together existing technologies to create the reliable digital twin that utilities need in the age of the smart grid.

Our chief guest speaker is Jim Anspach, Chair of the ASCE 38 standards group, who will bring us up to date on the imminent 38-21 utility as-built standard. Carsten Roensdorf and Neil Brammall will introduce the Open Geospatial Consortium's work in progress, the MUDDI standard for exchanging location data about underground infrastructure. Lawrence Arcand, who has led Subsurface Utility Engineering (SUE) investigations on a national level since 2003, will provide an overview of the Canadian standard CSA S250:20 Mapping of underground utility infrastructure.

Rob Martindale, Program Manager - Utilities/Railroads at the Colorado Department of Transportation, will present an overview on how a mix of legislation and regulation has enabled Colorado to revolutionize the capture and sharing of accurate underground utility location data. Page Tucker, President & CEO at ProStar which implemented the mobile and cloud solution adopted by the Colorado DoT will speak about how the partnership between Colorado DoT and Prostar made this successful. One of the remarkable new technologies applied to mapping underground infrastructure was developed by 4M-Analytics. Harel Dan, Head of GIS and Remote Sensing, and Ophir Wainer, Director of North America Business Development, will explain how 4M-Analytics uses satellite and aerial imagery to map underground infrastructure. Leica Geosystems' IDS Georadar was one of the early innovators in ground penetrating radar. Simon Pedley, Detection Ambassador at Leica Geosystems will talk about some of the innovations in hardware and software that Leica Geosystems has developed in the last couple of years.

Government has a crucial role to play in locating and mapping underground infrastructure. Gord Reynolds and Alon Gat will discuss how Infrastructure Ontario partnered with Metrolinx and the Ministry of Transportation to establish the Utility Coordination Program (UCP), a new initiative which oversees the development of utility relocation standards and the supporting process to improve the location, treatment and relocation of utility infrastructure. Gordon McElravy, of buildingSMART Canada, will discuss buildingSMART's efforts in implementing the Municipal Infrastructure Council Program to encourage building information model (BIM) initiatives at the municipal level. Capping the first day of CUF Prashant Shukle, who was responsible for mapping and earth observation in the Canadian government for many years, will moderate a panel discussion on the role of government in underground infrastructure with panelists from the State of Colorado, the U.K. Government's Geospatial Commission, Public Services and Procurement Canada, and Infrastructure Ontario.

Using satellite imagery to prioritize vegetation management for utilities

A recent startup LiveEO is focusing on the digitalization of energy and transportation network assets using satellite imagery. LiveEO uses current multi-band imagery including radar and hyperspectral data at various resolutions from earth observation satellites to provide services to railway, electric power and pipeline operators.  The services include vegetation, third party encroachment, and ground distortion monitoring. I recently had the opportunity to chat with Sven Przywarra, co-founder of LiveEO.

Transmission line inspections are essential in ensuring grid reliability and resilience.  They are generally performed by manned helicopters often together with a ground crew.  There are serious safety issues when inspections are conducted by helicopter. Data may be collected with cameras and analyzed to detect a variety of conditions including corrosion, evidence of flash over, cracks in cross arms, and right-of-way issues such as vegetation encroachment. in North America annual inspections are mandated by NERC  and are not optional. With over 200,000 miles of high-voltage transmission lines and 5.5 million miles of distribution lines in the United States, improving the efficiency and reducing the risk of inspections would have a major impact on the reliability of the power grid. 

Vegetation is a significant source of outages for many utilities. In some regions from one quarter to one half of all outages can be ascribed to vegetation. Vegetation encroachment initiated the massive North American northeastern outage of 2003. It is so critical for utility operations that some utilities have a vice president responsible for vegetation management. It can be costly for utilities to identify areas of high risk vegetation encroachment. Previously I have blogged about an electric power utility in Brazil that used Airbus Pléiades satellite imagery combined with GIS data to identify high risk areas for vegetation encroachment in a power grid.

A recent startup LiveEO is focusing on the digitalization of energy and transportation network assets using satellite imagery. LiveEO analyzes current multi-band imagery including radar and hyperspectral data for vegetation, third party encroachment, and ground distortion monitoring. LiveEO uses multi-band imagery at various resolutions to distinguish vegetation and network assets, stereo imagery to estimate tree heights, difference detection to identify changes, and interferometry to detect subsidence and other ground distortions at the cm level.  LiveEO is developing its own inhouse training data for machine learning algorithms. 

For electric power utilities LiveEO applies machine learning multi-band satellite imagery from multiple sources to automatically identify areas of high risk for vegetation encroachment and fallover anywhere on the planet. The results of these analyses are available to clients over the web on handheld and other devices.  This enables follow-up ground inspections of areas of high criticality to develop pruning and other response programs. LiveEO has applied this approach to transmission as well as distribution circuits.  To demonstrate the wide ranging capability of their approach LiveEO is offering a demonstration of the analysis of 50 km of line anywhere in the world.

A number of companies, realizing that earth observation imagery in itself is of limited value, are not in the business of selling imagery, but the results of analytics applied to the imagery.  I have blogged about a new company GHGSat who use their own nanosatellite to estimate emissions from industrial plants, leaks from flow and gathering infrastructure in natural gas fields, and even livestock.  LiveEO is another example of a company who apply analytics to the huge amount of earth observation data to provide a valuable service that is safer and less expensive than the traditional approach to transmission and distribution line vegetation monitoring.

Automated drone systems improve power line inspections and reduce costs

Transmission line inspections are mandatory in North America. Traditionally helicopters are used for these inspections, but with helicopters the risk of accidents is significant and helicopters are costly to maintain and operate.  Drone inspections dramatically reduce the risk of accidents.  Aeriosense Technologies has developed an automated drone inspection system that can reduce the cost of inspections by up to 55%. 

Background

Transmission line inspections are essential in ensuring grid reliability and resilience.  They are generally performed by manned helicopters or by a ground crew.  Data may be collected with cameras and analyzed to detect a variety of conditions including corrosion, evidence of flash over, cracks in cross arms, and right-of-way issues such as vegetation encroachment. Inspections every year or 18 months are mandated by NERC in North America and are not optional. 

With over 200,000 miles of high-voltage transmission lines and 5.5 million miles of distribution lines in the United States, improving the efficiency and reducing the risk of inspections would have a major impact on the reliability of the power grid.  Drone technology, legislation and regulations enabling beyond visual line of sight (BVLOS) flights, software for planning flights and analytics for feature identification are making it conceivable that the expensive process of inspections could be completely automated.  Even now current technologies make it possible to dramatically reduce the cost of power line inspections. According to Navigant Research, globally electric utilities are expected to spend over $13 billion a year on drones and robotics by 2026, a dramatic increase from about $2 billion currently. 

Automated drone inspections

Flight plan for transmission tower inspection

Using automated unmanned aerial systems (UAS) Aeriosense Technologies can reduce the cost and improve the safety of vertical infrastructure inspection.  Unlike many other companies which offer drone services to many different industries, Aeriosense focuses only on the electric power sectors, currently transmission but in the future distribution.  I had a chance to chat with Alex Babakov, a co-founder of Aeriosense, who has many years of experience in the electric power sector, including working for BC Hydro Research/ Powertech’s High Voltage Laboratory (HVL). 

Alex explained to me that Aeriosense provides a complete visual inspection of pylons and poles, cables, and right of way including vegetation encroachment using a UAS typically with a photo camera.  Examples of the types of problems that can be identified include broken strands in lines crossing canyons and rivers, corrosion on insulators and other equipment, evidence of flash over, cracks in cross arms, and problems in the right of way such as human or vegetation encroachment.  Aeriosense also conducts inspections as part of storm response to identify damage resulting from extreme weather conditions.

Flight plan for transmission line inspection

Automation enables Aeriosense's drone systems to provide higher quality inspections at lower cost. In preparation for an inspection flight, the available GIS and LiDAR data is analyzed with the aid of machine learning algorithms to create an optimal flight path.  With a flight path optimized in this way cost reductions of about 36% can be achieved compared to manually operated drone-based inspections.  Furthermore compared to manual drone inspections automated inspections reduce pilot fatigue and safety risks while conducting inspections.

Aeriosense is also active in distribution inspections. The sheer size of the distribution network in terms of total number of kilometers and number of pieces of equipment makes it clear that an automated approach is required. With current technology and regulations, automated inspections can be applied to some but not all distribution circuits. In the future with new technology developments and regulations automated inspections will be able to be applied to more distribution circuits.

Cost reduction

Based on data of inspection efficiencies from our past projects and the data coming from the field trials done by Electric Power Research Institute (EPRI) the reduction in cost with using an automated UAS compared to a helicopter inspection is between 55% and 28%.  The challenge is doing these cost comparisons is that there are intangibles that are difficult to quantify. Safety is one of these. Helicopter accidents are not infrequent in the electric utility industry and more often than not they lead to fatalities.  BC Hydro, for example, has a requirement for helicopters to be dual engine for redundancy and rates for this equipment are much higher.  Other utilities may not be willing to pay for the extra security whichl reduces the rates of the helicopters.

Improved safety

One of the most important benefits of Aeriosense's automated UAS approach is safety. Because of the relatively high risk of helicopter inspections, utilities have been looking for alternatives. Drone-based inspections are much safer and this provides additional motivation for utilities to move toward adopting drone-based inspections.

Development of training datasets for machine learning

Another benefit of the drone-based inspections are that utilities get a historical, unbiased record with the imagery that is captured by the drones. This enables engineers to go back and review the condition of assets not from a written record but from photos.  This provides a more objective view of how the condition of an asset has changed over time.

Collecting historical data has another important benefit in that it can provide a body of imagery that can be used to train machine learning algorithms. Collecting the data now will allow them to apply the AI tools currently being developed in the future.

Future applications of automated drone systems

The impact of more frequent, less costly inspections is improved reliability. Combining the efficiency of automated UAS inspections with defect detection using machine learning will allow utilities to get a very clear, timely situation assessment of their distribution system.  Aeriosense is currently working with the Centre for Energy Advancement through Technological Innovation on a project to help utilities identify current opportunities of automated drone applications and to develop best practices in implementing automated drone inspection programs.  Aeriosense is currently working in the transmission and distribution sector, but is actively exploring applications in the substations and generation verticals.

Another area that Aeriosense is currently piloting is the use of beyond visual line of sight (BVLOS) UAS flights for transmission inspections. FAA rules permit beyond visual line of sight (BVLOS) UAVs through Part 107 waivers.  I have blogged about Xcel Energy which obtained one of the first three waivers to fly UAVs for beyond visual line of sight operations. Xcel Energy estimates that flying drones beyond line of sight will eventually cost between $200 and $300 per mile, compared to helicopter flights that cost an average of $1,200 to $1,600 per mile.  More than 30 of these waivers have been granted and more than 20 companies are operating beyond BVLOS drones, still with some restrictions.  FAA very recently began allowing BVLOS operation using radar to track the location of the UAV.

Summary

Automated drone technology is already impacting transmission line inspections by improving safety, reducing the cost and time required to complete inspections, and by collecting an objective historical record that can provide training data for future machine learning applications. It not only enables more frequent inspections and improved reliability, but it also enables more rapid inspections, which is especially critical after a storm or other emergency situation. Research is underway at Aeriosense to determine the applicability of automated drone technology in other sectors, such as substations and generation, of the electric power industry.

Automating scan-to-BIM for brownfield substations

With about 55,000 substations in the U.S. and the average age of the transformers in these substations over 40 years old, many substations are going to be refurbished, retrofitted, extended or upgraded. Utilities are opting to create digital twins of their existing substations as part of this process to enable maintenance and upgrading activities to be performed more efficiently. For older substations records may consist in outdated 2D paper drawings or documents may be missing altogether.  New digital scanning technology together with machine learning is making it possible to semi-automate the process of creating intelligent BIM models from point clouds derived from laser scanning or photography.

primtech is a CAD-based design tool for designing high voltage substations.  An important recent enhancement Optical Symbol Recognition (OSR) allows point clouds from laser scanning or photography to be imported and then to apply machine learning to semi-automate the process of creating an intelligent BIM model. 

The software includes an extensive library, which contains more than four thousand intelligent 2D and 3D electrical devices from different manufacturers, steel structures, foundations, and wires and tubes.  The library has been growing for over 15 years and includes new equipment and elements as well as for legacy assets.  Each symbol (device, steel or foundation, etc.) has a specific geometric signature in the database in addition to meta data, file links, connection points and defined object behaviours. Together these comprise smart objects that enables objects in point clouds to be recognized and converted into an intelligent substation information model. The resulting intelligent model provides the basis for the calculations required in the substation engineering process.

The OSR process consists of two phases; first conductors are created and then equipment is added. A wire can be created by simply selecting three points, first anchor point, maximum sag point and end anchor point. For tubes, two or more points can be selected. Wires and tubes are available in the library for different conductor types according to international standards.  After conductors has been created and placed, the OSR tool provides a list of compatible library symbols for each symbol type. It is up to the user to choose which symbol should be selected. The user can select the perfect matching symbol or another one in the list. A similar workflow applies to all the other symbols like support structures and foundation heads. The system then combines the symbols to create assemblies (electrical device, support structure, foundation head) and places them in the model. Once an assembly has been created, the OSR tool will recognize identical assemblies in the point cloud and propose it for those positions in the substation. The wires are automatically attached to the respective electrical connection points on the equipment upon the insertion of the assemblies.  After the OSR tool has examined all objects in the point cloud and they have been mapped to an assembly in the model, a logical and intelligent primtech BIM model has been created. The model can be used as a full-fledged template for extension or retrofit - to create parts lists and plans as well as to carry out calculations including phase checking, clearance calculation, lightning protection calculation, sag and short-circuit effects calculations.

The new OSR technology is not fully automatic but drastically improves efficiency and reliability of converting point clouds into intelligent 3D models.This is an important advance that could dramatically improve the process of scanning brownfield substations to intelligent models by making the process more efficient and less error-prone.

M. Kokorus, F. Pizarro, W. Eyrich and S. Heuser, "From Optical Symbol Recognition (OSR) of Point Clouds to the Substation Information Model," 2018 IEEE/PES Transmission and Distribution Conference and Exposition (T&D), Denver, CO, USA, pp. 1-4., 16-19 April, 2018

Open source geospatial technology enables cost-effective solutions in many domains

At the GITA Pipeline Forum in Houston, an enthralling presentation by Rich Henry of Pivvot, a recent startup that builds solutions for pipeline routing and other applications, highlighted how open source geospatial technology is transforming applications in a variety of sectors. Pivvot builds SaaS solutions using open-source components on Google Maps and Amazon Web Services to provide a web-based cloud computing platform with a curated database from over 900 public and private sources.  The advantages of this approach for siting pipeline routes are dramatic; applying infinite computing resources, completely avoiding the arcane art of managing ESRI ArcGIS licences, and integrating hundreds of data layers making it possible to run alternative pipeline right of way scenarios in hours rather than weeks or months.

Recent technical advances have made it possible to implement cost-effective solutions for utility, telecom and pipeline asset management, integrating BIM and geospatial, and geospatially-enabled machine learning.  Standards are a key underpinning in the development of geospatial applications for a number of vertical industries.  There are now many widely adopted geospatial standards; WMS, WFS, GML, KML, InfraGML, PipelineML, WaterML to name just a few from the Open Geospatial Consortium (OGC) that were not available two decades ago.  Furthermore, open source geospatial developers like open standards.  Many open source libraries and applications implement standards from the Open Geospatial Consortium (OGC), W3C and GeoJSON. Since 1999 there has been rapid development of open source geospatial projects including MapServer, GDAL, PostGIS, GeoServer, OpenLayers, Leaflet, QGIS and others.  The Open Source Geospatial Foundation (OSGEO) was founded in in 2006 and the open source code-base has enabled entrepreneurs to rapidly develop low cost geospatial applications because the cost of maintaining most of the code is shared across a broad developer community.  I'll describe a few of the solutions built on open source components that I have encountered in the last few years.

Records management for utilities

One of the challenges faced by large utilities with multiple GIS, outage management, customer information and other systems, which are often the result of mergers, is how to make spatial and associated data, in different, often incompatible systems, accessible to all users not just GIS professionals.  At Distributech 2018 I discovered how Duke Energy is running an off-the-shelf web-based product based on open source components that is being used by 8000 users, 2000 of them using it 24/7 every day, to access GIS and associated data in several GIS systems, both online and offline, using laptops, tablets and phones. A single, common easy-to-use web tool permits viewing geospatial asset data stored in GE Smallworld, ESRI ArcGIS, Intergraph GTechnology, spatialNET and Google Maps as well as non-GIS data in Maximo, SAP, three different outage management systems, four or five customer information systems, CAD drawings and photos and images.

At Distributech 2019 a new entrant IQGeo emerged into the utility and communications space and announced an alternative to traditional GIS.  myWorld Capture is a mobile solution that prioritizes direct data maintenance from the field.  Building on an open source geospatial platform that supports data versioning and interfaces to legacy utility/communications GISs such as ArcGIS and Smallworld, myWorld provides tools that make it possible to develop and maintain a high quality near real-time GIS with the information that is required for a complete network asset data model including equipment location, descriptions, connectivity, condition, and status.

Open access to municipal geospatial data

At GITA 2018 in Phoenix, Bob Basques, GIS Systems Developer at the City of St Paul, described a system called COMPASS he and his team have developed an easy to use tool that allows city employees and the public access to all of the City's spatial and associated data including, for example, scans of surveyors' notebooks, 2.2 million street level photos, and permitting and licensing information from 200 different applications. Based completely on open source components the system is compact and efficient enough to run on a Raspberry Pi.

Integrating BIM and 3D geospatial

At the GEO|Design+BIM conference I discovered mago3D a 3D web-based geo-platform developed by Gaia3D.  It is open source and integrates many well-known open source geospatial products and APis. It has an engine for handling massive, complex BIM objects that supports LOD (level of detail) control and efficient spatial queries and culling (the removal of objects that are not visible to the observer which is key to efficient visualization or large, complex structures.) It integrates with two of the best known open source 3D virtual globe tools, Cesium and NASA WorldWind, for visualization in a geospatial context.  Viewing BIM structures in mago3D has some similarities with a Google Earth experience. The source code is available under open source licenses.

Machine learning

Deep learning algorithms have been developed by academia and as a result the code for the most part is open source. For example, a deep neural network model developed originally for medical image segmentation called U-Net is open source and has been applied to identifying building footprints. Successful training of deep networks requires thousands of labeled training samples. Labeled data involves people on the ground manually ground-truthing land use types and other features so that the deep learning algorithms can learn what to recognize.  Machine learning is computationally intensive and requires a distributed engine.  Spark is a open source (Apache) project which enables distributed processing for global scale computation. Machine learning has been used successfully to identify building footprints and transportation networks and deforestation in satellite imagery.

RasterFrames is a free and open source toolkit allowing scientists, data scientists, and software developers to process and analyze geospatial-temporal raster data with the same flexibility and ease as any other data type.  Open source training data is beginning to become available, for example, the System for Terrrestrial Ecosystem Parameterization (STEP) has 2000 manually labeled sites covering 17 different land cover types including five forest types scattered across all continents.

Pipeline siting

To identify an optimal route for a pipeline on your own local hardware involves stitching together many DEMs, calculating slopes, finding, downloading and integrating hundreds of data layers, and avoiding certain areas (protected or uncooperative owners).  You have to worry about having enough disk space and the calculations can take a long time with no certainty of completion.  Running these calculations on several machines in parallel requires an ArcGIS licence for each of the machines.  Together this could result in months of data collection and processing.  In contrast even for long pipelines an open source solution from Pivvot running in the cloud means you don't have to worry about file management and disk capacity or getting enough machines and licenses to run it in parallel. Using Pivvot's curated datasets data collection and processing can be done in a few hours and it is scalable to many data layers.

Migrating to an open source platform is easier than you think

Samuel Song is responsible for the National Cemetery GIS implementation within Veterans Affairs and like many in government increased use of GIS means a greater sensitivity to cost.  Samuel outlined his experience in migrating to open source at a GIS in the Rockies conference in Denver.

 He had been using ArcGIS Desktop as his desktop GIS, SQL Server as a database, ArcGIS Enterprise as a web server, and ArcGIS API for Javascript for his web GIS. To look at ways to lower his costs Samuel experimented with open source geospatial software; QGIS for his desktop GIS, PostgreSQL/PostGIS for his database, GeoServer for his web server, and Leaflet or OpenLayers for his web GIS (all part of the OSGEO open source geospatial stack). His most important findings are that QGIS is a very stable, powerful alternative to ArcMap, PostgreSQL/PostGIS is a stable alternative to SQL Server and perhaps most importantly migrating data to an open source spatial database from an ESRI Geodatabase was much easier than he expected. His overall conclusion was that with the currently available open source software stack it is possible to move to a fully open source platform from an existing ArcGIS implementation and that the migration is much simpler than expected.

Records management for utilities

One of the challenges faced by large utilities with multiple GIS, outage management, customer information and other systems, which are often the result of mergers, is how to make spatial and associated data, in different, often incompatible systems, accessible to all users not just GIS professionals.  At Distributech 2018 I discovered how Duke Energy is running an off-the-shelf web-based product based on open source components that is being used by 8000 users, 2000 of them using it 24/7 every day, to access GIS and associated data in several GIS systems, both online and offline, using laptops, tablets and phones. A single, common easy-to-use web tool permits viewing geospatial asset data stored in GE Smallworld, ESRI ArcGIS, Intergraph GTechnology, spatialNET and Google Maps as well as non-GIS data in Maximo, SAP, three different outage management systems, four or five customer information systems, CAD drawings and photos and images.

Samuel Song is responsible for the National Cemetery GIS implementation within Veterans Affairs and like many in government increased use of GIS means a greater sensitivity to cost.  Samuel outlined his experience in migrating to open source at a GIS in the Rockies conference in Denver.

 He had been using ArcGIS Desktop as his desktop GIS, SQL Server as a database, ArcGIS Enterprise as a web server, and ArcGIS API for Javascript for his web GIS. To look at ways to lower his costs Samuel experimented with open source geospatial software; QGIS for his desktop GIS, PostgreSQL/PostGIS for his database, GeoServer for his web server, and Leaflet or OpenLayers for his web GIS (all part of the OSGEO open source geospatial stack). His most important findings are that QGIS is a very stable, powerful alternative to ArcMap, PostgreSQL/PostGIS is a stable alternative to SQL Server and perhaps most importantly migrating data to an open source spatial database from an ESRI Geodatabase was much easier than he expected. His overall conclusion was that with the currently available open source software stack it is possible to move to a fully open source platform from an existing ArcGIS implementation and that the migration is much simpler than expected.

New regulations for pipeline safety creates opportunities for geospatial professionals

New federal regulations in the U.S. will likely more than double the areas where higher levels of pipeline integrity maintenance standards are mandated by federal legislation.  Identifying these areas is more complex creating opportunities for geospatial professionals to apply new technologies including machine learning.

On June 10, 1999, a gasoline pipeline operated by Olympic Pipeline Company exploded in Bellingham, Washington. Three people died in the accident.  On August 19, 2000 a natural gas pipeline owned by the El Paso Corporation exploded near Carlsbad, New Mexico killing 12 people.  After these incidents the government moved quickly to improve the quality of geospatial and other data about underground pipeline assets. The result was within about two years federal regulation from the Pipeline and Hazardous Materials Safety Administration (PHMSA) which required companies operating pipelines carrying hazardous materials to identify high consequence areas (HCAs) where there were 20 or more structures intended for human occupancy within a radius (potential impact radius or PIR) defined by the diameter and pressure of the pipe. Within the HCAs utility companies were required to maintain facilities data at a higher level of integrity. The cost to identify HCAs for a relatively small utility in Pennsylvania that I was familiar with at the time cost $2 million over two years.  This represented a significant opportunity for experienced geospatial professionals.

The Pipeline Safety, Regulatory Certainty, and Job Creation Act of 2011 mandated telephonic or electronic reporting of accidents and incidents including the amount of product released, number of fatalities and injuries, but not an accurate location, within 48 hours of occurrence.  The act also required the Secretary of Transportation to maintain a map of high-consequence areas in which pipelines are required to meet integrity management program regulations.  At this time the locational accuracy requirement for pipelines was  ± 500 feet.

At the GITA Pipeline Forum in Houston, Darrel Donaho of G2 Integrated Solutions presented an overview of the latest regulations from PHMSA.  Referred to as the Gas Mega Rule the new regulation extends the higher standards of pipeline maintenance to moderate consequence areas (MCAs). Pipeline operators are now required to identify areas where 1) the number of occupied structures within the PIR is five or greater and less than 20 or 2) where any portion of the pipeline PIR intersects the paved area of interstate highways, freeways and expressways and other principal four-lane arterial roads.  They are also required to report longitude/latitude to five decimal places for all incidents.

Whereas in the past identifying HCAs involved hand drawing building footprints from imagery, machine learning can now be used to automate the process of identifying building footprints prior to using a buffer analysis to identify areas qualifying as MCAs.  The analysis is aided by the current accuracy requirement for locating pipelines of ± 50 feet.  Identifying MCAs where highways intersect pipeline PIRs is more complex because automating the identification of transportation corridors is more difficult.  There are alternative, but more expensive ways to find this information.

An analysis of a 14.91 mi transmission line suggests that the impact of the new regulation will be significant, more than doubling the areas where higher integrity maintenance applies.  This creates a significant opportunity for geospatial professionals to apply new technologies including machine learning.

HCA	MCA (>=5)	MCA roads	Total MCA
5.65 mi	2.63 mi	3.03 mi	5.66 mi

 

Toward automating scan-to-BIM for brownfield substations

With about 55,000 transmission substations in the U.S. and the average age of the transformers in these substations over 40 years old, many substations are going to be refurbished, retrofitted, extended or upgraded. Utilities are opting to create digital twins of their existing substations as part of this process to enable maintenance and upgrading activities to be performed more efficiently. For older substations records may consist in outdated 2D paper drawings or documents may be missing altogether.  New digital scanning technology together with machine learning is making it possible to semi-automate the process of creating intelligent BIM models from point clouds derived from laser scanning or photography.

Companies are recognizing the benefits of an integrated geospatial and building information modeling (BIM) approach to design and construction. I blogged about the winner of the Year in Infrastructure 2019 award who applied BIM processes and reality modeling on a brownfield substation expansion project. The integrated approach enabled the organization to cut design time, minimize site visits, reduce land requirement and material waste, and, most importantly, eliminate change orders during construction.

Now reality capture and deep learning have been incorporated in a substation design product to semi-automate the scan-to-BIM process for brownfield substations.

primtech is a CAD-based design tool for high voltage substations.  An important recent enhancement Optical Symbol Recognition (OSR) allows point clouds from laser scanning or photography to be imported and then to apply machine learning to semi-automate the process of creating an intelligent BIM model. 

The software includes an extensive library, which contains more than four thousand intelligent 2D and 3D electrical devices from different manufacturers, steel structures, foundations, and wires and tubes.  The library has been growing for over 15 years and includes new equipment and elements as well as for legacy assets.  Each symbol (device, steel or foundation, etc.) has a specific geometric signature in the database in addition to meta data, file links, connection points and defined object behaviors. Together these comprise smart objects that enables objects in point clouds to be recognized and converted into an intelligent substation information model. The resulting intelligent model provides the basis for the calculations required in the substation engineering process.

The OSR process consists of two phases; first conductors are created and then equipment is added. A wire can be created by simply selecting three points, first anchor point, maximum sag point and end anchor point. For tubes, two or more points can be selected. Wires and tubes are available in the library for different conductor types according to international standards.  After conductors has been created and placed, the OSR tool provides a list of compatible library symbols for each symbol type. It is up to the user to choose which symbol should be selected. The user can select the perfect matching symbol or another one in the list. A similar workflow applies to all the other symbols like support structures and foundation heads. The system then combines the symbols to create assemblies (electrical device, support structure, foundation head) and places them in the model. Once an assembly has been created, the OSR tool will recognize identical assemblies in the point cloud and propose it for those positions in the substation. The wires are automatically attached to the respective electrical connection points on the equipment upon the insertion of the assemblies.  After the OSR tool has examined all objects in the point cloud and they have been mapped to an assembly in the model, a logical and intelligent primtech BIM model has been created. The model can be used as a full-fledged template for extension or retrofit - to create parts lists and plans as well as to carry out calculations including phase checking, clearance calculation, lightning protection calculation, sag and short-circuit effects calculations.

The new OSR technology is not fully automatic but drastically improves efficiency and reliability of converting point clouds into intelligent 3D models.This is an important advance that could dramatically improve the process of scanning brownfield substations and generating  intelligent models by making the process more efficient and less error-prone.

M. Kokorus, F. Pizarro, W. Eyrich and S. Heuser, "From Optical Symbol Recognition (OSR) of Point Clouds to the Substation Information Model," 2018 IEEE/PES Transmission and Distribution Conference and Exposition (T&D), Denver, CO, USA, pp. 1-4., 16-19 April, 2018

Read more about substation design