Between the Poles

The future is fewer maps

At the first Longitude meetup in the U.K., organized by Ben Flanagan, Kenneth Field, and Chris Wesson, Ed Parsons presented his perspective on one of the directions to expect from geospatial in the future.  It was a bit counterintuitive, for consumers more geospatial data and technology, but fewer maps.

To start with he reminded us that the public has never had as much access to maps as we do now thanks primarily to Google Maps - either as standalone or embedded in other apps. What he foresees is that in future we will be seeing fewer maps or no maps at all.

The maps we have used up to now are large scale maps presenting a large scale view - probably very useful to Winston Churchill and people planning things on a large scale. But in our day to day life, we don't need maps at this scale. Instead we need to know what we will encounter in the next 15 minutes, whether that be the next exit off the highway or the next grocery store. There will be more interaction between us and the world around us - for example showing which of the nearby bars or coffee shops are very busy or less busy.

Ed gave a practical example that anyone who uses a smart phone to navigate in a place they have not been before can relate to. When you come out of a tube station or any GNSS-denied place you are always faced with a quandary - how to orient yourself to know whether to walk left or right. I mentally flip a coin and walk until my smart phone tells me if I am headed in the direction I want to go. If I am in a part of the city where GNSS is unreliable I walk to the end of the block and the street number or name usually will tell me if I am headed in the right direction or that I have to retrace my steps in the other direction.  In the future your navigation app won't rely solely on GNSS signals but will recognize buildings and give you the right direction. In fact this could all happen through an audible interface so that you needn't look at a map at all, "turn right as you leave exit 3 from the tube station".  Of course this will require a lot more spatial data about the city you are in and will require some serious geoanalytics.

What this suggests is that as consumers we will use more geospatial data and analytics, but it will become part of the technology stack and most of us won't realize how much geospatial technology we are using.

Geospatial + BIM interoperability essential for the development of smart cities

Integrated geospatial and BIM is increasingly being recognized as essential for full lifecycle construction and for enabling urban digital twins.  At this year's Geo Business in London the importance of integrating these technologies was asserted by several speakers.  From the perspective of developing smart cities, Hong Kong has made integrating these technologies a policy priority for its goal of becoming a smart city.

In an award winning paper at a 2011 conference organized by Britain’s Association for Geographic Information (AGI), Ann Kemp, then head of GIS at Atkins Global, the design and engineering firm, asked the question ‘BIM isn’t geospatial — or is it?’ and then argued that integration of geospatial and BIM was essential to address the challenges of the 21st century.  At that time within the UK construction company this was a radical suggestion, but since then the need to integrate geospatial and BIM has been gaining traction in the construction industry.  Among the construction vendor community it was adopted by Bentley, Hexagon, and Trimble a number of years ago and in 2017 Autodesk also jumped on board, forming an alliance with ESRI.  At this year's Geo Business event in London the importance of an integrated geospatial and BIM approach in construction was recognized by several speakers.  For example, Marek Suchocki, Autodesk argued that GIS and BIM are not separate, quite a change for Autodesk compared to a couple of years ago.

Hong Kong has the highest population density (26,100 per km²) in the world.  In Hong Kong 50% of the construction industry is government providing government with a key role in moving the entire construction industry, both public and private, forward. The city is quite advanced in adopting BIM.  BIM is mandatory for projects over HK$30 million. The city has developed a BIM Data Repository prototype to allow BIM models of public structures to be shared among government departments and with the public.  BIM has been accepted for General Building Plan submission (permitting) and all of this has been done in collaboration with the Construction Industry Council (CIC).  Finally a study of BIM/GIS integration has been completed.

In his inspiring talk at Geo Business Thomas Chan, Director of Lands, Hong Kong SAR Government, asserted geospatial has to go hand in hand with BIM in creating the common spatial data infrastructure (CSDI) that will underpin Hong Kong as a smart city. As a basis for moving toward a smart city Hong Kong has adopted common spatial data infrastructure,  3D GIS city model, and open data as public policies.  The city has already initiated a pilot open CSDI data portal with over 70 data types available.   But the BIM-GIS integration study found that it is currently difficult to bring BIM models into GIS.  Thomas Chan mentioned the OGC standards CityGML and IndoorGML as essential smart city standards.  Research has shown these standards have some incompatibilities with buildingSmart BIM standards. He is strongly encouraging the geospatial and BIM communities to collaborate to ensure that BIM models submitted to the city can be easily brought into the city's CSDI.

I would suggest that the most important takeaway from this is that in addition to the now widely recognized benefits of geospatial+BIM integration for the construction industry, it is also essential for creating a common spatial data basis for smart cities.  If we cannot resolve AEC+geospatial interoperability now through collaborative work between the two communities, we will continue to be plagued by this issue for decades to come much as we have been by CAD/GIS incompatibility for the past 25 years.

Improving and sharing geospatial information about the subsurface in England

NEDO estimates that about 50% of time over-runs on commercial building projects are caused by unforeseen ground conditions.* There are two aspects to the problems associated with the subsurface.  One is geotechnics, soil, ground water, and geology conditions.  The other is the location of underground infrastructure, utilities, telecommunications and other cables and pipes. There are growing government, private sector and standards initiatives in both of these areas in the U.K. to improve the information and sharing of information about the subsurface.

Social driver: Public safety

Public safety is an important social driver for improving knowledge about the underground.  I remember attending a Pipeline Industries Guild (PIG) conference in Birmingham, U.K. in 2005 where the focus of the entire event was to motivate the pipeline operators industry in England to voluntarily participate in a call-before-you-dig centre that would reduce the risk of damage to underground infrastructure during excavation.  There had been a near major disaster in Birmingham (Hunton Hill) when a gas company crew had attempted what was supposed to be a routine tap into a gas main to connect a new customer.  Three drill bits later, the crew knocked off.  Fortunately an ESSO contractor just happened to pass by and recognized that the gas crew had been attempting to tap into a high pressure fuel line.  At the conference the section of pipe with dimple caused by the gas crew was shown.  A number of experts described the the extent of the disaster that would have followed if the crew had tapped into the line.  The 2003 gas explosion in Belgium would have paled by comparison.  In just the last month in the Americas, there has been an explosion in downtown San Francisco, another one in Durham, North Carolina, one in downtown Sao Paulo, and an evacuation of a neighbourhood in Calgary, Canada when a 6" gas main was ruptured during excavation.

Economic driver: construction productivity

UK construction productivity compared to other sectors

The major problem facing the construction industry in advanced economies is productivity which has plateaued or even declined over the past 40 years. With the growth of private investment in infrastructure, improved productivity has become urgent. McKinsey has made the case that BIM and geospatial are key technologies in transforming construction.  The U.K. has been a leader in the digitalization of the construction industry, mandating that from 2016 all public construction in the U.K. must use a BIM Level 2 process.  The subsurface has been for the most part ignored in the application of digital technology to construction, but there are signs now that this is changing in the U.K., as people recognize the drag on construction projects and the broader national economy that this represents.  Improving the reliability of and sharing location information about the underground can be seen as 'low hanging fruit' in the quest to improve construction productivity. 

Sharing subsurface geotechnical information

One of the problems is that every construction project involves considerable effort to discover subsurface geotechnical conditions and the location of underground infrastructure in the area to be excavated, but a large proportion of the data generated by the UK ground engineering industry is used for one project and not shared.  In the Netherlands the problem is being addressed by the Key Registry for the Subsurface (BRO) which came into force in January, 2018. The BRO registry consists of 26 data types, which will become mandatory in installments over five years and all of which include location. On 1 January 2018,it become mandatory to report the first three data types, geotechnical surveys (CPT), groundwater monitoring wells and soil drilling sample profiles. On 26 June 2018, this data became publicly available via the Dutch open data portal PDOK.

In the UK the Dig to Share project, supported by Atkins, British Geological Survey (BGS) and Morgan Sindall, is addressing this problem. Its aim is to develop a fully digital workflow, which is accessible to the whole industry, to upload and access data from the BGS web-based system. This will be developed on top of the existing system hosted and maintained by BGS. 

I should add that in the domain of geotechnics Keynetix is a UK-based provider of cloud-based software for capturing, visualizing, modeling, and sharing of geotechnical data. Bentley Systems has just announced the acquisition of Keynetix which adds to Bentley's products for the subsurface including borehole report management (gINT) and geotechnical analysis applications (PLAXIS and SoilVision) with which Bentley is targeting enabling a digital twin of the underground. 

Sharing information about the location of underground infrastructure

The Geospatial Commission has just announced its intention to create an Underground Assets Register.  The intention is to show where electricity and telecom cables, and gas and water pipes are buried to help prevent both accidents and disruption to the economy.  The project will start with £3.9 million pilot projects split between London and the North East.

Pipes and cables beneath a road crossing in North East England - Thanks to Holger Kessler for sharing.

Related to this is the North East Underground Infrastructure Hub (NEUIH) , the Ordnance Survey's and Northumbrian Water's project to create a ‘Common Infrastructure Map’ in Newcastle.  The current phase of the initiative is called ‘Sunderland sand box’ and implements a dataset that contains almost full water, gas, electricity and a great deal of telco data for the city of Sunderland.

Collecting underground utility damage statistics

The U.K. has not had access to good statistics on underground utility strikes similar to the Common Ground Alliance in North America.  Since 2014 the Utility Strike Avoidance Group (USAG) has begun collecting statistics on underground utility damage. USAG currently operates on a voluntary basis with no direct funding other than the support offered by member organizations.  Its 2015/2016 report includes over 2700 utility strikes across the UK during 2015 & 16 from 32 participants.  For comparison in the U.S. there were 316,422 incidents of damage to underground infrastructure reported to the Common Ground Alliance in 2017.  According to the Japan Construction Industry Association (Edson Enohi personal communication), in all of Japan in 2016 there were 134 incidents.

One-call centre for England

England does not have legislation mandating a one-call centre as just about every state and province in North America does.  This puts the onus on excavators to track down and contact utilities and others who may have infrastructure in the area of the planned excavation.  To fill the gap a private company LinesearchbeforeUdig (LSBUD) offers something similar to a North American one-call centre. I had the opportunity to chat with Richard Broome, Managing Director of LSBUD.  LSBUD offers a voluntary free to use service which anyone planning excavation can use to check for underground utility assets that LSBUD's 75 members (network operators) have in the area of the planned excavation.

The way this works has some similarities with a North American one-call centre.  The excavator provides details of the planned excavation to an online system.  The system immediately responds with a list of LSBUD members that have assets near where the excavation is planned.  Subsequently each member sends plans of their infrastructure assets in the area of the planned excavation.

LSBUD is owned by Fisher German LLP and PelicanCorp, an Australian company that runs one-call centres in Indiana, Kentucky, Alberta, British Columbia, New Zealand, Australia, Ireland, Singapore in addition to the UK. The assets that LSBUD's member manage include hundreds of thousands of kilometers of underground and overhead electricity, gas, high pressure fuel, water and fibre optic pipes and cables.  LSBUD is supported by its members which include all fuel transmission companies, 60-70% of electric power network operators, and some water and telecom operators in England. The benefits that the members derive from LSBUD are improved safety and reduced utility damage costs. The public also benefits from fewer outages.  The LSBUD service processes over 2.5 million inquiries per annum.

There are risks of sharing information for both commercial (competitive) or security purposes. To manage this balance, the Member always controls who accesses their data. Members always controls who accesses their data. Users can get very high level information about a Member’s network through the LSBUD oce call process, but members are in full control of the access rights to their detailed information. They also have a live view of those requesting and receiving their information together with a detailed audit trail.

Much of the liability is on the contractor to locate the assets on site. In terms of the accuracy/ validity of the data, the asset owner carries that themselves with relevant disclaimers. The asset owner determines both the Area of Interest (AOI) on the service, as well as any guidance/literature/disclaimers that are sent with their responses.

Coordinating street works

One area that would directly benefit from shared location information about the subsurface is street works. In 2004 the UK Parliament passed the Traffic Management Act, which gave highway authorities greater power to manage traffic disruption caused by 2.5 million roadworks that take place in England each year.  In particular it requires excavators to apply for a permit (as Bern has done) for road work providing information about the location and timing of the proposed road work.  Late last year the Department for Transport announced an investment of £10 million in Street Manager, a digital planning service that will generate real time data about street construction.  It will be free for technology companies and app developers to use. This will allow services like Waze (whose vision is to eliminate traffic congestion) and startups to build app that help motorists avoid congestion caused by road construction.

The Traffic Management Act also enables local councils to charge a fee for permitting road work.  One way of using this power is referred to as lane rental.  Pilots have been carried out in several location in England and have been found to reduce congestion by half on busy roads.  For this to be used effectively requires accurate locate information about underground utilities and communications infrastructure.

Standard relating to the underground

A process to update the 2014 Publicly Available Specification (PAS) 128, has just been initiated.  PAS 128, developed under the auspices of the British Standards Institution (BSI) and sponsored by the Institution of Civil Engineers (ICE) and others, is one of the few that reflects the advances in underground detection technology - hardware and software - that have occurred in the last few years.  That it is now being updated suggests that PAS128 will continue to stay current with the underground detection technology curve.

The Open Geospatial Consortium MUDDI initiative to develop a standard for sharing information about the subsurface, including both geotechnics and underground infrastructure, is supported by the Ordnance Survey and includes several U.K. and INSPIRE standards.

Together these government, private sector and standards initiatives reflect the accelerating recognition in the UK of the importance of sharing information about subsurface geotechnics and infrastructure for public safety and construction productivity.

Reference

* N.M.H. Alhalaby and I.L.Whyte, The impact of ground risks in construction on project finance in Risk and reliability in ground engineering, Edited by B.O.Skipp, Proceedings of the conference organised by the Institution of Civil Engineers, London, 1993.

"A statistical review of projects revealed that 50% of commercial buildings and 37% of industrial buildings experienced delay due to unforeseen ground conditions. Reports from NEDO state that the most frequent origin of overruns and long delays (greater than 10 weeks) are unforeseen obstacles in the ground."

NEDO Faster building for industry. National Economic Development Office London 1983

NEDO Faster building for commerce. National Economic Development Office London 1988

British standard PAS128 for underground utility location to be updated

Belgium underground gas line explosion during excavation 2003

As we are reminded by the recent underground utility hits in San Francisco, CA, Durham, NC, and Calgary, Alberta, not knowing the location underground utilities represents a serious risk for the public. There are 400,000 hits per year in the U.S. alone.  Standards for reporting the reliability of the locational information about underground utilities have been in place for decades but these standards do not reflect recent advances in underground utility detection technology. In the U.S. the 2003 ASCE 38-02  ,which is been used for classifying the quality of location information about underground infrastructure, is widely seen as being out of date. In France the 2012 presidential decree defines three explicit levels of cartographic accuracy for underground structures: A - less than 40 centimeters, B - 40 centimeters to 1.5 meters, and C - greater than 1.5 meters. In Canada the CSA S250 (2011) standard also specifies absolute precision with three different quality levels for exposed utilities and one for remote sensed detection. In the UK the 2014 Publicly Available Specification (PAS) 128, developed under the auspices of the British Standards Institution (BSI) and sponsored by the Institution of Civil Engineers (ICE) and others, not only includes the A,B, C, D quality levels of the U.S. standard, but extends it with explicit precision levels B1 to B4 similar in this respect to the French decree and the Canadian S250 standard.  I have just had a chance to chat with Andy Rhoades who was responsible for locating and mapping underground infrastructure at Heathrow airport for many years and is now leading the process to update PAS128.  Andy told me that this process will kickoff in a couple of weeks with a target of completion roughly six weeks after that.

Just as a reminder PAS128 defines quality levels A through D based on the type of survey used to locate underground utilities.

QLD - Desktop search techniques were used to identify existing utility data by identifying known utility owners within the survey area, requesting asset information from each of them, typically as-builts, and compiling the collected information.  Locational accuracy generally poorly known.

QLC - Requires site reconnaissance to identify above-ground visible physical features that are indicative of and help locate utilities within the survey area. Locational accuracy undefined.

QLB - For this quality level PAS requires that at least two remote sensing techniques must be used in detecting utilities.  Electro-magnetic locating (EML) and ground penetrating radar (GPR) are required, but others may be used in addition. Most EML systems only detect metal, do not have the capability of digitally recording what was detected, and can only provide rough estimates of depth.  GPR is able to provide 3D location, is not able by itself to identify the type of utility, and does not work well in certain types of soils such as clay.  Post-processing is often used with GPR to improve the interpretation of GPR data by resolving weak and intermittent signals or distinguishing multiple targets.

In addition PAS128 breaks the QLB level into several sublevels with specified absolute precision depending on the results of remote sensing.

QLB4 -  A utility cable or pipe is believed to exist but has not been detected. Locational accuracy undefined.
QLB3 - Only the horizontal location of the utility has been detected by one of the geophysical techniques used. Locational accuracy ±500 mm (XY),  Z locational accuracy undefined.
QLB2 - Horizontal and vertical location of the utility was detected by one of the geophysical techniques used. Locational accuracy for X,Y ±250 mm or ±40% of detected depth whichever is greater, for Z ±40% of detected depth.
QLB1 - Horizontal and vertical location of the utility was detected by multiple geophysical techniques. Locational accuracy for X,Y ±150 mm or ±15% of detected depth whichever is greater, for Z ±15% of detected depth.

QLA - Involves exposing the target utilities within the survey area by excavation to confirm and record the location and other attribute data.  Locational accuracy for X,Y ±50 mm, for Z ±25 mm.

In contrast with the standards in other countries PAS128 reflects the advances in underground detection technology - hardware and software - that have occurred in the last few years.  That it is now being updated suggests that PAS128 will continue to stay with the underground detection technology curve.

Urgent: Standards-based geospatial + BIM interoperability requires broad vendor participation

At the International Standards Summit 2019 in Düsseldorf a group of people representing owners, engineering and construction firms, vendors and services providers in the AEC space, and standards organization Open Geospatial Consortium (OGC) and buildingSMART International (bSI) were invited to meet to discuss ways to move forward in promoting a standards-based approach to geospatial + BIM interoperability.

The BIM and geospatial interoperability challenge is the latest symptom of the broader problem of integrating AEC and geospatial workflows, that has contributed to low productivity in the construction sector.  One of the areas of broad agreement at the meetup was that developing a standards-based approach to the BIM+geospatial interoperability problem is urgent.  There are two reasons for this.

The first is that there is growing evidence of the significant business benefits of an integrated BIM+geospatial full lifecycle approach to construction. Design-build-finance-maintain (DBFM), build-operate-transfer (BOT), and public-private-partnerships (PPP) are among the projects that have reported positive benefits of an integrated approach.

The second reason for urgency is that without standards proprietary interoperability will become the de facto standard. In November 2017 the BIM and geospatial market-leaders Autodesk and ESRI announced an agreement to address the interoperability problem. Since then both ESRI and Auto desk have been busy implementing proprietary ways to bring each other's data into their software environment.

ArcGIS is ESRI's market-leading GIS product.  Revit is Autodesk's market-leading BIM application that is utilized by designers for vertical BIM (buildings), although there are landscape architects that have managed to use it as well. Revit models include spatial data and related non-spatial information.  With the release of ArcGIS Pro 2.2 in June 2018 direct reading of Revit BIM (.RVT) files is now supported. In the past Revit models had to be imported through conversion to IFC (Industry Foundation Classes) files and then go through a second conversion to an ESRI geodatabase.  ArcGIS Pro 2.2 makes it possible to open a Revit BIM model directly. Tools are provided for correctly georeferencing BIM models in ArcGIS. Revit data added to ArcGIS behaves like any other 3D GIS data and can be queried, measured and analyzed.

InfraWorks is Autodesk's application that allows civil engineers to design roads and highways in context. InfraWorks 19.2, released in September 2018, now supports the Autodesk Connector for ArcGIS. Designers and engineers can bring GIS content from ArcGIS Online into InfraWorks allowing content such as roads, pipelines, and electrical transformers to be directly imported. The link is dynamic making it possible to build an InfraWorks model directly from GIS data published into ArcGIS Online. When the GIS data is updated, the InfraWorks model can be updated and refreshed. With the release of Autodesk Civil 3D 2019 InfraWorks models can be brought into Civil 3D, Autodesk's civil design product. Together these enhancements enable workflows from ArcGIS through InfraWorks to Civil 3D.

Together these implementations represent a proprietary (but incomplete) solution to the BIM+geospatial interoperability challenge.  If you happen to use Tekla, Bentley BIM, Vectorworks or Graphisoft as your BIM software or QGIS, GeoServer/MapServer/PostGIS, GeoMedia, MapInfo, Manifold, or Smallworld as your GIS software, this does not begin to solve your problem.  The objective of the OGC/buildingSMART partnership is to provide a standards-bases solution to the interoperability problem. The Integrated Digital Built Environment Subcommittee (IDBE SC) was founded to increase interoperability between the geospatial and built environment domains, specifically through coordination of standards development activities of the OGC and buildingSMART International (bSI).

At the meetup in Düsseldorf IDBE was represented by Hans-Christoph Gruler of Leica Geosystems, Leif Granholm of Trimble and Jim Plume of bSI Australasia. Also present were representatives of Bentley Systems, NavVis, Solibri(Nemetschek), Schiphol Airport, Hong Kong Housing Authority, Royal Haikoning, CRB, think project!, and others.  Together these folks represent owners, the construction sector, and vendors. 

A second conclusion of the meetup is that the IDBE SC needs more involvement from all sectors of the construction industry, but especially broader representation from vendors and owners. To date most of the efforts that I am aware of involve attempting to map the OGC standard CityGML to bSI's IFC standard.  It is fair to say that the research to date has shown that this is not possible in general.  Leif Granholm has championed another approach to the problem that moves interoperability up the stack from data models to the software level.  To be successful this requires participation from the major vendors in the geospatial  and BIM spaces. Owners, especially government, can encourage a standards-based as opposed to a proprietary approach to interoperability by mandating or requiring standards in RFIs, RFPs and contracts.

Bringing reliable near real-time GIS within reach of every utility

Workflows for ensuring utility GIS quality are fundamentally broken and have been for decades.  Integrating CAD and GIS into an efficient workflow has been a challenge and it has simply not been a priority in the utility industry and in some parts of the communications sector to ensure real-time, high quality asset GIS data. Recently business drivers for high quality GIS data such as implementing a smart grid, estimating service availability and other applications have made GIS quality a top priority.  And now non-traditional geospatial companies are offering geospatial-based technologies that are revolutionizing the capture and maintenance of GIS information about utility and communications assets that can provide a foundation for the real-time network digital twin that is essential in the era of the smart grid and our increasing dependence on communications and utility networks. 

Broken business processes degrades GIS data quality

Maintaining accurate, up-to-date asset data has been a challenge for utility and communications firms for decades.  Over the years the lack of CAD/GIS interoperability has cost communications and utility companies hundreds of millions of dollars trying to cleanup their GIS data and streamline their CAD+GIS workflows.  One of the persistent themes from the very beginning of the Between The Poles was the challenge of integrating CAD and GIS data in a common workflow (some examples; 2006, 2007, 2008 ).  Critical metrics for utility readiness for the smart grid is the as-built backlog, field update backlog and the location accuracy of the assets in the utility's GIS.  It is not uncommon for as-built backlogs to stretch into months even years, for updates from the field to take months to be entered into the records database if they are entered at all, and for the location of assets as recorded in the GIS to be city blocks from its actual location.   The challenge is to fashion workflows that include both engineering and geospatial data and applications into an efficient data flow from planning through design and construction to operations and maintenance with feedback loops that ensures an accurate, near real-time GIS.  For many utilities and communications firms traditional GIS, which has been the standard platform for utility GIS for the past few decades, has simply not delivered cost-effective, real-time, high quality asset location and associated data.

A key workflow found at all utilities and communications firms is the field maintenance process.  In many utilities this is a broken feedback loop which results in the quality of GIS data degrading over time.  When a field crew performs maintenance on network infrastructure, which includes routine and emergency maintenance, or encounters equipment incorrectly located or described in the GIS, the changes or the corrections to the GIS data from the field need to get back into the GIS quickly.  Too frequently this is still a paper-based process typically involving redlining as-builts, which get to sent to records for updating in the GIS.  The backlog associated with field updates can stretch into months.  The slow turnaround of field updates discourage field staff from reporting corrections and changes further exacerbating the GIS quality problem.  The result of this and other broken workflows such as the as-buillt workflow is that not only is GIS data incomplete, inaccurate and out-of-date, but that its quality degrades with time.

Business drivers for accurate, real-time GIS

For communications companies a major driver for an accurate and up-to-date GIS is providing rapid availability of service estimates to prospective customers.  I remember a large U.S. telecom that because it could not rely on the information in its GIS had to resurvey all the locations of its fiber terminations in order to respond to potential ADSL customers indicating whether they were close enough to a fiber termination for the service.

For electric power utilities a powerful business driver that has emerged in the last few years is the smart grid.  Digitalizing the grid has made the role of geospatial data and technology strategic for utilities. As a Navigant Research report stated the smart grid is all about situation awareness and effective anticipation of and response to events that might disrupt the performance of the power grid. Since spatial data underlies everything an electric utility does, GIS is the only foundational view that can potentially link every operational activity of an electric utility including design and construction, asset management, workforce management, and outage management as well as supervisory control and data acquisition (SCADA), distribution management systems (DMSs), renewables, and strategy planning. As John Simmins of EPRI has reiterated on a number of occasions a prerequisite to implementing a smart grid is cleaning up GIS data and developing business processes (feedback processes) to ensure that it remains accurate and up-to-date.

Technology enablers: Open standards, open source geospatial and handhelds

Recent technical advances have made it possible to implement cost-efficient processes for ensuring that utility asset data including GIS data is accurate and up-to-date.  There are now many widely adopted geospatial standards; Simple Features for SQL, WMS, WFS, GML, KML, and InfraGML to name just a few from the OGC that were not available two decades ago.  Secondly, since the founding of the Open Source Geospatial Foundation (OSGEO) in 2006 there has been rapid development of open source geospatial projects including PostGIS, GeoServer, MapServer, Leaflet and others.  This open source code-base has enabled entrepreneurs to rapidly develop low cost geospatial applications because the cost of maintaining most of the code is shared across a broad developer community.  Thirdly the advent of the handheld touch screen computer and phone provides a low cost universally accessible hardware platform for field staff to access geospatial and other information stored in a utility's databases.  With the appropriate software smart phones and tablets make it possible for field staff to see all of the information about an asset in the field at the touch of a finger.  For example, I blogged about how Duke Energy field staff are using an open source based mobile application which provides a single, common easy-to-use web tool for viewing geospatial asset data stored in GE Smallworld, ESRI ArcGIS, Intergraph GTechnology, spatialNET and Google Maps as well as non-GIS data in Maximo, SAP, three different outage management systems, four or five customer information systems, CAD drawings and photos and images.

Shifting the paradigm: GIS data maintenance from the field

The paradigm shift is empowering field staff to take responsibility for the quality of data in the utility GIS. Since they have direct knowledge about the location and condition of utility infrastructure, the key to revolutionizing data maintenance and enabling near real-time, accurate GIS and other data is "cutting out the middle man" and enabling field staff to maintain the utility GIS and other data in the field as they are working. Enabling field folks to do this without adding significantly to their workload (and over time reducing their workload as the quality of the GIS improves) requires a simple "touch and go" user interface which minimizes typing, responsive applications running on handhelds, and workflows that enable field staff to see their changes with a minimum delay measured in hours or days at the maxium.  Of course, operational data can be protected from problematic updates using data versioning technology (long transactions) which provides a rapid way for records folks to review and identify potentially problematic updates.

At Distributech 2019 a new entrant IQGeo emerged into the utility and communications space and announced an alternative to traditional GIS.  myWorld Capture is a mobile solution that prioritizes direct data maintenance from the field.  Building on an open source geospatial platform that supports data versioning and interfaces to legacy utility/communications GISs such as ArcGIS and Smallworld, myWorld provides tools that make it possible to develop and maintain a high quality near real-time GIS with the information that is required for a complete network asset data model including equipment location, descriptions, connectivity, condition, and status. 

The key to ensuring that the data is high quality and up-to-date is empowering field workers to make updates at the source as the work happens and to automate the data maintenance workflow as much as possible.  This approach can eliminate backlogs in field updates, encourage field staff to take responsibility for ensuring the quality of the utility GIS and enable an effective feedback loop that improves GIS data quality over time.   With the promise of near real-time updates directly from the field this approach provides a key foundation for the digital twin of the grid that integrates sensors reporting in real-time with simulation and analytics that makes it possible to create a accurate real-time situation assessment and to model the electrical state of the grid and possible future states based on reported or forecast outages and planned expansions and other changes to the electric network.

Operationalizing the new paradigm

How does this work in real world practice? I had a chance to chat with Dick Rohm, Director of Engineering at Cable One about how they are progressing toward their goal of no backlogs for field updates or for as-builts.  The business driver for this initiative, which they have been working on for the past several years, is service availability - being able to tell a customer immediately whether a fiber service is available at the customer's location or whether it will require an addition to the network.

Cable One provides commercial fiber services to businesses as well as cable services in 21 states, primarily in small cities and big towns.  Sales success requires being able to respond rapidly to potential customers requests for business fiber service, and say yes we already have the fiber capacity in your neighbourhood or no we have to build out a fiber line to support you.  Improving the quality of the data in their GIS is an important business driver for Cable One.  

By prioritizing as-built and update workflows Cable One has managed to ensure that as-builts and updates are in the GIS quickly, typically within 2-3 business days.  To achieve this, they first changed the as-built process, making the design process paperless about five years ago. Their engineering GIS allows engineers to use AutoCAD to design fiber and RF networks directly into GIS .  For construction, contractors are provided with AutoCAD drawings (DWG) on thumb drives and it is up to them if they wish to print them.  For small jobs, contractors can work completely electronically. For large jobs, contractors still use paper, but they print the DWG files themselves. Contractors must register with Cable One's Project Central which is used to track all projects. When paper as-builts are returned, they are spot-checked, and the GIS is updated based on any redlines. A project will not be closed out (and contractors paid) until redlines are in the GIS.  The GIS is used by Marketing and Sales as an added application to myWorld.

For updates from the field the initial idea was to use a web extension to their GIS, but they found that it was too cumbersome and slow.  It was replaced by myWorld on a handheld working with Google or Bing Maps which provided front end for viewing all engineering geospatial and associated data in the field. The myWorld solution only requires about an hour training time using a video. 

Based on the success of the myWorld implementation, user acceptance testing of myWorld Capture is currently underway. myWorld Capture enables field staff to update GIS and other data as they work.  Ease-of-use and rapid response are its key features. It makes it possible for field validators to do most of their work by tapping and drag and drop with very little typing.  Field engineering staff have to do more typing but even for them it is minimal.

The objective of the myWorld Capture implementation is 24-hour turnaround for updates from the field. It could even be less, but it is currently limited by the process of posting updates to the GIS and to the MyWorld database, which occur every night.  myWorld Capture will roll out to all field staff by July 1 of this year.
Dick Rohm emphasized that the key to making a transformative project, such as this at Cable One, successful is changing thought processes in the field. Data needs to be treated as just as important as physical assets.  When something changes or an error is found, the priority for field staff is to update it.  To encourage field staff to make changes in the GIS, tools need to be provided on handhelds that are fast and easy to use. Update processes must be fast so that their changes can be seen by field staff quickly in the GIS. Over time, field folks will increasingly rely on the GIS because they will see corrections that they make from the field appear in the database within 24 hours or sooner.  As data quality improves as a result of this feedback loop, field staff will find that they can rely on the GIS for locating and accessing up-to-date information about assets.

Conclusion

Prioritizing updates from the field to reduce the field update backlog to 24 hours or better together with an as-built backlog on the order of two to three days is a remarkable achievement.  It shows that with a business perspective that prioritizes a near real-time, reliable digital twin of utility and communications networks and geospatial technology supporting open standards and based on opensource and implemented for handhelds in the field, a cost-effective solution to the decades old GIS data quality challenge is within the grasp of any utility or communications firm.

Automating vegetation management for electric power transmission lines using deep learning with LIDAR

With FAA rules permitting beyond visual line of sight (BVLOS) UAVs available through Part 107 waivers the cost of capturing the data required for vegetation management for power lines, whether transmission or distribution, could drop dramatically.  However, manual post processing of the imagery to identify areas of vegetation encroachment, grow-ins and fall-ins, remains an error prone, tedious and time consuming process.  I have come across several firms (for example EPRI, SiteSee, Trimble RealWorks) over the years that have been working towards automating the process for identifying vegetation encroachments including grow-ins and fall-ins, but I have not encountered a utility that is using this technology operationally.  At Distributech 2019 in New Orleans I had a chance to chat with a startup Enview, based in San Francisco,  who described to me how their software has been implemented  over 7,000 miles of transmission lines last year to automate the identification of vegetation encroachment from LiDAR point clouds.  Furthermore, they expect the total distance covered to double this year.

Vegetation is a significant source of outages for many utilities. In some regions from one quarter to one half of all outages can be ascribed to vegetation.  Vegetation encroachment initiated the massive North American northeastern outage of 2003.  It is so critical for utility operations that utilities even have a vice president responsible for vegetation management.  It can be costly for utilities to identify areas of high risk vegetation encroachment which typically are carried out by manned helicopters.  Scheduling and routing crews to prune the trees that require attention is often sub-optimal adding to the cost of vegetation management.  Optimizing these processes can produce significant benefits in the form of fewer outages and reduced costs. I have blogged about Energisa in Brazil who used satellite stereo imagery to prioritize and optimize tree pruning and ComEd's pilot project in Chicago that uses LiDAR imagery to not only prioritize and optimize but also to change the pricing structure for procuring distribution tree trimming.

Enview applies machine learning, computer vision, and geospatial big data analytics to highly reliable feature extraction from LiDAR point clouds to identify the exact location, height and extent of trees and then to determine proximity to power lines and other utility equipment.  Utility GIS data can be integrated and results visualized using colour-coding to identify the level of risk for each tree.  This enables the utility to prioritize their tree trimming and optimize contractor routing.  In environmentally sensitive areas the visualization can be used to inform discussions with the various stakeholders responsible for different land uses.

At Distributech I had a chance to chat with Krassimir Piperkov, COO at Enview, who described how this technology has been deployed on more than 7,000 miles of infrastructure last year, and is on track to be deployed on over twice that much this year. He gave a very recent example of annual vegetation management analytics performed on 230kV transmission lines in California. These lines run through suburban and environmentally-sensitive, heavily forested areas.

3D visualization of a 230kV span (blue lines) with potential grow-in (yellow) and fall-in (red) vegetation that falls within client-determined clearance boundaries.

LIDAR data was acquired by a third party through a helicopter-mounted Teledyne Optech LiDAR sensor package. Four band imagery was co-collected during the LIDAR survey and registered into an orthomosaic of RGB and NIR bands. The data collection generated almost 2GB per linear mile. The Enview data analytics engine processed this data to (i) first classify the massive amounts of LiDAR data and (ii) then automatically calculate potential grow-in and fall-in vegetation points with high speed and accuracy. Insights and data were delivered to the operator not only via GIS-compatible file format for incorporation into existing workflows but also via an intuitive web portal accessible across silos. As a result of the superior accuracy and speed, the client was able to confidently prioritize vegetation prescription work to the areas that need it most and do so much sooner than otherwise possible with traditional methods.

The results being achieved with Enview's technology suggest that we are on the cusp of the widespread adoption and application of deep learning technology to automate specific tasks. The combination of UAVs for data collection and machine learning for feature extraction dramatically reduces the cost of inspections for utilities. This would permit them to be done more frequently than with manned aircraft and would lead to improvement in the reliability of above ground electric power and communications networks.

Augmented reality aids disaster damage assessment by inexperienced utility workers

One of the challenges of disaster response for utilities is situation assessment which involves sending utility personnel to the areas affected by a storm to assess the state of utility infrastructure. Skilled staff are often in short supply at such times and office and other staff without field experience are often assigned to perform damage assessment. These staff find it difficult to perform a reliable situation assessment because of unfamiliarity with the equipment they are looking for. At Distributech 2019 in New Orleans Rodney James of Duke Energy and John Simmins of EPRI described research they have been doing to use augmented reality (AR) to help with damage assessment by using specially equipped glasses to superimpose a map of the utility's infrastructure on the real world making it easy for an inexperienced staff member to see where equipment has been destroyed or otherwise affected by a storm.

Augmented reality used to require large heavy headsets or helmets but now there are a number of manufacturers of specially equipped light-weight glasses that can be used for AR. Duke Energy uses Realwear HMT-1 glasses that can be attached to the helmet that utility field workers are required to wear and that are safety rated. The glasses are smart running the Android operating system and are able to connect via a wide area network for real-time video and audio making it possible for someone back in the office to view what the staff member in the field is seeing. However, this can be a drain on the battery, so the setup can work with just audio communications as well.

I have blogged previously about Duke Energy's MapView mobile GIS application which enables anyone with a web browser to see a queryable map of all of the utility's infrastructure. A worker equipped with glasses sees MapView superimposed on the real world showing the location of all utility infrastructure with appropriate labelling. This enables an untrained staff member to determine very easily, quickly and reliably which equipment has been carried away by a flood, for example.  Of course this assumes that the utility's GIS data is accurate and uptodate.  I have blogged about this problem many times and about the recent research at EPRI to apply deep learning to improve the quality of utility GIS data.

Rodney James described other AR applications of the technology as well such as supervised routine field maintenance enabling new field workers to be mentored from the office by an experienced engineer.  It appears that we may soon see some practical applications of AR in the utility space deployed in the field.  A question then will be will all field staff including linesmen be equipped with glasses or just some.  In Duke's experience field staff have been enthusiastic about the glasses.

Machine learning cost-efficiently creates high quality utility GIS data for smart grid applications

Many utilities' GIS contains only a small part of what is required for managing a smart grid.  Often even the data in the GIS is too unreliable to use for smart grid applications.  For example, utility GISs may not include which customers are connected to which transformer, or if they do include this information it is incomplete or incorrect.  Many utility GISs cannot identify the phase(s) each customer is using and they may not include network topology - what is connected to what.  Other missing data may includes street lights, distributed energy generation such as rooftop PV panels, the type and condition of poles, joint use of poles, and encroachment.  To correct these problems typically requires a foot survey which is expensive.  For example, a pole inventory costs at least $20 per pole.  At the Open Geospatial Consortium (OGC) Energy Summit at EPRI in Charlotte, John Simmins of EPRI described the research he is leading at EPRI to use modern technology including machine learning to reduce this cost to pennies per pole using available imagery such as Google Streetview.

I began working in the utility and telecom sector in 1993, specifically in developing software for records management (called network documentation outside of North America), and at that time the magnitude of the AEC+geospatial cultural mismatch was just beginning to be appreciated by utilities and telecoms. Planners tend to use GIS tools, engineers and designers CAD tools, construction folks in the field paper CAD drawings, and asset managers FM, GIS or integrated tools. Fashioning AEC and geospatial data into an efficient data flow from planning through design and construction to operations and maintenance represented a challenge that remains a problem for utilities.  For many utilities the backlog of as-builts and updates waiting to be entered into the GIS stretches into months, with the result that GIS data is permanently out of date inhibiting management and field staff from relying on this data operationally.  The Between The Poles blog is over ten years old and one of the persistent themes from its very beginning in 2006 was the challenge of integrating CAD and GIS data in a common workflow (some examples; 2006, 2007, 2008 ).

There are signs that modern technology is beginning to offer a solution to the problem of developing and maintaining an accurate digital model of electric power grids.

I wrote recently about a utility in western Pennsylvania that intends to implement an Advanced Distribution Systems ( ), but realized that to this requires a network data model. Duquesne Light is typical of many North American small to medium electric power utilities.  It does not have an outage management system or an electrical data model which means it is difficult to connect customers to transformers and to determine upstream devices when customers report power problems.  Currently Duquesne Light's circuit maps are AutoCAD drawings.  Duquesne Light is moving to modernize its grid and plans to deploy an Advanced Distribution Management Systems (ADMS) including outage management (OMS), DMS and DSCADA.  But the ADMS will require developing a complete distribution electrical data model and to that in the traditional way can be expensive.

To quickly, safely and cost efficiently develop a network data model, Duquesne Light looked at several alternatives including reality capture technology.  A pickup truck was equipped with with side mounted photo and IR cameras and LiDAR laser scanners. The reality capture survey could be completed rapidly and safely, with no boots on the pavement  Actual development of the data model was completed in the office without the necessity of going to the field for quality checks because the imagery provided all the data that was required to create and verify the data model.  Duquesne Light found several significant advantages of the reality capture approach.  The first was asset accuracy - both location and asset attributes.  In addition the captured imagery could also be used for other purposes.  For example, it could be used for a joint-use audit.

But this is still basically a manual process requiring a human being to scan images.  At the Energy Summit John Simmins described his research using neural network technology to automatically identify poles and attached equipment from available imagery such as Google Streetview.  Normally applying neural networks requires large quantities of training data.  To reduce this requirement, John has evaluated "biologically inspired", forward pass, neural networks.  In this approach the algorithm first looks (foveates) for vertical things that have the characteristics of a pole, and then for the characteristics of equipment attached to the pole, for example, cross members, street lights, transformers and large cables.  From this analysis is possible to determine pole location, height and width, cross member position on the pole and projected length, street light support strut position and length, position of transformer and large cable connection locations.  The ultimate goal of the research is in addition to automatically detecting each element of equipment to identify the type, size and manufacturer of each piece.

This is not a simple problem and there is no silver bullet.  Using a standard neural network approach requires large volumes of training data.  For poles it requires images of poles with different types of damage under different lighting conditions and in different types of weather.  An Australian company SiteSee claims to be able to do this for communications towers and transmission pylons - identifying the type and manufacturer of each antenna or insulator on a tower or pylon and comparing it to the manufacturer's specification to detect signs of corrosion and other types of damage.  When what you are looking for has a distinctive geometry that differentiates it from the surroundings, a biologically inspired approach may provide a way of identifying equipment without large volumes of training data.  Another area where several different approaches have been applied to automate feature extraction is electric power transmission lines for vegetation management.

Creating a digital twin of an electric grid is essential for developing and maintaining a smart grid.  Utilities have been wrestling with this challenge for decades and in many cases their GIS data is still not much closer to what is required.  If it would be possible to use the technology that John Simmins is investigating or other technology to reduce the cost of improving the quality of their GIS data, it would accelerate the adoption of smart grid technology at many utilities.  In his presentation at the Energy Summit Jared Green described his research in using smart meter and AMI network data to map secondaries from the transformer to the home, identify sources of unmetered load, and use impedance data from smart devices to correct utility GIS data.  Given the history of the challenge to develop and maintain accurate GIS data at many utilities, these approaches represent a beacon of hope which may enable utilities to cost effectively develop a digital twin of their networks that is an essential prerequisite for a  smart grid.

Firm realizes ROI of 203% by integrating CAD and GIS for fiber network management

Integrating CAD and GIS into an efficient workflow for records management (aka network documentation) has been a challenge for utility and telecom operators for decades.  While utility and telecom operators have found ways to address this problem, it is rare to find the benefits for these systems quantified.  At Year in Infrastructure this year iForte Solusi Infotek presented concrete benefits of a records management system that has been in operation for several years for managing their fiber optic networks in Jakarta and on the rest of Java.

I began working in the utility and telecom sector in 1993, specifically in developing software for records management (called network documentation outside of North America).  At that time the magnitude of the AEC+geospatial cultural mismatch problem was just beginning to be appreciated by utilities and telecoms.  Planners tend to use GIS tools, engineers and designers CAD tools, construction folks in the field paper CAD drawings, and asset managers GIS integrated with their FM tools. Fashioning engineering and geospatial data into an efficient data flow from planning through design and construction to operations and maintenance represented a challenge that remains a problem for utilities and telecom companies. The Between The Poles blog is over ten years old and one of the persistent themes from the very beginning in 2006 was the challenge of integrating CAD and GIS data and applications in an efficient workflow (some examples; 2006, 2007, 2008 ).

The winner of this year's Year in Infrastructure Award for Communications Networks was a project of iForte Solusi Infotek.   iForte is a fast-growing fiber network operator with many large corporate customers all over Java island including Jakarta.  From this base it plans to expand into the rest of Indonesia.  iForte was facing many of the problems associated with paper-based records management in a fast-growing network operator. Prior to implementing the new system, their business processes were characterized by a plethora of paper-based manual processes. The data available for network planning was continually out of date making it difficult to compile an accurate situational assessment. Maintenance crews in the field had to rely on their own hand written notes on paper records. When sales staff met with a prospective customer it took an hour or more to collect the required information to determine the availability of fiber optic capacity at the customer's location requiring subsequent exchanges of emails to get the requisite information to the customer by which time the customer may have contacted a competitor.

The primary benefits of the new system were realized in planning, network maintenance and sales. 

• Maintenance crews could redline existing as-builts in the field so that engineering could see immediately where a problem had occurred and any temporary infrastructure the maintenance team had put in place to address the problem.
• Planning could see real-time cable utilization to identify congested networks. When there was an outage they could see the total capacity that was at risk.  It also enabled them to find alternatives for re-routing the network.  It was also found that network expansion planning could be completed four times faster. They were also able to use less fiber, saving on material costs and CAPEX. For example in one case, by looking at available capacity and utilization, it was possible to reduce new cable requirements from 1,400 meter to 100.
• For sales prior to implementing the records management system, determining fibre availability for a customer required on average of 60 minutes necessitating telephone calls to records. After implementing the new system, they were able to determine availability in six minutes and automatically generate a bill of materials and a job estimate.

For these activities iForte estimated that it has realized a return on investment of 203% over three years.

The primary benefits of the new system were realized in planning, network maintenance and sales, but there were intangible benefits as well.  Management was able to see the status of the network in real-time, review progress in building out the network on a map with demographics, assess the status of their network assets, and identify regions for network expansion.  The intangible value to the company achieved with the new system was operational excellence and improved strategic planning which provided for better performance and growth compared to competitors.

Geolocation plays a key role in integrating GIS and engineering data and applications for optimizing asset management workflows.  One of the reasons for the success of iForte's system is that Bentley decided many years ago to geospatially-enable all of their products including Microstation.  Bentley calls this capability geocoordination. It provides a location-aware foundation which facilitates workflows that require integrated engineering and geospatial data and applications.