Between the Poles

Grid resilience demands we understand our infrastructure above and below ground

Jay Cadman, SVP Enterprise, IQGeo

As the glue interlinking everything from the industrial ‘Internet of Things’ to smart grids and underpinning the real-time social and economic services of the future, full fiber and 5G is increasingly recognized as an essential national utility. With superfast broadband forming the enabling technology for social and economic progress, any outages could have an equally devastating effect to power or water shortages. Europe is already pioneering ways of improving infrastructure’s resilience to climate change. This is becoming increasingly critical as superfast broadband faces a more distributed, fluctuating and unpredictable threat landscape than ever before.

Full-fiber cables are vulnerable to underground water while 5G will move radio signals to even higher frequencies where they are more susceptible to weather interference. For example, Hurricane Sandy damaged Verizon’s Northeast network and some basement-level communications hubs, while in 2018 Hurricane Michael caused major flood damage to underground fiber-optic cables in Florida. Achieving resilience across the telecoms sector will mean understanding the complex intersection of infrastructure above and below ground. Future fixed and mobile connectivity networks will all be 5G so that everything from underground fiber cables to radio waves will need to be managed as a single interconnected ecosystem above and below ground. Knowing the location of all infrastructure will be critical to understanding where all our fiber optic cables and 5G antennae intersect with the people and places that will need them. It is also vital to understand the common vulnerabilities that could produce a single point of failure affecting multiple overground and underground networks. Infrastructure planners will need to know the location of 5G networks in relation to nearby threats and hazards. Hazard data may even need to be shared with rival telecoms operators and authorities to ensure common resilience across the sector.

The Data Gap

This presents a major challenge because many telcos do not have current, comprehensive data on overground and underground assets in an integrated, easily accessible and shareable format. Many current systems fail to fully capture live insights from both workforce and network assets. Network data is rarely held in a modern, mobile-friendly format which is accessible to field workers or contractors. Many networks do not link all network data with location, which is critical to understanding risks in all regions of their network. Smaller operators are frequently not documenting their network in a digital SoR (System of Record). All too often, network data is not only siloed within an organization, but also incompatible with external datasets such as meteorological or ecological satellite data. This also means it cannot easily be shared with other organizations or authorities to help combat common vulnerabilities and ensure sector-wide resilience.

For example, there are 3.4 million miles of underground infrastructure including data lines and fiber-optic cables in the UK alone, yet there is no up-to-date or integrated map of this subterranean labyrinth. This makes it impossible to achieve common resilience against underground threats such as flooding or to see how other present or planned underground utility infrastructure might pose hazards to telecoms networks.

To help prioritize critical services and customers for maintenance or upgrades, operators need comprehensive and current location-based information on underground fiber-optic networks in relation to all the critical settlements and services that depend on them. Similarly, without a single map of infrastructure above ground, operators cannot predict how local weather might affect antennae.

In a pioneering example of what could be achieved, London is working with companies to create a combined mobile-friendly map of its subterranean infrastructure where the site and source of hazards in relation to overground services can be swiftly identified.

Achieving resilience

Telecoms’ unpredictable, varied, and fluctuating threat landscape means operators must now go beyond reactive risk management and recovery and adopt forward thinking and predictive network resilience. Creating resilient 5G networks will require a holistic overview of the site and the source of potential threats across overground and underground infrastructures. Crucially, all fixed and mobile networks will need to be managed as an interconnected ecosystem regardless of the location of physical infrastructure. Key to this challenge is the ability to combine data from multiple sources and harness all of a network’s assets as a source of real-time threat intelligence. This means adopting geospatial information systems capable of capturing a wide array of network data from employee smartphones to IoT sensors and matching it with precise and accurate location data to reveal the locus and the solution for every problem anywhere in the network. Accurate SoRs and ‘as-builts’ are critical to ensuring a comprehensive and current risk overview across all infrastructure and effectively targeting mitigation strategies, from repositioning antennae away from threats to replacing copper wire with fiber-optic cable. Matching all telecoms data with location data is essential to contextualizing threats, revealing interdependencies with neighboring infrastructure that could create shared vulnerabilities, and identifying priority customers or communities to protect. Using open platforms capable of integrating network information with external sources is also vital to allowing important information to be shared with authorities or other operators. This also allows network data to be merged with relevant external sources of risk intelligence such as weather or vegetation data.

Below I have outlined some steps companies can take to create data-driven resilient grids.

Step 1: Democratize network data

Operators must devolve responsibility for updating network information across the workforce, from field technicians to office managers. This involves harnessing open, mobile-friendly geospatial information platforms that can capture information from a decentralized array of sources, from IoT sensors to field worker smartphones. As-builts and systems of record should be digitized and easily accessible to workers on the ground who can ensure they stay accurate and up to date. This empowers workers, creating a sense of collective responsibility for network intelligence and resilience. Opening up network data should also extend to the use of open APIs to integrate location-based network data with important information from external sources such as weather hazard data.

Step 2: Risk assessment and resilience

Assess the network to identify and remove systemic vulnerabilities and single points of failure. Since all fixed and mobile networks will ultimately be 5G, organizations must ensure they have integrated, comprehensive and current geospatial information on all infrastructure above and below ground so they can easily identify threats. Companies should be creating network-wide digital ‘dashboards’ tracking live, location-based intelligence revealing the site and source of hazards such as storms or cyber-attacks. Operators must ensure they can capture live information from any device in the field, and that their network location data is accurate and up-to-date, so they can take rapid remedial action or mitigate risks specific to each region. Virtualization of networks also means the equipment itself can indicate risk or damage and this can be matched with location data to create a digital twin of networks that encourages more agile, adaptable risk management.

Step 3: System resilience & security

Whether organizations adopt third-party cloud environments or bring this in-house, they must support geographic resilience, system redundancy, and the latest advances in cyber security. Critical network software must ensure data is duplicated and encrypted to drastically lower the likelihood of singular points of failure.

Step 4: Incident response

Have a company-wide damage assessment and incident response strategy, both to preempt current threats and to provide insights which can be used to predict and prevent future risks. Organizations should start by creating a live, location-based overview of damage, degradation, or hazards to the network across both overground to underground infrastructure. Incorporating this data into incident reports not only helps create more rapid and even predictive maintenance of networks, but also encourages a proactive and predictive risk management model that can draw on past incidents to inform future solutions. For example, data on regions at risk of flooding can be used to protect nearby fiber-optic cables but also to inform planning of future networks that mitigates identified risks. In this way, networks grow collectively stronger and smarter from every incident.

Step 5: Practice

Stress-test systems, people, and processes with regular rehearsals to ensure resilience is drilled into all company procedures, workers, and systems. Just as companies perform health and safety training, they should test and train all assets for network resilience. This will ensure that best practice is baked into the organization.

The threats facing the telecoms industry today are constantly changing, extreme weather and other recent trends pose the threat of a ‘black swan event’ that affects entire systems of networks. However, by digitizing and decentralizing network data and creating proactive resilience strategies, telecoms organizations can take control. As 5G networks increasingly comprise a mix of networks and infrastructures above and below ground, the key is to integrate live data from all assets and manage telecoms grids holistically as a single living ecosystem. It is vital that network geospatial systems are a ‘living document’ constantly drawing on live intelligence from the field.

Contributed article by Jay Cadman, SVP Enterprise, IQGeo

 

Ontario moves to reduce risk of delays due to underground utilities for public construction projects

Utility locates and relocation work have historically been the cause of construction delays on public construction projects, particularly in urban areas with dense population and the commensurate underground utility and telecom infrastructure.  In recent bills the Ontario Legislature has addressed this problem of underground utility locates and relocations by escalated enforcement options when the work is not completed accurately and in a timely manner.

Forcing faster utility relocations in Greater Toronto transit projects

In July 2020, Bill 171 Building Transit Faster Act went into force.  The objective of the bill was to remove impediments that historically have slowed down transit projects by designating a transit corridor within which special regulations apply.  For example, once lands have been included in a transit corridor, if a property owner plans to carry out construction activities on these lands, they need to apply to obtain a permit before the work can begin. Bill 171 specifically identified four provincial transit projects in the Greater Toronto Area to which the new regulations apply; the Ontario Line, the Scarborough Subway Extension, the Yonge Subway Extension and the Eglinton Crosstown West Extension.

Among the new regulations were measures allowing transit builders to relocate utilities more quickly in preparation for projects. Specifically the bill provided a mechanism by which utility companies may be required to move utility infrastructure for the transit. It also provided for escalated enforcement. The bill enabled the Ministry of Transportation to delegate some of its powers to Metrolinx, the Greater Toronto and Hamilton transit authority, including enforcement tools such as stop-work orders, inspections, and administrative penalties.

Forcing more timely utility relocations on provincial transit and highway projects

Five months later in December 2020, Bill 222 Ontario Rebuilding and Recovery Act, went into force.  It extended the special regulations applicable to designated transit corridors in the GTA to other provincial “priority transit projects”.  It also provided measures to enforce compliance on highway projects by amending the Public Service Works on Highways Act to allow a judge of the Ontario Superior Court to order a utility company to comply with an order to relocate utility infrastructure.  In the case of noncompliance with the court order the judge was empowered to authorize a road authority such as the Ministry of Transportation to carry out the utility relocation work itself.

Metrolinx/Infrastructure Ontario approach to underground utilities

Underground utilities represent one of the top transit project delivery risks identified by Metrolinx and Infrastructure Ontario (IO).  Mitigating utility risk for transit projects is a continuous process involving many stakeholders.  They key elements in Metrolinx and Infrastructure Ontario (IO) approach to utility risk mitigation are early collaboration,subsurface utility engineering (SUE) surveys prior to engineering design, options to avoid, protect or relocate utilities, and making underground utility location data available to engineering firms and contractor bid teams.

Involving utility companies in the project as early as possible is crucial. Building collaborative relationship among project stakeholders and utility companies has been found in a number of jurisdictions to be key to avoiding major utility problems during construction. To enable the open sharing of information confidentiality agreements between Metrolinx/IO and utility companies are essential.

On all transit projects Metrolinx/IO conducts subsurface utility engineering (SUE) surveys in accordance with ASCE 38-02 along the entire transit alignment. The level of SUE required is typically a minimum QLC/QLB with QLA at locations of interest including stations and transmission utilities. Combining the results of the SUE survey with proposed alignments results in a conflict matrix identifying the locations of utility conflicts.

For each conflict, several options are available; the project scope can be adjusted to avoid the conflict, the conflicting utility can be left in place and protected if necessary, or the utility can be relocated. With this approach based on a SUE survey, the risk of unexpected utility problems occurring during construction can be reduced and unnecessary and expensive utility relocations avoided.

Enabling bid teams to have access to the available data on the location of underground utilities can improve the quality of bids and reduce risk and uncertainty that can inflate estimates.  Metrolinx/IO makes the available utility location information to all the bid teams.  In addition topic meetings allow for direct interaction between bid teams and utility companies with Metrolinx/IO present.

One of the major issues recognized by Metrolinx/IO is incomplete, inaccurate and out of data records of the location of underground utilities. To address this issue, Metrolinx/IO reports that asset management systems are being developed that will improve data tracking including the location of underground utilities.

Forcing timely underground utility locates for provincial broadband projects

Ontario one call legislation gives infrastructure owners five days to produce locate information, but the deadline is often ignored. Late locates can lead to long delays and extra costs for contractors, excavators, locators, utility owners and other stakeholders.The recently passed Bill 257 Building Broadband Faster Act, 2021 which is intended to expedite the delivery of broadband to under served areas of Ontario, revises the 2012 Ontario One Call Act by requiring network owners to accurately locate their equipment within 10 business days after notification by Ontario One Call or face penalties and other actions that are available to the Minister of Infrastructure. It enables the broadband project owner to claim financial compensation if the utility or telecom either failed to accurately mark the ground and provide a written document showing the location of its underground infrastructure or incorrectly stated that none of its underground infrastructure would be affected by the excavation.  The act enables the Ministry to authorize excavation or digging to proceed when locates are not performed within 10 days.

Conclusion

With these bills, the measures relating to locating and relocating underground utilities initially applied to GTA transit projects have been extended to province-wide transit, highway and broadband projects. Together they cover most of the infrastructure in the provincial  public right of way.

Bringing reliable near real-time GIS within reach of every utility

Workflows for ensuring utility GIS quality are fundamentally broken and have been for decades.  Integrating CAD and GIS into an efficient workflow has been a challenge and it has simply not been a priority in the utility industry and in some parts of the communications sector to ensure real-time, high quality asset GIS data. Recently business drivers for high quality GIS data such as implementing a smart grid, estimating service availability and other applications have made GIS quality a top priority.  And now non-traditional geospatial companies are offering geospatial-based technologies that are revolutionizing the capture and maintenance of GIS information about utility and communications assets that can provide a foundation for the real-time network digital twin that is essential in the era of the smart grid and our increasing dependence on communications and utility networks. 

Broken business processes degrades GIS data quality

Maintaining accurate, up-to-date asset data has been a challenge for utility and communications firms for decades.  Over the years the lack of CAD/GIS interoperability has cost communications and utility companies hundreds of millions of dollars trying to cleanup their GIS data and streamline their CAD+GIS workflows.  One of the persistent themes from the very beginning of the Between The Poles was the challenge of integrating CAD and GIS data in a common workflow (some examples; 2006, 2007, 2008 ).  Critical metrics for utility readiness for the smart grid is the as-built backlog, field update backlog and the location accuracy of the assets in the utility's GIS.  It is not uncommon for as-built backlogs to stretch into months even years, for updates from the field to take months to be entered into the records database if they are entered at all, and for the location of assets as recorded in the GIS to be city blocks from its actual location.   The challenge is to fashion workflows that include both engineering and geospatial data and applications into an efficient data flow from planning through design and construction to operations and maintenance with feedback loops that ensures an accurate, near real-time GIS.  For many utilities and communications firms traditional GIS, which has been the standard platform for utility GIS for the past few decades, has simply not delivered cost-effective, real-time, high quality asset location and associated data.

A key workflow found at all utilities and communications firms is the field maintenance process.  In many utilities this is a broken feedback loop which results in the quality of GIS data degrading over time.  When a field crew performs maintenance on network infrastructure, which includes routine and emergency maintenance, or encounters equipment incorrectly located or described in the GIS, the changes or the corrections to the GIS data from the field need to get back into the GIS quickly.  Too frequently this is still a paper-based process typically involving redlining as-builts, which get to sent to records for updating in the GIS.  The backlog associated with field updates can stretch into months.  The slow turnaround of field updates discourage field staff from reporting corrections and changes further exacerbating the GIS quality problem.  The result of this and other broken workflows such as the as-buillt workflow is that not only is GIS data incomplete, inaccurate and out-of-date, but that its quality degrades with time.

Business drivers for accurate, real-time GIS

For communications companies a major driver for an accurate and up-to-date GIS is providing rapid availability of service estimates to prospective customers.  I remember a large U.S. telecom that because it could not rely on the information in its GIS had to resurvey all the locations of its fiber terminations in order to respond to potential ADSL customers indicating whether they were close enough to a fiber termination for the service.

For electric power utilities a powerful business driver that has emerged in the last few years is the smart grid.  Digitalizing the grid has made the role of geospatial data and technology strategic for utilities. As a Navigant Research report stated the smart grid is all about situation awareness and effective anticipation of and response to events that might disrupt the performance of the power grid. Since spatial data underlies everything an electric utility does, GIS is the only foundational view that can potentially link every operational activity of an electric utility including design and construction, asset management, workforce management, and outage management as well as supervisory control and data acquisition (SCADA), distribution management systems (DMSs), renewables, and strategy planning. As John Simmins of EPRI has reiterated on a number of occasions a prerequisite to implementing a smart grid is cleaning up GIS data and developing business processes (feedback processes) to ensure that it remains accurate and up-to-date.

Technology enablers: Open standards, open source geospatial and handhelds

Recent technical advances have made it possible to implement cost-efficient processes for ensuring that utility asset data including GIS data is accurate and up-to-date.  There are now many widely adopted geospatial standards; Simple Features for SQL, WMS, WFS, GML, KML, and InfraGML to name just a few from the OGC that were not available two decades ago.  Secondly, since the founding of the Open Source Geospatial Foundation (OSGEO) in 2006 there has been rapid development of open source geospatial projects including PostGIS, GeoServer, MapServer, Leaflet and others.  This open source code-base has enabled entrepreneurs to rapidly develop low cost geospatial applications because the cost of maintaining most of the code is shared across a broad developer community.  Thirdly the advent of the handheld touch screen computer and phone provides a low cost universally accessible hardware platform for field staff to access geospatial and other information stored in a utility's databases.  With the appropriate software smart phones and tablets make it possible for field staff to see all of the information about an asset in the field at the touch of a finger.  For example, I blogged about how Duke Energy field staff are using an open source based mobile application which provides a single, common easy-to-use web tool for viewing geospatial asset data stored in GE Smallworld, ESRI ArcGIS, Intergraph GTechnology, spatialNET and Google Maps as well as non-GIS data in Maximo, SAP, three different outage management systems, four or five customer information systems, CAD drawings and photos and images.

Shifting the paradigm: GIS data maintenance from the field

The paradigm shift is empowering field staff to take responsibility for the quality of data in the utility GIS. Since they have direct knowledge about the location and condition of utility infrastructure, the key to revolutionizing data maintenance and enabling near real-time, accurate GIS and other data is "cutting out the middle man" and enabling field staff to maintain the utility GIS and other data in the field as they are working. Enabling field folks to do this without adding significantly to their workload (and over time reducing their workload as the quality of the GIS improves) requires a simple "touch and go" user interface which minimizes typing, responsive applications running on handhelds, and workflows that enable field staff to see their changes with a minimum delay measured in hours or days at the maxium.  Of course, operational data can be protected from problematic updates using data versioning technology (long transactions) which provides a rapid way for records folks to review and identify potentially problematic updates.

At Distributech 2019 a new entrant IQGeo emerged into the utility and communications space and announced an alternative to traditional GIS.  myWorld Capture is a mobile solution that prioritizes direct data maintenance from the field.  Building on an open source geospatial platform that supports data versioning and interfaces to legacy utility/communications GISs such as ArcGIS and Smallworld, myWorld provides tools that make it possible to develop and maintain a high quality near real-time GIS with the information that is required for a complete network asset data model including equipment location, descriptions, connectivity, condition, and status. 

The key to ensuring that the data is high quality and up-to-date is empowering field workers to make updates at the source as the work happens and to automate the data maintenance workflow as much as possible.  This approach can eliminate backlogs in field updates, encourage field staff to take responsibility for ensuring the quality of the utility GIS and enable an effective feedback loop that improves GIS data quality over time.   With the promise of near real-time updates directly from the field this approach provides a key foundation for the digital twin of the grid that integrates sensors reporting in real-time with simulation and analytics that makes it possible to create a accurate real-time situation assessment and to model the electrical state of the grid and possible future states based on reported or forecast outages and planned expansions and other changes to the electric network.

Operationalizing the new paradigm

How does this work in real world practice? I had a chance to chat with Dick Rohm, Director of Engineering at Cable One about how they are progressing toward their goal of no backlogs for field updates or for as-builts.  The business driver for this initiative, which they have been working on for the past several years, is service availability - being able to tell a customer immediately whether a fiber service is available at the customer's location or whether it will require an addition to the network.

Cable One provides commercial fiber services to businesses as well as cable services in 21 states, primarily in small cities and big towns.  Sales success requires being able to respond rapidly to potential customers requests for business fiber service, and say yes we already have the fiber capacity in your neighbourhood or no we have to build out a fiber line to support you.  Improving the quality of the data in their GIS is an important business driver for Cable One.  

By prioritizing as-built and update workflows Cable One has managed to ensure that as-builts and updates are in the GIS quickly, typically within 2-3 business days.  To achieve this, they first changed the as-built process, making the design process paperless about five years ago. Their engineering GIS allows engineers to use AutoCAD to design fiber and RF networks directly into GIS .  For construction, contractors are provided with AutoCAD drawings (DWG) on thumb drives and it is up to them if they wish to print them.  For small jobs, contractors can work completely electronically. For large jobs, contractors still use paper, but they print the DWG files themselves. Contractors must register with Cable One's Project Central which is used to track all projects. When paper as-builts are returned, they are spot-checked, and the GIS is updated based on any redlines. A project will not be closed out (and contractors paid) until redlines are in the GIS.  The GIS is used by Marketing and Sales as an added application to myWorld.

For updates from the field the initial idea was to use a web extension to their GIS, but they found that it was too cumbersome and slow.  It was replaced by myWorld on a handheld working with Google or Bing Maps which provided front end for viewing all engineering geospatial and associated data in the field. The myWorld solution only requires about an hour training time using a video. 

Based on the success of the myWorld implementation, user acceptance testing of myWorld Capture is currently underway. myWorld Capture enables field staff to update GIS and other data as they work.  Ease-of-use and rapid response are its key features. It makes it possible for field validators to do most of their work by tapping and drag and drop with very little typing.  Field engineering staff have to do more typing but even for them it is minimal.

The objective of the myWorld Capture implementation is 24-hour turnaround for updates from the field. It could even be less, but it is currently limited by the process of posting updates to the GIS and to the MyWorld database, which occur every night.  myWorld Capture will roll out to all field staff by July 1 of this year.
Dick Rohm emphasized that the key to making a transformative project, such as this at Cable One, successful is changing thought processes in the field. Data needs to be treated as just as important as physical assets.  When something changes or an error is found, the priority for field staff is to update it.  To encourage field staff to make changes in the GIS, tools need to be provided on handhelds that are fast and easy to use. Update processes must be fast so that their changes can be seen by field staff quickly in the GIS. Over time, field folks will increasingly rely on the GIS because they will see corrections that they make from the field appear in the database within 24 hours or sooner.  As data quality improves as a result of this feedback loop, field staff will find that they can rely on the GIS for locating and accessing up-to-date information about assets.

Conclusion

Prioritizing updates from the field to reduce the field update backlog to 24 hours or better together with an as-built backlog on the order of two to three days is a remarkable achievement.  It shows that with a business perspective that prioritizes a near real-time, reliable digital twin of utility and communications networks and geospatial technology supporting open standards and based on opensource and implemented for handhelds in the field, a cost-effective solution to the decades old GIS data quality challenge is within the grasp of any utility or communications firm.

Firm realizes ROI of 203% by integrating CAD and GIS for fiber network management

Integrating CAD and GIS into an efficient workflow for records management (aka network documentation) has been a challenge for utility and telecom operators for decades.  While utility and telecom operators have found ways to address this problem, it is rare to find the benefits for these systems quantified.  At Year in Infrastructure this year iForte Solusi Infotek presented concrete benefits of a records management system that has been in operation for several years for managing their fiber optic networks in Jakarta and on the rest of Java.

I began working in the utility and telecom sector in 1993, specifically in developing software for records management (called network documentation outside of North America).  At that time the magnitude of the AEC+geospatial cultural mismatch problem was just beginning to be appreciated by utilities and telecoms.  Planners tend to use GIS tools, engineers and designers CAD tools, construction folks in the field paper CAD drawings, and asset managers GIS integrated with their FM tools. Fashioning engineering and geospatial data into an efficient data flow from planning through design and construction to operations and maintenance represented a challenge that remains a problem for utilities and telecom companies. The Between The Poles blog is over ten years old and one of the persistent themes from the very beginning in 2006 was the challenge of integrating CAD and GIS data and applications in an efficient workflow (some examples; 2006, 2007, 2008 ).

The winner of this year's Year in Infrastructure Award for Communications Networks was a project of iForte Solusi Infotek.   iForte is a fast-growing fiber network operator with many large corporate customers all over Java island including Jakarta.  From this base it plans to expand into the rest of Indonesia.  iForte was facing many of the problems associated with paper-based records management in a fast-growing network operator. Prior to implementing the new system, their business processes were characterized by a plethora of paper-based manual processes. The data available for network planning was continually out of date making it difficult to compile an accurate situational assessment. Maintenance crews in the field had to rely on their own hand written notes on paper records. When sales staff met with a prospective customer it took an hour or more to collect the required information to determine the availability of fiber optic capacity at the customer's location requiring subsequent exchanges of emails to get the requisite information to the customer by which time the customer may have contacted a competitor.

The primary benefits of the new system were realized in planning, network maintenance and sales. 

• Maintenance crews could redline existing as-builts in the field so that engineering could see immediately where a problem had occurred and any temporary infrastructure the maintenance team had put in place to address the problem.
• Planning could see real-time cable utilization to identify congested networks. When there was an outage they could see the total capacity that was at risk.  It also enabled them to find alternatives for re-routing the network.  It was also found that network expansion planning could be completed four times faster. They were also able to use less fiber, saving on material costs and CAPEX. For example in one case, by looking at available capacity and utilization, it was possible to reduce new cable requirements from 1,400 meter to 100.
• For sales prior to implementing the records management system, determining fibre availability for a customer required on average of 60 minutes necessitating telephone calls to records. After implementing the new system, they were able to determine availability in six minutes and automatically generate a bill of materials and a job estimate.

For these activities iForte estimated that it has realized a return on investment of 203% over three years.

The primary benefits of the new system were realized in planning, network maintenance and sales, but there were intangible benefits as well.  Management was able to see the status of the network in real-time, review progress in building out the network on a map with demographics, assess the status of their network assets, and identify regions for network expansion.  The intangible value to the company achieved with the new system was operational excellence and improved strategic planning which provided for better performance and growth compared to competitors.

Geolocation plays a key role in integrating GIS and engineering data and applications for optimizing asset management workflows.  One of the reasons for the success of iForte's system is that Bentley decided many years ago to geospatially-enable all of their products including Microstation.  Bentley calls this capability geocoordination. It provides a location-aware foundation which facilitates workflows that require integrated engineering and geospatial data and applications.

Avoiding a repeat of the CAD/GIS quaqmire of the past quarter century

Yesterday at Distributech 2018 in San Antonio I listened to a talk from a consulting company contracted by an electric power utility to clean up GIS data originally sourced from CAD drawings.  This reminded me once again that is a challenge that telecommunications and cable companies and electric and water utilities have been attempting to deal with at least since the early 1990s and for some even before that.  Telecommunications and cable and leading electric power utilities resolved this problem at least a decade ago, but for some electric power and even more water utilities it remains a problem that is going to hold these utilities back when they should be accelerating their adoption of digital technologies.

I began working in the utility and telecom sector in 1993, specifically in developing software for records management (called network documentation outside of North America), and at that time the magnitude of the AEC+geospatial cultural mismatch problem was just beginning to be appreciated by utilities and telecoms. In utilities and telecoms planners tend to use GIS tools, engineers and designers CAD tools, construction folks in the field paper CAD drawings, and asset managers GIS integrated with their FM tools. Fashioning AEC and geospatial data into an efficient data flow from planning through design and construction to operations and maintenance represented a challenge that remains a problem for utilities. The Between The Poles blog is over ten years old and one of the persistent themes from the very beginning in 2006 was the challenge of integrating CAD and GIS data in a common workflow (some examples; 2006, 2007, 2008 ).

A workflow that is found at every utility and telecom involves the design-build-maintain information flow. The workflow begins with an engineer and drafter producing a design, typically a CAD drawing, which is printed for the construction contractor to use in building a facility. The contractor uses the drawing in the field and then returns it as an "as-built" to the utility's records management group, who enter it, traditionally by manually redigitizing it from the paper drawing, into a GIS that is intended to contain the digital twin of the network. The data in the GIS is used for network maintenance, outage management and other functions. The reason for the manual redigitization step is the "errors" that CAD drawings inevitably contain unless drafters are forced to adhere to "geospatial rules". This entire process is slow and error-prone and backlogs of as-builts waiting to be entered into the GIS stretch into months and even years. The result is that utility GIS data is more often than not incomplete, out-of-date, and unreliable. For example, at the beginning of the broadband revolution, telecoms needed to be able to tell customers whether they were close enough to a terminal and fiber to be eligible for ADSL. In one of the largest telecoms in the U.S. the GIS records were so incomplete and unreliable that survey teams had to be sent out at considerable cost to resurvey the location of their terminals and fiber cables.

Because there was no standards-based solution to the CAD+GIS problem, every utility and telecom has had to try to find its own solution to the problem. This problem has persisted from the 1990s and remains a problem for utilities - as I said I heard a presentation yesterday that wasn't that different from presentations on the same topic I and others gave a quarter of a century ago.

Over the years the poor CAD/GIS interoperability has cost the utility and telecom sectors considerable amounts of money. Large telecoms have spent hundreds of millions of dollars trying to cleanup their GIS data and streamline their CAD+GIS workflows. A rough back-of-the-envelope calculation suggests this problem has cost directly and indirectly trillions of dollars over the past 20 years. In addition unreliable and out-of-date network infrastructure data that results from this problem has safety implications.  

The juncture between distinct sectors each with its own technologies and business models, AEC (Autodesk, Bentley, and others) and geospatial (ESRI, Hexagon, and others), has analogies with the clash of tectonic plates and the associated incessant seismic events. In the mid-1990s pressure from customers motivated Autodesk and ESRI, 800 lb gorillas in their respective areas of CAD and GIS, to attempt to collaborate to address the CAD+GIS problem. The result of the agreement between Jack Dangermond and Carol Bartz at the time resulted in an unsuccessful product called ArcCAD, which satisfied neither CAD nor GIS users.

Averting a tectonic clash repeat: BIM+geospatial interoperability

The prospect that, by analogy with CAD+GIS problem, we could still be individually finding solutions to BIM/geospatial integration 20 years from now is discouraging. While there are parallels between CAD/GIS and BIM/geospatial, there are grounds for optimism that this time around we may find ways the address the problem in a general way. Firstly, in the mid 1990s there were virtually no geospatial or 3D AEC data standards. The Open Geospatial Consortium (OGC) and builsingSmart were both only founded in 1994. It required quite a while for the OGC geospatial and buildingSmart's IFC standards to broadly penetrate industry and government.  Nowadays, there are many widely adopted geospatial standards; Simple Features for SQL, WMS, WFS, GML, KML, InfraGML, CityGML to name just a few from the OGC. buildingSmart's IFC standard is supported by all major BIM design tools including the market leader Revit.

Perhaps most significantly there has been significant progress in developing common conceptual models for geospatial and AEC views of city infrastructure. The major breakthrough in bringing the architectural and geospatial views onto a common footing is the OGC LandInfra Conceptual Model developed by the OGC in cooperation with buildingSMART International and approved as an OGC standard in August, 2016. The LandInfra conceptual model was developed by Bentley Systems, Leica Geosystems, Trimble, Australian Government Department of Communications, Autodesk, Vianova Systems AS, and buildingSMART International and provides a unifying basis for land and civil engineering standards including the OGC's InfraGML and buildingSmart International's IFC for infrastructure standards.  InfraGML is the OGC's application schema supporting land development and civil engineering infrastructure facilities. InfraGML supports a subset of the existing LandXML standard. buildingSmart International's IFC-Alignment project uses this common conceptual model of alignments for roads, railways, tunnels and bridges. The objectives of the IFC-Alignment project is to enable the exchange of alignment information through the full infrastructure lifecycle from planning through design and construction to asset management.

Secondly, in the 1990s and the early 2000s there were few open source geospatial projects. The Open Source Geospatial Foundation (OSGEO) was only founded in 2006. Since then there has been rapid development of open source geospatial projects. There is enough open source geospatial code now that at the GEO|Design+BIM conference in Amsterdam, Sanghee Shin described a web-based geo-platform integrating massive, complex BIM models using open source geospatial libraries and tools including PostGIS, GeoServer, and Cesium and NASA WorldWind virtual globes. The importance of this project is that it shows that it is possible using the available open source geospatial APIs and libraries - together with some genuine innovative development - to create an open, non-proprietary, 3D geospatial platform for integrating geospatial and BIM. Given the critical importance of addressing the cultural and technical divide between the AEC and geospatial worlds, clearly many believe that a viable open source alternative will be essential for developing the innovative solutions to the challenge of interfacing the two worlds. 

Finally, major players in the AEC and geospatial industries, Autodesk and ESRI, have again agreed to collaborate. This time around they are probably more aware of the technical issues hindering BIM+geospatial integration. Both have adopted open standards when it made business sense - often under pressure from government clients - and both have also adopted open source technology where it made business sense. Together the combination of major players supporting the effort, standards organization like the OGC and buildingSmart already working to address the problem, and a broad range of open source code and a vibrant developer community may be enough to address the technical problems. If building and infrastructure owners and facilities managers including government as well as regulators become aware of the risk of not solving the problem, a solution may be found so that we will not be having the same BIM+geospatial integration discussion a quarter of a century from now.

Open source spatial data versioning coming for utilities

The Research and Markets report Global GIS Market in the Utility Industry 2014-2018 identifies an important new market trend in the increasing penetration of open-source geospatial software into the utility market. According to the report many small and medium-sized organizations that cannot afford to invest in expensive proprietary GIS solutions are now choosing open-source software. In the U.S. most of the 3000 electric power utilities are small to medium-sized organizations, primarily municipal and rural electric coops.

Spatial data versioning (long transactions) for utility records management

Two essential capabilities for utility and telecom records management are storing spatial data in an SQL database and long transactions, more commonly referred to as data versioning these days  For a utility data versioning makes it possible to store spatial design data in the same database as as-built data, enabling design engineers to work on new designs at the same time as operations staff are relying on as-built information to operate and maintain active utility networks.  Many of the largest telecoms and utilties have outside plant solutions based on these technologies, in some cases since the 1990s.

Open source spatial data versioning

In the most recent Eclipse LocationTech Newsletter, Paul Ramsey describes the latest development on a major open source initiative to support spatial data versioning.  Paul points out that software developers have long had extremely versatile tools, such as Git, for managing software versions which tracks the details of all changes made to source code including dependencies in a distributed environment.  He and others have asked why don’t we have this capability for spatial data management ? 

The answer is GeoGig which takes the concepts used in distributed version control such as Git and applies them to versioned spatial data.  GeoGig is written in Java and is available under an open source  BSD License.

At the Location Intelligence Summit earlier this year in Washington, Juan Marin, CTO at Boundless,  gave an overview and demonstration of what was then called GeoGit, a distributed system for geospatial data versioning using the same philosophy as Git.

To further leverage the abilities of GeoGig, Boundless has announced Versio for organizations to manage their spatial data.  According to Paul Versio is a "hosted service built on GeoGig that provides easier toolks for data authors to collaborate on spatial data while streamlining the creation, maintenance, and distribution of spatial data."

Git

By way of background Git is a distributed revision control and source code management (SCM) system.  Git was initially designed and developed by Linus Torvalds for Linux kernel development in 2005.  It is free open source software distributed under the GNU General Public License (GPL)  V 2.

Every Git working directory is a full-fledged repository with complete history and full version tracking capabilities. Each local repository is standalone and is not dependent on network access to a central server. Git is known for very fast performance and for scalability.

History of long transactions for telecom/utility records management

Two capabilities that are essential for utility and telecom records management, storing spatial data in an SQL database and long transactions, were introduced in the early days of Automated Mapping/Facilities Mapping/Geographic Information Systems (AM/FM/GIS).  The first commercial systems implementing this technology in the early 1990's were IBM's GFIS which ran on DB2 and VISION* which managed versioned spatial data in an Oracle RDBMS (this was long before Oracle Spatial and Workspace Manager).  VISION* systems are still running at First Energy, supporting hundreds of concurrent utility network designers, and at CenturyTel (at the time US West then Qwest) and Telefonica Sao Paulo (at the time Telebras where the far-sighted visionary was the late Geovane Magalhaes) where each VISION* system supports about a thousand concurrent telecom outside plant designers.  VISION* was acquired by Autodesk in 1999.

GITA: Knowledge transfer, the aging workforce and data quality

As I blogged previously there were several very important and topical presentations at the GITA conference this year addressing the challenge of the aging workforce.  The problem is severe in many of the world's economies, but especially in Japan which has the highest proportion of citizens over the age of 65, about 23% of the population.  The population of Japan is declining and there is data that suggests this trend is accelerating.  The National Institute of Population and Social Security Research has estimated that Japan's population will drop to below 100 million by 2046.

Japan is also facing the same challenges that other advanced economies are plus some unique to Japan including renewable energy, safe nuclear power, increased broadband access, addressing sea level rise, intelligent networks/smart grid, water management, and rebuilding after the recent major earthquake and resulting tsunami.  For Japan's telecommunications and utility companies this means more things to do and fewer people to do them. 

A related problem is the quality of the data in the facilities databases maintainted by utility and telecommunications companies.  Historically network operators have managed to muddle along with data that can be up to several years out of date because of as-built backlogs and inaccurate because current business processes are not targetted on optimizing data quality, for example, operations staff are often not encouraged to report errors they see out in the field.  But with the next generation of intelligent networks or smart grids, many in the industry believe that utilities and telecommunications firms are going to have to get serious about maintaining what one long time utility specialist calls "100% accurate, real time" facilities data.

At the GITA conference this year, Toyokazu Fukui, Director of Development at NTT Infranet, described the workforce challenges at NTT Infranet.  The size of the company's workforce is expected to decline from its current 1700 employees to 1400 by 2016.  In addition many employees are over 52 and about half will be eligible to retire in the next ten years. In his presentation Fukui-san describes how NTT Infranet has changed its data management practices and  implemented an IT system to enable knowledge transfer from the older cohort of workers to the younger generation.

NTT Infranet realized that a critical requirement to acheve business benefits from a geospatially enabled facilities database is that the data be reliable and current. Some of the specific problems that Fukui-san mentioned NTT Infranet experienced with its first geospatial implementation include data quality issues resulting from delayed data entry, redundant data, a large headcount dedicated to data entry, data chronically out of date and unable to be used for day to day operations, and limited use across the organization.  None of these are unfamiliar.

Fukui-san described how NTT Infranet is making it possible to capture the "knowhow" of experienced workers in the company's facility database so that it can be accessed by all workers, especially younger, less experienced workers.  The ciritical areas that NTT Infranet identified as priorities for enabling this to happen are (1) a single location aware infrastructure (logical) database containing all the company's information about its facilities, (2) simplified data capture that enables faster and more timely entry of the data critical for business operations, and (3) adapting business processes to optimize data quality.  With this approach NTT Infranet is realizing business benefits such as lower costs of operations.   But this approach also provides a comprehensive, current view of network operations which enables the younger generation to suggest improved ways to do things that had not occurred to their seniors.

Map Asia 2010 Expands to Include Utility and Infrastructure Symposium

Non-vendor general geospatial conferences are having a challenging time these days.  There are alternative explanations for why this is happening.  Some say it is the result of geospatial no longer being special, that it is now part of standard IT functionality in most vertical application areas, others that customers have for the most part made their "GIS decision" and are now going to the user conference of their vendor of choice.

Several years ago GIS Development, the organizer of Map Asia and other international geospatial conferences, decided to broaden the appeal of what had been originally GIS conferences by including more vertically focussed sub-conferences.  For example, this year Map Asia, which is taking place in Kuala Lumpur, includes several vertical sub-conferences.

• Symposium on Utility and Infrastructure
• Agriculture and Plantation Seminar
• Ocean Information Services Seminar
• Special Session on Geospatial for Socioeconomic Development

The Utility and Infrastructure Symposium, which took place today,  was a day long symposium with speakers presenting on a wide range of infrastructure related topics including roads and highways, energy, water, electricity, gas, port infrastructure, and telecommunications.  The presentations all focussed on addressing vertical domain specific problems using geospatial and other technology.

The opening session at which I spoke was very well attended, in fact packed, with over a hundred people by my estimate.  It was a very attentive, mature audience and there were several excellent questions afterward.  My conclusion is that GIS Development are onto something, that geospatial has matured to the point where people are no longer primarily interested in GIS per se, but in learning about how problems in specific vertical sectors can be addressed using geospatial together with other technology.

FCC: Current State of Broadband in the US

The American Recovery and Reinvestment Act (ARRA) directed the FCC to submit a National Broadband Plan to Congress by February 17, 2010 that addresses broadband deployment, adoption, affordability, and the use of broadband to advance solutions to national priorities, including health care, education, energy, public safety, job creation, investment, and others. 

The Federal Communications Commission (FCC) has released an initial report on the current state of broadband in the U.S.

• Actual broadband speeds lag advertised speeds by at least 50% and possibly more during the busy hours.
• About 1% of users drive 20% of traffic, while 20% of users drive up to 80% of traffic.
• Preliminary analysis indicates that approximately three to six million people are unserved by basic broadband (speeds of 768 Kbps or less).  
• Preliminary estimates showing that the total investment required for universal broadband access ranging from $20 billion for 768 Mbps-3 Mbps service to $350 billion for 100 Mbps or faster. 
• The cost of providing consumers with a choice of infrastructure providers, and/or ensuring that all consumers have access to both fixed and mobile broadband would be significantly higher than these initial estimates. 
• Universal Service Fund recipients have made progress bringing broadband to rural America, but the fund faces systemic and structural problems.
• Nearly 2/3 of Americans have adopted broadband at home, while 33% have access but have not adopted it, and another 4% say they have no access where they live. 
• Wireless is increasingly moving to broadband, with smartphone sales projected to overtake sales of standard phones by 2011. 
• Bandwidth-hungry devices, applications and users are putting pressure on existing network capacity and driving many to cite the need for additional spectrum.
• The driving force behind national broadband plans in other nations has been competitiveness, job creation and innovation. 
• Successful national plans need four or more years of continuous effort and consistent funding sources.
• Broadband can be part of the solution to many of the nation’s challenges, creating economic and social benefits.
• Electronic health records, telemedicine, and mobile monitoring result in better, more affordable health care.
• Smart grids, smart homes and smart transportation, all of which require broadband, will be a critical part of our clean energy future.
• Delivery of government services, civic engagement, transparency in public policy can all be improved by broadband access and adoption.
• Over 70 percent of all high school students use the Internet as a primary source for homework. 
• Digital textbooks, online learning, teacher support and communications, digital student records can improve weak U.S. educational outcomes. 
• Internet use usage among people with disabilities is less than half that of the general population.
• Networks, equipment, services, devices and software are not designed to be accessible to people with disabilities.
• Consumers say online purchases save time and money, yet 39% have strong worries about giving out personal or credit card information. 
• 82% of upper income users compared to only 29% of low-income users have made purchases online.
• There are no mobile, wireless broadband communications services that meet the public safety community’s specialized needs
• As of 2005, over 77 percent of Fortune 500 companies post jobs and accept applications solely online.

Autodesk and SpatialInfo Announce Partnership for Telco and Cable Network Design and Management

Greg Foster, CEO of SPATIALinfo, and Alan Saunders

, Utility and Telecom Industry Manager at Autodesk announced the next stage in the partnership between Autodesk and SPATIALinfo at the SPATIALinfo User Conference this morning.  SPATIALinfo is Autodesk’s first Preferred Industry Partner in the cable/telco space.  SPATIALinfo has been developing telco/cable industry solutions to work with Autodesk products since 1997, and their

primary network design and management tool spatialNET is AutoCAD-based and is designed for network engineers to design and manage fiber, coax, copper and power networks, and supports both inside and outside plant.

SPATIALInfo and Autodesk also demonstrated a workflow showing design information flowing between SpatialNET, Autodesk Design Review, and Microsoft Office using Autodesk Buzzsaw to manage the workflow including email notification.