Between the Poles

Calculating forest loss using open source deep learning tools and publicly available training data

Deep learning algorithms have been developed by academia and as a result the code for the most part is open source but the successful training of deep networks requires thousands of labeled training samples and at the present time this training data is typically not open. In their presentation at this year's FOSS4GNA (Free and Open Source Software for Geospatial North America) get together in St Louis Jason Brown and Courtney Whalen, data scientists at astraea, showed how deep learning using publicly available labeled data for training was applied to track deforestation and reforestation in Mato Grosso, a state in the central amazon region of Brazil.

Chris Holmes of Planet Labs, in his insightful talk at FOSS4GNA in St Louis about the application of deep learning to geospatial data, identified a challenge in making this technology open. The deep learning algorithms have been developed by academia and as a result the code for the most part is open source. For example, a deep neural network model developed originally for medical image segmentation called U-Net is open source and has been applied to identifying building footprints. Successful training of deep networks requires thousands of labeled training samples. Labeled data involves people on the ground manually ground-truthing land use types and other features so that the deep learning algorithms can learn what to recognize. At the present time this training data is typically not open source. In this presentation by Jason Brown and Courtney Whalen, both data scientists at astraea, deep learning using publicly available labeled data for training was used to track deforestation and reforestation in Mato Grosso, a state in the central amazon region of Brazil.

This is computationally intensive and a distributed engine was used. The computation engine used open source components. Spark is a top level Apache project which enables distributed processing for global scale computation. RasterFrames is a free and open source toolkit allowing scientists, data scientists, and software developers to process and analyze geo

Forest cover in Mato Grosso in 2002

patial-temporal raster data with the same flexibility and ease as any other data type in Spark DataFrames. This is a LocationTech raster project and is built on GeoTrellis. Using this software each year required 6 to 7 hours of computation using 48 cores.

The imagery that was used was captured by the MODIS satellite for the years 2001 through 2017. MODIS monitors the reflection back from ground cover for several bands including red, green, blue, short wave infrared and near infrared. Its cameras have a spatial resolution of 500 by 500 meters and a revisit rate of one to two days. From the bands that it captures the normalized difference vegetation index (NDVI) can be calculated. From the data monthly means and yearly aggregates can be calculated.

Forest cover in Mato Grosso in 2011

The training data used came from the System for Terrrestrial Ecosystem Parameterization (STEP) which has 2000 manually labeled sites covering 17 different land cover types including five forest types scattered across all continents. The model was trained on MODIS 2012 data. 80% of the data was used for training. After training was completed the remaining 20% was used to test the model.

Comparison of rate of forest loss with Global Forest Watch for 2001 to 2016

After training and testing the first application was to Mato Grosso in central Brazil, a large state that has seen a lot of deforestation. The rate of deforestation tracked the rate estimated independently by the Global Forest Watch for the years 2001 to 2017. The major feature, the slowing down of the rate of deforestation in 2011 probably as a result of increased enforcement by the state government, is very clearly discernible.

The successful application of the deep learning technology in Mato Grosso has encouraged astraea to aim at applying this approach globally. They also intend to use satellite data with higher resolution and to handle seasonal differences better.

About STEP

The System for Terrestrial Ecosystem Parameterization (STEP) is a model for deriving vegetation and land surface parameters from remote sensing data for use in remote sensing-based classification of land cover, ecosystems, and vegetation types. The model defines parameters that relate to important ecological and biogeophysical parameters and that can be reliably measured or inferred from remote sensing, collateral, and field plot data. STEP is maintained as a database of training polygons drawn on high spatial resolution imagery that can be extracted with GIS to produce a global land cover classification. STEP is periodically reviewed to filter out inconsistent sites and augmented to fill gaps in biogeographical coverage. The database was originally created to follow the International Geosphere-Biosphere Programme (IGBP) land cover legend but it has since evolved to support any number of additional classifications.

Low cost sensor enables citizen scientists to measure and share urban air pollution data over the web

The World Health Organization (WHO) estimates that more than seven million deaths every year are linked to air pollution exposure from household and ambient (outdoor) air pollution. But in spite of these statistics the average city has only one or two air quality monitoring devices which are very expensive costing about $1.5 million each. At this year's FOSS4GNA (Free and Open Source Software for Geospatial North America) get together in St Louis Steve Liang, a professor at the University of Calagary and CEO of SensorUp, described a way that citizen scientists can contribute to measuring air pollution using a low cost board and CPU and share via open source geospatial web software to map the resulting measured air pollution in real time.

Based on the latest results from WHO air pollution is now the world’s largest single environmental health risk, linked to 12% of all global deaths. Around 4.3 million deaths every year are attributed to exposure to household (indoor) air pollution, from heating, cooking and lighting using solid fuels. Around 3.7 million deaths every year are linked to outdoor air pollution , including exposure to fine particulate matter from fuel combustion from vehicles and from power plants, industry and biomass burning.

Steve described the sensors, which are typically built by interested people in workshops led by staff from SensorUp, a startup that Steve leads. The devices consist of a sensor that measures PM2.5 (particles less than 2.5 microns in size), temperature and humidity, a CPU and Wifi. Each device costs less than $100. After building the device, each participant in the workshop brings it home and installs it outside within wifi range of their internet router. The sensor reports ambient PM2.5, temperature and humidity every 5 minutes to a central server maintained by SensorUp. It uses the Open Geospatial Consortium (OGC) standard SensorThings API which provides an open and unified way to interconnect Internet of Things (IoT) devices, data, and applications over the Web. SensorUp SensorThings platform is the most advanced OGC SensorThings API implementation.

A web application built on the open source geospatial software Leaflet allows users to view the data on a map in real time, investigate air pollution historically and compare different cities. The data is open and accessible to anyone.  To date about 500 people across Canada have built their own devices and are sharing measured PM2.5. The real-time feed from these devices is mapped here.

Open developer-friendly standard for querying satellite imagery

One of the things that is required to make the vast quantity of satellite imagery easily searchable is a common way to query satellite data.  The SpatioTemporal Asset Catalog, known as STAC, is an open specification that came about when fourteen different organizations came together to increase the interoperability of searching for satellite imagery. At this year's FOSS4GNA (Free and Open Source Software for Geospatial North America) get together in St Louis, Matt Hanson, of developmentSEED, gave a technical overview of the STAC standard and described one of the first implementations.  The context for Matt's STAC presentation was provided by Chris Holmes' keynote in the morning.

Chris Holmes and others have been working on a standard for searching satellite imagery. Currently when a user wants to search for all the imagery in their area and time of interest they can’t make just one search — they have to use different tools and connect to API’s that are similar but all slightly different. The STAC specification aims to make that much easier, by providing common metadata and API mechanics to search and access geospatial data. This and other standards are essential for opening up satellite data to processing and visualization.  This is one of a new breed of standards that are designed to be developer-friendly to encourage the open source community and others to get involved with the Open Geospatial Consortium (OGC) to further interoperability.

The STAC standard is minimalist, but balances that with extensibility.  Basically it is comprised of metadata, catalog, and API.  The core of STAC metadata is very simple, with only three mandatory fields; ID, geometry (for example, a bounding box in a defined projection), and date and time.  The standard supports extensions, of which the earth observation standard "eo" has been defined at this time.  Future extensions could include point cloud, mosaic and video extensions.  The eo extension includes parameters such as camera viewing angle, resolution (dist between pixels), percent cloud cover, and band descriptions such as RGB, SWIR, band frequency and band accuracy.

The STAC standard and cloud optimized GeoTiff (COG) files makes it possible to search and stream imagery. COGs are GeoTiff files optimized for the cloud, for example, by streaming them from Amazon Web Services S3.  Planet, Google Earth, and QGIS already support COGs and there are other open source tools such as COG-Explorer that support them as well.

Sat-api is developmentSEED's partial implementation of STAC.  It currently supports Landsat-8 and Sentinel-2 imagery.  The API allows these imagery libraries to be queried by any of the metadata fields such as date and time, location (by defining a polygon), bands, viewing angle, and so on. It returns GeoJSON.  Matt provided some example queries.

 

Find all Landsat-8 scenes from 2017 with 0-20% cloud cover

https://sat-api.developmentseed.org/search/stac?datetime=2017&collection=landsat-8&eo:cloud_cover=0/20

Find all scenes from Dec 31, 2016 through Jan  1, 2018

https://sat-api.developmentseed.org/search/stac?datetime=2016-12-31/2018-01-01

The next developments that developmentSEED is planning for the STAC standard is to add more data CBERS and MODIS and to develop some applications based on the API.

Enabling the rest of us to find actionable information in huge volumes of satellite data

There is a huge volume of satellite imagery that is being captured and has been captured over the past couple of decades.  The challenge is to automatically find objects such as ships, trees, buildings, or containers in this vast amount if data and track changes in them over time.  This morning at at this year's FOSS4GNA (Free and Open Source Software for Geospatial North America) get together in St Louis, Chris Holmes of Planet Labs, gave an insightful overview of the application of deep learning to geospatial data, identifying opportunities for the open source community to lead in the application of this data and technology to addressing this challenge.

Outside of Google, Planet Labs, DigitalGlobe and some others deep learning has been applied primarily to recognizing people in selfies and dogs and cats from peoples photos on online repositories such as Flickr. I have blogged previously about competitions DigitalGlobe (which has 100 petabytes of imagery) and Nvidia have hosted into automating building footprint and road network recognition. Planet Labs has a computer vision and analytics team that has been applying the results of academic research to geospatial data, primarily imagery from Planet's constellation of 150+ satellites. For example, they have used deep learning to detect building footprints and road networks in Dar es Salaam. Chris showed examples of airplane, ship and container recognition achieved using deep learning. But these are research pilots. One company has applied deep learning to scale for a practical application. Google has been developing and applying the technology to building recognition - automatically identifying building footprints and features such as aerials. The deep learning algorithms have been developed by academia and as a result the code for the most part is open source.  For example, a  deep neural network model developed originally for medical image segmentation called U-Net  has been applied to identifying building footprints.  Successful training of deep networks requires many thousand labeled training samples and this training data is not open source. Labeled data involves people on the ground manually ground-truthing building footprints and other features so that the deep learning algorithms can learn what to recognize.

Chris believed that it is time to open these models to a broader group. There are some technical things that need to happen to make this possible. We need new formats for exchanging raster and vector data. Cloud optimized GeoTiff (COG) makes it possible to stream imagery from cloud sources such as Amazon S3. Planet, Google Earth, and QGIS already support COGs and there are other open source tools such as COG-Explorer that support them as well. Chris and others are working on WFS 3.0, a completely new developer-friendly version of an existing OGC (Open Geospatial Consortium) standard for exchanging vector data over the web.

Another technical advance that is required is a common way to query satellite data based on date, time, location, type of bands, and so on.  Chris and others have been working on a standard for searching satellite imagery for objects. The SpatioTemporal Asset Catalog, known as STAC, is an open specification that came about when fourteen different organizations came together to increase the interoperability of searching for satellite imagery. Currently when a user wants to search for all the imagery in their area and time of interest they can’t make just one search — they have to use different tools and connect to API’s that are similar but all slightly different. The STAC specification aims to make that much easier, by providing common metadata and API mechanics to search and access geospatial data.  Chris emphasized that standards are essential for opening up satellite data to processing and visualization and encouraged the open source community to get involved with the OGC to further developer-friendly standards.

Currently in order to use imagery data, a lot of data preparation is required of the end user such as lining up pixels, assigning a projection, removing clouds and mist, and other corrections. We need to move toward analysis ready data where data derived from different sources can be combined for analysis.

Because of the huge volumes of computational intensity we need cloud-native engines to perform the analyses. Google, DigitalGlobe (Maxar), Planet have these deep learning engines, but Chris encouraged the open source community to get more involved in developing these engines.

An essential piece that is required to open up the data and technology to a broader audience is open labeled data, Chris referred to it as a "open labeled data commons". There are analogies with OpenStreetMap initiative. Chris suggested that something like this is required for labeled data - it needs a repository of open data, tools to capture and manage the data, and a community around this to capture and curate that data.

The other key piece is converting this data into actionable information.  For example, dashboards that would alert municipal governments to un-permitted development, forestry enforcement to illegal forest cutting, or commodity price monitoring agencies to changes into the flow of ships into harbours or to large changes in containers at container ports.

Taken together Chris suggestions and work underway represent a roadmap for the open source geospatial community to lead in making the huge volumes of satellite imagery and other data queryable and able to generate actionable information.

How much does it cost to run a FOSS4G conference ?

Since the formation of the Open Source Geospatial Foundation (OSGEO) in 2006, there has been an annual FOSS4G event that brings together open source geospatial developers, users and supporters from around the world. The first event in Lausanne, Switzerland attracted 560 participants and last year's event in Boston, USA brought together over 1,100. The first FOSS4G event I attended was in 2007 in Victoria, British Columbia and what I remember most was the energy and collaborative spirit among all the participants. These are all volunteer initiated and run events and usually generate a positive financial return.

There are also regional FOSS4G events around the world. At this year's FOSS4GNA (FOSS4G North America) get together in St Louis, Michael Terner, who helped organize the Boston FOSS4G event, gave an overview of what is involved in organizing a FOSS4G event including a financial breakdown of the revenue and expenses for the Boston meeting.

This year's FOSS4G meetup will be held in Dar es Salaam, Tanzania, the second time the event has been held in Africa. The venue for the 2019 meeting has been decided - Bucharest, Romania. If you are interested in staging this event, Michael's breakdown of the revenue and costs will be very helpful in budgeting for such an event.

The Boston event generated about $1 million in revenue primarily from registrations, but 25% of the total was provided by sponsors. The space for the event was provided by Harvard University. It employed a professional conference organizer, but a lot of the work in making the event happen including finding sponsors came from volunteers. OSGEO provided about $70,000 as an advance to help get things started. The single biggest cost, nearly half a million dollars, was the venue and food. The next biggest costs were the social events and the audio-visual plus recording, each of which cost over a hundred thousand dollars. The Boston event generated about a $140,000 surplus for OSGEO.

Extensive use of geospatial open source software by the National Geospatial Intelligence Agency

The National Geospatial-Intelligence Agency (NGA) is a U.S. Government intelligence agency and Department of Defence combat support agency. But it also provides support for disaster relief and humanitarian aid. At FOSS4GNA today in St. Louis I found out that NGA is the largest consumer of geospatial data in the U.S. and the first intelligence agency to contribute to Github.

Mark Munsell of NGA gave a very interesting keynote at FOSS4GNA this afternoon. Until a decade ago NGA did not use open source software. But NGA found that relying on proprietary software left it lagging behind. About 10 years go NGA decided to begin using open source software. It acts as a user of and contributor to open source code and as a supporter of the geospatial open source community. Boundless, Postgres, OSSIM, MapStory, OpenSensorHub, geomesa, GeoTrellis and others have all been supported by NGA. Internally NGA provides geospatial services to other intel services as well as the Department of Defence. NGA has implemented a Data Corps consisting of data scientists and specialists that can work on specific problems in any intel agency.

In the mid 1990s the U.S. government decided to outsource most development work and laid off most the the coders working in the intelligence community. They subsequently realized that they had lost an incredible wealth of expertise and knowledge as a result of that policy decision. Under the name of Dev Corps NGA now wants to hire developers, especially from the geospatial open source development community Mark emphasized.  Mark finished off with an impassioned plea for geospatial open source developers to continue contributing as they have been, but if they are interested in working for an agency that does do humanitarian and disaster relief in addition to intel and defence, he wants to talk to them.

Deep learning enables automated extraction of building footprints and road networks from satellite imagery

Automated feature extraction from satellite imagery has made major progress in the last year. Accurate building footprints extracted from high resolution satellite imagery are available from companies such as Ecopia (which has just announced a partnership with DigitalGlobe).  Also NVIDIA has demonstrated the ability to automate detection of many road networks using sophisticated algorithms and multi-spectral high resolution imagery.

Building footprints

SpaceNet is a repository commercial satellite imagery and labeled training data being made available through Amazon Web Services at no cost to the public to foster innovation in the development of computer vision algorithms to automatically extract information from remote sensing data.  It is the result of a collaboration between DigitalGlobe, CosmiQ Works, and NVIDIA.  Two competitions for automated building footprint extraction from high resolution satellite imagery and a third for road network extraction have been held based on SpaceNet data. 

The first SpaceNet Challenge was launched in November 2016.  Imagery of Rio de Janeiro from the WorldView 2 satellite at 50cm resolution using eight spectral bands was made publicly available on Amazon Web Services (AWS).  42 developers competed to create algorithms that extract building footprints from the imagery.  The source code of the winning algorithms was made available on the SpaceNetChallenge GitHub repository. 

The winning implementation was developed by Brazilian developer using an implementation based on random forests with brute force polygon search and did not leverage deep learning frameworks.  The organizers' conclusion based on the results of Round 1 was that automated building footprint extraction remained a challenge.

Round 2 winner. Most building footprints are precisely detected. Number on each building footprint represents the overlap between the prediction and the ground truth.

Round 2 of the SpaceNet challenge involved higher resolution 30 cm imagery from WorldView 3 for four cities across the globe. The competition challenged developers to improve performance from the first competition using the higher-resolution imagery and more geographically diverse training data samples. SpaceNet on AWS held imagery and building footprints for Las Vegas, Paris, Shanghai, and Khartoum with over 180,000 buildings181,619 footprints.  In addition additional bands were made available (panchromatic, 3-band pan-sharpened, and 8-band pan-sharpened).

Results from Round 2 winning solution: top left: Las Vegas, top right: Las Vegas, lower right: Khartoum, and lower left: Paris. Blue outlines: ground truth, green outlines: correct identifications, red outlines: false positives and yellow outlines: false negatives.

The winner of Round 2 applied a  deep neural network model developed originally for medical image segmentation called U-Net.  Successful training of deep networks requires many thousand annotated training samples.  Training was aided significantly by augmenting the imagery with OpenStreetMap data.  The winning solution used OpenStreetMap layers and Worldview-2 multispectral layers as the input of the deep neural network algorithm.  In Las Vegas, the winning solution was outstanding in outlining suburban homes, but also did well in identifying odd-shaped buildings. In Khartoum, the solution was very good in identifying the footprints of stand-alone apartment complexes, but had trouble with small buildings, with building footprints that are close together, and with horizontally large buildings.  One of the problems was inconsistent ground truth which affected training. The solutions offered by the developers in the competition were sufficiently successful that the organizers concluded that the winning algorithms achieved performance with potential for automated mapping tasks such as keeping maps up-to-date and to assist first responders during natural disasters.  The source code for the winning implementations can be found in the SpaceNetChallenge GitHub repository.

DigitalGlobe has just announced a partnership with Ecopia, which has established an automated process to create building footprints quickly and at scale by leveraging DigitalGlobe's Geospatial Big Data platform (GBDX) and advanced machine learning in combination with DigitalGlobe’s imagery library. The two companies plan to automatically extract accurate 2D building footprints globally, then refresh the datasets periodically to find and track changes over time.  These datasets would be valuable to municipal governments for permitting purposes and for first responders after disasters.   Ecopia is developing a database for all of Australia (7.6 million square kilometers) which will include every building with a roof area greater than 9 square meters.  The database will contain building footprints for about 15 million buildings. The database includes linkages to other datasets including geocoded address, property data and administrative boundaries. Ecopia intend to update the dataset regularly to be able to track changes, especially urban sprawl.

Road networks

In November 2017 CosmiQ Works, Radiant Solutions, and NVIDIA announced a third round competition, “Road Detection and Routing Challenge” to explore automated methods for extracting map-ready road networks and routing information from high-resolution satellite imagery.  

Detecting road networks in Paris. Photo image, ground truth, and the results achieved with three deep learning algorithms

NVIDIA has released the results of several deep learning algorithms that illustrate just how difficult identifying road networks is and the sophistication of the available tools.  These include using multi-band spectral imagery to identify the material properties of the road surface itself (asphalt, gravel, packed earth) and using more sophisticated processing including conditional random fields (CRF), percolation theory, and reinforcement learning. 

Detecting road networks in Las Vegas. Photo image, ground truth, and the results achieved with three deep learning algorithms

NVIDIA utilized high performance GPU compute resources provided by the NVIDIA GPU Cloud and Amazon Web Services.

Open source viewer sharing all city's geospatial data can run on Raspberry Pi

Locations of Cross Section surveyors' notebooks

At GITA 2018 in Phoenix, Bob Basques, GIS Systems Developer at the City of St Paul, described a system called COMPASS he and his team have developed that provides a shared, easy to use tool that allows city employees and the public access to all of the City's spatial and associated data including, for example, scans of surveyors' notebooks, 2.2 million street level photos, and permitting and licensing information from 200 different applications. Based completely on open source components the system is compact and efficient enough to run on a Raspberry Pi.

One of the problems that hampers efficient operations at municipalities is accessing geospatial data originating from multiple sources such as infrastructure maintenance, planning and zoning, property ownership, engineering, permitting, licensing and code enforcement. Spatial data is generated by CAD drafters, GIS users, surveyors and even users with smart phone apps. Imagery data can come from a variety of sources including earth observation satellites, aerial overflights, street photography, and drones. This data is constantly changing which means that any process that involves making copies such as converting it to a common format creates a bigger problem than it solves.

Location and maintenance data for each street light

The City of Saint Paul is responsible for infrastructure maintenance for all of the infrastructure within the public right-of-way. It is also responsible for maintaining parcel information linking and addressing assignment. St Paul is only 56 square miles, but it is densely developed and there is a lot of infrastructure and the associated regulations to manage. In addition St Paul is part of the metro area of the Twin Cities and users of St Paul data often need access to data outside of St Paul.

Organizations participating in GeoMoose project

The application is intended to improve the efficiency of common tasks including infrastructure maintenance management, planning and zoning, property ownership, engineering, permitting. licensing and code enforcement. To do this the system accesses multiple disparate data sources from different data custodians including Public Works, Safety and Inspections, Planning and Economic Development, Parks & Recreation, Office of Technology, Fire, Police, Real Estate, and District Councils. The public has access to 73 layers of data through OGC web standard interfaces. The tool, called GeoMoose, is an open source project (MIT license) on OSGeo and uses only open source components. There are quite a few governmental agencies around the U.S participating in the GeoMoose project.

Creating a project polygon and buffer to identify affected parcels and owners

The application is extremely compact and light weight - Bob demonstrated it running an a Raspberry Pi with a 64 GB drive (just about any size MicroSD card will work) - which means that users working in areas with restricted wireless can still access important data. GeoMoose runs on Linux and Windoes and can be accessed via Wifi from Windows, Linux, Android, iOS and Mac OS - actually any device with Wifi and a browser including a USB stick. The core of GeoMoose is comprised of HTML and Javascript code. It can access data stored in the file system as well as from OGC WMS/WFS sites over the internet. GeoMoose included an editor that allows users to create geometric objects such as polygons. A typical application would be to define a polygon representing a new project, put a buffer around it and find all the parcels that would be impacted by the project.

The system handles many different data types including 305 layers ( a new layer is added every week or two), 32 live database connections, 39 imagery layers, and 31 infrastructure layers. Besides Public Works users, users from more than 40 other organizations access the system. The system is an OSGeo project and is built on open source components including OpenLayers and MapServer, and can be integrated with PostgreSQL/PostGIS, Webdav, svn and Apache.

Locations of surveyors' notebooks

COMPASS exposes data to the world via OGC standard tools. Most of the data in the COMPASS system of tools is available as a WMS service. Selected datasets are also available as WFS layers. Some layers are protected for security or contractual reasons. GeoMoose 3.0, which was released in August of last year is mobile friendly and supports modern Javascript frameworks such as React.

Digital images of surveyors' notebooks can be accessed

The system has become widely used and trusted. It gets about 50,000 hits per week not only within Public Works but also by other users inside and outside of the city government. It has eased many previously labour intensive processes. Because more users are accessing their data, custodians of the data are taking greater responsibility for data which has led to an overall improvement in data quality and timeliness. Open source has allowed customization for specific users that would not have been possible without access to the underlying code. For example, it has made it possible to create specialized visualizations and reports, connecting to various proprietary databases, mixing data sources on output and building custom lookup processes.

Open source-based mobile viewer creates value for electric power utilities with multiple GISs

One of the challenges faced by large utilities with multiple GIS, outage management, customer information and other systems, which are often the result of mergers, is how to make spatial and associated data, in different, often incompatible systems, accessible to all users not just GIS professionals.  At Distributech 2018 Duke Energy described an off-the-shelf web-based product based on open source components that is being used by 8000 users, 2000 of them using it 24/7 every day, to access GIS and associated data in several GIS systems, both online and offline, using laptops, tablets and phones.  An important advantage of the the selection process Duke used in selecting the product was that it made it possible for a relative newcomer to compete successfully with the traditional GIS incumbents.

Background

Large investor-owned utilities are often the result of mergers. For example, in 2012 Duke Energy and Progress Energy merged to create the largest U.S. utility with 7.4 million U.S. customers and approximately 52,700 megawatts of electric generating capacity. The challenge for the IT folks is how to manage the many disparate IT systems that such mergers bring together. Historically Duke Energy and Progress Energy have employed all the major GIS systems including Synercom, Intergraph FRAMME, GE Smallworld, ESRI, Intergraph GTechnology, and IBM GFIS. Accessing the valuable operational data in multiple systems is a challenge because each system has its own user interface and command structure. Over the years I have seen a number of solutions that have attempted to address the challenge of providing a common viewer for visualizing data from multiple data sources. But each has had limitations, for example, all data had to be synced to a central Oracle database, or it could only run online making it difficult to deploy to the field, or it had limited support for the many GISs and user files used by utilities, or it supported iOS on handheld devices but not Android, or did not support open standards such as OGC WMS and WFS, or did not support Google Maps, or was so complex users required extensive training and it was like pulling teeth to geth them to use iit.

At Distributech 2018 Ron York, Lead Consultant/ GIS, and Francisco Sarmento, Lead Consultant, both at Duke Energy described how Duke has resolved the problem of a common viewer for several different GIS systems which was brought to a head by the merger of Duke and Progress. They also described the somewhat unusual process by which Duke selected the vendor whose product provided the successful solution.

Open source geospatial components

A key differentiator of the product Duke chose is that it is leverages open source components; PostGIS, GeoServer and Leaflet being the main ones. The product builds on the collective wisdom, code and support provided by the worldwide open source geospatial community.  Leaflet is the leading open-source JavaScript library for mobile-friendly interactive maps. PostGIS adds spatial capabilities to the PostgreSQL relational database. It extends PostgreSQL so it can store, query, and manipulate spatial data. GeoServer is a web server that allows maps and data in a variety of formats to be served to standard clients such as web browsers and desktop GIS programs. Data is published via Open Geospatial Consortium (OGC) standard interfaces, such as WMS and WFS.  These are all Open Source Geospatial Foundation (OSGEO) projects.

Multiple GIS support

The product Duke selected supports ESRI ArcGIS, GE Smallworld, Intergraph GTechnology, spatialNET and Google Maps. At Duke it is used to access data in a couple of GISs as well as non-GIS data in Maximo, SAP, three different outage management systems, four or five customer information systems, and can link to CAD drawings and photos and images.  Importantly for engineering users it preserves the original symbology and background maps of the different GIS systems.

Online and offline

Another key advantage of Duke's solution is that it can run equally well offline and online and is able to seamlessly combine and switch between online and offline data.

Mobile and office devices

It supports mobile as well as office devices, running on laptops, tablets and phones running iOS, Android, and Windows. The user interface adapts automatically for each device, displaying map layers, forms and other data in a way that is appropriate for each device.

Synced or direct access to databases

It provides two ways of accessing data; one is to periodically import or sync data into the system's spatial database, which uses PostGIS, and the other is to directly access the data via a web service. It supports ESRI REST Services, which are used by ArcGIS Online and ArcGIS Enterprise, and the industry standard OGC WMS and WFS web services, which are supported by all major geospatial vendors.

It does not support direct edit of spatial data, but does allow a GIS data correction requests to be routed through Maximo following Duke standard business procedure for making corrections.

It is flexible enough to support different business processes for collecting and organizing data. For example, pole inspections are conducted by circuit in what was Progress service territories and by square area in what was Duke's service territory.

Simplicity of use

Another of the key differentiators of Duke's web-based solution is that it is not targeted on GIS professionals, but on the 95% of people within Duke who can get a lot of value from a really simple Google Maps style interface onto their enterprise data.

Leading up to and during deployment Duke put on road shows, made web demos available online, utilized computer-based-training, and provided job-aid documents to make people aware of the new system.  The tool caught on surprisingly rapidly. Within five months of it being deployed there were 5000 users. Currently there are 8000 users with over 2000 users accessing it 24/7 every day.

Anticipated and unanticipated benefits

The major expected benefit that was realized by the system is that eases the integration of Duke and Progess. All GIS asset data is exposed through a single tool to designers and asset engineers. But there were some unanticipated benefits as well.  It has had a direct impact on tax calculation - it shortened the cycle for internal auditors to complete tax assessments.  It is also widely used during storms and other natural disasters. It enables crews from different area to use a familiar tool to analyze unfamiliar data in areas where they haven't been before.  It is used to provide driving directions for field crews.  Storm managers overlay outages, assets, flood plains, wind, rain, traffic  and other maps for situational assessment.  It allows manager to share staging sites during storms. It has also  proved extremely useful in locating "unmatched outages" - outages called in by first responders with just an address.

Process for choosing a vendor levels playing field for new entrants

Duke used an interesting process to select the vendor. Preliminary vendor interviews were conducted to exclude vendors that clearly didn't have a solution that could meet Duke's needs. Remote web demos provided a cheap and easy way to narrow down the list of vendors. A request for proposal (RFP) process was used to solicit technical-only proposals from product vendors. (Duke was only interested in off-the-shelf products not bespoke solutions.)  Duke picked the top five and asked them to come to Duke for a scripted demonstration. Instead of asking the vendors for a benchmark demonstrating how their product would address Duke's requirements using Duke data, they asked the vendors to demonstrate their product using their own data. They picked the top two vendors and began contract negotiations to decide on one vendor which turned out to be Ubisense MyWorld, a relatively new entrant in the geospatial world.  An important advantage of the selection process Duke used is that it made it possible for a relative newcomer to compete successfully with traditional GIS incumbents.

Avoiding a repeat of the CAD/GIS quaqmire of the past quarter century

Yesterday at Distributech 2018 in San Antonio I listened to a talk from a consulting company contracted by an electric power utility to clean up GIS data originally sourced from CAD drawings.  This reminded me once again that is a challenge that telecommunications and cable companies and electric and water utilities have been attempting to deal with at least since the early 1990s and for some even before that.  Telecommunications and cable and leading electric power utilities resolved this problem at least a decade ago, but for some electric power and even more water utilities it remains a problem that is going to hold these utilities back when they should be accelerating their adoption of digital technologies.

I began working in the utility and telecom sector in 1993, specifically in developing software for records management (called network documentation outside of North America), and at that time the magnitude of the AEC+geospatial cultural mismatch problem was just beginning to be appreciated by utilities and telecoms. In utilities and telecoms planners tend to use GIS tools, engineers and designers CAD tools, construction folks in the field paper CAD drawings, and asset managers GIS integrated with their FM tools. Fashioning AEC and geospatial data into an efficient data flow from planning through design and construction to operations and maintenance represented a challenge that remains a problem for utilities. The Between The Poles blog is over ten years old and one of the persistent themes from the very beginning in 2006 was the challenge of integrating CAD and GIS data in a common workflow (some examples; 2006, 2007, 2008 ).

A workflow that is found at every utility and telecom involves the design-build-maintain information flow. The workflow begins with an engineer and drafter producing a design, typically a CAD drawing, which is printed for the construction contractor to use in building a facility. The contractor uses the drawing in the field and then returns it as an "as-built" to the utility's records management group, who enter it, traditionally by manually redigitizing it from the paper drawing, into a GIS that is intended to contain the digital twin of the network. The data in the GIS is used for network maintenance, outage management and other functions. The reason for the manual redigitization step is the "errors" that CAD drawings inevitably contain unless drafters are forced to adhere to "geospatial rules". This entire process is slow and error-prone and backlogs of as-builts waiting to be entered into the GIS stretch into months and even years. The result is that utility GIS data is more often than not incomplete, out-of-date, and unreliable. For example, at the beginning of the broadband revolution, telecoms needed to be able to tell customers whether they were close enough to a terminal and fiber to be eligible for ADSL. In one of the largest telecoms in the U.S. the GIS records were so incomplete and unreliable that survey teams had to be sent out at considerable cost to resurvey the location of their terminals and fiber cables.

Because there was no standards-based solution to the CAD+GIS problem, every utility and telecom has had to try to find its own solution to the problem. This problem has persisted from the 1990s and remains a problem for utilities - as I said I heard a presentation yesterday that wasn't that different from presentations on the same topic I and others gave a quarter of a century ago.

Over the years the poor CAD/GIS interoperability has cost the utility and telecom sectors considerable amounts of money. Large telecoms have spent hundreds of millions of dollars trying to cleanup their GIS data and streamline their CAD+GIS workflows. A rough back-of-the-envelope calculation suggests this problem has cost directly and indirectly trillions of dollars over the past 20 years. In addition unreliable and out-of-date network infrastructure data that results from this problem has safety implications.  

The juncture between distinct sectors each with its own technologies and business models, AEC (Autodesk, Bentley, and others) and geospatial (ESRI, Hexagon, and others), has analogies with the clash of tectonic plates and the associated incessant seismic events. In the mid-1990s pressure from customers motivated Autodesk and ESRI, 800 lb gorillas in their respective areas of CAD and GIS, to attempt to collaborate to address the CAD+GIS problem. The result of the agreement between Jack Dangermond and Carol Bartz at the time resulted in an unsuccessful product called ArcCAD, which satisfied neither CAD nor GIS users.

Averting a tectonic clash repeat: BIM+geospatial interoperability

The prospect that, by analogy with CAD+GIS problem, we could still be individually finding solutions to BIM/geospatial integration 20 years from now is discouraging. While there are parallels between CAD/GIS and BIM/geospatial, there are grounds for optimism that this time around we may find ways the address the problem in a general way. Firstly, in the mid 1990s there were virtually no geospatial or 3D AEC data standards. The Open Geospatial Consortium (OGC) and builsingSmart were both only founded in 1994. It required quite a while for the OGC geospatial and buildingSmart's IFC standards to broadly penetrate industry and government.  Nowadays, there are many widely adopted geospatial standards; Simple Features for SQL, WMS, WFS, GML, KML, InfraGML, CityGML to name just a few from the OGC. buildingSmart's IFC standard is supported by all major BIM design tools including the market leader Revit.

Perhaps most significantly there has been significant progress in developing common conceptual models for geospatial and AEC views of city infrastructure. The major breakthrough in bringing the architectural and geospatial views onto a common footing is the OGC LandInfra Conceptual Model developed by the OGC in cooperation with buildingSMART International and approved as an OGC standard in August, 2016. The LandInfra conceptual model was developed by Bentley Systems, Leica Geosystems, Trimble, Australian Government Department of Communications, Autodesk, Vianova Systems AS, and buildingSMART International and provides a unifying basis for land and civil engineering standards including the OGC's InfraGML and buildingSmart International's IFC for infrastructure standards.  InfraGML is the OGC's application schema supporting land development and civil engineering infrastructure facilities. InfraGML supports a subset of the existing LandXML standard. buildingSmart International's IFC-Alignment project uses this common conceptual model of alignments for roads, railways, tunnels and bridges. The objectives of the IFC-Alignment project is to enable the exchange of alignment information through the full infrastructure lifecycle from planning through design and construction to asset management.

Secondly, in the 1990s and the early 2000s there were few open source geospatial projects. The Open Source Geospatial Foundation (OSGEO) was only founded in 2006. Since then there has been rapid development of open source geospatial projects. There is enough open source geospatial code now that at the GEO|Design+BIM conference in Amsterdam, Sanghee Shin described a web-based geo-platform integrating massive, complex BIM models using open source geospatial libraries and tools including PostGIS, GeoServer, and Cesium and NASA WorldWind virtual globes. The importance of this project is that it shows that it is possible using the available open source geospatial APIs and libraries - together with some genuine innovative development - to create an open, non-proprietary, 3D geospatial platform for integrating geospatial and BIM. Given the critical importance of addressing the cultural and technical divide between the AEC and geospatial worlds, clearly many believe that a viable open source alternative will be essential for developing the innovative solutions to the challenge of interfacing the two worlds. 

Finally, major players in the AEC and geospatial industries, Autodesk and ESRI, have again agreed to collaborate. This time around they are probably more aware of the technical issues hindering BIM+geospatial integration. Both have adopted open standards when it made business sense - often under pressure from government clients - and both have also adopted open source technology where it made business sense. Together the combination of major players supporting the effort, standards organization like the OGC and buildingSmart already working to address the problem, and a broad range of open source code and a vibrant developer community may be enough to address the technical problems. If building and infrastructure owners and facilities managers including government as well as regulators become aware of the risk of not solving the problem, a solution may be found so that we will not be having the same BIM+geospatial integration discussion a quarter of a century from now.