Between the Poles

Enabling the rest of us to find actionable information in huge volumes of satellite data

There is a huge volume of satellite imagery that is being captured and has been captured over the past couple of decades.  The challenge is to automatically find objects such as ships, trees, buildings, or containers in this vast amount if data and track changes in them over time.  This morning at at this year's FOSS4GNA (Free and Open Source Software for Geospatial North America) get together in St Louis, Chris Holmes of Planet Labs, gave an insightful overview of the application of deep learning to geospatial data, identifying opportunities for the open source community to lead in the application of this data and technology to addressing this challenge.

Outside of Google, Planet Labs, DigitalGlobe and some others deep learning has been applied primarily to recognizing people in selfies and dogs and cats from peoples photos on online repositories such as Flickr. I have blogged previously about competitions DigitalGlobe (which has 100 petabytes of imagery) and Nvidia have hosted into automating building footprint and road network recognition. Planet Labs has a computer vision and analytics team that has been applying the results of academic research to geospatial data, primarily imagery from Planet's constellation of 150+ satellites. For example, they have used deep learning to detect building footprints and road networks in Dar es Salaam. Chris showed examples of airplane, ship and container recognition achieved using deep learning. But these are research pilots. One company has applied deep learning to scale for a practical application. Google has been developing and applying the technology to building recognition - automatically identifying building footprints and features such as aerials. The deep learning algorithms have been developed by academia and as a result the code for the most part is open source.  For example, a  deep neural network model developed originally for medical image segmentation called U-Net  has been applied to identifying building footprints.  Successful training of deep networks requires many thousand labeled training samples and this training data is not open source. Labeled data involves people on the ground manually ground-truthing building footprints and other features so that the deep learning algorithms can learn what to recognize.

Chris believed that it is time to open these models to a broader group. There are some technical things that need to happen to make this possible. We need new formats for exchanging raster and vector data. Cloud optimized GeoTiff (COG) makes it possible to stream imagery from cloud sources such as Amazon S3. Planet, Google Earth, and QGIS already support COGs and there are other open source tools such as COG-Explorer that support them as well. Chris and others are working on WFS 3.0, a completely new developer-friendly version of an existing OGC (Open Geospatial Consortium) standard for exchanging vector data over the web.

Another technical advance that is required is a common way to query satellite data based on date, time, location, type of bands, and so on.  Chris and others have been working on a standard for searching satellite imagery for objects. The SpatioTemporal Asset Catalog, known as STAC, is an open specification that came about when fourteen different organizations came together to increase the interoperability of searching for satellite imagery. Currently when a user wants to search for all the imagery in their area and time of interest they can’t make just one search — they have to use different tools and connect to API’s that are similar but all slightly different. The STAC specification aims to make that much easier, by providing common metadata and API mechanics to search and access geospatial data.  Chris emphasized that standards are essential for opening up satellite data to processing and visualization and encouraged the open source community to get involved with the OGC to further developer-friendly standards.

Currently in order to use imagery data, a lot of data preparation is required of the end user such as lining up pixels, assigning a projection, removing clouds and mist, and other corrections. We need to move toward analysis ready data where data derived from different sources can be combined for analysis.

Because of the huge volumes of computational intensity we need cloud-native engines to perform the analyses. Google, DigitalGlobe (Maxar), Planet have these deep learning engines, but Chris encouraged the open source community to get more involved in developing these engines.

An essential piece that is required to open up the data and technology to a broader audience is open labeled data, Chris referred to it as a "open labeled data commons". There are analogies with OpenStreetMap initiative. Chris suggested that something like this is required for labeled data - it needs a repository of open data, tools to capture and manage the data, and a community around this to capture and curate that data.

The other key piece is converting this data into actionable information.  For example, dashboards that would alert municipal governments to un-permitted development, forestry enforcement to illegal forest cutting, or commodity price monitoring agencies to changes into the flow of ships into harbours or to large changes in containers at container ports.

Taken together Chris suggestions and work underway represent a roadmap for the open source geospatial community to lead in making the huge volumes of satellite imagery and other data queryable and able to generate actionable information.

Open source viewer sharing all city's geospatial data can run on Raspberry Pi

Locations of Cross Section surveyors' notebooks

At GITA 2018 in Phoenix, Bob Basques, GIS Systems Developer at the City of St Paul, described a system called COMPASS he and his team have developed that provides a shared, easy to use tool that allows city employees and the public access to all of the City's spatial and associated data including, for example, scans of surveyors' notebooks, 2.2 million street level photos, and permitting and licensing information from 200 different applications. Based completely on open source components the system is compact and efficient enough to run on a Raspberry Pi.

One of the problems that hampers efficient operations at municipalities is accessing geospatial data originating from multiple sources such as infrastructure maintenance, planning and zoning, property ownership, engineering, permitting, licensing and code enforcement. Spatial data is generated by CAD drafters, GIS users, surveyors and even users with smart phone apps. Imagery data can come from a variety of sources including earth observation satellites, aerial overflights, street photography, and drones. This data is constantly changing which means that any process that involves making copies such as converting it to a common format creates a bigger problem than it solves.

Location and maintenance data for each street light

The City of Saint Paul is responsible for infrastructure maintenance for all of the infrastructure within the public right-of-way. It is also responsible for maintaining parcel information linking and addressing assignment. St Paul is only 56 square miles, but it is densely developed and there is a lot of infrastructure and the associated regulations to manage. In addition St Paul is part of the metro area of the Twin Cities and users of St Paul data often need access to data outside of St Paul.

Organizations participating in GeoMoose project

The application is intended to improve the efficiency of common tasks including infrastructure maintenance management, planning and zoning, property ownership, engineering, permitting. licensing and code enforcement. To do this the system accesses multiple disparate data sources from different data custodians including Public Works, Safety and Inspections, Planning and Economic Development, Parks & Recreation, Office of Technology, Fire, Police, Real Estate, and District Councils. The public has access to 73 layers of data through OGC web standard interfaces. The tool, called GeoMoose, is an open source project (MIT license) on OSGeo and uses only open source components. There are quite a few governmental agencies around the U.S participating in the GeoMoose project.

Creating a project polygon and buffer to identify affected parcels and owners

The application is extremely compact and light weight - Bob demonstrated it running an a Raspberry Pi with a 64 GB drive (just about any size MicroSD card will work) - which means that users working in areas with restricted wireless can still access important data. GeoMoose runs on Linux and Windoes and can be accessed via Wifi from Windows, Linux, Android, iOS and Mac OS - actually any device with Wifi and a browser including a USB stick. The core of GeoMoose is comprised of HTML and Javascript code. It can access data stored in the file system as well as from OGC WMS/WFS sites over the internet. GeoMoose included an editor that allows users to create geometric objects such as polygons. A typical application would be to define a polygon representing a new project, put a buffer around it and find all the parcels that would be impacted by the project.

The system handles many different data types including 305 layers ( a new layer is added every week or two), 32 live database connections, 39 imagery layers, and 31 infrastructure layers. Besides Public Works users, users from more than 40 other organizations access the system. The system is an OSGeo project and is built on open source components including OpenLayers and MapServer, and can be integrated with PostgreSQL/PostGIS, Webdav, svn and Apache.

Locations of surveyors' notebooks

COMPASS exposes data to the world via OGC standard tools. Most of the data in the COMPASS system of tools is available as a WMS service. Selected datasets are also available as WFS layers. Some layers are protected for security or contractual reasons. GeoMoose 3.0, which was released in August of last year is mobile friendly and supports modern Javascript frameworks such as React.

Digital images of surveyors' notebooks can be accessed

The system has become widely used and trusted. It gets about 50,000 hits per week not only within Public Works but also by other users inside and outside of the city government. It has eased many previously labour intensive processes. Because more users are accessing their data, custodians of the data are taking greater responsibility for data which has led to an overall improvement in data quality and timeliness. Open source has allowed customization for specific users that would not have been possible without access to the underlying code. For example, it has made it possible to create specialized visualizations and reports, connecting to various proprietary databases, mixing data sources on output and building custom lookup processes.

Open source-based mobile viewer creates value for electric power utilities with multiple GISs

One of the challenges faced by large utilities with multiple GIS, outage management, customer information and other systems, which are often the result of mergers, is how to make spatial and associated data, in different, often incompatible systems, accessible to all users not just GIS professionals.  At Distributech 2018 Duke Energy described an off-the-shelf web-based product based on open source components that is being used by 8000 users, 2000 of them using it 24/7 every day, to access GIS and associated data in several GIS systems, both online and offline, using laptops, tablets and phones.  An important advantage of the the selection process Duke used in selecting the product was that it made it possible for a relative newcomer to compete successfully with the traditional GIS incumbents.

Background

Large investor-owned utilities are often the result of mergers. For example, in 2012 Duke Energy and Progress Energy merged to create the largest U.S. utility with 7.4 million U.S. customers and approximately 52,700 megawatts of electric generating capacity. The challenge for the IT folks is how to manage the many disparate IT systems that such mergers bring together. Historically Duke Energy and Progress Energy have employed all the major GIS systems including Synercom, Intergraph FRAMME, GE Smallworld, ESRI, Intergraph GTechnology, and IBM GFIS. Accessing the valuable operational data in multiple systems is a challenge because each system has its own user interface and command structure. Over the years I have seen a number of solutions that have attempted to address the challenge of providing a common viewer for visualizing data from multiple data sources. But each has had limitations, for example, all data had to be synced to a central Oracle database, or it could only run online making it difficult to deploy to the field, or it had limited support for the many GISs and user files used by utilities, or it supported iOS on handheld devices but not Android, or did not support open standards such as OGC WMS and WFS, or did not support Google Maps, or was so complex users required extensive training and it was like pulling teeth to geth them to use iit.

At Distributech 2018 Ron York, Lead Consultant/ GIS, and Francisco Sarmento, Lead Consultant, both at Duke Energy described how Duke has resolved the problem of a common viewer for several different GIS systems which was brought to a head by the merger of Duke and Progress. They also described the somewhat unusual process by which Duke selected the vendor whose product provided the successful solution.

Open source geospatial components

A key differentiator of the product Duke chose is that it is leverages open source components; PostGIS, GeoServer and Leaflet being the main ones. The product builds on the collective wisdom, code and support provided by the worldwide open source geospatial community.  Leaflet is the leading open-source JavaScript library for mobile-friendly interactive maps. PostGIS adds spatial capabilities to the PostgreSQL relational database. It extends PostgreSQL so it can store, query, and manipulate spatial data. GeoServer is a web server that allows maps and data in a variety of formats to be served to standard clients such as web browsers and desktop GIS programs. Data is published via Open Geospatial Consortium (OGC) standard interfaces, such as WMS and WFS.  These are all Open Source Geospatial Foundation (OSGEO) projects.

Multiple GIS support

The product Duke selected supports ESRI ArcGIS, GE Smallworld, Intergraph GTechnology, spatialNET and Google Maps. At Duke it is used to access data in a couple of GISs as well as non-GIS data in Maximo, SAP, three different outage management systems, four or five customer information systems, and can link to CAD drawings and photos and images.  Importantly for engineering users it preserves the original symbology and background maps of the different GIS systems.

Online and offline

Another key advantage of Duke's solution is that it can run equally well offline and online and is able to seamlessly combine and switch between online and offline data.

Mobile and office devices

It supports mobile as well as office devices, running on laptops, tablets and phones running iOS, Android, and Windows. The user interface adapts automatically for each device, displaying map layers, forms and other data in a way that is appropriate for each device.

Synced or direct access to databases

It provides two ways of accessing data; one is to periodically import or sync data into the system's spatial database, which uses PostGIS, and the other is to directly access the data via a web service. It supports ESRI REST Services, which are used by ArcGIS Online and ArcGIS Enterprise, and the industry standard OGC WMS and WFS web services, which are supported by all major geospatial vendors.

It does not support direct edit of spatial data, but does allow a GIS data correction requests to be routed through Maximo following Duke standard business procedure for making corrections.

It is flexible enough to support different business processes for collecting and organizing data. For example, pole inspections are conducted by circuit in what was Progress service territories and by square area in what was Duke's service territory.

Simplicity of use

Another of the key differentiators of Duke's web-based solution is that it is not targeted on GIS professionals, but on the 95% of people within Duke who can get a lot of value from a really simple Google Maps style interface onto their enterprise data.

Leading up to and during deployment Duke put on road shows, made web demos available online, utilized computer-based-training, and provided job-aid documents to make people aware of the new system.  The tool caught on surprisingly rapidly. Within five months of it being deployed there were 5000 users. Currently there are 8000 users with over 2000 users accessing it 24/7 every day.

Anticipated and unanticipated benefits

The major expected benefit that was realized by the system is that eases the integration of Duke and Progess. All GIS asset data is exposed through a single tool to designers and asset engineers. But there were some unanticipated benefits as well.  It has had a direct impact on tax calculation - it shortened the cycle for internal auditors to complete tax assessments.  It is also widely used during storms and other natural disasters. It enables crews from different area to use a familiar tool to analyze unfamiliar data in areas where they haven't been before.  It is used to provide driving directions for field crews.  Storm managers overlay outages, assets, flood plains, wind, rain, traffic  and other maps for situational assessment.  It allows manager to share staging sites during storms. It has also  proved extremely useful in locating "unmatched outages" - outages called in by first responders with just an address.

Process for choosing a vendor levels playing field for new entrants

Duke used an interesting process to select the vendor. Preliminary vendor interviews were conducted to exclude vendors that clearly didn't have a solution that could meet Duke's needs. Remote web demos provided a cheap and easy way to narrow down the list of vendors. A request for proposal (RFP) process was used to solicit technical-only proposals from product vendors. (Duke was only interested in off-the-shelf products not bespoke solutions.)  Duke picked the top five and asked them to come to Duke for a scripted demonstration. Instead of asking the vendors for a benchmark demonstrating how their product would address Duke's requirements using Duke data, they asked the vendors to demonstrate their product using their own data. They picked the top two vendors and began contract negotiations to decide on one vendor which turned out to be Ubisense MyWorld, a relatively new entrant in the geospatial world.  An important advantage of the selection process Duke used is that it made it possible for a relative newcomer to compete successfully with traditional GIS incumbents.

Avoiding a repeat of the CAD/GIS quaqmire of the past quarter century

Yesterday at Distributech 2018 in San Antonio I listened to a talk from a consulting company contracted by an electric power utility to clean up GIS data originally sourced from CAD drawings.  This reminded me once again that is a challenge that telecommunications and cable companies and electric and water utilities have been attempting to deal with at least since the early 1990s and for some even before that.  Telecommunications and cable and leading electric power utilities resolved this problem at least a decade ago, but for some electric power and even more water utilities it remains a problem that is going to hold these utilities back when they should be accelerating their adoption of digital technologies.

I began working in the utility and telecom sector in 1993, specifically in developing software for records management (called network documentation outside of North America), and at that time the magnitude of the AEC+geospatial cultural mismatch problem was just beginning to be appreciated by utilities and telecoms. In utilities and telecoms planners tend to use GIS tools, engineers and designers CAD tools, construction folks in the field paper CAD drawings, and asset managers GIS integrated with their FM tools. Fashioning AEC and geospatial data into an efficient data flow from planning through design and construction to operations and maintenance represented a challenge that remains a problem for utilities. The Between The Poles blog is over ten years old and one of the persistent themes from the very beginning in 2006 was the challenge of integrating CAD and GIS data in a common workflow (some examples; 2006, 2007, 2008 ).

A workflow that is found at every utility and telecom involves the design-build-maintain information flow. The workflow begins with an engineer and drafter producing a design, typically a CAD drawing, which is printed for the construction contractor to use in building a facility. The contractor uses the drawing in the field and then returns it as an "as-built" to the utility's records management group, who enter it, traditionally by manually redigitizing it from the paper drawing, into a GIS that is intended to contain the digital twin of the network. The data in the GIS is used for network maintenance, outage management and other functions. The reason for the manual redigitization step is the "errors" that CAD drawings inevitably contain unless drafters are forced to adhere to "geospatial rules". This entire process is slow and error-prone and backlogs of as-builts waiting to be entered into the GIS stretch into months and even years. The result is that utility GIS data is more often than not incomplete, out-of-date, and unreliable. For example, at the beginning of the broadband revolution, telecoms needed to be able to tell customers whether they were close enough to a terminal and fiber to be eligible for ADSL. In one of the largest telecoms in the U.S. the GIS records were so incomplete and unreliable that survey teams had to be sent out at considerable cost to resurvey the location of their terminals and fiber cables.

Because there was no standards-based solution to the CAD+GIS problem, every utility and telecom has had to try to find its own solution to the problem. This problem has persisted from the 1990s and remains a problem for utilities - as I said I heard a presentation yesterday that wasn't that different from presentations on the same topic I and others gave a quarter of a century ago.

Over the years the poor CAD/GIS interoperability has cost the utility and telecom sectors considerable amounts of money. Large telecoms have spent hundreds of millions of dollars trying to cleanup their GIS data and streamline their CAD+GIS workflows. A rough back-of-the-envelope calculation suggests this problem has cost directly and indirectly trillions of dollars over the past 20 years. In addition unreliable and out-of-date network infrastructure data that results from this problem has safety implications.  

The juncture between distinct sectors each with its own technologies and business models, AEC (Autodesk, Bentley, and others) and geospatial (ESRI, Hexagon, and others), has analogies with the clash of tectonic plates and the associated incessant seismic events. In the mid-1990s pressure from customers motivated Autodesk and ESRI, 800 lb gorillas in their respective areas of CAD and GIS, to attempt to collaborate to address the CAD+GIS problem. The result of the agreement between Jack Dangermond and Carol Bartz at the time resulted in an unsuccessful product called ArcCAD, which satisfied neither CAD nor GIS users.

Averting a tectonic clash repeat: BIM+geospatial interoperability

The prospect that, by analogy with CAD+GIS problem, we could still be individually finding solutions to BIM/geospatial integration 20 years from now is discouraging. While there are parallels between CAD/GIS and BIM/geospatial, there are grounds for optimism that this time around we may find ways the address the problem in a general way. Firstly, in the mid 1990s there were virtually no geospatial or 3D AEC data standards. The Open Geospatial Consortium (OGC) and builsingSmart were both only founded in 1994. It required quite a while for the OGC geospatial and buildingSmart's IFC standards to broadly penetrate industry and government.  Nowadays, there are many widely adopted geospatial standards; Simple Features for SQL, WMS, WFS, GML, KML, InfraGML, CityGML to name just a few from the OGC. buildingSmart's IFC standard is supported by all major BIM design tools including the market leader Revit.

Perhaps most significantly there has been significant progress in developing common conceptual models for geospatial and AEC views of city infrastructure. The major breakthrough in bringing the architectural and geospatial views onto a common footing is the OGC LandInfra Conceptual Model developed by the OGC in cooperation with buildingSMART International and approved as an OGC standard in August, 2016. The LandInfra conceptual model was developed by Bentley Systems, Leica Geosystems, Trimble, Australian Government Department of Communications, Autodesk, Vianova Systems AS, and buildingSMART International and provides a unifying basis for land and civil engineering standards including the OGC's InfraGML and buildingSmart International's IFC for infrastructure standards.  InfraGML is the OGC's application schema supporting land development and civil engineering infrastructure facilities. InfraGML supports a subset of the existing LandXML standard. buildingSmart International's IFC-Alignment project uses this common conceptual model of alignments for roads, railways, tunnels and bridges. The objectives of the IFC-Alignment project is to enable the exchange of alignment information through the full infrastructure lifecycle from planning through design and construction to asset management.

Secondly, in the 1990s and the early 2000s there were few open source geospatial projects. The Open Source Geospatial Foundation (OSGEO) was only founded in 2006. Since then there has been rapid development of open source geospatial projects. There is enough open source geospatial code now that at the GEO|Design+BIM conference in Amsterdam, Sanghee Shin described a web-based geo-platform integrating massive, complex BIM models using open source geospatial libraries and tools including PostGIS, GeoServer, and Cesium and NASA WorldWind virtual globes. The importance of this project is that it shows that it is possible using the available open source geospatial APIs and libraries - together with some genuine innovative development - to create an open, non-proprietary, 3D geospatial platform for integrating geospatial and BIM. Given the critical importance of addressing the cultural and technical divide between the AEC and geospatial worlds, clearly many believe that a viable open source alternative will be essential for developing the innovative solutions to the challenge of interfacing the two worlds. 

Finally, major players in the AEC and geospatial industries, Autodesk and ESRI, have again agreed to collaborate. This time around they are probably more aware of the technical issues hindering BIM+geospatial integration. Both have adopted open standards when it made business sense - often under pressure from government clients - and both have also adopted open source technology where it made business sense. Together the combination of major players supporting the effort, standards organization like the OGC and buildingSmart already working to address the problem, and a broad range of open source code and a vibrant developer community may be enough to address the technical problems. If building and infrastructure owners and facilities managers including government as well as regulators become aware of the risk of not solving the problem, a solution may be found so that we will not be having the same BIM+geospatial integration discussion a quarter of a century from now.

Key registry created in the Netherlands to help model the subsurface

Increasing urbanization throughout the world challenges the sustainable development and resilience of cities. Despite this, the importance of the ground beneath cities is often overlooked.  A lot of information about the underground is captured during excavation and drilling. But this data is typically stored in project or private databases and is not shared. Capturing this data in a shared, open database is a first step toward developing a model of the subsurface that can be reliably used for urban planning, design, construction and maintenance.  The Netherlands has recently embarked on a national program supported by legislation and standards to do just that.

At the preconference workshop “Utilizing subsurface data for urban planning, design, construction and maintenance” organized by Geodan and TNO at the GEO|Design+BIM conference in Amsterdam, Michiel van der Meulen of the COST Sub-Urban program presented an overview of a new program supported by legislation to develop and maintain an open database of information about the subsurface in the Netherlands.

Current data about the subsurface in the Netherlands comes from borehole logs, cone penetration tests, and groundwater measuring points. Most of the currently available information about the subsurface in the Netherlands comes from boreholes drilled typically to over 100 meters using light drilling equipment as well as some boreholes to much greater depths using heavy drilling equipment. This information is available for free through the DINOloket of the TNO, Geological Survey of the Netherlands.

Based on interpolating from the existing borehole data, there are several models of the subsurface in the Netherlands developed by the Geological Survey of the Nethelands. For example, there is a model of the subsurface down to 50 meters. It has a resolution of 100 by 100 by half a meter. It was intended to assist in identifying aquifers and gas fields. This is too course for city planning purposes. What an urban planner needs is much higher resolution and that will require more data.

In 2015 a new law was passed by the States General or parliament in the Netherlands creating a Basisregistratie Ondergrond (BRO) or Key Registry for the Subsurface. The law mandates that if you excavate or drill you have to share your data (relating to soils, geotechnics, and groundwater) with the registry. In addition if when using the data in the registry you find something is incorrect you have to report it. The registry is required to be open and accessible to all citizens of the Netherlands. This new registry will require new standards for recording data and there is a major effort underway in the Netherlands to develop these standards. The implementation of the new subsurface registry will begin in 2018.

The Key Registry for the Subsurface or BRO is intended to provide transparent and accessible information about the subsurface. It is subject to the government's open data policy. In addition to providing data for urban planning, design, construction and maintenance, it is intended to include an inventory of opportunities and risks relating to subsurface, provide data for sustainable water management and to support the governments goal of moving toward a carbon neutral economy at the national regional and local levels. It is also intended to reduce the costs associated with infrastructure management for soil remediation.

The overall goal is to initiate the process of organizing an information chain that will harness geological survey and city efforts in building databases and creating the necessary standards for information production about the subsurface. But to better serve cities with the subsurface information they need, Michiel van der Meulen said that we need to rethink geological mapping, rethink urban information management, and create new alliances among organizations involved in urban development. The new BRO key registry is an important first step in the process of developing models of the subsurface that can be joined seamlessly with above surface information to create the comprehensive digital twins that will be required by the smart cities of the future.

Do you really need a GIS to do spatial analytics ?

Geospatial technology - spatial data management and analytics - is being used in many industries outside of the traditional GIS sector. Analysts and IT folks in these industries may be surprised to find that the spatial analytics and data management they require is already available from their database management system whether it is Oracle, SQL Server, PostgreSQL, or SQLite

At the GIS in the Rockies conference in Denver Dave Murray from the City of Westminster reminded us how powerful the spatial analytics in modern relational database systems are.

In the late 1980s the first implementations supporting spatial data in a relational database were Vision* on Oracle RDBMS developed by Geovision and GFIS on Db2 by IBM. Subsequently the Open Geospatial Consortium defined a simple feature specification for spatial SQL. First Oracle and then Microsoft and the open source community have added spatial SQL based on the OGC standard to relational databases. Oracle Spatial, PostGIS+PostgreSQL, Microsoft SQL Server, and Spatialite+SQLite all support spatial SQL.

Folks wishing to employ geospatial techniques may find that they don't need a traditional GIS product and all the complexity involved in integrating a GIS with their database management system. They very likely will find that they can do all the geospatial processing they require using spatial extensions to standard SQL on their existing database. Finding the number of automobile accidents along a stretch of road, the census tracts a road passes through, or the burglaries in a neighbourhood are straightforward spatial SQL queries. The results can be visualized using Oracle's own visualization tools or open source applications like QGIS sitting on top of Oracle, SQL Server or PostGIS+PostgreSQL.  Dave mentioned spatial SQL courses on Udemy for those interested in learning more about spatial SQL.  Gordon Luckett's web site has a wealth of information about spatial queries, especially for integration with AutoCAD and Esri ArcMap.  Gordon also has developed courses on Lynda such as Using SQL Server with AutoCAD and Learning QGIS.

 

Progress in advancing integrated BIM+geospatial standards for infrastructure

There has been impressive progress in developing open standards for the integration of geospatial and AEC (architecture, engineering and constcruction) views of city infrastructure which provides a standards-based basis for full-lifecycle management from design through to operations and maintenance of infrastructure projects.

At GeoBusiness 2017 in London, Nigel Clifford, CEO of the Ordnance Survey, described a project that is intended to create a smart city model for four square kilometers of Manchester called cityverve.  The objective is to apply Internet of Things (IoT) technologies to a real city.  It is supported by Manchester City Council, Manchester Science Partnerships, the University of Manchester, Ordnance Survey, Cisco, BT and other tech players and well as government and Innovate UK.  It is intended to provide an open platform of platforms.  Among the fundamanental things that will be required to do this is open standards for modeling city infrastructure and to make this accessible to the geospatial and AEC communities.

Years ago private Dutch engineering and construction companies began adopting an integrated geospatial/BIM approach to construction.  I blogged previously about the firm Royal BAM Group nv / BAM Infraconsult that adopted integrated BIM and GIS because many of its projects require full life-cycle BIM.  At the first GeoBIM conference in Amsterdam, Jothijs van Gaalen gave some real world examples of Design, Build, Finance and Maintain (DBFM) highway construction that included GIS+BIM integration. BAM's motivation for investing in BIM+GIS are market developments especially more complex construction assignments and an increasing demand from customers for service provision throughout the entire life cycle of a project.   As another example, ARCADIS Netherlands has delivered a number of projects that integrate geospatial into the design process.

The Netherlands has also led in the development and application of open BIM/geospatial standards‎ in the construction industry including NL/SfB, BS&I, ETIM, CB-NL, IMGeo, CityGML, NLCS, GB-CAS, COINS, IFC, VISI, SALES, National Model BIM Protocol, and National Model BIM Implementation Plan.  The Netherlands has created a BIM Loket (BIM Gateway) that is intended to be a national portal for information about BIM and open BIM standards in the Netherlands including related geospatial standards. 

In the AEC world Industry Foundation Classes (IFC) are the open and neutral data format standard for the exchange of building information models (BIM) that is widely used in the AEC (architecture, engineering, and construction)  sector.  The IFC standard was developed and is supported by buildingSMART International.  The IFC BIM standard is used in the construction industry in many countries and is mandated in some such as Finland.  The first versions of IFC were intended for buildings (vertical BIM).  Recently buildingSMART initiated the IFC for Infrastructure project to extend the IFC standard to support linear transportation infrastructure (horizontal BIM or BIM for Infrastructure) such as rail, road, bridges, and tunnels. 

In the transportation infrastructure world the widely used LandXML standard, which is supported by  800 members in the roads and highway transportation sector, includes alignment, road, rail, survey, terrain, land parcels, drainage, wastewater, and water distribution systems.  LandXML is not associated with a recognized international standards organization and has had limited support for the past few years. 

In the geospatial world a widely used international geospatial standard for cities is CityGML.  Thomas Kolbe and co-workers are the developers of the CityGML standard that has been adopted by the Open Geospatial Consortium.  Kolbe et al. are translating a vision of the city as an interactive system comprised of functional components, utility networks connecting components, and interdependencies between utility networks into standards-based intelligent models that can be used to analyze urban environments for a variety of purposes including risk- and disaster management energy consumption, carbon balancing and city life-cycle management.

Currently CityGML includes 3D geometry, topology, semantics, and appearance for urban environments. CityGML also supports a standard mechanism for adding extensions, called Application Domain Extensions (ADEs).  There are several Application Domain Extensions (ADEs) that have been developed to extend CityGML to other domains.  For example,  I blogged about a basic extension UtilityNetworksADE that was proposed for city utility networks.  The INSPIRE Data Specifications Annex III contains use case encodings and a data model for Buildings the development of which was strongly influenced by CityGML. There have been attempts to integrate or at least develop a mapping between IFC and CityGML. 

The major breakthrough in bringing the architectural and geospatial views onto a common footing is the OGC LandInfra Conceptual Model developed by the OGC in cooperation with buildingSMART International and approved as an OGC standard in August, 2016.  LandInfra  was developed by Bentley Systems, Leica Geosystems, Trimble, Australian Government Department of Communications, Autodesk, Vianova Systems AS, and buildingSMART International and provides a unifying basis for land and civil engineering standards including the OGC's InfraGML and buildingSmart International's IFC for infrastructure standards.  

buildingSmart International's IFC-Alignment project uses this common conceptual model of alignments for roads, railways, tunnels and bridges.  The objectives of the IFC-Alignment project is to enable the exchange of alignment information through the full infrastructure lifecyle from planning through design and construction to asset management. A Candidate IFC-Alignment Standard is open for review now.

InfraGML is the OGC's application schema supporting land development and civil engineering infrastructure facilities. InfraGML supports a subset of the existing LandXML standard. InfraGML is comprised of several parts;

• Part 0: LandInfra Core Encoding Standard
• Part 1: LandInfra Land Features Encoding Standard
• Part 2: LandInfra Facilities and Projects Encoding Standard
• Part 3: LandInfra Alignments Encoding Standard
• Part 4: LandInfra Roads Encoding Standard
• Part 5: LandInfra Railways Encoding Standard
• Part 6: LandInfra Survey Encoding Standard
• Part 7: LandInfra Land Division - Encoding Standard

The Candidate OGC InfraGML encoding Standards was released for public comment in January of this year.  In April the Candidate OGC InfraGML 1.0: Part 7 – LandInfra Land Division - Encoding Standard was released.

At GeoBusiness 2017 in London, Phil Jackson of buildingSmart UK and UK Representative Infrastructure Room provided an update on the progress in extending IFC for infrastructure based on a common BIM/geospatial conceptual model;

• Linear alignments for road and rail - under deployment testing
• Common infrastructure architecture - developed jointly with OGC
• IFC for rail - an initial Chinese specification has been published
• IFC Bridges - just started

and several more are in active planning or ready to start;

• IFC for ports and harbours - starts this summer
• IFC for roads - ready to start
• IFC tunnel - planned

Thus it appears that building geospatial and AEC standards for transportation and land management based on a common conceptual model and developing in parallel is well advanced.

New underground infrastructure standard to be launched in April

In a previous post I alluded to the imminence of the new PAS 256 standard "Buried services – Collection, recording and sharing of location information  data" which is intended to complement the existing PAS 128 standard.

In the UK the total length of buried infrastructure is in excess of 4 million kilometres. The new PAS is intended to provide recommendations for the collection, recording and sharing of location information data relating to:underground assets and any associated above ground assets; decommissioned assets; and abandoned assets.The PAS is expected to set out a consistent, accessible data protocol to enable effective recording and sharing of the location, state, and nature of buried assets, and recommends how existing asset  records should be updated, recorded and collated. The PAS is also intended to cover the gathering of geospatial data using absolute or relative accuracy, plus associated evidence (such as photographic evidence).

The new standard has been through a public review and will be launched April 11, 2017 at the Institution of Civil Engineers in London..

OGC issues RFI for underground infrastructure maps and models

The Open Geospatial Consortium (OGC) has announced the initiation of a three-phase project to develop interoperability standards for underground infrastructure. The underground infrastructure data interoperability project will take two and a half years to complete and will involve the collaboration of many cities, utilities, and engineering and technology companies.

In the U.S. it is estimated that an underground utility is hit about every minute.  Underground utility conflicts and relocations are the number one cause for project delays during road construction.  Assuming the average cost of underground strikes is roughly $1000 per strike, the estimated total cost to the U.S. economy is $1.5 trillion annually.  Currently, the exchange of underground utility information between infrastructure organizations within the same jurisdiction or in adjacent jurisdictions has been greatly hampered by incompatible and incomplete data. OGC anticipates that this project will make a significant contribution towards facilitating improved information management and secure sharing and collaboration, which should make infrastructure planning, operations & maintenance, and emergency response less costly and time consuming, and more effective.

The Open Geospatial Consortium (OGC) has issued a Request for Information (RFI) to inform a Concept Development Study (CDS) that will assess the current state and future direction of information standards for modelling, mapping, and managing underground infrastructure.

The CDS will define the scope of a multi-phase underground infrastructure interoperability project. The purpose of the study is to develop an in-depth understanding of all the components necessary to enable infrastructure data interoperability and standards in an underground environment. The CDS is initially focused on the urban landscape. This RFI is a first step in the CDS process. Any organization with an interest in underground infrastructure is invited to respond to the RFI before 15 March 2017.

This initiative is supported by the Fund for the City of New York, the Singapore Land Authority, the Ordnance Survey and the OGC Innovation Program.

Developing a national 3D strategy in the Netherlands

At the GeoBIM Building and Infrastructure Conference in Amsterdam Rob van de Velde and Friso Penninga of Geonovum presented an overview of what has been achieved to date with respect to 3D data and technology at the national level in the Netherlands and what they see as the next major step, the development of a national 3D strategy. 

 

BIM and geospatial (2D and 3D) are key underpinnings of 3D.  The Netherlands has been at the forefront of implementing BIM in the construction industry.  Just as importantly in the long term, the Netherlands has been leading the geospatial-BIM integration initiative.  It has long been recognized that geospatial/BIM integration is essential for building and infrastructure operation and maintenance or what is referred to as full-lifecycle BIM.

 

Years ago private Dutch engineering and construction companies began adopting an integrated geospatial/BIM approach to construction.  I blogged previously about a firm Royal BAM Group nv / BAM Infraconsult that adopted integrated BIM and GIS because many of its projects require full life-cycle BIM.  At the first GeoBIM conference in Amsterdam, Jothijs van Gaalen gave some real world examples of Design, Build, Finance and Maintain (DBFM) highway construction that included GIS+BIM integration. BAM's motivation for investing in BIM+GIS are market developments especially more complex construction assignments and an increasing demand from customers for service provision throughout the entire life cycle of a project.   As another example, ARCADIS Netherlands has delivered a number of projects that integrate geospatial into the design process. 

The Netherlands has also been leading in the development and application of open BIM/geospatial standards‎ in the construction industry including NL/SfB, BS&I, ETIM, CB-NL, IMGeo, CityGML, NLCS, GB-CAS, COINS, IFC, VISI, SALES, National Model BIM Protocol, and National Model BIM Implementation Plan.  The Netherlands has created a BIM Loket (BIM Gateway) that is intended to be a national portal for information about BIM and for management of open BIM standards in the Netherlands.  It provides a central information centre for open BIM standards including related geospatial standards. 

Geospatial

Within government the Environment Planning Act is responsible for a growing focus on the physical environment.  There is also a greater desire for an integrated approach to modeling the physical environment.  Citizens are asking questions that require accurate 3D modeling, for example, noise, air quality, light and shadow, and viewports.

There have been several initiatives at the national level in the Netherlands.

• 2010 3D pilot exploring possibilities of 3D including IMGeo and CityGML
• 2012 3D pilot exploring Geo-BIM integration
• 2014 "Breakthrough 3D" - application of 3D in many domains

The last "Breakthrough 3D" initiative revealed that 3D technology had matured and that there were many initiatives asking for uniform 3D data.

BIM

The Dutch Building Information Council (BIR) has been encouraging BIM for a long time - I blogged about BIR and BIM in the Netherlands previously.  One of my takeways from the recent RICS BIM conference in London was that there is a desperate need for standards, especially to classify the huge amount of date in addition to the 3D geometry that accumulates during a construction project.  Much of this data would be useful to the FM folks who are responsible for operating and maintaining the building after construction if it were accessible.

From both the perspective of geospatial data for modeling the environment and BIM data for modeling the built environment, it's time for the next step:  a government initiative for a national 3D strategy.  The concept is to create a 3D production chain, capable of creating timely, accurate and reliable 3D geo-information.

Gap analysis - what's available vs what's needed

The Netherlands has a digital elevation model ( AHN or Actual Height Model) with nationwide coverage but the update cycle is on the order of eight years.  What's needed is a nationwide point cloud that is updated at least once a year.  There are a number of ways including new technologies this could help achieve this.

The Netherlands has national 2D core data, large scale topographic data that is open and universally accessible.  It has no equivalent in 3D.  One way to generate this rapidly are reliable techniques for automating the extraction of buildings, infrastructure and natural features from point clouds.

The Netherlands has a national portal for disseminating 2D data.  The national geoportal PDOK has experienced  3-4 billion hits in 2016.  For 3D data, PDOK needs to be upgraded to be able to serve 3D data. 

 

For 3D data there are standards for exchanging 3D data about cities and BIM model such as CityGML and IFC.  What's needed are standards for distributing and viewing 3D data.  Another important outstanding challenge is developing an interface between CityGML and IFC.

Rob emphasized that developing a national 3D standard is about identifying and involving stakeholders, understanding user requirements, and realizing the individual components, but more importantly it is about aligning individual initiatives, thus optimizing 3D supply chain.