Between the Poles

Deep learning enables automated extraction of building footprints and road networks from satellite imagery

Automated feature extraction from satellite imagery has made major progress in the last year. Accurate building footprints extracted from high resolution satellite imagery are available from companies such as Ecopia (which has just announced a partnership with DigitalGlobe).  Also NVIDIA has demonstrated the ability to automate detection of many road networks using sophisticated algorithms and multi-spectral high resolution imagery.

Building footprints

SpaceNet is a repository commercial satellite imagery and labeled training data being made available through Amazon Web Services at no cost to the public to foster innovation in the development of computer vision algorithms to automatically extract information from remote sensing data.  It is the result of a collaboration between DigitalGlobe, CosmiQ Works, and NVIDIA.  Two competitions for automated building footprint extraction from high resolution satellite imagery and a third for road network extraction have been held based on SpaceNet data. 

The first SpaceNet Challenge was launched in November 2016.  Imagery of Rio de Janeiro from the WorldView 2 satellite at 50cm resolution using eight spectral bands was made publicly available on Amazon Web Services (AWS).  42 developers competed to create algorithms that extract building footprints from the imagery.  The source code of the winning algorithms was made available on the SpaceNetChallenge GitHub repository. 

The winning implementation was developed by Brazilian developer using an implementation based on random forests with brute force polygon search and did not leverage deep learning frameworks.  The organizers' conclusion based on the results of Round 1 was that automated building footprint extraction remained a challenge.

Round 2 winner. Most building footprints are precisely detected. Number on each building footprint represents the overlap between the prediction and the ground truth.

Round 2 of the SpaceNet challenge involved higher resolution 30 cm imagery from WorldView 3 for four cities across the globe. The competition challenged developers to improve performance from the first competition using the higher-resolution imagery and more geographically diverse training data samples. SpaceNet on AWS held imagery and building footprints for Las Vegas, Paris, Shanghai, and Khartoum with over 180,000 buildings181,619 footprints.  In addition additional bands were made available (panchromatic, 3-band pan-sharpened, and 8-band pan-sharpened).

Results from Round 2 winning solution: top left: Las Vegas, top right: Las Vegas, lower right: Khartoum, and lower left: Paris. Blue outlines: ground truth, green outlines: correct identifications, red outlines: false positives and yellow outlines: false negatives.

The winner of Round 2 applied a  deep neural network model developed originally for medical image segmentation called U-Net.  Successful training of deep networks requires many thousand annotated training samples.  Training was aided significantly by augmenting the imagery with OpenStreetMap data.  The winning solution used OpenStreetMap layers and Worldview-2 multispectral layers as the input of the deep neural network algorithm.  In Las Vegas, the winning solution was outstanding in outlining suburban homes, but also did well in identifying odd-shaped buildings. In Khartoum, the solution was very good in identifying the footprints of stand-alone apartment complexes, but had trouble with small buildings, with building footprints that are close together, and with horizontally large buildings.  One of the problems was inconsistent ground truth which affected training. The solutions offered by the developers in the competition were sufficiently successful that the organizers concluded that the winning algorithms achieved performance with potential for automated mapping tasks such as keeping maps up-to-date and to assist first responders during natural disasters.  The source code for the winning implementations can be found in the SpaceNetChallenge GitHub repository.

DigitalGlobe has just announced a partnership with Ecopia, which has established an automated process to create building footprints quickly and at scale by leveraging DigitalGlobe's Geospatial Big Data platform (GBDX) and advanced machine learning in combination with DigitalGlobe’s imagery library. The two companies plan to automatically extract accurate 2D building footprints globally, then refresh the datasets periodically to find and track changes over time.  These datasets would be valuable to municipal governments for permitting purposes and for first responders after disasters.   Ecopia is developing a database for all of Australia (7.6 million square kilometers) which will include every building with a roof area greater than 9 square meters.  The database will contain building footprints for about 15 million buildings. The database includes linkages to other datasets including geocoded address, property data and administrative boundaries. Ecopia intend to update the dataset regularly to be able to track changes, especially urban sprawl.

Road networks

In November 2017 CosmiQ Works, Radiant Solutions, and NVIDIA announced a third round competition, “Road Detection and Routing Challenge” to explore automated methods for extracting map-ready road networks and routing information from high-resolution satellite imagery.  

Detecting road networks in Paris. Photo image, ground truth, and the results achieved with three deep learning algorithms

NVIDIA has released the results of several deep learning algorithms that illustrate just how difficult identifying road networks is and the sophistication of the available tools.  These include using multi-band spectral imagery to identify the material properties of the road surface itself (asphalt, gravel, packed earth) and using more sophisticated processing including conditional random fields (CRF), percolation theory, and reinforcement learning. 

Detecting road networks in Las Vegas. Photo image, ground truth, and the results achieved with three deep learning algorithms

NVIDIA utilized high performance GPU compute resources provided by the NVIDIA GPU Cloud and Amazon Web Services.

Elevated risk in road construction lacking knowledge of the subsurface

The challenge of increasing urbanization in a world where anthropogenic pressure on the environment has become threatening is driving the digitalization of urban planning and design, construction and maintenance and operations of urban infrastructure. Building information modeling and laser and photogrammetric scanning of entire cities is resulting in comprehensive digital twins of the above ground infrastructure of cities. But the below ground has been neglected. There is a growing recognition that the subsurface needs to be included in urban planning and construction to reduce the risks of conflicts with subsurface infrastructure and to maximize the benefits of subsurface resources.

One of the challenges identified by the European COST Sub-Urban program is a communications gap between urban planners and subsurface experts. To begin to improve the level of communication between urban planners and subsurface experts, Geodan and TNO organized a preconference workshop “Utilizing subsurface data for urban planning, design, construction and maintenance” at the GEO|Design+BIM conference in Amsterdam.

Alternative routes for proposed highway

The workshop was a hands-on event with real data from a part of the Netherlands. It was led by Cindy Teeven and Ceciel Fruijtier from Geodan and Rob van der Krogt from TNO. In the simulation in which all participants in the workshop participated, a decision had to be made about the preferred route for a new highway. The alternatives were a route that skirted existing buildings and followed the route of an existing railway and regional road and a route that went through the countryside. An online GIS system provided layers of typical 2D information from public databases that might be relevant to the decision process including the location of existing residential and business buildings, historical and conservation sites, existing infrastructure such as roads and railways, and so on. Each of the participants was assigned a role to play: people of the place, traffic engineer, civil engineer, regional government, environmental authority, and local business council. After an hour with the data we were asked to report on the decision that had been reached on a preferred route. Primarily for reasons relating to reducing traffic through populated areas, the choice of most of the stakeholders was the route through the countryside. However, the “civil engineers” raised a red flag about this route because of the risk of poor soil conditions that could make this route challenging and expensive. Clay and peat, which are prevalent in the Netherlands, are susceptible to subsidence when a load is placed on them and are therefore to be avoided whenever possible when building any kind of structure. In contrast a great deal was known about the soil conditions of the alternative route because infrastructure and houses had been built along it for many years.

Surface elevations from lowest (blue) to highest (red)

After the first hour it was announced additional information was provided including a digital terrain model (DTM) showing surface elevation for the area and a model of the subsurface composition of the terrain – whether sand, peat or clay - to roughly 50 meters interpolated from bore holes and groundwater sampling sites in the region.

The DTM showed that the route skirting existing infrastructure was generally higher by a couple of meters than the alternative route through the countryside where the land was low lying suggesting subsidence. The subsurface soil composition model was used to create profiles showing the probability of clay and peat in the subsurface along the proposed routes. The 3D model revealed a high probability of clay and peat in the subsurface along the route through the countryside. In light of the new information about the subsurface, participants were requested again to arrive at a decision for the preferred route. Armed with the new subsurface information several of the stakeholders changed their minds. For the “civil engineers” the new subsurface information confirmed their fears about the risk of the route through the countryside.

The cost of an elevated highway or viaduct over the extensive peat and clay fields would be much more expensive than the alternative route skirting existing infrastructure. If the simulation had included cost and it had been made clear that the cost would have to be borne through a rate increase, I would expect that more of the stakeholders would have changed their minds.

Probability of clay in the subsurface along route from lowest (blue) to highest (red). Vertical dimension is depth.

It was explained at the end of the workshop that the workshop was based on a historical event in the north of Holland. At the time it was decided to build the route through the countryside. After extensive construction delays and rising costs, it was ultimately resolved in a high profile court case. The outcome was that the construction contractor and the regional government agreed to split the much higher cost of building a viaduct through the countryside.

The workshop was effective in illuminating the risk of decision making without information about the subsurface. In this case there was some data about the subsurface although historically it was only consulted after the initial decision. The reality in much of the world is that there is little information publicly available about the subsurface. Many construction projects proceed blindly with respect to what lies below the surface. In the United States poor information about what lies below the surface is the leading cause of construction delays and cost overruns. Construction bids are routinely inflated significantly to accommodate the risks associated with the lack of knowledge about the subsurface. Poor information about the subsurface costs the world economy trillions of dollars. What is underway in the Netherlands with the Key Registry for the Subsurface or BRO is the beginning of a practical solution to this problem. The world should be watching.

Fully automated classification of road and highway features for autonomous vehicles

Point clouds are fine for visualization to provide context, but to be really useful they need to be classified.  Feature extraction, as the process is often referred to,  is currently a mostly manual process that has been used for many years in the remote sensing sector. A trained person visually inspects the point cloud and graphically selects features which are identified by type of feature, sign, light pole, lane marking, and so on. 

Automated classification of point clouds

Image processing of 3D point clouds has advanced to the point where it has been possible to semi-automate this tedious feature extraction process.  The reliability of the automation process depends on the point density of point cloud, often expressed as points per square meter or per square centimeter.  Point density depends on the technology used to capture the point cloud. Terrestrial lidar, for example, typically captures points at the rate of hundreds of thousands to millions of point per second and is capable of millimeter to centimeter precision. Phodar generates lower point densities and is less precise. Aerial lidar captures many points per second but with less precision because it is covering much greater areas. Single photon lidar is a new technology that can collect 6 million points per second which means that planes can fly quite high, cover very broad areas and still collect 12 - 30 points per square meter.

At the Year in Infrastructure 2017 (YII2017) conference in Singapore, Sharad Oberoi of Sanborn gave an insightful overview of the state of the art for automated feature extraction from combined aerial and mobile scans of highways and adjacent areas.  Different types of laser scanners are appropriate for capturing different types of features. Terrestrial and mobile laser scanners can capture point clouda that are suitable for extracting curbs, gutter drains, traffic signals, road and local terrain information, parking meters, walls, obstructions in the right of way, manholes, sidewalks, overhead clearances, lights, utility wires and connections, garbage cans, benches, fences, guardrails and barriers, retaining walls, and line of sight vegetation obstructions. Aerial lidar will only be able to get gutter drains, road and local terrain information, walls, obstructions in the right of way, and sidewalks with limited ability to get curbs, lights, utility wires and connections, retaining walls, and vegetation.  Lidar does not do well for metal structures such as communications towers and fir these a combination of lidar and phodar is preferred.

Sanborn has developed feature extraction technology that extracts transportation objects from combined aerial and mobile data sets in a semi-automated manner. The mobile and aerial data sets meet the accuracy and information content required for geospatial information for mapping applications as well as the additional information that can be mined for potential asset inventory and infrastructure information content. Mobile technology allows for the low-risk, rapid collection of geospatial information, limiting safety impacts to workers. The aerial LiDAR allows for the rapid collection of elevation information for detailed surface modeling as well as feature extraction. Both technologies collect enormous amounts of point-cloud data that can be merged using the Terrascan capability in Bentley MicroStation. The combined data set is then passed through the automated feature extraction modules to identify transportation objects.  Sanborn feature extraction modules achieve a high proportion of correct feature extraction for road and highway scans.

In the future Sharad expects that it will be possible to improve data collection efficiency by using terrestrial robots or autonomous UAVs equipped with SLAM technology to enable them to operate in GPS-denied areas such as under bridges and in tunnels.  Sanborn has been working with deep learning to improve the automated extraction process.  So far this has yielded mixed results, but Sharad expressed optimism that with AI full automation of the classification process will be achieved.

High precision maps for autonomous vehicles

An application of this technology is high precision maps for the autonomous vehicle market.

Sanborn develops high precision maps for the autonomous vehicle market.  Autonomous vehicles will require high precision maps which contain significantly more detailed information and true-ground-absolute accuracy than current road maps. Sanborn has developed proprietary mapping technology that leverages aerial imagery, aerial lidar data, and mobile lidar data to create standardized, high-precision 3D base-maps focusing specifically on the self-driving vehicle market. These maps include detailed inventories of all stationary physical assets related to roadways such as road lanes, road edges, shoulders, dividers, traffic signals, signage, paint markings, poles, and all other critical data needed for the safe navigation of roadways and intersections by autonomous vehicles.

Progress in advancing integrated BIM+geospatial standards for infrastructure

There has been impressive progress in developing open standards for the integration of geospatial and AEC (architecture, engineering and constcruction) views of city infrastructure which provides a standards-based basis for full-lifecycle management from design through to operations and maintenance of infrastructure projects.

At GeoBusiness 2017 in London, Nigel Clifford, CEO of the Ordnance Survey, described a project that is intended to create a smart city model for four square kilometers of Manchester called cityverve.  The objective is to apply Internet of Things (IoT) technologies to a real city.  It is supported by Manchester City Council, Manchester Science Partnerships, the University of Manchester, Ordnance Survey, Cisco, BT and other tech players and well as government and Innovate UK.  It is intended to provide an open platform of platforms.  Among the fundamanental things that will be required to do this is open standards for modeling city infrastructure and to make this accessible to the geospatial and AEC communities.

Years ago private Dutch engineering and construction companies began adopting an integrated geospatial/BIM approach to construction.  I blogged previously about the firm Royal BAM Group nv / BAM Infraconsult that adopted integrated BIM and GIS because many of its projects require full life-cycle BIM.  At the first GeoBIM conference in Amsterdam, Jothijs van Gaalen gave some real world examples of Design, Build, Finance and Maintain (DBFM) highway construction that included GIS+BIM integration. BAM's motivation for investing in BIM+GIS are market developments especially more complex construction assignments and an increasing demand from customers for service provision throughout the entire life cycle of a project.   As another example, ARCADIS Netherlands has delivered a number of projects that integrate geospatial into the design process.

The Netherlands has also led in the development and application of open BIM/geospatial standards‎ in the construction industry including NL/SfB, BS&I, ETIM, CB-NL, IMGeo, CityGML, NLCS, GB-CAS, COINS, IFC, VISI, SALES, National Model BIM Protocol, and National Model BIM Implementation Plan.  The Netherlands has created a BIM Loket (BIM Gateway) that is intended to be a national portal for information about BIM and open BIM standards in the Netherlands including related geospatial standards. 

In the AEC world Industry Foundation Classes (IFC) are the open and neutral data format standard for the exchange of building information models (BIM) that is widely used in the AEC (architecture, engineering, and construction)  sector.  The IFC standard was developed and is supported by buildingSMART International.  The IFC BIM standard is used in the construction industry in many countries and is mandated in some such as Finland.  The first versions of IFC were intended for buildings (vertical BIM).  Recently buildingSMART initiated the IFC for Infrastructure project to extend the IFC standard to support linear transportation infrastructure (horizontal BIM or BIM for Infrastructure) such as rail, road, bridges, and tunnels. 

In the transportation infrastructure world the widely used LandXML standard, which is supported by  800 members in the roads and highway transportation sector, includes alignment, road, rail, survey, terrain, land parcels, drainage, wastewater, and water distribution systems.  LandXML is not associated with a recognized international standards organization and has had limited support for the past few years. 

In the geospatial world a widely used international geospatial standard for cities is CityGML.  Thomas Kolbe and co-workers are the developers of the CityGML standard that has been adopted by the Open Geospatial Consortium.  Kolbe et al. are translating a vision of the city as an interactive system comprised of functional components, utility networks connecting components, and interdependencies between utility networks into standards-based intelligent models that can be used to analyze urban environments for a variety of purposes including risk- and disaster management energy consumption, carbon balancing and city life-cycle management.

Currently CityGML includes 3D geometry, topology, semantics, and appearance for urban environments. CityGML also supports a standard mechanism for adding extensions, called Application Domain Extensions (ADEs).  There are several Application Domain Extensions (ADEs) that have been developed to extend CityGML to other domains.  For example,  I blogged about a basic extension UtilityNetworksADE that was proposed for city utility networks.  The INSPIRE Data Specifications Annex III contains use case encodings and a data model for Buildings the development of which was strongly influenced by CityGML. There have been attempts to integrate or at least develop a mapping between IFC and CityGML. 

The major breakthrough in bringing the architectural and geospatial views onto a common footing is the OGC LandInfra Conceptual Model developed by the OGC in cooperation with buildingSMART International and approved as an OGC standard in August, 2016.  LandInfra  was developed by Bentley Systems, Leica Geosystems, Trimble, Australian Government Department of Communications, Autodesk, Vianova Systems AS, and buildingSMART International and provides a unifying basis for land and civil engineering standards including the OGC's InfraGML and buildingSmart International's IFC for infrastructure standards.  

buildingSmart International's IFC-Alignment project uses this common conceptual model of alignments for roads, railways, tunnels and bridges.  The objectives of the IFC-Alignment project is to enable the exchange of alignment information through the full infrastructure lifecyle from planning through design and construction to asset management. A Candidate IFC-Alignment Standard is open for review now.

InfraGML is the OGC's application schema supporting land development and civil engineering infrastructure facilities. InfraGML supports a subset of the existing LandXML standard. InfraGML is comprised of several parts;

• Part 0: LandInfra Core Encoding Standard
• Part 1: LandInfra Land Features Encoding Standard
• Part 2: LandInfra Facilities and Projects Encoding Standard
• Part 3: LandInfra Alignments Encoding Standard
• Part 4: LandInfra Roads Encoding Standard
• Part 5: LandInfra Railways Encoding Standard
• Part 6: LandInfra Survey Encoding Standard
• Part 7: LandInfra Land Division - Encoding Standard

The Candidate OGC InfraGML encoding Standards was released for public comment in January of this year.  In April the Candidate OGC InfraGML 1.0: Part 7 – LandInfra Land Division - Encoding Standard was released.

At GeoBusiness 2017 in London, Phil Jackson of buildingSmart UK and UK Representative Infrastructure Room provided an update on the progress in extending IFC for infrastructure based on a common BIM/geospatial conceptual model;

• Linear alignments for road and rail - under deployment testing
• Common infrastructure architecture - developed jointly with OGC
• IFC for rail - an initial Chinese specification has been published
• IFC Bridges - just started

and several more are in active planning or ready to start;

• IFC for ports and harbours - starts this summer
• IFC for roads - ready to start
• IFC tunnel - planned

Thus it appears that building geospatial and AEC standards for transportation and land management based on a common conceptual model and developing in parallel is well advanced.

2017 ASCE Infrastructure Scorecard reveals slight improvement

The 2017 ASCE Infrastructure Scorecard has been released this morning.  Grades for seven categories of infrastructure – Hazardous Waste, Inland Waterways, Levees, Ports, Rail, Schools, and Wastewater – improved slightly. The grades for three categories – Parks, Solid Waste, and Transit – declined this year.  The overall cumulative GPA remained constant at D+.

2013 and 2017
Water and the environment

    Dams                    D  D
    Drinking Water      D  D
    Hazardous Waste D  D+
    Levees                  D- D
    Solid Waste          B- C+
    Wastewater          D  D+

Transportation

    Aviation                 D  D
    Bridges                 C+ C+
    Inland Waterways D- D
    Ports                     C  C+
    Rail                       C+ B
    Roads                   D  D
    Transit                   D  D-

Public Facilities

    Public Parks & Recreation C- D+
    Schools                 D  D+

Energy

    Energy                  D+ D+

3D model-based design is transforming construction in the Netherlands

At the GeoBIM Building and Infrastructure Conference in Amsterdam, Ingeborg Ligtenberg, Senior Project manager for Mega Projects, described how the Dutch highway authority Rijkswaterstaat (RWS) is piloting a very different approach to highway construction.  Rijkswaterstaat is responsible for the design, construction, management and maintenance of the main infrastructure facilities in the Netherlands including the national road network, waterway network and sluices. 

Background

By way of background, the highway plan-design-build process that has been used over at least the past 25 years, starts off with a complete (re)survey of the project terrain to create a basemap on which the design is based. Existing as-builts in the area are rarely if ever consulted.  The deliverables from the design process are paper drawings which are used by construction contractors to build the project.  After construction is complete construction as-builts are submitted to the owner, but these are not "sealed", meaning nobody signs off on them putting their professional reputation behind their accuracy and reliability.  Practically what this means is that in the transportation industry, as-builts are treated as unreliable information and are rarely consulted.

At SPAR 3D 2013 Ron Singh, Engineering Automation Manager/Chief of Surveys, Oregon Department of Transportation (DoT) made a startling proposal that would stand the traditional highway construction process on its head. He proposed that the DoT should specify the data it needs and this data including BIM models and LIDAR scans would be collected by the DoT from the contractors during and after a highway construction project.  The data would be saved in a geospatially-enabled database maintained by the DoT.  One of the implications of this approach is that when a new project is initiated in the same area over 80 % of the necessary survey-grade information would already be available in the DoT's database, which would make the traditional complete resurvey unnecessary - saving time and money. 

As an example of how data reuse can save time and money, the Oregon DoT's first automated machine guidance (AMG) project was in 1997.   At that time in Oregon a few forward-looking contractors had invested in AMG technology foreseeing that it would save them money.  These forward-looking contractors were winning bids by lowering their estimates just enough to beat other contractors. They found that AMG technology did save a lot of money.  But the savings were going to the contractors, not to the DoT.  To redress this, starting in early 2015, the DoT began specifying on its bid web site that projects were "AMG ready" meaning that the DoT was making available provide all the data it had which could be download to see design BIM models, mobile LiDAR, overflight data, and so on.  The effect has been to drive bids down, because there is less risk for the contractor.  As a result the DoT is now sharing in the savings.

Rijkswaterstaat pilots

At the first GeoBIM conference in Amsterdam, Jothijs van Gaalen gave some real world examples of highway construction transformation based on GIS+BIM integration including using laser scanning/LiDAR for reality capture at the beginning of a design/construction project. 

The Rijkswaterstaat (RWS) board is convinced of the benefits of BIM and BIM/geospatial integration and decided initially to roll out BIM on four design-build-finance-maintain (DBFM) projects with the intention to require BIM on all DBFM-contracts in the future.

One of these projects is a highway expansion project for the highway connecting Schiphol-Amsterdam-Almere or SAA.  The objective of the SAA project was to improve the accessibility and quality of life within the SAA corridor. This includes the improvement of the natural and social environment and economic development and employment opportunities in the northern Randstad region.  A stretch of the highway is shown before and after the highway widening.

On many highway projects RWS had experienced information flow problems due to different formats and data models.  This not only necessitated major data conversion efforts, but also inhibited collaboration.  One of the goals of the SAA project was to improve information flow which was expected to also improve the flow of asset information for reuse during the maintenance phase of the highway.

A critical part of the RWS' new approach was the Information Delivery Specification (ILS). The ILS specified the general aspects of the required information including formats, standards, structure, and expected quality (timeliness and reliability) of the information.  The ILS was a contractual obligation that also required the contractor to accept new versions of standards on a yearly basis.  The standards that were adopted to ensure a uniform flow of structured information include VISI (an open communication standard for the construction industry based on XML), National Concept Type Library (CB-NL), Object Type Library (OTL), and COINS.

Today RWS is receiving a quarterly delivery of data from the contractor compliant with the ILS.  This is a remarkable achievement that required a lot of hard work during this first project.  But now that contractors are familiar with the standards and the ILS concept, it is expected that ensuring a uniform flow of information will be easier in future projects.  The next challenge is enabling this data to be used for asset management during the maintenance phase of the project.

Reusing data on future projects

 In response to a question, Ingeborg Ligtenberg said that this new approach was still in its initial phases but that the data would likely be made available to designers and contractors for future projects.  It would be made available on an "as is" basis, which sounds similar to how the Oregon DoT makes its data available on its bid site. 

As Ron Singh has pointed out, there is tremendous value in this data, but the value is only there if in the future engineers feel confident in using the data.  If very accurate and rich data were collected during and after construction including, critically, its lineage in the form of metadata about how and when it was captured, so that a future engineering user would have no reason not to trust its provenance and reliability, then there should be nothing to prevent the data from being reused in future projects - saving time and money.

Autonomous vehicles

One of the consumers of this digital highway information may be autonomous vehicles.  In-vehicle sensing is limited in its ability to enable safe and reliable driving under all conditions.  For example, on snowy roads, lane markings may not be visible. Such driving conditions will require the autonomous vehicle to be equipped with ultra-accurate maps to augment in-vehicle sensing systems.  Pre-loaded high-precision maps would enable autonomous vehicles to overcome many of these challenges – staying within lanes when wintry conditions obscure the lane markers. This will require sophisticated mapping systems and ultra-accurate and reliable DoT data to enable vehicles to navigate under challenging conditions.

Low carbon climate-resilient infrastructure requires private funding

A Report by the Global Commission on the Economy and Climate has recommended that since more than 60% of the world’s greenhouse gas (GHG) emissions are sourced from the world's existing stock of infrastructure, ensuring infrastructure is built to deliver sustainability is the only way to meet the COP21 goal  of keeping warming below 2°C and to guarantee long-term, inclusive and resilient growth. Infrastructure underpins core economic activity and is an essential foundation for achieving inclusive sustainable growth.  Investing in sustainable infrastructure is key for reigniting global growth and reducing climate risk.

The Global Commission is chaired by former President of Mexico Felipe Calderón and co-chaired by the climate economist Lord Nicholas Stern.

The report estimates that $90 trillion in sustainable infrastructure investment will be required over the next 15 years.  It stresses that this will require a shift in the world's construction and financial sectors to ensure that this money, which will be increasingly private, is spent on low-carbon, energy-efficient projects. 

Change in infrastructure spending required for a 2°C scenario (percentage change in expenditure over 2015-2030 compared to Business-as-usual)

The report has adopted a broad definition of infrastructure which includes both traditional types of infrastructure; energy, public transport, buildings, water supply, and sanitation and also natural infrastructure; forest landscapes, wetlands and watershed protection.  Infrastructure underpins core economic activity and is an essential foundation for achieving inclusive sustainable growth. It is indispensable for development and poverty elimination, as it enhances access to basic services, education and work opportunities, and can boost human capital and quality of life.

Investment is needed to replace ageing infrastructure in advanced economies and to accommodate higher growth and structural change in emerging market and developing countries. This will require a significant increase globally, from the estimated US$3.4 trillion per year currently invested in infrastructure to about US$6 trillion per year. The Global Commission found that it does not need to cost much more to ensure that this new infrastructure is compatible with climate goals, and the additional up-front costs can be fully offset by efficiency gains and fuel savings over the infrastructure lifecycle.

Importance of private infrastructure financing

Transforming the financial system and its intermediaries is essential to scaling up sustainable infrastructure finance.  Public finance and investment will continue to play a critical role, particularly in low-income countries.  But large amounts of private capital are needed and this will only flow if the right market signals are present within the financial system. The report suggests that reforms are needed in the financial regulatory system as well as other policy reforms, specifically, to price carbon to level the playing field between sustainable and unsustainable options in order to encourage private investment in sustainable infrastructure.

Private investment requires financial returns.  Efficiency in construction and operation and maintenance of infrastructure is essential. Increasing productivity drives technological innovation. Many governments as well as private sector engineering and construction firms have identified building information modeling (BIM) as a key foundation for increasing construction productivity.  Owners are beginning to recognize the benefits of full lifecycle BIM especially when augmented by geospatial technology.

Energy

The report estimates that energy infrastructure will require investment of US$25 trillion, or nearly a third of total core infrastructure investment over the coming 15 years. In addition energy efficiency will require investment of about the same magnitude.  Building sustainable energy infrastructure encourages economic growth, reduces air pollution and greenhouse gas emissions, and plays a key role in building resilience.

Energy efficiency

The report found that energy efficiency is a highly cost-effective way to manage demand and reduce the investment requirements for overall energy supply. Increasing energy efficiency in industry, buildings and transport could achieve up to half of the emission reductions needed globally to peak greenhouse gas emissions by 2020 according to the IEA.  Energy efficiency investments in IEA member countries since 1990 have avoided US$5.7 trillion of energy expenditure.  The IEA estimates further investments could boost global GDP by US$18 trillion by 2035, increasing growth by as much as 1.1% per year.  Energy efficiency measures are estimated to create up to three times as many jobs as fossil fuel supply investments per dollar of investment.

A key recommendation of the Global Commission’s 2015 report was to scale up commitments by development banks working with governments and the private sector to invest at least US$1 trillion per year by 2030 in clean energy, including energy efficiency.

Air pollution

Outdoor air pollution, much of which is associated with fossil fuels, is linked to nearly 4 million premature deaths per year. In China, it is estimated that air pollution killed around 1.6 million people in China in 2013. In India, the air pollution toll in 2013 stood at 1.4 million deaths.  In Europe, coal plant emissions account for more than 18,200 premature deaths, about 8,500 new cases of chronic bronchitis, and over 4 million lost working days each year.  Analysis for the Global Commission shows that the health and mortality burden of air pollution amounts to as much as 4% or more of GDP in some countries. Recent analysis by the OECD has found that globally air pollution-related healthcare costs alone are projected to increase from US$21 billion in 2015 to US$176 billion in 2060.

Cities

It is estimated that by 2050, two-thirds of the global population will live in cities, and over 70% of the global demand for infrastructure over the next 15 years is expected to be in urban areas.  How cities develop is important both for growth and for climate change.  The report emphasizes that investment in sustainable infrastructure is essential to make cities inclusive, safe, and resilient.

Global Commission

The global commission is supported by governmental and private sector leaders: Sharan Burrow, General Secretary, International Trade Union Confederation (ITUC), Australia; Suma Chakrabarti, President, European Bank for Reconstruction and Development (EBRD); Helen Clark, Administrator, UN Development Programme (UNDP); Former Prime Minister of New Zealand; Jamshyd Godrej Chairman and Managing Director, Godrej & Boyce Mfg Co. Ltd., India; Stuart Gulliver, Group Chief Executive, HSBC Holdings plc and Chairman of The Hongkong and Shanghai Banking Corporation Limited; Angel Gurría, Secretary-General, Organisation for Economic Co-operation and Development (OECD); Chad O. Holliday, Chairman, Shell, United States; and Sri Mulyani Indrawati, Finance Minister of Indonesia

HxGN Live: Hyper-spectral camera mounted on a UAV for precision agriculture and other applications

The typical multi-spectral camera used on satellites and airborne platforms captures three or four bands - typically the Landsat bands: green, red and near-infrared.   But hyperspectral cameras can capture 270 bands and are light enough to be mounted on a UAV.  At the annual Hexagon conference HxGN Live in Anaheim, Headwall Photonics gave an overview of the capabilities and advantages of hyper-spectral remote sensing.  Headwall and Leica Geosystems have partnered to offer a complete UAV-based hyper-spectral package which includes the Aibotix X6, Headwall hyper-spectral camera, storage, IMU/GPS, and software. 

A hyper-spectral camera passes the light it captures through a diffraction prism that breaks the light into a spectrum that ranges from infrared through visible to ultraviolet.  The Headwall camera records 270 bands ranging from the infrared through the visible spectrum. This broad spectrum allows the chemical composition of materials to be detected.  Spectral fingerprints can be defined based on chemical composition which enables detection of different species of trees, stages of crop development, disease incidence in crops, specific chemical compounds in mining tailings, and many other materials.  The Headwall camera can capture up to 300 frames per sec.  That generates a lot of data.  The key to extracting information from this volume of imagery is spatial analytics software which is part of the package.

The Headwall Leica Geosystems package is an integrated multi-rotor UAV and visible and near infrared (VNIR) hyper-spectral imaging system with software control.  It includes Headwall’s Nano-Hyperspec VNIR sensor with the Leica-Geosystems Aibot X6 multi-rotor UAV. The Nano-Hyperspec VNIR Sensor captures bands in the wave length range 400-1000 nanometers (nm) with a 12 bit pixel and a maximum frame rate of 300 Hz.  It also includes 480GB of on-board storage and an attached GPS/IMU. 480 GB corresponds to about 130 minutes of imagery at 100 fps.

Typical applications of hyper-spectral imagery are
precision agriculture, mineral exploration, and civil engineering.  Specific examples of the application of the technology include marijuana plant detection, crop health monitoring, citrus species mapping, invasive species monitoring, growth cycle and monitoring of crop yield, mine tailings monitoring, storage tank leakage detection, environmental monitoring, disaster monitoring, and mapping urban areas. 

A fascinating example is monitoring crop health where a scan of a field shows areas where the plants are not doing well.  This enables the selective application of fertilizer to only those areas where it is required.  Similarly a hyper-spectral scan reveals areas of insect infestation, which enables the selective application of insecticide.  Another application that would interest grape growers in the wine industry is using hyper-spectral imaging to identify and monitor leaf roll virus infestations.

Hyper-spectral imaging can also be used in an urban environment roof type mapping, insurance assessments, identifying types of road and monitoring road conditions, monitoring tree conditions in urban forests, and for urbanflash flood and disaster preparation and monitoring.

The key differentiator of hysper-spectral imaging is much greater precision in recognizing specific chemical compounds whether in plants, rocks, mine tailings, or urban environments.  However, the price is managing a much greater volume of data. Spatial analytics is key to extracting information from the huge volume of data collected in the field.

SPAR2016: Digitalizing highway construction at the Oregon DoT

At SPAR 2013Ron Singh, Engineering Automation Manager/Chief of Surveys, Oregon Department of Transportation (DoT) made a startling proposal that would stand the traditional highway construction process on its head by making the post-construction as-built survey the critical source of reliable survey and asset information in the form of a geo-referenced 3D BIM model (also known as horizontal BIM, heavy BIM, VDC, civil information modeling, BIM on its side, and BIM for infrastructure) and associated scanned and other data.  One of the implications of this is that when a new project is initiated in the same area over 80 % of the necessary survey-grade information is already available in the DoT's geospatially-enabled database making the traditional complete resurvey unnecessary - saving time and money.   Ron foresees that one of the consumers of this digital highway information would be autonomous vehicles.

This year at the Spar 3D 2016 Expo and Conference in Houston I had a chance to sit down with Ron to see how his and other DoTs have progressed on his vision since then.

Ron, the last time we chatted you mentioned you were organizing an event for Western State DoTs in July 2014. How did that go?

It was very successful. I did this on behalf of the Federal Highway Administration (FHA) who approached me and said they planned to do an event similar to the one I organized in 2010 which was called Design to Dozer. In that one we used digital technology including automated GPS-equipped dozers, graders, and excavators to build a sub grade of a road.  It was well attended and it really made a change for my DoT.

The FHA asked me to do another one to promote the use of intelligent construction technologies by providing information and training relating to 3D design, automated machine guidance (AMG) and related technologies for highway construction.  This time I decided to go all the way to paving including ditches, culverts, and drainage. I called it Design to Paver.  We invited the DoTs from the18 Western States plus Florida, North Carolina, and Michigan.  From each Dot we invited their chief engineer and their chief construction engineer.  Altogether we had about 250 people at the event.  It was a two day event and combined classroom work in the morning with on-site in afternoon.

This event included building a demonstration road but I wanted to build one we wouldn't have to tear up and restore. I found a piece of land that was a training facility for the Oregon National Guard. It was a big complex of about 40 or 50 acres and had been a training site since WWII. I went out to see the site and met with them and their environmental people.  I sketched out a plan for a road about 1500 feet long. I picked a site where the road would gain 200 feet in elevation because I wanted it to be fairly complex. I wanted to make some radical curves with superelevations and exaggerated superelevations, so we would have some drainage issues to deal with.  I wanted it to look like a real road and have some interesting geometry. No underground utilities. I wanted to take it from bare earth to a finished 1500 foot road with several curves, ditches, culverts, and paved in less than 2 days.  I don't know exactly but with a traditional approach this would probably have required about a week's work.  We could actually have done it in less than two days, but we were demonstrating.

We did all the surveying for the site months in advance.  We also had a geo-referenced 3D design model with rendering and animation.  We brought in the construction equipment the day before.  We had training sessions for the operators on safety including unexploded ordnance since this was a military training site.

I chunked the 1500 feet of road out in pieces and choreographed the whole thing.  For example, the dozer operator on each section of the road did not break ground until the audience showed up.  They stepped out of the buses and on their badges they had colour codes which allocated them to a team of about 25 people. Each group would go to station A at staggered times.  At station A they would be showed automated dozer operation. Then at a certain time they would go to station B and so on. It was very well coordinated.

What impact do you think the event had on the highway engineers who attended ?

It was amazing - the feedback through the emails I received was so positive.  People still talk about it. If you go to any of the DoTs who attended, the staff who had been there were just amazed.  One of the reasons is that what we were able to show them was something that would have taken a single individual many, many years to see, because none of them had experienced a project with automated equipment from breaking ground to paving.  They were able to experience it all in 2 days. We had all types of equipment, dozers, excavators, curb machines, paving machines, and so on.  We even had unmanned aircraft system (UAVs) for monitoring the construction and ground penetrating radar systems for detecting underground utilities.

Is there a way to find out more about this event?

We created a web site DesignToPaver.org which we intend to leave up for two more years. There is a guide book which you can download. It has more than the agenda. It has all the names of all the companies and  presenters.  If you would like to ask somebody questions, you can find that information in there.

Do you think the road map and time frame for 3D models and construction automation that you outlined is still realistic or is it accelerating?

The technology behind autonomous vehicles is really accelerating as is the engineering automation of highway construction, but the social and legal aspects of highway construction may be the barriers that are keeping it in check. If it were only up to to the technology we would be there much quicker, but I think the social and legal aspects are actually providing good balance, by making sure we are not breaking something that we can't replace.

You mentioned the legal aspects, do you think new legislation will be required to enable to the full digitialization of highway construction and maintenance ?

As long as you put a box around the question and restrict it to engineering automation, not autonomous cars, I think we are well-suited to get there with the existing laws.  I could imagine that to further capitalize on this technology, legislation to register another profession, such as remote sensing, might be beneficial.  I don't necessarily feel that it has to be a surveyor's function to do, let's say, mobile mapping or UAV mapping. I feel it needs to be somebody who is licensed, who has gone through training, who is able to prove that they know how to do this work, and who maintains ongoing professional development training.  The key requirement is that they stand behind their products in a legal sense.

There are many different technologies for capturing data now, even a consumer digital camera can be used to create a point cloud. If I  am a highway engineer how do I decide what data I can rely on for designing a highway ?

The data that has the level of rigour required for engineering design is what I call authoritative data. As an example, if you intend to use a boat with a sonar system and cameras for underwater bathymetric surveying in search of treasure, that's fine. I don't think anyone should regulate that. But if you are going to use the same tools to model bathymetry for the design of a public bridge, then it better be authoritative.  It better be data that you can track the lineage and source of.

One of the key questions I have is about the database of digital data including, in the case of Oregon, data from overflights and mobile LiDAR for the entire state highway system plus the digital as-built models and other data from actual construction projects.  Who do you see is responsible for maintaining this database including updates from ongoing highway maintenance work ?

In Oregon we think we at the DoT are responsible for maintaining it. As long as it is inside our corridor, it was either created by us or by our agents, companies that we hired. There is tremendous value in this data, but the value is only there if in the future people feel confident in using the data.  For that they need to know its provenance.  For example, if you consider our as-built plans, we have had these for generations, but we don't rely on them for anything but to help us find things. We don't rely on them for survey data, because the current as-builts were never created with that in mind.

If we collected very accurate, very rich data and we didn't track its lineage including metadata about how it was captured and when, a future engineering user would legitimately say I don't trust it, because I have no narrative about this data, for example, even something as fundamental as what coordinate system it is in.  But if we can compile all this information together in the form of comprehensive metadata that gives the future user trust in its provenance and reliability, then there should be nothing to prevent the data from being reused.

Are contractors who bit to build roads in Oregon interested in the data you are collecting - do they ask you for it ?

For construction project work, when we use mobile LiDAR or terrestrial laser scanning, we are very careful about precision, for example, we put in ground control points. We do the entire survey in such a way that we end up with a certain level of accuracy, a certain level of point density, and a certain level of reliability. 

When we collect mobile LiDAR data for the whole state, it is impossible to do collect data with the same rigour for the whole state.  We don't put ground control points in and we don't do other things that are required for high precision and reliability.  The data is still very usable for many things, but it is not authoritative, high accuracy survey data, that is required for engineering design.

If you are planning a project in an area where there was a previous project with a digital model, high precision data including ground control points and high density laser scans, would you tell contractors planning to bid on the project about the existing data ?

Absolutely. If we don' t tell them then we might as well as had not collected, documented (metadata) and stored it. We have to tell them because it helps drive the cost down.  To give you some background, our first automated machine guidance (AMG) project was in 1997.  In Oregon some contractors had invested in AMG technology and wanted to use it. In those days contractors who had done this were winning the bids by lowering their bid just enough to beat the other contractors.  When they did the work with AMG technology, they saved a lot of money, but most of the savings was going to the contractors, not to the DoT.  To redress this, starting in early 2015, we began specifying on our bid web site that projects were "AMG ready". We provide all the data we have which they can download to see the models, mobile LiDAR, overflight data, and so on.  The effect is to drive bids down, because there is less risk for the contractor.  Asa result we at the DoT are now sharing in the savings.

Is Oregon alone among DoTs in applying these technologies or are other DoTs also applying them ?

There are about five or six that are leaders in using different aspects of digital technology.  For example, the Wisconsin DoT is focusing on 3D design - they are ahead of us in this technology. Michigan is leading in some areas,  for example, data storage and data management. We met with them because we were interested to finding out what they were doing with engineering data management.  In return they were interested in what we were doing with 3D design and AMG. Utah his way ahead of most DoTs in asset management.  None of the DoTs including us are applying every aspect of 3D digital highway construction and maintenance.

Are there any ROI studies demonstrating the financial benefits of these digital technologies ?

We are doing one. We have a research project we call the OSU Engineering Automation project with Oregon State University in Corvalis.  They are studying our entire engineering automation process. They have already put a good project team together.   The final report is due by next April, about a year from now.

SPAR 3D: Virtual reality, scan to BIM, infrastructure models, and mapping indoor spaces

I have just experienced two remarkable days at the Spar 3D Expo and Conference in The Woodlands, Texas, in itself a remarkable location near Houston.  I will blog in more detail in the near future about the things and people I had the opportunity to experience over the past two days.

First of all, virtual reality.  Everyone is aware of the Oculus Rift (see the new review at CNET if you aren't).  There are other technologies that have come or are coming from Samsung, Sony and Google.  But at SPAR 3D the most remarkable VR technology I personally experienced was what David Smith, CTO of Wearaility, talked about and allowed me and others to try - a set of incredibly light-weight glasses with acrylic lenses designed to work with a smartphone that gives you a virtual experience comparable to IMAX without the IMAX theatre.  I tried it and found it an absolutely amazing surround experience.  And if you have wondered where Michael Jones, of Keyhole and Google Maps fame, has landed, he is now CEO of Wearality.

Secondly, my personal passion is the geolocation of underground utilities.  At SPAR 3D Mark Klusza, Founder & CTO, Real-time Metrology, Inc. described a major new advance in ground penetrating radar (GPR) technology he has developed that can detect a 3/4 inch rebar (among other things) four feet below the surface in clay.

For existing buildings creating a BIM model remains an art rather than science.  But at SPAR 3D a number of new technologies were presented that can help automate, improve the quality, and speed the process.  Foremost among them in my mind was presented by Nicolas Arnold, VP of Product Development, at SKUR.  With SKUR's technology, which runs in the cloud, you can compare the 3D model that you have developed with the point cloud you derived the model from and identify all the points of significant variance.  The same technology can also be used during a construction project to compare what has been built to what was designed - even when the building is only partially completed.

Ron Singh, Engineering Automation Manager/Chief of Surveys, Oregon Department of Transportation (DoT) described progress on his remarkable vision of how automation is upending how we design, build and monitor, operate and maintain U.S. highway systems.  Richard Arrowsmith, Asset Information Group Team Leader, Highways England described the digital model of the English highway system.  Stan Burns, President, Integrated Inventory, who has just retired from Utah DoT, described Utah DoT's comprehensive database of every piece of highway furniture and signage in the Utah highway system.  Together these represent the future of the digitalization of national highway systems.

A new company, Indoor Reality, is the third startup of Dr Avideh Zakhor, who is on leave from the University of California Berkeley.  Her new device is a backpack loaded with sensors including lasers and infrared that enables you to map the interior of buildings by simply walking through a multi-floor building, including up and down stairs.  The resulting data can be used to automatically generate floor plans and 3D models and captures everything you need for an Energy Plus analysis.

Mark Shell and Darrell Gadberry, both of the City of Fort Worth Water Department, described a very versatile device with laser and sonar and camera devices on board that can be used for sewer line inspections of small and large diameter sewer pipes.  Fundamentally this device enables condition-based maintenance of municipal sewer systems.  Mark and Darrell reported that it has saved the Fort Worth Water Department $42 million.

I was able to interview Dr Avideh Zakhor, Nicolas Arnold, Ron Singh, Larry Kleinkemper of Lanmar Services who specialize in creating BIM models for existing buildings, and Greg Bentley of Bentley who has a broad, forward-looking perspective on the "digitalization" of infrastructure.