Between the Poles

Above and underground infrastructure comprise Rotterdam's open digital twin

The City of Rotterdam is developing a digital twin of the city.  3D vector and image data of the above- and underground infrastructure, zoning regulations and city by-laws, data streams from real-time sensors, applications and simulations will comprise Rotterdam's digital twin. It is envisaged that the digital twin will enable the city to assess the impact of proposed changes in bylaws and regulations by simulating them before they are implemented. Rotterdam, and in fact the whole of the Netherlands have adopted transparency and open data as a pillar of government and are basing 3D models on open national and international standards, intend for the 3D data to be open and accessible to the public, and intend to use open source software wherever possible.

Many cities have 3D models of their aboveground infrastructure; buildings, transportation networks, parks, and other infrastructure typically captured in 2D or 3D imagery from overflights or in 2D GIS maps.  Imagery is not very smart and extracting 3D vector objects such as building footprints, heights, profiles and transportation networks requires sophisticated post-processing.  Even more importantly, underground infrastructure is usually neglected, even though it is recognized that water and wastewater, energy (gas, electric power, and district heating),  and communications (fibre and copper) networks provide the life blood of the city.

Rotterdam has been developing its 3D model, which underpins the digital twin, for several years.  Currently it contains street furniture such as street lights, trees and underground parts of street furniture, a large scale basemap, a complete map of underground networks of cables and pipes, buildings, and non-physical content such as zoning regulations and  bylaws.  The 3D model is kept in sync with data repositories maintained by city departments and others.  In the future it will also include real-time data streams from a variety of sensors. 

The 3D model is open, based on open standards and many of the applications that have been developed for it are open source projects. For example, visualizations rely on Cesium, an open source initiative which has just announced support for visualizing underground utilities, mines, and geological layers. The rationale for the data being based on open standards is that an standards-based environment creates an open data commons where developers can devote their time to developing innovative applications rather than spending time converting data from one format to another.

The Dutch 3D standard is an integration of CityGML Application Domain Extension (ADE) and the existing national IMGeo standard for large-scale representations of roads, water bodies, and land use. The Dutch Kadaster, Geonovum, the Netherlands Geodetic Commission and the Dutch Ministry of Infrastructure and Environment initiated a "3D Pilot NL" to develop a national 3D standard.  Existing intnernational standards including CityGML(Open Geospatial Consortium), IFC (buildingSMART), and others were investigated. The findings of the pilot  resulted in a national 3D standard implemented as a CityGML Application Domain Extension (ADE) that integrates the latest version of the existing Dutch national standard IMGeo 2.0. 

Current applications (virtualcitySYSTEMS, ESRI, IMAGEM and Unity) use the 3D model as input for analytics and visualization. One can imagine many use cases for the data in the Rotterdam 3D model. Among these three use cases have been identified as low-hanging fruit; climate adaptation, viewsheds and energy performance of buildings. 

Much of Rotterdam is below sea level and managing rain water accumulation during heavy rainfalls is a major issue. which affects building basements, subway entrances, low-lying roads, electrical transformers, and other infrastructure.  To address this problem, the concept of "water squares" has been developed and the 3D model used to determine the optimum location for these..  Water squares are open areas where water can safely be allowed to accumulate during heavy rainfalls to avoid overstressing storm sewers. 

In another application of the 3D model data, building data can be filtered and streamed to energy performance applications that provide insights into how buildings perform.

Open standards and open data also enables the sharing of data among cities in the Netherlands.  The Dutch Kadaster has developed an open 3D map of the whole of the Netherlands, with scales ranging from 1:500 to 1:25,000 in collaboration with academic, private sector, and public sector partners.  All data in the 3D map is built on the basis of open standards.  The map was created by combining existing 2D data with the Elevation Model of the Netherlands resulting from a LiDAR survey which covers the whole of the Netherlands with a resolution of eight elevation points per square meter.

Open data and open standards also relate to a critical issue that is of growing importance. Many governments have adopted transparency and open data as a cornerstone of democratic policy.  But often digital twins for smart cities are being developed by private-sector firms or by public-private initiatives.  As William Davies points out in Nervous States, "the IT infrastructure used to build e-geovernment projects or "smart cities" comes from the commercial sector, and the companies that provide it often retain control over the data that results." 

For example, recently a joint Alphabet/Google/Sidewalk Labs and City of Toronto initiative to develop a smart city on Toronto's waterfront foundered and was cancelled.  One of the reasons, it is argued, is that a technology giant such as Google (or Facebook, Amazon, ...) is not a vendor in the traditional sense, but a "surveillance capitalist" that collects and analyzes behavioural data captured from users for its own economic imperatives.  In the context of a smart city the opportunities for data collection from citizens and from systems such as water and wastewater, transit, waste removal, traffic management, and others are immense.  If this data is appropriated and becomes proprietary, it puts a private entity in a position to supplant democratic government, which doesn't have access to this huge volume of data and the analytics which extracts information from it.  Relying on open standards and ensuring that data is open and accessible to the public are essential to avoid citizens' and municipal data becoming proprietary.  Furthermore, using open source software and, I would add, open training data for machine learning applications helps avoid the problem of the analytics also becoming proprietary.

Based on a Webinar by Christian Veldhuis, Advisor City of Rotterdam, Rick Klooster, CCO, FUTURE INSIGHT and Antje Kunze, Director Sales & Marketing, virtualcitySYSTEMS.

White paper on reducing damage to underground utilities during construction

It is estimated that unreliable location information about underground infrastructure represents a $50 billion drag on the U.S. economy. Comparing the United States and Japan reveals a startling difference in the annual incidents of underground utility damage during construction - over 400,000 incidents in the U.S. and less than 150 in Japan. Clearly something can be done to reduce underground utility damage. Important advances in technology now make it possible to capture ground penetrating radar scans at highway speeds; simultaneously capture of above and below-ground scans using LiDAR and GPR to create a complete pre-design survey; generate centimetre accurate as-builts from a video taken with an Android smartphone; map underground pipelines for upto two km using inertial locating; detect underground objects 30 feet deep with acoustic sensing; and integrate existing 2D and site investigation results in a 3D model visualized in augmented reality. On-line systems for sharing information about the location of underground infrastructure have been implemented to manage underground utilities in the public right of way. 

Many jurisdictions in the Americas, Europe, and Asia Pacific have implemented policies and organizational structures for capturing and sharing information about the location of underground infrastructure.  Statistics reveal that a comprehensive approach involving policies, technologies and construction practices is required to be successful for continuing improvement in reducing utility damage during construction. Based on a review of international initiatives for managing location information about underground infrastructure, a set of 25 recommended elements have been compiled that should be considered by anyone intending to to reduce the risk of underground utility damage during construction.

A white paper prepared by Geoff Zeiss and Dr. Sakura Shinoaki and published by the Geospatial Information and Technology Association entitled Reducing Damage to Underground Utility Infrastructure during Excavation assesses costs, benefits, technical advances, and case studies to develop 25 recommendations for reducing damage to underground infrastructure. The white paper can be downloaded for free here and a summary is available here.

Centimetre-accurate as-builts efficiently captured with a smartphone

Without accurate maps of underground infrastructure, every construction project has the potential to become a disaster site. Incidents of underground utility damage are more frequent than many people realize. In the U.S. there are between 400,000 and 800,000 incidents every year. Every year there are injuries and fatalities attributable to hitting underground infrastructure during excavations. Inaccurate and missing information about underground infrastructure is also responsible for construction project schedule and budget overruns. According to the Federal Highway Administration (FHWA) missing or inaccurate location information about underground utilities is a major source of highway construction project delays. The drag on the U.S. economy resulting from underground utility damage is estimated at $50 to $100 billion annually.

90% of underground utility damage during construction is attributable to (1) incomplete, inaccurate and out of date records maintained by utilities and telecoms and (2) data format issues including paper, digital ink (pdf) and raster (imagery) records. As one customer said, existing records are typically 15-30% off the mark. Another contractor reported that records are not updated after relocation of assets. The result of these and other issues is that excavators are often working blindly.

AVUS is a recent startup that has been incubated by Vinci Group , a French construction company with annual revenue of €48.1 billion. In 2019 Vinci ran 290 000 construction worksites. AVUS is a service company providing services for capturing accurate as-builts and visualizing underground infrastructure.   Recently I had a chance to chat with Sebastien Magat, the entrepreneur behind AVUS.

As-built service

AVUS have developed a reality capture solution that allows the creation of as-builts accurate to ± 5 centimetres from a video taken with an Android smartphone. Currently in use at 30 construction sites, the workflow is very simple; open the AVUS app on a smartphone, connect to a local RTK base station or use known control points, enter project and site information, use the app to create a video by walking along the open trench after installation of the utility, upload the video to the AVUS cloud site where a point cloud is created, georeferenced and scaled using RTK or known or recorded ground control points, and receive the as-built in the form of point cloud (.las), orthophoto, and CAD drawing (.dwg). A similar process can be used with utilities exposed during construction to amend and correct existing records.

The process of generating a DWG file from the point cloud is currently a manual process, but with the rapid advances in feature extraction technology, aided by machine learning algorithms and the compilation of training data by EPRI and others, in the future this process is expected to be increasingly automated.

A benefit of the AVUS as-built services is that it provides reliable records of underground infrastructure certified to ±5 centimetres horizontally and vertically. For new infrastructure this provides a reliable record of what was installed in the ground, in other words, "as-constructed" instead of "as-designed".  For existing infrastructure this enables discrepancies and omissions to be reported back to asset owners. The as-built records are PAS 256 compliant and integrate with existing utility GIS databases.

Another important benefit of the AVUS as-built service is that it does not add significantly to construction costs. It is very efficient compared to the time required to call a surveyor to the site - the AVUS video recording takes less time than it would take for the surveyor to drive to the site. Furthermore, the video approach works in every weather condition, even at night in artificial lighting. This enables the construction crew to continue with their work, instead of having to wait for the surveyor to get to the site and conduct the survey.

Augmented reality

AVUS compiles existing 2D and 3D records and site investigation results into a 3D model that can be visualized in augmented reality on a handheld.  An Android app has been developed that uses Google AR Core to project highly accurate holograms on a smartphone. Accurate positioning is achieved by connecting to a local RTK base station or by calibrating to known features in the field of view.

The app is easy to implement and use and it integrates with existing utility GIS databases. The benefits of this AR approach is that increases productivity by enabling more rapid location of underground networks and results in reduced risk of underground utility damage.

AVUS provides an important service that captures the location of underground infrastructure efficiently, improves the quality of location data, and makes the data easily accessible to engineers and contractors in an augmented reality view in the field.

Augmented reality enables simplified design workflows

Engineering design is typically done on a desktop computer, using CAD or BIM software. Models are viewed in 3D or 2D on a monitor. Increasingly they can be viewed in mixed reality.  At a SPAR3D conference three companies in the construction sector, Gensler (a collaborative design firm), McCarthy (a commercial construction firm), and Martin Brothers Construction (a heavy civil construction firm), demonstrated how they were using mixed reality on real-world projects.  For example, Martin Brothers demonstrated building a bathroom pod with Hololens. The bathroom pod was designed using a standard design application such as Revit or Sketchup, then projected onto the reality of a shop floor were it was constructed using the full-scale virtual reality projection of the design into the shop. The next step is bidirectional between the mixed reality environment and the authoring design tool. This makes it possible for changes made in the mixed reality environment to be visible in Revit or Sketchup where they can be incorporated into the design. It is also possible to reverse this process where design changes made in Revit or Sketchup are simultaneously visible in the mixed reality environment.

Stéphane Côté of Bentley has taken this one step further and suggested that the design and visualization process should really be a continuum involving design, markup, and visualization.  To enable this to happen Stéphane's team developed a simple set of drawing tools along with features that enhance drafting accuracy, such as angle, length and element snapping.  But most innovative was the ability to snap to real world objects in augmented reality.  This is illustrated in a video by designing an extension to a table.  To do this the designer simply snaps to the table in augmented reality and then designs the extension seeing exactly what it looks like including accurate measurements such as lengths and angles.

As a real world example, when an engineer needs to design a bypass system between two existing pipes, normally he/she would have to take measurements of the pipes or scan them using a laser scanner. These measurements would then be imported into a CAD application where they would be used as the basis for the design.  It might even be necessary to iterate this process once or twice to ensure the accuracy of the design.  With an augmented reality design tool such as that created by Stéphane's team the design could be done on-site, avoiding the initial measurement step and being able to see immediately if the design fits accurately between the existing pipes.

Global underground utility mapping market estimated at US$ 842 million

Markets and Markets has just released a report Underground Utility Mapping Market Global Forecast To 2024 in which it is estimated that the 2019 market size for underground utility mapping, which refers to a complete detection, processing/post-processing and visualization solution, to be US$ 842 million and growing at an annual rate of 11% (CAGR) expecting it to reach US$1.4 billion by 2024.  Currently underground utility mapping is estimated to represent over 15% of the underground utility locate market but it is forecasted to be growing significantly faster, 11% vs 6% CAGR. These estimates are stated to be conservative.

By underground mapping application M&M means an integrated hardware + software solution that detects and locates underground utilities and provides a way of processing/post-processing and visualizing the results using a map view, and increasingly a 3d map view, of underground utilities.  It relies on one of the remote sensing techniques for identifying, detecting, and labeling underground utility lines (telecommunications, storm sewers, water, natural gas, electricity, cable television, and fiber optics.)  GIS is specifically referenced because of its ability to assist with 3D data acquisition, management,  and visualization as well as image processing.  The study sited technological advances including the increasing availability of utility mapping and detecting applications and improving 3D viewing experience as important factors driving the growth of the underground mapping market.  M&M suggests that the non-technical drivers for the growing market are rising concerns over the safety of underground utilities and government initiatives for the implementation of utility mapping tools.

The M&M research focused on technological solutions, which includes ground penetrating radar, electromagnetic induction, and others with associated software and professional services including firms providing surveying and mapping, training and education and consulting services.  The vertical industries that were investigated include oil and gas, electricity, government and public safety, and construction.  The geographic focus of the study is broad including North America, Europe, and Asia Pacific.  The analysis does not include the revenue generated by the hardware from vendors like Leica Geosystems, Sensors and Software, and GSSI, but does include their software revenue.

The major vendors providing underground utility mapping solutions that were involved in the study include Hexagon Geosystems (Leica Geosystems), Sensors & Software Inc. (Sensors & Software), Guideline Geo AB (Guideline Geo), Geophysical Survey Systems, Inc. (GSSI), and US Radar Inc. (US Radar) and services firms Plowman Craven Limited (Plowman Craven), Cardno Limited (Cardno), Utility Mapping, Maverick Inspection Ltd. (Maverick Inspection), multiVIEW Locates Inc. (multiVIEW), , Vivax-Metrotech Corp. (Vivax-Metrotech), Enviroscan, Inc. (Enviroscan), Global Detection Solution Sdn Bhd (GDS), ProStar Geocorp, Inc. (ProStar Geocorp), and Underground Services, Inc. (SoftDig).

To put this in context an analysis also by Markets and Markets estimated that the global utility locator market amounted to US$ 5.17 billion in 2017 and predicted it would grow by about 6% per annum to US$ 7.50 billion by 2023.

Read more about locating underground infrastructure

Automating vegetation management for electric power transmission lines using deep learning with LIDAR

With FAA rules permitting beyond visual line of sight (BVLOS) UAVs available through Part 107 waivers the cost of capturing the data required for vegetation management for power lines, whether transmission or distribution, could drop dramatically.  However, manual post processing of the imagery to identify areas of vegetation encroachment, grow-ins and fall-ins, remains an error prone, tedious and time consuming process.  I have come across several firms (for example EPRI, SiteSee, Trimble RealWorks) over the years that have been working towards automating the process for identifying vegetation encroachments including grow-ins and fall-ins, but I have not encountered a utility that is using this technology operationally.  At Distributech 2019 in New Orleans I had a chance to chat with a startup Enview, based in San Francisco,  who described to me how their software has been implemented  over 7,000 miles of transmission lines last year to automate the identification of vegetation encroachment from LiDAR point clouds.  Furthermore, they expect the total distance covered to double this year.

Vegetation is a significant source of outages for many utilities. In some regions from one quarter to one half of all outages can be ascribed to vegetation.  Vegetation encroachment initiated the massive North American northeastern outage of 2003.  It is so critical for utility operations that utilities even have a vice president responsible for vegetation management.  It can be costly for utilities to identify areas of high risk vegetation encroachment which typically are carried out by manned helicopters.  Scheduling and routing crews to prune the trees that require attention is often sub-optimal adding to the cost of vegetation management.  Optimizing these processes can produce significant benefits in the form of fewer outages and reduced costs. I have blogged about Energisa in Brazil who used satellite stereo imagery to prioritize and optimize tree pruning and ComEd's pilot project in Chicago that uses LiDAR imagery to not only prioritize and optimize but also to change the pricing structure for procuring distribution tree trimming.

Enview applies machine learning, computer vision, and geospatial big data analytics to highly reliable feature extraction from LiDAR point clouds to identify the exact location, height and extent of trees and then to determine proximity to power lines and other utility equipment.  Utility GIS data can be integrated and results visualized using colour-coding to identify the level of risk for each tree.  This enables the utility to prioritize their tree trimming and optimize contractor routing.  In environmentally sensitive areas the visualization can be used to inform discussions with the various stakeholders responsible for different land uses.

At Distributech I had a chance to chat with Krassimir Piperkov, COO at Enview, who described how this technology has been deployed on more than 7,000 miles of infrastructure last year, and is on track to be deployed on over twice that much this year. He gave a very recent example of annual vegetation management analytics performed on 230kV transmission lines in California. These lines run through suburban and environmentally-sensitive, heavily forested areas.

3D visualization of a 230kV span (blue lines) with potential grow-in (yellow) and fall-in (red) vegetation that falls within client-determined clearance boundaries.

LIDAR data was acquired by a third party through a helicopter-mounted Teledyne Optech LiDAR sensor package. Four band imagery was co-collected during the LIDAR survey and registered into an orthomosaic of RGB and NIR bands. The data collection generated almost 2GB per linear mile. The Enview data analytics engine processed this data to (i) first classify the massive amounts of LiDAR data and (ii) then automatically calculate potential grow-in and fall-in vegetation points with high speed and accuracy. Insights and data were delivered to the operator not only via GIS-compatible file format for incorporation into existing workflows but also via an intuitive web portal accessible across silos. As a result of the superior accuracy and speed, the client was able to confidently prioritize vegetation prescription work to the areas that need it most and do so much sooner than otherwise possible with traditional methods.

The results being achieved with Enview's technology suggest that we are on the cusp of the widespread adoption and application of deep learning technology to automate specific tasks. The combination of UAVs for data collection and machine learning for feature extraction dramatically reduces the cost of inspections for utilities. This would permit them to be done more frequently than with manned aircraft and would lead to improvement in the reliability of above ground electric power and communications networks.

State DOT saves millions by creating 3D map of underground utilities prior to design

Unknown or inaccurately located underground utilities can make or break a highway construction project's schedule.  For the largest construction project in the state of Alabama's history, the Alabama Department of Transportation (ALDOT) used ground penetrating radar (GPR) and other technology to create an accurate 3D model of underground infrastructure prior to designing a replacement for one of the busiest interchanges in the state.  ALDOT estimates that the 3D model, which was provided to all contractors bidding on the project, saved millions of dollars and reduced the risk of budget and schedule over-runs.

The location of existing underground utility infrastructure is more often than not poorly known which creates significant risk for infrastructure and highway construction projects. Typically on highway construction projects, the right of way is chosen, the highway designed and the project readied for construction before the one-call centre is contacted and the affected utilities arrive to detect and locate their underground facilities, usually in 2D.  The utilities that have been detected are then moved to make way for building the highway.  Not infrequently the location of the identified utilities turns out to be unreliable and during construction other unidentified utilities are encountered both of which can lead to delays and to budget overruns.

I blogged previously about a gas line construction project that was a Year in Infrastructure Award finalist in 2015.  For this project a 3D model of the existing underground and above ground infrastructure was created before design and construction.  The benefit of having the 3D model was clear; during construction there were no utility strikes, there were no change orders and the project was completed in 7 weeks instead of the expected 10 weeks. 

At this year's Year in Infrastructure Awards in London an outstanding project submitted by the Alabama Department of Transportation (ALDOT) was selected as finalist. The I-20/I-59 Corridor is the largest construction project in the State of Alabama’s history with a cost of $750 million.   The project has a very tight timeframe.  ALDOT's Design Bureau’s Visualization Group was tasked to provide a complete 3D proposed model for building information modeling (BIM). The BIM model needed to provide the information necessary for multiple uses including: visualizations, design checks, construction analysis, clash detection, right-of-way negotiations, lawsuits, and aesthetics. The model had to be very accurate because it was intended to be provided to the construction bidders to enable more accurate cost estimates and lower bids.

One of the important challenges faced by this project was utility coordination because the interchange is situated in Birmingham's business district.  Realizing that underground utilities can make or break a project's schedule, ALDOT invested a considerable effort in locating utilities and creating a 3D model of above and below-ground utilities. Potholing and scanning with ground penetrating radar augmented existing as-built records to enable ALDOT to create accurate 3D model of underground utilities.  3D is important because what appear to be clashes in 2D may not actually be.  Also it was much easier to explain to utilities in 3D that a costly move of a utility line is not necessary.   ALDOT’s Visualization Group was tasked with detecting clashes between the proposed design with the existing conditions. Every contractor bidding on the project got the full 3D model.  Because of the reduced risk associated with underground infrastructure ALDOT estimated that it saved over $10 million by using 3D modeling of underground and aboveground utilities.  To date the project is on budget and on schedule. After completion of the project ALDOT plans to retain the 3D model which can be reused for other projects in the same area.

ALDOT found other advantages of 3D modeling as well.  Public and stakeholder perspectives of the project was a major concern and using the 3D model allowed the Visualization Group to provide accurate photomatches, renderings, and animation to relay projects impacts to all stakeholders.  While the project is under construction the 3D model is also being used by inspectors in the field so that construction details are relayed accurately to the workers.

This project provides further evidence of the significant benefits of a 3D model of underground infrastructure which can be used during design and construction to reduce the risk of utility hits, reduce the cost of utility relocates, and avoid unnecessary utility relocates.

Mobile app uses consumer 360° cameras for visual construction documentation

At this year's BIMForum in Las Vegas, I came across an intriguing new company in the construction technology space called HoloBuilder.  I had an opportunity to follow up with Harry Handorf to gain a better understanding of how their solution, a combination of an app for iOS and Android with cloud processing, makes it possible to use a consumer 360° camera such as Ricoh Theta cameras to quickly document progress on a construction site.

The handheld app is called JobWalk, which makes it possible after uploading a floor plan to walk through a job site placing the camera and tripod in the middle of each room to take a series of 360° photos.  After the photos are uploaded to the HoloBuilder cloud, virtual tours of the construction site are available to anyone with a web browser on their platform.  In the virtual walkthroughs, dimensions can be measured with 99% accuracy.  The only input data that is required to make this possible is the distance between the camera lens and the floor.  Plug-ins are available for Revit and Navisworks which make it possible to view as-designed BIM model renderings side-by-side with as-constructed.  It is also possible to time travel over the lifetime of a construction site comparing images taken at different times to track construction progress.

HoloBuilder introduced its technology to the broad market in 2014. In 2015 it won the grand prize at the RICOH THETA Developers Contest.  Since then it has focused its solution entirely on the construction market.  It even offers a solution for real-time construction monitoring.  Harry told me that it has many customers in the construction market including the likes of Skanska, McCarthy, AECOM, Hensel Phelps, Mortenson and others.

As a practical example of how HoloBuilder is used, Skanska is currently building a 185,000-sq. ft. hotel with a construction contract value of $52.3 million.  Skanska is using HoloBuilder on the project to document existing site conditions, share progress with stakeholders and make decisions requiring the agreement of remote stakeholders scattered around North America. One of the use cases where Skanska found a lot of value with HoloBuilder is capturing images as part of pre-pour inspections - necessary inspections that occur before pouring concrete.  The challenge is to document all the in-slab deck conditions prior to pouring in a very limited window of time.  Cables, heating and ventilation, plumbing and electrical, rebar, and other equipment need to be captured.  Skanska’s Project Engineers use HoloBuilder to take 360° photos in conjunction with the bi-monthly concrete pour cycle.  Each of the floors scheduled for concrete pouring is captured and uploaded during a one-hour site walk using the JobWalk app and a standard 360° camera. HoloBuilder estimates that one 360° photo contains approximately the same visual information as 11 single lens photos. The resulting HoloBuilder project provides information about the contents and layout of materials in the slab including mechanical, electrical and plumbing as well as the rebar size, spacing and number. The images also contain additional context related to site conditions, weather, and location where the 360° photos were captured. Skanska is able to share the site conditions with the architect and structural engineer as a supplement to the inspection.  With HoloBuilder, Skanska’s project team reduced photo documentation time by 50%.  It also was able to offer a valuable deliverable that could be shared with the owner, architect, facilities management team and other project stakeholders.

This is another example of how consumer devices are changing the construction industry. The number of startups like HoloBuilder, Dot Product, Enscape, Revizto, Fuzor, Skur, Cupix and others in the construction industry is increasing and many are using consumer technologies such as gaming engines (Unity, Unreal Engine). As new technology gains traction in a $10 trillion industry that has traditionally been ranked at the bottom of R&D spending and digitization, construction productivity will rise and infrastructure will become increasingly attractive to private investors who are expected to become the source of most of the money required to develop, renew and maintain our infrastructure for the next two decades.

Bentley's digital twin capabilities for cities includes underground infrastructure

Proposed West Link tunnel under central Gothenburg

More and more cities are recognizing the tremendous benefits that a digital twin provides to its planners, private architects and engineers striving for compliance with city ordinances and by-laws, and for communicating with the public.  I remember several years ago when the Prime Minister of Singapore in his annual National Day address included a 3D visualization of what the proposed Marina Bay towers would look like - perhaps the first time 3D visualization has been used to inform citizens on a national scale.  At the Year in Infrastructure 2018 in London, Bentley announced the acquisition of Agency9, a Swedish company that provides web and mobile solutions for 3D visualization of geographic information and maps for collaboration and communication with its product CityPlanner (now OpenPlanner).  With this acquistion, Bentley augments its competitive positioning relative to CityZenith (SmartWorldPro), ESRI (CityEngine), and Autodesk (Infraworks) and open source projects such as mago3D in offering a 3D visualization and analytics platform for the massive data sources required for cities including underground infrastructure.

The requirements of a comprehensive platform for city visualization and analysis for planners, architects and engineers, and the public are the ability to support OGC (KML, CityGML and others), BuildingSmart (IFC), defacto (Shape, DWG, DGN, Revit and others) and other standards such as GeoJSON making it possible to integrate GIS, CAD, and BIM (vertical and horizontal) data, content creating and editing tools, for example, to allow buildings to be extruded from footprints and rooves and facades customized, support for open data sources such as OpenStreetMap, ability to integrate satellite, aerial and ground photo imagery and LiDAR point clouds,  support for both above- and below-ground infrastructure, gaming engine support (such as Unity or Unreal Engine) for interactive 3D visualization, cloud processing for managing, processing and querying massive amounts of data, web and mobile access, and APIs for analytics.

From a technical perspective the acquisition made a lot sense because Bentley's ContextCapture (from the acquisition of Acute3D several years ago), which creates 3D meshes for both inside and outside buildings from aerial and ground photo images and LiDAR point clouds, has been used by Agency9 for many years as the basis for its visualization capabilities.

Proposed West Link tunnel under Gothenburg

The West Link (Swedish: Västlänken) is a railway tunnel under construction under central Gothenburg. As an example of its support for underground infrastructure the City of Gothenburg used CityPlanner to publish a 3D visualization of the planned tunnel going below the entire city which showed stations with points of interest with images in a slideshow. During the first 24 hours that the project was published on its web site, it had over 10,000 unique visitors.

Documenting construction projects with a consumer 360° camera

Many see handheld scanners as revolutionizing the construction sector.  Portable and handheld laser scanners offer millimeter accuracy and can be used for engineering purposes.  Cameras with less precision are being used for rapidly documenting a construction site for a variety of purposes.   Most recently a couple of startups Cupix and HoloBuilder are lowering the bar by offering cloud software for the AEC industry to enable images of a construction site to be captured with a consumer grade 360° camera like the Ricoh Theta S or V. Walking through a construction site capturing 360° images along the way and uploading the images to the web is all that is required to create 3D walk-throughs of a site.  The 3D walk-through can be used to measure dimensions, it can be annotated, it can be compared with a BIM model, and 3D images captured at different times can be compared to monitor construction progress.

 

For engineering purposes, handheld laser scanners provide the requisite precision but are relatively expensive.  The scanning process requires time and produces large point cloud files and imagery that need to be managed and processed for visualization.

• The Leica Geosystems BLK360 weighs a mere 2.2 lbs (1 kg), is incredibly small 6.5 x 4 inches and lists for US$16,000.  It is a full laser scanner and imager, capturing 360,000 points/second with a range of 0.6 - 60 meters.  The scanner can be attached to a Microsoft Surface Pro tablet to visualize what you have scanned. Up to 16 of these can be used at the same time.
• Dot Product DPI-8 and DPI-8X (short range) are multi-sensor devices (IR laser and photo cameras).  Targeted at professionals, especially in the engineering and construction sector, the devices incorporate SLAM technology and works with Android handhelds.  All the processing is done locally on the handheld - no cloud processing is required.
• The Faro Freestyle can scan objects within a range of half a meter to 3 meters away with a resolution of  better than 1.5mm with a image point density of up to 45,000 points/m² at 0.5m and up to 10,500 points/m² at 1m.  The scanner is attached to a Microsoft Surface Pro tablet for visualizing the point cloud as you scan.

For rapid site documentation, Matterport allows you to scan a floor of a building in an hour or less, upload the captured data to the cloud (Amazon Web Services) and a couple of hours later have a fully-rendered 3D walk-through that can be accessed over the web.  It was originally targeted on the real estate sector but recently has been showing up at construction industry and other events including the recent BIMForum in Las Vegas.  It provides a dollhouse view to see a whole floor at once, an inside view for walk-throughs, and a floor plan view for a traditional top-down perspective. It relies on IR and photo cameras and multiple scans in each room - scanning every 1.5 to 2.5 meters.  It is capable of rough measurement, but not for engineering or construction purposes.  Its biggest advantages are the speed with which you can scan a site and the small size of the resulting file which has to be uploaded to Matterport's cloud site.  In addition to real estate and construction site documentation, it is used by FM professionals who need a 3D context for facilities management.  For example, it can provide a mechanical room walk through with tags on different pieces of equipment and a way of directing anyone to any location in the facility.  The camera is several thousand dollars and cloud processing requires a subscription.

 

The most recent additions to the technologies that can be used to document a construction site are Cupix and Holobuilder which lower the bar by only requiring a consumer grade 360° camera such as a Ricoh Theta.  In the case of Cupix cloud processing requires a subscription.  Similarly to the Matterport, you walk through the site capturing images with 360° camera as you go.  The resulting images are uploaded to the Cupix site and the result is 3d tours or walk-throughs accessible over the web.  You can compare 3D tours taken on different days to monitor construction progress, compare a 3D walk-through with a BIM model (Revit or IFC) you have uploaded, and measure dimensions with a precision of 1% or better at 20 meters.  To reach this level of precision in measurement requires including a marker when taking pictures.  Alternatively if a dimension is known it can be used to scale the entire walk-through.  Cupix can use images captured indoors and outdoors.  Cupix says that if GPS metadata is available from the photos, they can georeference the walk-throughs and generate a KML file. I used Cupix with a low cost 360Fly4K camera and a Samsung Note 5 phone to capture the interior and a bit outside of a cabin in a nearby park.  It took a couple of attempts to get the correct spacing between camera placements in capturing 360° images, but after that it was straightforward. I uploaded a rough picture of a floor plan and I was able to walk-through the resulting 3D tour using the Cupix web viewer.  Without doing anything additional I was able to get rough measurements accurate to about 5%.  Cupix only requires a consumer grade 360° camera.  A subscription is necessary for cloud processing. Cupix is targeting a broad market including travel and hospitality, real estate, and arts and culture in addition to the AEC industry

 

I just came across HoloBuilder at the recent BIMForum in Las Vegas.  Like Cupix it only requires a consumer 360° camera. At first glance it appears to be focused on the AEC industry and makes creating and managing 3D tours of construction sites very easy.  It offers 360° SiteStream which is a unique live streaming solution for real-time construction progress monitoring in 3D that runs as a plug-in app on the Ricoh Theta V.