Between the Poles

Towards a digital twin of infrastructure: efficient reality capture of new pipelines

Traditionally when a new pipeline is put in the ground, either its location and depth is only approximately known or it has to be surveyed by a registered surveyor.  Now there is another alternative. At GeoAlberta 2018 is Edmonton Joseph Hlady of Lux Modus described how his company has developed a rig and software running in the cloud that allows this information to be captured in "near real-time" as the pipeline is constructed.  The rig includes LiDAR and photo cameras which capture the trench and the pipeline by driving along each section before the trench is filled to record a point cloud and photos.  The point cloud and images are uploaded for cloud processing and within minutes to hours a digital twin of the pipeline can be viewed by anyone with a browser that supports HTML5 including mobile devices.

By automatically detecting the top of the ditch and the top of the pipe, the location (x,y) and depth (z) of the pipe can be computed in real-time by the LuxGear or later in a GIS.  Lux Modus uses the Unity gaming engine to enable 3D walk-throughs in virtual reality (VR) of the pipeline after the ditch has been covered.  For example, this makes it possible for QA/QC people to use VR to identify weld points in the office rather than in the field.

Open source geospatial coordinate sytem libraries are being revised

A "GDAL Coordinate System Barn Raising" has raised $144,000 to enable improvements to coordinate system libraries needed for the GDAL, PROJ, libgeotiff, PostGIS, and Spatialite open source toolchain. Support came from 25 players in the geospatial market including SAFE Software and ESRI, Boundless, Azavea, LINZ, MapBox, Riegl, Radiant Solutions, and others.

The objective of this effort is to provide GDAL, PROJ, and libgeotiff with modern capabilities including support for the updated OGC well-known text standard WKT2, replacing CSV tables with SQLite-based databases, not requiring projection transformations to pivot through WGS84, and preparing for the new North American datum NATRF2022 and NAPGD2022 which I blogged about previously.

The use of a SQLite-based database for coordinate systems definitions will allow projects to add more capability, transition custom project coordinate systems to something more universally consumable, and promote interoperability between different software.

Well-known text (WKT) is a text markup language for spatial reference systems of spatial objects and transformations between spatial reference systems. The updated standard, known as "WKT 2", was adopted by the Open Geospatial Consortium and ISO in 2015. OGC WKT2 fixes longstanding interoperability coordinate system definition discrepancies and contains tools for describing time-dependent coordinate reference systems.  Several countries are updating their geodetic infrastructure to include time-dependent coordinate systems. For example, Australia and the United States are adapting time-dependent coordinate systems in 2020 and 2022, respectively. The familiar NAD83 and NAVD88 in North America are being replaced by NATRF2022 and NAPGD2022, and the industry will have to adapt to these changes.

The new libraries provide for higher accuracy transformations between coordinate systems that avoid transforming everything through WGS84 and eliminate the extra steps that this entails.

Thanks to Dale Lutz of SAFE Software for pointing me to this during a chat at GeoAlberta 2018.

Bentley's digital twin capabilities for cities includes underground infrastructure

Proposed West Link tunnel under central Gothenburg

More and more cities are recognizing the tremendous benefits that a digital twin provides to its planners, private architects and engineers striving for compliance with city ordinances and by-laws, and for communicating with the public.  I remember several years ago when the Prime Minister of Singapore in his annual National Day address included a 3D visualization of what the proposed Marina Bay towers would look like - perhaps the first time 3D visualization has been used to inform citizens on a national scale.  At the Year in Infrastructure 2018 in London, Bentley announced the acquisition of Agency9, a Swedish company that provides web and mobile solutions for 3D visualization of geographic information and maps for collaboration and communication with its product CityPlanner (now OpenPlanner).  With this acquistion, Bentley augments its competitive positioning relative to CityZenith (SmartWorldPro), ESRI (CityEngine), and Autodesk (Infraworks) and open source projects such as mago3D in offering a 3D visualization and analytics platform for the massive data sources required for cities including underground infrastructure.

The requirements of a comprehensive platform for city visualization and analysis for planners, architects and engineers, and the public are the ability to support OGC (KML, CityGML and others), BuildingSmart (IFC), defacto (Shape, DWG, DGN, Revit and others) and other standards such as GeoJSON making it possible to integrate GIS, CAD, and BIM (vertical and horizontal) data, content creating and editing tools, for example, to allow buildings to be extruded from footprints and rooves and facades customized, support for open data sources such as OpenStreetMap, ability to integrate satellite, aerial and ground photo imagery and LiDAR point clouds,  support for both above- and below-ground infrastructure, gaming engine support (such as Unity or Unreal Engine) for interactive 3D visualization, cloud processing for managing, processing and querying massive amounts of data, web and mobile access, and APIs for analytics.

From a technical perspective the acquisition made a lot sense because Bentley's ContextCapture (from the acquisition of Acute3D several years ago), which creates 3D meshes for both inside and outside buildings from aerial and ground photo images and LiDAR point clouds, has been used by Agency9 for many years as the basis for its visualization capabilities.

Proposed West Link tunnel under Gothenburg

The West Link (Swedish: Västlänken) is a railway tunnel under construction under central Gothenburg. As an example of its support for underground infrastructure the City of Gothenburg used CityPlanner to publish a 3D visualization of the planned tunnel going below the entire city which showed stations with points of interest with images in a slideshow. During the first 24 hours that the project was published on its web site, it had over 10,000 unique visitors.

Bentley goes open source for its digital twin products

One of the biggest announcements in recent memory in the design and construction world was made by Keith Bentley of Bentley Systems this morning at Year in Infrastructure 2018 in London.  After describing some of the key technologies that Bentley plans to underpin its digital twin offering iTwin Services including ContextCapture for reality modeling and iModelHub for sharing  model information, Keith really opened up when he acknowledged that the world had gone open source and that Bentley was following Microsoft's lead in joining Google, Amazon and many other major companies in embracing open source. 

Bentley intends to build it products supporting digital twins around an open source project already on Github called iModel.js.  Keith has committed Bentley resources to developing an ecosystem around iModel.js. I have heard Keith Bentley at a number of events talking about his vision for Bentley technology, but I have never seen him so excited and committed to a new direction like this morning's announcement. To quote Keith this is a huge thing - for him the biggest thing in 35 of these Bentley events that he has spoken at since founding Bentley Systems.

Bentley had puts its toes in the open source water previously.  Bentley used open source geospatial components to develop a new front-end for its leading road and rail asset management system AssetWise Lifecycle Information Management solution (ALIM).  Bentley also partnered with AGI to support the open source Cesium project.

Bentley has clearly thought through open sourcing its digital twin products.  They have a business model for iTwin Services - not surprisingly for the rest of the IT world, but a significant step for Bentley - it is a subscription model.

To start the iModel.js ecosystem off, Bentley is working with Hack Partners to sponsor and support an Infrahack Hackathon early in 2019.

A tale of BIM4D and two iconic buildings in the heart of London's financial district

At the Year in Infrastructure 2018 conference this morning, I heard a fascinating tale of two buildings in a talk by Tom Dengenis of SYNCHRO. which was recently acquired by Bentley. I had always wondered about an abandoned construction project in the middle of the City, London's financial district. Construction on the Pinnacle, at 22 Bishopsgate in the heart of the City and the now iconic Shard began about the same time in 2006. Both had significant demolition to do before commencing work, but the Shard had more. But what we heard this morning is that because the Shard construction team adopted an innovative approach to building the substructure called "top down bottom up", the construction team on the Shard managed to start on the superstructure six months ahead of the Pinnacle and to complete the project on schedule in July 2012. SYNCHRO provided the graphic simulation along with a real time based 4D BIM model for this project. In contrast construction on the Pinnacle was halted abruptly in 2007. The lesson here is that in today's competitive environment with project durations getting shorter and shorter, being innovative in design and build is essential, an approach that Bentley calls constructioneering.

After construction was abruptly halted on 22 Bishopsgate, the site remained deserted with a seven story core concrete structure which came to be known as the "Stump". In 2015 the superstructure of the Stump  demolished to be replaced by a classic American-style, big floorplate building which will be second highest building in Europe (after the Shard) but which will be the most uninteresting building from an architectural perspective in the City - the City is the site of several iconic buildings including the Cheesegrater, Walkie-talkie, Scalpel, and the Gherkin.

Storm warning for the construction industry

At this year's BIMForum conference in Las Vegas Dr. Barbara Jackson of the University of Denver offered what I would categorize as a hurricane warning to the construction industry; change is coming like a very fast train and either the industry has to change itself or someone from outside will do it as Uber and Airbnb have done to the taxi and accomodation businesses and as Amazon is doing to the grocery business. Because in construction resistance to change is much greater than the motivation to innovate and change, under 30 year olds are saying that the construction companies they want to work for don't exist, so they leave.  One of the ways she hopes to help the industry to transform from within is by effecting a culture change by encouraging the under 30s folks not to leave but to stay and transform the industry.

Barb Jackson spent 25 years running her own construction company where she experienced the problems that were facing the industry as a whole. Seriously concerned for the future of the industry, she decided to go into academia to try to do something about them. In her view the key factors that make construction ripe for disruption are

• high segmentation and fragmentation
• low productivity gains - labour-productivity growth in construction has averaged only 1 percent a year over the past two decades, compared with growth of 2.8 percent for the total world economy.
• low adoption of digital technologies - McKinsey Global Institute places construction second to last just ahead of hunting and agriculture
• high level of waste - estimates of 30 % waste and rework are typical
• lack of leadership - construction is good at management (the mantra is plan to work, work to plan) but lacking in leadership
• a culture that discourages change and innovation.

Some have argued that the $10 trillion/year construction industry is not like other businesses, but Barbara's frank response to this is "this is bull, it is."  It is difficult to place all the blame on owners, who the designers and contractors say discourage them from doing anything innovative that raises the risk or the industry folks who the owners say resist any change.  She offered that perhaps the real culprit is lowest bid procurement which resulted such low margins that the only way to make money on projects was through change orders.

The construction business model is outdated and the collaborative economy as exemplified by Airbnb, Uber, Facebook, and Alibaba and what happened to the accomodation, taxi, media, and retail industries is a real threat for construction and an opportunity for an outside entrepreneur.  Companies like Artiman who look for industries that are ripe for disruption view the construction industry as low tech, archaic, wasteful, characterized by low productivity, with an out-dated business model, in a word ripe for disruption.

She cited the example of what is happening to the grocery business as another example of someone from outside effecting change to an industry that has been resistant to change. When Amazon acquired Whole Foods, the stock of Kroger, Super-value, and others lost 14% of its value overnight. Even Costco and Wal-Mart dropped 6%. Massive change had come to the grocery industry, and it was coming from outside-in. It was happening to the industry, not by the industry and investors were placing their bets on the outsiders.

There are already some examples of new business models impinging on the construction industry. The most notable example is Elon Musk's Boring Company that estimates that it can reduce the cost of tunneling by a factor of 10X and the time it takes to bore a tunnel by a factor of 14X. It is already working on a hyper-loop project from New York to Washington DC that would enable travel between the cities in 29 minutes.

Because of her real world experience, Barb's perspective is not a voice from academia crying in the wilderness, but is a serious warning from an industry insider.  Echoes of what Barb is warning about could be found in the opening address at the conference from Eddie Stewart, President of the Associated General Contractors of America (AGC).  The AGC represents 26,000 firms in the construction industry including general contractors, specialty-contracting firms, and service providers, but predominantly small contractors.  He also noted that the construction industry lags most industries in adopting digital technology.  He added that the industry as a whole is facing a labour shortage and the duration of projects is getting shorter.  He also warned that disruption is coming or maybe is already here.

At a BIM conference in Toronto several years ago, Patrick MacLeamy's, Chairman of buildingSMART and CEO of HOK, message was very simple.  In the U.S. buildings are too expensive, too inefficient to operate and maintain, and don't last long. The problem is that the construction industry is too fragmented.  The process of orchestrating many disciplines in an adversarial, litigation-prone environment is too complex and leads to frequent budget and schedule overruns, evidence of a seriously broken business model.

Young people in the industry are asking why things are still done in the traditional way, which they find stupid.  The resistance to change is so-inbuilt that under 30 year olds are saying that the construction companies they want to work for don't exist, so they leave the industry. One of the ways she hopes to help the industry to change from within is by effecting a culture change by encouraging the under 30s folks to remain and change the industry.  Barb is working on putting together education programs to keep young people in the industry hoping that the next generation will help construction disrupt itself before it is disrupted by somebody from outside.

Bentley open sources its web-based digital twin javascript support library

I am at the Year in Infrastructure 2018 conference in London this week.  This morning there have been some very interesting announcements relating to digital twins.  First of all Greg Bentley announced Bentley's foray into infrastructure digital twins.  iTwin Services provides cloud-based support for digital twins and integrates with ProjectWise and AssetWise.  Secondly, Keith Bentley announced what I think is Bentley's first open source project.  iModel.js, which is hosted on Github, is a javascript library to support web-based visualization and analytics for infrastructure digital twins. 

Previously Bentley used open source geospatial components to develop a new front-end for its leading road and rail asset management system AssetWise Lifecycle Information Management solution (ALIM).  Bentley also partnered with AGI to support the open source Cesium project.

Microsoft becomes a fully fledged open source company

Things do change. Microsoft, once the perceived enemy of open source, has been moving into open source over the last two decades.  The first infamous Halloween document requested by Jim Allchin was leaked to Eric Raymond in October 1998.  This and other internal documents recognized that that open software products such as Linux were technologically competitive with Microsoft's products and set out a strategy to combat them. But there were folks within Microsoft that disagreed with this strategy.  Microsoft began partnering with open source companies back in 2006. It announced support for Git in 2013.  It now has over 2,000 open-source projects on Github and claims to be the largest open-source project supporter in the world. As further evidence that It is now a fully fledged open source company, Microsoft has joined the Open Invention Network (OIN), an open-source patent consortium that includes the Open Source Geospatial Foundation (OSGEO), Google, IBM, Red Hat, Oracle and over 2600 other organizations as members or licencees. Microsoft is bringing something on the order of 60,000 patents to OIN, effectively granting royalty-free and unrestricted license to its patent portfolio to all other OIN members.  Since in the past Microsoft has derived significant royalty revenue from protecting these patents including many used in Android, this really is a case of putting its money where its mouth is.

Measuring emissions from natural gas superemitters from space

Four countries have launched satellites which can measure concentrations of greenhouse gas emissions. Canada's GHGSat Inc is the first private company that can measure emissions from any location on earth. To date GHGSat's Clair satellite has measured emissions from 2,500 locations.  Among these are emissions from so-called superemitters in the natural gas industry in the U.S.

I have blogged extensively about methane emissions from the shale gas industry. There is increasing evidence that leakages from production wells, of which there are about half a million in the U.S., erodes the advantage that natural gas has over coal with respect to contributing to greenhouse gas warming. Studies suggest that a small number of wells, socalled superemitters, are responsible for more than half of the total volume of leaked methane gas in the United States.  This is an image of a heat map created from measurements made by GHGSat's Claire August 2018 over a site in Texas.  The heat map has been superimposed on imagery captured by ESA's Sentinel-1.

Documenting construction projects with a consumer 360° camera

Many see handheld scanners as revolutionizing the construction sector.  Portable and handheld laser scanners offer millimeter accuracy and can be used for engineering purposes.  Cameras with less precision are being used for rapidly documenting a construction site for a variety of purposes.   Most recently a couple of startups Cupix and HoloBuilder are lowering the bar by offering cloud software for the AEC industry to enable images of a construction site to be captured with a consumer grade 360° camera like the Ricoh Theta S or V. Walking through a construction site capturing 360° images along the way and uploading the images to the web is all that is required to create 3D walk-throughs of a site.  The 3D walk-through can be used to measure dimensions, it can be annotated, it can be compared with a BIM model, and 3D images captured at different times can be compared to monitor construction progress.

 

For engineering purposes, handheld laser scanners provide the requisite precision but are relatively expensive.  The scanning process requires time and produces large point cloud files and imagery that need to be managed and processed for visualization.

• The Leica Geosystems BLK360 weighs a mere 2.2 lbs (1 kg), is incredibly small 6.5 x 4 inches and lists for US$16,000.  It is a full laser scanner and imager, capturing 360,000 points/second with a range of 0.6 - 60 meters.  The scanner can be attached to a Microsoft Surface Pro tablet to visualize what you have scanned. Up to 16 of these can be used at the same time.
• Dot Product DPI-8 and DPI-8X (short range) are multi-sensor devices (IR laser and photo cameras).  Targeted at professionals, especially in the engineering and construction sector, the devices incorporate SLAM technology and works with Android handhelds.  All the processing is done locally on the handheld - no cloud processing is required.
• The Faro Freestyle can scan objects within a range of half a meter to 3 meters away with a resolution of  better than 1.5mm with a image point density of up to 45,000 points/m² at 0.5m and up to 10,500 points/m² at 1m.  The scanner is attached to a Microsoft Surface Pro tablet for visualizing the point cloud as you scan.

For rapid site documentation, Matterport allows you to scan a floor of a building in an hour or less, upload the captured data to the cloud (Amazon Web Services) and a couple of hours later have a fully-rendered 3D walk-through that can be accessed over the web.  It was originally targeted on the real estate sector but recently has been showing up at construction industry and other events including the recent BIMForum in Las Vegas.  It provides a dollhouse view to see a whole floor at once, an inside view for walk-throughs, and a floor plan view for a traditional top-down perspective. It relies on IR and photo cameras and multiple scans in each room - scanning every 1.5 to 2.5 meters.  It is capable of rough measurement, but not for engineering or construction purposes.  Its biggest advantages are the speed with which you can scan a site and the small size of the resulting file which has to be uploaded to Matterport's cloud site.  In addition to real estate and construction site documentation, it is used by FM professionals who need a 3D context for facilities management.  For example, it can provide a mechanical room walk through with tags on different pieces of equipment and a way of directing anyone to any location in the facility.  The camera is several thousand dollars and cloud processing requires a subscription.

 

The most recent additions to the technologies that can be used to document a construction site are Cupix and Holobuilder which lower the bar by only requiring a consumer grade 360° camera such as a Ricoh Theta.  In the case of Cupix cloud processing requires a subscription.  Similarly to the Matterport, you walk through the site capturing images with 360° camera as you go.  The resulting images are uploaded to the Cupix site and the result is 3d tours or walk-throughs accessible over the web.  You can compare 3D tours taken on different days to monitor construction progress, compare a 3D walk-through with a BIM model (Revit or IFC) you have uploaded, and measure dimensions with a precision of 1% or better at 20 meters.  To reach this level of precision in measurement requires including a marker when taking pictures.  Alternatively if a dimension is known it can be used to scale the entire walk-through.  Cupix can use images captured indoors and outdoors.  Cupix says that if GPS metadata is available from the photos, they can georeference the walk-throughs and generate a KML file. I used Cupix with a low cost 360Fly4K camera and a Samsung Note 5 phone to capture the interior and a bit outside of a cabin in a nearby park.  It took a couple of attempts to get the correct spacing between camera placements in capturing 360° images, but after that it was straightforward. I uploaded a rough picture of a floor plan and I was able to walk-through the resulting 3D tour using the Cupix web viewer.  Without doing anything additional I was able to get rough measurements accurate to about 5%.  Cupix only requires a consumer grade 360° camera.  A subscription is necessary for cloud processing. Cupix is targeting a broad market including travel and hospitality, real estate, and arts and culture in addition to the AEC industry

 

I just came across HoloBuilder at the recent BIMForum in Las Vegas.  Like Cupix it only requires a consumer 360° camera. At first glance it appears to be focused on the AEC industry and makes creating and managing 3D tours of construction sites very easy.  It offers 360° SiteStream which is a unique live streaming solution for real-time construction progress monitoring in 3D that runs as a plug-in app on the Ricoh Theta V.