Between the Poles

Democratizing 3D scanning to non-surveyor professionals

At the SPAR3D 2017 conference, Burkhard Boeckem, CTO of Hexagon Geosystems, gave a vision of the future of 3D technologies developed by technology firms such as Hexagon.  Among the well-known brands that form Hexagon Geosystems is Leica Geosystems, which recently introduced two new ground-breaking LiDAR scanners, the SPL100 and the BLK360. These devices are targeted on very different markets.   He used these devices as examples of how a new technology becomes democratized among a much broader professional community than the small number of specialists it was initially targeted on.

By way of background, up until recently LiDAR scanning was pretty well restricted to the surveying community and a few other specialist groups.  Leica has made LiDAR devices for these specialist communities for over a decade.

Single photon LiDAR is a new technology that differs from the current waveform LiDAR mainstream in that it can detect weaker reflected light where current devices need much stronger light.  The SPL100 is a single photon LiDAR sensor that collects 6 million points per second which means that planes can fly higher and still collect 12 - 30 points per square meter, depending on flying height.  Leica says it is 10 times more efficient than conventional waveform LiDAR sensors for this type of mapping.  The SPL100 is initially targeted on a very specialist, exclusive group of mapping professionals who scan large areas - states, countries and regions. 

In contrast the BLK360 is a very small (1 kg) imaging LiDAR scanner, which is targeted on a potentially much broader professional audience for 3D data, including many people who haven't heard of LiDAR or even considered whether 3D data might help them. It is very simple to use - it has only one button.  It can complete a full 360 scan of a space with a range of 60 meters in three minutes.  All subsequent processing of the point cloud and imagery to visualize the data is performed on an iPad.  The BLK360 is the culmination of laser scanning development at Leica which began in the early 2000s targeting the surveyor community.   The BLK360 is intended to dramatically expand the domain of users of high precision 3D data.

Burkhard Boeckem used these two devices as a model for how he sees new technology first being developed and introduced for specialist professionals and then being democratized among the broader professional market. 

Looking toward the future he sees single photon LiDAR and Simultaneous Localization and Mapping (SLAM) as important breakthrough technologies that are very new and are only now being introduced into the specialist market. For GNSS-denied areas such as inside buildings, tunnels and mines Leica has just introduced the Pegasus Backpack which relies on SLAM to track the wearer in a building or other areas where GNSS does not penetrate.  The Pegasus Backpack appears to be targeted currently on a relatively small market of specialists.  Based on 15 years of laser scanning development at Leica, Burkhard Boeckem's message is that by analogy with the process that culminated in the BLK360, in the future we will likely see very simple "one-button" devices descended from the Pegasus Backpack and the SPL100 aimed at the broader professional market.

Getting technologies like the BLK360 out to people who could benefit from 3D data has become the current challenge according to Shabtay Negry, Senior Vice President at Mantis Vision who manufacture one of the earliest handheld 3D scanners and whose 3D technology is used in the Google Tango project.  The target for the BLK360 and other small, easy to use, and inexpensive devices are professionals who could benefit from  3D technology.  The introduction of the Xbox 360 Kinect in 2010 lowered the technology price bar for 3D scanners.  The Kinect and many of the low end scanners currently on the market use very inexpensive infrared plus photography technology to create 3D point clouds with associated images. As an example, the Matterport Pro, which lists for $3600, is such a device targeted on Real Estate professionals.  With these types of devices Shabtay Negry expects the 3D market to expand by 100,000 units in two to three years.  As examples of the types of professionals for whom 3D technology provides benefits he mentioned plumbers, upholsterers, insurance adjusters and many more.  Honeywell Logistics, who are not 3D specialists, use 3D scanning for space optimization.  Price as a barrier to the adoption of 3D scanners has been virtually eliminated.  The Structure sensor is $400 and attaches to iPad devices.  Mantis Vision promises to make its technology available for everyone - being able to scan an object, share the 3D model online, and send it to a 3D printer—from a smartphone.  Mantis Vision was working with the Google Tango project (now deprecated and replaced by ARCore) to make 3D scanning available on an Android tablet.

Are point clouds going away ?

At the SPAR3D 2017 conference in Houston, one of the interesting questions posed to a panel comprised of Greg Bentley (CEO of Bentley), Burkhard Boeckem (CTO of Hexagon Geosystems), and Shabtay Negry (Senior VP at Mantis Vision) is whether point clouds, whether from digital photography or LiDAR, will effectively disappear, being replaced by meshes for most reality modeling applications.  Meshes are mathematical constructs, typically 3D triangular networks, that are smaller and much easier to manipulate than point clouds, which are typically huge and a challenge to edit.
 
I blogged previously about a digital model comprised of meshes of downtown Philadelphia that was used to orchestrate and secure the Papal visit.  All the data was collected as photographs which were then used to create a single reality mesh using Bentley ContextCapture.  At SPAR3D there are presentations on two other projects which used a similar approach to create models of the city of Helsinki and the Cologne Cathedral.

Another fascinating example is the USS Arizona digital project which combined underwater lidar, sonar, aerial imaging, and existing photography and surveys to produce a survey-grade 3D model of the USS Arizona in the form of a mesh using Autodesk Remake.  Surveys were conducted in 2014 and 2016.  Just recently the two digital mesh models of the ship were compared to create a difference mesh to show areas where there has been change in the two years since 2014.

At SPAR3D there was a consensus among the panelists that meshes are replacing point clouds for most applications. Point clouds are just too big and have too little intelligence. They will remain as an intermediate data type because most scanners generate point clouds.  Replacing point clouds for many applications meshes will grow in importance in the future.  But realizing their full potential will require interactive, editable meshes that allow small bits to be changed or replaced without having to regenerate the entire mesh (which is often the case with triangulated surfaces).  Easy to use analytic tools like those used to create the difference map of the Arizona will also be required.  The panelists thought that with the appropriate edit and analytic tools people will be able to manipulate surfaces easily and may not  even be aware of what the form of the underlying data structures is.  Smaller files will also make cloud processing easier because in the future huge point cloud files won't have to be uploaded to the cloud.

Handheld 3D scanners rapidly approaching consumer pricing

I have just spent an incredible three days in Houston at the SPAR3D 2017 conference.  There is so much going on right now in the 3D world with several key technologies just coming to fruition and finding broad application in the real world. 

 

Many see handheld scanners as potentially revolutionizing the construction sector.  Handheld scanners had a high profile at SPAR3D this year, with several vendors showing new products and many promising future products in the $400 price range.  One vendor is already selling a 3D imaging product that attaches to an iPad and sells for $400.

 

First exhibited at the 2013 SPAR3D, Dot Product has released a products DPI-8 and DPI-8X (short range) similar in its basic technology, multisensor (IR and photo cameras) to their first handheld.  I saw it first at the SPAR3D 2014 conference. Targetted at professionals, especially in the engineering and construction sector, their latest product incorporates SLAM and works with Android handhelds and tablets.  All the processing is done locally on the handheld - no cloud processing is required.  The captured point clouds are small - you can even cull the point clouds on the fly. The working range of the DPI-8 is from 0.6 m to 3.7 m. It export in PTS, PTX, PLY, and PTG. Binary files integrate directly with AVEVA LFM, Autodesk ReCap, Trimble RealWorks, CloudCompare,  ClearEdge3D, Veesus Arena4D, and PanoMap. The DPI-8X lists for US$ 5150.  Dot Product has been working with Google on Project Tango and says there is a $450 scanner in the future.

Mantis Vision has also offered a handheld scanner F5 for professionals for several years.  Their latest offering is the Mantis Vision F5-SR scanner/imager. It has a range from 0.5 meters (m) up to 4.5 m with an accuracy of a few millimeters (mm).  It also relies on IR and photo cameras.  It uses triangulation to locate objects and is in lower end of the the F5 price range of $20-50,000.  Mantis Vision promises to make its technology available for everyone - being able to scan an object, share the 3D model online, and send it to a 3D printer—from a smartphone.  At the end of May Mantis Vision will be releasing a new scanner, the F6 at a price of $15,000.  The F6 can perform dynamic as well as static scanning and it is possible to scan collaboratively with up to 16 of the devices.

 

A handheld scanner that garnered a lot of attention at SPAR3D was the Matterport Pro from Matterport Immersive Media Systems.    Originally targetted on the real estate industry, the Matterport Pro allows you to scan a house in an hour or less, upload the captured data to the cloud (Amazon Web Services) and a couple of hours later you will have a fully-rendered model that can be used for walkthroughs with potential buyers.  It provides a dollhouse view to see a whole property at once, an inside view for walkthroughs, and a floorplan view for a traditional top-down perspective. It relies on IR and photo cameras.  It is pretty well restricted to indoor scans and requires multiple scans for a room - scanning every 1.5 to 2.5 meters.  It is claimed to be 95.5 % accurate - it can be used for measurement for sales not for engineering or construction purposes.  Its biggest advantages are the speed with which you can scan a house, for example, and the small size of the resulting file.  It can also provide a 3D context for facilities management.  For example, it can provide a mechanical room walk through with tags on different pieces of equipment and a way of directing anyone to any location in the facility.   The Matterport Pro 3D Camera lists for US$ 3600.00 plus US$ 49/mo to US$ 149/mo for cloud processing.

 

I have already blogged about the Leica Geosystems BLK360 which includes Autodesk Recap Pro.  It is a bit of a stretch to call it a handheld scanner but it weighs only 2.2 lbs (1 kg) and is incredibly small 6.5 inches (in) by 4 in.  Unlike the other devices described here it is a full laser scanner and imager. It captures 360,000 points/second with a range of 0.6 - 60 meters.  It has one button and can capture a full 360 degree scan in less than 3 minutes.  All aspects of the operation of the scanner can be controlled remotely from an iPad Pro.  It can perform all the processing required for visualization in the field on an iPad Pro.  It lists for US$16,000 and includes a one year license for Recap Pro. 

 

I blogged about the Faro Freestyle previously.  It can scan objects within a range of half a meter to 3 meters away with a resolution of  better than 1.5mm with a image point density of up to 45,000 points/m² at 0.5m and up to 10,500 points/m² at 1m.  The scanner is attached to a Microsoft Surface Pro tablet for visualizing the point cloud as you scan.  It sells for $11,000 to $12,000.  It is targetted on the architecture, accident reconstruction, civil engineering, construction, facility management, and industrial manufacturing sectors.

 

The least expensive scanner shown at SPAR3D was the Structure sensor with Canvas software from Occipital.  A new screw-on lens increases the field of view to 120º so that it requires fewer scans to capture a room.  Its range is 0.3 - 5 meters with a precision of 0.17%. It is targetted on the home owner as well as interior designers, architects, and contractors.  It can be used to capture 3D models of interiors, share them, measure with them or convert them to formats supported by AutoCAD and Sketchup.  The vendor offers a Scan To CAD service, that converts scans into editable CAD files (.skp, .dwg and .dae) for use with  tools like SketchUp, AutoCAD, and Revit.  It retails for $379 - $499 and attaches to several models of iPad including the iPad mini 4.  At the end of April it will be possible to purchase Bridge, a headset that makes it possible to mix real world and virtual realities with an iPhone.

3D model-based design is transforming construction in the Netherlands

At the GeoBIM Building and Infrastructure Conference in Amsterdam, Ingeborg Ligtenberg, Senior Project manager for Mega Projects, described how the Dutch highway authority Rijkswaterstaat (RWS) is piloting a very different approach to highway construction.  Rijkswaterstaat is responsible for the design, construction, management and maintenance of the main infrastructure facilities in the Netherlands including the national road network, waterway network and sluices. 

Background

By way of background, the highway plan-design-build process that has been used over at least the past 25 years, starts off with a complete (re)survey of the project terrain to create a basemap on which the design is based. Existing as-builts in the area are rarely if ever consulted.  The deliverables from the design process are paper drawings which are used by construction contractors to build the project.  After construction is complete construction as-builts are submitted to the owner, but these are not "sealed", meaning nobody signs off on them putting their professional reputation behind their accuracy and reliability.  Practically what this means is that in the transportation industry, as-builts are treated as unreliable information and are rarely consulted.

At SPAR 3D 2013 Ron Singh, Engineering Automation Manager/Chief of Surveys, Oregon Department of Transportation (DoT) made a startling proposal that would stand the traditional highway construction process on its head. He proposed that the DoT should specify the data it needs and this data including BIM models and LIDAR scans would be collected by the DoT from the contractors during and after a highway construction project.  The data would be saved in a geospatially-enabled database maintained by the DoT.  One of the implications of this approach is that when a new project is initiated in the same area over 80 % of the necessary survey-grade information would already be available in the DoT's database, which would make the traditional complete resurvey unnecessary - saving time and money. 

As an example of how data reuse can save time and money, the Oregon DoT's first automated machine guidance (AMG) project was in 1997.   At that time in Oregon a few forward-looking contractors had invested in AMG technology foreseeing that it would save them money.  These forward-looking contractors were winning bids by lowering their estimates just enough to beat other contractors. They found that AMG technology did save a lot of money.  But the savings were going to the contractors, not to the DoT.  To redress this, starting in early 2015, the DoT began specifying on its bid web site that projects were "AMG ready" meaning that the DoT was making available provide all the data it had which could be download to see design BIM models, mobile LiDAR, overflight data, and so on.  The effect has been to drive bids down, because there is less risk for the contractor.  As a result the DoT is now sharing in the savings.

Rijkswaterstaat pilots

At the first GeoBIM conference in Amsterdam, Jothijs van Gaalen gave some real world examples of highway construction transformation based on GIS+BIM integration including using laser scanning/LiDAR for reality capture at the beginning of a design/construction project. 

The Rijkswaterstaat (RWS) board is convinced of the benefits of BIM and BIM/geospatial integration and decided initially to roll out BIM on four design-build-finance-maintain (DBFM) projects with the intention to require BIM on all DBFM-contracts in the future.

One of these projects is a highway expansion project for the highway connecting Schiphol-Amsterdam-Almere or SAA.  The objective of the SAA project was to improve the accessibility and quality of life within the SAA corridor. This includes the improvement of the natural and social environment and economic development and employment opportunities in the northern Randstad region.  A stretch of the highway is shown before and after the highway widening.

On many highway projects RWS had experienced information flow problems due to different formats and data models.  This not only necessitated major data conversion efforts, but also inhibited collaboration.  One of the goals of the SAA project was to improve information flow which was expected to also improve the flow of asset information for reuse during the maintenance phase of the highway.

A critical part of the RWS' new approach was the Information Delivery Specification (ILS). The ILS specified the general aspects of the required information including formats, standards, structure, and expected quality (timeliness and reliability) of the information.  The ILS was a contractual obligation that also required the contractor to accept new versions of standards on a yearly basis.  The standards that were adopted to ensure a uniform flow of structured information include VISI (an open communication standard for the construction industry based on XML), National Concept Type Library (CB-NL), Object Type Library (OTL), and COINS.

Today RWS is receiving a quarterly delivery of data from the contractor compliant with the ILS.  This is a remarkable achievement that required a lot of hard work during this first project.  But now that contractors are familiar with the standards and the ILS concept, it is expected that ensuring a uniform flow of information will be easier in future projects.  The next challenge is enabling this data to be used for asset management during the maintenance phase of the project.

Reusing data on future projects

 In response to a question, Ingeborg Ligtenberg said that this new approach was still in its initial phases but that the data would likely be made available to designers and contractors for future projects.  It would be made available on an "as is" basis, which sounds similar to how the Oregon DoT makes its data available on its bid site. 

As Ron Singh has pointed out, there is tremendous value in this data, but the value is only there if in the future engineers feel confident in using the data.  If very accurate and rich data were collected during and after construction including, critically, its lineage in the form of metadata about how and when it was captured, so that a future engineering user would have no reason not to trust its provenance and reliability, then there should be nothing to prevent the data from being reused in future projects - saving time and money.

Autonomous vehicles

One of the consumers of this digital highway information may be autonomous vehicles.  In-vehicle sensing is limited in its ability to enable safe and reliable driving under all conditions.  For example, on snowy roads, lane markings may not be visible. Such driving conditions will require the autonomous vehicle to be equipped with ultra-accurate maps to augment in-vehicle sensing systems.  Pre-loaded high-precision maps would enable autonomous vehicles to overcome many of these challenges – staying within lanes when wintry conditions obscure the lane markers. This will require sophisticated mapping systems and ultra-accurate and reliable DoT data to enable vehicles to navigate under challenging conditions.

Leica lowers the bar with a $16k imaging laser scanner

Leica Geosystems has announced the “BLK360,” a miniaturized 3D imaging laser scanner. The BLK360 captures full-color panoramic images overlaid on a high accuracy point cloud. The one-button Leica BLK360 and included Autodesk software is intended to be easy to operate so that anyone who can operate an iPad can use the imaging laser scanner.  It is incredibly small - it weighs only about a kilogram.

It collects 360,000 points per second and has a range of 60 meters with 4 mm accuracy.  In addition to a LiDAR sensor, the BLK360 includes infrared sensors for thermal imaging and 360° cameras.  It is completely wireless and streams image and point cloud data to an iPad. A complete 360° laser scan with panoramic image capture and transfer to the iPad Pro requires 3 minutes. 

With an anticipated retail price of the scanner and software is $15,990/€15,000, this really lowers the entry barrier to laser scanning.

HxGN LIVE Precision mapping underground utilities with combined LiDAR and GPR

One of the most exciting presentations at the annual Hexagon conference HxGN Live in Anaheim from the viewpoint of practical application of new technology was a talk given by Gilbert Roulier of Grunder Ingenieure AG, who reported on a pilot project in Berne, Switzerland, to map underground utilities using combined mobile laser-scanning (mobile LiDAR) and ground penetrating radar (GPR). The objective of the pilot was to asses the capability of the equipment to capture in 3D culverts and water pipelines under streets in Berne with an accuracy of +/- 10 cm.

Equipment

The equipment used was a Leica Pegasus: Stream comprised of a Leica Pegasus:Two integrated with a Stream EM and towed by a vehicle.  One of the advantages of the Stream EM is that it is comprised of three antenna arrays - two for main line detection and one array for crossing line detection.  The Stream EM captures real-time tomography of the subsurface to about 4 meters in depth.  The combined LiDAR and GPR array can be towed at a speed of about 15 km/h.  In addition to capturing subsurface structures, the combined device also captured above ground structures.  Manholes and other visible surface infrastructure were used as ground control points.

After data capture there is a combined workflow that post processes the data captured by the Pegasus:Stream.  The final result is 3D data that can be imported into ArcGIS and AutoCAD.  As part of the pilot the data was compared to existing records and maps prepared showing the divergence.

Results and Benefits

The pilot revealed significant benefits from using combined LiDAR and GPR array.  First of all that the laser scanning and GPR equipment worked simultaneouly reduced traffic disruption.  It also dramatically improved productivity - up to 100 km could be captured and processed per day - including both above ground information about road structure and condition and below location of underground utilities in 3D.  An important advantage of the Stream EM array from a productivity perspective is that data collection was only required in one direction since the Stream EM array includes an array for transverse scanning of infrastructure crossing the direction of the scan.  An important advantage of the combined system is that it is capable of +/- 10 cm in locating underground infrastructure in 3D. Finally, the combined Pegasus: Stream array was much safer for the operator.  

From Grunder's perspective the pilot was successful and they report that they are seriously considering investing in the Pegasus:Stream.  Grunder sees the potential users of the combined system surveying companies and utility companies - both public and private.  It believes that the equipment could be used routinely for mapping of underground assets including pipe and cable detection and mapping, but also for road maintenance and renewal, subsurface cavity detection and in a modified form for ballast monitoring on rail lines.

Future

In his keynote at HxGN LIve Hexagon Geosystems President Juergen Dold made it clear that mapping digital realities above and below streets is a key technology for Hexagon Geosystems.  Very recently Hexagon announced the establishment of a georadar portfolio with three releases to provide applications in utility mapping, asset detection and mapping, and monitoring.  Included in the georadar portfolio is the Leica Pegasus:Stream which Hexagon sees as enabling mass digitization of infrastructure assets, such as telephone cabinets above and cable conduits below, by collecting data in less time without needing to stop traffic, increasing the safety of the road work crew.  They emphasize that the Pegasus:Stream has the potential to collect up to 100 km per day at 15 km/hr, providing digital documentation for GIS and CAD modelling.

I had an opportunity for a brief chat with Stuart Woods, Leica Geosystems Vice-President of Mobile Mapping, about Hexagon's plans for the Leica Pegasus: Stream.  According to Stuart Hexagon is planning to invest in the technology such as enhancements to the antenna arrays to increase the precision of the radar record.  They also intend to investigate translating the radar record into something more understandable like a point cloud.

Conclusion

The very exciting conclusion I draw from this is that the technology for underground remote sensing has finally gotten good enough that a major company sees an important market opportunity and is seriously investing in the technology and business of developing and manufacturing devices for detecting and mapping underground utilities. 

HxGN Live: 3D underground utility mapping, indoor mapping, UAVs and utility solutions

Last week I had the opportunity to present at the annual Hexagon conference HxGN Live in Anaheim. Hexagon is a rapidly growing company with broad technology reach with core strengths in geospatial surveying and laser scanning technology (Leica Geosystems), UAVs,  software GIS systems for network engineering, network operations, and operations and maintenance of utilities (Intergraph),  design and operation of nuclear power plants (Integraph) and enterprise management of large scale construction projects among other technologies and solutions.

Underground utility mapping

Underground utility conflicts and relocations are the number one cause for project delays during road construction. In 2005 Hexagon acquired UK-based Cable Detection.  Since then Leica Geosystems has offered both electromagnetic and ground penetrating radar hardware and software for detecting and mapping underground utilities.  At HxGN Live this year one of the exciting technology highlights was the Pegasus: Stream, which combines a Leica Pegasus: Two mobile mapping device incorporating laser scanners, optical cameras and GNSS receivers and an IDS Stream EM ground penetrating radar (GPR) array.  I first saw this towable combined above and below surface scanner at Geo Business in London.  According to Stuart Woods, Leica Geosystems Vice-President of Mobile Mapping, Hexagon has worked with IDS for years and is in the process of acquiring IDS' GeoRadar division, which developed and manufactures the Stream array among other radar technology.  This advance in technology puts Hexagon in the forefront of underground utility detection and mapping.

Indoor mapping

The other major hardware highlight of this year's HxGN LIve was the Pegasus Backpack.  This is a wearable reality capture platform that collects data indoors, outdoors and underground.  Klaas de Weerd, COO of Prisma Van Steenis, Prisma Groep, reported the experience that his team in mapping structures in the Netherlands.  Among other structures they mapped the Eye Film Institute in Amsterdam inside and out.  Leica mobile, P20 and P40 scanners were used outside, and the Backpack was used inside and on the roof.

Slope instability detection and monitoring

Enrico Boi, of the Georadar Division of IDS, reported on several applications using interferometric radar to detect slope deformation.   Slope failures are complicated and hard to predict, which is the reason radar is important.  For example, the I-70 through the Rockies is subject to rockfalls especially in the Glenwood Canyon in Colorado.  IDS Interferometric Radar (IBIS) equipment was installed across the Canyon to identify the location of slope instabilities which then could then be stabilized to prevent rockfalls.

Laser scanning during construction

Fokke Broersma of Arcadis NL (Arcadis is also melding geospatial and BIM in construction.) reported on how his company, which is committed to using BIM throughout the construction life cycle, uses laser scanning in conjunction with BIM.  One of the projects he described was the New Rijksmuseum in Amsterdam where laser scanning was used for capturing the existing structure prior to construction, then for as-builts after construction. 

In a related software product Hexagon at HxGN announced the release of HxGN Smart Build.  Smart Build is a cloud-based solution for large scale construction projects that integrates BIM models, point clouds, and as-built measurements to monitor construction progress and identify deviations from plan.

Opening electric network data

Adrienne Behan of ESB, an Irish energy company, described how her company manages its 2 million wood poles.  Their GIS provides multiple views into a single network model which feeds into the outage management system.  It also interfaces to other critical business systems.  Any changes to the network are available to business users of the system within 24 hours.  This is remarkable among utilities where backlogs often result in the network model being months even years out of date.

Mobile sensor platform robot

A highlight of President of Hexagon Safety and Infrastructure Steve Cost's keynote was a product of Hexagon SI's partnership with Gamma 2 Robotics.  The robot that came on stage is a mobile sensor platform.  One application of the product is a security patrol robot branded RAMSEE which can patrol autonomously providing real-time data on intruders, motion, heat, fire, smoke, gas and more.

UAVs

There were several UAVs on display at HxGN.  The one I blogged about several years ago, about the time that Hexagon acquired the German company that had developed it, is the Aibotix.  The Aibotix X6 has turned out to be a very stable platform capable of carrying a variety of sensors.  It is used primarily for industrial inspection such as transmission pylons, dams, bridges, and other structures.  It can hover, the camera or other sensor can be held below or above the frame, it can fly a path guided by GPS even in windy conditions and has fully automated takeoff and landing capabilities.

Historical building preservation Palazzo dei Normanni in Palermo

One of the outstanding examples of using scanning technology to record and preserve information about historical buildings was presented by Italian company Technoart.  The Palazzo dei Normanni in Palermo, Sicily is an incredible structure with an intriguing multi-cultural history.  The first building was built by the Arabs who occupied Sicily in the 9th,10th and 11th centuries.  The Normans conquered Sicily in 1072 and made Palermo the capital of their kingdom. The Norman kings built upon the former Arabian palace. In 1132 Roger II, famous for sponsoring one of the earliest maps of the world based on Arab and Greek sources by Muhammad al-Idrisi (نزهة المشتاق في اختراق الآفاق‎‎,Tabula Rogeriana, Book of Roger), added the incredible Cappella Palatina.  The laser and optical scan using a Pegasus One by Technoart resulted in a point cloud with optical imagery that with the aid of gaming software allows users to virtually walk through the palace and the Palatine Chapel and explore the structure including the gorgeous frescos in detail. 

Background

Over the past 16 years, Hexagon has evolved from an auto parts and industrial measurement (metrology) hardware provider in Sweden to an international hardware and software solutions provider with annual revenue of € 3 billion and remarkably an operating margin of 23%.  It operates in the agriculture, geospatial, geosystems, manufacturing intelligence (metrology), mining, positioning intelligence, process power and marine, and safety and infrastructure sectors.  Among the many acquisitions it has acquired in the last decade are widely known Leica Geosystems (was Wild), Intergraph and Erdas.  It has expanded from industrial metrology to include geospatial sensors including Leica total stations and laser scanners; and software-centric enterprise geospatial solutions.  Its industrial enterprise solutions include industrial metrology hardware and software targeting optimizing manufacturing processes.  It has acquired software for designing, constructing and operating industrial plants and offshore facilities.  

Automating the process of comparing digital models to reality

At the Spar 3D Expo and Conference in Houston I had the opportunity to chat with Nicolas Arnold, VP of Product Development at SKUR.  SKUR is a young software company that has just released its first product, which automates comparing a 3D digital model to reality as represented by a point cloud from terrestrial scanning technologies (typically LiDAR).  The scan can be of an existing structure or a new structure in construction. The variances are prioritized and can be inspected in an industry standard 3D visualization application.

Nicolas, SKUR provides technology that finds the differences between digital models and point clouds from scans.  What are some of the applications of this technology ?

We are in the business of comparing 3D building models with point clouds that are registered to the building model. We do this for two very specific use cases.

The first use case is what I refer to as construction QC.  Let's say you have a design model and during construction you capture a point cloud by laser scanning or some other technology.  What you are interested in is comparing the design model to what was actually built out in the field. To do this you simply upload the design file and the point cloud, specify three tolerances that are used to prioritize the differences between the model and point cloud, and then run our cloud-based application.  The application outputs colour-coded deviations, which you can visualize in Navisworks.

The second use case is what I refer to as model QC where you have the reverse situation.  Let's say there is an existing structure, a building or almost anything.  But there is no digital model and you need a model for a renovation or retrofit.  You can use a scan-to-BIM process to scan the building and generate a point cloud from which a model can be created.  The model creation could be done by your own people, by a third party provider, or by an automated process such as Clearedge.  In this use case what you are interested in is how  well the model fits with reality. You follow the same process, uploading the model and the point cloud, to compare the model to the point cloud to see where there might be modeling errors and other types of deviations.

These are the two use cases that SKUR serves. We target the sophisticated construction contractor (GC).  One application of the technology for a GC is to perform construction QC on their projects to look for problems.  Another application is pre-construction, when the GC is considering bidding on a renovation or retrofit project and wants to provide a digital model to the architect.  The GC wants to be confident that the model represents the structure accurately so he can tell the architect that the model is a reliable representation of reality. These are the folks that we are targeting for these two use cases.

Does SKUR provide data capture services ?

No, we are a software company.  If you are interested in this technology, you can create an account, it's up to you to capture the point cloud data however you want to do it, upload it onto our web-based system together with a digital model and then run our application.   You then download the variances and visualize them using an industry standard tool. We don't do any consulting.

I'm interested in how SKUR would be used during construction. What if the construction is only 10 % or 20 % complete, do I have to specify which part of the incomplete structure I have scanned ?

We have very smart algorithms that figure out which part of the structure to look at.  For example, if you bring us a point cloud of just the corner of a building and the model of the entire structure, we only focus on that corner of the building.  Another example is a building that is 12 stories high, but I only have a scan of the basement. Our software is smart enough to figure out the part of the model that relates to the scan.

What about the opposite case where during a scan-to-BIM process for an existing building, I have not finished the model so I only have a model for part of the building ?

It's basically the same algorithm. The software is smart enough to figure out which part of the scan relates to the partial model.  When you run our software, you simply upload the model and the point cloud and specify the tolerances.  The process is the same for construction QC or for model QC. From the perspective of the software, it's really the same thing.

Where does your software run, on a desktop or in the cloud ?

We are completely web-based, that is to say, cloud-based. Our software require heavy-duty processing that would not have been possible 5 years ago. It is all in the cloud and it's high performance multi-core computing.   We bring in the largest computers that are available on the Amazon cloud for these problems and they can generate the variances in a few hours.

What types of digital models do you support, 3D CAD, Revit, Bentley BIM, or IFC models ?

We are a young company, and we've just started.  Our first release supports 3D DWG models. But I think Revit models are the next logical step. And then we also plan to support the Bentley world.  The 3D DWG model really can be anything, with any level of complexity. I have had people asking me about things other than construction, like planes, such as an Airbus. I talked to a gentleman yesterday, who wants to use our software for a golf course.  As long as you have a DWG model and a point cloud that's registered to the DWG, we can find the variances between the two.  The registration between the two is critical. It has to be done accurately.

How difficult is it to choose the tolerances ?

It's very easy.  You choose three values corresponding to different degrees of variance which are colour coded, green, yellow, and red in the output.

How do you visualize the variances, I imagine for a large project there could be thousands of these ?

SKUR output is a visual representation and report of the detected locations where there are discrepancies showing the precise variance from design and, very importantly, the direction of the variance.  We colour code the discrepancies so you can spot the problem areas very rapidly. For example, in the graphic red and yellow coloring indicates degree of variance. Arrows show the direction of issue. In our experience most of the variances are green, so you can filter these out and just look at the ones colour-coded red, which are the big discrepancies.  This allows you to focus just on those important areas.  We're like an insurance company.  Most of the time it's fine, but when it's not you need to know.

Where did the algorithms come from  and where was the implementation done ?

We've invented everything from scratch in Oakland, California

I can imagine this technology being of tremendous value in many different sectors of the construction industry as well as in non-construction sectors.  Which sectors and geographies are you targeting as the market for your technology ?

The sector we have targeted so far are construction contractors (GCs). But going toward we will be increasingly focussing on owners.  We think that our software provides a view into the construction process that enables them to see during construction how well the designs by the architects and engineers compare to what is actually being built.

We are focussing on North America, but we also have clients in Europe. Europe is definitely high on our list, but as a young company we are focusing on close to home first.

If people want to try your software, how do they engage with you?

It's very simple. They sign up and create an account.  Then they can upload a model and a point clouds and hit the magic button. We do handholding in the beginning to help new customers.  But we don't do any data capture and or any data preparation. We simply provide a web-based software service.

Geo Business 2016: Geo-enabling BIM - Call to action

At this year's Geo Business 2016 event in London two sessions were held focussing on geo-enabling building information modeling (BIM).  Sponsored by Survey4BIM, an initiative supported by the UK Government BIM Task Group, the meetings were intended to encourage the geospatial community to become involved in defining standards for survey, collection, management, processing and delivery of geospatial information within a BIM context.

Since accurate 3D geospatial data underpins the ability to define level of detail (LoD), generalization, and preservation of designs through to construction and operation and maintenance in the real world, it needs to be defined and standardized in the context of construction.  Surveyors understand accuracy and precision and that it can be quantified and needs to be controlled in a process.  New technology such as laser-scanning is increasingly being used within the construction process and this is changing processes such as as-builting. 

The objectives of the geoenabling BIM initiative are to define best standard for consistent accuracy unit and work with Open Geospatial Consortium (OGC) to establish common approach to accuracy meta-data and attribution.  It also intends to look at available standards from industry and  professional organizations and BIM software vendors on how they currently support generalization.   Another area of focus is 3D data.  The groups intends to identify and engage with 3D specialists outside of the direct construction community who specialize in generalizing data for visualization and other areas where 3D  data is used.

Survey4BIM is a cross-industry group open to all organisations involved in the survey, collection, management, processing and delivery of geospatial information within a BIM context.  The group's make up is reflective of the sector, involving clients, contractors, consultants, suppliers, sub-contractors, institutions and other organizations.

The mission is to provide a forum for survey organizations and industry professionals to collaborate and share their journey putting “BIM into practice”, and to provide best practice guidance documents on survey matters relating to BIM.  To achieve this the groups has identified several goals;

• Provide leadership in establishing how surveyors provide a critical role in the BIM process
• Increase awareness of the importance of geospatial information in the BIM process
• Share knowledge and learning
• Establish collaborative links to academia
• Produce and publish best practice guidance to organizations, clients and industry professionals on geospatial matters
• Coordinate the Survey4BIM related activities undertaken by geospatial organizations
• Promote survey related case studies, which demonstrate best practices
• Promote the integrated management of geospatial information across all stages of the asset lifecycle.
• Represent the geospatial industry in wider discussions on BIM
• Act as a partner to the Cabinet Office BIM Task Group.

You can register to participate in the Geo-enabling BIM initiative here.

Spar 3D: 3D mapping the inside of multi-story buildings with a backpack

At the Spar 3D Expo and Conference in Houston The Woodlands, Texas, Avideh Zakhor from the University of California Berkeley was onsite with some remarkable products from her new startup Indoor Reality.  I blogged previously about the technology when it was still under development.  The "man portable system" she has developed is a 32 pound backback with a collection of sensors.  The most important feature of the technology are sensors and algorithms for accurately tracking the six degrees of spatial freedom which determine the location and orientation of the backpack in the building.  In addition there are sensors for recording optical and thermal point clouds. She has also developed a handheld device that supplements  the capabilities of the backpack.

The technology can track accurately the location and orientation of the backpack as the operator walks through the building going into all rooms, halls, up and down stairs and into other human accessible spaces.  Software tools then generate different 3D data products including optical and thermal 3D point clouds, a colourized 3D point cloud, surface texture map,  and triangulated mesh surfaces.  The package includes analytical software tools for automatically generating floor plans, distinguishing rooms, identifying staircases, identifying windows, finding heat sources such as computers and human occupants, and identifying lights. Surface textures can be added to create a 3D photo-realtistic rendering.  The sensors and software can capture and generate all required input for the Department of Energy's EnergyPlus energy performance analysis program.  It has GPS and can generate real world coordinates.  Indoor Reality is an ESRI Emerging Business Partner.

I had a chance to sit down with Avideh to talk about her new company and the remarkable products she was demonstrating at SPAR2016.

Avideh, what are the key features of the products you are demonstrating here that distinguish them from some of the other devices vendors are showing here at SPAR ?

We have developed a unique way to track where the operator is in a building and the orientation of the backpack.  We have a lot of sensors. That is the key to all of our intellectual property (IP). It's called sensor fusion and most of our IP is in that area. It does not suffer from drift. It tracks accurately where you are in the building even if you go upstairs. You can even go up in an elevator because it has a barometer.  In addition we detect wifi.  It can tell you what the wifi strength is in different parts of the building. We have an infrared sensor and we have developed an app that will give you real-time occupancy of the building, for example, we can tell you there are four people in this room, two people in that room. We have patents on that.

Based on the data collected by the operator's walk through the building, we allow users to virtually navigate through the space and spatially collaborate with each other through our web based visualization tool. During virtual navigation, you can look around in 3D as if you were there.  That's our second data product. We don't just record dumb point clouds.  We generate intelligent meshes.  Not only can I can pan around, look up and down at each location, but the shape of the cursor shows you the orientation of the surface - for a horizontal surface it's a circle, for vertical surfaces it's a square.  The cursor gets bigger as it gets nearer, smaller as it goes farther away.  We know the normals of all the surfaces because of the 3D mesh behind what you see.  We recognize doors and windows.  Of course, we can measure distances and we allow users to annotate objects and locations.   So this mesh is very intelligent, so intelligent that we can automatically generate a Revit model with a floor plan, rooms, ceilings, doors and windows. That is our third data product.

How do you foresee people will use the tool ?

We went to the University of Illinois just a month ago and we did an energy model for a 14 story bldg. We can do 3d geometry, we can detect lights automatically, we can detect windows automatically, and we can detect AC plugs automatically.   This gets fed into an energy simulation model which is a complete model for an Energy Plus performance audit.  Typically when people do an energy audit they are only interested in the lowest hanging fruit, for example, replacing incandescent light bulbs with LEDs.   This is because collecting the data for a full Energy Plus model has been time consuming.  With the Indoor Reality backpack you can collect whatever you need for a complete Energy Plus audit very simply. For example, we can superimpose the infrared record on a ceiling to show a hot area which is very likely a hot water pipe behind the dry wall.

We have just come out with a visual building documentation tool for stakeholders to collaborate  with each other throughout the lifecycle of a building - during design, construction and operations and maintenance.  If I want to remodel a building using an architect, designer and contractor they can communicate with each other by leaving notes for each other in the virtual building.  For example, I can put a tag on a door for John the contractor. "Please fix this door."  John can come along in the virtual building and edit that note "Avideh, I can do this Tuesday."  So this makes it possible to collaborate virtually on a building project in 3D,  in much the same way that we can collaborate on a Word document.

There is another dimension to this.  Let's say we want to remodel a space in a building. The first thing the architect will want to see is the existing floor plan and the other professionals will also want to see other aspects of the space.  Normally at this point the building owner will ask everyone involved in the planning to come over for a  walk-through the building.   That requires a lot of valuable time from very busy people.  With the virtual model that we can generate, most of this won't be necessary any more. The owner can ask a relatively junior person equipped with the backpack to walk the building to collect the data.  Then the resulting model can be put online and everyone can virtually walk through it.  For bidding on a big renovation project, a relatively junior operator does the walk through and puts it on the cloud and the construction companies who are interested in the project can virtually walk through it online for bidding purposes.

For a new building,  during the actual construction you can conduct a walk through every day and update visually what's going on.  Then if there are lawsuits in the future, you will have a permanent record of who did what and when.   You can show the legal folks where a component was supposed to go and where it was actually installed. It is also useful for monitoring progress, so you know who did what when to determine who gets paid and how much.   If there is a lawsuit and insurance companies get involved, you can step through the record to see who did what.  Then when you want to sell the finished building, the real estate folks can do a virtual walk through of the building with prospective customers.  Property managers and facility managers can use the virtual model for maintaining the building. For example, we talked about the energy performance audit.  Facility managers can use it to identify lights that need to be replaced with more efficient bulbs. Then they can put a virtual tag on the lights for the maintenance folks.  With a virtual model like this, owners, contractors, designers, and facility managers can all collaborate throughout the lifecycle of the building.

There seems to be a lot of areas where the technology you have developed could be used ?

There are quite a number of other application areas for the technology we have identified including historical building and monument recording, scan-to-BIM,  variance detection and virtual inspection, real estate valuations, building plans for first responders, change management and dispute resolution, and detecting thermal anomalies including water leaks and insulation deficiencies to name just a few.