Between the Poles

Be Inspired Awards 2016 reveal rapid adoption of reality capture in construction

At the Year in Infrastructure 2016 conference in London the Be Inspired Awards set some new records. There were 311 submissions from 22 countries.  Of these 110 were completed and operational projects and 120 were under construction.  60 projects were chosen as finalists in 18 categories including asset performance, power generation, rail and transit, mining, roads, utilities and communications, water network analysis, and construction.

15 of the 60 finalists reported using reality capture which is evidence of remarkably rapid adoption since Bentley only acquired a reality capture capability (Acute 3D since renamed as ContextCapture) in February of last year.  At this year's event Bentley announced that ContextCapture can now process LiDAR point clouds.

Crowd behaviour modeling with BIM

I spent a fascinating day at the CanBIM Regional Session & Technology Exhibition in Toronto.  I was very impressed by how much things relating to BIM have progressed in Canada, which has generally lagged the countries spearheading BIM development, partly because of government inaction.

One of the most interesting presentations today was by Erin Morrow of Arup who talked about two pedestrian simulation products he has been responsible for developing, MassMotion and Flow.   These are 3D agent-based models.  They are based on real spaces with continuously connected 3D geometry and crowd interactions.  Agents have size and acceleration, personalities that prefer things like avoiding going up stairs.  They can dynamically assesses their environment and make routing decisions.  Social forces determine how agents respond to their environment, other people, and goals.  They try very hard not to hit obstacles or bump other agents.  Agents are able to make autonomous decisions based on this continuous space.  For example, MassMotion can simulate how agents will be behave when faced with a choice between a crowded elevator or empty stairs. 

There are also tools for assessing the result of a simulation including 3D dynamic visualization and various quantitative metrics such as density in various parts of the model, how long it takes people to get from one point to another, what route do most people take, flow rates through doors and up escalators, and and others.

The simulation tool defines portals, where people can enter or leave from, spaces where agents can go anywhere, connectors that connect rooms and other features.  Erin demonstrated live how to create a simple model with two rooms, connected by a door, with a portal where people entered and an exit portal.  He created a simulation where 100 people enter the first room every minute and visualized the crowd flow in 3D.  It was fascinating to watch how people bunched in front of one of the connecting doors and then the other.

The simulation geometry can be developed from scratch in MassMotion.  But for complex structures it is much simpler to import a BIM model (MassMotion imports IFC models from a variety of software including AutoCAD, Microstation, SketchUp, Rhino, and Revit. ) to create spaces and connectors including stairs and escalators.  Erin demonstrated doing this with a BIM model of an airport.  MassMotion was able to correctly identify 90% of the elements it recognizes such as rooms, doors, stairs and escalators.  In the demonstration some editing was required to make sure there was a continuous geometry.  For example, an escalator needed editing to logically connect to the upstairs floor. Then the user adds other information such as portals, sources where people come from, sinks where people go and rates of flow through the portals people enter from.  People may come from outdoors or train cars and they may be going to gates to catch planes.

The software has roles to play during design, during construction, and as a management tool throughout the lifespan of a building and surrounding areas.  It can be used operationally during construction to design alternate routes during construction.  The software was used to simulate passenger movements during a full day at the new Jetblue T5 terminal at JFK airport in New York with the objective of ensuring passenger comfort and fluidity of motion in the terminal.

There are many other applications used by engineers and planners including evacuation analysis.  One of the interesting results that Erin mentioned was that the Roman Coliseum built in 70 CE could be evacuated in 15 minutes, the same time it is estimated it takes to evacuate the recently built Olympic stadium (the Bird's Nest) in Beijing.

Autodesk utility design and substation products spun off to SBS

It has just been announced that Autodesk and Spatial Business Systems (SBS) have formed a strategic alliance.   SBS will be responsible for future development, support, and licensing of Autodesk^® Substation Design for Inventor^®, Autodesk^® Substation Design for Electrical, and AutoCAD ^® Utility Design (AUD) software.   SBS will focus on the ongoing development and support of Substation Design and AUD, while Autodesk will continue to focus on the underlying platform technologies. This is intended to enable SBS to employ innovative, customer-focused development processes to support the new product enhancements necessary to address the evolving needs of the utility industry. The new products offered by SBS will be renamed to reflect the new structure.

The Autodesk team responsible for product development, system deployment and support will join SBS and work to incorporate new enhancements that will more closely integrate with the SBS Utility DataHub™ suite of utility integration products.

This sounds like very good news for Autodesk utility customers.  I have known Dennis Beck, President of SBS, for years.  He and his team know the utility business intimately having developed spatially-enabled utility solutions integrating Smallworld, Autodesk, Oracle and other products for a wide range of utility customers.

BIM adoption in the UK surpasses 50%

The UK Government's goal for construction industry is to reduce the CAPEX for public construction projects.  To reach its goal for the construction industry the UK Government has undertaken several initiatives, one of which is a commitment to embrace Building Information Modelling (BIM) in Government projects. The drivers for this incredible effort to transform the UK public construction industry over a 4-year time frame are to reduce costs, improve delivery times, lower emissions and create export opportunities for UK firms. The Government’s mandate for Level 2 BIM came into force in April of this year.  Since then any public sector project that is funded by HM Government will need to be done through collaborative 3D BIM. 

The NBS does a National BIM survey annually to determine what the increasing adoption of BIM in the construction industry means for construction product manufacturers.  More than 1000 people responded to this year's survey.  It represents one of the most accurate depiction of BIM adoption and usage in the UK. 

According to the survey 54 % of designers are currently using BIM.  Another 42% are aware of BIM, but are not yet using it.  Only 4% don't know what BIM is.  It is interesting to compare this to 2011, when only 13% of people were using BIM and 43% did not know what it was.  Of designers who reponded that they are aware of BIM, 86% expect to be using BIM by next year, and 97% within five years. The Government is already reporting significant cost savings through BIM and is making further investment in it.  It is concluded that BIM adoption will continue to rise in the UK.

In the NBS National BIM survey, designers were asked whether they feel that BIM will help the UK construction industry reach the Government's targets. 63% of the respondents felt that BIM will lower costs by about a third.  57% responded that BIM will bring about a 50% reduction in the overall time, from inception to completion, for new build and refurbished assets.  39% believe  that BIM will reduce emissions in the built environment.  

The maturity of BIM in the UK and the rise in BIM adoption globally is believed to represent an opportunity for UK manufacturers, but fewer than a third of the respondents believe that BIM will help reduce the trade gap between UK exports and imports of construction products and materials.

On-site AR visualization of BIM designs in an urban environment

There are two advances in technology that are having almost as much impact on mapping our world and tracking the location of things in it as the introduction of the GPS.  The development of microelectromechanical systems (MEMS) in the 1980s and 1990s has made possible the mass production of micro inertial measurement units (IMU) comprised of a combination of accelerometers, gyroscopes, and magnetometers.   These have enabled the smart phone which contains a range of sensors, typically an accelerometer, thermometer, gyroscope, pressure sensor, humidity sensor, light sensor, location sensor (GPS),and a magnetometer.  The second major advance is fast structure-from-motion (SfM) algorithms.  One of the first applications is simultaneous localization and mapping (SLAM) technology which was developed for autonomous vehicles. In 2012 GeoSLAM was developed which made it possible to solve a major spatial problem, accurately tracking the location of a person inside a building or in an obstructed environment such as a city.  Recently SfM has been successfully applied to indoor mapping in the form of new professional-level products from Indoor Reality and Leica Geosystems (Pegasus Backpack).

It turns out that structure-from-motion technologies can also be used for on-site visualization of a virtual BIM design in a real urban context.  At this year's Geo Business 2016 I had a chance to talk to Crispin Hoult about an augmented reality product called UrbanPlanAR his company Linknode is developing with SfM technology from Heriot-Watt University that is designed to do this on consumer handheld smart devices such as an iPad.

Background

For years researchers and technologists have been struggling with a major spatial challenge - accurately tracking the location of a person in an obstructed environment such as a city or inside a building in 6 dimensions, 3D location and 3D orientation.  Stucture-from-motion solves this problem and enables you to track your position and your orientation in an obstructed environment where GNSS satellite signals are unavailable or unreliable. Typically input from an inertial measurement unit (IMU) and LiDAR is used to construct in real-time a 3D map of the immediate environment including objects such as walls and other objects.  As you move, the map is reconstructed and this is used to record your trajectory.  The algorithm is robust and helps to addresses some of the problems, such as drift, that results from using just an accelerometer.  

Indoor mapping products, such as those based on GeoSLAM, assume that you are working in an unknown environment.  But what if you already have information about the environment in the form of GIS maps, point clouds, and other georeferenced information ? Tracking your location in a known environment using just a smart handheld device is the problem that Crispin Hoult and his company Linknode are addressing.  The specific business opportunity he is targeting with this technology is on-site (in-field) augmented reality visualization of BIM designs for new buildings with a consumer smart handheld. 

Technically SfM algorithms require multi-sensor input.  The typical smart phone or tablet has many motion tracking MEMS devices built into it.  If you are outside in an unobstructed environment, the GPS sensor can reliably report your location to within 5 or 10 meters.  But when you enter an obstructed environment where you have reliable georeferenced information, you need an algorithm that uses the camera and the inertial navigation sensors in the handheld tablet to accurately track its location and orientation in real-time.  Crispin has been working with Heriot-Watt university to "mobilize" an algorithm that does just this. Basically the algorithm uses “visual-inertial tracking” for the online tracking of the 3D position and 3D orientation of the tablet. This is supported by ‘prior’ learning of the visual and structural appearance of the environment through structure-from-motion (SFM).

Crispin, what is the business problem that UrbanPlanAR addresses ?

We are targeting UrbanPlanAR at on-site visualization using only a consumer smart handheld.  We are making it possible for an architect, owner, or city planner to come to a site with a handheld tablet and in real-time overlay a BIM model showing the design for a new building in the context of the real world.  This is an application of augmented reality.  What you see on the tablet is the real world and into that we superimpose the BIM model.  Our focus right now is to visualize the building design in its urban context - making it possible to walk around in the area and see the building from different viewpoints. We add value by providing more trusted, more contextual visualizations so people can make decisions and engage more with the planning process.

The alternative way of doing this is to create a completely virtual world (often referred to as black box), but then you have problems with resolution and realism.  We have research carried out by the University of Glasgow that shows that by doing on-site visualization and not blackboxing it, it becomes more trustworthy and enables more engagement from stakeholders.

Another way to look at the business value of UrbanPlanAR is that it adds value to your BIM investment. Designers and contractors invest in BIM because it enables automated clash detection, material takeoff, quantity surveying, construction staging and many other applications. One of the important value-adds that hasn't been fully recognized is visualization.  Our technology can be another strong reason for creating a BIM model in the first place.  If you have already developed a BIM model, it adds value to your model. With our software you can take your BIM model, walk out to the proposed site and show all stakeholders - potential customers, city planners, city engineers, and nearby land and building owners what the proposed structure is going to look like on site, in a real world context.

How does this work, what do I need to do to make this augmented reality visualization of a building design ?

The first step is to choose the context.  Unlike the traditional approach where we would choose a location from which we want to view the building design, with UrbanPlanAR we choose an area of interest, which could be a city square, for example.  Then we collect georeferenced information about the area we have chosen. Currently we use imagery captured specifically for that viewpoint site, but this will soon allow other alternatives to be used, such as existing datasets, for example, a city model, LiDAR scan, oblique imagery, or satellite or overflight imagery.

The next step is to get the BIM model of the design from the architect. To do this we plug into BIM 360, which is the Autodesk data store for BIM models. For smaller projects we can take a model published directly from Revit.

At this point we have the context and we have the BIM model. Then we simply take the mobile device out into the field on a tripod or simply held in the hand, and UrbanPlanAR will use the context to work out exactly where it is and enable you to see in real-time your design within the real world – displaying the BIM model not only “onto”, but visually “within” existing reality.

Unlike Virtual Reality systems where there are cost and time limits to the digitization process, what you see in augmented reality is 3D, infinite resolution that all comes for free as the real world.  What we are showing is a combination of real and virtual. As users stand at the site, they see exactly what is there plus the change. That's what adds to the trust - you are on-site, out in the field or the city square, walking around and showing stakeholders what is actually there with the superimposed virtual model. You can also record it so you can replay it back in the office.  That real-world GIS-based augmented reality we call that GIality. Everything is georeferenced, everything is anchored in the real world. That is what links the virtual to the real world.  You need to know where the virtual building is in real-world coordinates.  The technical challenge is that you also need to know  where the observer is and his/her orientation as he or she walks around the site.

Who do you foresee is the end user of UrbanPlanAR ?

I would say that the primary application is early stage project feasibility and development for a new building. It could be used by the architect or by the developer who is funding the architect. It could also be used as a sales tool at this stage.

The second stage is the planning stage. When you are looking at permitting you need the convince the city or town council, who are the custodians of the city. Typically landscape architects are hired to develop and present the environmental statement and impact of the project to the city planner. But we now live in a world where we no longer have to rely on consultants to convey information.  Nowadays the developer may do this him or herself.  The city or town planners may also be users - as validators of what has been presented. UrbanPlanAR broadens the breadth and depth of 3D information available about the project.  It is also interactive in that the architect can make changes to the BIM model and synchronize that to the users.  Another advantage is that the users can go out to the site at any time to inspect some detail of the design that they may have missed.

What we have found useful is to interactively display different surfaces or finishes, different building heights, and different models. Preconstruction changes are very easy to do.  The architect goes away after getting feedback from stakeholders, creates a new or modified BIM model, uploads it the cloud, and then a user synchronizes with the store and takes it out to the site.  By doing this at the feasibility stage, early on in the planning process, communities become more involved and don't see what is being presented as almost a fait accompli. You want them to leave feeling they are actively involved in the design process - then you get more buy-in and there is less likely to be an appeal during the construction stage.  

Does the BIM model have to be a Revit model ?

We integrate with BIM 360 today and we are planning to support Bentley BIM.  In addition Industry Foundation Classes (IFC) is one of our supported file formats.  But this is not yet equivalent to importing files directly from Revit or Bentley BIM, because exported IFC files from some BIM vendors do not support texture.  

What types of hardware does your software run on ?

We support the iPad Pro and we are working on a Microsoft Surface port.

Does UrbanPlanAR require preprocessing in the cloud or on a desktop ?

Neither the BIM model nor the geoferenced context ends up being the operational model we need to do the localization, but both are valid inputs.  All the input data including GIS context and Revit model is preprocessed on a desktop or in the cloud and then synchronized to the handheld.

Crispin, you already have a product that is designed for rural environments but also relies on augmented reality.  Can you tell me about that ?

Yes it is called VentusAR, and it is designed to work in a rural environment.  It has been targeted on visualizing wind farms and solar farms for land owners, communities and planners because these farms are very contentious issues. Before our our product became available, the standard methodology was to take photographs out in the field, go back to the office, process them on the desktop to incorporate the proposed wind or solar farm, send the resulting images to the printer and then show the picttures to the stakeholders. Because of the time and cost it tended to be done late in the planning process, which made it difficult to make changes.

Late last year for the first time we used VentusAR for visualizing transmission towers.  And a few weeks ago we did our first modeling of transmission lines.  As with UrbanPlanAR, VentusAR uses the tablet’s integrated camera, GPS position and 3D gaming capabilities to create an accurate, location-specific, realistic view of transmission tower models in context.  You can investigate different scenarios with different stock models like towers, turbines, and panels. It works well in a rural environment because our georeferenced information about the real world is typically a digital terrain model. In a rural environment, if your GPS is out by 5 meters and you are looking at something a hundred meters away, it doesn't matter that much.  In an urban environment where you need more precision and can't rely on GNSS, you need a more sophisticated approach, which is what we have done together with Heriot-Watt University for UrbanPlanAR.

The project is supported by Innovate UK, the United Kingdom’s innovation agency (Project 102040).

SPAR2016: Digitalizing highway construction at the Oregon DoT

At SPAR 2013Ron Singh, Engineering Automation Manager/Chief of Surveys, Oregon Department of Transportation (DoT) made a startling proposal that would stand the traditional highway construction process on its head by making the post-construction as-built survey the critical source of reliable survey and asset information in the form of a geo-referenced 3D BIM model (also known as horizontal BIM, heavy BIM, VDC, civil information modeling, BIM on its side, and BIM for infrastructure) and associated scanned and other data.  One of the implications of this is that when a new project is initiated in the same area over 80 % of the necessary survey-grade information is already available in the DoT's geospatially-enabled database making the traditional complete resurvey unnecessary - saving time and money.   Ron foresees that one of the consumers of this digital highway information would be autonomous vehicles.

This year at the Spar 3D 2016 Expo and Conference in Houston I had a chance to sit down with Ron to see how his and other DoTs have progressed on his vision since then.

Ron, the last time we chatted you mentioned you were organizing an event for Western State DoTs in July 2014. How did that go?

It was very successful. I did this on behalf of the Federal Highway Administration (FHA) who approached me and said they planned to do an event similar to the one I organized in 2010 which was called Design to Dozer. In that one we used digital technology including automated GPS-equipped dozers, graders, and excavators to build a sub grade of a road.  It was well attended and it really made a change for my DoT.

The FHA asked me to do another one to promote the use of intelligent construction technologies by providing information and training relating to 3D design, automated machine guidance (AMG) and related technologies for highway construction.  This time I decided to go all the way to paving including ditches, culverts, and drainage. I called it Design to Paver.  We invited the DoTs from the18 Western States plus Florida, North Carolina, and Michigan.  From each Dot we invited their chief engineer and their chief construction engineer.  Altogether we had about 250 people at the event.  It was a two day event and combined classroom work in the morning with on-site in afternoon.

This event included building a demonstration road but I wanted to build one we wouldn't have to tear up and restore. I found a piece of land that was a training facility for the Oregon National Guard. It was a big complex of about 40 or 50 acres and had been a training site since WWII. I went out to see the site and met with them and their environmental people.  I sketched out a plan for a road about 1500 feet long. I picked a site where the road would gain 200 feet in elevation because I wanted it to be fairly complex. I wanted to make some radical curves with superelevations and exaggerated superelevations, so we would have some drainage issues to deal with.  I wanted it to look like a real road and have some interesting geometry. No underground utilities. I wanted to take it from bare earth to a finished 1500 foot road with several curves, ditches, culverts, and paved in less than 2 days.  I don't know exactly but with a traditional approach this would probably have required about a week's work.  We could actually have done it in less than two days, but we were demonstrating.

We did all the surveying for the site months in advance.  We also had a geo-referenced 3D design model with rendering and animation.  We brought in the construction equipment the day before.  We had training sessions for the operators on safety including unexploded ordnance since this was a military training site.

I chunked the 1500 feet of road out in pieces and choreographed the whole thing.  For example, the dozer operator on each section of the road did not break ground until the audience showed up.  They stepped out of the buses and on their badges they had colour codes which allocated them to a team of about 25 people. Each group would go to station A at staggered times.  At station A they would be showed automated dozer operation. Then at a certain time they would go to station B and so on. It was very well coordinated.

What impact do you think the event had on the highway engineers who attended ?

It was amazing - the feedback through the emails I received was so positive.  People still talk about it. If you go to any of the DoTs who attended, the staff who had been there were just amazed.  One of the reasons is that what we were able to show them was something that would have taken a single individual many, many years to see, because none of them had experienced a project with automated equipment from breaking ground to paving.  They were able to experience it all in 2 days. We had all types of equipment, dozers, excavators, curb machines, paving machines, and so on.  We even had unmanned aircraft system (UAVs) for monitoring the construction and ground penetrating radar systems for detecting underground utilities.

Is there a way to find out more about this event?

We created a web site DesignToPaver.org which we intend to leave up for two more years. There is a guide book which you can download. It has more than the agenda. It has all the names of all the companies and  presenters.  If you would like to ask somebody questions, you can find that information in there.

Do you think the road map and time frame for 3D models and construction automation that you outlined is still realistic or is it accelerating?

The technology behind autonomous vehicles is really accelerating as is the engineering automation of highway construction, but the social and legal aspects of highway construction may be the barriers that are keeping it in check. If it were only up to to the technology we would be there much quicker, but I think the social and legal aspects are actually providing good balance, by making sure we are not breaking something that we can't replace.

You mentioned the legal aspects, do you think new legislation will be required to enable to the full digitialization of highway construction and maintenance ?

As long as you put a box around the question and restrict it to engineering automation, not autonomous cars, I think we are well-suited to get there with the existing laws.  I could imagine that to further capitalize on this technology, legislation to register another profession, such as remote sensing, might be beneficial.  I don't necessarily feel that it has to be a surveyor's function to do, let's say, mobile mapping or UAV mapping. I feel it needs to be somebody who is licensed, who has gone through training, who is able to prove that they know how to do this work, and who maintains ongoing professional development training.  The key requirement is that they stand behind their products in a legal sense.

There are many different technologies for capturing data now, even a consumer digital camera can be used to create a point cloud. If I  am a highway engineer how do I decide what data I can rely on for designing a highway ?

The data that has the level of rigour required for engineering design is what I call authoritative data. As an example, if you intend to use a boat with a sonar system and cameras for underwater bathymetric surveying in search of treasure, that's fine. I don't think anyone should regulate that. But if you are going to use the same tools to model bathymetry for the design of a public bridge, then it better be authoritative.  It better be data that you can track the lineage and source of.

One of the key questions I have is about the database of digital data including, in the case of Oregon, data from overflights and mobile LiDAR for the entire state highway system plus the digital as-built models and other data from actual construction projects.  Who do you see is responsible for maintaining this database including updates from ongoing highway maintenance work ?

In Oregon we think we at the DoT are responsible for maintaining it. As long as it is inside our corridor, it was either created by us or by our agents, companies that we hired. There is tremendous value in this data, but the value is only there if in the future people feel confident in using the data.  For that they need to know its provenance.  For example, if you consider our as-built plans, we have had these for generations, but we don't rely on them for anything but to help us find things. We don't rely on them for survey data, because the current as-builts were never created with that in mind.

If we collected very accurate, very rich data and we didn't track its lineage including metadata about how it was captured and when, a future engineering user would legitimately say I don't trust it, because I have no narrative about this data, for example, even something as fundamental as what coordinate system it is in.  But if we can compile all this information together in the form of comprehensive metadata that gives the future user trust in its provenance and reliability, then there should be nothing to prevent the data from being reused.

Are contractors who bit to build roads in Oregon interested in the data you are collecting - do they ask you for it ?

For construction project work, when we use mobile LiDAR or terrestrial laser scanning, we are very careful about precision, for example, we put in ground control points. We do the entire survey in such a way that we end up with a certain level of accuracy, a certain level of point density, and a certain level of reliability. 

When we collect mobile LiDAR data for the whole state, it is impossible to do collect data with the same rigour for the whole state.  We don't put ground control points in and we don't do other things that are required for high precision and reliability.  The data is still very usable for many things, but it is not authoritative, high accuracy survey data, that is required for engineering design.

If you are planning a project in an area where there was a previous project with a digital model, high precision data including ground control points and high density laser scans, would you tell contractors planning to bid on the project about the existing data ?

Absolutely. If we don' t tell them then we might as well as had not collected, documented (metadata) and stored it. We have to tell them because it helps drive the cost down.  To give you some background, our first automated machine guidance (AMG) project was in 1997.  In Oregon some contractors had invested in AMG technology and wanted to use it. In those days contractors who had done this were winning the bids by lowering their bid just enough to beat the other contractors.  When they did the work with AMG technology, they saved a lot of money, but most of the savings was going to the contractors, not to the DoT.  To redress this, starting in early 2015, we began specifying on our bid web site that projects were "AMG ready". We provide all the data we have which they can download to see the models, mobile LiDAR, overflight data, and so on.  The effect is to drive bids down, because there is less risk for the contractor.  Asa result we at the DoT are now sharing in the savings.

Is Oregon alone among DoTs in applying these technologies or are other DoTs also applying them ?

There are about five or six that are leaders in using different aspects of digital technology.  For example, the Wisconsin DoT is focusing on 3D design - they are ahead of us in this technology. Michigan is leading in some areas,  for example, data storage and data management. We met with them because we were interested to finding out what they were doing with engineering data management.  In return they were interested in what we were doing with 3D design and AMG. Utah his way ahead of most DoTs in asset management.  None of the DoTs including us are applying every aspect of 3D digital highway construction and maintenance.

Are there any ROI studies demonstrating the financial benefits of these digital technologies ?

We are doing one. We have a research project we call the OSU Engineering Automation project with Oregon State University in Corvalis.  They are studying our entire engineering automation process. They have already put a good project team together.   The final report is due by next April, about a year from now.

Automating the process of comparing digital models to reality

At the Spar 3D Expo and Conference in Houston I had the opportunity to chat with Nicolas Arnold, VP of Product Development at SKUR.  SKUR is a young software company that has just released its first product, which automates comparing a 3D digital model to reality as represented by a point cloud from terrestrial scanning technologies (typically LiDAR).  The scan can be of an existing structure or a new structure in construction. The variances are prioritized and can be inspected in an industry standard 3D visualization application.

Nicolas, SKUR provides technology that finds the differences between digital models and point clouds from scans.  What are some of the applications of this technology ?

We are in the business of comparing 3D building models with point clouds that are registered to the building model. We do this for two very specific use cases.

The first use case is what I refer to as construction QC.  Let's say you have a design model and during construction you capture a point cloud by laser scanning or some other technology.  What you are interested in is comparing the design model to what was actually built out in the field. To do this you simply upload the design file and the point cloud, specify three tolerances that are used to prioritize the differences between the model and point cloud, and then run our cloud-based application.  The application outputs colour-coded deviations, which you can visualize in Navisworks.

The second use case is what I refer to as model QC where you have the reverse situation.  Let's say there is an existing structure, a building or almost anything.  But there is no digital model and you need a model for a renovation or retrofit.  You can use a scan-to-BIM process to scan the building and generate a point cloud from which a model can be created.  The model creation could be done by your own people, by a third party provider, or by an automated process such as Clearedge.  In this use case what you are interested in is how  well the model fits with reality. You follow the same process, uploading the model and the point cloud, to compare the model to the point cloud to see where there might be modeling errors and other types of deviations.

These are the two use cases that SKUR serves. We target the sophisticated construction contractor (GC).  One application of the technology for a GC is to perform construction QC on their projects to look for problems.  Another application is pre-construction, when the GC is considering bidding on a renovation or retrofit project and wants to provide a digital model to the architect.  The GC wants to be confident that the model represents the structure accurately so he can tell the architect that the model is a reliable representation of reality. These are the folks that we are targeting for these two use cases.

Does SKUR provide data capture services ?

No, we are a software company.  If you are interested in this technology, you can create an account, it's up to you to capture the point cloud data however you want to do it, upload it onto our web-based system together with a digital model and then run our application.   You then download the variances and visualize them using an industry standard tool. We don't do any consulting.

I'm interested in how SKUR would be used during construction. What if the construction is only 10 % or 20 % complete, do I have to specify which part of the incomplete structure I have scanned ?

We have very smart algorithms that figure out which part of the structure to look at.  For example, if you bring us a point cloud of just the corner of a building and the model of the entire structure, we only focus on that corner of the building.  Another example is a building that is 12 stories high, but I only have a scan of the basement. Our software is smart enough to figure out the part of the model that relates to the scan.

What about the opposite case where during a scan-to-BIM process for an existing building, I have not finished the model so I only have a model for part of the building ?

It's basically the same algorithm. The software is smart enough to figure out which part of the scan relates to the partial model.  When you run our software, you simply upload the model and the point cloud and specify the tolerances.  The process is the same for construction QC or for model QC. From the perspective of the software, it's really the same thing.

Where does your software run, on a desktop or in the cloud ?

We are completely web-based, that is to say, cloud-based. Our software require heavy-duty processing that would not have been possible 5 years ago. It is all in the cloud and it's high performance multi-core computing.   We bring in the largest computers that are available on the Amazon cloud for these problems and they can generate the variances in a few hours.

What types of digital models do you support, 3D CAD, Revit, Bentley BIM, or IFC models ?

We are a young company, and we've just started.  Our first release supports 3D DWG models. But I think Revit models are the next logical step. And then we also plan to support the Bentley world.  The 3D DWG model really can be anything, with any level of complexity. I have had people asking me about things other than construction, like planes, such as an Airbus. I talked to a gentleman yesterday, who wants to use our software for a golf course.  As long as you have a DWG model and a point cloud that's registered to the DWG, we can find the variances between the two.  The registration between the two is critical. It has to be done accurately.

How difficult is it to choose the tolerances ?

It's very easy.  You choose three values corresponding to different degrees of variance which are colour coded, green, yellow, and red in the output.

How do you visualize the variances, I imagine for a large project there could be thousands of these ?

SKUR output is a visual representation and report of the detected locations where there are discrepancies showing the precise variance from design and, very importantly, the direction of the variance.  We colour code the discrepancies so you can spot the problem areas very rapidly. For example, in the graphic red and yellow coloring indicates degree of variance. Arrows show the direction of issue. In our experience most of the variances are green, so you can filter these out and just look at the ones colour-coded red, which are the big discrepancies.  This allows you to focus just on those important areas.  We're like an insurance company.  Most of the time it's fine, but when it's not you need to know.

Where did the algorithms come from  and where was the implementation done ?

We've invented everything from scratch in Oakland, California

I can imagine this technology being of tremendous value in many different sectors of the construction industry as well as in non-construction sectors.  Which sectors and geographies are you targeting as the market for your technology ?

The sector we have targeted so far are construction contractors (GCs). But going toward we will be increasingly focussing on owners.  We think that our software provides a view into the construction process that enables them to see during construction how well the designs by the architects and engineers compare to what is actually being built.

We are focussing on North America, but we also have clients in Europe. Europe is definitely high on our list, but as a young company we are focusing on close to home first.

If people want to try your software, how do they engage with you?

It's very simple. They sign up and create an account.  Then they can upload a model and a point clouds and hit the magic button. We do handholding in the beginning to help new customers.  But we don't do any data capture and or any data preparation. We simply provide a web-based software service.

SPAR2016: Greg Bentley on geo-coordination,mapping underground infrastructure, and information flow in construction

At the Spar 3D Expo and Conference in Houston Greg Bentley gave a keynote in the morning in which he especially focused on reality modeling and how this technology had been applied to model in 3D the route and venues during the Pope's visit to Philadelphia.  He emphasized how useful the model had been for the event organizers and the security people.

Greg, this morning in your talk you mentioned better project delivery and better asset performance and reliability in the same sentence. Is Bentley moving in the direction of enabling a full lifecycle construction process ?

While almost all significant construction projects are done from a 3D model, almost always it is a model created from scratch by the construction contractor.  We use the term construction modeling for this.  But by losing the design model at the beginning of construction, you have just neutered the design intent and optioneering that the design professionals did with their design model.  You could say it's great that construction enterprises are always using 3D, and it is. But starting a new construction model prevents the information reuse of the design model for better asset performance. To the extent that the asset was designed for better asset performance, that potential is lost.

During the operations and maintenance phase of a facility, the digital engineering models should serve as the digital DNA and be the frame of reference for the baseline operating decisions needed to enable the asset to perform as it was designed. But if there is a disconnect at the construction phase, then the work of the engineers, which should be paying off over the full lifecycle of the asset, is absolutely truncated in terms of benefits from the engineering design.

I believe we need to attract digital natives to the infrastructure professions.  We hope that they are joining an engineering culture that is not going to continue to be paid only by the hour. Because if its value extends over the entire lifecycle, the engineers' work is not limited to design.  They deserve greater credit for their intellectual property in their models, because they will be useful to make operations and maintenance decisions.

For us the construction phase is a challenge. We started on the design side and then expanded into asset performance. We are number one in some categories of asset reliability, but the potential to link those can be limited by the actual construction process.

How do you see technology helping to surmount this problem of information loss at the construction phase ?

As software developers we have to make sure that it is feasible for the the construction enterprise to start with the design model and reference to it. We have new construction modeling software that is incremental for and not a replacement for the design model. But you have to make sure that this approach pays off during construction.

We are the leader in the world in workface planning with our ConstructSim product. 4D is traditionally construction sequencing and staging. 5D goes beyond that in level of granularity by assigning the actual work for each craft in the field. The difficulty with 5D is that it's an order of magnitude more data and transactions, and it has to fit into the enterprise environment of construction because it needs to relate to materials, layout, and so on.  We have solved that problem with better software engineering, but given the nature of construction in different parts of the world, we can't assume that workface planning is equally feasible for every project.

For industrial construction involving steel, piping and concrete we have very mature workface planning software. In this case there is an important benefit for owners which is visibility into the path of construction. In previous times when construction was simpler and occurred at one place, visibility into the path of construction visibility was physical. Nowadays with economically competitive construction requiring off-site fabrication and global sourcing, there is no longer a physical path of construction.   Now you need visibility into the virtual path of construction.  Owners on the industrial side are finding it worthwhile to provide a cloud service for our workface planning ConstructSim product to make it available to the contractor supply chain. It provides benefits to the contractor in the form of workplace planning and the owner benefits from visibility into the path of construction.  So in the case of industrial construction both the owner and the contractor benefit from the software. However, we don't presume that this approach will always provide benefits to the contractor in other construction sectors.

In the UK the Crossrail project has targeted ensuring information flows from design through to operations and maintenance.  How has Bentley supported this ?

In the UK they have come quite a way with protocols for information sharing. There is a document Government Soft Landings, which has some good ideas that can work in an existing construction supply chain, but the upskilling needed to take full advantage is daunting.  At our offices in London we hosted an academy for Crossrail. They onboard all of their contractors through this academy. For the Crossrail project the goal of the academy is not to train contractors in the tools but to show them a path of information mobility all the way to the Crossrail asset register.  This helps them understand the importance of the continuity of information from design through construction to operations. When they know and understand that, since they are already using 4D tools, it is not any harder for them.  From the Crossrail project we have first hand experience with the benefits of having a digital design model during operations. We made sure that the model did not get dumbed down during construction. That's been our work over the past 4 or 5 years.

This morning you said geo-coordination plays a key role in reality modeling.  Could you elaborate on that ? 

Geo-coordination is so key to integrating data from many different sources for reality modeling. Digital engineering models exist for all infrastructure that has been engineered in our lifetime, but nobody ever looks for them  because they know they won't be able to find them.  Geocoordination can make it possible to find and put in use existing digital engineering models that can add value for asset performance.  We see many advantages in geocoordination. People shouldn't assume that geo-coordination will be found only in things created going forward.  In Bentley applications in the last 12 years, every time anyone has created an infrastructure asset, it has geocoordination in the form of real world x y z. From the geocoordination you can infer things that are tagged or related to it. Suddenly to be able to navigate with your Hololens the information environment, not only the real world environment, but also 3D design models.  And geocoordination is not limited to 3D models.  You can find lots of other useful information such as drawings, schematics and data sheets. There is a very high return on that investment.

For the Pope's visit to Philadelphia reality modeling involved capturing structures and real world location.  A highly detailed,  photo-textured 3D “reality mesh” model was constructed from 28,000 digital photographs, with high geometrical accuracy. Base imagery was provided by Pictometry, high-resolution aerial photography was taken by helicopter, and ground footage was captured by Bentley volunteers – including building facades, street views, and the inside of the Cathedral Basilica of Saints Peter and Paul.  All of this data was geo-coordinated, which means that real world coordinates were essential for integrating the data from Pictometry, aerial photography and ground footage.

One of the big opportunities and challenges is mapping underground utilities and other subsurface structures.  Is this something Bentley is interested in ?

Yes, subsurface infrastructure is potentially the biggest and most important reality capture opportunity. While I was in India I spoke to the assembled young engineers at IIT Delhi during the spring break. (In India during the spring break, top young engineers in the region go to the IIT for three days of courses to explore new technologies.) I said to these digital natives that we need you to remain civil engineers.  This will be more rewarding in the future because engineers will be paid for the full value of their work. But I added that when you leave civil engineering (like many of them do unfortunately), do it to become an entrepreneur and create the underground equivalent of digital photography. (By the way digital photography works well underwater and a lot of our infrastructure that is at risk is underwater.) But subterranean mapping is the last frontier of remote sensing. It is so important - and the payoff will be huge. 

Virtual reality will be important in visualizing underground infrastructure. Devices like the Hololens can bring together virtuality and reality, but it begins with mapping underground infrastructure.  It requires advancing sensing technologies for detecting underground infrastructure. In the US and the UK there are now standards for the quality of subsurface data (ASCE and PAS128).   Regrettably subsurface engineering hasn't got much attention or much investment.  I hope that when we are here again next year that there will have been breakthroughs in subsurface sensing.  I can assure you that when that occurs software won't be the holdup.

Spar 3D: 3D mapping the inside of multi-story buildings with a backpack

At the Spar 3D Expo and Conference in Houston The Woodlands, Texas, Avideh Zakhor from the University of California Berkeley was onsite with some remarkable products from her new startup Indoor Reality.  I blogged previously about the technology when it was still under development.  The "man portable system" she has developed is a 32 pound backback with a collection of sensors.  The most important feature of the technology are sensors and algorithms for accurately tracking the six degrees of spatial freedom which determine the location and orientation of the backpack in the building.  In addition there are sensors for recording optical and thermal point clouds. She has also developed a handheld device that supplements  the capabilities of the backpack.

The technology can track accurately the location and orientation of the backpack as the operator walks through the building going into all rooms, halls, up and down stairs and into other human accessible spaces.  Software tools then generate different 3D data products including optical and thermal 3D point clouds, a colourized 3D point cloud, surface texture map,  and triangulated mesh surfaces.  The package includes analytical software tools for automatically generating floor plans, distinguishing rooms, identifying staircases, identifying windows, finding heat sources such as computers and human occupants, and identifying lights. Surface textures can be added to create a 3D photo-realtistic rendering.  The sensors and software can capture and generate all required input for the Department of Energy's EnergyPlus energy performance analysis program.  It has GPS and can generate real world coordinates.  Indoor Reality is an ESRI Emerging Business Partner.

I had a chance to sit down with Avideh to talk about her new company and the remarkable products she was demonstrating at SPAR2016.

Avideh, what are the key features of the products you are demonstrating here that distinguish them from some of the other devices vendors are showing here at SPAR ?

We have developed a unique way to track where the operator is in a building and the orientation of the backpack.  We have a lot of sensors. That is the key to all of our intellectual property (IP). It's called sensor fusion and most of our IP is in that area. It does not suffer from drift. It tracks accurately where you are in the building even if you go upstairs. You can even go up in an elevator because it has a barometer.  In addition we detect wifi.  It can tell you what the wifi strength is in different parts of the building. We have an infrared sensor and we have developed an app that will give you real-time occupancy of the building, for example, we can tell you there are four people in this room, two people in that room. We have patents on that.

Based on the data collected by the operator's walk through the building, we allow users to virtually navigate through the space and spatially collaborate with each other through our web based visualization tool. During virtual navigation, you can look around in 3D as if you were there.  That's our second data product. We don't just record dumb point clouds.  We generate intelligent meshes.  Not only can I can pan around, look up and down at each location, but the shape of the cursor shows you the orientation of the surface - for a horizontal surface it's a circle, for vertical surfaces it's a square.  The cursor gets bigger as it gets nearer, smaller as it goes farther away.  We know the normals of all the surfaces because of the 3D mesh behind what you see.  We recognize doors and windows.  Of course, we can measure distances and we allow users to annotate objects and locations.   So this mesh is very intelligent, so intelligent that we can automatically generate a Revit model with a floor plan, rooms, ceilings, doors and windows. That is our third data product.

How do you foresee people will use the tool ?

We went to the University of Illinois just a month ago and we did an energy model for a 14 story bldg. We can do 3d geometry, we can detect lights automatically, we can detect windows automatically, and we can detect AC plugs automatically.   This gets fed into an energy simulation model which is a complete model for an Energy Plus performance audit.  Typically when people do an energy audit they are only interested in the lowest hanging fruit, for example, replacing incandescent light bulbs with LEDs.   This is because collecting the data for a full Energy Plus model has been time consuming.  With the Indoor Reality backpack you can collect whatever you need for a complete Energy Plus audit very simply. For example, we can superimpose the infrared record on a ceiling to show a hot area which is very likely a hot water pipe behind the dry wall.

We have just come out with a visual building documentation tool for stakeholders to collaborate  with each other throughout the lifecycle of a building - during design, construction and operations and maintenance.  If I want to remodel a building using an architect, designer and contractor they can communicate with each other by leaving notes for each other in the virtual building.  For example, I can put a tag on a door for John the contractor. "Please fix this door."  John can come along in the virtual building and edit that note "Avideh, I can do this Tuesday."  So this makes it possible to collaborate virtually on a building project in 3D,  in much the same way that we can collaborate on a Word document.

There is another dimension to this.  Let's say we want to remodel a space in a building. The first thing the architect will want to see is the existing floor plan and the other professionals will also want to see other aspects of the space.  Normally at this point the building owner will ask everyone involved in the planning to come over for a  walk-through the building.   That requires a lot of valuable time from very busy people.  With the virtual model that we can generate, most of this won't be necessary any more. The owner can ask a relatively junior person equipped with the backpack to walk the building to collect the data.  Then the resulting model can be put online and everyone can virtually walk through it.  For bidding on a big renovation project, a relatively junior operator does the walk through and puts it on the cloud and the construction companies who are interested in the project can virtually walk through it online for bidding purposes.

For a new building,  during the actual construction you can conduct a walk through every day and update visually what's going on.  Then if there are lawsuits in the future, you will have a permanent record of who did what and when.   You can show the legal folks where a component was supposed to go and where it was actually installed. It is also useful for monitoring progress, so you know who did what when to determine who gets paid and how much.   If there is a lawsuit and insurance companies get involved, you can step through the record to see who did what.  Then when you want to sell the finished building, the real estate folks can do a virtual walk through of the building with prospective customers.  Property managers and facility managers can use the virtual model for maintaining the building. For example, we talked about the energy performance audit.  Facility managers can use it to identify lights that need to be replaced with more efficient bulbs. Then they can put a virtual tag on the lights for the maintenance folks.  With a virtual model like this, owners, contractors, designers, and facility managers can all collaborate throughout the lifecycle of the building.

There seems to be a lot of areas where the technology you have developed could be used ?

There are quite a number of other application areas for the technology we have identified including historical building and monument recording, scan-to-BIM,  variance detection and virtual inspection, real estate valuations, building plans for first responders, change management and dispute resolution, and detecting thermal anomalies including water leaks and insulation deficiencies to name just a few.

SPAR 3D: Virtual reality, scan to BIM, infrastructure models, and mapping indoor spaces

I have just experienced two remarkable days at the Spar 3D Expo and Conference in The Woodlands, Texas, in itself a remarkable location near Houston.  I will blog in more detail in the near future about the things and people I had the opportunity to experience over the past two days.

First of all, virtual reality.  Everyone is aware of the Oculus Rift (see the new review at CNET if you aren't).  There are other technologies that have come or are coming from Samsung, Sony and Google.  But at SPAR 3D the most remarkable VR technology I personally experienced was what David Smith, CTO of Wearaility, talked about and allowed me and others to try - a set of incredibly light-weight glasses with acrylic lenses designed to work with a smartphone that gives you a virtual experience comparable to IMAX without the IMAX theatre.  I tried it and found it an absolutely amazing surround experience.  And if you have wondered where Michael Jones, of Keyhole and Google Maps fame, has landed, he is now CEO of Wearality.

Secondly, my personal passion is the geolocation of underground utilities.  At SPAR 3D Mark Klusza, Founder & CTO, Real-time Metrology, Inc. described a major new advance in ground penetrating radar (GPR) technology he has developed that can detect a 3/4 inch rebar (among other things) four feet below the surface in clay.

For existing buildings creating a BIM model remains an art rather than science.  But at SPAR 3D a number of new technologies were presented that can help automate, improve the quality, and speed the process.  Foremost among them in my mind was presented by Nicolas Arnold, VP of Product Development, at SKUR.  With SKUR's technology, which runs in the cloud, you can compare the 3D model that you have developed with the point cloud you derived the model from and identify all the points of significant variance.  The same technology can also be used during a construction project to compare what has been built to what was designed - even when the building is only partially completed.

Ron Singh, Engineering Automation Manager/Chief of Surveys, Oregon Department of Transportation (DoT) described progress on his remarkable vision of how automation is upending how we design, build and monitor, operate and maintain U.S. highway systems.  Richard Arrowsmith, Asset Information Group Team Leader, Highways England described the digital model of the English highway system.  Stan Burns, President, Integrated Inventory, who has just retired from Utah DoT, described Utah DoT's comprehensive database of every piece of highway furniture and signage in the Utah highway system.  Together these represent the future of the digitalization of national highway systems.

A new company, Indoor Reality, is the third startup of Dr Avideh Zakhor, who is on leave from the University of California Berkeley.  Her new device is a backpack loaded with sensors including lasers and infrared that enables you to map the interior of buildings by simply walking through a multi-floor building, including up and down stairs.  The resulting data can be used to automatically generate floor plans and 3D models and captures everything you need for an Energy Plus analysis.

Mark Shell and Darrell Gadberry, both of the City of Fort Worth Water Department, described a very versatile device with laser and sonar and camera devices on board that can be used for sewer line inspections of small and large diameter sewer pipes.  Fundamentally this device enables condition-based maintenance of municipal sewer systems.  Mark and Darrell reported that it has saved the Fort Worth Water Department $42 million.

I was able to interview Dr Avideh Zakhor, Nicolas Arnold, Ron Singh, Larry Kleinkemper of Lanmar Services who specialize in creating BIM models for existing buildings, and Greg Bentley of Bentley who has a broad, forward-looking perspective on the "digitalization" of infrastructure.