Between the Poles

Firm realizes ROI of 203% by integrating CAD and GIS for fiber network management

Integrating CAD and GIS into an efficient workflow for records management (aka network documentation) has been a challenge for utility and telecom operators for decades.  While utility and telecom operators have found ways to address this problem, it is rare to find the benefits for these systems quantified.  At Year in Infrastructure this year iForte Solusi Infotek presented concrete benefits of a records management system that has been in operation for several years for managing their fiber optic networks in Jakarta and on the rest of Java.

I began working in the utility and telecom sector in 1993, specifically in developing software for records management (called network documentation outside of North America).  At that time the magnitude of the AEC+geospatial cultural mismatch problem was just beginning to be appreciated by utilities and telecoms.  Planners tend to use GIS tools, engineers and designers CAD tools, construction folks in the field paper CAD drawings, and asset managers GIS integrated with their FM tools. Fashioning engineering and geospatial data into an efficient data flow from planning through design and construction to operations and maintenance represented a challenge that remains a problem for utilities and telecom companies. The Between The Poles blog is over ten years old and one of the persistent themes from the very beginning in 2006 was the challenge of integrating CAD and GIS data and applications in an efficient workflow (some examples; 2006, 2007, 2008 ).

The winner of this year's Year in Infrastructure Award for Communications Networks was a project of iForte Solusi Infotek.   iForte is a fast-growing fiber network operator with many large corporate customers all over Java island including Jakarta.  From this base it plans to expand into the rest of Indonesia.  iForte was facing many of the problems associated with paper-based records management in a fast-growing network operator. Prior to implementing the new system, their business processes were characterized by a plethora of paper-based manual processes. The data available for network planning was continually out of date making it difficult to compile an accurate situational assessment. Maintenance crews in the field had to rely on their own hand written notes on paper records. When sales staff met with a prospective customer it took an hour or more to collect the required information to determine the availability of fiber optic capacity at the customer's location requiring subsequent exchanges of emails to get the requisite information to the customer by which time the customer may have contacted a competitor.

The primary benefits of the new system were realized in planning, network maintenance and sales. 

• Maintenance crews could redline existing as-builts in the field so that engineering could see immediately where a problem had occurred and any temporary infrastructure the maintenance team had put in place to address the problem.
• Planning could see real-time cable utilization to identify congested networks. When there was an outage they could see the total capacity that was at risk.  It also enabled them to find alternatives for re-routing the network.  It was also found that network expansion planning could be completed four times faster. They were also able to use less fiber, saving on material costs and CAPEX. For example in one case, by looking at available capacity and utilization, it was possible to reduce new cable requirements from 1,400 meter to 100.
• For sales prior to implementing the records management system, determining fibre availability for a customer required on average of 60 minutes necessitating telephone calls to records. After implementing the new system, they were able to determine availability in six minutes and automatically generate a bill of materials and a job estimate.

For these activities iForte estimated that it has realized a return on investment of 203% over three years.

The primary benefits of the new system were realized in planning, network maintenance and sales, but there were intangible benefits as well.  Management was able to see the status of the network in real-time, review progress in building out the network on a map with demographics, assess the status of their network assets, and identify regions for network expansion.  The intangible value to the company achieved with the new system was operational excellence and improved strategic planning which provided for better performance and growth compared to competitors.

Geolocation plays a key role in integrating GIS and engineering data and applications for optimizing asset management workflows.  One of the reasons for the success of iForte's system is that Bentley decided many years ago to geospatially-enable all of their products including Microstation.  Bentley calls this capability geocoordination. It provides a location-aware foundation which facilitates workflows that require integrated engineering and geospatial data and applications.

Open source viewer sharing all city's geospatial data can run on Raspberry Pi

Locations of Cross Section surveyors' notebooks

At GITA 2018 in Phoenix, Bob Basques, GIS Systems Developer at the City of St Paul, described a system called COMPASS he and his team have developed that provides a shared, easy to use tool that allows city employees and the public access to all of the City's spatial and associated data including, for example, scans of surveyors' notebooks, 2.2 million street level photos, and permitting and licensing information from 200 different applications. Based completely on open source components the system is compact and efficient enough to run on a Raspberry Pi.

One of the problems that hampers efficient operations at municipalities is accessing geospatial data originating from multiple sources such as infrastructure maintenance, planning and zoning, property ownership, engineering, permitting, licensing and code enforcement. Spatial data is generated by CAD drafters, GIS users, surveyors and even users with smart phone apps. Imagery data can come from a variety of sources including earth observation satellites, aerial overflights, street photography, and drones. This data is constantly changing which means that any process that involves making copies such as converting it to a common format creates a bigger problem than it solves.

Location and maintenance data for each street light

The City of Saint Paul is responsible for infrastructure maintenance for all of the infrastructure within the public right-of-way. It is also responsible for maintaining parcel information linking and addressing assignment. St Paul is only 56 square miles, but it is densely developed and there is a lot of infrastructure and the associated regulations to manage. In addition St Paul is part of the metro area of the Twin Cities and users of St Paul data often need access to data outside of St Paul.

Organizations participating in GeoMoose project

The application is intended to improve the efficiency of common tasks including infrastructure maintenance management, planning and zoning, property ownership, engineering, permitting. licensing and code enforcement. To do this the system accesses multiple disparate data sources from different data custodians including Public Works, Safety and Inspections, Planning and Economic Development, Parks & Recreation, Office of Technology, Fire, Police, Real Estate, and District Councils. The public has access to 73 layers of data through OGC web standard interfaces. The tool, called GeoMoose, is an open source project (MIT license) on OSGeo and uses only open source components. There are quite a few governmental agencies around the U.S participating in the GeoMoose project.

Creating a project polygon and buffer to identify affected parcels and owners

The application is extremely compact and light weight - Bob demonstrated it running an a Raspberry Pi with a 64 GB drive (just about any size MicroSD card will work) - which means that users working in areas with restricted wireless can still access important data. GeoMoose runs on Linux and Windoes and can be accessed via Wifi from Windows, Linux, Android, iOS and Mac OS - actually any device with Wifi and a browser including a USB stick. The core of GeoMoose is comprised of HTML and Javascript code. It can access data stored in the file system as well as from OGC WMS/WFS sites over the internet. GeoMoose included an editor that allows users to create geometric objects such as polygons. A typical application would be to define a polygon representing a new project, put a buffer around it and find all the parcels that would be impacted by the project.

The system handles many different data types including 305 layers ( a new layer is added every week or two), 32 live database connections, 39 imagery layers, and 31 infrastructure layers. Besides Public Works users, users from more than 40 other organizations access the system. The system is an OSGeo project and is built on open source components including OpenLayers and MapServer, and can be integrated with PostgreSQL/PostGIS, Webdav, svn and Apache.

Locations of surveyors' notebooks

COMPASS exposes data to the world via OGC standard tools. Most of the data in the COMPASS system of tools is available as a WMS service. Selected datasets are also available as WFS layers. Some layers are protected for security or contractual reasons. GeoMoose 3.0, which was released in August of last year is mobile friendly and supports modern Javascript frameworks such as React.

Digital images of surveyors' notebooks can be accessed

The system has become widely used and trusted. It gets about 50,000 hits per week not only within Public Works but also by other users inside and outside of the city government. It has eased many previously labour intensive processes. Because more users are accessing their data, custodians of the data are taking greater responsibility for data which has led to an overall improvement in data quality and timeliness. Open source has allowed customization for specific users that would not have been possible without access to the underlying code. For example, it has made it possible to create specialized visualizations and reports, connecting to various proprietary databases, mixing data sources on output and building custom lookup processes.

Automating the process of comparing digital models to reality

At the Spar 3D Expo and Conference in Houston I had the opportunity to chat with Nicolas Arnold, VP of Product Development at SKUR.  SKUR is a young software company that has just released its first product, which automates comparing a 3D digital model to reality as represented by a point cloud from terrestrial scanning technologies (typically LiDAR).  The scan can be of an existing structure or a new structure in construction. The variances are prioritized and can be inspected in an industry standard 3D visualization application.

Nicolas, SKUR provides technology that finds the differences between digital models and point clouds from scans.  What are some of the applications of this technology ?

We are in the business of comparing 3D building models with point clouds that are registered to the building model. We do this for two very specific use cases.

The first use case is what I refer to as construction QC.  Let's say you have a design model and during construction you capture a point cloud by laser scanning or some other technology.  What you are interested in is comparing the design model to what was actually built out in the field. To do this you simply upload the design file and the point cloud, specify three tolerances that are used to prioritize the differences between the model and point cloud, and then run our cloud-based application.  The application outputs colour-coded deviations, which you can visualize in Navisworks.

The second use case is what I refer to as model QC where you have the reverse situation.  Let's say there is an existing structure, a building or almost anything.  But there is no digital model and you need a model for a renovation or retrofit.  You can use a scan-to-BIM process to scan the building and generate a point cloud from which a model can be created.  The model creation could be done by your own people, by a third party provider, or by an automated process such as Clearedge.  In this use case what you are interested in is how  well the model fits with reality. You follow the same process, uploading the model and the point cloud, to compare the model to the point cloud to see where there might be modeling errors and other types of deviations.

These are the two use cases that SKUR serves. We target the sophisticated construction contractor (GC).  One application of the technology for a GC is to perform construction QC on their projects to look for problems.  Another application is pre-construction, when the GC is considering bidding on a renovation or retrofit project and wants to provide a digital model to the architect.  The GC wants to be confident that the model represents the structure accurately so he can tell the architect that the model is a reliable representation of reality. These are the folks that we are targeting for these two use cases.

Does SKUR provide data capture services ?

No, we are a software company.  If you are interested in this technology, you can create an account, it's up to you to capture the point cloud data however you want to do it, upload it onto our web-based system together with a digital model and then run our application.   You then download the variances and visualize them using an industry standard tool. We don't do any consulting.

I'm interested in how SKUR would be used during construction. What if the construction is only 10 % or 20 % complete, do I have to specify which part of the incomplete structure I have scanned ?

We have very smart algorithms that figure out which part of the structure to look at.  For example, if you bring us a point cloud of just the corner of a building and the model of the entire structure, we only focus on that corner of the building.  Another example is a building that is 12 stories high, but I only have a scan of the basement. Our software is smart enough to figure out the part of the model that relates to the scan.

What about the opposite case where during a scan-to-BIM process for an existing building, I have not finished the model so I only have a model for part of the building ?

It's basically the same algorithm. The software is smart enough to figure out which part of the scan relates to the partial model.  When you run our software, you simply upload the model and the point cloud and specify the tolerances.  The process is the same for construction QC or for model QC. From the perspective of the software, it's really the same thing.

Where does your software run, on a desktop or in the cloud ?

We are completely web-based, that is to say, cloud-based. Our software require heavy-duty processing that would not have been possible 5 years ago. It is all in the cloud and it's high performance multi-core computing.   We bring in the largest computers that are available on the Amazon cloud for these problems and they can generate the variances in a few hours.

What types of digital models do you support, 3D CAD, Revit, Bentley BIM, or IFC models ?

We are a young company, and we've just started.  Our first release supports 3D DWG models. But I think Revit models are the next logical step. And then we also plan to support the Bentley world.  The 3D DWG model really can be anything, with any level of complexity. I have had people asking me about things other than construction, like planes, such as an Airbus. I talked to a gentleman yesterday, who wants to use our software for a golf course.  As long as you have a DWG model and a point cloud that's registered to the DWG, we can find the variances between the two.  The registration between the two is critical. It has to be done accurately.

How difficult is it to choose the tolerances ?

It's very easy.  You choose three values corresponding to different degrees of variance which are colour coded, green, yellow, and red in the output.

How do you visualize the variances, I imagine for a large project there could be thousands of these ?

SKUR output is a visual representation and report of the detected locations where there are discrepancies showing the precise variance from design and, very importantly, the direction of the variance.  We colour code the discrepancies so you can spot the problem areas very rapidly. For example, in the graphic red and yellow coloring indicates degree of variance. Arrows show the direction of issue. In our experience most of the variances are green, so you can filter these out and just look at the ones colour-coded red, which are the big discrepancies.  This allows you to focus just on those important areas.  We're like an insurance company.  Most of the time it's fine, but when it's not you need to know.

Where did the algorithms come from  and where was the implementation done ?

We've invented everything from scratch in Oakland, California

I can imagine this technology being of tremendous value in many different sectors of the construction industry as well as in non-construction sectors.  Which sectors and geographies are you targeting as the market for your technology ?

The sector we have targeted so far are construction contractors (GCs). But going toward we will be increasingly focussing on owners.  We think that our software provides a view into the construction process that enables them to see during construction how well the designs by the architects and engineers compare to what is actually being built.

We are focussing on North America, but we also have clients in Europe. Europe is definitely high on our list, but as a young company we are focusing on close to home first.

If people want to try your software, how do they engage with you?

It's very simple. They sign up and create an account.  Then they can upload a model and a point clouds and hit the magic button. We do handholding in the beginning to help new customers.  But we don't do any data capture and or any data preparation. We simply provide a web-based software service.

Oracle Database Cloud Service supports Spatial

At Oracle Open World in San Francisco 2014, September 28–October 2 the Oracle Spatial and Graph SIG held a Meetup.  By way of background Oracle Spatial and Graph support geospatial data and analytics for land management and GIS, mobile location services, sales territory management, transportation, LiDAR analysis and location-enabled Business Intelligence. The graph features include RDF graphs for applications ranging from semantic data integration to social network analysis to linked open data and network graphs used in transportation, utilities, energy and telcos and drive-time analysis for sales and marketing applications.  

There were several very interesting discussions at the Meetup that revealed some interesting aspects of Oracle Spatial itselef and its interoperability with other products such as ESRI ArcGIS, Autodesk Map3D, and Bentley.  These are taken from a summary recorded by Nick Salem moderator of the Meetup.

The Meetup was attended by about 20 users came from industries such as rail, local city government, national land management agencies, and defense. The SIG Board and members of the Oracle Spatial product development team joined up with them in the Oracle Technology Network Lounge.

Some of the most interesting discussion topics include new features in Oracle Spatial 12c and real world experience interoperating Oracle Spatial with CAD and GIS products from other vendors.

• Oracle’s new Database Cloud Service offering now includes Spatial. 
• A new Oracle Spatial 12c SDO_POINTINPOLYGON function can be used to ingest large point datasets (point clouds) without the overhead of creating a spatial index.
• Bechtel has developed a way to use ESRI ArcGIS versioning with Oracle Workspace Manager (OWM).  They have developed in-house custom tools to usefeature-level metadata to be able to retrieve spatial content from their project and public spatial databases, as well as to compile the revision history of layers in the database to retrieve maps impacted by changes to individual features.
• Dan Geringer, Oracle Spatial Solutions Architect, advised that other large enterprises have also successfully implemented OWM with ESRI ArcGIS.
• Bulk processing and interoperability are two of the important advantages of Oracle Spatial for enterprises. Bechtel has large project-based deployments of Autodesk and Bentley CAD systems, and has worked with Autodesk in recent years to successfully write spatial data to their Enterprise GIS Oracle Spatial database from Autodesk Map 3D. They also have developed a GIS Desktop Procedure that details the required Autodesk client software configuration for Oracle Spatial and ESRI ArcSDE database connections. They will be updating it for use with Autodesk 2014 and 2015 clients.
• Latest version of MapViewer (11.1.1.7) supports HTML5 and JSon and the ability to work in disconnected mode. Standalone, web-based vector editing is also available. Wish Li and Jayant Sharma demonstrated  MapViewer on mobile devices, an Android phone and an iPad.

 

ESRI UC: 3D and other themes of this year's conference

According to Jack Dangermond who led the plenary sessions on the first day,  this year in San Diego 12000 people from 130 countries were in San Diego for the 33rd Esri International User Conference.

I gravitated primarily to the utility sessions.  According to Bill Meehan, who is responsible for ESRI's utility business, there were 750 utility folks at this year's UC.

GIS is everywhere

GIS is being used in diveerse areas including environmental monitoring, climate change, permafrost melting, sea level rise, agriculture, water resources, aquaculture, pollution remediation, energy resources management, land information systems, urban design, utility network management, wasteshed modeling, utility pole inspection, sewer rehabilitation, port management, indoor GIS, iInsurance risk management, health-related mapping, law enforcement, mapping where space debris might fall, flood risk mapping, storm tracking and damage assessment,  geodesign, crowdsourcing for collecting a variety of data, interoperabilty, citizen engagement, government transparency, cartography, geology and mining, automatic generalization of different scale maps, story maps, portals for infrastructure, and open data, to mention just a few of the things that were singled out in the plenary session. 

Some of the areas that were singled out  for special recognition were

• Special Achievements in GIS award went to the Dartmouth Atlas for showing geographically how outcomes and the cost of medical procedures varies depending on location across the U.S. 
• The Hong Kong Lands Department received the Enterprise GIS award for their land management system which they started in the 1990's and is now in its second generation. 
• Tbe President's Award went to Direct Relief which distributes different types of equipment to people around the globe in need with 99% efficiency according to Forbes.

Themes

The theme of this UC was GIS transforming our world, not only changing the physical world as we know it, but also changing our perception of the world and how GIS has and will fundamentally help us collectively build a better future.  A classic example that has impacted everyone on the planet is GPS or more generally GNSS.   As a number of speakers including Dangermond said we are never lost any more and that is fundamental change to how we perceive the world.

Some of the major themes that emerged at this UC are common to many software and service providers in the IT sector, but some of these are new or unique to ESRI. 

Climate change

We are facing serious global challenges, in particular climate change, which both at the Geodesign Summit and at this UC conference appears to be a personal challenge for Dangermond who sees GIS as contributing in major way in creating a more sustainable future.  He sees GIS helping to change how we think and act and how we do basic things such as design. 

GIS across the organization

A theme that was pervasive throughout the conference including in the utility sessions was the need to make GIS pervasive across the organization because 90% of the data that organizations are collecting and managing has location.  This was also true in utility sessions I attended, where the thrust is to expand the use of GIS outside of the traditional area of records management into other departments.

Vertical industry "templates"

In the utility and telecommunications sectors there is a major effort underway to create preconfigured vertical "templates" for specific vertical industries.  In the utility sector one was announced yesterday for the electric power Industry, and others are underway for gas and telecommunications.  These templates are a collection of data and tools for vertical industries and are available as open source on Github.

Web

The web was pervasive, It was hard to find a session where the theme that desktop GIS is being transformed into web GIS did not come up.  It doesn't mean that the desktop is going to go away in the near future, but sharing data, applications, and apps is being recognized as a common goal and web GIS makes this very easy to do.  This includes web portals for accessing enterprise data over the web.

Cloud

A lot of the traditonal desktop GIS capabilities are now being made available in the cloud so that all you need is an iOS or Android device or a web browser.  This is also available to large enterprises who need to keep their data on their own servers for security reasons.  Cindy Salas from Centerpoint Energy demonstrated ESRI's cloud capability but running "on premise" completely wthin Centerpoint's fire wall. 

Integration

Being able to integrate location and the spatial dimension into enterprise applications such as ERP (SAP), business intelliegence (IBM Cognos), Salesforce, MS Dynamics, and Excel spreadsheets is a new focus for ESRI.   They have already created plugins for several well-known applications.

Mobile devices

In the past when you talked about mobile in the context of utilities, you were normally dealing with Panasonic Toughbooks which could cost up to $6 000 each.  At this conference just about every application was able to run on a variety of low cost mobile devices including iOS and Android mobile devices.  One of the sessions I attended was about the National Broadband rollout in Australia where the 500 field folks who are doing 300 000 pit inspections were all equipped with iPads, which you might have thought were too fragile for pretty rough field work..  This project has been underway for something like 18 months and todate only two of the iPads have been damaged.

Big data

Spatial data volumes whether from sensors at utilities, from LIDAR scanning of transmission lines, or from earth obsevration satellited are now reckoned in terabytes.  There were sessions on spatially enabling Hadoop, SAP HANA and other big data management systems for managing these data volumes.

Real-time

In the utility track managing real-tme data from sensors on transformers, smart meters, phasors, and other devices and enabling near real-time decision making was a common theme.  A general event processing capability that can be triggered by rules has been added to the quiver to make this possible for any data stream. 

In the area of imagery, Digital Globe imagery can be accessed with 4-5 hours of acquisition.

Analytics

Big data and real-time means that we need to be able to understand what these huge volumes of data are telling us and then enable decision-making, automated or human mediated, as rapidly as possible.  If a transformer is running hot, we can't wait a week until the next service check to reconfigure the network because by then we may have reduced it projected lifetime by 10%.  There were "location analytics" sessions in most of the vertical industry sessions including utilities.

Content

Software companies like ESRI and Hexagon are now in the content busines.  And with the data beng made available in the cloud, you can access it through web GIS or desktop GIS. The data includes traditional topographic maps, digital terrain models, imagery from earth observation satellites from Digital Globe and others some of it near real-time (fr example, from Digital Globe within 4-5 hours of acqusition), demographics, and others.

Dangermond's term for this is the "living atlas" because this body of content is intended to be dynamic, real-rime and comprehensive.  The goal seems to be all the data you would need to manage the planet.

Some of the content explicitly mentioned includes

• basemaps
• community. content
• imagery
• 30 cm coverage of the US
• 60 cm coverage of Western Europe
• imagery for the middle east and asia in total covering 2/3 of the world
• health
• lifestyle
• economics
• demographics, at the zip code, county, state, and national levels
• scientific maps including environmental, energy, infrastructure, and terrain models
• landscape layers

This data is accessible to web and desktop applications and makes it very easy to do standard spatial analytical things like multi-criteria suitability or site selection mapping.  Users can also publish their own data to the cloud.

Education

This has always been a strong focus area for ESRI and from Dangermond's perspective with the challenges the world is facing, GIS professionals will be even more critical in the future.  There is focus on advancing spatial literacy to a broader audience than in the past.

LiDAR

Laser scanning is a major source of important data for the construction industry (right of way determination and construction progress monitoring), utilities (transmission vegetation management), and other industries.  ESRI has made this a focus area and there were a number of sessions devoted to managing and analyzing point cloud data.

3D

Dangermond specifically singled out 3D as a major capability area for ESRI.  There are specialized 3D products, but 3D is also being built into ESRI main stream GIS and spatial analytical products.

3D city models

At the plenary there was a demonstration of how to quickly build a 3D city model from 2D data, including extruding buildings from a 3D building footprint and height, adding textures depending on building types, and then tools for analysis such as zoning and shadow analysis and visualization including flythroughs.  These models are sharable on the cloud as web scenes requiring only a web browser (with no plugins) or iOS mobile devices.

Geodesign

At the UC ESRI is demonstrating a prototype of a product designed specifically for geodesign.

Open data

Dangermond specifically mentioned the newest national open data initiative in Peru. 

Crowdsourcing/volunteered data

There are sessions specifically on editing and manipulating OpenStreetMap data.

Oracle Spatial and Graph to support NURBS

At the Oracle Spatial and Graph User Meeting, the folks from Oracle outlined some of the things to expect in the next version of Oracle 12c.  I was very impressed.  All of the below is my interpretation of what will be in Oracle 12c.

Parametric curve support

The really big news for the architecture, engineering and construction community (AEC) who use CAD and BIM applications is that Oracle 12c is expected to support NURBS (non-uniform rational basis spline) a type of parametric curve widely used in the design space.  According the Siva Ravada the support will be very general and will be able to handle different ways of representing NURBS. Up to now the only parametric curve supported by Oracle was circular arcs.

To me it looks like Oracle is taking a major step toward being able to store natively and manipulate (in some cases by stroking) the parametrized curves used by CAD and BIM applications.  There is a lot of CAD and BIM data out there and being able to store it and manipulate it in Oracle Spatial and Graph is going to be a game changer.

Orthorectification

Oracle is providing support for orthorectification embedded directly in the database.  This will have important implications for the industry.   Up to now, you had to pull the image out of where ever it was stored, transfer it over the network to someone's application often from an image processing company to orthorectify it, and then transfer it back to store the image or stream it to the end user.  With 12c you will be able to orthorectify images in the database.  This is in line with Oracle's objective of bringing processing to the data, rather than the data to processing.

Raster algebra

Oracle 12c will also have support for raster algebra, again in the database.  For example, you will be able to average in the database12 months of national temperature data stored in monthly raster files.

GDAL and PDAL support

Oracle supports the open source GDAL raster libraries, the industry standard developed by Frank Warmerdam (now with Google) and used by just about everyone for accessing raster images including ESRI.  PDAL is a similar type of open source library for point clouds developed by Howard Butler and Michael Gerlek.

3D

Oracle supports three types of 3D data.  3D vectors, point clouds and terrain (surfaces).  Oracle 12c will provide full support for 3D geodetic data, lon lat and elevation in meters or feet.

Performance

Oracle has developed a new engine for handling massive point clouds.  Dan Geringer showed impressive results for a 2.8 billion point point cloud (286 gigabyte file). 

Processing vector geospatial data is reported to be 50 to 100 times faster in 12c than in 11g. 

Raster operations have been parallelized which can dramatically reduce the time it takes for these operations.

Engineered Database

Since acquiring Sun, Oracle has become a hardware and software company and is offering integrated hardware and software solutions that offer extremely good performance characteristics.  For example, processing LiDAR data benefits from hardware acceleration on engineered machines such as Exadata boxes.

Bundled data

Oracle is planning to bundle vector data from Nokia (Navteq), TomTom and others.

Oracle Spatial and Graph

The name of the product is changing to Oracle Spatial and Graph partly because it has had graph capabilities (network and RDF semantic) already for several years.  But probably more importantly because the demand for graph capabilities is accelerating, especially RDF semantic graphs for linked data, text mining, and for social media analytics, according to Xavier Lopez.  Even Oracle's NOSQL database (aka BerkeleyDB) is apparently getting graph capabilities.

Integrating geospatial into construction: the challenge

I spent most of last week at the Geospatial World Forum 2013 (GWF 2013) in Rotterdam, which was an amazing event, because of its focus on monetizing geospatial value in vertical industries,  Industries for which there were symposia at GWF 2013 include construction and infrastructure, electic power and gas, mining and exploration, water as a resource, water distribution management, and agriculture in addition to more traditional sectors such as land management, photogrammetry and the environment.

I spent most of my time in Construction and Infrastructure sessions.  There were many absolutely fascinating pesentation on the applicaton of geospatial technology in this sector.  Among the most far-reaching in terms of potential impacts were three or four talks that explicitly addressed the challenge of finding a practical way to integrate geospatial into construction processes.

Value of geospatial information

To put this in context, there is a strong drive in the Netherlands to integrate geospatial information into govenrment organizations.  Geonovum is leading the charge in this area.   By way of motivation there is a very interesting cost-benefit analysis in the context of the European INSPIRE initiaive by Ecorys and Grontmij in November 2009.  The scope of study was the cost and benefits of the collection, maintenance and dissimilation of geographic information, but just within governmental organizations in the Netherlands.  The study did not attempt to estimate the value of geospatial information to society as a whole.  It concluded that over 15 years, the total net benefit to government organizations in the Netherlands would be € 34 million (Ralf Duinmeijer, Joulz ICT private communication).

Cost of poor interoperability

To put this in the context of interoperability and what poor interoperability costs the economy and individual firms, in 2004 in a remarkable study the U.S. National Institute of Standards and Technology (NIST) attempted to identify and estimate the efficiency losses in the U.S. capital facilities industry resulting from inadequate interoperability.  The NIST study focussed on interoperability problems attributed to the highly fragmented nature of the industry, the industry’s continued paperbased business practices, a lack of standardization, and inconsistent technology adoption among stakeholders.  It concluded that inadequate interoperability cost the U.S. capital facilities industry $15.8 billion in annually in 2002, but qualified the conclusion by saying that this is likely a conservative figure because there were additional significant inefficiency and lost opportunity costs associated with interoperability problems that were beyond the scope of NIST analysis.

At GWF 2013 it was reported that Rijkswaterstaat, responsible for public works and water management as part of the Dutch Ministry of Infrastructure, has estimated that poor interoperability costs it € 800 million per year.

Barriers to integrating BIM and geospatial

Bram Mommers, who is with ARCADIS Netherlands, set the stage by looking at what the barriers are that are hindering the integration of geospatial into the constrcution process.  (ARCADIS is a large international company that provides consultancy, design, engineering and management services in the fields of infrastructure, water, environment and buildings.)

Traditionally the challenge has been that civil engineering on one hand and geospatial on the other have been different cultures.  The way Carl Steinitz put is that they work at different scales.  Geospatial scientists deal with the universal, engineers with the very specific. 

Bram gave some examples of these parallel worlds. Geospatial folks make maps, engineers and architects make drawings.  Engineers and architects use CAD or BIM design tools.  Geospatial folks use GIS.  The geospatial standard for buildings and infrastructure is CityGML.  The AEC standard for BIM models is Industry Foundation Classes (IFC).

ARCADIS has been involved in projects that integrated geospatial into the desing process.  On the HOV Nijmegen project it was found that integrating geospatial and engineering design in a single database resulted in a single copy of each data element and multiple use.  It simplified communication and increased the quality of the final design.  It also enabled automated analysis of the consequences of design choices with the result that the planning cycle was shorter.   Bram mentioned several other projects that benefitted from the integration of geospatial with the construction process.

Based on their experience with these projects, Bram and his colleagues concluded that there are three main barriers to the integration of civll engineering and geospatial.

• Semantics - for example, different terms used for the me things by geospatial analysts and civil engineers and designers
• Different topology - examples,  (1) geospatial uses point, lines, and polygons; CAD/BIM uses splines, nurbs, and other parametric curves and treats polygons in a different way from geospatial topology; (2) features with location vs objects with location as an attribute
• Data formats - for example, geospatial data is stored in shape files, GML, and CityCML; CAD/BIM uses DWG, DGN, RVT files, and IFC

Mapping semantics

To address the first issue of semantics, a Dutch initiative called Concept Library (CB-NL) sponsored by the Dutch Council on Building Information (BIR), a joint government and industry initiative, has been created with the objective of developed an open, on-line system to map between the terminologies used in different domains.  The goal is a single integrated model for construction (buildings and infrastructure) that would allow design, construction, maintenance and operations to share the same data.  The concept library would be extended to include geospatial.  The overall goal as Bram stated it is

• Ontology for the build environment - multiple sets of domain terminologies mapped on to it
• One language
• Combines geospatial and construction

The impacts of this appproach that Bram sees are

• Store once, use multiple times - avoid redundant data
• Integrated information management - based on data custodians
• Geolocation as a property of an object - in additon to features

Bram listed a number of organizations in the private construction sector that are supporting this initiative.

SPAR 2013: Turning the construction process on its head

At SPAR 2013 I had the opportunity to hear a very forward looking presentation by Ron Singh, Chief of Surveys/Geometronics Manager at the Oregon Department of Transportation.  It was even more impressive because Ron's background is surveying.  His primary objective is speeding up the project development process.

Ron first looked at the project development process that has been used in the transportaion over the past 25 years.  It starts off with a complete survey of the project terrain to create a basemap - existing as-builts are rarely consulted.  Design of the new project is then conducted based on the basemap produced by the surveyors.  The result is paper drawings which are used by construction contractors to build the project.  After construction is complete construction as-builts are submitted, but they are almost an afterthought.  They are often as-designed rather than as-constructed, and most importantly they are not "sealed", meaning nobody signs off on them putting their professional reputation behind their accuracy and reliability.  Practically what this means is that in the transporation industry, as-builts are treated as unreliable information.

What Ron Singh is proposing is to stand this process on its head by making the post-construction survey the critical source of reliable asset information in the form of a 3D intelligent model. 

Then when a new project is initiated 80-90% of the necessary information is already available in the as-built database making a complete resurvey unnecessary.  All that is required before design can begin is minimal due diligence to validate the as-builts. 

It also implies that there is a reliable asset database that is maintained thoughout the operations and maintenance phase of the lifecycle.

Geospatial World Forum: Unlocking quality asset information at Heathrow

Airports are like cities.  They have all the infrastructure that cities have plus some.  Heathrow is the busiest airport in Europe with on average 181,000 passengers passing through it each day.  Heathrow has 13 different types of infrastructure, some of which are unique to the airport environment.  To provide some idea of the magnitude of the infrastructure at Heathrow, there are more than 45 000 man holes, 72 miles of high pressure fire water mains, 81 miles of aviation fuel pipelines, and power cables carry voltages ranging from 9V up to 400 kV.

That is a lot of infrastructure to manage.  The challenge is made more difficult because it is an environment where even a minor accident carries the risk of serious consequences.  (At last year's Autodesk University, Shashi Verma gave a talk about asset management at Heathrow called Flying High: Multi-Utility and Infrastructure Project Implementation at Heathrow Airport.)

At the Geospatial World Forum, Nigel Stroud, Geometry Information Manager with the British Airports Authority (BAA) at Heathrow gave a fascinating talk about the Heathrow Map Live system which provides web-based access to high quality information about Heathrow's infrastructure.  The primary reason that access to quality information is important is for safety reasons, but there are also legal and contractual requirements, such as CDM (Construction Design Management) regulations. And there is an important efficiency dimension to this as well.

Single point of truth

To enable interoperability and open access to various types of data, Heathow has defined a Common Data Environment (CDE), the objective of which is basically a single point of truth.  The objective of CDE is to cultivate a culture where data is created once only and used many times.  Another way to put this is that each data element has a single owner and is shared across the organization.  Implementing this is much more difficult than it sounds.  In many organizations, different groups independently capture and maintain  information about the same assets.  For example, in a study of a single utility Gartner found that nine different groups were independently capturing and maintaining similar data about power poles. At Heathrow CDE means that standards,  guidelines, and work processes are designed to support a single point of truth.  The important business benefit from a cost perspective is that having a common data environment minimizes rework, increases re-use of designs and provides for efficient handover from design and construction to operations.

Data quality

Because safety is Heathrow's first concern, data quality is a top priority. Reliable data about infrastructure is required to respond effectively to an emergency as well as to provide information about the location of existing infrastructure to the more than a thousand contractors working at Heathrow at any given time.  In 2002 40% of Heathrow's underground facilities were mapped to within half a meter. Now 72% of the underground facilities are mapped to half a meter. 

In my view, the most important thing that Heathrow has done is designing business processes to optimize data quality.  For example, contractors at Heathrow are contractually obligated to progressively deliver as-builts over the lifetime of construction projects.  In other words, as-builts are not only required at the end of the project, but during the progress of construction, so that they can reviewed by BAA as work proceeds.

Heathrow has had an effective asset management system for capturing and maintaining information about infrastructure including infrastructure and building models, drawings, operations and maintenance reference manuals, and asset maintenance reports for many years, but has found it a challenge to provide access to this information across the business.  This is what the Heathrow Map Live system accomplishes, a single, simple web-based tool based on an Oracle database that allows everyone within the business to query, retrieve and view information about Heathrow's infrastructure.

Avoiding underground infrastructure during excavation

One of the major hazards of all construction activity is inadvertently hitting underground infrastructure.  To give some idea of how frequent this occurs, in the US in 2010 there were 112,917 incidents where underground utility facilities were hit during excavation.

At an airport this is particularly hazardous because not only are there gas mains and electricity distribution cables, but also aviation fuel lines and very high voltage power lines.   This is one of the biggest benefits of the Heathrow Map LIve system.  At Heathrow statistics show that the proportion of events where underground facilities were hit during excavation as a result of inaccurate information decreased by a factor of 6x from 2002 to 2011, primarily as a result of access to high quality information including location about underground infrastructure provided by Heathrow Map Live.

Geospatial World Forum: How GDF Suez built a gas distribution asset management system in 10 months

At the Geospatial World Forum in Amsterdam, Radu Negoita, GIS Manager at GDF Suez Energy Romania, gave a fascinating overview of how GDF Suez has built a location aware asset management system in 10 months.  Natural gas is a rapidly growing source of energy that is cleaner from an emissions perspective than traditional energy sources such as coal.

GDF SUEZ Energy România has over 1.3 million customers and is a leader in the gas distribution and delivery market.  Its main buiness is the distribution of gas to consumers. The total gas distribution network managed by GDF Suez  is 16,000 km.  Natural gas distribution is a regulated activity to ensure the safe operation of the network which is comprised of  pipes, residential connections, regulator stations, sensors/measurement systems, and other types of assets.

Data acquisition

In 2007 GDF Suez initiated a data acquisition project to capture information including location about all their network assets.  It is an ongoing effort using using internal resources.  GDF Suez relies on GPS and total stations to capture accurate locations for their assets.

Asset management system

The objective of the asset management system is to improve the productivity of gas distribution operations by automating manual asset management processes.  The core is an asset management system that manages the assets in a centralized database (Oracle Spatial) and provides desktop (AutoCAD Map 3D) and web based tools (Autodesk Infrastructure Map Server) for optimizing data quality and providing secure access to the information across the organization. The core database relies on data versioning (long transactions) to manage asset updates safely in a production environment.  It relies on an open architecture to enable integration with SAP plant maintenance, SynerGEE network analysis, and other enterprise systems.  It supports design sketches and redlining of design drawings and provides multi-criteria analysis for investment planning.

Project development

The project lifetime was ten months which, based on my experience with similar asset management systems in a range of utilities, I find quite remarkable in terms of how fast the production system was delivered.  This involved a full project lifecycle including business requirements analysis, development of the hardware and software architecture, installation of Oracle Spatial database and configuring the gas network model, configuration and installation of the application, data migration and testing and training.  The solution was deployed and in production in 2010 and currently supports 600 active users.

The system provides automated data capture from field measurement systems, multi-user editing of asset location and properties, creation and editing of network topology, generation of job estimates, tabular and graphical reporting, and provides an operations dashboard for monitoring progress and the status of the network.

The desktop client is used for field data acquisition, creation of the new assets; geometry, attribute and topology editing and network state analysis.

The web client, which is used by the majority of the users, provides a viewer for attributes and geometry of the network; navigation and searching of maps, and report generation.

Plant maintenance

Plant maintenance is based on a maintenance plan for all assets of the network including pipes, residential connections, valves, regulator stations, and so on.  The master system which manages the assets is the geospatial asset management system. SAP- PM manages maintenance network assets in terms of preventive and corrective maintenance.  It also handles exceptions and other operations events on the network.

Future developoment

GDF Suez' future development plans include integration of the asset management system with the SCADA, SAP asset accounting, and workforce management enterprise systems.  They also intend to develop a crew routing and dispatch module and a mobile client for field staff.