Between the Poles

An open source web-based platform for integrating BIM and 3D geospatial

BIM+geospatial integration is being driven by a trend toward infrastructure construction projects that require maintenance and/or operations during an extended period post-construction.  At the GEO|Design+BIM conference in Amsterdam, Sanghee Shin, CEO of Gaia3D Inc., gave a riveting presentation about a web-based geo-platform integrating massive, complex BIM models using open source geospatial libraries and tools including PostGIS, GeoServer, and Cesium and NASA WorldWind virtual globes.

Drivers of BIM+geospatial integration

Since the world became aware of the importance of human impact on the environment, it has become increasingly difficult to design buildings without reference to geographic context.  Many cities have environmental zoning and other types of by-laws that require designers to take into account neighboring buildings, infrastructure and vegetation.  States and national governments are developing building codes that require designers to assess the energy, water, emissions, and other environmental impacts of buildings.  This means that when designing and constructing a building or renovating existing buildings, the traditional approach ( often associated with a 0,0 coordinate system ) of design in isolation is no longer adequate for sustainable design.

Furthermore integrated geospatial+BIM is essential during the operations and maintenance phase of a building.  Utilities, transportation, emergency planning, first-responder access, energy performance, and evacuation management all require information about internal and external infrastructure.  A common geospatial coordinate system enables a comprehensive operational view of all infrastructure including internal and external structures. 

BIM+geospatial integration provide greater value to projects that involve not just design and construction but also operations and maintenance.  A leader in this space, Rijkswaterstaat, the Dutch transportation authority, began offering design-build-finance-maintain (DBFM) projects a number of years ago which has motivated private Dutch engineering and construction companies to adopt an integrated geospatial+BIM approach to construction.  For example, the firm Royal BAM Group nv/BAM Infraconsult adopted integrated BIM+geospatial because of market developments including more complex construction assignments and an increasing demand from customers for service provision throughout the entire life cycle of a project.

It is indicative of the importance of the integration of these technologies that Autodesk and ESRI, 800 lb gorillas in their respective areas of BIM and GIS, have announced an agreement to collaborate to help bridge the gap between BIM and 3D GIS.  This is their seond attempt.  The first attempt, an agreement between Jack Dangermond and Carol Bartz in the mid 90s, resulted in a product called ArcCAD, intended for civil engineers and planners.

An open source geo-platform for integrating BIM+geospatial

mago3D is a new 3D geo-platform developed by Gaia3D.  It is open source and integrates many well-known open source geospatial products and APis. It is web-based so it is not necessary to install anything to run it. It has an engine for handling massive, complex BIM objects based on an internal format (F4D) that was created to reduce network data traffic, enable LOD (level of detail) control, support octrees, a 3D spatial structure that enables efficient spatial queries and (frustrum and occultation) culling.  Culling refers to the removal of objects that are not visible to the observer and is key to efficient visualization or large, complex structures. It integrates with two of the best known open source 3D virtual globe tools, Cesium (supported by AGI and Bentley) and NASA WorldWind, for visualization in a geospatial context.  Viewing BIM structures in mago3D has some similarities with a Google Earth experience. The source code is available under Apache and AGPL (Affero GPL) open source licenses.  Cesium is an open source project supported by AGI and Bentley.

mago3D includes a F4D Converter that converts BIM models in IFC (buildingSmart Industry Foundation Classes) format as well as 3D objects in .3ds (Autodesk 3ds Max), .obj (Alias/Wavefront) and .dae (Collada) formats into its internal F4D format.  Industry Foundation Classes (IFC) is a vendor neutral and open model that most industry software BIM tools including Revit and Tekla are able to export, although there is typically some information loss. The bimserver.org open source project provides an open repository of IFC models and offers a Java API to extract partial building information models from a repository.

The main components of mago3D are mago3D.js, a javascript plug-in for WebGL virtual globes and open source web server (NGiNX), map server (GeoServer), geospatially-enabled database (PostgreSQL+PostGIS), 2D mapping (OpenLayers, Leaflet) and 3D visualization (Cesium or NASA WorldWind). The mago3d.js javascript plugin is designed to run in a WebGL virtual globe environment in a browser so that moving/rotating objects such as BIM models and heading adjustment happen in-browser and are very fast. 

I went to the mago3D site and viewed a BIM model of a chemical manufacturing facility in South Korea.  I was impressed with the ability to zoom ala Google Earth from the entire virtual globe down to viewing the chemical facility in its geospatial context.  I could continue down to a very detailed, almost nuts and bolts, level showing that the LoD control and the octree-based culling algorithms make visualization very efficient, even for large complex objects.

The importance of this technology is that it shows that it is possible using the available open source, geospatial APis and tools - together with some genuine innovative development - to create an open, non-proprietary 3D geospatial platform for integrating geospatial and BIM.  Given the critical importance of addressing the cultural and technical divide between the AEC and geospatial worlds, a viable open source alternative is essential for developing the innovative solutions to the challenge of interfacing the two worlds. 

The other part of the solution is common, open standards. There has been encouraging progress on that front.  The major breakthrough in bringing the architectural and geospatial views onto a common footing is the OGC LandInfra Conceptual Model developed by the OGC in cooperation with buildingSMART International and approved as an OGC standard in August, 2016.  LandInfra  was developed by Bentley Systems, Leica Geosystems, Trimble, Australian Government Department of Communications, Autodesk, Vianova Systems AS, and buildingSMART International and provides a unifying basis for land and civil engineering standards including the OGC's InfraGML and buildingSmart International's IFC for infrastructure standards.  

Open source geospatial application reduces copper cable theft by 70%

A highlight of the SA Geotech conference in Johannesburg was a presentation by Ryno Goosen of Suritec Geospatial.  Ryno, who was awarded the Best Paper prize at SA Geotec, gave a fascinating presentation about how he developed an open source GIS solution to fight copper cable theft.

In Canada copper cable theft costs the Canadian electricity sector an estimated $40 million annually.  In South Africa in 2014 there were 72,533 reported incidents of copper cable theft resulting in 10,736 arrests.  Copper theft incidents in the U.S. have tripled in the last 5 years.

Goosen Goosen's solution architecture was based on open source geospatial software.  He chose open source for several reasons.  First and foremost, it enabled massive scalability by providing an elastic IT infrastructure.  Secondly it provided open interoperability with other open source software as well as proprietary systems. He also emphasized the importance of using software that was "battle-tested" in both military and commercial environments. 

His solution architecture was built on Cesium, GeoServer, Boundless Desktop/QGIS, Protobuf, Cassandra, WildFly, Solr and GeoTrellis.  Using this solution he was able to combine data from different sources including ground patrol and remote sensors.  By analyzing this data geographically and temporally he was able to determine spatial and temporal patterns of copper cable theft.  This enabled optimum deployment of resources.  The results were impressive.  Over a 30-month period cable theft was reduced by 69%.

The success of Goosen's shows that open source GIS software has become a viable alternative for building complex and highly scalable solution-oriented architectures.  He also suggested that it levels the playing field for small to medium size enterprises by providing them with similar intelligence capabilities to their larger counterparts.

Guide to selecting Free and Open Source Software for Geospatial Applications

The development, integration, and distribution of the information and spatial data infrastructure necessary to support the vision and goals of a digital Earth will occur in a distributed fashion, in very diverse technological, institutional, socio-cultural, and economic contexts around the world. Free and Open Source Software for Geospatial Applications (FOSS4G) offers mature, capable and reliable software to contribute to the creation of this infrastructure. A recent paper presents a selected set of some of the most mature and reliable FOSS4G solutions that can be used to develop the functionality for a digital earth.  It includes examples of large-scale, sophisticated, mission-critical applications of each software to illustrate their potential in support of a digital Earth. Information and resources are offered to help users select the best FOSS4G solutions for their particular contexts and system development needs.

Red Hat joins LocationTech geospatial open source initiative

Red Hat has joined the LocationTech open source geospatial initiative.  Red Hat joins Google, IBM, Oracle, and Boundless as a strategic member.

Other relatively new members are SensorUp and Planet Labs, who join Azavea, MapGears, Open Geospatial Consortium (OGC), Vivid Solutions, Glob3 Mobile, Mousebird Mobile, CCRI, Ordnance Survey, Mapbox, CartoDB, and RadiantBlue.

LocationTech is the Eclipse Foundation's industry working group focusing on location aware technologies. LocationTech is hosted by the Eclipse Foundation. Eclipse is a vendor neutral community for individuals and organizations who wish to collaborate on commercially-friendly open source software. The Eclipse Foundation is a not-for-profit, member supported corporation that hosts technology projects and helps cultivate both an open source community and an ecosystem of complementary products and services.

Red Hat has joined the LocationTech open source geospatial initiative.  Red Hat joins Google, IBM, Oracle, and Boundless as a strategic member.

Other relatively new members are SensorUp and Planet Labs, who join Azavea, MapGears, Open Geospatial Consortium (OGC), Vivid Solutions, Glob3 Mobile, Mousebird Mobile, CCRI, Ordnance Survey, Mapbox, CartoDB, and RadiantBlue.

LocationTech is the Eclipse Foundation's industry working group focusing on location aware technologies. LocationTech is hosted by the Eclipse Foundation. Eclipse is a vendor neutral community for individuals and organizations who wish to collaborate on commercially-friendly open source software. The Eclipse Foundation is a not-for-profit, member supported corporation that hosts technology projects and helps cultivate both an open source community and an ecosystem of complementary products and services.

Java rises to the top of programming languages in the open source community

Earlier this week Github published a graph showing the relative popularity of programming languages among GitHub development project repositories, both private and public, for 2008 to 2015. 

Some things don't seem too surprising. Javascript is at the top. C++ has stayed in the top 10 since 2008. 

But a couple of things really surprised me.  Of the three P's, PHP and Python have remained in the top 5 for the entire period, but Perl has fallen off the chart.  CSS has risen dramatically.  But the most surprising thing for me was the incredible rise of Java from 7th to 2nd since 2008. Not long ago it seemed that only the big system houses like HP, IBM  and Oracle were using Java. One reason for its spectacular rise may be that it is suited to building big distributed internet systems (the Hadoop framework is implemented in Java).  It is also used for Android which runs more than 50% of the world's smart phones.  Something which may be related is the recent spurt in C# (Microsoft's version of Java).  In the geospatial community the rise in Java popularity has been paralleled by the rise in the popularity of GeoServer and GeoTools both of which were developed using Java.

Finally, Apple's Swift development language for iOS, which Apple says it will open source, is off this graph, but apparently has already risen to 18th.  It's rapid rise perhaps accounts partially for the drop in Objective-C, which has like C++ stayed in the top 10 - until very recently.

Powering up the smart grid with geospatial technology

The electricity industry is undergoing a transformation. With utilities embracing geospatial technology and turning into data driven enterprises in the Smart Grid scenario, the sector is staring at an innovative future.

The National Academy of Engineering identifies the electric power grid as the first of 20 major engineering achievements that has had the greatest impact on the quality of life in the 20th Century. Modern society has reached a point where virtually every crucial economic and social function depends on the secure, reliable operation of electric power infrastructure.  But because it has become so crucial for modern life, it faces major challenges.
   
The major drivers for the fundamental change underway in the electric power industry are increasing demand, universal access, decarbonizing electric power, reducing revenue losses, and grid reliability and resilience.  Some of the  technologies that are contributing to this transformation are intelligent devices integrated with a communications network, distributed renewable power generation especially wind and solar PV, net zero energy buildings, microgrids, and the new remote sensing technologies of subsurface utility engineering.  

For an industry not known historically for rapid change, many utilities are in the midst of transforming themselves into data driven enterprises. Recently IDC published its future predictions for the development of the electric power industry over the period 2015-2018.  For many in the industry these are quite startling and clearly reflect an industry that is rapidly evolving.

The technology roadmap for the smart grid involves the deployment of increasing numbers of intelligent electronic devices for sensing and for control.  The challenge is federating the data from all of these devices, extracting information from it, and dispatching the information to the right control devices. 
 
With the changes that the electric power industry is undergoing now, analysts see geospatial technology poised to become a foundation technology for the smart grid.  The role of utility GIS is expected to touch every aspect of a utilities business, affecting customers, operations and management because geospatial is the logical technology that can provide  the basis for integrating data from intelligent electronic devices such as smart meters and the  information silos associated with proprietary applications.

You can read more about smart grid and relevance of geospatial data and technology, real-time big spatial data, standards for interoperability, the importance of data quality, open source geospatial technology,  spatial analytics and other aspects of the role of geospatial technology in the smart grid in Geospatial World.

Register for the FOSS4G 2015 open source geospatial conference in Seoul

Registration for the FOSS4G 2015 Conference in Seoul has opened.

There is only one week left to submit your abstracts for presentations (closing April 24th).  Some ideas for topics include

• Development: new development in projects.
• Hacks and Mashes: novel solutions to all our problems.
• Big Data : geospatial big data.
• New Trends: IoT, Indoor GIS, etc.
• Case Studies: relate your experiences.
• Benchmarks: comparisons between packages.
• Business Cases: building the economic case.
• Policies: FOSS4G related policies around government and international organizations.
• Visualization: effective presentation of information.
• Disaster Response: software, case studies, outcomes.
• Collaboration: data collection, data sharing, and open standards.
• NSDI: FOSS4G based Spatial Data Infrastructure.
• Education: FOSS4G in education and trainings.

Over 900 open source developers and others were at the Free and Open Source Software for Geospatial (FOSS4G) conference in Portland last year.  The previous year's FOSS4G 2013 in Nottingham attracted 833 delegates.  Last year's  attendance matched Denver's FOSS4G 2011 record.

FOSS4G-Europe 2015 in Como, Italy July 15th to 17th open for abstract submissions

The second FOSS4G-Europe Conference will be held at Politecnico di Milano in Como, Italy, from July 15th to 17th, 2015. A day of workshops will precede the conference on 14th July, and a code sprint will close the event on 18th July.

Como is on Lake Como, one of the most beautiful locations in Europe. It is right on the border with Switzerland close to Lugano.

The Conference aims to bring together FOSS4G users and developers worldwide and foster closer interactions with and amongst European communities in order to share ideas for improving geodata, software and applications openess.

This year's conference has six tracks.

• The General Track is the main event line with presentations on FOSS4G technologies and communities.
• The Academic Track relates progress of scientific research and experiences on using and teaching FOSS4G.
• The Open Data Track welcomes projects and case studies about geospatial open data and FOSS4G tools.
• The Water Track welcomes case studies related to water managment and related environmental modeling with FOSS4G.
• The Positioning Track welcomes projects and case studies on indoor/outdoor positioning with FOSS4G.
• The Library Track welcomes case studies related to Library and archives managment with FOSS4G tools.

The site is open for abstract submissions for these tracks.

Smart power spatial power

For the first time in a hundred years, the electric power utility industry is undergoing a momentous change. Distributed renewable power generation, especially solar photovoltaics (PV), is introducing competition into an industry that has been managed as regulated monopolies. Consumers with solar PV panels on their roofs (and in not-too-distant future with Tesla batteries in the basement) and companies like Solar City (co-founded by Tesla co-founder Elon Musk) are fundamentally changing the traditional utility business model. A recent report from the Edison Electric Institute (EEI) report refers to disruptive challenges that threaten to force electric power utilities to change or adapt the business model that has been in place since the first half of the 20th century.

As a result, every aspect of the the electric power industry is changing. One of these changes involves the role that geospatial data and technology play in the electricity industry. In the past, geospatial has been a tactical tool — it was (and still is) used in a variety of applications — in outage management, asset management, mobile work- force management, energy density modelling, vegetation management, demand modelling, transmission line siting, substation siting and design, energy performance modelling of buildings, disaster management, and mapping renewable resources, to name just a few. However, with the changes that the industry is undergoing now, geospatial is poised to become a foundation technology for the smart grid.

The Energy Issue of Geospatial World Magazine explores the impact that this momentous change is having on the application of geospatial technology in the electric power utility sector.  Below I'm providing an overview of the material relating to electric power you'll find in this issue.

GIS has been widely used by utilities for years for automated mapping/facilities management, back office records management, asset management, transmission line siting, and more recently for design and construction, energy conservation, vegetation management, mobile workforce management (MWFM), and outage management (OMS). Now, utilities are integrating GIS with automated meter infrastructure (AMI) and supervisory control and data acquisition (SCADA) systems. Intelligent design has crossed over from the office to the field in utilities, also enabled by the capabilities of GIS, says Smith. Geospatial-related analytics (spatial analytics) is seen as one of the key aspects of success for electric utility operations in the smart grid era. Looking for patterns and correlations between different land, weather, terrain, assets, and other types of geodata will be increasingly important for utilities. Power-related analytics with geospatial components include network fault tracing, load flow analysis, Volt/VAR analysis, real-time disaster situational awareness, condition-based maintenance, and vegetation management. The smart grid is all about situation awareness and effective anticipation of and response to events that might disrupt the performance of the power grid. Since spatial data underlies everything an electric utility does, GIS is the only foundational view that can potentially link every operational activity of an electric utility, including design and construction, asset management, workforce management, and outage management as well as supervisory control and data acquisition (SCADA), distribution management systems (DMSs), renewables, and strategy planning.

Peter Batty reports on the major growth in geospatially-enabled Web and mobile applications with a special focus on the open source geospatial community and the significant impact of these technologies in the utility sector. "In general, there are a lot of geospatial open source software components available now that have the capabilities and robustness to be used in serious enterprise applications."  John McDonald, Chairman of the Smart Grid Interoperability Panel has been a firm believer for a long time that geospatial information is part of the foundational platform for smart grid.  SGIP has signed a memorandum of understanding with the Open Geospatial Consortium with the goal of incorporating more geospatial standards into SGIP standards.  Cindi Smith of Bentley goes even further and argues that  “geospatial technology is already a foundational component of electric power utilities’ IT/OT systems.  Smart grid simply brings more focus to the role it can play by virtue of the visibility of smart grid projects and processes in a utility and their need to exploit the vast amounts of data produced by the smart grid." Loek Bakker & Jan van GelderIt of Alliander, a Dutch utility company, describe how essential it has been for Alliander to integrate GIS, ERP and SCADA systems for a correct picture of its assets.  As electric utilities evolve into increasingly data-driven organisations, Jeffrey Pires and G. Ben Binger describe how GIS is fast emerging as the backbone for data management platforms. 

Cities are beginning to develop 3D models of underground infrastructure motivated by new underground remote-sensing technologies and by ROIs of up to of US$21.00 saved for every US$1.00 spent on improving the quality level of subsurface utility information.  Steve Dibenedetto, Senior Geoscientist and Technology Manager, Underground Imaging Technologies (UIT), part of Caterpillar describes new remote-sensing technology for detecting and geolocating in 3D underground utility infrastructure such as Ground Penetrating Radar (GPR) and Electromagnetic Induction (EMI). 

The Indian on-going Restructured – Accelerated Power Development and Reforms Program (R-APDRP) is one of the largest IT initiatives by electric utilities anywhere in the world — in one integrated project, all state-owned distribution utilities in India are building IT infrastructure, IT applications and automation systems.  The programme set out to create baseline data in the form of consumer indexing, GIS mapping and asset mapping. Reji Pillai & C. Amritha assess how GIS can be applied in this context.  

Integrating  geospatial and BIM is a key enabler for energy performance modeling which is a fundamental instrument for reducing the energy consumption and improving the energy performance of new and existing buildings. According to a report from Navigant Research, global zero energy buildings revenue is expected to grow from $629.3 million in 2014 to $1.4 trillion by 2035.

Wolfgang Eyrich of Entegra shares how Entegra’s primtech product, which is designed to help substation designers deliver designs based on integrated product modelling, provides a geographical context to substation designing. 

Matt Zimmerman of Schneider Electric highlights one of Schneider Electric's key techologies "graphic work design" which is integrated geospatial and engineering design (CAD or BIM).  Schneider Electric's geospatial division focuses on developing integrated, location-aware enterprise solutions such as integrated outage management (OMS), customer information system (CIS), GIS, and external weather reporting and forecasting service to help plan crew deployment during a storm.  Matt foresees that location-aware predictive analytics for electric networks is going to be one of the major development areas for utilities in the future.  Brad Williams of Oracle points out that spatial analytics is becoming a key technology for electric utilities because everything a utility does - customers, assets, and operations - involves location. 

One of the biggest challenges that utilities are experiencing is increasing volumes of structured and unstructured data (big data) that is overwhelming traditional enterprise systems.  The structured data comes from smart meters and intelligent electronic devices, and the unstructured data from social networks including Twitter, Google, Facebook and other social applications.  Consumerization of geospatial technology (we are all GPS-enabled sensors) will enable crowd-sourcing new sources of information about electric power networks most of which involves location (big spatial data).

Open source spatial data versioning coming for utilities

The Research and Markets report Global GIS Market in the Utility Industry 2014-2018 identifies an important new market trend in the increasing penetration of open-source geospatial software into the utility market. According to the report many small and medium-sized organizations that cannot afford to invest in expensive proprietary GIS solutions are now choosing open-source software. In the U.S. most of the 3000 electric power utilities are small to medium-sized organizations, primarily municipal and rural electric coops.

Spatial data versioning (long transactions) for utility records management

Two essential capabilities for utility and telecom records management are storing spatial data in an SQL database and long transactions, more commonly referred to as data versioning these days  For a utility data versioning makes it possible to store spatial design data in the same database as as-built data, enabling design engineers to work on new designs at the same time as operations staff are relying on as-built information to operate and maintain active utility networks.  Many of the largest telecoms and utilties have outside plant solutions based on these technologies, in some cases since the 1990s.

Open source spatial data versioning

In the most recent Eclipse LocationTech Newsletter, Paul Ramsey describes the latest development on a major open source initiative to support spatial data versioning.  Paul points out that software developers have long had extremely versatile tools, such as Git, for managing software versions which tracks the details of all changes made to source code including dependencies in a distributed environment.  He and others have asked why don’t we have this capability for spatial data management ? 

The answer is GeoGig which takes the concepts used in distributed version control such as Git and applies them to versioned spatial data.  GeoGig is written in Java and is available under an open source  BSD License.

At the Location Intelligence Summit earlier this year in Washington, Juan Marin, CTO at Boundless,  gave an overview and demonstration of what was then called GeoGit, a distributed system for geospatial data versioning using the same philosophy as Git.

To further leverage the abilities of GeoGig, Boundless has announced Versio for organizations to manage their spatial data.  According to Paul Versio is a "hosted service built on GeoGig that provides easier toolks for data authors to collaborate on spatial data while streamlining the creation, maintenance, and distribution of spatial data."

Git

By way of background Git is a distributed revision control and source code management (SCM) system.  Git was initially designed and developed by Linus Torvalds for Linux kernel development in 2005.  It is free open source software distributed under the GNU General Public License (GPL)  V 2.

Every Git working directory is a full-fledged repository with complete history and full version tracking capabilities. Each local repository is standalone and is not dependent on network access to a central server. Git is known for very fast performance and for scalability.

History of long transactions for telecom/utility records management

Two capabilities that are essential for utility and telecom records management, storing spatial data in an SQL database and long transactions, were introduced in the early days of Automated Mapping/Facilities Mapping/Geographic Information Systems (AM/FM/GIS).  The first commercial systems implementing this technology in the early 1990's were IBM's GFIS which ran on DB2 and VISION* which managed versioned spatial data in an Oracle RDBMS (this was long before Oracle Spatial and Workspace Manager).  VISION* systems are still running at First Energy, supporting hundreds of concurrent utility network designers, and at CenturyTel (at the time US West then Qwest) and Telefonica Sao Paulo (at the time Telebras where the far-sighted visionary was the late Geovane Magalhaes) where each VISION* system supports about a thousand concurrent telecom outside plant designers.  VISION* was acquired by Autodesk in 1999.