Between the Poles

MEST 2009: Spatial Data, Convergence, Intelligent Models, Collaboration, Open Source Geospatial, GeoWeb, and Fusion

One of the distinguishing characteristics of the MEST conferences, is the daily wrap up where one of the organizers briefly reviews and summarizes the day's presentations.  I find the exercise very worthwhile because it gives you a chance to review and digest what you've heard during the day's proceedings.  There is also a session at the end of the conference, where the entire conference is reviewed by several of the speakers and organizers. 

By way of a little background, MEST is organized by the Bahrain Society of Engineers, so I find it more akin to the infrastructure conferences I attend than a traditional GIS conference.   Although it has commercial sponsors, it is non-denominational and this is reflected in most of the presentations.  It also tends to attract government people from many of the Gulf states, Bahrain, Yemen, Oman, Abu Dhabi, Dubai, and Saudi Arabia.

This year I was one of the panelists for the final wrap up, and I mentally did my own review which I presented as a personal precis of the conference.

Spatial Data

One of the themes that came up repeatedly was the importance of data, and especially of accurate, real-time data. Spatial data now includes not just traditional geospatial data sources but other sources such as architectural and engineering design, laser scanning, high resolution photogrammetry with multiple cameras, and ground penetrating radar.  Some of the newer technologies for acquiring spatial date were the subject of several talks including laser-scanning and LiDAR, high resolution photogrammetry, and earth observation satellites.  Dominic McPolin of the Central Planning Unit (CPU) in Bahrain and others emphasized the importance of reliable data in infrastructure planning.

Intelligent Models

Increasingly people want to do more than simply prepare maps.  They want to analyze and and simulate which requires more intelligent data.  For years utilities and telecommunications companies have maintained information about their outside facilities including location, properties such as maintenance records, and connectivity, because outage management, asset management and other utility applications require this information.  For example, outage management (OMS) is the primary user of connectivity because OMS typically needs to perform traces to identify controlling devices which may be the source of network traces and to identify customers affected by the failure of a device such as a transformer or pump. The increasing adoption of model-based design for structures such as buildings (building information modeling or BIM), roads and highways, electric power substations, nuclear power stations, hydroelectric plants, and renewable energy generation facilities, means that intelligent models are becoming available for many more types of infrastructure. Government permitting organizations are seeing the advantage of model-based design and are mandating (US GSA) or recommending (Singapore BCA) BIM to support building permit submissions.

Convergence

Another common theme is the convergence of architectural and engineering design, traditional GIS, and 3D technologies which enable what Dominic McPolin called a more holistic view of the built world.  World trends such as population growth, global climate change, and increasing complexity of our world are seen as major drivers motivating the broader interest in convergence.  Dominic McPolin emphasized that the complexity of modern economies requires a new fusion that will allow us to respond effectively to global climate change and other world challenges and predicted that because of its small size next year Bahrain will be a world leader in developing a new fusion of information and systems or conceptual business architecture that will provide a more holistic view of the national economy.

Collaboration

An important theme throughout the conference was the importance of collaboration between different disciplines such as engineering and business, different levels of government, different government ministries and agencies, and government and the private sector.  IT in many organizations is characterized by stovepipes, or islands of information, with limited communications between the different IT systems such as ERP, CRM, GIS, engineering design and others.  For example, a large utility or telecommunications firm may have thousands of applications, each running on one of ten or more operating systems, with its own proprietary database, which may be one of nine or more independent databases containing the same information.  This problem is exacerbated in government by multiple levels of government, national, state or provincial, and municipal.  In this respect small, centralized nations such as Singapore and Bahrain often have a significant advantage, because the problems associated with multiple levels of government are minimized.

Dr Al-Tamimi outlined a vendor-neutral architecture for the integration of different spatial systems within government including infrastructure, engineering, building permitting, mapping, and environmental, that is designed to be implemented in Bahrain and Saudi Arabia.  A key feature of the architecture Dr Al-Tamimi is developing is that it includes the full range of spatial information from the very detailed, engineering precision, large scale to very small scale.  He said that integrating model-based design or BIM is a priority and planned as the next step. 

Ron Lake, who has developed and has been proselytizing the concept of the GeoWeb or worldwide, web-based  integration of spatial information for several years, made what I think is a key point that the GeoWeb needs to be treated as a web of systems, rather than a web of documents.  Ron, who is the founder of the GeoWeb conferences, has been a proponent for the integration of model-based design and BIM, which was a central theme of the last GeoWeb in Vancouver, as a key component of the GeoWeb.

New Software Business Models

Several of the presentations at the conference either focussed on open source geospatial software or described projects that used open source software.  Daniel Ames, from Idaho State University, gave what I suspect was an eye-opening presentation for many in the audience, outlining some of the advantages of open source software.  He referred several times to the Open Source Geospatial Foundation (OSGEO) and very generously mentioned Autodesk's initial and continued support for the OSGEO.  Dan is the founder of  the MapWindow open source project, which has about 250 participants worldwide including 20 committers (developers with permission to contribute and modify code.) 

Innovation

Bahrain has announced that it intends to reduce CO2 emissions per capita by 60% by 2014. This will require the biggest infrastructure program (road, water, electricity) program the country has ever seen.  Dominic McPolin was unequivocal in identifying the key to achieving this aggressive goal as innovation and he made an impassioned plea to the private sector to provide the technology to enable this goal to be realized.

GITA Ontario Fall Forum 2009: Smart Grid, Asset Data Quality, Data Volume, and Mobile

At this year's GITA Ontario Chapter Fall Forum there were a number of fascinating presentations on topics such as smart grid, mobile work force, managing electric distribution systems with enterprise GIS, and asset management in utilities and local government. 

The day started off with a Industry Trends Analysis Forum, similar in intent to the ITAG session at the annual GITA Conference in the US.  I found this one run by the Ontario GITA Chapter to be particularly effective.  People were grouped by tables, each table with 3-6 people, and 30 minutes was allotted for round table discussions with the goal of each table coming up with a consensus on paper of they saw as the most important 2 or 3 applications and technologies.  A few facilitators circulated to help if tables needed help or suggestions.  After the 30 minutes was up, a few tables were selected to report orally and then the paper notes were collected from each table to be used to prepare a summary report.  I found the exercise very effective, because there was 100% active participation.

Some very interesting information that I collected at the talks I attended.

Smart grid

A smart meter read rate of 99.8% is the provincial goal in Ontario, which means that 99.8 % of meters need to be readable over the network at any given time.

Municipal utilities are not only deploying smart meters, but also demand response devices, which can shut down appliances to reduce load at peak.

A WiMAX communications network is being built by electric utilities across the province of Ontario to support deployment of the provincial smart grid.

Time of use billing for electric power is being implemented in Toronto, a city with a population of about 4 million, right now.

Reportedly smart device manufacturers cannot supply smart devices fast enough to meet demand.

Standards

Smart grid is going to make industry standards much more critical. In several talks the standards Multispeak and CIM were mentioned as critical for of sharing data among different applications.

Microsoft's vertical Smart Energy Reference Architecture (SERA) for smart grid devices with a common plug-in standard is starting to get some attention.

Mobile

Even small local governments are implementing mobile solutions.  Applications described include enabling editting in the field.

Data Quality

Improving the quality of asset databases is a high priority for most local governments in Ontario because of PSAB3150.  Unlike GASB 34 in the US, PSAB 3150 appears to have real teeth.  It not only requires local governments to dramatically improve the reliability and completeness of their asset databases by Jan 1, 2010, but also to implement systems for maintaining their asset data at a high level of data quality.

One utility has implemented a novel way to improve the quality of their network facilities database.  $10 will be paid for any verified correction submitted by a field worker who uses their geospatial field viewer.  They guarantee that 100% of these corrections will be on the monthly DVD within three months.  They are actually finding that 90% of corrections are being included on the DVD within a month.  They are also noticing that the frequency of "line hits", instances where mains are being hit during excavations is decreasing.

Data Volume

Typical smart meters are designed for 24 reads per day.

With smart grid the volume of data is expected to grow astronomically.  Chatham-Kent Energy, with tens of thousands of customers, is expecting data growth of 4 terabytes per year.

Reportedly PG&E with millions of customers is planning on 4 petabytes data growth per year.

Reportedly one local government in the US is dealing with a million data reads per second.

Google SOAP Search API to be Retired in Favour of REST and Javascript

Google has just announced that the Google's SOAP Search API , which has been deprecated since 2006, is being retired August 31st. It has been superseded by the Google AJAX Search API, which is a Javascript library but which exposes a raw RESTful interface for non-Javascript applications. The Javascript library allows you to embed Google Search in web pages and other web applications. The RESTful interface returns data using JSON encoding.

Impact of Open Standards and Open Source: Microsoft Promises "Greater Transparency"

Thursday Microsoft promised "greater transparency" in its development and business practices.  At least partially motivated by an antitrust case against Microsoft in the EU, it seems to suggest that the company is  finally recognizing the impact that open source and open standards have had on the IT industry. Some see this as possibly the end of Microsoft's patent threats against Linux.

Microsoft said it is implementing four new interoperability initiatives. These really seem to be huge moves for Microsoft.

• Microsoft will publish more than 30,000 pages of documentation for Windows protocols that used to be available only under a license.
• Microsoft is promising not to sue open-source developers for development and non-commercial distribution of applications using Microsoft protocols.   But if you intend to distribute commercial applications using these protocols, you need to get a license from Microsoft.
• Microsoft promised to create new APIs for Word, Excel and PowerPoint so developers can add other document formats and users will be able to set these formats as their default.
• Microsoft will use a new Open Source Interoperability Initiative to provide resources to the community for cooperative development. Microsoft says it is seeking a dialogue with customers, developers and open-source communities via an online Interoperability Forum.

OGC Cartographic Text

Two years ago several members of the Open Geospatial Consortium (OGC) including MapInfo, Autodesk, Intergraph, and Laser-scan began an interoperability initiative that was designed to make it possible for a standards-compliant application to display a map from any standards-compliant datastore.  The initial   objectives were standards for oriented points and cartographic text.  Cartographic text turned out to be a bit contentious because it conflicted with a traditional perspective that stylization information did not belong in the database with geospatial data. 

I am happy to report that the Open Geospatial Consortium OGC membership is revising the simple features (SF) specification and a key discussion item is how to best handle cartographic text.  This will have important implications for RDBMS vendors like Oracle that support spatial data as well as for geospatial vendors that consume spatial data.  I am optimistic that we will see the OGC discussion result in a modified SF that will include cartographic text and that we will see support for the revised SF specification in future product releases from Oracle, MapInfo, Autodesk, Intergraph, Laser-scan, Bentley, and others.

Interoperability using Tightly and Loosely Coupled (SOA) Architectures

One of the key trends of recent years is the integration of geospatial technology into mainstream IT. What this means practically is that standard IT architectures, tools, and standards are now being applied to geospatial solutions.

One of the major problems that IT in general faced since the beginning of IT is interoperability. Different applications, applications from different vendors, and applications running on different operating systems need to share data.  The most widely used approaches are paper and file exchange.  For example, many utilities and telecoms still rely on paper construction drawings as the solution for interoperability between engineering design (CAD) and records (GIS).

For data sharing and application integration two approaches have evolved in mainstream IT and are being applied to problems involving spatial data.  I think the simplest way to refer to them is as loosely and tightly coupled architectures.

Tightly coupled

A tightly coupled architecture integrates enterprise applications around a single point of truth, which is often a single spatially-enabled RDBMS. The types of applications that are linked include engineering design (CAD), facility records management (GIS), asset management, workflow, ERP, CRM, outage management, and other enterprise applications.  An EAI tool is sometimes used to link enterprise applications from specific vendors such as SAP.  A major advantage of a tightly coupled architecture is that it enables the rapid and efficient processing of large volumes of data, provides a single point of truth instead of several, often redundant, data sources, and enables open access to data throughout the organization.  This architecture has been widely adopted within large organizations.

Tightly coupled architectures rely on standards such as  SQL, ODBC, JDBC, and OLEDB, SQL/MM, and the Simple Feature Specification for SQL from the OGC, to provide open and secure access to data, including geospatial data, throughout the organization.  Most geospatial vendors provide the basic level of support for geospatially-enabled RDBMSs, which means practically that they support basic spatial data types, points, polylines, and closed polygons.  Many RDBMS vendors support the OGC's Sinple Feature Specfication for SQL.  The OGC with the support of geospatial and RDBMS vendors is working to extend this support to more complex data structures including oriented points and cartographic text in the short term and, I hope,  long-transactions, topology, linear referencing, and georeferenced rasters in the longer term.  The objective is to enable comprehensive geospatial interoperability, so that spatial data can be shared by different applications, applications from different vendors, and applications running on different operating systems.  At the recent Oracle Spatial User Conference, which took place right after the GITA 2006 Annual Conference, two muncipalities shared their experiences in implementing tighly coupled architectures centred around shared data in Oracle Locator/Spatial and involving applications from Autodesk, ESRI, Intergraph, GE Smallword, and MapInfo.

Loosely coupled

The loosely coupled architecture most people are familiar with currently is a service-oriented architecture (SOA) based on web services. Web services enables internet-based access to distributed data sources and applications using open standards from the W3C (http, XML, SOAP, WSDL, UDDI) and from the OGC (WMS, WFS, GML).  All geospatial vendors including support the basic geospatial web services (WMS, WFS, and GML). Autodesk and almost all of the major vendors demonstrated WMS/WFS/GML interoperability recently at GITA 2006 in Tampa.

Different Problems Means Different Solutions

I think it is fair to say that most IT professionals would agree that loosely and tightly coupled architectures address different problems and that both are necessary and they are not incompatible.  For example, a tightly coupled solution can expose web services.  I think that it is necessary for geospatial vendors to support both of these approaches. Specifically, this implies supporting web services available through mass market vendors such as Google and others, the OGC’s open web services initiatives, as well as working with the RDBMS vendors such as Oracle, the OGC, and with other geospatial vendors to move standards forward for tightly coupled architectures involving spatial data. 

Interoperability Using Open Standards

At GITA 2006 yesterday I witnessed a major milestone for the geospatial world thanks to the Open Geospatial Consortium (OGC).  Ten vendors in the geospatial arena viewed and updated shared maps.  In alphabetical order, ASA, Autodesk, Bentley, eSpatial, ESRI, Intergraph, Ionic, Oracle, Safe Software, and Telus participated in a demonstration of a disaster management scenario that starred the City of Tampa.   Replete with news casts and simulated oil spills and noxious gas clouds, the demonstration showed how geospatial data could be shared live among geospatial products from most of the major vendors using open web services standards WMS/WFS/GML defined by the OGC.  Not only was the demonstration extremely well presented, but the fact that OGC was able to achieve the cooperation of so many vendors is a remarkable accomplishment.  Congratulations are due to the OGC and its members in general, and to Greg Buehler in paricular, for enabiling this to happen.  It clearly showed the potential benefits of geospatial interoperability.