Between the Poles

Gartner on Microsoft Promises Greater Transparency

According to InformationWeek , Gartner in a new research report warns open source software makers that Microsoft's pledge to open up its documentation library to third parties carries legal risks for developers.

FDO Support for Native Spatial Data Types in SQL Server 2008

Autodesk has developed and donated to the OSGEO an open source Feature Data Object (FDO) Provider for Microsoft SQL Server 2008.  The SQL Server 2008 Data Provider allows access to the native spatial data types and spatial indexing available in SQL Server 2008.  When SQL Server 2008 is released, all of the major RDBMSs including Oracle, MySQL, PostGIS/PostgreSQL, Informix, and DB2 will support native spatial data types and spatial indexing.  The new FDO data provider adds to the 17 existing FDO data providers, including the seven developed since FDO was donated to the open source community.

Web Processing Service (WPS) Approved by OGC

The Open Geospatial Consortium (OGC) have approved the first version of the OpenGIS® Web Processing Service (WPS) Interface Standard, which defines a standard for publishing, discovery and binding to geospatial processes.   This means that there is standards based way for defining a web services interface for geospatial processing, such as creating a buffer, conducting a trace, or finding a shortest path.

I believe there are several open source implementations of WPS already including a Python one PyWPS and a Java version Java WPS .

Booting up in 10 seconds

Having gotten a bit frustrated with waiting minutes or worst case hours for my Windows machine to complete security checks, for Outlook to finish having two or three coffees, and for an administrator scan, I decided to try something a little speedier.  A month ago we bought a Linux notebook, because my wife is taking a course at the University of Toronto and didn't want to lug an expensive and heavy Windows machine between Ottawa and the U of T. 

This machine comes with pretty much everything you need for everyday computing including Firefox, mail clients, Skype, Open Office 2 (which reads .doc, .ppt, and .xls files), WiFi and wired networking, and pretty much works right out of the box.  But best of all it boots up in 10 seconds, shuts down in 3 seconds, has no moving parts, fits in a purse, weighs less than kg, is lime green and cost $300. 

All so very cool, that I bought another one a week later and have been carrying it around and using it as a second computer (for example between flights when I don't have time to boot up XP) for about three weeks.  It runs GNU/Linux, which looks to be a Debian distro with some Xandros apps.  The OS source code is available for download OS Download.

Consortech Hosts Geospatial Conference: Standards, Single Point of Truth and 3D

Last Monday and Tuesday Consortech , one of the most creative providers of geospatial solutions I have been fortunate to know in North America, hosted conferences in Quebec City and Montreal.  Pierre Genest, President of Consortech, arranged for Sam Bacharach of the Open Geospatial Consortium (OGC), Don Murray of Safe Software,  Xavier Lopez of Oracle , Carl Pelletier of the Ministere des Ressource Naturelles et de la Faune de Quebec, and myself to speak at the conference.  Between speakers, Denis Piche of Consortech demonstrated the technology discusses in the presentations.

Sam Bacharach (OGC)

Sam is one of the few people who can make a presentation on standards interesting.   His main point was that the motivation for standards is economic, and that it makes financial sense for OGC members to support standards.  The OGC was founded in 1994, and Sam said that Oracle and Autodesk joined in October and November, respectively, of that year. 

There were two of Sam's slides in particular that I found interesting.  The first is the list of organizations with which the OGC has alliance partnerships, which includes well known organizations like the IEEE, W3C, ISO, and GITA.  But the list also includes organizations that geospatial folks may not have heard of, like the International Alliance for Interoperability (IAI).  The IAI is responsible for a standard called the Industry Foundation Classes (IFC). which defines standards for classes of objects used in buildings like heating and ventilation, utilities, and structural components.  One of the interesting outcomes of the alliance between the IAI and the OGC is the upcoming IFC2x3g standard which allows building coordinate systems to be georeferenced to a known geographical coordinate system. This is a major turning point because it implies that architects and engineers now need to be concerned about where on the globe the building they are designing is situated.  The UK's "right to light", Noise abatement programs in the EU , and the City of Vancouver's Vancouver View Cones are examples of how important it is to know where a building or infrastructure is going to be located geographically.  Sam said that Microsoft and Google probably spend around half a billion dollars annually on data including increasingly 3D, but it is clear that they cannot collect and maintain this data all by themselves.  Clearly we are going to need  new standards to democratize the creation and maintenance of geospatial data including buildings and utility and telecommunications infrastructure.

The other slide of Sam's that is worthy of wider dissemination is the list of currently supported OGC standards.  This is quite an impressive list because a high proportion of these standards including the Simple Feature Specification for SQL (SFS), the Web Mapping Services (WMS), Geographic Markup Language (GML), and the Web Feature Service (WFS), have become the standard for geospatial data exchange for governments worldwide.  I would also suggest that a recently adopted standard, Web Geoprocessing Service (WPS), is very worthwhile taking a look at.

An area that is notably not well represented in the OGC standards is 3D, probably for the reason that this is an area where the technology churn is still very high.  However, LandXML is an example of a widely adopted standard for roads and highways, the IAI's Industry Foundation Classes is an example of a well-defined Building Information Model  (BIM) standard and the latest IFC standard, IFC2x3g, includes location.  Another standard called CityGML is going through the review process at the OGC, and is already supported by some organizations such as Oracle.

Xavier Lopez (Oracle)

There are two topics that I felt were particularly important to note in Xavier's presentations.   The first was  that geospatially-enables relational database management systems  support the concept of a single point of truth, and allow the exchange of information between geospatial products  from different vendors.  Because of the wide adoption of the OGC's Simple Feature Specification for SQL,  ESRI, Intergraph, Autodesk, MapInfo (Pitney Bowes Software), Bentley, and others all support storing geospatial data in an RDBMS.  This means that you really can have a single copy of geospatial data accessed by multiple applications from different vendors.  The advantage of using an RDBMS as opposed to a simple file like Shape, for example, is that data access is standards-based, SQL and ODBC or JDBC.

But if you want to stay with files and have the flexibility to move to a spatial RDBMS in the future, the OSGEO FDO API is designed to support the concept of single point of truth, but supporting a wide range of file formats and well as spatial RDBMSs. FDO differs from the widely used OSGEO GDAL/OGR API in that it is designed as a read/write API for geospatial data stores, including files and spatially-enabled RDBMSs.

The other part of Xavier's talks that I found interesting were that while Oracle 11g supports 3D coordinate systems, 3D spatial data types, TINs, and point clouds, none of the spatially-enabled RDBMs support the data types that are necessary for architectural and engineering applications, whether 2D or 3D.  In my opinion this is a significantly larger market than traditional GIS, and because building and infrastructure models are becoming much more widely adopted and much larger, I would suggest that being able to store architectural and engineering information in a queryable database is becoming essential in the future, especially as we are now able to model entire cities, including the inside of buildings and underground infrastructure.  Xavier suggested that Oracle is moving in this direction.

Don Murray (Safe)

Don Murray from Safe Software, who develop and maintain FME, which is the de facto standard for geospatial data translation software, made several important points.  First of all Don showed the latest version of one of my favourite slides which I have plagiarized several times.  The number of file formats supported by FME has reached 220, and there are no signs of abatement in the world's creation of new file formats.  Don's important point about this is that the challenge is not format, but data models, and that (in my words) the single largest contribution to the progress of world civilization made by Safe is what I would call their metamodel, because the data contained in all of the formats supported by Safe flows through this model.

A very important point that Don made is that although FME users do use FME to translate between different file formats, therby creating multiple copies of the same data, Safe's preferred architecture is what Don calls on-the-fly data translation which avoids the creation and storage of redundant data in new files.  For example, FME supports FDO, which means that users have on-the-fly read/write access to all of the file types that FME supports.

I can't help adding that Don also said that FME now supports AutoCAD Map 3D Object Data, which is something that will make all Map3D users and other folks needing to access data in DWG files very happy.

Another very interesting point that Don made  and that is a harbinger of things to come is is that Safe is moving to support 3D data formats including BIM standards such as IFC.  This means that the line between architectural and engineering data, which is also spatial, typically includes the insides of structures, and where precision is essential and geospatial data, which is typically outside of structures, is blurring.

Carl Pelletier (MRNF)

A speaker coming from the trenches as it were was Carl Pelletier who works for MRNF.  One of his slides was so clear in presenting what the problems are with silos or islands of information that I'm including the original here with my own attempt at an English translation.  At MRNF Carl said that they currently support 120 different systems maintaining about a terabyte of data.

Impact of islands of information

The development of systems without an architectural vision results in

• Many different applications with many different data formats
• Data redundancy
• Problems finding data
• Information exchange is based on the import/export of paper or files

I think that Carl has captured the fundamental problems arising from silos or islands of information and the reasons that many folks are now looking at ways to implement a single point of truth within their organization.

Impact of Open Standards and Open Source: Microsoft Promises "Greater Transparency"

Thursday Microsoft promised "greater transparency" in its development and business practices.  At least partially motivated by an antitrust case against Microsoft in the EU, it seems to suggest that the company is  finally recognizing the impact that open source and open standards have had on the IT industry. Some see this as possibly the end of Microsoft's patent threats against Linux.

Microsoft said it is implementing four new interoperability initiatives. These really seem to be huge moves for Microsoft.

• Microsoft will publish more than 30,000 pages of documentation for Windows protocols that used to be available only under a license.
• Microsoft is promising not to sue open-source developers for development and non-commercial distribution of applications using Microsoft protocols.   But if you intend to distribute commercial applications using these protocols, you need to get a license from Microsoft.
• Microsoft promised to create new APIs for Word, Excel and PowerPoint so developers can add other document formats and users will be able to set these formats as their default.
• Microsoft will use a new Open Source Interoperability Initiative to provide resources to the community for cooperative development. Microsoft says it is seeking a dialogue with customers, developers and open-source communities via an online Interoperability Forum.

British Columbia Introduces North America's First Full-fledged Carbon Tax

Yesterday the Government of British Columbia passed a bill that will phase in a carbon tax on all fossil fuels including gasoline, diesel, natural gas, coal, propane, and home heating fuel.   Beginning  this summer the rate starts at $10/tonne of carbon-equivalent emissions and will rise by $5/year for the next four years.  By 2012 the rate for gasoline will be 7.34 cents/litre.  This is projected to raise $1.85B over three years, two thirds of which will come from business.  The money raised will not go be used for program spending, but will be returned in some form to individuals and businesses.  Very innovative attempt to get British Columbians to cut their emissions. 

Apparently all the remaining US presidential candidates support some form of cap and trade, which according to Wikipedia means that "a central authority sets a limit or cap on the amount of a pollutant that can be emitted. Companies or other groups are issued emission permits and are required to hold an equivalent number of allowances which represent the right to emit a specific amount. The total amount of allowances and credits cannot exceed the cap, limiting total emissions to that level. Companies that need to increase their emissions must buy credits from those who pollute less. The transfer of allowances is referred to as a trade. In effect, the buyer is paying a charge for polluting, while the seller is being rewarded for having reduced emissions by more than was needed. Thus, in theory, those that can easily reduce emissions most cheaply will do so, achieving the pollution reduction at the lowest possible cost to society." Apparently the major oil companies and other companies are preparing for this believing cap and trade to be inevitable.

Map India 2008

I just spent two days in Delhi at Map India 2008.   I always enjoy  attending and presenting at Map India because India is very advanced in geospatial technologies and I always come across different and interesting perspectives. 

Economic Expansion

The conference took place this year in Noida, which is not in Delhi proper,  but in Uttar Pradesh.  What I found remarkable was the amount of construction going on in Greater Noida.  I have been in Dubai a number of times in recent years, and the level of construction there is astounding.  Something like 15% of the world's construction cranes are in Dubai, which has a population of only several million.  My impression of the level of construction in Greater Noida is that while not as frenetic as Dubai, it is still quite impressive.  There was evidence of construction everywhere. Even in Delhi things are happening.  For example, work has already started on a world class airport. I have heard that the same thing is happening to airports across India including Mumbai.  THis is all being driven by an Indian economy that has been expanding at an annual rate of 9%.

Suresh Prabhu

One of the most interesting talks I heard was presented by Suresh Prabhu, currently an MP, but who has held several ministerial positions in the Union government in the past.  His key point was that India cannot sustain the current level of growth without investing in infrastructure, which means that land  currently devoted to agriculture will have to be repurposed.  Also climate change means that for sustainability, something like a third of the land in India will need be dedicated to forests, which leaves only 2/3 of Indian land resources available for agriculture and infrastructure.  Another sustainability challenge is that 2/3 of Indian agriculture relies on ground water because surface water resources are drying up.  Planning and managing natural resources in the context of these challenges is another example of the benefits of convergence and the ability to design and simulate built structures in their geospatial context.

Mobile

There were several talks on mobile which interestingly is more critical for the transportation sector in India than in other countries.  There are about 3.75 million trucks on the road in India with an additional 200 000 being added each year.  80% of these vehicles are owned by companies with less than 5 vehicles.  The challenge from a macroeconomic perspective is efficiency.  In India logistics represent about 15% of GDP.  For comparison in the US logistics is 8.7% of GDP.  According to a speaker It is estimated that in India annually a staggering Rs200 000 crores (US$ 40B)are wasted because of inefficiencies in fleet management and warehousing.

Currently penetration of automated vehicle tracking is less than 1%.  As a speaker explained the main business driver for automated vehicle tracking in India is that although most drivers have mobile phones the information obtained over a telephone from a  truck driver is "99.9% unreliable."  This is why automated vehicle tracking is perceived to be more critical in India than the US and why penetration is expected to rise rapidly once some fundamental hurdles are overcome.  One of the hurdles is the price of mobile solutions, but it appears that equally challenging is the mythology surrounding GPS technology among small business owners who represent the overwhelming majority of the transportation sector.  GPS is perceived as an unproven technology or as too complicated for small businesses.  I can't help but think that this  appears to be a classic technology adoption problem in the SMB sector, not that diffferent from what Autodesk experienced in its early years.  The following excerpt is from The Autodesk File by John  Walker .

The Autodesk Customer

He or she is probably not an employee of a large, manufacturing-oriented company.  Rather, they are more likely to be a member of a small shop, or the manager of a small independent business where the expenditure of even $3,000 is significant.

When you pay for a product yourself, and see the checkbook balance drop, you tend to have a different perspective than if someone bought the system for you.  It is more personal, and the commitment to that product, and the company that produced it, is greater.

FDO Open Geospatial Data API Graduates from OSGEO Incubation

Frank Warmerdam, president of the OSGEO, has announced that the FDO project has graduated from the OSGEO incubation process and is now a full fledged OSGEO project.  The FDO open geospatial data API provides an open read/write interface to geospatial data sources including Oracle Spatial, shape files, and others.  A number of FDO data providers for widely used geospatial data stores have been developed by third parties around the world and FDO is supported by Safe Software, 1Spatial, and other geospatial application developers worldwide.

Graduating OSGEO Incubation includes requirements for open community operation, a responsible project governance model, code provenance and license verification and general good project operation. Graduating incubation is the OSGeo seal of approval for a project and gives potential users of the project added confidence in the viability and safety of the project.

Streamlining Workflows in Utilities: Technology is No Longer the Excuse

Jonathan Pickus from Las Vegas Valley Water District (LVVWD) gave one of the most insightful presentations I have seen on the typical challenges facing utilities and what the LVVWD has achieved in streamlining the flow of engineering design data within their organization.

Las Vegas is a very unique place, not only because of the "strip", the casinos and other entertainment establishments that attract over 40 million visitors per year, but also because it has been one of the most rapidly growing residential areas in the US.  Between 2000 and 2006 there were on average 17 000 new water connections every year.  (The sub prime loan credit squeeze has slowed this dramatically. In 2007 there were only 6000 new water connections.) 

Ninety percent of Southern Nevada’s water comes from Lake Mead created by the Hoover Dam on the Colorado River, with releases regulated by the Southern Nevada Water Authority. Lake Mead storage is currently 50 percent of capacity with a water elevation of 1116.9 ft. To put this in context when the lake top drops below 1,125 feet, a Drought Emergency goes into effect, which is the situation at the present time.

When Jonathan joined the LVVWD in 2001, he found the classic problems that I have seen at utilities worldwide, poor data quality, as-built backlogs, redundant data, and unhappy customers and an unhappy regulator.  But he had the perception to recognize that technology is no longer the excuse, the biggest challenge was institutional.  He also realized that he had a major advantage in that engineering (CAD), records (GIS), and survey were all part of the same organization with a single head.  This is not typical of many utilities.  In many utilities and other organizations managing infrastructure, the organization reinforces the problem of islands of technology.  Engineering (CAD), records (GIS), assets (ERP), and operations (paper) are separate organizations each with its own head.  One of the rules of thumb of IT is that the IT architecture implemented by an enterprise reflects its organizational structure.  The result is the problems that Jonathan found at LVVWD.

What Jonathan achieved at LVVWD sounds extremely simple once you've recognized the problem;

1. integrate CAD and GIS to allow as-builts to flow freely
2. allow facilities data to flow freely between the field (operations) and records. 

Similarly to what Brad Lawrence implemented at ENMAX Power, he gave priority to facilities data coming back from the field.  The results were no data redundancy, improved data quality, no as-built backlog, and the as-built team was freed up to devote their efforts to more strategic activities, doing more with less, a really remarkable achievement.

LVVWD also implemented a geospatial portal that integrated many of their internal systems including records, operations, and ERP, and focussed on streamlining two key activities;  vehicle dispatch and tracking and call before you dig.  Another amazing result of what LVVWD has done is that leakage has been reduced to 3%, which is critical given the severe drought situation in Southern Nevada.