Between the Poles

GIAC Day on the Hill

Yesterday I and about 25 other people representing the membership of the Geomatics Industry Association of Canada (GIAC) spent the day talking to civil servants, Members of Parliament, and Deputy Ministers and Ministers about a budget submission that GIAC has prepared asking the Government of Canada to fund a multi-sector National Geomatics Strategy and a National Imagery Program.

We had several teams and the team I was on talked to policy advisers at the Ministry of Public Safely,  Department of National Defence, and five Members of Parliament representing most parties.  I was very impressed with everyone we met, they listened and took notes, and either offered suggestions about whom to talk to or offered their support.

I talked about geomatics and infrastructure, specifically the utilities and telecommunications industries, and the abysmal data that we have about underground infrastructure. One of our team talked about geomatics and health, another about imagery technology, and another about lidar and other geospatial data sources.

Geo-awareness in Government

Among the people we talked to the level of geo-awareness was interesting.  Everyone knew about GPS and applications and devices for getting directions.  All were aware of Google Map.  Several had a geology or surveying background and so had a pretty advanced understanding of geomatics.  Most were aware of the use of geospatial technology to support election campaigns.  We were very surprised by one MP who was entirely familiar, in detail, with what GIAC is proposing, because she was involved in a similar geomatics policy effort in one of the provinces.

Smart Grid: A View from the Inside "It's All about Data"

I’ve invited Kevin Miller, who has many years experience in the electric power utility industry and who now works for Autodesk advising utilities on design information management challenges and solutions, to give his perspective on what the smart grid will mean for electric power utilities.

I have worked in the utility industry for the past 24 years.  I worked at a major electric utility in several capacities in the Transmission & Distribution organization, but for the majority of the time, I was involved in the implementation and support of Distribution systems including GIS, Outage Management System (OMS), Work Management, and Design.  During this time I have seen the concept of smart grid develop and take shape.

How we design, operate and maintain today’s electrical grid  for the most part hasn’t changed in the past 75 years.  Over the years, devices and equipment have evolved slowly (the level of R&D investment in the electric power industry is one of the lowest of any major industry),  but there really haven’t been major changes in how the grid functions.  Of course there was a massive investment in building out the grid to provide universal electrical power under the impetus, for example, of the Rural Electrification Act, which provided federal funds for installation of electric power in rural areas of the United States. 

But, in the 90’s, there was a major change when deregulation and decoupling deflected capital investment from the grid.  The “gold-plate” that had been lavished on the grid in the proceeding decades now was used to finance preparations for freer markets with the result that the technical evolution of the grid stagnated.  In the last two years, stimulus monies, green initiatives, and energy conservation are combining to create significant pressures to quickly catch up, to make the grid smarter to address today’s pressing problems, including reliability, security, customer empowerment, and global climate change.

The systems used to design, capture, maintain, and analyze the grid in use today are partial automations of 75 year old procedures.  The business processes for the design, construction, operation, and maintenance of the grid are carried out by different teams within utilities, for example, engineering design, construction, records, outage management, operations, and billing. Over the years the different functional teams have evolved procedures that support their own specific missions and informational and operational needs.  What has been missing is a holistic view of the entire business process for managing the life cycle of network assets.  Individual teams optimize their own sub-processes, but the optimized sub-processes do not take into consideration the “production and capture” business process  to capture and manage the digital data required to operate and maintain the grid.  The focus on sub-processes and not looking at the bigger picture of the overall business processes and information flows has resulted in enterprise data bases with stale, incomplete, and error laden data and network models.

In the past with the existing grid, we have been able to scrape by with out-of-date and inaccurate data.  Smart grid changes all that.  From a data perspective, smart grid is ultimately just much, much more data, much of it real time.  Getting the value (operational efficiencies and improved operating metrics such as fewer and shorter outages) out of smart grid investment relies on being able to utilize the digital data collected from smart devices to monitor, analyze and simulate the electrical grid in real time.  If your current digital model of your company's electric power network is based on inaccurate, out-of-date, and incomplete data because your business processes for managing the information flow across the organization are archaic and inefficient, it is going to get much, much worse with the smart grid. 

Systems optimized over the years at the sub-process level may have appropriate technology to utilize and analyze network data, but I find that utilities fall down significantly in their ability to “produce and capture” accurate and timely information to feed these systems. I am continually surprised at how bad a job utilities are doing at maintaining their network facilities data.  When companies are inundated by a sea of unreliable information and experience the difficulty in making operational decisions based on this information, they will quickly realize that they have to fix their business processes’ “produce and capture” problems.  But I expect that when that realization happens it will be too late.

Utilities need to address these looming problems now.   It is critical for forward looking organizations to assess the quality of their network facilities data and review their business processes from the perspective of operational efficiency, optimizing business processes and information flows for data quality, and making sure that they have the appropriate supporting technology.  Redundant data and inefficient data and work hand-offs are prime symptoms of an organization focusing on sub-process optimization and ignoring the big picture.  Reviewing your overall business process from soup to nuts with a perspective above the sub-processes (team level) is critical.  As well it is essential to develop a technology architecture that enables automating the overall business process in addition to supporting and optimizing the productivity of each operational team.  

Latest Release of the FDO Toolbox v0.8.8 Available

The latest release of the FDO Toolbox is now available.

FOSS4G: Google's Raul Vera Predicts Paradigm Shift - Mobile Geospatially Enabled Applications without Maps

Sydney is the the home of the Google technical team that were originally Where 2 Technologies who developed Google Maps and now are the folks developing Google Wave.  Raul Vera, who is a member of the Google Sydney team, gave a presentation in which he predicted a paradigm shift in spatial applications.

First of all he said that 70-80% of all Google searches have a geographic component.  In 2011 he expects that there will be more usage of Google Maps on mobile phones than on desktop/laptop computers.  As a result the future will be web applications (web apps), that are loaded on demand and run in a browser.  This means that you could be using a new or updated web app within hours of it being developed.  This defines a new level in agile development.

The future also means mobile geospatial applications without maps.  His example is Google Skymap that runs on the Android OS and provides a map of the sky for your location.  If you're into astronomy you will know that you need your location to provide a map showing the constellations and stars visible to you,  but otherwise you may be unaware that location has been used by the application.  The key to applications like this this is a location API, which can be either the W3C's open location api or an underlying location api developed by Google or another vendor.  The location api returns your location, but different vendors compute this is different ways.  In my experience you need to know how each vendor's location api works, because in the case of Google, for example, it returns the location of your server, which is not necessarily where you are. 

FOSS4G: Jackie Ng on the FDO Toolbox

Wednesday at FOSS4G Jackie Ng gave a presentation on the FDO Toolbox.  It was presented so well that I was flabbergasted to discover subsequently from one of Jackie's blog that this was his first public presentation.  

There are a couple of important reasons why it is worthwhile talking a look at FDO Toolbox.  First of all, it is built on the .Net platform which differentiates it from the alternative C++ and Java tools.  If you are used to working in a Windows .Net environment, you will feel at home here. Secondly, Jackie has put a lot of effort into the UI so the FDO Toolbox is simple to use. Thirdly, it is a comprehensive schema and data management tool that can handle geospatial and non-geospatial data with equal facility.  I was very impressed with how many things you can do with FDO Toolbox including 

Data Management (ETL)

• Create and edit data models (schemas)
• Read and write geospatial data for up to 150 data sources supported by FDO

Data Preview (maps for geospatial data)

• FDO query
• SQL query

Batch automation

• Batch scripting to automate common geo-processing tasks

Translation (FDO2FDO)

• Bulk copy selected geospatial data from one FDO data source to another
• Fast copy one FDO data source to another, not quite one-click but pretty close

Extensibility

• FDO Toolbox can be extended through extension modules. With extension modules you can add new commands in C# for custom functionality and include new menu entries to expose them in the user interface.

Jackie demonstrated the latest release of the FDO Toolbox 0.8.7 which includes a dramatically improved bulk copy with a completely revamped user interface and XML file definition.  Also FDO Toolbox now supports Python scripting.

FOSS4G Sydney: Senator Kate Lundy on Government 2.0 and Open Geospatial Data

I've been at FOSS4G 2009 here in Sydney for the last few days.  This has been an exceptionally interesting conference and given that Sydney is a long way for many people, there has been a very good turnout.

Thursday morning I chaired the morning session where we heard very interesting speakers.  The first speaker I had the honour to introduce was Senator Kate Lundy who has represented the Australian Capital Territory in the Australian Parliament since 1996.  Senator Lundy is currently Chair of the Joint Standing Committee for the National Capital and External Territories and a long-standing active member of the Senate Environment,  Communications and the Arts Committee.  Most importantly she is a strong advocate of the internet, digital technology and ICT innovation as an important driver of the Australian economy. 

A favourite web site of hers that I find very innovative in a Web 2.0 sense is the National Archives of Australia site called Mapping Our Anzacs, where the public can not only retrieve information about the Australian and New Zealand Army Corps (which took the brunt of Winston Churchill's unsuccessful Gallipoli campaign in the First World War), but can contribute their own relatives' wartime experiences to a public scrapbook or create a tribute to relatives or others who participated as Anzacs in the war.

I have singled out a couple of things that Senator Lundy said  because they appear to represent a clear statement of government policy with respect to Government 2.0 and open geospatial data.

Government 2.0

The current government appears to be a major supporter of government 2.0, which is seen as an enabler of openness, transparency, and accountability.   But Senator Lundy stressed that making government data open and available to citizens not only provides for more transparent government, but it also creates opportunities for innovation which in turn drives economic development.  Senator Lundy said that "as has been evident in the US for many years, open access to government data can dramatically increase the value created from the data both socially and economically. This means the society as a whole benefits from access to the data."

Open Geospatial Data

I have blogged previously about initiatives in the Government of Queensland and the Victorian Spatial Council about enabling open access to government data within government and outside of government.  But Australia has not yet established open and free access to geospatial data as has happened in the US at the federal level, California, Canada, South Africa, Japan, Brazil, and other jurisdictions.  Senator Lundy supports open access to data and in her presentation listed several initiatives the current government is undertaking to move in the direction of making geospatial data publicly available.  One thing in particular she said suggests a comittment on the part of the current government to establishing a policy of open access to geospatial data. "The Australian Special Minister of State has articulated the government’s view that the default position of government ultimately ought to be that all public sector information should be available online unless a case can be made not to release it. This view has been echoed in several portfolios and we have currently before parliament a Freedom of Information Bill and a Bill for the establishment of an Information Commissioner. These bills begin these reforms and provide the framework for future progress in this crucial area of public policy."

Web 2.0, Participation, Google Maps, Quality of Asset Data in Utilities and Smart Grid

If you remember, Eric Raymond in the Cathedral and the Bazaar floated the "many eyeballs" theory that all bugs are shallow if you have a large enough community involved in finding solutions. I also remember that Tim O'Reilly in his seminal paper What is Web 2.0 identified some of the important differences between Web 1.0 and Web2.0, among which a key one in my view was participation. In the early days of the web, we were passive consumers of data.  Web 2.0 brought Wikipedia, where users contributed as well as consumed information.  Like Wikipedia the new paradigm has become so successful that wikis are now ubiquitous and the advantage of  involving users in creating and maintaining data has been demonstrated over and over.  A now classic example is OpenStreetMap. Most recently Google has realized that users are "remarkable data sources themselves", and Google Maps and Google Earth have a new tool that lets you the user report errors in Google's maps. There is a new menu item "Report a Problem" that allows you to suggest edits, like a new highway on-ramp, new names of parks or buildings, and so on. Google promises to vet the edit within a month and if you submit your email address, they'll keep you posted on their progress.

I have blogged about the poor quality of network facility data in utilities and telecom and about how critical up to date reliable network facility data is going to be for the smart grid.  Many years ago I was struck by the simple low tech way Brad Lawrence of ENMAX Power Corp addressed the data quality problem.  He guaranteed field staff a 24 hour turnaround on all updates from the field by instructing records staff to give updates from the field highest priority.  A subsequent audit by an external auditor, in which a field survey sampled a subset of ENMAX's database and compared it to what was actually in the field, reported 99.6% reliability for ENMAX's outside plant database.  For an industry in which 40-70% data accuracy is typical, this struck me as a remarkable achievement. What Brad did was to empower field staff by enabling them to be participants in maintaining asset data. 

Google Maps and Google Earth have provided a simple online tool that can be used by the user community (typically field staff in the case of utilities) to report errors and changes in network facility data.  In the case of utilities that provide open access to assset data like North Shore City in New Zealand, not only field staff, but citizens could become active participants in maintaining utility asset data.  Given the long history of poor quality asset data in utilities, it seems to me that a radically new approach along the lines of what Google Maps has done is a promising way to achieve and maintain the level of data quality that we need to make the the promise of the smart grid a reality.

Sub-meter Remote Sensing Satellites Launched

DigitalGlobe has succcessfully launched and deployed WorldView-2 on a Boeing Delta II  rocket from Vandenberg Air Force Base in California.  WorldView-2 will collect multispectral imagery at 1.8 meter resolution and panchromatic imagery at .46 meters*. Geolocation accuracy is 6.5m, with predicted performance in the range of 4.6 to 10.7 meters.  Geolocation accuracy with registration to ground control points is 2.0 meters.

This is about a year after the launch of GeoEye-1 Sep 6, 2008.  GeoEye can collect images with a ground resolution of 0.41 meters* in the panchromatic or black and white mode. It collects multispectral or color imagery at 1.65-meter resolution. GeoEye-1 has three-meter geolocation accuracy without ground control points.

*For sub .5 meter resolution imagery the U.S. Government requires re-sampling the imagery to 0.5-meter for all customers not granted a waiver by the U.S. Government. 

Vancouver Makes Spatial Data Freely Available

I blogged earlier about Vancouver's committment to open data, open standards, and open source.  The City is committed to freely sharing its data in accessible formats and has just opened their Open Data Catalogue Beta, Vancouver's data.gov,  where you can find and download the data (shp, kml, csv, xls, dwg, ecw, Mr. Sid) used on VanMap.  The data is available for free and with an unrestrictive license.  This follows the lead of the government of Canada and the provinces and territories.

Terms of Use

The terms of use are very open, equivalent to a BSD, MIT, or Creative Commons Attribution licence.

"The City of Vancouver (City) now grants you a world-wide, royalty-free, non-exclusive licence to use, modify, and distribute the datasets in all current and future media and formats for any lawful purpose. You now acknowledge that this licence does not give you a copyright or other proprietary interest in the datasets. If you distribute or provide access to these datasets to any other person, whether in original or modified form, you agree to include a copy of, or this Uniform Resource Locator (URL) for, these Terms of Use and to ensure they agree to and are bound by them but without introducing any further restrictions of any kind."

Open Data in the UK

I've blogged about open spatial data several times in the past and about the contribution of spatial data to the national economy.

In June of this year the UK Prime Minister Gordon Brown announced that in order to ensure that "Government information is accessible and useful for the widest possible group of people, I have asked Sir Tim Berners-Lee who led the creation of the World Wide Web, to help us drive the opening up of access to Government data in the web over the coming month."  This seems to have been driven by examples such as data.gov implemented by the Obama Administration in the US as well as the issues around MPs' expense claims.  The Prime Minister just had another meeting with Tim Berners-Lee who briefed the Cabinet about progress toward the goal of delivering a single online access point to Government information and proposals to extend the open data approach.  To support this the government's announced plan is to ensure broadband access for all UK households by 2012 and build a nationwide high-speed broadband network by 2016.  According to the 2008 ITIF Broadband Rankings broadband penetration in the UK is 13th worldwide slightly behind the US.

The UK government expects that universal broadband access and open data will lead to economic growth through reusing government data for innovative products and services.