Between the Poles

FDO Toolbox 0.9.1 Available

Jackie Ng has released FDO Toolbox 0.9.1.

Autodesk University: Migrating CAD to Enterprise GIS

At AU I made time to attend an excellent presentation by Lou Ball, who has been in the utility business for over 20 years, on migrating data from CAD drawing files (typically DWGs) to GIS.  CAD applications,  whether you're talking about AutoCAD or Microstation, are (too) flexible and the challenges are identifying where the data elements you intend to migrate are, cleaning up the data, preparing them for migration. Lou outlined the steps to follow in migrating CAD drawings.

• Understand the source data
• Map features from source to target
• Plan for data cleanup
• Develop a migration strategy

Drawing Assessment

Assessing drawings is the first, very important part of migration. Just because a printed drawing looks clean, it still may require a significant effort to identify where the data you're interested in is stored and to prepare the data to be migrated to the GIS. For example, in the case of AutoCAD-based applications, data may be in object data, extended entity data, block attributes, externally linked databases, stored in labels (see the attached image), or something you might have to infer from linestyle and colour.

Lou talked about other typical problems, for example, common line problems, unsnapped lines, lines not split, and no point feature where lines are joined.  Blocks also represent challenges.

Data Cleanup Tools

There are several very good tools that can help cleanup data.  Lou outlined his experience with each and the types of problems each can help with.

• AutoCAD CAD Standards
• AutoCAD DesignCenter
• Autodesk Map3D Drawing Cleanup
• Drawing Cleanup Utility (DCU)

Migration Tools

He outlined the migration tools he has used and the advantages of each.

• AcClassify Enterprise
• SHP2SDO – Shape to Oracle Spatial
• Safe Software® FME™
• Autodesk Topobase

Really this is a wealth of information and experience that is invaluable for anyone who has to migrate CAD drawing data to a GIS.  You can find Lou's presentation on the Autodesk University web site.

FDO Toolbox 0.9.0 Released

Jackie Ng has announced the release of a new version 0.9.0 of the FDO Toolbox. You can also see Jackie Ng's presentation on the FDO Toolbox at FOSS4G here.

Chile: Open Standards, Open Source, and Open Data

I have been in Chile most of last week in Santiago, where I gave a presentation at the International Cartographic Conference (ICC 2009).  This is my first time in Chile, and I am extremely impressed by the natural environment, the Andes on one side and the Pacific Ocean on the other, and secondly by the quality of the infrastructure, especially the electric power network, I have seen here in Santiago.

I have had the opportunity to visit several government and commercial sites.  One in particular I found particularly interesting. I met Ivan Alonso Lienlaf Nova of SNIT (Sistema Nacional de Coordination de Informacion Territorial), which is a federal government organization that was created in 2006 to coordinate geospatial data among eleven ministries of the federal government.  From a technical perspective they have created a data catalog for all government geospatial data.  The national catalog currently contains metadata and references for 10 000 geospatial products in diverse areas such as agriculture, biology, administrative boundaries, roads, educational institutions, and cadastre.  SNIT is reaching out to other non-government sectors as well, including private companies and universities.

Open Standards

SNIT supports open standards including OGC and ISO standards, and in particular the ISO 19115 and 19139 standards for metadata. 

Open Source

GeoNodo is SNIT's web-based application portal, that is designed to allow developers to build web-based applications for accessing the SNIT metadata catalog and indirectly government geospatial data.  GeoNodo is based on open source geospatial tools,  MapServer, PostGIS, and Apache. The Regional Government of the Region del Los Rios has developed a web site using GeoNodo.

Open Data

Chile's freedom of Information bill was passed in 2008, after a decade long campaign by Proacceso, Chile’s right-to-know movement.  Ivan Alonso was optimistic that the bill will result in free and open access to government geospatial data, perhaps in a similar way to the November, 2008 decree published in Brazil that established the principle that federal government geospatial data should be made available without cost to Brazilian citizens.

Latest Release of the FDO Toolbox v0.8.8 Available

The latest release of the FDO Toolbox is now available.

FOSS4G: Open Source GeoREST Project Announced

I have blogged on previous occasions about the RESTful API for accessing raw geospatial data which Haris Kurtagic developed and which Jason Birch has used to provide open access to geospatial data at the City of Nanaimo, BC.  At FOSS4G I gave a presentation on behalf of Haris and Jason, who couldn't make it to Sydney, which announced the creation of the open source GeoREST project (www.geoREST.org).

Open Access to Raw Geospatial Data

The principle of free and open access to government geospatial data has been adopted by many governments including US Federal, Canada, Japan, South Africa, Brazil, California, and most recently by the City of Vancouver.  The benefits of free and open access to government geospatial data as identified by Senator Kate Lundy are

• Citizen engagement
• Transparency and accountability
• Economic development

But being able to access prepared maps and other derived material is just a first step, governments need to provide access to raw geospatial data in commonly used Web-friendly formats. Tim Berners-Lee, in Government Data Design Issues, is a strong advocate of making raw data accessible over the web including supporting standard Web methods, most critically, searching.  The thesis on which GeoREST is based is that standard Web protocols, HTTP and MIME, are the simplest way of providing searchable raw geospatial data on the web.

REST

Representational State Transfer (REST) is an architecture style of networked systems which relies on the Web protocols HTTP and MIME. The best example of a RESTful implementation is the Web itself.  REST uses URIs, HTTP methods (GET, POST, PUT, DELETE), HTTP response codes, and MIME.

The single major advantage of REST is that because it relies only on standard Web protocols, standard services available on the Web are available "out of the box" including

• Searching
• Passing through firewalls
• Caching
• Scaling/load balancing

GeoREST

GeoREST is a framework for accessing, distributing, and editing raw geospatial data over the web. Because it is based on REST, it supports searching using standard Web search engines such as Google and Bing. 

In its current form it provides feature-based access to FDO spatial data sources including editing. Examples of FDO data sources include PostGIS, SHP, Oracle Spatial, Microsoft SQL Server Spatial, MySQL, SDF, GML, and others.  GeoREST returns geospatial data over the Web in a number of Web friendly formats including KML, GeoRSS, HTML, GeoJSON, XML, PNG, HTML, CSV, and other text-based formats.  

GeoREST relies on open source libraries including CTemplate, POCO, FDO, MapGuide Open Source, and libkml.  GeoREST source code is available at www.geoREST.org.

Example Application Using GeoREST

You can find an example of an application built on GeoREST at the City of Nanaimo Property Search Site. More information about the implementation can be found on Jason Birch's Blog.

If you want to verify that Nanaimo's raw geospatial data is searchable, enter "2323 Rosstown Road, Nanaimo, BC" into Google, Bing or any other search engine, and then click on the first item that gets returned by the search engine.

FOSS4G Sydney: Senator Kate Lundy on Government 2.0 and Open Geospatial Data

I've been at FOSS4G 2009 here in Sydney for the last few days.  This has been an exceptionally interesting conference and given that Sydney is a long way for many people, there has been a very good turnout.

Thursday morning I chaired the morning session where we heard very interesting speakers.  The first speaker I had the honour to introduce was Senator Kate Lundy who has represented the Australian Capital Territory in the Australian Parliament since 1996.  Senator Lundy is currently Chair of the Joint Standing Committee for the National Capital and External Territories and a long-standing active member of the Senate Environment,  Communications and the Arts Committee.  Most importantly she is a strong advocate of the internet, digital technology and ICT innovation as an important driver of the Australian economy. 

A favourite web site of hers that I find very innovative in a Web 2.0 sense is the National Archives of Australia site called Mapping Our Anzacs, where the public can not only retrieve information about the Australian and New Zealand Army Corps (which took the brunt of Winston Churchill's unsuccessful Gallipoli campaign in the First World War), but can contribute their own relatives' wartime experiences to a public scrapbook or create a tribute to relatives or others who participated as Anzacs in the war.

I have singled out a couple of things that Senator Lundy said  because they appear to represent a clear statement of government policy with respect to Government 2.0 and open geospatial data.

Government 2.0

The current government appears to be a major supporter of government 2.0, which is seen as an enabler of openness, transparency, and accountability.   But Senator Lundy stressed that making government data open and available to citizens not only provides for more transparent government, but it also creates opportunities for innovation which in turn drives economic development.  Senator Lundy said that "as has been evident in the US for many years, open access to government data can dramatically increase the value created from the data both socially and economically. This means the society as a whole benefits from access to the data."

Open Geospatial Data

I have blogged previously about initiatives in the Government of Queensland and the Victorian Spatial Council about enabling open access to government data within government and outside of government.  But Australia has not yet established open and free access to geospatial data as has happened in the US at the federal level, California, Canada, South Africa, Japan, Brazil, and other jurisdictions.  Senator Lundy supports open access to data and in her presentation listed several initiatives the current government is undertaking to move in the direction of making geospatial data publicly available.  One thing in particular she said suggests a comittment on the part of the current government to establishing a policy of open access to geospatial data. "The Australian Special Minister of State has articulated the government’s view that the default position of government ultimately ought to be that all public sector information should be available online unless a case can be made not to release it. This view has been echoed in several portfolios and we have currently before parliament a Freedom of Information Bill and a Bill for the establishment of an Information Commissioner. These bills begin these reforms and provide the framework for future progress in this crucial area of public policy."

Web 2.0, Participation, Google Maps, Quality of Asset Data in Utilities and Smart Grid

If you remember, Eric Raymond in the Cathedral and the Bazaar floated the "many eyeballs" theory that all bugs are shallow if you have a large enough community involved in finding solutions. I also remember that Tim O'Reilly in his seminal paper What is Web 2.0 identified some of the important differences between Web 1.0 and Web2.0, among which a key one in my view was participation. In the early days of the web, we were passive consumers of data.  Web 2.0 brought Wikipedia, where users contributed as well as consumed information.  Like Wikipedia the new paradigm has become so successful that wikis are now ubiquitous and the advantage of  involving users in creating and maintaining data has been demonstrated over and over.  A now classic example is OpenStreetMap. Most recently Google has realized that users are "remarkable data sources themselves", and Google Maps and Google Earth have a new tool that lets you the user report errors in Google's maps. There is a new menu item "Report a Problem" that allows you to suggest edits, like a new highway on-ramp, new names of parks or buildings, and so on. Google promises to vet the edit within a month and if you submit your email address, they'll keep you posted on their progress.

I have blogged about the poor quality of network facility data in utilities and telecom and about how critical up to date reliable network facility data is going to be for the smart grid.  Many years ago I was struck by the simple low tech way Brad Lawrence of ENMAX Power Corp addressed the data quality problem.  He guaranteed field staff a 24 hour turnaround on all updates from the field by instructing records staff to give updates from the field highest priority.  A subsequent audit by an external auditor, in which a field survey sampled a subset of ENMAX's database and compared it to what was actually in the field, reported 99.6% reliability for ENMAX's outside plant database.  For an industry in which 40-70% data accuracy is typical, this struck me as a remarkable achievement. What Brad did was to empower field staff by enabling them to be participants in maintaining asset data. 

Google Maps and Google Earth have provided a simple online tool that can be used by the user community (typically field staff in the case of utilities) to report errors and changes in network facility data.  In the case of utilities that provide open access to assset data like North Shore City in New Zealand, not only field staff, but citizens could become active participants in maintaining utility asset data.  Given the long history of poor quality asset data in utilities, it seems to me that a radically new approach along the lines of what Google Maps has done is a promising way to achieve and maintain the level of data quality that we need to make the the promise of the smart grid a reality.

FOSSLC: Introduction to OpenStreetMap

At FOSSLC Summercamp, Richard Weait presented an overview of OpenStreetMap.  You can watch his presentation here.  He relates a fascinating story about a map edit disagreement in Cyprus where two contributors disagreed about the name of a town, Greek or Turkish.

Whitehouse Using OpenStreetMap Data to Track Change in Communities

The Whitehouse web site is using OpenStreetMap data to track how people are delivering on change in their communities.