Between the Poles

Inexpensive Digital Camera with a GPS

Until recently the only lower end digital camera with a built-in GPS that I was aware of is the Ricoh 500SE, which is still rather expensive. But there is now a less expensive alternative.  I've acquired a Nikon P6000 which has a built-in GPS.  I've just taken it along on a trip to Las Vegas and other destinations. 

The camera embeds WGS84 lon lat coordinates in the image files which you can upload to MyPictureTown, which recognizes the embedded coordinates and displays the locations in Google Maps and allows you to click on a marker and see the picture(s) at that location and vice versa.  I managed to find enough satellites with strong signals over southern Utah to take a few pictures with embedded GPS coordinates.

Open Geospatial Data

I spend a lot of time in many countries around the World and it is really fascinating how the availability of geospatial data is changing dramatically.  There are a number of reasons for this. 

Accessibility

Geospatial data is also much more accessible than ever before thanks to the Web.  All of the mass market web vendors, Google, Microsoft, Yahoo, and AOL MapQuest provide web access to geospatial data and most enable users to integrate their own geospatial data via mashups.  Google tends not to collect its own data, but to provide access to commercial data from commercial data vendors. Also Google has been encouraging governments to make their data available through Google's applications.  Microsoft on the other hand spends a lot of money to collect geospatial data.

One of the things that software entrepreneurs need to be aware of is that if they develop an application that uses commercial data from Google, Microsoft, Yahoo, or MapQuest, they need to read the fine print and may have to work out an agreement with the vendor, just like they would with Digital Globe or another data provider.

Commercial Geospatial Data Providers

The rise of commercial geospatial data providers like Radarsat, Intermap, TeleAtlas, Navteq, Digital Globe, GeoEye, and the Ordnance Survey (UK) has dramatically increased the volume and quality of geospatial data.  Every year the number of earth observation satellites increases and resolution improves.  In the past year, we have seen the launch of Radarsat II, GeoEye I, Worldview I, Theos, and others.  You can find data from some of these vendors at this location.

There are also new technologies such as high resolution photogrammetry from companies such as Vexcel (now Microsoft) and Pictometry, and laser scanning, terrestial, mobile, and LIDAR. 

The result is that the quality of geospatial data is improving and the sheer volume is going up exponentially.

Open Geospatial Data

Another contributor to the increasing availability of geospatial data is the exponential growth in open geospatial data, the data equivalent of open source in the programming realm.  There are key technologies that have enabled this to happen.  GPSs have become so inexpensive that geospatial data acquisition is now accessible to virtually everyone.  Secondly, the Web 2.0 phenomenon has enabled the widespread sharing of user content, in this case geospatial data collected by you and me. 

But equally important are open geospatial data licensing, which defines the legalities (intellectual properties rights or IPR) for sharing open geospatial data, and the open geospatial data pricing model that determines how much it is going to cost you and me. Open geospatial data derives from two sources, government and what I'll call non-government open geospatial data organizations, which tend to follow a Web 2.0 model, where the data is collected and contributed by users, typically equipped with a GPS.

A motivation for geospatial software entrepreneurs to be interested in open geospatial data is open access.  If you develop a web application that requires geospatial data and you use TIGER or OpenStreetMap data you can deploy your application to the web and be pretty confident that you are not infringing on anyone's copyright.  If you want to do this with geospatial data from Google Earth, Microsoft Virtual Earth, Yahoo, or MapQuest, you need to read the fine print and may have to work out an agreement with the vendor or the data provider.

Government: US

In the US, geospatial data collected by the Federal government is available at no cost and is in the public domain, which means you can do what you like with it including derivative works and resale.  The best known example is TIGER data collected by the US Census. 

Government: California

In the US at the state and local government level, the situation is quite different and data collected by local governments is still characterized by a variety of business models.  However, there is now an important precedent in the State of California.  The California Public Records Act (CPRA) requires that state and local government records should be accessible to anyone requesting them for no more than the cost of duplication.  The Attorney General of California issued an official opinion to the effect that digital parcel data is covered by the CPRA.  The California First Amendment Coalition (CFAC) took one of the counties in California to court after the county denied CFAC's request for the county's parcel basemap data at the (nominal) cost of copying.  The California Superior Court for Santa Clara County supported CFAC and directed the county to provide the parcel basemap data to CFAC charging only for the cost of duplication. So there is now a legal precedent in California for free access to geospatial data.

Government: Canada

As of April 1, 2007, geospatial data from Natural Resources Canada and other Federal, Provincial, and Territorial Governments is available for free, subject to a (made in Canada) license.  That there is a license reflects an important difference between the US on one hand and and British Commonwealth countries like Canada, Australia, New Zealand, and South Africa on the other, in that in the latter all government data is automatically covered by crown copyright.

Government: Japan

In Japan Government Federal government data is available from the Geographical Survey Institute (GSI) at a nominal charge.  High quality data is maintained by the the GSI which has an annual budget for collecting and maintaining this data.  Recently, what is called a Basic Law was passed by the Japanese Diet establishing the principle that data collected by the government should be available to the public at nominal cost.

Government: South Africa

As in Canada all data is covered by crown copyright.  In South Africa the National Government has adopted the principle that digital geospatial data is freely available.

Government: Netherlands

In the Netherlands some geospatial data collected by the government appears to be available for free. But it's not clear.  For example, currently local government, municipalities, and engineering companies have to pay for the topographic maps and Large Scale Base Maps (called GBKN ) that are maintained by the Dutch Cadastre.  There is also Basiskaart On Line; a WMS-service for GBKN maps that you are required to pay for.

Government: Australia

As in Canada all Government data is covered by crown copyright, but because of a variety of licenses and the lack of licences, it is difficult to share geospatial data even between departments within government.

Government: Victoria

Based on the 2008 Victorian Spatial Information Strategy, the State Government has initiated a Framework for Sharing, which has been developed by the Victorian Spatial Council. It comprises a framework of licenses and metadata and incorporates the concept of custodianship.  It is inclusive and is not restricted to government, including besides the State Government, utilities, telecommunications firms, the National Government, other government, quasi-government, and non-government organizations.  At this point it is a voluntary program, and it does not mandate a particular licensing or pricing model.

Government: Queensland

In Queensland there is an initiative by the State Government to create a foundation for sharing geospatial data within the government.  The focus of this initiative is a common licensing model, and the really innovative recommendation that has been made is to implement Creative Commons licensing.  Creative Commons is widely used for books, music, and other media and includes six different licenses including Creative Commons Attribution-Share Alike, which is analogous to Gnu Public License (GPL) in the open source world.

Non-Government

GPS technology has enabled a third type of geospatial data collection organization, which is non-governmental and relies primarily on user collected and contributed geospatial data.  The two I am most familiar with are MalSingMaps.com and OpenStreetMap.

MalSingMaps

MapSingMaps provides Garmin compatible road and highway geospatial data collected by members for Singapore and Malaysia. It is free for personal use, but it is copyrighted and derivative products are expressly not permitted.

OpenStreetMap

Currently probably the best known non-government open geospatial data site is OpenStreetMap

I heard a presentation from OpenStreetMap at Intergeo in Bremen, Germany and I was impressed by the volume of data that has been collected, their World coverage, and by the rate at which new data is being collected.

The total number of registered users is 65 000, of which 10% were active in the last month.  Worldwide 268 million nodes and 21 million ways have been collected.  There appears to be coverage in North America, Europe, Australia, Philippines, and Argentina.

 As you can see from the second slide the volume of OpenStreetMap data appears to be increasing dramatically.

OpenStreetMap data is "a free editable map of the whole world" and "allows you to view, edit and use geographical data in a collaborative way from anywhere on Earth."   It's IPR is covered by the Creative Commons Attribution-Share Alike license.

With respect to licensing there has been a discussion within the OpenStreetMap community about a new license that is specifically tailored for geospatial data.  I believe that two issues with CC licensing motivated this initiative.  The first is the need to distinguish between collective and derivative work for geospatial data.  More often than not, when you're working with geospatial data you are using more than one dataset which represents a collective work.  The second motivation is that in some jurisdictions, geospatial data is not copyrightable. The new OpenStreetMap license means that in the immediate future we will have three different types of open licenses for source code, creative media, and geospatial data.

Future of Open Geospatial Data

Probably the most important technical innovation that has enabled the proliferation of open geospatial data is the availability of inexpensive GPSs. If you look at the decreasing cost of terrestial laser scanners and the geospatial data collection techniques that are being developed using low cost digital cameras, it seems to me highly probably that we can expect an explosion of open geospatial data.

Mapping Underground Utility Networks in Penang

As I have blogged previously, a difficult challenge in many parts of the world is to find a single database which records where all underground assets, such as water and waste water, power, gas, oil, steam, and telecommunications, are located.  There are a number of reasons why this is important. 

The most important one is safety.  You may remember to explosion in Belgium when a routine road excavation resulted in a disastrous explosion with a number of fatalities. 

Secondly, the lack of reliable location information about underground facilities costs utility and telecom companies a lot of money, because when reliable information is not available, they have to roll a truck and send someone to the excavation site to (try) to determine if there is infrastructure that the excavator needs to be aware of.  For each site this can involve seven or more trucks from different utilities and telecos converging on the future excavation site.   Most utility and telecom companies have tens or even hundreds of staff doing this on a full time basis. 

Thirdly, for the company or government agency doing the excavation, encountering unexpected underground infrastructure can seriously delay the project (and negatively impact the budget).

Fourthly, in many jurisdictions utilities and telcos are taxed based on the infrastructure located in the jurisdiction.

Tokyo

Tokyo has had a mainframe-based ROADIC system for many years. GITA sponsored a study trip to ROADIC a few years ago.

Sarajevo, Bosnia

Sarajevo, in Bosnia, has recorded the location of all utility and telecommunications infrastructure operating in the city on paper maps for over 40 years.  This was tied into the building permitting process in Sarajevo.  A few years ago Sarajevo started converting these maps to digital format.

Calgary, Alberta

Calgary, Alberta has had the JUMP (Joint Utility Mapping Project) for many years, though not as long as Sarajevo.

State of Jalisco, Mexico

The Instituto de Información Territorial del Estado de Jalisco is developing an integrated infrastructure database for the State of Jalisco.

Edmonton, Alberta

Edmonton, Alberta also has a shared facilities mapping database. 

Penang, Malaysia

Recently, while I was in Penang I encountered an interesting approach for mapping and maintaining a database of underground facilities that is unlike anything I have seen elsewhere.  It's called Sutra D'Bank (PENANG STATE GOVERNMENT SUBTERRANEAN DATA BANK) and is maintained by a joint venture company EQUARATER (PENANG) SDN BHD (EPSB) formed by Equarater Sdn Bhd and the Penang Development Corporation.

Sutra D' Bank's customers are utilities or any other party undertaking excavations in areas under the jurisdiction of the local government.  The operator of Sutra D'Bank does two things.  They will identify the location of underground facilities in the planned excavation area, using the Sutra D'Bank database of undergound facilities supplemented by an on-site survey using a variety of technologies.  This is similar to the service that many utilities and telcos provide,  but identifies all underground facilities, not just those of one utility or telco. 

And they do more than that. To ensure that the Sutra D'Bank underground facilities database is up to date, the operator of Sutra D'Bank conducts an onsite as-built survey to record the position (X,Y,Z) and all relevant attributes of the new installation which they then upload to the Sutra D' Bank database.  This is unique and very interesting, because it addresses a common problem - what's being captured as as-built, as-designed or as-constructed drawings ? - that many utilities and telcos face who rely on their construction contractors for as-builts.

Report Cards on the Status of Infrastructure - International

There has been widespread awareness in the US about what is popularly referred to as crumbling infrastructure. In the presidential election campaign the issue came up repeatedly.  One of the reasons, perhaps the most important reason, for the high public awareness about the state of US infrastructure is the Report Card on American Infrastructure produced in 2003 and 2005 by the American Society of Civil Engineers (ASCE). I've referred to it multiple times in this blog.

Actually there are similar report cards or efforts underway to assess the state of infrastructure in a number of countries around the World.

Canada

Literature Review of Methodology to Review the Status of Infrastructure

CUTA/ACTU Transit Infrastructure Needs for the Period 2006-2010

Municipal Water and Wastewater Infrastructure: Estimated Investment Needs 1997 to 2012

Life-Cycle State of the Infrastructure Report on Public Works Assets - 2006: City of Hamilton

Edmonton City Council's Infrastructure Strategy

Study of the Status of Full Cost Recovery and Sustainability of Ontario Municipal Water and Wastewater Systems

Japan

The 2007 Report on Japanese infrastructure by the Japanese Society of Civil Engineers (JSCE)

Motivation for the JSCE Report

Australia

 2005 Australian Infrastructure Report Card

New Zealand

Infrastructure Stocktake: Infrastructure Audit

South Africa

The SAICE Infrastructure Report Card for South Africa: 2006

United Kingdom

State of the Nation Report: Transport

Autodesk announces second Digital City pilot

Autodesk announced it is working with the Incheon Free Economic Zone (IFEZ) and the
Incheon Urban Development Corporation (IUDC) to cooperate in the creation of Asia's first Digital City for Incheon, Korea -- the 'Tomorrow City' in Song-do City. Incheon is the second pilot city of Autodesk's Digital City initiative. A similar agreement with Salzburg, Austria was announced in July.

Autodesk National Geospatial Conference: Geospatial Data Sharing in Government - Penang JPBD GeoPortal

This past week Precision Design Solution (M) SDN BHD based in Kuala Lumpur hosted an Autodesk National Geospatial Conference in Penang which attracted major players in the Malaysian telecommunications, oil and gas, power, water, government, and transportation sectors. 

PEGIS Map Browser

I already knew something about geospatial in the Penang State Government because in 2006 I attended MapAsia in Bangkok and I had the opportunity to hear a presentation about a web-based system called PEGIS that really impressed me because it represented an evolution of an expensive, complex GIS requiring high end work stations and experts and focussed on producing printed maps to a modern web-based geospatial server that anyone with a web browser could use for accessing geospatial services. It has meant that the number of users who have access to geospatial data was broadened, the cost of providing access to geospatial data was reduced, and a foundation was built for open government, for providing transparent access to land data and land management processes.  I was so impressed that I blogged about it.

JPBD GeoPortal

This week I had the opportunity to hear about another web-based system developed by the Department of Town and Country Planning (JPBD) of the State of Penang, called JPBD GeoPortal. The presentation was given by Rossli bin Haron, responsible for GIS applications in the Penang JPBD..

There are several unique aspects of the JPBD GeoPortal that dramatically broaden access to geospatial data in the State of Penang.   First of all the JPBD GeoPortal is designed to share geospatial data not only among JPBD's 60 internal users as well as other government agencies, but also with the general public.  The opening screen of the JPBD web site provides five options, one is for the general public, which does not require an account and password, and four that are designed for particular government agencies that are registered with the JPBD and do require login.

Another unique aspect of the JPBD GeoPortal is that it supports the concept of data custodians, in other words, agencies within the Penang government that have been identified as responsible for maintaining specific datasets accessible through the JPBD GeoPortal.  A unique capability of the JPBD GeoPOrtal  is that custodians can use it to update the data they are responsible for.

Motivation for the GeoPortal

The immediate business driver for the GeoPortal was to facilitate timely and efficient access to geospatial data for JPBD’s 60 employees and external clients.  JPBD uses a variety of desktop products including ArcGIS, MapInfo, and AutoCAD, but decided that they needed a cost-effective way to provide access to geospatial services to a much broader audience, a majority of whom had no GIS experience.

The system also had to be able to support  a variety of different geospatial data formats from a number of external sources, including  JUPEM (land registry), PTG (state land office), MPPP (local council for the island of Penang) , MPSP (local council for the mainland of Penang State), other State’s JPBD Departments, PTD (district land Office), Telekom Malaysia, TNB (power company), JKR (public works), PBA (water company), and external consultants. 

Because of limited access to IT resources, JPBD required a web-based development environment that would enable non-programmers not only to develop applications, and also to support and maintain these applications.  High performance was a basic requirement, and it had to perform well on low-end x86 hardware. 

Based on these requirement JPBD decided to implement a web mapping system, and decided on MapGuide as the platform.  They found that they were able to develop and maintain web applications themselves.  In fact the only external help that they required was an outside contractor that helped with data preparation and hardware configuration and setup.

Functionality

Working form the principle that a simple and reliable implementation of a small number of basic functions could address 80% of users requirements, the GIS team at JPBD implemented suppport for base map printing, querying and reporting, parcel information,  property tax information, measuring distances and areas,  land use and zoning information , access for custodians to update the data they are responsible for, and general geospatial data about Penang.  The GeoPortal also supports metadata, and is moving toward the the Malaysian standard MS1759.  From what I saw the GeoPortal is very easy to use and provides a lot of valuable information including thematic maps, photographs of facilities, and textual information, as well as links to other sources.

The development of the GeoPortal was supported by JPBD, PEGIS, MPPP, MPSP through a MaCGDI project in 2004 led by JPBD.

Challenges

One of the things that impressed me about the GeoPortal was the challenges that JPBD faced in developing it.  Besides the general problems of limited awareness among management of the benefits of geospatial technology, limited funding, the difficulty of convincing government agencies to share their data, different standards for metadata in different government agencies, and the difficulty in identifying data custodians who are willing to take responsibility for data sets, the GIS team at JPBD faced some challenges that were unique to their environment.  For example, for security reasons they are not permitted to publish aerial photos.  They also have to follow JUPEM and other government guidelines for data security.  For example, they are not allowed to display data on Google Earth.  Government guidelines require a five year rotation of government employees, so that GIS skills are lost every five years and new people have to be trained.

Two key reasons why government agencies were reluctant to share data were the lack of guidelines for sharing data, basically intellectual property rights, and pricing, what to charge to government geospatial data.  These are not new issues, but were particularly challenging for JPBD because there really were no precedents or defined policies for addressing these issues.  Ultimately, a practical solution was adopted.  A set of guidelines was developed and agreed upon that defined standard operating procedures (SOP), with defined rights and costs for sharing data among MPPP, MPSP, JPBD, PEGIS, technical departments (JKR), and other user groups.  This is really a huge accomplishment, finding a way for multiple government agencies voluntarily to work together to share data.

Future

The GIS team at JPBD has challenging plans for the future.  First for all they intend to continue to enhance support for MyGDI, the national MaCGDI portal for state and local governments.

Secondly, they intend to develop 3D city models starting with heritage areas including support for high resolution photogrammetry and laser scanning.

They are also planning to develop an "eInventory" for open space and traditional villages (kampung) in Malasia to help prevent the loss of these important resources as Malaysia undergoes rapid development.

They are also planning to develop a common web mapping platform based on MapGuide for all of the web mapping applications in the Penang Government including JPBD Geoportal, PEGIS Map Browser, and MPSP GeoPortal.

Finally they are thinking very creatively of new ways to even further boraden access to geospatial data and services and to expand the concept of custodianship.

Net Net

The GIS team at Penang JPBD has overcome some incredible hurdles to improve data sharing in Penang.  They have dramatically expanded access to geospatial data and services both within the government and with the general public and have begun the process of establishing the concept of custodianship within the Penang goverment.  They have also begun to address the difficult issues of intellectual property rights and pricing for government geospatial data.  Really a very impressive achievement.

GITA Japan: Critical Infrastructure

Last week I had the opportunity to present at GITA Japan's 19th annual conference, held this year at Tokyu University. I found it very well attended both vendors and service providers and utility and telecommunications companies.  There were some very thought provoking presentations by international as well as Japanese speakers.

Introduction - Professor Sakura Shinoaki

Professor Shinoaki is well-known in Japan and in GITA because he was instrumental in designing the Road Administration Information Center (ROADIC)  system which was deployed first in Tokyo and then in most major cities in Japan.  I visited the ROADIC site in Tokyo several years ago and GITA sponsored a ROADIC Study Mission in 2003. 

Thanks to the ROADIC system Tokyo in one of the few places in the world where there is a single location where you can find accurate maps of all outside infrastructure including telecommunications and utilities, and in what I would call 2.5D.   The other cities I am aware of are Calgary and Edmonton, Alberta, and Sarajevo, Bosnia, where until recently the location of all utilities was recorded on paper maps.  This has been done for the past 40 years.  In the UK the Traffic Management Act, which received Royal Assent two years ago, but has only been implemented in April of this year, is a first step toward achieving a similar capability on a national scale.

The business benefits of a single site mapping all utilities and telecom infrastructure, for example, for  call-before-you-dig services, are so compelling that I continue to be surprised that every jurisdiction has not mandated this.  But I think that when you consider the additional benefits in a disaster situation such as Ike or Katrina in the US or the 2007 floods in the UK, I really think it is just a matter of time.

The Importance of Infrastructure Interdependencies - Bob Samborski

In 2004 GITA responded to Federal Government initiatives by beginning what became GITA's Geospatially Enabling Community Collaboration (GECCo) to enhance a coordinated response to emergencies using geospatial data.  Bob Samborski and Dave DiSera has been instrumental in this initiative and their presentations focussed on issues relating to critical infrastructure protection.

Perhaps the most startling slides that Bob showed  was the size of the challenge in the US.  The sheer numbers of facilities is staggering. 

One of the very persuasive examples of the interdependence of critical infrastructure is the Baltimore Howard Street Tunnel train derailment.  The derailment caused a fire, which lead to a water main fracture, which resulted in a flood, which led to a power outage which affected 100 000 people.  As well a fibre optic cable running through the tunnel was destroyed, and train services was disrupted for a significant period of time with the result that coal could not be delivered to a thermal power plant, ...

The Geospatial Dimensions of Critical Infrastructure - Dave DiSera

Dave discussed how proximity is an important factor in the interdependence of critical infrastructure and gave as one example a high voltage power line, a gas pipeline, and a waterway all within close proximity so that a major problem with the pipeline, for example, would most certainly impact power transmission and thereby a large number of people, as well as a significant transporation artery.

He also discussed models that have been developed for critical infrastructure interdependence by a number of organizations, but singled out those developed by Idaho National Lab, Argonne National lab, the Government of Canada, and Las Alamos National Lab.

An Overview of the Victorian Government Spatial Strategy...Initiatives and Issues - Neil Smith

Neil Smith is with the Department of Transportation of the Victorian Government.  I have blogged previously about the industry leading initiatives in the area of spatial data sharing by the Victorian Government.  Neil provided insight into the organizational structure of the Victorian Government that have enabled these initiatives to happen.  But the thing that Neil talked about that really rang a bell with me is how pervasive data redundancy is within governments, indeed organizations, everywhere.  The concepts of a single point of truth and custodianship appear to be the central foundation on which Victorian Government is attempting to do to address the problem of data redundancy.

Geographical Survey Institute - Yamato Tanaka

One of the interesting presentation in Japanese that I only partly understood was by Yamato Tanaka who talked about an initiative at the Federal government level to create a shared database that integrates geospatial data from all levels of data including Prefectural and Municipal.  From what I understood tallking to people in local government is Japan, one of the challenges is how this effort is going to be funded, by the Federal government or my local government.  It struck me that there are there are analogies with the attempt by DHS to create and implement a common data model GDM for sharing geospatial data at the state and local level in the US.

Crumbling Infrastructure

I've blogged several times about crumbling infrastructure in North America and the score card prepared by the American Society of Civil Engineers assessing the state of US infrastructure.  I mentioned this during my presentation in Japan and I was surprised to learn that there is a similar effort going on in Japan.  I say surprised, because Japanese infrastructure, at least what I have seen of Japanese streets, roads, and highways, and electric infrastructure, appears to be so much better maintained that what we are famliiar with in much of North America, both in the US and Canada.

UMPC for utilities

Finally, I'll mention that I have been interested in UMPCs for several years.  I have carried an OQO around with me for several years.  I've blogged about UMPCs and have encouraged utilities and telecommunications firms that I meet with to take a look at them.  At least two vendors at GITA Japan were showing mobile applications running on handheld devices.  The one I looked at was running on a Fujitsu U810 with a 5.6" screen, 40 GB h/d and running Vista.  Quite an impressive small device for about C$1200 which is about US$1000 these days, much cheaper than Panasonic Toughbooks, the standard mobile device for many utilities, that cost 4X to 6X times more.