Between the Poles

Dubai

Map Middle East 2006 was held this year in Dubai, which I discovered is one of the most remarkable places in the world.  First of all it has literally risen out of the desert in the last ten years. Everywhere you look there is construction.  If you look closely at this picture, you'll notice that every one of the buildings in the distance is surrounded by cranes because all are under construction.  There is a shortage of office space and residential space, and buildings are going up at a phenomenal rate.  One evening Haris Kurtagic, GM of SL-King and an Autodesk partner, and I went to the Dubai Marina for dinner, and not only was there a major building under construction every direction you looked, but at 22:00 at night construction was continuing at all of these sites.  It looks like construction is 24 x 7 in Dubai.  An incredible statistic that I heard was that right now over 15% of the world's cranes are in Dubai.

The result is a skyline that changes nightly.   At Map Middle East Haris and I were fortunate to  meet  Rana Mostafa Mnini  who is  In-charge of the drawing offices of the Health and Medical  Services  Department of the Government of  Dubai and an Autodesk user.   Rana explained that one of the reasons that makes it possible for buildings to go up at this frenetic pace is incredible pressure to deliver and that  a critical part of this are reliable design tools that enable people to be productive.

Map Middle East 2006

Since Monday I've been at Map Middle East 2006 in Dubai.  There are people here from several parts of the world,  but primarily from the Middle East as you might expect.  Some very interesting themes have been apparent in the talks I have attended.  First of all, Google Earth is on everyone's mind.  Ed Parsons from the Ordnance Survey is taking bets that every speaker will mention Google Earth.   So far he is ahead.  What was most telling was speaker after speaker, who have been geospatial specialists for years, reporting that their children's first (happy) experience with geospatial was Google Earth.  Some people were surprised that Vanessa Lawrence from the OS mentioned Google Earth and perhaps she was motivated by Ed not wanting to lose his bet, but I think that Google Earth is a phenomenon that simply can't  be ignored.

Michael Jones, CTO of Google Earth, is here and presented to a full house.  He spoke about Google Earth's mission, which is to reach the 6 billion inhabitants of Earth that are not familiar with geospatial technology, not the 1 million who are.  Another way to look at this is from the perspective of how to scale to billions,

                 Application                Number of users

                    Expert                                1

            Expert -> experts                       00's

              Expert -> users                    000,000's  (Traditional GIS)

               Users -> users                      billions    (Google Earth)

This is interesting because it makes clear not only who Google Earth is going after, but also who they're not.  It also shows the power of user participation enabled by Web 2.0.  When asked who does QA on Google Earth's data he replied that the users QA it.  He gave an example.  Google Earth recently deployed a new high resolution database for Germany, and within 12 hours they were getting updates and corrections.

Michael also reported that new spatial content is added to Google Earth every second, which means that in the 50 or so minutes he was speaking, 3000 new features were added to Google Earth.   As an example of the type of data people are loading into Google Earth, the location where every scene of Shakespeare's plays occurs are available in Google Earth.

A few other themes recurred.   Spatial data is ubiquitous and getting cheaper.  Someone mentioned a terabyte of new data every day.  Sam Bacharach of the OGC thought this was an underestimate.  The challenge,  which several speakers mentioned, is how to convert terabytes of data to megabytes of useful information and to provide user interactions to make this accessible to the average cell phone user.  It was interesting that several speakers mentioned that maps are not the only or even the best way to present useful information derived from spatial data to users.

I'll leave you with a fascinating bit of information that suggests that the tsunami that was triggered by an earthquake near western Sumatra was not totally unexpected.  The map to the right , which was published in 1979, identified using red through green the areas of highest probability of earthquake activity.  I've included a subset of the map showing the west coast of Sumatra. Thanks to David Rhind of the City University of London for providing these slides.

MapGuide Open Source

March 7, 2006 was a big day for the geospatial community, because that was when Bob Bray, MapGuide Technical Architect at Autodesk, hit the switch that released MapGuide Open Source to the Open Source Geospatial Foundation ( http://mapguide.osgeo.org .)

There are several reasons why I see this as an important milestone for the geospatial IT community.  First it shows a serious commitment by a major geospatial vendor to the Open Source Geospatial Foundation and to the wider open source community.   The code that was released on that day represents many person years of development effort by Autodesk engineers.

Next generation web mapping

MapGuide OS introduces the next generation of web mapping.  For those people whose vision of the future foresees field staff at utilities, telcos, and other organizations responsible for maintaining network infrastructure empowered to take responsibility for the facilities database, MapGuide OS represents the platform on which these solutions can be built.  I would go so far as to draw an analogy with Web 2.0.  Web 1.0 is about publishing, which is what MapGuide 6.x, MapServer, and web mapping products from other geospatial vendors are about.   The item in Tim O'Reilly's list (What is Web 2.0 ? ) that really brought this home is

               Web 1.0        Web 2.0

           publishing --> participation

MapGuide Open Source is about enabling participation where the people in the field will be able not only to view and markup, but also to update spatial, attribute, and connectivity data from a handheld in the field.

If you want to see some early, but exciting,  MapGuide OS sample applications, go to MapGuide OS Sample Apps .  These examples include DWF and AJAX clients and how to link MapGuide to Google Earth.

Landbases and Infrastructure

Landbase evolution

One of the problems that just about every organization responsible for managing network infrastructure has to face is the evolution of its landbase.  By the landbase I mean maps of the roads, water bodies, public buildings, and other features that appear on maps produced by TeleAtlas, NAVTEQ, US Census TIGER, the Ordnance Survey in the UK, BAKOSURTANAL in Indonesia, and other private and public mapping agencies.  Utilities, telcos, and municipalities capture the location of their infrastructure against a landbase.  But a landbase is a moving target because landbases evolve over time, and I'm not referring to continental drift.  The accuracy of the maps is continually being improved, maps are becoming more detailed, roads get widened and straightened, and waterbodies get filled in and created.  For example, as a result of the availability of global positioning (GPS) data there is currently an ongoing effort called positional accuracy improvement (PAI), which is creating a new generation of maps.  The Ordnance Survey in the UK and the US Census are two organizations that have ongoing PAI projects.

Dependency of Infrastructure Networks on a Landbase

Organizations responsible for maintaining network infrastructure adopt a specific landbase and capture the location of their facilities against this landbase.  When the organization decides to adopt a new landbase, which can happen for several reasons such as improved accuracy or because it decides to buy a landbase from TeleAtlas or NAVTEQ rather than maintaining its own, it finds that the locations of its facilities relative to recognizable geographic features such as road centrelines change.  In the example I've included, the old landbase is blue, facilities are black, and the landbase is red.  In many cases facilities, which typically run under or alongside roads, are no longer on or even near the correct road.  In a nutshell the problem is that the productivity of field staff is diminished, because, somewhat paradoxically, although the accuracy of the landbase improves, it becomes more difficult to locate facilities. 

Traditional Solutions

The traditional approach to this problem is to use a technique called conflation, which involves  identifying  control points, features that are recognizable in both the "old" and the "new" landbases. A least-squares algorithm is used to compute a "best-fit" mathematical transformation which it is hoped will move the facilities closer to the relative position they had when displayed against the original landbase.   But this approach is frequently unsuccessful in producing the quality of maps that organizations require to maintain the productivity of their field staffs.

When the traditional approach is unsuccessful, organizations are known to resort to simply re-digitizing their entire network against the new landbase.  Needless to say, this can be expensive and emphasizes how serious a problem landbase evolution is.  Secondly, landbases will continue to evolve, and with each new landbase organizations are faced with a similar problem.  Do they move to a more accurate landbase, which they know will diminish the reliability of their facilities maps.

Relative and Absolute Location

An alternative approach that Serguei Sokolov and I have encouraged people to consider is based on the recognition that the relative location of facilities objects is what is important to field staff and  not the absolute location, which is typically what is captured in spatial databases.  For example, a linesman is not really interested that a transformer is located at 45.4234 N, -75.6789 W, but that it is on the east side of Bank St., 27.5 m south of the intersection with Catherine St.  The key point is that if you capture the location of facilities relative to recognizable features in the landbase such as centrelines and intersections, then  your landbase can evolve without diminishing the usefulness of your facilities maps.

Modelling Infrastructure Networks

Models and Intelligent Databases

A key concept that characterizes next generation design tools is model-driven design.  First generation CAD tools were designed to produce pieces of paper, called construction drawings in the case of telco, utility, road, and highway networks.  The objects that were manipulated were graphical objects, coloured symbols and lines.  By introducing the concept of a model, we can create intelligent databases which are comprised of real world objects such as conductors, transformers, and service points.

Class

Several years ago, Autodesk Map and later AutoCAD introduced support for classification.  This represented a major advance because now an object was not simply a red line or a green triangle, but a real world object like a primary conductor or a transformer, something meaningful to an electrical engineer.   We use the term class to refer to a type of equipment associated with a domain.  For example, a simple electric network is comprised of several classes including primary and secondary conductors, transformers, junctions, terminations, service points, fuses, poles, and substations.  It is often useful to differentiate between classes of objects that can carry an electric current such as conductors and transformers, and structural elements such as poles and transformer pads.

Location

Objects such as transformers, service points, fuses, and poles are typically modelled as  point objects (x,y).  Linear features such as conductors are modelled as polylines (x1,y1,..xn,yn).

Properties

A class of objects has common properties. For example, properties of poles might include the year installed, type of material (wood, metal, concrete), and height. 

Relationships

Objects relate to other objects.  For example, a cable can connect to other cables and to transformers and fuses.  It is useful to differentiate connectivity, which is a relationship between classes of objects that carry an electrical current, and asssociation, which a relationshio between electrical objects and structural objects such as poles and substations.  A pole supports an electrical cable, but does not carry electrical current.

Constraints

Also associated with intelligent databases is a set up constraints which are designed to ensure the consistency of the database.  The most common constraints are of two types, property constraints and relationship constraints. 

Property constaints limit the values that a property can have. For example, the length of a pole must be greater than zero or the year that a pole was installed must be between 1940 and 2006.

Relatonship constraints are constraints on how classes of objects relate to one another.  For example, a conductor can connect to another conductor or a transformer, but not to road or a river.  A pole can be associated with a transformer. 

UML Modelling

A simple graphical way to model infrastructure networks is using UML (unified modelling language), an industry standard (OMG) way of representing object models.  There are several software tools that support UML including Rational Rose.  A very simple introduction to general UML modeling is Objects First with Java: A Practical Introduction using BlueJ.

Benefits of Intelligent Databases

The advantage of a model-based design compared to a traditional CAD drawing is that operations that are standard day to day processes for organizations responsible for infrastructure networks can  be modelled and operated.  For example, a typical operation at a utlility is an upstream trace.  When a customer calls in reporting a power outage at a residence or business, the power utility' needs to follow the electrical circuit from the reported power outage upstream to determine where the failure may have occured.  With an intelligent database which models connected electrical components, you can auomatically perform an upstream trace to identify the network components that may have failed.  You can also export electrical network information to electrical analysis applications such as Millsoft.  In the case of a telco, a typical operation that can be easily performed with an intelligent database is a loop makeup.  Both upstream traces and loop makeups are operations that need to be performed many times a day so that automating these operations improves the productivity of engineering staff.

Domain Models

Dat models are domain specific.  For example, there are different models for the water, wastewater, electric, and gas industries.  Also within an industry there are different models.  Telcos have developed different models for copper, fibre, cable, and wireless networks.  In addition each utility and telco typically defines its own model to fit its business processes.

Infrastructure Management Applications

Infrastructure applications run on intelligent database and are used for data maintenance and automating operations such as upstream trace and loop makeup.  Intelligent databases and infrastructure management applications are not new. The earliest ones date from the early 1980's.  In the early 1990's relational database manaement systems (RDBMS) were first used to store intelligent databases.  What is new is application development tools that enable developers to easily define a data model and an infrastructure management application to support the unique business processes of an organization.  Autodesk has two products that support customized data models. 

For the high end Autodesk GIS Design Server (AGDS) supports highly customized solutions for large telcos and utilities and is able to scale to very large databases and thousands of concurrent users.

–

For the midrange, Topobase supports configurable, as opposed to customized, solutions and will offer out of the box data models for water, waste water, electric, and other infrastructure networks.  Both Topobase and AGDS rely on spatially-enabled Oracle RDBMS.

For the low end, you can expect to see more and more model-based functionality being supported in Autodesk Map3D, MapGuide, and applications like Autodesk Utility Design.  For example, FDO, which is included in Map3D and MapGuide, supports features with class, location, properties and relationships.

To wrap up, if you haven't already, I would strongly encourage you to start thinking about using data models. If you're an AutoCAD or Autodesk Map3D user, consider using model-based features such as classificaton.  It is not difficult to retrofit a set of classes to your existing data, especially if you have been consistent in in how you have represented similar types of equipment and the properties assigned to them.  The benefits of intelligent databases are significant.

Handheld Devices

Up until a year ago the only handheld alternatives that utilities and telcos had for their field staff were either PDAs or notebooks such as Panasonic Toughbooks.

PDAs

PDAs, while fine for tabular information, simply don't have enough graphic real estate for most graphic applications.  Also they run specialized operation systems such as Windows CE, PocketPC, Mobile5, or PalmOS. I know of several companies that tried PDAs, but ultimately replaced them with notebooks.

Notebooks

There are problems with notebooks as well. They are too bulky to take up poles or down manholes. They are also expensive. Some companies bolt them to their trucks so they can't be lost or dropped.

Pocketable

Since early 2005 there has been an alternative.  For over a year  I've been carrying an OQO (www.oqo.com), which is a handheld small enough to go in your pocket (called "pocketable") , with enough screen resolution, 800 x 480, to support serious graphical applications like DWF Viewer and Autodesk Design Review, and able to run full XP Professional.   I have shown the device to many utilities and telcos, and the things that appeal most about it are the form factor, the graphic real estate, and its ability to run full XP.  The latest device, Model 01+, has 512 MB RAM and a 30 GB hard drive and supports Wi-Fi, Bluetooth, Firewire,and USB.  Literally you can run AutoCAD on this device.  The screen is transflective, which means you can see it in bright  sunlight.  And it's reliable. I picked up a USB GPS and have taken the device cross country skiing, hiking, and carried it around the world on airplanes (I like to stick the GPS up against the window on planes to track where we are).  The only serious problem I have had is losing the stylus.

Carry with you UMPCs

Now there is another alternative. Last week at Cebit in Hannover, three devices that are similar to the OQO, but larger were shown.  These devices are called ultra mobile PCs or UMPCs and are "carry with you" as opposed to pocketable.  All three UMPCs at CeBit had touch-sensitive color screens, measure 7 inches across diagonally, and support Wi-Fi, Bluetooth and USB. They weigh just under a kilo, which is twice as much as the OQO, and have 30GB to 60GB hard drives, run on Intel's ultralow-power Pentium and Celeron chips, and support Windows XP Tablet PC Edition with an additional piece of software called the Touch Pack, which provides an on-screen keyboard that you can type with your thumbs.  Microsoft refers to this platform as Origami. Intel sees a tremendous future for these small mobile devices.  Last year they created a new division for mobile devices, called the LPIA Division. 

With these devices currently you can get about two hours of usage.  But as a result of reduced power consumption especially by the screen and improved battery technology, Intel is projecting lower power consumption and improved battery technology so that over the next few years you can expect to see "all day" devices.

UMPCs are in the US$1100 to 1200 range, which compares favourably with most notebooks.

The important trend I see here is that increasingly you will see low cost all day handhelds with a range of form factors, but able to run standard operating systems like XP, Linux, and Max OS X.  This will enable a revolution in mobile computing solutions with full connectivity over W-Fi and 3G and intelligent applications on the handheld that I expect will dramatically improve the productivity of field workers.

National Highway Authority of India

2005 Report Card on America's Infrastructure

The American Society of Civil Engineers (ASCE) publishes a biennial report card on the condition of US infrastructure including roads.  In 2001 ASCE assigned a grade of D+ to the condition of US roads.  In 2003 they reported that the trend for the condition of US roads was downward, and in 2005 the ASCE revised the grade to D.  The 2005 Report Card on America's Infrastructure reported that poor roads cost $275 per motorist.  That's $54 Billion per year.  US motorists spend 3.5 billion hours a year stuck in traffic.  That is a $63 billion a year hit to the economy.  Even more importantly, poor roads are dangerous.  There are more than 43,000 fatalities on US roadways every year.  The FHWA reports that outdated and substandard road and bridge design, pavement conditions, and safety features are factors in 30% of all fatal highway accidents. 

To me this makes it clear that this is a critical problem and that one of the key requirements for maintaining a national highway and road system is the ability to track the condition, traffic, maintenance history and other key parameters in order to prioritize maintenance of a highway system. 

National Highway Authority of India (NHAI)

The NHAI is a relatively young organization, but it has already managed the construction of significant part of the Golden Quadrilateral, India's equivalent of the US Interstate Highway System. NHAI is an autonomous organization responsible not only for the development, but also the maintenance of national highways in India.  In a forward looking decision, the NHAI decided to develop a road information system, called appropriately RIS to track and monitor the condition of the National Highway system.

RIS (Road Information System)

RIS is a web-based system that integrates a linear referencing system for tracking the condition of pavement and geospatial data and analysis.  RIS was developed using Oracle Spatial and MapGuide.  NHAI has just deployed RIS (www.nhai-ris.org). Iinitially it will manage about 20,000 km of national highways.  It is planned to add an additional 45,000 km in the future, so that ultimately RIS will manage over 65,000 km of India’s national highways system.  RIS is a comprehensive highway management system which includes ten subsystems including asset management, traffic, pavement, environment, bridges, tolls, accidents, ad-hoc spatial queries, and HDM-4, which is an economic modelling tool.  The users of RIS are intended to include ministries of the Central and State Governments, financial institutions, highway contractors, toll companies, truck operators, state police, real estate developers, city managers, academic and research Institutes, and the general public.  This is a remarkable system in the range and amount of information and the analytical functionality that is integrated in one site.  It is worth your while to take a look at it.

 

Field Force Automation and the Aging Worforce

At the recent Eastern Utility Forum one of the critical areas that was identified by participants, which included power, gas, and railroad companies, is the aging workforce.  This is a bigger problem for utilities than for other industries.  A UTC  Research Report "The Aging US Workforce and Utilities Industries" reported that the average age in the utility sector is 3.3 years higher than the US national average, that half of the utility workforce is over 45, and not enough young people are being recruited to replace experienced staff who are approaching retirement.  What this means for utilities is that they are facing a critical loss of knowledge and skills.

The 2005 GITA Technology Review reports that field force automation was among the top three applications for the electric, gas, and water industries.  A plausible explanation for the critical importance of field force automation for utilities is the aging workforce.

One approach that some utility companies have used is mentoring, where experienced workers are encouraged to assist in knowledge transfer to new employees.  Another is to encourage older workers to remain on the job through flexible working hours and other enticements.   

But in my opinion the solution is ultimately going to involve IT.  At the Forum this was brought home in a very tangible way by Kodi Ekker and Mike Nelson from Idaho Power and Vito Lee, Denis McCombs, and Mohan Inguva of Nevada Power.  They showed how they have addressed this problem and how it is going to be addressed by more and more utility companies in the future.  They demonstratied what I would call intelligent desktop solutions.  Intelligent solutions like these help younger workers be productive, but without the necessity of years of on the job training to acquire the knowledge that more experienced workers have absorbed through many years of experience.  These systems integrate the production of a construction drawing, engineering analysis, and generation of a bill of materials and job estimates in a single, accessible package.  In the context of the aging workforce problem that utilities are facing, the critical benefit for the future is that with basic AutoCAD skills a relatively inexperienced person can efficiently perform the design, engineering, and financial estimating tasks that are required to complete a work order. 

Looking further down the road, with a handheld computer with enough computing power and graphical real estate to run intelligent applications like this in the field together with wireless communications to exchange information with a central data repository, I think you can foresee the possibilities for enabling younger, inexperienced workers to be productive, dramatically improving the flow of engineering design information between the field and the office, and improved data quality.  The business benefit is improved quality of service to the customer.

 

Common Misconceptions about Open Source Software

I have talked to quite a few people in IT about open source and there are some common misconceptions that I run into. 

1. The first is that open source is only a small part of the software industry. 

The last time I looked there were about 100,000 projects hosted by Source Forge, which is only one of many sites hosting open source projects.  About 70% of the world's web sites run the Apache Foundation's web server software.  I saw a statistic recently that most of the Fortune 500 companies have deployed open source software, Linux, Apache, MySQL, PHP, Perl, Python, Sendmail, and others.  As another example, the latest data from IDC show that worldwide server revenue for Linux servers was $5.7 billion in 2005, a 20.8% increase year over year.   (Interestingly. this represents the 14th consecutive quarter of double digit growth for Linux servers.)

2. Another common misconception is that the opposite of open source software is commercial software. 

The reality is that there are two types of commercial software, open source and closed source (sometime called proprietary).  Many commercial companies base their business entirely around open source software.  Commercial open source companies include Red Hat, Sleepycat, and MySQL, which are internationally well-known examples.  In the geospatial realm, DMSolutions is an example of a company that has based a very successful business entirely around open source geospatial software.   Some companies support both closed and open source software, for example,  Novell (who acquired SUSE), Oracle (who just acquired  Innodb  and Sleepycat), Sun ( who sell Star Office which is based on Open Office), IBM (who distribute Apache web server and Eclipse), Autodesk (who will sell MapGuide Enterprise which is based on MapGuide Open Source), and others.

3. Also common is the misconception that all open source projects are comprised of volunteers working at home in the evenings or weekends.  The implication of this is that the quality of open source software is not up to the standards of closed source software.

At the last MapServer User Meeting in Minneapolis, Dirk-willem van Gulik, President of the Apache Foundation presented a fascinating overview of the Apache Foundation.  If my memory serves, one of the things he mentioned that stuck in my mind is that of the more than 1000 Apache committers (developers who can create and modify Apache code), only one of them is a volunteer.  The rest are full-time developers employed by major corporations to work full or part-time on Apache.  In the geospatial arena Autodesk's contribution to the Open Source Geospatial Foundation was developed, and will continue to be maintained, by full-time, professional Autodesk developers.

With respect to quality, as I alluded to earlier, many of the world's major corporations have deployed open source software.  Open source proponents like Eric Raymond argue plausibly that open source is capable of achieving higher standards of quality because more developers look at the code.  The reality is that, like closed source software, you have to assess each open source project or product based on its own merits.

A simple way to summarize the situation is that the commercial software industry uses two models, closed and open source, and that both play an important role in delivering solutions to the IT industry.  Some software companies prefer the closed source model, some the open, and some rely on both.

 

Eastern Utility Forum

Earlier this week Autodesk hosted a forum for large organizations responsible for managing infrastructure networks.  Participants included power, gas, municipal, and railway companies.  The aspect of the forum that I found most interesting was the common perspective on the business problems that companies responsible for managing infrastructure networks are facing.   Lisa Campbell of Autodesk led a discussion of the the most urgent business issues.  For many of these there was a consensus among the participants.  I would single out two key IT issues for which there was a consensus among the participants.  Integration is a major issue for large utilities.  Mark Julian of First Energy in his introductory remarks  emphasized the importance of tailoring integrated IT solutions to the companies business processes and objectives and not the other way around.  A key aspect of this challenge is the integration of solution from different vendors.  For example, all the participants are faced with integrating ERP systems from SAP, Peoplesoft and others, geospatial technology from several vendors, workflow, asset  and outage management systems from Indus, LogicaCMG, and Invensys, as well as other systems form other vendors.   The business benefits are improved customer service, increased competitiveness, greater productivity and reduced costs.

Field force automation is a key issue that is driven by an aging work force at most utilities and telecoms.  Mobile solutions are also driven rapidly improving technology.  For example, reliable wireless communications, either WiFi and 3G, is becoming increasingly prevalent in many jurisdictions.   I've attached a general architecture for addressing the immediate challenges of field force automation, getting reliable facilities data to the field force and enabling feedback from the field in the form of red lines or markups.  This architecture is scalable, general in the sense that it can support a variety of geospatial tools and hardware, and simple.  The key components are a single point of truth such as  an Oracle spatially-enabled RDBMS containing data about the network infrastructure , a field viewer for viewing and redlining in the field on handheld devices, for example an OQO (www.oqo.com), a cache of light-weight design drawings, and an enterprise synchronization tool ,such as iAnywhere's Afaria (www.iAnywhere,com), to automate the distribution of software and data to the field and to post red lines and markups back to the records people in the central office.  At the forum, Brad Lawrence outlined ENMAX's field force automation solution, a key aspect of which was ENMAX's policy of prioritizing red lines and markups from the field by guaranteeing 24 hour turnaround.  Brad outlined the business benefits of  a basic field force automation solution such as this.  One of the most important is improved data quality which means fewer "returns", more efficient cable locates, improved ability to respond to emergencies,  happier regulators, and reduced costs.