Between the Poles

Oracle Spatial and Graph to support NURBS

At the Oracle Spatial and Graph User Meeting, the folks from Oracle outlined some of the things to expect in the next version of Oracle 12c.  I was very impressed.  All of the below is my interpretation of what will be in Oracle 12c.

Parametric curve support

The really big news for the architecture, engineering and construction community (AEC) who use CAD and BIM applications is that Oracle 12c is expected to support NURBS (non-uniform rational basis spline) a type of parametric curve widely used in the design space.  According the Siva Ravada the support will be very general and will be able to handle different ways of representing NURBS. Up to now the only parametric curve supported by Oracle was circular arcs.

To me it looks like Oracle is taking a major step toward being able to store natively and manipulate (in some cases by stroking) the parametrized curves used by CAD and BIM applications.  There is a lot of CAD and BIM data out there and being able to store it and manipulate it in Oracle Spatial and Graph is going to be a game changer.

Orthorectification

Oracle is providing support for orthorectification embedded directly in the database.  This will have important implications for the industry.   Up to now, you had to pull the image out of where ever it was stored, transfer it over the network to someone's application often from an image processing company to orthorectify it, and then transfer it back to store the image or stream it to the end user.  With 12c you will be able to orthorectify images in the database.  This is in line with Oracle's objective of bringing processing to the data, rather than the data to processing.

Raster algebra

Oracle 12c will also have support for raster algebra, again in the database.  For example, you will be able to average in the database12 months of national temperature data stored in monthly raster files.

GDAL and PDAL support

Oracle supports the open source GDAL raster libraries, the industry standard developed by Frank Warmerdam (now with Google) and used by just about everyone for accessing raster images including ESRI.  PDAL is a similar type of open source library for point clouds developed by Howard Butler and Michael Gerlek.

3D

Oracle supports three types of 3D data.  3D vectors, point clouds and terrain (surfaces).  Oracle 12c will provide full support for 3D geodetic data, lon lat and elevation in meters or feet.

Performance

Oracle has developed a new engine for handling massive point clouds.  Dan Geringer showed impressive results for a 2.8 billion point point cloud (286 gigabyte file). 

Processing vector geospatial data is reported to be 50 to 100 times faster in 12c than in 11g. 

Raster operations have been parallelized which can dramatically reduce the time it takes for these operations.

Engineered Database

Since acquiring Sun, Oracle has become a hardware and software company and is offering integrated hardware and software solutions that offer extremely good performance characteristics.  For example, processing LiDAR data benefits from hardware acceleration on engineered machines such as Exadata boxes.

Bundled data

Oracle is planning to bundle vector data from Nokia (Navteq), TomTom and others.

Oracle Spatial and Graph

The name of the product is changing to Oracle Spatial and Graph partly because it has had graph capabilities (network and RDF semantic) already for several years.  But probably more importantly because the demand for graph capabilities is accelerating, especially RDF semantic graphs for linked data, text mining, and for social media analytics, according to Xavier Lopez.  Even Oracle's NOSQL database (aka BerkeleyDB) is apparently getting graph capabilities.

Geodesign Summit: Creating 3D BIM models for the entire LA community college system

Michael Rendler is the Director of e7 Architectural Studio which is a non-profit organization working with the Los Angeles Community College District (LACCD).  Its goals are to assist in designing environmentally sound and sustainable additions to current and future campuses.  Michael gave a fascinating overview of how 3D models for all the buildings on LACCD's mulitple campuses were developed.

The LACCD is the largest community college district in the United States and serves more than 250,000 students annually at nine colleges, spread throughout 36 cities in the greater Los Angeles area.  About eight years ago it was decided to initiate a project ro create 3D BIM models of all the buildings on the LACCD  campuses.  One of the most important motivations for creating these models was that BIM models would faciliate energy performance analyses of LACCD buildings. 

There are several very interesting and unique aspects to the development of these models.  First of all the models were created entirely by LACCD students.  Existing structures were laser scanned and BM models were developed from the point clouds. 

Secondly, to ensure a uniform representation of all the buildings across all of the campuses, it was decided to adopt standards wherever possible.  The standards used include IFC, SDSFIE, OMNIClass, and COBIE.

Thirdly, the BIM models were stored in a geospatial database (Oracle Spatial) using real world coordinates.  Real world coordinates enabled the models to be easily integrated with geospatial data from a variety of sources.

The models could then be used for a variety of purposes including visualization, energy performance modeling, predictive maintenance,  and safety and security simulations.

For the safety and security simulation the models were brought into a gaming environment to allow  gamification of safety procedures.

FDO Provider for GE Smallworld makes VMDS data available to AutoCAD clients

The FDO Provider for GE Smallworld, developed by SBS, uses the Autodesk FDO data access technology to provide seamless, dynamic data access to GE's Smallworld. The FDO Provider for Smallworld eliminates the need for data translation, Smallworld Magik programming and custom integration development to access data in a Smallworld's VMDS database.

The FDO provider relies on the Smallworld Connector (SWConnector), a comprehensive API for GE Smallworld.  SWConnector is available in a specialized version for Autodesk’s AutoCAD Map 3D, AutoCAD Utility Design, AutoCAD Map 3D Enterprise and Autodesk Infrastructure Map Server applications. The FDO SW Connector Provider for Autodesk provides access to Smallworld data, allowing display, query, analysis and update via Autodesk products.

Feature Data Objects (FDO) technology is an open source project that enables Autodesk Geospatial products and enterprise applications to query, retrieve and update natively spatial data stored in relational databases, files and web-based services.

An introductory video to the FDO Provider for Smallworld is available on Youtube.  It demonstrates retrieving facilities from a Smallworld VMDS database and creating new features and updating and deleting existing features using an AutoCAD Map 3D client on the desktop   The updates are made in a short transaction database that is then played back and reviewed by a Smallworld operator while loading the job into the VMDS database.

Autodesk announces Project Artoo for geospatial data cleanup

Project Artoo for AutoCAD Map 3D is a free technology preview available on Autodesk Labs.  Artoo is designed to cleanup geospatial data to correct common errors that can be introduced during digitizing, importing, surveying, scanning, and drafting such as duplicates, undershoots, snapping, dangles, node removal, sliver removal, and removing zero-length and short lines.

Project Artoo supports cleanup operations on FDO-connected data stores, including Oracle Spatial, Microsoft SQL Server, SDF, SQLite, and ESRI SHP files and geodatabases.

Burlington Hydro's real-time smart grid for operations and analytics: Managing big data real-time

At the Oracle Spatial User Conference in Washington DC this week, Don Guatto, COO and VP Engineering at Burlington Hydro and Geoff Cameron, EVP Intelligent Networks at AGSI gave a rivteing presentation on Burlington Hydro's (BHI) smart grid implementaton for the utility's GridSmartCity initiatives.

BHI is a utility near Toronto with 65000 meters, 32 substations and 93 employees.  The GridSmartCity initiatives include

• self healing networks - this initiative has reduced average outage times in BHI's service area to 20 seconds per year
• smart meters - Burlington is in Ontario which has deployed smart meters and time-of-use pricing province wide
• distributed generation - Ontario has a Feed-in-Tariff program which has led to the deployment of over 7000 micro-generation projects, mostly solar
• electric vehicle (EV) charging stations
• factory ride-through systems - enabling factories to continue functioning through outages
• battery-based electric storage

Smart grid management system for operations and analytics

An independent consultant hired by BHI made a recommendation with far reaching implications for BHI. "Two of the main factors affecting GIS today are the move towards using internet technology to be able  to serve GIS data to a much broader user base and efforts by the Open Geospatial Consortium (OGC) to promote interoperability.  BHI needs to implement a new GIS that will fit its technology vision and provide the benefits of today's GIS to BHI operations."

Based on this recommendation BHI determined that the foundation for their smart grid management system would be an open architecture based on open standards.  This would allow them to integrate their SCADA system, 65 000 smart meters reporting power use every 15 minutes, the supporting automated meter infrastructure (AMI), intelligent devices, power line sensors, a customer information system (CIS), their ERP system, engineering analysis, and other systems.  It would utilize bidirectional communications to both receive information from and control smart devices. The system would have handle much larger data volumes than BHI had ever experienced before.  For example, some devices were reporting 60 times per second.  And perhaps the most challenging requirement is that it also had to operate real-time, which as Geoff emphasized, is beyond the capability of today's traditional enterprise GIS systems.

Implementation

AGSI's solution uses their Go360 products which provide a scalable, geospatially-enabled solution built on an open service oriented web architecture. It uses Oracle Spatial for a shared, geospatia data repository and AutoCAD Map 3D for design and geospatial data maintenance.  A web browser, either on the desktop or on a mobile device, is all that's required to access tools for asset maintenance, cable locate, asset managemet, operations, financials and drill downs, automating network pinning and work protection tagging, SCADA, schematic views, automated CAIDI, SAIDI and SAIFI reporting, an outage management system (OMS), mobile workforce automation, automated as-built management, real-time asset monitoring and analytics, and an executive dashboard.

One of the most impressive Go360 applications that Geoff demonstrated was a transformer status monitoring dashboard that not only showed a map with transformer loading in real- in the form of a heat map, but could also report on historical loading and even estimate, based on the history of oveloading on a particuler transformer, how much the lifetime of the transformer had been shortened as a result of the overloading.  It also allowed the operator to reconfigure the grid in real time to reduce the load on overlloaded transformers and redistribute it to others with available capacity.

What impressed me most was to see how an open IT architecture based on open standards together with industry standard off-the-shelf components, a bidirectional communications network and real-time smart devices and sensors has enabled BHI to develop a state of the art real-time geospatially-enabled smart grid operations and management system that integrates with their existing enterprise systems and provides a common point of access to all their operational data. 

BHI has plans to contnue to develop smart grud applications in the future, but I think that they have built a strong IT foundation for their future smart grid development.

Technical University of Berlin wins Oracle Spatial Excellence Award for CityGML

Last week I blogged about an extension to the OGC CityGML standard for utilities.  Today at the Oracle Spatial User Conference the Institut für Geodäsie und Geoinformationstechnik - Technische Universität Berlin, where Thomas Kolbe and his team are responsible for the CityGML specificiation, won an Oracle Spatial Excellence Award for Education and Research.

Berlin 3D City Model

I blogged about the 3D city model of Berlin before.  The 3D City Database is a free 3D geo database to store, represent, and manage virtual 3D city models on top of a standard spatial relational database.  The schema of the 3D City Database is based on the City Geography Markup Language (CityGML), an OGC standard for representing 3D city models.  The 3D City Database was developed for the Berliner Senatsverwaltung für Wirtschaft, Technologie und Frauen and the Berlin Partner GmbH.   The 3D City Database has been implemented as an Oracle Spatial relational database schema. It is shipped as a collection of SQL scripts which allow for creating and dropping instances of the 3D City Database on top of an Oracle Spatial DBMS.  The 3D City Database is open source and released under the GNU Lesser General Public License v3 (LGPL).

Data creation is not information use

At the Global Geospatial Conference 2012 in Québec City, Ed Parsons of Google argued in favour of web access to micro data including geospatial. 

Google's goal is to organize the world's information, of which geospatial is just one type and not really so special, as Jack Pellicci then of Oracle, has been advancing for many years. Ed emphasized the the importance of access to data by paraphrasing the well-known movie adage, if you build it, they will not necessarily come.  In the case of spatial data infrastructure (SDI), a lot of this data has been available for a long time, but without access it has sat on shelves gathering dust.  Data creation is not information use and the focus on data creation has obscured the importance of analysis and visualization, and above all access.  Fortunately, the global information infrastructure already exists to enable this to happen in the form of the Web, which Google used to fundamentally change the way the world accesses spatial data.

Web access to micro data

He then gave an example of being able to access information about one if his favourite pubs in London by a single click, and said that we should be able to access information about individual items of infrastructure, whether a utility pole, power transformer, or water valve in the same way.  Fundamentally this is the internet of things where everything has an internet address or URI.  He gave another example, accessing information about a particular property parcel in Nanaimo, BC with a single click, which is also a good example of how the web allows you to visualize the same data in different ways.

He also made the case for adding more structure to the web, as a way of making the information you retrieve on the web more meaningful.  Many sites are generated from structured data, typically stored in databases. When this data is formatted into HTML, it becomes very difficult to recover the original structured data, but many applications, especially search engines, can benefit greatly from direct access to this structured data.  One way to do this is to use a standard set of html tags, defining schemas, that you can use to markup web pages in ways recognized by major search providers.

National Highway Authority of India's Road Information System

I blogged previously about the Road Information System (RIS) developed to assist in monitoring and managing information about the highway system known as the Golden Quadrialteral connecting Kolkata, Chennai, Mumbai and Delhi.  The National Highway Authority of India (NHAI) is an autonomous organization responsible not only for the development, but also the management and maintenance of national highways in India.  RIS was developed for the NHAI by CES Technologies.

Road Information System

NHAI's Road Information System (RIS) is a web-based geospatially-enabled system that integrates a linear referencing system for tracking the condition of pavement and geospatial data and analysis.  RIS contains data about the highways included in the Golden Quadrilateral (GQ), North-South (NS) and East-West (EW) corridors of the National Highways Development Project (NHDP) which was initiated in 1998.  RIS is a comprehensive highway management system which included linear referencing, asset management, pavement management, traffic information, bridge monitoring, environmental information, toll management, document management, accident infomration, contract database, and HDM-4, an industry standard economic analysis package.  HDM-4 provides tools for estimating funding requirements, budget allocations, predicting road network performance, and assessing policy impacts.

Decision making

RIS includes tools for decision making including Impact of available budget on highways performance that allows the NHAI to estimate the trade-off between maintenance and "roughness" or road condition.  The decision support tools also assist the NHAI in allocating funds to sub networks, optimzing multi-year work programs, and various budget reporting including breakdowns by work type and State and CMU.

California Supreme Court to hear case for free and open access to county geospatial data

I blogged previously about a court case in 2009 involving Santa Clara County in California that was decided in favour of free and open access to county GIS data under California's Public Records Acts (PRA). 

On May 31, 2011 a different California appelate court decided in favour of  Orange County which charges $375,000 for their parcel database. After trying to obtain Orange County's parcel file at the cost of reproduction, in 2009 the Sierra Club initiated a lawsuit against Orange County under the PRA.  In 2010, the Superior Court decided that the PRA exempts GIS databases from public record disclosure because the court concluded that they are "computer software developed by a state or local agency".  

A California appellate court agreed with the Orange County Supreme Court that a GIS database is covered by the PRA software exclusion because it concluded that "computer mapping systems" as defined by the PRA includes software and data.  Since this decision conflicts with the other California appellate court's decision in the case of Santa Clara County, the Sierra Club has appealed the question of access to Orange County's parcel basemap to the California Supreme Court. 

According to a recent article by Bruce Joffe, Orange County's logic in the case is that GIS includes software and data (citing Esri's definition that says "GIS is a collection of software and data"); that the County's Landbase is a GIS; that GIS is a type of Computer Mapping System; that CMS is excluded bySection 6254.9 of the PRA; therefore, the County's GIS Landbase data is excluded. The Sierra Club argued that "Computer Mapping System" means a system of software modules, which does not include data; that GIS-formatted data is necessary for the public to analyze the government's decisions using its GIS database; that "includes" means an illustrative example, not an expansion of the definition of software; and that the California Legislature did not intend to exclude data when it passed the software exemption.

Sierra Club's final appeal has been made to the California Supreme Court based on the two County's public records lawsuits for the same kind of data resulted in opposite opinions being an important reason. On September 14, 2011 the Court agreed to hear the case. 

GIS professionals who may be willing to co-sign a GIS amicus brief can contact Bruce Joffe (founder of the Open Data Consortium project) at 510-508-0213, or email Bruce at GIS.Consultants@joffes.com.

AU: Hungarian State Railways geolocates 1.35 million objects to 5 cm precision in 14 months

One of the major challenges facing companies that are responsible for managing the world's infrastructure networks is data quality.  I have blogged about data quality on multiple occasions (here, here, here and here).  At Autodesk University Peter Barcza, Managing Director of BPMC Ltd., gave a fascinating presentation of how Hungarian State Railways (MAV), with a very tight budget and a legally binding deadline of January 2, 2011, which translated into a tight timeframe of a little over a year, was able to geolocate 1.35 million objects to 5 cm (2 inches) precision.

Data collection and data quality are the critical success factors for most GIS projects. From 50 to 90 percent of resources are often used for data collection and ensuring data quality. A railway is a multi-utility environment and the MAV team had to classify and geolocate 200 different types of equipment including 7800 km of track, bridges, signaling systems, energy supply, overhead power lines, telecommunications, real estate and facilities such as stations, and other non-MÁV objects.  The original paper maps, some of which dating from the Austro-Hungarian Empire, were unreliable for a number of reasons and the precision was in the 1.5- to 2-meter range, inadequate for MAV's purposes. Traditional surveying is time-consuming and MAV would have had to hire expensive external resources to conduct the surveying. To address these issues, Peter re-geolocated all of MAV's facilities using an integrated GPS/GIS data collection process designed for a multi-utility environment.

After analyzing various alternatives, the MAV team decided that given the tight budget and the tight timeframe, GPS/GLONASS based data collection was the only feasible strategy. A country wide GNSS RTK system of 54 earth stations (≤2 cm [1σ] horizontal, ≤3 cm [1σ] vertical) meant that the measurement precision was estimated to be within 5 cm (2 inches), which was adequate for MAV's purposes. Data would have to be collected in the field for later processing because of the difficulty in geolocating and collecting attributes of many of the objects. The data collection process was designed to ensure efficient, semi-automatic, data upload to the GIS system.  A reliable country wide GPRS communicatiions network made this approach feasible.  The GIS system was comprised of an Oracle Spatial database, AutoCAD Map 3D Enterprise, and Autodesk Infrastructure Map Server.

Quality systems

The key to the success of the project was the quality systems focussed on design, field data collection ("measurement") and data upload.  Multiple layers of control ensured that all data was reviewed, accepted or revised within eight days of collection.  The first level of control was local teams from seven differenmt departments in six different territoreis reviewed collected data independently.  The field data collector did not partcipate in this review.  A second level of control reviewed data at a national level.

Results

MAV was able to complete the survey of 1.35 million objects by January, 2011.  In the process 20% of the assets were "discovered", meaning that they were previously undocumented by MAV.  The discovered objects included 500 properties (land parcels) that belonged to MAV but about which MAV had no records.  

The work was done entirely by MAV employees which made it much less expensive than if external contractors would have had to have been used.  Overall it is estimated that as a result of using internal resources and a GPS/GLONASS-based approach the project was 16 times more efficient than a traditional survey.  And the project generated a lot of pride among MAV employees who carried out the work because it was such a remarkable achievement from a data collection, management and quality perspective.