Between the Poles

Digitally capturing the location of underground utilities detected using electromagnetic techniques

The industry standard for locating underground utilities is a manual process involving the use of an electromagnetic (EM) wand to detect underground cables and pipes.  The detected location is typically recorded by painting or otherwise marking the ground.  I recently had the opportunity to chat with Kieren Tinning of Terraflow Geomatics, whose Utility Mapper product adds significant value to an industry standard underground locating tool by enabling the digital capture of the location of underground pipes and cables detected with an EM wand and the generation of 3D data for integration with CAD and GIS applications. On a recent depth of cover survey of a thousand kilometre gas transmission line it was found that because Utility Mapper automated most of the data capture and geoprocessing staff time was reduced by 42 %.

Background

In the U.S., underground utilities are a major cause of highway construction delays. Knowing where things are underground has become important enough that in several countries around the world - France, the Netherlands, Singapore, and the U,K., initiatives are already underway to create national digital twins of above and below-ground infrastructure. Recently there have been important technical advances in several areas that are have brought the feasibility of cost-effective mapping of underground infrastructure at the municipal, regional, and national level within reach.  One of these is to digitally capture the location of underground utilities detected using remote sensing technologies such as electromagnetic (EM) and ground penetrating radar (GPR).

Remote-sensing tools for underground utility detection have been available for decades. Electromagnetic (EM) detection wands can be used in several different ways. Passively detecting utilities with EM is used for services that emit naturally emit a frequency such as electrical cables (50 or 60 Hz).  Another approach is to physically connect a signal generator to a service.  This method requires physical access to the service, usually by way of a surface feature that is connected to the service such as a valve. Another technique is to introduce an active signal into a line through electromagnetic induction (EMI). The advantages of EM detection are that it is simple to use and effective for detecting conductive underground infrastructure.  Ground penetrating radar is an alternative technique for underground detection, but in some soil types such as aggregate and clay GPR does not perform well. Without supplementary information GPR does not identify the type of utility (gas, water, electric, fiber optic). In addition GPR requires an expert, typically a trained geophysicist, to interpret the scans. As a result EM detection has become the industry standard.

One of the serious deficiencies of EM is that most wands do not permit the digital capture of the location of detected underground objects.  Typically EM detection is a manual process in which the location of underground infrastructure is recorded by painting or otherwise marking the ground.  Terraflow Geomatics' Utility Mapper changes this by enabling the digital capture of the location of underground pipes and cables detected with an EM wand. Using together RTK devices such as Trimble's Catalyst enables highly accurate location information to be simultaneously captured.  Terraflow's Data Engine automates the synchronization of this data back to the desktop for integration with AutoCAD, Microstation,  ArcGIS and other applications.

Customer experience

PVS Contractors performs large scale depth of cover and other locate services for pipeline operators. Prior to acquiring Utility Mapper, calculating the depth of cover was a manual process in which the depth was recorded manually and correlated with the GPS location. For a survey of a thousand kilometre pipeline, this typically would require eight full time staff for eight months, plus an admin staff member and supervisor for several days per week to check and confirm the data. For a recent large scale depth of cover survey for a gas transmission pipeline running from Windsor to Montreal, about a thousand kilometres, PVS deployed Utility Mapper together with Vivax-Metrotech electromagnetic locators and Trimble Catalyst, which enabled survey-grade positioning with an Android smartphone.  Utility Mapper automated most of the capture and processing - integrating the GPS and electromagnetic locate information and wirelessly synchronizing the location and depth of cover information directly to ESRI’s ArcGIS Online for quality assurance and quality control. This reduced staffing requirements from 64 to 47 person-months. Furthermore, the Utility Mapper solution reduced the data review time for a supervisor (prior to sending the data to the client) to one hour per week .

Summary

Terraflow's Utility Mapper is an important advancement.  It adds significant value to the industry standard EM underground utility locating tool by enabling the cost-effective digital capture of 3D location data captured by EM devices and RTK GNSS and streaming this data to users' CAD and GIS applications.

ESRI UC: 3D and other themes of this year's conference

According to Jack Dangermond who led the plenary sessions on the first day,  this year in San Diego 12000 people from 130 countries were in San Diego for the 33rd Esri International User Conference.

I gravitated primarily to the utility sessions.  According to Bill Meehan, who is responsible for ESRI's utility business, there were 750 utility folks at this year's UC.

GIS is everywhere

GIS is being used in diveerse areas including environmental monitoring, climate change, permafrost melting, sea level rise, agriculture, water resources, aquaculture, pollution remediation, energy resources management, land information systems, urban design, utility network management, wasteshed modeling, utility pole inspection, sewer rehabilitation, port management, indoor GIS, iInsurance risk management, health-related mapping, law enforcement, mapping where space debris might fall, flood risk mapping, storm tracking and damage assessment,  geodesign, crowdsourcing for collecting a variety of data, interoperabilty, citizen engagement, government transparency, cartography, geology and mining, automatic generalization of different scale maps, story maps, portals for infrastructure, and open data, to mention just a few of the things that were singled out in the plenary session. 

Some of the areas that were singled out  for special recognition were

• Special Achievements in GIS award went to the Dartmouth Atlas for showing geographically how outcomes and the cost of medical procedures varies depending on location across the U.S. 
• The Hong Kong Lands Department received the Enterprise GIS award for their land management system which they started in the 1990's and is now in its second generation. 
• Tbe President's Award went to Direct Relief which distributes different types of equipment to people around the globe in need with 99% efficiency according to Forbes.

Themes

The theme of this UC was GIS transforming our world, not only changing the physical world as we know it, but also changing our perception of the world and how GIS has and will fundamentally help us collectively build a better future.  A classic example that has impacted everyone on the planet is GPS or more generally GNSS.   As a number of speakers including Dangermond said we are never lost any more and that is fundamental change to how we perceive the world.

Some of the major themes that emerged at this UC are common to many software and service providers in the IT sector, but some of these are new or unique to ESRI. 

Climate change

We are facing serious global challenges, in particular climate change, which both at the Geodesign Summit and at this UC conference appears to be a personal challenge for Dangermond who sees GIS as contributing in major way in creating a more sustainable future.  He sees GIS helping to change how we think and act and how we do basic things such as design. 

GIS across the organization

A theme that was pervasive throughout the conference including in the utility sessions was the need to make GIS pervasive across the organization because 90% of the data that organizations are collecting and managing has location.  This was also true in utility sessions I attended, where the thrust is to expand the use of GIS outside of the traditional area of records management into other departments.

Vertical industry "templates"

In the utility and telecommunications sectors there is a major effort underway to create preconfigured vertical "templates" for specific vertical industries.  In the utility sector one was announced yesterday for the electric power Industry, and others are underway for gas and telecommunications.  These templates are a collection of data and tools for vertical industries and are available as open source on Github.

Web

The web was pervasive, It was hard to find a session where the theme that desktop GIS is being transformed into web GIS did not come up.  It doesn't mean that the desktop is going to go away in the near future, but sharing data, applications, and apps is being recognized as a common goal and web GIS makes this very easy to do.  This includes web portals for accessing enterprise data over the web.

Cloud

A lot of the traditonal desktop GIS capabilities are now being made available in the cloud so that all you need is an iOS or Android device or a web browser.  This is also available to large enterprises who need to keep their data on their own servers for security reasons.  Cindy Salas from Centerpoint Energy demonstrated ESRI's cloud capability but running "on premise" completely wthin Centerpoint's fire wall. 

Integration

Being able to integrate location and the spatial dimension into enterprise applications such as ERP (SAP), business intelliegence (IBM Cognos), Salesforce, MS Dynamics, and Excel spreadsheets is a new focus for ESRI.   They have already created plugins for several well-known applications.

Mobile devices

In the past when you talked about mobile in the context of utilities, you were normally dealing with Panasonic Toughbooks which could cost up to $6 000 each.  At this conference just about every application was able to run on a variety of low cost mobile devices including iOS and Android mobile devices.  One of the sessions I attended was about the National Broadband rollout in Australia where the 500 field folks who are doing 300 000 pit inspections were all equipped with iPads, which you might have thought were too fragile for pretty rough field work..  This project has been underway for something like 18 months and todate only two of the iPads have been damaged.

Big data

Spatial data volumes whether from sensors at utilities, from LIDAR scanning of transmission lines, or from earth obsevration satellited are now reckoned in terabytes.  There were sessions on spatially enabling Hadoop, SAP HANA and other big data management systems for managing these data volumes.

Real-time

In the utility track managing real-tme data from sensors on transformers, smart meters, phasors, and other devices and enabling near real-time decision making was a common theme.  A general event processing capability that can be triggered by rules has been added to the quiver to make this possible for any data stream. 

In the area of imagery, Digital Globe imagery can be accessed with 4-5 hours of acquisition.

Analytics

Big data and real-time means that we need to be able to understand what these huge volumes of data are telling us and then enable decision-making, automated or human mediated, as rapidly as possible.  If a transformer is running hot, we can't wait a week until the next service check to reconfigure the network because by then we may have reduced it projected lifetime by 10%.  There were "location analytics" sessions in most of the vertical industry sessions including utilities.

Content

Software companies like ESRI and Hexagon are now in the content busines.  And with the data beng made available in the cloud, you can access it through web GIS or desktop GIS. The data includes traditional topographic maps, digital terrain models, imagery from earth observation satellites from Digital Globe and others some of it near real-time (fr example, from Digital Globe within 4-5 hours of acqusition), demographics, and others.

Dangermond's term for this is the "living atlas" because this body of content is intended to be dynamic, real-rime and comprehensive.  The goal seems to be all the data you would need to manage the planet.

Some of the content explicitly mentioned includes

• basemaps
• community. content
• imagery
• 30 cm coverage of the US
• 60 cm coverage of Western Europe
• imagery for the middle east and asia in total covering 2/3 of the world
• health
• lifestyle
• economics
• demographics, at the zip code, county, state, and national levels
• scientific maps including environmental, energy, infrastructure, and terrain models
• landscape layers

This data is accessible to web and desktop applications and makes it very easy to do standard spatial analytical things like multi-criteria suitability or site selection mapping.  Users can also publish their own data to the cloud.

Education

This has always been a strong focus area for ESRI and from Dangermond's perspective with the challenges the world is facing, GIS professionals will be even more critical in the future.  There is focus on advancing spatial literacy to a broader audience than in the past.

LiDAR

Laser scanning is a major source of important data for the construction industry (right of way determination and construction progress monitoring), utilities (transmission vegetation management), and other industries.  ESRI has made this a focus area and there were a number of sessions devoted to managing and analyzing point cloud data.

3D

Dangermond specifically singled out 3D as a major capability area for ESRI.  There are specialized 3D products, but 3D is also being built into ESRI main stream GIS and spatial analytical products.

3D city models

At the plenary there was a demonstration of how to quickly build a 3D city model from 2D data, including extruding buildings from a 3D building footprint and height, adding textures depending on building types, and then tools for analysis such as zoning and shadow analysis and visualization including flythroughs.  These models are sharable on the cloud as web scenes requiring only a web browser (with no plugins) or iOS mobile devices.

Geodesign

At the UC ESRI is demonstrating a prototype of a product designed specifically for geodesign.

Open data

Dangermond specifically mentioned the newest national open data initiative in Peru. 

Crowdsourcing/volunteered data

There are sessions specifically on editing and manipulating OpenStreetMap data.

Mobile geospatial solution improves data quality at water utility

At the HxGN conference in Las Vegas Chris Bodely of Anglian Water which provides water and wastewater services to 6.3 million customers in the southeast of England gave an overview of GIS at the water utilty and in particular their implementation of a mobile GIS. This is relevant for any utility that is still sending paper maps and work orders to the field.

Background

In 1992 Anglian began digitizing their paper maps and drawings.  In 1995 they implemented an AM/FM/GIS solution using Intergraph FRAMME.  Then in 2003/2004 they developed a new GIS strategy, a key element of which was a mobile GIS solution.  At that time field work orders were paper and the process to develop them was manual.  Because it was a  manual, paper-based process, it was difficult to process feedback from the field staff when something was changed or when errors in the records were documented by field staff.  The result was that forms returned from the field were often incomplete and unreliable.  This is a problem that most utilities relying on a paper-based process are experiencing.

Mobile GIS

In 2006 Anglian provided Panasonic Toughbooks to their field staff with a desktop mapping application and facilities data.  The application enables field staff to see as-built maps and other information for Anglian's water and wastewater networks.  In 2012 the application was replaced with an integrated application that includes accces to SAP. 

A key feature of the mobile GIS is that it also enables field staff to record errors and changes as redline directly on the digital map in the field as part of a two step automated process.  The redline is which is then forwarded electroniaclly to Anglian's Central Mapping group.  After reviewing the proposed changes Central Mapping update the faciltiies database with the new information.  Providing direct access to the database has empowered the field staff, who are now taking greater responsibility for the reliability of the data in the facilities database. 

Benefits

Anglian found that the mobile GIS resulted in producitivity improvments across most process areas of 2-5%.  But the biggest benefit was in the area of data quality.   Form completion and quality improved from the low 60s% to the high 90s% over a period of 6 months. This resulted in improved regulatory and statutory data capture, for example, to comply with the Traffic Management Act (TMA), and directly impacts the bottom line, because it means fewer fines for non-compliance with the TMA.  Another major benefit was that empowerment resulted in a happier field workforce

The remaining major challenge is to provide the updated facilities information to field staff automatically.  Currently it is a manual process requring ditribution of the updated facilities database via DVD.

AU: Bringing the field into the office

One of the most important goals of utilities in this time of aging and shrinking workforces is improving the productivity of operations staff.  Many utilities are finding that many tasks that used to require sending staff into the field can now be done much more efficiently in the office.  I've blogged previously about obligue high resolution imagery from Pictometry, which has just released a plug-in for AutoCAD Map 3D.  

At the Utility Symposium at Autodesk University last week, David Petterson from Pictometry outlined some of the uses that utilities and local giovernments are finding for Pictometry's high resolution oblique imagery.   High resolution oblique imagery not only can provide orthophotography with resolution down to 4 inches (10 cm), but allows quantitative measurement in three dimensions.

Utility applications of oblique imagery

Using Pictometry imagery can provide greater coverage of right of ways and allows the utility to validate encroachments.  In many parts of the U.S. Pictometry provides historical data which can be used to pinpoint in time when the encroachment first occurred.  Tree heights and offsets to lines can also be measured.  Parcel owners can be identified and access points reviewed.

After storm events, Pictometry will refly locations that have suffered damage to provide a quick response to storm events.

Permitting is another important application area of oblique imagery.  Identifying, validating, and preparing preliminary designs are all areas where the imagery can help.

But most importantly, high resolution oblique imagery can reduce the number of field visits (truck rolls) and reduce the time required per visit for visits that cannot be avoided.  Oblique imagery means that everything above ground can be measured.  The lighting is always good and it's available 24/7.

Fukushima Daiichi: Radiation dosage readings at plant perimeter continue to fall

JAIF has reported the latest radiation readings at plant perimeter as of 15:00, Apr. 15th (02:00:00 Friday April 15 ET)

• 530μSv/h at the south side of the office building
• 70μSv/h at the Main gatt
• 28μSv/h at the West gate

Previous readings

15:00, Apr. 7th (02:00 am Thursday April 7 ET).

• 0.67 mSv/h (670 μSv/h) at the south side of the office building
• 108 μSv/h at the Main gate
• 43 μSv/h at the West gate

09:00, Apr. 3rd (20:00 Saturday April 2 ET)

• 0.83 mSv/h (830 μSv/h) at the south side of the office building
• 127 μSv/h at the Main gate
• 59 μSv/h at the West gate

21:00, Mar. 31st (08:00 am Thursday March 31 ET)

• 0.94 mSv/h (940 μSv/h) at the south side of the office builiding
• 148 μSv/h at the Main gate
• 70 μSv/h at the West gate

21:00, Mar. 30th (8:00 am Wednesday March 30 ET)

• 1.00 mSv/h (1000 μSv/h) at the south side of the office building
• 156 μSv/h at the main gate
• 72 μSv/h at the west gate

15:00, Mar. 30th (02:00 am Wednesday March 30 ET)

• 1.05mSv/h (1050 μSv/h) at the south side of the office building
• 163 μSv/h at the main gate
• 75 μSv/h at the west gate

09:00, Mar. 30th  (20:00:00 Tuesday March 29 ET)

• 1.05 mSv/h (1050 μSv/h) at the south side of the office building
• 169 μSv/h at the Main gate
• 78 μSv/h at the West gate

Earlier readings at the West Gate

• 120.0 μSv/h at 21:30, Mar. 28 (08:30 am Monday March 28 ET)
• 122.8 μSv/h at 15:30, Mar. 28 (02:30 am Monday March 28 ET)
• 132.5 μSv/h at the West gate at 16:00, Mar. 27 (03:00 am Sunday March 27 ET)

Fukushima Daiichi: JAIF reports perimeter radiation measurement stable

The latest JAIF report as of 10 am March 19 (9 pm March 18 ET) says that

Units 1,2,3 Reactors

Status of fuel is reported to be "Fuel exposed partially or fully".

Unit 1,2,3 Containment

Unit 1 containment is reported as Not Damaged

Unit 2 containment is reported as Damage Suspected

Unit 3 containment is reported as Might be ”Not damaged”

Units 5, 6

Open a vent hole on the rooftop for avoiding hydrogen explosion

Off-site power

External power supply of Unit-1 and 2 are scheduled to be
connected until March 19. Unit 3 to 6 are scheduled to be connected until March 20.

Radiation reading at the plant east gate

  304 μSv/h at 03：30, Mar. 19 (02:30:00 p.m. Friday March 18 ET)

  287 μSv/h at 12:00, Mar. 18 (23:00:00 Thursday March 17 ET)

Previously reported readings at the plant perimeter are

  271 μSv/hr at 7:30 AM Mar 18 (06:30 p.m. Thursday March 17 ET)

  289 μSv/hr at 11pm Mar 17 (10:00:00 a.m. Thursday March 17 ET)

  292 μSv/hr at 8:40pm Mar 17 (07:40:00 a.m. Thursday March 17 ET)

  646.2μSv/h at 11:10am Mar. 17 (10:10 p.m. Wednesday March 16 ET)

 1472μSv/h at 16:20, Mar 16 (03:20 am Wednesday March 16 ET)

Fukushima Daiichi Nuclear Reactors Status from JAIF: Damage to Unit 3 Containment Suspected

JAIF Report on status of Fukushima Daiichi as of 12:30 March 16 (11:30:00 p.m. Tuesday March 15 ET)

Unit 3

White smoke was seen rising from the vicinity of Unit-3 at around 8:30, Mar. 16. Damage to the containment vessel of the unit is suspected.  This is a special concern, because this reactor uses MOX (mix of uranium and plutonium oxides) as fuel.

Unit 4

The level of water in the Spent Fuel Pool remains low, and damage to fuel rods is suspected.  It is reported that they are "Preparing Water Injection".

Units 5 and 6

Spent fuel pool temperatures continue to increase

Unit 2

Building is reported to be slightly damaged

Radiation measurement at the plant perimeter

3391μSv/h at 11:00, Mar. 16 (10:00:00 p.m. Tuesday March 15 ET)

Argentine Electric Power Industry: Data Quality and Reliability

I have blogged previously about the importance of data quality in the context of the smart grid and that the electric power regulator ANEEL in Brazil is getting serious about data quality.

In Argentina the power industry regulators have mandated a high level of data quality since the 1990's.  Beginning with the Privatization Program in 1990 most state owned companies were transferred to the private sector including state telephone company, airlines, railroads, and electric power production and distribution. There are about 20 electric  power distribution companies in Argentina, all of which were privatized at this time.  After privatization, power distribution companies in Argentina were expected to be profitable.  The national  power Industry regulator Enre and the regulators in each of the states are responsible for overseeing the power industry.  The regulators focussed on two areas, reliability and data quality, and these two factors effectively determine the rate of profitability that the power distribution companies can achieve.

The regulators require frequent reports from the power utilities, for example, equipment utilization reports are required daily, outage reports are required monthly, and so on.  The regulators have their own call centers so that consumers unhappy with some aspect of electric power service can call the regulator to complain.  Also the regulator does spot checks of electric service and conducts audits.  The acceptable threshold for data quality is 98%.  Companies not meeting this level of data quality may face fines of up to millions of US dollars and reduced rates.

I would like to thanks Alejandro Spiritoso for enlightening me about the importance of data quality in the Argentine power industry.

GITA Ontario: Mobile Solutions for Road Asset Management

I was at GITA Ontario Fall Forum yesterday near Toronto, which is an event I look forward to every year. This year was no exception, there was some exceptionally good content especially in the electric power utility area.

Every year I have a chance to catch up with some folks that are developing and deploying geospatial applications in Ontario, a province with about 10 million people.  This time I spend some time chatting with Scott MacPhee, Web Applications Engineer at Hunter GIS, which develops web-based geospatial applications for the municipal, utility, telco and resource management sectors using Autodesk MapGuide 6.5, MapGuide Open Source,  and MapGuide Enterprise.  They have developed several applications for local governments including tax parcel assessment, municipal permit tracking, development tracking, water and sewer asset management, gas infrastructure management, electrical infrastructure management, and environmental and natural resource management. One of the applications they have developed is a Road Asset Management System (RAMS), a comprehensive, web-based road asset management systems that allows municipal road departments to manage inventories and inspections, maintain an asset database including roads, bridges, culverts, signs, and lights,and help with prioritization and budgeting.  RAMS integrates with the parcel tax assessment database, automatically computes ratings, needs and resurfacing/rehabilitation costs, and allows you query the roads database for spatial analysis of road conditions.  RAMS has been in use for some time by local governments in Ontario.

I have blogged about the poor quality of network facility data in utilities and telecom, how critical up to date reliable network facility data is becoming, and how essential it is to use technology to empower field staff  to be full participants in improving the quality of asset data. What Hunter GIS has done is develop a simple online web-based application that can be used by field staff to report to report the condition of road facilities. Scott showed me a mobile extension to RAMS that provides field access to RAMS databases via a Blackberry or other wireless handheld device.  This means that field staff can view and select assets graphically, list all the assets on a street, view inspection reports on road assets, create a new inspection report, for example, report a pot hole, and attach a photograph all from a Blackberry.  Impressive. I fully expect that in the future they are going to find rapidly increasing use of handheld devices for these types of applications and proportionately less use of desktops.  Hunter GIS also have mobile solutions for municipal permit tracking and service request tracking.

Smart Grid Communications Networks

Utilities implementing a smart grid need to be able to link customers, their equipment and data centers and often external organizations on a bidirectional communications network.  Some utilities are looking at using telephone company communications assets to do this, but many are looking at other alternatives including laying their own fibre network.  In the case of electric power utilities serving largely rural areas technologies being considered include broadband over powerlines and wireless technology. 

A wireless technology that I frequently hear about from utilities is WiMAX.  A major manufacturer of power utility equipment manufactures WiMAX-based smart meters. Among the companies that have been looking at or testing WiMAX technology include Australia-based SP AusNet, Energy Australia, American Electric Power, Consumers Energy,  Centerpoint, National Grid, San Diego Gas & Electric (SDG&E), Southern California Edison (SCE), and Hydro One.  A major reason for using WiMAX for the smart grid is high bandwidth which means that WiMAX could be used with major assets like substations to collect large volumes of data from video monitoring units, phasor units which collect information like voltage, current and frequency in real time, and other smart devices. WiMAX could also be used to support mobile work force applications including mapping, digital imagery and video.  WiMAX is an open standard, which differentiates it from proprietary smart grid network technology often found in North America and which the European smart grid initiative is trying to avoid. (Image Hydro One)