Between the Poles

Applying machine learning to identifying housing starts from satellite imagery

Deep learning is increasingly finding practical applications using geospatial data, specifically, satellite imagery. At the GeoIgnite conference in Ottawa, Nicolas Martinez of Statistics Canada described a project scheduled to kick-off in July to use machine learning to identify housing starts across Canada.  The goal of the project is to improve the accuracy and coverage of housing starts survey, specifically to fill data gaps for smaller remote and indiginous communities which are generally excluded from the current survey program. 

Canada Mortgage and Housing Corporation (CMHC) currently employs a significant number of people at their head office in addition to field agents across the country who physically confirm residential starts and completions.  The objective of the project is to apply convolutional neural network (CNN) machine learning to satellite imagery to extract information about housing starts and completions. The imagery will be preprocessed to prepare it to be used in developing a machine learning processing model.  The largest challenge will be developing training and testing imagery data sets.

The original application of deep learning to photo imagery was by Geoffrey Hinton at Google who used it to distinguish dogs and other objects in photos that people uploaded.  More recent applications of satellite imagery targeted landuse and agricultural identification. For example, remote sensed imagery from a satellite can be used to create a model that will differentiate between corn and potatoes, using factors that can be calculated from satellite imagery such as normalized difference vegetation index (NDVI) (min, max, mean), texture (min, max, mean), vegetation height, and geometric factors such as orientation.  I blogged about how open source code and publicly available training data has been applied to track deforestation and reforestation in Mato Grosso, a state in the central amazon region of Brazil. A study in the Netherlands used CNN to identify blocked waterways using overflight imagery with a success rate of 97 %.

An important application of satellite imagery is identifying built structures. I have blogged about how satellite imagery has been used to identify buildings and transportation networks by applying a neural network model together with OpenStreetMap layers and high resolution Worldview multispectral imagery.  NVIDIA has demonstrated the ability to automate detection of many road networks using deep learning algorithms and multi-spectral high resolution imagery.  Another application of CNN and imagery in the Netherlands was able to differentiate residential roofs with dormer windows from those without.

The essential key to effective application of deep learning is good training and test data - ground-truthed data that involves, for example, someone on the ground identifying whether the fields seen in the imagery are corn or potatoes or something else. There are  publicly available training data sets that can be used to train satellite imagery CNN models. For example, I blogged about the System for Terrrestrial Ecosystem Parameterization (STEP) dataset which has 2000 manually labeled sites covering 17 different land cover types scattered across all continents. A large publicly available training dataset derived from Sentinel 2 imagery contains 30,000 polygons of land use training data for ten classes of land use: annual crop, forest, herbaceous vegetation, highway, industrial, pastures, permanent crop, residential, river and sea or lake.

The Statistics Canada project is scheduled to begin in July with initial work on a proof of concept and CNN testing to begin in August and September.  The communities chosen for the initial phase are  Kitchener-Cambridge-Waterloo, Red Deer, and Iqaliut.  The first phase is scheduled to be completed by winter 2019.  The project plans to use Tensorflow and Keras, open source libraries and a python framework for applying machine learning.  Artificial intelligence (AI) has often suffered from inflated expectations, complex code, heavy processing requirements and not very practical applications. There is a growing body of practical applications in the geospatial domain that shows that as a result of the immense processing power available in the cloud machine learning is able to generate fairly easily practical and useful results.  As Chris Holmes pointed out in his talk at FOSS4GNA, the challenge for Statistics Canda will be to develop reliable and publicly available training and test datasets to enable deep learning models to be created for a broader range of applications using the huge volumes of satellite imagery and other geospatial data that are now available.

Deep learning is increasingly finding practical applications using geospatial data, specifically, satellite imagery. At the GeoIgnite conference in Ottawa, Nicolas Martinez of Statistics Canada described a project scheduled to kick-off in July to use machine learning to identify housing starts across Canada.  The goal of the project is to improve the accuracy and coverage of housing starts survey, specifically to fill data gaps for smaller remote and indiginous communities which are generally excluded from the current survey program. 

Canada Mortgage and Housing Corporation (CMHC) currently employs a significant number of people at their head office in addition to field agents across the country who physically confirm residential starts and completions.  The objective of the project is to apply convolutional neural network (CNN) machine learning to satellite imagery to extract information about housing starts and completions. The imagery will be preprocessed to prepare it to be used in developing a machine learning processing model.  The largest challenge will be developing training and testing imagery data sets.

The original application of deep learning to photo imagery was by Geoffrey Hinton at Google who used it to distinguish dogs and other objects in photos that people uploaded.  More recent applications of satellite imagery targeted landuse and agricultural identification. For example, remote sensed imagery from a satellite can be used to create a model that will differentiate between corn and potatoes, using factors that can be calculated from satellite imagery such as normalized difference vegetation index (NDVI) (min, max, mean), texture (min, max, mean), vegetation height, and geometric factors such as orientation.  I blogged about how open source code and publicly available training data has been applied to track deforestation and reforestation in Mato Grosso, a state in the central amazon region of Brazil. A study in the Netherlands used CNN to identify blocked waterways using overflight imagery with a success rate of 97 %.

An important application of satellite imagery is identifying built structures. I have blogged about how satellite imagery has been used to identify buildings and transportation networks by applying a neural network model together with OpenStreetMap layers and high resolution Worldview multispectral imagery.  NVIDIA has demonstrated the ability to automate detection of many road networks using deep learning algorithms and multi-spectral high resolution imagery.  Another application of CNN and imagery in the Netherlands was able to differentiate residential roofs with dormer windows from those without.

The essential key to effective application of deep learning is good training and test data - ground-truthed data that involves, for example, someone on the ground identifying whether the fields seen in the imagery are corn or potatoes or something else. There are  publicly available training data sets that can be used to train satellite imagery CNN models. For example, I blogged about the System for Terrrestrial Ecosystem Parameterization (STEP) dataset which has 2000 manually labeled sites covering 17 different land cover types scattered across all continents. A large publicly available training dataset derived from Sentinel 2 imagery contains 30,000 polygons of land use training data for ten classes of land use: annual crop, forest, herbaceous vegetation, highway, industrial, pastures, permanent crop, residential, river and sea or lake.

The Statistics Canada project is scheduled to begin in July with initial work on a proof of concept and CNN testing to begin in August and September.  The communities chosen for the initial phase are  Kitchener-Cambridge-Waterloo, Red Deer, and Iqaliut.  The first phase is scheduled to be completed by winter 2019.  The project plans to use Tensorflow and Keras, open source libraries and a python framework for applying machine learning.  Artificial intelligence (AI) has often suffered from inflated expectations, complex code, heavy processing requirements and not very practical applications. There is a growing body of practical applications in the geospatial domain that shows that as a result of the immense processing power available in the cloud machine learning is able to generate fairly easily practical and useful results.  As Chris Holmes pointed out in his talk at FOSS4GNA, the challenge for Statistics Canda will be to develop reliable and publicly available training and test datasets to enable deep learning models to be created for a broader range of applications using the huge volumes of satellite imagery and other geospatial data that are now available.

A killer application for satellite imagery collected daily for the entire earth

At the GeoIgnite conference in Ottawa, Wade Larsen of Urthecast described what may be a killer application for satellite imagery captured daily. Farmers in the US over-irrigate by 25% per year. A recent study has demonstrated that monitoring vegetation growth on a daily basis can save farmers $50 000 in water per year. With 260,000 pivots operating in the US, this amounts to billions of dollars of savings every year.

Urthecast has been known for having two cameras (Iris and Theia) on the International Space Station and two Deimos earth observation satellites. Last year Urthecast announced the acquisition of Geosys Technology, a software company that processes earth observation imagery for agricultural applications.  Their latest initiative is called UrtheDaily with the objective of capturing nine bands of imagery from six satellites covering the Earth's entire landmass every day. The nine bands are identical to Sentinel-2's visual and near-infrared bands. The cameras are cross-calibrated with Sentinel-2 to assure data compatibility. The six cameras enable a daily revisit for higher latitudes and somewhat less for tropical areas. It captures imagery at the same time, from the same altitude every day.  About 150 terabytes will be acquired and downloaded every day. The imagery has five meter resolution (GSD) and represents scientific grade data quality that will be available on the cloud for user access within hours of acquisition. UrtheDaily is designed to support machine-learning and AI ready geo-analytics applications on a global scale. The UrtheDaily platform will provide the ability to discover data, run user algorithms, and deliver data to customers.

UrtheCast reports that it has already signed contracts primarily in the agriculture sector. An example of a potential killer app in agriculture is monitoring irrigation. Farmers in the U.S. overwater by 25 %. A four year in Texas by HydroBio which assessed the value of frequent monitoring of plant health and growth using satellite data concluded that to be effective this required high-frequency (daily) monitoring with highly calibrated sensors. With its resolution UrtheDaily will be able to monitor vegetation for individual pivots on a daily basis in order to optimize water utilization. It is estimated that this could save farmers $50,000 per pivot per year. (A pivot typically irrigates 120 acres.) There are 260,000 pivot irrigation systems in the U.S. which translates into the potential to save farmers billions of dollars per year.

To date UrtheCast reports that it has signed $350 million worth of contracts in the U.S., Russia, India and other countries. It plans to launch six satellites in 2021 and become operational in 2022.

Chattanooga quantifies major benefits from smart grid investment

Chattanooga's power company Electric Power Board (EPB) is one of the most advanced with respect to adopting smart grid technology in North America. At HXGNLive in Las Vegas, Ken Jones reported quantified benefits that EPB has found from its investment in smart grid technology. These include 65% reduction in customer outages and 52% reduction in outage minutes and savings of $23 million in restoring outages after a recent severe storm.

About five years ago I first heard about the remarkable transformation of the power grid in the Chattanooga, Tennessee, a city with a population of about 180,000. With the help of the Department of Energy and other partners the local power company Electric Power Board (EPB) saw the smart grid as not only benefiting the utility and improving the quality of life of its cusomers, but also as a driver of economic development. Early on EPB decided to deploy one of the country’s highest capacity fiber-optic networks to improve power quality, reliability, customer service and energy efficiency. Chattanooga is one of the few municipalities anywhere with 10 Gbit/sec internet speeds for both urban and rural customers. Currently over 50% of EPB customers subscribe to the fiber service.  EPB has estimated that the incremental economic and social benefits of the high capacity backbone for Chattanooga lie between $865.3 million to $1.3 billion. Furthermore, somewhere between 2,832 and 5,228 jobs have been created linked to EPB's infrastructure investment.

In addition to the fiber backbone, EPB has installed 1400 automatic switches, an AMI management system, a distribution management system (DMS), and support for demand management (DM) and distributed energy (DRE).

At HxGNLive in Las Vegas Ken Jones of EPB presented some of the very impressive quantified benefits that EPB has realized as a result of these investments in fiber and smart grid technology.

Reliability

Reduction in customer outages	65%
Reduction in outage minutes	52%

Annual operating savings

Meter reading	$2.0 million
Field services	$0.7 million
Demand charges	$3.0 million

Environmental benefits (CO2 emissions reduction)

Reduced truck mileage	400 tons/year
Demand management kWh reduction	1000 tons/year
Power factor improvement	2000 tons/year

In addition EPB has developed a solar farm of 4,400 panels generating about 2 million kWh/year. Anyone can buy into this starting at just $5/month. Ken gave examples of local firms that completely offset their power usage with solar power.

One of the interesting steps that EPB has introduced to help customers monitor their power usage is a mobile app MyEPB. The app provides real-time energy usage and usage comparison by day, week, and month, real-time outage reporting for the EPB service territory, support for customers to report outages, and billing alerts, for example, notify me when my monthly bill reaches $50. To date there have been 30,000 downloads of MyEPB.

Progress of outage restoration after a recent storm showing areas automatically restored (violet) and manually restored (green)

An area where the investment in smart grid technology has realized a major savings is resilience. Ken presented a map showing the areas in the EPB service territory where power was automatically restored by the automated distribution switches after a recent severe summer storm. The outage and restoration statistics were equally impressive and showed that automation resulted in an estimated savings of $23 million for this storm.

	W/o automation	W/ automation	Difference	Improvement(%)
Customers with an outage	72,622	32,043	40,579	55.88%
Cost of all outages	$69.3 million	$46.1 million	$23.2 million	33.48 %
Outage minutes	16,986,240	12,059,524	4,926,716	29.00 %

One of the important steps that the EPB has taken with its partners is to implement a microgrid testbed at the Chattanooga airport comprising a solar farm, batteries and supporting infrastructure.  This is one of the few airports anywhere that is able function completely off the grid.

These quantified benefits provide tangible evidence of the benefits of smart grid technology deployment.  I would recommend that any power utility operating in a small to medium-sized city seriously consider the Chattanooga experience.

Technology breakthrough: lowering the bar to underground GPR surveys for land surveyors

Traditional output from a GPR scanner showing hyperbolas reflected from underground utilities

The most exciting technology announcement at HXGNLive is a new ground penetrating radar (GPR) solution that dramatically lowers the bar for GPR scanning, enabling surveyors and other professionals who have hesitated to get into this lucrative market because of the complexity of interpreting GPR scans to take the leap.  Before this announcement today by Agata Fischer of Leica Geosystems, non-geotechnical professionals including surveyors were put off by GPR scans consisting of images with hyperbolas showing the reflections of RF waves from underground objects.  This left GPR pretty much to highly trained geotechnical and other underground engineering professionals.

As I discovered at a recent talk I gave to the Alberta Land Surveyors Association, there is a lot interest among the surveying community to expand their professional offerings beyond the traditional above ground survey to include below ground surveys.  Two years ago the one-button BLK360 opened up engineering grade laser scanning to a much broader professional audience.  Leica Geosystems is betting that with a similar focus on simplicity the DSX device and DXplore software announced today, is going to open GPR scanning to a much broader professional community, in particular, surveyors. 

Output from the DSX with DXplore software showing tomographic image of detected underground utilities

The key features of the DSX GPR device and DXplore software announced today are simplicity of use and interpretation, detection results that can be relied on,  immediate 2D and 3D undergound utility maps, and interoperability with CAD and other engineering products.  The DSX workflow is very simple: import available as-builts in the area as DXF or other CAD format files, define a grid for the area to be surveyed, walk the grid with the DSX device with real-time fedback on the DSX's display, show the tomographic display (no hyperbolas !) of what was detected, manually identify suspected utilities on the display,  the software then processes the captured GPR images to confirm the utility pipe or cable, and export a 2D or 3D vector file in a CAD compatible file format.  Very simple.

But too simple for experienced GPR practitioners. I have talked to very experienced GPR people who don't find the DSX interesting for them.  As GPR experts they are trained to work with B-scan/radargram data representation and interpretation, thus expert systems such as Leica DS2000 or IDS GeoRadar Stream are more suitable. But for people who don't have GPR experience, believe that underground surveying represents a significant business opportunity, and already use or know Leica Geosystems total stations or other survey equipment this is a way to get their toes in the water.  Furthermore I fully expect that Leica will not stop here but will continue to develop easy to use GPR software for detecting and mapping underground infrastructure including support for the multi-channel Leica IDS Georadar arrays that are being applied to very sophisticated mapping of underground infrastructure.

Proven processes for reducing underground utility damage

Are there things that an owner, agency or municipal government responsible for an airport, industrial campus, or town can do to reduce the risk of underground utility damage, the associated disruptions to operations and business and the attendant danger for construction workers and the public ?  At this year's Geo Business in London I had a chance to chat with Andy Rhoades, who was Head of Service Protection Knowledge & Information Management Technical Standards and Assurance at Heathrow Airports from 2001 to 2017.  During the first ten years of Andy's tenure at Heathrow the annual number of services strikes (underground utility damage) at Heathrow declined by a factor of six and has continued to decline even as the amount of construction has grown.

Airports are like cities except that they have a greater density of underground utilities and more types of underground equipment; communications cables, aeronautical ground lighting cables, gas mains, low voltage electric power cables, high voltage electric power cables, storm water mains, sanitary waste water mains, potable water mains, grey water mains, fuel mains, fire fighting water mains, and a variety of underground structures. In addition striking a main in an airport carries with it a larger risk than in an urban area because of the incoming and outgoing aircraft and the large number of people in a concentrated area.  I have previously blogged about what Andy was doing at Heathrow based on his talk at Geospatial World Forum 2014.

Andy introduced a seven step process, from early design to handover, for excavations at Heathrow that continues to be followed at Heathrow.   One of important features is the concept of exclusion zones.  Based on PAS 128 quality levels, it details what type of excavation can be safety used within 0 to 3 meters of a utility.  These include hand-digging (foot pressure only), powered hand tools, vacuum or hydraulic extraction, and powered excavation. 

Another requirements is that all people involved in detecting and mapping underground infrastructure must be trained.  Heathrow has helped to develop National Vocational Qualifications (NVQs) relevant to underground asset management. National Vocational Qualifications (NVQs) are work based awards in England, Wales and Northern Ireland that are achieved through assessment and training. To achieve an NVQ, candidates must prove that they have the competence to carry out their job to the required standard. NVQs are based on National Occupational Standards that describe the 'competencies' expected in any given job. In addition in the UK there is a formal structure Qualifications and Credit Framework (QCF).  Training levels that Andy mentions are QCF Utility Mapping Technician, QCF Level 3 Utility Mapping Surveyor, and QCF Level 5 Senior Utility Mapping Surveyor.

Asset damage compared to the amount of construction activity at Heathrow.  Graph also shows number of incidents halved 2015-2016 when using predictive algorithms.

Based on his experience at Heathrow, for his consulting practice Andy has produced two manuals that document his over 15 years of experience at Heathrow, and 9 years at Thames Water, Londons Water company.  Services and Buried Infrastructure Protection Standard is a recommended standard for owners of large number of underground assets such as airports, towns, universities, industrial campuses, and other facilities.  The second is a Clients Guide to Utility Survey that gives owners/clients a guide to what they should require from an underground utility survey.  Between them these manuals contain a wealth of practical information for owners/clients who want to reduce the risk of underground utility damage during excavation.  They also contain examples of what has happened if the guidelines are not followed, for example, air traffic control tower loss of communications, when a backhoe operator decided to take out a big concrete slab which happened to contain fiber cables carrying the airport's communications.  He also provides best practices for surveys in rural areas, suburban areas, busy urban areas and congested city areas and airports. 

One of the very interesting sections in the Guide covers different technologies for underground utility detection including GPR, EML (Cat and Genny), Acoustic pipe locator, using a dye to trace drains, earth resistance, gyroscopic pipe locator, infrared imaging, magnetometry, metal detectors, RFID detectors, vibration acoustic.  There are some good rules of thumb, for example, underground utilities can be detected through concrete, utilities can't be detected through salty water (w/ electrolytes) with GPR, but can through fresh water (w/o electrolytes), and underground detection generally is not very accurate for depth without expert processing of raw data.  That radio waves used in GPR travel at the speed of light through air, two thirds the speed of light in ground (average depending on ground make up), and one third the speed of light in water, can result in serious errors in reported depths of detected infrastructure.

He also discusses how the commonly used techniques GPR and EML can generate unreliable data including ghost utilities.  EML in particular has problems when utilities are close to each other or cross each other.  The result is that the reported location can be off and in the worst case, EML can report three utilities when in fact there are only two, or two where there is only one.  EML can report an S shaped warp in a cable when it crosses another utility.  He has a very detailed decision tree that can be followed to improve the confidence level in detected underground utilities by taking into account frequencies used, ground conditions, depth, separation between utilities, type of equipment (plastic, metal, PVC, optical cable, ceramic), water content, temperature and air content.

Airports are worst cases because of the many different types and density of underground utilities.  Recognizing that the reliability of underground utility detection and mapping is about 70% if using EML only, it is essential that anyone involved in owning, managing, locating, mapping and reducing the risk of utility damage during construction follow procedures like those defined by Andy at Heathrow and now through his own company Buried Asset Protection Ltd. [email protected].

Currently Andy is leading the update of the British quality standard for underground infrastructure PAS 128.

Accelerating disruption in the electric power industry

Back in 2016 Gartner predicted that by 2020 the largest electric power company will be an Uber-like behemoth that will not own assets but will simply manage energy suppliers and energy consumers in an open market.

At that time I blogged about three large Las Vegas casinos' plans to leave the NV Energy utility grid.  MGM Resorts left the grid in 2016 even though required to pay $86.9 million in exit fees to do so.  This trend has accelerated.  In 2018, 10 companies began efforts to leave Nevada’s utility monopoly, NV Energy.

Regulators in some jurisdictions have recognized that the traditional business model for electric power no longer works in this age of decentralized power generation.  New York's REV initiative and Texas' ERCOT deregulation initiative have become models for other states for modernizing power generation, transmission, distribution, and retail.  The central tenet of modern utility regulation is to restrict traditional utilities to owning and maintaining power transmission and distribution lines.  Matching power generation and consumption power is through an open, unregulated transactive energy market.  Open markets tend to encourage decentralized, renewable energy generation.

In 1999, the Texas Legislature began restructuring the state’s electric industry which allowed consumers to begin choosing their retail electricity provider. As of Dec. 31, 2001, deregulation of the ERCOT electricity markets required investor-owned utilities to unbundle their operations. The provision of service to end-use retail customers became competitive. There are over 100 retail electricity providers of which more than a quarter offer a 100% renewable plan to their customers as an option. Since 2002, approximately 85% of commercial and industrial consumers have switched power providers at least once.

In Illinois in 1997 the Customer Choice Act mandated that state utilities could no longer own generation facilities. During the Mandatory Transition Period 1997–2007, utilities were required to sell their electricity generation assets to other energy companies. Utilities no longer sold power and became responsible for delivering electricity only. Since 2002 in the deregulated market, customers were permitted to buy electricity from competing retail electric providers. In 2006 the Retail Electric Competition Act encouraged residential and small business customers to switch to alternative electric providers.

in July 1999, Ohio restructured its energy market to give consumers choice with their energy provider. The law took effect on January 1, 2001. Customers could now choose to buy energy from retail electric suppliers instead of automatically receiving it from the utility company (primarily AEP Ohio, Dayton Power & Light, Duke Energy, and FirstEnergy) in their area. In May 2008, each incumbent utility was required to shed its power generation operations and become a purely electric distribution utility.

All of the big technology players (Google, Apple, Amazon, Facebook) are already indirectly disrupting the electric power generation industry with commitments to billions of dollars of renewable power generation which has driven over $15 billion in investment in wind and solar power generation.

The latest battleground is Virginia where Costco's and Walmart's requests to leave Dominion Energy's grid have been rejected by the Virginia's State Corporation Commission (SCC) Division of Public Utility Regulation. Kroger, Harris Teeter, Target, Albertsons and Cox Communications all currently have exit applications pending at the commission.

The Virginia Energy Reform Coalition (VERC) includes policy experts from across the ideological spectrum aimed at breaking the monopoly held by two utilities, Dominion Energy and Appalachian Power in favour of a deregulated energy market.   VERC cites ERCOT as an inspiration, for restricting utilities to owning transmission and distribution lines and no longer selling electric power.

I remember back in 2013 John Wellinghoff, then Chairman of FERC, saying that these changes in the electric power industry are driven by power consumers - industrial, commercial and consumer.  "People are going to continue to drive towards having these kinds of technologies available to them. And once that happens through the technologies and the entrepreneurial spirit we are seeing with these companies coming in, I just don't see how we can continue with the same model we have had for the last 100 or 150 years...You need whatever you do need - it may be a wind system providing power remotely, it may be a battery system for storage, it may be a local generation system, solar PV or natural gas, or a combined heat and power system. But whatever it is, it is what the consumer needs at the cost level that is appropriate and the reliability level they think is appropriate based upon their choices."

Consumers wanting to be in control of their energy, corporations wanting to reduce their power costs and reduce their emissions, the world wide drive for more sustainable energy, and dropping wind and solar energy costs are all making the traditional business model unsustainable.  Regulators who see the future as a transactive energy market with traditional utilities providing infrastructure such as New York and Texas are prepared for the future.  Others are going to experience increasing pressure from consumers and corporations who want the freedom and flexibility of going off the grid.