Between the Poles

Planet Labs to capture near-continuous whole-Earth imagery from 28 five kilogram satellites

A very recent article is Scientific American provides more information about Planet Labs' plans for "continuous whole Earth imagery".  The constellation of 28 Earth-imaging satellites called “Flock 1” which just rode into space Jan 9 is comprised of 28 “Doves”  each weighing about five kilograms.  As part of the payload on an Antares rocket the satellites are on their way to the International Space Station from which they will be individually launched into Earth orbits. 

By the end of the month “Flock 1” will be in position to photograph the complete surface of the planet at a resolution of 3-5 meters per pixel. This will require storing the equivalent of a 10-terapixel image.  Planet Labs plans for the satellites to provide near-continuous pictures of Earth’s surface.  This will make Planet Labs the first to capture high-resolution whole-Earth images nearly continuously.  Existing Earth observation satellites can provide higher resolution and more spectral range but they only photograph specific targets which are selected by customers and their revisit times are on the order of a day.  In effect customers rent the use of a satellite to capture detailed images of very specific small areas of the Earth at particular times. Planet Labs' constellation will photograph the entire Earth's surface very frequently.  According to Planet Labs continuous whole-Earth images have the potential to serve many purposes simultaneously, from a single set of data.

Antares launches Planet Labs 28 satellite Earth observation constellation

In April 2013 Planet Labs launched two demonstration satellites, “Dove 1” and “Dove 2”.  November 21 two more satellites Dove 3 and Dove 4 were placed into orbit by a Dnepr rocket. Dove 4 conforms to the 3U CubeSat specification, with a launch mass of 5.2 kg. Basic physical dimensions are 100 mm × 100 mm × 340 mm, with two 260 mm × 300 mm deployable solar arrays.  Power storage is provided by Lithium-Ion cells. The batteries will be recharged by solar cells mounted on the body of the satellite and on the two deployable solar panels.

Yesterday January 9 Orbital Sciences Corp. launched its Cygnus cargo spacecraft aboard its Antares rocket at 1:07 p.m. EST Thursday from NASA’s Wallops Flight Facility in Virginia.  Its primary mission is to resupply the International Space Station.  Solar array deployment has been reported as complete for the Cygnus spacecraft.  It is scheduled to rendezvous with the International Space Station on Sunday, Jan. 12.

In addition to the suppplies for the Space Station the Cygnus carries CubeSats for 23 experiments designed by students for a variety of technology demonstrations.

And it also carries a constellation of CubeSat based satellites that will form Planet Labs' 28 satellite Earth observation constellation.  These mini-Earth observing satellites will orbit the Earth at an altitide of 400 km.  The satellites can provide much more frequent snapshots (revisits) of the same location on Earth than current Earth observation satellites.  This means basically 4D (2D/3D+time) allowing users to track changes—from traffic jams to deforestation—in close to real time. The satellites will send their images to at least three ground stations—two in the U.S. and one in the U.K. The data will be processed and uploaded for use by customers almost immediately.

Evolution of data analytics at an electric power utility and the importance of data governance

At the Second Annual Summit on Data Analytics for Utilities in Toronto, Lincoln Frost-Hunt, Director of Enterprise IT at Hydro One, gave a riveting presentation on how one utility has evolved from a company that in the past tended to use "gut feel" decision-making to a company that is increasingly relying on data-driven decision making,

Hydro One manages both transmission, almost 30 000 km or 96% of Ontario's transmission network, and distribution, representing about 75% of Ontario, mostly rural but some urban as well.  The total value of their assets is C$17 billion and they manage C$3 billion investment in their assets every year.

Data governance

Lincoln emphasized that a key prerequisite for a successful data-driven analytical approach is quality data.  Bad data means bad decisions.  They found that when they started to make investment decisions based on their analysis of their operational data, the deficiencies in their data sets immediately become apparent.  For example, they found that only 76 % of their connectivity data was reliable, the remaining quarter was unusable.  To ensure the necessary level of data quality and reliability data governance (processes that ensure that important data assets are formally managed)  was a key underpinning of Hydro One's analytics strategy.    They implemented major projects to cleanup their connectivity and other data.  Realizing that data quality is perishable, they also implemented a data governance policy and focused on improving business processes that produced bad data such as the processes for managing as-builts, which I have blogged about on multiple occasions, most recently here.  In addition to ensure that their data quality level does not degrade overtime and that it is maintained at the required level of reliability, they have implemented a system of quarterly audits.

Data analytics

Lincoln described several areas where Hydro One's data-driven approach has yielded value.

Customer billing

Since this is Ontario, all of Hydro One's customers have smart meters.  Hydro One processes 65 000 bills per day.  This is a key essential process because this is a major source of revenue.  A certain proportion of these get kicked out because of problems.  Since seven years ago when they first installed smart meters, tracking down the problems that caused these bills to be rejected was a slow manual process that used a lot of resources.  In the last two years Hydro One has implemented an analytical tool set that allows them to resolve these problems and get the bills out to the customers much more rapidly.  Since bills generates revenues, the value derived from the analytical approach was immediately apparent to managment and helped gain support for the data-driven approach to decision-making.

Asset management

Improving utilization of assets is a key objective of the regulator, the Ontario Energy Board (OEB).  Hydro One took an analytical approach based on modeling equipment failure so that they could estimate the probability of failure based on external conditions such as age and average load.  This allowed them to generate risk curves which enables them to demonstrate to the OEB that their investment strategy had found the sweet spot between the cost of maintenance and reducing the risk of major failure.  Geospatial technology was a key part of the asset analytics toolset that  helped them optimize the lifecycles of their assets.

Workflow optimization

Like most utilities, Hydro One is facing the challenge of an aging workforce, so improving productivity is a key corporate objective.  This is an especial challenge for Hydro One because of its largely rural service area.  Sending a crew to resolve a problem or respond to a service request could require hours of driving to the site and back.  Putting together "work bundles" comprising all the service requests and trouble calls in a geographic area reduces the time the crew spends on the road and maximizes their time actually resolving issues and servicing requests.  To do this they installed GPS, communications and ruggedized tablets on every truck so they knew where every crew was and the crews had access to the information they needed.  They also were able to show all trouble calls and service requests on a map for a three month period so they could easily assemble work bundles by geographic areas.  Another major benefit they found was that when they received an emergency trouble call, they could determine immediately which crew was nearest, thus minimizing travel time.  They also invested in route optimization so that the crew knew the most efficient route to follow.  The bottom line is that crews spent significantly less time driving and more time resolving trouble calls and responding to service requests.

Enterprise architecture

Lincoln described Hydro One's current enterprise architecture as comprised of three major enterprise systems with many heavily customized point solutions.  I suspect that many utilities would find themselves with a similar architecture.  Each point solution is a silo and the enterprise systems are big silos.  Whatever interoperability there is has either been customized or acquired from a third party by Hydro One.  Every time a vendor upgrades a point solution, Hydro One has to retest the links to other systems.

They evaluated three alternative architectures ranging from a best of breed approach with many point solutions focused on automating key business processes to a single enterprise system with bolt-ons or point solutions designed to run with the single enterprise system.  They decided on a hybrid approach, a single enterprise system with minimal point solutions when justified by incremental value.

Basically Hydro One relies on SAP for key business functions like ERP, customer information management, and asset management.  Their analytical tool set is Business Objects, another SAP module, integrated with a GIS designed to integrate with SAP. 

As an example of a bolt-on, they use an advanced visualization tool (Space-Time Insight which is an SAP partner) as part of their analytics tool set.  Lincoln said that visualization had really changed people's view of analytics because it enabled them to visualize operational data in space and time in ways they had been able to do before.  Visualizing based on geolocation, time and network topology has helped them gain insights about their operations that they would never have seen by looking at Excel spreadsheets.   It also created a bit of a challenge for Hydro One's IT folks, because it created an IT traffic jam - everyone wants to use the visualization tool and no one wants to look at Excel spreadsheets.

Smart zone

Hydro one has implemented what they call a "smart zone" around Barrie, north of Toronto, where all equipment are intelligent electronic devices (IEDs) including automated substations, basically a distribution smart grid.  They intend to roll this out to the rest of their service territory.

Real-time big data

Hydro One realizes that the volume of data they need to manage is increasing astronomically, and with IEDs they need to be able to analyze and make decisions in real-time or at least near real-time.  Lincoln said that their next major  IT development focus is  to be better able to manage "big data" in real-time.  I have blogged on a couple of occasions (real-time monitoring smart devices for decision making and using social media streams to improve customer experience) about an example of technology that is designed to help do this.

Planet Labs launches two more earth observation satellites

In April 2013 Planet Labs launched two demonstration satellites, “Dove 1” and “Dove 2”.  Yesterday two more satellites Dove 3 and Dove 4 were placed into orbit by a Dnepr rocket. 

It appears the Dove-3 and 4 mission is a technology demonstration.   They will operate from longer lived sun synchronous orbits. Both satellites are licensed to collect images of the Earth and will undertake an experimental mission in an 800 km by 597 km and a 700 km circular orbit, respectively.  Dove 3 conforms to the 3U cubesat specification, with a launch mass of 5.2 kg. Basic physical dimensions are 100 mm × 100 mm × 340 mm, with two 260 mm × 300 mm deployable solar arrays.  Power storage is provided by Lithium-Ion cells. The batteries will be recharged by solar cells mounted on the body of the satellite and on the two deployable solar panels.

Automating image-processing work flows for the Earth observation sensor web

Just about every industry from construction to utilities to roofers to insurance is being impacted, and often transformed, by the availability of high resolution Earth observation (EO) imagery from satellites, planes and helicopters, and UAVs.  Initially 2D, then 3D and in 2014 4D (2D/3D + temporal) imagery is available from many sources.  Imagery from international satellites,  low cost satellite constellations and UAVs is poised to dramatically reduce the cost of EO imagery.

An exponentially increasing volume of EO imagery is being captured every day from a variety of devices. The Committee on Earth Observation Satellites (CEOS) reports 286 Earth observation devices in orbit. The next Digital Globe satellite Worldview-3 scheduled for some time in 2014 will be capable of 31 cm resolution and can store two terabits of data between downloads. Two start-up satellite companies have already started putting satellite constellations in space, that promise to provide high frequency revisits to every point of the Earth's surface at low cost.  Planet Labs and Skybox Imaging each plan to launch constellations of 24+ low cost satellites that will capture high-resolution imagery of every spot on earth many times per day.  They promise to deliver the first ever HD-video of any spot on earth,  allowing users to track changes—from traffic jams to deforestation—in near real time. 

A few years ago I remember that the total amount of imagery being downloaded daily from all EO satellites was estimated to amount to about a terabyte.  Now a single satellite could be responsible for a terabyte daily.   If you add advances in aerial photogrammetry and the rapid development of UAV-based photogrammetry, we are looking at huge volumes of imagery every day, probably approaching a petabyte.

All of this data requires processing before it is made available to customers and end users.  Currently this processing is time consuming and a significant proportion of it stil involves manual and semi-automated processing.  This results in delays between the time the data is captured and when it is made available to customers which can range from days to weeks or even months.  Customers are becoming impatient with the time it takes to process data and are increasingly pushing for near real-time availability.

At the PCI Geomatics Reseller Meeting in Ottawa Wolfgang Lück, whose company Forest Sense cc customizes image processing workflows and provides remote sensing training, gave an overview of the sensor web and how automating image processing is enabling near real-time imagery availability.

Wolfgang says that most of the sensor web including different types of satellites, ground-bases sensors, mobile devices and satellite monitoring systems called sentinels are in place.  

• EDRS Geostationary Relay and supercomputer
• Wide area sytematic Optic/SAR monitoring
• VHR SAR
• Optical VHR/hyperspectral
• Field sensors
• Mobile devices
• DRS (earth receiving station)

According to Wolfgang, the communications protocols and the standards (for example, the OGC Sensor Observation Service) are in place for these systems to talk to each other.  Most of the satellite systems are in space, with just the sentinel systems still needing to go up. These are wide area systematic Optic/SAR monitoring systems being launched into space by ESA.  On the ground there are receiving stations (DRS), field sensors, and mobile devices carried by people.  The mobile devices capture as well as receive information.

Imagery requires involves complex processing to make it usable.  With the huge volumes of data that the sensor web is capable of capturing this processing has to be automated if the data is to be available in near real-time at low cost.  There are many actors that need to be accounted for, but the most important actors are end user customers (machines or human beings) because customers determine the products that are to be delivered.

Typical image processing workflow

There are different work flows depending on the sensors that are used and the final product requested by the customer.  To provide a feel for just how complex this processing is and the challenges involved in automating these work flows,  Wolfgang went through a typical work flow beginning with raw satellite imagery and following through to producing an orthorectified image suitable for classification, for example to identify different types of vegetation.  Even for this pretty basic image processing work flow, there are many steps.

1. Ingestion of data from pick-up point
2. Binary data to supported format conversion
3. Relative radiometric correction and artifact removal
4. Band alignment
5. DN to ToA reflectance conversion
6. Haze cloud and water classification
7. Auto-ground control point collection
8. Orthorectification
9. DSM generation
10. Topographic correction
11. Spectral preclassification
12. Level 4 product generation
13. Image compositing / mosaicking
14. Delivery of data to pick-up point

In the past many of these steps were manual or semi-automated.  The important breakthrough that Wolfgang was able to demonstrate is that he has been able to automate the entire process, which means that end user products, whether orthorectified images, digital surface models or classified vegetation maps are available much faster and at lower cost.

From Wolfgang's perspective the key to enabling this to happen is an image processing platform that provides a modular architecture, robust scripting, "big data" management, support for a broad range of sensor models (cameras),  support for Linux as well as Windows, comprehensive image processing functionality, the ability to incorporate custom leading edge algorithms -  integrated with an accessible user interface and API.  In addition because this level of automation pushes the envelope, responsive support from the vendor is also essential.

In the example Wolfgang used a scripting language, Python or EASI, to implement a work flow primarily comprised of PCI-Geomatica modules, but also custom components implementing leading edge algorithms from current research.

The first step is format conversion.  The data often comes in some proprietary format which the processing engine may not support.  A converter is required to bring it into a format that the processing engine supports. 

The next step is radiometric corrections and artifact removal.  Forest Sense supports a wide range of systems put together by emerging space nations, such as African or Middle Eastern countries.  In the example, you can see two lines lines are artifacts.  New gains and biases have to be calculated for those lines and applied to the image.  Other examples are random noise detection and dropped line removal.

Another problem is systematic noise.  A fast Fourier transform (FFT) reveals low a certain pattern in the high frequency domain which is characteristic of  systematic noise.  We can detect that automatically, do a FFT, create a mask to filter it out, and then do a reverse FFT to reconstruct the corrected image.

The next correction is the flat field correction that ensures that the sensors are calibrated correctly relative to one another.

The next step is band alignment.  The different bands need to be aligned because they are acquired at slightly different angles by different sensors.

The next step is total atmosphere reflectance correction.  What the sensor actually gets is relative voltage, and that is converted to relative radiance, which need to be normalized to total atmosphere reflectance.  This allows images acquired at different times, or from different sensors to be compared and to be used for quantitative analysis.   But the effect of the atmosphere is still there.  We can do atmospheric correction, but it is a bit of a black art and applied incorrectly can introduce artifacts.  For this reason, many people in the domain prefer to use total atmospheric reflectance data because they can trust it. 

At this point the image can be classified.

The first step in classifying the image is haze, water and cloud classification.  To do this requires both blue and red bands. These bands correlate with one another very highly, but the blue band is affected stronger by haze and water vapour.  What is called a clear sky vector can be applied to the image to calculate the haze vector and remove haze.  The right of the side shows the haze that is removed from the image.

The next step is collection of ground control points (GCPs) so that accurate geolocations can be calculated.  Unlike many systems which collect GCPs even over water or cloud which introduces errors, Wolfgang only collects ground control points over land. 

GCPs enable automated orthorectification as well as extracting digital surface models (DSMs) automatically from stereo pairs. 

The next step is preclassification. Spectral preclassification requires topographic normalization using a digital terrain model (DTM).  The image has to be preclassfied into different surface types that have different BRDF (bidirectional reflectance distribution function) properties.  Light reflects in different directions at different intensities on different surfaces at different wavelengths.  This has to be done using bands that are not affected by topography, which makes this step quite complicated. 

The slide shows a Landsat image of a hilly terrain with a lot of shade and exposed slopes which need to be normalized topographically.  Traditionally people have used a range of vegetation indices, such as the normalized difference vegetation index (NDVI).  From comparison with the DTM, it is possible to see that the NDVI is affected by topography.  Although it is a ratio between bands, it is still possible to see shady and sunny slopes.  The reason for this is that there is a lot of scattering in the blue band and shorter wavelengths but no scattering in the red and longer wavelengths so the ratio between the red band and the infrared band is not a pure ratio because the red band is affected by scattering. 

The TVI index was developed to be  resistant to topography.  Of course it doesn't work in areas of full shade.  A DSM is used to to calculate which areas are in complete shade so they can be masked out.  At this point all of these techniques can be applied to generate an automatic spectral preclassification of the image which then can be used for  topographic normalization.  The skylight adaptive topographic normalization is based on techniques that Wolfgang developed using all of these preprocessing steps.  In the corrected image it is still possible to still see some shady areas, but most of the shadow has been removed. 

The next step is image compositing and mosaicing and the generation of Level 4 (geoinformation)  products.  These products might include vegetation indices, biophysical parameters, masks, pixel-based classifications or object-based classifications. 

The final step is to package the data and deliver the data to the customer, typically by transmitting the imagery to a pickup point where it is catalogued.

At this point the product is available for the customer to collect from the pickup point.

Wolfgang demonstrated that downloading raw data and transforming it into something usable by a farmer, construction contractor, or insurance adjustor requires many steps and a lot of processing.  This example of a typical image processing work flow shows just what nearly a petabyte of the raw imagery captured daily must go through to create something usable by the end user customer (people or machines). To get this to customers in near real-time is incredibly computationally intensive and involves rapidly processing massive volumes of data.  This is why algorithmic performance is critical and why multiple processors with hyperthreading and most recently GPUs (graphics processing units) are being harnessed to improve throughput.

Near real-time satellite imagery coming soon

The Committee on Earth Observation Satellites (CEOS) includes 53 space agencies which together operate 286 Earth observation devices. One of the things that is changing rapidly is the resolution of these satellites.  Geoeye-1 which was launched  in 2008 is capable of 41 cm resolution.  Worldview-2 launched about a year later is capable of 46 cm resolution.  The next Digital Globe satellite Worldview-3 scheduled for some time in 2014 will be capable of 31 cm resolution.  Unless the Department of Defense changes the rules, you won't be able to buy imagery at this resolution.  it is still restricted to 50 cm. 

The other important characteristic of these satellites is the revisit time which reflects the frequency with which the satellite passes over the same spot on earth.  For Geoeye-1 the average revist time is less than 3 days.  For Worldview-2, it is 1.1 days.  For Worldview-3 it is less than a day.  If you consider all of the Digital Globe satellites, Ikonos, Quickbird, Workdview-1, Worldview-2, and Worldview-3 parts of Earth are passed over several times per day. 

Another important characteristic of satellite imagery is that it is not inexpensive.  In fact one of the drivers for the explosion of UAVs for earth observation that we are experiencing is the cost of imagery from other sources, satellite and aerial.
 
But there is a new type of satellite cluster that has the potential to be disruptive.  Two startup satellite companies have already started putting satellite constellations in space, that promise to provide more frequent revisits per day than Digital Globe satellites can provide and at a much lower cost. 

In April 2013 Planet Labs launched two demonstration satellites, “Dove 1” and “Dove 2”.  In early 2014, Planet Labs plans to launch 28 mini- Earth observing satellites at an altitide of 400 km.  The satellites will provide frequent snapshots of the planet at a resolution aof about 5 m, allowing users to track changes—from traffic jams to deforestation—in close to real time. The satellites will send their images to at least three ground stations—two in the U.S. and one in the U.K. The data will be processed and uploaded for use by customers almost immediately.
 
Skybox Imaging plans to launch a constellation of 24+ satellites that will capture high-resolution imagery (sub-meter) and the first ever HD-video of any spot on earth, multiple times per day. Skybox will capture the planet on a near real-time basis to provide a tool for addressing global challenges in areas including security, humanitarian efforts, and environmental monitoring.   In both cases it is expected that the cost of the imagery will be signficantly less than current pricing.