Between the Poles

Economic value of big geospatial data could reach $700 billion/yr by 2020

McKinsey Global Institute published a report Big data: The next frontier for innovation, competition, and productivity that analyzes the state of digital data, how it is used in different domains in both the private and public sectors, and offers suggestions for policy makers. In addition to looking at big data as a whole, the report examines five areas in detail including personal location data.

Real-time location data generated by mobile phones and other devices has created an entirely new set of location-based services from navigation to pricing property and casualty insurance. Personal location data is a new horizontal application domain that includes industry sectors from utilities to insurance to transportation. The growing smart phone market is creating an exponentially growing volume of location-aware data.  McKinsey sees this domain as a hotbed of innovation that has the potential to create significant value for consumers.  For example, use of real-time  traffic information for navigation will create a quantifiable consumer benefit by saving time spent traveling and fuel consumption. McKeinsey expects the economic value these innovations create to far exceed the revenue generated by service providers.

Economic value

Mobile phones with GNSS chips have created a revolution will continue over the next  decade. This personal location domain offers the potential for huge new value creation over the next ten years that McKinsey estimates at more than $100 billion/yr (about 0.14% of world GDP) in revenue to service providers and as much as $700 billion/yr (about 1% of world GDP) in value to consumer and business end users. Realizing this value will require investment in technology, infrastructure, and personnel as well as appropriate government action.

McKinsey's detailed analysis of the major applications of personal location data finds that, in ten years’ time, these applications could create value of $100 billion/yr or more for service providers  primarily from sales of navigation hardware and revenue from LBS, mobile LBS premiums, and geo-targeted advertising.

McKinsey estimates that by 2020 the benefits (difference between willingness to pay and the cost of applications) that customers, consumers and businesses, will enjoy from location-based applications will reach as as $700 billion/yr. Most of this, about 70 %, will be the consumer benefit from time and fuel saved by using GNSS navigation systems including those with real-time traffic information and mobile LBS applications. The remaining 30 % of the benefits will be through location-enabled applications such as geo-targeted mobile advertising.  Smart navigation applications are estimated to provide $500 billion in value to global consumers in time and fuel saved by 2020. Geo-targeted advertising could represent more than 5 percent of total global advertising spending by 2020.

For comparison, a recent study prepared for Google by Oxera Consulting Ltd in the UK estimated global revenues from geospatial products and services to be $150-$270 billion per year.  The geospatial industry is estimated to be growing by 13% per year through 2016.  These numbers were estimated using a different methodology to McKinsey's. The lower estimate of $150 billion/year was estimated using a bottom-up approach based on the Bloomberg BICS classification of companies worldwide and assumptions about allocating the revenue from companies that provide geospatial in addition to other services such as Trimble.  The $270 billion number was estimated by scaling up the US revenue estimated by Boston Consulting Group to come up with a world estimate.

Personal location data

McKinsey includes sources of personal location data that locate an individual to at least within a couple of city blocks in an urban settingin its analysis.

• Credit and debit card payments from cards processed at point-of-sale (POS) terminals and  automated teller machines. In 2008, there were 90 billion to 100 billion transactions of this type worldwide.
• Mobile phones - the location of such devices can be triangulated from cell-tower signals.  In 2011 there were about 6 billion of these.
• Smartphones with GPS - about 600 million devices in 2010 and the number is growing at about 20 % per year. Many smartphones also have Wi-Fi networking capability, which can be linked to a physical location of a Wi-Fi network.

In the future McKinsey expects indoor location to become more important.  They mention Shopkick, which allows merchants to track their customers from when they walk into a store, and Path Intelligence, which can track foot traffic within malls or amusement parks.

McKInsey's research estimates that the global pool of personal location data was at least 1 petabyte in 2009 and that this pool is growing by about 20 % a year. 

Location data generated by smartphones is growing at double the average rate of growth for all location data.  Asia is the leading region for the generation of personal location data because so many mobile phones are in use there. In 2010 the regions with the largest number of cell phone users were

• China 800 million
• India 650 million
• North America 300 million.

Applications of personal location data

McKInsey has identified three major categories of applications of personal location data.

• Location-based applications and services for individuals including smart routing, automotive telematics, and mobile-phone based location services
• Organizational use of individual personal location data that includes geo-targeted advertising, electronic toll collection, insurance pricing, and emergency response.
• Macro-level use of aggregate location data that includes urban planning and retail business intelligence.

Barriers to use of personal location data

Concerns about privacy and security was identified by McKinsey as a major barrier to future expansion of the use of personal location data. Laws are unclear about who owns that data and how it can be used.  McKinsey believes that a policy and legal framework that clearly describes the permissible and prohibited use of these data would be beneficial for all stakeholders.

For policy makers, McKinsey sees that a priority is incentives for innovation are in place including developing an up-to-date framework for intellectual property rules and rights, funding R&D in potential breakthrough areas, and ensuring that the infrastructure, including spectrum policies, is optimal.

Methodology

By looking at the value chain of location-based applications, McKinsey examined both the amount of data generation and the potential to create value.  McKInsey estimated the amount of personal location data generated in each category of application orsource device and in different geographic regions using the installed base of devices, usage behavior, and the frequency with which data are generated. To assess the potential value that data can generate, McKinsey estimated the economic value of each application to end users—individual consumers, enterprises, and government organizations.

Privacy and personal geographic data

At the Creating the Policy and Legal Framework for a Location-enabled Society conference in Boston, Kirk Goldsberry, who is a Visiting Scholar at the Center for Geographic Analysis at Harvard, gave a fascinatng presentation with the help of two of his students on the topic of personal geographic data and privacy.

According the the Federal Trace Commission, which is responsible for protecting privacy at the Federal level, the 4 holy data types are

• Financial data
• Medical data
• Data about children
• Geographic data

Goldsberry's main point was that the first three are treated as very sensitive and there is a legal framework to ensure these data types are protected.  When it comes to geographic data, on the other hand, there is very little by way of policy or legal infrastructure ro protect personal geographic data.

As an example he cited the iPhone.  About two years ago it was disclosed that iOS devices maintain a database that potentially could be used to track the movements of users. In response Steve Jobs in an email said that "We don't track anyone. The info circulating around is false."

Without the user's explicit consent iOS doesn't record GPS location, but does record the location of cell towers and Wi-Fi access points, which can be used to create a clear general track of a user's movements. 

To show what was possible with this iOS database, two of Goldsberry's students voluteered to carry an iPhone (with location disabled) for a couple of weeks.  Goldsberry used the iOS database recording cell towers and Wi-Fi access points to put together maps that showed pretty clearly where the students had been including long distance trips and their daily routines.  For example, one student had travelled to Seattle for a conference and had stayed in a hotel.  The student's location at and around the hotel and conference were not hard to discern.  The student went to a Mariner's game and from the location determined through the iOS database it was very clear that the student was at Safeco Field.

Limited legal and policy protection for personal geographic data

I interpreted what Goldsberry showed as not targetting Apple specifically, but that there is very little policy and legal protection for personal geographical data.

A number of speakers made the point that technology, including GPS and GIS, has moved so rapidly and regulation and legislation move so slowly that for personal geographic data, the horse has already left the barn.  It is too late to close to the door. 

Experiment

There is the risk that hasty legislation or regulation on privacy and personal geographic data could prevent or inhibit the very real social benefits the technology is capable of delivering.  The approach that several speakers recommended is to enable governments to experiment, so that regulation or legislation on privacy of personal geographic data is able to optimally walk the fine line between protecting privacy on one hand, and enabling the social beneifts the technology is capable of on the other.

I commend Kevin Pomfret, Excecutive Director of the Center for Spatial Law and Policy, for organizing such an important conference at such a critical time for geospatial technology.

Location-enabled Society: Mapping tweets using real-time big data technology and privacy

In a very appropriately titled session Taking Society's Pulse in Real-time at the Creating the Policy and Legal Framework for a Location-enabled Society conference in Boston, one item really got people's attention focussed on the issues that arise from combining location with personal data. TweetMap is a technology that visualizes a large number of tweets on a map of the world. This is not the first time I have seen tweets mapped.  But what really got people's attention is that a speaker showed how to drill down to your neighbourhood and see what the people next door are tweeting about.  Of course you can do this without a map by simply following your neighbors on Twitter.  But somehow mapping it makes it seem more intrusive, even invasive to some people. In another demonstration, Tweetmap was used to show how a person sailing in the Caribbean could be tracked as they tweeted from their boat.

To be tracked geospatially on Twitter requires that you explicitly agree to geolocation tracking for Twitter on your iPhone or Android device.  For example, on an iPhone the geolocation option allows you to make your location known to others. Apps such as Facebook or Twitter for iPhone can report your location as well as the messages you post.  The iPhone requires that these applications obtain your permission before they can be enabled.  Apparently only about 2-4% of smart phone users, mostly younger folks, agree to this.  After seeing a Tweetmap drill down, I would expect that some people might have second thoughts about agreeing to the gelocation option. And in the not too distant future this will be possible in real-time on inexpensive hardware with open souce software.

TweetMap is an instance of MapD, a massively parallel database platform being developed through a collaboration between Todd Mostak, (a researcher at MIT), and the Harvard Center for Geographic Analysis (CGA).  Currently the tweet database starts on 12/10/2012 and ends on 12/31/2012 and contains 125 million tweets.  The database can be queried by time, space, and keyword.  The ultimate objective is to be able to query billions of tweets live in real-time.  In this case real-time means from tweet to tweet on a map in under a second.

MapD is a general purpose SQL database that can be used to provide real-time visualization and analysis of very large data sets.  MapD uses commodity Graphic Processing Units (GPUs) to parallelize compute intensive tasks such as querying and rendering very large data sets on-the-fly.  MapD runs on inexpensive hardware ( ~$1000) and is intended to be open source.

President Obama signs Executive Order on cybersecurity for critical infrastructure

IPrior to the President's State of the Union Address, President Obama signed an Executive Order on cybersecurity. This has  important implications for the electric power industry and is a wakeup call for utilties that have not yet developed or updated their cybersecurity policy.

Michael Daniel, Special Assistant to the President and Cybersecurity Coordinator, on the White House Blog, hs given an overview of the Executive Order including the motivation for it and its key provisions.

According to Mr Daniel, the government’s senior-most civilian, military, and intelligence professionals all agree that inadequate cybersecurity within the nation's 18 critical infrastructure areas poses a threat to the security of the United States.  Because of the seriousness of the threats, the President issued an Executive Order directing federal departments and agencies to use their existing authorities to provide better cybersecurity for the Nation.  The Administration received input from a broad range of stakeholders in industry, the public sector, the legislative branch, and the advocacy community including over 30 organizations representing all 18 critical infrastructure sectors.

The Executive Order focusses on the three areas, information sharing, a framework of core security practices based on existing standards, and privacy protections.

Information sharing

The Order makes it is a national priority to increase the cyber threat information shared with authorized individuals and companies.  In particular, it aims to improve information sharing between the private sector and all levels of government.  It expands the Department of Homeland Security’s (DHS) Enhanced Cybersecurity Services program to provide near real-time sharing of information on cyber threats with critical infrastructure companies and state and local governments.

Cybersecurity framework

The Executive Order directs the National Institute of Standards and Technology (NIST) to lead the development of a framework to reduce cyber risks to critical infrastructure. NIST is directed tol work with industry to identify existing voluntary consensus standards and industry best practices to incorporate into the framework.  The Order puts private-sector cyber leaders in critical infrastructure sectors at the core of  the development of voluntary best practices for the framework.  The DHS Secretary is directed tol establish a voluntary program to support the adoption of the Cybersecurity Framework by owners and operators of critical infrastructure.  This has direct and immediate implications for utilities.

Privacy

The Executive Order directs departments and agencies to incorporate privacy and civil liberties protections into cybersecurity activities based upon widely-accepted Fair Information Practice Principles as well as other applicable privacy polices.

Distributech 2013: Smart grid becomes mainstream

If there was one consistent message at Distributech this year, it was that smart grid has become mainstream - every utilty is doing something related to smart grid.

A very good example of some the things that utilities are abe to do because of their investment in smart grid technology is the keynote given by Michael Niggli, COO and President of San Diego Gas and Electric Company (SDGE).  SDG&E is a regulated public utility with 3.4 million customers and 1.4 million electric meters and more than 840,000 natural gas meters in San Diego and southern Orange counties.

Smart grid

According to Mr. Niggli smart grid is accelerating like never before.  SDG&E has deployed 2.3 million smart meters to electric and gas consumers. And there are other smart-grid related initiatives that SDG&E has undertaken including distributed generation, primarily solar photvoltaic (PV), plug-in electric vehicles (PEVs), and a new outage management system that leverages real-time data from smart meters..

Social networking and customer engagement

Having smart meters has helped in a way that SDG&E probably didn't originally project.  When SDG&E had to take the San Onofre nuclear plant, which represents 20% of their generation capacity, off-line in January 2012, they had to look for ways to dramatically reduce load.  SDG&E leveraged their smart meter network to help reduce power demand.  I have blogged before about utilities using social networking to communicate with the public.  In this case one of the ways that SDG&E took advantage of their AMI network was to use social networking to help reduce load.  The smart meter network (AMI) allowed them to  compare different schools' power usage so they used social gaming to create a competition among schools to see who could reduce their energy usage the most.

Green button

Another area where SDGE and a number of other utilities are helping consumers become more energy consciousus is the Green Button program, originally modeled on Veteran Affairs' Blue Button program.  SDGE's Green Button program not only allows users to access their own usage data, but also to make it available to 3rd parties, typically companies who analyze consumers' historical usage to help them reduce their power usage.  Across the country about 16 million consumers have access to their electric usage data via a Green Button program and another 30 mllion are exptected to get access soon.

Distributed generation

California is leading the country in clean energy.  California has mandated 33% renewable energy by 2020, and has also mandated reducing emissions to 1990 levels, by about 30% from current levels, in the same timeframe.  In addition new federal regulations targeting 54.5 miles per gallon in all new vehicles by 2025 is dramatically changing the energy landscape in California.  And as Mr. Niggli and other speakers emphasized, what is happening to utilities in Califiornia will happen to utilities in the rest of the country.

Distributed generation is an important source of renewable energy for SDGE.  According to Mr. Niggli In SDG&E's service territory there are 21'000 rooftop PV installations with a capacity of 160 MW.  New rooftop PV installations  are increasing at a rate of 3% per month.

Plug-in electric vehicles

Presently there are 2'500 plug-in electric vehicles in San Diego currently and SDGE is projecting that this could rise to 200'000 by 2020.  SDGE has used time of use pricing to motivate PEV owners to charge at super off peak when electricity is at a very low rate.  As a result 84% of vehicle charging is occuring between 12 midnight and 5am.

Wind and solar intermittent energy sources changes the daily demand profile

When intermiitent sources like wind and solar generation contribute more than about 20%, they begin to significantly impact the daily generation profile.  On the demand side, plug-in electric vehicles and increasing customer awareness of electric usage are changing daily load profiles.  The difference between the two is critically important to SDGE because they need to provide generation capacity to fill the gap.   Mr. Niggli gave two examples from the Californi ISO where generation dropped by 4'500 MW in 2 hrs, and then ramped up by 12'500 MW in 2.5 hrs, as a result of wind and solar generation.

SDG&E expects to start seeing the impact of the new daily power profile in 2015 and is working now to find dispatchable energy sources that can ramp up and down rapidly.  To put this in context, it takes months to ramp up nuclear, weeks for coal-fired plants, minutes for hyroelectric and natural-gas turbines, and microseconds for batteries.

Wild fires

Wild fires are a big problem for SDG&E because that makes SDGE liable for damage caused by fires started by SDGE equipment, for example, by poles being blown over. 

SDGE has implemented 144 weather stations to monitor local weather conditions in their service territory.  If winds get too high in an area increasign the risk of dmage to poles and other equipment, SDG&E will proactively shut down part of the network.  They have also put cameras on transmission lines and nstalled software that can automatically detect fires on transmission lines.

Outage management leveraging smart meters

SDG&E has installed a new outage management system that leverages their 2.3 million smart meters.  The major advantage that the new system provides is that they can detect an outage before the first customer calls in.  They can also differentiate without rolling a truck between outages due to customer problems and those caused by their own network.  The net effect is that it reduces the number of outages that SDG&E needs to respond to and reduces the average outage time (CAIDI and SAIDI).

Privacy by Design: Protecting the privacy of consumer electricity usage data

I blogged recently about the rules for protecting consumer electric power usage data collected by utilities in California.  I found this on the Center for Spatial Law and Policy site.  The California Public Utilities Commision, which regulates electric power in California, has defined the rules for providing access to and protecting the privacy of consumer electric power usage data collected by smart meters in California.

At  the Data Analytics for Utilities conference in Toronto last week, Ann Cavoukian, the Information and Privacy Commissioner of Ontario and one of the people who have raised the profile of data privacy, gave an enlightening overview of the important privacy issues as they relate to smart grid, and especially what they mean in Ontario. 

As I have blogged before,  smart grid, expecialy distributed generation and smart meters, is more advanced in Ontario than most other jurisdictions in North America and comparable to Sweden and Denmark in the EU.  Ann Cavoukian provided important information about smart grid in Ontario.   Ontario has invested $1 billion in smart meter infrastructure and is the first jurisdiction in North America to equip every small business and home (except for condos and multi-unit buildings) with a smart meter.   That's 4,770,289 installed smart meters and 4,258,094 customers on time-of-use (TOU) billing. 

Nearly 5 million smart meters means a lot of data and much of it is personally identfiable information (PII), defined as any data that can be tied to an individual, directly or indirectly. Electric usage data tells people when you are using electric power at a granularity of 15 minute intervals and can be used for nefarious purposes.

Privacy = control

Ann Cavoukian's fundamental point is that privacy means that electric usage data belongs to the consumer, and that for the consumer it means being able to control who has access to their data.

When a utility collects electricity usage PPI data for billing purposes, the customer is right to expect that this primary purpose is the only use that the utility will make of the data.  If the utility wants to use the data for another purpose, it needs to ask the consumer for permission.

Privacy in the big data era

The challenge in this era of huge volumes of data that smart meters and other smart devices can generate is reconciling protecting the privacy of consumer energy usage data with  the importance to utilities of analytics which requires access to meter data.

Privacy by design

Ann Cavoukian's recommendation for doing this is what she calls Privacy by Design.  The first principle of which is minimizing the collection of PPI data and using it only for the primary purpose for which it was collected. 

De-identification

But there is a technique which enables the utility to use meter data, simply by aggregating the data so that it is de-identified and cannot be linked to an individual, for example, by aggregating all meter data by postal code and rejectjng any postal codes with less than three meters.  National statistical agencies have been doing this kind of things for years.  For example, Stats Canada does not report demographic data for census divisions  that have less than three respondents.  There are a variety of mathematical techniques for de-identifying data.

Ann Cavoukian said that the principles of Privacy by Design underlie not only Ontario's privacy laws, but also privacy laws in the U.S., E.U., New Zealand and Australia.

Protecting consumer access to and privacy of electric power usage data collected by smart meters in California

I blogged some time ago about the Federally-initiated Green Button initiative.  Aneesh Chopra, then CTO of the US, in his keynote at the Gridweek conference in Washington DC, challenged the electric utility industry to do what the Federal government initiated at  Veterans' Affairs and what was taken up by the private health industry - "blue button" access to personal health records.  For electirc power utility customers with smart meters he challenged the electric utility industry to implement a "Green Button" program, so that every consumer with a "click of a green button" can access and download personal electric usage data.  And not only that, but give permission for thrid parties to see their data anonymously as well. To date, utilities in seventeen states including Arkansas, California, Illinois, Indiana, Kentucky, Louisiana, Maryland, Massachusetts, Michigan, North Carolina, Ohio, Oklahoma, Pennsylvania, Tennessee, Texas, Virginia, West Virginia, and Washington, D.C. have implemented Green Button programs.

Green Button web sites were launched at Pacific Gas and Electric and San Diego Gas and Electric.  Southern California Edison offers a Green Button program to certain of its customers who have smart meters.  By clicking on the green button, customers can view online or download their historical electricity usage data in a standard form that allows them to view their data on their own desktop computer.  You can see simple examples at the Green Button web site. People are also being encouraged to share their energy usage data with third party companies who would like access to consumer data because it helps them target their products to real consumer needs.  With the Green Button system, a consumer can download previous electric power usage information from their utility and then make it available to an energy company (using a process designed to protect consumer privacy).

California Public Utilities Commission Rulemaking to Protect Consumer Privacy

What I wasn't aware of, and just came across on the Center for Spatial Law and Policy site (an incredible trove of information about legal and policy issues related to spatial data) is the rulemaking by the California Public Utilities Commision, which regulates electric power in California, that defines the rules for providing access to and protecting the privacy of consumer electric power usage data collected by smart meters.  Some extracts;

"This decision adopts rules to protect the privacy and security of customer data generated by Smart Meters concerning the usage of electricity that are deployed by Pacific Gas and Electric Company (PG&E), Southern California Edison Company (SCE), and San Diego Gas & Electric Company (SDG&E). The rules adopted implement the protections ordered by Senate Bill (SB) 1476 (Chapter 497, Statutes of 2010)....

"the decision adopts policies to govern access to customer usage data by customers and by authorized third parties. PG&E and SCE must continue to provide and SDG&E must provide access to customer usage data. Each utility must provide pricing, usage and cost data to customers in the customer-friendly manners discussed below..

"PG&E, SCE, and SDG&E must ... make price, usage and cost information available to its customers online and updated at least on a daily basis. Each day’s usage data, along with applicable price and cost details, and with hourly or 15-minute granularity (matching the time
granularity programmed into the customer’s smart meter), must be available by
the next day...

"PG&E, SCE and SDG&E must ... provide third parties access to a customer’s usage data via the utility’s backhaul when authorized by the customer. The three utilities should propose a common data format to the extent possible and be consistent with ongoing national standards
efforts...

"Customer Authorization.
(1) Authorization. Separate authorization by each customer must be obtained for all disclosures of covered information...
(2) Revocation. Customers have the right to revoke, at any time, any previously granted authorization."

In other words the date belongs to the consumer, not the utility, and the consumer decides who has access to it, not the utility.

Electronic surveillance: GPS and smart meters and the Fourth Amendment

The Fourth Amendment to the U.S. constitution protects citizens from unreasonable searches and seizures and requires a judicially sanctioned warrant to search private premises.  Tracking someone's location or use of electricity using electronic devices have instigated two court cases that are determining the direction of judicial thinking on electronic surveillance.

GPS tracking

Over a year ago a conviction was overturned because the police did not obtain a warrant before installing a GPS in a suspect's car.  In that case the Supreme Court without dissent agreed that the police violated a citizen's rights when they attached a GPS device to his Jeep and monitored his movements for 28 days.

Electric power usage tracking

The Maine Public Utilities Commission (PUC) authorized the Central Maine Power Company to install smart meters in homes.  Customer can opt out, but they were required to pay both an initial fee and a recurring monthly fee. 

• Standard wireless smart meter - no additional charge
• Smart meter without radio transmitter - $20 one-time charge and $10.50 monthly charge
• Electro-mechanical meter - $40 one-time charge and $12 monthly charge

Some customers objected because of privacy concerns, but the Maine Public Utilities Commission dismissed the complaint. The case has now come to the Maine Supreme Court.  The appellants contend that the smart meter records electricity usage information which can be used to reveal when a person is at home, when they are asleep, and at what times they are using specific appliances, that this information gets transmitted to the smart grid, and that it represents an impermissible search for which no consent has been given.  The appellents cited Kyllo v. United States (2001), a U.S. Supreme Court privacy decision in which the Court held that using technology to collect data or information that otherwise could not be obtained without a physical intrusion constitutes a “search” under the Fourth Amendment.

GPS tracking and privacy

Apparently the US Administration has asked the Supreme Court to determine if the police need a warrant before installing a GPS to track a vehicle.  A criminal conviction was overturned by a federal appeals court on the grounds that the police did not obtain a warrant before secretly installing a GPS device on a man's car and monitoring his whereabouts for a month.  The appeals court said that the police violated the Fourth Amendment which protects citizens from unreasonable searches and seizures, as well as requiring any warrant to be judicially sanctioned and supported by probable cause.  Some people are concerned that not requiring a warrant would open the door to wide spread tracking of citizens with or without legal cause.

Thanks to Steve Swazee for pointing me to this.