Between the Poles

Are plug-in electric vehicles beginning to takeoff in Southern California ?

At  the Data Analytics for Utilities conference in Toronto today, David O. Jermain gave a fascinating presentation on the state of plug-in electric vehicles (PEVs) in Southern California and how they have affected the local utility, Southern California Edison (SCE). 

Utility folks have been of two minds about PEVs, whether to treat them as a massive electricity load growth opportunity, like air conditioners several decades ago, or as just another type of hair dryer.  (But PEVs are actually quite different from either air conditioners and hair dryers because hybrid PEVs like the Chevy Volt can generate as well as consume electricity and PEV's batteries could potentially supply electricity to external devices.)

PEVS in Southern California

David provided some very suggestive statistics about the deployment of PEVs in SCE's service area and the number of calls related to PEVs that the SCE call centre gets.

                       #PEVs

• 2010 1000
• 2011 1650 (65% increase)
• 2012 5200 (300% increase)

Two thirds of these PEVs charge at level 1 (120 V).  One third charge at level 2 (240 V).

Calls received by the SCE call centre suggest that at long last, PEVs may be beginning to take off.  Calls to the SCE call centre that relate to PEVs have increased by 350% over the last year.  The number of email inquiries relating to PEVs has increased by 360%.  The number of pageviews on the SCE web site that relate to PEVs has increased 14-fold.

         # pageviews

• 2011    1,600
• 2012 23,000

Since California probably leads the U.S. in the deployment of electric vehicles, what is happening there right now may be what the rest of North America will see in a few years.

How do PEVs affect the local distribution utility

Some people at utilities worry about whether the local distribution network would be able to handle a scenario in which everyone goes home after work and plugs in their PEV.   The concern is that the resulting surge or spike could overload transformers and worse.  There is also a concern that customers would require a lot of support from customer service folks.  SCE has developed a very elaborate workflow to integrate PEVs and PEV customers, but so far it really hasn't been necessary to use it..  So far PEVs haven't created problems for the local distribution grid and SCE's customer satisfaction statistics have not been negatively affected either.

But from the perspective of the market, PEVs present challenges.  David broke these down into issues relating to home fueling, public charging and how to handle fleets, for example,  the Department of Defence and companies like Frito-lay have bought a considerable number of PEVs.  The issues involve rates, submetering, the relationship with third party vendors who provide charging services, demand response, and so on.

Load shaping

One of the very interesting opportunities where PEVs could actually improve the reliability of the grid is in load shaping to balance intermittent energy sources like solar PV and wind.  Germany has been using a variety of techniques for years to sculpt load to balance wind and other intermittent energy sources.  A fascinating study by the Pacific Northwest National Laboratory (PNNL) looked at the possibility of using battery capacity in plug-in electric vehicles to mitigate imbalances from wind generation in the power grid.

Another interesting possibility is where PEVs could act as residential batteries during outages or to reduce costs where time of use pricing is available.

Why are utilities implementing analytics programs and how are they doing it ?

At  the Data Analytics for Utilities conference in Toronto yesterday, several speakers explained what is driving utilities to invest in analytics programs, and gave some real world examples of how utilities are implementing analytics programs.

Why analytics ?

In a nutshell, as a number of speakers emphasized, the goal is to enable utilities to make decisions based on information derived from the increasing volume of reliable data that is available to utilities as a result of their smart grid activities, rather than "gut" intuition, which is what utilities had to rely on in the past because they had limited and out-of-date data.

Lloyd Tokerud of Hudson Energy referenced a seminal paper, an MIT Sloan Management Review "Big Data, Analytics, and the Path from Insight to Value" that found that top performing organizations are twice as likely to apply analytics to activities. 

The study also found that the biggest challenges in adopting analytics are managerial and cultural.  Lloyd pointed out that to enable an analytics prorgam to be successful, it is necessary to educate people on the use of data and to promote a data-driven culture rather than intuition.  The important benefit is that an anlytics program can enable the utility to make better decisions faster.

Lloyd also said that analytics requires people with different skills than traditional IT.   He called them  "purple people".  Purple people understand both IT (red) and the business (blue).

Guidelines for implementing an analytics program

Mary Rich, who was responsible for smart grid and analytics initiatives at Centerpoint Energy in Texas gave a very insightful overview of how Centerpoint implemented their first anaytics program subsequent to their rollout of smart meters and AMI.

Based on her experience at Centrepoint she made several points that are important for any utility planning to embark on an analytics program, 

First of all, implementing an analytics initiaive is not a departmental exercise, it has to be company-wide.  It needs top-level support and strong governance.  Without these it will either fail or drag on for years.  Seondly it is not cheap.  It requires new tools and new people.  The big payback will not be short term, but long term.  Creating a data-driven culture is a cultural transformation that requires breaking down traditional data silos in engineering, operations, GIS, and across the company. 

One of the most important objectives is ensuring that the data is accurate and comprehensive.  Most importantly there can be only one authoritative source for each piece of data.  Conflicting reports are not going to help management make decisions rapidly and even worse they will raise red flags with the regulator.

How did Centerpoint Energy implement an analytics program

At Centerpoint they created a team comprised of subject matter experts (SMEs) from all divisions of the company.  It was ensured that these people were relieved of some of their normal committments so they could devote quality time to the analytics initiative.

The team spent about a month and a half identifying areas which potentially could benefit significantly from the application of analytics.  They then winnowed these down to the top 25.

For each of these opportunities they wrote detailed use cases,  Mary emphasized that this was not easy.  It was time-consuming and often required external support people to help SMEs who were not familiar with writing use cases.

As a practical example of implementing a use case, ons of the uses cases related to what Centerpoint refers to as "diversion" and what others call non-revenue-generating, non-technical losses.  With analog meters Centerpoint had to rely on the meter readers to note signs of tamperring with a meter and to report a possible diversion problem to Centerpoint.  A crew would roll a truck and investigate,but in many cases it would turn out not to be a diversion probem.  Wth smart meters meter tampering can be detected auomatically., but it is an involved process because it is necessary to make sure that there isn't an outage or a crew working in the area, because these can trigger events that look like meter tampering.  The SME team working on the use case estimated that by using autmated techniques they could increase detection by 20%.

The reality was quite different. They found that their autmated algorithms were much more effective than expected, because they were able to identify real diversion events with 80% accuracy.  They found that the big business benefit was fewer truck rolls.  Mary strongly recommended that every utility should know the cost of a truck roll because reducing truck rolls is one of the most important areas where utiltiies can reduce costs, often dramatically.  Mary said that in Houston alone, they were able to eliminate 4.5 million truck rolls.

Big data and trends in data analytics for utilities

I had the opportunity to catch up on current trends in analytics in the utility indusrty at  the Data Analytics for Utilities conference in Toronto today.

Jane Allen and Tom Peters from Deloitte gave a high level overview of what utilities are experiencing as they move toward the smart grid and decarbonizing energy generation through intermittent renewable energy sources.  

Big data

The smart grid means intelligent devices that can report their status sometimes as frequently as many times per second.  Smart meters, which are many utilties' first experience with intelligent devices, typically capture electricity usage in 15 minute intervals, which for a utility wih a million meters translates into 3.2 terabytes of data. They may be required by law or by the regulator to maintain historical usage data on-line for a significant period of time.  The result is that the data volumes that utilities need to manage are going up exponentially.  For example, Oncor, the largest distribution and transmission utility in Texas said that their operational database which used to be 30 terabytes is now 300 terabytes.

Analytics

To make sense of the huge volumes of data that utilities are collecting requires specialized analytical tools that are able to handle large data volumes.  The first wave of analytical tools were not designed to do this in real time or even near real-time, but the next wave will be able to.

Apparently Germany was the first country to use analytics to help sculpt electric power demand to the supply of intermittent energy from wind and solar.  This is critical in Germnay because renewables contribute a greater proportion of the electric power in Germany than any other major country.

Deloitte sees five main areas where analytics are being applied in the utility industry, customer, supply chain, financial, workforce and risk. 

Tom Peters oulined some of the trending application areas where analytics are being used by utilities. 

Pricing analytics

Segmenting demand using demographcs and looking at dynamic pricing to manage peak demand (demand response) in different sectors.

Marketing anlaytics

Monitoring the response to web-based marketing campaigns in real time.

Customer service analytics

Analyzing unstructured text data in social media looking for "bags of words" that identify topics in on-line discussions to monitor and assess how the on-line community is responding to utility programs and initiaitves.

Workplace injury analytics

Using analytics to identify high-cost work-related injuries that are preventable.

Fraud analytics

For example, analyzing a company's general ledger for an entire year looking for duplicate invoices, invoices sent to the wrong company, billings for products and services never delivered, and so on.  Tom Peters said that in the case of one large company this saved $11 million in a year.

Geospatial analytics

Using location information to look at big data in a different way from a traditional spreadsheet makes it possible to see things from a much more interesting and enlightening perspective.

In summary utilities are having to deal with a lot more data.  They need analytics to winnow the wheat from the chaff, to turn huge volumes of data into actionable information.  The good news is that the software analytical tools available now that are designed specifically for big data are getting better and more affordable.

IEA projects that current energy trends will lead to a 3.6 °C rise in Earth's temperature

The World Energy Outlook 2012 just released by the IEA makes for fascinating reading.

Primary energy demand

First of all the IEA projects global energy demand to increases by over one‐third between now an 2035 driven by emerging economies. There are projected to be dramatic shifts in the energy mix as collective share of oil and coal drop by fifteen percentage points to 42%.  However, because of increased use of natural gas fossil fuels remain the principal source of energy worldwide.  Unconventional gas (shale gas) is projected to surposs coal in the primary energy supply mix by 2035.

The United States, which imports about 20% of its total energy supply, is projected to become nearly  self‐sufficient in net terms by 2035, primarily because of unconventional oil and gas, bioenergy, and improved fuel efficiency in transport.

Large‐scale investment in energy‐supply infrastructure is required.  It is estimated to amount to about $37 trillion over 2012‐2035, which is equivalent to 1.5% of global GDP.   Of this the oil and gas supply infrastructure will require $19 trillion; while $17 trillion will be required by the power sector for generation, transmission and distribution.

Electricity

Demand for electricity is projected to grow by over 70% to almost 32 000 TWh by 2035.  Over half of the increase is projected to come from China and India alone.  Generation from renewables globally is projected to grow  from 20% to 31% of total generation.

Water and energy

Global freshwater use for energy production in 2010 totalled 583 billion cubic metres (bcm), or some 15% of the world’s total water use. Of that, water consumption – the amount used but not returned to its source – was 66 bcm.  It is projected that water use will increase by 20% over 2010‐2035 (compared to 70%  more energy production), while consumption will increase by 85%.   These trends are the result of more efficient power plants with advanced cooling systems that reduce overall water use but increase consumption.

Energy efficiency

Several major energy‐consuming countries (China, United States, the European Union and Japan) have adopted new energy efficiency measures over the last year.  Implementation of these programs is projected to reduce global energy intensity (energy consumption per unit of GDP) by1.8% a year through to 2035.  This is a major improvement compared with only 0.5% per year over the last decade.

The buildings sector is singled out as having tremendous potential for energy efficiency gains (one third of the world's total energy consumption is by buildngs), but four‐fifths of this are not projected to be realized, primarily due to non-technical barriers.

Emissions

The IEA projects that energy‐related CO₂ emissions will rise from an estimated 31.2 Gt in 2011 to 37.0 Gt in 2035.  It is estimated that this would result in a long‐term average temperature increase of 3.6 °C at the Earth's surface.

The IEA has investigated an alternative scenario which if implemented could result in a lower estimated temperature rise at the Earth's surface.  The "Efficient World Scenario" assumes a more efficient allocation of resources which will increase global economic output by $18 trillion by 2035.  It requires an added investment of $11.8 trillion in efficient end‐use technologies.   This investment is projected to be  more than offset by a $17.5 trillion reduction in fuel bills and a $5.9 trillion cut to supply‐side investment.

In this scenario global primary energy demand growth is halved compared to the most probable scenario.  Energy intensity is assumed to improve at 2.6 times the rate over the last 25 years.  It is projected that oil demand will peak before 2020 and decline through 2035,  Coal demand in 2025 is projected to be lower than today. Natural gas demand is projected to continue to increase but at a lower rate than currently projected in the most probable scenario.

As a result, energy‐related CO₂ emissions are projected to peak before 2020 and decline to
30.5 Gt in 2035.  The IEA estimates that this would result in a long‐term average temperature increase of 3 °C.

Is the future of the grid direct current (DC) ?

In the late 1880's when electricity was just beginning to be introduced primarily for lighting and for running motors, there were two alternative incompatible technologies that were being promoted as the basis for the electic grid.  Direct current (DC), associated with Thomas Edison, took an early lead because the first practical electric motors used direct current. DC had a major disadvantage in that because there was no way to convert between high and low voltages, the latter typically used by the consumer, it could not be transmitted over long distances efficiently.  Alternating current, associated with Nicola Tesla and George Westinghouse, had a significant advantage in that it was possible to transform between high and low voltages which made it possible to transmit electricity efficiently at high voltages over long distances.  Ultimately AC won out and is the basis for our current electric grid.

Direct current is everywhere

But this might be changing. Most of the digital equipment we typically use such as computers, gaming devices, televisions, control and sensing devices use DC.  If I remember, something like 40% of the power in the U.S. goes to chip devices, and this is expected to rise to 60% in the not very distant future.   Many renewable energy sources such as solar PV generate DC.  High voltage direct current (HVDC) transmission is significantly more efficient than AC transmission over long distances.  A significant breakthrough in high voltage DC technology was announced recently by ABB, which has developed a practical high voltage DC circuit breaker.  The fastest way to charge an electric vehicle is DC.

Solid state batteries

But perhaps the device that may dramatically change how consumers use power in the world of smart grid and distributed generation is the solid state transformer (SST).  The breakthrough that has enabled this to happen is silicon carbide (SiC) MOSFET transistors that can handle high frequency, high voltage transformation.  Transformers using this technology are 1% the size and weight of a traditional iron core transformer.   They are capable of higher efficiencies than iron core, can handle different voltage inputs and both DC and AC, and support bi-directional power flow.  All of which makes them highly suitable for a distributed generation environment.

According to the market research report "Solid State Transformer Market - Global Forecast & Analysis (2012 - 2020)" published by MarketsandMarkets, the total solid state transformers applications market is expected to reach $ 5 billion by 2020, growing at a CAGR of 82.3% from 2012 to 2020.

Major water organizations ask Congress to ensure funding for water and wastewater infrastructure

I have blogged previously about the issues facing water/wastewater systems in the U.S. and the challenge of funding replacement of aging infrastructure and upgrading to meet the requirements of the Clean Water Act. One of the most important funding mechanisms are state revolving funds which provide low interest loans for water and wastewater infrastructure. The Clean Water State Revolving Fund (CWSRF) provides loans for water quality improvement projects including municipal wastewater facilities, estuary protection projects, and nonpoint source pollution control.  The Drinking Water State Revolving Fund (DWSRF) supports loans to improve drinking water quality,

According to the ASCE, the problem is that most jurisdictions are spending between 1 and 1.5 percent on infrastructure, which is down from 5 to 6 percent spent in the 1960s and '70s.  In September 2002, an EPA Gap Analysis concluded that if there is no increase in investment, there will be a $6-billion gap between current annual capital expenditures for wastewater treatment ($13 billion annually) and projected spending needs.  The EPA has estimated that total investments of $187.9 billion for wastewater and $334.8 billion for drinking water is required over the  next 20 years.  The American Water Works Assn (AWWA) has estimated that the cost of repairing and expanding U.S. drinking water infrastructure will exceed $1 trillion in the next 25 years and $1.7 trillion over 40 years.

In late June of this year the House Appropriations Subcommittee on Interior, Environment and Related Agencies of the U.S. House of Representatives approved a FY2013 funding bill that will cut state revolving funds for water and wastewater.  The cuts are significant

• CWSRF - 53 percent cut from the $1.47 billion 2012 to $689 million in 2013
• DWSRF - 9.7 percent cut from $918 million in 2012 to $829 million in 2013.

Eight U.S. water organizations including the AWWA have just sent a letter to Congress encouraging Congress to ensure funding for the Clean Water State Revolving Fund (CWSRF) and the Drinking Water State Revolving Fund (DWSRF) is maintained at the Senate proposed figures of $1.47 billion and $918 million.

In addition they asked for $238 million for the Clean Water Act §106 Operating Grant program and  $190 million for the Public Water Work System Supervision Program.

California's first carbon auction judged to have been a success

California held its first carbon auction Nov 14 and it seems to have been a success.  According The California Air Resources Board the first auction attracted lots of participants, all the greenhouse gas allowances available (23 million allowances) were sold in about three hours and were sold for slightly more than the opening price of $10 per ton of carbon.  And the state has raised $233 million.  

AB32 is the legislation that mandated carbon trading for about 350 companies in California.  In the first two years 90 percent of the credits will be free. In other words, based on their emssions history companies will get 90% of their emission allowances for free from the state.  But by 2016, all allowances will be sold.

Realizing the value of smart meters and AMI by integration with geospatial and other enterprise systems

In September 2011, I blogged about one of the most amazing applications of smart meters and AMI that I had come across.

Alabama Power had deployed about 1.4 million smart meters in its operating territory. One of the initiatives they have undertaken to take advantage of their smart meters was to integrate their automated meter infrastructure (AMI) system with their outage management system (OMS).  The smart meters give them immediate information about an outage, helping identify without sending a crew out whether it was a network or customer problem  The integrated AMI/OMS system helps in reducing unnecessary truck rolls.

Disaster management

But the smart meters and AMI also helped in another, perhaps from a business planning perspective, unforseen way.   In April 2011 thirty tornados hit Alabama Power's operating area completely destroying two substations, flattening transmission pylons, breaking 7500 poles, and leaving 400,000 cutomers without power. 

By looking for smart meters that could be read the day before and comparing them with the meters that could not be read after the tornados, Alabama Power was able to put together a detailed picture using Google Maps of where power has been lost - all without making telephone calls.  The application could also provide emergency response officials with infrmation about whether the power was on or off in specific buildings - critical information that first responders require before entering a damaged building.  In addition, Alabama Power was able to track power restoration trends as customers started coming back on-line.

Integration of AMI with geospatial and other enterprise systems

There is an interesting recent interview with Arshad Mansoor of EPRI, who makes the case that to gain the full advantage of smart grid-related systems such as AMI, geographic information systems (GIS), outage management systems (OMS), data analytics and workforce management systems, they must all be well integrated.  The Alabama Power case is a very good example of the benefits of integration of AMI and geospatial technology specificially.  Mansoor sees integrating AMI with a geographic information system (GIS) as the first step in integrating AMI with enterprise systems..

He says that one of the most important benefiys of inegrating AMI and geospatial technology is the ability to link the utility's service point and metered account with a customer's phyical address.  Mansoor says that getting 100 % of customers correctly linked without integrating AMI with GIS is almost impossible.  He estimates that most utilities maybe have 5 percent incorrectly linked.  This could be even as high as ten % in some cases.

Once you can link service points to customers physical addresses and geolocation, then a whole range of other systems can be integrated with the AMI and GIS, including the outage management system (OMS), data analytics and the work management system.  Together these will all contribute to reducing the restoration time after an outage.

For example, if the outage management system is integrated with the AMI and GIS, then an upstream trace from multiple failed service points to identify the common point of failure is feasible.  The reverse is also feasible, identifying customers to contact when a piece of equipment or a substation fails.

With the workforce management system also integrated with AMI, GIS and other sytems, it is possible to send the crews to the exact point of failure.  

But there is the data quality issue here, about which I have blogged about on multiple occasions especially relating to the quality of location information, which is a major challenge for many utilities.  Poor quality location information about outdoor faciltiies can leave crews struggling to find failed equipment, especially at night or in snow, and  can lead to severe delays in service restoration.

Canadian cities ask for long term infrastructure plan from the federal government

In 2007 the Federation of Canadian Municipalities (FCM) commissioned McGill University to survey Canadian municipal governments to determine their infrastructure needs.   Based on the responses to the survey, FCM estimated that the municipal infrastructure deficit in Canada in 2007 was C$123 billion.  The FCM broke this down by key infrastructure categories,

• water and waste water systems - $31 billion
• transportation - $21.7 billion
• transit - $22.8 billion
• waste management - $7.7 billion
• community, recreational, cultural and social infrastructure - $40.2 billion

New Infrastructure Needs

The FCM McGill University survey also attempted to estimate new infrastructure needs, defined as all infrastructure that needs to be expanded or built to meet the changing needs of a community for demographic, socio-economic, environmental and other related reasons. Capital investments required to provide an enhanced level of service or meet new regulations would also be included.  The FCM estimated the cost of new infrastructure to be C$115 billion.

FCM 20 year infrastructure plan submitted to the federal government

November 13 the FCM submitted a long term municipal infrastructure plan (LTIP) entitled The Road to Jobs and Growth: Solving Canada's Municipal Infrastructure Challenge to the federal government as its contribution to the federal government's six months of consultations on long-term infrastructure planning.

The FCM plano calls for an increase in annual federal investments in municipal infrastructure  from $3.25 billion to $5.75 billion over 20 years.  This would bring funding, as a percentage of GDP, in line with historical levels from the 1950s to the mid-1970s.  To achieve this FCM is proposing maintaining the existing permanent gas tax transfer, but with cost-of-living indexing.  In addition it is proposing a new federal program that would matching investments from municipalities and provinces. 

The FCM plan also places great emphasis on encouraging public-private-partnerships (P3) and sustainable development.

The recommendations include

1. Funding should be long-term and predictable
Commit funding to LTIP for 15 to 20 years with five-year planning cycles.

2. Invest to leverage additional funds
An annual federal investment of $5.75 billion in LTIP will leverage an additional $7.5 billion in new provincial, territorial and municipal investments. These investments are on top of the $12 to $15 billion that municipalities already invest each year in local infrastructure and billions more contributed by provincial and territorial governments.

3. Renew and improve the Gas Tax Fund and the Building Canada Fund
a. Direct 100 percent of the Building Canada Fund (BCF) to municipal infrastructure.
b. Protect the current purchasing power of the Gas Tax Fund (GTF) and BCF against inflation.

4. Core Economic Infrastructure Fund
a. Invest $2.5 billion annually in a Core Economic Infrastructure Fund (CEIF), to be matched by municipal governments and by provincial and territorial governments, for a total program value of $7.5 billion a year.
b. Focus CEIF on core economic infrastructure, such as transit, roads, bridges and other municipal transportation infrastructure, and on water, wastewater and storm-water infrastructure.

5. Reduce gridlock, build transit
To reduce congestion and improve local mobility, allocate the $1 billion Cutting Commute Times component of the CEIF to transit.

6. Meeting new needs
Prioritize projects that meet new federal wastewater regulations.

7. P3s and alternative financing
a. Ensure the majority of LTIP is delivered through program models that maximize predictability and certainty. This increases municipal financing options, especially for P3s.
b. Create a “P3 screen” that requires all municipal projects with a value of $200 million or more that are receiving federal funding to develop a thorough business case that includes analysis on the viability of P3s.

8. Innovative Infrastructure
a. Create the Centre for Municipal Infrastructure Innovation and Sustainability (CMIIS) to help build the capacity of municipalities to improve asset management and innovative infrastructure practices.
b. Renew and expand the National Guide to Sustainable Municipal Infrastructure (InfraGuide).
c. Create the Innovative Infrastructure Fund (IIF) by expanding the FCM Green Municipal Fund endowment to make revolving loans and grants to municipalities for innovative, sustainable infrastructure pilot projects.

Canada's cities call for a 20-year infrastructure plan

Last week leaders from 22 of the largest municipalities in Canada met in Ottawa for the Big City Mayors' Caucus to discuss what the cities need from the Federal Government in next Spring's budget..  They praised the Federal government for investing in Canadian cities through successful programs such as the Gas Tax Fund, the Building Canada Fund and the Economic Action Plan stimulus funding but are concerned that most of this funding expires by 2014.  They are even more concerned about that the state of municipal infrastructure in many Canadian cities represents a significant threat both to economic development and to quality of life. Their first priority is to modernize transportation networks. Traffic gridlock costs the economy $10 billion a year. One out of every four roads is congested, and 50% need immediate repair.  The mayors said that only 8 cents out of very tax dollar currently goes to municipalities.

The municipal leaders said that they support the long term infrastructure plan recently submitted to the federal government by the FCM which asks the Federal government to commit $5.75 billion annually to municipal infrastructure.  THE FCM estimates that that level of committment by the Federal government would leverage an additional $7.5 billion annually from other sources including provincial, territorial, and municipal governments.

The evolution of the open source software foundation

Steve Walli of the Outercurve Foundation has published a very perceptive discussion of why the phenomenon of the open source software foundation (neutral non-profit) has become so important in the free and open source software (FOSS) community.

Open source is the best software re-use strategy

Steve's fundamental premise is that liberally-licensed collaboratively developed software is the best re-use strategy.  There are a number of different ways to look at this, but the one that has stuck in my mind was the result of attending a military open source conference in Washington, DC some years ago.  The U.S. military, which is perhaps the biggest user of open source software in the world, uses FOSS because they judge it to be more secure and to have fewer bugs than proprietary software all other things being equal.  The reason most often given for this is Eric Raymond's "many eyeballs" thesis that more people seeing code reduces bugs and security problems.

Economic benefit

As I remember from the history of the development of the Apache foundation, there is often a very strong economic motivation for FOSS.  In the early days of the web, there was the opportunity for every major vendor, IBM, HP, Microsoft, Sun, Oracle, and others to develop their own http web server.  Instead IBM helped form the Apache Founation and then encouraged other major vendors to join which just about everyone did (with one important exception).   The major vendors jointly contributed to developing one web server based on the NCSA http server and found it much less expensive than if each vendor had developed its own web server.

Intellectual property rights management is what FOSS foundations do

Steve makes the case that the core capability of FOSS foundations is to manage IP - he refers to them as IP Management Machines.  He says that this was the primary motivation for the Free Software Foundation (FSF), Apache Software Foundation (AS F), OSDL/Linux, and the Eclipse Foundation.  Corporations have more stringent software intellectual property (IP) needs and. neutral non-profit FOSS foundations have proved to be a solution to this problem, providing for the IP management needs of corporations.  FOSS foundations remove the barriers to corporate engagement for FOSS projects.

For example, for all contrbutions to Eclipse projects, the contributor has to agree to a standard IP statement for every submission.  It also must be contributed through standard channels, either Bugzilla or Gerrit, either of which which provide a complete audit trail for any contribution.   

For large contributions (over 250 lines) or code from external projects, then a complete IP review is required.  This can take a  long time because the provenance of all the code is checked, the licences are checked, and the code is scanned for IP problems.  Th reason Eclipse insists that this soemwhat involved process be followed for all contributions is that it guarantees that all Eclipse code can be used legally for commercial purposes under the EPL.

Growing projects into mega projects

Steve refers to Henrik Ingo's research which I blogged about recently.  Henrik found that the largest software projects are governed by FOSS foundations.   He concluded that there appears to be a glass ceiling for single vendor projects prohibiting their growth from the Large category upwards.  Based on his research findings he recommends that companies who want their projects to become mega project should contribute these projects to a FOSS foundation.

What is a FOSS foundation ?

A FOSS foundation is a neutral (not dominated by one vendor) non-profit (but run as a business) organization that provides legal structures, business operations, and technical services to its members, most of whom are firms or individuals involved in some aspect of software development or as software users.   FOSS foundations are different from corporate open source projects like OpenSUSE, Fedora, JBoss, or MySQL in several important ways. 

• FOSS foundations are neutral which encourages participation. 
• FOSS foundations put a lot of effort into managing IP openly and cleanly because clean IP encourages adoption among the user community. 
• FOSS foundations encourage broad participation, something that it is hard for a corporate project to do.

FOSS foundations have many projects they are responsible for.  The Eclipse Foundation for example has 270 active projects.  FOSS foundations also provide technical services and infrastructure such as source code repositories, build tools, test harnesses, code signing, bug tracking,  and so on. 

For example, the Eclipse foundation has implemented a very efficient process for contributors that minimizes for the contributor and the committer the time it take to submit the code, verify that it compiles and links correctly, conduct code reviews, run standard tests, and if all that succeeds actually submit the code to the main tree on Github.

Legal structures

One of the most important services that a FOSS foundation provides is legal liability and risk management.  This includes providing and managing licenses for contributors and users, provenance tracking, and indemnification.

Strong communities

In my view one of the most important things that FOSS foundations provide is a governance structure that creates an architecture of participation.  The objective is inclusion not exclusion.  The governance structure makes it easy to join conversations and to participate in various ways based on merit.  It  also encourages and grows committers (developers who can commit code changes). 

At the Eclipse Foundation there are something like 6 million users, some of whom contribute to the Eclipse ecosystem in various ways,by configuring the IDE, extending in in various ways, developing and publishing plugins, integrating Eclipse in products, and at the highest level fixing problems and developing new modules for Eclipse project code itself.  There is a fascinating video of a presentation on what it means to be a committer by a couple of Eclipse committers at a recent EclipseCON conference in Europe.

One of the most important aspects of being an Eclipse participant is the Eclipse software development process which is part of the Eclipse governance structure. 

This development process is used by all Eclipse projects.  The key objectives of the Eclipse development process are transparency (all decisions, code, and discussions can be seen by everyone), openness (encouraging participation by making it easy to join discussions), and diversity (encouraging broad participation, encouraging as many firms and individuals as possible to particpate).