Between the Poles

Future of Utilities: Utility asset management will be based on shared geospatial platform

Currently geospatial data and technology are used tactically by electric power utilities for many purposes including outage management, vegetation management, disaster management, renewable energy facility siting, universal electrification planning, asset management, and energy density mapping, to name just a few.  But the smart grid is fundamentally transforming the role of geospatial data and technology in electric power utilities. According to a report from Navigant Research “The smart grid is all about situation awareness and effective anticipation of and response to events that might disrupt the performance of the power grid. Since spatial data underlies everything an electric utility does, GIS is the only foundational view that can potentially link every operational activity of an electric utility including design and construction, asset management, workforce management, and outage management as well as supervisory control and data acquisition (SCADA), distribution management systems (DMSs), renewables, and strategy planning.”  In other words the smart grid makes geospatial strategic.  Location is the basis on which data is organized and spatial analytics is how information is extracted from the data.  As Bradley Williams has so succinctly put it, "spatial analytics is becoming a key technology for electric utilities because everything a utility does - customers, assets, and operations - involves location."

At the Future of Canada's Utilities Summit in Toronto I had the opportunity to chat with Alexander Bakulev, VP of Strategy and Assets at METSCO, in Mississisaug, Ontario about asset management, location and GIS, and the future of utility IT.  Alex has deep experience with asset management at utilities in Ontario and as VP of Strategy he brings a forward-looking perspective to the changing role of location and GIS in asset management in electric power utilities. 

Alex, what type of asset management service does METSCO provide to your customers ?

We provide a framework for asset management based on a methodology, risk-based asset management. When a decision needs to be made such as whether to replace an asset or maintain it and run it to failure, those decisions are made by engineers to mitigate the risk of failing as well as for safety and environmental reasons. We enable our customers to translate those risks into dollars so they can compare dollars to dollars and make an economic decision whether to replace or maintain an asset.  Of course, this has to be done within a regulatory environment.  In Ontario this means PSAB 3150 and ISO 55000.

Can you give a perspective on how advanced your customers are with respect to asset management ?

Asset data

We have a maturity hierarchy that helps us understand where a utility company is in managing its assets.  When we begin working with a customer, the first thing we look at is their asset data, what do they have, in what media, and where it resides.  Companies have different maturity in term of the asset records they have and how they manage them. Usually what we see is that companies may have good inspection data, but the information is stored in a decentralized mode -  paper documents, Excel spreadsheets, special lab-testing software files, and so on - but not in a centralized system. In many cases we have to compile paper documents and files to order to combine it all into a centralized asset management system.  But we see that is changing. Companies are starting to invest in document management systems and capturing data digitally. They are also collecting data with mobile devices in a way that goes directly to the online ERP system.  In addition newer equipment is smarter and able to report inline so it's a paperless process.

Risk of failure

Once you have the basic information about the condition of each asset, the next step is to try to predict each asset's failure probability. We want to understand what's the chance that they will fail next year, after 5 years,  or 10 years.  If the company has historical data on failures including the type of asset, when it failed and why it failed, we can use that information to statistically predict failures.  To do this we first find data about which assets have been replaced. Then we try to determine the reasons why the assets were replaced. Often this information is even less available than condition information. With this information we can estimate the expected failure rate for existing assets.

Consequence of failure

Once you know the risk for every asset to fail, the next step is to look at the consequence of failure.  We not only look at what is being fed downstream from an asset such as a transformer, but also at the protection schema on the feeder.  For example, when a radial feeder fails, the result is a minimum of 4 hours outage.  If it the feeder is a loop design, the outage may only be 30 minutes. If the feeder is automated, the outage could be seconds.  And it may not be just power that customers lose, there could also be environmental and safety issues. For example, if a corner pole fails, the consequences are more serious than a pole in a street line. There are also financial factors. How fast do you need to replace the asset ? Do you keep it in stock or order it ?  For expensive assets, it is very expensive to keep spares in a warehouse.

Monetized risk

Once we know the condition of each asset, the probability that it will fail, and the consequence of failure,  we can monetize the risk. With monetized risk, when we have to make a decision, we make an economic decision.  Maybe it's better to run to failure, or increase the frequency of maintenance to watch the asset closer and accept the risk of failure for the next 5 years and after that replace it, or if it is high risk, replace the asset immediately. 

It sounds like you can classify utility companies based on the data they have and how they use it to make decisions ?

We have a maturity model that helps us determine where a customer is so we can determine how best to help them.

Run to failure -  Assets are only replaced when they fail.  From the perspective of financial costs, this alternative looks good, but if you look at customer value, the cost is very high.  There are many unplanned outages of relatively long duration.

Age-based - In this case assets are replaced based on the age of each asset.  In this case investment requirements can be three times higher than run to failure, but the customer is benefiting by improved service.  There are much fewer unplanned outages.

Condition-based - With this approach you only replace assets that are in poor condition. Investment requirements are less than age-based, but still quite high compared to run to failure.  Customer value is similar to age-based.

Risk-based -  For example, you may decide to run 70 % of assets to failure in those areas where there is low risk, but for the 30% of assets that are high risk, you try to avoid any failures. This requires focussed investments.  Investment requirements are significantly lower than condition-based, but still higher than run to failure.  But with this alternative, customer value is optimal within the budgeted investment level.

Where are most of your customers also in the maturity model when you first start working with them ?

When we first begin to work with them, most are somewhere between age-based and condition-based. Many are making age-based decisions, but are trying to move to condition-based asset management for many asset classes.  Surprisingly there are some large companies that still follow a run to failure approach.  We have customers that have gotten to the highest level, risk-based asset management, and we have helped them get there.

What size utility finds it easiest to do this ?

In our experience, mid-sized utilities - with half a million to a million meters - find this transition easiest.  They are more flexible.  For large utilities the transition  is challenging because it they have to change what they have been doing for the past ten years, mostly age and condition-based asset management, and they have implemented large expensive systems to help them do that.

How does location and GIS fit onto this ?

Location and GIS are fundamental in several areas of asset management.  First of all geospatial technology provides a communication tool.  Most critically geospatial visualization is used to justify investments. It makes it possible to show stakeholders high risk areas. It makes it possible to show people recommended investments and where to invest.  

Secondly, it helps to plan for the future.  By visualizing changing demographics it helps identify what parts of a city require investment. For example, if in the next 10 years most of the work is going to be done in a particular rapidly growing area of the city, it can be planned for by creating a yard in the area, for example.

We also use GIS for core planning.  When engineers are designing the electric power network for a new subdivision, they require very detailed information about existing electric power assets in and near the area.  This information is collected from many different sources and visualized on a map.  The engineers can look at the risk profile of every existing asset on the map and start planning which assets need to be replaced immediately, which in five or ten years, and where they need to install new equipment.

Another critical area for GIS is connectivity,  Connectivity is about which customers are connected to which service point, which service points are downstream from each transformer, and which transformers are on each feeder.  Connectivity information is essential because it enables utilities to know who is affected by a planned or unplanned outage. It also provides a link between customers and infrastructure for financial planning purposes.   Many utilities don't have reliable connectivity information or they think they do but really don't.  GIS is the primary tool that we use to establish or correct connectivity.

Another critical area that is growing in importance for GIS is spatial analytics.   GIS can help estimate the condition of assets and identify high risk assets. For a simple example, in a tabular ERP system, all poles are the same, but in the GIS we can see some poles are corner poles which are higher risk.  As another example, if an asset is near a highway which is salted in winter, this can affect the condition of the asset. Here we are getting into predictive analytics where we can make estimates based on proximity to things that may affect the expected lifetime of assets.  In this case assets near roads with salt in winter are at higher risk.

What's your perspective on the future for utility asset management  ?

Many companies are beginning to see the advantage of the mapping environment for asset management. In the past asset management was pretty much done with Excel or similar table-based tools. What we are seeing now is that increasingly decisions are being made based on maps, increasingly interactive maps.  There are several drivers for this.

The first driver is simplification.  Asset management is simpler when you put everything on a map.  And for the new generation of engineers, maps facilitate knowledge transfer - it 's easier for them to design, manage and maintain the grid with assets visulaized on a map.

The other driver is grid complexity.  With the smart grid and distributed energy generation, the grid is getting much more complicated. There is a lot more data that needs to be analyzed.  Thefe are also increasingly complex reporting requirements from the regulator.  There should be one environment that can handle all of this - data management, analytics, and regulatory reporting.  Maps help alleviate complexity and achieve simplification.

Do you foresee in the future that utility applications such as outage management, customer information system, crew management and dispatch systems, and others will be best of breed or an integrated solution from a single vendor ?

The future as I see it is all about map-based decisions - where are the outages, where are our marketing opportunities, where are assets likely to fail, where do our employees live, where are the customers that have lost power, where should we position crews in preparation for a storm ?   To enable this to happen, I believe the future for utility IT is a map-based platform which applications like asset management, design, outage management, and disaster management from different vendors can plug into and share information in a shared geospatial environment.

HxGN Live: Hyper-spectral camera mounted on a UAV for precision agriculture and other applications

The typical multi-spectral camera used on satellites and airborne platforms captures three or four bands - typically the Landsat bands: green, red and near-infrared.   But hyperspectral cameras can capture 270 bands and are light enough to be mounted on a UAV.  At the annual Hexagon conference HxGN Live in Anaheim, Headwall Photonics gave an overview of the capabilities and advantages of hyper-spectral remote sensing.  Headwall and Leica Geosystems have partnered to offer a complete UAV-based hyper-spectral package which includes the Aibotix X6, Headwall hyper-spectral camera, storage, IMU/GPS, and software. 

A hyper-spectral camera passes the light it captures through a diffraction prism that breaks the light into a spectrum that ranges from infrared through visible to ultraviolet.  The Headwall camera records 270 bands ranging from the infrared through the visible spectrum. This broad spectrum allows the chemical composition of materials to be detected.  Spectral fingerprints can be defined based on chemical composition which enables detection of different species of trees, stages of crop development, disease incidence in crops, specific chemical compounds in mining tailings, and many other materials.  The Headwall camera can capture up to 300 frames per sec.  That generates a lot of data.  The key to extracting information from this volume of imagery is spatial analytics software which is part of the package.

The Headwall Leica Geosystems package is an integrated multi-rotor UAV and visible and near infrared (VNIR) hyper-spectral imaging system with software control.  It includes Headwall’s Nano-Hyperspec VNIR sensor with the Leica-Geosystems Aibot X6 multi-rotor UAV. The Nano-Hyperspec VNIR Sensor captures bands in the wave length range 400-1000 nanometers (nm) with a 12 bit pixel and a maximum frame rate of 300 Hz.  It also includes 480GB of on-board storage and an attached GPS/IMU. 480 GB corresponds to about 130 minutes of imagery at 100 fps.

Typical applications of hyper-spectral imagery are
precision agriculture, mineral exploration, and civil engineering.  Specific examples of the application of the technology include marijuana plant detection, crop health monitoring, citrus species mapping, invasive species monitoring, growth cycle and monitoring of crop yield, mine tailings monitoring, storage tank leakage detection, environmental monitoring, disaster monitoring, and mapping urban areas. 

A fascinating example is monitoring crop health where a scan of a field shows areas where the plants are not doing well.  This enables the selective application of fertilizer to only those areas where it is required.  Similarly a hyper-spectral scan reveals areas of insect infestation, which enables the selective application of insecticide.  Another application that would interest grape growers in the wine industry is using hyper-spectral imaging to identify and monitor leaf roll virus infestations.

Hyper-spectral imaging can also be used in an urban environment roof type mapping, insurance assessments, identifying types of road and monitoring road conditions, monitoring tree conditions in urban forests, and for urbanflash flood and disaster preparation and monitoring.

The key differentiator of hysper-spectral imaging is much greater precision in recognizing specific chemical compounds whether in plants, rocks, mine tailings, or urban environments.  However, the price is managing a much greater volume of data. Spatial analytics is key to extracting information from the huge volume of data collected in the field.

Citizen geodata science an emerging big data trend

Gartner's latest hype cycle identifies a new trend "citizen data science" that has important implications for the geospatial sector.

At the recent Canadian Institute of Geomatics (CIG) Ottawa Branch Workshop on Big Data: What is it? Technology and Applications, Monica Wachowicz from the University of New Brunswick pointed out a brand new entry Data Science in the Gartner Hype Cycle 2014 that wasn't in the Gartner Hype Cycle 2013. Then in the Gartner Hype Cycle 2015 Data Science and Big Data have disappeared and a new entry with the intriguing name Citizen Data Science has appeared.

2013 Big Data

2014 Big Data and Data Science

2015 Citizen Data Science

What is data science ?

Monica referenced research by Harlan Harris et al “Analyzing the Analyzers” An Introspective Survey of Data Scientists and Their Work which surveyed data scientists to try to determine what data scientist do.  They divided data scientists into five categories based on their self-ranked skill sets; Statistics (which includes spatial statistics), Math/Operations Research, Business, Programming, and Machine Learning/Big Data, and four categories based on their respondents' self-identification; Data Researchers, Data Business people, Data Engineers, and Data Creatives.  

• Data Business people are most focused on the organization and how data projects yield profit.
• Data Creatives excel at applying a wide range of tools and technologies to a problem, or creating innovative prototypes at hackathons.
• Data Developers are focused on the technical problem of managing data — how to get it, store it, and learn from it.
• Data Researchers backgrounds are in statistics or mathematics, 75% of them have published in peer-reviewed journals, and over half have a PhD.

The resulting landscape of data scientists shows a broad range of skills distributed among the self-identified professional categories.  The researchers found that

• a defining feature of data scientists is the breadth of their skills, and their ability to single-handedly do at least prototype-level versions of all the steps needed to derive new insights or build data products.
• the evidence points in favor of a scientific versus a tools-based education for data scientists. 
• 70% of the respondents had at least a Master’s degree. 
• 40% had undergraduate degrees in scientific fields, specifically, physical or social sciences (but not mathematics, computer science, statistics, or engineering).
  

The survey showed that the most successful data scientists are those with substantial, deep expertise in at least one aspect of data science; statistics, big data, or business communication.  The authors concluded that data science is a collaborative and creative field, where the successful professional is able to work with database administrators, business people, and others to get data projects completed in innovative ways. 

What is citizen data science ?

One of the biggest drivers for a new type of data specialist is that there are simply not enough data scientists to satisfy the demand from virtually all sectors of the economy.  Gartner has recommended cultivating “citizen data scientists” which they identify as people on the business side who may have some undergraduate mathematics or social science background and who can be assigned to exploring and analyzing data with the appropriate software tools.  Last year Gartner predicted that the demand for citizen data scientists will increase five times more rapidly than that of the highly skilled data scientists Harlan Harris studied.

Software tools are critical for helping citizen data scientists find real insights and avoid simple statistical mistakes.  For example, a technology that Gartner identified is "smart data discovery" which is a next-generation data discovery capability that provides business users or citizen data scientists with insights from advanced analytics and helps them avoid some of the common statistical pitfalls.

Citizen geodata science

There is at least ancedotal evidence that there has been for some time a trend toward "citizen geodata science".  At the India Geospatial Forum in Hyderabad in 2014, I moderated a session on electric power.   One of the speakers was Arup Ghosh, Chief Technology Officer at Tata Power Delhi Distribution Ltd (TPDDL) who presented his perspective on implementing geospatial technology in a private utility.   The geospatial group at TPDDL had about 60 field personnel and 18 analysts and support staff,  none of which had an educational background in geospatial data and technology.  Twelve were electrical engineers and the rest were people with electric power experience.  All learned geospatial data management and simple analytics "on the fly".

At a recent GoGeomatics Social Jonathan Murphy gave a presentation about his experience working in Northern Alberta preparing terrain for seismic surveys.  Geospatial data and technology was used in all aspects of field operations.   The staff were experienced in seismic surveying, winter drilling programs, wildfire management, and road and facility construction, but had minimal education in geospatial data management and analytics.  They also had picked up enough geospatial knowledge "on the fly" to do their jobs.

These are two examples of what could be called "citizen geodata science" which I suspect is part of Gartner's citizen data science trend and is almost certainly growing more rapidly than traditional "geospatial data science" as geospatial technology is adopted into vertical industries.  The challenge is how to reach these people, perhaps through MOOCs, community colleges, conferences focused on vertical industries and including geospatial technology vendors, presentations and hand-on training, or vendor marketing to help them avoid the common pitfalls of geodata management and geoanalytics.

Automating the process of comparing digital models to reality

At the Spar 3D Expo and Conference in Houston I had the opportunity to chat with Nicolas Arnold, VP of Product Development at SKUR.  SKUR is a young software company that has just released its first product, which automates comparing a 3D digital model to reality as represented by a point cloud from terrestrial scanning technologies (typically LiDAR).  The scan can be of an existing structure or a new structure in construction. The variances are prioritized and can be inspected in an industry standard 3D visualization application.

Nicolas, SKUR provides technology that finds the differences between digital models and point clouds from scans.  What are some of the applications of this technology ?

We are in the business of comparing 3D building models with point clouds that are registered to the building model. We do this for two very specific use cases.

The first use case is what I refer to as construction QC.  Let's say you have a design model and during construction you capture a point cloud by laser scanning or some other technology.  What you are interested in is comparing the design model to what was actually built out in the field. To do this you simply upload the design file and the point cloud, specify three tolerances that are used to prioritize the differences between the model and point cloud, and then run our cloud-based application.  The application outputs colour-coded deviations, which you can visualize in Navisworks.

The second use case is what I refer to as model QC where you have the reverse situation.  Let's say there is an existing structure, a building or almost anything.  But there is no digital model and you need a model for a renovation or retrofit.  You can use a scan-to-BIM process to scan the building and generate a point cloud from which a model can be created.  The model creation could be done by your own people, by a third party provider, or by an automated process such as Clearedge.  In this use case what you are interested in is how  well the model fits with reality. You follow the same process, uploading the model and the point cloud, to compare the model to the point cloud to see where there might be modeling errors and other types of deviations.

These are the two use cases that SKUR serves. We target the sophisticated construction contractor (GC).  One application of the technology for a GC is to perform construction QC on their projects to look for problems.  Another application is pre-construction, when the GC is considering bidding on a renovation or retrofit project and wants to provide a digital model to the architect.  The GC wants to be confident that the model represents the structure accurately so he can tell the architect that the model is a reliable representation of reality. These are the folks that we are targeting for these two use cases.

Does SKUR provide data capture services ?

No, we are a software company.  If you are interested in this technology, you can create an account, it's up to you to capture the point cloud data however you want to do it, upload it onto our web-based system together with a digital model and then run our application.   You then download the variances and visualize them using an industry standard tool. We don't do any consulting.

I'm interested in how SKUR would be used during construction. What if the construction is only 10 % or 20 % complete, do I have to specify which part of the incomplete structure I have scanned ?

We have very smart algorithms that figure out which part of the structure to look at.  For example, if you bring us a point cloud of just the corner of a building and the model of the entire structure, we only focus on that corner of the building.  Another example is a building that is 12 stories high, but I only have a scan of the basement. Our software is smart enough to figure out the part of the model that relates to the scan.

What about the opposite case where during a scan-to-BIM process for an existing building, I have not finished the model so I only have a model for part of the building ?

It's basically the same algorithm. The software is smart enough to figure out which part of the scan relates to the partial model.  When you run our software, you simply upload the model and the point cloud and specify the tolerances.  The process is the same for construction QC or for model QC. From the perspective of the software, it's really the same thing.

Where does your software run, on a desktop or in the cloud ?

We are completely web-based, that is to say, cloud-based. Our software require heavy-duty processing that would not have been possible 5 years ago. It is all in the cloud and it's high performance multi-core computing.   We bring in the largest computers that are available on the Amazon cloud for these problems and they can generate the variances in a few hours.

What types of digital models do you support, 3D CAD, Revit, Bentley BIM, or IFC models ?

We are a young company, and we've just started.  Our first release supports 3D DWG models. But I think Revit models are the next logical step. And then we also plan to support the Bentley world.  The 3D DWG model really can be anything, with any level of complexity. I have had people asking me about things other than construction, like planes, such as an Airbus. I talked to a gentleman yesterday, who wants to use our software for a golf course.  As long as you have a DWG model and a point cloud that's registered to the DWG, we can find the variances between the two.  The registration between the two is critical. It has to be done accurately.

How difficult is it to choose the tolerances ?

It's very easy.  You choose three values corresponding to different degrees of variance which are colour coded, green, yellow, and red in the output.

How do you visualize the variances, I imagine for a large project there could be thousands of these ?

SKUR output is a visual representation and report of the detected locations where there are discrepancies showing the precise variance from design and, very importantly, the direction of the variance.  We colour code the discrepancies so you can spot the problem areas very rapidly. For example, in the graphic red and yellow coloring indicates degree of variance. Arrows show the direction of issue. In our experience most of the variances are green, so you can filter these out and just look at the ones colour-coded red, which are the big discrepancies.  This allows you to focus just on those important areas.  We're like an insurance company.  Most of the time it's fine, but when it's not you need to know.

Where did the algorithms come from  and where was the implementation done ?

We've invented everything from scratch in Oakland, California

I can imagine this technology being of tremendous value in many different sectors of the construction industry as well as in non-construction sectors.  Which sectors and geographies are you targeting as the market for your technology ?

The sector we have targeted so far are construction contractors (GCs). But going toward we will be increasingly focussing on owners.  We think that our software provides a view into the construction process that enables them to see during construction how well the designs by the architects and engineers compare to what is actually being built.

We are focussing on North America, but we also have clients in Europe. Europe is definitely high on our list, but as a young company we are focusing on close to home first.

If people want to try your software, how do they engage with you?

It's very simple. They sign up and create an account.  Then they can upload a model and a point clouds and hit the magic button. We do handholding in the beginning to help new customers.  But we don't do any data capture and or any data preparation. We simply provide a web-based software service.

GeoBuiz 2016: Geospatial and consumer devices, reality modeling, big data and analytics

Reality modeling

In the future, the deliverables at the end of a construction project will not only include the completed building itself, but also a digital "reality model" of the structure.  The digital model will include digital as-builts comprised of georeferenced point clouds captured by laser-scanning or meshes generated from images captured with a digital camera on a smartphone.  That was the prediction of Bhupinder Singh of Bentley at the GeoBuiz 2016 Summit in Bethesda.  This is a vision that is shared by other professionals in the construction industry including Ron Singh, Chief Surveyor at the Oregon Department of Transportation, who sees it as dramatically changing the construction process from surveying through design and construction to operations and maintenance.

Consumer devices and professionals

Consumer mobile devices will increasingly be the platform for many professional activities which currently require specialized devices and a PC computing platform.  Bryn Fosburgh of Trimble pointed out that while currently the location accuracy of GPS equipped smartphones is 10 meters, centimeter precision is surely coming because all that is required is a relatively simple antenna redesign.  Bryn sees a continuum in these devices between consumer and professional applications.   But while consumers tend to blindly accept whatever is available on their smartphones, professionals using these devices will have to be aware of the limits of the hardware, what it can be used for and what not.  For example, while smartphones in the future will be capable of centimeter accuracy in ideal locations, the accuracy will be less where there is a lot of scatter in the RF signals from the GPS satellites.  To ensure reliable results a professional surveyor relies on multiple sources including GPS linked to an earth station and traditional total station measurements.  Increasingly we will require other professionals to be involved.  Bryn reported a rising demand for photogrammetrists, as a result of the growing volume and availability of high resolution imagery which needs professional to help interpret.

Platform as a service

For virtually all of the world's big IT companies including IBM, Oracle, SAP, and Microsoft and the world's big content companies such as Digital Globe, the cloud is central to their vision of the future.  Together with others in the IT industry Xavier Lopez of Oracle sees the future in platform as a services (PaaS), but there are challenges.  A major challenge is integrating data in different databases including traditional SQL databases from Oracle and Microsoft and nonSQL databases such as Hadoop.  Another challenge is semantics.  Frequently different disciplines use different terms for the same thing.  I have blogged about the CB-NL project in the Netherlands aimed at solving the semantics problem in the construction industry.  Another big issue that Xavier identified is data fusion without co-location - how do you enable efficient processing of a logically federated database when the data resides in different locations ?

Monetizing the cloud

But perhaps the biggest challenge is finding an appropriate business model.  It is not completely transparent how the cloud is going to be successfully monetized, though the trend is toward some form of subscription or pay-per-use.   An interesting model of the successful application of the cloud in the construction industry is Textura, just acquired by Oracle.  Textura’s cloud serves as a place in which all of the participants in a project from the funding, through design, to construction can come together and access the shared assets necessary to complete the project.  Textura’s network includes about 85,000 general and subcontractors who using Textura to collaborate on construction projects.  Textura handles on average about 6,000 projects each month. The company’s shared database, which hosts project budgets, contractors, invoices, payments, change orders and compliance documents, resides in the cloud and is accessible to contract administrators, contractors, developers, subcontractors, consultants, title companies, and banks.

Big data and analytics

One of big drivers for the cloud is enabling the extraction of information from the huge volumes of data (most of which includes location) which are now available.  I have blogged about the nearly 100 petabytes of earth observation imagery available from Digital Globe.  According to Dr Richard W. Spinrad, Chief Scientist at the National Oceanic and Atmospheric Administration (NOAA) , NOAA is expecting to have 160 petabytes of archived geospatial data by 2020. 

As another example, Jeff Jonas of IBM described a publicly available database which includes 600 billion transactions per day collected by telephone companies.  This data, which includes location, can be mined (after being anonymized) for very interesting and commercially valuable information.  Jeff Jonas described some examples - using it to analyze where people spend most of their time (their "pattern of life"), how many people go into a store, and how long they remain in a store. 

Dipanshu Sharma, Founder & CEO, xAd illustrated live some of the things his company, which is in the location-based marketing business, does with this type of data.  xAd can customize ads to target people who are inside a store ("There's a great deal on aisle 5"), just outside one ("Come on in, we have great deals on umbrellas") or close to a competitor's store ("Checkout XXXX, our prices are lower than YYYY") - the words are mine.

In all of these example, geospatial data and technology are key to the applications.  Jeff Jonas foresees that the apps of the future on your smartphone will require geolocation and will be so compelling that just about everyone will enable GPS location on their smartphones.

GeoBuiz 2016: High volume geospatial imagery processing requires new architecture

DigitalGlobe owns and operates a constellation of commercial earth observation satellites.  The latest is Worldview-3 which is capable of about 30 cm precision.  Digital Globe's satellites collect 3,000,000 square kilometers of Earth imagery every day, that's about 70 terabytes of data per day.  Digital Globe also provides online access to 15 years of earth imagery, about 90 petabytes.  Of all high-resolution commercial imagery collected since 2010, Digital Globe has collected approximately 80% of it.  Clearly as a user you can't download all of this data, so how can users do things like search through this huge data set and generate a product such as a vegetation map for Tanzania ?

At the GeoBuiz 2016 Summit in Bethesda this week, Walter Scott, CTO and Founder, outlined how  Digital Globe is making this vast amount of high resolution data accessible to and usable by customers.  To make it possible for customers to search and process this data efficiently, Digital Globe has put all of the data on the public cloud (Amazon) and has provided a platform that supports efficient search, processing and workflows. GBDX is an online environment for efficiently running advanced algorithms for information extraction from imagery datasets at scale.  It is a set of RESTful APIs, and algorithms that have already been implemented by Digital Globe or its partners include car counting, orthorectification, land use, land cover, and atmospheric compensation. 

One of the most important efficiencies of the GBDX architecture is that it avoids having to move huge volumes of data over the internet.  It does this by bringing the application to the data, rather than the data to the application.  For example, to search through the data, GBDX provides a catalog API that uses 39 different image attributes such as geographic location, cloud cover percentage, sensor type, sun angle and others.  The search runs locally on Amazon, avoiding pushing data over the internet.

The GBDX API includes support for workflows.   A workflow is a sequence of tasks executed against sets of data. A typical workflow is initiated with a search for the data you want to process.  The selected data is assigned to a working set, and then processed  through a sequence of steps to generate your desired product.   For example, a typical product would be an orthorectified vegetation map.  Again the workflow runs locally on Amazon, avoiding pushing data over the internet during processing.  Only the final product resulting from the workflow, such as a vegetation map, would be transferred over the internet to the customer.

GBDX uses the cloud infrastructure of Amazon Web Service (AWS).  Data is stored in S3 ‘buckets’ that are accessed through GBDX RESTful web service APIs. GBDX includes built in tools supporting remote sensing algorithms from DigitalGlobe and ENVI.  GBDX is python-friendly and allows you create your own tools.

One of the application areas that the platform makes possible is deep learning.  The basic concept is that software can simulate the brain's large array of neurons as an artificial “neural network".  Large volumes of input data and some way of quantitatively assessing the results are required to train the system.  With the greater depth enabled by new computers and algorithms, remarkable advances in speech and image recognition are being achieved.  For example, a Google deep-learning system that had been shown 10 million images from YouTube videos proved almost twice as good as any previous image recognition effort at identifying objects such as cats.  Microsoft has demonstrated speech software that transcribes spoken words into English text with an error rate of only 7 percent.

But deep learning requires a lot of compute power and a lot of training.  GBDX provides the computational power to apply deep learning to earth observation and Digital Globe's data library can provide the training data sets.  What is required is a scalable way of assessing and quantifying the results. Walter Scott suggested crowd sourcing as a way of achieving the scale required for earth observation training.

SPAR 3D: Virtual reality, scan to BIM, infrastructure models, and mapping indoor spaces

I have just experienced two remarkable days at the Spar 3D Expo and Conference in The Woodlands, Texas, in itself a remarkable location near Houston.  I will blog in more detail in the near future about the things and people I had the opportunity to experience over the past two days.

First of all, virtual reality.  Everyone is aware of the Oculus Rift (see the new review at CNET if you aren't).  There are other technologies that have come or are coming from Samsung, Sony and Google.  But at SPAR 3D the most remarkable VR technology I personally experienced was what David Smith, CTO of Wearaility, talked about and allowed me and others to try - a set of incredibly light-weight glasses with acrylic lenses designed to work with a smartphone that gives you a virtual experience comparable to IMAX without the IMAX theatre.  I tried it and found it an absolutely amazing surround experience.  And if you have wondered where Michael Jones, of Keyhole and Google Maps fame, has landed, he is now CEO of Wearality.

Secondly, my personal passion is the geolocation of underground utilities.  At SPAR 3D Mark Klusza, Founder & CTO, Real-time Metrology, Inc. described a major new advance in ground penetrating radar (GPR) technology he has developed that can detect a 3/4 inch rebar (among other things) four feet below the surface in clay.

For existing buildings creating a BIM model remains an art rather than science.  But at SPAR 3D a number of new technologies were presented that can help automate, improve the quality, and speed the process.  Foremost among them in my mind was presented by Nicolas Arnold, VP of Product Development, at SKUR.  With SKUR's technology, which runs in the cloud, you can compare the 3D model that you have developed with the point cloud you derived the model from and identify all the points of significant variance.  The same technology can also be used during a construction project to compare what has been built to what was designed - even when the building is only partially completed.

Ron Singh, Engineering Automation Manager/Chief of Surveys, Oregon Department of Transportation (DoT) described progress on his remarkable vision of how automation is upending how we design, build and monitor, operate and maintain U.S. highway systems.  Richard Arrowsmith, Asset Information Group Team Leader, Highways England described the digital model of the English highway system.  Stan Burns, President, Integrated Inventory, who has just retired from Utah DoT, described Utah DoT's comprehensive database of every piece of highway furniture and signage in the Utah highway system.  Together these represent the future of the digitalization of national highway systems.

A new company, Indoor Reality, is the third startup of Dr Avideh Zakhor, who is on leave from the University of California Berkeley.  Her new device is a backpack loaded with sensors including lasers and infrared that enables you to map the interior of buildings by simply walking through a multi-floor building, including up and down stairs.  The resulting data can be used to automatically generate floor plans and 3D models and captures everything you need for an Energy Plus analysis.

Mark Shell and Darrell Gadberry, both of the City of Fort Worth Water Department, described a very versatile device with laser and sonar and camera devices on board that can be used for sewer line inspections of small and large diameter sewer pipes.  Fundamentally this device enables condition-based maintenance of municipal sewer systems.  Mark and Darrell reported that it has saved the Fort Worth Water Department $42 million.

I was able to interview Dr Avideh Zakhor, Nicolas Arnold, Ron Singh, Larry Kleinkemper of Lanmar Services who specialize in creating BIM models for existing buildings, and Greg Bentley of Bentley who has a broad, forward-looking perspective on the "digitalization" of infrastructure.

Crossrail's huge BIM+geospatial model targeted on operations and maintenance

The UK Government as part of its building information modeling (BIM) initiative has said repeatedly that it expects the big payoff of a digital model, estimated at more than 40% savings, will be during operations and maintenance, typically representing 80% of the total cost of a facility.  Companies such as BAM who do Design, Build, Finance and Maintain (DBFM) projects report significant benefits from full lifecycle BIM + geospatial.  But there is little if any quantitative evidence supporting this conjecture.  I have asked people from Finland familiar with the very early BIM developments in that country if there were studies of the benefits of BIM for operations and maintenance, but apparently the BIM focus there has been entirely on design and build.

 

Crossrail with a budget of £14.8 billion is the biggest engineering project in Europe.  It involves 42 km of tunnels beneath one of the most densely populated parts of Europe.  It has wider tunnels  and its 40 stations have longer station platforms than the Tube has.  Crossrail trains are expected to start running next year and the full network should be open by 2019.

 

But the most interesting aspect of the Crossrail project is a 3D digital model with associated asset data that has not only been used during design and construction, but is intended to be used for operations and maintenance.  Crossrail appears to be the first major project that may be able to provide support for the conjecture that the biggest benefits of BIM are for operations and maintenance.

 

The Crossrail model is comprised of spatial and non-spatial data with links between the two.  The spatial data is made up of more than 250,000 3D BIM models as well as as-builts, together comprising a few terabytes. As construction of each facility is completed as-builts are collected by point-cloud survey using laser scanners.  The point clouds captured in the survey are compared to the design and divergences that need resolving are recorded for fixing. The detailed asset data and documentation add an additional 5 terabytes.  This represents one of the World's largest BIM model.  A critical aspect of the spatial database is that all assets are geolocated so that workers can query a particular location of London on a map and then navigate to the Crossrail assets there.

 

The model is intended to become a crucial tool for monitoring, operating and maintaining Crossrail’s systems once the railway is running.  Sensors monitor various aspects of the railway's operation and remote-controlled devices can change operating parameters from a central control room or from a handheld device.  Managers can view this information within the 3D model and can zoom in on an area which needs attention.  Crossrail is testing low-power wireless smart sensors called Utterberries that can monitor strain, temperature, humidity, acceleration, and other aspects of a facility. Utterberries weigh 15 grams and are smart - they have an ARM processor on-board and can operate for more than a year on one charge.  One of the coolest capabilities of the digital infrastrucure is an augmented-reality interface which allows workers to hold an iPad up to a wall or floor and see a view of the infrastructure (electricity, water, and communications) under the floor or behind the wall.

RICS BIM 2016: April 4th BIM mandate looms for the UK construction industry

At this year's RICS BIM conference the elephant in the room was April 4^th 2016, the deadline for all public construction projects to be BIM Level 2 compliant. This is the culmination of a five year process starting in 2011 focussed on developing a set of documents defining BIM Level 2 and providing best practices for achieving compliance.

This year's conference suggests that for many in the industry they are now wrestling with implementation issues and are reporting a shortage of BIM skills.  They are finding that BIM is really about data and are realizing that data management and analytics are essential to handling and interpreting the huge volume of data that is being generated. A shortage of data scientists to help them to do this has become an acute problem.

Transforming the UK Construction Industry in Five Years

The government's focus from the beginning has been on full life cycle BIM.  Back in 2010/2011, the Government's BIM roadmap was perceived to be very aggressive, mandating BIM Level 2 compliance for all national government projects after just five years. In the UK public projects comprise about 50% of the construction industry so this amounted to radically transforming the UK construction industry in half a decade. Put another way the construction industry had to move from thinking analog to thinking digital in just five years. 

The utility industry has been going through a similar transition in moving to what is called a smart grid and distributed generation.  Some think that this process will take 20 years.  But new technology and a younger generation steeped in digital is stepping in to make this happen much sooner than that.  Energy is going digital more rapidly than expected.  I think it likely that something similar will happen in the construction industry because digital enables all sorts of efficiencies that we often cannot even imagine.  Uber and Airbnb are classic examples.  Both are the largest or second largest companies in their respective industries.  Gartner has forecasted a similar type of company will emerge for managing energy flows in the electric power industry by 2020.

For 2012 the Government's strategy focussed on discovery through pilot projects, notably the Ministry of Justice Cookham Wood project, a £20 mllion project where it is estimated that BIM saved hundreds of thousands of pounds in CAPEX costs.  

At the 2013 conference David Philp,Head of BIM Implementation, Cabinet Office, UK Government, announced PAS 1192-2 a key document supporting the Construction BIM Strategy to achieve Level 2 compliance.  It specified requirements for achieving BIM Level 2, sets set out the framework for collaborative working on BIM enabled projects and provided specific guidance for the information management requirements associated with projects delivered using BIM.  David Philp made it clear that although the initial focus is on the design/build part of the lifecycle with the goal of saving 20% of Capex and reducing the 30% waste in most construction projects, the "largest prize for BIM lies in the operational stages of the project life-cycle".

At the 2014 conference Peter Hansford, Government Chief Construction Advisor, made a link between the overwhelming interest in BIM as represented by the sold-out RICS BIM conference and the industry’s enhanced interest and appetite for BIM. He saw a shift in emphasis from asking ‘’should we do BIM?’’ to ‘’how do we do BIM?’’  This was happengin inspite of the fact that as, Rich Saxon, the UK Government's BIM Ambassador for Growth, pointed out In a panel discussion on the future of BIM that some aspects of BIM Level 2 were still  incomplete. 

Deborah Rowland, Head of Facilities Management, Government Property Unit, Cabinet Office in the UK Government outlined the Government's motivation and plans for a program to improve the post construction handover and operation of newly constructed buildings. She described the Government Soft Landings (GSL) program which was intended to rectify this and was designed to put in place a legal, contractual, and technical framework (based on the BSRIA Soft Landings Framework)  incorporating building information modeling (BIM) to fix the problem by ensuring continuity thoughout the buiiding lifecycle from inception, though design, construction, commissioning, training and handover through to operations and maintenance for 1-3 years after handover.

At the 2015 conference seven key BIM Level 2 documents (PAS 1192:2, PAS 1192:3, BS 1192-4, CIC BIM Protocol, Classification (Uniclass), Digital Plan of Works / Levels of Detail (LoD), Government Soft Landings) which collectively represented the current repository of BIM best practices were announced.  Every speaker at the conference referenced these documents.

Seven major government projects using BIM, worth about £ 10 billion, have maintained records of the savings they have seen from using BIM. The measured benefits of BIM and factors associated with BIM range between15-20 %.   By 2014 the government estimated that it had achieved savings of £ 1.4 billion through BIM and related improvements in the 2013/2014 fiscal year.  But with full lifecyle BIM, the government saw an upside potential of 40%+ with the biggest benefits being realized in the operations phase of the building lifecycle.

This year it was announced that Government statistics show that in 2015 Her Majesty's Government saved GBP 3 billion on these demonstration projects as a result of gains in construction productivity related to BIM. 

Full Life Cycle BIM

But BIM Level 3 is where the gold is.  Level 3 will enable the interconnected digital design of different elements in a built environment and will extend BIM into the operation of assets over their lifetime. It will support the accelerated delivery of smart cities, services and grids. As the AGI has pointed out in a recent Foresight Report integrated BIM and geospatial technology will be fundamental for BIM Level 3.

Is the UK construction industry ready for April 14 2016 ? 

An industry survey in the UK in 2013 reported that 54% of respondents said that they were using BIM on projects.  Most of those not yet using BIM said that they would be within one to two years.  70 % of the respondents reporting using BIM said that BIM has given them a competitive advantage. But this year's conference suggests that most in the industry are doing BIM although for some it is not really clear what BIM Level 2 is and how one demonstrates compliance. 

Data and People

This year's RICS BIM conference was quite different from previous conferences.  The focus was heavily on implementation, not on progress in completing specifications or developing and working with 3D models as in previous RICS BIM conferences.

The event was led off by a panel discussion that set the tone for the rest of the conference - the focus was on data and people rather than 3D BIM models.  The industry people on the panel were patently up to their elbows in real world BIM projects including data, the analytics to make sense of it, and the shortage of the required skills (data scientists and people with BIM skills).

Alex Jones, Interserve, said that within the construction industry there is now a focus on data and how data is used downstream.  He said that clients (owners) are now aware of BIM and that both private and public procurements are asking for BIM.  But he qualified this - they are not so much asking for BIM (as in 3D models) as asking for data that can be used downstream. David Throssel, Skanska, added that deliverables now means data.

In the context of data, Simon Rawlinson, Arcadis UK, gave an example of utilities where smart devices and automated capture has resulted in a database of asset conditions, and which also includes geospatial data for geolocating those assets. This information allows utilities to move forward to condition-based maintenance where prioritizing maintenance and replacement depends on asset condition not the calendar.  Real-time monitoring is being adopted by virtually all industries.  For example, devices are being implanted in structures such as bridges to monitor distortions. He added that the leadership has not yet recognized that sensors and analytics enable a major change that will revolutionize asset maintenance.

Also the new digital economy is disintermediating certain skills. For example, digitalized construction requires fewer layers. Simon cited a KPMG study of disintermediation by technology. As an example, he mentioned data such as that collected by utilities from smart devices that allows owners/clients/utilities to do condition-based maintenance, which can lower field staff requirements significantly.

In the construction industry as a whole one of the major issues that emerged is a shortage of BIM related skills.Simon emphasized that now that we have lots of data, what we need urgently are data scientists to help make sense of it.  He identified this as a major problem because in his experience data scientists are as rare as “hens teeth”. 

David Hancock, UK Government Construction, said that construction industry still thinks analog. Nowadays we can all produce data, but analyzing it is the challenge.  We need people who can analyze to produce actionable items. He also identified BIM skills shortage as a major problem, and repeated that data scientists people are “as rare as hens' teeth.  He suggested that one solution is upskilling – training existing construction/bldg maintenance/utility folks in BIM and data science.  Another may be raiding the U.S. baseball industry where teams such as the Oakland Atheletics starting using analytics several years ago.  "Moneyball" analytics has been recommended as a strategy for the utility sector in the U.S.

In the question period a speaker argued that the 3D BIM model has been "massively overplayed".  BIM is really about data and collaboration, not about 3D. To maximize the benefits of BIM clients  (owners) have to be clear about what the model, and much more importantly the data associated with the model, are going to be used for. For example, if for facilities management (FM) then clients (owners) have to be specific about what data is required for maintaining and operating the building as part of defining deliverables.

BIM Level 2 for BIM Novices

What if you are a small company with no more experience with BIM than discovering what the acronym stands for ?  At RICS BIM 2016 there was a sequence of presentations by a group of mostly quantity surveyors (people who do quantity takeoff), all BIM novices, who came together to try to understand and share openly what BIM Level 2 means practically. (As one speaker pointed out, there is no legal definition of BIM Level 2 compliance.  They called their project KT4BIM, which you can find on LinkeIn. The objective is to answer the questions

• how do we do BIM Level 2

• how do we demonstrate compliance.

The target audience is BIM novices and for both the residential and commercial sectors.  The objective is to share all the material used and developed as part of this project with the industry.  More information about this very important project will be provided in my next blog post.

BIM Level 2 Documents

I've included this list just to show how comprehensive this set of BIM and BIM-related documents is.

Key documents

• BS 1192:2007+A1:2015     Collaborative production of architectural, engineering and construction information
• PAS 1192-2:2013     Specification for information management for the capital/delivery phase of construction projects using Building Information Modelling
• PAS 1192-3:2014      (Corrigendum No. 1) Specification for information management for the operational phase of assets using building information modelling
• BS 1192-4:2014     Collaborative production of information Part 4: Fulfilling employer’s information exchange requirements using COBie
• PAS 1192-5:2015     Specification for security minded building information modelling, digital built environments and smart asset management
• BS 7000-4:2013     Design management systems. Part 4. Guide to managing design in construction
• BS 8536-1:2015     Briefing for design and construction. Code of practice for facilities management (Buildings infrastructure)
• Uniclass 2015     Classification embedded within the NBS Toolkit
• Digital Plan of Work     Delivery plan embedded within the NBS Toolkit
• CIC/BIM Pro first edition 2013     Building Information Model (BIM) Protocol

Supporting documents

• CIC/INF MAN/S first edition 2013    Outline Scope of Services for the Role of Information Management
• Government Soft Landings    "Design for operation" to encourage better outcomes using BIM for built assets during the design and construction stages to ensure value is achieved in the operational lifecycle of an asset.
• AEC (UK) BIM Technology Protocol Version 2.1.1 June 2015    Practical implementation of BIM for the UK Architectural, Engineering and Construction (AEC) industry
• BS ISO 16757-1:2015    Data structures for electronic product catalogues for building services. Concepts, architecture and model.
• BS 8541-1:2012, BS 8541-2:2011, BS 8541-3:2012, BS 8541-4:2012, BS 8541-5:2015, BS 8541-6:2015    Library objects for architecture, engineering an construction

For the future of distributed energy watch the Hawaiian power utilities

Keep a close watch on the Hawaiian power utilities (HECO) because Hawaii has committed to 100% renewables by 2045 and 65% by 2030. That was the recommendation of Sharon Allan, CEO of the Smart Grid Interoperability Panel (SGIP) at a DistribuTECH2016 MegaSession on distributed energy resources (DER) in response to a question from the audience asking what should utilities do to get ready for GRID 3.0.

Hawaii already has 487 MW of solar PV capacity, 90% of which is residential rooftop panels. At  DistribuTECH2016, Talin Sakugawa from HECO and Matthew Shawver from in2lytics gave an insightful presentation on distributed energy generation (DER), big data and analytics at the Hawaiian power utility.  Hawaii has seen an exponential rise in rooftop solar PV. As a result since 2010 the utility has been experiencing a reduction in net load and revenue. The most dramatic drop in load is during daytime hours as is readily apparent by comparing the utility's load on clear and cloudy days which reflects the impact of residential solar PV.

Challenges facing utilities

The Rocky Mountain Institute (RMI) has published  an analysis of the economics of leaving the grid.  The central thesis is that solar PV and electricity storage enables consumers to leave the grid completely.  Solar PV has already reached grid parity in about 10% of the U.S. and declining PV prices suggest that this trend will continue.  However, without the ability to store electric power, consumers with rooftop PV still require the grid for nights and cloudy days.  The development of combined solar PV and batteries from Tesla and others promises to make solar + storage accessible for increasing numbers of consumers.  These consumers could form their own microgrid, either by themselves or with their neighbours and disconnect from the grid.  Alternatively they could become a source of dispatchable power and become an energy provider.  Increasingly they could find that either of these options is economically advantageous.  This has serious implications for local utilities because if this trend develops it will seriously erode the traditional utility revenue base.   It could also lead to a completely decentralized grid comprised of many microgrids.  From an operations perspective it increases the complexity of managing the grid compared to the centralized model in use today.

The other challenge for a utility with a high penetration of intermittent, distributed generation (DER) is load balancing, ensuring that generation meets demand. The German power industry, the U.S. Department of Energy, Hawaiian power utilities,  California power utilities, ERCOT, and many others are working to address this challenge.  HECO's presentation at DistribuTECH gave an insightful overview of what they are doing to meet this technical challenge.

Data sources

HECO collects a large amount of data from diverse sources. It is comprised of internal operational data plus data from esternal sources. It includes weather forecasts, data from customer sited PV, consumer/public resource data, phasor (PMU) data, renewables power quality, feeder data, irradiance meters, generator and substations power quality, SCADA, and vendor data such as forecasts and data from Independent Power Producers (IPPs). The data has widely different time resolutions ranging from real time such as PMUs sampling 30 times per second, SCADA reporting every 2 seconds, calculated gross load every 2 seconds, PV inverter data reporting every 5 minutes and aggregated monthly, weather forecast reporting every 15 minutes and aggregated daily, transformers collecting data every 15 mins and aggregated monthly, and smart meters reporting every 15 minutes and aggregated quarterly. Much of the data includes location. The data comes in different formats and has different levels of quality.

Fundamental to distributed energy are weather maps.  Maps of irradiance and wind velocity can be directly used to estimate or predict solar and wind generation in different parts of the islands.  Temperature maps are also important because the efficiency of solar panels depends on temperature.  The more accurate the weather forecasts are, the more predictable the generation.  Knowing that there it will be cloudy over the south of Oahu, but sunny over the rest of the island, or cloudy over the whole island, but with moderate to strong winds allows operations to estimate how much backup generation is required and where. 

EMS Integration

Data from new sources relating to distributed renewable generation are integrated with traditional sources in a distributed energy management system which enables operations and planning visibility into how distributed generation is impacting thte grid. It not only provides a view into the status of the grid (situational awareness) but also allows simulations to assess the impact of different future renewable generation scenarios on the grid. This helps to determine where backup generation may be required because high demand and low generation from renewables or curtailment when there is excess generation and insufficient load.  Location is fundamental to the analytics, because wind and irradiance varies widely over the islands.

Traditional input sources include SCADA and data from transmission, generators, protection, and so on. The new input data sources include GIS-based infrastructure models and data from the distributed generation sources. Also new is weather forecasting which enables predicting generation from renewable sources.  It also includes historical and actual weather reports, and satellite and other weather data.

HECO uses an IT platform from In2lytics (a spinoff from Referentia) which integrates all of these time series data with EMS, SCADA, CIS, and GIS and makes it available to operational users, planners, modelers and the public. in2lytics is a high performance time series database that is specially designed to provide instant data accessibility for planning and operational decision making.  in2lytics enables load, query, analysis using MATLAB, and sharing of the "big data" required to monitor and manage today’s complex electric grid. in2lytics has native high performance interfaces for MATLAB in addition to programming languages.  It translates data from different sources into its internal time series database. All data is archived in a spatially enabled time series database for future analysis. Matthew said that in2lytics is able to analyze large volumes of historical time series data very rapidly. For example, a longitudinal analysis on two years worth of archived data can be run in an hour or two.

Applications

Accounting for renewables requires a lot of data. Some areas that use this data include generation planning, load forecasting, distribution planning, and contract administration. One of the first applications is a customer facing web site called Renewable Watch - Oahu that reports the total renewables generation, solar and wind, on Oahu in real-time.

An example of an application that will be used for contract administration beginning next month estimates how much power is lost from Independent Power Producers (IPPs) in the case of curtailment. It uses weather information to estimate wind speed and irradiance meters to estimate wind and solar generation potential.

Another analytical application was used to study how solar PV variability affects transformer tap changes. This application allows HECO to relate frequency of tap changes to solar PV variability and to determine which transformers are most affected by solar variability.

HECO has a number of projects with the Department of Energy and other partners.

1. Department of Energy's Integrating System to Edge-of-Network Architecture and Management" (SEAMS) A federally-funded research initiative on high penetration grids which aims to streamline grid planning and operations for utilities in regions with high concentrations of distributed generation (DG) resources.

2. Department of Energy's Distributed Resource Energy Analysis and Management System (DREAMS) Department of Energy system to provide useful information about the distributed grid to grid operators.

   Partners: Siemens, Alstom, DNV, AWS Truepower, Referentia, Sunshot

   Advisory Team:SMUD, Southern California Edison, California ISO, TEP, APS, Kaua'i Island Utility Cooperative, Western APA

3. Making sense of synchrophasor data for utilities. The Synchrophasor Visual Integration and Event Evaluation for Utilities (SynchroVIEEU) with High Penetrations of Renewables. Accelerate the integration of synchrophasor information into production grade data visualization and analysis platforms/models. Leverage PMU capability at many substations – explore ways to tap resources and provide real-time visibility and real-time data. Make synchrophasor data accessible for efficient and reliable operations of a modern grid in light of high penetrations of renewable resources

   Partners: SEL, DNV GL, Referentia

4. Monitoring, planning, and modeling high PV penetration microgrid. Decision support for microgrid design and operation at Marine Corps Base Hawaii. The Office of Naval Research (ONR) has launched an ambitious program to demonstrate and evaluate energy technologies using Navy and Marine Corps facilities as test beds, known as the Energy Systems Technology and Evaluation Program (ESTEP). Program management is being handled by the Space and Naval Warfare Systems Command (SPAWAR) Systems Center Pacific (SSC Pacific). ESTEP was established in 2013. It brings together the Department of the Navy, academia, and private industry to investigate and test emerging energy technologies at Navy and Marine Corps installations. At present, ESTEP conducts over 20 in-house government energy projects, ranging from energy management to alternative energy and storage technologies.

Other projects include

• Remote light sensor technology at the Kahuku wind farm on Oahu to measure and forecast windspeed and direction to support reliability;
• Sonic detection and ranging technology to remotely measure wind speed and direction;
• Network of solar irradiance sensors to measure light and to make forecasts;
• EV Race to the Sun, a joint Japan-U.S. showcase event for electric vehicles,to take place on Haleakala;
• Battery storage technology development, including the JUMPSmartMaui project.

The ultimate objective for HECO is to become a company offering diversified services providing value to engaged customers, of course with satisfied regulators and sustainable costs and margins.  

Big data and spatial-temporal analytics

It was clear from this presentation that essential for HECO, or to any utility, with a high penetration of solar or wind are weather maps, forecasted, actual and historical because these enable the utility to project generation. The also allow utilities to analyze the historical record in conjunction with other data to discover trends that may help improve forecasting.  Collecting and analyzing temporal geospatial data is fundamental for distributed energy management.  This is in addition to the expanded application of geospatial technology for utility operations and planning in the smart grid era.