By Dr. Yu-An Sun
The Big Data analytics promise is all about plucking insights, knowledge and trends from deep reservoirs of raw data.
But the reality is that massive troves of data are very difficult for commercial and institutional computer systems to collect, sort, manage and analyze in a cost effective and speedy way. For example:
- Curating and “cleaning” data to improve quality requires access to powerful computers.
- Data analytics use complex techniques such as machine learning, visualization and cloud computing – to name a few.
- The core competencies required to process and analyze giant databases typically don’t exist in businesses, government and other organizations that want to gain insights and solve challenges with big data analytics. .
Experts at the Big Data Foundry
The Big Data Foundry pools together experts in software and hardware who can help enterprises harness their data in a more powerful and efficient manner. It was created by PARC, a Xerox company (R&D services), Cisco (IT leader), Hitachi Data Systems (big data storage) and Quantiply (enterprise architecture).Our mission is to help organizations develop innovative data analytics methods that find answers to business problems, and develop models that focus on solutions that can make a difference.
Case Use: Three Years of Smart Meter Readings on 5,000 Homes
A recent example of how the foundry works can be found in the analysis of energy meter data collected by Ireland’s Commission for Energy Regulation (CER) to assess the performance of smart meters and their impact on consumers’ energy consumption.
For more than three years, meter readings were taken every 30 minutes from more than 5,000 residences throughout Ireland. The foundry received permission from the CER to run various data analytic experiments on the massive data set. It’s important to note that the data set was stripped of personal information to maintain resident privacy.
Read Why and How to Make Data Privacy a Business Priority.
Download a free copy of Big Data in 2015: A European Study and turn to page 20. The research covered by this report, conducted by Forrester Research on behalf of Xerox, reveals:The breadth of the plans that most European corporations have for implementing big data initiatives.
Developments that will have a transformative effect on how businesses will make decisions.
The services they will offer in the near future.
One of the experiments involved determining consumption patterns across the 5,000 residences. Mining this sort of information from the massive data set might be useful to help anticipate strains on an energy grid that could possible cause an outage.
But before an experiment like this could begin, the Irish data had to be imported onto the cloud and parts of it had to be “cleaned” to make it machine-readable. Managing and analyzing the data for this single experiment was forecasted to take a week. But, when worked through resources at the Big Data Foundry, amount of time dropped to just two to three days (see chart below).
The efficiency and “one-stop-shop” approach to collecting, sorting, managing and analyzing data sets makes it more likely that the information is used. PARC data scientists now are exploring how to automate the analysis, making it easier for someone like a business analyst to glean insight into energy use.
The Big Data Foundry removes the barriers to exploring and implementing new Big Data technologies. You can engage world-class data scientists, data engineers, cloud experts, innovators, and ethnographers all in one place to help you discover new routes to revenue and profitability in your organization.
To learn more contact Dr. Yu-An Sun (YuAn.sun@xerox.com)
Subscribe to this blog and receive email updates when we publish a new article.
