TY - JOUR
T1 - A Big Data platform for smart meter data analytics
AU - Wilcox, Tom
AU - Jin, Nanlin
AU - Flach, Peter
AU - Thumim, Joshua
N1 - Publisher Copyright:
© 2018 Elsevier B.V.
PY - 2019/2
Y1 - 2019/2
N2 - Smart grids have started generating an ever increasingly large volume of data. Extensive research has been done in meter data analytics for small data sets of electrical grid and electricity consumption. However limited research has investigated the methods, systems and tools to support data storage and data analytics for big data generated by smart grids. This work has proposed a new core-broker-client system architecture for big data analytics. Its implemented platform is named Smart Meter Analytics Scaled by Hadoop (SMASH). Our work has demonstrated that SMASH is able to perform data storage, query, analysis and visualization tasks on large data sets at 20 TB scale. The performance of SMASH in storing and querying large quantities of data are compared with the published results provided by Google Cloud, IBM, MongoDB, and AMPLab. The experimental results suggest that SMASH provides industry a competitive and easily operable platform to manage big energy data and visualize knowledge, with potential to support data-intensive decision making.
AB - Smart grids have started generating an ever increasingly large volume of data. Extensive research has been done in meter data analytics for small data sets of electrical grid and electricity consumption. However limited research has investigated the methods, systems and tools to support data storage and data analytics for big data generated by smart grids. This work has proposed a new core-broker-client system architecture for big data analytics. Its implemented platform is named Smart Meter Analytics Scaled by Hadoop (SMASH). Our work has demonstrated that SMASH is able to perform data storage, query, analysis and visualization tasks on large data sets at 20 TB scale. The performance of SMASH in storing and querying large quantities of data are compared with the published results provided by Google Cloud, IBM, MongoDB, and AMPLab. The experimental results suggest that SMASH provides industry a competitive and easily operable platform to manage big energy data and visualize knowledge, with potential to support data-intensive decision making.
KW - Big data
KW - Meter data analytics
KW - Smart grid
UR - http://www.scopus.com/inward/record.url?scp=85060310485&partnerID=8YFLogxK
U2 - 10.1016/j.compind.2018.12.010
DO - 10.1016/j.compind.2018.12.010
M3 - Article
AN - SCOPUS:85060310485
SN - 0166-3615
VL - 105
SP - 250
EP - 259
JO - Computers in Industry
JF - Computers in Industry
ER -