Managing heterogeneous data on a big data platform: A multi-criteria decision making model for data-intensive science

Gautam Pal, Katie Atkinson, Gangmin Li

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

9 Citations (Scopus)

Abstract

This paper presents an approach to solving the data variety problem of big data through an offline and online decisionmaking system. We present a graph-based approach to imitate real-world problem domain with a set of criteria and problem solvers. We introduce a Multi-criteria decision-making model to select a set of problem solvers that meets the set of criteria most. Suppose a system is processing Twitter data that comes as a stream of JSON records from multiple data sources. The decision system determines which of the available methods to use for a list of requirements (criteria). When multiple criteria (must meet requirements) coexist in a problem domain, their order of importance against the criteria, the mutual influence on each other and level of indispensability forms a graphic structure. In the proposed model, we consider each vertex of the graph as a criterion or benefit of an agent against the criterion. The mutual influence of multiple agents is denoted by the connecting edges of the graph. We also proposed a fuzzy graph framework to model real-world unpredictability. The model produces benchmarking results for each of the problem solvers in terms of absolute values to support decision making. The model is implemented through TopBread, Resource Description Framework (RDF), and RDF Data Query Language (RDQL). The key advantage of the proposed model over the existing ones is that the framework can operate in a dual-mode - both as a standalone offline tool and as an online decision-making gateway, it can also be used in high-velocity ingestion scenarios.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE International Conference on Big Data and Smart Computing, BigComp 2020
EditorsWookey Lee, Luonan Chen, Yang-Sae Moon, Julien Bourgeois, Mehdi Bennis, Yu-Feng Li, Young-Guk Ha, Hyuk-Yoon Kwon, Alfredo Cuzzocrea
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages229-239
Number of pages11
ISBN (Electronic)9781728160344
DOIs
Publication statusPublished - Feb 2020
Event2020 IEEE International Conference on Big Data and Smart Computing, BigComp 2020 - Busan, Korea, Republic of
Duration: 19 Feb 202022 Feb 2020

Publication series

NameProceedings - 2020 IEEE International Conference on Big Data and Smart Computing, BigComp 2020

Conference

Conference2020 IEEE International Conference on Big Data and Smart Computing, BigComp 2020
Country/TerritoryKorea, Republic of
CityBusan
Period19/02/2022/02/20

Keywords

  • 3 Vs of big data
  • Fuzzy graph
  • NoSQL databases
  • Terms - Multi criteria decision making. Multi agent systems

Fingerprint

Dive into the research topics of 'Managing heterogeneous data on a big data platform: A multi-criteria decision making model for data-intensive science'. Together they form a unique fingerprint.

Cite this