Data loading and mapping using staging DBMS in the grid

Ejaz Ahmed*, Nik Bessis, Yong Yue, Des Stephens

*Corresponding author for this work

Research output: Chapter in Book or Report/Conference proceedingConference Proceedingpeer-review

5 Citations (Scopus)

Abstract

With the proliferation of international standards for grid-enabled databases, the need for data loading and data mapping in a large integrated environment of heterogeneous databases highlights issues of consistency and integrity. We discuss methods for providing semi-autonomous data integration by focusing on efficient data loading design and effective mapping strategies. In order to upload and integrate data from various small to large size data repositories, an intermediate staging facility is employed to temporarily store data before it is validated to ensure accurate and useful integration into the target data source. We propose a mechanism that semi-automates the integration process that includes not only new data, but legacy data as well. We expand the notation of a database management system (DBMS) to include the management of the data transfer or data transformation processes. The DBMS now must perform the task of data mapping in the form of value correspondences by using a staging schema or staging DBMS mapping procedure.

Original languageEnglish
Title of host publicationIEEE Canadian Conference on Electrical and Computer Engineering, Proceedings, CCECE 2008
Pages1887-1891
Number of pages5
DOIs
Publication statusPublished - 2008
Externally publishedYes
EventIEEE Canadian Conference on Electrical and Computer Engineering, CCECE 2008 - Niagara Falls, ON, Canada
Duration: 4 May 20087 May 2008

Publication series

NameCanadian Conference on Electrical and Computer Engineering
ISSN (Print)0840-7789

Conference

ConferenceIEEE Canadian Conference on Electrical and Computer Engineering, CCECE 2008
Country/TerritoryCanada
CityNiagara Falls, ON
Period4/05/087/05/08

Keywords

  • Components
  • Data integration
  • Global schema
  • Grid
  • Staging DBMS
  • Staging schema

Cite this