Large spatial datasets

Present Challenges, future opportunities

Research output: Contribution to journalConference article

Abstract

The key advantages of a well-designed multidimensional database is its ability to allow as many users as possible across an organisation to simultaneously gain access and view of the same data. Large spatial datasets evolve from scientific activities (from recent days) that tends to generate large databases which always come in a scale nearing terabyte of data size and in most cases are multidimensional. In this paper, we look at the issues pertaining to large spatial datasets; its feature (for example views), architecture, access methods and most importantly design technologies. We also looked at some ways of possibly improving the performance of some of the existing algorithms for managing large spatial datasets. The study reveals that the major challenges militating against effective management of large spatial datasets is storage utilization and computational complexity (both of which are characterised by the size of spatial big data which now tends to exceeds the capacity of commonly used spatial computing systems owing to their volume, variety and velocity). These problems fortunately can be combated by employing functional programming method or parallelization techniques.
Original languageEnglish
Pages (from-to)204-217
Number of pages14
JournalICCIIDT 2016
Publication statusPublished - 24 Feb 2017
EventInternational Conference on Change, Innovation, Informatics and Disruptive Technology - Sandell St, London, United Kingdom
Duration: 11 Oct 201612 Oct 2016
http://sriweb.org/londonconf/ (Link to Conference Website)

Fingerprint

Functional programming
Computational complexity
Big data

Cite this

@article{f3c5121d93254722af13bfed2fc11dad,
title = "Large spatial datasets: Present Challenges, future opportunities",
abstract = "The key advantages of a well-designed multidimensional database is its ability to allow as many users as possible across an organisation to simultaneously gain access and view of the same data. Large spatial datasets evolve from scientific activities (from recent days) that tends to generate large databases which always come in a scale nearing terabyte of data size and in most cases are multidimensional. In this paper, we look at the issues pertaining to large spatial datasets; its feature (for example views), architecture, access methods and most importantly design technologies. We also looked at some ways of possibly improving the performance of some of the existing algorithms for managing large spatial datasets. The study reveals that the major challenges militating against effective management of large spatial datasets is storage utilization and computational complexity (both of which are characterised by the size of spatial big data which now tends to exceeds the capacity of commonly used spatial computing systems owing to their volume, variety and velocity). These problems fortunately can be combated by employing functional programming method or parallelization techniques.",
keywords = "Spatial Database, Large Datasets, Hadoop, Cloud, Map-Reduce, Bulk-Loading",
author = "Grace Samson and Joan Lu and Qiang Xu",
year = "2017",
month = "2",
day = "24",
language = "English",
pages = "204--217",
journal = "ICCIIDT 2016",
issn = "2476-017X",

}

Large spatial datasets : Present Challenges, future opportunities. / Samson, Grace; Lu, Joan; Xu, Qiang.

In: ICCIIDT 2016, 24.02.2017, p. 204-217.

Research output: Contribution to journalConference article

TY - JOUR

T1 - Large spatial datasets

T2 - Present Challenges, future opportunities

AU - Samson, Grace

AU - Lu, Joan

AU - Xu, Qiang

PY - 2017/2/24

Y1 - 2017/2/24

N2 - The key advantages of a well-designed multidimensional database is its ability to allow as many users as possible across an organisation to simultaneously gain access and view of the same data. Large spatial datasets evolve from scientific activities (from recent days) that tends to generate large databases which always come in a scale nearing terabyte of data size and in most cases are multidimensional. In this paper, we look at the issues pertaining to large spatial datasets; its feature (for example views), architecture, access methods and most importantly design technologies. We also looked at some ways of possibly improving the performance of some of the existing algorithms for managing large spatial datasets. The study reveals that the major challenges militating against effective management of large spatial datasets is storage utilization and computational complexity (both of which are characterised by the size of spatial big data which now tends to exceeds the capacity of commonly used spatial computing systems owing to their volume, variety and velocity). These problems fortunately can be combated by employing functional programming method or parallelization techniques.

AB - The key advantages of a well-designed multidimensional database is its ability to allow as many users as possible across an organisation to simultaneously gain access and view of the same data. Large spatial datasets evolve from scientific activities (from recent days) that tends to generate large databases which always come in a scale nearing terabyte of data size and in most cases are multidimensional. In this paper, we look at the issues pertaining to large spatial datasets; its feature (for example views), architecture, access methods and most importantly design technologies. We also looked at some ways of possibly improving the performance of some of the existing algorithms for managing large spatial datasets. The study reveals that the major challenges militating against effective management of large spatial datasets is storage utilization and computational complexity (both of which are characterised by the size of spatial big data which now tends to exceeds the capacity of commonly used spatial computing systems owing to their volume, variety and velocity). These problems fortunately can be combated by employing functional programming method or parallelization techniques.

KW - Spatial Database

KW - Large Datasets

KW - Hadoop

KW - Cloud

KW - Map-Reduce

KW - Bulk-Loading

UR - http://proceedings.sriweb.org/repository/index.php/ICCIIDT/icciidtt_london/paper/view/24

M3 - Conference article

SP - 204

EP - 217

JO - ICCIIDT 2016

JF - ICCIIDT 2016

SN - 2476-017X

ER -