CDES: An approach to HPC workload modelling

John Brennan; Ibad Kureshi; Violeta Holmes

doi:10.1109/DS-RT.2014.15

CDES: An approach to HPC workload modelling

John Brennan, Ibad Kureshi, Violeta Holmes

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

Computational science and complex system administration relies on being able to model user interactions. When it comes to managing HPC, HTC and grid systems user workloads - their job submission behaviour, is an important metric when designing systems or scheduling algorithms. Most simulators are either inflexible or tied in to proprietary scheduling systems. For system administrators being able to model how a scheduling algorithm behaves or how modifying system configurations can affect the job completion rates is critical. Within computer science research many algorithms are presented with no real description or verification of behaviour. In this paper we are presenting the Cluster Discrete Event Simulator (CDES) as an strong candidate for HPC workload simulation. Built around an open framework, CDES can take system definitions, multi-platform real usage logs and can be interfaced with any scheduling algorithm through the use of an API. CDES has been tested against 3 years of usage logs from a production level HPC system and verified to a greater than 95% accuracy.

Original language	English
Title of host publication	Proceedings - IEEE International Symposium on Distributed Simulation and Real-Time Applications, DS-RT
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	47-54
Number of pages	8
ISBN (Electronic)	9781479961443
DOIs	https://doi.org/10.1109/DS-RT.2014.15
Publication status	Published - 13 Nov 2014
Event	18th IEEE/ACM International Symposium on Distributed Simulations and Real Time Applications - Aeronautics and Space Institute, Toulouse, France Duration: 1 Oct 2014 → 3 Oct 2014 Conference number: 18 http://ds-rt.com/2014/ (Link to Conference Website)

Conference

Conference	18th IEEE/ACM International Symposium on Distributed Simulations and Real Time Applications
Abbreviated title	DS-RT 2014
Country/Territory	France
City	Toulouse
Period	1/10/14 → 3/10/14
Internet address	http://ds-rt.com/2014/ (Link to Conference Website)

Access to Document

10.1109/DS-RT.2014.15Licence: Other

Cite this

@inproceedings{16850404869c4646b50761a3906f917c,

title = "CDES: An approach to HPC workload modelling",

abstract = "Computational science and complex system administration relies on being able to model user interactions. When it comes to managing HPC, HTC and grid systems user workloads - their job submission behaviour, is an important metric when designing systems or scheduling algorithms. Most simulators are either inflexible or tied in to proprietary scheduling systems. For system administrators being able to model how a scheduling algorithm behaves or how modifying system configurations can affect the job completion rates is critical. Within computer science research many algorithms are presented with no real description or verification of behaviour. In this paper we are presenting the Cluster Discrete Event Simulator (CDES) as an strong candidate for HPC workload simulation. Built around an open framework, CDES can take system definitions, multi-platform real usage logs and can be interfaced with any scheduling algorithm through the use of an API. CDES has been tested against 3 years of usage logs from a production level HPC system and verified to a greater than 95% accuracy.",

keywords = "HPC, HPC simulator, scheduler, WMS, workload modelling",

author = "John Brennan and Ibad Kureshi and Violeta Holmes",

year = "2014",

month = nov,

day = "13",

doi = "10.1109/DS-RT.2014.15",

language = "English",

pages = "47--54",

booktitle = "Proceedings - IEEE International Symposium on Distributed Simulation and Real-Time Applications, DS-RT",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "18th IEEE/ACM International Symposium on Distributed Simulations and Real Time Applications, DS-RT 2014 ; Conference date: 01-10-2014 Through 03-10-2014",

url = "http://ds-rt.com/2014/",

}

Brennan, J, Kureshi, I & Holmes, V 2014, CDES: An approach to HPC workload modelling. in Proceedings - IEEE International Symposium on Distributed Simulation and Real-Time Applications, DS-RT., 6957176, Institute of Electrical and Electronics Engineers Inc., pp. 47-54, 18th IEEE/ACM International Symposium on Distributed Simulations and Real Time Applications, Toulouse, France, 1/10/14. https://doi.org/10.1109/DS-RT.2014.15

CDES : An approach to HPC workload modelling. / Brennan, John; Kureshi, Ibad; Holmes, Violeta.

Proceedings - IEEE International Symposium on Distributed Simulation and Real-Time Applications, DS-RT. Institute of Electrical and Electronics Engineers Inc., 2014. p. 47-54 6957176.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - CDES

T2 - 18th IEEE/ACM International Symposium on Distributed Simulations and Real Time Applications

AU - Brennan, John

AU - Kureshi, Ibad

AU - Holmes, Violeta

N1 - Conference code: 18

PY - 2014/11/13

Y1 - 2014/11/13

N2 - Computational science and complex system administration relies on being able to model user interactions. When it comes to managing HPC, HTC and grid systems user workloads - their job submission behaviour, is an important metric when designing systems or scheduling algorithms. Most simulators are either inflexible or tied in to proprietary scheduling systems. For system administrators being able to model how a scheduling algorithm behaves or how modifying system configurations can affect the job completion rates is critical. Within computer science research many algorithms are presented with no real description or verification of behaviour. In this paper we are presenting the Cluster Discrete Event Simulator (CDES) as an strong candidate for HPC workload simulation. Built around an open framework, CDES can take system definitions, multi-platform real usage logs and can be interfaced with any scheduling algorithm through the use of an API. CDES has been tested against 3 years of usage logs from a production level HPC system and verified to a greater than 95% accuracy.

AB - Computational science and complex system administration relies on being able to model user interactions. When it comes to managing HPC, HTC and grid systems user workloads - their job submission behaviour, is an important metric when designing systems or scheduling algorithms. Most simulators are either inflexible or tied in to proprietary scheduling systems. For system administrators being able to model how a scheduling algorithm behaves or how modifying system configurations can affect the job completion rates is critical. Within computer science research many algorithms are presented with no real description or verification of behaviour. In this paper we are presenting the Cluster Discrete Event Simulator (CDES) as an strong candidate for HPC workload simulation. Built around an open framework, CDES can take system definitions, multi-platform real usage logs and can be interfaced with any scheduling algorithm through the use of an API. CDES has been tested against 3 years of usage logs from a production level HPC system and verified to a greater than 95% accuracy.

KW - HPC

KW - HPC simulator

KW - scheduler

KW - WMS

KW - workload modelling

UR - http://www.scopus.com/inward/record.url?scp=84913555795&partnerID=8YFLogxK

U2 - 10.1109/DS-RT.2014.15

DO - 10.1109/DS-RT.2014.15

M3 - Conference contribution

AN - SCOPUS:84913555795

SP - 47

EP - 54

BT - Proceedings - IEEE International Symposium on Distributed Simulation and Real-Time Applications, DS-RT

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 1 October 2014 through 3 October 2014

ER -

CDES: An approach to HPC workload modelling

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this