FPGA-Based Dynamic Deep Learning Acceleration for Real-Time Video Analytics

Yufan Lu, Cong Gao, Rappy Saha, Sangeet Saha, Klaus D. McDonald-Maier, Xiaojun Zhai

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)


Deep neural networks (DNNs) are a key technique in modern artificial intelligence that has provided state-of-the-art accuracy on many applications, and they have received significant interest. The requirements for ubiquity of smart devices and autonomous robot systems are placing heavy demands on DNNs-inference hardware, with high requirement for energy and computing efficiencies, along with the rapid development of AI techniques. The high energy efficiency, computing capabilities, and reconfigurability of FPGAs make these a promising platform for hardware acceleration of such computing tasks. This paper primarily addresses this challenge and proposes a new flexible hardware accelerator framework to enable adaptive support for various DL algorithms on an FPGA-based edge computing platform. This framework allows run-time reconfiguration to increase power and computing efficiency of both DNN model/software and hardware, to meet the requirements of dedicated application specifications and operating environments. The achieved results show that with the proposed framework is capable to reduce energy consumption and processing time up to 53.8% and 36.5% respectively by switching to a smaller model. In addition, the time and energy consumption are further elaborated with a benchmark test set, which shows that how input data in each frame and size of a model can affect the performance of the system.

Original languageEnglish
Title of host publicationArchitecture of Computing Systems
Subtitle of host publication35th International Conference, ARCS 2022, Heilbronn, Germany, September 13–15, 2022, Proceedings
EditorsMartin Schulz, Carsten Trinitis, Nikela Papadopoulou, Thilo Pionteck
PublisherSpringer, Cham
Number of pages15
Volume13642 LNCS
ISBN (Electronic)9783031218675
ISBN (Print)9783031218668
Publication statusPublished - 14 Dec 2022
Externally publishedYes
Event35th International Conference on Architecture of Computing Systems - Heilbronn, Germany
Duration: 13 Sep 202215 Sep 2022
Conference number: 35

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13642 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference35th International Conference on Architecture of Computing Systems
Abbreviated titleARCS 2022


Dive into the research topics of 'FPGA-Based Dynamic Deep Learning Acceleration for Real-Time Video Analytics'. Together they form a unique fingerprint.

Cite this