|
MultiTASC++: A continuously adaptive scheduler for edge-based multi-device cascade inference
|
Authors: Sokratis Nikolaidis, Stylianos I. Venieris, Iakovos S. Venieris
Status: Final
Date of publication: 7 March 2024
Published in: ITU Journal on Future and Evolving Technologies, Volume 5 (2024), Issue 1, Pages 26-46
Article DOI : https://doi.org/10.52953/TBYB6219
|
Abstract:
Cascade systems, consisting of a lightweight model processing all samples and a heavier, high accuracy model refining challenging samples, have become a widely-adopted distributed inference approach to achieving high accuracy and maintaining a low computational burden for mobile and IoT devices. As intelligent indoor environments, like smart homes, continue to expand, a new scenario emerges, the multi-device cascade. In this setting, multiple diverse devices simultaneously utilize a shared heavy model hosted on a server, often situated within or close to the consumer environment. This work introduces MultiTASC++, a continuously adaptive multi-tenancy-aware scheduler that dynamically controls the forwarding decision functions of devices to optimize system throughput while maintaining high accuracy and low latency. Through extensive experimentation in diverse device environments and with varying server-side models, we demonstrate the scheduler's efficacy in consistently maintaining a targeted satisfaction rate while providing the highest available accuracy across different device tiers and workloads of up to 100 devices. This demonstrates its scalability and efficiency in addressing the unique challenges of collaborative DNN inference in dynamic and diverse IoT environments. |
Keywords: Cascade, DNN inference, edge computing, multi-tenancy, scheduler
Rights: © International Telecommunication Union, available under the CC BY-NC-ND 3.0 IGO license.
|
|
| ITEM DETAIL | ARTICLE | PRICE | |
|---|
ENGLISH 
Full article (PDF) |
|
| 0
| Free of charge | DOWNLOAD |
|
|
| |
