Decision-driven fault-tolerant architecture for vision transformers with real-time error mitigation

Authors: Indhuja Gudluru, Chunyuan Shen, Ke Wang
Status: Final
Date of publication: 25 June 2025
Published in: ITU Journal on Future and Evolving Technologies, Volume 6 (2025), Issue 2, Pages 198-214
Article DOI : https://doi.org/10.52953/ZBPE9349

Abstract:
Vision Transformers (ViTs) have evolved in the field of computer vision by transitioning traditional Convolutional Neural Networks (CNNs) into attention-based architectures. This architecture processes input images as sequences of patches. ViTs achieve enhanced performance in many tasks such as image classification and object detection due to their ability to capture global dependencies within input data. While their software implementations are widely adopted, deploying ViTs on hardware introduces several challenges. These include fault tolerance in the presence of hardware failures, real-time reliability, and high computational requirements. Permanent faults that are in processing elements, interconnections, or memory subsystems lead to incorrect computations and degrading system performance. This paper proposes a fault-tolerant hardware implementation of ViTs to overcome these challenges. This hardware implementation integrates real-time fault detection and recovery mechanisms. The architecture includes four primary units: patch embedding, encoder, decoder, and Multi Layer Perceptron (MLP) which are supported by fault-tolerant components such as lightweight recompute units, a centralized Built-In Self-Test (BIST), and a learning-based decision-making system using machine learning model 'decision tree'. These units are interconnected through a centralized global buffer for efficient data transfer, ensuring seamless operation even under fault conditions.

Keywords: Decision tree, fault tolerance, hardware Accelerator, vision transformers
Rights: © International Telecommunication Union, available under the CC BY-NC-ND 3.0 IGO license.