Recommendation ITU-T F.748.12 (06/2021) Deep learning software framework evaluation methodology
Summary
History
FOREWORD
Table of Contents
1 Scope
2 References
3 Definitions
     3.1 Terms defined elsewhere
     3.2 Terms defined in this Recommendation
4 Abbreviations and acronyms
5 Conventions
6 Industrial realization of deep learning software framework in artificial intelligence
     6.1 Application architecture based on deep learning software framework
     6.2 Hardware
     6.3 Compilers
     6.4 Frameworks
     6.5 Fundamental applications
     6.6 Industry applications
7 Indicating requirements and evaluation methods of deep learning software framework
     7.1 Requirements overview
     7.2 Classic deep learning models
     7.3 Specific indicating items and evaluation methods for training framework
          7.3.1 Ecological construction
               7.3.1.1 Interface
               7.3.1.2 Core developers and contributors
               7.3.1.3 The situation in which issues are solved
          7.3.2 Ease of use
               7.3.2.1 Model building and conversion
               7.3.2.2 Secondary development based on high-level language
               7.3.2.3 Custom extension
               7.3.2.4 Cross-platform
               7.3.2.5 Model library support
               7.3.2.6 Tutorial documentation and training materials
               7.3.2.7 Dynamic graph and static graph
               7.3.2.8 Stability
               7.3.2.9 Debuggability
          7.3.3 Performance
               7.3.3.1 Model library operating performance
               7.3.3.2 Operating performance of a customized model
               7.3.3.3 Hardware acceleration support
          7.3.4 Supported architecture
               7.3.4.1 CPU/FPGA
               7.3.4.2 Single GPU/multi-GPU
               7.3.4.3 Distributed training
               7.3.4.4 Virtual environment support
               7.3.4.5 Operating systems support
          7.3.5 Security and stability
               7.3.5.1 Usage of third-party library
               7.3.5.2 Data security
     7.4 Specific indicating items and evaluation methods for inference framework
          7.4.1 Ease of use
               7.4.1.1 Model optimizing functionality
               7.4.1.2 Universal model representation
               7.4.1.3 Cross-platform
          7.4.2 Performance
               7.4.2.1 Inference speed
               7.4.2.2 Run-up speed
               7.4.2.3 System resources occupation
               7.4.2.4 Energy consumption
          7.4.3 Underlying optimization
               7.4.3.1 Support for different underlying hardware
               7.4.3.2 Optimization for instruction set
          7.4.4 Security and stability
               7.4.4.1 Model encryption
<\pre>