Page 99 - AI Standards for Global Impact: From Governance to Action

AI Standards for Global Impact: From Governance to Action Part 2: Thematic AI

Figure 43: Evolution of NPU reference chips of IntelliFusion

13�4�7 Day 0 support for novel models enabling instantaneous deployment
of edge AI

AI compiler technology has become mission-critical for deploying AI models at scale on edge
devices. The usual trend is for solutions to build on a legacy software stack that interprets AI
models but does not allow for compilation. This limits the application of the technology for AI
models, such as language models, as they are not compatible with the existing technology stack.

AI compilation optimizes model execution at different levels of abstraction – known as
"intermediate representations" - that represent specific features of the execution of an AI
workload. The compiler translates the AI model through various intermediate representations
to a low level that is close to the target hardware. Using AI compilation instead of model
interpretation allows users to adapt to the constant stream of new AI models.

Roofline’s flexible SDK enables models to be imported from any AI framework such as
TensorFlow, PyTorch, or ONNX. It enables deployment across diverse hardware, covering
CPUs, MPUs, MCUs, GPUs, and dedicated AI hardware accelerators - all with just a single line
of Python. Roofline’s flexible SDK that enables day-0 support for PyTorch models found on
Hugging Face, which solves the challenge of the inability to support emerging new AI models
and layer types, due to specific software features not compatible across systems.

13�4�8 Defining the boundaries of AI from fundamental limitations to resource
constraints

CSEM mentioned the challenges and solutions in deploying AI at the edge, focusing on the
limitations posed by neural scaling laws and the memory wall. The need for hardware-aware
AI design, efficient feature selection, and adversarial training to overcome resource constraints
while maintaining performance was highlighted. The proposed strategies enable scalable,
private, and energy-efficient AI solutions tailored for edge computing environments.

87