Page 547 - AI for Good Innovate for Impact

AI for Good Innovate for Impact

(continued)

Item Details
Model Training Effective shape and position control via magnetic system is provided with
and Fine-Tuning the use of Reinforcement Learning algorithms, mainly Soft Actor-Critic. Our 4.5: Manufacturing
approach leverages privileged information during training by equipping the
critic with access to full plasma state information without diagnostics noise
that is unavailable to the actor at inference time.
Tweaks and employed techniques include curriculum learning and plasma
configuration randomization for better domain transfer.
To integrate this into operational settings, we implement real-time plasma
control system (PCS) modules in C and C++ (depending on the target fusion
device), ensuring performance, reliability, and compatibility with existing
control infrastructure.
Testbeds or Pilot More information on the pilot deployment can be found at [2]
Deployments
Code reposito- Not available
ries

2 Use Case Description

2�1 Description

Fusion energy is a promising clean and sustainable energy solution, offering significant potential
for reducing greenhouse gas emissions on a large scale. Fusion Power Plants (FPPs) represent
the next frontier in energy production. Flexible and robust control of FPPs requires managing
the control object - plasma - under varying technical contexts, including diagnostic and material
degradation, failures, and evolving conditions. Sustaining and efficiently controlling a steady-
state fusion reaction in an FPP remains one of the most critical challenges in the fusion energy
industry. Existing plasma control systems are designed to satisfy specific requirements of
experimental devices, maintaining testing plasma discharges with finely tuned parameters.
FPP will be operating in long discharges, guided by a reduced set of available measurements,
which makes the conventional control approach less reliable and precise, creating a significant
barrier to the commercial adoption of fusion energy.

Machine learning (ML) methods, particularly those based on reinforcement learning (RL), offer
a promising alternative. To address this challenge, we are developing an innovative machine
learning-based Plasma Control System (PCS) for both current and next-generation fusion
devices, as well as future fusion power plants (FPPs). Our PCS is based on a machine-agnostic,
plasma-state-oriented control approach and uses our in-house technology stack: an advanced
NSFsim tokamak simulator and RL toolkit.

In this use case, we are developing the RL agent, a key component of PCS, for controlling
plasma shape and position. This approach offers a promising alternative to conventional control
methods by training end-to-end controllers that do not require additional data processing or
task decomposition. The RL-based approach is faster to train and more adaptable to different
plasma shapes, requiring no adjustments to training parameters, compared to conventional
systems with PID controllers. During 2023-24, as part of the DIII-D National Fusion Facility
Program, we applied RL algorithms for plasma parameter control on the DIII-D tokamak. Our

511