Double deep reinforcement learning twin-delayed agents for performance improvement of a grid-connected wave energy conversion system

Abstract

This study introduces a novel approach using two deep learning agents, trained with the twin-delayed deep deterministic policy gradient (TD3) algorithm, to replace the PI controllers used for the control of grid-connected Archimedes Wave Swing (AWS) wave energy conversion systems. The generator converter’s controller has two mandatory objectives: minimizing losses in the stator and maximizing energy extraction from incident sea waves. These goals are achieved by controlling the generator’s dq currents using a TD3 agent on the rectifier side. In addition, the grid-side inverter’s controller is responsible for regulating both the DC link and the point-of-common-coupling voltages. In the new configuration, two approaches are proposed in this work: either a single deep learning agent replaces the four proportional-integral (PI) controllers on the inverter side, or a hybrid approach combining two PI controllers with a TD3 agent. To verify the reliability of the TD3 agents, the system is analyzed in both steady and transient states under fault conditions. Furthermore, the TD3 agents’ performance is benchmarked against the classical PI controller configuration in MATLAB Simulink. The results demonstrate better dynamic and steady-state responses from the hybrid-TD3 agent on the grid side than from the full PI classical configuration.