A hierarchical reinforcement learning-based vehicle-to-grid dispatch architecture for car parks: Integrating proximal policy optimisation and a large language model within a dual-agent framework

This study proposes a comprehensive optimisation scheduling framework for large-scale vehicle-to-grid (V2G) scenarios, integrating deep reinforcement learning (DRL) with large language model (LLM) feedback mechanisms. The framework aims to coordinate bidirectional energy flows within a parking cluster comprising 300 electric vehicles and two battery swap stations, maximising operational revenue while ensuring battery health and charging/discharging reliability. For the constructed V2G car park scheduling model, a dual-agent hierarchical Proximal Policy Optimisation (DHPPO) algorithm architecture is designed: the upper layer employs a single-agent PPO algorithm, enhancing decision performance through dynamic reward adjustments guided by the LLM agent; the lower layer implements deterministic convex optimisation via CVXPY, ensuring the feasibility of scheduling outcomes and compliance with constraints. Furthermore, to enhance scalability and computational efficiency, electric vehicles within the car park are grouped into six clusters of 50 vehicles each, each receiving action commands from the upper-layer algorithm to achieve group-based collaborative optimisation. Simulation results demonstrate: (1) Compared to the baseline DQN algorithm, this method increases overall V2G economic benefits by 19.74%; (2) Battery charging frequency decreased by approximately 25%, extending the expected lifespan of electric vehicle batteries; (3) The proposed hierarchical PPO-LLM algorithm accelerates reward convergence under dynamic conditions while maintaining stable grid interaction. These results demonstrate the effectiveness of the proposed algorithm in enhancing V2G efficiency, reducing charging expenditure, and preserving battery health.