An actor-critic approach for control of residential photovoltaic-battery systems

The rationale of shifting towards green energy, along with the cost reduction and the increasing capacity of lithium-ion batteries, has motivated the end-users to go for energy storage systems integrated with solar technology solutions. Such systems provide the end-users with greater flexibility, thereby enhancing their role as prosumers in a range of grid-management programs. In this regard, we consider a residential household equipped with a battery and photovoltaic panels, collectively known as the photovoltaic-battery (PV-B) system. We further learn (off-line) a deterministic sub-optimal policy for charging/discharging of the residential battery using an actor-critic reinforcement learning based method. Such proposed approach, named polynomial deterministic policy gradient (PDPG), does not require any model of the system and uses polynomials as function approximator, as opposed to conventional neural networks. The usefulness of the proposed approach is tested on real power data (demand and PV generation) of a residential household in Australia. Numerical simulations indicate that the proposed PDPG algorithm outperforms the OFFON control approach in terms of electricity bill savings and the model-based receding horizon control in terms of computation time.