AI agents in Algorithmic Electricity Markets: On th... | AI Research

Jakub Seredyński and Georgios Tsaousoglou investigate whether autonomous AI agents, used for bidding in electricity markets, can independently learn to engage in "tacit collusion." Tacit collusion occurs when market participants coordinate to raise prices or increase profits without an explicit agreement or communication. The researchers hypothesize that because electricity markets are often oligopolistic and involve repeated interactions, learning-based algorithms may autonomously discover and sustain supra-competitive outcomes that harm consumers.

Assessing Tacit Collusion

The authors argue that identifying collusion is difficult because high profits alone do not prove coordination. To address this, they propose a multi-dimensional framework to evaluate whether agent behavior is truly collusive:

• Punishment of Deviation: This checks if agents respond to a competitor’s attempt to act more competitively by temporarily lowering their own bids to reduce the deviator's profit, eventually returning to the original strategy once the deviator complies.

• Unsustainability under Shortsight: This tests whether the collusive behavior disappears when agents are forced to act with limited memory and a low discount factor, which prioritizes immediate gains over long-term coordination.

• Short-term Profitability of Deviations: This examines if agents have profitable unilateral deviations that they choose not to execute because they anticipate that such moves would trigger a competitive response, ultimately leading to lower long-term profits.

Modeling Market Behavior

The researchers use Multi-Agent Reinforcement Learning (MARL) to simulate electricity market participants. In this model, agents learn bidding strategies through trial and error based on past observations, such as their own dispatch, profit, and the Locational Marginal Price (LMP) at their node. The agents operate in a repeated game with imperfect monitoring, meaning they do not see the specific actions or profits of their competitors, only the public market signals. The market itself is cleared using a standard DC optimal power flow (OPF) problem, which accounts for network constraints and transmission line capacities.

Experimental Findings

The study indicates that tacit collusion is a realistic risk in algorithmic electricity markets. Experimental results show that agents can learn to sustain supra-competitive outcomes—where profits exceed those found in Nash equilibria—without ever being instructed to collude. The authors note that these outcomes emerge purely from the agents' independent learning processes. However, they also observe that not all supra-competitive outcomes are necessarily collusive; some may arise from factors unrelated to coordination.

Limitations and Considerations

The researchers emphasize that identifying tacit collusion is a subtle task that cannot rely on a single metric. A primary limitation identified is that previous methods for detecting collusion, such as comparing profits against Nash equilibria, are difficult to apply to complex, realistic electricity markets. By moving beyond simple profit comparisons, the authors aim to provide a more robust way to analyze market risks. The study suggests that factors such as grid constraint tightness, demand levels, and the variance of marginal costs may influence the likelihood of these collusive behaviors emerging.