Deep Reinforcement Learning for Flexible Job Shop S... | AI Research

Deep Reinforcement Learning for Flexible Job Shop Scheduling with Random Job Arrivals
This paper addresses the Flexible Job Shop Scheduling Problem (FJSP), which involves assigning a sequence of tasks to various machines to minimize the total time required to complete all jobs. This task is notoriously difficult because it involves complex combinations of choices and must account for the unpredictable arrival of new jobs in real-time. The authors propose a new approach using Deep Reinforcement Learning (DRL) to make these scheduling decisions dynamically, aiming to outperform traditional methods that struggle with the complexity of modern, uncertain production environments.

How the Approach Works

The researchers model the scheduling problem as a Markov Decision Process, where an AI agent observes the state of the shop floor—such as which jobs are waiting and which machines are free—and makes decisions at specific events, like when a job arrives or an operation is finished.
Instead of trying to calculate every possible move from scratch, the agent is trained to select from a set of well-established "dispatching rules." These rules are proven heuristics that prioritize jobs or assign them to machines based on criteria like processing time or arrival order. By using the Proximal Policy Optimization (PPO) algorithm, the agent learns which combination of these rules works best for the current situation. The system uses lightweight neural networks to process this information, keeping the model efficient and easier to train than more complex, resource-heavy alternatives.

Key Results

The team tested their DRL approach against both individual dispatching rules and an arrival-triggered mixed-integer linear programming (AT-MILP) method. Simulations showed that the DRL agent consistently outperformed any single dispatching rule across various scenarios.
A particularly notable finding is the agent's performance on "heterogeneous" datasets. In real-world factories, jobs and machines are rarely identical; some jobs are much longer than others, and some machines are faster than others. The researchers found that their DRL method was especially effective in these complex, varied environments, providing high-quality schedules where traditional optimization methods might be too slow or less adaptable.

Considerations and Limitations

While the results are promising, the study highlights a few important factors. First, the agent’s performance is dependent on the set of dispatching rules provided to it; while the agent can choose the best rule for a given moment, it is still constrained by the quality of those rules.
Additionally, the researchers noted that they did not perform an exhaustive grid search for every hyperparameter due to the high computational cost of training, suggesting that further tuning might yield even better results. Finally, the framework is designed specifically for scenarios where processing times are known and deterministic, and it assumes that machine setup times and the time required to transport materials between machines are negligible.