Reinforcement Learning for Field Development Optimization

Research papers
We get used to hearing about reinforcement learning applied to solve some challenging but very distant problems from Oil and Gas like mastering GO and Dota, financial trading, robotics, etc. Many industries have high hope for RL, but it seems that the Oil and Gas community underestimates the benefits.

Reinforcement Learning is highly capable in an environment where some decision is at stake. It trains in a reward system, which provides positive/negative feedback to an agent for performing actions. An agent tries to maximize cumulative positive rewards.

I want to share with you the application of RL for better well placement. Drilling wells is a high investment procedure, and drilling to recover little or no oil is nearly catastrophic for some companies. The question of where to drill a well and a sequence of drilling steps to maximize NVP can be solved with traditional optimization algorithms like particle swarm optimization or genetic algorithm. However, those algorithms do not observe all the data changing in the reservoir (pressure, saturation, oil mobility, and accumulation, etc.) and hence do not generalize well to different reservoir conditions, economic situations, or operational constraints.

The paper "Nasir et al., 2021, Deep Reinforcement Learning for Constrained Field Development Optimization in Subsurface Two-phase Flow" presents a framework for training a reinforcement learning agent for a few million simulations run with alternating reservoir properties, economic conditions, and operational constraints to optimize drilling location and its sequence. The trained agent outperforms the well-pattern method in approximately 88% of the cases considered based on attained NVP values.

I hope that in subsequent papers, we would see the head-to-head comparison with traditional optimization algorithms in terms of optimization speed and a degree of generalization.

Kudos to the authors! Please keep up the development.

The link for the LinkedIn post