Process Controller in DQN Method

ZHUANG SHAO, FENG-QI SI, ZHI-GAO XU, DANIEL KUDENKO

Abstract


Because of its natural adaptive ability and flexibility in designing of control objective, reinforcement learning provides new ideas for process industry control and has the hope of achieving a wide range of applications. Process industry objects have the characteristics of continuous state, continuous action, non-episodic, etc., and the system state is irreversible. Meanwhile, serious problem of data imbalance appears in the sampling from the state trajectory. All of these make it a difficult problem. This paper designs a novel data processing and controller framework based on deep Q-learning combined with the data management method of condition library. The framework firstly collects time series data from the interaction between the agent and the environment, then updates the data in batches in the condition library, and uses the queuing mechanism to ensure the balance of the training data for Q function. In order to ensure the execution speed of the controller, the policy function is designed. This paper uses the classical water level control as an example to verify the effectiveness of the proposed framework.

Keywords


Reinforcement learning, DQN, Process control, Condition library.Text


DOI
10.12783/dtetr/pmsms2018/24909

Full Text:

PDF

Refbacks

  • There are currently no refbacks.