Intelligent Trader Model Based on Deep Reinforcement Learning

• Published in: Web Information Systems and Applications Vol. 11817; pp. 15 - 21
• Main Authors: Han, Daoqi, Zhang, Junyao, Zhou, Yuhang, Liu, Qing, Yang, Nan
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2019; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Deep reinforcement learning; DQN; One-step deterministic action value; Quantization strategy
• ISBN: 9783030309510; 3030309517
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-30952-7_2

The stock market has the characteristics of changing rapidly, having many interference factors, and yielding insufficient period data. Stock trading is a game process under incomplete information, and the single-objective supervised learning model is difficult to deal with such serialization decision problems. Reinforcement learning is one of the effective ways to solve these problems. This paper proposes an ISTG model (Intelligent Stock Trader and Gym) based on deep reinforcement learning, which integrates historical data, technical indicators, macroeconomic indicators, and other data types. The model describes evaluation criteria and control strategies. It processes long-period data, implements a replay model that can incrementally expand data and features, automatically calculate reward labels, constantly train intelligent traders, and moreover, proposes a method of directly calculating the single-step deterministic action values by price. Upon testing 1479 stocks with more than ten years’ data in the China stock market, ISTG’s overall revenue reaches 13%, which is better than overall - 7% of the buy-and-hold strategy.