A novel financial trading system based on reinforcement learning and technical analysis applied on the Tehran securities exchange market

Stock trading is a significant decision-making problem in asset management. This study introduces a financial trading system (FTS) that leverages artificial intelligence (AI) techniques to automate buy and sell orders specifically in Iran's stock market. Due to limited availability of labeled data in financial markets, the FTS utilizes reinforcement learning (RL), a subset of AI, for training. The model incorporates technical analysis and a constrained policy to enhance decision-making capabilities. The proposed algorithm is applied to the Tehran Securities Exchange, evaluating its efficiency across 45 periods using three different stock market indices. Performance comparisons are made against common strategies such as buy and hold, randomly selected actions, and maintaining the initial stock portfolio, with and without transaction costs. The results indicate that the FTS outperforms these methods, exhibiting excellent performance metrics including Sharp ratio, PP, PF, and MDD. Consequently, the findings suggest that the FTS serves as a valuable asset management tool in the Iranian financial market.