The Markov chain is widely used in state-dependent inventory control of spare parts because of its ability to model the gradual degradation process of components and predict their condition. Also, according to previous studies, considering system information causes a significant reduction in costs. Therefore, the present study tries to extract the system information using a machine learning algorithm and provide it as a transition matrix to the Markov decision process (MDP) to determine the future states of the critical spare parts inventory system. In the presented method, the machine learning algorithm, here Adaptive Neuro-Fuzzy Inference System (ANFIS), is in charge of the training data. The Markov chain uses the trained data to predict the future states of the inventory system. For this purpose, four states have been considered, each representing a level of tension and demand in the inventory system. Applying the model to the data collected for a critical component showed that the model has good accuracy in predicting the following states of the system. Also, the presented model offers a lower error rate, RMSE, and MAPE, compared to the ARIMA model for predicting the next state of the inventory system