q learning algorithm

Concrete, as one of the main materials in the construction industry, plays a vital role in the sustainability, safety, and welfare of urban spaces. This is because concrete has a direct impact on bearing gravitational and lateral loads, and improving the quality of concrete can prevent the premature destruction of buildings. Additionally, it can reduce the volume of construction waste and create a sustainable urban environment. However, numerous factors affect the compressive strength of concrete, and failing to identify these factors can lead to premature building destruction and adverse outcomes during natural disasters. A proper understanding of these factors is essential for enhancing concrete quality and ensuring the optimal performance of structures. Accordingly, the aim of this article is to analyze the factors influencing the quality and strength of concrete to improve the sustainability, safety, and welfare of urban spaces and to protect the urban environment. In this article, to achieve the research objectives, in addition to using a machine learning model based on the Extreme Gradient Boosting algorithm, metaheuristic algorithms have been employed to create an accurate predictive model.

The aim of selecting test items is to choose a subset that has the potential to detect errors due to changes within the software. In other words, the purposes of test selection methods is to reduce the number of test cases after changing the code and focus on identifying the modified parts of the program. Intelligent methods such as regression improve the accuracy of tests in software projects, and the use of optimization algorithms to find the optimal amount of test cases can be useful in terms of time and speed, and according to research by examining and optimizing this algorithm in the system. In this paper, a technique for reducing regression test cases based on teacher-student optimization method was presented. This method was studied in two stages of teacher (education phase) and student (learning phase) on the test set and was implemented with different parameters. The experimental results showed that the use of the teacher-student algorithm reduces the time required for the reduction parameters of the regression test to some extent, although it does not give us a definite answer and will give a near-optimal answer. Also, the results of teacher-student algorithm were compared with previous approaches of regression test case reduction. Experimental results show better average execution time for test case selection. .

Network routing of drones for search and rescue operations is a critical challenge. This challenge arises due to the physical limitations of drones, adverse environmental conditions, and time constraints. In this paper, a novel approach for network routing of drones using the Q-Learning algorithm is proposed. This algorithm enables drones to automatically determine optimal paths in complex environments and adapt to environmental changes. Simulation results demonstrate that the Q-Learning algorithm can find shorter and more efficient routes compared to genetic algorithms. These findings highlight Q-Learning as a promising method for improving network routing of drones in search and rescue operations

Considering the outstanding position of the country in the world from the point of view of energy resources, especially the gas field, the implementation of energy infrastructure projects, especially the gas pipeline, is essential. Nevertheless, one of the main challenges in this type of projects is the lack of proper productivity of resources (machinery, etc.). Therefore, the main goal of the current research is to improve the productivity of earthmoving machinery for gas pipeline construction projects, with the help of machine learning algorithms. In this research, by using library studies, documents (daily reports of seven gas transmission pipeline projects), expert judgment, text mining method (and Rapidminer software), effective criteria for determining the productivity of earthmoving machinery in gas pipeline construction projects, identified and final became. In short, the main result of the current research indicates that the prediction of the excavation volume, through the predictive algorithm (as a basis for optimizing the productivity of earthmoving machinery) and also the classification algorithm and using the deep learning model (as the selected model with the best performance in the prediction of the excavation volume), is applicable. In fact, the findings of the current research can be used in order to predict the volume of earthmoving, before the start of the project and prepare a general schedule that ultimately improves the productivity of earthmoving machinery in gas pipeline projects.

Heat transfer is an important and influential factor in the design of internal combustion engines. Accurate prediction of heat transfer has a great impact on the design of cooling systems of internal combustion engines. The simulation of internal combustion engines for their thermal analysis is done using computational fluid dynamics along with boiling simulation. This article investigates the subcooled boiling flow of pure water in a small channel and its dependence on different parameters such as working pressure, heat flux, and mass flux. The subcooling temperature of the inlet fluid is of interest. Numerical simulations were performed using axisymmetric models and the effect of different grid sizes on the accuracy of the results was investigated. The bubble separation diameter was also studied and its relationship with pressure, heat flux, mass flux, and temperature under the inlet coolant was analyzed. Finally, a database of generated simulation results and predictive models for bubble dynamics using machine learning algorithms were presented.

Sewage flow path is the main component of urban sewerage network infrastructure. Damage to sewers is less noticeable due to invisibility, and this failure to handle the damage leads to emergencies and unreasonable costs. These vital arteries need to be maintained and rebuilt during service for optimal performance in all dimensions. Nowadays, the methods of processing and classifying photos and videos taken by mobile videometer robots are widely used to inspect the sewer network. One of the successful algorithms in the field of image processing is the convolutional neural network algorithm, which is a subset of deep learning algorithm. In this paper, a convolutional neural network algorithm is used to classify images of sewer network damage and cases affecting the improvement, accuracy and performance of this algorithm. The images were obtained by a videometric robot from the sewer network. Results of using the proposed algorithm in the sewerage network, achieving 98% accuracy in classifying network faults and compared to other methods and also reducing the relatively low execution time of the proposed architecture (91 minutes) compared to other architectures valid ones are the same in deep learning on the same hardware platform. Also, in the future, the proposed algorithm will be used to analyze networks without the need for specialized personnel and also to control an automatic network videometry robot.

The conventional Tracking-Learning-Detection (TLD) algorithm is sensitive to illumination change and clutter and low frame rate and results in drift even missing. To overcome these shortcomings and increase robustness, by improving the TLD structure via integrating mean-shift and co-training learning can be achieved better results undergo low frame rate (LFR) condition and the robustness and accuracy tracking of the TLD structure increases. Because of, the Mean-Shift tracking algorithm is robust to rotation, partial occlusion and scale changing and it is simple to implement and takes less computational time. On the other, the co-training learning algorithm with two independent classifiers can learn changes of the target features in during the online tracking process. Therefore, the extended structure can solve the problem of lost object tracking in LFR videos and other challenges simultaneously. Finally, comparative evaluations of the proposed method to other top state-of-the-art tracking algorithms under the various scenarios from the TB-100 known dataset, demonstrate the superior performance of the proposed algorithm compared to other tracking algorithms in terms of tracking robustness and stability performance. Finally, the proposed structure based on the TLD architecture, in scenarios with the various challenges mentioned, will improve on average about 33% of the results, compared to the traditional TLD algorithm.

Transmitted vibration to driver body (whole-body vibration) in heavy-duty Off-road vehicles without primary suspension leads to health problems in long term such as spine disorder, back pain, heart abnormal beating, vision disable, digestive problem, etc. Commonly, secondary seat suspension is used to isolate vibration and remove health problems, s especially in the spinal column. Passive seat suspension is widely employed for this purpose, but its efficiency is low. Active seat suspension and semi-active suspensions have better performance in blocking unwanted oscillation. Recently, semi-active magnetorheological dampers are focused on by researchers which can adjust their viscosity by electromagnetic flux. It is called smart fluid due to having controllable parameters such as damping ratio by electricity variations. Thus, suspension can dampen oscillation by low electricity current efficiently. The current paper introduces a novel active force control (AFC) equipped with an iterative learning estimator for seat suspension via MR damper. Results of simulations show that it can cancel vibration perfectly. The suspension was coupled to the human body vibration model, and the driver vibration results have been obtained. It is important to note that active force control can also be used to eliminate high-velocity disturbances. Since the vibrations that occur in the car have a high rate of change, the results indicated that active force control equipped with an iterative learning estimator system can be effective in reducing the transmission vibration to the driver so that the first and second vibration peaks was reduced by about 60%.MethodologyAt the first, a dynamic model of an off-road vehicle seat suspension which is equipped with an MR damper has been developed. After that, a PID controller has been designed for semi-active seat suspension. Next, the PID controller has been integrated into the AFC method. Lastly, utilizing the ILA in mass estimation for the AFC approach and simulation results of that has been discussed.The simulation work has been developed employing MATLAB/Simulink software whereas passive suspension and semi-active suspension via the PID and AFCAIL controllers were simulated. To evaluate the performance of the control schemes, the seat suspension model was exposed to a number of various disturbances as road roughness listed as follows:Results and DiscussionIn this section, a comparison between passive suspension and all the control methods has been performed in time and frequency domains. ConclusionA novel controller called the active force control has been employed for vibration control of seat suspension equipped by a magnetorheological (MR) damper. The AFC method was integrated into the iterative learning algorithm to calculate the estimated mass of the system named AFC-IL . The designed controllers were simulated for the vibration canceling of an off-road vehicle seat suspension system. The AFC technique achieved to be uncomplicated in terms of calculation and change it suitable for real-time working conditions. Moreover, the AFC demonstrates high performance and produce robust and accurate response still in the presence of various disturbances. The simulation results express that for known parameters and situations, the proposed AFC-IL method efficiency enhanced in comparison to the popular PID controller. The results exemplify that the AFC-based scheme as an intelligent control technique has proper potential to eliminate the disturbances superiorly for the off-road vehicle seat suspension. Although complementary efforts should be accomplished to investigate the influences of other types of disturbances, parameters and uncertainties changes in real situations such as road trials. In continuing a test rig will be developed to evaluate the AFC scheme performance in real conditions in terms of the vibration attenuating and optimized parameters.

Automatic classification of power quality disturbances is the foundation to deal with the power quality problem. From a traditional viewpoint, the identification process of power quality disturbances should be divided into three independent stages: signal analysis, feature selection, and classification. However, there are some inherent defects in signal analysis and the procedure of manual feature selection is tedious and imprecise, leading to a low classification accuracy of multiple disturbances. To deal with these problems, this paper presents an automated system for the classification and identification of power quality disturbances. After receiving input signals, the proposed system requires some preprocessing such as changing the range of values by dividing the signals into their basic domains. In the next stage, the RMS value of the signal can be appraised to know the occurrence of the disturbance. If the RMS value of the input signal is not equal to the normal signal, the disturbance is occurring. To identify and classify disturbances, a novel deep learning-based method is developed. In this method, the activation function is expressed by a fuzzy approach. This makes the system more flexible. The benefits of the proposed strategy are separating the disturbances of basic frequency and using the nature of power quality signals as a tool for feature extraction. However, in the traditional method, for example, in empirical mode decomposition, the separation of signals from their components is not conveniently possible. To evaluate the proposed algorithm, a 33-bus distribution power network has been applied. The results reveal good agreement in comparison with other assessment tests.

This paper mainly aims to determine the optimal drug dosage for the purpose of reducing the population of cancer cells in melanoma patients. To do so, Reinforcement Learning method and the eligibility traces algorithm are employed, giving us the advantage of creating a compromise between the two algorithms of the reinforcement learning, being Monte-Carlo and Temporal Difference. Furthermore, it can be said that using this approach, there was no need to employ a mathematical model in the whole process. However, as its implementation on the real system was not possible, a delayed nonlinear mathematical model is used to investigate the performance of the proposed controller and simulate the behavior of the environment. It should be noted this mathematical model made use of no control method. This is the first time that population control of cancer cells is applied and tested on this model. To know of the optimal dosage of the drug, it should be mentioned that the drug is required to prevent the side effects on healthy/normal cells as much as possible. According to the obtained results, the eligibility traces algorithm is able to control and reduce the population of cancer cells through injecting the sub-optimal drug dose. This will increase the level of immunity in our body. Finally, to demonstrate the advantage of a selective method of increasing the rate of cancer cell death, this method is compared with the Q-learning algorithm and optimal control. By applying the fault to the sensor, the performance of the proposed controller to reduce cancer cells was investigated. The adaptability of the proposed method with the environment changes is checked afterwards. To this end, uncertainty in the system parameters and initial conditions are applied and the population of cancer cells are controlled in five melanoma patients. Moreover, having added noise to the system, it was shown that the eligibility traces algorithm is able to control the population of cancer cells and make it reach zero. Additionally, the convergence speed of both eligibility traces algorithm and Q learning algorithm in reducing the number of cancer cells for different learning rates was investigated.

Melanoma is one of the most dangerous types of cancers and every year, many people suffer from this cancer. Hence, quick diagnosis and treatment are significantly important for the physicians. In the recent decade, intelligent methods have attracted considerable attention for diagnosing and treating the melanoma. The main objective of this paper is determining the optimal dosage of the drug for the elimination of the cancer cells while preventing from the side effect of the drug on the normal cells. To this aim, the Q-learning algorithm is employed. In order to select the actions, a Case-Based Reasoning (CBR) policy is used, which is an accelerated heuristic policy. The considered policy has increased the learning speed and reduced the overall time, to reach the optimal policy. The half-life effect of the drug is also considered to obtain the side effect of the drug on the patientchr('39')s body, at each time step. In order to demonstrate Q-learning algorithm performance in cancer cells control and optimal dosage determination purposes, Q-learning is compared with two methods, including fix dosage injection method and Hamiltonian method, which is one of the most important optimal control methods. Finally, it is revealed that the total injected dosage by using Reinforcement Learning method (Q-learning) is significantly reduced within the whole period of time in comparison with employing the optimal control and a fixed dosage injection cases. The number of cancer cells is controlled, as well. It should be noted that by applying the noise and uncertainty to the system parameters and the initial conditions, the proposed method can successfully control the cancer cells.

Fuzzy cognitive maps (FCMs) that are soft computing techniques, by combining fuzzy logic and neural network theory, have been known as a powerful tool for modeling complex systems. Utilization of different learning algorithms to overcome the weaknesses of this model, is one of the active area of science. In this paper, a new hybrid algorithm based on nonlinear Hebbian learning and real-coded genetic algorithm is introduced, which operate in an entangled way and by improving the characteristics of each of these two algorithms, can be applied in different decision-making models with high precision. The proposed model is implemented on a process control problem.