autoencoder networks

Deep Neural Network (DNN) models are one of the most dominant unsupervised feature extraction methods that have been widely studied recently. Convolutional Neural Networks (CNN) combine learned features with input data and use 2D convolutional layers, and this architecture is usually well suited for processing 2D data such as images. Convolutional neural networks work by extracting features directly from images and do not require feature extraction by an observer. The corresponding properties are not predefined. Convolutional neural networks learn to recognize different features of an image using tens or hundreds of hidden layers. Each hidden layer increases the complexity of the learned image features. The initially hidden layers can learn how to detect edges, and the later layers  discover how to identify more complex shapes, particularly the shape of the object we are trying to detect.
Generally, CNNs in each layer recognize more detailed features from an image that conclude, analyze, and then decide.

The basis of this research is based on DenseNet deep learning architecture. DenseNet architecture consists of several dense transmission blocks placed between two adjacent dense blocks. Each layer uses all the previous feature maps as input. This new model provides elevatedaccuracy with a reasonable number of network parameters for object detection tasks.
With changes in settings, these algorithms achieved high accurate results in several datasets used for this purpose. Sets of features are collected in a flat layer and reconstructed layer by layer as a latent space of an Autoencoder network with UpSampling layers.
In a joint project with the support of the American Science Foundation, Stanford University, and several other scientific research centers, a website for the free sharing of brain and cognitive science data under the openfMRI was launched in 2011 and  then  renamed openNeuro. Accordingly, these data were related to confidence in perceptual decisions, which were simultaneously recorded in EEG and fMRI.
The task used during data recording was the Random Moving Dots (RDM) test, in which healthy volunteers were asked to judge the direction in which dots were moving across the screen.
This study used  the independent component analysis (ICA) methodto clean the existing EEG signals in the Python programming language using the fastICA algorithm. The self-sufficient analysiscomponents are called the separation of independent sources mixed by an unknown combination system. Besides, the the separation should be done only based on the observation of the combined signals, i.e. , both the combination system and the primary signals are unknown.
The Gramian Angular Sum/Difference Fields method and Markov Transfer Fields were used to reduce the size of the input without losing the basic data and to convert the EEG signal into images.

The test data in this deep neural network are tensorial, consisting of MTF images with a size of 64×64×3 and an fMRI image matrix with a size of 70×70×32. The total available data are 16962, and the number of test data is 11873, which for this network, the ratio of test data to test data is 70-30. GPUs from Google's Collaboratory service were used to process these number calculations.
The total number of parameters is 1593175, of which 1541275 parameters, are trained in the network.  The average accuracy of the model was 85.86% during 1000 iterations (Figure 1).








Latent







Figure 1. Model implementation steps

The above method using fastICA, MTF, and DenseNet deep learning algorithm and combining it with an autoencoder network is new for reconstructing brain images. According to the obtained results, it can be concluded that the deep learning algorithm performs better than other superficial methods in many applications. As mentioned earlier, the deep learning algorithm is a new method yet has much potential for improvement and development. The method used in this research is a novel method in image reconstruction and there is a need to improve it. Developing methods can help improve the algorithm performance in future work   to increase the accuracy model and a more appropriate weighting of the used neural network. Additionally, with newer deep structures that increase in number daily, their efficiency in image reconstruction using EEG signals can be evaluated and checked.
Ethical considerations
Compliance with ethical guidelines
There were no ethical considerations involved in the research related to this article.
Authors’ contributions
The design of the research implementation stages and the writing of the article were done by Mahdi Arjmand. All authors performed the research literature and background. Saeed Stayeshi, Manuchehr Kelarestaghi, and Javad Hatami completed the design and actively participated in advising and supervising the implementation of the research and writing stages.
Funding
This research is not under the financial support of any institution or organization.
Acknowledgments
The authors are grateful to all the dignitaries who helped us in conducting and consensus on the current research.
Conflict of interest
The authors declare no conflicts of interest.

The concentration of particulate matters has recently increased in the metropolitan area of Tehran resulting in many severe hazards for both the environment and citizens. Particulate matters (PM) with a diameter less than 2.5 microns (PM2.5) are considered to be one of the most dangerous types of pollution. Estimating the concentration of these particles in Tehran is challenging due to the existence of various sources of pollution and the lack of sufficient ground stations. Aerosol optical depth (AOD) data retrieved from satellite imagery can be an alternative. However, AOD are not easily convertible into surface pollution and requires the development of appropriate models such as those based on data-driven approaches and machine learning techniques. Thus, the present study seeks to create a model to estimate the concentration of PM2.5 in Tehran employing deep generative models and in-situ measurements, meteorological data, and AOD data extracted from MODIS satellite imagery. Reviewed literature has proved the ability of deep learning techniques to solve regression and classification problems. Deep learning techniques are divided into various categories, one of which is based on the generative models seeking to reconstruct the input features. In this way, high-level and efficient features can be employed to explore the relationship between PM2.5 and AOD. Thus, the present study has investigated the potential of deep generative models for estimating PM2.5 concentration from high resolution AOD data retrieved from satellite imagery. 

Materials and Study Area:

 As a metropolitan area suffering from air pollution particularly in winters, the capital city of Iran, Tehran was selected as the study area. PM2.5, the main source of pollution in Tehran, is mainly emitted from vehicles and especially old urban public transportfleet.Aerosol data collected by Aqua and Terra sensors of MODIS and retrieved by Multi-angle Implementation of Atmospheric Correction (MAIAC) algorithm were used in the present study. Meteorological data were obtained from the global ECMWF climate model, and the concentration of PM2.5 was measured at air quality monitoring stations. Data were collected for a time interval of January 2013 to January 2020.

The present study has investigated the potential of deep generative models used to provide an estimate of PM2.5 concentration based on satellite AOD data. To reach such an aim, three types of deep generative neural networks, deep autoencoder (DAE), deep belief network (DBN) and conditional generative adversarial network (CGAN) were developed. Moreover, the performance of deep generative modes was compared with linear regression techniques as typical models used to explore the relation between PM2.5 and AOD data. Finally, the most accurate model for the generation of high resolution (1km) PM2.5 maps from AOD data was selected based on the performance of models.

 Results and Discussion:   

The accuracy of each developed model was evaluated using the test data and the obtained results were compared with results obtained from other basic linear regression models. Accuracy evaluation indicated that the developed deep autoencoder (DAE) combined with support vector regression led to the highest correlation (R2 = 0.69) and lowest RMSE (10.34) and MAE (7.95) and thus, can be potentially used for high resolution estimation of PM2.5 concentration. Next was the developed deep belief network which with a performance close to DAE demonstrated its potential capability to estimate PM2.5 concentration from satellite AOD data. The CGAN network acted less accurately in the estimation of PM2.5 concentration as compared to other deep generative models, but outperformed the linear regression algorithms on the test data. To sum up, findings indicated that deep generative models have outperformed classical linear regression techniques used for high resolution estimation of PM2.5 from satellite AOD data. Among the linear methods, the highest accuracy was achieved by the Lasso algorithm with an RSME of 12.14 and MAE of 9.46 on the test data which showed the significance of regularization for the improvement of performance in linear regression algorithms. Nevertheless, the accuracy of linear regression techniques was much lower than deep generative models.

Finally, DAE was selected as the best model for the estimation of PM2.5 concentration across the study area and high resolution maps of PM2.5 concentration were generated using the developed model. Investigating the daily PM2.5 maps generated for two days with different air quality conditions (clean and polluted) demonstrated the efficiency of the developed DAE for PM2.5 modeling.