data pre-processing

The integration of advanced metering technology in power systems has enabled real-time data access for every node in a smart grid. As a result, the power system can now access large volumes of data. This vast amount of data requires an alternative method of analysis. Machine learning-based load forecasting technologies are being applied in this scenario. However, this massive data collection needs to be processed through the appropriate data pre-processing method, such as the removal of noise, outliers, and erroneous data, the detection of missing data, the normalization of widely divergent datasets, etc., to improve the effectiveness of the load forecaster. Thus, to eliminate the various kinds of errors and outliers present in the data that was directly obtained from smart meters, this study analyses and compares the efficacy of eight distinct smoothing and filtering techniques as a novel contribution of this work. Using the processed data acquired, a neural network-based load forecasting model was developed to compare the efficacy of the variouspreprocessing approaches. This study makes use of real-time data obtained from the smart meter placed at a node within the NIT Patna campus. The proposed moving average filter surpasses the other methods for filtering and smoothing the raw data by an average MAPE of 2.66, according to the load forecasting results that were obtained.