Big Data Analytics and Data Mining Optimization Techniques for Air Traffic Management

With the advancements in science and technology‎, ‎the industrial and aviation sectors have witnessed a significant increase in data‎. ‎A vast amount of data is generated and utilized continuously‎. ‎It is imperative to employ data mining techniques to extract and uncover knowledge from this data‎. ‎Data mining is a method that enables the extraction of valuable information and hidden relationships from datasets‎. ‎However‎, ‎the current aviation data presents challenges in effectively extracting knowledge due to its large volume and diverse structures‎. ‎Air Traffic Management (ATM) involves handling Big data‎, ‎which exceeds the capacity of conventional acquisition‎, ‎matching‎, ‎management‎, ‎and processing within a reasonable timeframe‎. ‎Aviation Big data exists in batch forms and streaming formats‎, ‎necessitating the utilization of parallel hardware and software‎, ‎as well as stream processing‎, ‎to extract meaningful insights‎. ‎Currently‎, ‎the map-reduce method is the prevailing model for processing Big data in the aviation industry‎. ‎This paper aims to analyze the evolving trends in aviation Big data processing methods‎, ‎followed by a comprehensive investigation and discussion of data analysis techniques‎. ‎We implement the map-reduce optimization of the K-Means algorithm in the Hadoop and Spark environments‎. ‎The K-Means map-reduce is a crucial and widely applied clustering method‎. ‎Finally‎, ‎we conduct a case study to analyze and compare aviation Big data related to air traffic management in the USA using the K-Means map-reduce approach in the Hadoop and Spark environments‎. ‎The analyzed dataset includes flight records‎. ‎The results demonstrate the suitability of this platform for aviation Big data‎, ‎considering the characteristics of the aviation dataset‎. ‎Furthermore‎, ‎this study presents the first application of the designed program for air traffic management‎.