An Enhanced Machine Learning Approach to Identify Noise and Detect Relevant Structures for Predictive Modeling

Abstract

The era of big data and social networking platforms have provided great repositories of the data for mining useful information for the real-world industry. However, along with this benefit comes the noise in the data. Generally, noise is the data-set that are redundant, false, bad, and/or outliers. Data cleaning, outlier identification, feature engineering, data slicing, etc. are few of many techniques used traditionally. End goal remains ensuring good data (signal) is not lost in bad data (noise) and less processing cost are incurred to extract useful knowledge out of given big data. This paper presents a follow up progress on existing work of the author in relevance of machine learning algorithms, academic and career data predictions and personality computing. All of that have been initially inspired by potential of useful relationships and data points in unstructured data and thus Noise becomes very relevant and may appear Signal in other contexts and predictors in goal. This proposed model is collectively titled as ‘Noise Removal and Structured Data Detection’ based on inherited parallel processing and unique n-Dimensional training approach. Personality features can be quantified into talent traits, matrix indicating the max/min for relevance factors in the academics/career of nD. The engine internals examine and train the algorithm that it minimizes the x,y co-ordinates and maximizes the z co-ordinate. It records and compares the engine internal metrics and reports it back to engine to further optimize the machine learning process until the optimum results are obtained or do not improve any further.