Improved Relief Weight Feature Selection Algorithm Based on Relief and Mutual Information
Hongbin Wang, Pengming Wang, Shengchun Deng et al. · Information · 2021
As the classic feature selection algorithm, the Relief algorithm has the advantages of simple computation and high efficiency, but the algorithm itself is limited to only dealing with binary classification problems, and the comprehensive distinguishing ability of the feature subsets composed of the former K features selected by the Relief algorithm is often redundant, as the algorithm cannot select the ideal feature subset. When calculating the correlation and redundancy between characteristics by mutual information, the computation speed is slow because of the high computational complexity and the method’s need to calculate the probability density function of the corresponding features. Aiming to solve the above problems, we first improve the weight of the Relief algorithm, so that it can be used to evaluate a set of candidate feature sets. Then we use the improved joint mutual information evaluation function to replace the basic mutual information computation and solve the problem of computation speed and correlation, and redundancy between features. Finally, a compound correlation feature selection algorithm based on Relief and joint mutual information is proposed using the evaluation function and the heuristic sequential forward search strategy. This algorithm can effectively select feature subsets with small redundancy and strong classification characteristics, and has the excellent characteristics of faster calculation speed.
Cited by 9 works. doi:10.3390/info12060228 Open-access PDF
Authors
References
- Wrappers for feature subset selection
- Estimating attributes: Analysis and extensions of RELIEF
- The feature selection problem: traditional methods and a new algorithm
- Feature selection for high-dimensional data: a fast correlation-based filter solution
- Relief-based feature selection: Introduction and review
- Feature selection for classification: A review
- Feature Selection for Clustering: A Review
- Benchmarking relief-based feature selection methods for bioinformatics data mining
- Embedded local feature selection within mixture of experts
- ReliefF-MI: An extension of ReliefF to multiple instance learning