High-Dimensional Diagonal Classifiers
The advancement in technology has necessitated the collection of massive high-dimensional
data in bioinformatics, medical sciences, business, climatology, and magnetic resonance images. These collec-
tions have given rise to high-dimensional (p) and small sample(n) (HDSS) problems in group classifications.
The classification coefficient of the HDSS data cannot be constructed due to the ill-conditioning or singu-
larity of the covariance matrix. The curse of dimensionality suffered by the classical methods has rendered
the formulation of the coefficient impossible as such different dimensionality reduction methods have been
proposed to avert the curse of dimensionality and ill conditioning. Dimensionality reduction methods such
as the principal component analysis have been applied as a prelude to reducing the dimension of the HDSS
data to enhance constructing the coefficients of any classification methods. Some of these dimension reduc-
tion methods with infused constants as a benchmark selection, may increase or decrease computational time
and improve classification accuracy at the cost of information loss. In this paper, we proposed two methods
that can solve the ill-conditioning and singularity, or dimensionality problems associated to HDSS data di-
rectly. These methods are diagonal FLCM (DFLCM) and point extraction method (PEM). We investigated
the robustness of these methods based on the optimal probability benchmark (OPB). The computational
time (CPU time) of all the methods is also investigated. We applied three real data sets to determine the
comparative performance, and the classification results revealed that the proposed DFLCM is very robust
more than the NaГЇve Bayes classifier in terms of the probability of correct classification and CPU time.
Therefore, this study affirmed that the OPB is suitable to determine the robustness of HDSS classification
problems.
1. El Anbari, M.; Bensmail, H. Computational and statistical challenges with high dimensionality: A new method
and efficient algorithm for feature selection in knowledge discovery. In Qatar Foundation Annual Research Forum
Proceedings (Vol. 2012), No. 1. Bloomsbury Qatar Foundation Journals.
2. Fan, J.; Li R. Statistical challenges with high dimensionality: Feature selection in knowledge discovery. In Pro-
ceedings of the International Congress of Mathematicians Madrid, August 22-30, 2006 (pp. 595-622). European
Mathematical Society-EMS-Publishing House GmbH.
3. Herbert, P.; Tiejun, T. Recent advances in discriminant analysis for high-dimensional data classification. J. Biom.
Biostat 3 2, 1-2.
4. Li, C. Q.; Xu, Q. S. High-dimensional spectral data classification with nonparametric feature screening. Journal
of Chemometrics 34 (2020), 3,: e3199.
5. Gad, H. A.; El-Ahmady, S. H.; Abou-Shoer, M. I.; Al-Azizi, M. M. Application of chemometrics in authentication
of herbal medicines: a review. Phytochemical Analysis 24 (2013) 1, 1-24.
6. Bommert, A.; Sun, X.; Bischl, B., Rahnenfehrer, J.; Lang, M. Benchmark for filter methods for feature selection
in high-dimensional classification data. Computational Statistics & Data Analysis 143 (2020), 106839
7. Harikumar, S.; AS, A.; Kaimal, R. A depth-based nearest neighbor algorithmfor high-dimensional data classifica-
tion. Turkish Journal of Electrical Engineering and Computer Sciences 27 (2019), 6, 4082-4101.
8. Fan, J.; Lv, J. Sure independence screening for ultrahigh dimensional feature space. Journal of the Royal Statistical
Society Series B: Statistical Methodology 70 (2008), 5, 849-911.
9. Mourao-Miranda, J.; Bokde, A. L.; Born, C., Hampel, H.; Stetter, M. Classifying brain states and determining
the discriminating activation patterns: support vector machine on functional MRI data. NeuroImage 28 (2005),
4, 980-995.
10. Pappu, V.; Pardalos, P. M. High-dimensional data classification. In Clusters, Orders, and Trees: Methods and
Applications: In Honor of Boris Mirkin’s 70th Birthday (2014), pp. 119-150. New York, NY: Springer New York.
11. Johnstone, I. M.; Titterington, D. M. Statistical challenges of high-dimensional data. Philosophical transactions.
Series A, Mathematical, physical, and engineering sciences 367 (2009), 1906, 4237-4253.
12. Okwonu, F. Z.; Othman, A. R. A model classification technique for linear discriminant analysis for two groups.
International journal of computer science issues (IJCSI) 9 (2012), 3 no. 2, 125-128.
13. Fan, J.; Fan, Y. High dimensional classification using features annealed independence rules. Annals of statistics
36 (2008), 6, 2605-2644.
14. Bickel, P. J.; Levina, E. Some theory for Fisher’s linear discriminant function,naive Bayes’, and some alternatives
when there are many more variables than observations. Bernoulli 10 (2004), 6, 989-1010.
15. Ghosh, D. Singular value decomposition regression models for classification of tumors from microarray experiments.
In Biocomputing 2002 (2002), pp. 18-29.
16. Bair, E.; Hastie, T.; Paul, D.; Tibshirani, R. Prediction by supervised principal components. Journal of the
American Statistical Association 101 (2006), 473, 119-137.
17. Ma, J.; Yuan, Y. Dimension reduction of image deep feature using PCA. Journal of Visual Communication and
Image Representation 63 (2019), 102578.
18. Monakhova, Y. B.; Godelmann, R.; Kuballa, T.; Mushtakova, S. P.; Rutledge, D. N. Independent components
analysis to increase efficiency of discriminant analysis methods (FDA and LDA): Application to NMR fingerprinting
of wine. Talanta 141 (2015), 60-65.
19. Seghouane, A. K.; Shokouhi, N.; Koch, I. Sparse principal component analysis with preserved sparsity pattern.
IEEE Transactions on Image Processing 28 (2019), 7, 3274-3285.
20. Velliangiri, S.; Alagumuthukrishnan, S. J. P. C. S. A review of dimensionality reduction techniques for efficient
computation. Procedia computer science 165 (2019), 104-111.
21. Ye, B.; Liu, P. Classification of High-Dimensional Data: A Random-Matrix Regularized Discriminant Analysis
Approach. Inter. Journal of Innov. Comp., Info. and Control 15 (2019), 3, 955-967.
22. Wang, C.; Jiang, B. On the dimension effect of regularized linear discriminant analysis. Electronic Journal of
Statistics 12 (2018) 2709-2742
23. Guo, Y.; Hastie, T.; Tibshirani, R. Regularized linear discriminant analysis and its application in microarrays.
Biostatistics 8 (2007), 1, 86-100.
24. Fu, R.; Tian, Y.; Bao, T.; Meng, Z.; Shi, P. Improvement motor imagery EEG classification based on regularized
linear discriminant analysis. Journal of medical system 43 (2019), 6, 169-182.
25. Wu, W.; Mallet, Y.; Walczak, B.; Penninckx, W.; Massart, D. L.; Heuerding, S.; Erni, F. Comparison of regular-
ized discriminant analysis linear discriminant analysis and quadratic discriminant analysis applied to NIR data.
Analytica Chimica Acta 329 (1996), (3), 257-265.
26. Elkhalil, K.; Kammoun, A.; Couillet, R.; Al-Naffouri, T. Y.; Alouini, M. S. A large dimensional study of regularized
discriminant analysis. IEEE Transactions on Signal Processing 68 (2020), 2464-2479.
27. Zhang, Z.; Dai, G.; Xu, C.; Jordan, M. I. Regularized discriminant analysis, ridge regression and beyond. The
Journal of Machine Learning Research 11 (2010), 2199-2228.
28. Fu, R.; Wang, H.; Bao, T.; Han, M. EEG intentions recognition in dynamic complex object control task by
functional brain networks and regularized discriminant analysis. Biomedical Signal Processing and Control 61
(2020), 101998, 1-12.
29. Krzeszowski, T.; Wiktorowicz, K. . Combined regularized discriminant analysis and swarm intelligence techniques
for gait recognition. Sensors 20 (2020), 23, 6794-67808.
30. Friedman, J. H.. Regularized discriminant analysis. Journal of the American statistical association 84 (1989), 405,
165-175.
31. Xu, J.; Xu, Q.; Yi, L.; Chan, C. O.; Mok, D. K. W. Correlation-assisted nearest shrunken centroid classifier with
applications for high dimensional spectral data. Journal of Chemometrics 30 (2016), 1, 37-45.
32. Tibshirani, R.; Hastie, T.; Narasimhan, B.; Chu, G. Class prediction by nearest shrunken centroids, with applica-
tions to DNA microarrays. Statistical Science 18 (2003), 1, 104-117.
33. Chung, D.; Keles, S. . Sparse partial least squares classification for high dimensional data. Statistical applications
in genetics and molecular biology 9 (2010), (1), 17, 1-30
34. Dudoit, S.; Fridlyand, J.; Speed, T. P. Comparison of discrimination methods for the classification of tumors using
gene expression data. Journal of the American statistical association 97 (2002), 457, 77-87.
35. Li, Y.; Chai, Y.; Zhou, H.; Yin, H. A novel dimension reduction and dictionary learning framework for high-
dimensional data classification. Pattern Recognition 112 (2021), 107793.
36. Chen, W.; Xu, Y.; Yu, Z.; Cao, W.; Chen, C. P.; Han, G. . Hybrid dimensionality reduction forest with pruning
for high-dimensional data classification. IEEE Access 8 (2020), 40138-40150.
37. Xiao, X.; Fu, D.; Shi, Y.; Wen, J. (2020). Optimized Mahalanobis-Taguchi System for High-Dimensional Small
Sample Data Classification. Computational Intelligence and Neuroscience 2020 1, 4609423, 1-15.
38. Okwonu, F. Z.; Othman, A. R.. Heteroscedastic variancecovariance matrices for unbiased two groups linear
classification methods. Applied Mathematical Sciences 7 (2013), 138, 6855-6865.
39. Huberty, C. J.; Holmes, S. E. Two-group comparisons and univariate classification. Educational and Psychological
Measurement, 43 (1983), 1) 15-26.
40. Dietterich, T.; Jain, A.; Lathrop, R.; Lozano-Perez, T. A comparison of dynamic reposing and tangent distance
for drug activity prediction. Advances in neural information processing systems 6 (1993), 216-223.
41. Jain, A. N.; Dietterich, T. G.; Lathrop, R. H.; Chapman, D.; Critchlow Jr, R. E.; Bauer, B. E.; Teresa A. W;
Lozano-Perez, T. Compass: A shape-based machine learning tool for drug design. Journal of Computer-Aided
Molecular Design 8 (1994), 6, 635-652.
42. Dietterich, T. G.; Lathrop, R. H.; Lozano-PГ©rez, T. Solving the multiple instance problem with axis-parallel
rectangles. Artificial intelligence 89 (1997), no. 1-2, 31-71.
43. Dua, D.; Graff, C. UCI Machine Learning Repository [http://archive. ics. uci. edu/ml]. Irvine, CA: University of
California, School of Information and Computer Science, zuletzt abgerufen am: 14.09. 2019. Google Sch (2019).
44. Gorman, R. P.; Sejnowski, T. J. Analysis of hidden units in a layered network trained to classify sonar targets.
Neural networks 1 (1988), 1, 75-89.
45. Zhang, K.; Fan, W. Forecasting skewed biased stochastic ozone days: analyses, solutions and beyond. Knowledge
and Information Systems 14 (2008), 3, 299-326.
46. Zanin Zambom, A.; Matthews, G. J. Sure Independence Screening in the Presence of Data That is Missing at
Random. Statistical Papers from Loyola University Chicago (2019), pg 1-27.
47. Nandy, D.; Chiaromonte, F.; Li, R. Covariate information number for feature screening in ultrahigh-dimensional
supervised problems. Journal of the American Statistical Association 117 (2022), 539, 1516-1529.
Copyright (c) 2026 «ВЕСТНИК НУУз»
Это произведение доступно по лицензии Creative Commons «Attribution-NonCommercial-ShareAlike» («Атрибуция — Некоммерческое использование — На тех же условиях») 4.0 Всемирная.
.jpg)
2.png)
