Abstract:
Logistic regression, a fundamental tool in binary classification tasks, faces challenges in real-world datasets where the assumption of predictor variable independence is often violated, leading to multicollinearity issues. Additionally, the inclusion of numerous predictors can destabilize the Maximum Likelihood Estimator (MLE), reducing predictive efficiency. To address these issues, we introduce an innovative logistic regression algorithm combining the Iteratively Reweighted Least Squares (IRLS) and Least Angle Regression (LARS) algorithms, leveraging the Least Absolute Shrinkage and Selection Operator (LASSO) concept. Through L1 regularization, our method selects relevant predictors while shrinking less influential ones towards zero, improving model interpretability and performance. Notably, our algorithm excels in handling imbalanced datasets, maintaining accuracy across class distributions. Comparative assessments demonstrate superior predictive accuracy and feature selection compared to established algorithms. Moreover, our algorithm exhibits resilience in managing imbalanced data, showcasing its potential to advance logistic regression in binary classification tasks. These findings highlight its contribution to real-world applications, offering valuable insights for practical scenarios.
