A new shrinkage-based estimator for logistic regression under multicollinearity: The James-Stein logistic Liu estimator (JSLLE)

Abstract

Logistic regression is a widely used method for analyzing binary outcomes in various fields, including healthcare and finance. The Maximum Likelihood Estimator (MLE) is the standard technique for estimating parameters in logistic regression. However, when predictor variables exhibit multicollinearity, MLE becomes inefficient, leading to inflated variances and unstable coefficient estimates. To address this issue, several biased estimators have been proposed in the literature, including the James-Stein Estimator (JSE) and the Logistic Liu Estimator (LLE). This study introduces a new estimator, the James-Stein Logistic Liu Estimator (JSLLE), which combines the shrinkage properties of JSE with the flexible biasing mechanism of LLE. We explore two distinct shrinkage values based on existing literature, resulting in two variants of the estimator: JSLLE1 and JSLLE2. The primary objective is to enhance the accuracy of parameter estimation in logistic regression using JSLLE by minimizing the Scalar Mean Squared Error (SMSE). A Monte Carlo simulation study was conducted to assess the performance of MLE, JSE, LLE, and JSLLE under moderate to high multicollinearity and sample sizes ranging from small to large. The findings indicate that JSLLE outperforms MLE, JSE, and LLE for moderate sample sizes with moderate to high multicollinearity. The relative efficiency of JSLLE1 compared to MLE based on SMSE shows clear improvement across different correlation levels and sample sizes. For correlation values of 0.5, 0.7, and 0.9, JSLLE1 achieves efficiencies of 30.83%, 39.28% and 55.81% for n=50; 19.73%, 24.07% and 39.46% for n=100; and 2.61%, 3.61% and 9.29% for n=1000. The corresponding efficiencies of JSLLE2 are 28.32%, 33.99%, and 47.53%; 17.72%, 23.45%, and 38.03%; and 2.58%, 3.62%, and 9.53%, respectively. Overall, the efficiency of the proposed estimator increases with higher correlation, demonstrating its robustness and practical value in addressing multicollinearity, making JSLLE a reliable and efficient alternative for logistic regression analysis.