Controlling Separation in Generating Samples for Logistic Regression Models

Abstract

Separation has a significant impact on parameter estimates for logistic regression models in frequentist approach and in Bayesian approach. When separation presents in a sample, the maximum likelihood estimation (MLE) does not exist through standard estimation methods. The existence of posterior means is affected by the presence of separation and also depended on the forms of prior distributions. Therefore, controlling the appearance of separation in generating samples from the logistic regression models has an important role for parameter estimation techniques. In this paper, we propose necessary and sufficient conditions for separation occurring in the logistic regression samples with two dimensional models and multiple dimensional models of independent variables. By using the technique of rotating Castesian coordinates of p dimensions, the characteristic of separation occurring in general cases is presented. Using these results, we propose algorithms to control the probability of separation appearance in generated samples for given sample sizes and multiple dimensional models of independent variables. The simulation studies show that the proposed algorithms can effectively generate the designed random samples with controlling the probability of separation appearance.