Pablo Livacic-Rojas, G. Vallejo, P. Fernández, Ellián Tuero-Herrero
{"title":"Power of Modified Brown-Forsythe and Mixed-Model Approaches in Split-Plot Designs","authors":"Pablo Livacic-Rojas, G. Vallejo, P. Fernández, Ellián Tuero-Herrero","doi":"10.1027/1614-2241/a000124","DOIUrl":"https://doi.org/10.1027/1614-2241/a000124","url":null,"abstract":"Low precision of the inferences of data analyzed with univariate or multivariate models of the Analysis of Variance (ANOVA) in repeated-measures design is associated to the absence of normality distribution of data, nonspherical covariance structures and free variation of the variance and covariance, the lack of knowledge of the error structure underlying the data, and the wrong choice of covariance structure from different selectors. In this study, levels of statistical power presented the Modified Brown Forsythe (MBF) and two procedures with the Mixed-Model Approaches (the Akaike’s Criterion, the Correctly Identified Model [CIM]) are compared. The data were analyzed using Monte Carlo simulation method with the statistical package SAS 9.2, a split-plot design, and considering six manipulated variables. The results show that the procedures exhibit high statistical power levels for within and interactional effects, and moderate and low levels for the between-groups effects under the different conditions analyzed. For the latter, only the Modified Brown Forsythe shows high level of power mainly for groups with 30 cases and Unstructured (UN) and Autoregressive Heterogeneity (ARH) matrices. For this reason, we recommend using this procedure since it exhibits higher levels of power for all effects and does not require a matrix type that underlies the structure of the data. Future research needs to be done in order to compare the power with corrected selectors using single-level and multilevel designs for fixed and random effects.","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"13 1","pages":"9–22"},"PeriodicalIF":3.1,"publicationDate":"2017-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42853409","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Effect of Partly Missing Covariates on Statistical Power in Randomized Controlled Trials With Discrete-Time Survival Endpoints","authors":"S. Jolani, M. Safarkhani","doi":"10.1027/1614-2241/A000121","DOIUrl":"https://doi.org/10.1027/1614-2241/A000121","url":null,"abstract":"Abstract. In randomized controlled trials (RCTs), a common strategy to increase power to detect a treatment effect is adjustment for baseline covariates. However, adjustment with partly missing covariates, where complete cases are only used, is inefficient. We consider different alternatives in trials with discrete-time survival data, where subjects are measured in discrete-time intervals while they may experience an event at any point in time. The results of a Monte Carlo simulation study, as well as a case study of randomized trials in smokers with attention deficit hyperactivity disorder (ADHD), indicated that single and multiple imputation methods outperform the other methods and increase precision in estimating the treatment effect. Missing indicator method, which uses a dummy variable in the statistical model to indicate whether the value for that variable is missing and sets the same value to all missing values, is comparable to imputation methods. Nevertheless, the power level to detect the treatm...","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"13 1","pages":"41-60"},"PeriodicalIF":3.1,"publicationDate":"2017-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46326950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Anabela Marques, A. Ferreira, Margarida M. G. S. Cardoso
{"title":"Performance of Combined Models in Discrete Binary Classification","authors":"Anabela Marques, A. Ferreira, Margarida M. G. S. Cardoso","doi":"10.1027/1614-2241/a000117","DOIUrl":"https://doi.org/10.1027/1614-2241/a000117","url":null,"abstract":"Diverse Discrete Discriminant Analysis (DDA) models perform differently in different samples. This fact has encouraged research in combined models which seems particularly promising when the a priori classes are not well separated or when small or moderate sized samples are considered, which often occurs in practice. In this study, we evaluate the performance of a convex combination of two DDA models: the First-Order Independence Model (FOIM) and the Dependence Trees Model (DTM). We use simulated data sets with two classes and consider diverse data complexity factors which may influence performance of the combined model – the separation of classes, balance, and number of missing states, as well as sample size and also the number of parameters to be estimated in DDA. We resort to cross-validation to evaluate the precision of classification. The results obtained illustrate the advantage of the proposed combination when compared with FOIM and DTM: it yields the best results, especially when very small samples are considered. The experimental study also provides a ranking of the data complexity factors, according to their relative impact on classification performance, by means of a regression model. It leads to the conclusion that the separation of classes is the most influential factor in classification performance. The ratio between the number of degrees of freedom and sample size, along with the proportion of missing states in the minority class, also has significant impact on classification performance. An additional gain of this study, also deriving from the estimated regression model, is the ability to successfully predict the precision of classification in a real data set based on the data complexity factors.","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"13 1","pages":"23–37"},"PeriodicalIF":3.1,"publicationDate":"2017-02-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41421439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using Conditional Association to Identify Locally Independent Item Sets","authors":"J. Straat, L. V. D. Ark, K. Sijtsma","doi":"10.1027/1614-2241/A000115","DOIUrl":"https://doi.org/10.1027/1614-2241/A000115","url":null,"abstract":"Abstract. The ordinal, unidimensional monotone latent variable model assumes unidimensionality, local independence, and monotonicity, and implies the observable property of conditional association....","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"12 1","pages":"117-123"},"PeriodicalIF":3.1,"publicationDate":"2016-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"57293444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Dealing with data streams: An online, row-by-row, estimation tutorial","authors":"Lianne Ippel, M. Kaptein, J. Vermunt","doi":"10.1027/1614-2241/A000116","DOIUrl":"https://doi.org/10.1027/1614-2241/A000116","url":null,"abstract":"Abstract. Novel technological advances allow distributed and automatic measurement of human behavior. While these technologies provide exciting new research opportunities, they also provide challenges: datasets collected using new technologies grow increasingly large, and in many applications the collected data are continuously augmented. These data streams make the standard computation of well-known estimators inefficient as the computation has to be repeated each time a new data point enters. In this tutorial paper, we detail online learning, an analysis method that facilitates the efficient analysis of Big Data and continuous data streams. We illustrate how common analysis methods can be adapted for use with Big Data using an online, or “row-by-row,” processing approach. We present several simple (and exact) examples of the online estimation and discuss Stochastic Gradient Descent as a general (approximate) approach to estimate more complex models. We end this article with a discussion of the methodolo...","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"12 1","pages":"124-138"},"PeriodicalIF":3.1,"publicationDate":"2016-12-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"57293496","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Meta-Analytic Investigation of the Relationship Between Scale-Item Length, Label Format, and Reliability","authors":"Tyler Hamby, R. Peterson","doi":"10.1027/1614-2241/A000112","DOIUrl":"https://doi.org/10.1027/1614-2241/A000112","url":null,"abstract":"Abstract. Using two meta-analytic datasets, we investigated the effect that two scale-item characteristics – number of item response categories and item response-category label format – have on the reliability of multi-item rating scales. The first dataset contained 289 reliability coefficients harvested from 100 samples that measured Big Five traits. The second dataset contained 2,524 reliability coefficients harvested from 381 samples that measured a wide variety of constructs in psychology, marketing, management, and education. We performed moderator analyses on the two datasets with the two item characteristics and their interaction. As expected, as the number of item response categories increased, so did reliability, but more importantly, there was a significant interaction between the number of item response categories and item response-category label format. Increasing the number of response categories increased reliabilities for scale-items with all response categories labeled more so than for oth...","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"12 1","pages":"89-96"},"PeriodicalIF":3.1,"publicationDate":"2016-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"57293435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. Poncet, D. Courvoisier, C. Combescure, T. Perneger
{"title":"Normality and Sample Size Do Not Matter for the Selection of an Appropriate Statistical Test for Two-Group Comparisons","authors":"A. Poncet, D. Courvoisier, C. Combescure, T. Perneger","doi":"10.1027/1614-2241/A000110","DOIUrl":"https://doi.org/10.1027/1614-2241/A000110","url":null,"abstract":"Abstract. Many applied researchers are taught to use the t-test when distributions appear normal and/or sample sizes are large and non-parametric tests otherwise, and fear inflated error rates if the “wrong” test is used. In a simulation study (four tests: t-test, Mann-Whitney test, Robust t-test, Permutation test; seven sample sizes between 2 × 10 and 2 × 500; four distributions: normal, uniform, log-normal, bimodal; under the null and alternate hypotheses), we show that type 1 errors are well controlled in all conditions. The t-test is most powerful under the normal and the uniform distributions, the Mann-Whitney test under the lognormal distribution, and the robust t-test under the bimodal distribution. Importantly, even the t-test was more powerful under asymmetric distributions than under the normal distribution for the same effect size. It appears that normality and sample size do not matter for the selection of a test to compare two groups of same size and variance. The researcher can opt for the t...","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"12 1","pages":"61-71"},"PeriodicalIF":3.1,"publicationDate":"2016-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"57293420","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Impact of the Number of Dyads on Estimation of Dyadic Data Analysis Using Multilevel Modeling","authors":"H. Du, Lijuan Wang","doi":"10.1027/1614-2241/A000105","DOIUrl":"https://doi.org/10.1027/1614-2241/A000105","url":null,"abstract":"Abstract. Dyadic data often appear in social and behavioral research, and multilevel models (MLMs) can be used to analyze them. For dyadic data, the group size is 2, which is the minimum group size we could have for fitting a multilevel model. This Monte Carlo study examines the effects of the number of dyads, the intraclass correlation (ICC), the proportion of singletons, and the missingness mechanism on convergence, bias, coverage rates, and Type I error rates of parameter estimates of dyadic data analysis using MLMs. Results showed that the estimation of variance components could have nonconvergence problems, nonignorable bias, and deviated coverage rates from nominal values when ICC is low, the proportion of singletons is high, and/or the number of dyads is small. More dyads helped obtain more reliable and valid estimates. Sample size guidelines based on the simulation model are given and discussed.","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"12 1","pages":"21-31"},"PeriodicalIF":3.1,"publicationDate":"2016-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"57293379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Methodological Challenges of Mixed Methods Intervention Evaluations","authors":"H. Boeije, S. Drabble, A. O’Cathain","doi":"10.1027/1614-2241/A000101","DOIUrl":"https://doi.org/10.1027/1614-2241/A000101","url":null,"abstract":"Abstract. This paper addresses the methodological challenges that accompany the use of a combination of research methods to evaluate complex interventions. In evaluating complex interventions, the question about effectiveness is not the only question that needs to be answered. Of equal interest are questions about acceptability, feasibility, and implementation of the intervention and the evaluation study itself. Using qualitative research in conjunction with trials enables us to address this diversity of questions. The combination of methods results in a mixed methods intervention evaluation (MMIE). In this article we demonstrate the relevance of mixed methods evaluation studies and provide case studies from health care. Methodological challenges that need our attention are, among others, choosing appropriate designs for MMIEs, determining realistic expectations of both components, and assigning adequate resources to both components. Solving these methodological issues will improve our research designs an...","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"11 1","pages":"119-125"},"PeriodicalIF":3.1,"publicationDate":"2015-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"57293358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Methodological Issues in Categorical Data Analysis","authors":"J. Hagenaars","doi":"10.1027/1614-2241/A000102","DOIUrl":"https://doi.org/10.1027/1614-2241/A000102","url":null,"abstract":"Abstract. The “General Linear Reality” view of the social world endorsed by analysis models assuming (underlying) continuous variables that are normally distributed is still prevailing in most of s...","PeriodicalId":18476,"journal":{"name":"Methodology: European Journal of Research Methods for The Behavioral and Social Sciences","volume":"11 1","pages":"126-141"},"PeriodicalIF":3.1,"publicationDate":"2015-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"57293370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}