{"title":"Using Generalizability Theory to Assess the Reliability of Student Ratings of Academic Advising","authors":"A. Sun, M. Valiga, Xiaohong Gao","doi":"10.1080/00220973.1997.10806611","DOIUrl":"https://doi.org/10.1080/00220973.1997.10806611","url":null,"abstract":"The coefficient alpha procedure has been used frequently to assess the reliability of student ratings of academic advising, despite the fact that in many cases, coefficient alpha is not an appropriate procedure for providing reliability information on this type of measurement data. Data were collected from 15 postsec ondary institutions' use of the Survey of Academic Advising, and a more advanced framework, that is, the generalizability theory, was used to analyze the reliability of the data. The results of the study demonstrate that the reliability of student ratings of advising can be estimated more appropriately and accurately by using the proce dures of generalizability theory. Advantages and disadvantages of the generalizabil ity theory approach in assessing the reliability of student ratings of advising are dis cussed in comparison with those of the coefficient alpha procedure. USING STUDENT RATINGS as an outcome assessment to evaluate the qual ity of advising programs is a common practice in many postsecondary educa tional institutions. The results of such assessment often play a critical role in decisions about the future of the individual advisor?decisions about promotions and tenure and about the advising program, such as whether or not to continue a specific program or service (Severy, Lee, Carodine, Powers, & Mason, 1994). To justify the credibility of a decision, the decisionmaker needs some assurance from the data that the same or similar results could be obtained if the same advi sors and advising programs were rated again under similar circumstances. In measurement theory terms, these ratings should have a desirable level of relia bility.","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"65 1","pages":"367-379"},"PeriodicalIF":2.2,"publicationDate":"1997-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220973.1997.10806611","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58956020","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Structure of the Theoretical Concept of Educational Goals: A Test of Factorial Validity.","authors":"Jianping Shen","doi":"10.1080/00220973.1997.10806609","DOIUrl":"https://doi.org/10.1080/00220973.1997.10806609","url":null,"abstract":"How educational goals are conceptualized as a construct has implica tions for developing and evaluating educational programs and activities. The author used EQS 3.0 to test the factorial validity of 4 models of conceptualization of educa tional goals by conducting confirmatory factor analysis on empirical data collected from 1,134 education faculty members. The 4-factor model espoused by Goodlad fit the data best, and the participants wanted academic, vocational, personal, and social goals to be in relative harmony. Implications for educational theory, practice, and research are discussed. IN EDUCATIONAL THEORY AND PRACTICE, educational goals have been conceptualized in various ways. Some educators conceptualize educational goals as an all-inclusive, undifferentiated entity; others tend to view educational goals as the personal versus the social, or the academic versus the vocational; still others hold that the concept of educational goals includes personal, social, aca demic, and vocational elements. My purpose in the present study was to test the factorial validity of different conceptions of educational goals. The concept of educational goals has evolved in the history of educational thought. According to Walker and Soltis (1986), in The Republic the Greek philosopher Plato distinguished between the academic and vocational dimen sions of educational goals. The ancient Greeks believed that human beings had bodies that were animated and driven by a three-part soul. Each soul had an appetitive part, which expressed desires and needs and sought their fulfillment; a spirited part, which put aside unnecessary needs in the interests of self-protection and survival; and a rational part, which rose above both appetite and physical action to provide good judgment through reason. Therefore, according to Plato, educational goals were to provide academic education to people whose souls were dominated by rationality and to offer vocational education to those whose","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"65 1","pages":"342-352"},"PeriodicalIF":2.2,"publicationDate":"1997-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220973.1997.10806609","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58955864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Tests for Comparing Dependent Correlations Revisited: A Monte Carlo Study.","authors":"K. May, J. Hittner","doi":"10.1080/00220973.1997.9943458","DOIUrl":"https://doi.org/10.1080/00220973.1997.9943458","url":null,"abstract":"Abstract A Monte Carlo evaluation of 4 test statistics for comparing dependent zero-order correlations was conducted. In particular, the power and Type I error rates of Hotelling's t; Williams' t; Olkin's z; and Meng, Rosenthal, and Rubin's Z were evaluated for sample sizes of 20, 50, 100, and 300 under 3 different population distributions (normal, uniform, and exponential). For the power analyses, 3 different magnitudes of discrepancy or effect sizes between ρy, x1 , and ρy, x2 were examined (values of .1, .3, and .6). Likewise, for the Type I error rate analyses, 3 different magnitudes of the predictor-criterion correlations were evaluated (ρy, x1 = ρy, x2 = .1, .4, and .7). All of the analyses were conducted at 3 different levels of predictor intercorrelation (ρx1, x2 = .1, .3, and .6). The results indicated that the choice as to which test statistic is optimal, in terms of power and Type I error rate, depends not only on sample size and population distribution but also on (a) the predictor intercorrel...","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"65 1","pages":"257-269"},"PeriodicalIF":2.2,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220973.1997.9943458","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58956486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Using SAS PROC MIXED To Demystify the Hierarchical Linear Model","authors":"Jianjun Wang","doi":"10.1080/00220979709601397","DOIUrl":"https://doi.org/10.1080/00220979709601397","url":null,"abstract":"Abstract Hierarchical data analyses in different disciplines are reviewed to compare statistical applications of the Hierarchical Linear Model (HLM) software and the SAS MIXED procedure. Similar features of the 2 programs are illustrated through use of the SAS MIXED procedure to confirm an HLM example. The SAS is a standard statistical package with a large group of users; discussions of the shared features in statistical computing may identify options to demystify existing methods for analyzing hierarchical data.","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"66 1","pages":"84-93"},"PeriodicalIF":2.2,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220979709601397","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58956833","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The effect of year-long instruction in mathematical problem solving on middle-school students' attitudes, beliefs, and abilities","authors":"K. Higgins","doi":"10.1080/00220979709601392","DOIUrl":"https://doi.org/10.1080/00220979709601392","url":null,"abstract":"Abstract Three classes of middle-school students who received 1 year of problem-solving instruction were compared with 3 classes of students who were taught mathematics in a more traditional manner. At the end of the school year, all the students completed a questionnaire that explored their mathematical beliefs. In addition, 3 students of varying ability levels per class were interviewed and asked to solve 4 nonroutine problems. Compared with the students who had received traditional mathematics instruction, the students who had received problem-solving instruction displayed greater perseverance in solving problems, more positive attitudes about the usefulness of mathematics, and more sophisticated definitions of mathematical understanding. A limitation of the problem-solving instruction is that the students tended to equate problem solving with the problem-solving skills they had learned, seeing them as “rules” to solve all problems. The implications of these findings for the reform of mathematics instr...","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"36 1","pages":"5-28"},"PeriodicalIF":2.2,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220979709601392","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58956948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Family involvement with middle-grades homework: Effects of differential prompting","authors":"S. Balli, Jonathan Wedman, David H. Demo","doi":"10.1080/00220979709601393","DOIUrl":"https://doi.org/10.1080/00220979709601393","url":null,"abstract":"A middle-grades homework intervention was investigated to determine if variations in prompting families to be involved with mathematics homework would influence their level of involvement. The extent to which family involvement was a predictor of student achievement in mathematics was also examined, as were the relationships among family involvement, student achievement, and parent education level. Families in the 2 prompted groups were significantly more involved with mathematics homework than were families in the no-prompt group. Level of family involvement was not significantly related to student achievement on the post-test. However, students across the 3 groups whose parent(s) held a 4-year college degree scored significantly higher on the post-test than did students neither of whose parents held a college degree, even though reported levels of family involvement were nearly identical across parent education levels. Qualitative data elicited in follow-up interviews with family members indicated that \"quality of involvement\" with homework merits examination in future research.","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"66 1","pages":"31-48"},"PeriodicalIF":2.2,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220979709601393","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58957027","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Effects of Reciprocal Peer Tutoring on Graduate Students' Achievement, Test Anxiety, and Academic Self-Efficacy.","authors":"B. W. Griffin, M. Griffin","doi":"10.1080/00220973.1997.9943454","DOIUrl":"https://doi.org/10.1080/00220973.1997.9943454","url":null,"abstract":"Abstract Two experiments were conducted to determine the effects of reciprocal peer tutoring (RPT) on graduate students' academic achievement, test anxiety, and academic self-efficacy. In Experiment 1, students were assigned to one of two conditions—RPT or non-RPT. RPT students developed questions on specific course topics; they then used these questions to quiz other students before taking midterm and final course examinations. Contrary to previous research findings, the RPT and control groups did not differ significantly on lower cognitive or higher cognitive achievement measures. Students who used RPT generally reported that RPT improved their understanding of course content In Experiment 2, the RPT procedures were modifled to better match the procedures used in earlier studies whose authors had found RPT to be superior to non-RPT conditions. Also, the instruments were expanded to include course-relevant outcome measures of student academic self-efficacy and test anxiety. As in Experiment 1, the RPT an...","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"65 1","pages":"197-209"},"PeriodicalIF":2.2,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220973.1997.9943454","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58956048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Alexander and Govern's approximation : Providing an alternative to ANOVA under variance heterogeneity","authors":"P. Schneider, D. A. Penfield","doi":"10.1080/00220973.1997.9943459","DOIUrl":"https://doi.org/10.1080/00220973.1997.9943459","url":null,"abstract":"A Monte Carlo simulation was conducted to compare the Type I error rate and power of the 1994 approximation developed by Alexander and Govern as an alternative to the ANOVA F test. It was compared with the ANOVA F, Kruskal-Wallis, Welch, Brown-Forsythe, and James second-order tests. The authors concentrated on the impact of various factors on Type I error rate and power. The factors included variance inequality, sample-size pairings with group variances, degree of skewness/kurtosis, and number of treatment groups. Under variance heterogeneity, Alexander-Govern's test was not only comparable to the performance of the Welch test and the James second-order test but was superior in certain instances.","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"65 1","pages":"271-286"},"PeriodicalIF":2.2,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220973.1997.9943459","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58956546","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Longitudinal Study of Persisting and Nonpersisting Teachers' Academic and Personal Characteristics.","authors":"R. N. Marso, F. L. Pigge","doi":"10.1080/00220973.1997.9943457","DOIUrl":"https://doi.org/10.1080/00220973.1997.9943457","url":null,"abstract":"Abstract This longitudinal study was designed to compare the academic and personal characteristics of teacher candidates who persist and do not persist through teacher preparation and the early years of teaching. The candidates' (N = 551) characteristics were collected upon commencement of teacher preparation; 7 years later the candidates were classified by degree of persistence as follows: (a) not certified as teachers, (b) certified but not teaching, (c) part-time teachers, and (d) full-time teachers. The candidates' gender, major, initial assurance about teaching, and time at which they decided to become teachers were found to be associated with their degree of persistence; their level of academic aptitude, basic academic skills, and expected effectiveness as future teachers were found not to be associated with their degree of persistence. Relationships between these findings and findings from the National Longitudinal Survey of teacher attrition are discussed, and implications for effective teacher re...","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"65 1","pages":"243-254"},"PeriodicalIF":2.2,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220973.1997.9943457","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58956442","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Mathematics Self-Efficacy and Mathematical Problem Solving: Implications of Using Different Forms of Assessment","authors":"Frank Pajares, M. David Miller","doi":"10.1080/00220973.1997.9943455","DOIUrl":"https://doi.org/10.1080/00220973.1997.9943455","url":null,"abstract":"Abstract The mathematics self-efficacy and problem-solving performance of 327 middle-school students were assessed using two forms of assessment (traditional multiple-choice vs. open-ended fill-in-the-blank). The purpose was to determine whether varying the assessment format would influence students' self-efficacy judgments or alter the relationship between self-efficacy and performance. No differences in self-efficacy resulted from the different forms of assessment Students who took the multiple-choice performance test obtained higher scores than did students who took the open-ended test The latter group had poorer calibration, that is, the degree to which students' judgments of their capability reflect their actual competence. This finding suggests that students' self-perceptions of their mathematics capability may be less accurate than has previously been reported or that students' familiarity with traditional assessment formats creates an expectancy of a performance task that is multiple choice in nat...","PeriodicalId":47911,"journal":{"name":"Journal of Experimental Education","volume":"65 1","pages":"213-228"},"PeriodicalIF":2.2,"publicationDate":"1997-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1080/00220973.1997.9943455","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"58956233","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}