Browse
Recent Submissions
Item The relationship between managers and operational performance: a proposed model at Australian public sports and leisure centres(University of Otago, 2004) Keeves, John Philip; Sharp, Colin Andrew; Crilley, G; Darmawan, I Gusti NgurahItem The multilevel analysis of students' achievement in learning the Chinese language.(Shannon Research Press, 2001-12) Yuan, Ruilan; Keeves, John PhilipA three-level hierarchical or multilevel model is employed to examine the factors that influence students' achievement in learning the Chinese language. The factors might be many or various, such as school factors, factors related to teachers, parents and peers. This study, however, only examines student level factors. The first level of the three-level multilevel model is the within-student level; the second level is between student level; and the third level is between-class level. The results show that different factors at different levels of the model have effects on the students' achievement. Interaction effects are also observed between some variables and the achievement.Item The development of scales to measure students', teachers' and scientists' views on STS.(Shannon Research Press, 2001-03) Tedman, Debra K; Keeves, John PhilipThe starting point for this work on the development of scales was an existing instrument concerned with Views on Science, Technology and Society (STS) which had been prepared in Canada. This Australian study developed scales to measure views towards science, technology and society, and, it was necessary initially to specify scores to the alternative responses or views for each of the statements included in the scales used in this study. The initial scores or codes for the scales were based upon preliminary analysis and the researcher's judgment derived from a review of the literature. Subsequently, a validation study used the opinions of experts to confirm the numerical codes assigned to the responses. It was also necessary to test the items in each of the scales to see whether the model of a unidimensional scale was consistent with recorded data. It was possible to show that by using the numerical codes, the chosen items fitted well their respective scales. Once the three scales (a) effects of Society on Science and Technology (Society), (b) the effects of Science and Technology upon Society (Science), and (c) characteristics of Scientists (Scientists), had been specified and items were identified that satisfied the requirement of unidimensionality, it was possible to calibrate the three scales and the items within them using the partial credit model for Rasch scaling. The construction and calibration of these three scales permitted an investigation to proceed that involved the accurate measurement of students', teachers' and scientists' views on STS. [ Author abstract]Item Accountability of teachers and schools : a value-added approach.(International Education Journal, 2006-06) Darmawan, I Gusti Ngurah; Keeves, John PhilipCurrently, there has been substantial interest, in Australia and internationally, in policy activities related to outcomes-based educational performance indicators and their link with growing demands for accountability of teachers and schools. In order to achieve a fair comparison between schools, it is commonly agreed that a correction should be made for lack of equity. It is argued that student performance is influenced by three general factors: the student background, classroom and school context, and identified school policies and practices. In this article the effects of these three factors on science achievement among students in Canberra, Australia have been addressed. The effects are discussed with reference to Type A, Type B, Type X, and Type Z effects. Type A effects are school effectiveness indicators controlled for student background. Type B school effects are controlled for both student background and context variables. Type X effects are estimated with student effects, context effects and non-malleable policy effects controlled for. Finally, Type Z effects invoke school effectiveness indicators, controlled for student, context, and all identified policy effects. [Author abstract]Item Suppressor variables and multilevel mixture modelling.(Shannon Research Press, 2006-06) Darmawan, I Gusti Ngurah; Keeves, John PhilipA major issue in educational research involves taking into consideration the multilevel nature of the data. Since the late 1980s, attempts have been made to model social science data that conform to a nested structure. Among other models, two-level structural equation modelling or two-level path modelling and hierarchical linear modelling are two of the techniques that are commonly employed in analysing multilevel data. Despite their advantages, the two-level path models do not include the estimation of cross-level interaction effects and hierarchical linear models are not designed to take into consideration the indirect effects. In addition, hierarchical linear models might also suffer from multicollinearity that exists among the predictor variables. This paper seeks to investigate other possible models, namely the use of latent constructs, indirect paths, random slopes and random intercepts in a hierarchical model. [Author abstract]Item Some problems in the analysis of cross-national survey data(Shannon Research Press, 2006-06) Keeves, John Philip; Lietz, Petra; Gregory, Kelvin David; Darmawan, I Gusti NgurahIn this article three emergent problems in the analysis of cross-national survey data are raised in a context of 40 years of research and development in a field where persistent problems have arisen and where scholars across the world have sought solutions. Anomalous results have been found from secondary data analyses that would appear to stem from the procedures that have been employed during the past 15 years for the estimation of educational achievement. These estimation procedures are briefly explained and their relationships to the observed anomalies are discussed. The article concludes with a challenge to the use of Bayesian estimation procedure, while possibly appropriate for the estimation of population parameters would appear to be inadequate for modelling scores that are used in secondary data analyses. Consequently, an alternative approach should be sought to provide data on the performance of individual students, if a clearer and more coherent understanding of educational processes is to be achieved through cross-national survey research. [Author abstract, ed]Item A view from outside the confines of South Australia.(Shannon Research Press, 2006-11) Keeves, John Philiphe SACE Review report, Success for All, completely ignores two important issues, namely, (a) the portability of the certificate, and (b) the nature of secondary schooling in a future that is set in a global world. The Review saw the South Australian education system operating in a context that was limited to the geographical and cultural boundaries of the state. This paper discusses both of these issues that appear to require the resolution of conflicting and incompatible problems. In conclusion the paper considers the changing nature of schooling and the role of alternative education in both schools and programs and rejects the continuance of comprehensive schooling at the upper secondary school level within a bureaucratic education system. The paper argues that different types of schools should be gradually established through the self-management and self-governance of schools by the communities served and the choices made by the students who attend the schools and their parents, and who support them while they are engaged in upper secondary education. [Author abstract]Item Research and national debate on Australian schooling(Shannon Research Press, 2006-11) Keeves, John Philip; Curtis, David DThis paper is a response to the paper prepared by Masters that is titled 'The case for an Australian Certificate of Education'. It argues that a national debate is needed urgently on the many issues that have arisen in Australian education. These issues include not only the curriculum provided for students at the final stages of secondary schooling, and the certification of attainment of educational outcomes on completion of 12 years of schooling, but also the curriculum of schools across Australia, particularly at the lower and middle secondary school levels. In addition, there are related issues associated with participation in higher education and the completion of a first degree at an Australian university. All too often, decisions are made at all levels of education on ideological grounds and without consideration of the body of research findings that are available to guide the making of decisions and the monitoring of development and change. This paper draws on readily available research to show the similarities and differences between the state education systems to argue a case for informed debate that draws on the large body of evidence that is available.Item Monitoring the learning and teaching of science in a changing world.(Shannon Research Press, 2004-09) Keeves, John PhilipThis article addresses the issues involved in monitoring the teaching and learning of science in a changing world. It examines the development of cross-cultural studies of educational achievement, particularly in the field of science, including the theoretical basis of the studies and the models advanced and used in evaluation and more recently in the monitoring of change over time. In addition ten specific issues are identified for investigation into the critical problems facing learning and teaching of science across the world at the beginning of the twenty-first century with particular reference to the PISA studies being conducted by the Organisation for Economic Cooperation and Development. [Author abstract]Item Learning in schools : a modelling approach(Shannon Research Press, 2002-07) Keeves, John PhilipThis article claims that constructivism is both incomplete and inadequate for the effective learning and teaching of mathematics and science at the upper secondary school level. The article briefly reviews the reforms that have occurred over the past 50 years on mathematics and science education from the perspectives provided by advances in knowledge on the physical and biological sciences, in developmental and cognitive psychology, in educational research and in the emerging field of neuroscience. It is argued that the finding from these many different fields of research must be brought together to advance learning through a modelling approach which requires that both individual and corporate knowledge must be tested not only for coherence, but also for adequacy against evidence obtained from the real world in which human beings are living and undertaking their inquiries. [Author abstract]Item Change in differences between the sexes in mathematics achievement at the lower secondary school level in Australia : over time.(Shannon Research Press, 2001-07) Afrassa, Tilahun M; Keeves, John PhilipIn this paper an investigation is reported on whether changes have occurred in the differences between the sexes in mathematics achievement at the lower secondary school level over the 30 year period from 1964 to 1994. In order to make meaningful comparisons the mathematics test scores from the three studies conducted in Australia under the auspices of the International Association for Evaluation of Educational Achievement were brought to a common interval scale using Rasch measurement procedures. The scale scores are used to examine differences between boys and girls in mathematics achievement on the three occasions as well as the changes that have occurred between occasions. No significant sex differences in mathematics achievement are found on each of the occasions. However, a significant decline in mathematics achievement is recorded for boys between 1964 and 1994, but not for girls. The decline in mathematics achievement over this 30 year period for boys is equivalent to nearly one year of mathematics learning, while the drop for girls is only approximately equivalent to half a year of mathematics learning. [Author abstract]Item Examining the validity of different assessment modes in measuring competence in performing human services.(Shannon Research Press, 2004-07) Hungi, Njora; Darmawan, I Gusti Ngurah; Keeves, John PhilipThis article addresses an important problem that faces educators in assessing students' competence levels in learned tasks. Data from 165 students from Massachusetts and Minnesota in the United States are used to examine the validity of five assessment modes (multiple choice test, scenario, portfolio, self-assessment and supervisor rating) in measuring competence in performance of 12 human service skills. The data are examined using two analytical theories, item response theory (IRT) and generalizability theory (GT), in addition a prior, but largely unprofitable examination using classical test theory (CTT) was undertaken. Under the IRT approach with Rasch scaling procedures, the results show that the scores obtained using the five assessment modes can be measured on a single underlying scale, but there is better fit of the model to the data if five scales (corresponding to the five assessment modes) are employed. In addition, under Rasch scaling procedures, the results show that, in general, the correlations between the scores of the assessment modes vary from small to very strong (0.11 to 0.80). However, based on the GT approach and hierarchical linear modelling (HLM) analytical procedures, the results show that the correlations between scores from the five assessment modes are consistently strong to very strong (0.53 to 0.95). It is argued that the correlations obtained with the GT approach provide a better picture of the relationships between the assessment modes when compared to the correlations obtained under the IRT approach because the former are computed taking into consideration the operational design of the study. Results from both the IRT and GT approaches show that the mean values of scores from supervisors are considerably higher than the mean values of scores from the other four assessments, which indicate that supervisors tend to be more generous in rating the skills of their students. [Author abstract]Item Two-level model of information technology adoption in local government of Bali(Shannon Research Press, 2002-03) Keeves, John Philip; Darmawan, I Gusti NgurahRecent advance in information technology has aroused much interest among policy-makers, the business sector, the media and the academic world in developed countries. However, very little is known about the obstacles to adopting information technology in developing countries, particularly the low-income economies. This research stems from the issues described above. A variety of factors may influence the adoption of IT in local governments. Those factors range from technological and institutional to personal, social and economic factors. These factors included not only the information on variables gathered at the employee level but also on questions regarding the characteristics of each organization involved in the study. Hence the data files contain information obtained at two different levels, namely individual level and organisational level. Therefore a two-level hierarchical linear model is used to examine the relationships among variables. [Author abstract]Item Errors: What are they and how significant are they?(Shannon Research Press, 2000-12) Keeves, John PhilipErrors in educational research and measurement arise from four main sources: (1) errors associated with the characteristic being measured or intrinsic errors, (2) errors arising from the instrument being used or instrumental errors, (3) errors involved in the act of measurement or observational errors, and (4) errors arising from the process of sampling or sampling errors. The word 'error' has many meanings. The most common meaning is concerned with the idea of a ' mistake' which does not apply in this context. A further meaning is concerned with 'the difference between an observed or estimated numerical result and the true or exact one'. However, in educational research the 'true value' is both unknown and unknowable and this meaning does not apply. This paper is concerned with the examination of errors in several recent Australian research studies.Item The Course Experience Questionnaire as an Institutional Performance Indicator(Shannon Research Press, 2000-07) Curtis, David D; Keeves, John PhilipData from the 1996 Course Experience Questionnaire (CEQ) were analysed using the Rasch measurement model. This analysis indicates that 17 of the 25 CEQ items fit a unitary scale that measures course quality as perceived by graduates. Graduates are located on the interval measurement scale produced in the Rasch analysis. The interval nature of the scale renders the graduates' scores amenable to analyses that are not wisely employed using ordered raw CEQ scores. Analysis of variance indicates that variations in graduates' responses are attributable to field of study and institutional factors. In order to compare universities, corrections are made for the course mix of each institution to produce expected institutional scores. These are compared with observed institutional scores to determine those universities that have performed above, at, or below expectation. (Individual institutions are not identified in this analysis). Important issues relating to the educational and statistical significance of the findings have emerged. The data collected through the CEQ do not represent a simple random sample of all graduates. Instead, the data model is a hierarchical one, with individual graduates nested within courses, which are nested within institutions. This requires analysis using multilevel analytical tools. Conventional analyses substantially underestimate the standard errors of aggregated measures (such as institutional means) and therefore report institutional differences as significant when they are not. The implications of the measurement and analytical problems for policy decisions over the distribution of funding among institutions and among courses within institutions are discussed.