Scales, measures and tests are commonly used in the social and health sciences as research and clinical tools. It is important that these are well designed and are reliable and valid for the purpose intended.
This course provides an overview of the scale development process and the statistical techniques used including Reliability analysis, Factor Analysis and Rasch Analysis. Rasch analysis is one of the modern psychometric techniques that form part of Item Response Theory. Using specially designed software (RUMM2030) this technique allows a formal assessment of the measurement properties of scales and tests. It provides detailed feedback on all aspects of the scale including the response format, the fit of the items, item bias, unidimensionality and the spread of items across the trait being measured. Rasch analysis also allows a linear transformation of ordinal scales to interval level measures, permitting the use of parametric statistical techniques and the calculation of change scores.
This course takes a practical applied approach (rather than mathematical) and participants are provided with detailed, hands-on experience. SPSS will be used for practical demonstrations, examples and exercises relating to reliability and factor analysis. RUMM2030 is used for the Rasch analysis training and participants will be provided with a limited licence version of the software.
The course is relevant to researchers working in the areas of psychology, health, medicine, business, management and the social sciences, who are interested in developing or refining scales and instruments used to assess human behaviour, emptions, opinions or experiences.
Day 1
Introduction to measurement and testing
Theoretical frameworks – Classical Test Theory (CTT), Item Response Theory (IRT)
Overview of the scale development process
Day 2
Development of a test plan, item generation, selection of response format
Reliability of a scale
Assessing the reliability of a scale using SPSS
- internal consistency (Cronbach alpha coefficient)
- test-retest reliability
Scale validity (construct, convergent, divergent, incremental)
Day 3
Testing the dimensionality of a scale using SPSS
- Exploratory factor analysis (EFA) vs Confirmatory factor analysis (CFA)
- Assessing the factorability of a scale
- Extracting factors – identifying the number of factors to retain
- Rotating factors to aid interpretation
- Refining a scale by removing items
- Writing up the results of an EFA
Overview of Rasch Analysis
- Underlying theory
- Comparison with classical test approaches
- Demonstration of a worked example
- Preparing the data file for Rasch analysis using SPSS
Day 4
Introduction to RUMM2030 software used to conduct Rasch Analysis
- Importing data into RUMM2030
- Evaluating overall model fit
- Checking the response format (thresholds) and rescoring items
- Detecting misfitting items and persons
- Identifying item bias (differential Item Functioning)
- Assessing the dimensionality and targeting of a scale
- Refining a scale based on the results of Rasch Analysis
Day 5
Exporting Rasch scores to SPSS for other analyses
Preparation of conversion tables
Writing up the results of Rasch Analysis
Practice using Rasch analysis using student's own data or provided data sets.
This course is run in a computer lab. All equipment will be provided.
An understanding of elementary statistics, equivalent to the syllabus of Fundamentals of Statistics is required. This course is designed for people with little or no experience of scale development, factor analysis or Rasch analysis.
There is no textbook for this course but participants will be provided with a detailed set of notes. Course participants are encouraged to bring existing scales, questionnaires and/or data sets to class for discussion.
Q: Do I need any prior knowledge before taking this course?
A: Please see recommended background for prior knowledge required
Good balance of theory & practice. (Summer 2014)
It was hands on, we could look at our own data and get advice throughout the week. (Summer 2014)
Excellent paced, resourced and explained. ’Pitched’ at a perfect level of difficulty. (Summer 2014)
The instructor provided easy to follow notes, whilst telling us through the content and theory using power points, examples and a straight forward and also entertaining explanations. (Summer 2015)
There was a good balance of lecture, hands on activities and time to consult Julie. Julie always challenged us appropriately by making sure we tried things ourselves. (Summer 2015)
I can see myself applying what I learnt in this course in my research work in the near future. (Summer 2015)
Focused and yet took into account varying backgrounds. (Summer 2015)
The instructor's bound, book length course notes will serve as the course text.