Scale Development, Rasch Analysis and Item Response Theory

Scales, measures and tests are commonly used in the social and health sciences as research and clinical tools. It is important that these are well designed and are reliable and valid for the purpose intended.

 
Level 2 - runs over 5 days
Instructor: 

Julie Pallant is an Assoc Prof (adjunct) with the School of Nursing and Midwifery, Griffith University and was previously employed as Research Support Coordinator at the Rural Health Academic Centre, Faculty of Medicine, University of Melbourne She is the author of the SPSS Survival Manual: A step by step guide to data analysis using SPSS.

Julie has extensive experience using, and teaching, Rasch analysis using the RUMM2030 package. She has published over 100 peer reviewed articles, with 40 of these using Rasch Analysis. She provides research and statistical consulting and training to a wide variety of individuals and organizations, both nationally and internationally.

Course dates: Monday 3 July 2017 - Friday 7 July 2017
Week: 
Week 2
Course status: Course completed (no new applicants)
About this course: 

This course provides an overview of the scale development process and the statistical techniques used including Reliability analysis, Factor Analysis and Rasch Analysis. Rasch analysis is one of the modern psychometric techniques that form part of Item Response Theory. Using specially designed software (RUMM2030) this technique allows a formal assessment of the measurement properties of scales and tests. It provides detailed feedback on all aspects of the scale including the response format, the fit of the items, item bias, unidimensionality and the spread of items across the trait being measured. Rasch analysis also allows a linear transformation of ordinal scales to interval level measures, permitting the use of parametric statistical techniques and the calculation of change scores.

 

This course takes a practical applied approach (rather than mathematical) and participants are provided with detailed, hands-on experience. SPSS will be used for practical demonstrations, examples and exercises relating to reliability and factor analysis. RUMM2030 is used for the Rasch analysis training and participants will be provided with a limited licence version of the software.

 

The course is relevant to researchers working in the areas of psychology, health, medicine, business, management and the social sciences, who are interested in developing or refining scales and instruments used to assess human behaviour, emptions, opinions or experiences.

Course syllabus: 

Day 1
Introduction to measurement and testing
Theoretical frameworks – Classical Test Theory (CTT), Item Response Theory (IRT)
Overview of the scale development process

 

Day 2
Development of a test plan, item generation, selection of response format
Reliability of a scale
Assessing the reliability of a scale using SPSS

  • internal consistency (Cronbach alpha coefficient)
  • test-retest reliability

Scale validity (construct, convergent, divergent, incremental)

 

Day 3
Testing the dimensionality of a scale using SPSS

  • Exploratory factor analysis (EFA) vs Confirmatory factor analysis (CFA)
  • Assessing the factorability of a scale
  • Extracting factors – identifying the number of factors to retain
  • Rotating factors to aid interpretation
  • Refining a scale by removing items
  • Writing up the results of an EFA

Overview of Rasch Analysis

  • Underlying theory
  • Comparison with classical test approaches
  • Demonstration of a worked example
  • Preparing the data file for Rasch analysis using SPSS

 

Day 4
Introduction to RUMM2030 software used to conduct Rasch Analysis

  • Importing data into RUMM2030
  • Evaluating overall model fit
  • Checking the response format (thresholds) and rescoring items
  • Detecting misfitting items and persons
  • Identifying item bias (differential Item Functioning)
  • Assessing the dimensionality and targeting of a scale
  • Refining a scale based on the results of Rasch Analysis

 
Day 5
Exporting Rasch scores to SPSS for other analyses
Preparation of conversion tables
Writing up the results of Rasch Analysis
Practice using Rasch analysis using student's own data or provided data sets.

Course format: 

This course is run in a computer lab unless otherwise notified. All equipment will be provided.

Recommended Background: 

An understanding of elementary statistics, equivalent to the syllabus of Fundamentals of Statistics is required. This course is designed for people with little or no experience of scale development, factor analysis or Rasch analysis.

Recommended Texts: 

There is no textbook for this course but participants will be provided with a detailed set of notes. Course participants are encouraged to bring existing scales, questionnaires and/or data sets to class for discussion.

FAQ: 

Q: Do I need any prior knowledge before taking this course?

A: Please see recommended background for prior knowledge required

Participant feedback: 

Factor analysis & Rasch will be very helpful for my PHD. (Summer 2016)

 

The instructor provided easy to follow notes, whilst telling us through the content and theory using power points, examples and a straight forward and also entertaining explanations. (Summer 2015)

 

There was a good balance of lecture, hands on activities and time to consult Julie. Julie always challenged us appropriately by making sure we tried things ourselves. (Summer 2015)

 

I can see myself applying what I learnt in this course in my research work in the near future. (Summer 2015)

 

Focused and yet took into account varying backgrounds. (Summer 2015)

 

Good balance of theory & practice. (Summer 2014)

 

It was hands on, we could look at our own data and get advice throughout the week. (Summer 2014)

 

Excellent paced, resourced and explained. ’Pitched’ at a perfect level of difficulty. (Summer 2014)

 

Program: 
Winter Program 2017
Notes: 

The instructor's bound, book length course notes will serve as the course text.