Revistes Catalanes amb Accés Obert (RACO)

Testing invariance between web and paper students satisfaction surveys. A case study

Mónica Martínez Gómez, Juan A. Marin Garcia, Martha Giraldo O'Meara


Purpose: This paper studied the measurement invariance (MI) across web-based and paperbased
surveys to evidence if both techniques of data collection can be regarded as equivalent.
Design/methodology: We develop a multigroup confirmatory factor analysis (MGCFA) with
Maximum Likelihood Estimation to asses measurement invariance of the Job Diagnostic
Survey (JDS) adapted to teaching, with data collected from paper and web surveys. Sample
from paper surveys was constituted by 294 student of a Spanish public university in the
academic years 2007-08, 2008-09 and 2009-10. Internet surveys were administered through an
open source survey application called Lime Survey. We received 241 completed questionnaires.
Findings: Results show that metric invariance, covariance invariance, variance of latent factors
invariance and measurement errors invariance can be established between two groups. We can
conclude that both methods of collecting data can be considered equivalent.
Research limitations/implications: This study was done with a particular sample and strict
focus questionnaire and we might not generalize the findings. It should be extended in the
future to include other universities and graduate students.
Originality/value: Results showed that the factor structures remained invariant across the
internet-based and paper-based groups, that is to say, both methods of collecting data can be
considered equivalent, with the same factor structure, factor loadings, measurement errors of
factors and the same reliability. These findings are useful for researchers since they add a new
sample in which web and paper questionnaires are equivalent and for teachers to desire to
change the teaching methodology at university, encourage students’ participation and teamwork
through active methodologies.

Text complet: Text complet