TY - JOUR
T1 - Investigation Into the Effects of Using Normal Distribution Theory Methodology for Likert Scale Patient-Reported Outcome Data From Varying Underlying Distributions Including Floor/Ceiling Effects
AU - DeWees, Todd A.
AU - Mazza, Gina L.
AU - Golafshar, Michael A.
AU - Dueck, Amylou C.
N1 - Publisher Copyright:
© 2020 ISPOR–The Professional Society for Health Economics and Outcomes Research
PY - 2020/5
Y1 - 2020/5
N2 - Objectives: Utilization of parametric or nonparametric methods for testing Likert scale data is often debated. This 2-part simulation study aims to investigate the sampling distribution of various Likert scale distributions (including floor/ceiling effects) and analyze the effectiveness of using parametric versus nonparametric tests with varying sample sizes. Methods: We simulated populations from parametric distributions binned into Likert scales. In study 1, replicates were sampled from each distribution with sizes ranging from 5 to 150 observations, calculating means with simulated 95% CIs at each sample size. In study 2, floor/ceiling effects were introduced such that the proportion of patients responding with the lowest rating varied from approximately 40% to 90%. Two-sample tests were then conducted for the 90% floor effect distribution against all other floor distributions to determine effectiveness of parametric versus nonparametric methods via 2-sided pooled t tests and Wilcoxon rank-sum tests. Coverage of the difference in means, realized P values, relative efficiency, measures of agreement in direction, and conclusion of tests were plotted by sample size. Results: The sampling distributions of the 1-sample means and SDs for most distributions converged quickly to Gaussian, with 95% coverage. One- and 2-sample t tests of the mean demonstrated acceptable coverage, type I error, and agreement. Conclusions: Simulations confirm that the sampling distribution of the mean rapidly approaches normality and appropriate tests provide adequate coverage and type I error. Two-sample t tests demonstrate appropriateness and increased statistical power gained by using parametric over nonparametric approaches, suggesting t tests should be implemented with few restrictions.
AB - Objectives: Utilization of parametric or nonparametric methods for testing Likert scale data is often debated. This 2-part simulation study aims to investigate the sampling distribution of various Likert scale distributions (including floor/ceiling effects) and analyze the effectiveness of using parametric versus nonparametric tests with varying sample sizes. Methods: We simulated populations from parametric distributions binned into Likert scales. In study 1, replicates were sampled from each distribution with sizes ranging from 5 to 150 observations, calculating means with simulated 95% CIs at each sample size. In study 2, floor/ceiling effects were introduced such that the proportion of patients responding with the lowest rating varied from approximately 40% to 90%. Two-sample tests were then conducted for the 90% floor effect distribution against all other floor distributions to determine effectiveness of parametric versus nonparametric methods via 2-sided pooled t tests and Wilcoxon rank-sum tests. Coverage of the difference in means, realized P values, relative efficiency, measures of agreement in direction, and conclusion of tests were plotted by sample size. Results: The sampling distributions of the 1-sample means and SDs for most distributions converged quickly to Gaussian, with 95% coverage. One- and 2-sample t tests of the mean demonstrated acceptable coverage, type I error, and agreement. Conclusions: Simulations confirm that the sampling distribution of the mean rapidly approaches normality and appropriate tests provide adequate coverage and type I error. Two-sample t tests demonstrate appropriateness and increased statistical power gained by using parametric over nonparametric approaches, suggesting t tests should be implemented with few restrictions.
KW - parametric versus nonparametric tests
KW - patient-reported outcomes
UR - http://www.scopus.com/inward/record.url?scp=85080993575&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85080993575&partnerID=8YFLogxK
U2 - 10.1016/j.jval.2020.01.007
DO - 10.1016/j.jval.2020.01.007
M3 - Article
C2 - 32389228
AN - SCOPUS:85080993575
SN - 1098-3015
VL - 23
SP - 625
EP - 631
JO - Value in Health
JF - Value in Health
IS - 5
ER -