< - - Back to     Measurement matrix 

Measurement Evaluation Grids (MEGs)

 

Construct

Back Pain    

Instrument/Scale

Kopec et. al., 2008

5 Domain Computerized Adaptive Test (CAT) using 5 item banks: daily activities, walking, handling objects, pain or discomfort, and feelings.

Methods of Translation

 

Qualitative Methods

Not reported in this paper

Data Collection and Sample

Data were collected from a Canadian sample of 888 individuals with physician diagnosed or self-reported arthritis; 543 of these people reported back pain. Only 215 completed the questionnaires.

Subjects were treated between 1999 & 2004 at the Vancouver General Hospital Spine Program.

Data were collected online, in person or by telephone

Sample Characteristics

The sample was younger (21.9% were 65 and over), fairly well educated (only 19.5% reported a high school education or less and 36.7% reported trade or technical school); 44.7% were female.

Response rates

The response rate was 57.2% or 29.9%, depending on the denominator used.

Classical Test Theory

 

        Reliability

Test-retest reliability was measured with the intra-class correlation coefficient from a 2- way analysis of variance model. Test –retest reliability coefficients were: Daily (0.88), Walking (0.90), Pain (0.92), Feeling (0.83) and Handling objects (0.91).

Standard errors of measurement or SEM= σ(1-ICC)½  ; σ= SD of the score at baseline. The SEM’s ranged from 2.0 (pain) to 3.8 (feelings).

Latent Variable Models

Method

Factor Analyses and reliability estimates

Item Response Theory Results

Information Function

 

 

Precision

 

The generalized partial credit model in Parscale was used.

  Unspecified factor analyses (reported in another publication) were conducted  and used to test for unidimensionality.

31 items were deleted because of lack of fit or differential item functioning (DIF). The final banks each were comprised of 31-45 items,

 All subjects were required to answer 3 items per domain. The maximum was 7. Selection of items was based on the information function with administration of the most informative items. The minimum standard error of individual scores was set to 2.0 for daily activities, 1.5 for walking, handling objects & for pain, and 2.5 for feelings. The stopping rule was 7 items or achieving the desired standard error between 3 & 6 items.

Validity Content Construct Criterion (e.g., ROC curves, sens, spec, PPV, NPV) Predictive &  

Other construct validity

Predictive, Concurrent validity

 

 

 Also included in the scales administered were the SF -36, the Modified Oswestry Disability Index & the Roland- Morris Disability Questionnaire. 

Examined were correlations of CATs with the legacy measures and discriminations assessed using the F statistic from linear regression. The relative discrimination was calculated as the ratio of the F statistic for the instrument tested and of the instrument with the largest F.

Correlations were high between the physical function and pain CAT measures and the legacy measures (ranging from 0.65 to 0.83). As expected, the feeling CAT correlated lower with the physical function measures (0.33 to 0.48) and high with the mental component of the SF-36 (0.72).

Sensitivity to change over time

The authors reported the standard errors of measurement (SEM) results as preliminary estimates of the distribution-based minimal clinically important differences (MCID). They were: daily activities 3.5, walking 3.3, handling objects 2.6, pain 2.0 & feelings 3.8.

Comments- Expert Review

The major limitation was the less than desirable response rate and small sample size, possibly resulting in local generalization of the results only. The findings are supportive of the merits of the 5 CATs.

 

Back to     TOP    Measurement matrix

 

PACSLAC – Pain Assessment Checklist for Seniors with Limited Ability to Communicate.

Instrument/Scale

Van Nispen, et. al.,  2008

PACSLAC – Pain Assessment Checklist for Seniors with Limited Ability to Communicate

Methods of Translation

N/A

Qualitative Methods

N/A

Data Collection

Pain observed during rest (T1), influenza vaccination (T2) & specific moment of pain (e.g., injection, washing) (T3)

Sample Characteristics

128 nursing home residents from 3 nursing homes (12 psycho-geriatric wards) who met DSM VI criteria for dementia were sampled. The study was performed in the Netherlands.

 

Response rates

 

 

Classical Test Theory

 

The original item set contained 60 items in four domains – 1) facial expressions 2) activity/body movements 3) social/ personality/ mood indicators and 4) physiological indicators of change including eating/ sleeping.

Used Classical Test Theory (CTT) to reduce the number of items on the basis of nursing ratings for VAS scores >30. Items were eliminated if frequencies were <5% or >95% in T2 & T3 conditions. Items were deleted if the corrected item-total correlation was <0.2 or if α would increase with removal.

Reliability

 

In other studies, the CTT total scale alpha = 0.82  to 0.92 ; subscale alphas = 0.55- 0.73

Latent Variable Models

 

 

Latent Analyses, Parallel Analyses and Item Response Theory (IRT) were performed.

Factor analysis with Oblimin rotation was performed for the remaining items.

Measure of sampling adequacy (MSA) for individual items was used; those <0.5 were deleted.  

The 3 factor bifactor model fit best the general and 3 group factors.

Method

 

Information Function

 

Precision

      Differential Item Functioning

 

      Measure

 

 

 

       Sample

      Studied Variables

      DIF Variables   

Magnitude/ Impact of DIF

      Recommendations/Actions taken:

 

Based on comparing IRT models, the authors decided equal discrimination was best and applied the Rash Model so that sum scores could be used.  1 PL was used. For DIF analyses LR differences in χ2 p<0.01 indicated uniform DIF, as recommended by Zumbo.

A reduced measure with 20 items was evaluated. DIF analyses were performed with each of the 3 scales; 2 of the items had DIF.  “Frustrated” showed uniform DIF for dementia severity and non-uniform DIF for type of rater (RN versus lower level nursing staff). “Pain expression” showed DIF for dementia severity.

 

 

 

Magnitude and impact of DIF were not examined

The authors removed the 2 items with DIF.

Validity Content Construct Criterion (e.g., ROC curves, sens, spec, PPV, NPV) Predictive &

 

ROC analysis showed that the shorter version performed somewhat better in terms of “wrong judgements”.

Sensitivity to change over time

 

Comments- Expert Review

According to the author, the work has the following limitations:

A small sample (n= 128) was evaluated.

This analysis of the Dutch version may not generalize to English.

The conditions evaluated may not reflect more severe pain.

Expert review:

While the analytic strategy was sound, the restricted sample and small sample size may limit generalizability.

 

Back to     TOP    Measurement matrix