< - - Back to     Measurement matrix 

Measurement Evaluation Grids (MEGs)

   

Construct

Nakazawa, et. al., 2009

Palliative care knowledge: 6 domains measure philosophy, pain, dyspnea, psychiatric problems, gastrointestinal problems and communication regarding palliative care were evaluated

Instrument/Scale

Palliative Care Knowledge Test (PCKT)

Development of a new measure

Methods of Translation

Developed in Japanese

Qualitative Methods

The design was a cross-sectional, anonymous, self-administered survey.

(See Expert Review below for additional information)

Data Collection and Sample

Nurses on inpatient palliative care units (PCU) constituted the sample

Sample Characteristics

The sample was drawn from two hospitals in Yokohama Japan. 940 Registered Nurses (97% women) were invited to respond to 40 items. The breakdown by clinical area is as follows: Palliative Care Unit 14%, Surgical 38%, Medicine 18%. 35% of respondents had less than 4 years clinical experience.

Response Rates

The response rate was 82% in the first round and 73% in the second round.

Among the 40 original items, the authors omitted items with:

(1) floor and ceiling effects ( >90% correct ≤10% correct)

(2) >1% missing data

(3) kappa coefficients for test-retest ≤0.3

(4) IRT discrimination parameter ≤ 0.5

There was repeated expert review of the remaining items.

 20 items were selected

Classical Test Theory

 

        Reliability

 

A pool of 148 items was developed

Two-week test- retest analysis was performed using the intraclass correlation coefficient (ICC). (n=204). KR-20 was used to test internal consistency.

For binary items, a two parameter logistic IRT model was applied. Item difficulties ranged from 2.9 - 3.09. Discrimination ranged from 0.33 to 2.25. Item level kappa statistics in test-retest analyses ranged from 0.26 – 0.74.

“Uncertain information should not be given to patient or family because it may cause additional anxiety” had the highest difficulty and the lowest discrimination.

The final measure contains 20 Items in 5 domains:

1)    philosophy 2) pain 3) dyspnea 4) psychiatric problems

5) gastrointestinal problems

The KR-20 internal consistency test = 0.81.

The ICC for the test-retest analyses =0.88 overall and 0.62 to 0.82 for each domain

Latent Variable Models

        Method

       Information Function

       Precision

       Differential Item Functioning

       Measure

       Sample

       Studied Variables

       DIF Variables

       Magnitude/ Impact of DIF

      Recommendations/Actions taken:  

 

 Item response theory two parameter logistic response model

 Not reported

 

   Not examined

Validity Content Construct Criterion (e.g., ROC curves, sens, spec, PPV, NPV) Predictive &

 

Face validity was evaluated by a panel of experts (3 palliative care physicians, 4 oncology nurse specialists, 2 home care nurses) rated appropriateness of items using a 4 grade scale. Items with < 2 (average scale) and those with 0 were omitted. The exercise was repeated- reducing the pool to 40.

Content validity- The final pool of items after analysis above (steps 1-4) was subjected to repeated expert review.

Known group validity- Means for PCU & other nurses were compared using unpaired t-tests. Linear regression was used to examine factors associated with the test. Significant differences in score for each domain and overall were found.

Sensitivity to change over time

 

Comments- Expert Review

This instrument was developed in Japan and one attitudinal item refers to Japanese. However, this item was not included in the final 20.

Although the use of IRT is admirable, no assumptions were tested and five domains were posited. There was no factor analysis to confirm the existence of these domains.

 Further, if they exist, this would invalidate the IRT essential unidimensionality assumption.  It is not known if the measure is unidimensional. More analysis would be required to address this issue. Thus, this test can only be viewed as experimental.

 No qualitative work in terms of translation was reported- nor was qualitative work performed to examine the conceptual meaning for nurses.  The authors also mention as a limitation that the measure was also intended for physicians who were not included in the sample. Finally, only two facilities were included.

  

Back to     TOP    Measurement matrix