This is a write-up for the first inter-rater reliability analysis as part of the MHIN screening. The rationale for this analysis has been provided elsewhere - along with some ideas which mostly could not be implemented due to the data available from the first screening round. The screening process is described here - including a decision rule indicating in which case an intervention is screened “in” or “out”.
The calculations for this analysis are found in this document on the sheet “Analysis”.
In total, this analysis is based on 58 attempted screenings for 9 interventions (6,44 attempted screenings/intervention).
9 Raters have contributed with at least two attempted screenings.
B) Key findings and inferences
2. With regards to individual rating patterns:
C) Most important things to consider for raters to improve future inter-rater reliability.
D) Please note:
Two aspects seem to stand out with regard to this, though: