Investigating various thresholds as immunohistochemistry cut-offs for observer agreement

Aim/Background

Clinical translation of immunohistochemistry (IHC) biomarkers requires a reliable and reproducible cut-off for interpretation of immunostaining. Most of the IHC biomarker research focuses on the clinical relevance (diagnostic or prognostic utility) of cut-offs with less emphasis on observer agreement using these cut-offs. We identified three cut-offs from our diagnostic IHC work and literature namely, 10% positive epithelial cells (10% hereafter), 20% positive epithelial cells (20% hereafter) and moderate to strong staining intensity (+2/ + 3 hereafter) for investigating observer agreement. The aim was to establish consensus based cut-off(s) that could potentially be used by pathologists.

Methods

A series of 36 IHC images of microarray cores for four IHC biomarkers with variable staining intensity and percentage of positive cells was used for investigating inter- and intra-observer agreement. Seven pathologists participated in the study and they scored the immunostaining of each image for the three cut-offs. Kappa statistic was used to assess the strength of agreement for each cut-off.

Results

The inter-observer agreement between all seven pathologists using the three cut-offs was reasonably good. A good agreement was observed for experienced pathologists using 10% cut-offs and the agreement was statistically higher than junior pathologists (p = 0.02). In addition, the mean intra-observer agreement for all seven pathologists using the three cut-offs was reasonably good. For all three cut-offs a positive correlation was observed with perceived ease of interpretations (p < 0.0001 for 10% cut-off, p = 0.001 for +2/ + 3 cut-off and p = 0.004 for 20% cut-off). Finally, cytoplasmic only staining achieved higher agreement using all three cut-offs than cytoplasmic/nuclear staining and cytoplasmic/membranous staining.

Conclusions

All three cut-offs investigated achieve reasonable strength of agreement modestly decreasing inter and intra-observer variability in IHC interpretation but 10% is slightly better than 20% and +2/ + 3 cut-offs. These cut-offs have previously been used in cancer pathology and we have provided evidence that they are reproducible between practising pathologists.

Disclosure

All authors have declared no conflicts of interest.