The measurement of observer agreement for categorical data.
Explore this paper's citation graph
Summary
A general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies is presented and tests for interobserver bias are presented in terms of first-order marginal homogeneity and measures of interob server agreement are developed as generalized kappa-type statistics.
- Type
- article
- Published
- 1977-03-01
- Cited by
- 80,928
- References
- 43
- Access
- Open access
- OpenAlex
- https://openalex.org/W2164777277
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:11077516
Keywords
Categorical variable, Kappa, Statistics, Homogeneity (statistics), Multivariate statistics
References
- A Note on the Equivalence of Two Test Criteria for Hypotheses in Categorical Data
- A review of statistical methods in the analysis of data arising from observer reliability studies (Part II)
- Hypotheses Of ‘No Interaction’ In Multi-dimensional Contingency Tables
- Measuring nominal scale agreement among many raters.
- Large sample standard errors of kappa and weighted kappa.
- Measures of response agreement for qualitative data: Some generalizations and alternatives.
- A general approach to estimation of variance components
- Errors of Measurement, Precision, Accuracy and the Statistical Comparison of Measuring Instruments
- Estimating Individual Rater Reliabilities from Analysis of Treatment Effects
- Tests of statistical hypotheses concerning several parameters when the number of observations is large
- An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers.
- The Analysis of Variance
- Measuring agreement between two judges on the presence or absence of a trait.
- Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit.
- Some Further Remarks Concerning "A General Approach to the Estimation of Variance Components"
- A Coefficient of Agreement for Nominal Scales
- Analysis of categorical data by linear models.
- The Measuring Process
- A general methodology for the analysis of experiments with repeated measurement of categorical data.
- An analysis for compounded functions of categorical data.
Cited by
- Communication failures in the speech-based control of smart home systems
- Biomarkers for inflammation and surveillance strategies in inflammatory bowel disease
- Socioeconomic position and mass media campaigns to prevent chronic disease
- What kind of goals do children and young people set for themselves in therapy? Developing a goals framework using CORC data
- Interobserver agreement in the examination of acute ankle injury patients.
- Caries incidence after topical application of varnishes containing different concentrations of sodium fluoride: 3-year results.
- Nonresponse in Business Tendency Surveys: Theoretical Discourse and Empirical Evidence
- Serum tumor markers may precede instrumental response to chemotherapy in patients with metastatic cancer.
- Evaporative tear film and contact lens factors associated with dry eye symptoms in contact lens wearers
- Degree of differentiation in colorectal adenocarcinomas: a multivariate analysis of the influence on survival.
- A Study of Adverse Occurrences and Major Functional Impairment Following Surgery
- Recognizing delirium superimposed on dementia: assessing nurses' knowledge using case vignettes.
- Systematic bias in recording the history in patients with chest pain.
- Development and validation of the French ASQoL questionnaire.
- Television news coverage of obesity in China, 1982-2009.
- Grounding affect recognition on a low-level description of body posture
- The use of portable 2D echocardiography and 'frame-based' bubble counting as a tool to evaluate diving decompression stress.
- "who is Key...?" - Characterizing Value Adding Users in Enterprise Social Networks
- Exploring Characteristics of Code Churn
- Interrater Reliability in Content Analysis of Healthcare Service Quality Using Montreal’s Conceptual Framework
Related papers
- Count on kappa
- Understanding interobserver agreement: the kappa statistic.
- The Kappa Coefficient and the Prevalence of a Diagnosis
- Statistical methods in epidemiology. v. Towards an understanding of the kappa coefficient
- Reproducibility studies in Manual/Musculoskeletal Medicine: a new method for kappa independence from prevalence
- More than Just the Kappa Coefficient: A Program to Fully Characterize Inter-Rater Reliability between Two Raters
- The Problems with the Kappa Statistic as a Metric of Interobserver Agreement on Lesion Detection Using a Third-reader Approach When Locations Are Not Prespecified.
- Interrater reliability: the kappa statistic
- Reliability of ophthalmic diagnoses in an epidemiologic survey.