# Cohen's Kappa for more than two categories

Cross Validated Asked by Asra Khalid on November 25, 2020

I have data set of teacher’s evaluation rated by 4 different raters. The teacher’s are evaluated for 13 different categories for example (Interaction with students, lesson delivery etc). All of the 4 raters are rating teachers in 13 categories from 1 to 5.

I want to find the agreement level between observers using cohen’s kappa. I know how to compute kappa for one category only but I am confused how can we do that for different categories? Do I have to compute it for each category separately? or is there any other method?

For computing kappa I am using STATA. Here’s an example of how my data looks like. Can’t share the original data.

From kappa - Stata "kap (second syntax) and kappa calculate the kappa-statistic measure when there are two or more (nonunique) raters and two outcomes, more than two outcomes when the number of raters is fixed, and more than two outcomes when the number of raters varies. kap (second syntax) and kappa produce the same results; they merely differ in how they expect the data to be organized."

Answered by Carl on November 25, 2020

## Related Questions

### Keeping baseline as predictor with change score as outcome in this peculiar scenario

0  Asked on November 21, 2021

### How to test if sum of two coefficients of ols model is greater than zero using R?

3  Asked on November 21, 2021

### maximum Corelation coefficient – how the numarator and denominator becomes equal?

0  Asked on November 21, 2021

### Which Object detection model will give the best result on images when the speed is not a problem for Text Images

1  Asked on November 21, 2021

### What is alpha in Vapnik’s statistical learning theory?

1  Asked on November 21, 2021

### Difference-in-Differences time-variant control variable

0  Asked on November 21, 2021 by cian

### Quantile Matching using the skewed t-distribution from Azzalini & Capitanio (2003)

0  Asked on November 21, 2021 by jj_okocha

### Non-independence of trial likelihoods in a staircase procedure?

0  Asked on November 21, 2021

### Should I reword the Factor if it shows all negative loadings?

1  Asked on November 21, 2021 by giulia-magnani

### How to correctly interpret rma.uni output?

1  Asked on November 21, 2021 by ena

### Backpropagation through time for stacked RNNs

0  Asked on November 21, 2021 by e-fresher

### averaging feature importance from different models

0  Asked on November 21, 2021 by henry50618

### Assumptions of OLS and linear mixed models

1  Asked on November 21, 2021 by molecularrunner

### requirements for simulating a covariance matrix

1  Asked on November 21, 2021 by apocalypsis

### Dirichlet distribution: Normalization of alpha values

2  Asked on November 20, 2021 by user60674

### Interpreting main effects in the presence of an interaction in logistic regression

1  Asked on November 20, 2021 by fcassidy

### How to interpret precision and recall for multiclass prediction?

1  Asked on November 20, 2021

### What standard deviation is used for calculating standard error?

3  Asked on November 20, 2021 by narayanpatra

### In a parametric model, if I do not have enough data, can I estimate the parameter, and simulate data from the estimated model and estimate again?

1  Asked on November 20, 2021