Calculating Significantly Above Chance for 3-AFC Test

I am trying to figure out which participants who took a 18-item multiple choice test scored significantly above chance. Each item has 3 choices, so the probability of getting each question correct is 0.33. To know how many items correct a person would have to get in order to be significantly above chance, does it make sense to use a binomial probability calculator like this: and try different numbers of successes until the probability is < p=0.05 (assuming this is my significance criteria)?

For 18 items where the probability correct is 0.33, the probability of getting 9 correct is 0.061 and the probability of getting 10 correct is p=0.027, so does it make sense to use “10 items correct” as my criteria for “performed significantly above chance?”

Psychology & Neuroscience Asked by bernice.anders on December 30, 2020

1 Answers

One Answer

The hypothesis would normally be expressed as a one-sided test as p(X≥10). The online calculator you have selected is fine and the significant result could be expressed in an article as follows: (p=0.33, q=0.66, K=10, n=18, p-value = 0.043).

Another way to think of this is that 9 or fewer correct answers is more likely to be random chance.

At the threshold of 10, it is unlikely that the result could be explained by random chance.

Answered by Tony Mobbs on December 30, 2020

Add your own answers!

Related Questions

Term for emotional distancing?

1  Asked on January 31, 2021 by randy-zeitman


Multiple “levels” of association in classical conditioning?

1  Asked on January 28, 2021 by prathamesh-kulkarni


How to teach yourself to keep secrets?

0  Asked on January 21, 2021 by user26710


Why is almost all therapy done as one-hour sessions weekly?

2  Asked on January 20, 2021 by jared-k


Ask a Question

Get help from others!

© 2022 All rights reserved.