TransWikia.com

What are the best practices for giving online tests?

Mathematics Educators Asked on September 6, 2021

Many of us our coming off our first semester of required-online classes; and at some of our institutions we are preparing for what is most likely a required-online semester in the fall. (That is: The first time we know in advance of the all-online requirement, and with some experiential knowledge of how things progressed in the spring.)

Assume that we’re going to be giving math tests online in the future. In my department there are a wide variety of opinions about how best to deploy those tests, and we’ve passed around competing published opinion papers on the matter (mostly from a time when courses were not all-online by necessity; i.e., prior publications assume students registered for online courses as a special case). Most of the conversation revolves around how to tamp down on cheating during those tests. Granted that, what are the best parameters under which to give tests? Some points that would be addressed in the best answer:

  • Short vs. long availability period? E.g.: Some instructors want the testing to occur within the normal class hour, and require students to begin within a 15 minute window. Others look to give students maximum flexibility, say, a 6-hour or day-long window in which to take the one-hour test (so as to work around other courses, family/work obligations, etc.)
  • Short vs. long timer? A short timer on the test may reduce opportunities for cheating, but likewise make a student more desperate and likely to resort to cheating; perhaps a longer timer would reduce that stress.
  • Present questions all at once or one at a time?
  • Randomize question order?
  • Prohibit backtracking?
  • Require academic integrity statement?

Now, my own situation involves teaching math and computing in the context of a community college STEM degree program; e.g., at the level of college algebra through calculus III, linear algebra, and discrete mathematics. Traditionally these tests are largely computational — a small number of proof-based questions are asked at the uppermost levels, and it is a minority of our students who can answer any of them (e.g., making all questions proof-based is not an option at this level). Hopefully this question can cover an array of similar-level courses at other institutions.

The online system we use is Blackboard, but likewise I hope this question is appropriate for any online platform (e.g., my bulleted points above are basically the Blackboard test deployment options, excepting the last one).

The larger institution has made an executive decision to not license or employ any online proctoring system (e.g., video surveillance of students taking tests), so that is not an option in our case.

Note some related but distinct questions on ME.SE that have been recently asked:

So: What are the best-practice parameters with which to deploy an online test in the current environment (all classes by necessity being fully online)?

Note: Suggestions to refrain from giving tests are out of scope for the current question, and will not be selected as the answer. Please address any such possible practices in a separate question.

6 Answers

Here is my experience:

Throughout the term I gave several quizzes with

  • long timer
  • short availability
  • questions all at once
  • academic integrity statement
  • few modifications

The average score was around 80 %.

For the final exam, I gave a quiz with majority (around 2/3) of the questions very similar to those given throughout the term (or even the same questions, but reworded). However, the conditions were different:

  • short timer
  • questions one at a time
  • randomized question order
  • prohibit backtracking
  • random questions (a random question from each category with 1 to 4 similar questions)
  • no academic integrity statement

Besides this, I was tracking the progress of students to make sure that there are no students waiting for someone else to answer their question. This led to quizzes being mostly individual.

This time, the average score was 45 %. (And of course, there were many examples of students failing questions which they had correctly answered before.)

I should admit that cheating is a real plague here, starting from primary school. I am trying to fight it as much as I can. For instance, besides the quiz, there were problems to solve in the final exam, which I made as variable as I could (the exam was in Statistics, so I used random samples). The students do know how much I despise cheating and that they would get F if they are caught, so most of them will prefer to return a blank paper if there is a chance to get caught cheating. And still there were 4 F's in the final exam because of cheating (out of around 40 students).

So, speaking of quizzes, I am not sure that the second scheme is better, but I'd rather be using it. However, I understand that this may not be the best idea in other environments where cheating is not so ubiquitous as here.

Correct answer by zhoraster on September 6, 2021

The main issue with using proctoring software is that some students perceive this as being required to install spyware on their computer with all the features of tracking keystrokes etc., and when there is a recording of the exam session there are questions of where these recordings are stored and who has access to them.

At my institution we are moving toward live proctoring over Zoom without recording. However, proctoring through a front-facing camera is not very effective because students can easily communicate with others or access unauthorized materials on their other devices on their desktop if we cannot see their hands. For this reason, our preferred method for proctoring will be through the Zoom mobile app where the students sets up their cell phone as a side camera, clearly showing their workspace and with their computer screen tilted slightly toward the camera. Some schools are already doing this and have published detailed guidelines, including: Texas A&M University, Hong Kong University of Science & Technology, and KTH Royal Institute of Technology in Sweden. We expect that we may need to accommodate a small minority of students that cannot create a suitable environment for taking an exam at home in this way.

Answered by J. Gray on September 6, 2021

Randomize questions

Other answers are good, but I think a very important aspect on online quizzes has not been covered: questions should be different for every student - as different as resources and fairness allow.

Questions can be randomized in several ways:

  • Random parameters.
  • Slightly different versions of the same question (e.g. one version with permutations with repeat and another version without repeat).
  • Mathematically equivalent questions with apparently different settings (e.g. one question about husbands and wives and another about locks and keys).
  • Actually different questions. If possible, a large pool of independent questions taken at random.

In online tests (and even more in online homework) my ideal goal is to make the students need to know statistics (or to learn them, in homework) even to able to cheat. This way, I need to minimize the "cheatable" part, ideally making it indistinguishable from learning the subject. For example, I like questions about computing a probability in a normal distribution because when parameters are different the only thing that can be copied is the procedure, which is what I want them to learn.

However, I must admit that preparing random quizzes is a lot of work.

Edit in response to comment:

The comment asks how to implement random quizzes for problems with answers that can't be expressed as a number with decimals.

Some ways that worked for me (in Moodle):

  • Wiris quizzes, that allow for answers including formulas and can check fore mathematical equivalency. For example it allows the answer to be $frac{3 cdot pi}{2}$ or even a function. Since Wiris is a commercial extension to Moodle, it is not installed in all institutions, but there may also be equivalent extension from other vendors.
  • Multiple option questions: The answer can be as complex as you want, because in the end the student just needs to choose between a finite set of answers, and that can be easily graded by Moodle.
  • Transforming the answer to a decimal number. If, instead of asking the derivative of a function, we ask for the value of the derivative at a given point, the answer is a real number and we just need to evaluate the answer against a given tolerance. However, I know that pure mathematicians usually dislike this approach.

To generate those kinds of questions I use three different tools:

  • Moodle calculated questions. Quite easy but very little mathematical capabilities.
  • R package exams. You can do in the question anything you can do in R. Great for statistics and numerical analysis, useful but not so great for calculus or algebra.
  • Wiris, which has built-in random capabilities.

And of course, few automatic quizzes will outperform the teaching and assessing effect of a hand-graded essay or a long problem. However, automatic quizzes are great at easing logistics and introducing randomness.

We have also used random data in hand-graded exercises, but those are slow to grade and randomness needs to be limited. In the end all the students have the same problem with some different parameters and teachers can grade using a table of results.

Answered by Pere on September 6, 2021

Do standard time exams in the normal course period. Keep as much the same and stable vise loosely goosey.

Email it as a pdf. Let the kids send handwritten work back, scanned.

Avoid the grad student impulse to give project style questions. Do normal problems similar to the drill work. Take it as a victory if they learn that.

Ask them to box their answers. If you want to make it easy on yourself, do short frequents questions and just grade the final result. Students can see the key after the exam to see where they went wrong. If still confused they can request a session to go over it. Not grade arguing but to learn.

Do a nonverbise integrity statement. Doesn't hurt.

Answered by guest on September 6, 2021

On The Timer

I tried a long timer (about 3-5 days) for my exams and final exam last semester. The reason for this is that my favorite assignments from college -- the ones where I learned the most -- were challenging assignments that had a long availability period and allowed incremental work over the course of several days.

However, an unbelievably large number of my students used Chegg, Mathway, Symbolab, and Wolfram|Alpha to cheat. I gave F's for academic integrity violations to almost 25% of my students. That's just the ones where I had evidence that I felt was strong enough to win an appeal. Many more clever students may have used the resources without being so obvious about it.

Some students probably got a lot of value out of the long-timer assignment, the same way that I did when I was in school. If you go this route, be prepared to buy a subscription to Chegg at a minimum.

Answered by Chris Cunningham on September 6, 2021

Generally about cheating in tests
if you are trying to prevent cheating, trying to limit the student's ability to do it is very hard and mostly counterproductive, as it can introduce frustration , specificity in an online setting.
Instead, I recommend to minimize the student's motivation to cheat, after the first time it is just get's easier, if you design tests to not be frustrating you'll get a much better result instead of trying to fight the student.
Hard $not Rightarrow$ Frustrating, even the opposite, giving the student's three hard integrals is mostly better then 20 easy ones.
Short vs. long availability period
I’m on the Long availability period side, a student in the online setting might have difficulties to attend during specific times and thus it’d be better to let them have time.
Long timer vs. Short timer
I’m on the long timer side, from my experience, students cheat either out of habit or desperation.
in the long timer scenario, the student is less likely to be desperate and resort to cheating.
Backtracking
if you prohibit backtracking you are just annoying the student, there isn't something more frustrating then realizing you were wrong and can't fix it.
Randomize question order
It really doesn't matter from the student's perspective and they'l probably not even notice it.
Require academic integrity statement
from my experience, this won't deter a student from cheating.
General tips to minimize cheating
A majority of online resources cannot answer “why?” Questions. So, for example, having a question such as $int frac{dx}{sin(x)}$ Could make a student just put the integral into WolframAlpha and get the solution.
On the other hand, if we add an intermediary question such as: “Why does substituting $u=sin(x)$ not work?” WolframAlpha can’t help the student cheat anymore.
This question is especially good because most answers will use the $cot(x) + csc(x)$ trick, which would light a red flag as the answer expected will probably be the $cos(x)$ substitution and partial fractions.

Answered by razivo on September 6, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP