Generative AI and teaching - Advice for educators

Is it possible to reliably detect AI-written work?

There are no tools available on the market which can reliably detect AI-written texts (Webb, 2023; Weber-Wulff et al, 2023). Such tools may have a tendency to incorrectly flag students who are writing in a second language as having cheated (Liang et al, 2023). ). If students' work is submitted to AI services (with which KI has no agreement) to check for cheating, the students' work is handed over to a third party who can then arbitrarily use the material. As it stands, KI uses Ouriginal (Urkund) and iThenticate  to detect possible plagiarism, neither of which offers AI detection at the moment.

It should also be noted that generative AI services are not designed to answer questions such as "Did you write this text?".

How can I adapt my examination practice?

As there are no current technologies available to reliably detect AI-generated work, and considering that students will need to use these technologies in their future jobs, we recommend supporting students in the use of AI in their studies, and in developing their AI literacy as a whole.

In the written assessment we often use the SOLO taxonomy (Biggs, 2011). In some courses, we need to ensure that students know the basics before starting the next level. One proposal could be to ask students not to use external resources (books, and conversation AI) during the assessment.

However, to reach the “relational” and “extended abstract” levels, it is good to use some kind of AI to help students organize their declarative knowledge first and then try to make it more functional. Since the validity of the information produced by AI is not always correct, students need to be able to analyze and evaluate the results, and are encouraged to use AI with caution.

You can use the diagram (below) to help you decide how to design assessments. If the purpose is to assess basic concepts that students should remember and understand, it is more reliable to carry out the assessment in an examination hall without access to the internet. Alternatively, we need to adapt and change how the assessment is carried out so that it can work in a situation where generative AI could otherwise affect the outcome.

Consider the purpose of the examination. What knowledge is it that you want to have evidence for? Then you need to reflect on the current shape of the examination and whether the use of AI would skew the results. If you find that the examination needs to be redesigned, keep in mind the intended learning outcomes (what you need evidence for) and think about alternative ways of finding that information. For essays and short answer questions in take-home exams you can consider adding the use of AI as part of the assessed task. 

How does generative AI affect students' learning?

How generative AI affects students’ learning when they write depends entirely on how students use AI tools. If students use AI tools to create long passages or even entire texts without further reflection, this will likely affect learning negatively. So, not even if the AI output is correct would copying and pasting be an effective way of learning, and AI output is often incorrect or heavily biased (Ho, 2023).  

On the other hand, students may use AI tools, for example, to brainstorm ideas or as a sounding board to polish their argument, while carefully considering each suggested change before implementing or rejecting it. In this case they may learn as much as they did when they wrote in more traditional ways – if not more.

In addition, if students use AI tools wisely, assessment may become fairer and students may be able to use their time more wisely. Menial tasks such as spelling and other basic grammatical errors can easily be fixed with the help of AI tools (Huang & Tan, 2023) - even though students must, of course, still check any AI suggestion. Using AI tools to make the more mechanical aspects of writing more efficient may level the field for students writing in other languages than their mother tongue (Nordling, 2023), potentially making assessment fairer. Doing so may also free up time to spend on more advanced aspects of writing, for example, analysis.

Students still need to know what good writing is

Using AI tools may also help students deal with writer’s block (Nordling, 2023). A student stressing over not being able to start writing may ask an AI tool to produce a few words on the topic, and in that way avoid having to start writing on a blank page.

For students to use AI tools well, they still need to know what good writing is. Students need to be aware of several aspects of writing, for example, how an academic text should be structured, what it means to adjust one’s text to an intended audience, how to use field-specific terminology, and that vocabulary is always contextual; that the meaning of words may change depending on the context.

To ensure that students use AI tools wisely - in a manner which enables them to retain critical skills and learn well, we need to help students make good choices (van Dis et al, 2023). We need to nuance the discussion about AI tools and try to keep from categorical statements such as “using AI tools is forbidden”. Instead, we can give clear examples of potentially useful ways of using AI, and potentially unsuitable ways. We also need to be clear about if, and in what ways, students may use AI tools in examinations. 

We also need to strive to encourage open communication about AI, so that students feel comfortable asking questions and having a dialogue with their teachers. If the message is that all use of AI is disallowed and that all use by default equals cheating, you will lose the opportunity to be invited into discussions with the students, and you will not be able to affect your students’ use of AI. 

Part of teaching students to use AI tools wisely is teaching them to be as transparent as they can regarding their use of generative AI tools in the writing. You will know how students used AI tools in each text, and will be able to fairly assess the work at hand. However, you may still of course tell the student that they have not fulfilled the course criteria and therefore need to repeat the examination. And, you will have the chance to have yet another discussion about effective AI use.