ChatGPT and assessment

What is ChatGPT?

ChatGPT is of the first ‘Large Language Model’ (LLM) artificial intelligence tools that are both freely available and user-friendly. Although several other models and tools are available now, ChatGPT, because of its high usage and notoriety is the focus of this article. LLM AI tools scrape the internet for texts to learn from, and create a model made up of words and the relationships between them. As such, the model gains the ability to make a prediction of the next word within a given context, a little like the predictive texting model on your phone, but with the advantage of being applicable to very specific contexts.

Unlike other AI-driven text services, such as Microsoft Editor or Grammarly, which students (and educators!) may use to refine their own texts, ChatGPT can generate a text within the context given from a given prompt. The user can then accept the output or refine the prompt in order to obtain a better answer.

In common with all AI services, Large Language Models operate in a so-called black box, where we cannot easily see the relationship between the input and the output.

Whilst ChatGPT (and similarly other LLM services) can produce texts which seem very human, it is worth remembering that the algorithm has absolutely no understanding of the material ^(1,2), but instead, it is a statistically sound output based on its input parameters. The other aspect which is worth keeping in mind is that although the AI has clearly trained on masses of text, it does have gaps in its knowledge that it has no ability to recognize ^(2,3). That said, the shift from the GPT-3.5 to the GPT-4 model appears to have improved accuracy a lot, and this is likely to improve in the future.

Limitations of Generative LLM AI services

• The output is statistically viable in any given situation, but in practice, that means it is also normative of its input text which also mirrors the norms of the texts found on the internet ⁽¹⁾. Biases, both subtle and visible can easily be seen towards western attitudes and values, though the programmers of some systems such as ChatGPT have clearly implemented safeguards to limit explicit sexism, racism, and other biases that are societally unacceptable, both through the programming and through pre-screening the material the system learns from ⁽⁴⁾.
  Similarly, given certain prompts, the service will tend towards the extreme (for example when asked to describe a day in the life of a patient with a specified illness, the patient may include every conceivable symptom).
• Since the AI does not understand, it can also something confidently write text which is wrong ^{(2, 5)}. These may be small factual errors, or bigger things, for example making up references to books that don’t exist. Since the model is statistical, it does not state when it is unsure about its statements. This highlights the importance of questioning our own tendency to conflate fluency with accuracy as highlighted in Martindale and Carpuat’s (2018) study of users’ trust in machine-translated tests ⁽⁶⁾.
• The AI does not always have access to specific texts ⁽³⁾. This can sometimes be shown in the answer.

Whilst there are a number of services that are now available that claim to be able to detect AI-generated output, these are often easy to avoid by making small changes to texts ⁽⁷⁾, and it is unclear how well they will hold up in the future as new LLM AI models emerge, and strategies to avoid detection are developed.

The current state of the assessment

As you may know, we have different assessment formats. In an essay, students write a response to the question without any clue from the teacher. This form of assessment is one of the most effective ways to check if the student can make a complex response to a challenging question ⁽⁸⁾. It is suitable for home exams and during the Covid pandemic, it was promoted by most of the universities as an alternative to in-site exams.

Recently, conversation AI developed to help us generate some text from existing literature on the web. It seems that this technology will help us to perform our jobs easier and better. So we need to include using it as a skill in higher education. Students, like other people, can use this tool to answer their exam questions.

There is a hot discussion about the impact of this tool on academic exams. As for now, only four types of them from our assessment portfolio could be affected by conversational AI (like ChatGPT). A simple solution is to use other alternatives (besides essays) to increase the validity and reliability of the assessment.