Possibly one of the trickiest design decisions (and one for which I don’t think I have yet found the best option) is about what wording to use around some of the unfamiliar features of this assessment type.
The certainty scale
One of the key features is the certainty scale. At the moment I’m using guess-think-know which I like because I think there is a good semantic distinction between them and they hopefully correspond to people’s natural ways of describing levels of certainty. The drawbacks are, potential differences in interpretations and clumsy phraseology needed to explain ‘think’ to users.
This is not the approach that previous implementations have used. I can see good arguments for simply calling the buttons 1,2,3. This is a much more objective description of what they deliver and might avoid unfair penalisation of those who perceive their knowledge using more self-deprecating language. However, I fear that it is a bit abstract for younger pupils and might be off-putting simply because of the technical appearance.
I think the titles for the categories are due for a change before I start the pilot. I wrote them in pretty quickly and a don’t think the language is really clear or motivational enough (given that the message is quite tough).
I happy with ‘Secure knowledge‘ for my top scores (3,2).
I think ‘Unsecured knowledge‘ might work better than ‘Guessing‘ as it implies that this knowledge will be secured in the future. Although scored differently, I don’t believe it is that useful to differentiate between correct and incorrect at this level of certainty. Effectively, I think we should communicate, ‘If you don’t know it with any certainty, you have work to do’.
Finally, I think ‘Misunderstanding‘ seems gentler than ‘Misconception‘. It allows ‘misunderstanding the question’ as an explanation so avoids criticising in a way that may seem unjust. I’m going to try sub-categories of ‘Misunderstanding‘ and ‘Serious misunderstanding‘ in the next iteration.
Adapting to audience
Ultimately, there will be no perfect solution that I can author. It seems fairly obvious to me that people will want to adapt the language to fit the context in which they are using these assessments. So a task, definitely on my to-do list, is to provide the feedback panel elements as an editable form so that users can adapt the language for their audience.