I just reviewed a paper submitted to a journal (one way to stay in touch with the latest developments), and all along they were doing research on the cognitive and motivational relationships in the game. They claimed it was a game, and proceeded on that assumption. And then the truth came out.
When designing and evaluating learning experiences, you really want to go beyond whether it’s effective or easy to use, and decide whether it’s engaging. Yes, you absolutely need to test usability first (if there’s a problem with the learning outcomes, is it the pedagogy or the interaction?), and then learning effectiveness. But ultimately, if you want it optimally tuned for success, pitched at the optimal learning level using meaningful activities, it should feel like a game. The business case is that the effectiveness will be optimized, and the tuning process to get there is less than you think (if you’re doing it right). And the only real way to test it is subjectively: do the players think it’s a game.
If you create a learning experience and call it game, but your learners don’t think it is, you undermine their motivation and your credibility. It can be relative (e.g. better than regular learning) as you might not have the resources to compete with commercial games, but it ought to be better than having to sit through a page turner, or you’ve failed.
There are systematic ways to design games that achieve both meaningful engagement and effective education practice. Heck, I wrote a whole book on the topic. It’s not magic, and while it requires tuning, it’s doable. And, as I’ve stated before: you can’t say it’s a game, only your players can tell you that.
So here were these folks doing research on a ‘game’. The punchline: “students, who started playing the game with high enthusiasm, started complaining after a short while, ‘this is not a game’, and stopped gameplay”. Fail.
Seriously, if you’re going to make a game, make it demonstrably fun. Or it’s not a game, whether you say so or not.