Open Access
Issue
Nat. Sci. Soc.
Volume 32, Number 3, Juillet/Septembre 2024
Dossier « L’évaluation des jeux sérieux sur les thématiques agro-environnementales, territoriales et alimentaires »
Page(s) 357 - 365
Section Regards – Focus
DOI https://doi.org/10.1051/nss/2024050
Published online 10 February 2025

Willy Christian Kriz, psychologist, professor of leadership and organizational development at Vorarlberg University of Applied Sciences, was born in 1968 in Vienna (Austria). He started as a research assistant at the Vienna International Future Conference where he worked with Dennis Meadows. After an awarded Ph.D. in 1999 on ‘Training of system competence with simulation games’, he worked at the Ludwig Maximilian University of Munich as assistant professor. He was an executive board member of the International Simulation and Gaming Association (ISAGA) since 1999-2016 (but has already attended annual conferences of ISAGA since 1996) and was also one of the 3 editors in chief of the journal Simulation & Gamingem (2016-2021) (currently he is associate editor). He developed numerous games to teach decision making, entrepreneurship and business and evaluated their effects. He wrote 15 books including The shift from teaching to learning: individual, collective and organizational learning through gaming simulation1, Back to the future of gaming2 with Richard Duke in 2014, or Gaming as a cultural commons. Risks, challenges, and opportunities3 in 2022.

Nicolas Salliou and Gilles Martel: Could you tell us what led you to use games and simulations in your research?

Willy Kriz: I first became involved into gaming when I was a student assistant on a project with Dennis Meadows, who is well-known for being a member of the Club of Rome and for authoring several books on the limits to growth. At that time, he was president of the Systems Dynamics Association and the International Simulation and Gaming Association (ISAGA4). He was interested in modelling the future, particularly concerning the depletion of resources. Additionally, he was keen on using simulation games as educational tools to raise awareness about environmental issues. I worked with him as a master student on a project for two years and he encouraged me to go to ISAGA. I became a member while I was still a student, and, as you can tell by my grey hair that was quite some time ago. So, I have been a member for over 30 years now.

N.S. & G.M.: Could you explain how researchers came to use games in their research?

Willy Kriz: According to one of my mentors, Jan Klabbers, gaming simulation can be used for a variety of reasons. The first reason comes from traditional science, which he refers to as analytical science. In this context, simulation games can serve as test scenario or quasi-experiment, similar to an experimental method. In that case, games function like a behaviour lab, where researchers can test and develop theories. The second reason comes from design science, where you have a specific idea on how to use games in order to change some situation in the real world. Both approaches — analytical and design science — are fields where games and simulations are utilized, and they are evolving together. The ISAGA conference hosts both communities but it primarily focuses on how to use games for transfer and real-world impact. First, as an intervention tool to raise awareness of some ecological or sustainable problems. In that case, researcher can still act as a scientist by evaluating whether games have an impact on consciousness, awareness or behaviour, or at least whether players learned something by playing. Second, games can be used as an experiment, like a game theory, where a researcher tests a theory within simplified game environment in which people communicate. Finally, games can be used as policy exercises to support decision-making among stakeholders, not just for learning purposes. Stakeholders can play through various scenarios and make real decisions, for example, about infrastructures. This process can also be scientifically evaluated.

N.S. & G.M.: Do you see any evolution in the way games and simulation are accepted as a research method?

Willy Kriz: In most of the European cultures, there is a growing acceptance of games and simulations, both as analytical research tools and as intervention tools for change. I come from the field of gaming simulation, but today, gamification is trending, which, although it shares some ideas with gaming simulation, is a different concept. There is some acceptance to gamify workplaces or to apply game elements, game principles and game techniques for various purposes, and this trend boosts the acceptance of simulation games as well. Additionally, there are serious games rooted partly in the entertainment video game industry. On the one hand, I think they complicate the understanding of what gaming simulation is. On the other hand, they help increase the acceptance of gaming simulation to some extent. I see an ongoing trend towards greater familiarity and acceptance of these ideas. The traditional approach to simulation games benefits from this trend, as long as we understand that there are significant differences between these methodologies.

N.S. & G.M.: How would you differentiate between serious games and simulation games?

Willy Kriz: They have fundamental differences and some in common. All serious games aim to achieve something beyond entertainment. In a serious game, it is more than just a good time, you want an effect in the real world outside after the gameplay ends. For example, if a game is used for educational purposes, it is a serious game. If a game is used for planning a highway, where stakeholders play through various scenarios that influence real-world decisions, it is a serious game. In that sense, even a simple TV quiz game like ‘who wants to be a millionaire’ could be used as a serious game because it tests participants’ knowledge, and viewers can learn something. Somehow, it can be more than just entertainment, or at least you can use those kinds of quiz games with students to test or build knowledge.

In that sense, all simulation games are serious games, but not all serious games are simulation games. The key difference is that simulation games go further by simulating real-world processes and structures to some degree. The simulation can be about the decision-making in a network of actors that is making decisions and is behaving in a simulated system. It can simulate resource dynamics, and the interconnection between rules, resources, actors and roles through a system of rules. So, if you simulate something with the game, while it is obviously a serious game, it is more than a quiz game. The quiz just tests knowledge and builds up knowledge, but does not simulate reality. That is the biggest difference.

N.S. & G.M.: Are there certain research communities that tend to mobilise more an inter- or transdisciplinary approach?

Willy Kriz: A major tradition is business games. In that field, considering stakeholders and multidisciplinarity has been important, although it has not always been fully implemented in games themselves. Another tradition is urban planning and infrastructure simulation, where there is a higher awareness of the need for multidisciplinarity. A third tradition comes from pedagogy and education where multidisciplinarity is particularly common in northern European countries, including the Netherlands. Unfortunately, I do not speak French but I saw an openness for multidisciplinarity in France in the last ISAGA conference organised by Nicolas Becu in La Rochelle5. It seems like he took on board a good mix of multidisciplinarity. In German speaking countries, multidisciplinarity was originally a little bit less prevalent, but it has been growing over time. Lastly, the military is another major origin of simulation gaming. While it was not traditionally multidisciplinary that is starting to change as well.

N.S. & G.M.: Do you think that recent state requirements for participation and transdisciplinarity, for example in urban planning, increase the legitimacy of the using games and simulation?

Willy Kriz: I agree and it is increasingly moving towards transdisciplinarity because of the challenging problems in this world. They seem more and more complex, changing rapidly, with all the ambiguity and uncertainty it generates. The idea of using simulation gaming as a language to model complex systems and involve various stakeholders to create pathways toward more sustainable solutions has been around since the 1960s and 1970s. Today, we are facing even more complex and wicked problems. Therefore, there is growing openness, not only to adopt a more participatory and multidisciplinary approach, but also to use gaming as a participative, multidisciplinary way to tackle these issues.

N.S. & G.M.: Are games so good at tackling complexity?

Willy Kriz: Yes, having real people take on roles in a game is a significant advantage. In this way, the human element is not only simulated but also has real emotions into the game. Even if artificial intelligence could simulate human behaviour, like how ChatGPT simulates dialogue, it is still different when you have real human beings being involved. A key benefit of incorporating the human factor in games is the ability to reflect on the experience during a debriefing. Players can discuss what triggered within themselves and within the group: certain emotions, the reasons behind certain decisions, discussing underlying assumptions and cultural factors. This is an advantage above other methods. However, as with any method, it is important to use a mix of methodology to cover the variety of objectives. Reading a book, attending a lecture or conducting experiments can be useful additions to a game to produce effect.

N.S. & G.M.: How do you ensure this transfer, or the ‘real-world effect’ you mentioned earlier?

Willy Kriz: In the short time, you can ask people after the game, and hopefully, they will all say they liked the game. But how do you create a substantial effect? How do you change mental models and behaviour of people, or even the structures and processes within a social system? The game is just the first step; serious game should also have a serious transfer. There is increasing recognition that simply using a game in order to create the transfer is not enough6. Unfortunately, while this awareness is growing, I am not very optimistic that it is really done in practice. Most of the time, games are played for a period of time and, if there is time left, a debriefing is held with the players. This is helpful, but it is not enough, especially since it is rarely followed by any formal evaluation. Consequently, we do not really know the long-term effect of games, and we are left hoping impact is there. I am a psychologist from pedagogic and educational psychology, I see many theoretical arguments supporting why games can be effective learning environment. However, it is rarely proven because evaluation is difficult.

N.S. & G.M.: We can sense the importance of transfer here, but before we go further into that, could you elaborate on the game debriefing, which is often considered as the most important step?

Willy Kriz: I am not sure if it is the most important, but it is certainly crucial. Mainly, it starts with co-learning, the experiential learning cycle. Games are great for providing experiences, and they are also great for applying decisions, insights that you gained in that safe environment. In games, you can immediately try things out, test new skills developed through the game, and learn from mistakes, learning by doing. However, research shows that games are time consuming, and much of the additional learning effect is lost if you play the game without debriefing7.

There is an unproven rule that the debriefing should take as much time as the gameplay. Debriefing session helps the participants to better understand what has happened and to draw some conclusions. Usually, the session starts by addressing emotions because some games may have a high degree of emotional involvement, both positive and negative. Addressing these emotions sets a good foundation for deeper learning on a cognitive level.

Then, it is important to explore the narrative of what happened during the game. A serious game is not completely the same as reality; after all, reality itself is a social construction. Therefore, we have to understand what is common between games and reality. In that sense, it is not bad if there are some differences between games and reality. Finding such differences is actually beneficial, as they create cognitive conflict, challenging what we thought we knew. This inner or even creative conflict, both within oneself and among players with differing perspectives, encourages deeper learning. When participants disagree, they learn from each other by articulating their mental models, assumptions and subjective theories about the world. Through dialogue, we can better elaborate and understand ourselves better, as well as understanding other perspectives and actually learning something new. From my point of view, it is good if the game is not seen as absolutely true or realistic. A group supported in that kind of dialogue about differences between game and reality can start a good learning effect. For me, it is the heart of the evaluation of the game in the sense of the players. They reflect on what the game means for their own reality. This form of evaluation is also happening in a good debriefing.

In the end, participants need to articulate: what did I really learn? The next step is to formulate: what will I do differently? There is a famous quote from Ivo Wenzler, ‘If you play a game on Friday and nothing has changed on Monday, the people still think and behave the same way like before’. Even if people enjoyed the game, if it does not lead to any change in behaviour or mental models, the serious game has not achieved its purpose. In a good debriefing, people make plans for the future: how will they act differently from now on considering that game experience and this debriefing dialogue. These new perspectives should lead to a higher probability of change.

N.S. & G.M.: Why do you think that the use of games can be particularly helpful to trigger this change of mental models?

Willy Kriz: A simulation game is a simulated reality where each player assumes a certain role, makes decisions, tackles challenges or solves problems, and acquires knowledge to apply in the game. Doing so, players will learn from their failures because it is a safer environment. Players will not destroy the whole earth if they try out a specific scenario, strategy or behaviour. Within this context, players can explore different hypothetical futures alongside others with other interests and perspectives, and eventually conflicts might arise. For me, conflicts are actually positive if managed well. Through debriefing, players better understand their own behaviour, mindsets and values. Having a series of games is very helpful because a player can try it out again with this better understanding. In the case of longer and more complex game, a debriefing may occur just after the first round. Often, something will happen in the game that players might not fully understand, leading to disagreements or even complaints like: ‘This is a bad simulation — reality is not like this’. But this curiosity and conflict can be used productively during the debriefing. By asking: ‘how is it for you?’, ‘how is your reality?’, ‘why is this game so bad?’. So, even if players do not accept the game as a good model of reality, it is not negative if you can facilitate dialogue around it in the debriefing. It is beneficial because the reason of the game is to start thinking about your perception of reality, how you understand the reality, its elements, interactions and behaviours of the system in reality. When differences arise, players can argue their points, defend their views, and listen to others with differing opinions. This is the start of a learning process: to think and expand your own knowledge, integrate different perspectives. Players may even adopt a meta-perspective, understanding that under some conditions this might be true, under other conditions the other one also has some good ideas. Good simulation games try to capture and to simulate elements of the real world but, ultimately, it is a construction. From an epistemological standpoint, knowledge is socially constructed and games explore how we construct reality. Through this process of reconstructing reality within the game, we may even change our perception of reality outside the game.

N.S. & G.M.: Is social constructivism shared among simulation and gaming communities? How is it applied in practice?

Willy Kriz: From the traditional perspective of simulation and gaming, I would say that early research and articles from the founding fathers of ISAGA, which has existed for 55 years, always had a strong connection between gaming simulation and system science8. Many shared the idea of a social construction of reality. Therefore, it is crucial in game design to include all the different stakeholders and perspectives on the particular system or problem being simulated. Additionally, testing the game with different key stakeholders for the simulated system is essential. The design phase should integrate as many meaningful and diverse perspectives as possible. When the simulation model is co-designed, many stakeholders often express that they finally understand their system for the first time or have a shared and common language to see their own reality. This model-building process — essentially reconstructing the reality of a system — serves as a highly valuable first step in understanding the constructed reality. After that, course, gaming elements are added to the model. The end of the game building process is a game artefact usable by participants who did not take part in the design of that game. That is why the debriefing is so important, because in the game design process, designers have reconstructed reality by connecting elements of the system. Then, this group defined system elements (modelling) and translated them into game elements (game design). For example, if you are creating a game to simulate an economic system, finances and money are an important element in reality and money must be part of the game. In a game like monopoly, little coloured papers with numbers on it symbolize real money. In this way, system elements are translated into game elements. Other game elements could include actors playing some roles, or game events that create unforeseen challenges that players must deal with. Building a model is a translation, and the model a reconstructed social reality, albeit more abstract than the reality. Once the translation of system elements into game elements is complete, the game can be played. The debriefing reverses this process, aiming to link the game experience back to the original system elements. The game is a reconstructed reality that players can experience, and it becomes real if there is a transfer back to the real system that was simulated.

N.S. & G.M.: Do you know if other scientific communities share a different epistemological perspective than social constructivism when they use games?

Willy Kriz: Yes, some communities, especially from the business gaming side, have lost the understanding that a perfect game is not necessarily the most realistic game. They think that a better game would be more realistic. They make games more and more realistic with more factors, elements and interconnections of reality in the game. It might be more realistic but from a didactic standpoint, it is counterproductive. Players are just overburdened, as they have not enough pre-experience or pre-knowledge to cope with that much complexity. Nowadays, American business schools use games with no debriefing at all, arguing that students can play without debriefing because there is no difference between game and reality and they have to learn by themselves. If they succeed in the game, it shows their competence. With no debriefing, students do not question the model behind the game. The model is just like a black box. They even make the grading based on the performance in the game. I think it is highly unethical because a team or a person often learns more from failure than success in games. For me, this mixing up between game and reality shows a more positivistic point of view about using games, which is completely unethical and wrong.

N.S. & G.M.: What are the risks associated with this mix up between game and reality? Do you think new technology can increase this risk?

Willy Kriz: For me, it is important that players can draw their assumptions and learning from debriefing and questioning the model behind the game. Assuming that the game is the reality and give to students a good or bad grade based on their game performance is a wrong way to use a game because they miss the opportunity to reflect on the model behind the reality or discuss their perspectives. Without this opportunity, students are not thinking about the model or the boundaries within which this model makes sense. 3D video games with very realistic technology, artificial intelligence and other devices like the 3D headsets can exacerbate this risk by making games highly realistic. It is fruitful and wonderful to make it more real and immersive as long as you do not skip the debriefing when players gather, learn from each other, and contemplate the differences between experience and reality. They should think and reflect on how this will change their behaviour from now on. It increases the risk that debriefing is no longer considered necessary because the game was feeling so real.

N.S. & G.M.: In that sense, by skipping the debriefing part, they seem to care even less about the transfer?

Willy Kriz: Yes, I think the learning comes not only from the debriefing but also from the transfer. The transfer is very tricky because the debriefing alone will not ensure a transfer. Debriefing makes the probability of a transfer more likely but you must allow players to behave in a different way in reality. For example, in the business world, people participate in a training game for one or two days to build knowledge or acquire skills. Let us assume that the game and the debriefing were good and participants acquired new ways of thinking and behaving. But if the company’s decision makers did not take part in the training, they will stick to the old processes, culture or mind-set, despite players’ acquired knowledge that work processes could work better. So, the question is: can you really change something? Very often, the game and its debriefing have no impact because they are not sufficiently interconnected with the other systems in reality. Participants probably had a good time for two days, like going to the theatre or having a nice trip in a different country and they are happy because they learned something new. But if you come back home, and everything is the same, with no idea or power to change something in your daily routine, I would say it is useless. That is why decision-makers in the reality of the system that you are simulating must be present in the design phase of the game. Stakeholders, who are the decision makers afterwards, must be integrated and further steps defined with them on how the game could be relevant to change something in the real world. It is very important to think about a good debriefing but there has to be a strategy to muster the motivation created in the game and to enable new skills to be applied in the real world.

N.S. & G.M.: Does it mean that the process from game design, to debriefing, to transfer, must be conducted in a transdisciplinary process?

Willy Kriz: Yes, I would say that is absolutely necessary. And that is also what the founding fathers of the International Simulation Gaming Association, like Dick Duke, emphasized from the beginning. They had this idea of multilogue, not just dialogue, but multilogue9. A multilogue is similar to transdisciplinarity in the sense that it involves different stakeholders and experts from reality. I understand expert in a broad sense; a child could be an expert for creating ideas on how to improve his or her kindergarten. The idea is not only to have diverse disciplines, but include different types and dimensions of diversity like different genders, levels of experience, religions, etc. Of course, it depends somewhat on the problem you are facing, what you are simulating, and the objectives of the simulation game you are using. Just as there is diversity of perspective in the real world, the game design phase should have the same diversity10.

N.S. & G.M.: Beyond the challenges of transdisciplinarity, how do researchers cope with analysing what happens during these games? How do they evaluate it?

Willy Kriz: Well, I would say that unfortunately, there is not much evaluation done, or at least not deeper evaluation with games. First of all, it is quite challenging, and second, who pays for that? Normally, a company paying for a training will not pay for a scientific evaluation. Evaluation is very complex: how can you prove from a scientific point of view what would have been different if you had not use a game? It is very tricky to gather enough data, especially for control groups. Still, I think you have to use all the tools available from traditional science.

Evaluation requires an evaluation model of the game and certain measurements over time11. These measurements can be done before the game, during the gaming and debriefing, and after the game. Post-game measurements can include at least one or two measurements some time after the game, maybe weeks, months or even years later. You have to think about input and output variables, outcome variables and process variables that you can measure during gameplay. A variety of methodologies can be applied. A qualitative approach might include interviews, observations with trained observers, or audio/video recordings. Questionnaires and more quantitative approaches are also possible. It is possible to test behaviours or knowledge, using pre- and post- game assessments.

There are many ways to evaluate, but it is tricky and not done very often because it is also expensive. I think the lack of funding is a big problem for conducting deep evaluation. Surface-level evaluations are fine, for example, asking if people would recommend this game, if they like this game, or if they think they learned something? While that is important, it is also very subjective. The deeper questions are: do they actually do something differently? Is there a change in behaviour? All these considerations are about the gameplay’s impact on reality but you can also test and evaluate other processes like the design phase.

Three different models need to be interconnected right from the start. The first model is the simulation model of the game, which deals with how and what parts of the reconstructed reality are incorporated into the game. The second model is a more didactic/pedagogical model: how do I apply this game? What is the right learning environment? What are the good debriefing methods to get the maximum learning effect for a particular group? How do I increase reflection on reality, make plans for the future, and change reality? Finally, the third model is an evaluation model. How do I evaluate the whole process of creating and using the game? And eventually, how do I improve the game? It is like a continuous improvement process, not just for the game as a model of reality but also for how you use the game with participants and debrief them about it. Unfortunately, there is not so much convincing research on that.

N.S. & G.M.: Can you tell us about your own experience with these evaluations?

Willy Kriz: I was fortunate to have funding for some more sophisticated evaluations of games, especially in a large project on entrepreneurship games in the business field. These games simulate the process of starting a company, developing a business model to kickstart the company, and then simulate the phases of market entry and competition with your company and its products. The game simulates a span of 2-3 years for starting a company and entering the market. This was part of a large, nationwide competition involving more than 100 universities, with students from various faculties playing in a cup system. Students first competed at the same university and the best teams of each university would compete against teams of other universities. Different business games with different scenarios were used until a final round with the best teams in the country. Throughout this cup system, we collected extensive data from over 10,000 students. We collected variables pre, post and during the gameplay, using questionnaires and interviews. We also had the opportunity to observe teams and students who qualified for the next level. So, while we had certain measurements at one level, we also tracked some teams as they progressed to higher levels. We also used personality tests from psychology, as well as knowledge and performance tests, such as the ability to write good business plans. One key aspect for us was to look at students who performed well compared to students who did not and investigate the later effect on real startups. We wanted to see if these students went to create their own companies, what kind of positions they held, and whether there was a higher rate of them building sustainable companies.

We also looked into whether there was a real transfer of the game. Students may be more motivated or knowledgeable, but does that transfer into behaviour? Do they actually start their own companies and have some business success? Fortunately, we were able to demonstrate this kind of long-term effects five years after their participation in this gaming cup system12. This was a fortunate situation as the research project was funded by the state. Normally, in-depth evaluation funds are not available and you can only do your best with what is available. From a scientific perspective, there is still a lot of room for good publications that prove real transfer. By ‘real transfer’, I mean not only to prove that participants are satisfied and learned something, but to have real change maybe three or five years after the gaming intervention. We are all convinced that games are great. Over the last decades, from practice and theory, you have a lot of research and publications arguing why games are good training and educational methods13. However, it is still mostly unproven on a more sophisticated scientific basis.

N.S. & G.M.: Is proof of effect always necessary? After all, companies or the military continue to invest in learning tools like simulation games.

Willy Kriz: Indeed, the military has been convinced about the value of simulations for many decades. It seems to be effective, otherwise they would not continue. The same goes for companies: why should they invest more money in the evaluation? They receive positive feedback from participants, who report learning something relevant and that is somehow enough. However, from a scientific perspective, there is a desire to have data showing that effect.

N.S. & G.M.: Are we, as scientists, focusing too much on rigorous evaluation to eliminate any doubt about the usefulness of games?

Willy Kriz: Exactly. As scientists, we want to prove that games trigger change, but I think that is not the main question. We already have enough data from the past decades to prove that, but the main question is trickier. The question is: under what conditions do you trigger change? Change varies for different target groups. For example, in my own research, we observed a gender effect: women gained competence and knowledge, but their motivation to start their own company slightly decreased. Of course, the goal is not just that students are more skilled; it is also about motivating them to apply those skills. Thus, the question became, why was the positive effect only for men and not for women? Evaluation can be used, not only to prove a summative effect, but for investigation. With a wealth of data and variables, we were able to identify patterns regarding why, and under what conditions, the impact of debriefing and gaming is stronger for certain target groups. We discovered that game scenarios, in our current socialisation system, are built in a way that it is more motivating for men14. Additionally, most of the trainers were men, so there were fewer women being facilitators of the game. In addition, there was not so much deep reflection. For the majority of men, competition was motivating enough. However, for most women, competition was not bad but not so important. What mattered most to them was the opportunity for reflection, gaining insights into their strengths and weaknesses and contemplating aspects of their own personality. Therefore, they sought more profound debriefing. To address this, we changed the debriefing procedures, modified the game scenarios, and improved the ratio between men and women facilitators, as facilitators serve as role models. Doing so, we achieved a better effect for women and, later on, for women’s entrepreneurship.

Evaluation is not just about proving that games are better than other methods, but about understanding the kind of debriefing needed and the facilitation variables to consider for maximum effect across different target groups. Different game scenarios, varied didactic game settings, and perhaps tailored debriefing methods to diverse social groups may all be necessary. I think much more formative evaluation is needed for this continuous improvement process.

N.S. & G.M.: Is there an advantage to using different games as well?

Willy Kriz: I strongly believe that the type and the methodology of the game should not be too similar. There are many different kinds of games: role-playing games, haptic games with tangible game pieces, digital computer simulations, etc. Regarding time, there are turn-based games, continuous games that progress through a storyline with different or similar phases, or real-time gaming. You can also use sophisticated 3D video game with 3D glasses. If possible, educational or corporate training program should aim at a maximum diversity in gaming approaches because there are many ways to simulate reality. I think it is also beneficial because different learners are sensitive to different forms of games. While I think this diversity of game is valuable, I do not have good research to back it up. That would be a great avenue for further research. From my own experience, we used very different games with high success. In this nationwide project about entrepreneurship games, we started with purely computer-simulated games. When we saw it was ineffective for some groups, we integrated haptic games, tangible games and board games for certain target groups, which led to much higher success in terms of skill-building, motivation, and awareness about the issues. We found that a mix of game types was more effective than just repeating the same kind of gaming approach. I believe there are compelling initial arguments from both theoretical and practice of evaluation, further scientific research is required.

N.S. & G.M.: Is it better to use games with students or to involve them in the process of creating the game?

Willy Kriz: I think it is more important for participants to create their own games and, if possible, play one or two games. It is always beneficial when they can create their own games and their own models about reality. Building a model and translating it into a game is very effective. Even if it is just a first prototype, I believe more in the game design process than playing a prepared game and debriefing it. If you take a prepared tool, the most creative part will be the debriefing to explore how the outcome would have been different if we changed some elements, or if we integrated neglected factors. Participants need to engage with the game in a way that encourages discussion about variations of the game. They can then debate about the advantages and disadvantages of incorporating different game and system elements. The debriefing is a moment to open up that creative process.

N.S. & G.M.: Can we consider there are two approaches to using games: normative and reflexive?

Willy Kriz: We do not want to have a normative model. We do not want to suggest that the game is telling the truth. In a quiz game asking you the capital of France, there is a truth. However, in a complex world where we want to understand the complexity and create a new alternative future, we do not want to repeat the same mistakes that created the mess we face today. So, it is not about being normative, it is about how we can better understand the situation and what can we do now to act differently.

N.S. & G.M.: What kind of advice would you give to someone interested in games regarding their effective utilization?

Willy Kriz: If you are dealing with a problem where the best or only solution is known, simple serious games motivate players to learn about it. However, this is not a simulation game. If you really want to use the full potential of gaming simulation, it is about creating futures, hypothetical scenarios, different alternatives which we can then discuss and draw conclusion from. Simulation games ask questions like: which way do we want to go? What kind of decisions do we have to take? What kind of behaviour do we have to change? This approach is useful to explore different futures when facing complex problems where there is no clear best solution, and where people do not even agree on a common model of reality due to diverse perspectives. You can bring these perspectives together through model building, game building, and again with different stakeholders in the use of the game. By debriefing those games, you create new ideas and alternative futures. From there, you can select what seems to be the best solution for most stakeholders, as there is often no one answer that fits for all15. Finally, you can use that as a blueprint to change elements in the reality. That is why I believe the gaming simulation approach is valuable to use. But I would advise to keep in mind that what we know is just a hypothesis and things can change quickly. We might have to build an improvement of the game or maybe a completely new and different kind of game in order to go further.


1

Kriz W.C., Eiselen T., Manahl W., 2014. The shift from teaching to learning: individual, collective and organizational learning through gaming simulation, Stuttgart, wbv Publikation.

2

Kriz W.C., Duke R.D. (Eds), 2014. Back to the future of gaming, Stuttgart, wbv Media GmbH & Company KG.

3

Kikkawa T., Kriz W.C., Sugiura J. (Eds), 2022. Gaming as a cultural commons. Risks, challenges, and opportunities, Singapore, Springer.

6

Dernat S., Etienne R., Hostiou N., Pailleux J.Y., Rigolot C., 2022. Ex-post consequences of participatory foresight processes in agriculture. How to help dairy farmers to face outcomes of collective decisions planning?, Frontiers in Sustainable Food Systems, 6, 776959, https://doi.org/10.3389/fsufs.2022.776959. Gee J.P., 2008. Learning and games, in Salen K. (Ed.), The ecology of games. Connecting youth, games, and learning, Cambridge, MIT Press, 21-40.

7

Crookall D., 2023. Debriefing: a practical guide, in Angelini M.L., Muñiz R. (Eds), Simulation for participatory education. Virtual exchange and worldwide collaboration, Cham, Springer, 115-214.

8

Duke R.D., 1974. Toward a general theory of gaming, Simulation & Games, 5, 2, 131-146, https://doi.org/10.1177/003755007452001. Klabbers J.H., 1989. Methodology-behavioural and social systems design and evaluation of games/simulations, classification, taxonomy, in Klabbers J., Takkenberg C., Scheper W., Crookall D. (Eds), Simulation-Gaming, Oxford, Pergamon, 215-218. Meadows D., 1989. Gaming to implement system dynamics models, in Milling P.M., Zahn E.O.K. (Eds), Computer-based management of complex systems. Proceedings of the 1989 international conference of the System Dynamics Society, Stuttgart, July 10-14, Berlin & Heidelberg, Springer, 635-640, https://doi.org/10.1007/978-3-642-74946-9_69.

9

Duke R.D., 2011. Origin and evolution of policy simulation: a personal journey, Simulation & Gaming, 42, 3, 342-358, https://doi.org/10.1177/1046878110367570.

10

Kriz W.C., 2003. Creating effective learning environments and learning organizations through gaming simulation design, Simulation & Gaming, 34, 4, 495-511, https://doi.org/10.1177/1046878103258201.

11

Flood S., Cradock-Henry N.A., Blackett P., Edwards P., 2018. Adaptive and interactive climate futures: systematic review of ‘serious games’ for engagement and decision-making, Environmental Research Letters, 13, 6, 063005, https://doi.org/10.1088/1748-9326/aac1c6. Kriz W.C., Hense J.U., 2006. Theory-oriented evaluation for the design of and research in gaming and simulation, Simulation & Gaming, 37, 2, 268-283.

12

Kriz W.C., Auchter E., 2016. 10 years of evaluation research into gaming simulation for German entrepreneurship and a new study on its long-term effects, Simulation & Gaming, 47, 2, 179-205, https://doi.org/10.1177/1046878116633972.

13

Sauvé L., Renaud L., Gauvin M., 2007. Une analyse des écrits sur les impacts du jeu sur l’apprentissage, Revue des sciences de l’éducation, 33, 1, 89-107, https://doi.org/10.7202/016190ar. Stanitsas M., Kirytopoulos K., Vareilles E., 2019. Facilitating sustainability transition through serious games: a systematic literature review, Journal of Cleaner Production, 208, 924-936, https://doi.org/10.1016/j.jclepro.2018.10.157.

14

Auchter E., Kriz W., 2013. Gender aspects by using start-up simulations for entrepreneurship education results of theory-based evaluation studies, Journal of Asia Entrepreneurship and Sustainability, 9, 1, 39-50, https://doi.org/10.25916/sut.26290309.

15

Kriz W.C., 2017. Types of gaming simulation applications, Simulation & Gaming, 48, 1, 3-7, https://doi.org/10.1177/1046878117689860.

Cite this article as: Kriz W.C., Salliou N., Martel G., 2024. Willy Christian Kriz: Current trends and challenges in evaluating the use of games in research. Nat. Sci. Soc. 32, 3, 357-365. https://doi.org/10.1051/nss/2024050.


© W.C. Kriz et al., Hosted by EDP Sciences

Licence Creative CommonsThis is an Open Access article distributed under the terms of the Creative Commons Attribution License CC-BY (https://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, except for commercial purposes, provided the original work is properly cited.

Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.

Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.

Initial download of the metrics may take a while.