At Nicosia Risk Forum 2022, INEGMA-E2 was presented in two different formats: First, there was an overall presentation on the project itself during the session dedicated to relevant EU projects invited to participate at the forum; and 2. As a side event designed by the INEGMA E2 team on the topic of exercise evaluation.
The presentation on the project sparked great interest from the audience. The question received from another member of the audience was on the network of evaluators and the platform that will be used after the end of the project to keep the network alive. The answer from our side was that this is being discussed in the moment internally within the team and we see the UCPKN as the most suitable platform for the sustainability of the network of evaluators.
For the side event on exercise evaluation the space given to us by the organizers was used to showcase the posters prepared for WP2/3/4 to present results and receive feedback. The key takeaways from that discussion include:
- The structure of the matrix in terms of system, structures and processes was seen as helpful, especially because of the possibility to apply these generic definitions to different exercise scenarios and evaluation focuses.
- Participants with a scientific background suggested that the definitions and methods developed in the project should be subjected to a peer review process in order to scientifically substantiate the practical methods. It was explained that this aspect is already integrated in the project planning and will be worked on in particular by the scientific partners within the consortium.
- The process of evaluation was presented starting from the preparations to the execution and after action review. The comments received were on using the term observer (as in Driver +) instead of evaluator and this needs to be changed to avoid confusion. Differences in terminology should be considered for all future developments within the project.
- Although the project results so far are backed up in terms of methodology and theory, they seem very academic and can create confusion among the evaluators which are practical people that appreciate simplified solutions.
- On the topic of the skillsets of the evaluator, the participants reflected on the importance of the: communication skills, the ability to present a constructive criticism, knowledge about exercise evaluation in general, expertise on the specific topic of the exercise (urban search and rescue, flood rescue, fire fighting etc.), the importance of the diversity of the team – to have not only specialists on one specific topic but also generalist, gender balance was also identified as important and participation of academic/pracademic people into the team of evaluators.
- On the topic of certification of future evaluators – the opinion was divided within the group. Yes, having certified evaluators creates the precondition for better quality of results but at the end of the day who will do that. Therefore, the consensus was not to go in that direction for the time being.
- On the topic of the network of evaluators the group talked that there is still not a profession: exercise evaluator. Most of the exercise evaluators by their opinion are coming from the UCPM roster of civil protection experts who have enough exercise/real-time experience and willingness to be in a such a role. Having the pool of evaluators was assessed as very important however that should not be only a list of names. As UCPKN has limited capacities to maintain such a network after project end some interim solutions need to be designed. The entire group considered creation of this network as very important output of the project itself.