Competency Assessment Tool

Quantitative Component

The quantitative component of the competency assessment tool is designed to track changes in fellows’ self-reported confidence, understanding, and ability to use modelling evidence over the course of the Fellowship. The same 14 Likert-scale questions are used at baseline, midline, and endline, allowing responses to be compared across time points.

The questions are organised around three competency areas that reflect the core aims of the RMMT Fellowship: understanding and interpreting modelling outputs; applying modelling to decision-making; and collaboration, communication, and influence. Together, these areas capture both technical engagement with modelling and the broader skills required to translate modelling evidence into policy-relevant action.

Each question is scored on a five-point Likert scale, where 1 = Strongly Disagree and 5 = Strongly Agree. Scores can be analysed by individual question, competency area, survey time point, and cohort. Changes in scores over time should be interpreted as changes in fellows’ perceived confidence and capability, rather than as a formal test of technical proficiency.

Interpreting Quantitative Results

The Likert-scale questions provide a structured way to assess perceived change over time. They are most useful when interpreted alongside qualitative feedback, which helps explain why fellows’ confidence may have increased, remained stable, or varied across competency areas.

Survey Questions

Competency Area 1. Understanding and Interpreting Modelling Outputs

Short Name	Question
Distinguish Analyses	I can distinguish between different types of analyses, such as mathematical modelling, machine learning, exploratory data analysis and statistical forecasting.
Basic Principles	I understand the basic principles of malaria transmission modelling.
Interpret Key Outputs	I am confident in interpreting key outputs from malaria models, such as incidence projections and intervention impact.
Uncertainty	I understand how uncertainty in data is included in malaria models.
Assumptions and Limitations	I understand the assumptions and limitations underlying malaria modelling outputs.

Competency Area 2. Applying Modelling to Decision-Making

Short Name	Question
Identify Policy Questions	I can identify policy questions that could be informed by modelling.
Translation to Recommendations	I am confident in translating model results into actionable recommendations.
Align Outputs with Timelines	I understand how to align modelling outputs with policy and programmatic timelines.
Support Decision-making	I feel capable of using modelling evidence to support strategic decision-making.

Competency Area 3. Collaboration, Communication, and Influence

Short Name	Question
Communicate Technical Results	I can effectively communicate technical modelling results to non-technical stakeholders.
Collaborate with Modellers	I am confident in initiating and maintaining collaborations with modelling experts.
Facilitate Dialogue	I understand how to facilitate dialogue between modellers and policy-makers.
Advocate for Modelling Use	I can advocate for the use of modelling in malaria planning and resource allocation.
Advocate for Data-driven Decision-making	I am confident in advocating for data-driven decision-making within my organisation or country.

Qualitative Component

The qualitative component is designed to provide richer insight into fellows’ experiences, learning needs, and reflections on the Fellowship. While the quantitative questions allow change to be tracked across time, the open-ended questions help explain how and why that change occurs.

The qualitative questions vary across baseline, midline, and endline surveys to reflect the different purpose of each evaluation point. At baseline, questions focus on expectations, prior experience, and areas where fellows feel least confident. At midline, questions focus on progress, challenges, and support needs while there is still time to adapt delivery. At endline, questions explore overall experience, perceived learning gains, programme design, broader impact, and future application.

This component is particularly important for understanding how fellows connect the curriculum to their own roles, country contexts, capstone projects, and policy environments. Responses can help identify which aspects of the Fellowship are most useful, which topics require additional support, and how future cohorts could be strengthened.

Using Qualitative Feedback

Open-ended feedback should be reviewed during the Fellowship, not only at the end. Midline responses are especially valuable because they allow facilitators to adjust pacing, revisit challenging topics, provide additional examples, or strengthen capstone support while the programme is still underway.

Suggested open-ended questions for each survey are shown below.

Survey	Suggested Open-Ended Questions
Baseline	What do you hope to gain from the Fellowship? What experience have you had with malaria modelling or model outputs? What areas do you feel least confident about at this stage?
Midline	Which aspects of the course do you feel: most confident about? least confident about? are most beneficial to your work? Which topics or activities would benefit from further clarification? What additional support would help you apply learning to your capstone project?
Endline	Overall Experience How would you describe your overall experience of the RMMT Fellowship? What three words best capture your experience of the fellowship? How well did the programme meet your expectations? Learning and Impact How has your understanding of malaria modelling and its policy application evolved during the fellowship? What have been the most useful or impactful skills, tools, or knowledge you have gained during the fellowship? How has your confidence and ability to engage with modelling evidence for policy and decision-making improved since the start of the fellowship? Programme Design and Delivery How effective was the structure and pacing of the programme (e.g., balance between theory and application, in-person weeks vs virtual sessions, pace and level of content)? Were there any logistical or delivery aspects that could be improved (timing, communication, workload, materials)? What suggestions would you make to enhance future cohorts? Broader Impact and Recommendations Who do you think would most benefit from participating in future RMMT fellowships (e.g., roles, sectors, experience levels)? Would you recommend the RMMT Fellowship to others? Why or why not? Would you be interested in attending a follow-up RMMT course and what should be the primary focus? Personal Reflections What has been your biggest personal or professional takeaway from this journey? How do you see yourself applying what you’ve learned in the next 6–12 months? Please share a short testimonial (2–3 sentences) that captures what this fellowship has meant to you.