Monitoring and Evaluation

Monitoring and evaluating (M&E) your project's K&A activity, such as a workshop, gives you information to assess that activity. M&E provides an opportunity to learn and improve as you go, feeding the results back into research and employing adaptive management.

Knowing how, where and by whom your research is being heard about, tested, or applied can also:

  • confirm some of your assessments with evidence,
  • guide your allocation of resources, and
  • provide information for future papers and publications.

I want to find out more:

  • Monitoring & Evaluation questions
  • How to measure your impact
  • Define your performance indicators
  • Select your methods
  • Feedback on your Monitoring & Evaluation
AttachmentSize
Some_M&E_Methods_V_0_1.pdf37.49 KB

Define your performance indicators

As with research, developing the most relevant objectives, performance indicators, questions and methods is usually well worth the time invested.

For example: if your knowledge and adoption objectives are to increase farmers’ awareness and adoption of native grasses for grazing in Newhaven:

  • One of your performance indicators may be that 'After six months from the start of the project, 40% of farmers in the Newhaven area will have heard about the value of native grasses for grazing in Newhaven.'

Your evaluation methods for this performance indicator may include polling farmers at a community meeting, conducting short phone interviews with a sample of local farmers, discussing with the key leaders in the farming community and/or advisers.

One of your performance indicators may be:

  • the number of farmers actually using/planting native grasses over the lifetime of your project, and whether this number increases over time
  • whether relevant farming groups and agricultural advisers are incorporating the value of native grasses for grazing in Newhaven in their information and advice
  • whether a workshop or webcast on native grasses for grazing in Newhaven was a success, and who for
  • whether new publications on the value of native grasses for grazing were received by the target audiences and seen as useful
  • the media and web pick-up on your research/issues

Feedback on your Monitoring & Evaluation

Discuss your survey questions with a sample of researchers and/or natural resource managers you know, and ask them for their feedback:

  • Are the evaluation questions hitting the mark?
  • Are your evaluation methods and questions going to give you information that you can use to improve your knowledge and adoption?
  • Can they see any gaps in your evaluation, or make any suggestions?

 

How to measure your impact

Research that contributes to different areas will need different approach

If your research contributes to government policy then the stakeholders, knowledge and adoption approaches, and performance indicators you use may be different to those you would use for a collaborative industry project that contributes to on-ground practice.

The two main types of monitoring and evaluation methods are qualitative and quantitative.

Qualitative evaluation methods

Allow stakeholders to explore issues and provide feedback in more depth and complexity, unbiased by set questions. They can give details of evidence, examples, problems and ideas, but can be more difficult and costly to analyse and report.

Qualitative evaluation methods include:

  • focus groups
  • individual in-depth interviews
  • written comments
  • most significant change
  • quick polling face to face

Quantitative evaluation methods

These are relatively easy to analyse and report, but don’t tell you ‘why’ results are as they are.

Quantitative evaluation methods include:

  • survey questionnaires
  • special instruments such as 360 degree surveys
  • electronic surveying
  • card sorts
  • quick voting

Quantitative evaluation methods allow you to:

  • get precise measurements
  • track progress over time
  • measure strengths and weaknesses
  • compare to benchmarks

Some quantitative evaluation questions:

  • Was a draft report submitted to X Committee by a specified date?
  • How many people attended a public meeting?
  • What was the increase in the number of requests to be put on the electronic newsletter list?

Monitoring & Evaluation questions

Think about your research project:

  • Do your stakeholders feel informed about developments in your project, and to what degree?
  • Do your stakeholders know more about an issue or practice after a period of time?
  • Which communication activities were effective, and for which audiences?
  • Does web traffic increase to local Landcare and regional NRM agency websites following the establishment of an online discussion group about an issue or practice?
  • Can visitors to your trial sites apply the principles of what they had learnt?
  • Can visitors see any barriers to adoption and what could encourage them to adopt related practices?
  • What are the key points that the field day participants had learnt?
  • Are there any unanswered questions, if so, what are they?

Select your methods

Knowledge and adoption outcomes can be measured in a variety of ways, from short electronic surveys to in-depth case studies. A combination of evaluation methods is often the best way to go.

  • Check that the evaluation methods will answer the evaluation questions, and that they relate back to your project's objectives.
  • Consider the cost, time, resources and skills available when choosing methods.
  • Consider the ethics of your monitoring and evaluation – the people who are using your research are probably busy, and many producers and regional groups are over-surveyed.