Planning

Your K&A plan is a critical component in turning knowledge into practice.

You need to complete a K&A plan as part of your early project milestones. Activities listed in the K&A plan will need to be reported against milestone reports. The K&A plan is considered an iterative document which may change over time, once submitted.

It is expected that the research team will undertake the majority of the work to implement a K&A plan and that activities undertaken will be funded from the project budget.

The programme in which your project sits will be responsible for the broader K&A work including synthesis work across projects, to which you might contribute.

The extent of your K&A plan and implementation will depend upon a number of considerations including:

Support is available from Land & Water Australia's Knowledge and Adoption Officers and Program Coordinators.

K&A plan template guidelines

The K&A plan template guidelines and the template are designed as a guide, not a rulebook. These guidelines will help you with writing your K&A plan.

AttachmentSize
K&A_plan_template.doc41 KB
Example_of_a_completed_KA_plan_V1.pdf42.87 KB

Understanding target groups and individuals

Understanding target groups and individuals

This step is often forgotten! To implement an effective K&A plan, it is important to have a good understanding of the perceptions, needs and concerns of your stakeholders.

Key questions to ask yourself

To better understand priority groups and individuals, make sure you can answer the following questions about each target group/individual:

  • Perceptions
    • What do they already know and understand about the project?
    • Do they have any misconceptions?
  • Concerns
    • What are their concerns about the project?
  • Communication needs
    • What do they want to know?
    • How do they want to be communicated with, consulted, engaged?
    • How do they want to interact with the project team?
    • What help do they need to be informed and involved?

Identifying perceptions, concerns and communication needs

Listed below are tactics for identifying what is already known about people’s perceptions, concerns and communication needs. Results from surveys and questionnaires are useful for finding out about perceptions while the results of qualitative methods of research are useful for finding out about concerns and communication needs. Surveys are the only approach that yields statistically significant data. Other approaches, including those listed below, do provide a ‘snapshot’ of each target group/individual, but, because they may not give the full picture, they should be seen as preliminary assessments rather than final analyses. If you use 2 or more of these approaches and the information seems consistent, your ‘snapshot’ is more likely to be an accurate reflection of the full picture.

  • Review newspaper clippings featuring a relevant issue and/or target groups/individuals - This is a good way to get a quick overview.
  • Discuss target groups and individuals with colleagues - who have dealt with similar target groups/individuals and their issues. Consider including those in other states. This can give you a sense of the concerns that have arisen in similar situations.
  • Meet informally with target groups/individuals - Informal meetings or telephone contacts can give you a first-hand idea of both substantive concerns and the feelings about those concerns.
  • Send a personal letter/email to individuals asking them to send you a list of their questions and concerns about the project - This can be a useful way to start a dialogue with a greater number of people.
  • Review relevant survey results - Ask people who often run surveys, such as the Australian Bureau of Statistics and the Australian Bureau of Agricultural Resource Economics.
  • Review any other relevant research - into the natural resource issues or target groups/individuals. For example results of focus groups.
  • Brainstorm questions and concerns - at the beginning or end of a scheduled meeting with groups/individuals. Ask people to write their questions on index cards that you distribute and collect. Often you will want to know their concerns in advance of a meeting, but this approach can be very useful for making sure that you meet their concerns and for showing them that you are doing so.
  • Consult pre-existing or specially formed advisory committees - To be useful, advisory committees must be representative of the target groups and individuals you will be communicating with.
  • Search the internet for relevant names/organisations.
  • Get target groups and individuals involved in the K&A planning process - Ask them how they want to be involved.

How do you know who to communicate with?

How do you know who to communicate with?

Identifying target groups and individuals is largely a process of thinking through, as specifically as possible, who you want to reach or influence through the project. You also need to identify the method(s) by which you want to reach them:

  • providing information - making information available for those who seek it out
  • communicating - targeting information at users directly or indirectly
  • engaging - designing research questions with potential users who are involved in the research

Consider too the sort of relationship you have, or want to have, with each group/individual. Is it an impersonal ‘informing’ one-way relationship or a strong interactive ‘engaging’ relationship? The stronger the relationship, the higher the priority of the group/individual.

The table below shows that, for your high-priority target groups/individuals (those you want to engage with), you should use all methods of communication.

 Type of relationship
Priority of groups/individualsInformingTargeted one-wayTargeted two-wayEngaging
High-priority (engaging)HighHighHighHigh
Medium-high priority(communicating)HighMediumMediumLow
Low-medium priority (providing information)HighHighLowLow
Low priority (providing information)HighLowLowLow

Answering the following questions will help you identify the groups/individuals you need to reach. Those groups/individuals that appear in more than one response are particularly important for you to reach.

High-priority groups/individuals (engaging)

Which groups/individuals:

  • are currently involved in your project's activities?
  • are likely to be affected directly by your research outcomes?
  • are likely to be marginalised if they are not consulted or communicated with about the project?

Medium-high priority groups/individuals (communicating)

Which groups/individuals:

  • within your own organisation make decisions about the project?
  • would be helpful for you to communicate with because they might have important information, ideas or opinions?
  • should you involve to make sure that you have a balanced range of opinions?

Low-medium priority groups/individuals (providing information)

Which groups/individuals:

  • may not especially want to have input, but need to know what is happening?
  • belong to networks - particularly local ones - that you need to communicate with?
  • have previously been involved with the project or a related project?

Low priority groups/individuals (providing information)

Which groups/individuals:

  • are interested in the project’s research?

Reality check

Natural resource management projects generally have 3 major types of user—policy makers, planners and practitioners. The wider community may also be interested in the project. The priority you assign to groups/individuals will depend on your project. For example, if farmers are involved through participatory action research, they will be a high-priority group for you to engage with.

Use the checklist provided to help you identify your target groups/individuals.

Then, take a reality check on your list of target groups/individuals by considering the following four questions:

  1. Have you included everyone who is likely to benefit from the research?
  2. Have you included everyone who could ‘lose’ from the research outcomes/recommendations?
  3. Have you included people who could provide relevant expertise and information to the research process?
  4. Have you included people who are important for cooperation in or funding of the research activities?

Checklist: Who to include in your K&A plan

Policy makers

Natural resource use regulators or policy makers – those with a responsibility for either regulating the resource, developing strategies or policies about the resource, or developing legislation:

  • Federal government departments/agencies/committees involved in regulation/policy/legislation
  • State government departments/agencies/committees involved in regulation/policy/legislation
  • Local government/municipal authorities/committees involved in regulation/policy/legislation
  • Regional natural resource management agencies (e.g. Catchment Management Authorities)
  • Inter-agency or issues-based statutory committees whose findings or deliberations impact on the quality of the resource or people’s use of the resource
  • Other

Planners

Natural resource managers – those who directly manage the resource related to the research issue and need information for their planning:

  • Special user groups (e.g. Landcare, integrated catchment groups, indigenous groups)
  • Federal government departments/agencies/committees who directly manage the resource
  • State government departments/agencies/committees who directly manage the resource
  • Local government/municipal authorities/committees who directly manage the resource
  • Quasi-government agencies overseeing specific functions (e.g. water authorities, regional planning commissions, environmental commissions)
  • Other

Practitioners

Natural resource users – those who directly use the natural resources that are being researched:

  • Individuals who use land, water or other natural resources
  • Local council authorities with direct responsibility for the resource (but who may not manage the resource)
  • Regional natural resource management agencies (e.g. Catchment Management Authorities)
  • Industry associations or bodies that may be subject to (a) regulation, (b) modification to access to, or (c) subject to charge for resource use
  • Other

Natural resource use advisors – including both private and government advisors about impacts on or use of the natural resource

  • Local businesses (including agribusiness and banks) that provide advice about the resource
  • Consultants
  • Extension personnel from state government agencies, Landcare and Catchment Management Groups etc
  • Research organisations (e.g. Cooperative Research Centres, state government departments, universities, CSIRO)
  • Trade associations (e.g. Commercial Fisheries Association)
  • Professional scientific/technical associations (e.g. Ecological Society of Australia)
  • Other

Natural resource use funders – those who fund your project

  • Research and development funding organisations (e.g. Land & Water Australia)
  • Government funding arrangements/bodies (e.g. Natural Heritage Trust)
  • Other

Community

The broader Australian community – those with an in interest in the natural resource issue through either their general interest in environmental/sustainability issues and/or their involvement in recreation, tourism or using the products of the resources (food and fibre):

Local residents/community groups

  • Country Women's Association
  • Associations such as Lions, Rotary
  • Associations of senior citizens
  • Indigenous groups
  • Ethnic groups
  • Other organisations or individuals who have stature in the community and can influence opinion

Conservation groups

  • Local conservation groups
  • National conservation groups (e.g. Australian Conservation Foundation, Greenpeace)
  • State-wide conservation groups (e.g. Queensland Conservation Council)
  • Groups related to specific issues (e.g. rainforest protection, biodiversity, limnology)
  • Other

Business groups

  • Manufacturing industries (e.g. food processing)
  • Export agents/companies
  • Real estate agencies
  • Chambers of Commerce
  • Industrial groups
  • Other

Education

  • Colleges and universities
  • Primary schools and high schools
  • Other

Media representatives

  • Local
  • State/national
  • Special media (e.g. rural)
  • Other

Internal

Internal – those within your organisation or in collaborating organisations who may need to be involved in the communication

Internal

  • Project staff
  • Links with other projects in your organisation
  • Management

Advisory committees

  • Organisational
  • Community reference groups/panels

Target Audiences

 The key audience segments which Land & Water Australia and its programs address represent areas of LWA’s market that use different channels and require different messages. 

Each Segment comprises sub-groups, and some programs may only address a subset of the groups in each segment. Nonetheless, the segments are homogenous enough that a communication for one sub-group will usually be appropriate for another in that segment. There is a small amount of cross-over in some segments, where sub-groups may belong to more than one audience.

It should be noted that this analysis assumes a second variable (subject of interest) is also considered when planning activities, however in most cases this is defined by the program (i.e. NPSI deals with Irrigation issues).

Segment: Agricultural Extension

Who?

  1. Public Extension Agents
  2. Agribusiness advisers
  3. NGOs dealing with Agriculture (such as Landcare)

What?

Accurate information packaged to serve their clients (the Farmers and Landholders segment)

Relevant LWA subjects include: Irrigation, Climate Variability, Weed management, Soil, Mixed farming, Salinity, River and Waterway management.

Potential Methods of Engagement

  • Presentations
  • eNewsletters
  • Workshops
  • Guides and tools.
  • Demonstrations of NRM Tools
  • At Industry-related events 

Segment: Community Engagement

This group is organisations and individuals who are involved in community engagement of NRM issues, in the same way NRM facilitators might engage agencies and organisations.

Who?

  1. Community groups
  2. Landcare etc.
  3. Community engagement staff in Regional bodies
  4. NGOs

What?

Increasing awareness and participation in local or regional NRM issues. Support community and other stakeholder involvement in NRM.

They generally use research findings for education and practical management of NRM issues and sustainable resource use.

Potential Methods of Engagement

  • Individual consultations
  • web and email information
  • workshops
  • forums and conferences

Potential benefits

Outcomes in this segment could be linked to the Community skills, knowledge and engagement Priority Area of the Caring for our Country program.

 

Segment: Farmers and Landholders

Who?

  1. Farmers including growers, graziers and mixed farming
  2. Non-farming Landholders

What?

Farmers and landholders require practical, on-the-ground approaches to sustainable land management. They do not use research findings specifically, but the knowledge or tools derived from that research.

Irrigation, Climate Variability, Weed management, Soil, Mixed farming, Salinity, River and Waterway management

Potential Methods of Engagement

  • Demonstration Sites and Field Days
  • Workshops
  • Rural Press & Regional Radio
  • LWA Web Portal for Landholders & Farmers
  • Landcare and Production Groups
  • Industry bodies (e.g. National Farmers Federation)
  • Participatory research

Notes

Communicating with this group is likely to be difficult, compared with most other segments, and it is worth considering whether another channel is more efficient, such as Agricultural Extension or Community Engagement.

 

Segment: Influencers and Decision Makers

Who?

Managers, politicians and other decision makers and leaders. This group tends to be quite similar across sectors.

What?

Their role is predominantly decision making/guiding/finding resources/spending money. They rely on the subject specialists in their team to come up with the answers to the problems their organisations have to manage. They require sufficient information to know when their subject specialists are doing the right thing.

This group will also often working at a high-level thinking/strategy level, however they may be unable to consider this activity in their day-to-day work. The strategy and thinking will often be done at specific times of the year, facilitated through special sessions or discussion (including conferences, strategy workshops etc).

How?

They are time-poor and increasingly turn to the internet to get fast, subject-specific information to support decision making processes. They are likely to take the advice of their staff on important issues, and often their own specialist subject is “institutional knowledge”, or how their own organisation works.

They will reference internal policy and guidelines, summary briefs (both internally generated and from news or specialist sources), daily newspapers and magazines.

They are increasingly unlikely to use libraries and information repositories, corporate magazines, scientific journals and annual reports, or other information-heavy documents.

Potential Methods of Engagement

  • eNewsletters
  • Conferences
  • Events specifically targeted at the executive level

 

Segment: NRM Facilitation

Who?

  1. Caring for our Country NRM Facilitators
  2. Public Extension Agents

What?

Communicate Government policies and initiatives and provide opportunities for direct community feedback on NRM issues to Government. Engage Local Government in NRM at the regional scale; promote and disseminate information about Australian Government NRM policies and programs, and assist Local Government with the development of NRM policies

Potential Methods of Engagement

  • Presentations (in-person and online)
  • eNewsletters
  • Consultations
  • Workshops
  • Guides and tools.
  • Demonstration of NRM Tools

 

Segment: Policy and Regulation

Who?

  1. Policymakers in State and Federal Governments
  2. Regulators (especially water regulators)
  3. Lobbyists & NGOs

What?

Determining priorities, legislation, allocation and development opportunities; guiding policies and frameworks (Federal); structures for on-ground and local/regional planning (State). Sustainability of state resources. Benchmarking.

Uses research findings to develop, evidence and influence policy and regulation decisions.

Social, Institutional, Governance Issues; Water regulation and irrigation, Climate issues,

Potential Methods of Engagement

  • Policy Fact sheets
  • LWA Web Channel – Policy
  • Program Websites
  • Conferences and seminars

Segment: Resource Management

Who?

  1. Local resource management specialists
  2. State planning and resource management staff
  3. State service providers
  4. Water regulators
  5. State agencies, statutory authorities, regional NRM bodies and CMAs
  6. Conservation bodies

Individuals in this group are likely to stretch from Environmental Science through to Engineering backgrounds, so there will be some variance in the types of messages constructed for this group. However, they are likely to be fairly similar within a given subject area.

What?

Resource planning (for specific regions), water planning and environmental flows, operational data, compliance, flood and drought response, conservation, environmental and ecosystem services.

Uses research for planning and practical delivery of NRM outcomes. Also sometimes called “on-the-ground”.

Service provider are also likely to be interested in: Operational, compliance data, annual water accounts, and environmental flows

Potential Methods of Engagement

  • Presentations (in-person and online)
  • Newsletters
  • Consultations
  • Workshops
  • Guides and tools.
  • Demonstration of NRM Tools
  • Conferences and seminars

Segment: Science and Research

This group is different from other audiences, as it represents individuals with whom LWA conducts a business relationship with (contracting research), rather than and Adoption relationship. Consequently, it should be treated differently.

Uses research findings to influence and guide new research, research priorities, or through synthesis.

Because information about research is quite different from the adoptable learning’s resulting from research, this group should be communicated with separately from other groups.

Who?

  1. Private researchers and consultants
  2. Government and academic researchers
  3. Government data agencies
  4. Land & Water Australia researchers (current, past, potential)
  5. Tertiary and Postgraduate students

What?

Usually, research and funding news relating to a specific program or subject area, but also research strategy, calls for funding and other administrative material.

Potential Methods of Engagement

  • eNewsletters
  • RSS Feeds
  • Conferences
  • Scientific Journals
  • “For Researchers” Portal on LWA Websites
  • Program Websites

 

Participatory Research

What is Participatory Action Research?

Participatory action research (PAR) is an approach to research in which a community or other group takes on the role of researcher or co-researcher i.e. they own the research and are responsible for putting the results of the research into action. If professional researchers are also involved, they are equal participants with the community or group doing the research. PAR requires that all research participants:

  • recognise the value of local knowledge
  • accept and own research results
  • are willing to be involved in all stages of the research
  • are willing to include a wide range of participants
  • choose research methods that suit the situation, and that communities or groups can learn to use without outside help

PAR is designed to break down the barriers between researchers and the users of their research. It makes the research immediately relevant and applicable to the local need. It is particularly useful when long-term change is needed and in situations where community groups can gradually apply the research methods to create change.

PAR is an iterative process with 4 phases:

  1. Reflection – participants decide what needs to be researched
  2. Planning – participants plan how to carry out the research
  3. Action – participants implement the plan
  4. Observation – participants collect information

After each of the 4 phases, it is important that participants reflect on what has happened and decide whether they can practically apply the collected information and/or whether more research is required. In this way, PAR is a continuing process of change.

Closely related terms

PAR resembles other types of research, such as action research, participant observation, participatory research, collaborative inquiry, emancipatory research, action learning, and contextual action research. But only PAR embraces all of the following five principles (Grundy 1982):

  • participation
  • collaboration
  • empowerment
  • knowledge
  • social change

Why use PAR?

When you use PAR, you gain access to the knowledge and expertise of the people who are most affected by your research problem. As a result, PAR:

  • gives you an increased depth and understanding of the issues, making your research more relevant
  • increases the chances of the community adopting practices and outcomes of your research
  • reduces logistical barriers to research, which can improve efficiency and reduce frustration for everyone
  • gives you access to observe behaviour as it is happening, increasing the efficiency of your data collection
  • minimises the risk of misinterpretation and misunderstanding because of the close relationship that you develop with the community

PAR also has some disadvantages compared to traditional research methods:

  • it can be more time consuming if participants get caught up in details
  • it can be susceptible to researcher bias because the researcher is involved
  • it is not as well recognised as quantitative research methods
  • it can produce large amounts of information that are difficult to manage

When is it appropriate to use PAR? PAR is particularly useful for:

  • building partnerships and trust
  • developing action plans for solving problems
  • empowering the community and developing community capacity
  • identifying community issues and relevant research
  • engaging the community
  • planning for, and increasing awareness of, upcoming change
  • accessing specialised local knowledge

PAR is not useful in the following situations (Whyte 1989):

  • where the community group is interested in the results, but not in developing the topic or design of the research
  • where the problem does not seem important to the community group
  • where the research methods and types of data being collected do not appear credible to the community group
  • where the research has had considerable theoretical development and needs testing

Who can participate?

Anyone who is connected to the research topic! Representatives of schools, organisations, businesses, industry, government agencies, farmers/landholders, community groups and research institutions can all participate. However, this does not mean that everyone who is connected to the topic needs to participate.

Where is PAR being used?

PAR is used extensively in health care and education research, and its popularity is increasing in other fields including rural research and development, on-farm client-oriented research, and urban community development.

Examples of PAR in research

Kimberley locals know their fish

Murdoch University’s David Morgan and Mark Allen embarked on a project to survey the fish of the Kimberley’s Fitzroy River. Consulting with the Kimberley Language Resource Centre and a large number of local people, both Indigenous and non-Indigenous, they developed a series of brochures illustrating the fish of the river system.

In the brochures, they provided English common names for the fish, scientific names, and Indigenous names in 5 of the 6 language groups of the west Kimberley—Bunuba, Gooniyandi, Ngarinyin, Nyikina and Walmajarri.

Three of the freshwater fish species discovered had Indigenous names but had never been previously described or given English or scientific names.

The research inspired an LWA-funded project to look at the cultural significance of the fish of the Kimberley’s King Edward River. By using PAR, the researchers gained access to expert knowledge that they might otherwise have missed. Reflecting on their project gave them a new perspective on the significance of fish in other river systems.

I want to find out more: Centre for Fish and Fisheries Research

Estuarine health in Little Swanport

Oyster farmers in Little Swanport on the east coast of Tasmania are playing an integral role in research that is looking at how their estuary responds to changed freshwater inflows.

The project started when a concerned farmer organised a meeting between key representatives and funding agencies. Now, the project is funded by LWA and the Fisheries Research and Development Corporation, and is a joint effort of the community and local researchers.

Jeff Ross from the University of Tasmania says, ‘[the farmers] get directly involved in the science at all levels, including preparing chlorophyll samples and sorting phytoplankton samples’. But their participation includes much more than field support. They provide access to their boat ramp and storage facilities, and they donate oysters and water samples of different qualities for experiments, among other activities.

‘They have worked in the estuary on a daily basis for over 20 years and have an incredible wealth of knowledge and insight about how the estuary operates. This knowledge is invaluable to us as scientists given the short-term nature of our project.’

‘You could say that we are collecting quantitative data to help prove or disprove some of their anecdotal observations, such as how important freshwater inflow is to oyster production.’

The Little Swanport catchment has a history of involving various target groups, including the community, in developing catchment management plans.

‘Without their participation, much of what we are doing in Little Swanport would be difficult or impossible.’


I want to find out more: Rip Rap magazine no 29

More PAR information:

K&A plan template guidelines

Intended impact of your project (objectives):

The intended impact of your project essentially links back to your project's objectives. In undertaking your research, you intend to provide some new knowledge which would improve the management of natural resources. What is this impact? The following prompts may assist you in determining your intended impact:

  1. Is the project seeking to influence on-ground practice, NRM policy or NRM planning? At what stage in the project's life?
  2. Who are you trying to reach/influence through this project?
  3. Is there more than one target group? (Define each group precisely.)
  4. Why would the target group want to be involved with the project or the uptake of this research?
  5. What is the best way of reaching the target group?

Examples of intended impacts:

  • to inform policy
  • to build capacity amongst planners
  • to improve decision making processes
  • to inform an emerging scientific field.

Who (Target)

Who is involved in, affected by or interested in your project research in natural resource management can contribute to a range of activities. In the strategy we categorise them into 3 areas:

  • Policy - The Australian and State governments
  • Planning - e.g. 56 regional NRM groups (CMAs) and local governments
  • Practitioner - e.g. land managers, extension staff and networks, and agencies or organisations that manage land assets.

Note: These are a guide only - some individuals and organisations sit in more than one category.

Your project may be relevant to some or all of these sectors. Understanding their attitudes and practices concerning the research and/or the issue the research addresses will assist you in reaching them more effectively.

A comprehensive contact list for your target participants and audiences is fundamental to undertaking engagement and communication activities. It also assists in monitoring and evaluation.

I want to find out more:
 

  • Examples of target audiences

Type of engagement/how (Method)

How are they to be engaged (method)

There is a broad range of methods to manage knowledge for adoption, from direct engagement or collaborative research through to tailored communication products and, finally, indirect information provision.

Selection of your methods depends on the content, target audience, required outcomes and resources available for implementation or delivery. Not all of the methods will be applicable to all projects.

The method may often be influenced by the ‘adoptability’ of the research e.g. relevance, trialability, skills required and cost implications.

I want to find out more:

  • Examples of methods

Monitoring and Evaluation (engagement & impact) (Measure)

Monitoring and evaluation (M&E) at the project level links to program and corporate monitoring and evaluation. The monitoring and evaluation column refers to each individual line item that you have identified a target and method against – it does not refer to evaluating your entire project. It shouldn’t duplicate any other M&E work or project work that you do either.

It is useful to consider measuring how you have undertaken these activities as well as the outputs and outcomes. For example, feedback on how a steering committee or workshop has been run can be useful for making immediate improvements while the project is still underway. Measuring outputs is the easiest step – e.g. how many events have been run and how many people attended. Capturing outcomes can be more difficult in the lifespan of a project, particularly adoption outcomes. These are more likely to be measured at a program or corporate level with techniques such as Return on Investment.

I want to find out more:
 

  • Examples of monitoring and evaluation techniques
     
  • Monitoring and Evaluation

Legacy

A K&A project plan should consider methods for project implementation and methods for project legacy when the research project is completed.

Your project may have important outcomes for adoption beyond the lifecycle of your project. A legacy plan ensures the research outcomes are not forgotten upon completion of your project. Managing the project’s legacy may be undertaken as part of the program.

I want to find out more:
 

  • Examples of legacy activities 
  • Legacy
     

Advice regarding media and branding

Before preparing a publication, presentation or media release contact Land & Water Australia to check any style, branding or media guidelines you should be using.

Formatting, style and branding issues are best handled early in production to minimise angst and cost.

AttachmentSize
K&A_plan_template.doc41 KB
Example_of_a_completed_KA_plan_V1.pdf42.87 KB

Examples of methods

Engagement:

  • Participatory action research
  • grower or user initiated research
  • research advisory committees
  • citizens’ juries
  • public hearings
  • web-based meetings
  • grower field sites

Communication:

  • policy briefings
  • tailored workshops
  • targeted issues papers
  • information centres and field offices
  • one-to-one surveys and response sheets
  • focus groups

Information Provision:

  • general publications
    • fact sheets
    • newsletters
    • project reports
    • journal articles
  • websites
  • general media
    • newspapers
    • television
    • press releases
    • advertisements

Examples of monitoring and evaluation techniques

Engagement:

  • most significant change
  • ex-poste adoption survey
  • feedback on effectiveness and appropriateness of engagement processes
  • surveys on likelihood of uptake and barriers to uptake

Communication/information provision:

  • longitudinal survey of attitudes
  • feedback on usefulness, structure and content of events
  • downloads from a website
  • numbers at a seminar
  • feedback on publications

Monitoring and Evaluation

Monitoring and evaluating (M&E) your project's K&A activity, such as a workshop, gives you information to assess that activity. M&E provides an opportunity to learn and improve as you go, feeding the results back into research and employing adaptive management.

Knowing how, where and by whom your research is being heard about, tested, or applied can also:

  • confirm some of your assessments with evidence,
  • guide your allocation of resources, and
  • provide information for future papers and publications.

I want to find out more:

  • Monitoring & Evaluation questions
  • How to measure your impact
  • Define your performance indicators
  • Select your methods
  • Feedback on your Monitoring & Evaluation
AttachmentSize
Some_M&E_Methods_V_0_1.pdf37.49 KB

Define your performance indicators

As with research, developing the most relevant objectives, performance indicators, questions and methods is usually well worth the time invested.

For example: if your knowledge and adoption objectives are to increase farmers’ awareness and adoption of native grasses for grazing in Newhaven:

  • One of your performance indicators may be that 'After six months from the start of the project, 40% of farmers in the Newhaven area will have heard about the value of native grasses for grazing in Newhaven.'

Your evaluation methods for this performance indicator may include polling farmers at a community meeting, conducting short phone interviews with a sample of local farmers, discussing with the key leaders in the farming community and/or advisers.

One of your performance indicators may be:

  • the number of farmers actually using/planting native grasses over the lifetime of your project, and whether this number increases over time
  • whether relevant farming groups and agricultural advisers are incorporating the value of native grasses for grazing in Newhaven in their information and advice
  • whether a workshop or webcast on native grasses for grazing in Newhaven was a success, and who for
  • whether new publications on the value of native grasses for grazing were received by the target audiences and seen as useful
  • the media and web pick-up on your research/issues

Feedback on your Monitoring & Evaluation

Discuss your survey questions with a sample of researchers and/or natural resource managers you know, and ask them for their feedback:

  • Are the evaluation questions hitting the mark?
  • Are your evaluation methods and questions going to give you information that you can use to improve your knowledge and adoption?
  • Can they see any gaps in your evaluation, or make any suggestions?

 

How to measure your impact

Research that contributes to different areas will need different approach

If your research contributes to government policy then the stakeholders, knowledge and adoption approaches, and performance indicators you use may be different to those you would use for a collaborative industry project that contributes to on-ground practice.

The two main types of monitoring and evaluation methods are qualitative and quantitative.

Qualitative evaluation methods

Allow stakeholders to explore issues and provide feedback in more depth and complexity, unbiased by set questions. They can give details of evidence, examples, problems and ideas, but can be more difficult and costly to analyse and report.

Qualitative evaluation methods include:

  • focus groups
  • individual in-depth interviews
  • written comments
  • most significant change
  • quick polling face to face

Quantitative evaluation methods

These are relatively easy to analyse and report, but don’t tell you ‘why’ results are as they are.

Quantitative evaluation methods include:

  • survey questionnaires
  • special instruments such as 360 degree surveys
  • electronic surveying
  • card sorts
  • quick voting

Quantitative evaluation methods allow you to:

  • get precise measurements
  • track progress over time
  • measure strengths and weaknesses
  • compare to benchmarks

Some quantitative evaluation questions:

  • Was a draft report submitted to X Committee by a specified date?
  • How many people attended a public meeting?
  • What was the increase in the number of requests to be put on the electronic newsletter list?

Monitoring & Evaluation questions

Think about your research project:

  • Do your stakeholders feel informed about developments in your project, and to what degree?
  • Do your stakeholders know more about an issue or practice after a period of time?
  • Which communication activities were effective, and for which audiences?
  • Does web traffic increase to local Landcare and regional NRM agency websites following the establishment of an online discussion group about an issue or practice?
  • Can visitors to your trial sites apply the principles of what they had learnt?
  • Can visitors see any barriers to adoption and what could encourage them to adopt related practices?
  • What are the key points that the field day participants had learnt?
  • Are there any unanswered questions, if so, what are they?

Select your methods

Knowledge and adoption outcomes can be measured in a variety of ways, from short electronic surveys to in-depth case studies. A combination of evaluation methods is often the best way to go.

  • Check that the evaluation methods will answer the evaluation questions, and that they relate back to your project's objectives.
  • Consider the cost, time, resources and skills available when choosing methods.
  • Consider the ethics of your monitoring and evaluation – the people who are using your research are probably busy, and many producers and regional groups are over-surveyed.