Chapter 5 of the Model Forest Toolkit
Monitoring and evaluation (M&E) of your Model Forest’s plans and activities helps gauge success and highlight what aspects of a program, project or activity might need to be adjusted in the future.
- Monitoring is the continuous assessment of program or project implementation and performance, and answers the question, “what is happening?”
- Evaluation is the periodic assessment of the relevance, cost-effectiveness and sustainability of a program or project, and answers the question, “did what was supposed to happen actually happen and why?”
Both of these M&E processes are important for obtaining a complete picture of your Model Forest’s success – and using them will help in designing and implementing better projects, enhanced accountability, greater capacity to demonstrate progress and direct financial savings for your Model Forest.
To get started, develop an M&E Framework, which will help you:
- Determine how you will measure impacts
- Build a monitoring strategy to track progress
- Learn and make programming adjustments on an ongoing basis
- Identify evaluation needs over the life of your Model Forest, and
- Support reporting and communication of impacts.
A typical M&E Framework consists of four main components:
- Logic Model– An illustration of how proposed activities are expected to help achieve the Model Forest’s outcomes and short-, medium- and long-term impacts
- Impact Monitoring Strategy– A plan for ongoing performance measurement by identifying targets and indicators of impacts
- Evaluation Strategy– A plan for evaluating your Model Forest’s performance
- Progress Reporting– A process to guide the sharing of your Model Forest results
View and download our M&E Framework templates from Google Drive.
To download Chapter 5 of the Model Forest Toolkit in PDF format, click here.
1. LOGIC MODEL
A logic model serves as a “road map”, showing the sequence of outcomes and impacts that are expected to logically flow from your Model Forest’s activities.
Model Forest Toolkit Logic Model Template
1.1. Start by listing the program areas (and activities) your Model Forest will undertake, as defined in your Strategic Plan, providing enough information to give an overview of the main components of your program – detailed project planning is not necessary. Some examples include: Science & Research, Communications & Public Awareness, Capacity Building & Training, and Management.
1.2. Next, identify the outputs associated with each of the program areas. Outputs are the range of tangible goods, services or products that will be generated as a result of a project or activity. There may be more than one per program area. These outputs are the evidence that a project or activity occurred. Some examples include: tools, workshops, conferences, field guides, manuals, websites, research reports, etc.
1.3. What are the outcomes that will occur from your Model Forest activities? These are the direct effects expected to see as a result of the project or activity and should be within the control of your Model Forest to obtain. Outcomes typically have an action word associated with them, for example: enhanced understanding of the Model Forest concept; increased availability of knowledge and tools to local landowners and decision-makers.
1.4. Finally, impacts are the developmental results of your Model Forest; that is, the real, sustainable changes that have resulted from the outcomes achieved by completing your projects and activities. For example: increased collaboration between partners on Model Forest projects; adoption / use of Model Forest products, tools, data, and processes beyond its boundaries; enhanced community capacity to manage local forest resources; reduced rates of deforestation.
- Short-term impacts would be seen within 1-2 years after the completion of an activity and extends in reach to the Model Forest’s target beneficiaries.
- Medium-term impacts would be seen within 2-5 years after the completion of and activity and effect both the Model Forest and surrounding area.
- Long-term impacts (5+ years) go beyond the life of an activity or project and are amplified within broader society. They should reflect the overall objectives of your Model Forest.
A logic model is usually laid out in columns, as a flow chart that shows the linking relationships from program areas through to impacts. See the template for some ideas.
2. IMPACT MONITORING STRATEGY
Impact monitoring provides the knowledge needed to effectively and efficiently manage your Model Forest projects/activities on an ongoing basis. This can provide reassurance that impacts are occurring as expected, or can serve as an early warning that the planned impacts are not occurring (and could lead to a decision for additional research, such as through an evaluation, to determine why).
Model Forest Toolkit Impact Monitoring Strategy Template
2.1. Start by listing the outputs, outcomes, and impacts from your logic model as elements and then identify your impact indicators by deciding what information or data would be needed to assess whether each output has been produced or each outcome / impact achieved.
At the output level, indicators can measure the quantity or quality of goods and services created or provided by your projects (e.g. number of people trained, opinions of teachers on training facilities provided, number of facilities in operating condition). An indicator at the outcome or impact level would measure the change (in quantity and/or quality) resulting from the goods and services produced by the projects. For example:
- If an output is a “workshop on improved bamboo shoot production”, an indicator might be the number of smallholders that were trained in improved bamboo shoot production techniques.
- If a short-term impact is “increased smallholder annual average production of bamboo shoots”, an indicator might be the “average annual production of bamboo
shoots”.
Indicators should, where possible, measure the impact of a situation rather than the input. For example, women’s literacy rate is a better measure of women’s educational status than female enrollment rates because literacy measures the effect resulting from a change in enrollment rates.
Note: Impact indicators should be designed to measure benefits and adverse effects on men and women separately whenever appropriate.
2.2. Next, identify targets or threshold values that can help differentiate acceptable from unacceptable progress. Targets are quantifiable levels of the indicator that your Model Forest stakeholders want to achieve in a given time frame. Targets make the goals concrete and allow for the comparison of actual performance and progress over time.
To set a good target, several pieces of information are required, including: impact indicator, target group (for whom), quantity (how much), quality (how well), time frame (by when), location (where), and baseline (from when and from what level). For example:
- Impact indicator: average annual bamboo shoot production by small landholder
- Target group: male and female small landholders (cultivating 2 hectares or less)
- Quantity: 500 small landholders increase average annual production by 50%
- Quality: maintaining same quality of harvest as baseline
- Time frame: by September 2020
- Location: within the Model Forest
- Baseline: 2018 baseline
Within the Model Forest, 500 male and female small landholders (cultivating 2 hectares or less) increase their average annual bamboo shoot production by 50% by September 2020, maintaining the same quality of harvest as at the 2018 baseline.
2.3. Finally, outline a realistic plan for data collection to inform your monitoring process. Answer the following questions:
- What sources are most appropriate? e.g., Who should be interviewed? Which documents should be reviewed? Does an organization already collect appropriate information?
- What methods for data collection should be used? e.g., sample surveys, administrative records, national statistics, workshops or focus groups, observation, etc.
- Who has responsibility for the data collection? e.g., staff, supervisors, independent evaluator, Model Forest partners, etc.
- What timing & frequency of data collection should be used? e.g., at start, monthly, annually, according to seasonal cropping cycles, during an evaluation, etc.
- What are the estimated costs for collecting the information?
- Are there any risk factors to consider?
Put all of these details into an Impact Monitoring Strategy table to help organize your answers. See the template for some ideas.
3. EVALUATION STRATEGY
Evaluation provides a periodic opportunity to take an in-depth look at the state of your Model Forest and achievements during a specified time period. Evaluations typically occur at two points:
- Relatively early in the life of a Model Forest (e.g., within the first two to three years). The focus is on examining how the Model Forest is being implemented, whether adjustments are necessary and whether progress toward the achievement of the impacts is occurring (called formative evaluations).
- After a Model Forest has been in place long enough that some impacts may have been achieved (e.g., within five years). The focus is on the degree to which these impacts have been achieved as well as to determine the contribution of the Model Forest to these achieved impacts (called summative evaluations).
Formative evaluations are focused on improving your Model Forest and feed back into ongoing work to enhance the probability that you will achieve your impacts. Summative evaluations look back on the level of achievement of impacts as part of a project or program ending.
Whether your evaluation is formative or summative, you will need an Evaluation Strategy.
Model Forest Toolkit Evaluation Strategy Template
3.1. The first step in developing an Evaluation Strategy involves identifying the issues and associated questions that need to be addressed during the evaluation.
There are four issue areas to consider:
- Relevance– The extent to which your Model Forest is meeting existing needs and is maintaining the core principles of a Model Forest. For example, is your Model Forest providing identified target audiences with the tools and information they require to implement the principles of sustainable forest management? Does your Model Forest realistically address an actual need? Is your Model Forest operating within its defined mission or mandate?
- Success– The extent to which your Model Forest is achieving its objectives. For example, is your Model Forest effective in meeting its intended outcomes and impacts, within budget and without unwanted negative impacts? Is your Model Forest making progress toward the achievement of long-term impacts?
- Cost-effectiveness– The extent to which impacts are being achieved efficiently and effectively. For example, are the most appropriate and efficient means being used to achieve impacts, relative to alternative design and delivery approaches?
- Sustainability– The extent to which projects, outputs, outcomes and impacts will continue after external support has ended. For example, has your Model Forest implemented mechanisms to ensure that e.g., skills, knowledge, and capacity will persist after an activity has been completed and/or funding ends?
3.2. Next, identify the data requirements that need to be collected in order to answer each evaluation question. It is important to identify any data requirements which rely on the initial or ongoing collection of information, to ensure that the necessary data are available at the time of an evaluation.
3.3. Lastly, outline a realistic plan for data collection needed to inform your evaluation process. Questions to consider:
- What sources of data are most appropriate? e.g., Who should be interviewed? Which documents should be reviewed? Does an organization already collect appropriate information?
- What methods for data collection should be used? e.g., sample surveys, administrative records, national statistics, workshops or focus groups, observation, etc.
- Who has responsibility for the data collection? e.g., staff, supervisors, an independent evaluator, Model Forest partners, etc.
- What timing & frequency of data collection should be used? e.g., monthly, annually, according to seasonal cropping cycles, during an evaluation, etc.
- What are the estimated costs for collecting the information?
Put all of these details into an Evaluation Strategy table to help organize your answers. See the template for some ideas.
4. PROGRESS REPORTING
Progress reporting feeds into your Model Forest’s communications and outreach plans and activities, ensuring that target audiences receive impact updates in a timely and appropriate way. Reporting on impacts and results helps assess where you are in comparison to where you planned to be, and it is a tool to stimulate discussion among Model Forest stakeholders.
An important aspect of progress reporting is the feedback of key messages into your Model Forest management process, allowing continuous improvement in the management and administration of the Model Forest and in achievement of identified impacts.
A Progress Reporting table can help you collect the information needed for these activities.
Model Forest Toolkit Progress Reporting Template
4.1. First, list the elements and their related impact indicators and targets as developed in your Impact Monitoring Strategy. For each item, collect and add the following information:
- Activities completed– A brief overview of the actual activities that were undertaken and the date they were completed. Activities completed are not impacts; they are highlighted to illustrate what the Model Forest has done, not what it has accomplished.
- Accomplishments– The impacts recorded from data collection during ongoing impact monitoring. This is the key column in the impact reporting summary. What changes are being seen?
- Impact analysis– An overview of the implications of the recorded accomplishments for the Model Forest, its beneficiaries, partners, etc. Have any additional, unanticipated impacts been identified?
4.2. Produce some descriptive text to accompany the Progress Reporting table. Consider including:
- Identification and analysis of key factors, issues or challenges affecting achievement of impacts
- Status update of financial and in-kind contributions from Model Forest participants and donors
- Update of the identified risks and mitigation plans
- The identification of new and emerging risks that are affecting or have potential to affect the achievement of impacts
- Information on required modifications to projects and associated resource allocation
- Lessons learned (for example, on the achievement and cost-effectiveness of impacts, relevance of results, partnerships, financial and human resources, and informed and timely action)
- Recommendations for action, and
- Degree of sustainability of impacts over a period of time.
Monitoring and evaluation should not be stand-alone, technical activities. They should be closely linked to decision-making processes at all levels and provide feedback to managers, Model Forest partners and others on, among other things, the performance of existing policies and programs.
The information posted here is simply a place to start. There are many monitoring and evaluation resources available online and professionals with expertise that can provide more support to help you plan around your own Model Forest’s needs and priorities.