How does SMILER improve M&E?
Project staff work hard to implement relief and development programs in effective, timely and cost-efficient ways. Because projects need information and data early in their implementation, technical and management staff often bring to a new project their own tools and approaches from past experiences. Their methods and tools may be valid, but each project needs a holistic and systematic approach. From an agency perspective, it is cost-effective to standardize the use of tools. SMILER is a field-tested solution that CRS has adopted agency-wide.
SMILER provides guidance about the essential but often-overlooked tasks of setting up and implementing an effective monitoring system. The approach enables staff to turn the M&E planning documents in the project proposal into a useful M&E system that can benefit all stakeholders.
SMILER is the backbone of the project. SMILER connects the objectives and their indicators to a transparent and easily understood system to collect, analyze and report on data. It includes mechanisms to turn data into useful evidence that supports sound project decision making and ensures that all staff have a clear understanding of the project and their role in M&E.
SMILER is endorsed by TOPS and approved for Food for Peace grantees. TOPS is a technical-support agency funded by USAID.
Success in Malawi
In Malawi, CRS and eight partners used SMILER to develop an M&E system for a five-year, $80-million program called Wellness and Agriculture for Life Advancement (WALA). The program is complex: it aims to improve food security in southern Malawi by using interventions in eight sectors to target five livelihood zones.
Staff held an initial SMILER session in November 2009 and then worked with consortium partners in the following months to finalize and implement a comprehensive M&E system. During this time, staff also created a joint M&E system with a new HIV and AIDS project and developed a shared database with this project.
As a result, the CRS Malawi program has a comprehensive M&E system in place that includes the following tools:
Assessments and evaluations
- Baseline survey and final survey
- Midterm evaluation
- Annual surveys
- Final evaluation
- Data flow maps
- Data gathering form and report formats, with instructions
Communication and reporting
- Communication and reporting maps for partners
- Knowledge and learning strategy
Process and tools to monitor M&E quality
- M&E technical working group
- Data-quality assessment
- Quality-improvement verification checklist
Information and communication technology
- Database for beneficiary numbers and selected indicators
- Mobile solution pilot to monitor household hunger
With this M&E system, the program is more efficient and effective. In Malawi, CRS now has a comprehensive set of data-gathering and report forms, and
a clear mechanism for conveying the information from its source all the way to the project’s reports. Staff have implemented surveys, collected accurate baseline data, and used the information to create annual updates and a midterm evaluation. Shane Lennon, WALA chief of party, notes, “The bottom line is that the USAID/Food for Peace office likes WALA’s M&E system.”
Staff are also implementing a system to track the quality of the data and attend to weaknesses in the system. With the database, project managers have an accurate census of the target population. They use this census to track which members of which households participate in project activities, without double counting.
—Shane Lennon, WALA chief of party
Staff successfully used SMILER to develop and implement an effective M&E system for this complex project. A review of the SMILER M&E system was conducted in 2011 and three important lessons emerged from the process.
1. Management and technical staff should ensure the integrity of the M&E system
Management has a central role in ensuring adherence to the M&E system. A fully operational, useful M&E system requires close oversight to make sure that staff are collecting the required data, using standard report formats, uploading data in the database and analyzing the findings for regularly scheduled events, such as annual meetings. While M&E staff lead the process to design and implement the system, the managers must ensure adherence.
2. M&E should be a priority throughout the life of the project, particularly in the first months of project implementation
Managers and technical staff are eager to get their project up and running. Establishing the M&E system is an important priority in the first months of project implementation and should not be displaced by activity planning.
Ensuring timely and accurate collection of data with many partners requires ongoing training and support. Community volunteers need initial training and ongoing support so they can gather and report on data in a standardized way.
3. Better data analysis can yield better insights
Partners use the data from the routine monitoring to measure their own progress and write reports. Quarterly meetings and technical working groups provide venues for discussing accomplishments and targets. But technical staff and management can push for a deeper analysis of the data to gain better insights about the program’s results.
There are more opportunities for rich discussions with communities and project volunteers who lead interventions and collect data. The CRS program is turning its attention to engaging participants in more meaningful ways regarding the M&E system.
Review of SMILER implementation
A team composed of regional and country-program staff will work with project staff to monitor progress. The team will evaluate the system’s strengths, weaknesses and gaps to develop an action plan for further improvement.
Information and Communication Technologies for Development