"To obtain partnership is an experience in communication."
Baselines are generally described as a collection of descriptive and quantitative data, usually collected at the early stage of the project, mainly for the purpose of establishing a benchmark by which to measure change. Baselines are often associated with questionnaire surveys and a great deal of descriptive writing. They are of use mainly to outsiders.
In the past, there have been several problems associated with the production and use of baselines. They have: asked too many questions and often missed the more important questions; produced large reports which are difficult to interpret; required expenditure of a great deal of time and energy collecting information which has not been used until the final evaluation (ie: it is not an ongoing management tool); and not had the resulting information communicated to field staff and/or communities.
Participatory baselines, when the beneficiaries/community take the lead in describing what they perceive as important, can remedy many of the past problems and can enhance the baseline information that outsiders may require. With participatory baselines the information needed by the beneficiaries to describe themselves, their condition, and to measure change will be more realistic, more accessible, more easily interpreted, and more understandable because the information will have been chosen, obtained, and produced by the community. In addition, the community and the project will gain more information that can be incorporated into project implementation. Finally, results will not be filed in some distant project office, but will be readily available at the local level.
A participatory baseline may look very different from other baselines, but it can perform many of the same functions. It can include:
The results of community problem analyses and needs identification exercises as synthesized into a community case study or monograph.
A number of drawings by school children or a series of photographs.
The enactment of a drama, a story or a fable.
Farm maps by beneficiaries showing tree inventories and farm uses of these trees.
A collection of one page, issue specific, community interviews. For example: cookstoves, livestock feeding, fuelwood gathering practies, market surveys.
Participatory baselines can serve many purposes. They can:
The field worker, as facilitator, generates and provides methods and tools to help the community decide whether to do a baseline, what their information needs are at a given time, how to gather the information and how to analyse and present it.
The field worker, as a participatory baseline facilitator, might want to:
Discuss the objectives of the project in a group meeting. If the beneficiaries have not established their own objectives and negotiated between competing objectives, it may be useful to go through an exercise to establish community objectives. The purpose of the group meeting is to clarify common and conflicting objectives, so that it is clear to everybody what the beneficiaries and project together hope to achieve, and what the limitations are that they both face.
Discuss the different reasons for doing a baseline. What are the benefits? What kinds of inputs are required?
If the purpose of the baseline is to help the beneficiaries assess their basic needs, the Community Problem Analysis methods and tools may be useful.
If the purpose of the baseline is to obtain more exact information (either numbers or descriptive), it might be useful to discuss how this information will be used, and how precise it has to be. The section on identifying information needs for Evaluation Events may provide useful methods and tools.
If the purpose of the baseline is to help people understand current situations and current practices in order to build on them through the implementation phase, discussions of information needs might focus on:
(a) the current range of technologies and practices, and their outputs,
(b) the availability of inputs required by the new technology or practice that the project proposes,
(c) how these technologies could be effectively combined.
If the purpose of the baseline is to provide information on a range of different activities, not all of which can be identified at the beginning of a project, it may be possible to do focused, subject specific, smaller baselines over the course of project implementation. This may involve periodical production of "one page" semi-structured interviews; or a series of drawings focused on a specific subject.
If the purpose of the baseline is to establish a benchmark for later monitoring and evaluation, it may be useful to ask the two questions below and gather and synthesize information to establish current practices and/or attitudes.
"How will we know that the objectives (or activities) we have decided upon have been achieved?"
"What is the minimum amount of information that will tell us this?"
It is not easy to identify essential information. But, one of the lessons that has been learned is that focused time spent at this stage can be time well spent. The first, most frequent, and biggest mistake that is sometimes made is that too much irrelevant information is gathered. This creates problems in analysis, delivery of timely feedback, and presentation.
HOW ACCURATE SHOULD INFORMATION BE?
Chambers (1986) suggests that information does not always have to be statistically valid, but can be based on:
Optimal ignorance: this refers to the importance of knowing what it is not worth knowing. It takes courage to implement. It is easier to demand more and more information than it is to abstain from demanding it.
Appropriate imprecision: much data collected has a degree of accuracy which is unnecessary.
Source: Chambers (1986)
Use group discussions and available material (eg. previous studies and records) to help determine information that must be obtained. Often, a number of questions are used to evaluate an issue. As a result, excess information is obtained. By carefully selecting one or two questions through which the crucial information can be deduced, unnecessary information can be weeded-out.
It is important that the community/beneficiaries choose not only the questions but their own units of measurement and their own form of expression.
These key questions can sometimes serve as indicators, gauging the efficacy of a certain approach or conclusion. There are many different kinds of indicators, some of the more common are:
Indicators of relevance: These show the relevance or appropriateness of an activity. For example, for a cookstove programme, one indicator of relevance might be the concern that the proposed beneficiaries express regarding fuelwood scarcity.
Indicators of effort: These show how much and what is being invested in order to achieve the objectives. Such as, how long it takes how many people to plant what number of seedlings in a week.
Indicators of efficiency: These show whether resources and activities are being put to the best possible use to achieve objectives. For example, the differences between direct seeding and nursery seedlings when inputs and survival rates are considered.
Indicators of utilization: These show to what extent something that has been made available is being used for that purpose. For example, an indicator of utilization might be the number of people who have joined a forest-based handicraft marketing cooperative organized with the assistance of the project.
Indicators of coverage: These show what proportion of those who need something are receiving it. For example, an indicator of coverage might be the estimated number of people in the community who are currently producing forest-based handicrafts, and the number known to belong to the project sponsored marketing cooperative.
Indicators of quality: These show the quality or standard of something. For example, an indicator of the quality of tree planting demonstrations might be the mortality of outplanted seedlings due to poor quality planting.
Field workers and project staff may have, or be able to obtain kinds of information to which beneficiaries do not have access.
This could be information available in town libraries, from local government offices, or from town markets. The communities may need information on the current legal status of communal lands or the price fluctuations of forest products on the urban markets.
When the community requires outside information, field staff may have to interpret or translate the information so that it is usable in the community context.
Statistical reports from a local bureau of land management have to be presented in a clear and straightforward form.
Both the information gathering tools and the form of presentation should be appropriate to the beneficiaries and/or community. Look over the guidelines for choosing appropriate tools.
If both qualitative and quantitative information are required, two or more information gathering tools may be needed. However, the presentation may combine both kinds of information in order to provide both contextual and quantitative information. For example, a drama might be presented to describe the community situation, and the "actors" might present the numbers that were generated for the baseline.
Since baselines are mainly used as reference material, it is important to record them in a way that assures that they are safe and can be easily accessed at a later date.
For example, a project to assist wood carvers with the marketing of their high quality handicrafts (Ebony masks) could use a participatory baseline in the following way:
The participants (carvers) could decide that "success" would be an increase in the money they receive for their work. A collection of information that gives them the average current breakdown of the prices each party (from the producer to the final buyer) pays for the masks would be a the kind of baseline information to collect.
It would help them establish what their share of the final price was and if and how much it could realistically be expected to increase. It would also give them a beginning point from which to measure change.
The carvers might collect information among themselves to establish the average price they received for a mask when selling to different markets or middlemen. The field workers might collect information from the urban centres where masks were sold. They could also survey the wholesale dealers.
With this information, the carvers could create a "Baseline mask" such as that shown below:
With this information the carvers could also develop a strategy to increase their portion of the profits from the masks. This same
information, collected at a later date, could help them monitor whether their overall profits were increasing, decreasing or staying the same.
Participatory baselines are relevant, useful and creative. They can be done at any time throughout the project, whenever new information is needed. When the initial exercise is completed, the community and field staff can know:
1. How they are going to measure the progress and success of the activity.
2. Information which will give them a starting point against which to measure change in the future.
Participatory monitoring and ongoing evaluation (POE) is a method to record and periodically analyse the information that the community or the beneficiary has determined to be important.
Participatory monitoring (PM) is the recording of useful information to keep track of activities and/or progress towards objectives on a day-to-day, week-to-week, or season-to-season basis. Ongoing evaluation (oe) is a series of periodic "breaks" to analyse the monitored information in order to assess how things are going. Are activities being accomplished on time? Is progress towards achievement of objectives satisfactory?
The diagram below shows how POE fits into the overall PAME concept. The arrows indicate the feedback of information from POE to activities and objectives, and the linking of POE to the other methods of PAME (Community Problem Analysis, Participatory Baselines, and Evaluation Events).
To use an example, at point A, the ongoing evaluation revealed nothing unusual, things were going according to plan. However, at point B, ongoing evaluation revealed that there were problems. An evaluation event was conducted to get more information on the problems. As a result a decision was made to change some of the activities. These changes also meant a slight change in what was monitored.
The reasons why POE is done will vary, depending for the most part, on the activities of the project. But basically, PMoe provides information to help make decisions such as:
"Are we making satisfactory progress towards our goals?"
"Should we change our strategy or our activities?"
"Should we reassess our objectives?"
PMoe is done for any or all of the following purposes:
1. PMoe is directed and conducted beneficiaries (or the community) in order to systematica y record and analyse the information which have they determined to be important.
2. PMoe is systematic and consistent over the life of the project, rather than a "one-shot" information gathering exercise. This means that participants must have decided at the beginning of the project what is to be monitored, and how and when it will be recorded and processed.
3. PMoe is also flexible. If what is being monitored is not giving useful information, there is room for adjustment.
4. PMoe is locally relevant. The terms of measurement are chosen by the community. For example, wood might be measured in bundles or "guntas" rather than in kilos. The tools of recording information are also chosen by the community.
5. PMoe is based on key indicators which have been established by the community. Key indicators are selected by prioritizing a number of relevant questions each of which substitutes for a "bundle of questions" related to a specific issue (key indicators are described in Chapter Four).
6. PMoe must balance between monitoring and ongoing evaluation. The "correct" balance will vary according to a number of factors such as the seasonality of the activities. Some projects, such as small-scale forest based enterprises, have daily recordkeeping (monitoring) and monthly meetings where information is summarized and discussed (ongoing evaluation). Other efforts, for example, reforestation activities, may only require record keeping (monitoring) during the planting seasons, with an evaluation at the end of each planting season.
Participatory action research is the term used to describe the process by which the themes and issues of research are influenced, if not completely determined, by the local people themselves. Field staff assist the farmer in carrying out local testing while project activities are beginning. The results of this research are then, regularly integrated into on-going activities. Staff members can then identify relevant topics for further research, passing them on to research agencies.
PMoe complements participatory action research by providing the method and tools for information gathering and analysis of locally inspired research. Some tools that will be useful for participatory action research are described in Section 8.
REASONS FOR PARTICIPATORY ACTION RESEARCH
In Kenya, participatory action research was found to be useful for the following reasons:
1. Local experts could adapt new methods to local conditions better than outsiders.
2. Local control of change was more important than rapid transformation.
3. Building upon and branching from existing technologies was more easily accepted by farmers than introducing entirely new technologies.
4. Participation between insiders and outsiders implied dialogue to establish trust and shared goals.
5. Dialogue took place within communities as well as between the community and outside researchers.
6. Strategies for change were elicited from "successful" members of disadvantaged groups.
Source: Roucheleau (1986)
Measuring beneficiary participation in forestry development projects is something that has been of special interest to outsiders. It is often a required element for continuation of a project's funding. However, beneficiaries are not necessarily as concerned with the interests of every segment of the population as are outsiders who have "participation" as a project objective. It is important to be aware that community participation does not necessarily mean equal representation of all elements or groups in the community. Sometimes groups, who may have the most to gain or the most to lose, are underrepresented or not fully involved in the decision making process. Even if community participation is encouraged throughout the duration of the project it may be important to ensure, through monitoring, that all involved and effected groups have input.
If it is decided that it will be useful or necessary to monitor participation, beneficiaries should help define "participation", and help identify appropriate participation indicators.
Local people may wish to know who participates in communal activities, so that benefits can be more equitably distributed. The hours of individual participation may not, however, be the accurate measure of contribution. For example, one person may supply skilled labour for a short period of time and still be considered by others to have equally participated. Communities may choose to evaluate participation based on the skill and quality of the work, rather than labour/time spent. A carpenter's contribution of 2 hours can sometimes be considered the same as an unskilled labourer's contribution of 20 hours.
Because it is a process which emerges and develops as the project progresses, information needs regarding "participation" may undergo significant change over the life of the project.
For example, there may be an enthusiastic turnout at meetings during the initial "introductory" phase, but as groups or committees take over decision making, the community only has to be "kept informed". Therefore, the turnout at meetings may drop off, especially if things are going well. Increases and decreases in turnout at community meetings are not always an effective indicator of participation.
The following steps provide the method for PMoe. These steps are followed by the beneficiaries. The field staff should facilitate and help out by suggesting tools, when and if necessary.
1. Establish the PURPOSE of PMoe.
2. Establish WHAT will be monitored.
3. Establish HOW the variables will be monitored.
4. Establish WHO will monitor and conduct ongoing evaluation.
5. Establish WHEN the monitoring will be done. 6. Establish the TOOLS that will be used to monitor.
6. Establish WHO will want the information from POE and how it should be presented.
1. Establish the purpose of POE. Why is PMoe useful in this community? Why is POE useful for these activities?
2. If the decision is made that POE is going to be useful, the next thing to decide is what information to monitor.
There are many factors and variables that can be monitored on a community forestry project (both technical and socio-economic data may be needed). These will have to be sorted out and the key pieces of information (key indicators) determined. Key indicators are described in Section 4.4.
To establish key indicators, the beneficiaries can go over the activities that have been agreed upon. For each of these activities, the beneficiaries can "brainstorm" the kinds of information that would help them to know, on an ongoing basis, if activities were progressing as planned.
Information needs must be prioritized in order to establish key indicators, the relative importance of different type of information should be clear to all the participants.
3. After determining WHAT will be monitored (the key indicators), it is time to decide HOW items will be monitored.
The beneficiaries choose the terms of measurement. The terms will vary from country to country, and. perhaps even from community to community. Some will measure in "guntas", some in sacks, some in tins. Most communities will have an existing system of measurement. As long as this is consistent within a community, it can be translated later for information comparison between communities.
It is not always necessary (or appropriate) to have exact measurements. Orders of magnitude and directions of change are often all that is needed. What is important is that the key indicators that are monitored are those that are relevant to beneficiaries.
The socio-economic terms of measurement, if qualitative information is monitored, are also established by the beneficiaries. These may vary a great deal between communities.
4. The next step is to decide WHO will monitor. If the project includes the whole community (such as a community nursery) it may be decided to charge a minimal amount for some seedlings, and pay a nursery person to manage the nursery and keep the records. If the project involves farm forestry, individual farmers may keep their own records.
It is important that the beneficiaries have a clear idea of who is responsibility for different monitoring activities.
Consistency in monitoring information can be encouraged by good field staff follow-up during extension visits.
5. WHEN should the monitoring be done? Again, this will vary from community to community, and according to the. nature of the activities. After it is decided when monitoring will take place, the times for ongoing evaluation can be planned. Should it be seasonally? Monthly? Every six months?
Ongoing evaluation will necessitate an "adding up" of the information collected, and an analysis and presentation of that information to the people who make decisions. It may also mean conducting a discrete "one shot" annual information gathering exercise such as a survival survey.
Ongoing evaluation can be done by small groups, who have been assigned this responsibility. It is important to decide in advance who this group will be, what inputs they will need from field staff, when they will meet to synthesize, analyse and present "results" to the beneficiary group as a whole. Input from field staff may be required for ongoing evaluation, or for discrete processes of information monitoring such as survival surveys. Methods for analysis and communication of results are described in Section 7.
6. Monitoring tools should be selected primarily because they appropriately relate to the project's activities. However, cultural, social, economic and educational factors operating in the community should be taken into account. Tools must be commensurate with project and beneficiary resources (time, skills, materials).
Many of the tools that are described in Section 8 are specific to monitoring and ongoing evaluation:
Farmers' Own Records
Nursery Record Books Financial Accounts
Community Environmental Assessment Survival Surveys
Participatory Action Research Maps and Mapping
Other tools can be combined with these, or adapted to be useful monitoring devices. Creativity is encouraged when choosing tools to "fit" the situation.
7. The question of WHO needs and should receive the information can be answered during any of the above steps. Should the beneficiaries share their information with other communities with similar activities? Should the beneficiaries share the information with their own community, or among each other? How will this be done?
When POE is implemented it will give, throughout the life of the project:
- Key indicators which will monitor activities/objectives on a consistent basis;
- Tools with which the community can monitor;
- A planned period to formally analyse and discuss the information that has been gathered through monitoring;
- Information to guide the project. It will indicate whether the project should change, re-organize, re-think, delete an activity, or remain on the same course.