Listening to the roosters crow behind me, the growing buzz of traffic drowning them out. Sitting in the dining area of the Wisma PGI (delightfully pronounced “piggy,” much to my merriment. Combined with the Wisma, it can come out as “Where’s my piggy” at times).
First day is over and we’ve covered a lot of ground. The participants of this evaluation workshop are here to assess the quality of their evaluation methods for a program called the AHRTP – Annual Human Rights Training Program, or PHAMT to use the bahasa Indonesia acronym.
We spent the first part of the day addressing basic evaluation questions. I split the dozen participants into three groups and each group answered two questions from the list below:
1. Why do we evaluate?
2. Who evaluates? (and who is evaluated?)
3. What is evaluated?
4. Where is it evaluated?
5. When is it evaluated?
6. How is it evaluated?
I naively put 1 hour for the small group work in order for them to answer the questions; it took more than the morning to complete the activity. Call me naive again, but asking these basic questions when developing an evaluation strategy for a program (whether it be related to human rights education or not) makes a whole lot of sense. Get these answered in the beginning and your evaluation will be effective and purposeful, not just an annoying add-on at the end of a program.
The other major part of the day was to have participants map the various types of evaluation tools at their disposal for the program. They focused their attention on four distinct types of evaluation: 1) needs assessment, 2) formative evaluation, 3) summative evaluation, and 4) impact and transfer. They collectively came up with over a dozen evaluation tools, and were in the process of analyzing one tool per group to determine whether or not the tool reflected certain key characteristics of good evaluations.
As we begin the day today we’ll be looking at the specific evaluation tools used during the 2-week AHRTP and see how to improve them. Improving the tools is not simply a matter of fine-tuning the questions. It’s making sure the right questions are asked, asking only the questions that are necessary, making sure the questions contribute towards the overall evaluation strategy, making sure the questions establish valid “baseline data” of the participants’ skills and knowledge, making sure the questions are answerable, and (this one’s just as important as all the others) making sure program organizers have the resources and time to properly input and analyze all the data.
I perused some completed questionnaires from the last AHRTP held in February and came across some useful (and not so useful) comments (bear in mind these have been translated). My comments in italics after each response:
- This 2-week training has enabled us to sharpen the analysis of the participants and to strengthen their capacity and knowledge. Thanks, but what does this mean?
- The food was not really good in terms of taste and variation. Oh I can relate to that.
- Through this training, I have shifted my basic perception on advocacy, since I had only applied national regulations under advocacy documents all this time, but now there are many international instruments that can be used in advocacy strategy. Now that’s encouraging!
- The education process was very limited due to insufficient time allowance. A valid concern. Time management (usually a gripe that there is insufficient time) is probably the most frequent comment I used to get from participants in workshops I facilitated. I’m gradually figuring out to leave more time for activities, give participants more space, and essentially reduce the amount of activities participants do.
Well we’ll see what the day brings. It’s a committed group, the kind that effortlessly continues past the end of the workshop day to continue working.
One more remark unrelated to the training. Have you ever noticed the stunningly comical array of signs identifying toilets for both sexes? The signs for “Gent” and “Ladies” here at the Piggy brought a smile to my face. Not only do I find it unfathomable for women to wear what looks like an uncomfortable triangular miniskirt, but both sexes make use of the same bow shape: a bowtie for the Gent and a cutesy bow in the Lady’s hair.Time for breakfast.
paul mcadams - a change is coming 