«Comparative Study and ‘Outcome and Impact’ Analysis of Six Vocational Training Projects in West Africa Synthesis report based on six case ...»
- Does the project/ program reach its objectives?
- Do the people who graduated from courses find adequate (self-) employment? What kind of employment? (quantitative analysis)
- Do males and females benefit differently and how? (quantitative analysis)
- Do graduates use their knowledge for generating income? Has the level of income increased?
- Which are the furthering / hindering factors for the generation of income after the completion of training?
- Do the communities of the trainees directly benefit from the project/ program?
- Are there any unexpected effects?
- Is the PME system sufficiently developed for assessing outcome?
- Are the costs of the project/ program appropriate compared to the benefits?
- Does cost consciousness play a part when choosing the approach?
- Is the duration of the courses appropriate /justifiable in the light of its benefits?
- Is the quantity of the trainings offered corresponding with the demands?
- What is the cost of a training measure per person?
- How much is the training fee compared to the overall cost of the course?
- When compared to other initiatives working with a similar approach are the costs higher or lower?
3.5 Selection and reaching of target groups
- Who has access to the training program?
- What are the criteria for the selection of participants (a.o. gender, age, refugee status)?
- Does the training reach the poorest or – for some reasons – the less poor?
- Are handicapped people and people faced with HIV/Aids infection specially considered?
- How does the project deal with participants getting pregnant during the duration of the training?
- Do “do-no-harm” considerations play a part in the selection of the target group?
- Do communities participate in the development of selection criteria?
- Which other groups have a say in defining the criteria of selection?
- Does the project/ program make use of contacts with the local economy?
- Does the project/ program systematically analyse the labour market and does it make use of that information?
- Do the curricula adhere to the national Vocational Education system?
- Are there interfaces to other services in the country such as microfinance schemes, employment schemes, job placement?
- Are women and men equally represented among the trainers? Does gender balance with the staff selection form quality criteria?
- How are the trainers qualified (technically, pedagogically, business wise)?
- Is the benefit for the target groups sustainable?
- Do other people but the participants benefit indirectly from the project/ program?
- Are there – positive or negative – changes with regard to gender relations due to the vocational training activities?
- Are there changes towards peace and reconciliation due to the vocational training activities?
- What other changes can be observed in the environment (economy, society) of the project/ program and in how far can they be attributed to the vocational training activities?
- Has the project/ program contributed to poverty reduction in the region?
- Has the project/ program enhanced gender justice in its environment?
- Is the PME system of the partner organisation sufficiently developed for assessing impact?
4 Comparative analysis of the approaches
How do the three main approaches to vocational training – formal, non-formal, and improved traditional apprenticeship – differ from each other with regard to
- demand from trainees
- satisfaction of trainees
- drop out rates
- effectiveness (in terms of employment and income generation) Berufsbildung | Evaluierung
- efficiency (cost-benefit-ratio)
- long-term change?
Which conclusions can be drawn from this?
- Drafting of ToR and receiving feedback from EED colleagues and partners
- Application for the financing of the study within EED September 2008
- After approval: Team meeting in EED for discussion of the ToR and adjustments if needed
- Contracting the evaluators October 2008
- Desk studies
- Identification of need for tracer studies
- Inception Report
- Starting of tracer studies and self evaluation (pre- assessments) with the partner organisations November 2008
- Tracer studies and self evaluation with the partner organisations December 2008
- Analysis of the results of the pre-assessments
- Building of hypotheses for the study
- Coordination meeting of the evaluation team January 2009
- Studies at the project places with involvement of the partner organisations February 2009
- Individual reports; receiving feedback from partner organisations involved
- Acceptance of the reports by EED evaluation unit March 2009
- Submission of synthesis report to EED
The evaluators are expected to produce the following:
Inception Report / Design of the Evaluation • The inception report (2 to 4 pages) reflects on how the evaluation team intends to fulfil the ToR. It informs about the methodology, working plan and division of tasks within the team. It points out problems and risks also that might jeopardize the achievement of the objectives of the study. In case the inception report suggests changes to the ToR they need prior consent by the EED evaluation unit.
Individual Reports • Individual reports should be submitted to EED for each partner organisation. Language should be English. They should cover the items 3.1 to 3.7 of the ToR and comprise approx. 20 pages. Before finalization the reports will be sent to the partner organisations for comments and consent.
Synthesis Report • The final report synthesizes the results of the individual reports and puts them in the context of common key topics. Focus areas of the report are: comparison of the different approaches to Vocational Training and ‘outcome and impact’ (‘Wirkungen’ in German).
The report should be submitted in English and not exceed 40 to 50 pages; reference documents should be put as annexes. The evaluators are asked to put a structure of contents, a list of abbreviations, a summary (approx. four pages) that contains the most important findings and recommendations and can be used as a separate document.
The report may be structured along the questions of the ToR but also differ from it. However, the results of the study should comprise the complete set of questions in the ToR. The evaluators are requested to value the results and draw conclusions from them. The recommendations should be based on those conclusions.
Presentation and finalization of the synthesis report • At the draft stage the synthesis report (basic findings, conclusions and recommendations) is submitted to a technical audience in EED for feedback and learning purposes. After that an in depth discussion of the report takes place with the involved staff in EED. On the basis of the comments and suggestions the report will be finalized and submitted to EED.
7 Profile of the evaluators
• Experience from evaluations in similar contexts
• Experience from consultancies
• Gender competence
• Special knowledge of Vocational Training
• Special knowledge on outcome and impact assessment
• Understanding for the development work of the Churches and Church based organisations Berufsbildung | Evaluierung Annex 2 Study Itinerary
Annex 3 Resources and internet links
Six case studies of the comparative study:
- Liberia Opportunities Industrialisation Centres Sinje and Gbanga
- Sierra Leone Opportunities Industrialisation Centres Bo and Mattru Jong
- Opportunities Industrialisation Centre Ghana, Livelihood Enhancement Project for Youth in the Kumasi Metropolis
- Vocational Training Programme for Females, Ghana
- Mason Technical School, Garkida, Nigeria
- Youth and Women Development Association for Social Transformation (YOWDAST), Ganye, Nigeria Promoting livelihood and employment in post conflict situations, FAKT/EED, 2007.
Skills Development in Sub-Saharan Africa, World Bank, 2004.
ADEA, Technical and vocational skills development in Africa, Workshop paper 2008 http://www.adeanet.org/adeaPortal/adea/Biennale%202008/Documentation/Papers%20for%2 0presentation/02.%20Session%202/Final%20PDF%20documents/Session%202%20Doc%204 %20OECD%20AfDB%20ENG.pdf OECD Economic Development in Africa, 2008 http://www.oecd.org/dataoecd/14/35/40573969.pdf Manual Tracer Studies, FAKT 2001 http://www.fakt-consult.de OIC International with information of its affiliates LOIC, SLOIC and OICG http://www.oicinternational.org/index.php?page=liberia-oic UN World Population Prospects http://esa.un.org/unpp/ Berufsbildung | Evaluierung
Republic of Sierra Leone Population Census:
http://www.statistics.sl/2004%20Pop.%20&%20Hou.%20Census%20Analytical%20Reports/2 004%20Census%20Report%20on%20Population%20Size%20Age%20and%20Sex%20Structu re.pdf http://www.statistics.sl/2004%20Pop.%20&%20Hou.%20Census%20Analytical%20Reports/2 004%20Population%20and%20Housing%20Census%20Report%20on%20Education%20and %20Literacy.pdf Berufsbildung | Evaluierung Evangelischer Entwicklungsdienst e.V. (EED) Ulrich-von-Hassell-Str. 76 53123 Bonn Telefon: +49 (0)228 8101-0 E-Mail: email@example.com www.eed.de