«Comparative Study and ‘Outcome and Impact’ Analysis of Six Vocational Training Projects in West Africa Synthesis report based on six case ...»
Berufsbildung | Evaluierung
winners in their families, which has an impact on poverty reduction at the family level, although these impacts are difficult to quantify. In cases where a substantial number of persons from one community has been trained and earn income poverty reduction at community level becomes feasible. In most cases it is assumed that the indirect impact of VT on poverty reduction is marginal, considering that the majority of graduates traced 1-5 years after graduation is still financially depending on their families. The indirect impact of VT on poverty alleviation is strongly influenced by external factors such as the overall economic situation. 24% of the self employed graduates are employing others, thus contributing to skills development in their communities. Taking the total sample of all graduates interviewed it is estimated that on average approx. 2% are employing others, which is a small minority.
With regard to achievement of project objectives it is concluded that the skills learned improve the livelihood perspectives of the majority of project beneficiaries. The aim of making youth self-reliant is possibly “half-way” achieved, as the majority of graduates are depending on family support in the first years after graduation.
As for the comparison of the three approaches it is concluded that:
With regard to effectiveness none of the three approaches has a clear advantage over others; the most decisive is the labour market orientation and the quality of the training provided.
The cost efficiency and the short-term cost benefit are higher for non-formal and apprenticeship training compared to formal vocational education and training.
Formal centre based VT offers graduates more prospects for continuous technical education but graduates have no real advantage over other competitors once they directly enter the largely informal labour market. If labour markets develop (e.g. in emerging economies) formal VT may become a condition for employment, but this is not yet the case in the countries studied, where the vast majority of the working population works in agriculture and the informal sector. Education as an entry criterion, the long duration of courses and the costs of training are excluding factors for the poor. Formal TVET is the most costly training model.
Non-formal VT is better accessible for the poor, for youth who has no or inadequate access to formal education and for married women who want to learn a new skill for income generation. It often lacks quality as there are no external control mechanisms and no benchmarks set for quality standards. New “competency based” TVET systems, as possibly implemented in the region in the long term, may certify skills acquired non-formally and informally and may create a link between non-formal and formal vocational training.
Improved apprenticeship schemes use community resources and are flexible by nature.
If well managed they can be an effective and low cost alternative to centre based training. They are especially suitable for: (1) providing marginalised groups with access to low cost skills development, and (2) for skills development in rural areas. They are less suitable for modern trades which require a more systematic process of teaching and learning and may be more beneficial for men than for women, unless projects place more emphasis on personality and life skill development of female trainees.
Cooperative training models which combine elements of centre based training with apprenticeship schemes may have the highest potential for employment, as cases from other Berufsbildung | Evaluierung contexts show. This requires from the training providers to proactively seek collaboration with the private sector, which is lacking in all programmes observed.
Key recommendations Before making strategic recommendations the results of this study need to be discussed with partners first.
Partners should decide on which target groups in which contexts to focus. If VT is to emphasise on poverty reduction non-formal VT and improved apprenticeship schemes may be more appropriate. Formal technical and vocational education should probably remain a domain of government funded institutions, unless government institutions do not have the capacities to provide adequate service. NGO and church based training providers depending on foreign donors should primarily target groups who have no access to mainstream training offers.
Low cost- community based training solutions are needed to increase outreach of VT to neglected rural communities. Training models such as apprenticeship training and nonformal courses organised in communities using existing facilities can also be applied by integrated rural development programmes.
Promotion of employment of females needs more than good quality training. Partners need to survey markets and identify employment and income opportunities beyond the stereotype set of trades offered to females at the moment. More emphasis is required for life skill and personality development, the quality of these sessions (e.g. use of participatory methods) needs to be improved. Social counselling should be gender specific. Advocacy is needed to address labour access and inequality issues.
To increase effectiveness and impact VT programmes need improvement in:
The labour market orientation of training programmes: staff need to be trained in conducting labour market surveys and in curriculum revision/ modification Vocational counselling and guidance Improvement of practical orientation of centre based training: attachments to small and medium enterprise of at least 3 months per training year should become a standard ´ Focus on self-employment: i.e. by better integrating gender specific life skill and entrepreneurship training Intensifying post training support with the aim to enhance graduate’s prospects for wage and self employment, priority should be given to female graduates as they face greater entry barriers into the labour market and possibly to vulnerable male youth depending on context Increasing the number and capacity of female staff in teaching and management Partners need to improve quality management of training and should intensify (or implement) measures of regular outcome monitoring. Tracer studies should become a management tool for steering projects towards the desired outcomes.
VT institutions and projects should take measures to better cooperate with community structures and other organisations providing complementary services.
1.1 Background, objectives and focus of the study The EED evaluation unit, in cooperation with the regional desks, regularly commissions studies on cross cutting issues which are of strategic importance. The topics may refer to a certain region (group of countries) as well as to certain sectors. The aim of those studies is to find answers to key questions that have come up in these projects in the passed years of cooperation and to give orientation for the coming cooperation – for EED as well as for the partner organisations. In some of the studies – like in this one – there is a special focus on finding out about the ‘outcome and impact’ with the purpose of learning about best practices.
Vocational training and employment promotion of youth is one of the programme priorities of EED in West Africa. EED has been funding vocational skills training initiatives (in short called “VT”) in the region for several decades. Although VT projects have been evaluated and consulted on an individual basis from time to time, there has not been a comprehensive study which, in a broader sense, assessed the effectiveness, impact and relevance of the different VT approaches in their specific contexts. This study is meant to fill the gap.
Main purpose of this study is to assess the effectiveness of different approaches of vocational training in their respective contexts. The study gives special attention to assessing the employment and income situation of VT graduates and the impacts on families and communities. It further analyses quality and the efficiency of the programmes/ projects.
Special attention is given to analysing the relevance of the VT approaches and methods in each context and to drawing lessons learned for improvement and good practice.
(for TOR see Annex 1)
1.2 Scope of the study
The comparative study covered four countries: Sierra Leone, Liberia, Ghana and Nigeria, and six EED partners. For each partner, a detailed case study report was produced by the consultants.
The VT approaches studied can be broadly categorised as follows:
• Formal, school-based vocational education with courses of rather long duration
• Non-formal centre based vocational skills training
• Improved traditional apprenticeship schemes This report summarises the key results of the six case studies and elaborates in more detail on
the following guiding questions:
1. What happened to the graduates (males/ females) of the vocational training courses? Are there any recognisable trends on how many of them (males/ females) find jobs or regular income through self employment?
Berufsbildung | Evaluierung
2. What are the factors that have a key influence on the results of vocational training? Which approaches are adequate for which context and circumstances?
3. What can be learned from the indirect effects – intended or unintended - of the vocational training measures and how can the lessons learnt become guiding for future projects?
It further elaborates on the relevance of the approaches, the key criteria for success and provides recommendations for improvement and good practice.
1.3 Methodological approach and overview of activities
In a first step, partners were asked to provide essential “pre-study information” on:
• criteria partners use to select trades and courses
• demand for VT
• the project/ programme outputs (number and field of people trained)
• the costs of training, and
• tracer data (employment status of graduates 6-12 months after graduation) A questionnaire was developed with inputs of the team of consultants and distributed to each partner. The review of the data received, of narrative reports and project evaluations showed that most of the six EED partners did not have valid information on the employment status of their graduates. The partners that were engaged in regular post training follow up did not collect information in a systematic manner, e.g. analysis was lacking to what extent a person employed works in the field of training or not. Thus it was decided to organise the study in
1. Tracer studies, to be conducted by each EED partner
2. Compilation of tracer data, formulation of hypothesis and discussion of arising issues
3. Main study, conducted by a team of international and regional/ local consultants together with the respective partner organisations.
Purpose of the tracer study was to establish a resource base for the main study on programme quality, effectiveness and impact. The main study was meant to verify the tracer study results with partners and target groups and to elaborate more in detail on project/ programme quality, impacts and relevance, using the expertise of international and regional/ local consultants.
1.3.1 Methodology and scope of the tracer study
The tracer studies were supposed to broadly cover three key areas:
the quality of VT measures: the degree of satisfaction with the training provided the effectiveness or direct outcomes: the utilisation of skills learned by graduates for income generation, the employment situation of graduates, change in the income situation and the quality of employment elements of impact, e.g. the creation of jobs by graduates The tracer studies were conducted from November to December 2008 (Liberia and Ghana) and January to February 2009 (Nigeria and Sierra Leone). Coordination was the task of FAKT;
the entire process took place over the distance (follow up largely by using mobile phones).
Tools for data collection and documentation A sample interview guide, a matrix for data compilation and a guide for conducting the tracer study was designed by FAKT together with other consultants. The draft was shared with one of the partners (VTF). The idea of conducting a pilot phase for testing and reviewing the tools was dropped because of the tight schedule for the tracer study. Initially, it was envisaged to finalise data collection and analysis till the end of December 2008. Delays, especially because of the security situation in Nigeria, and delays in communication with partners resulted in the postponement of the entire study.
It was planned to use Microsoft Excel or internet based IT systems for data compilation and IT based analysis. Because of the limited know how of some partners in using IT software it was decided to use simple WORD formats which could be filled even by hand, with the limitation According to the pre-study information there are 885 graduates; the case study report says 952 graduates.
Berufsbildung | Evaluierung that cross-analysis of the raw data after the initial compilation by partners/ local consultants was no longer possible.
Selection of respondents The method for selection of respondents was supposed to be a combination of random sampling and purposive sampling using three categories: year of graduation, gender and trades. The study was supposed to cover graduates from 2003 to 2007.
In practice, systematic random sampling was hardly possible. The main challenge was the tracing itself. Vocational training graduates, especially males, are mobile in their search for job and income opportunities. The other challenge was the lack of a reliable information base in most projects. Only those partners who carry out regular tracing studies and follow ups (two out of six) did have a data base with names and contacts of graduates, the others did not. Even for those who did, contacts had changed in many cases. Thus, respondents were finally chosen according to their availability.