FREE ELECTRONIC LIBRARY - Thesis, documentation, books

Pages:     | 1 | 2 ||

«The Development and Initial Validation of Competencies and Descriptors for Canadian Evaluation Practice Brigitte Maicher Net Results and Associates ...»

-- [ Page 3 ] --

Our initial goals, particularly with respect to the validation process, had been somewhat more ambitious than the resources could support. Context and environment determined much of what could be realized. Rather than implement the multistepped approach that was envisioned, we proceeded with a simplified methodology. Although it had been our intent to employ several data collection methods and analyses, in the end we did what was feasible. Initially, the selected experts were to be randomly assigned to three methods of data collection, one group for the questionnaire, one for interviews, and one for a Delphi study. In actuality our resources and volunteer fatigue allowed for only the survey. Even then, the number of survey questions taxed the respondents, and some reported that it took them over one hour to complete.

Considerable fluctuation in committee membership occurred during the 19-month process. Committee changes necessitated bringing everyone up to date repeatedly. Discussions on resources and debates that had already taken place were reopened. The core members worked to achieve a balance between efficiency and extensive open consultation in an effort to address the principles of inclusiveness and transparency. This lengthened the process.

doi: 10.3138/cjpe.29.3.54 CJPE 29.3, 54–69 © 2015 66 Maicher and Frank Doubts about some aspects of the taxonomy lingered among members of the

subcommittee. These limitations were discussed among the group:

• There may still be too many dimensions.

• There may be unnecessary overlap.

• Descriptors vary in appropriateness.

• Some descriptors seem applicable to senior evaluators and some to more junior evaluators.

• A definition section may be needed, particularly with reference to the competencies themselves and the Reflective Practice domain.

• There was some feeling that Reflective Practice refers to values and ethics rather than to competencies.

Irrespective of these challenges, every effort was made to achieve a common platform of practice for the evaluation community and the evaluation users. It is understood that the platform will need to be stabilized and built upon. Evaluation is a diverse field of knowledge with changing practices and theories that cover several disciplines. The present competencies and descriptors will, we hope, be refined and complemented in the future.


Building a framework for valid evaluator competencies presupposes certain knowledge, skills, and dispositions. We perceive validity as the single most important aspect of identified competencies. Validity helps ensure the competencies are actionable and serve their intended purposes. Judgements based on competencies should be useful to evaluation, and beneficial to the evaluator and to the evaluation community.

A precise description of what is meant by each competency helps ensure that expectations are clear. Descriptors increase the accuracy of the competencies and support judgements based on defensible criteria. Accuracy is particularly central, because in evaluation, as in other complex and variable systems, the quality of inferences is influenced by the precision of terms.

The Reflective Practice and Interpersonal domains are particularly infused with judgements drawn from competencies and descriptors. In those domains, ethics and values strongly affect behavioural outputs specific to contexts and programs but the linkage is more tenuous than in the technical domain. Thus, the guidance provided for ethics and values in particular must be seen as trustworthy. The ethical guidelines of the Canadian Evaluation Society and the Program Evaluation Standards from the Joint Committee on Standards for Educational Evaluation are critical and need to be regularly consulted by evaluators. Specificity of descriptors lessens individual interpretations and threats to their intended purpose.

Our hope is that the competencies and descriptors will fulfill all the purposes intended, helping evaluators to improve their evaluation skills, and guiding all © 2015 CJPE 29.3, 54–69 doi: 10.3138/cjpe.29.3.54 Developing Competencies and Descriptors 67 who seek to improve the quality of evaluations and the credibility of the field of evaluation. So far we have seen the competency document employed in the CES credentialing process and in new developments in evaluation education. We have also seen employers and agencies begin to orient their evaluation projects and hiring practices to this source. We are grateful to the many hands that contributed to the development of this important taxonomy over many years.


What can the future bring? We would like to see a systemic mechanism of review of competencies and descriptors—one that has a regular cycle and process. Potentially, working groups could be convened, one for each domain. These groups could each consist of two to three Credentialing Board members and Credentialed Evaluators and be organized through the Vice President. To ensure coherence and check for unnecessary repetition, one member from each working group could convene with the others to review all the recommendations together. Results could then be validated at a conference workshop or presentation and approved by National Council.

The field of evaluation may become a discipline with an elaboration of a theory that would encompass broad principles and at the same time reflect situationand context-specific parameters. It may be a theory of change or a theory of evaluation or both. The competencies and descriptors are the basic underpinnings meant to define evaluation competencies today; they will be influenced and modified by a theory as much as they will influence it. As competencies are refined and modified, further research to validate them is required. In this process, evaluation societies could expand their network of partnerships and collaborations and together produce research that brings the field of evaluation forward as a credible and essential part of all programs, policies, and initiatives.


American Evaluation Association. (2005). Guiding principles for evaluators. American Journal of Evaluation, 26(1), 5–7. http://dx.doi.org/10.1177/109821400502600101 Buchanan, H., & Kuji-Shikatani, K. (2014). Evaluator competencies: The Canadian experience. Canadian Journal of Program Evaluation, 28(3), 39–41.

Canadian Evaluation Society. (1999). Essential skills series. Retrieved from http://www.

evaluationcanada.ca Canadian Evaluation Society. (2007). An action plan for the CES with respect to professional standards for evaluators. Retrieved from http://www.evaluationcanada.ca/ professional-designations-project-archives Canadian Evaluation Society. (2008). Cross-walk of evaluator competencies. Retrieved from htttp://www.evaluationcanada.ca/txt/2_competencies_cdn_evaluation_practice.pdf Canadian Evaluation Society. (2010). CES competencies for Canadian evaluation practice.

Retrieved from http://evaluationcanada.ca/txt/2_competencies_cdn_evaluation_ practice.pdf doi: 10.3138/cjpe.29.3.54 CJPE 29.3, 54–69 © 2015 68 Maicher and Frank Canadian Evaluation Society Credentialing Sub-Committee. (2010). Competencies and descriptors. (Unpublished working document).

Eoyang, G. H., & Berkas, T. H. (1998). Evaluating performance in CAS. Retrieved from http://c.ymcdn.com/sites/www.plexusinstitute.org/resource/resmgr/files/evalincas.pdf Ghere, G., Stevahn, L., King, J., & Minnema, J. (2006). A professional development unit for reflecting on program evaluator competencies. American Journal of Evaluation, 27(1), 108–123. http://dx.doi.org/10.1177/1098214005284974 Gullickson, A. R., & Howard, B. B. (2009). The personnel evaluation standards: How to assess systems for evaluating educators. The Joint Committee on Standards for Educational Evaluation. Thousand Oaks, CA: Corwin Press Sage.

Huse, I., & McDavid, J. (2006). Literature review: Professionalization of evaluators. Victoria, BC: University of Victoria.

Joint Committee on Standards for Educational Evaluation. (1994). The program evaluation standards (2nd ed.). Thousand Oaks, CA: Sage.

King, J., Stevahn, L., Ghere, G., & Minnema, J. (2001). Toward a taxonomy of essential evaluator competencies. American Journal of Program Evaluation, 22(2), 229–247.

http://dx.doi.org/10.1177/109821400102200206 Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

Long, B. (2007). Minority report with respect to professional designations for evaluators.

Vancouver, BC: Canadian Evaluation Society.

Parry, B. (1996). Competency based education/training. Retrieved from http://ieeeltsc.files.

wordpress.com/2009/03/competency-paper-20070104-macallisterr.doc Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks: Sage.

Russ-Eft, D., Bober, M., de la Teja, I., Foxon, M., & Koszalka, T. (2008). Evaluator competencies: Standards for the practice of evaluation in organizations. San Francisco, CA:


Stevahn, L., King, J., Ghere, G., & Minnema, J. (2005). Establishing essential competencies for program evaluators. American Journal of Program Evaluation, 26(1), 43–59. http:// dx.doi.org/10.1177/1098214004273180 Wilcox, Y., & King, J. (2014). A professional grounding and history of the development and formal use of evaluation competencies. Canadian Journal of Program Evaluation, 20(3), 1–28.

Wordnet, Princeton University. (2014). A lexical database for English. Retrieved from http:// wordnetweb.princeton.edu/perl/webwn?s=validation.


Brigitte Maicher is principal of the firm Net Results and Associates. She has conducted evaluations as part of Net Results and through her employment with the New Brunswick provincial government since 1991, chiefly in health, including hospitals and community health centres, addiction services, health technology, Aboriginal programs, nursing homes © 2015 CJPE 29.3, 54–69 doi: 10.3138/cjpe.29.3.54 Developing Competencies and Descriptors 69 and, to a lesser extent, in education. She holds an MSc in Psychology and an MBA. She served as one of three members of the Core-committee of the Canadian Evaluation Society’s Professional Designation Program and chaired the Credentialing Committee.

Christine Frank is principal of the firm Christine Frank and Associates, conducting evaluations for organizations across several sectors including education, health, immigration/ settlement, international development, and social services. She holds a PhD in Education and was a core instructor in the postgraduate Research Analyst Program at Georgian College in Barrie from 2000 to 2008. She helped develop the Canadian Evaluation Society’s Professional Designation Program and has served as a member of its Credentialing Board since its inception.

–  –  –

Pages:     | 1 | 2 ||

Similar works:

«TEXTE Water Safety Texte 74 (Berlin, 28 – 30 April 2003) 03 Conference Abstracts ISSN 0722-186X UMWELTBUNDESAMT The publisher does not accept responsibility for the correctness, accuracy or completeness of the information, or for the observance of the private rights of third parties. The contents of this publication do not necessarily reflect the official opinions. Publisher: Federal Environmental Agency (Umweltbundesamt) Postfach 33 00 22 14191 Berlin Tel.: +49/30/8903-0 Telex: 183 756...»

«Hedge Fund Leverage∗ Andrew Ang† Columbia University and NBER Sergiy Gorovyy‡ Columbia University Gregory B. van Inwegen§ Ivy Asset Management This Version: 5 July 2010 JEL Classification: G11, G18, G23, G32 Keywords: capital structure, long-short positions, exposure, hedging, systemic risk ∗ We thank Suresh Sundaresan and Tano Santos and seminar participants at Columbia University for helpful comments. † Email: aa610@columbia.edu; WWW: http://www.columbia.edu/∼aa610. ‡ Email:...»

«International Journal of English Language and Linguistics Research Vol.2,No.3,pp. 13-28,December 2014 Published by European Centre for Research Training and Development UK (www.eajournals.org) ROLE OF VOCABULARY LEARNING STRATEGIES IN EFL LEARNERS’ WORD ATTRITION Thamer Alharthi Department of European Languages and Literature, King Abdulaziz University, Jeddah, Saudi Arabia Correspondence: P.O. BOX 80200, Jeddah 21589, Saudi Arabia. ABSTRACT: The use of vocabulary learning strategies (VLS)...»

«Read Now and Download Les Liaisons Dangereuses Book at Our Online Library. Get Les Liaisons Dangereuses PDF Book For FREE From Our Library Les Liaisons Dangereuses Book PDF #217251 Download Book Author: Pierre-Ambroise Choderlos de Laclos PDF File: Les Liaisons Dangereuses Book PDF Read Now and Download Les Liaisons Dangereuses Book at Our Online Library. Get Les Liaisons Dangereuses PDF Book For FREE From Our Library DOWNLOAD LES LIAISONS DANGEREUSES BOOK PDF #217251 BY: PIERRE-AMBROISE...»

«ACTA CHROMATOGRAPHICA, NO. 15, 2005 DEVELOPMENT AND VALIDATION OF AN HPTLC–DENSITOMETRY METHOD FOR ASSAY OF GLUCOSAMINE OF DIFFERENT FORMS IN DIETARY SUPPLEMENT TABLETS AND CAPSULES C. Sullivan and J. Sherma* Department of Chemistry, Lafayette College, Easton, PA 18042, USA SUMMARY A quantitative method using silica gel HPTLC plates, automated bandwise sample application, detection with ninhydrin chromogenic reagent solution, and automated visible mode densitometry has been developed for...»

«Reads book Information Technology Interview Questions With Answers at Online Library. Get file Information Technology Interview Questions With Answers PDF to free download INFORMATION TECHNOLOGY INTERVIEW QUESTIONS WITH ANSWERS PDF Complete data published is really a hard copy manual thats printed INFORMATION TECHNOLOGY INTERVIEW QUESTIONS WITH ANSWERS Document nicely bound, and functional. It operates as a reference manual skim the TOC or index, get the page, and stick to the directions detail...»

«Fundamental Components of the Gameplay Experience: Analysing Immersion Laura Ermi Hypermedia Laboratory 33014 University of Tampere Finland +358 3 215 8077 firstname.lastname@uta.fi Frans Mäyrä Hypermedia Laboratory 33014 University of Tampere Finland +358 3 215 7933 firstname.lastname@uta.fi ABSTRACT This paper presents a gameplay experience model, assesses its potential as a tool for research and presents some directions for future work. The presented model was born from observations among...»

«mietimmobilien mietimmobilien Mietwagen Preisvergleich Mietwagen vergleichen und sparen! Mietwagen vergleichen und sparen! Versicherung, km frei, ohne Storno! Wohnungen zur Miete Mietwohnung gesucht Mietwohnung gesucht Hier kostenlose Angebote finden! Mietimmobilien | immonet.de Mietwohnungen, Apartments mehr in Ihrer Nähe. Sofort online suchen! Mietimmobilien Günstige Wohnungen in Ihrer Nähe! Günstige Wohnungen in Ihrer Nähe! Hier werden Sie fündig. Mietimmobilien Halli Hallo, ich habe...»

«The Musician’s Résumé Handbook BOB BORDEN KATHY IVY Office of Careers and Professional Development Eastman School of Music University of Rochester Rochester, New York Table of Contents What is a Résumé? Introduction Where to Begin Brainstorming Format Types and Lengths of Résumés Selling Yourself Active Verbs Reproduction of Résumé The Finished Product How and When to Use your Résumé Example Résumés Bibliography -2WHAT IS A RÉSUMÉ? A résumé is a formal overview of your past...»

«hm denim hm denim Denim | mytheresa.com Die Must-Have Jeans der S/S Saison jetzt Online shoppen! HM Denim Jeans gr 152 schmal mamikreisel.de HM Denim Jeans gr 152 schmal Bund verstellbar #jeans #hm #kinder #hose #cool #swag #trend #denim #retro #hipster #aktuell #musthave Versand 2.50 HM Denim Slim Jeans gr.152 mamikreisel.de HM Denim Slim Jeans gr.152 Bund ist verstellbar #hm #hose #jeans #Denim #jungen #röhre #squinny #slim #schmal Versand 2.50 HM DENIM on Pinterest | Hm, Slim Legs HM Denim...»

«Ss Never Eat Alone And Other Secrets to Success, One Relationship at a Time by Keith Ferrazzi Ss Keith Ferrazzi is the quintessential networker. He has, he says, 5,000 contacts in his PDA who will answer the phone when he calls. And he has never been to one of those networking events specifically set up for the purpose of making contacts, nor is he, in his own words, a networking jerk, though he once was. The distinction between his approach and the formal approach taken by many others is the...»

«Georgia Standards of Excellence Curriculum Frameworks Mathematics GSE First Grade Unit 1: Creating Routines Using Data These materials are for nonprofit educational purposes only. Any other use may constitute copyright infringement. Georgia Department of Education Georgia Standards of Excellence Framework GSE Creating Routines Using Data • Unit 1 Unit 1: Creating Routines Using Data TABLE OF CONTENTS (*indicates new addition) Classroom video available here:...»

<<  HOME   |    CONTACTS
2016 www.thesis.xlibx.info - Thesis, documentation, books

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.