Decolonizing evaluation

Updated: Jun 8, 2021

Note: Our team is continuing to evolve our language around how we use the word "decolonizing" in reference to our evaluation practice. Please read our reflections on this blog (link) from March 2021. Based on these reflections, we believe the following blog would be more accurately named something like 'Identifying and addressing white supremacy in evaluation'.

A lot of our work happens in relationship with philanthropic partners, so we have invested a lot of time reading and considering new visions for and critiques of philanthropy in books like Winners Take All, Decolonizing Wealth, or publications like Leading with 100 Year Vision. When Edgar Villanueva referred to the Dismantling Racism workbook in his book Decolonizing Wealth, it got me thinking about how colonized mindsets show up in evaluation. Specifically, Villanueva writes, “In their Dismantling Racism workbook, Kenneth Jones and Tema Okun identified other characteristics of white supremacy culture, including perfectionism, sense of urgency, defensiveness, quantity over quality, worship of the written word, paternalism, either/or thinking, fear of open conflict, individualism, worship of unlimited growth, objectivity, and avoidance over discomfort.” These characteristics seemed familiar in the context of my experiences around evaluation. I thought of many ways we as evaluators are (un)consciously operating under these norms to the detriment of our evaluations with our partners, and I started to think through some strategies to consciously and proactively dismantle tendencies towards white supremacy culture. Below is our team’s initial thinking about what a colonized mindset versus a decolonized mindset looks like in evaluation.

Colonized Decolonized Evaluation application Either/or thinking Both/and thinking Move beyond themes and trends. Explore where there are tensions and contradictions in the data Use evaluation for learning and growth instead of as a pass/fail report Fear of open conflict Conflict as healthy Set up time for biweekly feedback using tools like The Management Center’s 2x2 feedback Regularly gather partner feedback during the evaluation Perfectionism High quality Regular brief reports and check-ins as opposed to long end-of-project reports Reframe mistakes as opportunities for learning Quantity over quality Quality over quantity Use a mixed methods evaluation approach to gather qualitative and quantitative data Expand collection beyond vanity metrics (e.g. number of participants) Build in time in budget to pivot evaluation based on community feedback Progress is bigger, more Progress is more just, increased wellbeing Consider positive evaluation outcomes that don’t only demonstrate growth Objectivity Strong objectivity Have multiple individuals analyze the same data set Actively discuss how bias plays a role in evaluation Involve participants, staff and community members in data interpretation Right to comfort Engage in discomfort Practice feeling uncomfortable. Use the “Risk/Learning Zone” as a model for individual and organizational growth Worship of the written word Communicate impact in multiple mediums Move away from lengthy written reports Include more visual presentations and reports with images and diagrams Expand data collection beyond surveys Sense of urgency Go slow to go fast Design realistic work plans. Distinguish between realistic short-term and long-term outcomes Invest in relationship building with the community voice at the outset

Other: Use systematic processes to collect data that is most relevant and useful, and that acknowledges the value of alternative data collection methods and sources. Read more about how statisticians are cautioning against strong conclusions drawn from statistical significance. Our team will continue to share how we’re working across some of these pieces to dismantle a colonized mindset in evaluation. We plan to dive in more deeply through a series of blog posts. As your team uses these tools and investigates these issues, we want to hear from you so that we can continue to deepen and improve our own ability to implement successful and decolonized evaluation.

Decolonizing evaluation

Recent Posts

Comments