What is 'vetting'?

By “vetting” we simply mean examining or checking. In other words, gathering information on an organization to determine their soundness. You’ll run into this word a lot when you browse the websites of meta-charities; organizations that do quality assessments on charitable organizations. Don’t let the very fancy jargon fool you though, the process is really quite simple. We try to explain the different steps of the vetting framework below.

Why we vet

At Kinder we empower people to instantly and effectively act on the global challenges of our time by facilitating donations to charitable organisations. The more effective the organisation performs, the more it will be promoted on our donation platforms.

How we vet

In collaboration with Impact Center Erasmus and Duke University our research team has developed a state-of-the-art vetting framework to assess the performance of charitable organisations. Here we explain the four steps of our framework in detail.

Step 1: Initial Screening

First off, we assess the transparency and accountability of charitable organisations. 

Why this step?

Accountability and transparency are two of our core values. Generally — and understandably — when a charitable organisation is asking for money, the donating party wants to know how the money is spent. They also want to know who to contact about their concerns, questions and suggestions. We are not suggesting that charitable organisations need to share all their internal documentation with the public. But there is a threshold to be met, before we at Kinder include an organisation in our donation tools. This is assessed as follows.

Method

In this first step we check the following five requirements: 

1. Donors should be able to easily contact the organisation to ask questions or voice concerns. So, the organisation needs to demonstrate openness to questions and have several communication channels in place.  

2. Donors should be able to find information on people that are accountable for the performance of the organisation. Therefore, information on board members and key staff should be easy to find on the organisation’s website.

3. Donors should be able to access the organisation’s financial statements so they can see how the organisation is funded and how the donations are spent. Therefore, charitable organisations need to publish their annual review in a timely manner.

4. Donors should be able to find the strategic plans of the organisation so they can see whether their money would be put to good use over time. As such, organisations need to make their plans for the future accessible to the public.

5. Donors want to avoid donating to charities that misuse raised funds. The charitable organisation should not be involved in any form of controversy — including attempts to convert people religiously or politically. In case of scandals, organisations need to clearly communicate how they were managed, and share the lessons learned.

While you might think we set the bar very low in this step, you’d be surprised how many organizations fail to meet one or more of these initial criteria. Of the first 1600 organizations currently vetted, only about 450 organizations met all five criteria as listed above. Therefore we decided for now to include those organisations that miss out on either financial statements or strategic planning.

How we will improve

We started with checking organisations for past controversy. However, such checks are only snapshots. The aid sector is always in flux, and while we may clear an organisation one day, the next day they could be complicit in a major scandal. For now, we crawl the web for controversy on a weekly basis. This process is partly automated, but still requires a lot of manual work in terms of data entry. We will design more efficient methods to do this, but we will never completely outsource human judgement.

Step 2: Organisational Competence

In this step we are looking to see if charitable organizations are particularly competent. We want to find organizations that know how to plan well for the future, monitor and evaluate their own programs closely, engage with research, and strive to act ethically and with integrity.

Why this step?

Organisational competence is an important pillar of performance. Organisations that are competent in terms of planning and execution of programs while interacting with research are more likely to getting things done effectively.

Method

We built an online vetting portal where trained vetters all over the globe can vet all possible charities you can imagine. Sounds too good to be true? You are correct. At this point the portal is not used by vetters all around the world, but by a core team of vetters in the heart of Amsterdam. Rest assured, we are working hard to make our open source “Vet-o-pedia” dreams come true!

How about those vetters then? Can anyone just log in and voice their opinion on everything? The short answer is no. The long answer is that before someone becomes a vetter, they have to do an initial assessment, and then, when they have been vetting for a while under supervision of a Senior Vetter they get to do a few benchmark vets. If these vets are within the acceptable limits, they can start as a Junior Vetter. As they continue vetting, and their performance remains within the acceptable limits, their trustworthiness will improve.

Are you interested in joining Kinder as a Vetter? Apply here, or have a look at our team page to learn more about our current vetters!

Our vetting portal is fully up and running. Vetters that log in can choose from a selection of four charitable organizations to vet. They vet according to specifically designed questions in four different thematic groups, namely: 'Strategy', 'Programs', 'Sector Research' and 'Ethics'.

How we will improve

When more research minded people enter our pool of vetters, the same organization can be vetted multiple times by different vetters. This will improve our accuracy in the established framework, but it will also show us which parts of the framework cause controversy. Questions vetters keep disagreeing on? Perhaps we should rephrase them, or scrap them completely. We will plug the vetting data back into our framework design and come up with different solutions to the problems we face.

Furthermore, we will perform regular noise audits where we select a random subset of charitable organizations to be vetted again. If the noise (difference between vets) is unacceptable, we will go back to the drawing board to improve our process!

Step 3: Intervention Effectiveness

Great! The charitable organizations have been scrutinized quite a bit. But we are not yet satisfied. The next part of the vetting process focuses on the effectiveness of interventions. 

Why this step?

Here we ask the crucial question: do the programs these organizations employ even work? And what about their cost-effectiveness compared to other interventions that tackle the same problem?

Naturally, this is a very tricky thing, and overstating our ability to deal with this knotty problem would be a big mistake. Thankfully, many organizations care about intervention effectiveness as well, such as GiveWell and 3IE impact, so we are not alone in our effort to gather information on effective interventions.

Method

Without getting into too much detail: Kinder asks charitable organisations to share evidence about their interventions (such as malaria bed nets or deworming pills), adds additional evidence and stores it all in an Evidence Database. We are most interested in interventions with rich quality evidence stating a positive effect. When organizations are added to our portal, the corresponding information on programs is cross-referenced with our Evidence Database. Any shining examples of effectiveness pop up? You will be sure to see them featured on our Kinder platforms!

How we will improve

The evidence database is updated and improved on a rolling basis. Is the report on a particular intervention rather shallow? Let’s push to go more in depth. Should a snippet of evidence surface online that contradicts our current analysis? We will take it into account. We aim to share our research as much as possible, so that we may be reviewed by our peers and enter a constructive dialogue.

Step 4: Cause Area Analysis

So... After completing the first three steps, we have hopefully landed on a few highly competent organizations that are very likely to have effective programs, right? What's left? We want to find the organizations that tackle the most urgent and important problems.
Again this is not an easy task. What is important or urgent to you, might not be of particular interest to, for instance, your neighbors: let alone a person on the other side of the world with a completely different set of values. We are aware of this. Kinder believes in letting everyone decide for themselves  about what they think is an important problem to solve. After all, we want you to give freely, unencumbered by the need to weigh a thousand different things to come to the best donation decision for you.

Why this step?

Having said that, we still feel that we are responsible to share the contextual information about cause areas candidly. So we will! Especially on a level of sector analysis. We will compare sectors within different cause areas. Therefore, if you are interested in animal welfare, we give you the deets on the relative performance of animal advocacy, compared to, for instance, wildlife conservation.

Method

Organizations like 80,000 Hours and Open Philanthropy Project do very comprehensive research on cause areas. It is important to note that generally speaking, organizations like the two mentioned above define cause areas in a narrow way.  Our cause areas list is meant to be inclusive, and not exclude cause areas that matter to many people based on our own fallible assumptions. We aim to do our bit by looking for gaps in the current data, and developing our own ideas about important, neglected and solvable problems.

How we will improve

Naturally, research is always in flux. We will monitor the cause area prioritization research that is being done globally and share important lessons on our platforms. We don’t claim 100% certainty, but we do attempt to claw our way out of the woods of misinformation, confusion and directionlessness. Just because there is no guarantee, does not mean we should not try, right?