The Gates Foundation paid $575 million to hurt public schools

The Bill and Melinda Gates Foundation is one of the best known charities in the world, not only because of the founder(s) names. It is also the richest private foundation in the world with 37 billion dollar capital.

The foundation aims to eliminate poverty and inequality and “Empower the Poorest”. In the US they mostly concentrate on public education improvement.

The foundation was often criticized for a “socially blind” investment strategy, including into stocks of companies that do a lot of the harm the foundation seeks to fight, or downright cooperating with those companies. The foundation also seems to promote products from companies it owns big amounts of stocks of.

The foundation itself makes a point of saying that they do what is most effective, based on extensive research and number crunching.

And that is the point where they wasted half a billion.

As I wrote above, in the US the foundation mainly invests into education. In their annual letter 2018, the Gates wrote:

Early on, we also supported efforts to transform low-performing schools into better ones. This is one of the toughest challenges in education. One thing we learned is that it’s extremely hard to transform low-performing schools; overall they didn’t perform as well as newly created schools. We also helped the education sector learn more about what makes a school highly effective. Strong leadership, proven instructional practices, a healthy school culture, and high expectations are all key.

This effort helped educators understand how to observe teachers, rate their performance fairly, and give them feedback they can act on. But we haven’t seen the large impact we had hoped for.

The Gates Foundation invested – in several years – hundreds of millions of dollar into evaluating and assessing what makes good schools good.

While this is undoubtedly a good goal, they may have fallen to the typical “big data” fallacy.

What do I mean? I mean the underlying assumption of big data proponents that the more data you have, the better the result will be.

Which is, to put it a bit bluntly, bullshit.

Just imagine the famous haystack: If you want to find a needle, does it help to put more haystacks on top of the first one?

Yes, you have (hopefully) more needles, but you also have more hay, and the pure size of the new heap may prevent you from doing the needed work.

The amount of data is far less important than the work you do with it. Or in other words, the questions you ask the data. To use the haystack: If you know the needle is magnetic and you have a big magnet, then you will have a way higher chance to succeed than if you don’t ask “Is that needle magnetic?”

Big Data proponents say that thanks to computers you can find every needle in even the biggest haystack, but this is wrong.
What you do find are more things that look like needles.
But you still need humans to interpret the results. A computer does not know if the correlation of the divorce rate in Maine with the consumption of margarine (1) is important, unimportant or a downright coincidence. A human may have a hunch.

But humans need a start. They need to have a point where they can start asking questions, preferably the right ones. And they need to be able to look at the results and understand them.
But the more data you have, the harder both activities get.

Equally important, as every particle physicist can tell you, just observing something can change the results.

This may have happened to the High School project the Gates Foundation did in the US that cost them $575 million.

The project gathered huge amounts of data. First of all the test scores. Then evaluations and observations of principals, students and parents.

Huh! Wow. Did you see that big, red flashing warning light?

I just wrote that it is important to ask the right questions. Putting aside the problem of asking this completely different set of peoples, do you get the main problem here?

Every single person asked here is heavily biased. And the Gates Foundation put their evaluation on the base of the data that was coming from those peoples, or in other words, the whole project is based on what those people ask.

By “ask” I mean the assumptions and bias they have. What is a good teacher?
A student may say one that gives good marks. A parent may say a teacher that upholds a strict discipline. A principal may say a good teacher it one that keeps the expenses in the budget. And so on.

To make it worse, except the students, every single person asked gets the majority of their information second-hand: from the students. I have yet to see a single parent that sits in the lessons for a few weeks for example.

And then there is the highly likely problem that factors that are independent from teachers influence their results.

There are a lot of possible reasons for this, especially in social sciences. Let me just think a moment about possible reasons why results can differ widely that are not related to actual teacher skill.

Every year schools get a bunch of new students. Even classes in one year are often very different from each other and sometimes it is just one single child that makes the big difference.

The ethnic background of the school’s students can be very different. A class of second generation Asien-immigrants will stomp every fourth generation European class in math, for example, because the parents have different expectations from their children.

The general state of the building may influence the students. My town just closed a school after students and teachers reported stinks and getting headaches. Turns out that after 50 years suddenly naphthalene is coming out of the floor.

Equally important are other school-externalities.
How long do the students have to travel to get to the school? When do lessons start? How noisy is it outside the school? How many of the students receive additional teaching? Are there different projects running that have an effect, maybe a parent has created learning circles that didn’t exist before?
Are there special motivational or influencing factors that are invisible to the observer and the teacher? Many a doctor’s careers where started by a sibling having a bad chronical illness and many writers got their passion started by a librarian who dropped the right book at the right time into the hands of a young library patron. (2)
Maybe the students are even subjected to poisoning by farmers that put pesticides on the fields on the other side of the school fence or their homes! (3)

A teacher that is super successful in one class may completely fail in a different one out of reasons that are out of his control or simply because he fits more into one type of class then another. Does this make him or her a bad teacher?

Another problem with the Gates Foundation approach was that the “results” were feeded back into schools, even going so far as to firing teachers because of it, likely creating “survivor bias”, among other problems like distrust from teachers towards the Foundation.

All that happened while the algorithms that were employed where kept a secret.

In a way that makes sense: You don’t want to teachers adjust to the “rules” to get a better assessment and such (partly) invalidating the study.
On the other hand: If knowledge of the rules leads to better results, then either you should distribute them as far as possible, or the rules do not measure teacher’s objective success but something else, whatever that may be!

All in all the Gates Foundation project has arguably done more bad than good and certainly costs a lot of money. Money that could have helped the schools in proven ways.

But honor where honor is due: At least the Gates Foundation published the results of the independent evaluation of their school project. If just they had done an evaluation of their methods before investing half a billion...

source and evaluation result

*1 more of this on spurious correlations

*2 The funniest story I know about writers is that of Terry Pratchett. As a (very) young boy he was an avid customer of a small store that sold “erotic literature”.
He was too young to appreciate the more steamy (picture) books on the shelves in the back, but he bought a lot of cheap SF from the front “excuse” stacks. Many a man who entered the shop, Pratchett observed, were leaving the store with a confused look on the face when seeing the young boy who was probably the only customer interested in the official merchandise.

*3 There are a lot of studies that found correlations especially about lower IQ and pesticides in pregnancy. Here is just one article from Yale

All pictures from pixabay.com.