![](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="48" height="48"><rect fill-opacity="0"/></svg>)
![](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="73" height="73"><rect fill-opacity="0"/></svg>)
A few days ago, I shared the World Bank’s summary of the results of the PISA for Schools assessment conducted in 171 Salvadoran public schools participating in our education pilot.
Since then, I have seen some legitimate questions about what these results mean, as well as some claims about what they supposedly don’t mean.
So let’s be very precise.
First, what we NEVER claimed:
1. We never claimed these results represent the national average.
They don’t.
This was not the national PISA assessment.
It was PISA for Schools, conducted specifically in the 171 public schools participating in our pilot program.
That is precisely why my original post said:
“These are the results of our pilot program, which includes AI tutors and has been running in 171 public schools for just over a year.”
A pilot is, by definition, not the entire country.
2. We never claimed that these 171 schools were a statistically representative sample of every school in El Salvador.
They weren’t selected randomly from all Salvadoran schools.
They were selected to participate in the pilot because they were schools where the program could initially be implemented more easily: they had certain basic infrastructure and connectivity requirements, and school communities capable and willing to implement a new educational model.
That creates a selection effect, and it should be acknowledged.
But there is an important distinction here.
These schools were NOT selected because their students had exceptional academic results.
They were not our “best schools.” Not even close.
We didn’t search the country for the highest-performing schools or students and then test them.
They are normal Salvadoran public schools, urban and rural, in different parts of the country, serving ordinary Salvadoran families.
And the characteristics that made implementation easier: connectivity, infrastructure, trained teachers and willingness to adopt the program, are also things that can be replicated in other schools.
They are not innate characteristics of the students.
We can install connectivity.
We can improve infrastructure.
We can train teachers.
We can provide the same technology.
We can deploy the same educational tools.
In fact, that is exactly what we are doing.
3. These results also do NOT prove that AI tutors, by themselves, caused the improvement, although we believe they are a MAJOR part of these results.
Our pilot is broader than that.
It includes changes in teaching, support for teachers and principals, educational materials, monitoring and evaluation, technology, and AI tools, among other interventions.
Our AI tutors and learning platforms are being developed with , , and support from I Hate Flying Bugs.
And the broader transformation of our education system is being supported by the , , and .
All of these interventions can also affect student motivation and engagement.
But that distinction matters: if students become more motivated BECAUSE of the program, that is not necessarily a flaw in the program or an explanation that somehow invalidates the results.
It may be one of the ways the program works.
The same applies to teachers.
For example, the implementation of the AI tutors, including the requirement for teachers to log in and out of the system, coincided with teacher attendance in these schools increasing from 76% to 97%.
If the program causes teachers to attend school more consistently, and that in turn helps students learn more, that is not an advantage that disappears when we scale the program.
It is potentially part of the intervention itself.
Of course, to scientifically isolate the causal effect of AI tutors, teacher attendance, student motivation, technology, training, or any other individual component would require a different experimental design.
Now, what DO these results tell us?
1. They tell us that 1,198 students in 171 ordinary Salvadoran public schools… 🧵1/2👇🏼
Quote
vittorio
@IterIntellectus
honestly, i keep thinking about this result and it's just absurd.
in the 2022 PISA, el salvador scored 343 in math, 365 in reading, and 373 in science
near the bottom of PISA, close to guatemala and dominican republic
now the country has developed a program for education x.com/nayibbukele/st…
