What we learned from undertaking a cost-effectiveness modelling of our SMS Reminder Programme
1. Why we updated our cost-effectiveness model
Earlier this year, we posted our under-immunisation analysis, which explored immunisation coverage in India. Our main finding was that the remaining gap in immunisation is very geographically concentrated: roughly 15 percent of the country’s districts account for close to half the country’s under-immunised children. We noted that it is important to factor in the scale of need and cost-effectiveness of operating in a geography when determining the geographic location of our programmes. This post explores the cost-effectiveness implications of selecting different geographies for one of our programmes, namely, SMS reminders.
As a reminder for the reader: Suvita’s flagship programme is SMS reminders to parents and caregivers, where we prompt them to complete their children’s routine immunisation schedule. We currently operate this programme in four districts of Bihar state and across all of Maharashtra state, reaching around 1.4M parents and caregivers every year. For this programme, our learnings from the under-immunisation analysis generated two decision-relevant questions (which we will refer to as ‘Set 1’):
Set 1
Within a state, should we operate across the whole state or in the districts in that state with the greatest need?
Should our SMS Reminder Programme expand into new states, and if so, which one(s)?
While the under-immunisation analysis helped us determine where in the country children are missing vaccines, it did not tell us what it would cost to reach the under-immunised children, or how much of an impact reaching them would make. Both vary considerably across geographies and are important for answering the questions above. The cost of enrolling a child depends on how birth records are held and whether they have to be digitised by hand, while the benefit of reaching a child depends on local disease burden and baseline coverage. A cost-effectiveness analysis (CEA) estimates how much "good” a programme achieves per unit of money spent. This enables us to compare our programme across geographies or with another intervention. Suvita is funded by GiveWell, an evaluator that directs funds to programmes that clear a cost-effectiveness bar, set as a multiple of a benchmark roughly the value of giving cash directly to people in poverty.[1] That bar is not fixed. We understand it moves because of several factors, including how much funding GiveWell expects to have, and has stood at six times the benchmark as of May 2026.[2] We applied the GiveWell approach to cost-effectiveness analysis, which sharpened the above two questions for us into a ‘Set 2’ list:
Set 2
District Targeting Question: Would operating in specific districts raise cost-effectiveness enough to justify the operational cost of targeting?
State Expansion Question: Would expanding to a new state clear GiveWell’s current bar of cost-effectiveness?
Luckily for us, we did not have to build a cost-effectiveness model from scratch. GiveWell had built a CEA for our SMS Reminder Programme as part of a grant assessment with Suvita in 2023, when their bar for funding interventions stood at ten times the benchmark. The 2023 model was designed to evaluate a single grant rather than to compare states in which we have never operated, and now, three years later, we have three additional years of cost data and new evidence.
We therefore refreshed the inputs, rebuilt parts of the calculation, and requested Rethink Priorities to independently review parts of the model we were most uncertain about. Our 2026 model is available here.
Before sharing the results, one point to note: We realise the most useful output of this exercise was not the final cost-effectiveness estimates themselves, but a clearer understanding by Suvita’s internal team of how much analysis a geographic prioritisation decision of this kind actually requires. We learned that a lighter version would have pointed us in the same direction on the geographic prioritisation questions we set out to answer. Sections 3 and 4 set out where additional depth in our analysis helped, and where it did not.
2. What the CEA told us
Most of all, the analysis gave us a clear sense of which factors drive the cost-effectiveness of our SMS Reminder Programme, and which do not. For instance, we had set out to compare states on need, disease burden, and cost. Among the states we shortlisted, all of which have large numbers of under-vaccinated children, the results suggested that the choice of state matters considerably less than whether we can obtain digital birth records once we are working there. The latter impacts costs the most. We explain this further in the sections below. Before sharing the results, it is also worth noting that where we actually decide to work in the future will depend on more than the model. That decision also rests on considerations such as what its government is already doing, and whether our team could operate there.
We now take in turn our District Targeting Question and our State Expansion Question.
2.1 Key lessons in answering our District Targeting Question
A major takeaway from the CEA was that it makes most sense to deliver SMS reminders to targeted districts when enrolling children in Suvita’s programme in that state is expensive. Because enrollment cost is different in the two states that Suvita currently works in, the strength of the case for targeting specific districts in each state is different.
At first glance, the model suggests that targeting districts with lower immunisation coverage is the more cost-effective option. While this is broadly in line with our expectation, there is an important nuance: targeting districts only becomes more cost-effective if costs are reduced by a proportionate amount to the number of children reached. A substantial part of what it costs to run our SMS Reminder Programme does not vary with the number of children enrolled, while the cost of sending reminders to one additional child is small.
Maharashtra state is a good illustration of this. The state government records births digitally and shares the information we need to enrol caregivers, which keeps the costs of enrolment to roughly $0.27 per child. Restricting the programme to selected districts in the state would reduce the number of children we reach without reducing our fixed costs to a comparable degree. It would also introduce costs of its own: identifying the appropriate districts, ensuring our district prioritisation is aligned with those of the government, building the systems required to enrol selectively, and maintaining them. On these estimates, continuing to operate across the whole state is more cost-effective.
The situation in Bihar state is different. Birth records in Bihar are held on paper and have to be manually digitised before a caregiver can be enrolled, which raises the cost of enrolment to roughly $0.66 per child, more than twice the figure in Maharashtra. Where each additional child costs that much to enrol, concentrating on the districts where the programme would have the highest health impacts could be a sensible approach from a cost-effectiveness standpoint.
We are currently in the process of determining what this implies for Bihar: Do we expand within the state, remain in the four districts we currently cover, or prioritise differently is a programmatic decision the team is working through.
Overall, the analysis indicates that finding solutions to reduce the cost of enrolling a child would improve cost-effectiveness in Bihar more than any other programmatic change available to us.
More generally, the cost-effectiveness case for targeting districts depends less on where under-vaccinated children live and more on how costs respond when fewer children are served.
2.2 Key lessons in answering our State Expansion Question
The main takeaway on expansion from the CEA was that it makes most sense where birth records can be accessed cheaply, and that this matters more than which state is chosen. The details are below.
Our under-immunisation analysis identified six states that together account for around two-thirds of India's under-immunised children, two of which are states where Suvita already delivers programmes. We modelled the remaining four to better understand the case for Suvita to extend our programmes to those states, namely Uttar Pradesh, Rajasthan, Madhya Pradesh and Gujarat.
We modelled each state under two cost scenarios. The first scenario assumes we can access digitised birth records, as we do in Maharashtra, applying a cost per enrolled child similar to Maharashtra's. The second assumes the records have to be digitised by hand, as in Bihar, applying a cost similar to Bihar's.
Among these, Uttar Pradesh demonstrated the most promise, provided we can obtain birth records there at low cost and the set-up costs are not too high.
Each row in the chart is one state. The dark dot is our estimate under the first scenario, where digitised birth records are available. The light dot is the same state under the second, where the records have to be digitised by hand, and the line between the two shows how far the estimate falls when we have to pay for that. The two vertical lines mark GiveWell's bar: the solid line is the bar as it stands today, at six times the benchmark, and the dashed line is the bar of ten times that applied when the 2023 model was built.
Start with the dark dots on the right. All four states landed above the current bar under the first scenario, and they sit close together, between 11.3 and 13.2 times GiveWell’s benchmark. Uttar Pradesh has the highest estimate of the four states, but the gap between it and Gujarat, the lowest, is small. On cost-effectiveness alone, there is little to choose between these states. What distinguishes Uttar Pradesh is the scale of need rather than the estimate: it has the largest number of under-vaccinated children of any state in the country, a little over 20 percent of the national total.
Now follow the lines left to the light dots. Only one of them, Uttar Pradesh, at 6.5 times the benchmark, still sits to the right of the solid line. Madhya Pradesh and Gujarat fall below it. On these estimates, expanding into either of those two states while paying to digitise records by hand would not be justified if we were to use GiveWell’s current cost-effectiveness bar, and Uttar Pradesh lands above the bar with little room to spare.
The State Expansion Question therefore depended to a large degree on the availability of digitised birth registration data within them. Whether Uttar Pradesh is a strong opportunity or a marginal one depends almost entirely on whether we can obtain birth records from the state at scale. We will be investigating this further over the coming months by exploring availability and access to robust data, including date of birth, address, and phone numbers.
One consideration the state comparison leaves out is the fixed cost of entering a new state at all: establishing a relationship with the state government and setting up operations. We have not modelled those costs and expect them to be material in the initial stage.
3. What we learned from the process
In some ways, the model confirmed what we already believed about our SMS Reminder Programme. In others, it surprised us. Below we share both.
3.1 What the analysis confirmed
Importance of cost-per-enrollee. We expected cost-per-enrollee to be a dominant lever, and it is. What we learned from the modeling was a better understanding of the why. A large share of what it costs to run our SMS Reminder Programme in a state, including staff time, systems, and the work of building and maintaining a relationship with the state government, does not vary much with the number of children we reach. Once those foundations are in place, reaching one additional child costs very little, and cost-effectiveness improves as we enrol more children in a state we already work in; cost of enrolment therefore determines whether a state clears the bar.
Importance of access to government data. We also expected access to government data to be materially significant to costs, but we didn’t realise just how crucial this was. The difference between accessing birth records that are already digitised versus having to digitise them by hand is large enough as a single factor to move an otherwise promising state from above our benchmark to below it. As a result, understanding how a state maintains its birth records and whether we can access them has become an explicit part of how we evaluate entering a new geography.
3.2 What surprised us
Targeting identified districts in states is not an obvious choice. We had assumed that concentrating on the districts with the greatest need in a state would improve overall cost-effectiveness. For most of our current operations, it would not, for the reasons explained above.
Our model is based on a fair degree of judgement: Some Suvita runs more than one programme, and resources are shared between them, so subjective judgments must be made about how to assign resources and costs to different programmes: how a field officer’s salary should be divided between programmes when a given week contains work on both, or how much of an M&E specialist’s time belongs to which state. We made considered choices, but a different set of reasonable choices would have produced somewhat different numbers on cost-effectiveness.
India-specific evidence is thin. The effect size in our model comes from GiveWell’s 2024 review, which pooled sixteen randomised trials and, after discounting for internal and external validity, landed on the effect of SMS reminders at a 15 percent reduction in the unvaccinated population. Reviewing our model, Rethink Priorities questioned whether the external validity discount was large enough for India. Most of the trials were conducted in sub-Saharan Africa, where not knowing about vaccination or not remembering when a dose is due are among the main reasons children are missed, and a reminder addresses those barriers directly. Our internal study found that demand-side barriers still account for roughly 65 percent of reasons, but that their composition is shifting with fear of side effects and awareness gaps becoming increasingly prominent. A text message has limited impact on those barriers, and neither of the two India-specific studies in this literature found a clear effect from SMS reminders as a standalone programme.
Rethink Priorities suggested the discount should be at least twice as large, while being explicit that they knew of no principled way to derive the right figure. As a result, we doubled it, bringing the effect size we use down to 11 percent. The input doing the most work in our model is therefore one that our reviewer and we arrived at by judgement, and better India-specific evidence would improve our estimates more than any further work on the model itself.
We had much to learn about our own programme. A cost-effectiveness analysis requires us to make explicit every step between spending money and generating impact. For Suvita, that meant setting out how government birth records are collected, how many of them we enrol, how many of those enrolled actually receive our reminders, how those reminders affect vaccination uptake, and how additional vaccinations translate into health impact. Several of these steps look straightforward until you have to demonstrate and explain exactly how one leads to the next. Two of the steps there required quantification: the proportion of eligible children we actually enrol, and the proportion of enrolled children who actually receive our messages. Answering those questions meant going back to our own monitoring data and several consultations with our programme teams. The answers we gathered turned out to differ substantially between Bihar and Maharashtra. A strong understanding of where our assumptions were weakest has been one of the more useful outputs of conducting the cost-effectiveness analysis.
4. What we would do differently next time
Most of the information and data we needed for decision-making was available without us building a detailed model. Our costs per enrolled child came from our own accounts, and the number of under-vaccinated children by district came from the internal assessment we had already completed. When we put them together with a rough effect size, we have an approximate ranking of states and the observation that the cost of obtaining birth records dominates everything else.
Our arrival at an answer to the District Targeting Question was less straightforward, but some time spent understanding our cost breakdown would have helped us realise earlier that most of our costs do not vary with the number of children enrolled. We now believe that a few hours spent on building a basic CEA model would have pointed us in the same direction on both the District Targeting Question and State Expansion Question we set out with. The full exercise took a couple of weeks.
What a full model added was an absolute number; in our case, that number is how many times the GiveWell benchmark our programme sits at, rather than a ranking. A rough calculation would have told us that Uttar Pradesh looks more promising than Gujarat, but it wouldn't have told us whether either clears a specific bar, which was a question we did want to answer.[3]
Building the full model is how we learned which parts matter the most. For future questions of this kind, we would start with a rough, back-of-the-envelope calculation, and add detail only where a better estimate could plausibly change our decision.
Finally, we welcome any feedback on our cost-effectiveness analysis and subsequent interpretations, and would be eager to hear further inputs.
Footnotes
[1] GiveWell's benchmark was originally the cost-effectiveness of GiveDirectly's unconditional cash transfers. In 2024 GiveWell revised its estimate of cash transfers upwards by a factor of three to four but retained the original figure as its benchmark, so the benchmark is best understood as GiveWell's pre-2024 estimate of cash transfers in Kenya rather than as current cash programmes.
[2] GiveWell lowered the bar from eight times the benchmark to six times in early 2026, a change it estimated would result in around $90 million in additional grants that year. The change in the benchmark seems to follow the funding available: Coefficient Giving, formerly Open Philanthropy, increased its 2026 allocation to GiveWell's recommendations from USD175 M to 1 Billion in 7 months, partly based on growing expectations of future funds.
[3] The bar moved while we were modeling, which limits how much that precision was worth.
