Case study: the Fragile Families Challenge and predictive models

Part of Agathon’s vision is “doing Good work … that makes the world better.” Of course, that looks different across the variety of clients we serve. We build websites and apps for ministries and companies changing the way we do healthcare. Part of our business is also hosting professional WordPress bloggers as they share information on a variety of topics. And we work with sociologists to build and maintain the platforms for their research.

Recently, we saw one such project come to fruition with the publication of the Fragile Families Challenge results.

We’ve been working with Matt Salganik from Princeton since 2010. This has included supporting his work with the All Our Ideas platform as well as the Open Review Toolkit.

In 2015, Matt referred one of his colleagues, Marta Tienda, to us for help with a project known as mDiary. This study sent biweekly surveys to a group of participants from the Fragile Families and Child Wellbeing Study about things like their familial and romantic relationships, substance use, school experience, and more. This research has been tracking individuals from birth, and they’re now teenagers. We worked with Marta and her team to build and support a mobile-friendly survey tool to support their research in a way that protected the privacy of participants and incentivized them to participate.

The Fragile Families Challenge

In 2017, Matt came to us with his own project related to the 15-year Fragile Families study: the Fragile Families Challenge.

In this case, Matt and his team didn’t need custom software development but a simple website to highlight the challenge. The goal of this challenge was to provide teams of computer scientists, statisticians, and computational sociologists with anonymized “big data” from the study. They were challenged to use that data to predict outcomes for other participants whose data was held back.

Our team was able to support Matt and his team in their work. We quickly spun up a new logo, branding, and website for the challenge and provide ongoing hosting for the site.

Applications to participate in the challenge opened in February of 2015. Over the course of 7 months, 150+ teams from 68 institutions in 7 countries participated. Recently, the results of that mass collaboration were published. Despite the diversity of researchers and methods, the results showed a lack of accuracy in using machine-learning to predict outcomes.

While the challenge didn’t result in a clear way to predict outcomes, knowing that “having a large amount of data and having complicated machine learning does not guarantee accurate prediction”1 is still an important conclusion. It turns out this is an area of research where there are still many problems to solve.

Helping you do good work

Good work takes a lot of different forms across different spheres, industries, and niches. We’d love to hear more about the Good work you’re doing. Contact us today to chat about your digital presence and how we can help you with strategy, design, and development!

Footnotes

  1. Matt Salganik, https://www.technologyreview.com/2020/04/02/998478/ai-machine-learning-social-outcome-prediction-study/

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.