Werner Vogels: “So do you actually build stuff?”
Dan Wagner: “We do!”
Werner Vogels: “So tell us the kinds of things that you build?”
Dan Wagner: “We left the campaign immediately. The lights went off, the curtains came down and we were 15 kids clutching our Macs walking down the street. And we began to get introduced to these companies that had a generic set of problems: an […] IT implementation over here […] and the decision making apparatus of that company over here, powered by spreadsheets and conditional formatting. There was a vast distance between these two things.
As we began to get introduced to these companies, they said, “well, how do we apply the same type of expertise at our company?” A college president approached me to say, “We are dealing with 40% attrition of kids that are dropping out. How can we figure out who these kids are ahead of time, build a predictive model that might say who they are, and then build intervention programs?”
The exercise in terms of the data goes beyond just a prediction: you have to pull all the data into one central place, you have to clean it, you have to impute for missing values, you have to match all the databases using record linkage from different spots, you have to run a linear and non-linear algorithm to predict who might drop out in the future, validate it on data, score the data set, provide it back to the university, run the intervention and see what what happens. That is software.
But you can’t do a lot of that stuff with the on-premise IT available at big organizations. We began by implementing some basic ETL software, pulling the data into Redshift, aggregating the data and running basic linear models. Over time, we’ve added functionality to our platform and made it available to these companies.”
Written by: Caroline Grey