On building stuff at AWS re:Invent 2014

December 22, 2014 Civis Analytics

Our founder, Dan Wagner, was recently featured at AWS re:Invent in a Founders Fireside Chat with Amazon CTO, Dr. Werner Vogels. Watch the full video, or read an excerpt below:

Werner Vogels: “So do you actually build stuff?”

Dan Wagner: “We do!”

Werner Vogels: “So tell us the kinds of things that you build?”

Dan Wagner: “We left the campaign immediately. The lights went off, the curtains came down and we were 15 kids clutching our Macs walking down the street. And we began to get introduced to these companies that had a generic set of problems: an […] IT implementation over here […] and the decision making apparatus of that company over here, powered by spreadsheets and conditional formatting. There was a vast distance between these two things.

As we began to get introduced to these companies, they said, “well, how do we apply the same type of expertise at our company?” A college president approached me to say, “We are dealing with 40% attrition of kids that are dropping out. How can we figure out who these kids are ahead of time, build a predictive model that might say who they are, and then build intervention programs?”

The exercise in terms of the data goes beyond just a prediction: you have to pull all the data into one central place, you have to clean it, you have to impute for missing values, you have to match all the databases using record linkage from different spots, you have to run a linear and non-linear algorithm to predict who might drop out in the future, validate it on data, score the data set, provide it back to the university, run the intervention and see what what happens. That is software.

But you can’t do a lot of that stuff with the on-premise IT available at big organizations. We began by implementing some basic ETL software, pulling the data into Redshift, aggregating the data and running basic linear models. Over time, we’ve added functionality to our platform and made it available to these companies.”

Written by: Caroline Grey

The post On building stuff at AWS re:Invent 2014 appeared first on Civis Analytics.

Previous Article
Civis or the Academy?
Civis or the Academy?

People at Civis come from a variety of backgrounds, and several of us left academia to work here. Myself, I...

Next Article
The Medicaid Counterfactual: How we ran the simulation
The Medicaid Counterfactual: How we ran the simulation

Last week, after the first article about our work on predicting the uninsured went out the Upshot posed an ...