Civis R&D Bookshelf: How Neural Nets Work and When You Might Not Need Them

November 10, 2017 Keith I.

This post is part of our Bookshelf series organized by the Data Science R&D department at Civis Analytics. In this series, Civis data scientists share links to interesting software tools, blog posts, scientific articles, and other things that they have read about recently, along with a little commentary about why these things are worth checking out. Are you reading anything interesting? We’d love to hear from you on Twitter.

A Visual Primer on the Math Behind Deep Learning

I’m a big fan of the edu-tainment universe on YouTube. Channels like SciShow, standupmaths, Vsauce, and Numberphile all have great content that is both scientifically rigorous and fun. 3Blue1Brown, in particular, has created a number of math-related videos with quirky, helpful animations. Recently, 3Blue1Brown released a series on neural nets that is a wonderful introduction to the mathematics of backpropagation. I’d also highly recommend the channel’s Essence of Linear Algebra series.

Imitating word2vec with Retro Natural Language Processing

While deep learning is a game changer for natural language problems, I find in my day-to-day work that earlier statistical methods are still very useful. Case in point: the blog post from stitchfix above describes a way to combine standard pieces of text analysis (PMI, SVD, word co-occurrence) into a model that works roughly as well as the neural net–based word2vec model. Neural nets may perform better on very large datasets, but for most datasets PMI + SVD is just faster to train and has fewer parameters to worry about.

Designing Data-Intensive Applications

A key, but often overlooked, component of data science is putting your awesome, 16-layer deep neural net into production. This means efficiently moving data to and from storage and compute environments, which is much easier said than done. This book gives a whirlwind tour of the tools and concepts of data wrangling in applications. From OLAP and OLTP to MapReduce to message brokers, this book is both engaging and comprehensive. It even has Tolkien-esque maps!

The post Civis R&D Bookshelf: How Neural Nets Work and When You Might Not Need Them appeared first on Civis Analytics.

Previous Article
Civis R&D Bookshelf: Project Management Edition
Civis R&D Bookshelf: Project Management Edition

This post is part of our Bookshelf series organized by the Data Science R&D department at Civis Analytics. ...

Next Article
Grid / Batch / Secure / Cloud / Fun / Powerful Computing with Civis Platform
Grid / Batch / Secure / Cloud / Fun / Powerful Computing with Civis Platform

We’ve all been there. You are running some compute-heavy code on your laptop at work. Now, your computer is...