Re-thinking How We Average (A New Approach to Stein's Paradox)

Dr. Maya Gupta
Researcher, Google
Given on: January 10, 2013


We consider one of the simplest and most common estimation problems: estimating the mean of a random variable from IID samples. In practice, one often wants to estimate many means for many random variables, for example estimating energy consumption for all the households on a block. In a 1956 result called Stein's paradox, Stein showed that when multiple mean estimations are needed, one can do better than the sample averages by combining data from un-related tasks. Similarly, we propose and analyze a new multi-task regularization approach to produce better mean estimates that is provably better than taking the independent sample averages with very mild assumptions on the underlying distributions. Experiments on real data and simulations show that significant practical gains are possible compared to sample averages and James-Stein estimation. We discuss open questions, including whether the proposed multi-task averaging is useful for algorithms that take multiple averages, such as k-means, wavelets, kernel density estimation, etc.


Gupta joined Google Research in 2012. Before Google, Gupta was an Associate Professor of Electrical Engineering at the University of Washington (2003-2012). In 2007, Gupta received the PECASE award from Pres. George Bush for her work in classifying uncertain signals, and the 2007 Office of Naval Research YIP Award. Her Ph.D. in Electrical Engineering is from Stanford University (2003), where she was a National Science Foundation Graduate Fellow. Gupta has also worked for Ricoh Research, NATO (NURC), HP R&D, AT&T Labs, and Microsoft, and runs Artifact Puzzles.