# Minimum Description Length

**Information Theory Accomplishment **| 1978

**IBM researcher:**** **Jorma Rissanen

** Where the work was done: **Almaden Research Center

**What we accomplished**: Intuitively, the Minimum Description Length (MDL) principle suggests that more complex statistical models are less likely and therefore the empirical fitness of a candidate model on a sample has to be evaluated with respect to the complexity of the model. For example, a high-degree polynomial may fit a given finite set of points in the plane better than a linear function, but may not be useful for generalization because of overfitting. Rissanen's theory provides a very rigorous foundation for model selection based on the number of bits required to describe the model.

**Related links**: 1993 IEEE Richard W. Hamming Medal

1998 IEEE Information Theory Society Golden Jubilee Award for Technological Innovation

2006 The Kolmogorov Medal of the University of London

2009 IEEE Claude E. Shannon Award

1989 (Book): J. Rissanen, "Stochastic Complexity in Statistical Inquiry," World Scientific Publishing Company, 1989.

1978 - 1986: Selected Journal Articles

J. Rissanen, Modeling by shortest data description, Automatica 14 (1978) 465-471.

J. Rissanen, A Universal Prior for Integers and Estimation by Minimum Description Length, The Annals of Statistics 11 (1983) 416-431.

J. Rissanen, Stochastic Complexity and Modeling, The Annals of Statistics 14 (1986) 1080-1100.

See also: Algorithmic Information Theory

Image credit: Engineering and Technology History Wiki

BACK TO IBM RESEARCH ACCOMPLISHMENTS