The web contains a treasure trove of design data, with many web pages being the product of careful thought about layout, font, and color scheme. Not only does the current web document current design trends, historical snapshots of the web are a lens into past fashions. The Internet Archive cite{internetarchive} has captured snapshots of the public Internet each year going back to 1996. In this paper, we present a curated dataset of 21 years of web design, scraped from the Internet Archive. We report initial analysis of design trends apparent in this data, and we demonstrate how the data can be modeled with deep neural networks to enable novel design applications, such as predicting the apparent year of a web design. The novelty of our work is two-fold: (1) mining the long-term temporal evolution of designs on the Internet, and (2) using deep neural networks as a tool for discovering design elements, which can complement the hand-curated features so far used in data-driven design mining.