We present a historical review of Einstein's 1917 paper 'Cosmological Considerations in the General Theory of Relativity' to mark the centenary of a key work that set the foundations of modern cosmology. We find that the paper followed as a natural next step after Einstein's development of the general theory of relativity and that the work offers many insights into his thoughts on relativity, astronomy and cosmology. Our review includes a description of the observational and theoretical background to the paper; a paragraph-by-paragraph guided tour of the work; a discussion of Einstein's views of issues such as the relativity of inertia, the curvature of space and the cosmological constant. Particular attention is paid to little-known aspects of the paper such as Einstein's failure to test his model against observation, his failure to consider the stability of the model and a mathematical oversight concerning his interpretation of the role of the cosmological constant. We recall the response of theorists and astronomers to Einstein's cosmology in the context of the alternate models of the universe proposed by Willem de Sitter, Alexander Friedman and Georges Lemaitre. Finally, we describe the relevance of the Einstein World in today's 'emergent' cosmologies.