Walter Smith was born in 1842 at Ferry Port-on-Craig in Scotland, and served as an apprentice with an engineering company in Glasgow. After working with Neilson's for a short while, he joined the Edinburgh & Glasgow Railway when S.W. Johnson was Locomotive Superintendent. A lifelong friendship grew from this association, and would later influence locomotive design on both the North Eastern Railway (NER) and the Midland Railway. Smith moved with S.W. Johnson to the Great Eastern Railway (GER) in 1866, before leaving in 1874 to become the Locomotive, Carriage, and Wagon Superintendent for the Imperial Government Railways of Japan. As one of the first British engineers in Japan, Smith was responsible for setting up the workshops, machinery, and running sheds.

Smith returned to Britain in 1883 to join the NER. Initial duties included improvements to the NER's Gateshead Works, but Smith was quickly influencing NER locomotive design and he would eventually become the NER's Chief Draughtsman. This influence increased greatly in 1890 when T.W. Worsdell retired and was replaced by Wilson Worsdell as the Locomotive Superintendent.

Smith's most notable influences concerned the use of piston valves, and compounding. Although early NER attempts to use piston valves received mixed success, repeated testing and trials led to the NER being an early adopter of piston valves in Britain. Wilson Worsdell quickly discarded the two cylinder Worsdell-von Borries compound system used by his brother (T.W. Worsdell). Although simple expansion would be used on many of his express locomotives, he also adopted a three cylinder compound system developed by Smith. This was characterised by two low pressure cylinders and a high pressure cylinder in the middle. All three cylinders were a similar size, but the use of two low pressure cylinders resulted in a greater low pressure cylinder area. A valve allowed high pressure steam to enter the outer cylinders when extra power was required (e.g. when starting a heavy load). This system proved particularly successful, and it is probably fair to say it was the only successful compound system used in British locomotive design. Smith's friend, S.W. Johnson would famously adopt his compound system for use on the Midland, and it would continue to be used by the London Midland Scotland (LMS) railway.

Smith died in 1906 at the age of 64, whilst still in the employment of the NER.