Drummond was born and educated in Edinburgh but emigrated to Canada in 1854. He became the technical manager and later a partner in his brother-in-law's sugar refinery business, John Redpath & Son of Montreal. After 1872 he was active in Canadian politics.
He was also a keen art collector and amassed one of the most important private collections of European art in Canada at the time. He served as president of the Art Association of Montreal from 1896 to 1899. A significant part of his collection was sold nine years after his death at Christie's on 26 June 1919.