Go to the Wall (To).
To be pushed on one side, laid on the shelf, passed by. Business men, and those in a hurry, leave the wall-side of a pavement to women, children, and loungers.
· 
· 
Entry taken from Dictionary of Phrase and Fable, edited by the Rev. E. Cobham Brewer, LL.D. and revised in 1895.