Your wits are gone wool-gathering. As children sent to gather wool from hedges are absent for a trivial purpose, so persons in a “brown study” are absent-minded to no good purpose.
“But, my dear, if my wits are somewhat wool-gathering and unsettled, my heart is as true as a star.”—Harriet B. Stowe.