properly means “sprinklings” or “scatterings.” Its present meaning is base insinuations or slanders.
“No sweet aspersion [rain] shall the heavens let fall
To make this contract grow.”
Casting aspersions on one, i.e. sprinkling with calumnies, slandering or insinuating misconduct.
“I defy all the world to cast a just aspersion on my character.”—Fielding: Tom Jones.