Hmm I'd say I do give them plenty to dry. Washed them yesterday around 2pm and didn't sleep till 3am-ish. Like you I want to get it done early in the day. Can't say I've never slept with them wet. I probably have slept a few times when they were moist in the past but I've stopped that right away.
Oh that's exactly what I did yesterday, driving with the windows down! Best blow dryer
And an update on the Rot Know product: It did away with the funky smell and buildup