h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

I cycled a lot when I lived in East Anglia - big open largely flat spaces with minimal hills. But the killer, when I lived on the Suffolk coast for a blissful and lovely two years, were the unpredictable cross-winds coming in off the sea with nothing to stop them, unpredictable gusts that could push you way over the road to the wrong side and for which the only response was to ride at an angle trying to generate force in the opposite direction.

Yes, I would disagree that a 20 mph crosswind will not slow you down at all - yes it will because you have to expend a significant amount of energy trying to stay in a straight line rather than be blown across the road. Not as bad as headwind obviously but its effect is not zero for going straight forward.