Heat capacity matters here. Cold air has some but not near as much as a block of something solid. Heat capacity is how much heat energy a material contains at a certain temperature per unit mass. Solids and liquids tend to be dense while air isn’t that dense. Air can certainly carry heat energy (or absorb it) but it’s the difference in standing in front of an A/C register and feeling the cool vs holding a handful of ice and feeling that cold.
Unless you have a fair jet of very cold air, it’s not a big deal. If it was a flame the energy content would be fairly low. You can sweep your hand through fire and be fine because it’s not very energy dense compared to something like molten lead. Stick your hand in molten lead for even a fraction of a second and you’re in big trouble even though it’s much cooler than a flame. It’s because of heat capacity.
Kind of tough to explain but that’s the key words to use if you want to learn more.