Noise is the sound of your old synths breathing. It's what tells your ear the sound is being produced by real circuitry and not ones and zeros. Noise is character. Noise is organic. Noise is interesting.
More importantly, noise surrounds us every day of our lives. The ambient hum of the city traffic... the eerie nightime calls of spring peepers (frogs) in the countryside... the whir of your computer's fans... noise is an inescapable part of our lives. So why do we want to get rid of it so much?
I think there is a tendency for technology to take things a step too far, at least initially. Those of us old enough to remember recording on tape can attest that noise reduction was a sort of holy grail in those days. Every year or so a company (usually Dolby) would come out with some new improved recipe for their noise reduction process. Of course, this was necessary as tape is an inherently noisy medium. But once digital recording came around, it was possible to not just reduce the noise, but to eliminate it almost entirely. That's the step too far I mentioned earlier.
Look, I'm not saying we should all start making our mixes sound like they were recorded on Thomas Edison's wax cylinder, just that maybe we should stop worrying so much about trying to cleanse our recordings of one of the things that can end up making them more interesting.In fact, everyone, at least once, should try to deliberately rough up their recordings and make them noisier. Try recording an old toy keyboard through the noisy headphone jack. Have a lush pad you want to warm up? Lay it down to cassette tape and resample it, hiss and all. Even consider sampling the sound of a blank tape's hiss and laying it behind your recordings or samples so it's barely audible. Imperfections give a sound more life. So embrace the noise!