I think that Cleaning your cymbals keeps them as they were intended to sound. After all..they were made that way.
As for the sound, I think it does change if you Don't clean them. They don't vibrate quite as much due to the oxidation in the tonal grooves.
That's not a bad thing at all..You may have heard of some people burying they're ride cymbals in the ground to dry the sound out..most notably for Jazz applications..
But today there are so many options for getting the sound you want that you don't have to go to that extreme. I have a 20'' Zildjian K Constantinople Ride that I haven't cleaned since the day I bought it about 10 years ago, and when I first got that cymbal, I used to put a small piece of tape on it to dry out the sound. Now I don't have to do that anymore due to the build up.
Cleaning Cymbals actually takes a small amount of metal out of them.. That black stuff you see on your cloth when you clean them is actually metals.. So if you clean them a lot, this will weaken the cymbal over time.
I prefer to keep my cymbals relatively clean, I think they do sound a little clearer, and I like they way they look clean as well.