If your cabin temperature is close to your climate control setting (accounting for heat from the sun as well), the fan speed will go to the minimum setting. If you don't want the A/C on, just press the button to disable it and keep everything else on Auto. Keep in mind that A/C also helps window defrosting by drying the air that comes out of your vents - it's not just used for summer cooling. As per the user manual, the compressor will not come on if the outside temperature is below freezing.