It isn't that the heater valve is moving, it is that it is allowing water to circulate when off.
Why is that happening? The heater valve cable isn't fully allowing it to the stop, or fully closed.
The heater valve even though it is off is not fully stopping the flow of water.
Heater valves get gummed up just like the rest of the system. The difference is that they made 2 types of heater valves over the years.
The original one is a flag type, that basically shuts off the flow to a trickle (yes it doesn't fully close) as there is a small hole left when closed, this allows for water to semi circulate, if it didn't then you could face a really corroded heater core, and that isn't a good thing.
The second type is like a ball stop, it too allows for a trickle. But that trickle is over come by the a/c.
Now, how can you get your a/c colder.
Proper charging of the system.
Your fresh/recirc valve must be functioning.
The recirc servo is in the rain tray infront of the fresh air plenum (duct). When the a/c slide switch is fully to the right, that tells the servo to engage, and to pull the door up. This shuts off air coming in across the hood and heated by the engine bay. It also tells the system that you can re-use the cooler cabin air. Cooling chilled air is better than chilling air coming off the hood. No it isn't going to get stale, as the cabby's aren't air tight any way.
Now I have only been driving these little guys since 1980, and have seen this issue for years. If you don't want to have water circulate in the system, then you have to by-pass the heater core. But then in the winter you would have to re-connect it.
I have placed a a/c cut out switch in my cars, so I can choose to run the a/c or not. If I want defrosted air with no a/c running I turn the a/c compressor off.
If your car is aged, then I would consider replacing the heater valve, on a cold car with 2 pinch clamps, it is a 10 minute job to replace it with a known good unit. Shoot the next time you are in a wrecking yard, take a minute to cut a used one out and look at it while you turn the valve.
They do break internally, and even thou the cable pulls the arm, the valve doesn't move.
When the car is parked in the sun, The plenum heats up. The air around the heater core gets hot, the heater core since it doesn't have colder air blowing over it warms up the surrounding plastic. When you get in your car to start it, you get cold air that can't compete with the pre-warmed box.
To compensate I usually turn the center vents off, and run with the side vents only for a few minutes. The side vents are colder as they don't run over the heater core. The air from the a/c will start to cool the air box down. Once it is cool about 5 minutes, the center vents should only be about 5 degrees different than the sides.
Now if you want to completely shut of the heater in the summer, then again the only way to do that is to place a by-pass on the hoses.