Polishes usually contain some kind of oil. Oil can penetrate small fissures in the varnish and end up in the wood, damping the sound, or making future gluing and repair more difficult. Some oil will remain on the surface and catch dirt, essentially turning your violin into an air filter. If the oil is a vegetable base, it may harden over time, forming a new "varnish" layer incorporating dirt from the air, perspiration and rosin. Yuck! This can be very difficult to remove, even by a pro!