You design for the impedance change my making the filter go over above your desired operating band and making it slightly over damped into 8 ohms, and slightly under damped into your smaller load, then use the opamp filters on the input of the amp to limit the audio to a band that stops before the under damped output filter becomes too problematic (Spice is your friend here).