Change filament by height AND use a modifier for colour change

Say I make a model, and slice it, change the colour by height by using the right hand side bar (right click) change filmament, while also having a modifier block on the object with a different colour, but now when I add the modifier, the colour changes by height are removed automatically. It was handy to do this if you had text you needed a different colour to the height range.