Think about reducing the brim distance to object. For filament that warps a lot, I put 0 brim distance.
The other thing is that your brim is not useful if it is not adhering to the build plate enough. Do you have brim on the outside only, or inside as well? Apply brim on inside too if you aren’t already.