Nozzle touch the part after purge when printing in part mode

I noticed that when I print in sequence by object, and I have various parts of different colors on the surface, after purging the nozzle to change color and start printing the new object, if there is a piece already printed in the nozzle path/way to print the new piece, the nozzle slides for a millimeter on the already printed part, releasing a flash of the new color on the printed piece.
Is there a way to prevent this from happening?