If you are getting a perfect first layer and problems with the second one chances are you are trying to print too fast. The first layer prints much slower than subsequence ones.
Also matte filament usually has to be printed slower than normal filament. What brand of filament are you printing?
The area in red is really hard to print as there is too much of an overhang. With a tall enough layer height the outer wall doesn’t have anything to adhere to and will pull off the model as you have seen. Speed probably isn’t your problem, model geometry is.
In addition to the already good advice, Ensure that pressure advance is enabled in your filament profile and is within reason. The gaps between your walls and solid infill are too large to be normal, they look almost completely unattached.
I have a new P2S, I had exactly the same issue with the same model and also a “fidget ring”. Did you find any more of these issues with other models? Did you fix it?