I might be having this problem as well, I thought it was related to wall order, but it seems like the tip of the hot end is dragging near the seam on my tall thin build. No problem with the first layers tho.
Here’s a small cross section I re-printed of the problem area