With out seeing it in person I would say the roller pivot stop has worn enough to allow over pivoting of that roller .
Remove the roller assembly and turn 180 degrees and re-install and test the angle.
Before I rebuilt my Sparton trailer I had the same issue,sometimes loading boat with the trailer in to far will cause the same issue, but suspect a wore pivot stop.
JMHO Jeff