Its all personal preference really. I prefer my fish to be free of these parasites.
Even on the Mississippi there are areas where the sunfish and crappies and even perch stay free of infestations of parasites and if I am looking for some fish for dinner I will seek out those areas. Some of this has to do with current. Sluggish, near current-free backwaters can be rife with the black-spot and yellow worms.
A lot of the parasitic issue can be how certain species of fish are more bottom related too. There are a lot of fisheries that have the parasites all over sunfish and perch but not a bit of it in crappies caught in the same area.