I think a lot of it may have to do with perceived changes in the cost necessary to participate effectively.
My 1st set-up was a Zebco 404. I fished off the dock and with my Dad out of a 14′ boat with a 10hp outboard. Hook and minnow, hook and angle worm, jig and minnow, etc.
50 years later, add much bigger boat, more powerful outboard(s), depth finder potentially costing over $1,000, GPS, rod and reel set-ups for every scenario… If you watch a fishing show on TV, pay attention to the chatter on websites like this, or go fishing with one of us, it appears the cost to do this is pretty high. I think that scares new participants away.
The new wave of people fishing from kayaks has me hopeful that they will lead a new wave of people to fishing at a lower cost.