Sometimes when fish are not actively feeding, it’s tough to convince them to bite. Spoons might be too aggressive for these fish.
One tactic is to get less aggressive – more moderate action baits, smaller size baits, and a less aggressive presentation.
I mentioned in another thread that the Acme Tungsten Pendu Jig (AKA Sling Blade) in the smaller 2 or 3 gram sizes has caught me fish when fish were approaching but wouldn’t bite. The VMC Tungsten Mongo jig with a Clam Silkie tipped with a spike or the VMC Tungsten Fly jig with a spike are pretty good at coaxing bites from less aggressive fish also.
You may find that early morning and late afternoon is a better time than mid day to try catching these same fish.