My 15′ glass boat does exactly the same thing. The sort of strange thing was that with a higher pitch prop, I could trim it out more and get a little more speed (but the motor overheated – bad news)!
The basic principles at work are the prop thrust and the trim angle need to work together with the hull design to steady the boat in the water. In my case, lower pitch prop = less thrust = less possible trim angle.
Weight distribution will affect it some, but for me, the 100lbs or so that I’m shuffling around really is a small percentage of the total weight of the rig when it’s in the water.
I’ve considered adding a fin/hydrofoil, but have heard mixed reviews and don’t want to throw money at something that’s not going to help. I’ll be interested to see if anyone else has a fix for this.