This may be dramatic but the writing is on the wall for small sellers. Amazon has an underlying performance platform that is definitely used for gating and buy box (probably other area's as well). It has also been used in an obvious way with metric thresholds on our accounts. They have increased fee's across the board and now have low volume shipping fee's and low volume listing/warehouse fee's. (FBA specifically).
Fundamentally they only want very high volume, accurate, simple shipments being received through FBA. They want low volume and mixed stuff going through FBM or elsewhere. I'd venture to guess these inbound shipping costs are going to continue to get more punishing.
Think about how you are operating and figure out a way to fit the mold.