I have been experiencing a very similar problem on an almost identical setup.
After commissioning a new but similar 2-axis frame hanging from the rail.
I can not increase the gains enough for stable tracking on Pitch without introducing high frequency noise into the loop.
I am using a 2-axis 8 bit non-genuine board w/ 2.408b firmware.
Even with low power and gains, oscillations are easily induced, both high frequency and an odd twitching motion.

Things I have tried and tested:
- smoothened 12V power supply
- 6 point Acc & Gyro re-calibration
- new motor + measured phase resistances (13 ohm for DYS 2208-70 motor)
- swapped Roll and Pitch terminals from board, problem stays at Pitch.
- Gryo Low-Pass Filtering. worsens the problem oddly enough.
- ferrite ring on cabling
- different cabling
- reflashing firmware
- external FC acc signals.

Problem remains; bad tracking, twitching and high frequency jitter.
Turning the motors OFF doesn't show any more noise on the Pitch channel for the acc and gyro readings than the roll, so the IMU seems fine.

*** I noticed the AUTO DETECT function consistently designates 15 poles to the pitch motor, while correctly deducing 14 poles for the same Roll motor.
This suggests a bad motor, however I ordered it brand new to solve this very problem...

Sorry to not be of help, if you solve plz lemme know!

I still haven't solved this.
Tried, tested and swapped the motor but problem remains.
My best guess is a bad board, but swapping motor outputs doesn't make a difference so even that seems odd.
Will order a new board i guess