Your Yaw D is at its limit. Set gain multiplier bigger on advanced for yaw and retune. set the multiblyer big enough so that after retuning D is less than 255. Try 2 and divide PID values by 2 and retune.

You have frame IMU, you should not need estimate frame angles from motors. I am not sure what Use Frame IMU if possible does, try without it.

Try limiting acceleration under RC.

you may want to considered follow potentiometer or encoders.

"In the Gains Multiplier settings, should I leave the Outer P for YAW set at 100?"

They are more for RC response...