Hello,

The parameter name "Smooth transition" may be a bit confusing, because it relates only to the IMU calculations, not stabilization. When you enable it, IMU does not go crazy in gimbal lock position. Another problem is that stabilization suffers when two motors become close to parallel - their interference significantly increased. In this position only adjusting PID values may help.

A good solution may be installing 2nd frame IMU in the position "Below outer motor" (YAW in your case). It starts working in the gimbal lock position and improves the system response. W/out 2nd IMU, encoders are used as a closed-loop signal source, which makes it less effective than the normal gyro-based closed loop.

today I make a downgrade to 2.56 firmware and all is working done and properly, all motor and axis i correct workin in gimbal lock mode without artefacts. maybe Alex can fix this problem in next firmware?

i make a video https://youtu.be/hNt2qqaeK-Y

Hi! This parameter doesn't have any downside and is reccomended to be enabled. It starts working when PITCH angle approaches +-60-70 degrees, where the attitude expressed in Euler angles, becomes unstable.
If you have vibrations at a certain position (what I have guessed after checking your video showing RMS error plot), it may relate to PID tuning. The solution would be adjusting PIDs in this problematic position. Using the GUI's "optimize" tool you can do it automatically, by capturing system response in two or more positions and optimizing PID parameters for them all at once (i,e, load several responses for one axis and run tool for them, with the "Do for all loaded data files" option checked).