One simplifying assumption to speed up analysis is small deflection, also called small rotation. The shaft you analyzed benefits from this assumption because the rotation of the end with the torsion is very small. Mechanical can multiply these very small deflections by a large factor to make the displacement visible. When it does this, the node travels in the direction it moved, but keeps going in a straight line when multiplied by the large factor to make it visible. When the nodes are arranged in a circle, and the nodes move in a tangential direction, when you multiply that motion by a large factor, you get a visually larger diameter. Draw a circle through the ends of the red lines that were tangent to the black circle, you can see a larger circle. Mechanical is doing this to make the very small displacement visible. You can set the displacement scale factor to 1.0 True Scale, then the diameter will not change.
If you have material that is actually undergoing large deflection, then you turn on large deflection under analysis settings and the analysis will take more time, but nodes on a cylindrical shaft made of an elastomer for example, will move around the circle and will not travel in a straight line when the displacement scale factor is set to 1.0 True Scale.
Ansys will often issue warnings that can be safely ignored if the constraints are not in conflict.