Something in your full model is not properly constrained. The diagnostic technique to find the cause of the problem is to drop a Modal analysis on the Model cell of the Static Structural analysis. Now drag all the support boundary conditions from the Static Structural category to the Modal category in the Outline. Set the Modal Analysis Settings to deliver 12 modes. Solve the Modal analysis then plot Deformation for all the modes. If you get six modes with almost 0 frequency, that is a body that is unconnected to ground.
The most accurate model is to add the pin and make contact in the holes. That is the true behavior of the motion of the nodes in each hole. The flexing will cause more pressure to be on the inner circles of the holes and less or no pressure on the outer circles of the holes.