Does the model now compute incore, if you issue DSPOPTION,,INCORE ?
Does the warning message "out of core memory mode" disappear?
Is the model linear elastic, small deformations?
To check, try to build another model (completely stand alone, do not copy from this one).
Make a simple block, held at the base. Refine the mesh in order to have approximately the same number of nodes (better: the same number of DOF). Apply one force on the side opposite the base. Do you still get the message "out of core"?
This will eliminate the doubt that something is "special" in your model. Neither symmetry, nor pretension should be the reason.
Hope this helps, Regards