Try switching to Level Set Method. It might be more robust for your convection load. Refer to the Mechanical USers Guide=>Optimization Analysis Workflow=>Apply Design Objectives and Constraints...Thermal Compliance...
For the and methods, the option uses a unique formula regardless of the context. That is, it executes in the presence of Heat Flux, Heat Flow, prescribed Displacement, Convection conditions, and/or Internal Heat Generation.