The patch option only tends to cause problems if you patch a high volume fraction on/in a fluid jet OR the volume fraction is very near the packing limit.
With your model I think you need to hook that directly into the Granular Temperature as you're replacing the Fluent model.