Thanks Lito, I think we are getting somewhere close to solving this. I have two questions.

a) Firstly, I have not enabled "Use python provided by lumerical option". So the Python exectuable path is located in my usual python folder installed in my machine. When I enabled this option, the executable path is located in a folder named "python-3.6.8-embed-amd64" in my D drive. This is not like your path which is in Programfiles directory of C-drive. Not sure, how I got this folder into my system though?

Now, when I run the simple pytest script, it works fine. I get a new FDTD opened up with a ring in it. This is as expected.

b) The goal is to use an external IDE (like Spyder/Visual code) to interface with FDTD. It still does not work. Do you think it has to do the interpreter path? The IDE like Spyder/VScode were installed in a bundle with Anaconda. So the python interpreter used is the default python.exe from conda. Anyway, I tried three different inerpreter paths:
C:\Program Files\Lumerical\v222\python-3.9.9-embed-amd64\python.exe
I still end up getting the error.