PyQt5 run sklearn calculations on a separate QThread

I am creating a PyQt5 application and want to run some ML code in a separate thread when a button is clicked. I am unable to do so. This is my threading code:

newthread = QtCore.QThread()
runthread = RunThread()
runthread.moveToThread(newthread)
runthread.finished.connect(newthread.quit)
newthread.started.connect(runthread.long_run)
newthread.start()

And the class RunThread is as follows:

class RunThread(QtCore.QObject):

finished = QtCore.pyqtSignal()

def __init__(self):
    super().__init__()

@pyqtSlot()
def long_run(self):
    #ML code called in the line below
    simulation.start_simulation()
    self.finished.emit()

Running it normally doesnt work. Pycharm quits with the following error:

process finished with exit code -1073740791 (0xc0000409)

Running it in debug mode throws thousands of warnings thrown by sklearn saying:

Multiprocessing-backed parallel loops cannot be nested below threads,
setting n_jobs=1

The code runs eventually, but it takes a lot more time (at least 4x times) to do so.

Could someone please let me know what the issue here is?

The warning from sklearn is pretty explicit here. Basically you cannot train sklearn models with hyperparameter n_jobs set to anything greater than 1 when executing from a nested thread.

That being said, without seeing what is going on inside of simulation.start_simulation() it's hard to conjecture.

My best guess would be to look for anything in start_simulation() that is using multiple jobs and see what happens when you set that to n_jobs=1.

Alternatively, you may try writing your ML code as a standalone script and execute it using QProcess. That may allow you to utilize n_jobs greater than 1.