ML.NET - crashing when multithreaded

0

I used the ml.net model builder to make a simple image recognition program, but when I goto multi-thread it to check the images from multiple folders at the same time, it crashes. It returns "Exception thrown at 0x00007FFAB191F155 (tensorflow.dll) in Test.exe: 0xC0000005: Access violation reading location 0x0000000002020202."

The code is just the code that the ml.net model builder automatically generates:

public class ModelInput {
    [ColumnName("Label"), LoadColumn(0)]
    public string Label { get; set; }


    [ColumnName("ImageSource"), LoadColumn(1)]
    public string ImageSource { get; set; }
}

 public class ModelOutput {
    // ColumnName attribute is used to change the column name from
    // its default value, which is the name of the field.
    [ColumnName("PredictedLabel")]
    public String Prediction { get; set; }
    public float[] Score { get; set; }
}

class ConsumeModel {
    private static Lazy<PredictionEngine<ModelInput, ModelOutput>> PredictionEngine = new Lazy<PredictionEngine<ModelInput, ModelOutput>>(CreatePredictionEngine);

    public static string MLNetModelPath = Path.GetFullPath("MLModel.zip");

    public static ModelOutput Predict(ModelInput input) {

            ModelOutput result = PredictionEngine.Value.Predict(input);
            return result;
       
    }

    public static PredictionEngine<ModelInput, ModelOutput> CreatePredictionEngine() {
        // Create new MLContext
        MLContext mlContext = new MLContext();

        // Load model & create prediction engine
        ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);
        var predEngine = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(mlModel);
        return predEngine;
    }
}

the code I am using to determine the label of the image and the multi threading mechanism is a followed:

    public static void determine(string Dictpath) {
        for (int j = 0; j < 9; j++) {
            ModelInput sampleData = new ModelInput() {
                ImageSource = Directory.GetFiles(Dictpath)[j],
            };
            var predictionResult = ConsumeModel.Predict(sampleData);
            Console.WriteLine(predictionResult.Prediction);
        }
    }

And to multi-thread the application, I am using:

  string[] Directories = Directory.GetDirectories("Saved/");
   for (int i = 0; i < 10; i++) {
            Task.Run(() => {
                determine(Directories[i]);
            });
        }

I tried slowing down the amount of time it takes to start each thread, but I still get the same result. It seems to only work if it is single-threaded and crashes if its multi-threaded.

c#
multithreading
ml.net
asked on Stack Overflow Oct 30, 2020 by Matthew Ransley

1 Answer

3

You are using the same PredictionEngine instance from multiple threads concurrently, without synchronization. This is only viable if the PredictionEngine class is thread-safe. Unless the documentation states explicitly that "all public and protected members of the class are safe and may be used concurrently from multiple threads", you should assume that the class is not thread-safe, and not use it this way. Otherwise you'll get undefined behavior. You can either synchronize all access to a single instance by using a lock or other means, or create a dedicated instance per thread.

answered on Stack Overflow Oct 31, 2020 by Theodor Zoulias

User contributions licensed under CC BY-SA 3.0