Scalar instead of vector type for bitmap input with ml.net and TensorFlow model

1

I started to get familiar with ml.Net, especially in context of image classification with a pretrained TensorFlow model.

The existing example (https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/getting-started/DeepLearning_ImageClassification_TensorFlow) for that task is working fine. I also managed to adapt it to use an image from memory instead of loading it in pipeline.

The next step is to consume an internal TensorFlow model and that's where the question begins.

The model can have several images as input instead of one and I don't get the input data structure running.

The model request as input type:

Vector<Single, 275, 384, 3>

The description for the input is: <Number of images, image height, image width, image channels (RGB)>.

In my data structure I tried to use an array of Bitmaps (instead of single Bitmap like in modified example mentioned above). With mapping for input column and definition of image size.

public class ImageData
{

    [ColumnName("serving_default_input_4"),  ImageType(275, 384)]
    public Bitmap[] Input { get; set; }
}

public class ImagePrediction : ImageData
{
    [ColumnName("StatefulPartitionedCall")]
    public float[] Scores { get; set; }
}

With this I get an exception when I try to create empty data for fitting (in line: var data = mlContext.Data.LoadFromEnumerable(new List());) that a scalar is expected as data type.

//Pipeline
IEstimator<ITransformer> pipeline = mlContext.Transforms.ResizeImages(outputColumnName: "serving_default_input_4", imageWidth: 384, imageHeight: 275, inputColumnName: "serving_default_input_4")
                    .Append(mlContext.Transforms.ExtractPixels("serving_default_input_4", "serving_default_input_4"))
                    .Append(mlContext.Model.LoadTensorFlowModel(_testTensorFlowModel)
                    .ScoreTensorFlowModel(outputColumnNames: new[] { "StatefulPartitionedCall" }, inputColumnNames: new[] { "serving_default_input_4" }, addBatchDimensionInput: true));

//Fitting
var data = mlContext.Data.LoadFromEnumerable(new List<ImagePrediction>()); // This line crashes
ITransformer model = pipeline.Fit(data);

The Exception I receive is:

System.ArgumentOutOfRangeException
  HResult=0x80131502
  Message = Column 'serving_default_input_4' is supposed to be scalar, but type of associated field 'Input' is vector 
  Source= Microsoft.ML.Data
  

I guess my problem is related to how the image list in ImageData is defined, but I don't see it... My research about how to declare a scalar data type was not really helpful so far.

Maybe somebody has an idea or hint.

Thanks.

Some system information:

  • Microsoft.ML 1.5.1
  • Microsoft.ML.ImageAnalytics 1.5.1
  • Microsoft.ML.TensorFlow 1.5.1
  • SciSharp.TensorFlow.Redist 2.3.0
  • Visual Studio 2019
  • Windows 10 64 bit
c#
tensorflow
ml.net
asked on Stack Overflow Aug 31, 2020 by Phil

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0