Ggtcf

I'm new to tensorflow.js and tensorflow

The context : We have trained a model using custom vision to recognized from an image, the hairlength : short, mid, long. This model was exported and we would like to use it in local with tensorflow js. The exported files from custom vision are a *.pb file and a labels.txt file.

I have used tensorflowjs_converter python script, here is the command I have used to convert a frozen model *.pb in a json model :

tensorflowjs_converter --input_format=tf_frozen_model --output_node_names='model_outputs' --output_json OUTPUT_JSON C:pythontf_modelshairlengthmodel.pb C:pythontf_modelsexports

Then I paste this model.json and shards in the assets folder of my angular client. Then I try to load the model and I give him an image to get the prediction but all I get are indexes values that are way out of bound since I only need 0: long, 1: mid, 2: short hairlength. Here is a capture of the console

This is the class I have used in my client (typescript) for predictions:

import * as tf from '@tensorflow/tfjs';

// import HAIRLENGTH_LABELS from './hairlength';
import FrozenModel from '@tensorflow/tfjs';

const MODEL = 'assets/models/hairlength/model.json';
const INPUT_NODE_NAME = 'model_outputs';
const OUTPUT_NODE_NAME = 'model_outputs';
const PREPROCESS_DIVISOR = tf.scalar(255 / 2);

export class MobileNetHairLength 

 private model: FrozenModel;
 private labels = ['long', 'mid', 'short'];

 constructor() 

 async load()
 this.model = await tf.loadGraphModel(MODEL);
 

 dispose() 
 if (this.model) 
 this.model.dispose();
 
 

 /**
 * Infer through MobileNet. This does standard ImageNet pre-processing before
 * inferring through the model. This method returns named activations as well
 * as softmax logits.
 *
 * @param input un-preprocessed input Array.
 * @return The softmax logits.
 */
 predict(input) 
 const preprocessedInput = tf.div(
 tf.sub(input, PREPROCESS_DIVISOR),
 PREPROCESS_DIVISOR);
 const reshapedInput =
 preprocessedInput.reshape([1, ...preprocessedInput.shape]);
 // tslint:disable-next-line:no-unused-expression
 return this.model.execute([INPUT_NODE_NAME]: reshapedInput, OUTPUT_NODE_NAME);
 

 getTopKClasses(logits, topK: number) 
 const predictions = tf.tidy(() => 
 return tf.softmax(logits);
 );

 const values = predictions.dataSync();
 predictions.dispose();

 let predictionList = [];
 for (let i = 0; i < values.length; i++) 
 predictionList.push(value: values[i], index: i);
 
 predictionList = predictionList
 .sort((a, b) => 
 return b.value - a.value;
 )
 .slice(0, topK);

 console.log(predictionList);
 return predictionList.map(x => 
 return label: this.labels[x.index], value: x.value;
 );
 

And this the class that calls the above one, I just give the canvas element :

import 'babel-polyfill';
import * as tf from '@tensorflow/tfjs';
import MobileNetHairLength from './mobilenet-hairlength';

export class PredictionHairLength 

 constructor() 

 async predict(canvas) 
 const mobileNet = new MobileNetHairLength();
 await mobileNet.load();
 const pixels = tf.browser.fromPixels(canvas);

 console.log('Prediction');
 const result = mobileNet.predict(pixels);
 const topK = mobileNet.getTopKClasses(result, 3);

 topK.forEach(x => 
 console.log( `$x.value.toFixed(3): $x.labeln` );
 );

 mobileNet.dispose();
 

My questions are :

Is the convert python command correct ?

Did I miss something in my client to get the correct indexes ?

Thank you for your time and answers

If you need more informations, I would be glad to give them to you

I think it's a simple bug in your code: const INPUT_NODE_NAME = 'model_outputs'; should probably be 'model_inputs' or whatever it actually is. Here, you're setting the output to be the input image and then reading it back without predicting anything.