TIDL: Trouble-Shooting Guide

Table of Contents

• Steps to Debug Error Scenarios
• Steps to Analyse Performance
• Steps to Debug Functional Mismatch
  • Weights Quantization statistic Analysis
  • Feature Map Scale Analysis
  • Feature Map Comparison with Reference
    • Example script to Reference output from Caffe
    • Example script to Reference output from MxNet for ONNX
    • Example script to Reference output from tensorFlow for frozen graph
    • Example script to Reference output from tensorFlow for TFLite model

Steps to Debug Error Scenarios

• We recommend testing the inference first in PC emulation mode with debugTraceLevel = 2 enabled. This shall help to identify the error scenario and layer in which the error is happening. Then user can validate model in EVM with TIDL standalone application and match result with PC. Finally the user can test the model in SDK. debugTraceLevel = 2 can be enabled in SDK level also.
• debugTraceLevel shall be set in inference configuration file

Steps to Analyse Performance

• set enableLayerPerfTraces = 1 in inference configuration file to enable layer level performance.
• This shall help in identifying the layers which is running slower than expected.
• When enableLayerPerfTraces set to one , then the network level shall be taken from first line of the trace "Network Cycles"
• debugTraceLevel shall be 1 to get the right performance numbers

Network Cycles 6294273 

Layer,   Layer Cycles,kernelOnlyCycles, coreLoopCycles,LayerSetupCycles,dmaPipeupCycles, dmaPipeDownCycles, PrefetchCycles,copyKerCoeffCycles
  1,          81811,          48850,          49277,           7779,          14969,                18,           1007,                16,   
  2,          71051,          52722,          53246,           1473,           3290,                16,              0,                 0,   
  3,          34063,          16700,          17307,           7379,           3952,                18,             17,                16,   
  4,          60926,          45133,          45431,           6625,           4176,                18,            777,                 9,   
  5,          29990,           5996,           6040,            871,           3432,                 9,              0,                 0,   
  6,          30806,          14975,          15275,           6575,           4114,                61,             10,                 9,   
  7,          20355,           5508,           5810,           6360,           3480,                11,             10,                 9,   
  8,          34670,          20921,          21031,           6222,           2291,                18,            727,                 9,   

Steps to Debug Functional Mismatch

• The following steps are suggested, when user find the functional issue (Lowe accuracy) while running inference using TIDL compared to Floating model inference on Training framework (Caffe, tensorflow, Pytorch etc). All these steps needs to performed on PC emulation mode rather than device. Once user achieves the accepted accuracy then, PC emulation and Target execution device can be bit-matched.
  • Try running the network with Higher Bit depth numParamBits = 12 or 16
    • If the results is acceptable with 16-Bits then the 8-Bit quantization is Not sufficient for your network. We recommend to try the suggestion provided in Here
    • The accuracy is not acceptable with 16-bit mode also then, follow below steps to narrow down the issue

Weights Quantization statistic Analysis

• The import tool generates parameters quantization statistics at end of model import. This information is saved as "*_paramDebug.csv" in the same location as output TIDL model files. Find the below for example file. This information calculated using the float weights and and quantized weights in function "TIDL_CompareParams". User can refer this code for more information each parameter.

Weights Quantization statistic

The important information that needs to be check during debugging is mean and max of all the absolute float parameters "meanOrigFloat" and " orgmax". If the orgmax is much higher than the meanOrigFloat then there is possibility of higher quantization loss. Refer (!tidl_fsg_quantization.md::did_tidl_quantization_Types) reduce the quantization loss in inference User can compared this file with 16-bit and 8bit parameters.

Feature Map Scale Analysis

• If user could not find any information regarding the quantization error, the next steps is to analyze the feature map range scales used during the quantization of the same.
• This can be achieved by enabling layer level traces from TIDL inference by setting below

debugTraceLevel = 1
writeTraceLevel = 3

• When user run the inference, below information is printed in the console. The first colum here is data ID for which the trace data being dumped. The second the and third colum is minimum and maximum value in the tensor. The fourth colum is the data element type of the tensor.

Feature Map Scale Console Log

• If the minimum and maximum values are in the range of the 1 to 4 then, minimum quantization error are expected for these layers. If the range is large (> 32), and has big variation between layers, then it is recommended to try inference with below in create time parameters.

RangeExpansionFactor = 1.5

Feature Map Comparison with Reference

• Even after setting above, if the quantization error is high then user need analyse the dumped trace. At end of above inference traces, layer levle traces are generated in the tidl/test/trace folder
• The inference software generates two files for each data ID

Feature Map Trace files

• These files can be viewed using generic binary file viewers
  

  Feature Map Trace Float view
• User can generate these traces for 16-bit and 8-bit settings in import configuration file and compare them using below script.

import numpy as np
import argparse
import matplotlib
import matplotlib.pyplot as plt

parser = argparse.ArgumentParser(description='My Arg Parser')
parser.add_argument('-i', '--in_file_list',                       default='trcae_files_list.txt', help='test file containinglist of files to compare', required=False)
args = vars(parser.parse_args())

#dir *float* /o:d /s/b

def save_error_plot(list,i, mean, var, mxd, mx):    
    plt.hist(list, color = 'blue', edgecolor = 'black',bins=60)    
    #image_txt = "mean = " + str(mean) +", Var = "+ str(var) +", MAx = "+ str(mx)
    image_txt = "MeanAbsDiff=%7.4f, MaxAbsDiff=%7.4f, MaxVal=%7.3f" %(mean, mxd, mx)
    plt.title(image_txt)
    plt.savefig("figs\\"+str(i).zfill(4)+"_abs_diff_hist.png")    
    plt.clf()    

def main():
    with open(args['in_file_list']) as f:
        content = f.readlines()
        f.close()
    print("%5s, %12s, %12s, %12s, %12s %12s, %12s, %12s" %("Idx", "Min", "Max", "max_abs_diff", "max_diff_idx", "mean_abs_diff",  "var_abs_diff", "Scale"))
    for i, line in enumerate(content):
        values = line.split()

        fileHandle = open(values[0], 'rb')   
        tidl_data = np.fromfile(fileHandle, dtype=np.float32)
        fileHandle.close()

        fileHandle = open(values[1], 'rb')   
        ref_data = np.fromfile(fileHandle, dtype=np.float32)
        fileHandle.close()

        mx = np.max(ref_data)
        mn = np.min(ref_data)
        org_diff = (tidl_data - ref_data)
        combined = np.vstack((ref_data, tidl_data, org_diff)).T     
        np.savetxt("figs\\"+str(i).zfill(4)+"_float.txt", combined, fmt='%10.6f, %10.6f, %10.6f') 
        abs_diff = abs(tidl_data - ref_data)
        maxIndex      = np.argmax(abs_diff)
        max_abs_diff  = np.max(abs_diff)
        mean_abs_diff = np.mean(abs_diff)
        var_abs_diff  = np.var(abs_diff)
        save_error_plot(abs_diff, i,mean_abs_diff,var_abs_diff,max_abs_diff,mx)
        rng = max(np.abs(mx), np.abs(mn))
        if(mn < 0):
            scale = 127/rng if rng!=0 else 0
            tidl_data = np.round(tidl_data * scale)
            tidl_data = tidl_data.astype(np.int8)  
        else:
            scale = 255/rng if rng!=0 else 0
            tidl_data = np.round(tidl_data * scale)
            tidl_data = tidl_data.astype(np.uint8)  

        tidl_data = np.asarray(tidl_data, order="C")      
        with open(values[0]+"viz.y",'wb') as file:
            file.write(tidl_data)
            file.close()


        print("%5s, %12.5f, %12.5f, %12.5f, %12d, %12.5f, %12.5f %12.5f" %(i, mn, mx, max_abs_diff, maxIndex, mean_abs_diff,  var_abs_diff, scale))


if __name__ == "__main__":
    main()

The input list file shall contain the trace file names as below

D:\trace_8-bit\onnx_tidl_infer_resnet18v1.txt_0001_00064_00112x00112_float.bin D:\trace_16-bit\onnx_tidl_infer_resnet18v1.txt_0001_00064_00112x00112_float.bin
D:\trace_8-bit\onnx_tidl_infer_resnet18v1.txt_0002_00064_00056x00056_float.bin D:\trace_16-bit\onnx_tidl_infer_resnet18v1.txt_0002_00064_00056x00056_float.bin

• If user passes the file paths to 8-bit and 16-bit traces , then this scripts would write the float values and differences to a text files like below

Feature Map Trace Float view - Using numPy

• This script also generates below histogram for each tensor. If the mean difference is close to maximum difference then we have, then this particular tensor has higher quantization loss.

FM Difference Histogram

Reference Traces from Frames works.

• If the 16-bit flow also not providing expected accuracy, then there is possibility that some of the layer has very high quantization loss or functionally not working as expected. In this case, user need to generate the layer level traces from training frame works/ PC Runtime codes like caffe, tensorflow, MXNet etc.
• Since TIDL merges may layers during import of the model, user need know the tensor in original model corresponding to each data ID in the TIDL model. User can refer the "*_netLog.txt" in the same path as imported model
• The out Data Names, here corresponds to the tensor name in the original network. User cab use this information and once of the reference scripts below to generate trace information

Network Log

• All the scripts provided below are for reference purpose only. May not directly work wit any version. User many need to update it as per the latest API in the corresponding framework by taking help from community forum.

Example script to Reference output from Caffe

import os
import os.path
import time
import sys
import ntpath

model_path = "/mnt/d/work/vision/c7x-mma-tidl/ti_dl/test/testvecs/models/public/caffe/resNet10/deploy.prototxt"
pretrained_path = "/mnt/d/work/vision/c7x-mma-tidl/ti_dl/test/testvecs/models/public/caffe/resNet10/resnet10_cvgj_iter_320000.caffemodel"
input_name = "/mnt/d/work/vision/c7x-mma-tidl/ti_dl/test/testvecs/input/airshow.jpg"

import caffe
caffe.set_mode_cpu()
from caffe.proto import caffe_pb2
import cv2
import numpy as np
import math
import string
from google.protobuf import text_format

def writeNPAryAsRaw(ipFrame, fileName, opDataType=np.float32, opScale=1):
    if opDataType != np.float32:
        qFrame = np.rint(ipFrame * opScale)
    else:
        qFrame = ipFrame

    fileHandle = open(fileName, 'wb')
    ip1DAry = np.reshape(qFrame, (1, np.prod(qFrame.shape)))
    ip1DAry = ip1DAry.astype(opDataType)
    fileHandle.write(ip1DAry)
    fileHandle.close()

def predict(model_path, pretrained_path, image, frameNum, blobs=None):
    net = caffe.Net(model_path, pretrained_path, caffe.TEST)

    input_dims = net.blobs['data'].shape
    print ("input dim from desc", input_dims[2], input_dims[3])

    batch_size, num_channels, input_height, input_width = input_dims
    caffe_in = np.zeros(input_dims, dtype=np.float32)

    caffe_in[0] = image.transpose([2, 0, 1])
    out_blobs = net.forward_all(blobs, **{net.inputs[0]: caffe_in})

    return out_blobs, net

def getLayerByName(net_proto, layer_name):
    for layer in net_proto.layer:
    if layer.name == layer_name:
        return layer
    return None

def infer():
    caffe.set_mode_cpu()
    mean_pixel = [0, 0, 0]
    num = 0
    use_cur_scale = True 

    net_proto = caffe_pb2.NetParameter()
    text_format.Merge(open(model_path).read(), net_proto)

    # moved image reading out from predict()
    image = cv2.imread(input_name, 1);
    image = cv2.resize(image, (224, 224))

    image = image.astype(np.float32)- mean_pixel
    layer_names=['prob', 'data',  'data_scale', 'conv1_relu', 'layer_64_1_relu2','layer_64_1_conv2','layer_64_1_sum','layer_128_1_relu1','layer_128_1_relu2']    
    blob_names =['prob', 'data',  'data_bn', 'conv1','layer_64_1_conv1','layer_64_1_conv2', 'layer_64_1_sum','layer_128_1_bn1', 'layer_128_1_conv1']
    blob_type  =['int8', 'uint8', 'int8', 'uint8', 'uint8', 'int8', 'int8', 'uint8', 'uint8']
    out_blobs, net = predict(model_path, pretrained_path, image, num, blobs=blob_names)

    dataOut = out_blobs['prob']
    print(dataOut.shape)
    argIndex = np.argsort(np.squeeze(dataOut))[::-1][:10]

    print (argIndex)   
    if 'data' in out_blobs.keys():
        writeNPAryAsRaw(out_blobs['data'], 'data'+'_orgIn'+'.y', opDataType=np.uint8, opScale=1)       

    for blobName in out_blobs.keys():
    layerIndex = blob_names.index(blobName)
    layerName = layer_names[layerIndex]
    print(layerName, blobName)
    layerParam =  getLayerByName(net_proto, layerName)  
    min_val = np.min(out_blobs[blobName])
    max_val = np.max(out_blobs[blobName])

    elementType = blob_type[layerIndex]
    scale   = 255/np.abs(max_val) if elementType=='uint8' else 127/np.maximum(np.abs(max_val), np.abs(min_val))

    print(scale)
    writeNPAryAsRaw(out_blobs[blobName], blobName+'_float32'+'.y', opDataType=np.float32)  
    if elementType=='int8':
        out_blobs[blobName] = out_blobs[blobName] + 128/scale
        writeNPAryAsRaw(out_blobs[blobName], blobName+'_'+elementType+'.y', opDataType=np.int8, opScale=scale)  
    else :
        writeNPAryAsRaw(out_blobs[blobName], blobName+'_'+elementType+'.y', opDataType=np.uint8, opScale=scale)  

def main():
    infer()

if __name__ == '__main__':
    main()

Example script to Reference output from MxNet for ONNX

import mxnet as mx
import numpy as np
from collections import namedtuple
from mxnet.gluon.data.vision import transforms
from mxnet.contrib.onnx.onnx2mx.import_model import import_model
import os
import argparse

parser = argparse.ArgumentParser(description='My Arg Parser')
parser.add_argument('-m', '--model',                       default='../../test/testvecs/models/public/onnx/squeezenet1.1.onnx', help='Input Onnx model to load', required=False)
parser.add_argument('-i', '--image',                       default='../../test/testvecs/input/airshow.jpg', help='Input Image to infer', required=False)
parser.add_argument('-t', '--trace_enable',  type=int,     default=1, help='Set 1 to enable trace', required=False)
parser.add_argument('-d', '--dir_trace',                   default='trace/', help='Base Directory for trace', required=False)
parser.add_argument('-b', '--input_tensor_name',           default='data', help='Input tensor name', required=False)
parser.add_argument('-r', '--rgb_input',     type=int,     default=1, help='Input tensor RGB or BGR, set to 1 for RGB and 0 for BGR', required=False)
parser.add_argument('-p', '--pre_proc_type',     type=int, default=0, help='Pre-Processing type for Input', required=False)
args = vars(parser.parse_args())

print(args['model'])

#mx.test_utils.download('https://s3.amazonaws.com/model-server/inputs/kitten.jpg')
#mx.test_utils.download('https://s3.amazonaws.com/onnx-model-zoo/synset.txt')
with open('synset.txt', 'r') as f:
    labels = [l.rstrip() for l in f]

# Enter path to the ONNX model file
model_path= args['model']
sym, arg_params, aux_params = import_model(model_path)


Batch = namedtuple('Batch', ['data'])
def get_image(path, show=False):
    img = mx.image.imread(path, to_rgb=args['rgb_input'])
    if img is None:
        return None
    return img


def preprocess(img):   
    transform_fn = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
    img = transform_fn(img)
    img = img.expand_dims(axis=0)
    return img

def preprocess_1(img):   
    transform_fn = transforms.Compose([
    transforms.Resize(224, keep_ratio=True),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.5, 0.5, 0.5], [0.00390625, 0.00390625, 0.00390625])
    ])
    img = transform_fn(img)
    img = img.expand_dims(axis=0)
    return img

def predict(path):
    img = get_image(path, show=False)
    if args['pre_proc_type'] == 0 :
        img = preprocess(img)   
    elif args['pre_proc_type'] == 1 :
        img = preprocess_1(img)
    mod.forward(Batch([img]))
    internal_dict = dict(zip(all_names, mod.get_outputs()))
    #print(internal_dict[output_names[0]])
    # Take softmax to generate probabilities
    scores = mx.ndarray.softmax(np.squeeze(internal_dict[output_names[0]])).asnumpy()
    if(args['trace_enable']):   
        print('Writing')
        for name in all_names:
            act = internal_dict[name]
            act = act.asnumpy()
            maxval = max(act.ravel())
            minval = min(act.ravel())
            rng = max(np.abs(maxval), np.abs(minval))
            if minval >= 0 : 
                scale = 255/rng if rng!=0 else 0
            else :
                scale = 127/rng if rng!=0 else 0

            #print(name, act.shape, minval, maxval)  
            act_shape = act.shape
            file_name = name+str(act_shape)
            file_name = file_name.replace("/","_").replace("(","_").replace(")","").replace(", ","x").replace(",","x").replace(" ","_")+".y"
            file_name = "trace/"+file_name
            print(file_name, act_shape, minval, maxval, scale)
            with open(file_name+"float.bin",'wb') as file:
                act_float = act.astype(np.float32)  
                act_float = np.asarray(act_float, order="C")      
                file.write(act_float)
                file.close()

            file_name = name+str(act_shape[-2:])
            file_name = file_name.replace("/","_").replace("(","_").replace(")","").replace(", ","x").replace(",","x").replace(" ","_")+".y"
            file_name = "trace/"+file_name
            with open(file_name,'wb') as file:
                #act = np.round(act * scale) + 128
                act = np.round(act * scale)
                act = act.astype(np.uint8)  
                act = np.asarray(act, order="C")      
                file.write(act)        
    # print the top-5 inferences class
    scores = np.squeeze(scores)
    a = np.argsort(scores)[::-1]
    print(a[0:5])
    for i in a[0:5]:
        print('class=%s ; probability=%f' %(labels[i],scores[i]))


# Determine and set context
if len(mx.test_utils.list_gpus())==0:
    ctx = mx.cpu()
else:
    ctx = mx.gpu(0)

output_names = sym.list_outputs()
#print('Output : ')
#print(output_names)
if args['trace_enable']:
    sym = sym.get_internals()
    blob_names = sym.list_outputs()
    sym_group = []
    for i in range(len(blob_names)):
        if blob_names[i] not in arg_params:
            x = sym[i]
            if blob_names[i] not in output_names:
                x = mx.symbol.BlockGrad(x, name=blob_names[i])
            sym_group.append(x)
    sym = mx.symbol.Group(sym_group)

all_names = sym.list_outputs()
#print('All Output : ')
#print(all_names)

# Load module
mod = mx.mod.Module(symbol=sym, data_names=(args['input_tensor_name'],), context=ctx, label_names=None)

mod.bind(for_training=False, data_shapes=[(args['input_tensor_name'], (1,3,224,224))], 
        label_shapes=mod._label_shapes)
mod.set_params(arg_params, aux_params, allow_missing=True, allow_extra=True)



# Enter path to the inference image below
#img_path = 'kitten.jpg'
img_path = args['image']

predict(img_path)

Example script to Reference output from tensorFlow for frozen graph

import math
import tensorflow as tf

import time
import os
import numpy as np
import PIL
import PIL.Image as Image

tf.app.flags.DEFINE_string(   'graph_path', 'tmp\keras_frozen_optimized.pb',  'The directory where the graph was written to or an absolute path to a ')
tf.app.flags.DEFINE_string(   'input_file', '00010.png',  'Input image to be inferred ')
tf.app.flags.DEFINE_integer(  'infer_image_height', 32, 'Infer image height')
tf.app.flags.DEFINE_integer(  'infer_image_width', 32, 'Infer image width')
tf.app.flags.DEFINE_bool(     'write_activations', True, 'Write the activations to file')
tf.app.flags.DEFINE_string(   'input_node_names', '00010.png',  'Input image to be inferred ')
tf.app.flags.DEFINE_string(   'output_node_names', '00010.png',  'Input image to be inferred ')
tf.app.flags.DEFINE_string(   'preproc_type', 'inception',  'Type of pre proc to be applied ,eg: inception or vgg ')

FLAGS = tf.app.flags.FLAGS

def all_nodes():
nodes = [n for n in tf.get_default_graph().as_graph_def().node]
node_names = [n.name for n in tf.get_default_graph().as_graph_def().node]  
ops = []
for node in nodes:
    #print(node.name, node.op)
    outputs = tf.get_default_graph().get_operation_by_name(node.name).outputs
    if len(outputs) > 0:
    op_name = outputs[0]
    ops.append(op_name)
return ops, nodes, node_names


def infer_frozen_graph(graph_filename):
ext = os.path.splitext(graph_filename)[-1]

if ext == ".pbtxt":
    with tf.gfile.GFile(graph_filename, 'r') as gfilep:   
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(gfilep.read())
elif ext == ".pb":
    with tf.gfile.GFile(graph_filename, 'rb') as gfilep:   
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(gfilep.read())

with tf.Graph().as_default() as graph:
    _ = tf.import_graph_def(graph_def, name='')

    infer_image_height = FLAGS.infer_image_height 
    infer_image_width = FLAGS.infer_image_width

    image = Image.open(FLAGS.input_file)
    image = image.convert('RGB')  

    #image Net center crop 
    if FLAGS.preproc_type == "inception" :
        image = crop_image_by_factor(image) 

    image = image.resize((infer_image_height,infer_image_width),resample=PIL.Image.BICUBIC)
    image = np.array(image, dtype=np.float32)

    if FLAGS.preproc_type == "inception" :
        mean_array = [128, 128, 128] 
    elif FLAGS.preproc_type == "vgg" : 
        mean_array = [123.68, 116.78, 103.94] 
    else :
        mean_array = [0, 0, 0] 


    mean = np.array(mean_array, dtype=np.float32) #Mean Substraction
    mean.reshape(1, 1, 1, 3)
    image = image.reshape(-1,infer_image_height,infer_image_width,3)
    image = (image - mean) 
    with open("input_data.y",'wb') as file:
    inDtaUint8 = image.astype(np.int8)  
    inDtaUint8 = inDtaUint8.transpose([0,3,1,2])
    inDtaUint8 = np.asarray(inDtaUint8, order="C")      
    file.write(inDtaUint8)   
    if FLAGS.preproc_type == "inception" :
        image = image / 128.0

    tf.logging.info('Evaluating %s' % graph_filename)
    with tf.Session() as sess:
    input = graph.get_operation_by_name(FLAGS.input_node_names).outputs[0]
    #predictions = graph.get_operation_by_name("jacintonet_v1_11_custom_stride/conv1a/Pad").outputs[0]
    predictions, nodes, node_names = all_nodes()
    init = tf.global_variables_initializer()
    sess.run(init)      
    activations = sess.run(predictions, feed_dict={input:image})

    if(FLAGS.write_activations):   
    print('Writing')
    for act, node in zip(activations, nodes):
        print(node.name, act.shape)  
        act_shape = act.shape
        file_name = node.name+str(act.shape)
        file_name = file_name.replace("/","_").replace("(","_").replace(")","").replace(", ","x").replace(",","x").replace(" ","_")+".y"
        file_name = "trace/"+file_name
        mx = max(act.ravel())
        mn = min(act.ravel())
        rng = max(np.abs(mx), np.abs(mn))
        scale = 127/rng if rng!=0 else 0
        print(file_name, act_shape, mn, mx, scale)
        with open(file_name+"float.bin",'wb') as file:
            act_float = act.astype(np.float32)  
            if len(act.shape)==4:
                act_float = act_float.transpose([0,3,1,2])
            act_float = np.asarray(act_float, order="C")      
            file.write(act_float)
            file.close()

        with open(file_name,'wb') as file:
        if node.name!='input_ignore':
            #act = np.round(act * scale) + 128
            act = np.round(act * scale)
        act = act.astype(np.uint8)  
        if len(act.shape)==4:
            act = act.transpose([0,3,1,2])
        act = np.asarray(act, order="C")      
        file.write(act)
        #print(act)

    idx = 0
    for act, node in zip(activations, nodes):
    print(node.name)
    if node.name == FLAGS.output_node_names :
        act_idx = idx
    idx += 1
    inferred_label = np.argmax(activations[act_idx])
    inferred_label_top5 = np.argsort(activations[act_idx].ravel())[::-1][0:5]

    print(inferred_label)
    print(inferred_label_top5)



def crop_image_by_factor(image, factor=0.875): 
width  = image.size[0]
height = image.size[1]

crop_width  =  width*factor
crop_height =  height*factor
half_the_width = image.size[0] / 2
half_the_height = image.size[1] / 2

image = image.crop((half_the_width - crop_width/2,
    half_the_height - crop_height/2,
    half_the_width + crop_width/2,
    half_the_height + crop_height/2))
return image




def main(_):
    infer_frozen_graph(FLAGS.graph_path)

if __name__ == '__main__':
tf.app.run()

Example script to Reference output from tensorFlow for TFLite model

from __future__ import absolute_import, division, print_function
import tensorflow as tf
import numpy as np
import PIL
import PIL.Image as Image
import matplotlib.pyplot as plt

tflite_mnist_model = '2019-08-15_20-04-59unet_model.tflite'
interpreter = tf.lite.Interpreter(model_path=tflite_mnist_model)
interpreter.allocate_tensors()

print("\nInput Tensors")
print(interpreter.get_input_details())
print("\nOutput Tensors")
print(interpreter.get_output_details())

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
input_shape = input_details[0]['shape']

image = Image.open('.images/american_bulldog_124.jpg')
image = image.convert('RGB')  
image = image.resize((input_shape[2],input_shape[1]),resample=PIL.Image.BICUBIC)
image = np.array(image, dtype=np.float32)
image = image.reshape(-1,input_shape[2],input_shape[1],3)
image = image/128.0 - 1

interpreter.set_tensor(input_details[0]['index'], image)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
pred_mask = output_data
pred_mask = pred_mask[..., tf.newaxis]

• In case of TF lite user can not take generate output of any intermediate layers traces. Only final output layers canb be predicted uing TFLite model
• For getting layer levle traces, user need to make TFLite model by making intermeditate layers as output.
• User can refer code to make a ny layer in keras model as output layer

outputs = [model.get_layer("conv2d_4").output, model.get_layer("conv2d_5").output]
model_infer = tf.keras.Model(inputs=model.inputs, outputs=outputs)