1.2. Model BasicsΒΆ

Machine learning models usually take input data in tensor (multi-dimensional array) format and produce output data in tensor format. Before compiling and inferencing a model using TVM, it is recommended that you understand the basic requirements for your model:

  • model semantics, including the meanings of input and output

  • input name, shape

  • output shape

  • sample of input data

  • expected output data corresponding to the sample input data

Experimenting with pre-processing input data, running inference, and post-processing input data can be accomplished even without a TVM setup and TI devices. For example, if you have an ONNX model, you can write a Python script like the following to import the onnxruntime package, pre-process input data, run inferencing, post-process output data, and verify the results.

import onnxruntime
model_file = ...
input_name = ...
input_data = ...
sess = onnxruntime.InferenceSession(model_file)
output = sess.run([], {input_name : input_data})

In the topics that follow, we assume you understand the basics of your model, so we focus on TVM-related aspects. Although we show data pre-processing, model input names and shapes, and data post-processing in our examples, the details about these steps are likely to be different for your model, so it is your responsibility to set them correctly.