We started this series with a look at operators and kernels, the “instructions” used by models and the implementation of those instructions on the available hardware. We then explored the computation graph, which defines the sequence of operators for a given model, and explored how different model formats opt to include the explicit computation graph in the distributed file, or defer it to the inference application. With tflite-micro and the .| danielmangum.com
In our last post we explored operators and kernels in Tensorflow Lite, and how the ability to swap out kernels depending on the hardware capabilities available can lead to dramatic performance improvements when performing inference. We made an analogy of operators to instruction set architectures (ISAs), and kernels to the hardware implementation of instructions in a processor. Just like in traditional computer programs, the sequence of instructions in a model needs to be encoded and distribu...| danielmangum.com