TensorFlow Lite On ESP32

What is TensorFlow Lite?

Steps to Deploy on ESP32

Train a Model

• Command: pip install tensorflow
• Example code:

  import numpy as np
  import tensorflow as tf
  from tensorflow import keras
  
  angles = np.random.uniform(low=-np.pi, high=np.pi, size=(1000, 1))
  sines = np.sin(angles)
  model = keras.Sequential([keras.layers.Dense(1, input_shape=(1,))])
  model.compile(optimizer='adam', loss='mean_squared_error')
  model.fit(angles, sines, epochs=100, verbose=0)
  model.save('sine_model.h5')
  
  
  

  Copy Code ✔ Copied!

Convert to TensorFlow Lite

• Install tool: pip install tflite
• Convert: tflite_convert --keras_model_file sine_model.h5 --output_file sine_model.tflite

Set Up ESP32

• Add URL: http://dl.espressif.com/dl/package_esp32_index.json in Preferences.
• Install via Board Manager.
• Add tflite-micro library from GitHub repository to Arduino libraries.

Integrate and Run

• Code:

  #include 
  #include "tflite-micro.h"
  #include "sine_model.h"
  
  tflite::MicroErrorReporter micro_error_reporter;
  tflite::ErrorReporter* error_reporter = µ_error_reporter;
  const unsigned char* model_data = sine_model;
  const size_t model_size = sine_model_len;
  const tflite::Model* model = tflite::GetModel(model_data);
  tflite::AllOpsResolver resolver;
  constexpr int kTensorArenaSize = 2000;
  uint8_t tensor_arena[kTensorArenaSize];
  tflite::MicroInterpreter interpreter(model, resolver, tensor_arena, kTensorArenaSize, error_reporter);
  
  float randomFloat(float min, float max) {
      return min + (max - min) * (float)random(0, 10000) / 10000.0;
  }
  
  void setup() {
      Serial.begin(115200);
      interpreter.AllocateTensors();
  }
  
  void loop() {
      float angle = randomFloat(-3.1416, 3.1416);
      TfLiteTensor* input = interpreter.input(0);
      input->data.f[0] = angle;
      if (interpreter.Invoke() != kTfLiteOk) {
          error_reporter->Report("Failed to invoke model");
          return;
      }
      TfLiteTensor* output = interpreter.output(0);
      float prediction = output->data.f[0];
      Serial.print("Angle: ");
      Serial.print(angle);
      Serial.print(", Predicted sine: ");
      Serial.println(prediction);
      delay(1000);
  }
  
  
  

  Copy Code ✔ Copied!

• Upload to ESP32, open Serial Monitor to see results.

Unexpected Detail

Deploying TensorFlow Lite on ESP32

Background and Context

Detailed Steps for Deployment

Step 1: Training a Machine Learning Model

• Installation: Install TensorFlow with the command pip install tensorflow.
• Training Code: The following code generates a dataset and trains the model:

  import numpy as np
  import tensorflow as tf
  from tensorflow import keras
  
  # Generate dataset
  angles = np.random.uniform(low=-np.pi, high=np.pi, size=(1000, 1))
  sines = np.sin(angles)
  
  # Create a simple neural network model
  model = keras.Sequential([
      keras.layers.Dense(1, input_shape=(1,))
  ])
  
  # Compile the model
  model.compile(optimizer='adam', loss='mean_squared_error')
  
  # Train the model
  model.fit(angles, sines, epochs=100, verbose=0)
  
  # Save the model
  model.save('sine_model.h5')
  
  
  

  Copy Code ✔ Copied!

• Notes: This model is simple, with one dense layer, suitable for the ESP32's limited resources. The dataset consists of 1000 random angles and their sine values, trained for 100 epochs with the Adam optimizer and mean squared error loss.

Step 2: Converting the Model to TensorFlow Lite Format

• Installation: Install the conversion tool with pip install tflite.
• Conversion Command: Use the following to convert:

tflite_convert --keras_model_file sine_model.h5 --output_file sine_model.tflite

• Details: The .tflite file is optimized for size and performance, crucial for microcontrollers like ESP32 with limited flash and RAM.

Step 3: Setting Up the ESP32 Development Environment

• ESP32 Board Support:
    Open Arduino IDE, go to File > Preferences, and add http://dl.espressif.com/dl/package_esp32_index.json to Additional Board Manager URLs.
    Go to Tools > Board > Board Manager, search for "esp32", and install the latest version.
• TensorFlow Lite Micro Library:
    Download from GitHub repository and add to Arduino libraries directory, or use Library Manager if available.
    Ensure compatibility with ESP32, as Espressif provides specific support for tflite-micro.

Step 4: Integrating the Model into ESP32 Code

• Convert to C Array: Use xxd -i sine_model.tflite > sine_model.h to generate a header file containing the model data as a C array, typically:

#include 
#include "tflite-micro.h"
#include "sine_model.h"

tflite::MicroErrorReporter micro_error_reporter;
tflite::ErrorReporter* error_reporter = µ_error_reporter;
const unsigned char* model_data = sine_model;
const size_t model_size = sine_model_len;
const tflite::Model* model = tflite::GetModel(model_data);
tflite::AllOpsResolver resolver;
constexpr int kTensorArenaSize = 2000;
uint8_t tensor_arena[kTensorArenaSize];
tflite::MicroInterpreter interpreter(model, resolver, tensor_arena, kTensorArenaSize, error_reporter);

float randomFloat(float min, float max) {
    return min + (max - min) * (float)random(0, 10000) / 10000.0;
}

void setup() {
    Serial.begin(115200);
    interpreter.AllocateTensors();
}

void loop() {
    float angle = randomFloat(-3.1416, 3.1416);
    TfLiteTensor* input = interpreter.input(0);
    input->data.f[0] = angle;
    if (interpreter.Invoke() != kTfLiteOk) {
        error_reporter->Report("Failed to invoke model");
        return;
    }
    TfLiteTensor* output = interpreter.output(0);
    float prediction = output->data.f[0];
    Serial.print("Angle: ");
    Serial.print(angle);
    Serial.print(", Predicted sine: ");
    Serial.println(prediction);
    delay(1000);
}

• Notes: The randomFloat function generates random angles for testing, using a simple scaling method. The tensor arena size (2000 bytes) may need adjustment for larger models, given ESP32's 520 KB SRAM.

Step 5: Upload and Run the Code

• Procedure: Ensure the board is selected correctly (e.g., ESP32 Dev Module), compile, and upload. Open Serial Monitor at 115200 baud to see outputs.

Considerations and Potential Challenges

• Memory Constraints: ESP32 has 520 KB SRAM and limited flash, so ensure the model and tensor arena fit within these limits. Optimize by reducing model complexity or increasing tensor arena size if needed.
• Library Compatibility: Ensure the tflite-micro library version matches ESP32 requirements, as Espressif provides specific support.
• Random Number Generation: The randomFloat function uses a simple scaling method, which may not be precise for all applications. Consider using a more robust random number generator for production.

Unexpected Observation

Conclusion

Key Citations

• Easy TinyML on ESP32 and Arduino
• TensorFlow Lite Micro for Espressif Chipsets
• First steps with ESP32 and TensorFlow Lite for Microcontrollers
• TensorFlow Lite TinyML for ESP32