RAISEuP Fellow – Matt Harrison (CHaSE Manufacturing)

RAISEuP Fellow – Matt Harrison

Description & Goals

The Applied Analytics and Emerging Technology Lab partnered with Matt Harrison from CHASE Manufacturing to prototype a quality measuring system that integrates computer vision with machine learning and image quality analysis to flag low levels of quality in their manufacturing process.  Specifically, we focused on the color or stain applied to wood products going into components for high end RVs.  Our goals were to create a prototype workstation that could be deployed in the manufacturing plant with the following high level objectives:

  • Raise the level of consistent quality of products produced within CHaSE Mfg.
  • Lower the burden on our manufacturing personnel and management.
  • The objective is to implement artificial intelligence to augment, not replace our workforce.
  • Lowering the cost of deploying this technology by using off the shelf maker materials versus purchasing a dedicated platform.
  • To create a framework to support developing cutting edge AI assisted edge devices whose purpose is to help with providing manufacturing personnel support in reducing the decision space. 

Our initial design included running pre-trained data models on a Jetson Edge Device with ODOO as a backend administration system:

Software on Edge Device
  1. An AI Edge device that is using computer vision to qualify products by implementing techniques including unsupervised machine learning and mask generation.
  2. The software receives and interprets visual data captured by an external camera and applies the unsupervised machine learning techniques stated above.
  3. Some input data may also be received from other sensors within the edge device (e.g., temperature, humidity)
  4. The output of machine learning then determines current state or measurements is then stored and available for a logic engine to trigger any immediate subsequent actions such as turn light indicator to red for low quality result(s).  Other parts of results may be stored for deeper analysis.
Hardware on Edge Device
  1. A Jetson nano/xavier AI edge computer with peripherals including a tower light that is used as indicator light for quality level of analysis result of image processing machine learning algorithm(s) (green: high quality, yellow: low quality, a red indicator would be indication for further inspection of results), foot pedal switch for dismissing yellow or red indicator. The device will also employ temperature and humidity sensor to gather data to be used to indicate if any adjustments to the environment are needed.  This data will be available for analysis.
  2. A containerized application that contains the run-time environment for the software described above that connects with IoT server to receive its configuration and serve the configuration details.  The IoT server will be able to connect to a variety of devices that may perform a variety of tasks (some not stated within this scope).
    1. This can be several different configuration details like:
      1. RTSP Camera Stream with Model Serving.
      2. RFID endpoint for relating realtime antenna data of cart location and applicable products on carts with model predictions.
        1. All CHaSE Finish Carts are labeled with RFID tags. 
        2. Most All CHaSE Products will be labeled wtih RFID tags that can identify the finish specification for the products.
        3. As the carts move beneath the camera this tool will collect product data that includes what the target finish is so that if there are any conflicts between the model predicted color class and the class that is read in through the RFID antenna a Red Light indicator turned on.
      3. Indicator Lights for User Notification.
        1. Red/Yellow/Green.
      4. Foot pedal switch for User Interface.
        1. To dismiss the Yellow/Red warnings.
      5. Temperature and Humidity Sensor.
        1. To provide environment data for analysis.
Connection to IoT server running within ERP system (ODOO)
  1. Odoo addon, IoT Server that
    1. Serves the IoT Edge devices their device specific configuration.
    2. Receives the data from the edge devices to further notify employees, trigger events in the system and reporting.


Outcomes & Technology

Tower Light Control Plugin

Plugin designed for managing tower lights using a GPIO board. The plugin runs within a Docker container and communicates with the lights via MQTT messages. It includes a Docker Compose configuration for local development, allowing you to easily set up and test the plugin. The plugin code is written in Python and includes functionality to control red, yellow, and green tower lights based on MQTT messages. You can find detailed usage instructions in the README.md file of the Github branch here.

In summary, this project provides a Dockerized solution for controlling tower lights using GPIO pins and MQTT communication. It simplifies development with clear instructions in the README.md file and includes necessary configurations for running the plugin locally in a development environment.
Implementation demo video recording for the plugin can be found here.

Storage Management Plugin

This plugin offers a comprehensive storage management system that includes a plugin designed for cleaning up storage space based on predefined thresholds. The system leverages the MQTT protocol for communication and employs a Dockerized worker to execute storage cleanup tasks periodically. The primary components of the project consist of Mosquitto, an MQTT broker for communication; a Python-based plugin responsible for storage space management; and a Dockerized Python worker responsible for executing storage cleanup tasks while interacting with both the plugin and the MQTT broker.

You can find detailed usage instructions in the README.md file of the Github branch here. The system will automatically monitor storage space and trigger cleanup processes when it falls below the specified threshold. You can customize the storage management behavior by modifying the StoragePlugin class in the plugin/storage.py file.

The project is designed to be configurable, and you can adjust various settings, such as storage thresholds and cleanup frequencies, in the respective Python files.

Color Variation in Sequence: A Case Study

This study serves as an evaluation of the color histogram developed as part of a computer vision color processing pipeline. The histogram design is omitted here, instead discussed in the pipeline documentation.

The goal of the study is to better understand how the color profile (histogram) of a fixed object varies throughout a sequence of images. This analysis is primarily qualitative, informed by the quantitative work carried out in the accompanying Jupyter notebook.

Documentation & Demos