Skip to main content Link Menu Expand (external link) Document Search Copy Copied
AWS Solutions Library Samples

Overview

This guidance recommends using IoT Device Shadows from AWS IoT Core to configure a physical device remotely. These types of operations are called “configurations”. IoT Device Shadows will provide the infrastructure (e.g. reserved MQTT topics) to manage the desired, reported, and delta states of the device (even when the device is offline). Example configurations include video encoder configurations, analytics module configurations, and imaging configurations.

This capability requires edge and cloud components of the guidance to work together. The edge implementation provided in the guidance supports video encoder configurations of a device. For cloud, customers need to implement their own by following the provided Java reference sample code or utilize the guidance provided UpdateDeviceShadow API.

Customers can add more types of configurations by mimicking the edge implementation for video encoder configurations and the cloud Java reference sample code provided in Run. See Instructions to implement a new type of configuration for instructions on adding new configuration types.

Assumptions:

  1. Device has been added to SpecialGroup_EnabledState. If the cloud formation deployment was succesfull and the device was created succesfully using API StartCreateDevice, by default the device should already be in SpecialGroup_EnabledState. If the device is not in SpecialGroup_EnabledState, see State Management (wip-still need to add url once published) on how to move the device to SpecialGroup_EnabledState.
  2. The computer used for development is x86_64.
  3. ONVIF device supports SetVideoEncoderConfiguration and GetVideoEncoderConfigurations APIs

Pre-requisites:

  1. Use a binary that is compiled with the configuration capability. run CLI:

    a. for armv7

     cross build --features config --target armv7-unknown-linux-gnueabihf --release

    If successfully compiled, the binary can be found at guidance-for-video-analytics-infrastructure-on-aws/source/edge/target/armv7-unknown-linux-gnueabihf/release/edge-process. Instructions to install cross-rs can be found at cross.

    b. for x86_64

     cargo build --features config

    If successfully compiled, the binary can be found at guidance-for-video-analytics-infrastructure-on-aws/source/edge/target/debug/edge-process

  2. An ONVIF compliant device to run binary on.

Instructions to run and verify the capability

Run:

  1. Run the binary (can also be run after Java code is executed)
     ./edge-process -c <path to config yaml file>

  2. Execute the Java code below

    [Tip] Customers can choose to 1/embed the Java code provided here to an existing application or 2/implement the Java code following this guidance’s server-less pattern of APIGW → Lambda for synchronous operation or APIGW -> Lambda -> DDB -> StepFunction for asynchronous operations.

import software.amazon.awssdk.core.SdkBytes;
import software.amazon.awssdk.services.iot.IotClient;
import software.amazon.awssdk.services.iot.model.*;
import software.amazon.awssdk.services.iotdataplane.IotDataPlaneClient;
import software.amazon.awssdk.services.iotdataplane.model.IotDataPlaneException;
import software.amazon.awssdk.services.iotdataplane.model.UpdateThingShadowRequest;
import com.google.gson.JsonObject;

public void updateShadowForConfiguration(
    final String thingName,
    final String region,
    final String accountId,
    final IotClient iotClient,
    final IotDataPlaneClient iotDataPlaneClient
) {
    // Validate if device exists
    System.out.println("Checking if device exist.");
    try {
        iotClient.describeThing(DescribeThingRequest.builder().thingName(thingName).build());
    } catch (ResourceNotFoundException e){
        System.out.printf("Thing %s doesn't exist.%n", thingName);
        return;
    }

    System.out.println("Updating IoT Device Shadow");

    // Set this value to the shadow name
    String shadowName = "videoEncoder";

    // Creating a json object with the desired configuration
    // Recommended to validate desired configuration input if not hardcoded
    JsonObject configurationJson = new JsonObject();
    JsonObject videoSettings = new JsonObject();
    JsonObject vec1 = new JsonObject();
    vec1.addProperty("name", "GuidanceConfiguration");
    vec1.addProperty("bitRateType", "CBR");
    vec1.addProperty("bitRate", 512);
    vec1.addProperty("frameRate", 15);
    vec1.addProperty("gopRange", 30);
    vec1.addProperty("resolution", "1920x1080");
    videoSettings.add("vec1", vec1);
    configurationJson.add("videoSettings", videoSettings);

    // Create desired state
    JsonObject desired = new JsonObject();
    desired.add("desired", configurationJson);

    // Create state document
    JsonObject messagePayload = new JsonObject();
    messagePayload.add("state", desired);

    UpdateThingShadowRequest updateThingShadowRequest = UpdateThingShadowRequest.builder()
        .thingName(thingName)
        .shadowName(shadowName)
        .payload(SdkBytes.fromUtf8String(messagePayload.toString()))
        .build();

    try {
        iotDataPlaneClient.updateThingShadow(updateThingShadowRequest);
    } catch (IotDataPlaneException e) {
        System.out.println(e);
    }
}

For validating configuration input structure, we recommend using JSON schema. Example:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "$id": "#ConfigurationPayloadSchema.json",
    "type": "object",
    "anyOf": [
        {"required": ["videoSettings"]}
    ],
    "additionalProperties": false,
    "properties": {
        "videoSettings": {
            "type": "object",
            "anyOf": [
                {"required": ["name"]},
                {"required": ["codec"]},
                {"required": ["bitRateType"]},
                {"required": ["bitRate"]},
                {"required": ["frameRate"]},
                {"required": ["gopRange"]},
                {"required": ["resolution"]}
            ],
            "additionalProperties": false,
            "properties": {
                "name": {
                    "type": "string"
                },
                    "codec": {
                    "enum": ["H264"]
                },
                "bitRateType": {
                    "enum": ["CBR", "VBR"]
                },
                "bitRate": {
                    "type": "integer",
                    "minimum": 100,
                    "maximum": 8000
                },
                "frameRate": {
                    "type": "integer",
                    "minimum": 1,
                    "maximum": 30
                },
                "gopRange": {
                    "type": "integer",
                    "minimum": 1,
                    "maximum": 120
                },
                    "resolution": {
                    "type": "string",
                    "enum": ["1920x1080", "1280x1024", "1280x720", "720x480", "640x480", "320x240"]
                }
            }
        }
    }
}

Verify:

  1. Sign in to AWS IoT console. In the menu bar on the left under Manage, find All devices > Things. Search the device using deviceId. In the menu bar in the middle under Device Shadows, find the “videoEncoder” shadow. The desired state of the state document should contain the videoSettings initialized in the Java code.
  2. After Java code is executed, if LOG_LEVEL on device is set to INFO, DEBUG, or TRACE, customers should see below logs printed on the device terminal:
      INFO  Successfully set video encoder configuration

    printed in the terminal on device. Alternatively, customers can also find the logs in cloud trail if environement variable LOG_SYNC environment variable is set to TRUE.

  3. Sign in to AWS IoT console, and find the “videoEncoder” shadow (same steps as 1). The reported state of the state document should match the videoSettings initialized in the Java code. The delta state of the state document should disappear (no diff between desired and reported). Note: all 3 events may occur within a very short duration, so you may not observe the delta state between steps 1 and 2.

Instructions to implement a new type of configuration

Using AI (analytics module) configurations (supported by ONVIF compliant devices) as an example. Ref: ONVIF GetAnalyticsModules and ModifyAnalyticsModules APIs.

Edge Rust code

  1. Add the ONVIF API request and response Rust struct to source/edge/crates/onvif-client/src/wsdl_rs/analytics20.rs. See this on how to create Rust structs for ONVIF APIs.

  2. Add new functions to OnvifClient as an interface for the ONVIF API that supports setting and getting the new configuration.

  3. Add functions to DeviceStateModel.

  4. Add functions to IotClientManager for checking if an incoming MQTT message is for the new configuration.

  5. Add logic in source/edge/crates/edge-process/src/connections/aws_iot.rs to handle sending the configuration through a new tokio channel if the incoming mqtt message is for AI configurations.

pub async fn setup_and_start_iot_event_loop(
    ...
    video_config_tx: Sender<Value>,
    ai_config_tx: Sender<Value>,
    ...
) -> anyhow::Result<JoinHandle<()>> {
    ...
    let handle = tokio::spawn(async move {
        ...
        loop {
            ...
            loop {
                select! {
                    ...
                    Ok(msg_in) = iot_client.recv() => {
                        if let Some(message) = pub_sub_client_manager.received_ai_settings_message(msg_in.as_ref()) {
                            info!("Ai settings message received {:?}", message);
                            let _ = ai_config_tx.send(message).await;
                        }
                    }
                }
            }
        }
    });
    ...
}

fn trigger_shadows() {
    ...
    let shadows = &[PROVISION_SHADOW_NAME, VIDEO_ENCODER_SHADOW_NAME, AI_SHADOW_NAME];
    ...
}
  1. Add logic in source/edge/crates/edge-process/src/bin/main.rs to handle the new configuration type. Example for AI configurations:
     ...
 let (ai_config_tx, mut ai_config_rx) = channel::<Value>(BUFFER_SIZE);

 let _config_join_handle = tokio::spawn(async move {
     loop {
         select! {
             ...
             Some(ai_settings) = ai_config_rx.recv() => {
                 let _res = model.set_ai_configuration(ai_settings).await;
                 let ai_settings_res = model.get_ai_configurations().await;
                 match ai_settings_res {
                     Ok(ai_settings) => iot_shadow_client_ai_config.update_reported_state(ai_settings).await.expect("Failed to update configuration shadow"),
                     Err(_) => error!("Could not retrieve ai settings after configuration"),
                 }
             }
             ...
         }
     }
 });

 let ai_config_tx_clone = ai_config_tx.clone();
 ...
 let _iot_loop_handle = setup_and_start_iot_event_loop(
     ...
     video_config_tx_clone,
     ai_config_tx_clone,
     ...
 )
 .await?;
 ...
  2. Compile the binary and test it on device.

Cloud java code

  1. Assign a device shadow for the new configuration. This value should match constant AI_SHADOW_NAME in the edge code. Note: please read over IoT Fleet Indexing quotas and IoT Core Device Shadow Service quotas when determining whether you should reuse an existing shadow or add a new shadow for the configuration. Notable limits:
    1. Maximum of 10 names in the named shadow names filter for IoT fleet indexing
    2. Maximum shadow document size of 8 Kilobytes
  2. To add a new configuration category to the json schema for input validation, add a new item in “anyOf” and “properties”. Example for aiSettings:
     ...
 "anyOf": [
     {"required": ["videoSettings"]},
     {"required": ["aiSettings"]}
 ],
 ...
 "properties": {
     "videoSettings": {
         ...
     },
     "aiSettings": {
         ...
     }
 }
 ...
  3. To modify GetDevice to return a new type of configuration, modify getDevice in VideoAnalyticsDeviceManagementControlPlane/src/main/java/com/amazonaws/videoanalytics/devicemanagement/dependency/iot/IotService.java to read from the appropriate device shadow and add the new configurtaion to deviceSettings. Example for aiSettings:
     // Retrieving aiSettings in ai iot shadow.
 JSONObject getThingShadowResponseAiSettings;
 try {
     getThingShadowResponseAiSettings = getThingShadow(deviceIdentifier, AI_SHADOW_NAME);
 } catch (software.amazon.awssdk.services.iotdataplane.model.ResourceNotFoundException e) {
     getThingShadowResponseAiSettings = new JSONObject();
 }
 JSONObject shadowStateReportedAiSettings = getThingShadowResponseAiSettings
         .optJSONObject(SHADOW_STATE_KEY, new JSONObject())
         .optJSONObject(SHADOW_REPORTED_KEY);

 if (shadowStateReportedAiSettings != null && !shadowStateReportedAiSettings.isEmpty()) {
     String aiSettingsStr = shadowStateReportedAiSettings.optString(AI_SETTINGS);
     deviceSettings.put(AI_SETTINGS, aiSettingsStr);
 }