Tried OTA (Over The Air) firmware update with Amazon FreeRTOS

• English
• 日本語

What is OTA Update?

OTA (Over The Air) is a function to update to the latest firmware wirelessly.

In the case of IoT, a large number of terminals may be placed here and there, and it would be very inefficient if a person had to go around each terminal to update the program when replacing it.

For this reason, some OS for the IoT have an OTA Update function built in.
It is also necessary to have a mechanism to verify that the data received is correct in order to safely deliver the update data and prevent malicious programs from being executed.

Amazon FreeRTOS OTA Update uses a private key and code signing certificate to authenticate the connection, thus ensuring a secure connection.

Also, since it uses the MQTT (Message Queuing Telemetry Transport) protocol over TLS (Transport Layer Security), it uses TLS device authentication to make sure that the destination device is not a fake.

In the previous article, I installed Amazon FreeRTOS on ESP-32 DevKitC and ran a sample program that sends messages to AWS IoT Core.

For details, please refer to the previous article, but I added the “OTA Updates” library when I selected the demo project.

Adding this library makes it relatively easy to implement OTA Update.

Translated with www.DeepL.com/Translator (free version)

Overall configuration diagram

The OTA Update mechanism of Amazon FreeRTOS is as follows.

1. Create a private key and code-signing certificate in advance and register them with AWS Certificate Manager.
2. Save the modified updated firmware compiled on PC to a disk (s3) in the AWS cloud.
3. From the AWS console, specify the target device (or group) and create a job.
4. OTA Update agent running on ESP32 will download the latest module from s3 via MQTT protocol and replace the firmware.

Advance preparation

The AWS CLI (Command Line Interface) is required to import the code-signing certificate into AWS Certificate Manager, so install it from this page in advance.

I downloaded 64-bit Windows installer.

Preparing the initial firmware

Prepare an ESP32 DevkitC running a pre-update version of the firmware with a certificate for signature verification and the OTA Update agent.

The general steps are as follows.

1. Create an s3 bucket
2. Create roles and policies
3. Create a user and attach a policy
4. Create a private key
5. Create a code signing certificate
6. Import into AWS Certificate Manager
7. Grant access to FreeRTOS code signing
8. Convert and copy certificates
9. Compile & Flush (Write)

It is assumed that the steps in this article have been completed in advance, including downloading FreeRTOS from the AWS console, building the ESP-IDF environment on a PC, and building the CMake environment.

Directory Structure

Following the steps in the previous article, I downloaded Amazon FreeRTOS to PC (Windows 10) and extracted it to the following directory.

I unzipped it to D:\GoogleDrive\M2B\FreeRTOS, but you can use any directory you like.

├─d:
│  │      
│  ├─GoogleDriveD
│  │  │
│  │  ├──M2B
│  │  │  │
│  │  │  ├──build　・・・　For storing build files
│  │  │  │
│  │  │  ├──FreeRTOS ・・・ Unzip it here.
│  │  │  │  │
│  │  │  │  ├──demos　・・・Demo Application
│  │  │  │  │
│  │  │  │  ├──device
│  │  │  │  │
│  │  │  │  ├──freertos_kernel　・・・ FreeRTOS kernel
│  │  │  │  │
│  │  │  │  ├──libraries
│  │  │  │  │
│  │  │  │  ├──tests
│  │  │  │  │
│  │  │  │  ├──tools　・・・　Conversion tools for authentication files, etc.
│  │  │  │  │
│  │  │  │  ├──vendors　・・・ ESP-IDF
│  │  │  │  │

Create the s3 bucket

Create an area on AWS s3 to store the updated firmware.

From the AWS console, go to the s3 service and click the “Create Bucket” button.

• Bucket name: esp-wroom-32-ota-en
• AWS Region: US East(N. Virginia)us-east-1

and scroll down to the bottom.

The bucket name needs to be unique around the world.

If you run into the same name, add the date, etc. to make it a unique bucket name.

Check “Block all public access” and scroll down.

Create a separate policy and use that policy for access.

• Bucket versioning: Enable

Scroll down.

• Tags are not set (you may set them if necessary).
• Default encryption is disabled

Click on the “Create bucket” button.

The bucket has been created.

Creating Roles

The next step is to create a role that will be used for the OTA Update job.

This role will be used to update the firmware, so it will need access to Amazon FreeRTOS Update, IAM, and s3.

From the AWS console, go to the IAM service page and select the Roles.

Click the “Create role” button.

• Select type trusted entity: AWS Service

and scroll down.

Select IoT from the list of AWS services.

Select IoT from the list of use cases below, and click on the “Next Permissions” button.

The following policies are attached by default to the IoT selection.

• AWSIoTLogging
• AWSIoTRuleActions
• AWSIoTThingsRegistration

Click on the “Next Tags” button.

Click the “Next Review” button without setting any tags (you can set them if you want).

• Role name: esp-32-ota-role-en
• Description: Allows IoT to call AWS services on your behalf.

and click the “Create role” button.

A role was created.

Attaching Policies

Attaching a few policies.

Attach the AmazonFreeRTOSOTAUpdate policy to the role

Select the role esp-32-ota-role-en just created.

Click on the “Attach policies” button.

Enter “AmazonFreeRTOSOTAUpdate” in the Filter policies, search for it, check the policies that appear, and then click the “Attach policy” button.

AmazonFreeRTOSOTAUpdate policy has been attached.

Adding IAM permissions

Add permissions to IAM.

Click “+Add inline policy.”

Click on the JSON tab.

Paste the policy document below and click the “Review Policy” button.

{
    "Version": "2012-10-17",
    "Statement": [
      {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:PassRole"
            ],
            "Resource": "arn:aws:iam::Your AWS Account ID:role/esp-32-ota-role-en"
      }
    ]
}

• Name: esp-32-ota-iam-Policy-en

Click on the “Create policy” button.

An inline policy has been created and attached.

Adding permissions to Amazon S3

Let’s add more permissions to s3.

Note that if your Amazon S3 bucket name starts with “afr-ota”, the “AmazonFreeRTOSOTAUpdate” administrative policy that you just attached already contains the necessary permissions, so you don’t need to do the following.

I have set the s3 bucket name as “esp-wroom-32-ota-en”, so I had to add the policy.

Click “+Add inline policy”.

Click on the JSON tab.

Paste the policy document below and click the “Review policy” button.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObjectVersion",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::esp-wroom-32-ota-en/*"
            ]
        }
    ]
}

• Name: esp-32-s3-Policy-en

and click the “Create policy” button.

The policy has been created and attached.

Creating an OTA User

Create an OTA user.

This user will be used to run the OTA update, which will be run from Amazon FreeRTOS on ESP32, and must have the following permissions

• Access the S3 bucket where the firmware update (update) is stored.
• Access the certificates stored in AWS Certificate Manager.
• Access to AWS IoT MQTT-based file delivery capabilities
• Access FreeRTOS OTA Update
• Accessing AWS IoT Jobs
• Accessing IAM
• Accessing Code Signing for AWS IoT
• List FreeRTOS hardware platforms

From the AWS console, navigate to the IAM service and click the “Add users” button in the User menu.

• User name: esp32otaupdateen
• Programmatic access: Check

Click on the “Next Permissions” button.

Click the Next Tags button.

Do not add any tags (you may do so), and click the “Next Review” button.

“This user has no permissions” A warning message will appear, but click the “Create user” button.

The permissions will be added after this.

A user will be created and the Access key ID and Secret access key will be saved.

Creating & attaching a policy

Create and attach a policy.

Select esp32otaupdateen from the list of user names

Click on the “Add permissions” button.

“Attach existing policies directly” – > Click the “Create policy” button.

Select the JSON tab.

Paste the following policy code and click the “Next Tags” button.

{
    "Version":"2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListAllMyBuckets",
                "s3:CreateBucket",
                "s3:PutBucketVersioning",
                "s3:GetBucketLocation",
                "s3:GetObjectVersion",
                "s3:ListBucketVersions",
                "acm:ImportCertificate",
                "acm:ListCertificates",
                "iot:*",
                "iam:ListRoles",
                "freertos:ListHardwarePlatforms",
                "freertos:DescribeHardwarePlatform"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::esp-wroom-32-ota-en/*"
        },
        {   
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::AWS account ID:role/esp-32-ota-role-en"
        }
    ]
}

Do not add any tags (optional) and click the “Next Review” button.

• Name: esp32otaupdate-Policy-en

Scroll down.

Click on the “Create policy” button.

Go back to the previous screen and click the refresh icon, then search for the policy name just created “esp32otaupdate-Policy-en”, check it when it appears in the list, and click the “Next Review” button.

Click on the “Add permissions” button.

Added permissions.

Authentication during update

Here’s how to do it on a PC (Windows 10).

When downloading the latest firmware from the device (ESP32-DevKitC), I used the private key and code signing certificate to check if the image is correct.

In other words, the code signing certificate is created to prevent the device from mistakenly reflecting the malicious firmware.

Create cert_config.txt

Create a D:\GoogleDriveD\M2B\FreeRTOS\OTA_Cert\cert_config.txt file with the following contents using a text editor.

以下の内容で D:¥GoogleDriveD¥M2B¥FreeRTOS¥OTA_Cert¥cert_config.txt ファイルをテキストエディターで作成する。

[ req ]
prompt             = no
distinguished_name = my_dn
                    
[ my_dn ]
commonName = your mail address
                    
[ my_exts ]
keyUsage         = digitalSignature
extendedKeyUsage = codeSigning

Creating a Private Key

Create an ECDSA code-signed private key using the OpenSSL command from PowerShell on Windows 10.

Execute it in D:\GoogleDriveD\M2B\FreeRTOS\OTA_Cert>

openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -pkeyopt ec_param_enc:named_curve -outform PEM -out ecdsasigner.key

Creating a code signing certificate

Based on the private key (ecsdsasigner.key) that I created earlier, I created a code signing certificate (ecsdsasigner.crt) for ECDSA using the following command.

openssl req -new -x509 -config cert_config.txt -extensions my_exts -nodes -days 365 -key ecdsasigner.key -out ecdsasigner.crt

Importing into AWS Certificate Manager

Import the code signing certificate, private key, and certificate chain into AWS Certificate Manager using AWS CLI.

In advance, I used the aws configure command to set the access key ID, secret access key, region, etc. for the OTA user (esp32otaupdateen) that I just created.

Note that once the aws configure command is configured, the previous settings are saved even if the computer is rebooted.

aws configure

• AWS Access Key ID [None]: Specify the access key ID when creating a user
• AWS Secret Access Key [None]: Specify the secret access key when creating a user
• Default region name [None]: us-east-1
• Default output format [None]: json

Then import the signing certificate, private key, etc. into AWS Certificate Manager using the following command, and save the certificate’s ARN (Amazon Resource Name) as it is returned.

aws acm import-certificate --certificate fileb://ecdsasigner.crt --private-key fileb://ecdsasigner.key

Allow access to FreeRTOS code signing

When updating firmware, digital signatures are used to check the authenticity of the update data.

In order to do this, the account for the update (esp32otaupdateen) must be allowed access to FreeRTOS code signing (Code Sigining for AWS IoT).

Select esp32otaupdateen from the IAM user in the AWS console.

Click on the “Add permissions” button.

“Attach existing policies directly” -> Click the “Create policy” button.

A new tab will open, click on “JSON Tab”.

Paste the following into the box and click the “Next Tags” button.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "signer:*"
      ],
      "Resource": "*"
    }
  ]
}

Do not add any tags (optional) and click the “Next Review” button.

• Name: esp32otaupdate-signer-Policy-en

and click the “Create policy” button.

Go back to the previous tab and click on the Refresh Icon, the policy you just created (esp32otaupdate-signer-Policy-en) will be displayed, check it and click on the “Next Review” button.

Click on the “Add permissions” button.

Policies have been added.

Demo program for OTA Update

Edit the following file to make the OTA Update demo program work.

D:\GoogleDriveD\M2B\FreeRTOS\vendors\espressif\boards\esp32\aws_demos\config_files\aws_demo_config.h

In the above file

#define CONFIG_OTA_UPDATE_DEMO_ENABLED

Define.

This AWS sample program is designed to work differently depending on the constants that are defined.

D:\GoogleDriveD\M2B\FreeRTOS\demos\include\iot_demo_runner.h

The demo program to be executed is sorted in the above file.

• In line 111, it checks if CONFIG_OTA_UPDATE_DEMO_ENABLED is defined or not.

Copy the certificate

Copy the contents of the code signing certificate (ecsdsasigner.crt) to the Paste code signing certificate here. location in the following file.

D:\GoogleDriveD\M2B\FreeRTOS\demos\include\aws_ota_codesigner_certificate.h

It is important to note that if you copy and paste the file as is, it does not match the format of the C language, and the certificate part will not be recognized during the Build process, resulting in an error due to the reason that “error missing terminating”.

Converting a certificate

Double-click on the following file (HTML) to open it in your browser for preliminary conversion with the certificate conversion tool.

D:\GoogleDriveD\M2B\FreeRTOS\tools\certificate_configuration\PEMfileToCString.html

Select the code signing certificate (ecsdsasigner.crt) from the “Choose File” button, and then click the “Display formatted PEM string …” button.

The certificate with the characters at the end of the line converted for C Language will be displayed.

Copy and paste it into the file( aws_ota_codesigner_certificate.h).

Paste it into aws_ota_codesigner_certificate.h and save it.

Note that the “;” (semicolon) at the end is required.

CMake

Start the power shell, go to the directory where FreeRTOS installed, and create the build file with the following command.

d:
cd .\GoogleDriveD\M2B\FreeRTOS\
cmake -DVENDOR=espressif -DBOARD=esp32_devkitc -DCOMPILER=xtensa-esp32 -G Ninja -S . -B ..\build\

See the previous article for an explanation of the parameters of cmake.

Build

Next, compile the Build file with the following command.

cmake --build ..\build\

Flush

After successfully compiling and creating aws_demos.bin file, connect the ESP32-DevkitC to the PC via USB and write.

.\vendors\espressif\esp-idf\tools\idf.py flash -b 115200 -B ..\build\

Please refer to the previous article for an explanation of the parameters.

When the new window opens and the writing is finished, the initial firmware preparation is complete.

Updating the firmware

The next step is to verify that the firmware is updated by OTA.

Running ESP32-DevkitC

Run ESP32-DevkitC with the firmware before the update (Ver 092) installed and powered from the micro USB Type-B connector.

Updating the version file

Open the following file to change the current version.

D:\GoogleDriveD\M2B\FreeRTOS\demos\include\aws_application_version.h

The current version was 092, so I changed it to 093 and saved it.

Re-Build

Then, re-Build with the following command to create aws_demos.bin of Ver 093.

cmake --build ..\build\

Job registration to AWS

Create a job for OTA update.

From the AWS console, go to the IoT Core service page and click the Manage -> Jobs -> “Create Job” button.

Select “Create FreeRTOS OTA update job” and click the “Next” button.

OTA Job Properties

• Job name: esp32-ota-updatejob-en

Click the “Next” button without setting the description and tags (optional).

Select “Thing Groups” or “Things” for the device to be updated.

In this case, I selected “ESP-WROOM-32-GGTest” and “Things”, so the things (devices) will be updated with pinpoint accuracy.

Specify the name that you set in the previous article on registering devices.

Select “MQTT” as the secure protocol and scroll down.

• Sign a new file for me.

and click the “Create new profile” button in the Existing code signing profile.

• Profile name: esp32_ota_profile_en
• Hardware platform of the device: ESP32-DevKitC

and select “Import a new code signing certificate” and scroll down.

• Certificate body: Code signing certificate (ecsdsasigner.crt)
• Certificate private key: Private key (ecsasigner.key)

and click the “Import” button to upload the file, then scroll down.

• Path name of codes signing certificate on device: /OTA_Cert/auth.pem

and click the “Create” button.

Go back to the previous screen and scroll down to the bottom where the profile (esp32_ota_profile_en) has been created and selected.

Select “Choose file” and upload aws_demos.bin of the rebuilt Ver 093.

• S3 URL: Bucket name (s3://esp-wroom-32-ota-en)
• Path name of file on device: /device

and scroll down.

• Role: esp-32-ota-role-en

Select the first role you created and click the “Next” button.

Creating a Job

Now that the configuration is complete, create the job.

• Job run type: Your job will complete after deploying to the devices and groups that you chose(snapshot)
• Job start rollout configuration: Not configured (optional)
• Job stop configuration: Not set (optional)
• Job run timeout configuration: Not set (optional)

Click the “Create Job” button.

A job will be created and OTA Update will be executed.

OTA Update

After waiting for a while, the firmware update will be done automatically, so check the log of the ESP32-DevkitC serial console.

You can see that the job has been executed and the version has been upgraded to 0.9.3 via MQTT protocol.

The End

With the Amazon FreeRTOS OTA Update demo project, I was able to confirm that it automatically updates to the latest firmware.

Setting up roles and policies in advance is complicated and a bit of a hassle, but once it’s done, OTA Update itself is not that difficult as it is automatically updated via MQTT protocol by simply creating a job.

In addition, the private key and code signing certificate will be checked by AWS Certificate Manager, so it is nice to have a secure IoT device environment.

This concludes this article.

Translated with www.DeepL.com/Translator (free version)

I hope this article will be useful to someone somewhere.