R-Car/Boards/Yocto-Gen3/AWS IoT Greengrass/v5.5.0

From eLinux.org
Jump to: navigation, search


Introduction

This page describes how to setup the Yocto environment to use the AWS IoT Greengrass with R-Car and run it.

This page contains information abot building and running AWS IoT Greengrass on Yocto environment on:

Topic

EOL Notification of the M3SK

Production of M3SK is discontinued.
See M3SK page for detail.

The new version of R-Car Starter Kit Premier is now on sale !!

  • Equipped with R-Car H3e-2G
    (En) https://www.renesas.com/jp/en/about/press-room/renesas-launches-r-car-gen3e-20-percent-higher-cpu-speed-automotive-infotainment-cockpit-and-digital
    (Zh) https://www.renesas.com/jp/zh/about/press-room/renesas-launches-r-car-gen3e-20-percent-higher-cpu-speed-automotive-infotainment-cockpit-and-digital
    (Jp) https://www.renesas.com/jp/ja/about/press-room/renesas-launches-r-car-gen3e-20-percent-higher-cpu-speed-automotive-infotainment-cockpit-and-digital
  • CPU performance is increased 20% by supporting up to 2GHz frequency over past products.
  • You can buy from here.

SW Release Information

Board name SW name Release date Note
R-Car Starter Kit ( Premier / Pro ) Yocto v5.9.0 (stable) [New!!] 2022/02/08
Kingfisher Infotainment Board Yocto v5.9.0 (stable) [New!!] 2022/02/14 To check for latest information, please refer to the meta-rcar/tree/v5.9.0.
Android 10 (stable) 2021/07/26 R-Car Starter Kit Premier(R-Car H3) + Kingfisher is supported.
R-Car Starter Kit Pro(RTP8J77961ASKB0SK0SA05A) + Kingfisher is also supported from 2021/11/25.
Android P (stable) 2020/09/29 R-Car Starter Kit Premier(R-Car H3) + Kingfisher is supported.
R-Car Starter Kit Pro(RTP8J77961ASKB0SK0SA05A) + Kingfisher is also supported from 2021/03/16.
CCPF-SK Board Yocto v5.9.0 (stable) [New!!] 2022/02/08 Prebuilt binary is available in Quick startup guide page. (Updated on 2022/03/18)


Software revisions

Software Revision
Yocto Project 3.1.8
aarch64-poky-linux-gcc (GCC) 9.3
Kernel Ver 5.10.41
Userland 64/32bit 64
U-Boot 2020.10
OP_TEE 3.13.0
Greengrass 2.4.0

Environment

Host PC

OS Ubuntu 20.04 LTS (64bit)
Memory 8 GB or more
Storage At least 100 GB free

Boards confirmed to work

Board SoC Confirmed
R-Car Starter Kit Premier(H3) H3e-2G v3.0 OK
R-Car Starter Kit Premier(H3) H3 v3.0 (1rank DDR) OK
R-Car Starter Kit Premier(H3) H3 v3.0 (2rank DDR) NT
R-Car Starter Kit Premier(H3) H3 v2.0 with 4GB DDR OK
R-Car Starter Kit Pro M3 v3.0 OK
R-Car Starter Kit Pro M3 v1.0 OK
CCPF-SK + R-Car Starter Kit Premier(H3) H3e-2G v3.0 OK
CCPF-SK + R-Car Starter Kit Premier(H3) H3 v3.0 (1rank DDR) OK
CCPF-SK + R-Car Starter Kit Premier(H3) H3 v3.0 (2rank DDR) NT
CCPF-SK + R-Car Starter Kit Premier(H3) H3 v2.0 with 4GB DDR OK
CCPF-SK + R-Car Starter Kit Pro M3 v3.0 OK
CCPF-SK + R-Car Starter Kit Pro M3 v1.0 OK

18px <translate> Note:</translate> NT='Not Tested' but will work

See also:

Required packages

  1. Install required packages
    $ sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib \
    build-essential chrpath socat cpio python3 python3-pip python3-pexpect \
    xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa \
    libsdl1.2-dev pylint3 xterm docker.io
    
    Refer to Yocto Project Quick Start for more information.
    18px <translate> Note:</translate>If the OS version of the host PC is earlier than "Ubuntu 20.04", the default gcc version is old, so use "gcc-9/g++-9" or later version.
  1. Set up initial git configuration
    $ git config --global user.email "you@example.com"
    $ git config --global user.name "Your Name"
    

Building the BSP

  1. Create the following script(build.sh) for building
    #!/bin/bash
    
    BOARD_LIST=("h3ulcb" "m3ulcb" "h3ulcb-ccpf-sk" "m3ulcb-ccpf-sk")
    TARGET_BOARD=$1
    WORK=`pwd`/${TARGET_BOARD}
    
    # Commit ID
    POKY_COMMIT=6ebb33bdaccaeadff0c85aab27acf35723df00d8
    META_OE_COMMIT=c38d2a74f762a792046f3d3c377827b08aade513
    META_RENESAS_COMMIT=0fe77668f5d9a31a5d10449988c3d8fb8dc475c5
    META_RENESAS_CCPF_COMMIT=b49b57d6e68d5cda70aefbed34e06903484c573b
    META_VIRTUALIZATION_COMMIT=c4f156fa93b37b2428e09ae22dbd7f5875606f4d
    META_JAVA_COMMIT=62d6c0653ad69e14c21db2d4482e578400116a1b
    META_AWS_COMMIT=09a9e8845e1c0685d279a5fec12dc2764e67675c
    META_DOCKER_COMMIT=1ca1b5caf6f373dcc49db82dce50f4d8ab9f25cd
    
    Usage () {
        echo "Usage: $0 \${TARGET_BOARD_NAME}"
        echo "BOARD_NAME list: "
        for i in ${BOARD_LIST[@]}; do echo "  - $i"; done
        exit
    }
    
    # Check Param.
    if ! `IFS=$'\n'; echo "${BOARD_LIST[*]}" | grep -qx "${TARGET_BOARD}"`; then
        Usage
    fi
    
    mkdir -p ${WORK}
    cd ${WORK}
    
    # Clone basic Yocto layers in parallel
    git clone git://git.yoctoproject.org/poky &
    git clone git://git.openembedded.org/meta-openembedded &
    git clone git://github.com/renesas-rcar/meta-renesas &
    git clone git://git.yoctoproject.org/meta-virtualization &
    git clone git://git.yoctoproject.org/meta-java &
    git clone git://github.com/aws/meta-aws &
    git clone git://github.com/tkomagata/meta-docker &
    if [ "${TARGET_BOARD}" = "h3ulcb-ccpf-sk" ] || [ "${TARGET_BOARD}" = "m3ulcb-ccpf-sk" ]; then
        git clone git://github.com/renesas-rcar/meta-renesas-ccpf &
    fi
    
    # Wait for all clone operations
    wait
    
    # Switch to proper branches/commits
    cd ${WORK}/poky
    git checkout -b tmp ${POKY_COMMIT}
    cd ${WORK}/meta-openembedded
    git checkout -b tmp ${META_OE_COMMIT}
    cd ${WORK}/meta-renesas
    git checkout -b tmp ${META_RENESAS_COMMIT}
    cd ${WORK}/meta-virtualization
    git checkout -b tmp ${META_VIRTUALIZATION_COMMIT}
    cd ${WORK}/meta-java
    git checkout -b tmp ${META_JAVA_COMMIT}
    cd ${WORK}/meta-aws
    git checkout -b tmp ${META_AWS_COMMIT}
    cd ${WORK}/meta-docker
    git checkout -b tmp ${META_DOCKER_COMMIT}
    if [ "${TARGET_BOARD}" = "h3ulcb-ccpf-sk" ] || [ "${TARGET_BOARD}" = "m3ulcb-ccpf-sk" ]; then
        cd ${WORK}/meta-renesas-ccpf
        git checkout -b tmp ${META_RENESAS_CCPF_COMMIT}
    fi
    
    if [ "${TARGET_BOARD}" = "h3ulcb-ccpf-sk" ] || [ "${TARGET_BOARD}" = "m3ulcb-ccpf-sk" ]; then
        TEMPLATECONF=${WORK}/meta-renesas-ccpf/meta-rcar-gen3/docs/sample/conf/${TARGET_BOARD}/bsp/
    fi
    
    cd ${WORK}
    source poky/oe-init-build-env ${WORK}/build
    
    if [ "${TARGET_BOARD}" = "h3ulcb" ] || [ "${TARGET_BOARD}" = "m3ulcb" ]; then
        cp ${WORK}/meta-renesas/meta-rcar-gen3/docs/sample/conf/${TARGET_BOARD}/poky-gcc/bsp/bblayers.conf ${WORK}/build/conf/
        cp ${WORK}/meta-renesas/meta-rcar-gen3/docs/sample/conf/${TARGET_BOARD}/poky-gcc/bsp/local.conf ${WORK}/build/conf/
    fi
    
    bitbake-layers add-layer ../meta-openembedded/meta-networking
    bitbake-layers add-layer ../meta-openembedded/meta-filesystems
    bitbake-layers add-layer ../meta-virtualization
    bitbake-layers add-layer ../meta-docker/meta-rcar-gen3
    bitbake-layers add-layer ../meta-java
    bitbake-layers add-layer ../meta-aws
    
  2. Using setup script
    $ chmod a+x setup.sh
    $ ./setup.sh <target_board_name>
    
    target_board_name is specified for each board according to the following table:
    R-car-sk-target-board-name.png
  3. Append the following to local configuration(<target_board_name>/build/conf/local.conf)
    # Docker presumes systemd
    DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"
    VIRTUAL-RUNTIME_initscripts = "systemd-compat-units"
    
    # add docker to the image
    DISTRO_FEATURES_append = " virtualization docker"
    IMAGE_INSTALL_append = "  docker"
    CORE_IMAGE_EXTRA_INSTALL_append = " kernel-modules"
    
    # Add Greengrass and required packages to the image
    IMAGE_INSTALL_append = " greengrass-bin texinfo sudo openjdk-8"
    EXTRA_IMAGE_FEATURES_append = " ssh-server-openssh"
    
    # Possible provider: cacao-initial-native and jamvm-initial-native
    PREFERRED_PROVIDER_virtual/java-initial-native = "cacao-initial-native"
    
    # Possible provider: cacao-native and jamvm-native
    PREFERRED_PROVIDER_virtual/java-native = "jamvm-native"
    
    # Optional since there is only one provider for now
    PREFERRED_PROVIDER_virtual/javac-native = "ecj-bootstrap-native"
    
    IMAGE_INSTALL_append = " python3-pip"
    
  4. Build
    $ cd <target_board_name>
    $ source poky/oe-init-build-env
    $ bitbake core-image-minimal
    
    Depending on the performance of the host PC, it may take several hours for the build to complete.
    When the build completes successfully, you will see the following output:
    NOTE: Tasks Summary: Attempted 5383 tasks of which 5 didn't need to be rerun and all succeeded.
    
  5. Check the built images
    The built images are stored in the following:
     <target_board_name>/build/tmp/deploy/images/<target_board_name>/
  6. Write the images to the SD card
    Write the images to the SD card with reference to Loading kernel and rootfs via eMMC/SD card.
    18px <translate> Note:</translate> Please replace "core-image-weston" with "core-image-minimal".

Confirmation of Greengrass Core activation

  1. Turn on the target board
    1. Insert the SD card into the target board, and connect the LAN cable to it.
    2. Use a microUSB cable to connect the target board, connect a power supply to it, and connect to serial console.
    3. The PC will start the terminal software and make a serial connection.
    4. Power on.
      Short-press SW8 "Power" to switch the board on.
      If the board is attached to the CCPF board, short-press SW4 "Power" instead of SW8.
  2. Configure U-Boot to boot from SD card
    Refer to the Configure U-Boot to boot from SD card.
  3. Check that the Greengrass Core is running.
    $ systemctl status greengrass.service --no-pager
    
    Aws-status.png

Configure and check AWS IoT Greengrass

In subsequent chapters, the tags will be included in the title as follows.

  • Work on the target board (R-Car SK) is described as <Terminal>.
  • Work on the AWS Cloud is described as <Web>.
  • Work on the host PC is described as <PC>.
  1. <Web> Create an AWS account
    https://console.aws.amazon.com/console/home
    Create an AWS account and log in to the AWS Management Console as the root user.
  2. <Web> Creating an IAM user
    Create an IAM user for work with reference to the following.
    https://docs.aws.amazon.com/ja_jp/IAM/latest/UserGuide/id_users_create.html#id_users_create_console
    The following policies should be attached at the time of creation.
    • AWSLambda_FullAccess
    • IAMFullAccess
    • AWSIoTFullAccess
    • AWSGreengrassFullAccess
    • IAMAccessAnalyzerFullAccess
    18px <translate> Note:</translate>Be sure to write down the "Access Key ID", "Secret Access Key", and "Password" that are displayed when users access the system. Note that the secret access key and password can only be confirmed at this time.
    Warning Warning: These three items are very important information for security reasons, and should be managed with great care.
    Sign out the AWS root user and log in with the IAM user you created.
    https://console.aws.amazon.com/console/home
  3. <Web> Setting up a region
    Set the desired region from the upper right corner of the screen.
    In this procedure, we will set "Asia Pacific(Tokyo) ap-northeast-1".
  4. <Web><Terminal> Creating an IoT Core device
    Run "Confirmation of Greengrass Core activation" beforehand and make sure that the terminal can connect to the Internet by connecting a LAN cable.
    1. <Web> Select "IoT Core" from AWS services.
      Aws-iot-core.png
    2. <Web> Select "Greengrass" → "Getting started" from the side menu.
    3. <Web> Press "Set up one core device".
    4. <Web><Terminal> Follow the instructions on the screen to perform the following tasks.
      1. <Web> "Step 1: Register a Greengrass core device"
        Enter a core device name of your choice.
        In this procedure, the core device name shall be "R-CarH3SKDevice".
      2. <Web> "Step 2: Add to a thing group to apply a continuous deployment"
        Check "Enter a new group name" and enter a name of your choice.
        In this procedure, the thing group name shall be "R-CarH3SKDeviceGroup".
      3. <Terminal> "Step 3: Install the Greengrass Core software"
        1. "Step 3.1: Install Java on the device"
          No work required as it is included in the image generated by the R-Car SK build.
        2. "Step 3.2: Configure AWS credentials on the device"
          Execute the following command to set the information to access AWS as an environment variable.
          $ export AWS_DEFAULT_REGION=ap-northeast-1
          $ export AWS_ACCESS_KEY_ID=<your "Access Key ID">
          $ export AWS_SECRET_ACCESS_KEY=<your "Secret Access Key">
          
        3. "Step 3.3: Run the installer"
          "Download the installer" requires no work.
          "Run the installer" does not match the path displayed in the web browser, so execute the following command in the terminal.
          $ java -Droot="/greengrass/v2" -Dlog.store=FILE -jar /greengrass/v2/alts/init/distro/lib/Greengrass.jar \
          --aws-region ap-northeast-1 --thing-name R-CarH3SKDevice --thing-group-name R-CarH3SKDeviceGroup \
          --component-default-user ggc_user:ggc_group --provision true --setup-system-service true \
          --deploy-dev-tools true
          
          Replace "R-CarH3SKDevice" and "R-CarH3SKDeviceGroup" with the values set in "Step 1" and "Step 2".
          When the above command is executed, the R-Car SK terminal communicates with AWS IoT and is registered as a Greengrass core device.
          After the command finishes, Greengrass CLI will be deployed to the terminal, and check if the following commands can be executed.
          $ /greengrass/v2/bin/greengrass-cli -V
          
          18px <translate> Note:</translate>You can check the deployment status in the following log.
          $ tail -f /greengrass/v2/logs/greengrass.log
          
  5. <Web> Add policies to be sent to IoT Core
    18px <translate> Note:</translate> The examples in this document are intended only for dev environments.
        All devices in your production fleet must have credentials with privileges that authorize only intended actions on specific resources.
        The specific permission policies can vary for your use case. Identify the permission policies that best meet your business and security requirements.
        For more information, refer to Example policies and Security Best practices.
    Add a policy for the Lambda function running on the terminal to send messages to the IoT Core in the AWS cloud.
    1. Select "IAM" from AWS services.
    2. Select "Policies" from the side menu.
    3. Select "Create Policy".
    4. Select "IoT" for Service, "Publish" for Actions, and "All Resources" for Resources.
      Aws-iot-core-policy.png
    5. Click on "Next: Tags" and "Next: Review", then enter any "Name" in the "Review policy", and click on the "Create Policy" button.
      In this procedure, the name shall be "R-CarH3SKDevice-IoT-Publish".
    6. Return to the IAM screen and select "Roles" from the side menu.
    7. Enter "GreengrassV2TokenExchangeRole" in the search and select the role with the same name.
    8. Click "Attach Policies", search for the policy you just created, and click "Attach Policy".
  6. <Terminal> Preparing the execution environment for Lambda functions
    Build an environment to run the sample program on a terminal.
    Execute the following commands.
    $ mkdir -p /home/ggc_user
    $ chown ggc_user:ggc_group /home/ggc_user
    $ pip3 install numpy
    
  7. <Web><PC> Creating a Lambda Function
    Create a Lambda function to deploy to the terminal.
    1. <PC> Run the following command to create a template for a Lambda function.
      $ mkdir -p ~/lambda && touch ~/lambda/lambda_function.py
      
    2. <PC> Copy and paste the following code into the lambda_function.py that you created.
      import datetime
      import json
      import numpy
      import boto3
      
      class DummyTemperatureSensor(object):
          def __init__(self, loc=25, scale=1, size=1):
              self.loc = loc
              self.scale = scale
              self.size = size
      
          def get_value(self):
              return numpy.random.normal(self.loc, self.scale, self.size)[0]
      
      
      def lambda_handler(event, context):
          print(event)
      
          # AWS IoT connection
          iot = boto3.client('iot-data', endpoint_url='https://xxxxxxxxxxxxxxx-ats.iot.ap-northeast-1.amazonaws.com')
      
          # get temperature
          sensor = DummyTemperatureSensor()
      
          # set publish paramater
          topic='topic/sensor/temperature'
          payload = {
              "timestamp": str( datetime.datetime.now() ), 
              "temperature": sensor.get_value()
          }
          print(payload)
      
          # publish to AWS IoT
          iot.publish(
                  topic=topic,
                  qos=0,
                  payload=json.dumps(payload, ensure_ascii=False)
              )
      
      18px <translate> Note:</translate>The following part of the above code needs to be replaced with its own endpoint.
            https://xxxxxxxxxxxxxxx-ats.iot.ap-northeast-1.amazonaws.com
      The endpoints can be found below.
      1. <Web> Select "IoT Core" from AWS services.
      2. <Web> Select "Settings" from the side menu.
      3. <Web> Refer to "Endpoint" in Device data endpoint.
    3. <PC> Zip the packages.
      Zip the lambda_function.py and boto3 packages you have created.
      $ cd ~/lambda
      $ sudo docker run --rm -v $(pwd):/var/task amazon/aws-sam-cli-build-image-python3.8:latest \
        pip install boto3 -t ./
      $ zip -r lambda_artifact.zip ./
      
      The zip file is copied to a PC that has access to the AWS cloud for uploading to the cloud.
  8. <Web> Registering the Lambda function
    Register the created Lambda function to the AWS cloud.
    1. Select "Lambda" from AWS services.
    2. Select "Functions" from the side menu.
    3. Select "Create function".
      Aws-lambda-function-create-1.png
    4. Check the "Author from scratch" box.
    5. Give an arbitrary name to the "Function name". In this procedure, "DummyTemperatureSensor" is used.
    6. Select "Python 3.8" as the Runtime.
    7. Select "arm64" as the Architecture.
    8. Other items keep the default values and press "Create Function".
    9. When the screen changes, from the right side of "Code source", select "Upload from" → ".zip file", and upload → save the zip file you created.
    10. Select "Actions" → "Publish new version" in the upper right corner and press Publish.
  9. <Web> Deploying the Lambda function
    Deploy the registered Lambda function to the terminal.
    1. Select "IoT Core" from AWS services.
    2. Select "Greengrass" → "Components" from the side menu.
    3. Click on "Create component".
      Aws-lambda-function-deploy-1.png
    4. Check the "Import Lambda function" box.
    5. Select the function that you created in "Lambda function".
      Aws-lambda-function-deploy-2.png
    6. Enter "topic/sensor/get" in the "Topic" field of the Event sources.
    7. Select "AWS IoT Core MQTT" as the "Type" of the Event sources.
      Aws-lambda-function-deploy-3.png
    8. Set the "Memory size" of Container parameters to "64 MB".
      Aws-lambda-function-deploy-4.png
    9. Other items keep the default values and press "Create component".
    10. When the screen changes, press "Deploy" in the upper right corner.
    11. Select the group you created for the deployment destination and click "Next".
    12. Other items keep the default values and press "Next" several times, and press "Deploy" at the last confirmation screen.
  10. <WEB> <Terminal> Operation check
    Confirm that the function you created will be deployed on the terminal side.
    Also, check the log on the terminal side to see if messages are being received by AWS IoT → terminal.
    1. <Terminal> Checking the operation of R-Car SK
      1. Confirm that the Lambda function has been deployed.
        Execute the following command to confirm.
        $ /greengrass/v2/bin/greengrass-cli component list
        
        When you execute the command the following log will be output, and you can see that the deployed function (DummyTemperatureSensor) is displayed in the list.
        root@m3ulcb:~# /greengrass/v2/bin/greengrass-cli component list
            :
            :
        
        Component Name: DummyTemperatureSensor
            Version: 1.0.0
            State: RUNNING
            Configuration: {"containerMode":"GreengrassContainer","containerParams":{"devices":{},"memorySize":64000.0,"mountROSysfs":false,"volumes":{}},"inputPayloadEncodingType":"json","lambdaExecutionParameters":{"EnvironmentVariables":{}},"maxIdleTimeInSeconds":60.0,"maxInstancesCount":100.0,"maxQueueSize":1000.0,"pinned":true,"pubsubTopics":{"0":{"topic":"topic/sensor/get","type":"IOT_CORE"}},"statusTimeoutInSeconds":60.0,"timeoutInSeconds":3.0}
        
      2. Check the log.
        Run the following command to check the log.
        $ cd /greengrass/v2/logs/
        $ tail -f <name of created function>.log
        
        Verify that the following log is output.
        serviceName=<name of created function>, currentState=RUNNING
        
    2. <Web><Terminal> Checking the operation of the AWS cloud
      1. <Web> Select "IoT Core" from AWS services.
      2. <Web> Select "Test" → "MQTT Test Client" from the side menu.
      3. <Web> Enter "topic/sensor/temperature" in the Topic filter and press "SUBSCRIBE".
      4. <Web> Enter "topic/sensor/get" in the "Topic name" of "Publish to a topic" and click "Publish.
        18px <translate> Note:</translate>The default message payload will be garbled on the terminal side, so change it to any alphanumeric text if necessary.
      5. <Terminal> The message received will be displayed in the log as follows.
        root@m3ulcb:/greengrass/v2/logs# tail -f DummyTemperatureSensor.log
        2021-12-06T04:50:40.524Z [INFO] (pool-2-thread-38) DummyTemperatureSensor: lambda_function.py:17,{'message': 'AWS IoT ??????????????????????????????'}. {serviceInstance=0, serviceName=DummyTemperatureSensor, currentState=RUNNING}
        2021-12-06T04:50:41.172Z [INFO] (pool-2-thread-38) DummyTemperatureSensor: lambda_function.py:31,{'timestamp': '2021-12-06 04:50:41.168002', 'temperature': 27.381898093845034}. {serviceInstance=0, serviceName=DummyTemperatureSensor, currentState=RUNNING}
        2021-12-06T04:51:14.939Z [INFO] (pool-2-thread-38) DummyTemperatureSensor: lambda_function.py:17,{'message': 'hello world'}. {serviceInstance=0, serviceName=DummyTemperatureSensor, currentState=RUNNING}
        2021-12-06T04:51:14.969Z [INFO] (pool-2-thread-38) DummyTemperatureSensor: lambda_function.py:31,{'timestamp': '2021-12-06 04:51:14.964931', 'temperature': 25.003559249938885}. {serviceInstance=0, serviceName=DummyTemperatureSensor, currentState=RUNNING
        
      6. <Web> A message is received in the subscription.
        Aws-result1.png