Create CI test

examples/ci-tests/README.md

@@ -0,0 +1,58 @@
+Hardware CI Tests
+=================
+
+This folder holds tests that run as part of Tock's
+[hardware-in-the-loop CI](https://github.com/tock/tock/blob/master/doc/CI_Hardware.md).
+
+These are applications that are automatically loaded onto test hardware.
+
+Each test has an application which is loaded onto the Tock hardware platform
+under test (`main.c`) and a test harness runner (`test.py`) which is run on the
+supporting test-runner platform (currently expected to be a Raspberry Pi –
+this is because the test runner must have low-level hardware access, i.e. GPIO,
+I2C, SPI, etc support, and the test harnesses all use Raspberry Pi-specific
+libraries to support this).
+
+While these tests are written to be run by the automated CI infrastructure,
+they should generally also be abe to be run standalone as well (again, if run
+on a Raspberry Pi). The tests also assume physical pin connections (i.e. a wire
+between a pin on the board under test and the driving Raspberry Pi header) as
+documented in each board's test harness description in the main Tock repo.
+
+To run a test, enter the test directory and invoke the unit test corresponding
+to the hardware platform attached to the Raspberry Pi you are logged into:
+
+    [rpi01] $ cd uart-rx-tx
+    [rpi01] $ python3 test.py Nrf52840Test
+
+    Starting Uart Rx/TX Test...
+
+    0.000187 INFO -- Initiating UART test...
+    0.016512 INFO -- Setting up for nrf52840dk Uart Rx/Tx test...
+    0.016805 INFO -- Waiting for message...
+    0.017386 INFO -- UART Transaction Beginning
+    5.022696 INFO -- Waiting for message...
+    5.023263 INFO -- UART Transaction Beginning
+    10.028633 INFO -- Message sent: R
+    12.031758 INFO -- Message Received (r[character]): rR
+    12.032129 INFO -- Echoed: R
+    Correct Serial Communication Message Received
+    .
+    ----------------------------------------------------------------------
+    Ran 1 test in 12.021s
+
+    OK
+
+Tests use the standard Python [`unittest`](https://docs.python.org/3/library/unittest.html) framework.
+
+To the extent possible, tests SHOULD be hardware-agnostic. The only
+per-platform specification should be pin assignment. Board configurations in
+the main Tock repo specify supported hardware and tests. For this reason, if a
+test attempts to run on a platform that inherently cannot support it (e.g. BLE
+tests on a platform with no Bluetooth radio), the test MUST fail.
+
+Tests may assume full ownership of the hardware. Each folder is a logical unit
+of testing and is designed to be run in isolation. Currently, there are no
+multitenant hardware-CI tests. If and when these are added, current thinking is
+to put concurrent test configurations into a folder such that the folder
+remains the atomic unit of a test, but this is certainly subject to change.

examples/ci-tests/ble/Makefile

@@ -0,0 +1,17 @@
+# Makefile for user application
+
+# Specify this directory relative to the current application.
+TOCK_USERLAND_BASE_DIR = ../../..
+
+# Which files to compile.
+C_SRCS := $(wildcard *.c)
+
+# External libraries used
+EXTERN_LIBS += $(TOCK_USERLAND_BASE_DIR)/simple-ble
+
+# Include userland master makefile. Contains rules and flags for actually
+# building the application.
+include $(TOCK_USERLAND_BASE_DIR)/AppMakefile.mk
+
+# Include simple-ble's Makefile so it's rebuilt automatically
+include $(TOCK_USERLAND_BASE_DIR)/simple-ble/Makefile

examples/ci-tests/ble/README.md

@@ -0,0 +1,68 @@
+# BLE Scan Test
+
+## Test Description
+
+`test.py` provides the following tests:
+
+### BleTest
+
+This test utilizes the test harness (RPi) capabilities to run a ble scan of the device-under-test to verfiy ble connectivity works as intended. The test harness simply runs a scan of all bluetooth devices in the vacinity, and locates the device-under-test (tock device) for ble connection.
+
+## Example
+     0.000152 INFO -- Initiating BLE test...
+     0.016245 INFO -- Setting up for nrf52840dk BLE test...
+     0.016528 INFO -- Bluetooth Status on RPi Harness
+
+     ● bluetooth.service - Bluetooth service
+         Loaded: loaded (/lib/systemd/system/bluetooth.service; enabled; vendor preset: enabled)
+         Active: active (running) since Fri 2021-08-27 18:17:17 UTC; 4min 38s ago
+         Docs: man:bluetoothd(8)
+     Main PID: 1912 (bluetoothd)
+         Status: "Running"
+         Tasks: 1 (limit: 973)
+         CGroup: /system.slice/bluetooth.service
+                  └─1912 /usr/lib/bluetooth/bluetoothd
+
+     Aug 27 18:17:17 ubuntu systemd[1]: Starting Bluetooth service...
+     Aug 27 18:17:17 ubuntu bluetoothd[1912]: Bluetooth daemon 5.53
+     . (other information..)
+     .
+     .
+     Aug 27 18:17:17 ubuntu systemd[1]: Started Configure Bluetooth Modems connected by UART.
+     6.719571 INFO -- Scan result error:
+     LE Scan ...
+     08:99:10:53:EB:97 (unknown)
+     40:8E:59:DA:D9:F0 (unknown)
+     F0:00:00:00:02:F0 TockOS
+     65:83:87:B5:B6:22 (unknown)
+     . (more devices..)
+     .
+     .
+
+     6.720389 INFO -- BLE scan ended.
+     6.720653 INFO -- Restarting test harness bluetooth.
+     .
+     ----------------------------------------------------------------------
+     Ran 1 test in 6.899s
+
+
+     OK
+
+## Notes
+
+When manually invoking the test harness, you must specify the board under test.
+Otherwise, python unittest will attempt to run tests for all platforms.
+
+To run the test,
+```bash
+sudo python3 test.py Nrf52840Test
+```
+
+
+### Supported Boards
+
+CI has been validated and runs on the following hardware platforms:
+
+Board | Test Name
+------|----------
+nrf52840dk | Nrf52840Test

examples/ci-tests/ble/main.c

@@ -0,0 +1,74 @@
+#include <ble.h>
+#include <gap.h>
+#include <stdbool.h>
+#include <stdio.h>
+#include <tock.h>
+
+// Sizes in bytes
+#define DEVICE_NAME_SIZE 6
+#define UUIDS_SIZE 4
+#define MANUFACTURER_DATA_SIZE 2
+#define FAKE_TEMPERATURE_DATA_SIZE 2
+
+/*******************************************************************************
+ * MAIN
+ ******************************************************************************/
+
+int main(void) {
+  int err;
+  printf("[Tutorial] BLE Advertising\n");
+
+  // declarations of variables to be used in this BLE example application
+  uint16_t advertising_interval_ms = 20;
+  uint8_t device_name[]            = "TockOS";
+  uint16_t uuids[]                 = {0x1800, 0x1809};
+  uint8_t manufacturer_data[]      = {0x13, 0x37};
+  uint8_t fake_temperature_data[]  = {0x00, 0x00};
+
+  static uint8_t adv_data_buf[ADV_DATA_MAX_SIZE];
+
+  // configure advertisement interval to 300ms
+  // configure LE only and discoverable
+  printf(" - Initializing BLE... %s\n", device_name);
+  AdvData_t adv_data = gap_adv_data_new(adv_data_buf, sizeof(adv_data_buf));
+
+  gap_add_flags(&adv_data, LE_GENERAL_DISCOVERABLE | BREDR_NOT_SUPPORTED);
+
+  // configure device name as TockOS
+  printf(" - Setting the device name... %s\n", device_name);
+  err = gap_add_device_name(&adv_data, device_name, DEVICE_NAME_SIZE);
+  if (err < RETURNCODE_SUCCESS)
+    printf("ble_advertise_name, error: %s\r\n", tock_strrcode(err));
+
+  // configure list of UUIDs */
+  printf(" - Setting the device UUID...\n");
+  err = gap_add_service_uuid16(&adv_data, uuids, UUIDS_SIZE);
+  if (err < RETURNCODE_SUCCESS)
+    printf("ble_advertise_uuid16, error: %s\r\n", tock_strrcode(err));
+
+  // configure manufacturer data
+  printf(" - Setting manufacturer data...\n");
+  err = gap_add_manufacturer_specific_data(&adv_data, manufacturer_data,
+                                           MANUFACTURER_DATA_SIZE);
+  if (err < RETURNCODE_SUCCESS)
+    printf("ble_advertise_manufacturer_specific_data, error: %s\r\n",
+           tock_strrcode(err));
+
+  // configure service data
+  printf(" - Setting service data...\n");
+  err = gap_add_service_data(&adv_data, uuids[1], fake_temperature_data,
+                             FAKE_TEMPERATURE_DATA_SIZE);
+  if (err < RETURNCODE_SUCCESS)
+    printf("ble_advertise_service_data, error: %s\r\n", tock_strrcode(err));
+
+  // start advertising
+  printf(" - Begin advertising! %s\n", device_name);
+  err = ble_start_advertising(ADV_NONCONN_IND, adv_data.buf, adv_data.offset, advertising_interval_ms);
+  if (err < RETURNCODE_SUCCESS)
+    printf("ble_start_advertising, error: %s\r\n", tock_strrcode(err));
+
+  // configuration complete
+  printf("Now advertising every %d ms as '%s'\n", advertising_interval_ms,
+         device_name);
+  return 0;
+}

examples/ci-tests/ble/test.py

@@ -0,0 +1,137 @@
+# BLE Test
+# This tester corresponds to libtock-c/examples/ci-tests/ble test.
+
+import logging
+import time
+import unittest
+import os
+import subprocess
+
+TARGET_NAME = 'TockOS'
+
+################################################################################
+# Helper classes and functions
+################################################################################
+
+def time_gap(start_time):
+    """Return time gap between current time and start_time
+
+    Argument:
+    start_time - Start time
+    """
+    return "{:.6f}".format(time.time() - start_time)
+
+# END
+
+################################################################################
+# Start test and logger
+################################################################################
+
+# Test Start Time
+TEST_START_TIME = time.time()
+
+# Logger set format
+LOG_FORMAT = "%(timegap)s %(levelname)s -- %(message)s"
+logging.basicConfig(format=LOG_FORMAT)
+
+# Logger add formatter
+logger = logging.getLogger('BLE Test')
+logger.setLevel('INFO')
+
+logger.info('Initiating BLE test...',
+            extra={'timegap': time_gap(TEST_START_TIME)})
+
+# END
+
+################################################################################
+# Test Case Module
+################################################################################
+
+class BleTest(unittest.TestCase):
+    def test_ble_advertise(self):
+        """Check if the advertised device name can be found"""
+        # Change line from docs to logging info
+        print()
+
+        logger.info('Bluetooth Status on RPi Harness\n',
+            extra={'timegap': time_gap(TEST_START_TIME)})
+        os.system('sudo systemctl status bluetooth')
+        print() # Line change
+        os.system('sudo systemctl status hciuart')
+        print() # Line change
+        # os.system('sudo timeout 5 stdbuf -oL hcitool lescan')
+        # print() # Line change
+        scan_cmd = 'sudo timeout 5 stdbuf -oL hcitool lescan'
+
+        # BLE scan flag
+        found = False
+
+        try:
+            scan_result = subprocess.check_output(scan_cmd, 
+                                              stderr=subprocess.STDOUT,
+                                              shell=True)
+            scan_result_str = scan_result.decode('ascii')
+            logger.info('Scan result:\n' + scan_result_str,
+                        extra={'timegap': time_gap(TEST_START_TIME)})
+
+            # Search for target board name
+            scan_entries = scan_result_str.split('\n')
+            print(scan_entries)
+
+            for entry in scan_entries:
+                mac_addr, name = entry.split(' ')
+                print(name)
+
+                if name == TARGET_NAME:
+                    found = True
+
+        except subprocess.CalledProcessError as err:
+            # Print error
+            scan_result_str = err.output.decode('ascii')
+
+            logger.info('Scan result error:\n' + scan_result_str,
+                        extra={'timegap': time_gap(TEST_START_TIME)})
+
+            # Search for target board name
+            scan_entries = scan_result_str.split('\n')
+
+            for entry in scan_entries:
+                if entry != '':
+                    mac_addr, name = entry.split(' ', 1)
+
+                    if name == TARGET_NAME:
+                        found = True
+
+        finally:
+            logger.info('BLE scan ended.',
+                extra={'timegap': time_gap(TEST_START_TIME)})
+
+            # Restart bluetooth
+            # Note: the scanning process is corrupted whenever we try to kill it, so
+            #       for now, we resort to restarting bluetooth every test, but if
+            #       there is a better implementation, feel free to change this.
+            logger.info('Restarting test harness bluetooth.',
+                extra={'timegap': time_gap(TEST_START_TIME)})
+            os.system('sudo hciconfig hci0 down; sudo hciconfig hci0 up')
+
+            self.assertTrue(found)
+
+# END
+
+################################################################################
+# Test Case Setup
+################################################################################
+
+class Nrf52840Test(BleTest):
+    def setUp(self):
+        logger.info('Setting up for nrf52840dk BLE test...',
+            extra={'timegap': time_gap(TEST_START_TIME)})
+
+# END
+
+################################################################################
+# Main
+################################################################################
+
+if __name__ == '__main__':
+    unittest.main()

examples/ci-tests/gpio/Makefile

@@ -0,0 +1,11 @@
+# Makefile for user application
+
+# Specify this directory relative to the current application.
+TOCK_USERLAND_BASE_DIR = ../../..
+
+# Which files to compile.
+C_SRCS := $(wildcard *.c)
+
+# Include userland master makefile. Contains rules and flags for actually
+# building the application.
+include $(TOCK_USERLAND_BASE_DIR)/AppMakefile.mk