tests: add support for test generation

[anisa-su993]

Usage

python run_tests.py [OPTIONS]
-/--test: opcode of test
-s/--suite: test suite
-e/--endpoint: endpoint type (ioctl devname | mctp nid eid)
-v/--verify-rsp: check expected rsp payloads?
-r/--rc: expected rc
-d/-mctpd: mctpd compile flag (openbmc | codeconstruct)

run_tests.py will generate, compile and run specified tests.

python generate_tests.py has same usage as run_tests but will only generate test files and will not compile or run

5 test cases are currently defined in 2 suites:

• Generic: Identify (0001h), Set Response Msg Limit (0004h), Get Supported Features (0500h), Get Feature (0501h)
• FMAPI: Get Host DC Region Config (5601h)

Tests:

This was tested by starting a QEMU VM, cloning this branch on it, and running "run_tests.py" with different options (generating a suite or all tests, w/ioctl or mctp, etc.)

The VM topology:

"-device usb-ehci,id=ehci \
 -object memory-backend-file,id=cxl-mem1,mem-path=/tmp/t3_cxl1.raw,size=4G \
 -object memory-backend-file,id=cxl-lsa1,mem-path=/tmp/t3_lsa1.raw,size=1M \
 -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
 -device cxl-rp,port=0,bus=cxl.1,id=cxl_rp_port0,chassis=0,slot=2 \
 -device cxl-type3,bus=cxl_rp_port0,volatile-dc-memdev=cxl-mem1,id=cxl-dcd0,lsa=cxl-lsa1,num-dc-regions=2,sn=99 \
 -device usb-cxl-mctp,bus=ehci.0,id=usb0,target=cxl-dcd0\
 -machine cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=1k"

Output from generating single test case:

root@deb-101020-bm01:~/libcxlmi# python3 tests/run_tests.py -t 5601 -e mctp 1 9 --verify-rsp
Generating Test Case for Opcode 5601
Directory already configured.

Just run your build command (e.g. ninja) and Meson will regenerate as necessary.
If ninja fails, run "ninja reconfigure" or "meson setup --reconfigure"
to force Meson to regenerate.

If build failures persist, run "meson setup --wipe" to rebuild from scratch
using the same options as passed when configuring the build.
To change option values, run "meson configure" instead.
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /usr/bin/ninja -C /root/libcxlmi/build
ninja: Entering directory `/root/libcxlmi/build'
[0/1] Regenerating build files.
The Meson build system
... snip ....
Opened MCTP EP with NID:EID =  1:9
Test 5601 passed!

Output from running Generic Commands suite:

python3 tests/run_tests.py -s generic -e ioctl mem0
Generating Suite GENERIC
Directory already configured.

Just run your build command (e.g. ninja) and Meson will regenerate as necessary.
If ninja fails, run "ninja reconfigure" or "meson setup --reconfigure"
to force Meson to regenerate.

... snip ...
Running Suite GENERIC
libcxlmi: ioctl returned non zero retval 3
Error: Function cxlmi_cmd_identify (0001h) returned rc of 3 but expected 0
Opened endpoint on device mem0
libcxlmi: ioctl returned non zero retval 3
Error: Function cxlmi_cmd_set_response_msg_limit (0004h) returned rc of 3 but expected 0
Opened endpoint on device mem0
Opened endpoint on device mem0
Suite GENERIC Results: 2 / 4

This output is expected because the identify and set rsp msg limit are prohibited on the mailbox.

Output from running all tests

python3 tests/run_tests.py -e mctp 1 0
... snip ...
[11/11] Linking target tests/output/test-0501
Opened MCTP EP with NID:EID =  1:9
Opened MCTP EP with NID:EID =  1:9
Opened MCTP EP with NID:EID =  1:9
libcxlmi: Error code in response: 3
Error: Function cxlmi_cmd_get_supported_features (0500h) returned rc of 3 but expected 0
Opened MCTP EP with NID:EID =  1:9
libcxlmi: Error code in response: 3
Error: Function cxlmi_cmd_get_feature (0501h) returned rc of 3 but expected 0
Opened MCTP EP with NID:EID =  1:9
Total tests passed: 3 / 5
        Suite GENERIC Results: 2 / 4
        Suite FMAPI Results: 1 / 1

This output is expected, because the commands are sent through the MCTP EP, so Identify (0001h), Set Rsp Msg Limit (0004h) and Get DC Region Config (5601h) all pass but the 2 Feature Commands are not part of the MCTP CCI, so they fail.

For an example test file generated: the file generated from python3 tests/run_tests.py -t 5601 -e mctp 1 9 --verify-rsp is as follows:

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include <libcxlmi.h>

#define MAX_PAYLOAD_SIZE 4096

static inline void freep(void *p)
{
	free(*(void **)p);
}
#define _cleanup_free_ __attribute__((cleanup(freep)))

#define UUID(time_low, time_mid, time_hi_and_version,                    \
  clock_seq_hi_and_reserved, clock_seq_low, node0, node1, node2,         \
  node3, node4, node5)                                                   \
  { ((time_low) >> 24) & 0xff, ((time_low) >> 16) & 0xff,                \
    ((time_low) >> 8) & 0xff, (time_low) & 0xff,                         \
    ((time_mid) >> 8) & 0xff, (time_mid) & 0xff,                         \
    ((time_hi_and_version) >> 8) & 0xff, (time_hi_and_version) & 0xff,   \
    (clock_seq_hi_and_reserved), (clock_seq_low),                        \
    (node0), (node1), (node2), (node3), (node4), (node5)                 \
  }


#define ASSERT_EQUAL(expected, actual) \
    if (expected != actual) { \
        printf("Assertion failed: %s = %llu, %s = %llu\n", \
               #expected, (unsigned long long)(expected), \
               #actual, (unsigned long long)(actual)); \
        rc = EXIT_FAILURE; \
    }

int main() {
    struct cxlmi_ctx *ctx;
    struct cxlmi_endpoint *ep;
    _cleanup_free_ void *req = calloc(1, MAX_PAYLOAD_SIZE);
    _cleanup_free_ void *rsp = calloc(1, MAX_PAYLOAD_SIZE);
    _cleanup_free_ void *expected = calloc(1, MAX_PAYLOAD_SIZE);
    int rc = EXIT_FAILURE;

    assert(req != NULL);
    assert(rsp != NULL);
    assert(expected != NULL);
    ctx = cxlmi_new_ctx(stdout, DEFAULT_LOGLEVEL);
    assert(ctx != NULL);

    ep = cxlmi_open_mctp(ctx, 1, 9);

    if (!ep) {
        printf("Failed to open MCTP EP with NID:EID =  %d:%d\n", 1, 9);
        goto exit_free_ctx;
    }

    printf("Opened MCTP EP with NID:EID =  %d:%d\n", 1, 9);

    
    struct cxlmi_cmd_fmapi_get_host_dc_region_config_req *request_1 = (struct cxlmi_cmd_fmapi_get_host_dc_region_config_req *)req;
    *request_1 = (struct cxlmi_cmd_fmapi_get_host_dc_region_config_req) {
        .host_id = 0,
        .region_cnt = 2,
        .start_region_id = 0,
    };



    struct cxlmi_cmd_fmapi_get_host_dc_region_config_rsp *actual_1 = (struct cxlmi_cmd_fmapi_get_host_dc_region_config_rsp *) rsp;

    rc = cxlmi_cmd_fmapi_get_dc_reg_config(ep, NULL, request_1, actual_1);
    if (rc != 0) {
        fprintf(stdout, "Error: Function cxlmi_cmd_fmapi_get_dc_reg_config (5601h) returned rc of %d but expected 0\n", rc);
        goto cleanup;
    }
    rc = EXIT_SUCCESS;

cleanup:
    cxlmi_close(ep);

exit_free_ctx:
    cxlmi_free_ctx(ctx);
    return rc;
}