Problem Compiling for Alveo U280

[ATosun00]

Hello everyone,

I'm encountering an issue where the codebase fails to compile. I'm trying to build the GEMM kernel for my Alveo U280 board, using Vitis HLS and Vivado (both 2023 versions). My system is running on Ubuntu 22.04.5.

Help would be appreciated.

cd GEMM/
mkdir build
cd build/
cmake .. -DNUM_REPLICATIONS=4 -DFPGA_BOARD_NAME=xilinx_u280_gen3x16_xdma_1_202211_1 -DGLOBAL_MEM_UNROLL=1 -DDATA_TYPE=float -DXILINX_COMPILE_SETTINGS_FILE=../../settings/settings.compile.xilinx.gemm_cannon.hbm.ini -DXILINX_LINK_SETTINGS_FILE=../settings/settings.link.xilinx.gemm_cannon.hbm.generator.ini
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Found MPI, so binaries will be built with MPI support
-- Fetching mandatory build dependency GoogleTest
-- Found Python: /usr/bin/python3 (found version "3.10.12") found components:  Interpreter
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Fetching mandatory build dependency hlslib
-- Fetching mandatory build dependency cxxopts
-- cxxopts version 2.2.0
-- Fetching mandatory build dependency json
-- Using the multi-header code from /scratch-local/tosun/HPCC_FPGA/GEMM/build/_deps/extern_json-src/include/
-- Found Git: /usr/bin/git (found version "2.34.1")
-- e301a3c-dirty
-- Could NOT find IntelFPGAOpenCL (missing: IntelFPGAOpenCL_AOCL IntelFPGAOpenCL_AOC IntelFPGAOpenCL_INCLUDE_DIRS IntelFPGAOpenCL_LIBRARIES IntelFPGAOpenCL_RPATH IntelFPGAOpenCL_VERSION IntelFPGAOpenCL_MAJOR_VERSION IntelFPGAOpenCL_MINOR_VERSION)
-- Found Xilinx Runtime (XRT): /opt/xilinx/xrt
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: /usr/lib/x86_64-linux-gnu/libOpenCL.so (found version "2.2")
-- Found Vitis: /scratch-local/software/Xilinx/Vitis/2023.1/bin/v++
-- Found Python3: /usr/bin/python3 (found version "3.10.12") found components:  Interpreter
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of cl_float
-- Check size of cl_float - done
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /usr/lib/x86_64-linux-gnu/libblas.so
-- /usr/lib/x86_64-linux-gnu/libblas.so
-- Configuring done
-- Generating done
-- Build files have been written to: /scratch-local/tosun/HPCC_FPGA/GEMM/build
make
make[1]: Entering directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
make[2]: Entering directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
Scanning dependencies of target hpcc_fpga_base
make[2]: Leaving directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
make[2]: Entering directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
[  4%] Building CXX object lib/hpccbase/CMakeFiles/hpcc_fpga_base.dir/setup/fpga_setup.cpp.o
[  9%] Building CXX object lib/hpccbase/CMakeFiles/hpcc_fpga_base.dir/hpcc_settings.cpp.o
[ 13%] Linking CXX static library libhpcc_fpga_base.a
make[2]: Leaving directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
[ 13%] Built target hpcc_fpga_base
make[2]: Entering directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
Scanning dependencies of target ge_xilinx
make[2]: Leaving directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
make[2]: Entering directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
[ 18%] Building CXX object src/host/CMakeFiles/ge_xilinx.dir/execution_default.cpp.o
In file included from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution.h:30,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:24:
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:171:88: error: wrong number of template arguments (2, should be at least 5)
  171 | class GEMMBenchmark : public hpcc_base::HpccFpgaBenchmark<GEMMProgramSettings, GEMMData> {
      |                                                                                        ^
In file included from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:31,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution.h:30,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:24:
/scratch-local/tosun/HPCC_FPGA/shared/include/hpcc_benchmark.hpp:97:7: note: provided for ‘template<class TSettings, class TDevice, class TContext, class TProgram, class TData, class> class hpcc_base::HpccFpgaBenchmark’
   97 | class HpccFpgaBenchmark {
      |       ^~~~~~~~~~~~~~~~~
In file included from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution.h:30,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:24:
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:181:5: error: ‘void gemm::GEMMBenchmark::addAdditionalParseOptions(cxxopts::Options&)’ marked ‘override’, but does not override
  181 |     addAdditionalParseOptions(cxxopts::Options &options) override;
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:191:5: error: ‘std::unique_ptr<gemm::GEMMData> gemm::GEMMBenchmark::generateInputData()’ marked ‘override’, but does not override
  191 |     generateInputData() override;
      |     ^~~~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:200:5: error: ‘void gemm::GEMMBenchmark::executeKernel(gemm::GEMMData&)’ marked ‘override’, but does not override
  200 |     executeKernel(GEMMData &data) override;
      |     ^~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:210:5: error: ‘bool gemm::GEMMBenchmark::validateOutput(gemm::GEMMData&)’ marked ‘override’, but does not override
  210 |     validateOutput(GEMMData &data) override;
      |     ^~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:217:5: error: ‘void gemm::GEMMBenchmark::printError()’ marked ‘override’, but does not override
  217 |     printError() override;
      |     ^~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:219:10: error: ‘void gemm::GEMMBenchmark::collectResults()’ marked ‘override’, but does not override
  219 |     void collectResults() override;
      |          ^~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:226:5: error: ‘void gemm::GEMMBenchmark::printResults()’ marked ‘override’, but does not override
  226 |     printResults() override;
      |     ^~~~~~~~~~~~
In file included from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:24:
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution.h:54:65: error: wrong number of template arguments (1, should be at least 4)
   54 | calculate(hpcc_base::ExecutionSettings<gemm::GEMMProgramSettings> const& config, HOST_DATA_TYPE* a, HOST_DATA_TYPE* b, HOST_DATA_TYPE* c,
      |                                                                 ^
In file included from /scratch-local/tosun/HPCC_FPGA/shared/include/hpcc_benchmark.hpp:50,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:31,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution.h:30,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:24:
/scratch-local/tosun/HPCC_FPGA/shared/include/hpcc_settings.hpp:160:7: note: provided for ‘template<class TSettings, class TDevice, class TContext, class TProgram, class> class hpcc_base::ExecutionSettings’
  160 | class ExecutionSettings {
      |       ^~~~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:46:65: error: wrong number of template arguments (1, should be at least 4)
   46 | calculate(hpcc_base::ExecutionSettings<gemm::GEMMProgramSettings> const& config, HOST_DATA_TYPE* a, HOST_DATA_TYPE* b, HOST_DATA_TYPE* c, HOST_DATA_TYPE* c_out,
      |                                                                 ^
In file included from /scratch-local/tosun/HPCC_FPGA/shared/include/hpcc_benchmark.hpp:50,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/gemm_benchmark.hpp:31,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution.h:30,
                 from /scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:24:
/scratch-local/tosun/HPCC_FPGA/shared/include/hpcc_settings.hpp:160:7: note: provided for ‘template<class TSettings, class TDevice, class TContext, class TProgram, class> class hpcc_base::ExecutionSettings’
  160 | class ExecutionSettings {
      |       ^~~~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp: In function ‘std::map<std::__cxx11::basic_string<char>, std::vector<double> > bm_execution::calculate(const int&, cl_float*, cl_float*, cl_float*, cl_float*, cl_float, cl_float)’:
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:53:30: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   53 |     for (int i=0; i < config.programSettings->kernelReplications; i++) {
      |                              ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:54:59: error: request for member ‘context’ in ‘config’, which is of non-class type ‘const int’
   54 |         compute_queues.push_back(cl::CommandQueue(*config.context, *config.device, 0, &err));
      |                                                           ^~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:54:76: error: request for member ‘device’ in ‘config’, which is of non-class type ‘const int’
   54 |         compute_queues.push_back(cl::CommandQueue(*config.context, *config.device, 0, &err));
      |                                                                            ^~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:58:36: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   58 |     cl_int size_in_blocks = config.programSettings->matrixSize / config.programSettings->blockSize;
      |                                    ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:58:73: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   58 |     cl_int size_in_blocks = config.programSettings->matrixSize / config.programSettings->blockSize;
      |                                                                         ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:59:65: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   59 |     size_t number_blocks_per_kernel = ((size_in_blocks + config.programSettings->kernelReplications - 1)/(config.programSettings->kernelReplications));
      |                                                                 ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:59:114: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   59 |     size_t number_blocks_per_kernel = ((size_in_blocks + config.programSettings->kernelReplications - 1)/(config.programSettings->kernelReplications));
      |                                                                                                                  ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:60:37: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   60 |     size_t out_buffer_size = config.programSettings->matrixSize *
      |                                     ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:61:69: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   61 |                                 (number_blocks_per_kernel) * config.programSettings->blockSize;
      |                                                                     ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:70:30: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   70 |     for (int i=0; i < config.programSettings->kernelReplications; i++) {
      |                              ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:93:30: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   93 |         if (i == 0 || config.programSettings->replicateInputBuffers) {
      |                              ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:94:52: error: request for member ‘context’ in ‘config’, which is of non-class type ‘const int’
   94 |             a_buffers.push_back(cl::Buffer(*config.context, CL_MEM_READ_ONLY | memory_bank_info[0],
      |                                                    ^~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:95:64: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   95 |                                 sizeof(HOST_DATA_TYPE) *config.programSettings->matrixSize*config.programSettings->matrixSize, NULL, &err));
      |                                                                ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:95:99: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   95 |                                 sizeof(HOST_DATA_TYPE) *config.programSettings->matrixSize*config.programSettings->matrixSize, NULL, &err));
      |                                                                                                   ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:97:52: error: request for member ‘context’ in ‘config’, which is of non-class type ‘const int’
   97 |             b_buffers.push_back(cl::Buffer(*config.context, CL_MEM_READ_ONLY | memory_bank_info[1],
      |                                                    ^~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:98:64: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   98 |                                 sizeof(HOST_DATA_TYPE) *config.programSettings->matrixSize*config.programSettings->matrixSize, NULL, &err));
      |                                                                ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:98:99: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
   98 |                                 sizeof(HOST_DATA_TYPE) *config.programSettings->matrixSize*config.programSettings->matrixSize, NULL, &err));
      |                                                                                                   ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:100:52: error: request for member ‘context’ in ‘config’, which is of non-class type ‘const int’
  100 |             c_buffers.push_back(cl::Buffer(*config.context, CL_MEM_READ_ONLY | memory_bank_info[2],
      |                                                    ^~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:101:64: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  101 |                                 sizeof(HOST_DATA_TYPE) *config.programSettings->matrixSize*config.programSettings->matrixSize, NULL, &err));
      |                                                                ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:101:99: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  101 |                                 sizeof(HOST_DATA_TYPE) *config.programSettings->matrixSize*config.programSettings->matrixSize, NULL, &err));
      |                                                                                                   ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:104:50: error: request for member ‘context’ in ‘config’, which is of non-class type ‘const int’
  104 |         out_buffers.push_back(cl::Buffer(*config.context, CL_MEM_WRITE_ONLY | memory_bank_info[3],
      |                                                  ^~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:111:30: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  111 |     for (int i=0; i < config.programSettings->kernelReplications; i++) {
      |                              ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:120:39: error: request for member ‘program’ in ‘config’, which is of non-class type ‘const int’
  120 |         cl::Kernel gemmkernel(*config.program, (std::string(KERNEL_NAME) + "0:{" + KERNEL_NAME + "0_" +  std::to_string(i + 1) + "}").c_str(),
      |                                       ^~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:141:53: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  141 |         err = gemmkernel.setArg(0, a_buffers[config.programSettings->replicateInputBuffers ? i : 0]);
      |                                                     ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:143:53: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  143 |         err = gemmkernel.setArg(1, b_buffers[config.programSettings->replicateInputBuffers ? i : 0]);
      |                                                     ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:145:53: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  145 |         err = gemmkernel.setArg(2, c_buffers[config.programSettings->replicateInputBuffers ? i : 0]);
      |                                                     ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:168:32: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  168 |     for (int i = 0; i < config.programSettings->numRepetitions; i++) {
      |                                ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:200:35: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  200 |         for (int i=0; i < (config.programSettings->replicateInputBuffers ? config.programSettings->kernelReplications : 1); i++) {
      |                                   ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:200:83: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  200 |         for (int i=0; i < (config.programSettings->replicateInputBuffers ? config.programSettings->kernelReplications : 1); i++) {
      |                                                                                   ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:202:71: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  202 |                                         sizeof(HOST_DATA_TYPE)*config.programSettings->matrixSize*config.programSettings->matrixSize, a);
      |                                                                       ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:202:106: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  202 |                                         sizeof(HOST_DATA_TYPE)*config.programSettings->matrixSize*config.programSettings->matrixSize, a);
      |                                                                                                          ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:205:71: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  205 |                                         sizeof(HOST_DATA_TYPE)*config.programSettings->matrixSize*config.programSettings->matrixSize, b);
      |                                                                       ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:205:106: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  205 |                                         sizeof(HOST_DATA_TYPE)*config.programSettings->matrixSize*config.programSettings->matrixSize, b);
      |                                                                                                          ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:208:71: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  208 |                                         sizeof(HOST_DATA_TYPE)*config.programSettings->matrixSize*config.programSettings->matrixSize, c);
      |                                                                       ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:208:106: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  208 |                                         sizeof(HOST_DATA_TYPE)*config.programSettings->matrixSize*config.programSettings->matrixSize, c);
      |                                                                                                          ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:211:34: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  211 |         for (int i=0; i < config.programSettings->kernelReplications; i++) {
      |                                  ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:216:34: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  216 |         for (int i=0; i < config.programSettings->kernelReplications; i++) {
      |                                  ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:219:34: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  219 |         for (int i=0; i < config.programSettings->kernelReplications; i++) {
      |                                  ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:247:30: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  247 |     for (int i=0; i < config.programSettings->kernelReplications; i++) {
      |                              ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:248:83: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  248 |         long max_bytes_to_read = (static_cast<long>(sizeof(HOST_DATA_TYPE)*config.programSettings->matrixSize*config.programSettings->matrixSize))
      |                                                                                   ^~~~~~~~~~~~~~~
/scratch-local/tosun/HPCC_FPGA/GEMM/src/host/execution_default.cpp:248:118: error: request for member ‘programSettings’ in ‘config’, which is of non-class type ‘const int’
  248 |         long max_bytes_to_read = (static_cast<long>(sizeof(HOST_DATA_TYPE)*config.programSettings->matrixSize*config.programSettings->matrixSize))
      |                                                                                                                      ^~~~~~~~~~~~~~~
make[2]: *** [src/host/CMakeFiles/ge_xilinx.dir/build.make:63: src/host/CMakeFiles/ge_xilinx.dir/execution_default.cpp.o] Error 1
make[2]: Leaving directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
make[1]: *** [CMakeFiles/Makefile2:635: src/host/CMakeFiles/ge_xilinx.dir/all] Error 2
make[1]: Leaving directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
make: *** [Makefile:141: all] Error 2

[Mellich]

Seems that the template parameters for GEMM have not been properly updated in the latest release.
I pushed a hotfix to the master that should get rid of the error. Let me know if this works for you.

[ATosun00]

Thanks for the quick reply. I haven't tested this yet, but this wasn't only the case for GEMM, I did the same with STREAM the way it was specified in the Documentation.
But I can still test the Hotfix.

[Mellich]

You are right, this was also the case for STREAM and RandomAccess. Should be fixed now.

[ATosun00]

Yes, I was able to go further with the Build Process. I got this message

WARNING: [v++ 60-1600] The option 'jobs' was used directly on the command line, where its usage is deprecated. To ensure input line works for supported operating systems or shells, v++ supports specification for some
options in a configuration file. As an alternative, please use options 'hls.jobs', 'vivado.synth.jobs' in a configuration file.
Option Map File Used: '/scratch-local/software/Xilinx/Vitis/2023.1/data/vitis/vpp/optMap.xml'

So I removed the line XILINX_COMPILE_FLAGS:STRING=-j 40 in build/CMakeCache.txt.

After this I got this Error Message

INFO: [v++ 60-423] Target device: xilinx_u280_gen3x16_xdma_1_202211_1
ERROR: [v++ 60-1139] Failed to parse --connectivity.slr option: The compute unit specified as gemm0_$PY_CODE_GEN i + 1$ does not exist.
ERROR: [v++ 60-703] Failed to finish linking
INFO: [v++ 60-1653] Closing dispatch client.
make[2]: *** [src/device/CMakeFiles/gemm_base_emulate_xilinx.dir/build.make:65: bin/gemm_base_emulate.xclbin] Error 1
make[2]: Leaving directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
make[1]: *** [CMakeFiles/Makefile2:479: src/device/CMakeFiles/gemm_base_emulate_xilinx.dir/all] Error 2
make[1]: Leaving directory '/scratch-local/tosun/HPCC_FPGA/GEMM/build'
make: *** [Makefile:141: all] Error 2

So I changed the placeholders in the .ini file, like this.

# Set number of available SLRs
# PY_CODE_GEN num_slrs = 3

[connectivity]

nk=gemm0:4

# Assign kernels to the SLRs
# PY_CODE_GEN block_start [replace(local_variables=locals()) for i in range(num_replications)]

slr=gemm0_1:SLR0
slr=gemm0_2:SLR1
slr=gemm0_3:SLR2
slr=gemm0_4:SLR0

# PY_CODE_GEN block_end

# Assign the kernels to the memory ports
# PY_CODE_GEN block_start [replace(local_variables=locals()) for i in range(num_replications)]

sp=gemm0_1.m_axi_gmem0:HBM[0]
sp=gemm0_1.m_axi_gmem1:HBM[1]
sp=gemm0_1.m_axi_gmem2:HBM[2]
sp=gemm0_1.m_axi_gmem3:HBM[3]

sp=gemm0_2.m_axi_gmem0:HBM[4]
sp=gemm0_2.m_axi_gmem1:HBM[5]
sp=gemm0_2.m_axi_gmem2:HBM[6]
sp=gemm0_2.m_axi_gmem3:HBM[7]

sp=gemm0_3.m_axi_gmem0:HBM[8]
sp=gemm0_3.m_axi_gmem1:HBM[9]
sp=gemm0_3.m_axi_gmem2:HBM[10]
sp=gemm0_3.m_axi_gmem3:HBM[11]

sp=gemm0_4.m_axi_gmem0:HBM[12]
sp=gemm0_4.m_axi_gmem1:HBM[13]
sp=gemm0_4.m_axi_gmem2:HBM[14]
sp=gemm0_4.m_axi_gmem3:HBM[15]

# PY_CODE_GEN block_end

After this the Build Process finished, you got similar issues?

[ATosun00]

I am still having Problems with executing the the Tests.

This time i just did cmake .. -DFPGA_BOARD_NAME=xilinx_u280_gen3x16_xdma_1_202211_1 without specifying settings-files.

After getting rid of the "jobs" Specification in the CMakeCache and modifying in the .ini file, the build finished succesfully.

I also have access to the Alveo Card.

[bash tosun@codesigns46:bin]$ xbutil examine 
System Configuration
  OS Name              : Linux
  Release              : 5.15.0-102-generic
  Version              : #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024       
  Machine              : x86_64
  CPU Cores            : 16
  Memory               : 128512 MB
  Distribution         : Ubuntu 22.04.5 LTS
  GLIBC                : 2.35
  Model                : X299 DESIGNARE EX

XRT
  Version              : 2.15.225
  Branch               : 2023.1
  Hash                 : adf27adb3cfadc6e4c41d6db814159f1329b24f3
  Hash Date            : 2023-05-03 10:13:19
  XOCL                 : 2.15.225, adf27adb3cfadc6e4c41d6db814159f1329b24f3
  XCLMGMT              : 2.15.225, adf27adb3cfadc6e4c41d6db814159f1329b24f3

Devices present
BDF             :  Shell                            Platform UUID                         Device ID       Device Ready  
-------------------------------------------------------------------------------------------------------------------------
[0000:e1:00.1]  :  xilinx_u280_gen3x16_xdma_base_1  283BAB8F-654D-8674-968F-4DA57F7FA5D7  user(inst=128)  Yes


 Devices that are not ready will have reduced functionality when using XRT tools
[bash tosun@codesigns46:bin]$

However when i execute the following test

[bash tosun@codesigns46:bin]$ ./GEMM_test_xilinx -f gemm_base_emulate.xclbin
THIS BINARY EXECUTES UNIT TESTS FOR THE FOLLOWING BENCHMARK:









XRT build version: 2.15.225
Build hash: adf27adb3cfadc6e4c41d6db814159f1329b24f3
Build date: 2023-05-03 10:13:19
Git branch: 2023.1
PID: 291700
UID: 15599
[Thu Nov  7 15:09:19 2024 GMT]
HOST: codesigns46
EXE: /usr/bin/orted
[XRT] ERROR: clGetDeviceInfo: invalid param_name    
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
XRT build version: 2.15.225
Build hash: adf27adb3cfadc6e4c41d6db814159f1329b24f3
Build date: 2023-05-03 10:13:19
Git branch: 2023.1
PID: 291699
UID: 15599
[Thu Nov  7 15:09:19 2024 GMT]
HOST: codesigns46
EXE: /scratch-local/tosun/HPCC_FPGA/GEMM/build/bin/GEMM_test_xilinx
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
[==========] Running 36 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 13 tests from BaseHpccBenchmarkTest/0, where TypeParam = std::tuple<cl::Device, cl::Context, cl::Program>
[ RUN      ] BaseHpccBenchmarkTest/0.SetupSucceedsForBenchmarkTest
-------------------------------------------------------------
General setup:
C++ high resolution clock is used.
The clock precision seems to be 1.00000e+01ns
-------------------------------------------------------------
An error occured while setting up the benchmark:
        Communication type could not be converted from string: UNSUPPORTED
An error occured while setting up the benchmark: 
        Communication type could not be converted from string: UNSUPPORTED
/scratch-local/tosun/HPCC_FPGA/shared/tests/hpcc_base_benchmark_test.cpp:179: Failure
Value of: success
  Actual: false
Expected: true
[  FAILED  ] BaseHpccBenchmarkTest/0.SetupSucceedsForBenchmarkTest, where TypeParam = std::tuple<cl::Device, cl::Context, cl::Program> (1 ms)
[ RUN      ] BaseHpccBenchmarkTest/0.AllExecutedWhenNotTestOnly
-------------------------------------------------------------
General setup:
C++ high resolution clock is used.
The clock precision seems to be 1.00000e+01ns
-------------------------------------------------------------
An error occured while setting up the benchmark:
        Communication type could not be converted from string: UNSUPPORTED
[codesigns46:291699]  Process received signal 
[codesigns46:291699] Signal: Segmentation fault (11)
[codesigns46:291699] Signal code: Address not mapped (1)
[codesigns46:291699] Failing at address: (nil)
[codesigns46:291699] [ 0] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7faa7efe7520]
[codesigns46:291699] [ 1] ./GEMM_test_xilinx(+0x134a22)[0x556b8d10ea22]
[codesigns46:291699] [ 2] ./GEMM_test_xilinx(+0x1344dc)[0x556b8d10e4dc]
[codesigns46:291699] [ 3] ./GEMM_test_xilinx(+0x133f42)[0x556b8d10df42]
[codesigns46:291699] [ 4] ./GEMM_test_xilinx(+0x139018)[0x556b8d113018]
[codesigns46:291699] [ 5] ./GEMM_test_xilinx(+0xcfc0b)[0x556b8d0a9c0b]
[codesigns46:291699] [ 6] ./GEMM_test_xilinx(+0xc852f)[0x556b8d0a252f]
[codesigns46:291699] [ 7] ./GEMM_test_xilinx(+0x9d50c)[0x556b8d07750c]
[codesigns46:291699] [ 8] ./GEMM_test_xilinx(+0x9df9a)[0x556b8d077f9a]
[codesigns46:291699] [ 9] ./GEMM_test_xilinx(+0x9e8a1)[0x556b8d0788a1]
[codesigns46:291699] [10] ./GEMM_test_xilinx(+0xae7a7)[0x556b8d0887a7]
[codesigns46:291699] [11] ./GEMM_test_xilinx(+0xd10d9)[0x556b8d0ab0d9]
[codesigns46:291699] [12] ./GEMM_test_xilinx(+0xc977d)[0x556b8d0a377d]
[codesigns46:291699] [13] ./GEMM_test_xilinx(+0xacf0f)[0x556b8d086f0f]
[codesigns46:291699] [14] ./GEMM_test_xilinx(+0x124361)[0x556b8d0fe361]
[codesigns46:291699] [15] ./GEMM_test_xilinx(+0x1241fc)[0x556b8d0fe1fc]
[codesigns46:291699] [16] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7faa7efced90]
[codesigns46:291699] [17] /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7faa7efcee40]
[codesigns46:291699] [18] ./GEMM_test_xilinx(+0x1db55)[0x556b8cff7b55]
[codesigns46:291699]  End of error message 
Segmentation fault (core dumped)
[bash tosun@codesigns46:bin]$ 

or even when i do

[bash tosun@codesigns46:bin]$ ./GEMM_xilinx -h
XRT build version: 2.15.225
Build hash: adf27adb3cfadc6e4c41d6db814159f1329b24f3
Build date: 2023-05-03 10:13:19
Git branch: 2023.1
PID: 292445
UID: 15599
[Thu Nov  7 15:21:01 2024 GMT]
HOST: codesigns46
EXE: /usr/bin/orted
[XRT] ERROR: clGetDeviceInfo: invalid param_name    
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
XRT build version: 2.15.225
Build hash: adf27adb3cfadc6e4c41d6db814159f1329b24f3
Build date: 2023-05-03 10:13:19
Git branch: 2023.1
PID: 292444
UID: 15599
[Thu Nov  7 15:21:01 2024 GMT]
HOST: codesigns46
EXE: /scratch-local/tosun/HPCC_FPGA/GEMM/build/bin/GEMM_xilinx
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
-------------------------------------------------------------
General setup:
C++ high resolution clock is used.
The clock precision seems to be 1.00000e+01ns
-------------------------------------------------------------
Implementation of the GEMM benchmark proposed in the HPCC benchmark adapted for FPGA
Version: 1.3

MPI Version:  3.1
Config. Time: Thu Nov 07 14:46:14 UTC 2024
Git Commit:   0c2b781-dirty

Usage:
  ./GEMM_xilinx [OPTION...]

  -f, --file arg          Kernel file name
  -n, arg                 Number of repetitions (default: 10)
      --skip-validation   Skip the validation of the output data. This will
                          speed up execution and helps when working with
                          special data types.
      --device arg        Index of the device that has to be used. If not
                          given you will be asked which device to use if there
                          are multiple devices available. (default: -1)
      --platform arg      Index of the platform that has to be used. If not
                          given you will be asked which platform to use if
                          there are multiple platforms available. (default: -1)
      --platform_str arg  Name of the platform that has to be used (default:
                          )
  -r, arg                 Number of used kernel replications (default: 4)
      --dump-json arg     dump benchmark configuration and results to this
                          file in json format (default: )
      --test              Only test given configuration and skip execution
                          and validation
  -h, --help              Print this help
  -m, arg                 Matrix size in number of blocks in a single
                          dimension (default: 8)
  -b, arg                 Block size in number of values in one dimension
                          (default: 512)
      --replicate-inputs  Also replicates the input buffer for each kernel

[bash tosun@codesigns46:bin]$ 

I would really appreciate it if you could help me out.

[ATosun00]

Sorry for the Bad formatting, don't know why it outputs like that. 😃

[Mellich]

Hi, please try out the current branch. There was a bug in the setup code, which should be fixed now.

So I changed the placeholders in the .ini file, like this.

After this the Build Process finished, you got similar issues?

The current version of HPCC FPGA uses jinja2 for generating the settings files. The settings have to be updated to use jinja2 syntax, thanks for pointing this out. Of course it is also fine to provide your own non-generated link settings for now.

If you build the benchmark for Xilinx devices make sure to use the old (deprecated) OpenCL C++ header because the new header is not compatible with the provided drivers. You can do that by also providing the the option -DUSE_DEPRECATED_HPP_HEADER=Yes to cmake. Otherwise, the benchmarks will very likely segfault.

[ATosun00]

I will Check once I‘m home. Thanks for your help, appreciate it. 💪🏻

[ATosun00]

So I tried this with with the deprecated OpenCL headers and my modified .ini file (like shown above in a comment above).

[bash tosun@codesigns46:bin]$ ./GEMM_test_xilinx -f gemm_base_emulate.xclbin
THIS BINARY EXECUTES UNIT TESTS FOR THE FOLLOWING BENCHMARK:

XRT build version: 2.15.225
Build hash: adf27adb3cfadc6e4c41d6db814159f1329b24f3
Build date: 2023-05-03 10:13:19
Git branch: 2023.1
PID: 887500
UID: 15599
[Sat Nov  9 16:56:00 2024 GMT]
HOST: codesigns46
EXE: /usr/bin/orted
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
XRT build version: 2.15.225
Build hash: adf27adb3cfadc6e4c41d6db814159f1329b24f3
Build date: 2023-05-03 10:13:19
Git branch: 2023.1
PID: 887499
UID: 15599
[Sat Nov  9 16:56:01 2024 GMT]
HOST: codesigns46
EXE: /scratch-local/tosun/HPCC_FPGA/GEMM/build/bin/GEMM_test_xilinx
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
[==========] Running 36 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 13 tests from BaseHpccBenchmarkTest/0, where TypeParam = std::tuple<cl::Device, cl::Context, cl::Program>
[ RUN      ] BaseHpccBenchmarkTest/0.SetupSucceedsForBenchmarkTest
-------------------------------------------------------------
General setup:
C++ high resolution clock is used.
The clock precision seems to be 1.00000e+01ns
-------------------------------------------------------------
Selected Platform: Xilinx
-------------------------------------------------------------
Selection summary:
Platform Name: Xilinx
Device Name:   xilinx_u280_gen3x16_xdma_base_1
-------------------------------------------------------------
-------------------------------------------------------------
FPGA Setup:gemm_base_emulate.xclbin
[XRT] ERROR: See dmesg log for details. err = -22
[XRT] ERROR: failed to load xclbin: Invalid argument
ERROR in OpenCL library detected! Aborting.
/scratch-local/tosun/HPCC_FPGA/shared/setup/fpga_setup.cpp:170: CL_OUT_OF_HOST_MEMORY
An error occured while setting up the benchmark: 
	An OpenCL error occured: CL_OUT_OF_HOST_MEMORY
Selected Platform: Xilinx
-------------------------------------------------------------
Selection summary:
Platform Name: Xilinx
Device Name:   xilinx_u280_gen3x16_xdma_base_1
-------------------------------------------------------------
-------------------------------------------------------------
FPGA Setup:gemm_base_emulate.xclbin
[XRT] ERROR: See dmesg log for details. err = -22
[XRT] ERROR: failed to load xclbin: Invalid argument
ERROR in OpenCL library detected! Aborting.
/scratch-local/tosun/HPCC_FPGA/shared/setup/fpga_setup.cpp:170: CL_OUT_OF_HOST_MEMORY
An error occured while setting up the benchmark: 
	An OpenCL error occured: CL_OUT_OF_HOST_MEMORY
/scratch-local/tosun/HPCC_FPGA/shared/tests/hpcc_base_benchmark_test.cpp:179: Failure
Value of: success
  Actual: false
Expected: true
[  FAILED  ] BaseHpccBenchmarkTest/0.SetupSucceedsForBenchmarkTest, where TypeParam = std::tuple<cl::Device, cl::Context, cl::Program> (753 ms)
[ RUN      ] BaseHpccBenchmarkTest/0.AllExecutedWhenNotTestOnly
-------------------------------------------------------------
General setup:
C++ high resolution clock is used.
The clock precision seems to be 1.00000e+01ns
-------------------------------------------------------------
Selected Platform: Xilinx
-------------------------------------------------------------
Selection summary:
Platform Name: Xilinx
Device Name:   xilinx_u280_gen3x16_xdma_base_1
-------------------------------------------------------------
-------------------------------------------------------------
FPGA Setup:gemm_base_emulate.xclbin
[XRT] ERROR: See dmesg log for details. err = -22
[XRT] ERROR: failed to load xclbin: Invalid argument
ERROR in OpenCL library detected! Aborting.
/scratch-local/tosun/HPCC_FPGA/shared/setup/fpga_setup.cpp:170: CL_OUT_OF_HOST_MEMORY
An error occured while setting up the benchmark: 
	An OpenCL error occured: CL_OUT_OF_HOST_MEMORY
[codesigns46:887499] *** Process received signal ***
[codesigns46:887499] Signal: Segmentation fault (11)
[codesigns46:887499] Signal code: Address not mapped (1)
[codesigns46:887499] Failing at address: (nil)
[codesigns46:887499] [ 0] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fc214c43520]
[codesigns46:887499] [ 1] ./GEMM_test_xilinx(+0x13405c)[0x55eaa35d205c]
[codesigns46:887499] [ 2] ./GEMM_test_xilinx(+0x133b16)[0x55eaa35d1b16]
[codesigns46:887499] [ 3] ./GEMM_test_xilinx(+0x13357c)[0x55eaa35d157c]
[codesigns46:887499] [ 4] ./GEMM_test_xilinx(+0x138652)[0x55eaa35d6652]
[codesigns46:887499] [ 5] ./GEMM_test_xilinx(+0xcec15)[0x55eaa356cc15]
[codesigns46:887499] [ 6] ./GEMM_test_xilinx(+0xc7539)[0x55eaa3565539]
[codesigns46:887499] [ 7] ./GEMM_test_xilinx(+0x9c516)[0x55eaa353a516]
[codesigns46:887499] [ 8] ./GEMM_test_xilinx(+0x9cfa4)[0x55eaa353afa4]
[codesigns46:887499] [ 9] ./GEMM_test_xilinx(+0x9d8ab)[0x55eaa353b8ab]
[codesigns46:887499] [10] ./GEMM_test_xilinx(+0xad7b1)[0x55eaa354b7b1]
[codesigns46:887499] [11] ./GEMM_test_xilinx(+0xd00e3)[0x55eaa356e0e3]
[codesigns46:887499] [12] ./GEMM_test_xilinx(+0xc8787)[0x55eaa3566787]
[codesigns46:887499] [13] ./GEMM_test_xilinx(+0xabf19)[0x55eaa3549f19]
[codesigns46:887499] [14] ./GEMM_test_xilinx(+0x1237bb)[0x55eaa35c17bb]
[codesigns46:887499] [15] ./GEMM_test_xilinx(+0x123656)[0x55eaa35c1656]
[codesigns46:887499] [16] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7fc214c2ad90]
[codesigns46:887499] [17] /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7fc214c2ae40]
[codesigns46:887499] [18] ./GEMM_test_xilinx(+0x1cb75)[0x55eaa34bab75]
[codesigns46:887499] *** End of error message ***
Segmentation fault (core dumped)

Still got Errors 😅

[Mellich]

Hm, do other simple OpenCL codes work for you? You get a CL_OUT_OF_HOST_MEMORY error from the OpenCL runtime. This seems not to be related to the benchmarks at first glance.

[ATosun00]

So one of the fixes was changing the Enviroment Variable XCL_EMULATION_MODE.

export XCL_EMULATION_MODE=sw_emu

When I execute the Host Application.

[bash tosun@codesigns46:bin]$ ./GEMM_xilinx -f gemm_base_emulate.xclbin
XRT build version: 2.15.225
Build hash: adf27adb3cfadc6e4c41d6db814159f1329b24f3
Build date: 2023-05-03 10:13:19
Git branch: 2023.1
PID: 73203
UID: 15599
[Thu Nov 14 12:32:16 2024 GMT]
HOST:
EXE: /usr/bin/orted
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
XRT build version: 2.15.225
Build hash: adf27adb3cfadc6e4c41d6db814159f1329b24f3
Build date: 2023-05-03 10:13:19
Git branch: 2023.1
PID: 73202
UID: 15599
[Thu Nov 14 12:32:16 2024 GMT]
HOST:
EXE: /scratch-local/tosun/HPCC_FPGA/GEMM/build/bin/GEMM_xilinx
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
[XRT] ERROR: clGetDeviceInfo: invalid param_name
Authorization required, but no authorization protocol specified
-------------------------------------------------------------
General setup:
C++ high resolution clock is used.
The clock precision seems to be 1.00000e+01ns
-------------------------------------------------------------
Selected Platform: Xilinx
-------------------------------------------------------------
Selection summary:
Platform Name: Xilinx
Device Name:   xilinx_u280_gen3x16_xdma_1_202211_1
-------------------------------------------------------------
-------------------------------------------------------------
FPGA Setup:gemm_base_emulate.xclbin
Kernel Name: gemm0_1, CU Number: 0, Thread creation status: success
Kernel Name: gemm0_2, CU Number: 1, Thread creation status: success
Kernel Name: gemm0_3, CU Number: 2, Thread creation status: success
Kernel Name: gemm0_4, CU Number: 3, Thread creation status: success
Prepared FPGA successfully for global Execution!
-------------------------------------------------------------
Implementation of the GEMM benchmark proposed in the HPCC benchmark adapted for FPGA
Version: 1.3
MPI Version:  3.1
Config. Time: Thu Nov 14 12:19:07 UTC 2024
Git Commit:   eb1f375-dirty

Summary:
Block Size                              512
Communication Type                      UNSUPPORTED
Kernel File                             gemm_base_emulate.xclbin
Kernel Replications                     4
MPI Ranks                               1
Matrix Size                             4096
Repetitions                             10
Replicate Inputs                        No
Test Mode                               No
Device                                  xilinx_u280_gen3x16_xdma_1_202211_1

-------------------------------------------------------------
Start benchmark using the given configuration. Generating data...
-------------------------------------------------------------
Generation Time: 9.20680e+00 s
-------------------------------------------------------------
Execute benchmark kernel...
-------------------------------------------------------------
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_1, CU Number: 0, State: Start
Kernel Name: gemm0_1, CU Number: 0, State: Running
Kernel Name: gemm0_2, CU Number: 1, State: Start
Kernel Name: gemm0_2, CU Number: 1, State: Running
Kernel Name: gemm0_3, CU Number: 2, State: Start
Kernel Name: gemm0_3, CU Number: 2, State: Running
Kernel Name: gemm0_4, CU Number: 3, State: Start
Kernel Name: gemm0_4, CU Number: 3, State: Running
Kernel Name: gemm0_1, CU Number: 0, State: Idle
Kernel Name: gemm0_3, CU Number: 2, State: Idle
Kernel Name: gemm0_4, CU Number: 3, State: Idle
Kernel Name: gemm0_2, CU Number: 1, State: Idle
Execution Time: 7.65157e+01 s
-------------------------------------------------------------
Validate output...
-------------------------------------------------------------
      norm. residual          res. error           mach. eps
         6.13703e+00         2.24177e+01         1.19209e-07
Validation Time: 4.54715e+01 s
-------------------------------------------------------------
Collect results...
-------------------------------------------------------------
 best                mean                GFLOPS
7.21907e+00 s       7.34923e+00 s       1.90383e+01 GFLOP/s
-------------------------------------------------------------
ERROR: VALIDATION OF OUTPUT DATA FAILED!
device process sw_emu_device done
Kernel Name: gemm0_1, CU Number: 0, Status: Shutdown
Kernel Name: gemm0_2, CU Number: 1, Status: Shutdown
Kernel Name: gemm0_3, CU Number: 2, Status: Shutdown
Kernel Name: gemm0_4, CU Number: 3, Status: Shutdown

/scratch-local/tosun/HPCC_FPGA/GEMM/tests/test_kernel_functionality_and_host_integration.cpp:71: Failure
The difference between data->C_out[i * matrix_size + j] and 2.0 * data->C[i * matrix_size + j] is nan, which exceeds std::numeric_limits<cl_float>::epsilon(), where
data->C_out[i * matrix_size + j] evaluates to -nan,
2.0 * data->C[i * matrix_size + j] evaluates to 2, and
std::numeric_limits<cl_float>::epsilon() evaluates to 1.1920928955078125e-07.

/scratch-local/tosun/HPCC_FPGA/GEMM/tests/test_kernel_functionality_and_host_integration.cpp:136: Failure
The difference between data->C_out[i * matrix_size + j] and c_ref_out[i * matrix_size + j] is 14536, which exceeds 0.001, where
data->C_out[i * matrix_size + j] evaluates to 0,
c_ref_out[i * matrix_size + j] evaluates to 14536, and
0.001 evaluates to 0.001.

[       OK ] Default/GEMMKernelTest.JsonDump/1 (12181 ms)
[----------] 18 tests from Default/GEMMKernelTest (282819 ms total)

[----------] Global test environment tear-down
[==========] 36 tests from 3 test suites ran. (443572 ms total)
[  PASSED  ] 24 tests.
[  FAILED  ] 12 tests, listed below:
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectCtimesBeta/0, where GetParam() = 1
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectCtimesBeta/1, where GetParam() = 2
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectAtimesAlpha/0, where GetParam() = 1
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectAtimesAlpha/1, where GetParam() = 2
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectBtimesAlpha/0, where GetParam() = 1
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectBtimesAlpha/1, where GetParam() = 2
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectAmulB/0, where GetParam() = 1
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectAmulB/1, where GetParam() = 2
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectCplusA/0, where GetParam() = 1
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectCplusA/1, where GetParam() = 2
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectbetaCplusalphaAB/0, where GetParam() = 1
[  FAILED  ] Default/GEMMKernelTest.FPGACorrectbetaCplusalphaAB/1, where GetParam() = 2

12 FAILED TESTS

C_out always equates to 0.