Skip to content

Conversation

@carl-tud
Copy link
Contributor

@carl-tud carl-tud commented Jul 7, 2025

This PR is the second in a series to introduce unicoap, a unified and modular CoAP implementation for RIOT. An overview of all PRs related to unicoap is presented in #21389, including reasons why unicoap is needed and a performance analysis.

What does this PR include?

  • RFC 7252 messaging implementation
  • Implementation of the CoAP over UDP and CoAP over DTLS drivers, plus an additional zero-copy optimization for UDP
  • Basic server functionality, including XFA resource definitions and helpful debug logs
  • A sample server application
  • Structured documentation, including a tutorial from the first line of code to running the example and testing it using the included client script
  • Support for running unicoap on a thread of your choice, e.g., the main thread.

The new API is more flexible. CoAP endpoints are abstracted into a unicoap_endpoint_t structure and transport-specific settings are controlled by flags. For example, this is a simple resource that responds with "Hello, World!".

// Define request handler for /hello
static int handle_hello_request(unicoap_message_t* message, 
    const unicoap_aux_t* aux, unicoap_request_context_t* ctx, void* arg) {
    // The aux parameter provides access to auxiliary information, such as local and remote endpoints,
    // transport-specific data and internal properties such as the message token.

    // Retrieve remote (client) endpoint and log string description of protocol number.
    printf("/hello resource invoked over %s\n", unicoap_string_from_proto(aux->remote->proto));

    // Craft response using convenience initializer. Vectored payloads are supported, too.
    unicoap_response_init_string(message, UNICOAP_STATUS_CONTENT, "Hello, World!");

    // Send response.
    return unicoap_send_response(message, ctx);
}

// Statically define resource, but dynamic registrations are also possible.
UNICOAP_RESOURCE(hello) {
    .path = "/hello",
    
    // Instruct unicoap to send confirmable messages when communicating over UDP or DTLS.
    // This flag abstracts the transport-specific message type.
    .flags = UNICOAP_RESOURCE_FLAG_RELIABLE,
    
    // Specify what methods to allow. In unicoap, there are no duplicate defines for method flags.
    .methods = UNICOAP_METHODS(UNICOAP_METHOD_GET, UNICOAP_METHOD_PUT),
    
    // Optionally, you can also restrict the resource to a set of transports.
    .protocols = UNICOAP_PROTOCOLS(UNICOAP_PROTO_DTLS, UNICOAP_PROTO_UDP),
    
    .handler = handle_hello_request,
    .handler_arg = NULL
};

More in the documentation (CI build now available).

@github-actions github-actions bot added Area: network Area: Networking Area: doc Area: Documentation Area: tests Area: tests and testing framework Area: build system Area: Build system Area: CoAP Area: Constrained Application Protocol implementations Area: sys Area: System Area: examples Area: Example Applications Area: Kconfig Area: Kconfig integration labels Jul 7, 2025
@carl-tud carl-tud changed the title net/unicoap: Messaging and Minimal Server (pt 2) net/unicoap: Unified and Modular CoAP stack: Messaging and Minimal Server (pt 2) Jul 7, 2025
@crasbe crasbe added Type: new feature The issue requests / The PR implemements a new feature for RIOT CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jul 7, 2025
@riot-ci
Copy link

riot-ci commented Jul 7, 2025

Murdock results

✔️ PASSED

c307339 fixup: forgot to check in util_macros.h

Success Failures Total Runtime
10533 0 10534 12m:14s

Artifacts

@carl-tud carl-tud force-pushed the unicoap-02-server-minimal branch from da014c7 to 4d00949 Compare July 9, 2025 14:37
@github-actions github-actions bot removed the Area: tests Area: tests and testing framework label Jul 9, 2025
@carl-tud carl-tud marked this pull request as ready for review July 9, 2025 14:42
@carl-tud carl-tud requested a review from Teufelchen1 as a code owner October 19, 2025 18:48
@github-actions github-actions bot added the Area: tests Area: tests and testing framework label Oct 19, 2025
@carl-tud
Copy link
Contributor Author

carl-tud commented Oct 19, 2025

List of open discussion points from very slowly scrolling through the Github webview on a Czech train:

(you may check the ones that are resolved)

I fixed most of these but GitHub wouldn't let me mark them as resolved. My guess is -- not enough JavaScript.

slowly scrolling through the Github webview on a Czech train

good god...

const uint8_t* component = NULL;
ssize_t res = -1;
while ((res =unicoap_options_get_next_by_number(iterator, number,
(const uint8_t**)&component)) >= 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(const uint8_t**)&component)) >= 0) {
&component)) >= 0) {

or why would you still need this cast now that component is const uint8_t *?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I remember you saying you a doing this kind of casts in other places across unicoap, have you double-checked and changed?

Comment on lines +31 to +43
if (length > 0) {
size_t i = length - 1;
while (i > 0) {
if (path[i] == '/') {
i -= 1;
} else {
break;
}
}
return i + 1;
} else {
return length;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (length > 0) {
size_t i = length - 1;
while (i > 0) {
if (path[i] == '/') {
i -= 1;
} else {
break;
}
}
return i + 1;
} else {
return length;
}
if (length == 0) {
return length;
}
size_t i = length - 1;
while (i > 0) {
if (path[i] == '/') {
i -= 1;
}
else {
break;
}
}
return i + 1;

less indentation

return false;
}

/* The actual path (LHS) is now either as long or longer. If it is longer, then we
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/* The actual path (LHS) is now either as long or longer. If it is longer, then we
/* The actual path (LHS) is now either as long or longer than RHS. If it is longer, then we

assert(resource);
assert(lhs_path);

/* We are comparing the left-hand side (path from request) to the right-hand side (resource). */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is pretty bound to LHS = request path and RHS = resource definition path, why not reflecting that in the variable name instead of LHS and RHS? Would probably ease understanding.

Comment on lines +65 to +66
* expect a slash to indicate a subtree. Either the RHS path already ends with a slash
* or the LHS has a slash that succeeds the last RHS character.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get that part. Isn't LHS trimmed to have no slashes at the end?

Comment on lines +70 to +71
* LHS: /a -> RHS ends in slash, every path that is longer than RHS is a subpath
* RHS: /
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* LHS: /a -> RHS ends in slash, every path that is longer than RHS is a subpath
* RHS: /
* RHS: /, LHS: /a -> RHS ends in slash, every path that is longer than RHS is a subpath

similar below

Comment on lines +73 to +74
* LHS: /a/a -> RHS ends in slash, every path that is longer than RHS is a subpath
* RHS: /a/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this basically the same case as the one above?

Comment on lines 97 to 164
return cursor == (end + 1);
/* Make sure we read all options, i.e., the actual path is not longer than the resource's.
*/
return unicoap_options_get_next_uri_path_component(&iterator, &component) == -1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're welcome for the unittest nagging :P

};
UNICOAP_OPTIONS_ALLOC(options, 10);

printf("not working\n");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
printf("not working\n");

?

@mguetschow
Copy link
Contributor

mguetschow commented Oct 20, 2025

Followup of #21582 (comment) regarding dTLS:

The Network error: NoRequestInterface was due to me not installing aiocoap[all] as recommended on https://aiocoap.readthedocs.io/en/latest/installation.html. However, now I get (probably same as you)

$ python3 examples/networking/coap/unicoap_server/client.py -m GET -u "coaps://[fe80::5c64:bbff:fe11:9231%tap0]/.well-known/core"
usage: client.py -m <GET|PUT|POST|DELETE|PATCH|iPATCH|FETCH> -u <URI> [--type <NON|CON>] [--observe] [-p <PAYLOAD>]
DEBUG:asyncio:Using selector: EpollSelector
using NON GET request
timeout set to 4.0s
INFO:websockets.server:server listening on [::]:8600
DEBUG:coap-server:Server ready to receive requests
/home/mikolai/TUD/Code/RIOT/examples/networking/coap/unicoap_server/client.py:107: DeprecationWarning: Initializing messages with an mtype is deprecated. Instead, set transport_tuning=aiocoap.Reliable oraiocoap. Unreliable.
  request = Message(
INFO:coap-server:No DTLS connection active to (fe80::5c64:bbff:fe11:9231%tap0, 5684, b'Client_identity'), creating one
DEBUG:coap-server:Sending request - Token: be49, Remote: <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fdf1f8cc5d0>
DEBUG:coap-server:Sending message <aiocoap.Message: GET to <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fdf1f8cc5d0>, 1 option(s), token be49, NON, MID 0xa9e1>
WARNING:coap-server:Unhandled alert level 1 code 0
error: timeout exceeded after waiting 4.0s
Exception ignored in: <function DTLSClientConnection.__del__ at 0x7fdf1fab9580>
Traceback (most recent call last):
  File "/home/mikolai/.local/share/virtualenvs/RIOT-Q3sUo0rz/lib/python3.11/site-packages/aiocoap/transports/tinydtls.py", line 263, in __del__
    self.shutdown()
  File "/home/mikolai/.local/share/virtualenvs/RIOT-Q3sUo0rz/lib/python3.11/site-packages/aiocoap/transports/tinydtls.py", line 256, in shutdown
    self._transport.close()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 839, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Interestingly enough, for CONFIG_UNICOAP_DEBUG_LOGGING=y, CONFIG_UNICOAP_ASSIST=y, I get coap.transport.dtls: establishing session... and then unicoap does not respond to any requests anymore (even not UDP requests). With both set to n, I was able to get responses to UDP requests even after failed dTLS requests.


On my machine, dTLS with your script and gcoap_server does (mostly) work, so indeed seems to be a unicoap-related issue :/

$ python3 examples/networking/coap/unicoap_server/client.py -m GET -u "coaps://[fe80::5c64:bbff:fe11:9231%tap0]/.well-known/core"
usage: client.py -m <GET|PUT|POST|DELETE|PATCH|iPATCH|FETCH> -u <URI> [--type <NON|CON>] [--observe] [-p <PAYLOAD>]
DEBUG:asyncio:Using selector: EpollSelector
using NON GET request
timeout set to 4.0s
INFO:websockets.server:server listening on [::]:8600
DEBUG:coap-server:Server ready to receive requests
/home/mikolai/TUD/Code/RIOT/examples/networking/coap/unicoap_server/client.py:107: DeprecationWarning: Initializing messages with an mtype is deprecated. Instead, set transport_tuning=aiocoap.Reliable oraiocoap. Unreliable.
  request = Message(
INFO:coap-server:No DTLS connection active to (fe80::5c64:bbff:fe11:9231%tap0, 5684, b'Client_identity'), creating one
DEBUG:coap-server:Sending request - Token: 8647, Remote: <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fb24d0d03d0>
DEBUG:coap-server:Sending message <aiocoap.Message: GET to <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fb24d0d03d0>, 1 option(s), token 8647, NON, MID 0x9f40>
DEBUG:coap-server:Incoming message <aiocoap.Message: 2.05 Content from <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fb24d0d03d0>, 1 option(s), 46 byte(s) payload, token 8647, NON, MID 0x9f40>
DEBUG:coap-server:Received Response: <aiocoap.Message: 2.05 Content from <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fb24d0d03d0>, 1 option(s), 46 byte(s) payload, token 8647, NON, MID 0x9f40>
DEBUG:coap-server:Response <aiocoap.Message: 2.05 Content from <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fb24d0d03d0>, 1 option(s), 46 byte(s) payload, token 8647, NON, MID 0x9f40> matched to request <Pipe at 0x7fb24d0bb490 around <aiocoap.Message: GET to <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fb24d0d03d0>, 1 option(s), token 8647, NON, MID 0x9f40> with 2 callbacks (thereof 1 interests)>
response: 2.05 Content
b'</cli/stats>;ct=0;rt="count";obs,</riot/board>'
/home/mikolai/.local/share/virtualenvs/RIOT-Q3sUo0rz/lib/python3.11/site-packages/aiocoap/transports/tinydtls.py:169: UserWarning: DTLS module did not shut down the DTLSSocket perfectly; it still tried to call _write in vain
  warnings.warn(
Exception ignored in: <function DTLSClientConnection.__del__ at 0x7fb24d2bd580>
Traceback (most recent call last):
  File "/home/mikolai/.local/share/virtualenvs/RIOT-Q3sUo0rz/lib/python3.11/site-packages/aiocoap/transports/tinydtls.py", line 263, in __del__
    self.shutdown()
  File "/home/mikolai/.local/share/virtualenvs/RIOT-Q3sUo0rz/lib/python3.11/site-packages/aiocoap/transports/tinydtls.py", line 256, in shutdown
    self._transport.close()
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 839, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
    self._check_closed()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

@mguetschow
Copy link
Contributor

The assertion failure problem from #21582 (comment) seems fixed, so now I was able to test dTLS on the nrf52840dk as well. The output suggests some kind of buffer overflow

Client side

 python3 examples/networking/coap/unicoap_server/client.py -m GET -u "coaps://[2001:db8::2]/greeting?name=RIOT"
usage: client.py -m <GET|PUT|POST|DELETE|PATCH|iPATCH|FETCH> -u <URI> [--type <NON|CON>] [--observe] [-p <PAYLOAD>]
DEBUG:asyncio:Using selector: EpollSelector
using NON GET request
timeout set to 4.0s
INFO:websockets.server:server listening on [::]:8600
DEBUG:coap-server:Server ready to receive requests
/home/mikolai/TUD/Code/RIOT/examples/networking/coap/unicoap_server/client.py:107: DeprecationWarning: Initializing messages with an mtype is deprecated. Instead, set transport_tuning=aiocoap.Reliable oraiocoap. Unreliable.
  request = Message(
INFO:coap-server:No DTLS connection active to (2001:db8::2, 5684, b'Client_identity'), creating one
DEBUG:coap-server:Sending request - Token: 03d5, Remote: <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fd3db0d0490>
DEBUG:coap-server:Sending message <aiocoap.Message: GET to <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7fd3db0d0490>, 2 option(s), token 03d5, NON, MID 0xe54d>
error: timeout exceeded after waiting 4.0s

server (unicoap) side:

...
coap.transport.dtls: establishing session...
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000074020020000000002EB0517BD95B01002861002000000000000000000000000000000000B4100020C132010060FA0100B410002000000000FFFFCF075833010058330100986100200000000000001500515F010050610020000000FA25000000986E12831A0000000000000074610020C7432700C4....

looks like some kind of buffer overflow to me? like in, undefined behavior, that just happens to behave differently on native and on hardware


(Aaaah, I cannot reply to a comment that is not to a file in Github? o.O)

@mguetschow
Copy link
Contributor

mguetschow commented Oct 21, 2025

looks like some kind of buffer overflow to me? like in, undefined behavior, that just happens to behave differently on native and on hardware

there we go (with debug/assist enabled):

https://github.com/carl-tud/RIOT-Pull-Requests/blob/47b3b000c353020c00eca8e739e77cd7ddc29785/sys/net/application_layer/unicoap/drivers/rfc7252/dtls/transport.c#L70

note that res is expected to return -18 here.


Here is the pcap after commenting that line out: unicoap-dtls.pcapng.zip

Python side gives me WARNING:coap-server:Unhandled alert level 1 code 0 now, too.

@mguetschow
Copy link
Contributor

For future reference: mguetschow@92e63e5 contains some fixes to the dtls stack to make unicoap over dtls mostly work: I've encountered spurious deadlocks (I think) when sending multiple dtls requests one after another.

Other open issues encountered on the way:

  • unicoap sends a piggy-backed ACK for a CON request, even if the resource is marked RELIABLE. If the request is NON, unicoap rightfully sends CON.
  • unicoap holds stale dTLS session until they are explicitly closed by the peer, or until the session triage timeout is hit. I would expect it to drop stale sessions as soon as it obtains a new request (session initiation).

@mguetschow
Copy link
Contributor

#21827 together with mguetschow@30044de should be the right fixes to get dTLS to work properly.

@mguetschow
Copy link
Contributor

mguetschow commented Oct 30, 2025

Testing on native with LWIP_IPV6=1 or LWIP_IPV4 gives

/home/mikolai/TUD/Code/RIOT/sys/net/application_layer/unicoap/drivers/rfc7252/udp/transport.c:29: error: "UDP_DEBUG" redefined [-Werror]
   29 | #define UDP_DEBUG(...) _UNICOAP_PREFIX_DEBUG(".transport.udp", __VA_ARGS__)
      | 
In file included from /home/mikolai/.riot/pkg/lwip/src/include/lwip/api.h:40,
                 from /home/mikolai/TUD/Code/RIOT/pkg/lwip/include/sock_types.h:22,
                 from /home/mikolai/TUD/Code/RIOT/sys/include/net/sock/udp.h:845,
                 from /home/mikolai/TUD/Code/RIOT/sys/include/net/sock/dtls.h:546,
                 from /home/mikolai/TUD/Code/RIOT/sys/include/net/sock/async/event.h:174,
                 from /home/mikolai/TUD/Code/RIOT/sys/include/net/unicoap/transport.h:22,
                 from /home/mikolai/TUD/Code/RIOT/sys/net/application_layer/unicoap/drivers/rfc7252/udp/transport.c:20:
/home/mikolai/.riot/pkg/lwip/src/include/lwip/opt.h:3535: note: this is the location of the previous definition
 3535 | #define UDP_DEBUG                       LWIP_DBG_OFF

but otherwise works like a charm (after applying #21833)

@mguetschow
Copy link
Contributor

mguetschow commented Oct 30, 2025

@carl-tud May I also interest you about tiny_strerror? see #21833 (comment)

@mguetschow
Copy link
Contributor

mguetschow commented Oct 30, 2025

On nrf52840dk, it works using gnrc over slipdev, but lwip (ipv6) currently gives me coap.transport.udp: udp_sendv_aux failed: -118. Will need further investigation.

Edit: this is with slipdev enabled, when not using that one but only the default 802.15.4 radio, this PR needs to be rebased on current master to include #21533, to avoid an assertion failure.

With that applied, I still get -118, though. Might very well be an lwIP (setup) issue.

Comment on lines +331 to +337
error:
SERVER_DEBUG("failed to send response, trying to send 5.05 unreliably\n");

/* try to send 5.05, */
unicoap_response_init_empty(packet->message, UNICOAP_STATUS_INTERNAL_SERVER_ERROR);
unicoap_messaging_send(packet, _messaging_flags_resource(resource->flags) &
~UNICOAP_MESSAGING_FLAG_RELIABLE);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
error:
SERVER_DEBUG("failed to send response, trying to send 5.05 unreliably\n");
/* try to send 5.05, */
unicoap_response_init_empty(packet->message, UNICOAP_STATUS_INTERNAL_SERVER_ERROR);
unicoap_messaging_send(packet, _messaging_flags_resource(resource->flags) &
~UNICOAP_MESSAGING_FLAG_RELIABLE);
error:
SERVER_DEBUG("failed to send response, trying to send 5.00 unreliably\n");
/* try to send 5.00 */
unicoap_response_init_empty(packet->message, UNICOAP_STATUS_INTERNAL_SERVER_ERROR);
unicoap_messaging_send(packet, _messaging_flags_resource(resource->flags) &
~UNICOAP_MESSAGING_FLAG_RELIABLE);

5.05 (Proxying not supported) would not fit here, I guess.

Also it may be debatable if it makes sense to try sending a message after sending a message failed, but I guess that depends on the error condition. With lwip, I get

coap.messaging.rfc7252: received <NON REQ mid=11684 token=D453 code=0.01 GET payload=(0 bytes) options=(0; 0 bytes)>
coap.messaging: received in channel:
        remote=UDP <sock_tl_ep port=5683 netif=2 ipv6=fe80::5c2f:e804:c321:2ab>
        local=UDP <sock_tl_ep port=5683 netif=0 ipv6=fe80::6462:cc4f:30e5:f533>
coap.server: /: found
coap.server: invoking handler
app: GET /, 0 bytes
coap.server: sending immediate response
coap.messaging: sending in channel:
        remote=UDP <sock_tl_ep port=5683 netif=2 ipv6=fe80::5c2f:e804:c321:2ab>
        local=UDP <sock_tl_ep port=5683 netif=0 ipv6=fe80::6462:cc4f:30e5:f533>
coap.messaging.rfc7252: sending <CON RESP mid=7598 token=D453 code=2.05 Content payload=(13 bytes) options=(0; 0 bytes)>
coap.transport.udp: sendv: 20 bytes
coap.transport.udp: udp_sendv_aux failed: -EHOSTUNREACH
coap.messaging.rfc7252: sending failed
coap.messaging.rfc7252: [MID 7598] transmission ended
coap.server: error: could not send response
coap.server: failed to send response, trying to send 5.05 unreliably
coap.messaging: sending in channel:
        remote=UDP <sock_tl_ep port=5683 netif=2 ipv6=fe80::5c2f:e804:c321:2ab>
        local=UDP <sock_tl_ep port=5683 netif=0 ipv6=fe80::6462:cc4f:30e5:f533>
coap.messaging.rfc7252: sending <ACK RESP mid=7598 token=D453 code=5.00 Internal Server Error payload=(0 bytes) options=(0; 0 bytes)>
coap.transport.udp: sendv: 6 bytes
coap.transport.udp: udp_sendv_aux failed: -EHOSTUNREACH
coap.messaging.rfc7252: sending failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: build system Area: Build system Area: CoAP Area: Constrained Application Protocol implementations Area: doc Area: Documentation Area: examples Area: Example Applications Area: Kconfig Area: Kconfig integration Area: network Area: Networking Area: sys Area: System Area: tests Area: tests and testing framework Type: new feature The issue requests / The PR implemements a new feature for RIOT

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants