feat(logs): support log streaming

[vgrozdanic]

Adds support for live streaming of logs by polling the API every 2 seconds (by default).

Polling interval is configurable via command line argument.

[vgrozdanic]

this is done to prevent OOM in case this keeps running for a long time

[vgrozdanic]

We need to do custom formatting, because every other formatting adds empty lines after printing out the table. We want it to seem like it's infinite table, without empty lines in between printing multiple rows, so we need to use FORMAT_NO_BORDER, and define our own line separator for header

[szokeasaurusrex]

Can we revisit whether it makes sense to output streamed logs in the table format? I really think it would be better, certainly in terms of making the implementation easier, but possibly also from a usability standpoint, if we just outputted them in an unstructured format. Or, if we also want to support a structured format for AI agents, I would suggest a separate mode that outputs in JSON or CSV, which could be enabled with a --structured flag.

My concern with table format is that the implementation is complex, it's not human readable unless you widen your terminal window, and also it probably is not a very standard format for LLMs compared with something like JSON or CSV. I think we can do better by having the default log format look like the logs that Sentry CLI outputs (human-readable), with a structured logging option.

What are your thoughts?

[szokeasaurusrex]

Deferring review until after HackWeek – please ping me if you need a review more urgently

[szokeasaurusrex]

Requesting changes, as the deduplication logic seems strange/possibly wrong, and we can improve it. Please see inline comment for details 👍

[szokeasaurusrex]

Can we revisit whether it makes sense to output streamed logs in the table format? I really think it would be better, certainly in terms of making the implementation easier, but possibly also from a usability standpoint, if we just outputted them in an unstructured format. Or, if we also want to support a structured format for AI agents, I would suggest a separate mode that outputs in JSON or CSV, which could be enabled with a --structured flag.

My concern with table format is that the implementation is complex, it's not human readable unless you widen your terminal window, and also it probably is not a very standard format for LLMs compared with something like JSON or CSV. I think we can do better by having the default log format look like the logs that Sentry CLI outputs (human-readable), with a structured logging option.

What are your thoughts?

[szokeasaurusrex]

Why not have the max size here be the set to the number of logs we are fetching per request? We probably would need to add that as a parameter to the signature.

[szokeasaurusrex]

Just my thinking process: If we are always fetching the most recent $x$ logs, and we assume that logs can neither be deleted nor can there timestamps be modified, then I am pretty sure the following holds:

1. Suppose fetch $n$ includes logs $\ell_1, \ell_2, ..., \ell_m$, where $m$ is maximum number of logs.
2. Suppose fetch $n+1$ then includes logs $\ell_{m+1}, \ell_{m+2}, ..., \ell_{2m}$. Importantly, no logs are repeated.
3. If the timestamps cannot be changed and no logs can be changed, then it is not possible for fetch $n+2$ to contain any of $\ell_1, \ell_2, ..., \ell_m$, as they would have also been included in $n+1$.
4. That means, we only need to cache at most $m$ entries.

[vgrozdanic]

Yeah, makes sense, i'll update the code

[szokeasaurusrex]

It would be ideal to update the max_rows field in the struct to also have a NonZeroUsize type. That way, we don't need the expect, because the nonzero value is enforced through the type system. But, we can do that in a future PR.

[vgrozdanic]

i have updated it as a part of this PR