Bug fix/3322 web and tui report token usage differently

[riatzukiza]

addresses #3322

Summary of Changes Since opentui Branch

The main focus has been on standardizing token usage metrics across the TUI and Web interfaces:

Key Changes:

1. Token Metrics Alignment - Unified how token usage is calculated and displayed between TUI and Web UI
2. Enhanced TUI Sidebar - Added total token usage display in the TUI session sidebar
3. Improved Share UI - Enhanced the Web Share component to surface token usage information
4. Comprehensive Testing - Added extensive test coverage for token metrics consistency

Visual design

[rekram1-node]

hm I dont thinking making the tui super verbose is ideal.

I didn't look into the issue enough what was the issue?

[riatzukiza]

hm I dont thinking making the tui super verbose is ideal.

I didn't look into the issue enough what was the issue?

The issue is that people think that the counts on the web share ui and the counts on the tui ui mean the same thing, and that they are different makes them think that the tui is misreporting their token usage.

The tui reports context size, the share ui reports API token usage.
API token usage looks way bigger than context size, and that difference only gets larger and larger through each compaction cycle.

Yea, I wasn't really sure how you all wanted to address this, because from what I could tell neither of the two numbers are "wrong"

But people are misunderstanding what they mean.

I just made a utility that could get all the metrics, and the web and the tui both use that utility to display the information.