Skip to content

feat: call AI streamtools manually #1898

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

YousefED
Copy link
Collaborator

@YousefED YousefED commented Jul 31, 2025

This PR exposes low-level functionality to apply operations (add / update / delete blocks) suggested by an LLM to the editor manually.

This makes it possible to completely offload LLM invocation to the client application. The client application can then (based on an LLM response), manually invoke the applicable StreamTools.

Demo @ https://blocknote-git-feature-call-streamtools-manually-typecell.vercel.app/ai/manual-execution (view example source)

Future work
Note that this will not integrate with the BlockNote AI UI components (UI toolbar) and is a very low-level way where the consumer will have full responsibility of setting up the LLM call (including prompts), and BlockNote only handles the "diffing" + display of document updates.

We'll also work on a more higher level abstraction that will make it easier to integrate BlockNote AI with custom backends. The proposed abstraction levels / integration points:

  • Lowest level: invoke streamtools directly to apply updates to an editor
  • Mid-level: an API that allows you to hook up your own backend / LLM integration to customize LLM requests (TBD)
  • Highest level: current version of BlockNote AI: BlockNote handles everything including calling the LLM (which is called either directly or via a Proxy)

Copy link

vercel bot commented Jul 31, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
blocknote ✅ Ready (Inspect) Visit Preview Jul 31, 2025 10:37am
blocknote-website ❌ Failed (Inspect) Jul 31, 2025 10:37am

Copy link

pkg-pr-new bot commented Jul 31, 2025

Open in StackBlitz

@blocknote/ariakit

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/ariakit@1898

@blocknote/code-block

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/code-block@1898

@blocknote/core

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/core@1898

@blocknote/mantine

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/mantine@1898

@blocknote/react

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/react@1898

@blocknote/server-util

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/server-util@1898

@blocknote/shadcn

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/shadcn@1898

@blocknote/xl-ai

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/xl-ai@1898

@blocknote/xl-docx-exporter

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/xl-docx-exporter@1898

@blocknote/xl-email-exporter

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/xl-email-exporter@1898

@blocknote/xl-multi-column

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/xl-multi-column@1898

@blocknote/xl-odt-exporter

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/xl-odt-exporter@1898

@blocknote/xl-pdf-exporter

npm i https://pkg.pr.new/TypeCellOS/BlockNote/@blocknote/xl-pdf-exporter@1898

commit: e152550

*
* (alternative to writing to the writable stream using {@link writable})
*/
async executeOne(chunk: StreamToolCall<T>): Promise<void> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be noted that running this once, will close the stream, so you can't call it in a loop or anything (since the .execute method closes the writer once it is done).

imo, this is a bit unexpected, I'd expect to be able to apply multiple executeOne calls, but I see how this is overlapping with just a writer.write.

@YousefED YousefED changed the title feature: call AI streamtools manually feat: call AI streamtools manually Jul 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants