|
| 1 | +# Design patterns for AI SDKs and signal APIs |
| 2 | + |
| 3 | +Interacting with AI and Large Language Model (LLM) APIs introduces unique challenges, such as managing asynchronous operations, handling streaming data, and designing a responsive user experience for potentially slow or unreliable network requests. Angular [signals](guide/signals) and the [`resource`](guide/signals/resource) API provide powerful tools to solve these problems elegantly. |
| 4 | + |
| 5 | +## Triggering requests with signals |
| 6 | + |
| 7 | +A common pattern when working with user-provided prompts is to separate the user's live input from the submitted value that triggers the API call. |
| 8 | + |
| 9 | +1. Store the user's raw input in one signal as they type |
| 10 | +2. When the user submits (e.g., by clicking a button), update a second signal with contents of the first signal. |
| 11 | +3. Use the second signal in the **`params`** field of your `resource`. |
| 12 | + |
| 13 | +This setup ensures the resource's **`loader`** function only runs when the user explicitly submits their prompt, not on every keystroke. You can use additional signal parameters, like a `sessionId` or `userId` (which can be useful for creating persistent LLM sessions), in the `loader` field. This way, the request always uses these parameters' current values without re-triggering the asyncronous function defined in the `loader` field. |
| 14 | + |
| 15 | +Many AI SDKs provide helper methods for making API calls. For example, the Genkit client library exposes a `runFlow` method for calling Genkit flows, which you can call from a resource's `loader`. For other APIs, you can use the [`httpResource`](guide/signals/resource#reactive-data-fetching-with-httpresource). |
| 16 | + |
| 17 | +The following example shows a `resource` that fetches parts of an AI-generated story. The `loader` is triggered only when the `storyInput` signal changes. |
| 18 | + |
| 19 | +```ts |
| 20 | +// A resource that fetches three parts of an AI generated story |
| 21 | +storyResource = resource({ |
| 22 | + // The default value to use before the first request or on error |
| 23 | + defaultValue: DEFAULT_STORY, |
| 24 | + // The loader is re-triggered when this signal changes |
| 25 | + params: () => this.storyInput(), |
| 26 | + // The async function to fetch data |
| 27 | + loader: ({params}): Promise<StoryData> => { |
| 28 | + // The params value is the current value of the storyInput signal |
| 29 | + const url = this.endpoint(); |
| 30 | + return runFlow({ url, input: { |
| 31 | + userInput: params, |
| 32 | + sessionId: this.storyService.sessionId() // Read from another signal |
| 33 | + }}); |
| 34 | + } |
| 35 | +}); |
| 36 | +``` |
| 37 | + |
| 38 | +## Preparing LLM data for templates |
| 39 | + |
| 40 | +You can configure LLM APIs to return structured data. Strongly typing your `resource` to match the expected output from the LLM provides better type safety and editor autocompletion. |
| 41 | + |
| 42 | +To manage state derived from a resource, use a `computed` signal or `linkedSignal`. Because `linkedSignal` [provides access to prior values](guide/signals/linked-signal), it can serve a variety of AI-related use cases, including |
| 43 | + * building a chat history |
| 44 | + * preserving or customizing data that templates display while LLMs generate content |
| 45 | + |
| 46 | +In the example below, `storyParts` is a `linkedSignal` that appends the latest story parts returned from `storyResource` to the existing array of story parts. |
| 47 | + |
| 48 | +```ts |
| 49 | +storyParts = linkedSignal<string[], string[]>({ |
| 50 | + // The source signal that triggers the computation |
| 51 | + source: () => this.storyResource.value().storyParts, |
| 52 | + // The computation function |
| 53 | + computation: (newStoryParts, previous) => { |
| 54 | + // Get the previous value of this linkedSignal, or an empty array |
| 55 | + const existingStoryParts = previous?.value || []; |
| 56 | + // Return a new array with the old and new parts |
| 57 | + return [...existingStoryParts, ...newStoryParts]; |
| 58 | + } |
| 59 | +}); |
| 60 | +``` |
| 61 | + |
| 62 | +## Performance and user experience |
| 63 | + |
| 64 | +LLM APIs may be slower and more error-prone than conventional, more deterministic APIs. You can use several Angular features to build a performant and user-friendly interface. |
| 65 | + |
| 66 | +* **Scoped Loading:** place the `resource` in the component that directly uses the data. This helps limit change detection cycles (especially in zoneless applications) and prevents blocking other parts of your application. If data needs to be shared across multiple components, provide the `resource` from a service. |
| 67 | +* **SSR and Hydration:** use Server-Side Rendering (SSR) with incremental hydration to render the initial page content quickly. You can show a placeholder for the AI-generated content and defer fetching the data until the component hydrates on the client. |
| 68 | +* **Loading State:** use the `resource` `LOADING` [status](guide/signals/resource#resource-status) to show an indicator, like a spinner, while the request is in flight. This status covers both initial loads and reloads. |
| 69 | +* **Error Handling and Retries:** use the `resource` [**`reload()`**](guide/signals/resource#reloading) method as a simple way for users to retry failed requests, may be more prevalent when relying on AI generated content. |
| 70 | + |
| 71 | +The following example demonstrates how to create a responsive UI to dynamically display an AI generated image with loading and retry functionality. |
| 72 | + |
| 73 | +```angular-html |
| 74 | +<!-- Display a loading spinner while the LLM generates the image --> |
| 75 | +@if (imgResource.isLoading()) { |
| 76 | + <div class="img-placeholder"> |
| 77 | + <mat-spinner [diameter]="50" /> |
| 78 | + </div> |
| 79 | +<!-- Dynamically populates the src attribute with the generated image URL --> |
| 80 | +} @else if (imgResource.hasValue()) { |
| 81 | + <img [src]="imgResource.value()" /> |
| 82 | +<!-- Provides a retry option if the request fails --> |
| 83 | +} @else { |
| 84 | + <div class="img-placeholder" (click)="imgResource.reload()"> |
| 85 | + <mat-icon fontIcon="refresh" /> |
| 86 | + <p>Failed to load image. Click to retry.</p> |
| 87 | + </div> |
| 88 | +} |
| 89 | +``` |
| 90 | + |
| 91 | + |
| 92 | +## AI patterns in action: streaming chat responses |
| 93 | +Interfaces often display partial results from LLM-based APIs incrementally as response data arrives. Angular's resource API provides the ability to stream responses to support this type of pattern. The `stream` property of `resource` accepts an asyncronous function you can use to apply updates to a signal value over time. The signal being updated represents the data being streamed. |
| 94 | + |
| 95 | +```ts |
| 96 | +characters = resource({ |
| 97 | + stream: async () => { |
| 98 | + const data = signal<ResourceStreamItem<string>>({value: ''}); |
| 99 | + // Calls a Genkit streaming flow using the streamFlow method |
| 100 | + // expose by the Genkit client SDK |
| 101 | + const response = streamFlow({ |
| 102 | + url: '/streamCharacters', |
| 103 | + input: 10 |
| 104 | + }); |
| 105 | + |
| 106 | + (async () => { |
| 107 | + for await (const chunk of response.stream) { |
| 108 | + data.update((prev) => { |
| 109 | + if ('value' in prev) { |
| 110 | + return { value: `${prev.value} ${chunk}` }; |
| 111 | + } else { |
| 112 | + return { error: chunk as unknown as Error }; |
| 113 | + } |
| 114 | + }); |
| 115 | + } |
| 116 | + })(); |
| 117 | + |
| 118 | + return data; |
| 119 | + } |
| 120 | +}); |
| 121 | +``` |
| 122 | + |
| 123 | +The `characters` member is updated asynchronously and can be displayed in the template. |
| 124 | + |
| 125 | +```angular-html |
| 126 | +@if (characters.isLoading()) { |
| 127 | + <p>Loading...</p> |
| 128 | +} @else if (characters.hasValue()) { |
| 129 | + <p>{{characters.value()}}</p> |
| 130 | +} @else { |
| 131 | + <p>{{characters.error()}}</p> |
| 132 | +} |
| 133 | +``` |
| 134 | + |
| 135 | +On the server side, in `server.ts` for example, the defined endpoint sends the data to be streamed to the client. The following code uses Gemini with the Genkit framework but this technique is applicable to other APIs that support streaming responses from LLMs: |
| 136 | + |
| 137 | +```ts |
| 138 | +import { startFlowServer } from '@genkit-ai/express'; |
| 139 | +import { genkit } from "genkit/beta"; |
| 140 | +import { googleAI, gemini20Flash } from "@genkit-ai/googleai"; |
| 141 | + |
| 142 | +const ai = genkit({ plugins: [googleAI()] }); |
| 143 | + |
| 144 | +export const streamCharacters = ai.defineFlow({ |
| 145 | + name: 'streamCharacters', |
| 146 | + inputSchema: z.number(), |
| 147 | + outputSchema: z.string(), |
| 148 | + streamSchema: z.string(), |
| 149 | + }, |
| 150 | + async (count, { sendChunk }) => { |
| 151 | + const { response, stream } = ai.generateStream({ |
| 152 | + model: gemini20Flash, |
| 153 | + config: { |
| 154 | + temperature: 1, |
| 155 | + }, |
| 156 | + prompt: `Generate ${count} different RPG game characters.`, |
| 157 | + }); |
| 158 | + |
| 159 | + (async () => { |
| 160 | + for await (const chunk of stream) { |
| 161 | + sendChunk(chunk.content[0].text!); |
| 162 | + } |
| 163 | + })(); |
| 164 | + |
| 165 | + return (await response).text; |
| 166 | +}); |
| 167 | + |
| 168 | +startFlowServer({ |
| 169 | + flows: [streamCharacters], |
| 170 | +}); |
| 171 | + |
| 172 | +``` |
0 commit comments