feat(expand): Enables the use of Pocketbases 'Expand' property #41

cf-david-easton · 2025-06-22T19:33:46Z

Pocketbase has a useful way to automatically expand relation fields with their data, this PR explores enabled the pocketbase-loader to do the same, this has to be implemented on both schema generation and fetching of the actual data.

At present I am only allowing one level deep, however it would not be terribly complex to allow deeper chaining but there would still need to be a cut off point.

Also one decision I made whilst doing this was to preserve the way Pocketbase returns expanded fields in their own object. However I am wondering if for a more streamlined experience, we should expand the values directly in their original field.

…pocketbase

pawcoding

I had a quick look over the code for some initial feedback, but I'll have a second look on the weekend when I have a bit more time to experiment with it. I'm a bit confused about all that array stuff, but it probably just needs a bit more thinking to understand it.

Regarding the "how deep should we nest", PocketBase itself allows for 6 levels of nesting, so we could also do that. Should be easy since it's just a recursive call. But before that I definitely want to finish a PR myself that reduces the number of calls to get a new token.

About the "overwrite the original value of expanded fields with their relations", I would leave it as is with the current implementation. I don't want to transform the original object to much and only introduced the file URL transformer since the file is almost useless without the actual URL (at least in my use cases so far where I uploaded e.g. an icon for a project).

Can you also turn on GitHub Actions on your fork so that the tests can run? You don't need to set up anything special, just enabling them and maybe pushing another commit should trigger them.

pawcoding · 2025-06-23T17:51:36Z

src/schema/generate-schema.ts

+            collectionName: expandedFieldDefinition.collectionId,
+            superuserCredentials: options.superuserCredentials,
+            localSchema: options.localSchema,
+            jsonSchemas: options.jsonSchemas,


It's probably fine, but using the same jsonSchemas would leave us unable to define a schema with the same name for the "parent" and the expanded "child" with different values.

I haven't dived too much into the implications of expand with the localSchema and jsonSchemas, how would we go about. resolving this? If its a case of name collisions, can we cleverly modify the collection names for the linked field or check if a pre-existing schema exists before making a new one? Sorry if this sounds like nonsense I need to look much more into the jsonSchema stuff

pawcoding · 2025-06-23T17:54:26Z

src/schema/generate-schema.ts

+          expandedFields[expandedFieldName] = z.union([
+            expandedchema,
+            z.array(expandedchema)
+          ]);


What is this used for? So that when you have a relation with multiple records it's typed as an array instead of the value?
Since this information is encoded in the original database schema we can use it to only use a single value or array for the type.

Yes that was my intention with the union to show it could be a single value or an array of the schema type.
Sorry could you elaborate on your 2nd point? Is there a better way I can infer this?

src/schema/generate-schema.ts

src/types/pocketbase-entry.type.ts

src/types/pocketbase-schema.type.ts

src/schema/generate-schema.ts

pawcoding · 2025-06-23T18:03:59Z

src/schema/generate-schema.ts

+            `The provided field in the expand property "${expandedFieldName}" does not have an associated collection linked to it, we need this in order to know the shape of the related schema.`
+          );
+        } else {
+          const expandedchema = await generateSchema({


Note for myself: Now that the generateSchema function can be called multiple times recursively it's definitely time to refactor the whole authentication. I already have a rough draft for this, were I can pass in a promise that resolves the token. This way we only need to do one authentication request.
I'll hopefully finish this up on the weekend.

test/schema/generate-schema.e2e-spec.ts

pawcoding · 2025-06-23T18:23:18Z

src/schema/generate-schema.ts

@@ -129,10 +133,48 @@ export async function generateSchema(
    }
  }

-  // Combine the basic schema with the parsed fields
+  if (options.expand) {


Doesn't this expand run for every nested layer? So it will always print the error message that the expand property cannot be found on the already expanded collection? 🤔
In other words: where is the cut-of point for x-level nesting?

I have address this with new additions today for supporitng 6 levels deep of expand nesting.
I essentially split all expand args as they nest until we are left with an empty array which I check the lenght of and pass undefined to this option so that the very next generateSchema call, expand is undefined for it.

cf-david-easton · 2025-06-27T12:12:43Z

Thanks for the great feedback @pawcoding, will give it a proper read over the weekend. Thanks for addressing my initial questions regarding the formatting of the expanded field, was tempted to start work on doing the opposite but glad you addressed it.

A lot of the array stuff came from me realising that relation fields can have multiple relations inside so I had to factor in if it was a singular object or multiple. Maybe a better way of representing and handling this though.

One thing I missed out on my schema gen was the fact that a relational field can be allowed to be null, so when I throw my fork into another project to try it out, it worked like a charm when seeing the collections type for expand but it would fail loading the schema because not all my pockbase items had a relation on them which isn't right so still a few little bits alongside your feedback, thanks again!

Co-authored-by: pawcode <[email protected]>

…d with better error protection

…ments exist

…ent the single or multi value

src/types/pocketbase-entry.type.ts

src/types/pocketbase-collection.type.ts

src/loader/load-entries.ts

pawcoding · 2025-07-12T14:51:07Z

src/schema/generate-schema.ts

+/**
+ * Builds a Zod object schema from expandedFields, where each property is either a ZodSchema or an array of ZodSchema.
+ */
+export function buildExpandSchema(
+  expandedFields: ExpandedFields
+): ZodObject<Record<string, ZodSchema | z.ZodArray<ZodSchema>>> {
+  const shape: Record<string, ZodSchema | z.ZodArray<ZodSchema>> = {};
+
+  for (const key in expandedFields) {
+    const value = expandedFields[key];
+    if (Array.isArray(value)) {
+      // If it's an array, wrap the first element as a ZodArray (assuming all elements are the same schema)
+      shape[key] = z.array(value[0]);
+    } else {
+      shape[key] = value;
+    }
+  }
+
+  return z.object(shape);
+}


That is the purpose of this function?

This was when I was still trying to figure out how to get the conditional shapes of array or singular elements. It was a core misunderstanding by me that I could read the maxSelect property of the field to understand if its a singular or multiple select, have removed and simplified the expanded property.

src/schema/generate-schema.ts

pawcoding · 2025-07-12T14:53:20Z

src/schema/generate-schema.ts

+  return schema.transform((entry) => {
+    if (Array.isArray(entry)) {
+      return entry.map((e) => transformFiles(options.url, fileFields, e));
+    }
+
+    return transformFiles(options.url, fileFields, entry);
+  });


Do we really need this? My assumption would be that the transformation for expanded collections is already done it the recursive call 🤔

Yes probably, not sure why I added this originally, reverted

pawcoding · 2025-07-12T15:00:00Z

test/loader/load-entries.e2e-spec.ts

+    (parseEntry as Mock).mockImplementation((entry: PocketBaseEntry) => {
+      parsedEntries.push(entry);
+      return entry; // or whatever parseEntry should return
+    });


Does this also work with spyOn?

vi.spyOn(parseEntry).mockImplementationOnce(/* whatever */)

Almost, had to do some fiddling and landed on extracting the arg out without a mock implmentation. Made it a global one for the file so the other checks for the call to use it.

src/schema/generate-schema.ts

feat(expand): added fetching and schema gen for expand property from …

6b3284e

…pocketbase

pawcoding reviewed Jun 23, 2025

View reviewed changes

cf-david-easton and others added 5 commits June 29, 2025 10:39

Apply suggestions from code review

4134e77

Co-authored-by: pawcode <[email protected]>

feat(expand): added ability to call multiple layers of depth on expan…

75d2917

…d with better error protection

feat(expand): added length guard for expand option

258f04d

feat(expand): added guard for expand nesting when no more chained seg…

9d20391

…ments exist

feat(expand): updated expand schema parsing to more accurately repres…

3933c35

…ent the single or multi value

pawcoding requested changes Jul 12, 2025

View reviewed changes

pawcoding assigned cf-david-easton Jul 12, 2025

cf-david-easton added 4 commits July 12, 2025 18:45

feat(expand): implemented feedback and simplified expand typing

e6b541f

feat(expand): removed unused const

bc1cca0

Merge branch 'master' into add-expand-option-to-loader

0f342cb

feat(expand): corrected typing for collection in test

3f51a05

feat(expand): Enables the use of Pocketbases 'Expand' property #41

Are you sure you want to change the base?

feat(expand): Enables the use of Pocketbases 'Expand' property #41

Uh oh!

Conversation

cf-david-easton commented Jun 22, 2025

Uh oh!

pawcoding left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cf-david-easton commented Jun 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!