-
|
I see cocoindex has ExtractByLlm function, but maybe for some complex scenario, is it possible to integrate a dspy program to do transform? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Thanks for the question! Currently the easiest way to run Dspy within CocoIndex is to call it within a custom function. It's very easy to define a custom function, and you can call any Python library from it (example of converting PDFs into list of images) - so you can also call Dspy from it. We'll also create a new example that calls Dspy soon. Note that we'll also consider adding builtin functions that integrate with Dspy later. Since Dspy is very flexible and provides many modules / optimizers / etc., it's likely to be a series of builtin functions. We may do this if we see it'll provide significant convenience. But wrapping the custom Dspy invocation code with custom functions should already be quite simple and offers maximum control. Please let us know if you have more questions and if there's anything else we can help. Thanks! |
Beta Was this translation helpful? Give feedback.
Thanks for the question!
Currently the easiest way to run Dspy within CocoIndex is to call it within a custom function. It's very easy to define a custom function, and you can call any Python library from it (example of converting PDFs into list of images) - so you can also call Dspy from it. We'll also create a new example that calls Dspy soon.
Note that we'll also consider adding builtin functions that integrate with Dspy later. Since Dspy is very flexible and provides many modules / optimizers / etc., it's likely to be a series of builtin functions. We may do this if we see it'll provide significant convenience. But wrapping the custom Dspy invocation code with custom functions should …