-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Add option to batch dune calls #4222
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The pancakeswap-v3.ts adapter exports: |
|
The pancakeswap adapter exports: |
|
The pumpdotfun.ts adapter exports: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.
| SUM(daily_revenue_sol) as total_sol_revenue | ||
| FROM daily_revenue | ||
| ` | ||
| `, { extraUIDKey: 'pump-fees' } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we directly use the file name instead of passing the extraUIDKey?
|
|
||
| export const queryDune = async (queryId: string, query_parameters: any = {}) => { | ||
|
|
||
| export function queryDune(queryId: string, query_parameters: any, options: FetchOptions, { extraUIDKey = '' }: { extraUIDKey?: string } = {}) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if one query fails? other dune queries will also fail with them, and max sql query length limit by dune if any? and while running cron for dimension, all dune query would run at the same time, would this overwhelm the dune query response, and it could take very long time of max 30 minutes, after that time it would time out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the batch needs to be enabled, and it would work only for the same adapter(same query to be exact), so what you have mentioned wont happen.
|
@g1nt0ki maybe we could use the existing flag(isExpensiveAdapter) and run it normally, and have max 5 dune query in batch, to solve the timeout and reduce the errors |
|
atm, the support is limited to batching only the same adapter calls (different time periods) as union all wont work for varying columns (correct me if I am wrong here)
atm, the support is limited to batching only the same adapter calls (different time periods) as union all wont work for varying columns (correct me if I am wrong here) |
in theory, it could work but would be complicated as too much nesting of CTE and duplicate CTE names conflicts. |
No description provided.