feat(cat-voices): proposals query filtering #3641
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…pository. Index is only available in DocumentDataRemoteSource
…ments_indexing_3528
damian-molinski
added this to the
M5: Voting & Delegation - Production Readiness milestone
…zation_3528_part2
…put-output-hk/catalyst-voices into feat/database_optimization_3528_part2
…se_optimization_3528_part3
Base automatically changed from
feat/database_optimization_3528_part2
to
feat/database_optimization_3528
November 5, 2025 09:40
LynxLynxx
reviewed
Nov 5, 2025
Contributor
LynxLynxx
left a comment
LynxLynxx
left a comment
There was a problem hiding this comment.
LGTM just a few nit picks or questions
catalyst_voices/apps/voices/lib/pages/proposals/proposals_page.dart
Outdated
Show resolved
Hide resolved
catalyst_voices/packages/internal/catalyst_voices_blocs/lib/src/proposals/proposals_cubit.dart
Outdated
Show resolved
Hide resolved
catalyst_voices/packages/internal/catalyst_voices_blocs/lib/src/proposals/proposals_cubit.dart
Show resolved
Hide resolved
bstolinski
reviewed
Nov 5, 2025
catalyst_voices/apps/voices/lib/pages/proposals/widgets/proposals_tabs.dart
Outdated
Show resolved
Hide resolved
catalyst_voices/packages/internal/catalyst_voices_blocs/lib/src/proposals/proposals_cubit.dart
Show resolved
Hide resolved
...es/packages/internal/catalyst_voices_repositories/lib/src/database/dao/proposals_v2_dao.dart
Outdated
Show resolved
Hide resolved
...es/packages/internal/catalyst_voices_repositories/lib/src/database/dao/proposals_v2_dao.dart
Outdated
Show resolved
Hide resolved
Contributor
📚 Docs PreviewThe docs for this PR can be previewed at the following URL: https://docs.dev.projectcatalyst.io/voices/feat/database_optimization_3528_part3 |
bstolinski
approved these changes
Nov 6, 2025
damian-molinski
merged commit 2a8547a
into
feat/database_optimization_3528
45 checks passed
This was referenced Nov 10, 2025
Closed
Closed
damian-molinski
added a commit
that referenced
this pull request
Nov 26, 2025
* feat(cat-voices): database migration (#3585) * docs: capture initial times * chore: remove cacheDocument * turn off logging * chore: use debugPrint instead of logger * feat: bulk documents save * batching sync * chore: move exact ref resoling to getDocumentData instead of index * fix: DocumentRepository * chore: simplify getting documents data * remove getAllDocumentsRefs and getCachedDocumentsRefs from DocumentRepository. Index is only available in DocumentDataRemoteSource * chore: update docs * simplified document index endpoint * remove randomness from LocalCatGateway * indexing by batch size * wip: filtering by types * Rework filtering refs + checking all refs if already cached in parallel * docs * update indexing.csv * update indexing csv * chore: cleanup * trailing new line * chore * move performance tab to docs * bulk saving typed docs in parallel * chore: revert hardcoded timestamp * chore: typos * split _sync into smaller functions + add documentation * little refactor * fix: analyzer * initial v2 tables * wip * wip * feat: database migration * chore: cleanup * bump batch size * cleanup * chore: remove defensive content decoding * spelling * test on platform * chore: update build scripts * more migration test data * clean up constructors * cleanup * fix: template tests * fix: spelling * fix code-generator earthly target * use logger in migration + wrap in transaction * chore: move migration transaction level higher * feat(cat-voices): database queries migration and optimization (#3614) * docs: capture initial times * chore: remove cacheDocument * turn off logging * chore: use debugPrint instead of logger * feat: bulk documents save * batching sync * chore: move exact ref resoling to getDocumentData instead of index * fix: DocumentRepository * chore: simplify getting documents data * remove getAllDocumentsRefs and getCachedDocumentsRefs from DocumentRepository. Index is only available in DocumentDataRemoteSource * chore: update docs * simplified document index endpoint * remove randomness from LocalCatGateway * indexing by batch size * wip: filtering by types * Rework filtering refs + checking all refs if already cached in parallel * docs * update indexing.csv * update indexing csv * chore: cleanup * trailing new line * chore * move performance tab to docs * bulk saving typed docs in parallel * chore: revert hardcoded timestamp * chore: typos * split _sync into smaller functions + add documentation * little refactor * fix: analyzer * initial v2 tables * wip * wip * feat: database migration * chore: cleanup * bump batch size * cleanup * chore: remove defensive content decoding * chore: daos * spelling * saveAll * test on platform * chore: update build scripts * feat: DocumentsV2Dao methods * simple proposals pagination query * chore: create a JoinedProposalBriefEntity * rename method * feat: exclude hidden proposals * more tests * renaming and splitting logic into smaller parts * feat: per language strategy * remove CatalystDatabaseLanguage in favor of raw queries as they are easier to mange * remove Index Strategy Documentation * handle case where ref is empty * migration now includes indexes * use v2 documents table for saveAll and isCachedBulk * adds ActionType to JoinedProposalBriefEntity * adds versionIds to JoinedProposalBriefEntity * comments count * adds isFavorite to JoinedProposalBriefEntity * add template to JoinedProposalBriefEntity * adds documentsLocalMetadata table for auto updates * Update docs * use v2 proposals query for discovery most recent section * feat: simplify most recent proposals section * add proposal fav status for v2 tables * local proposal fav status update * update fav state locally for faster feedback * self review * fix tests * update times * fix: analyzer * more migration test data * clean up constructors * cleanup * fix: template tests * spelling * fix: spelling * chore: PR review adjustments * fix: add discovery specific colors (#3637) * fix code-generator earthly target * use logger in migration + wrap in transaction --------- Co-authored-by: Ryszard Schossler <[email protected]> * feat(cat-voices): proposals query filtering (#3641) * docs: capture initial times * chore: remove cacheDocument * turn off logging * chore: use debugPrint instead of logger * feat: bulk documents save * batching sync * chore: move exact ref resoling to getDocumentData instead of index * fix: DocumentRepository * chore: simplify getting documents data * remove getAllDocumentsRefs and getCachedDocumentsRefs from DocumentRepository. Index is only available in DocumentDataRemoteSource * chore: update docs * simplified document index endpoint * remove randomness from LocalCatGateway * indexing by batch size * wip: filtering by types * Rework filtering refs + checking all refs if already cached in parallel * docs * update indexing.csv * update indexing csv * chore: cleanup * trailing new line * chore * move performance tab to docs * bulk saving typed docs in parallel * chore: revert hardcoded timestamp * chore: typos * split _sync into smaller functions + add documentation * little refactor * fix: analyzer * initial v2 tables * wip * wip * feat: database migration * chore: cleanup * bump batch size * cleanup * chore: remove defensive content decoding * chore: daos * spelling * saveAll * test on platform * chore: update build scripts * feat: DocumentsV2Dao methods * simple proposals pagination query * chore: create a JoinedProposalBriefEntity * rename method * feat: exclude hidden proposals * more tests * renaming and splitting logic into smaller parts * feat: per language strategy * remove CatalystDatabaseLanguage in favor of raw queries as they are easier to mange * remove Index Strategy Documentation * handle case where ref is empty * migration now includes indexes * use v2 documents table for saveAll and isCachedBulk * adds ActionType to JoinedProposalBriefEntity * adds versionIds to JoinedProposalBriefEntity * comments count * adds isFavorite to JoinedProposalBriefEntity * add template to JoinedProposalBriefEntity * adds documentsLocalMetadata table for auto updates * Update docs * use v2 proposals query for discovery most recent section * feat: simplify most recent proposals section * add proposal fav status for v2 tables * local proposal fav status update * update fav state locally for faster feedback * self review * fix tests * update times * fix: analyzer * more migration test data * clean up constructors * cleanup * fix: template tests * spelling * fix: spelling * chore: PR review adjustments * add order parameter * ProposalsOrder docs * GetProposalsBriefPage supports order * add filters object * proposalsBriefPage filtering * fix: status filtering * more draft proposals filtering tests * Campaign proposals filter * update docs * expose getVisibleProposalsCount and tests * expose filters parameter * integrate proposals page with v2 queries * chore: increase time diff between proposals * chore: reduce count query tables watched when not needed * local proposals cubit * local proposal fav staus update * docs * fix: add discovery specific colors (#3637) * fix code-generator earthly target * use logger in migration + wrap in transaction * spelling * rename category to categoryId for better consistency * proposals per tab selector * release completed in close * extract early return logic into function --------- Co-authored-by: Ryszard Schossler <[email protected]> * feat(cat-voices): voting space (#3648) * docs: capture initial times * chore: remove cacheDocument * turn off logging * chore: use debugPrint instead of logger * feat: bulk documents save * batching sync * chore: move exact ref resoling to getDocumentData instead of index * fix: DocumentRepository * chore: simplify getting documents data * remove getAllDocumentsRefs and getCachedDocumentsRefs from DocumentRepository. Index is only available in DocumentDataRemoteSource * chore: update docs * simplified document index endpoint * remove randomness from LocalCatGateway * indexing by batch size * wip: filtering by types * Rework filtering refs + checking all refs if already cached in parallel * docs * update indexing.csv * update indexing csv * chore: cleanup * trailing new line * chore * move performance tab to docs * bulk saving typed docs in parallel * chore: revert hardcoded timestamp * chore: typos * split _sync into smaller functions + add documentation * little refactor * fix: analyzer * initial v2 tables * wip * wip * feat: database migration * chore: cleanup * bump batch size * cleanup * chore: remove defensive content decoding * chore: daos * spelling * saveAll * test on platform * chore: update build scripts * feat: DocumentsV2Dao methods * simple proposals pagination query * chore: create a JoinedProposalBriefEntity * rename method * feat: exclude hidden proposals * more tests * renaming and splitting logic into smaller parts * feat: per language strategy * remove CatalystDatabaseLanguage in favor of raw queries as they are easier to mange * remove Index Strategy Documentation * handle case where ref is empty * migration now includes indexes * use v2 documents table for saveAll and isCachedBulk * adds ActionType to JoinedProposalBriefEntity * adds versionIds to JoinedProposalBriefEntity * comments count * adds isFavorite to JoinedProposalBriefEntity * add template to JoinedProposalBriefEntity * adds documentsLocalMetadata table for auto updates * Update docs * use v2 proposals query for discovery most recent section * feat: simplify most recent proposals section * add proposal fav status for v2 tables * local proposal fav status update * update fav state locally for faster feedback * self review * fix tests * update times * fix: analyzer * more migration test data * clean up constructors * cleanup * fix: template tests * spelling * fix: spelling * chore: PR review adjustments * add order parameter * ProposalsOrder docs * GetProposalsBriefPage supports order * add filters object * proposalsBriefPage filtering * fix: status filtering * more draft proposals filtering tests * Campaign proposals filter * update docs * expose getVisibleProposalsCount and tests * expose filters parameter * integrate proposals page with v2 queries * chore: increase time diff between proposals * chore: reduce count query tables watched when not needed * local proposals cubit * local proposal fav staus update * docs * fix: add discovery specific colors (#3637) * fix code-generator earthly target * use logger in migration + wrap in transaction * spelling * wip * rename category to categoryId for better consistency * adding vote data to ProposalBriefData * fix: dependencies * voting page + cubit integration - wip * proposals per tab selector * release completed in close * extract early return logic into function * Voting tabs selector + releasing completer on close * move spaces blocs to shell page level * fix: voting in grid view * dummy voting integration * chore: remove showComments * docs: ProposalOrDocument --------- Co-authored-by: Ryszard Schossler <[email protected]> * feat(cat-voices): database queries cleanup (#3672) * docs: capture initial times * chore: remove cacheDocument * turn off logging * chore: use debugPrint instead of logger * feat: bulk documents save * batching sync * chore: move exact ref resoling to getDocumentData instead of index * fix: DocumentRepository * chore: simplify getting documents data * remove getAllDocumentsRefs and getCachedDocumentsRefs from DocumentRepository. Index is only available in DocumentDataRemoteSource * chore: update docs * simplified document index endpoint * remove randomness from LocalCatGateway * indexing by batch size * wip: filtering by types * Rework filtering refs + checking all refs if already cached in parallel * docs * update indexing.csv * update indexing csv * chore: cleanup * trailing new line * chore * move performance tab to docs * bulk saving typed docs in parallel * chore: revert hardcoded timestamp * chore: typos * split _sync into smaller functions + add documentation * little refactor * fix: analyzer * initial v2 tables * wip * wip * feat: database migration * chore: cleanup * bump batch size * cleanup * chore: remove defensive content decoding * chore: daos * spelling * saveAll * test on platform * chore: update build scripts * feat: DocumentsV2Dao methods * simple proposals pagination query * chore: create a JoinedProposalBriefEntity * rename method * feat: exclude hidden proposals * more tests * renaming and splitting logic into smaller parts * feat: per language strategy * remove CatalystDatabaseLanguage in favor of raw queries as they are easier to mange * remove Index Strategy Documentation * handle case where ref is empty * migration now includes indexes * use v2 documents table for saveAll and isCachedBulk * adds ActionType to JoinedProposalBriefEntity * adds versionIds to JoinedProposalBriefEntity * comments count * adds isFavorite to JoinedProposalBriefEntity * add template to JoinedProposalBriefEntity * adds documentsLocalMetadata table for auto updates * Update docs * use v2 proposals query for discovery most recent section * feat: simplify most recent proposals section * add proposal fav status for v2 tables * local proposal fav status update * update fav state locally for faster feedback * self review * fix tests * update times * fix: analyzer * more migration test data * clean up constructors * cleanup * fix: template tests * spelling * fix: spelling * chore: PR review adjustments * add order parameter * ProposalsOrder docs * GetProposalsBriefPage supports order * add filters object * proposalsBriefPage filtering * fix: status filtering * more draft proposals filtering tests * Campaign proposals filter * update docs * expose getVisibleProposalsCount and tests * expose filters parameter * integrate proposals page with v2 queries * chore: increase time diff between proposals * chore: reduce count query tables watched when not needed * local proposals cubit * local proposal fav staus update * docs * fix: add discovery specific colors (#3637) * fix code-generator earthly target * use logger in migration + wrap in transaction * spelling * wip * rename category to categoryId for better consistency * adding vote data to ProposalBriefData * fix: dependencies * voting page + cubit integration - wip * proposals per tab selector * release completed in close * extract early return logic into function * Voting tabs selector + releasing completer on close * move spaces blocs to shell page level * fix: voting in grid view * dummy voting integration * chore: remove showComments * docs: ProposalOrDocument * create proposals/count query .csv files * update proposals query times * category indexes * CatId parse and tryParse * create new tables for authors * update migration * run database analyze after sync * DocumentAuthors docs * simpler copyWith * cat.id keeps schema * get rid of authorCatIdWithoutUsername column * console profiler * catId -> id * times * results notes * spelling * fix: analyzer * update results (#3673) * fix: readme * remove CatalystRuntimeProfiler * catId schema -> scheme * rename * bring back profiler to SignedDocumentManager, with debounce --------- Co-authored-by: Ryszard Schossler <[email protected]> * feat(cat-voices): campaign total ask (#3734) * remove totalAsk and proposalsCount from campaign model * proposalsCount -> finalProposalsCount * CampaignCategoryViewModel.id -> CampaignCategoryViewModel.ref * little DiscoveryCubit state build cleanup * watch campaign total ask in discovery * smaller voting category model * new proposal campaign * safe check * proposal builder category total ask * chore: missing TODO * category details * watch proposals template total ask * finish integration of total_ask * close * remove print and add TODO * documents getLatestOf * categoryId -> categoryRef * chore: make logger private * feat: Make MultiCurrencyAmount immutable * chore(cat-voices): remove old daos (#3745) * remove totalAsk and proposalsCount from campaign model * proposalsCount -> finalProposalsCount * CampaignCategoryViewModel.id -> CampaignCategoryViewModel.ref * little DiscoveryCubit state build cleanup * watch campaign total ask in discovery * smaller voting category model * new proposal campaign * safe check * proposal builder category total ask * chore: missing TODO * category details * watch proposals template total ask * finish integration of total_ask * close * remove print and add TODO * documents getLatestOf * remove old tables and daos * deleteWhere tests * remove old pagination and count methods * local draft documents dao * categoryId -> categoryRef * chore: notInType -> typeNotIn * chore: move DriftDocumentsV2LocalMetadataDao setup into group * separate get and getWhere * docs: documents sources interfaces documentation * chore: methods rename * chore: refTo -> referencing * chore: typeNotIn -> excludeTypes * update content * feat(cat-voices): smaller proposals query scope (#3747) * smaller proposals page query * update PR nr * test(cat-voices): enable back index tests (#3766) * chore: remote source tests * SyncManager and DocumentService tests * chore(cat-voices): cleanup miss leading "ref" / "selfRef" with "id" (#3762) * remove totalAsk and proposalsCount from campaign model * proposalsCount -> finalProposalsCount * CampaignCategoryViewModel.id -> CampaignCategoryViewModel.ref * little DiscoveryCubit state build cleanup * watch campaign total ask in discovery * smaller voting category model * new proposal campaign * safe check * proposal builder category total ask * chore: missing TODO * category details * watch proposals template total ask * finish integration of total_ask * close * remove print and add TODO * documents getLatestOf * remove old tables and daos * deleteWhere tests * remove old pagination and count methods * local draft documents dao * categoryId -> categoryRef * chore: notInType -> typeNotIn * chore: move DriftDocumentsV2LocalMetadataDao setup into group * separate get and getWhere * docs: documents sources interfaces documentation * chore: methods rename * chore: refTo -> referencing * chore: typeNotIn -> excludeTypes * update content * feat(cat-voices): smaller proposals query scope (#3747) * smaller proposals page query * update PR nr * silence warning * chore: version -> ver * fix: ref test * chore: remove id and version getters from DocumentDataMetadata * selfRef -> id part1 * chore: selfRef -> id * chore: use more of ids * chore: use Map.of instead of Map.from * dtos migration tests * fix: analyzer --------- Co-authored-by: Ryszard Schossler <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Proposals query filtering options.
Related Issue(s)
Part of #3528
Description of Changes
ProposalsCubit, build and close, locally byProposalsPageto fee up resourcesProposalsFiltersV2and integration queries withProposalsPagePlan for following PRs
VotingPageCampaignfounds requested + categories funds requests queryDemo
proposals_page.mov
Related Pull Requests
Must be merged after #3614
Please confirm the following checks