Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
97deea2
feat: primitive parquet reader with page pruning
kszucs Jun 10, 2025
3677975
add poetry build for libviewer
lhoestq Jul 7, 2025
b2d49be
add libviewer to rows
lhoestq Jul 7, 2025
92b633d
refactor: only extract metadata and don't try to calculate offset index
kszucs Sep 8, 2025
d5d4397
ci: update dockerfiles to include the rust toolchain and libviewer
kszucs Sep 15, 2025
faeeb0c
chore: add cargo files to the rows dev docker image
kszucs Sep 15, 2025
1b23098
chore: add build-essentials
kszucs Sep 15, 2025
26242ff
chore: pin python to 3.12.11 in libviewer and update lockfile
kszucs Sep 23, 2025
c01f7bc
feat: use PageIndexPolicy to optionally read offset index
kszucs Oct 10, 2025
1536cd0
feat: support querying RowsIndex with page pruning
kszucs Oct 10, 2025
1492897
build: add libviewer as a dependency to libcommon
kszucs Oct 11, 2025
b15d7fc
style: ruff format libcommon changes
kszucs Oct 11, 2025
d0e7930
chore: use query_with_page_pruning from the rows endpoint
kszucs Oct 11, 2025
c6579ec
chore: fix mypy errors
kszucs Oct 11, 2025
c56021e
style: import Sequence from collections.abc
kszucs Oct 11, 2025
25943b5
build: don't use libviewer as an editable dependency
kszucs Oct 11, 2025
9f15627
build: try to configure poetry to properly install libviewer
kszucs Oct 12, 2025
92b0667
ci: temporarily disable poetry cache
kszucs Oct 12, 2025
802a823
style: fixx ruff check errors
kszucs Oct 12, 2025
191add5
build: relock projects depending on libcommon
kszucs Oct 12, 2025
ff056c6
build: add rust toolchain to more dockerfiles
kszucs Oct 12, 2025
c48408d
build: copy the entire libviewer directory in dockerfiles because poe…
kszucs Oct 12, 2025
7458864
build: add cargo to PATH
kszucs Oct 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,7 @@ assets
**/.pytest_cache
**/.coverage
**/coverage.xml
**/target
**/*.rs.bk
**/.cargo/registry
**/.cargo/git
6 changes: 3 additions & 3 deletions .github/workflows/_unit-tests-python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ env.python-version }}
cache: "poetry"
cache-dependency-path: |
${{ inputs.working-directory }}/poetry.lock
# cache: "poetry"
# cache-dependency-path: |
# ${{ inputs.working-directory }}/poetry.lock
- name: Install packages
run: sudo apt update; sudo apt install -y ffmpeg libavcodec-extra libsndfile1
- name: Install dependencies
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ e2e:

.PHONY: install
install:
$(MAKE) -C libs/libviewer install
$(MAKE) -C libs/libcommon install
$(MAKE) -C libs/libapi install
$(MAKE) -C jobs/cache_maintenance install
Expand Down
37 changes: 36 additions & 1 deletion jobs/mongodb_migration/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 36 additions & 1 deletion libs/libapi/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 36 additions & 2 deletions libs/libcommon/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions libs/libcommon/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ environs = "^14.3.0"
filelock = "^3.18.0"
fsspec = {version = "2024.3.1", extras = ["s3"]}
huggingface-hub = {version = "^0.31.0", extras = ["hf-transfer"]}
libviewer = {path = "../libviewer", develop = true}
mongo-types = "0.15.1"
mongoengine = "^0.29.1"
networkx = "^3.0"
Expand Down
Loading
Loading