Skip to content

Commit 3a2b6d8

Browse files
authored
[wheel] limit build artifacts duplicated in example directory for ray_cpp (#54465)
## Why are these changes needed? Currently, the built wheels include the `ray/cpp/example/`, which includes a total copy of the `lib` and `include` directories (because they are copied here in `//cpp:ray_cpp_pkg`. This just about doubles the size of the wheels being distributed right now (~35MB for `include` and ~35MB for `lib`) This change modified the `//cpp:ray_cpp_pkg` target to not copy the `lib` and `include` directories into the `example/thirdparty` directory. This example dir is used to generate an example bazel project with `ray cpp --generate-bazel-project-template-to /tmp/ray`. However, the script already finds the "real" `include`/`lib` directories and copies those in at execution time (just deleting the ones we package in anwyway). You can see this happen [here](https://github.com/ray-project/ray/blob/2d1ca70e9507480eac37b65796ae0da1dffc5084/python/ray/scripts/scripts.py#L2621). ## Related issue number none, but can create if preferred ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR. - [x] I've run `scripts/format.sh` to lint the changes in this PR. - [ N/A ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy Run any build as per the docs ``` docker run -ti --rm -e BUILDKITE_COMMIT="$(git rev-parse HEAD)" -e BUILD_ONE_PYTHON_ONLY=py311 -w /ray -v "$(pwd)":/ray quay.io/pypa/manylinux2014_x86_64:2024-07-02-9ac04ee /ray/python/build-wheel-manylinux2014.sh ``` install the wheel, and see `<site>/ray/cpp` has `include` and `lib`, and `example`, but no `example/thirdparty/{include,lib}` then running `ray cpp --generate-bazel-project-template-to /tmp/ray` to see `thirdparty/{include,lib}` was still available there. ``` $ docker run -it --rm -w /ray -v "$(pwd)":/ray python:3.9.23-bookworm /bin/bash ... root@fb2a260f25a6:/ray# pip install ./.whl/ray* Processing ./.whl/ray-3.0.0.dev0-cp39-cp39-manylinux2014_x86_64.whl Processing ./.whl/ray_cpp-3.0.0.dev0-cp39-cp39-manylinux2014_x86_64.whl ... root@fb2a260f25a6:/ray# ray cpp --generate-bazel-project-template-to /tmp/ray Project template generated to /tmp/ray To build and run this template, run cd /tmp/ray && bash run.sh root@fb2a260f25a6:/ray# ls -al /tmp/ray/thirdparty/ total 16 drwxr-xr-x 4 root root 4096 Jul 11 19:52 . drwxr-xr-x 3 root root 4096 Jul 11 19:52 .. drwxr-xr-x 6 root root 4096 Jul 11 19:52 include drwxr-xr-x 2 root root 4096 Jul 11 19:52 lib root@fb2a260f25a6:/ray# ls -al /tmp/ray/thirdparty/lib total 36496 drwxr-xr-x 2 root root 4096 Jul 11 19:52 . drwxr-xr-x 4 root root 4096 Jul 11 19:52 .. -rwxr-xr-x 1 root root 37363304 Jul 11 19:52 libray_api.so root@fb2a260f25a6:/ray# ls -al /tmp/ray/thirdparty/include/ total 32 drwxr-xr-x 6 root root 4096 Jul 11 19:52 . drwxr-xr-x 4 root root 4096 Jul 11 19:52 .. drwxr-xr-x 27 root root 4096 Jul 11 19:52 boost drwxr-xr-x 8 root root 4096 Jul 11 19:52 msgpack -rw-r--r-- 1 root root 526 Jul 11 19:52 msgpack.h -rw-r--r-- 1 root root 648 Jul 11 19:52 msgpack.hpp drwxr-xr-x 2 root root 4096 Jul 11 19:52 nlohmann drwxr-xr-x 3 root root 4096 Jul 11 19:52 ray ``` Signed-off-by: Mark Molinaro <[email protected]>
1 parent 7b07e12 commit 3a2b6d8

File tree

1 file changed

+0
-5
lines changed

1 file changed

+0
-5
lines changed

cpp/BUILD.bazel

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -164,11 +164,6 @@ genrule(
164164
cp -f -r $$WORK_DIR/external/boost/boost/*.hpp "$$BOOST_DIR" &&
165165
cp -f $(locations libray_api.so) "$$PY_CPP_DIR/lib/" &&
166166
cp -f -r "$$WORK_DIR/cpp/include/ray" "$$PY_CPP_DIR/include" &&
167-
THIRDPARTY_DIR="$$WORK_DIR/cpp/example/thirdparty" &&
168-
rm -rf $$THIRDPARTY_DIR &&
169-
mkdir $$THIRDPARTY_DIR &&
170-
cp -f -r "$$PY_CPP_DIR/include" $$THIRDPARTY_DIR &&
171-
cp -f -r "$$PY_CPP_DIR/lib" $$THIRDPARTY_DIR &&
172167
cp -f -r "$$WORK_DIR/cpp/example" "$$PY_CPP_DIR" &&
173168
echo "$$WORK_DIR" > $@
174169
""",

0 commit comments

Comments
 (0)