[Core] StreamingObjectRefGenerator not working over network

[Famok]

What happened + What you expected to happen

StreamingObjectRefGenerator works when using ray locally (using ray.init()) or on a local cluster (conneciting using ray.init('auto')
but not when connecting over network using ray.init('ray://...:10001). it will then throw a TypeError: 'str' object cannot be interpreted as an integer error.

The script below throws this error message when connecting PC B to PC A (Head Node) and executing either the same script or accessing the actor by name.

Traceback (most recent call last):
  File "<myenv>\lib\site-packages\IPython\core\interactiveshell.py", line 3460, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-8-7fe69e992715>", line 1, in <module>
    test = a.stream.remote()
  File "<myenv>\lib\site-packages\ray\util\client\common.py", line 544, in remote
    return return_refs(ray.call_remote(self, *args, **kwargs))
  File "<myenv>\lib\site-packages\ray\util\client\api.py", line 100, in call_remote
    return self.worker.call_remote(instance, *args, **kwargs)
  File "<myenv>\lib\site-packages\ray\util\client\worker.py", line 562, in call_remote
    return self._call_schedule_for_task(task, num_returns)
  File "<myenv>\lib\site-packages\ray\util\client\worker.py", line 575, in _call_schedule_for_task
    id_futures = [Future() for _ in range(num_return_refs)]
**TypeError: 'str' object cannot be interpreted as an integer**

For me this looks like the StreamingObjectRefGenerator cannot be used of network?

Versions / Dependencies

ray context:

ClientContext(dashboard_url='xxx:8265', python_version='3.10.13', ray_version='2.8.1', ray_commit='82a8df138fe7fcc5c42536ebf26e8c3665704fee', protocol_version='2023-06-27', _num_clients=1, _context_to_restore=<ray.util.client._ClientContext object at 0x000001A0359049D0>)

Windows

Edition	Windows 10 Enterprise
Version	22H2
Installiert am	‎22.‎09.‎2021
Betriebssystembuild	19045.3693
Leistung	Windows Feature Experience Pack 1000.19053.1000.0

Packages

# Name                    Version                   Build  Channel
adwin                     0.18.0                   pypi_0    pypi
aiohttp                   3.8.6                    pypi_0    pypi
aiohttp-cors              0.7.0                    pypi_0    pypi
aiorwlock                 1.3.0                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
alabaster                 0.7.13             pyhd8ed1ab_0    conda-forge
ansicon                   1.89.0                   pypi_0    pypi
anyio                     3.7.1                    pypi_0    pypi
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asgiref                   3.7.2                    pypi_0    pypi
astroid                   2.14.2          py310haa95532_0    defaults
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
async-timeout             4.0.3                    pypi_0    pypi
atomicwrites              1.4.1              pyhd8ed1ab_0    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
autopep8                  2.0.4              pyhd8ed1ab_0    conda-forge
babel                     2.13.1             pyhd8ed1ab_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
bcrypt                    4.0.1           py310h87d50f1_1    conda-forge
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
binaryornot               0.4.4                      py_1    conda-forge
black                     23.10.1         py310h5588dad_0    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
blessed                   1.20.0                   pypi_0    pypi
brotli-python             1.1.0           py310h00ffb61_1    conda-forge
bzip2                     1.0.8                hcfcfb64_5    conda-forge
ca-certificates           2023.7.22            h56e8100_0    conda-forge
cachetools                5.3.2                    pypi_0    pypi
certifi                   2023.7.22                pypi_0    pypi
cffi                      1.16.0          py310h8d17308_0    conda-forge
chardet                   5.2.0           py310h5588dad_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
colorful                  0.5.5                    pypi_0    pypi
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
contourpy                 1.2.0                    pypi_0    pypi
cookiecutter              2.4.0              pyhca7485f_0    conda-forge
cramjam                   2.7.0                    pypi_0    pypi
cryptography              41.0.5          py310h6e82f81_0    conda-forge
cycler                    0.12.1                   pypi_0    pypi
debugpy                   1.8.0           py310h00ffb61_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
diff-match-patch          20230430           pyhd8ed1ab_0    conda-forge
dill                      0.3.7              pyhd8ed1ab_0    conda-forge
distlib                   0.3.7                    pypi_0    pypi
django                    4.2.7                    pypi_0    pypi
django-extensions         3.2.3                    pypi_0    pypi
docstring-to-markdown     0.13               pyhd8ed1ab_0    conda-forge
docutils                  0.20.1          py310h5588dad_2    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
fastapi                   0.104.1                  pypi_0    pypi
fastparquet               2023.10.1                pypi_0    pypi
filelock                  3.13.1                   pypi_0    pypi
flake8                    6.0.0           py310haa95532_0    defaults
fonttools                 4.44.0                   pypi_0    pypi
frozenlist                1.4.0                    pypi_0    pypi
fsspec                    2023.10.0                pypi_0    pypi
google-api-core           2.12.0                   pypi_0    pypi
google-auth               2.23.4                   pypi_0    pypi
googleapis-common-protos  1.61.0                   pypi_0    pypi
gpustat                   1.1.1                    pypi_0    pypi
grpcio                    1.59.3                   pypi_0    pypi
h11                       0.14.0                   pypi_0    pypi
httptools                 0.6.1                    pypi_0    pypi
icu                       73.2                 h63175ca_0    conda-forge
idna                      3.6                      pypi_0    pypi
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
importlib_resources       6.1.0              pyhd8ed1ab_0    conda-forge
inflection                0.5.1              pyh9f0ad1d_0    conda-forge
intervaltree              3.1.0              pyhd8ed1ab_1    conda-forge
ipykernel                 6.26.0             pyha63f2e9_0    conda-forge
ipython                   8.17.2             pyh5737063_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
isort                     5.12.0             pyhd8ed1ab_1    conda-forge
jaraco.classes            3.3.0              pyhd8ed1ab_0    conda-forge
jedi                      0.18.1          py310h5588dad_1    conda-forge
jellyfish                 1.0.1           py310h87d50f1_1    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
jinxed                    1.2.0                    pypi_0    pypi
jpeg                      9e                   hcfcfb64_3    conda-forge
jsonschema                4.20.0                   pypi_0    pypi
jsonschema-specifications 2023.11.2                pypi_0    pypi
jupyter_client            8.6.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.5.0           py310h5588dad_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
keyring                   24.2.0          py310h5588dad_1    conda-forge
kiwisolver                1.4.5                    pypi_0    pypi
krb5                      1.20.1               heb0366b_0    conda-forge
lazy-object-proxy         1.9.0           py310h8d17308_1    conda-forge
libclang                  14.0.6          default_hb5a9fac_1    defaults
libclang13                14.0.6          default_h8e68704_1    defaults
libffi                    3.4.2                h8ffe710_5    conda-forge
libpng                    1.6.39               h19919ed_0    conda-forge
libpq                     12.15                h906ac69_1    defaults
libsodium                 1.0.18               h8d14728_1    conda-forge
libspatialindex           1.9.3                h39d44d4_4    conda-forge
libsqlite                 3.44.0               hcfcfb64_0    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
llvmlite                  0.41.1                   pypi_0    pypi
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3           py310h8d17308_1    conda-forge
matplotlib                3.8.1                    pypi_0    pypi
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mccabe                    0.7.0              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.0              pyhd8ed1ab_0    conda-forge
mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
more-itertools            10.1.0             pyhd8ed1ab_0    conda-forge
msgpack                   1.0.7                    pypi_0    pypi
multidict                 6.0.4                    pypi_0    pypi
mypy_extensions           1.0.0              pyha770c72_0    conda-forge
nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
nbconvert                 7.11.0             pyhd8ed1ab_0    conda-forge
nbconvert-core            7.11.0             pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.11.0             pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
numba                     0.58.1                   pypi_0    pypi
numpy                     1.26.2                   pypi_0    pypi
numpydoc                  1.5.0              pyhd8ed1ab_0    conda-forge
nvidia-ml-py              12.535.133               pypi_0    pypi
opencensus                0.11.3                   pypi_0    pypi
opencensus-context        0.1.3                    pypi_0    pypi
opencv-python             4.8.1.78                 pypi_0    pypi
openssl                   3.1.4                hcfcfb64_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.2                    pypi_0    pypi
pandoc                    3.1.3                h57928b3_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
paramiko                  3.3.1              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pathspec                  0.11.2             pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0                   pypi_0    pypi
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
ply                       3.11                       py_1    conda-forge
prometheus-client         0.19.0                   pypi_0    pypi
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
protobuf                  4.25.1                   pypi_0    pypi
psutil                    5.9.6                    pypi_0    pypi
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py-spy                    0.3.14                   pypi_0    pypi
pyarrow                   6.0.1                    pypi_0    pypi
pyasn1                    0.5.1                    pypi_0    pypi
pyasn1-modules            0.3.0                    pypi_0    pypi
pycodestyle               2.10.0             pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydantic                  1.10.13                  pypi_0    pypi
pydocstyle                6.3.0              pyhd8ed1ab_0    conda-forge
pydot                     1.4.2                    pypi_0    pypi
pydot-ng                  2.0.0                    pypi_0    pypi
pyflakes                  3.0.1           py310haa95532_0    defaults
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pylint                    2.16.2          py310haa95532_0    defaults
pylint-venv               3.0.3              pyhd8ed1ab_0    conda-forge
pyls-spyder               0.4.0              pyhd8ed1ab_0    conda-forge
pynacl                    1.5.0           py310h635b8f1_3    conda-forge
pyparsing                 3.1.1                    pypi_0    pypi
pyqt                      5.15.10         py310hd77b12b_0    defaults
pyqt5-sip                 12.13.0         py310h2bbff1b_0    defaults
pyqtwebengine             5.15.10         py310hd77b12b_0    defaults
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.10.13         h4de0772_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-dotenv             1.0.0                    pypi_0    pypi
python-fastjsonschema     2.18.1             pyhd8ed1ab_0    conda-forge
python-graphviz           0.20.1                   pypi_0    pypi
python-lsp-black          1.3.0              pyhd8ed1ab_0    conda-forge
python-lsp-jsonrpc        1.1.2              pyhd8ed1ab_0    conda-forge
python-lsp-server         1.7.2           py310haa95532_0    defaults
python-slugify            8.0.1              pyhd8ed1ab_2    conda-forge
python-statemachine       2.1.2                    pypi_0    pypi
python_abi                3.10                    4_cp310    conda-forge
pytoolconfig              1.2.5              pyhd8ed1ab_0    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pywin32                   306             py310h00ffb61_2    conda-forge
pywin32-ctypes            0.2.2           py310h5588dad_1    conda-forge
pyyaml                    6.0.1           py310h8d17308_1    conda-forge
pyzmq                     25.1.1          py310h2849c00_2    conda-forge
qdarkstyle                3.0.3              pyhd8ed1ab_0    conda-forge
qstylizer                 0.2.2              pyhd8ed1ab_0    conda-forge
qt-main                   5.15.2              h19c9488_10    defaults
qt-webengine              5.15.9               h5bd16bc_7    defaults
qtawesome                 1.2.3              pyhd8ed1ab_0    conda-forge
qtconsole                 5.4.2           py310haa95532_0    defaults
qtpy                      2.4.1              pyhd8ed1ab_0    conda-forge
ray                       2.8.1                    pypi_0    pypi
referencing               0.31.1                   pypi_0    pypi
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
rich                      13.6.0             pyhd8ed1ab_0    conda-forge
rope                      1.11.0             pyhd8ed1ab_0    conda-forge
rpds-py                   0.13.2                   pypi_0    pypi
rsa                       4.9                      pypi_0    pypi
rtree                     1.1.0           py310h1cbd46b_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
sip                       6.7.12          py310h00ffb61_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smart-open                6.4.0                    pypi_0    pypi
sniffio                   1.3.0                    pypi_0    pypi
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
sphinx                    7.2.6              pyhd8ed1ab_0    conda-forge
sphinxcontrib-applehelp   1.0.7              pyhd8ed1ab_0    conda-forge
sphinxcontrib-devhelp     1.0.5              pyhd8ed1ab_0    conda-forge
sphinxcontrib-htmlhelp    2.0.4              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1              pyhd8ed1ab_0    conda-forge
sphinxcontrib-qthelp      1.0.6              pyhd8ed1ab_0    conda-forge
sphinxcontrib-serializinghtml 1.1.9              pyhd8ed1ab_0    conda-forge
spyder                    5.4.3           py310haa95532_1    defaults
spyder-kernels            2.4.4           win_pyh7428d3b_0    conda-forge
sqlite                    3.44.0               hcfcfb64_0    conda-forge
sqlparse                  0.4.4                    pypi_0    pypi
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
starlette                 0.27.0                   pypi_0    pypi
text-unidecode            1.3                pyhd8ed1ab_1    conda-forge
textdistance              4.5.0              pyhd8ed1ab_0    conda-forge
three-merge               0.1.1              pyh9f0ad1d_0    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomlkit                   0.12.2             pyha770c72_0    conda-forge
tornado                   6.3.3           py310h8d17308_1    conda-forge
traitlets                 5.13.0             pyhd8ed1ab_0    conda-forge
types-python-dateutil     2.8.19.14          pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023.3                   pypi_0    pypi
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
ujson                     5.8.0           py310h00ffb61_0    conda-forge
urllib3                   2.1.0                    pypi_0    pypi
uvicorn                   0.24.0.post1             pypi_0    pypi
vc                        14.3                h64f974e_17    conda-forge
vc14_runtime              14.36.32532         hdcecf7f_17    conda-forge
virtualenv                20.21.0                  pypi_0    pypi
vs2015_runtime            14.36.32532         h05e6639_17    conda-forge
watchdog                  3.0.0           py310h5588dad_1    conda-forge
watchfiles                0.21.0                   pypi_0    pypi
wcwidth                   0.2.12                   pypi_0    pypi
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websockets                12.0                     pypi_0    pypi
whatthepatch              1.0.5              pyhd8ed1ab_0    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
wrapt                     1.15.0          py310h8d17308_1    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
yapf                      0.24.0                     py_0    conda-forge
yarl                      1.9.2                    pypi_0    pypi
zeromq                    4.3.5                h63175ca_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hcfcfb64_5    conda-forge
zstd                      1.5.5                h12be248_0    conda-forge

Reproduction script

Executing this script on any machine works (creates a local cluster)

PC A:

import ray
ray.init() # or ray.init("auto") if a local cluster is allready running
@ray.remote
class streamer:
    @ray.method(num_returns="streaming")
    def stream(self):
        for i in range(10):
            yield i

a = streamer.options(name='streamer',namespace='somenamespace').remote()
a.stream.remote()

When i try to do the same on PC B but connect to PC A:

import ray
ray.init("ray://<ip pc a>:10001")
@ray.remote
class streamer:
    @ray.method(num_returns="streaming")
    def stream(self):
        for i in range(10):
            yield i

a = streamer.remote()
a.stream.remote()

or getting the same actor using

a = ray.get_actor(name='streamer',namespace='somenamespace')
a.stream.remote()

Issue Severity

High: It blocks me from completing my task.

[jjyao]

StreamingObjectRefGenerator doesn't work with Ray client for now so the error is expected.

[Famok]

Thanks for the clarification, have I missed that in the documentation?

What is the intended use case for the streaming operator then? Can it be used from node to node? Or do I have to use dynamic (if that work for clients)?

[Famok]

Sorry for digging up this issue, I'm still confused:
I still want to use generators. I've read https://docs.ray.io/en/latest/ray-core/ray-generator.html#generators it does not mention it doesn't work as a client.

I can run the examples when executed using a local machine (using ray.init() to start one) but does not work when connection to the same or remote machine using ray.init('ray://...').

@ray.remote
def task():
    for i in range(5):
        time.sleep(5)
        yield i

task.remote()

It should yield a <ray._raylet.ObjectRefGenerator at 0x24c784784f0>.
but executing it as a client it throws AttributeError: 'ObjectRefGenerator' object has no attribute 'binary'.

trying 'gen = task.remote() is even weirder, because this won't throw an exception but when I try iterating:

for g in gen:
    print(g)

it will tell me: 'TypeError: 'ClientObjectRef' object is not iterable'

Complete Traceback from task.remote():

Traceback (most recent call last):
  File "c:\environments\ray\lib\site-packages\IPython\core\formatters.py", line 223, in catch_format_error
    r = method(self, *args, **kwargs)
  File "c:\environments\ray\lib\site-packages\IPython\core\formatters.py", line 708, in __call__
    printer.pretty(obj)
  File "c:\environments\ray\lib\site-packages\IPython\lib\pretty.py", line 410, in pretty
    return _repr_pprint(obj, self, cycle)
  File "c:\environments\ray\lib\site-packages\IPython\lib\pretty.py", line 778, in _repr_pprint
    output = repr(obj)
  File "python\ray\includes/unique_ids.pxi", line 79, in ray._raylet.BaseID.__repr__
  File "c:\environments\ray\lib\site-packages\ray\util\client\common.py", line 121, in hex
    self._wait_for_id()
  File "c:\environments\ray\lib\site-packages\ray\util\client\common.py", line 194, in _wait_for_id
    self._set_id(self._id_future.result(timeout=timeout))
  File "c:\environments\ray\lib\concurrent\futures\_base.py", line 458, in result
    return self.__get_result()
  File "c:\environments\ray\lib\concurrent\futures\_base.py", line 403, in __get_result
    raise self._exception
AttributeError: 'ObjectRefGenerator' object has no attribute 'binary'

[jjyao]

Hi @Famok,

Is it possible to not use Ray client but Ray job submission? We currently don't encourage people to use Ray client anymore so we didn't spend effort to support streaming generator for ray client.

I'll update the doc to explicit mention that Ray client is not supported.

[Famok]

Hi @jjyao,

Thanks for the clarification! Either I missed it, or it its absent from most of the documentation that there is a functional difference between client and job submission.
May I ask why you don't want to support the client anymore? I find the interactive usage a great feature, because I don't need to ssh to my node and can directly do stuff on my machine too. (As likereviewing the results or saving results to a network storages my node has no access to, because of network policies).

[anyweez]

I recently ran into the same issue and found that it's now possible to use generators in client mode, at least in some cases. Here are the related unit tests demonstrating how it works.

This following script works for me on Ray 2.31.0. The very surprising piece I was missing was the requirement to set num_returns="dynamic":

import ray
import ray.data


@ray.remote
def generator():
    for i in range(10):
        yield i

def main():
    gen = generator.options(num_returns='dynamic').remote()
    for num in ray.get(gen):
        print(ray.get(num))

if __name__ == '__main__':
    ray.init(
        address="ray://my-cluster:10001"
    )

    main()