str + Series -> Never

[zmoon]

Example:

import pandas as pd

df = pd.DataFrame({"col": ["b"]})
reveal_type("a" + df["col"])
reveal_type(df["col"] + "c")
df["new"] = "a" + df["col"] + "c"  # mypy error

> mypy t.py
t.py:4: note: Revealed type is "Never"
t.py:5: note: Revealed type is "Never"
t.py:6: error: Unsupported left operand type for + ("Never")  [operator]
Found 1 error in 1 file (checked 1 source file)

Is this like in the discussion in #1355 , a cast is needed now?

pandas 2.3.2
pandas-stubs 2.3.2.250827

[Dr-Irv]

Is this like in the discussion in #1355 , a cast is needed now?

Yes. See https://github.com/pandas-dev/pandas-stubs/blob/main/docs/philosophy.md#generic-series-have-restricted-arithmetic

[zmoon]

In my real case where I noticed this, I know my Series has pandas string dtype. Is the idea that pd.Series[str] refers to any text dtype? Or will there be a way to refer to pandas string dtype at some point? It's just a bit confusing since .astype(str) specifically results in an object dtype Series (the objects being strs).

[Dr-Irv]

In my real case where I noticed this, I know my Series has pandas string dtype. Is the idea that pd.Series[str] refers to any text dtype? Or will there be a way to refer to pandas string dtype at some point? It's just a bit confusing since .astype(str) specifically results in an object dtype Series (the objects being strs).

You have to separate the internal dtype in pandas from the type used in the stubs.

Consider this code:

from typing import reveal_type
import pandas as pd

s = pd.Series([1,2,3])
ss = s.astype(str)
reveal_type(ss)
print(ss)

At runtime, the output is:

Runtime type is 'Series'
0    1
1    2
2    3
dtype: object

Output from pyright is:

issue1359.py:7:13 - information: Type of "ss" is "Series[str]"

So the type checker sees the result of astype(str) to be Series[str] even though it has dtype object. Note that with pandas 3.0 (hopefully released soon), pandas will have a string dtype. Here's the output of the same program using the dev copy of pandas:

Runtime type is 'Series'
0    1
1    2
2    3
dtype: str

Note how the dtype changed. So by tracking series that contain str now in the stubs, we are better prepared for the future and can do a whole bunch of other type checks with string operations.

[zmoon]

Thanks @Dr-Irv this is helpful!

My question was more about the best way satisfy the type checker using cast. In my case, I know that my Series has dtype pd.StringDtype(). So I would prefer not to use .astype() (though str, "string" and pd.StringDtype() all work to satisfy the type checker). But cast("pd.Series[pd.StringDtype()]", s) does not work, and cast("pd.Series[str]", s) works but feels a bit weird, which was what I was trying to ask about in my previous comment.

If I modify your example to use .astype("string") instead:

Runtime type is 'Series'
0    1
1    2
2    3
dtype: string

[Dr-Irv]

My question was more about the best way satisfy the type checker using cast. In my case, I know that my Series has dtype pd.StringDtype().

You should use cast("pd.Series[str]", s). We are using Series[str] to represent any series holding strings.

If you are using pd.StringDtype() and the stubs are not recognizing that usage to infer that the corresponding series is Series[str], then that's a separate issue to report.

Having said that, if you have a DataFrame variable df and a column "a" that is non-numeric, you must use cast to tell the type checker the type of df["a"]