BUG: Cannot interpret string dtype as a valid data type

[jeremiedbb]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

df = pd.DataFrame([[1,2], [3,4]], columns=["first", "second"])

np.dtype(df.columns.dtype)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[10], line 1
----> 1 np.dtype(df.columns.dtype)

TypeError: Cannot interpret '<StringDtype(storage='python', na_value=nan)>' as a data type

Issue Description

Hi, this issue was caught in scikit-learn's CI (for instance here) a couple of days ago and only involves the dev version of pandas. It looks like there was a recent change in pandas string dtypes that make them not recognized as numpy dtypes.

ping @jorisvandenbossche, I saw that you merged several PRs about pandas strings last week :)

Installed Versions

INSTALLED VERSIONS

commit : 930c8a4
python : 3.13.5
python-bits : 64
OS : Linux
OS-release : 6.8.0-60-generic
Version : #63~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 22 19:00:15 UTC 2
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : fr_FR.UTF-8
LOCALE : fr_FR.UTF-8

pandas : 3.0.0.dev0+2249.g930c8a479d
numpy : 2.3.1
dateutil : 2.9.0.post0
pip : 25.1.1
Cython : 3.1.2
sphinx : None
IPython : 9.4.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : None
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
psycopg2 : None
pymysql : None
pyarrow : None
pyiceberg : None
pyreadstat : None
pytest : 8.4.1
python-calamine : None
pytz : None
pyxlsb : None
s3fs : None
scipy : 1.16.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
qtpy : None
pyqt5 : None

[jorisvandenbossche]

This is actually an intentional change (see https://pandas.pydata.org/docs/dev/whatsnew/v2.3.0.html#upcoming-changes-in-pandas-3-0), and with the default string dtype being a pandas extension dtype, it is then expected behaviour that you get this error in numpy functions that are expecting a np.dtype.

In the migration guide, there is a section about the consequence of the dtype no longer being object dtype (https://pandas.pydata.org/docs/dev/user_guide/migration-3-strings.html#the-dtype-is-no-longer-object-dtype), but probably we should make it more explicit there that it is also no longer a numpy dtype object.

We already have several other data types in pandas that are pandas extension dtypes (eg categorical), so in general sklearn is already equipped to handle those, but I certainly expect there will be some corner cases where this is not yet handled (and specifically the error in the column transformer is for the dtype of the columns, where it is currently much less likely to have a pandas dtype).

Will open a PR with some fixes on the sklearn side

[jorisvandenbossche]

In the migration guide, there is a section about the consequence of the dtype no longer being object dtype (https://pandas.pydata.org/docs/dev/user_guide/migration-3-strings.html#the-dtype-is-no-longer-object-dtype), but probably we should make it more explicit there that it is also no longer a numpy dtype object.

-> #61921

[jorisvandenbossche]

Going to close this as there is further no action to be taken on the pandas side (apart from better documenting, which is one in #61921)