BUG: factorize does not preserve extension dtypes

[batterseapower]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
x = pd.Series([1, None], dtype='Int32').to_frame(name='col')

# This is 'Int32Dtype()' as expected
print(pd.MultiIndex.from_frame(x).to_frame()['col'].dtype)

# This is float64
pd.MultiIndex.from_frame(x).factorize()[1].to_frame().iloc[:, 0].dtype

Issue Description

If you factorize an index, it should always be the case that the factorized index has the same dtypes as the original index, but this example shows that sometimes an extension dtype will be dropped and replaced with a more generic one.

(A related bug is that factorize of an Index should preserve column names.)

pd.factorize of a DataFrame with Int32 columns shows similar behaviour.

Expected Behavior

'Int32Dtype()' in both cases

Installed Versions

INSTALLED VERSIONS

commit : 4665c10
python : 3.9.7
python-bits : 64
OS : Darwin
OS-release : 24.6.0
Version : Darwin Kernel Version 24.6.0: Mon Jul 14 11:30:29 PDT 2025; root:xnu-11417.140.69~1/RELEASE_ARM64_T6000
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 2.3.2
numpy : 2.0.2
pytz : 2024.2
dateutil : 2.8.2
pip : 24.3.1
Cython : 0.29.24
sphinx : 4.2.0
IPython : 7.29.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.12.3
blosc : None
bottleneck : 1.4.2
dataframe-api-compat : None
fastparquet : None
fsspec : 2024.6.1
html5lib : 1.1
hypothesis : None
gcsfs : None
jinja2 : 3.1.6
lxml.etree : 5.3.0
matplotlib : 3.9.4
numba : None
numexpr : 2.10.2
odfpy : None
openpyxl : None
pandas_gbq : None
psycopg2 : None
pymysql : None
pyarrow : 19.0.0
pyreadstat : None
pytest : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.13.1
sqlalchemy : 2.0.41
tables : N/A
tabulate : 0.9.0
xarray : None
xlrd : 2.0.1
xlsxwriter : None
zstandard : None
tzdata : 2024.2
qtpy : None
pyqt5 : None

Replace this line with the output of pd.show_versions()

[jorisvandenbossche]

@batterseapower thanks for the report!

That indeed seems to be a place where we don't have full support for extension dtypes. Looking at the implementation of the factorize method, I suppose this is because we don't have a custom implementation for MultiIndex, but use the base one that passes self._values to the factorize algorithm:

pandas/pandas/core/base.py

Lines 1288 to 1295 in cc40732

	def factorize(
	self,
	sort: bool = False,
	use_na_sentinel: bool = True,
	) -> tuple[npt.NDArray[np.intp], Index]:
	codes, uniques = algorithms.factorize(
	self._values, sort=sort, use_na_sentinel=use_na_sentinel
	)

For a Series or simple Index, the _values will be the ExtensionArray. But for a MultiIndex, this _values gives you a 2d numpy array, and so that looses the extension dtype information.

But at least, as a stopgap, for MultiIndex input, it should cast the temporary uniques back to the original dtypes of the input MultiIndex.

[batterseapower]

Interesting. I was concerned that a simple cast might lose info so I implemented a workaround like this:

def factorize(index: pd.Index, sort: bool = False):
    codes, uniques = index.factorize(sort=sort)

    # Work around issues discussed in https://github.com/pandas-dev/pandas/issues/62337
    assert isinstance(uniques, pd.Index)
    if uniques.nlevels == 1 and uniques.dtype == index.dtype:
        # Not preserved for some reason
        uniques.names = index.names
    else:
        example_indexes = np.full(len(uniques), -1, dtype=np.intp)
        example_indexes[codes] = np.arange(len(codes))
        assert (example_indexes >= 0).all()

        uniques = index[example_indexes]

    return codes, uniques

But it seems the correctness of factorize is dependent on _values not losing info anyway (e.g. due to casting ints to float types), so maybe post-casting the result is a fine approach.