BUG: Remove special-casing for date objects in DatetimeIndex indexing (GH#62158)

[Aniketsy]

This PR resolves (GH#62158) by removing the legacy special-casing for Python date objects in DatetimeIndex indexing logic.

What’s Changed:
Removed the special-case handling of date objects in DatetimeIndex.getitem.

Updated/added tests in pandas/tests/frame/indexing/test_indexing.py to reflect the new, consistent behavior.

Please let me know if my approach or fix needs any improvements . I’m open to feedback and happy to make changes based on suggestions.
Thankyou !

[Aniketsy]

@jbrockmendel hi
Could you please review my PR for the datetime/date indexing deprecation?

I have a question about handling the new deprecation warning
Do I need to add @pytest.mark.filterwarnings("ignore:Indexing/slicing with datetime.date is deprecated") to every individual test that triggers this warning, or is there a preferred way to handle this globally for the test suite?
Also should I update every test that uses the deprecated behavior, or is there a more efficient approach?

Thanks!

[jbrockmendel]

Mostly looks good, will need a whatsnew note

[Aniketsy]

@jbrockmendel Thanks for the feedback! I’ve applied the suggested changes and updated the PR. Please let me know if there’s anything else that needs adjustment.

[jbrockmendel]

Can you get the CI passing?

[Aniketsy]

Sure, I’ll look into the CI failures.

[Aniketsy]

Hi @jbrockmendel , I’ve identified that one of the CI failures is due to the pyarrow import error, which I’ve skipped. However, I’m still having trouble understanding the cause of the other three failed tests. Could you help me take a look?

[jbrockmendel]

test_align_date_objects_with_datetimeindex involves alignment that will change when dates no longer match to datetimes

[Aniketsy]

@jbrockmendel Thanks for the help! I’ve updated the changes, but CI is still failing. I’ll look into this in more detail.

[Aniketsy]

@jbrockmendel i’m stuck- getting 3 check fails, i tried to fix them but haven't been successful. Could you please take a look and guide me on this?

[jbrockmendel]

The test_align_date_objects_with_datetimeindex ones look like the behavior is correct and the test expected needs to be updated (with comment about the history and whatsnew note about the change).

The pyarrow one looks like you need to add a check to exclude pa.types.is_date in a place where it is implicitly (incorrectly) assuming that dtype.kind =="M" \Rightarrow pa.types.is_timestap

[Aniketsy]

@jbrockmendel still getting CI check fails, there are 3 test failure. I've updated as per suggestions.

[jbrockmendel]

I've updated as per suggestions.

Are you sure? The CI logs show pretty clearly what's wrong.

[Aniketsy]

@jbrockmendel i may have missed something, could you please point out where the mistake is ?

[jbrockmendel]

this override really shouldnt be needed

[jbrockmendel]

Look in the CI log:

    def _is_comparable_dtype(self, dtype: DtypeObj) -> bool:
        """
        Can we compare values of the given dtype to our own?
        """
        if isinstance(dtype, ArrowDtype):
            # GH#62277
            if dtype.kind != "M":
                return False
    
            pa_dtype = dtype.pyarrow_dtype
>           if (pa_dtype.tz is None) ^ (self.tz is None):
                ^^^^^^^^^^^
E           AttributeError: 'pyarrow.lib.DataType' object has no attribute 'tz'

[jbrockmendel]

shouldn't this depend on self.dtype?

[Aniketsy]

i have updated with self.dtype.kind or dtype.kind, please correct me if i should only use self.dtype.kind

[jbrockmendel]

i think the condition on L6315 implies that the attribute always exists?

[jbrockmendel]

FAILED pandas/tests/indexes/datetimes/test_indexing.py::TestGetIndexer::test_get_indexer_date_objs - AssertionError: numpy array are different

[jbrockmendel]

i think object dtype is always comparable?

[jbrockmendel]

this seems weird. wouldn't this return False if both are int64 dtypes?

[Aniketsy]

i added checks inside the arrow dtype, also in _should_compare hope I'm not going in the wrong direction. please correct me if I am.