Skip to content

Conversation

@correctmost
Copy link

Type of Changes

Type
🔨 Refactoring

Description

_is_setuptools_namespace can end up reading the same files over and over.

For example, when running pylint's import-error checks on yt-dlp, ~23,500 redundant open() calls were performed prior to caching.

Closes pylint-dev/pylint#9603.

Stats

Before

Command Mean [s] Min [s] Max [s] Relative
pylint --recursive=y . 17.126 ± 0.084 17.047 17.292 1.00
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    28477    0.140    0.000    0.512    0.000 {built-in method _io.open}

After

Command Mean [s] Min [s] Max [s] Relative
pylint --recursive=y . 16.969 ± 0.065 16.882 17.079 1.00
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     4949    0.047    0.000    0.064    0.000 {built-in method _io.open}

Cache stats at Pylint exit

CacheInfo(hits=23528, misses=1165, maxsize=1024, currsize=1024)

_is_setuptools_namespace can end up reading the same files over
and over.

For example, when running pylint's import-error checks on yt-dlp,
~23,500 redundant open() calls were performed prior to caching.

Closes pylint-dev/pylint#9603.
Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌

@jacobtylerwalls jacobtylerwalls merged commit 8628425 into pylint-dev:main Mar 16, 2025
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E0401 (import-error) checks perform repeated file reads

3 participants