Parse error if comment # starts in the middle of the line #179
Description
Version: 0.10.0, 0.12.0
To reproduce, the following repository can be used for scanning:
https://github.com/danielhrisca/asammdf
This contains a dependency to numexpr that itself has a line in its requirement.txt has has a comment that starts in the middle of the line:
numpy >= 1.23.0 # keep in sync with NPY_TARGET_VERSION (setup.py)
Here is the output of a scan with python-inspector:
asammdf % python-inspector --python-version 311 --operating-system linux --json-pdt /tmp/my-dependencies.json --analyze-setup-py-insecurely --requirement requirements.txt --verbose
Resolving dependencies...
direct_dependencies:
DependentPackage(purl='pkg:pypi/canmatrix', extracted_requirement='canmatrix[arxml,dbc]>=1.0', scope='install')
DependentPackage(purl='pkg:pypi/lz4', extracted_requirement='lz4', scope='install')
DependentPackage(purl='pkg:pypi/numexpr', extracted_requirement='numexpr', scope='install')
DependentPackage(purl='pkg:pypi/numpy', extracted_requirement='numpy>=1.23.0', scope='install')
DependentPackage(purl='pkg:pypi/pandas', extracted_requirement='pandas', scope='install')
DependentPackage(purl='pkg:pypi/typing-extensions', extracted_requirement='typing_extensions', scope='install')
DependentPackage(purl='pkg:pypi/python-dateutil', extracted_requirement='python-dateutil', scope='install')
DependentPackage(purl='pkg:pypi/isal', extracted_requirement='isal; platform_machine == "x86_64" or platform_machine == "AMD64"', scope='install')
DependentPackage(purl='pkg:pypi/lxml', extracted_requirement='lxml>=4.9.3', scope='install')
environment: Environment(python_version='311', operating_system='linux')
repos:
PypiSimpleRepository(index_url='https://pypi.org/simple', credentials=None)
Traceback (most recent call last):
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/packvers/requirements.py", line 102, in __init__
req = REQUIREMENT.parseString(requirement_string)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/pyparsing/util.py", line 256, in _inner
return fn(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/pyparsing/core.py", line 1200, in parse_string
raise exc.with_traceback(None)
pyparsing.exceptions.ParseException: Expected string_end, found '#' (at char 16), (line:1, col:17)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/resolve_cli.py", line 257, in resolve_dependencies
resolution_result: Dict = resolver_api(
^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/api.py", line 280, in resolve_dependencies
resolution, purls = resolve(
^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/api.py", line 344, in resolve
resolved_dependencies, packages = get_resolved_dependencies(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/api.py", line 385, in get_resolved_dependencies
resolver_results = resolver.resolve(requirements=requirements, max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/resolvelib/resolvers.py", line 427, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/resolvelib/resolvers.py", line 229, in _get_updated_criteria
for requirement in self._p.get_dependencies(candidate=candidate):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/resolution.py", line 652, in get_dependencies
return list(self._iter_dependencies(candidate))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/resolution.py", line 643, in _iter_dependencies
for r in self.get_requirements_for_package(purl=purl, candidate=candidate):
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/resolution.py", line 502, in get_requirements_for_package_from_pypi_simple
yield from get_reqs_insecurely(
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/resolution.py", line 277, in get_reqs_insecurely
yield from parse_reqs_from_setup_py_insecurely(setup_py=setup_py_location)
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/resolution.py", line 128, in parse_reqs_from_setup_py_insecurely
for req in iter_requirements(level="", extras=[], setup_file=setup_py):
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/python_inspector/setup_py_live_eval.py", line 151, in iter_requirements
pkg = Requirement(req)
^^^^^^^^^^^^^^^^
File "/Users/klw1imb/.pyenv/versions/3.11.1/lib/python3.11/site-packages/packvers/requirements.py", line 104, in __init__
raise InvalidRequirement(
packvers.requirements.InvalidRequirement: Parse error at "'# keep i'": Expected string_end
Is it allowed to have comments start in the middle of the line? I guess the Requirements File Format allows it when saying
A line that begins with # is treated as a comment and ignored. Whitespace followed by a # causes the # and the remainder of the line to be treated as a comment.