Validate that dependencies are in the correct section in pyproject.toml

[ddog-nasirthomas]

What does this PR do?

Update ddev validate dep to validate if third-party dependencies are listed in project.dependencies

Motivation

AI-5661

Third-party libraries go in the [project.optional-dependencies] section instead of [project].

If a dependency is misplaced, CI won’t catch it at the PR stage. As a result, it could be merged into master, leaving it in an unbuildable state.

This adds validation to detect this mistake

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[codecov]

Codecov Report

❌ Patch coverage is 60.37736% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.04%. Comparing base (b9e94e3) to head (5214a5f).
⚠️ Report is 20 commits behind head on master.

Additional details and impacted files

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sarah-witt]

Could you assert the whole error message, maybe with a regex, so we know exactly what error we are expecting?

[ddog-nasirthomas]

Sarah asked over slack to check integrations-extras and marketplace to make sure it works there.

I checked both and they do have quite a few errors. Not sure if we have to fix all of these since they are in the extras and marketplace repos but I will check.

Posting an example below from each repo

Extras:

ddev -e validate dep

upbound_uxp/pyproject.toml has the following errors:
  - Found third-party dependencies in project.dependencies: `kubernetes`. 
     - Third-party dependencies belong in project.optional-dependencies`

Marketplace:

ddev -m validate dep

crest_data_systems_cloudflare_ai_gateway/pyproject.toml has the following errors:
  - Found third-party dependencies in project.dependencies: `datadog-api-client`. 
     - Third-party dependencies belong in project.optional-dependencies

crest_data_systems_cofense_triage/pyproject.toml has the following errors:
  - Found third-party dependencies in project.dependencies: `datadog-api-client`. 
     - Third-party dependencies belong in project.optional-dependencies

crest_data_systems_commvault/pyproject.toml has the following errors:
  - Found third-party dependencies in project.dependencies: `datadog-api-client`. 
     - Third-party dependencies belong in project.optional-dependencies

[github-actions]

⚠️ Recommendation: Add qa/skip-qa label

This PR does not modify any files shipped with the agent.

To help streamline the release process, please consider adding the qa/skip-qa label if these changes do not require QA testing.

[sarah-witt]

Hi @ddog-nasirthomas thanks for checking those repos! And great that the new validation has already caught some errors! Can you fix those now so once we merge and release this, those repos aren't failing the validation?

[ddog-nasirthomas]

Extras PR is here
Marketplace PR is here

[ddog-nasirthomas]

Extras and Marketplace PR got approved. Currently in a code freeze but I will follow up on this PR once they get merged after