Skip to content

Add parameter validation #577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 3, 2025

Conversation

spline2hg
Copy link
Contributor

@spline2hg spline2hg commented Jun 24, 2025

Added parameter validation for acquisition functions

Summary by CodeRabbit

  • New Features
    • Added support for custom acquisition functions in Bayesian optimization setup.
  • Bug Fixes
    • Improved input validation for acquisition function parameters to ensure values are within valid ranges, preventing misconfiguration and enhancing reliability.

Copy link

coderabbitai bot commented Jun 24, 2025

"""

Walkthrough

Input validation has been added to the constructors of three acquisition function classes in the bayes_opt/acquisition.py file. These validations check that parameters such as xi, exploration_decay, and exploration_decay_delay meet specific constraints, raising descriptive ValueErrors if violated. Additionally, the BayesianOptimization class constructor in bayes_opt/bayesian_optimization.py was extended with an optional acquisition_function parameter to allow custom acquisition functions. Corresponding tests were added to verify these validations. No other logic or API changes were made.

Changes

File(s) Change Summary
bayes_opt/acquisition.py Added input validation in constructors for UpperConfidenceBound, ProbabilityOfImprovement, and ExpectedImprovement classes.
bayes_opt/bayesian_optimization.py Added optional acquisition_function parameter to BayesianOptimization constructor with updated docstring.
tests/test_acquisition.py Added parameterized tests to validate error handling for invalid inputs in acquisition function constructors.

Poem

In the garden of code where the logic grows,
Rabbits check inputs before the function flows.
No more sneaky values slipping through the gate,
Validation now ensures parameters are great!
And now a choice to pick the function's way,
Custom hops to guide the Bayesian play. 🐇✨
"""


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1169da5 and 44b57e8.

📒 Files selected for processing (2)
  • bayes_opt/acquisition.py (4 hunks)
  • tests/test_acquisition.py (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • bayes_opt/acquisition.py
🔇 Additional comments (8)
tests/test_acquisition.py (8)

380-385: Well-implemented parameter validation test for exploration_decay.

The test correctly validates that exploration_decay must be in the range (0, 1] by testing boundary and invalid values. The parametrized approach efficiently covers multiple edge cases.


388-393: Comprehensive validation test for exploration_decay_delay parameter.

The test appropriately validates that exploration_decay_delay must be a non-negative integer by testing negative values, non-integer types, and float values. Good coverage of edge cases.


396-399: Correct validation test for xi parameter in ProbabilityOfImprovement.

The test properly validates that xi must be non-negative by testing negative values, which aligns with the PR discussion that xi=0 should be allowed.


402-407: Consistent exploration_decay validation for ProbabilityOfImprovement.

The test mirrors the validation logic for UpperConfidenceBound, ensuring consistent parameter validation across different acquisition functions.


410-415: Appropriate exploration_decay_delay validation for ProbabilityOfImprovement.

The test maintains consistency with the UpperConfidenceBound validation while correctly testing the parameter within the ProbabilityOfImprovement context.


418-421: Proper xi parameter validation for ExpectedImprovement.

The test ensures consistent validation behavior across both ProbabilityOfImprovement and ExpectedImprovement classes for the xi parameter.


424-429: Consistent exploration_decay validation across all acquisition functions.

The test maintains uniform validation logic for exploration_decay across UpperConfidenceBound, ProbabilityOfImprovement, and ExpectedImprovement classes.


432-437: Complete exploration_decay_delay validation coverage.

The test completes the comprehensive parameter validation coverage by testing exploration_decay_delay for ExpectedImprovement, ensuring all three acquisition functions have consistent validation behavior.

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
bayes_opt/acquisition.py (4)

455-460: Good parameter validation, but fix line length issue.

The validation logic is mathematically sound and uses descriptive error messages. However, there's a line length issue that needs to be addressed.

Apply this diff to fix the line length:

-        if exploration_decay_delay is not None and (not isinstance(exploration_decay_delay, int) or exploration_decay_delay < 0):
+        if exploration_decay_delay is not None and (
+            not isinstance(exploration_decay_delay, int) or exploration_decay_delay < 0
+        ):

613-622: Good validation logic, but address formatting issues.

The parameter validation is correct and consistent with other acquisition functions. The xi > 0 check is appropriate for an exploration parameter.

Apply this diff to fix line length and remove extra blank line:

-        if exploration_decay_delay is not None and (not isinstance(exploration_decay_delay, int) or exploration_decay_delay < 0):
+        if exploration_decay_delay is not None and (
+            not isinstance(exploration_decay_delay, int) or exploration_decay_delay < 0
+        ):
             error_msg = "exploration_decay_delay must be an integer greater than or equal to 0."
             raise ValueError(error_msg)
-

797-806: Fix formatting issues and consider code duplication.

The validation logic is correct and consistent with other acquisition functions.

Apply this diff to fix line length and remove whitespace from blank line:

-        if exploration_decay_delay is not None and (not isinstance(exploration_decay_delay, int) or exploration_decay_delay < 0):
+        if exploration_decay_delay is not None and (
+            not isinstance(exploration_decay_delay, int) or exploration_decay_delay < 0
+        ):
             error_msg = "exploration_decay_delay must be an integer greater than or equal to 0."
             raise ValueError(error_msg)
-        
+

Note: Consider extracting the repeated validation logic into a helper function to reduce code duplication across the three acquisition function classes.


455-460: Consider refactoring to reduce code duplication.

The validation logic is duplicated across all three acquisition function classes. Consider extracting this into a helper function for better maintainability.

Here's a suggested refactor:

def _validate_common_params(
    exploration_decay: float | None = None,
    exploration_decay_delay: int | None = None,
    xi: float | None = None,
) -> None:
    """Validate common acquisition function parameters."""
    if xi is not None and xi <= 0:
        raise ValueError("xi must be greater than 0.")
    
    if exploration_decay is not None and not (0 < exploration_decay <= 1):
        raise ValueError("exploration_decay must be greater than 0 and less than or equal to 1.")
    
    if exploration_decay_delay is not None and (
        not isinstance(exploration_decay_delay, int) or exploration_decay_delay < 0
    ):
        raise ValueError("exploration_decay_delay must be an integer greater than or equal to 0.")

Then each constructor could call:

_validate_common_params(
    exploration_decay=exploration_decay, 
    exploration_decay_delay=exploration_decay_delay,
    xi=xi  # only for ProbabilityOfImprovement and ExpectedImprovement
)

Also applies to: 613-622, 797-806

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8e7a286 and a344b8a.

📒 Files selected for processing (1)
  • bayes_opt/acquisition.py (3 hunks)
🧰 Additional context used
🪛 Ruff (0.11.9)
bayes_opt/acquisition.py

458-458: Line too long (129 > 110)

(E501)


619-619: Line too long (129 > 110)

(E501)


803-803: Line too long (129 > 110)

(E501)


806-806: Blank line contains whitespace

Remove whitespace from blank line

(W293)

@spline2hg
Copy link
Contributor Author

@till-m Just to confirm — xi should be positive, right? I noticed that in the tests we're creating acquisition function instances with xi=0. Since we're adding validation, should we update those test cases accordingly?

@till-m
Copy link
Member

till-m commented Jun 25, 2025

I did some quick googling (and thinking) and I think xi=0 is possibly reasonable, whereas negative xi is theoretically possible but probably never useful. For now I propose we allow anything non-negative :)

Sidenote: I realize that one of the decay_delay's has lost it's docstring (presumably that is my fault): link. Would you mind also fixing that in this PR?

Also, feel free to tag me when you push changes since I need to manually approve the workflow runs.

@till-m till-m changed the title Add paramter validation Add parameter validation Jun 25, 2025
Copy link

codecov bot commented Jun 27, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.81%. Comparing base (8e7a286) to head (44b57e8).
Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #577      +/-   ##
==========================================
+ Coverage   97.76%   97.81%   +0.04%     
==========================================
  Files          10       10              
  Lines        1164     1188      +24     
==========================================
+ Hits         1138     1162      +24     
  Misses         26       26              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@spline2hg
Copy link
Contributor Author

spline2hg commented Jun 27, 2025

@till-m For the type annotations, would you recommend doing it like this?

kappa: Annotated[float, "non-negative"] = 2.576
exploration_decay: Annotated[float, "range (0, 1]"] | None = None
exploration_decay_delay: Annotated[int, "non-negative"] | None = None

Or should we define reusable types in a separate typing module, like:

NonNegativeFloat = Annotated[float, "non-negative"]

we can also use annotated_types for Ge and Gt , what do u recommend ?

@till-m
Copy link
Member

till-m commented Jul 2, 2025

Hey @spline2hg, how do you feel about not additionally annotating the types? It's technically still a float, right?

annotated_types for Ge and Gt

What are Ge and Gt?

@spline2hg
Copy link
Contributor Author

yes it would still be float, so how do we have type hints for NonNegativeFloat and UnitInterval ?

What are Ge and Gt?

Greater than equal to and Greater than

@spline2hg
Copy link
Contributor Author

@till-m Just to clarify — are you saying that we can keep the types as they are, like:

kappa: float = 2.576
exploration_decay: float | None = None
exploration_decay_delay: int | None = None

and not necessarily use any NonNegativeFloat or UnitInterval?

@till-m
Copy link
Member

till-m commented Jul 2, 2025

yes, exactly

@spline2hg
Copy link
Contributor Author

okay so is there anything more i can add in this pr?

@till-m
Copy link
Member

till-m commented Jul 3, 2025

No, all good! Thanks for your contribution! :)

@till-m till-m merged commit d60b99d into bayesian-optimization:master Jul 3, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants