Skip to content

Conversation

@samc
Copy link

@samc samc commented Nov 3, 2023

Looks like we stumbled across a bug in our production middleware that was resulting in some unwarranted internal service errors - looks to us like there's an overflow in the min / max value calculation for random mutually exclusive groups.

I'll use an example from our production code that was giving us trouble to explain the problem.

  1. We have a mutually exclusive group (random assignment) with exactly 6 tenants
  2. shortlistedCampaigns would then have their weights adjusted to 16 based on the following:

campaign.weight = Math.floor(100 / shortlistedCampaigns.length);

3) based on that weight, the following would adjust the lower and upper bounds of startVariationAllocation & endVariationAllocation respectively:

const startRange = Math.ceil(variationWeight * 100);

4) this all results in a maximum upper bound of 9600 when it comes time to decide on a variation:

if (bucketValue >= variation.startVariationAllocation && bucketValue <= variation.endVariationAllocation) {
return variation;
}

  1. which, looking at the algo from _generateBucketValue, looks to default to a max of 10000 based on Constants.MAX_TRAFFIC_PERCENT. This leaves a 4% chance that an assigned bucketed value falls outside the acceptable bounds of all active tests in a group and returns null - failing on this line:

campaignKey: winnerCampaign.key

Here's a snapshot of the overflow in action on our production site:

image

@samc samc marked this pull request as ready for review November 3, 2023 13:52
@softvar softvar force-pushed the master branch 2 times, most recently from 33a5f91 to 1d54f4e Compare March 25, 2025 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant