Reworks balloon for sending balloons in freeze

[tuupke]

Replaces show_balloons_postfreeze with minimum_number_of_balloons which allows for sending out balloons to teams even during the freeze while still supporting the old behavior if desired.

This is useful as a motivational tool for teams that have not yet solved 'enough' problems before entering the freeze. e.g. 2 problems before the freeze while the expected number of problems to solve before the freeze is 3. By setting this new value to 3 the team will receive a balloon as long as the problem they've solved has been solved before the freeze.

Kept on draft until I fix the documentation and do some minor cleanup.

[tuupke]

Mhh come to think of it, we don't really have the full old behavior anymore. Because now problems that are only solved in the freeze will never receive a balloon regardless of the value.

What do you think, should the old config var be reinstated?

[cubercsl]

Possibly related to #2778

[tuupke]

Possibly related to #2778

Yes thank you! I was looking for this exact issue but couldn't find it for some reason. These indeed serve the same purpose.

[tuupke]

I'm quite sure the remarks phpstan makes are incorrect. Since otherwise it would not work at all while it does 🙃

[nickygerritsen]

I'm quite sure the remarks phpstan makes are incorrect. Since otherwise it would not work at all while it does 🙃

Probably the docblocks for the methods being called are wrong.

Use

cd webapp
vendor/bin/phpstan analyze --configuration phpstan.dist.neon

to check locally btw.

[Kevinjil]

Just a general question: has something (e.g. at SEERC) influenced why we want this feature compared to the last time it was discussed IRL at the hackathon?

[tuupke]

Just a general question: has something (e.g. at SEERC) influenced why we want this feature compared to the last time it was discussed IRL at the hackathon?

Partly. Some were just because we we're already looking at this code because of a bug (hence Nicky's PR) Some discussion and talks 'inspired' me on how to implement and that this is a nice feature. It was also the case that there were some teams that only solved some of the easy problems in the freeze. Because this is somewhat common -especially on the early rungs of the tournament ladder- it might be a nice motivational feature to add.

The replacing of the old config flag came after (incorrectly) thinking that this change offered at least the same functionality as before.

Tbh I cannot recall this discussion, but if we're opposed we can close the PR; I don't think it hurts anything while it does add a new functionality. When declining the pr we should cherry-pick 706147e since it DRYs up the retrieving of balloon updates.

[vmcj]

I think we shouldn't merge this without tests. I do agree it happens relatively often but some in the team had strong opinions on such a feature, because of the same opinion we don't test this ourselves at competitions so I think if we ship this we should test it in a different way (with Unit tests for example..)

[eldering]

I'm not in favor of adding this because it adds extra complexity and weird exceptions. For example, what if a teams solves their first problem in the freeze, but it is an as yet unsolved problem? Highly unlikely maybe, but then that exposes crucial information during the freeze. There are likely similar edge-case issues, which I think is the reason we should not add this.

[eldering]

I think a better way to solve this, is when show_balloons_postfreeze is enabled to show clearly which balloons are post-freeze and how many problems the team already solved. Then it becomes an easy non-technical detail to choose which teams to hand a post-freeze balloon out to.

[Kevinjil]

I think a better way to solve this, is when show_balloons_postfreeze is enabled to show clearly which balloons are post-freeze and how many problems the team already solved. Then it becomes an easy non-technical detail to choose which teams to hand a post-freeze balloon out to.

So, this proposal basically shifts the balloon selection from a technical problem to an organizational problem. One could e.g. decide that such balloons are not handed out, unless the contest jury decides to grant it.

[vmcj]

I think a better way to solve this, is when show_balloons_postfreeze is enabled to show clearly which balloons are post-freeze and how many problems the team already solved. Then it becomes an easy non-technical detail to choose which teams to hand a post-freeze balloon out to.

So, this proposal basically shifts the balloon selection from a technical problem to an organizational problem. One could e.g. decide that such balloons are not handed out, unless the contest jury decides to grant it.

I think the balloon runners are not always the most experienced with all the rules and edge cases on when something would disclose information. So in that case I would say we should also include that information (handing this out provides additional crucial information), and if you already have that information you also have the edge cases. So leaving the choice up to the balloon runners feels wrong to me.

[tuupke]

I think we shouldn't merge this without tests. I do agree it happens relatively often but some in the team had strong opinions on such a feature, because of the same opinion we don't test this ourselves at competitions so I think if we ship this we should test it in a different way (with Unit tests for example..)

Sure, I'll go ahead and start adding some tests. Good to have for balloons anyways. Will probably/hopefully also help address some of @eldering's concerns.

[tuupke]

I'm not in favor of adding this because it adds extra complexity and weird exceptions. For example, what if a teams solves their first problem in the freeze, but it is an as yet unsolved problem? Highly unlikely maybe, but then that exposes crucial information during the freeze. There are likely similar edge-case issues, which I think is the reason we should not add this.

This is exactly why these are only shown for problems that have been solved pre-freeze by teams in the same sortorder. So in your example the team would not get a balloon. The entire goal is to not leak any information that was not available pre-freeze while still being able and hand out balloons when desired. I've tried making this abundantly clear in the code and the doc -more in the first than in the second- any suggestions for how to improve?

Adding the testcases as suggested by @vmcj will help discover and prevent regressions on these possible edge-cases. I've given this quite some thought and could not come up with relevant edge-cases within the scope of the feature. I'm open to any and all edge-cases you can think of @eldering. Would be great to address those beforehand.

[tuupke]

I think a better way to solve this, is when show_balloons_postfreeze is enabled to show clearly which balloons are post-freeze and how many problems the team already solved. Then it becomes an easy non-technical detail to choose which teams to hand a post-freeze balloon out to.

So, this proposal basically shifts the balloon selection from a technical problem to an organizational problem. One could e.g. decide that such balloons are not handed out, unless the contest jury decides to grant it.

I think the balloon runners are not always the most experienced with all the rules and edge cases on when something would disclose information. So in that case I would say we should also include that information (handing this out provides additional crucial information), and if you already have that information you also have the edge cases. So leaving the choice up to the balloon runners feels wrong to me.

I completely agree! Solving this at an organizational level is not a good idea. The expectation that (balloon) volunteers are aware of enough intricacies and act accordingly does not hold. We've all organized and been to plenty of contests where this does not even hold for the pre-freeze situation let alone for things that happen post-freeze.

[tuupke]

To add to the edge-case discussion. The following scenario would potentially leak information:

1. Problem A is easy to solve.
2. Team Foo -a good team- focuses on more difficult problems first.
3. Many teams submit correct solutions for A. (pre-freeze)
4. The scoreboard is frozen.
5. Foo submits a correct solution for A.
6. Some information has leaked once Foo receives their balloon for A.

The major issue with this scenario is that it assumes the value of balloons to hand out post freeze is set to a number so high that this well performing team still gets a balloon. Any team with a sufficient number of accepted submissions will not receive any more balloons regardless of which problem they solve.

I'd like to stress that the behavior is disabled by default. Only after setting the configuration variable to a strictly positive value will balloon notifications appear for correct post-freeze submissions. There is thus less potential for information leaking than with the old implementation of handing out all balloons post-freeze.

---

For completeness sake, I will be also be adding back the old behavior of handing out balloons to all accepted post-freeze submissions somehow. Not yet sure whether this is with the old or a new config flag.