2025 proposal selection process

[nairnandu]

Based on the retrospective and feedback from proposal authors, this is a draft of the 2025 proposal review process to be reviewed by the team

[jgraham]

Based on the feedback from last year, going forward I think we'll have more success if we're more successful at developing a shared understanding of value proposition of different proposals, and where we have room for discussion. Of course that won't always work, but I'd like the process to focus on finding those points of agreement and ensuring that as far as possible we end up with a shared consensus.

With that in mind, here's a different take on what the process could be:

1. After the proposal deadline each participating org shares a list of focus areas they'd like to champion. Those can correspond to one or more proposals. Where different orgs have similar focus area proposals they can work together out of band to develop one proposal, but in each case there should be one primary champion for later rounds. It's also OK to drop proposals at any stage if no one wants to champion it further.
2. Champions work on their focus areas, gathering evidence that the focus area represents an Interop priority (e.g. bug reports, developer feedback, positive platform impact on areas like a11y; we can discuss what people would like to see in advance). They are also responsible for ensuring the focus area has an identified set of tests. We do this in a shared space so that non-champions can provide feedback if desired (e.g. point to relevant standards-positions).
3. We have a fixed amount of f2f time (e.g. 15 minutes per participant over two meetings) to formally present the focus area proposals and explain why they're considered a priority for Interop.
4. We have some time for further async adjustments e.g. if there's feedback like "we like X, but can't support it with subfeature Y, please remove that".
5. Each participant now buckets the final proposals as P1, P2, P3 or "veto".
6. A successful outcome at this point looks like lots of proposals where orgs have given similar rankings, and few vetos. We sort by the buckets and pick focus area proposals with lots of P1/P2 rankings.

Some notes:

1. There is no formal elimination stage. It's up to participants to not champion proposals that don't meet the requirements to be accepted.
2. There's no limit on the number of proposals you can champion. It's up to individual orgs whether they'd prefer to put more time into fewer proposals or less time into more proposals. But at the presentation stage you get a fixed time to explain to others why you consider each proposal an Interop priority. This reflects the fact that time is the actual limited resource (both for decision making and implementation).

[foolip]

As discussed in today's meeting, there's some ambiguity here about whether adjusting up and down is the only way proposals end up being accepted or rejected, in particular as we approach the final decision on Dec 19th.

I think allowing for adjusted rankings in response to argument is good, and that will hopefully move a few things into the accepted state.

I would suggest that all remaining limbo proposals are individually decided by consensus, meaning two supporting and none opposed. This ensures that we have a decision on each proposal that reached this stage, and don't implicitly leave out the bottom N if we run out of time, for example.

[jgraham]

I've clarified at the end that we have to actually make a decision about any remaining proposals, that is they can't just be ignored. But I'm really hoping that we don't have to use that. If it's clear that everyone is "meh" about a proposal to the extent that we don't even discuss it that seems like a pretty clear sign that it's not important enough to be part of Interop.

[foolip]

Does majority here mean 4 given that we have 6 participating organizations? The details here might change the number of proposals that are covered by this by a lot.

[jgraham]

I've changed the wording to make it clear that it remains a judgement call, not an algorithm. Given that it's a new process it seems better to allow additional flexibility here.

[dandclark]

1741ff1 removed all the language about which parts of the process are confidential and which are not. I think it's still important that this all be defined beforehand.

Microsoft's position is still that the Interop process should follow the model of web standards development by doing things in the open such that positions and discussions are public. But at the very least, organizations should be able to share which proposals they are championing and why, since revealing that information won't give full visibility into how the final votes and vetoes stack up.

[foolip]

I agree it's important that any confidential parts are explicitly marked. Per our charter we operate in public except where a process document like this one says otherwise.

Google's preference is to operate in public, including our all or positions on specific proposals. If there is some confidentiality in the process, we think it's important to at the very least clarify that it only applies to sharing the positions of other organizations who want them to be confidential. This wording from the original revision of this PR would do the job:

However, organizations can choose to publish their own priorities, for the Interop proposals, outside of the Interop program.

[jgraham]

To be clear there's no intended change to the confidentiality compared to last year i.e. the details around proposal ranking and selection remain confidential to participants.

Of course there's not, and never has been, any restriction on commenting positively or negatively about specific web technologies in general.

I need to update the PR to make that clear.

However, the possibility of specifically allowing people to talk about the proposals they are championing is something I'd also considered, and would be worth discussing as a group.

[dandclark]

Of course there's not, and never has been, any restriction on commenting positively or negatively about specific web technologies in general.

I need to update the PR to make that clear.

However, the possibility of specifically allowing people to talk about the proposals they are championing is something I'd also considered, and would be worth discussing as a group.

Last meeting I believe we were discussing these points but ran out of time. To help focus the discussion for next meeting I've pushed a suggestion for language that further clarifies the first point and opens up sharing of championed proposals. Let's dig into this more Thursday.

[foolip]

Thank you @dandclark! I've included this in the agenda for tomorrow, #674.

The wording you've suggested looks good to me, and is in line with Google's position/preference in #657 (comment).