Skip to content

feat: UTC-371: Low Agreement Resolution section in Quality Settings #666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

feat: UTC-371: Low Agreement Resolution section in Quality Settings #666

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
114ebdd
SDK regeneration
fern-api[bot] Nov 6, 2025
c6b12a1
SDK regeneration
fern-api[bot] Nov 7, 2025
3f0ec10
SDK regeneration
fern-api[bot] Nov 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 52 additions & 1 deletion .mock/definition/__package__.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,13 @@ types:
Serializer get numbers from project queryset annotation,
make sure, that you use correct one(Project.objects.with_counts())
properties:
agreement_threshold:
type: optional<string>
docs: >-
Minimum percent agreement threshold for which minimum number of
annotators must agree
validation:
pattern: ^-?\d{0,1}(?:\.\d{0,2})?$
allow_stream: boolean
annotation_limit_count:
type: optional<integer>
Expand Down Expand Up @@ -263,6 +270,11 @@ types:
label_config:
type: optional<string>
docs: Label config in XML format. See more about it in documentation
max_additional_annotators_assignable:
type: optional<integer>
docs: >-
Maximum number of additional annotators that can be assigned to a low
agreement task
maximum_annotations:
type: optional<integer>
docs: >-
Expand Down Expand Up @@ -3815,6 +3827,13 @@ types:
Serializer get numbers from project queryset annotation,
make sure, that you use correct one(Project.objects.with_counts())
properties:
agreement_threshold:
type: optional<string>
docs: >-
Minimum percent agreement threshold for which minimum number of
annotators must agree
validation:
pattern: ^-?\d{0,1}(?:\.\d{0,2})?$
annotation_limit_count:
type: optional<integer>
validation:
Expand Down Expand Up @@ -3890,6 +3909,11 @@ types:
label_config:
type: optional<string>
docs: Label config in XML format. See more about it in documentation
max_additional_annotators_assignable:
type: optional<integer>
docs: >-
Maximum number of additional annotators that can be assigned to a low
agreement task
maximum_annotations:
type: optional<integer>
docs: >-
Expand Down Expand Up @@ -4230,6 +4254,13 @@ types:
Serializer for project response, combining all the serializers for
different roles. Don't use it except for Spectacular/Fern definitions.
properties:
agreement_threshold:
type: optional<string>
docs: >-
Minimum percent agreement threshold for which minimum number of
annotators must agree
validation:
pattern: ^-?\d{0,1}(?:\.\d{0,2})?$
allow_stream: boolean
annotation_limit_count:
type: optional<integer>
Expand Down Expand Up @@ -4306,6 +4337,11 @@ types:
label_config:
type: optional<string>
docs: Label config in XML format. See more about it in documentation
max_additional_annotators_assignable:
type: optional<integer>
docs: >-
Maximum number of additional annotators that can be assigned to a low
agreement task
maximum_annotations:
type: optional<integer>
docs: >-
Expand Down Expand Up @@ -4430,6 +4466,13 @@ types:
Serializer get numbers from project queryset annotation,
make sure, that you use correct one(Project.objects.with_counts())
properties:
agreement_threshold:
type: optional<string>
docs: >-
Minimum percent agreement threshold for which minimum number of
annotators must agree
validation:
pattern: ^-?\d{0,1}(?:\.\d{0,2})?$
annotation_limit_count:
type: optional<integer>
validation:
Expand Down Expand Up @@ -4500,6 +4543,11 @@ types:
label_config:
type: optional<string>
docs: Label config in XML format. See more about it in documentation
max_additional_annotators_assignable:
type: optional<integer>
docs: >-
Maximum number of additional annotators that can be assigned to a low
agreement task
maximum_annotations:
type: optional<integer>
docs: >-
Expand Down Expand Up @@ -5741,7 +5789,9 @@ types:
openapi: openapi/openapi.yaml
MetricParam:
properties:
agreement_threshold: string
agreement_threshold:
type: string
availability: deprecated
allowed: string
max_additional_annotators_assignable:
type: optional<integer>
Expand All @@ -5751,6 +5801,7 @@ types:
validation:
min: -2147483648
max: 2147483647
availability: deprecated
metric_name:
type: optional<string>
docs: Evaluation metric chosen for this project
Expand Down
2 changes: 2 additions & 0 deletions .mock/definition/projectTemplates.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,7 @@ service:
workspace_id: 1
response:
body:
agreement_threshold: agreement_threshold
annotation_limit_count: 1
annotation_limit_percent: annotation_limit_percent
annotator_evaluation_minimum_score: annotator_evaluation_minimum_score
Expand Down Expand Up @@ -375,6 +376,7 @@ service:
is_draft: true
is_published: true
label_config: label_config
max_additional_annotators_assignable: 1
maximum_annotations: 1
members: members
members_count: 1
Expand Down
20 changes: 19 additions & 1 deletion .mock/definition/projects.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ service:
next: http://api.example.org/accounts/?page=4
previous: http://api.example.org/accounts/?page=2
results:
- allow_stream: true
- agreement_threshold: agreement_threshold
allow_stream: true
annotation_limit_count: 1
annotation_limit_percent: annotation_limit_percent
annotator_evaluation_minimum_score: annotator_evaluation_minimum_score
Expand Down Expand Up @@ -109,6 +110,7 @@ service:
is_draft: true
is_published: true
label_config: label_config
max_additional_annotators_assignable: 1
maximum_annotations: 1
members: members
members_count: 1
Expand Down Expand Up @@ -450,6 +452,7 @@ service:
id: 1
response:
body:
agreement_threshold: agreement_threshold
allow_stream: true
annotation_limit_count: 1
annotation_limit_percent: annotation_limit_percent
Expand Down Expand Up @@ -488,6 +491,7 @@ service:
is_draft: true
is_published: true
label_config: label_config
max_additional_annotators_assignable: 1
maximum_annotations: 1
members: members
members_count: 1
Expand Down Expand Up @@ -584,6 +588,13 @@ service:
docs: Maximum number of members to return
body:
properties:
agreement_threshold:
type: optional<string>
docs: >-
Minimum percent agreement threshold for which minimum number of
annotators must agree
validation:
pattern: ^-?\d{0,1}(?:\.\d{0,2})?$
annotation_limit_count:
type: optional<integer>
validation:
Expand Down Expand Up @@ -640,6 +651,11 @@ service:
label_config:
type: optional<string>
docs: Label config in XML format. See more about it in documentation
max_additional_annotators_assignable:
type: optional<integer>
docs: >-
Maximum number of additional annotators that can be assigned to
a low agreement task
maximum_annotations:
type: optional<integer>
docs: >-
Expand Down Expand Up @@ -729,6 +745,7 @@ service:
request: {}
response:
body:
agreement_threshold: agreement_threshold
annotation_limit_count: 1
annotation_limit_percent: annotation_limit_percent
annotator_evaluation_minimum_score: annotator_evaluation_minimum_score
Expand Down Expand Up @@ -762,6 +779,7 @@ service:
is_draft: true
is_published: true
label_config: label_config
max_additional_annotators_assignable: 1
maximum_annotations: 1
min_annotations_to_start_training: 1
model_version: model_version
Expand Down
4 changes: 3 additions & 1 deletion .mock/definition/projects/metrics.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ service:
id: integer
display-name: ✨ Update project metrics configuration
request:
name: ProjectMetricParamUpdateRequest
name: MetricParamUpdateRequest
body:
properties:
additional_params: optional<map<string, unknown>>
Expand All @@ -64,10 +64,12 @@ service:
validation:
min: 0
max: 100
availability: deprecated
max_additional_annotators_assignable:
type: optional<integer>
validation:
min: 0
availability: deprecated
metric_name:
type: optional<string>
validation:
Expand Down
4 changes: 3 additions & 1 deletion .mock/definition/prompts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,8 @@ service:
next: http://api.example.org/accounts/?page=4
previous: http://api.example.org/accounts/?page=2
results:
- allow_stream: true
- agreement_threshold: agreement_threshold
allow_stream: true
annotation_limit_count: 1
annotation_limit_percent: annotation_limit_percent
annotator_evaluation_minimum_score: annotator_evaluation_minimum_score
Expand Down Expand Up @@ -365,6 +366,7 @@ service:
is_draft: true
is_published: true
label_config: label_config
max_additional_annotators_assignable: 1
maximum_annotations: 1
members: members
members_count: 1
Expand Down
Loading
Loading