Dev (#952)

* add task to write relatedness check to tsv (#930)

* add task to write relatedness check to tsv

* fix requirements

* relatedness_check_table_path

* add relatedness check file path to metadata.json

* Benb/use metadata as source of family table load (#936)

* use run metadata as source of family table load

* ruff

* Support gcs dirs in rsync (#932)

* Support gcs dirs in rsync

* ws

* Gencode refactor to remove gcs (#934)

* Gencode refactor to remove gcs

* Fix

* additional semi join (#947)

* metadata parameters refactor (#946)

* metadata parameters refactor

* fix missing param

* tweak

* missed one

* last one

* fix test

* last few bugfixes

* fix

* bump

* missed one

* change parameter type due to confusing bug

* push

* enum

* Parse clinvar version from header (#949)

* Parse clinvar version from header

* responses activate

* fix test

* Dependency reordering so that `ValidateCallsetTask` runs before updating the reference data. (#950)

* Parse clinvar version from header

* Dependency reordering for reference data updates and validation

* ruff

* missed one

* Revert relatedness changes

* push

* Fix import issue

* Fix sample type

* ruff

* Fix import mocking

* imports

* responses activate

* fix test

* Tweaks

* comment

* Benb/check parsed clinvar version in complete (#951)

* Parse clinvar version from header

* First pass

* Bump hail tables to https

* correct dataset/dataset types

* Fix clinvar mito

* Fix combined

* Dependency reordering for reference data updates and validation

* ruff

* missed one

* Revert relatedness changes

* push

* Fix import issue

* Fix sample type

* ruff

* Fix import mocking

* imports

* Missed one

* First mocking pass

* Finish mocks in reference data

* responses activate

* ruff

* commas

* fix test

* Update compare_globals.py

* import

---------

Co-authored-by: Julia Klugherz <[email protected]>

Author: bpblanken

pyproject.toml

@@ -29,7 +29,6 @@ extend-exclude = [
     'luigi_pipeline/lib/*',
     'luigi_pipeline/seqr*.py',
     'luigi_pipeline/tests/data/*',
-    'v03_pipeline/lib/reference_data/gencode/*',
 ]
 ignore = [
     # Individual Rules

requirements.in

@@ -3,6 +3,5 @@ google-api-python-client>=1.8.0
 hail==0.2.132
 luigi>=3.4.0
 gnomad==0.6.4
-google-cloud-storage>=2.14.0
 aiofiles==24.1.0
 pydantic==2.8.2