more dynamically allocated mem

[a-frantz]

Some of our tasks allocate a large static amount of RAM that is often an over-allocation for many inputs.
One example here: https://github.com/stjudecloud/workflows/blob/main/tools/ngsderive.wdl#L338

[DuyLe0710edu]

Hi the team, I am Duy , a sophomore Bucknell Computer Science & Computer Engineering. I am really sorry to contribute to the project this late because I was having a family issue during March. I have experienced in full-stack and some ML/DL research. I also contribute to some project on github and I want to be the GSOCs 2025 for Jude Research Hospital this year. I have experienced with medical application and familiar with hospital settings. I will also put my LinkedIn here: https://www.linkedin.com/in/duy-le-dev/

[DuyLe0710edu]

Can you assign me this issues ? I think I can get this, RAM over-allocation is the issue and I have handled one of them before. I came up with a possible solution of we can implement the dynamic memory allocation inside star.wdl because the start using the fixed memory of 50GB allocation so it should be adequate for that. Please correct me if I am wrong.

[adthrasher]

@DuyLe0710edu - This issue is lacking some information. To properly implement dynamic memory allocation where we have not previously implemented it, someone will need to either dig in to the underlying tool implementations to understand the memory consumption as a function of the input data size or run a significant amount of real world data through to profile the memory usage. The latter will be difficult for an outside contributor as we cannot share patient data in an uncontrolled manner due to privacy concerns. Without St. Jude data, a contributor would need to gather external data sources and determine that they are likely to perform reasonably similar to St. Jude data.

[DuyLe0710edu]

Thank you for the feedback on the dynamic memory allocation implementation. I understand the challenges you've outlined, and here's how I propose it. I think I will make it simple by using multiplication values in memory allocation formulas that intentionally allocate more memory than might be minimally required. If you can provide me the mock data or the format of that I think it could be easier. Do yo recommend any public dataset out there, I can alter the format and try it on. Please assign me this problem.

[a-frantz]

Thank you for the feedback on the dynamic memory allocation implementation. I understand the challenges you've outlined, and here's how I propose it. I think I will make it simple by using multiplication values in memory allocation formulas that intentionally allocate more memory than might be minimally required. If you can provide me the mock data or the format of that I think it could be easier. Do yo recommend any public dataset out there, I can alter the format and try it on. Please assign me this problem.

It is not simple or easy for us to supply "mocked" data for you, and I am unaware of any suitable repositories of genomic data which would be comparable enough to St Jude data for this. We don't have a way to help you with this.

You will not be assigned this issue as you don't have any of the means necessary to complete it. Thank you for your interest.