Skip to content

chore: Add troubleshooting entry for Fusion non-resync and env var workaround #770

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

chore: Add troubleshooting entry for Fusion non-resync and env var workaround #770

wants to merge 5 commits into from

Conversation

robnewman
Copy link
Member

Describe problem and using FUSION_REFRESH_TIMEOUT env var workaround

@robnewman robnewman requested a review from justinegeffen August 8, 2025 18:37
@netlify Netlify
Copy link

netlify bot commented Aug 8, 2025 •
edited
Loading

✅ Deploy Preview for seqera-docs ready!

Name Link
🔨 Latest commit 628ee79
🔍 Latest deploy log https://app.netlify.com/projects/seqera-docs/deploys/689a5bac33c6a50008d24c93
😎 Deploy Preview https://deploy-preview-770--seqera-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@robnewman robnewman requested a review from robsyme August 8, 2025 18:37
robsyme
robsyme reviewed Aug 8, 2025
View reviewed changes
Copy link
Member

@robsyme robsyme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but would be helpful to clarify where the FUSION_REFRESH_TIMEOUT environment needs to be set.

platform-cloud/docs/troubleshooting_and_faqs/studios_troubleshooting.md Outdated
@@ -74,6 +74,14 @@ ERROR system error 2 (No such file or directory) [path:/sys/fs/cgroup/memory/mem

This is displayed because logging is set to `stderr` by default to ensure all logs are shown during the session, and can safely be ignored.

## Running session does not show new data in object storage

By default, Fusion does not resync objects from the remotely mounted data-link(s) after initial mounting. This means that when running a session with data mounted, if the underlying storage updates, the data is not re-synced to the studio session. This behavior can be changed by setting the `FUSION_REFRESH_TIMEOUT` environment variable in your studio session to a number of seconds. This will force Fusion to refresh the view of the mounted data-link(s).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting the env "in your studio session" may be misleading. Setting the env var inside a running session with "export FUSION_REFRESH_TIMEOUT" won't change the behaviour of the outer fusion session. My understanding is that it needs to be set outside of the fusion process (so it can be inherited by fusion). Setting FUSION_REFRESH_TIMEOUT in the Compute Environment, or as a variable while configuring the Studio will work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting FUSION_REFRESH_TIMEOUT in the Compute Environment

I don't think we want to recommend this approach, because of the potential unknown impact if a user inadvertently runs a pipeline in that CE. I'll update to be more explicit that it needs to be added as an env var when the session is added.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, pushed, and re-review requested.

@robnewman robnewman requested a review from robsyme August 8, 2025 20:56
@justinegeffen justinegeffen added 1. Editor review Needs a language review 1. Dev/PM/SME Needs a review by a Dev/PM/SME labels Aug 8, 2025
justinegeffen
justinegeffen reviewed Aug 11, 2025
View reviewed changes
platform-cloud/docs/troubleshooting_and_faqs/studios_troubleshooting.md
@@ -77,6 +77,18 @@ ERROR system error 2 (No such file or directory) [path:/sys/fs/cgroup/memory/mem

This is displayed because logging is set to `stderr` by default to ensure all logs are shown during the session, and can safely be ignored.

## Running session does not show new data in object storage

By default, Fusion does not resync objects from the remotely mounted data-link(s) after initial mounting. This means that when running a session with data mounted, if the underlying storage updates, the data is not re-synced to the studio session. This behavior can be changed when [adding a studio session](../studios/managing#add-a-studio) by defining the `FUSION_REFRESH_TIMEOUT` environment variable to N number of seconds (e.g. `30`). This will force Fusion to refresh the view of the mounted data-link(s) every N seconds.
Copy link
Contributor

@justinegeffen justinegeffen Aug 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
By default, Fusion does not resync objects from the remotely mounted data-link(s) after initial mounting. This means that when running a session with data mounted, if the underlying storage updates, the data is not re-synced to the studio session. This behavior can be changed when [adding a studio session](../studios/managing#add-a-studio) by defining the `FUSION_REFRESH_TIMEOUT` environment variable to N number of seconds (e.g. `30`). This will force Fusion to refresh the view of the mounted data-link(s) every N seconds.
By default, Fusion does not resync objects from remotely mounted data-link(s) after initial mounting.
If you have a running session with data mounted and the underlying storage is updated, the data will not be resynced to the Studio session.
You can change this behavior when you are [adding a Studio session](../studios/managing#add-a-studio) by defining the `FUSION_REFRESH_TIMEOUT` environment variable to a specified number of seconds (e.g., `30`). This will force Fusion to refresh the view of the mounted data-link(s) at the specified interval.

justinegeffen
justinegeffen reviewed Aug 11, 2025
View reviewed changes
platform-cloud/docs/troubleshooting_and_faqs/studios_troubleshooting.md
By default, Fusion does not resync objects from the remotely mounted data-link(s) after initial mounting. This means that when running a session with data mounted, if the underlying storage updates, the data is not re-synced to the studio session. This behavior can be changed when [adding a studio session](../studios/managing#add-a-studio) by defining the `FUSION_REFRESH_TIMEOUT` environment variable to N number of seconds (e.g. `30`). This will force Fusion to refresh the view of the mounted data-link(s) every N seconds.

:::note
Setting the environment variable _inside_ an already running studio session by executing the command `export FUSION_REFRESH_TIMEOUT=30` won't change the behaviour of the outer Fusion session.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Setting the environment variable _inside_ an already running studio session by executing the command `export FUSION_REFRESH_TIMEOUT=30` won't change the behaviour of the outer Fusion session.
Setting the environment variable _inside_ an already running Studio session by executing the command `export FUSION_REFRESH_TIMEOUT=30` won't change the behavior of the outer Fusion session.

justinegeffen
justinegeffen reviewed Aug 11, 2025
View reviewed changes
...m-enterprise_versioned_docs/version-25.2/troubleshooting_and_faqs/studios_troubleshooting.md
@@ -77,6 +77,18 @@ ERROR system error 2 (No such file or directory) [path:/sys/fs/cgroup/memory/mem

This is displayed because logging is set to `stderr` by default to ensure all logs are shown during the session, and can safely be ignored.

## Running session does not show new data in object storage

By default, Fusion does not resync objects from the remotely mounted data-link(s) after initial mounting. This means that when running a session with data mounted, if the underlying storage updates, the data is not re-synced to the studio session. This behavior can be changed when [adding a studio session](../studios/managing#add-a-studio) by defining the `FUSION_REFRESH_TIMEOUT` environment variable to N number of seconds (e.g. `30`). This will force Fusion to refresh the view of the mounted data-link(s) every N seconds.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
By default, Fusion does not resync objects from the remotely mounted data-link(s) after initial mounting. This means that when running a session with data mounted, if the underlying storage updates, the data is not re-synced to the studio session. This behavior can be changed when [adding a studio session](../studios/managing#add-a-studio) by defining the `FUSION_REFRESH_TIMEOUT` environment variable to N number of seconds (e.g. `30`). This will force Fusion to refresh the view of the mounted data-link(s) every N seconds.
By default, Fusion does not resync objects from remotely mounted data-link(s) after initial mounting.
If you have a running session with data mounted and the underlying storage is updated, the data will not be resynced to the Studio session.
You can change this behavior when you are [adding a Studio session](../studios/managing#add-a-studio) by defining the `FUSION_REFRESH_TIMEOUT` environment variable to a specified number of seconds (e.g., `30`). This will force Fusion to refresh the view of the mounted data-link(s) at the specified interval.

justinegeffen
justinegeffen reviewed Aug 11, 2025
View reviewed changes
...m-enterprise_versioned_docs/version-25.2/troubleshooting_and_faqs/studios_troubleshooting.md
By default, Fusion does not resync objects from the remotely mounted data-link(s) after initial mounting. This means that when running a session with data mounted, if the underlying storage updates, the data is not re-synced to the studio session. This behavior can be changed when [adding a studio session](../studios/managing#add-a-studio) by defining the `FUSION_REFRESH_TIMEOUT` environment variable to N number of seconds (e.g. `30`). This will force Fusion to refresh the view of the mounted data-link(s) every N seconds.

:::note
Setting the environment variable _inside_ an already running studio session by executing the command `export FUSION_REFRESH_TIMEOUT=30` won't change the behaviour of the outer Fusion session.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Setting the environment variable _inside_ an already running studio session by executing the command `export FUSION_REFRESH_TIMEOUT=30` won't change the behaviour of the outer Fusion session.
Setting the environment variable _inside_ an already running Studio session by executing the command `export FUSION_REFRESH_TIMEOUT=30` won't change the behavior of the outer Fusion session.

justinegeffen
justinegeffen requested changes Aug 11, 2025
View reviewed changes
Copy link
Contributor

@justinegeffen justinegeffen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some copy and structure suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@justinegeffen justinegeffen justinegeffen requested changes

@robsyme robsyme Awaiting requested review from robsyme

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
1. Dev/PM/SME Needs a review by a Dev/PM/SME 1. Editor review Needs a language review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@robnewman @robsyme @justinegeffen