Security considerations for trust_remote_code=True

[JoeGaffney]

Hi Diffusers team,

I’m reaching out with a question and some concerns about the security implications of allowing trust_remote_code=True when loading pipelines via ModularPipeline.from_pretrained(). As recently noticed this being used in PR descriptions.

https://huggingface.co/docs/diffusers/main/en/modular_diffusers/modular_pipeline#creating-a-modularpipeline-with-frompretrained

From what I understand, enabling this option allows execution of arbitrary Python code from remote, which seems similar in risk to running exec() on unvetted code fetched from the internet. This raises potential security concerns around malicious code injection or data leaks.

I’m curious if the team has considered these risks, and whether there are safeguards in place to prevent accidental or silent execution of potentially harmful code. For example:

• Should this feature require more explicit user consent or warnings before being enabled?
• Would adding an additional environment variable or other mechanism to gate this behaviour help improve safety?
• While requiring trust_remote_code=True as an argument is a step toward explicit consent, in practice this alone does not provide strong security. It’s easy for users especially those copying examples or working quickly to enable this flag without fully understanding the risks. This “soft” consent mechanism lacks safeguards against accidental or silent execution of arbitrary remote code.
• Given Diffusers’ widespread use in production, might it be safer overall to reconsider having this feature at all?

I appreciate the flexibility this feature provides, but want to better understand the trade-offs and any best practices recommended to keep users safe.

Thanks for your time and insights. Please feel free to correct me if I’m mistaken about any of this.

[asomoza]

This is an interesting discussion, first I must clarify that this is my own personal opinion but I'm intrigued why do you refer as a "soft" consent to add a specific argument with the value True when loading custom code. This is not like pressing a button by mistake, you have to literally write that part, in my eyes, there's little more that we can do to make this less "soft".

I understand your concerns about copy & paste some code from the tutorials, but my piece of advice here is that you should never copy & paste any code on production without understanding fully what you're doing and reviewing the code your using. For example, the link you posted is for a section that has this Loading custom code is also supported just, the custom code should alert a developer what they're using something risky and if they can use it in production. Tutorials should be used for learning not for grabbing code to use in real use case scenarios.

The env idea seems good but not sure if it will add some security and could be annoying for people that really knows what they're doing, if people are copy & pasting code without knowing what they're doing, copying a env file doesn't seem like something they won't do also.

About this part:

Given Diffusers’ widespread use in production, might it be safer overall to reconsider having this feature at all?

This feature is so people can just use their custom code directly as blocks with modular diffusers, this doesn't mean you should use public anonymous code, you can just use your own code and be able to reuse it with multiple pipelines and scripts, you're not forced to:

• Upload your custom code anywhere.
• Use others people code.
• Use this feature at all

Also you would be surprised on how people view this, we promote a lot the use of the safetensors format which prevents arbitrary code execution when loading checkpoints, but people still use the pickle format that is unsafe an even get mad at us because torch made a change so the weights_only option defaults from False to True and we don't provide an argument to revert this.

But as you can see, even torch.load() still has the option where you can enable it to execute arbitrary code that you don't know where it comes from.

Just so you know, I'm just discussing this and giving my point of view, I would love to read more opinions about this matter, so feel free to discuss further what I'm saying if there's something wrong with it.

[JoeGaffney]

Hey @asomoza

I agree that setting trust_remote_code=True is an explicit action, but I still see it as a soft form of consent when it means running remote code. The problem is this flag can be hidden inside wrapper libraries or copied without users realizing it will execute code from external sources. Especially as this is pulling down live code from the web.

This kind of issue has come up before in ecosystems like NPM and PyPI, where people have exploited similar risks around executing hidden or remote code. I’m just flagging this for discussion because, while it might not be a big risk for me or most people right now, it could become a problem as the ecosystem and number of repos keep growing.

[asomoza]

I see your point, in the case of wrapper libraries, yeah this is a concern if you just install them without reviewing them.

I know that package managers have this issue, there isn't that much they can do either though, you just audit every package and make the ecosystem really slow or just let the users care about security themselves, this is the equivalent to something like redhat versus arch linux, one is really secure with old packages and the other one is on the cutting edge with security issues if you don't know what you're doing.

I think that maybe using a environment variable to enable/disable this globally seems like a good idea then, do you think that's a good solution?

[JoeGaffney]

Hey @asomoza

Sorry took a while to reply it's been a busy week.

I think an opt in ENV controlling it at the lowest level or maybe it would be a more opt out given that it many people are probably relying on it.

Looking as the code a bit more it looks like its actually in the transformers repo that actually would load and run the remote code. Maybe I could mock up some idea of how to control it with an ENV var?

[DN6]

Hi @JoeGaffney when you get a chance, would you be able to point me to the place where the transformers repo would load and run remote code? With the modular blocks, we should only be downloading code from a Diffusers repo. If the repo points to a model that also uses custom code, we don't automatically download that code unless trust_remote_code=True is passed again when loading components.

I think as a first step we can set an opt out env variable and make it very prominent in the modular docs.
Thanks for bringing up this discussion btw.