Skip to content

Add custom docker image support #1193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Add custom docker image support #1193

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
34ae07e
Add custom docker image support
devsnek Nov 24, 2020
947bda1
patch dep
devsnek Nov 24, 2020
0c7d7e4
squash! fix dir
devsnek Nov 24, 2020
52e6065
delete image after build
devsnek Nov 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions crates/metadata/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ pub enum MetadataError {
/// targets = [ "x86_64-apple-darwin", "x86_64-pc-windows-msvc" ]
/// rustc-args = [ "--example-rustc-arg" ]
/// rustdoc-args = [ "--example-rustdoc-arg" ]
/// docker-image = "rustops/crates-build-env"
/// ```
///
/// You can define one or more fields in your `Cargo.toml`.
Expand Down Expand Up @@ -128,6 +129,9 @@ pub struct Metadata {
/// List of command line arguments for `rustdoc`.
#[serde(default)]
rustdoc_args: Vec<String>,

/// Custom docker image.
docker_image: Option<String>,
}

/// The targets that should be built for a crate.
Expand Down Expand Up @@ -277,6 +281,11 @@ impl Metadata {
map.insert("DOCS_RS", "1".into());
map
}

/// Return the custom docker image, if provided.
pub fn docker_image(&self) -> Option<String> {
self.docker_image.clone()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's return a borrow here, since it's not dynamic.

Suggested change
pub fn docker_image(&self) -> Option<String> {
self.docker_image.clone()
pub fn docker_image(&self) -> Option<&str> {
self.docker_image.as_ref()

}
}

impl std::str::FromStr for Metadata {
Expand Down
28 changes: 20 additions & 8 deletions src/docbuilder/rustwide_builder.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,11 +106,15 @@ impl RustwideBuilder {
self.skip_build_if_exists = should;
}

fn prepare_sandbox(&self, limits: &Limits) -> SandboxBuilder {
SandboxBuilder::new()
fn prepare_sandbox(&self, metadata: &Metadata, limits: &Limits) -> Result<SandboxBuilder> {
let mut builder = SandboxBuilder::new()
.cpu_limit(self.config.build_cpu_limit.map(|limit| limit as f32))
.memory_limit(Some(limits.memory()))
.enable_networking(limits.networking())
.enable_networking(limits.networking());
if let Some(image) = metadata.docker_image() {
builder = builder.image(SandboxImage::remote(&image)?)
}
Ok(builder)
}

pub fn update_toolchain(&mut self) -> Result<()> {
Expand Down Expand Up @@ -208,11 +212,15 @@ impl RustwideBuilder {
let krate = Crate::crates_io(DUMMY_CRATE_NAME, DUMMY_CRATE_VERSION);
krate.fetch(&self.workspace)?;

let metadata = Metadata::from_crate_root(&build_dir.build_dir())?;

build_dir
.build(&self.toolchain, &krate, self.prepare_sandbox(&limits))
.build(
&self.toolchain,
&krate,
self.prepare_sandbox(&metadata, &limits)?,
)
.run(|build| {
let metadata = Metadata::from_crate_root(&build.host_source_dir())?;

let res = self.execute_build(HOST_TARGET, true, build, &limits, &metadata)?;
if !res.result.successful {
failure::bail!("failed to build dummy crate for {}", self.rustc_version);
Expand Down Expand Up @@ -322,14 +330,18 @@ impl RustwideBuilder {

let local_storage = tempfile::Builder::new().prefix("docsrs-docs").tempdir()?;

let metadata = Metadata::from_crate_root(&build_dir.build_dir())?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, are you sure this is right? I think builds happen in a temporary directory, not in the source directory.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested it, but Build::host_source_dir returns self.dir.source_dir() so i just made that method public.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have build_dir here, not source_dir - is that intentional?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately this won't work. source_dir() is populated during the run() call (build.rs => prepare.rs), so it's going to be empty where you call it. What we could do instead is to make rustwide's Crate::copy_source_to method public, copy the source to a temporary directory, fetch the metadata from it, and then delete the temporary directory.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about a BuildDir::create_or_get_source_dir (or whatever name) which both docs.rs and run() can call so we don't waste cpu cycles moving everything twice

let res = build_dir
.build(&self.toolchain, &krate, self.prepare_sandbox(&limits))
.build(
&self.toolchain,
&krate,
self.prepare_sandbox(&metadata, &limits)?,
)
.run(|build| {
use docsrs_metadata::BuildTargets;

let mut has_docs = false;
let mut successful_targets = Vec::new();
let metadata = Metadata::from_crate_root(&build.host_source_dir())?;
let BuildTargets {
default_target,
other_targets,
Expand Down