Huge performance regression in rustdoc between 1/28 and 2/25 nightlies

[jdm]

When using the 1/28 Rust nightly for a Servo CI job on linux, the job that does a variety of builds and unit tests as well as generates docs for every single dependency took 20 minutes total. After updating to the 2/25 nightly, just the doc generation by itself takes 37 minutes, often causing our CI limit of 60 minutes to be exceeded.

[jdm]

Looks like the doc generation used to take 6m 28s.

[euclio]

I added a new docs pass to check codeblock syntax a few days before the 1/28 nightly. The timing's a little off, but you could try running rustdoc without the check-code-block-syntax pass to see if that makes up the difference.

Never mind, that pass is present in the 1/28 nightly.

[Manishearth]

cc @QuietMisdreavus

[Mark-Simulacrum]

It looks like the number of patches landing in master between those two nightlies is actually fairly small when constraining to just src/librustdoc. I might have messed this up though.

f573049 Auto merge of #58232 - ljedrz:HirIdification_continued, r=Zoxc
aadbc45 Auto merge of #57051 - Eijebong:parking_lot, r=nikomatsakis
fcccf06 Auto merge of #56514 - ollie27:rustdoc_test_libdir, r=QuietMisdreavus
32471f7 Auto merge of #58503 - varkor:const-generics-hir, r=petrochenkov
b244f61 Auto merge of #58341 - alexreg:cosmetic-2-doc-comments, r=steveklabnik
0e5a209 Auto merge of #58058 - QuietMisdreavus:use-attr, r=GuillaumeGomez
4b1e39b Auto merge of #57851 - Aaron1011:fix/clean-lifetime, r=GuillaumeGomez

[GuillaumeGomez]

It's very certainly because of my minification process. Although, I didn't expect it to have a noticeable impact...

[QuietMisdreavus]

The only minifier change that landed into rustdoc between those dates was #57884. The minifier was actually in place before then.

[Manishearth]

This may actually be a regression in quote/proc-macro2.

I'm not yet sure if this is the thing causing long doc builds (some things in our server build history disagree with this hypothesis), but our mozjs_sys crate hits a ton of intra-doc-links failures in jsapi.rs, a bindgen-generated file. Normally this is fine, but on some versions of proc-macro2 bindgen is outputting a file that's basically a single giant line (+ some headers). The intra doc link failures have a field day with this, printing out the entire (already giant) file for each warning. There are a lot of such warnings because the autogenerated documentation has things like [[Get]] in it (used in JS specs to talk about properties/methods/slots). This is probably a lot of work for rustc (since it also has to do span computation), as well as a lot of IO traffic, slowing everything down.

[Manishearth]

I've narrowed down the regression range to between 2019-02-07 and 2019-02-10

[Manishearth]

No, the mozjs_sys issues are not the problem. They started becoming a problem at some point, but the commit that did the rust upgrade does not have a giant log, and still takes 55minutes to build

[Manishearth]

These are all the PRs landing in master in that range

d173180 Auto merge of #58266 - GuillaumeGomez:rollup, r=GuillaumeGomez
ad43389 Auto merge of #58010 - Zoxc:parallel-passes, r=michaelwoerister
626e74d Auto merge of #58254 - kennytm:rollup, r=kennytm
825f355 Auto merge of #57998 - niklasf:align-enum, r=nagisa
1efdda1 Auto merge of #58125 - taiki-e:libsyntax-2018, r=Centril
ff9158c Auto merge of #58224 - pietroalbini:revert-appveyor-debug, r=alexcrichton

The first rollup contains a lot of rustdoc PRs.

[Manishearth]

It's probably either #58143 or #57884 from #58266 , verifying that the problematic commit is indeed the rollup (sadly we don't store travis builds for rollup contents)

Servo does have crates with a ton of items in them, so we may be hitting ranges where algorithmic complexity actually matters.

[Manishearth]

Confirmed, the perf regression is inside #58266

[Manishearth]

The regression is caused by #57884 cc @GuillaumeGomez

Unsure what's casing it so far.