# 2023-11-07 Triage Log

A difficult week for triage, due to what appears to be system-level disruption to
measurement apparatus, yielding transient noise (and potentially masking actual
problems). The main non-noise performance change was huge regression to bitmaps introduced
by PR 117131, and that already has a fix in-flight (PR #117542). The other thing
worth noting is that the parallel rustc front-end has been enabled in the nightly builds,
which has introduced some overhead that was expected by wg-parallel-rustc.

Triage done by **@pnkfelix**.
Revision range: [650991d6..7b97a5ca](https://perf.rust-lang.org/?start=650991d62c3a2c80ba27009d06839adbb038bf5e&end=7b97a5ca8422d1495a8918106d3249aa405812d4&absolute=false&stat=instructions%3Au)

**Summary**:

| (instructions:u)                   | mean | range          | count |
|:----------------------------------:|:----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 5.8% | [0.3%, 104.7%] | 234   |
| Regressions ❌ <br /> (secondary)  | 2.7% | [0.5%, 11.6%]  | 219   |
| Improvements ✅ <br /> (primary)   | -    | -              | 0     |
| Improvements ✅ <br /> (secondary) | -    | -              | 0     |
| All ❌✅ (primary)                 | 5.8% | [0.3%, 104.7%] | 234   |


10 Regressions, 4 Improvements, 3 Mixed; 3 of them in rollups
68 artifact comparisons made in total

#### Regressions

Accept less invalid Rust in rustdoc [#117450](https://github.com/rust-lang/rust/pull/117450) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=d7d9f15be205b45d44700e7ac33b4f79b91f9680&end=50be22964035027398754e54cd3a497efb00f83e&stat=instructions:u)

| (instructions:u)                   | mean | range         | count |
|:----------------------------------:|:----:|:-------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 4.1% | [0.6%, 21.0%] | 17    |
| Regressions ❌ <br /> (secondary)  | 3.0% | [1.0%, 7.7%]  | 6     |
| Improvements ✅ <br /> (primary)   | -    | -             | 0     |
| Improvements ✅ <br /> (secondary) | -    | -             | 0     |
| All ❌✅ (primary)                 | 4.1% | [0.6%, 21.0%] | 17    |

* primary regressions are to doc benchmarks, were anticipated, and are unavoidable.
* marking as triaged.


Fix order of implementations in the "implementations on foreign types" section [#117521](https://github.com/rust-lang/rust/pull/117521) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=c5afe0a61e39bdd912803eae8d1887a513bdd172&end=b20f40dba9feabf4b446d9b09d15996f0c046eff&stat=instructions:u)

| (instructions:u)                   | mean | range        | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 2.4% | [0.7%, 3.3%] | 3     |
| Regressions ❌ <br /> (secondary)  | -    | -            | 0     |
| Improvements ✅ <br /> (primary)   | -    | -            | 0     |
| Improvements ✅ <br /> (secondary) | -    | -            | 0     |
| All ❌✅ (primary)                 | 2.4% | [0.7%, 3.3%] | 3     |

* the primary regressions are to diesel-doc-full (3.35%), syn-opt-full (3.07%) and serde-doc-full (0.69%).
* the syn-opt-full "regression" is noise; the other two primary benchmarks are doc; this is rustdoc fix and that team has already said they don't think they can do much to address the situation there.
* marking as triaged.

dropck_outlives check whether generator witness needs_drop [#117134](https://github.com/rust-lang/rust/pull/117134) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e8418e092a643ef87019192af7ed6bbd23cc4a63&end=a2f5f9691b6ce64c1703feaf9363710dfd7a56cf&stat=instructions:u)

| (instructions:u)                   | mean | range        | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.3% | [0.2%, 0.3%] | 5     |
| Regressions ❌ <br /> (secondary)  | -    | -            | 0     |
| Improvements ✅ <br /> (primary)   | -    | -            | 0     |
| Improvements ✅ <br /> (secondary) | -    | -            | 0     |
| All ❌✅ (primary)                 | 0.3% | [0.2%, 0.3%] | 5     |

* regressed ripgrep check for incr-patched:println (0.35%), incr-full (0.26%), and full (0.21%); regressed regex check for incr-patched:Job (0.29%) and incr-patched:println (0.24%).
* this is easily justified; it is fixing a code-acceptance regression
* marking as triaged

Add track_caller to transmute_copy [#117510](https://github.com/rust-lang/rust/pull/117510) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1d6f05fd3735efc8f5af2f6278a3391543c2790b&end=49112241e96aaca646b1aa8ef704e494f56feed4&stat=instructions:u)

| (instructions:u)                   | mean | range        | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.4% | [0.4%, 0.4%] | 3     |
| Regressions ❌ <br /> (secondary)  | -    | -            | 0     |
| Improvements ✅ <br /> (primary)   | -    | -            | 0     |
| Improvements ✅ <br /> (secondary) | -    | -            | 0     |
| All ❌✅ (primary)                 | 0.4% | [0.4%, 0.4%] | 3     |

* regressed bitmaps incr-full for check (0.37%), opt (0.36%), and debug (0.36%).
* this seems likely to be noise. It certainly paled in comparison to the regression associated with #117131 for this benchmark (see below).
* marking as triaged.

Guarantee representation of None in NPO [#115333](https://github.com/rust-lang/rust/pull/115333) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=9c20ddd956426d577d77cb3f57a7db2227a3c6e9&end=1bb6553b967b69eed9ed8147e78b4f65cfc48e11&stat=instructions:u)

| (instructions:u)                   | mean  | range         | count |
|:----------------------------------:|:-----:|:-------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 12.2% | [0.9%, 30.1%] | 8     |
| Regressions ❌ <br /> (secondary)  | -     | -             | 0     |
| Improvements ✅ <br /> (primary)   | -     | -             | 0     |
| Improvements ✅ <br /> (secondary) | -     | -             | 0     |
| All ❌✅ (primary)                 | 12.2% | [0.9%, 30.1%] | 8     |

* this is a docs change.
* There was a spike of noise that somehow significantly (but temporarily) affected three instances of stm32f4 debug, as well as clap opt (all of which immediately went away in the next commit).
* there is more [discussion of why on zulip](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/perf.20is.20slow.20again/near/400329622)
* already marked as triaged.

clone less [#117540](https://github.com/rust-lang/rust/pull/117540) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=5020f7c3b8bd8bf57077389bca5c19f7911fde7a&end=f1b104f52350c800f78d4372aec52080bd6f9164&stat=instructions:u)

| (instructions:u)                   | mean  | range         | count |
|:----------------------------------:|:-----:|:-------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 16.5% | [1.1%, 30.2%] | 3     |
| Regressions ❌ <br /> (secondary)  | -     | -             | 0     |
| Improvements ✅ <br /> (primary)   | -     | -             | 0     |
| Improvements ✅ <br /> (secondary) | -     | -             | 0     |
| All ❌✅ (primary)                 | 16.5% | [1.1%, 30.2%] | 3     |

* the regressions here seem to be correlated with the same spike of noise that afflicted #115333.
* marking as triaged

Run tidy license checker on more workspaces [#115274](https://github.com/rust-lang/rust/pull/115274) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=f1b104f52350c800f78d4372aec52080bd6f9164&end=3aaa0f57b7b877ef58532a8de075d1e5a79142bf&stat=instructions:u)

| (instructions:u)                   | mean | range         | count |
|:----------------------------------:|:----:|:-------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 8.8% | [3.3%, 17.8%] | 3     |
| Regressions ❌ <br /> (secondary)  | -    | -             | 0     |
| Improvements ✅ <br /> (primary)   | -    | -             | 0     |
| Improvements ✅ <br /> (secondary) | -    | -             | 0     |
| All ❌✅ (primary)                 | 8.8% | [3.3%, 17.8%] | 3     |

* the regressions here seem to be correlated with the same spike of noise that afflicted #115333.
* already marked as triaged

Warn users who set `non_exhaustive_omitted_patterns` lint level on a match arm [#117094](https://github.com/rust-lang/rust/pull/117094) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=2db26d3d55387930f1b1dfb84810bcde5a787a09&end=f81d6f0cb00b1e314107b81bcb9aa72b9df9f564&stat=instructions:u)

| (instructions:u)                   | mean | range        | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.2% | [0.2%, 0.3%] | 3     |
| Regressions ❌ <br /> (secondary)  | 0.1% | [0.1%, 0.1%] | 4     |
| Improvements ✅ <br /> (primary)   | -    | -            | 0     |
| Improvements ✅ <br /> (secondary) | -    | -            | 0     |
| All ❌✅ (primary)                 | 0.2% | [0.2%, 0.3%] | 3     |

* regressed check incr-full for cranelift-codegen (0.25%), cargo (0.22%) and hyper (0.21%).
* this small amount does not appear to be noise
* but it shouldn't have any (significant) effect if the `non_exhaustive_omitted_patterns` lint is not in use.
* marking as triaged

Rollup of 6 pull requests [#117590](https://github.com/rust-lang/rust/pull/117590) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=a42d94ebb883116f2867cbe3016d9bde9ca6d3f7&end=4b85902b438f791c5bfcb6b1c5b476d5b88e2bef&stat=instructions:u)

| (instructions:u)                   | mean  | range         | count |
|:----------------------------------:|:-----:|:-------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 12.7% | [0.4%, 30.9%] | 6     |
| Regressions ❌ <br /> (secondary)  | 16.2% | [8.1%, 24.2%] | 2     |
| Improvements ✅ <br /> (primary)   | -     | -             | 0     |
| Improvements ✅ <br /> (secondary) | -     | -             | 0     |
| All ❌✅ (primary)                 | 12.7% | [0.4%, 30.9%] | 6     |

* the primary regressions (bitmaps and cargo), while huge, are also clearly noise.
* marking as triaged.

enable parallel rustc front end in nightly builds [#117435](https://github.com/rust-lang/rust/pull/117435) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=fcca978aa0c9a7443649305dd0ed48c4691a1e89&end=f9b644636f8c74d1d9a3283dc65a6a6f02a97718&stat=instructions:u)

| (instructions:u)                   | mean | range        | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 2.9% | [0.4%, 5.1%] | 209   |
| Regressions ❌ <br /> (secondary)  | 2.7% | [0.4%, 9.1%] | 219   |
| Improvements ✅ <br /> (primary)   | -    | -            | 0     |
| Improvements ✅ <br /> (secondary) | -    | -            | 0     |
| All ❌✅ (primary)                 | 2.9% | [0.4%, 5.1%] | 209   |

* I was not sure that wg-parallel-rustc was anticipating there being this much of a regression here.
* after followup in [zulip for wg-parallel-rustc](https://rust-lang.zulipchat.com/#narrow/stream/187679-t-compiler.2Fwg-parallel-rustc/topic/performance.20results/near/400790214), they clarified that their expectation of a 1.65% regression was based on wall-time that includes so-called "non-relevant results". (Our default rustc-perf presentation filters out the non-relevant results, which will boost outliers and thus inflate the true mean effect.)
* concluded that this was anticipated and marking as triaged

#### Improvements

bootstrap: do not purge docs on CI environment [#117471](https://github.com/rust-lang/rust/pull/117471) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=b20f40dba9feabf4b446d9b09d15996f0c046eff&end=e8418e092a643ef87019192af7ed6bbd23cc4a63&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -3.0% | [-3.0%, -3.0%] | 1     |
| Improvements ✅ <br /> (secondary) | -     | -              | 0     |
| All ❌✅ (primary)                 | -3.0% | [-3.0%, -3.0%] | 1     |

* single improvement to syn-opt-full that is almost certainly noise.
* and was reverted anyway in PR #117533

Remove support for compiler plugins. [#116412](https://github.com/rust-lang/rust/pull/116412) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1bb6553b967b69eed9ed8147e78b4f65cfc48e11&end=5020f7c3b8bd8bf57077389bca5c19f7911fde7a&stat=instructions:u)

| (instructions:u)                   | mean   | range           | count |
|:----------------------------------:|:------:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -      | -               | 0     |
| Regressions ❌ <br /> (secondary)  | -      | -               | 0     |
| Improvements ✅ <br /> (primary)   | -10.1% | [-23.2%, -0.9%] | 8     |
| Improvements ✅ <br /> (secondary) | -      | -               | 0     |
| All ❌✅ (primary)                 | -10.1% | [-23.2%, -0.9%] | 8     |

* perf improvements here are mostly noise.

Rollup of 3 pull requests [#117564](https://github.com/rust-lang/rust/pull/117564) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=3aaa0f57b7b877ef58532a8de075d1e5a79142bf&end=a9b03ffe9b23a0832c897e25f246439d99d3026d&stat=instructions:u)

| (instructions:u)                   | mean  | range           | count |
|:----------------------------------:|:-----:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -               | 0     |
| Regressions ❌ <br /> (secondary)  | -     | -               | 0     |
| Improvements ✅ <br /> (primary)   | -9.6% | [-15.4%, -3.0%] | 4     |
| Improvements ✅ <br /> (secondary) | -     | -               | 0     |
| All ❌✅ (primary)                 | -9.6% | [-15.4%, -3.0%] | 4     |

* again, perf improvements here seem to be noise

Hint optimizer about try-reserved capacity [#117503](https://github.com/rust-lang/rust/pull/117503) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=4b85902b438f791c5bfcb6b1c5b476d5b88e2bef&end=f5ca57e153afaed818f8be88abf5ce46715c0f9a&stat=instructions:u)

| (instructions:u)                   | mean   | range           | count |
|:----------------------------------:|:------:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -      | -               | 0     |
| Regressions ❌ <br /> (secondary)  | -      | -               | 0     |
| Improvements ✅ <br /> (primary)   | -10.4% | [-23.6%, -0.4%] | 6     |
| Improvements ✅ <br /> (secondary) | -13.5% | [-19.5%, -7.5%] | 2     |
| All ❌✅ (primary)                 | -10.4% | [-23.6%, -0.4%] | 6     |

* improvement here is transient noise on bitmaps and cargo.


#### Mixed

Update cargo [#117462](https://github.com/rust-lang/rust/pull/117462) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=50be22964035027398754e54cd3a497efb00f83e&end=9d83ac217957eece2189eccf4a7232caec7232ee&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 0.9%  | [0.8%, 1.1%]   | 2     |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -0.5% | [-0.6%, -0.4%] | 2     |
| Improvements ✅ <br /> (secondary) | -1.5% | [-1.5%, -1.5%] | 1     |
| All ❌✅ (primary)                 | 0.2%  | [-0.6%, 1.1%]  | 4     |

* regressions are to unicode-normalization-opt-incr-patched:println (1.06%) and clar-opt-full (0.81%)
* the regressions to instruction counts seem small but they persist; i.e. this is not transient noise. (It is not clear whether the cost is borne from cargo itself, or if something has changed about how cargo is invoking rustc that incurs this cost.)
* on the other hand, cycle counts and wall time are not impacted at all, which implies to me that this is probably not worth investigating.
* marking as triaged.


Rollup of 5 pull requests [#117459](https://github.com/rust-lang/rust/pull/117459) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=9d83ac217957eece2189eccf4a7232caec7232ee&end=09ac6e4b6dbec74dc4718191a6650a1b7e04a7a6&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | -     | -              | 0     |
| Regressions ❌ <br /> (secondary)  | 0.4%  | [0.3%, 0.5%]   | 4     |
| Improvements ✅ <br /> (primary)   | -0.2% | [-0.3%, -0.1%] | 3     |
| Improvements ✅ <br /> (secondary) | -     | -              | 0     |
| All ❌✅ (primary)                 | -0.2% | [-0.3%, -0.1%] | 3     |

* regressions are all minor and to variants of secondary benchmark coercions
* marking as triaged

Add all RPITITs when augmenting param-env with GAT bounds in `check_type_bounds` [#117131](https://github.com/rust-lang/rust/pull/117131) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=a2f5f9691b6ce64c1703feaf9363710dfd7a56cf&end=2520ca8566e596b10b0a163d31d9ce216876fafc&stat=instructions:u)

| (instructions:u)                   | mean  | range          | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary)    | 32.3% | [0.3%, 99.8%]  | 19    |
| Regressions ❌ <br /> (secondary)  | -     | -              | 0     |
| Improvements ✅ <br /> (primary)   | -     | -              | 0     |
| Improvements ✅ <br /> (secondary) | -1.4% | [-1.7%, -1.0%] | 6     |
| All ❌✅ (primary)                 | 32.3% | [0.3%, 99.8%]  | 19    |

* the regression to bitmaps (to 13 variants, ranging from 10.8% to 99.8%) appears persistent (non-noise)
* it carries over to cycle-count and wall-clock time.
* compiler-errors has been investigating, and it looks like PR #117542 will address it.
* marking as triaged
