Skip to content

CP-308875: set Xen PCI MMIO BAR to WB #6591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

edwintorok
Copy link
Contributor

The default for the Xen PCI MMIO BAR is UnCachable. Setting this to WriteBack in the MTRR shows a massive performance improvement on AMD, at least for Linux guests.

On Intel this is already set to WriteBack by Xen via another mechanism.

To be effective this also requires the corresponding Xen commit, old versions will just ignore this xenstore key.

The optimization is not enabled by default in Xen due to the wide range of guests it supports, but XAPI supports a much narrower set of guest OSes.

Setting the cache attribute to WB is done by setting UC=0.

@robhoes robhoes requested a review from andyhhp July 15, 2025 09:51
@last-genius
Copy link
Contributor

there is a parallel set of PRs where this was decided to be made configurable (either through xenopsd.conf or through the platform key): #6566 (update of #6555).

@edwintorok edwintorok force-pushed the private/edvint/xsi-1947/xenstore-0 branch from 09fd590 to b494899 Compare July 15, 2025 10:32
@edwintorok
Copy link
Contributor Author

there is a parallel set of PRs where this was decided to be made configurable (either through xenopsd.conf or through the platform key): #6566 (update of #6555).

We can make it configurable (just in case someone wants to revert to UC=1), although we should also change the default to be UC=0.

@edwintorok
Copy link
Contributor Author

edwintorok commented Jul 15, 2025

I'll rebase this on #6566 (comment) once that is in and change the defaults there.

@edwintorok edwintorok force-pushed the private/edvint/xsi-1947/xenstore-0 branch 2 times, most recently from 0be2cd6 to 7c15ef6 Compare July 15, 2025 12:47
@edwintorok
Copy link
Contributor Author

Rebased on the other PR by tweaking the newly introduced xenopsd.conf default, thanks for pointing it out.

The default for the Xen PCI MMIO BAR is UnCachable.
Setting this to WriteBack in the MTRR shows a massive performance improvement on AMD,
at least for Linux guests.

On Intel this is already set to WriteBack by Xen via another mechanism.

To be effective this also requires the corresponding [Xen
commit](https://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=22650d6054625be10172fe0c78b9cadd1a39bd63),
old versions will just ignore this xenstore key.

The optimization is not enabled by default in Xen due to the wide range of
guests it supports, but XAPI supports a much narrower set of guest OSes.

Setting the cache attribute to WB is done by setting UC=false.

Signed-off-by: Edwin Török <[email protected]>
@edwintorok edwintorok force-pushed the private/edvint/xsi-1947/xenstore-0 branch from 7c15ef6 to 37a42e0 Compare July 15, 2025 14:33
@edwintorok edwintorok marked this pull request as ready for review July 15, 2025 14:34
@edwintorok edwintorok enabled auto-merge July 15, 2025 14:39
@edwintorok edwintorok added this pull request to the merge queue Jul 15, 2025
Merged via the queue into xapi-project:master with commit 19fa4d2 Jul 15, 2025
16 checks passed
Comment on lines +53 to +54
This is useful for AMD, and mostly a noop on Intel (which achieves a similar
effect using Intel-only features in Xen)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The part in brackets is going to go stale quickly. In order to support encrypted memory (and later, encrypted VMs), we're going to have to stop using iPAT on Intel, at which point Intel will become like AMD and this control will be relevant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, so doing this uniformly now on both Intel and AMD actually future proofs us against that. You can then change the Intel side from iPAT in Xen without further toolstack changes?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants