add CPU platforms to instances (#8728)

this materializes RFD 314 and in some respects, 505. 

builds on #8725 for CPU
family information, which is a stand-in for the notion of sled families
and generations described in RFD 314. There are a few important details
here where CPU platforms differ from the sled CPU family and I've
refreshed RFD 314 (and 505) as appropriate. 

## hardware CPU families are less linear than Oxide CPU platforms.

We can (and do, in RFD 314) define Milan restrictively enough that we
can present Turin (and probably later!) CPUs to guests "as if" they were
Milan. Similarly I'd expect that Turin would be defined as roughly
"Milan-plus-some-AVX-512-features" and pretty forward-compatible.
Importantly these are related to but not directly representative of real
CPUs; as an example I'd expect "Turin"-the-instance-CPU-platform to be
able to run on a Turin Dense CPU. Conversely, there's probably not a
reason _to_ define a "Turin Dense" CPU platform since from a guest
perspective they'd look about the same.

But at the same time the lineage through the AMD server part family
splits at Zen 4 kind of, with Zen 4 vs Zen 4c-based parts and similar
with Zen 5/c. It's somewhat hard (I think) to predict what workloads
would be sensitive to this. And as
#8730 gets into a bit,
the details of a processor's packaging (core topology, frequency, cache
size) can vary substantially even inside one CPU family. The important
part here is that we do not expect CPU platforms to cover these
details and it would probably be cumbersome to try; if the instance's
constraint is "I want AVX256, and I want to be on high-frequency-capable
processors only", then it doesn't actually matter if it's run on a Turin
or a Milan and to tie it to that CPU platform may be overly restrictive.

On instance CPU platforms, the hope is that by focusing on CPU features
we're able to present a more linear path as the microarchitectures grow.

## instance platforms aren't "minimum"

I've walked back the initial description of an instance's CPU platform
as the "minimum CPU platform". As present in other systems, "minimum CPU
platform" would more analogously mean "can we put you on a Rome Gimlet
or must we put you on a Milan Gimlet?", or "Genoa Cosmo vs Turin Cosmo?"
- it doesn't seem _possible_ to say "this instance must have AVX 512,
but otherwise I don't care what kind of hardware it runs on.", but
that's more what _we mean_ by CPU platform.

In a "minimum CPU platform" interpretation, we _could_ provide a bunch
of Turin CPUID bits to a VM that said it wanted Milan. But since there's
no upper bound here, if an OS has an issue with a future "Zen 14" or
whatever, a user would discover that by their "minimum-Milan" instance
getting scheduled on the new space-age processor and exploding on boot
or something. OSes _shouldn't_ do that, but...

Implementation-wise, this is really just about the names right now. You
always get Milan CPUID leaves for the time being. When there are Turin
CPUID leaves defined for the instance CPU platform, and Cosmos on which
they make sense, this becomes more concrete.

## "are these CPU platforms compatible?"

RFD 314 has a section now, and I've added a stub function, covering
some more obvious ways that CPU platforms would be *incompatible*.
This is particularly fraught if we consider being incorrect about
topology an incompatibility, but even setting that aside several bits in
CPUID are descriptive of architectural behaviors and are not easily (or
at all) able to be emulated.

`functionally_same()` and the CPUID profiles here may be fated to move
out of Omicron and into another crate which can be shared with Propolis,
where it can ensure that a requested profile is consistent with the
hardware on which Propolis would create a VM (not to mention test uses)

---------

Co-authored-by: Greg Colombo <[email protected]>

Author: iximeow

Cargo.lock

@@ -653,6 +653,7 @@ rand_distr = "0.5.1"
 rand_seeder = "0.4.0"
 range-requests = { path = "range-requests" }
 ratatui = "0.29.0"
+raw-cpuid = { git = "https://github.com/oxidecomputer/rust-cpuid.git", rev = "0a8dbd2311263f6a59ea58089e33c8331436ff3a" }
 rayon = "1.10"
 rcgen = "0.12.1"
 reconfigurator-cli = { path = "dev-tools/reconfigurator-cli" }

Cargo.toml

@@ -1197,6 +1197,10 @@ pub struct Instance {
 
     #[serde(flatten)]
     pub auto_restart_status: InstanceAutoRestartStatus,
+
+    /// The CPU platform for this instance. If this is `null`, the instance
+    /// requires no particular CPU platform.
+    pub cpu_platform: Option<InstanceCpuPlatform>,
 }
 
 /// Status of control-plane driven automatic failure recovery for this instance.
@@ -1261,6 +1265,51 @@ pub enum InstanceAutoRestartPolicy {
     BestEffort,
 }
 
+/// A required CPU platform for an instance.
+///
+/// When an instance specifies a required CPU platform:
+///
+/// - The system may expose (to the VM) new CPU features that are only present
+///   on that platform (or on newer platforms of the same lineage that also
+///   support those features).
+/// - The instance must run on hosts that have CPUs that support all the
+///   features of the supplied platform.
+///
+/// That is, the instance is restricted to hosts that have the CPUs which
+/// support all features of the required platform, but in exchange the CPU
+/// features exposed by the platform are available for the guest to use. Note
+/// that this may prevent an instance from starting (if the hosts that could run
+/// it are full but there is capacity on other incompatible hosts).
+///
+/// If an instance does not specify a required CPU platform, then when
+/// it starts, the control plane selects a host for the instance and then
+/// supplies the guest with the "minimum" CPU platform supported by that host.
+/// This maximizes the number of hosts that can run the VM if it later needs to
+/// migrate to another host.
+///
+/// In all cases, the CPU features presented by a given CPU platform are a
+/// subset of what the corresponding hardware may actually support; features
+/// which cannot be used from a virtual environment or do not have full
+/// hypervisor support may be masked off. See RFD 314 for specific CPU features
+/// in a CPU platform.
+#[derive(
+    Copy, Clone, Debug, Deserialize, Serialize, JsonSchema, Eq, PartialEq,
+)]
+#[serde(rename_all = "snake_case")]
+pub enum InstanceCpuPlatform {
+    /// An AMD Milan-like CPU platform.
+    AmdMilan,
+
+    /// An AMD Turin-like CPU platform.
+    // Note that there is only Turin, not Turin Dense - feature-wise there are
+    // collapsed together as the guest-visible platform is the same.
+    // If the two must be distinguished for instance placement, we'll want to
+    // track whatever the motivating constraint is more explicitly. CPU
+    // families, and especially the vendor code names, don't necessarily promise
+    // details about specific processor packaging choices.
+    AmdTurin,
+}
+
 // AFFINITY GROUPS
 
 /// Affinity policy used to describe "what to do when a request cannot be satisfied"

common/src/api/external/mod.rs

@@ -4774,6 +4774,7 @@ async fn cmd_db_instance_info(
                     propolis_ip: _,
                     propolis_port: _,
                     instance_id: _,
+                    cpu_platform: _,
                     time_created,
                     time_deleted,
                     runtime:
@@ -7376,6 +7377,7 @@ fn prettyprint_vmm(
     const INSTANCE_ID: &'static str = "instance ID";
     const SLED_ID: &'static str = "sled ID";
     const SLED_SERIAL: &'static str = "sled serial";
+    const CPU_PLATFORM: &'static str = "CPU platform";
     const ADDRESS: &'static str = "propolis address";
     const STATE: &'static str = "state";
     const WIDTH: usize = const_max_len(&[
@@ -7386,6 +7388,7 @@ fn prettyprint_vmm(
         INSTANCE_ID,
         SLED_ID,
         SLED_SERIAL,
+        CPU_PLATFORM,
         STATE,
         ADDRESS,
     ]);
@@ -7399,6 +7402,7 @@ fn prettyprint_vmm(
         sled_id,
         propolis_ip,
         propolis_port,
+        cpu_platform,
         runtime: db::model::VmmRuntimeState { state, r#gen, time_state_updated },
     } = vmm;
 
@@ -7425,6 +7429,7 @@ fn prettyprint_vmm(
     if let Some(serial) = sled_serial {
         println!("{indent}{SLED_SERIAL:>width$}: {serial}");
     }
+    println!("{indent}{CPU_PLATFORM:>width$}: {cpu_platform}");
 }
 
 async fn cmd_db_vmm_list(
@@ -7500,6 +7505,7 @@ async fn cmd_db_vmm_list(
                 sled_id,
                 propolis_ip: _,
                 propolis_port: _,
+                cpu_platform: _,
                 runtime:
                     db::model::VmmRuntimeState {
                         state,

dev-tools/omdb/src/bin/omdb/db.rs

@@ -79,6 +79,7 @@ async fn instance_launch() -> Result<()> {
             start: true,
             auto_restart_policy: Default::default(),
             anti_affinity_groups: Vec::new(),
+            cpu_platform: None,
         })
         .send()
         .await?;

end-to-end-tests/src/instance_launch.rs

@@ -134,6 +134,7 @@ oxide-tokio-rt.workspace = true
 oximeter.workspace = true
 oximeter-instruments = { workspace = true, features = ["http-instruments"] }
 oximeter-producer.workspace = true
+raw-cpuid = { workspace = true, features = ["std"] }
 rustls = { workspace = true }
 rustls-pemfile = { workspace = true }
 update-common.workspace = true

nexus/Cargo.toml

@@ -5,7 +5,7 @@
 use super::InstanceIntendedState as IntendedState;
 use super::{
     ByteCount, Disk, ExternalIp, Generation, InstanceAutoRestartPolicy,
-    InstanceCpuCount, InstanceState, Vmm, VmmState,
+    InstanceCpuCount, InstanceCpuPlatform, InstanceState, Vmm, VmmState,
 };
 use crate::collection::DatastoreAttachTargetConfig;
 use crate::serde_time_delta::optional_time_delta;
@@ -68,6 +68,12 @@ pub struct Instance {
     #[diesel(column_name = boot_disk_id)]
     pub boot_disk_id: Option<Uuid>,
 
+    /// The instance's required CPU platform. If this is `None`, Nexus will not
+    /// constrain placement decisions by CPU platform. Instead, after selecting
+    /// a sled by any other constraints the instance will be incarnated with the
+    /// most general CPU platform supported by the selected sled.
+    pub cpu_platform: Option<InstanceCpuPlatform>,
+
     #[diesel(embed)]
     pub runtime_state: InstanceRuntimeState,
 
@@ -139,6 +145,7 @@ impl Instance {
             // Intentionally ignore `params.boot_disk_id` here: we can't set
             // `boot_disk_id` until the referenced disk is attached.
             boot_disk_id: None,
+            cpu_platform: params.cpu_platform.map(Into::into),
 
             runtime_state,
             intended_state,
@@ -493,4 +500,6 @@ pub struct InstanceUpdate {
     pub ncpus: InstanceCpuCount,
 
     pub memory: ByteCount,
+
+    pub cpu_platform: Option<InstanceCpuPlatform>,
 }

nexus/db-model/src/instance.rs

@@ -0,0 +1,65 @@
+// This Source Code Form is subject to the terms of the Mozilla Public
+// License, v. 2.0. If a copy of the MPL was not distributed with this
+// file, You can obtain one at https://mozilla.org/MPL/2.0/.
+
+use crate::SledCpuFamily;
+
+use super::impl_enum_type;
+use serde::{Deserialize, Serialize};
+
+impl_enum_type!(
+    InstanceCpuPlatformEnum:
+
+    #[derive(
+        Copy,
+        Clone,
+        Debug,
+        PartialEq,
+        AsExpression,
+        FromSqlRow,
+        Serialize,
+        Deserialize
+    )]
+    pub enum InstanceCpuPlatform;
+
+    AmdMilan => b"amd_milan"
+    AmdTurin => b"amd_turin"
+);
+
+impl InstanceCpuPlatform {
+    /// Returns a slice containing the set of sled CPU families that can
+    /// accommodate an instance with this CPU platform.
+    pub fn compatible_sled_cpu_families(&self) -> &'static [SledCpuFamily] {
+        match self {
+            // Turin-based sleds have a superset of the features made available
+            // in a guest's Milan CPU platform
+            Self::AmdMilan => {
+                &[SledCpuFamily::AmdMilan, SledCpuFamily::AmdTurin]
+            }
+            Self::AmdTurin => &[SledCpuFamily::AmdTurin],
+        }
+    }
+}
+
+impl From<omicron_common::api::external::InstanceCpuPlatform>
+    for InstanceCpuPlatform
+{
+    fn from(value: omicron_common::api::external::InstanceCpuPlatform) -> Self {
+        use omicron_common::api::external::InstanceCpuPlatform as ApiPlatform;
+        match value {
+            ApiPlatform::AmdMilan => Self::AmdMilan,
+            ApiPlatform::AmdTurin => Self::AmdTurin,
+        }
+    }
+}
+
+impl From<InstanceCpuPlatform>
+    for omicron_common::api::external::InstanceCpuPlatform
+{
+    fn from(value: InstanceCpuPlatform) -> Self {
+        match value {
+            InstanceCpuPlatform::AmdMilan => Self::AmdMilan,
+            InstanceCpuPlatform::AmdTurin => Self::AmdTurin,
+        }
+    }
+}

nexus/db-model/src/instance_cpu_platform.rs

@@ -44,6 +44,7 @@ mod image;
 mod instance;
 mod instance_auto_restart_policy;
 mod instance_cpu_count;
+mod instance_cpu_platform;
 mod instance_intended_state;
 mod instance_state;
 mod internet_gateway;
@@ -125,6 +126,7 @@ mod utilization;
 mod virtual_provisioning_collection;
 mod virtual_provisioning_resource;
 mod vmm;
+mod vmm_cpu_platform;
 mod vni;
 mod volume;
 mod volume_repair;
@@ -183,6 +185,7 @@ pub use image::*;
 pub use instance::*;
 pub use instance_auto_restart_policy::*;
 pub use instance_cpu_count::*;
+pub use instance_cpu_platform::*;
 pub use instance_intended_state::*;
 pub use instance_state::*;
 pub use internet_gateway::*;
@@ -252,6 +255,7 @@ pub use v2p_mapping::*;
 pub use virtual_provisioning_collection::*;
 pub use virtual_provisioning_resource::*;
 pub use vmm::*;
+pub use vmm_cpu_platform::*;
 pub use vmm_state::*;
 pub use vni::*;
 pub use volume::*;

nexus/db-model/src/lib.rs

@@ -16,7 +16,7 @@ use std::{collections::BTreeMap, sync::LazyLock};
 ///
 /// This must be updated when you change the database schema.  Refer to
 /// schema/crdb/README.adoc in the root of this repository for details.
-pub const SCHEMA_VERSION: Version = Version::new(189, 0, 0);
+pub const SCHEMA_VERSION: Version = Version::new(190, 0, 0);
 
 /// List of all past database schema versions, in *reverse* order
 ///
@@ -28,6 +28,7 @@ static KNOWN_VERSIONS: LazyLock<Vec<KnownVersion>> = LazyLock::new(|| {
         // |  leaving the first copy as an example for the next person.
         // v
         // KnownVersion::new(next_int, "unique-dirname-with-the-sql-files"),
+        KnownVersion::new(190, "add-instance-cpu-platform"),
         KnownVersion::new(189, "reconfigurator-chicken-switches-to-config"),
         KnownVersion::new(188, "positive-quotas"),
         KnownVersion::new(187, "no-default-pool-for-internal-silo"),