run_td.sh working on one machine but not another

[fwoodruff-ab]

I previously followed the instructions on a Xeon 8570 machine. I am now on a Xeon 6731E machine and run_td.sh is no longer working. I have moved from a machine obtained for testing purposes through a private source, to a machine obtained from a small bare metal cloud provider.

I SSHed into the Xeon 8570 machine with a Ubuntu 24.04 operating system already installed. I followed the instructions in README.md. I had direct remote access to the BIOS via a console and it looked just like what is listed in the instructions in step 4.3. I used an image generated using step 5.1 on a different machine and copied (scp) this across. I ran run_td.sh without issue.

I then obtained a Xeon 6731E machine through a cloud provider. I ran step 4.1 using do-release-upgrade, step 4.2 as stated, and step 4.3 I had to ask the cloud provider to action, since the BIOS was password protected.

Running step 6 I get:

ubuntu@ab-tdx-test-machine:~/tdx/guest-tools$ ./run_td.sh
Error: Failed to create TD VM. Please check logfile "/tmp/tdx-guest-td.log" for more information.
ubuntu@ab-tdx-test-machine:~/tdx/guest-tools$ cat /tmp/tdx-guest-td.log
qemu-system-x86_64: KVM_TDX_INIT_VM failed: Invalid argument
ubuntu@ab-tdx-test-machine:~/tdx/guest-tools$ 

Searching for the error online I found this and not much else.

How do I run a TDX guest on this Intel TDX machine?

System report

Git ref

e7c4dc6c4a1adbc24d07ac034f33df3c0fb90fae

Operating system details

Distributor ID:	Ubuntu
Description:	Ubuntu 24.04.1 LTS
Release:	24.04
Codename:	noble

Kernel version

6.8.0-1013-intel #20-Ubuntu SMP PREEMPT_DYNAMIC Thu Oct  3 17:38:00 UTC 2024 x86_64 x86_64 GNU/Linux

TDX kernel logs

[    1.247418] virt/tdx: BIOS enabled: private KeyID range [32, 64)
[    1.247421] virt/tdx: Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.
[    7.573694] systemd[1]: Hostname set to <ab-tdx-test-machine>.
[    8.341104] virt/tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 5, build_date 20240725, build_num 784
[    8.341108] virt/tdx: CMR: [0x100000, 0x77800000)
[    8.341110] virt/tdx: CMR: [0x100000000, 0x3ffe000000)
[    9.282380] virt/tdx: 1042424 KB allocated for PAMT
[    9.282388] virt/tdx: module initialized
...
[    1.247418] virt/tdx: BIOS enabled: private KeyID range [32, 64)
[    1.247421] virt/tdx: Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.
[    7.573694] systemd[1]: Hostname set to <ab-tdx-test-machine>.
[    8.341104] virt/tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 5, build_date 20240725, build_num 784
[    8.341108] virt/tdx: CMR: [0x100000, 0x77800000)
[    8.341110] virt/tdx: CMR: [0x100000000, 0x3ffe000000)
[    9.282380] virt/tdx: 1042424 KB allocated for PAMT
[    9.282388] virt/tdx: module initialized

TDX CPU instruction support

CPU supports TDX according to /proc/cpuinfo

Model specific registers (MSRs)

MK_TME_ENABLED bit: 1 (expected value: 1)
SEAM_RR bit: 1 (expected value: 1)
NUM_TDX_PRIV_KEYS: 20
SGX_AND_MCHECK_STATUS: 0 (expected value: 0)
Production platform: Production (expected value: Production)

CPU details

 Intel(R) Xeon(R) 6731E

QEMU package details

Status: Installed
Package: qemu-system-x86
Version: 2:8.2.2+ds-0ubuntu1.4+tdx1.0
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-release/ubuntu noble/main amd64 Packages

Libvirt package details

Status: Installed
Package: libvirt-clients
Version: 10.0.0-2ubuntu8.3+tdx1.2
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-release/ubuntu noble/main amd64 Packages

OVMF package details

Status: Installed
Package: ovmf
Version: 2024.02-3+tdx1.0
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-release/ubuntu noble/main amd64 Packages

sgx-dcap-pccs package details

Status: Installed
Package: sgx-dcap-pccs
Version: 1.21-0ubuntu1
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-attestation-release/ubuntu noble/main amd64 Packages

tdx-qgs package details

Status: Installed
Package: tdx-qgs
Version: 1.21-0ubuntu2.2
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-attestation-release/ubuntu noble/main amd64 Packages

sgx-ra-service package details

Status: Installed
Package: sgx-ra-service
Version: 1.21-0ubuntu2.2
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-attestation-release/ubuntu noble/main amd64 Packages
Description: Intel(R) Software Guard Extensions Multi-Package Registration Agent Service

sgx-pck-id-retrieval-tool package details

Status: Installed
Package: sgx-pck-id-retrieval-tool
Version: 1.21-0ubuntu2.2
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-attestation-release/ubuntu noble/main amd64 Packages

QGSD service status

● qgsd.service - Intel(R) TD Quoting Generation Service
     Loaded: loaded (/usr/lib/systemd/system/qgsd.service; enabled; preset: enabled)
     Active: active (running) since Tue 2024-11-26 12:07:02 UTC; 2h 30min ago
    Process: 2063 ExecStartPre=/bin/chown -R qgsd:qgsd /var/opt/qgsd/ (code=exited, status=0/SUCCESS)
    Process: 2133 ExecStartPre=/bin/chmod 0750 /var/opt/qgsd/ (code=exited, status=0/SUCCESS)
    Process: 2152 ExecStartPre=/usr/share/qgs/linksgx.sh (code=exited, status=0/SUCCESS)
    Process: 2264 ExecStart=/usr/bin/qgs (code=exited, status=0/SUCCESS)
   Main PID: 2282 (qgs)
      Tasks: 5 (limit: 306471)
     Memory: 3.1M (peak: 4.3M)
        CPU: 109ms
     CGroup: /system.slice/qgsd.service
             └─2282 /usr/bin/qgs

Nov 26 12:07:02 ab-tdx-test-machine systemd[1]: Starting qgsd.service - Intel(R) TD Quoting Generation Service...
Nov 26 12:07:02 ab-tdx-test-machine qgsd[2282]: Added signal handler
Nov 26 12:07:02 ab-tdx-test-machine qgsd[2282]: About to create QgsServer with num_thread = 4
Nov 26 12:07:02 ab-tdx-test-machine qgsd[2282]: About to start main loop
Nov 26 12:07:02 ab-tdx-test-machine systemd[1]: Started qgsd.service - Intel(R) TD Quoting Generation Service.

PCCS service status

● pccs.service - Provisioning Certificate Caching Service (PCCS)
     Loaded: loaded (/usr/lib/systemd/system/pccs.service; enabled; preset: enabled)
     Active: active (running) since Tue 2024-11-26 12:07:02 UTC; 2h 30min ago
       Docs: https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/pccs/README.md
   Main PID: 2061 (node)
      Tasks: 15 (limit: 306471)
     Memory: 112.2M (peak: 120.3M)
        CPU: 2.463s
     CGroup: /system.slice/pccs.service
             └─2061 /usr/bin/node /opt/intel/sgx-dcap-pccs/pccs_server.js

Nov 26 12:07:02 ab-tdx-test-machine systemd[1]: Started pccs.service - Provisioning Certificate Caching Service (PCCS).
Nov 26 12:07:03 ab-tdx-test-machine node[2061]: 2024-11-26 12:07:03.546 [info]: HTTPS Server is running on: https://localhost:8081

MPA registration logs (last 30 lines)

[26-11-2024 10:24:11] INFO: SGX Registration Agent version: 1.21.100.3
[26-11-2024 10:24:11] INFO: Starts Registration Agent Flow.
[26-11-2024 10:24:11] INFO: Registration Flow - PLATFORM_ESTABLISHMENT or TCB_RECOVERY passed successfully.
[26-11-2024 10:24:11] INFO: Finished Registration Agent Flow.
[26-11-2024 11:24:33] INFO: SGX Registration Agent version: 1.21.100.3
[26-11-2024 11:24:33] INFO: Starts Registration Agent Flow.
[26-11-2024 11:24:33] INFO: Registration Flow - Registration status indicates registration is completed successfully. MPA has nothing to do.
[26-11-2024 11:24:33] INFO: Finished Registration Agent Flow.
[26-11-2024 12:07:02] INFO: SGX Registration Agent version: 1.21.100.3
[26-11-2024 12:07:02] INFO: Starts Registration Agent Flow.
[26-11-2024 12:07:02] INFO: Registration Flow - Registration status indicates registration is completed successfully. MPA has nothing to do.
[26-11-2024 12:07:02] INFO: Finished Registration Agent Flow.

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/PEK-1494.

This message was autogenerated

[bktan8]

Hello @fwoodruff-ab - The TDX early preview isn't ready to support Xeon 6731E yet. This product is codenamed Sierra Forest, which is Xeon 6 with E-Cores. We only have support for Xeon 6 with P-Cores (codenamed: Granite Rapids) for time being.

[bktan8]

#288

[pprincipeza]

@bktan8 And are there any plans from Intel to support TDX on Sierra Forest / 6731E family?

[hector-cao]

@pprincipeza We are working on delivering the support for Sierra Forest on 24.10, it should be released soon

[pprincipeza]

@hector-cao Thanks for the heads-up! Do you believe this could ever be backported to the TDX stack for 24.04? I believe customers will most likely adopt TDX in an LTS stack instead of a non-LTS (like 24.10).