Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to Passthrough /dev/tpm0 to Guest OS on Intel TDX Machine #275

Open
syambabu8k opened this issue Nov 15, 2024 · 1 comment
Open

Unable to Passthrough /dev/tpm0 to Guest OS on Intel TDX Machine #275

syambabu8k opened this issue Nov 15, 2024 · 1 comment
Assignees
@hector-cao

Comments

@syambabu8k
Copy link

Testing on a bare-metal Intel TDX machine and encountered issues while attempting to use TPM passthrough for a guest OS.

  1. Environment Details:
    • Hardware: Bare-metal Intel TDX machine.
    • Software: Using QEMU with Intel TDX-enabled support.
    • Documentation: Followed the Intel TDX documentation provided.
    • Objective: Test TPM passthrough and use tpm2-tools to read and update PCR values.
  2. Steps Taken:
    • Created an Intel TDX guest OS and used the run_td.sh script to launch it.
    • Successfully created and logged into the Guest VM.
    • Attempted to run tpm2-tools (e.g., tpm2_pcrread) inside the Guest OS but encountered the following error:

ERROR:tcti:src/tss2-tcti/tcti-device.c:451:Tss2_Tcti_Device_Init() Failed to open specified TCTI device file /dev/tpmrm0: No such file or directory

Updated the run_td.sh script to modify the qemu-system-x86_64 command with TPM passthrough options:

-tpmdev passthrough,id=tpm0,path=/dev/tpm0,cancel-path=/dev/tpmrm0
-device tpm-tis,tpmdev=tpm0

  1. Issue:
    • After updating the QEMU command for TPM passthrough:
    • The VM was created, but I could no longer connect to it.
    • Logs showed the following errors:

error: kvm run failed Bad address
EAX=00000000 EBX=00000000 ECX=00000000 EDX=000806f8
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 ffff0000 0000ffff 00009b00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 00000000 0000ffff
IDT= 00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??

  1. Expected Behavior:
    • The TPM device should be successfully passed through to the Guest OS.
    • The Guest OS should be able to detect /dev/tpmrm0 or /dev/tpm0 and allow tpm2-tools to execute.
  2. Additional Information:
    • The /dev/tpm0 device is present on the host machine and functional.
    • No issues observed when running tpm2-tools directly on the host machine.

Request:

1.	Assistance in debugging the “KVM run failed” error when using the TPM passthrough configuration.
2.	Guidance on ensuring /dev/tpm0 and /dev/tpmrm0 are successfully passed to the Guest OS and accessible.
3.	Suggestions for any missing configurations or updates required for Intel TDX environments.

System-Report:

Git ref

5c3ac230e645841c0fb81ea3ca23e1d72dfb6f90

Operating system details

Distributor ID: Ubuntu
Description:    Ubuntu 24.04 LTS
Release:        24.04
Codename:       noble

Kernel version

6.8.0-1008-intel #15-Ubuntu SMP PREEMPT_DYNAMIC Fri Jul 12 09:47:38 UTC 2024 x86_64 x86_64 GNU/Linux

TDX kernel logs

[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.8.0-1008-intel root=UUID=cb735d25-09f9-4618-861b-a24d9bf6567a ro kvm_intel.tdx=1 nohibernate nomodeset
[    1.246791] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.8.0-1008-intel root=UUID=cb735d25-09f9-4618-861b-a24d9bf6567a ro kvm_intel.tdx=1 nohibernate nomodeset
[    2.515807] virt/tdx: BIOS enabled: private KeyID range [16, 32)
[    2.515810] virt/tdx: Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.
[    2.576446] smpboot: CPU0: Intel(R) Xeon(R) Platinum 8480CTDX (family: 0x6, model: 0x8f, stepping: 0x8)
[    9.195200] virt/tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 5, build_date 20240129, build_num 698
[    9.195204] virt/tdx: CMR: [0x100000, 0x77800000)
[    9.195206] virt/tdx: CMR: [0x100000000, 0x2076000000)
[    9.195206] virt/tdx: CMR: [0x2080000000, 0x4078000000)
[    9.195207] virt/tdx: CMR: [0x4080000000, 0x6078000000)
...
[    9.195206] virt/tdx: CMR: [0x100000000, 0x2076000000)
[    9.195206] virt/tdx: CMR: [0x2080000000, 0x4078000000)
[    9.195207] virt/tdx: CMR: [0x4080000000, 0x6078000000)
[    9.195208] virt/tdx: CMR: [0x6080000000, 0x8078000000)
[   10.736154] virt/tdx: 2101268 KB allocated for PAMT
[   10.736159] virt/tdx: module initialized
[ 8622.101341] audit: type=1400 audit(1729124407.139:140): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_binary-amd64_Packages" pid=28511 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[ 8622.101385] audit: type=1400 audit(1729124407.139:141): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_i18n_Translation-en" pid=28511 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[100090.976804] audit: type=1400 audit(1729215876.598:179): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_binary-amd64_Packages" pid=58880 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[100090.976841] audit: type=1400 audit(1729215876.598:180): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_i18n_Translation-en" pid=58880 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[287920.198925] audit: type=1400 audit(1729403707.003:214): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_binary-amd64_Packages" pid=74398 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[287920.198963] audit: type=1400 audit(1729403707.003:215): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_i18n_Translation-en" pid=74398 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[449563.470829] audit: type=1400 audit(1729565351.297:247): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_binary-amd64_Packages" pid=88510 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[449563.470866] audit: type=1400 audit(1729565351.297:248): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_i18n_Translation-en" pid=88510 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[539287.567254] audit: type=1400 audit(1729655075.907:282): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_binary-amd64_Packages" pid=101039 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[539287.567290] audit: type=1400 audit(1729655075.907:283): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_i18n_Translation-en" pid=101039 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[646845.447784] audit: type=1400 audit(1729762634.397:317): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_binary-amd64_Packages" pid=110132 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[646845.447820] audit: type=1400 audit(1729762634.397:318): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_i18n_Translation-en" pid=110132 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[801982.334741] audit: type=1400 audit(1729917772.167:352): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_binary-amd64_Packages" pid=129685 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
[801982.334782] audit: type=1400 audit(1729917772.167:353): apparmor="DENIED" operation="open" class="file" profile="ubuntu_pro_apt_news" name="/var/lib/apt/lists/ppa.launchpadcontent.net_kobuk-team_tdx-release_ubuntu_dists_noble_main_i18n_Translation-en" pid=129685 comm="python3" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

TDX CPU instruction support

CPU supports TDX according to /proc/cpuinfo

Model specific registers (MSRs)

MK_TME_ENABLED bit: 1 (expected value: 1)
SEAM_RR bit: 1 (expected value: 1)
NUM_TDX_PRIV_KEYS: 10
SGX_AND_MCHECK_STATUS: 0 (expected value: 0)
Production platform: Production (expected value: Production)

CPU details

 Intel(R) Xeon(R) Platinum 8480CTDX

QEMU package details

Status: Installed
Package: qemu-system-x86
Version: 1:8.2.2+ds-0ubuntu2+tdx1.0
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-release/ubuntu noble/main amd64 Packages

Libvirt package details

Status: Installed
Package: libvirt-clients
Version: 10.0.0-2ubuntu8.3+tdx1.2
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-release/ubuntu noble/main amd64 Packages

OVMF package details

Status: Installed
Package: ovmf
Version: 2024.02-3+tdx1.0
APT-Sources: https://ppa.launchpadcontent.net/kobuk-team/tdx-release/ubuntu noble/main amd64 Packages

sgx-dcap-pccs package details

Status: Installed
Package: sgx-dcap-pccs
Version: 1.20-0ubuntu1
APT-Sources: /var/lib/dpkg/status

tdx-qgs package details

Status: Installed
Package: tdx-qgs
Version: 1.20-0ubuntu1
APT-Sources: /var/lib/dpkg/status

sgx-ra-service package details

Status: Installed
Package: sgx-ra-service
Version: 1.20-0ubuntu1
APT-Sources: /var/lib/dpkg/status
Description: Intel(R) Software Guard Extensions Multi-Package Registration Agent Service

sgx-pck-id-retrieval-tool package details

Status: Installed
Package: sgx-pck-id-retrieval-tool
Version: 1.20-0ubuntu1
APT-Sources: /var/lib/dpkg/status

QGSD service status

● qgsd.service - Intel(R) TD Quoting Generation Service
     Loaded: loaded (/usr/lib/systemd/system/qgsd.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-10-17 06:15:53 UTC; 4 weeks 1 day ago
   Main PID: 45891 (qgs)
      Tasks: 5 (limit: 617857)
     Memory: 640.0K (peak: 3.1M)
        CPU: 43ms
     CGroup: /system.slice/qgsd.service
             └─45891 /usr/bin/qgs

Oct 17 06:15:53 b49691f5dc3c qgsd[45891]: Added signal handler
Oct 17 06:15:53 b49691f5dc3c qgsd[45891]: About to create QgsServer with num_thread = 4
Oct 17 06:15:53 b49691f5dc3c qgsd[45891]: About to start main loop
Oct 17 06:15:53 b49691f5dc3c systemd[1]: Started qgsd.service - Intel(R) TD Quoting Generation Service.

PCCS service status

● pccs.service - Provisioning Certificate Caching Service (PCCS)
     Loaded: loaded (/usr/lib/systemd/system/pccs.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-10-17 06:15:53 UTC; 4 weeks 1 day ago
       Docs: https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/pccs/README.md
   Main PID: 45805 (node)
      Tasks: 15 (limit: 617857)
     Memory: 56.9M (peak: 61.8M)
        CPU: 12.451s
     CGroup: /system.slice/pccs.service
             └─45805 /usr/bin/node /opt/intel/sgx-dcap-pccs/pccs_server.js

Nov 14 01:00:01 b49691f5dc3c node[45805]: 2024-11-14 01:00:01.095 [info]: Request-ID is : d44fcdd3c6834f54b1a2cc14732dc86c
Nov 14 01:00:01 b49691f5dc3c node[45805]: 2024-11-14 01:00:01.477 [info]: Request-ID is : 4b485b76770b490cb280b80b83ffc723
Nov 14 01:00:01 b49691f5dc3c node[45805]: 2024-11-14 01:00:01.812 [info]: Request-ID is : a1ed0b57e14d4b87b44fcf67473dd7b2
Nov 14 01:00:01 b49691f5dc3c node[45805]: 2024-11-14 01:00:01.908 [info]: Scheduled cache refresh is completed successfully.
Nov 15 01:00:00 b49691f5dc3c node[45805]: 2024-11-15 01:00:00.403 [info]: Request-ID is : f681bb56a7cd472f9e978f2a9adac2cf
Nov 15 01:00:00 b49691f5dc3c node[45805]: 2024-11-15 01:00:00.715 [info]: Request-ID is : ac32a3f45be2433889baacc438357da1
Nov 15 01:00:01 b49691f5dc3c node[45805]: 2024-11-15 01:00:01.056 [info]: Request-ID is : ac32d8c9c95d4b79b4f1a62edf3bd45a
Nov 15 01:00:01 b49691f5dc3c node[45805]: 2024-11-15 01:00:01.381 [info]: Request-ID is : 96ae4554523f4f1482881dc28915b0f7
Nov 15 01:00:01 b49691f5dc3c node[45805]: 2024-11-15 01:00:01.736 [info]: Request-ID is : 3c186bc6266d41b48ee5004bf642cb63
Nov 15 01:00:01 b49691f5dc3c node[45805]: 2024-11-15 01:00:01.826 [info]: Scheduled cache refresh is completed successfully.

MPA registration logs (last 30 lines)

[15-08-2024 08:08:58] INFO: Please use management tool or PCKCertIDRetrievalTool to read PLATFORM_MANIFEST.
[15-08-2024 08:08:58] INFO: Finished Registration Agent Flow.
[15-08-2024 08:14:42] INFO: SGX Registration Agent version: 1.20.100.2
[15-08-2024 08:14:42] INFO: Starts Registration Agent Flow.
[15-08-2024 08:14:42] INFO: SGX MP Server configuration flag indicates that Registration Server won't save encrypted platform keys.
[15-08-2024 08:14:42] INFO: Platform registration request (PLATFORM_MANIFEST) won't be send to Registration Server.
[15-08-2024 08:14:42] INFO: Please use management tool or PCKCertIDRetrievalTool to read PLATFORM_MANIFEST.
[15-08-2024 08:14:42] INFO: Finished Registration Agent Flow.
[15-08-2024 09:00:04] INFO: SGX Registration Agent version: 1.20.100.2
[15-08-2024 09:00:04] INFO: Starts Registration Agent Flow.
[15-08-2024 09:00:04] INFO: SGX MP Server configuration flag indicates that Registration Server won't save encrypted platform keys.
[15-08-2024 09:00:04] INFO: Platform registration request (PLATFORM_MANIFEST) won't be send to Registration Server.
[15-08-2024 09:00:04] INFO: Please use management tool or PCKCertIDRetrievalTool to read PLATFORM_MANIFEST.
[15-08-2024 09:00:04] INFO: Finished Registration Agent Flow.
[15-08-2024 09:11:42] INFO: SGX Registration Agent version: 1.20.100.2
[15-08-2024 09:11:42] INFO: Starts Registration Agent Flow.
[15-08-2024 09:11:42] INFO: SGX MP Server configuration flag indicates that Registration Server won't save encrypted platform keys.
[15-08-2024 09:11:42] INFO: Platform registration request (PLATFORM_MANIFEST) won't be send to Registration Server.
[15-08-2024 09:11:42] INFO: Please use management tool or PCKCertIDRetrievalTool to read PLATFORM_MANIFEST.
[15-08-2024 09:11:42] INFO: Finished Registration Agent Flow.
[15-08-2024 09:17:51] INFO: SGX Registration Agent version: 1.20.100.2
[15-08-2024 09:17:51] INFO: Starts Registration Agent Flow.
[15-08-2024 09:17:51] INFO: SGX MP Server configuration flag indicates that Registration Server won't save encrypted platform keys.
[15-08-2024 09:17:51] INFO: Platform registration request (PLATFORM_MANIFEST) won't be send to Registration Server.
[15-08-2024 09:17:51] INFO: Please use management tool or PCKCertIDRetrievalTool to read PLATFORM_MANIFEST.
[15-08-2024 09:17:51] INFO: Finished Registration Agent Flow.
[16-10-2024 02:56:33] INFO: SGX Registration Agent version: 1.20.100.2
[16-10-2024 02:56:33] INFO: Starts Registration Agent Flow.
[16-10-2024 02:56:59] INFO: Registration Flow - PLATFORM_ESTABLISHMENT or TCB_RECOVERY passed successfully.
[16-10-2024 02:56:59] INFO: Finished Registration Agent Flow.
@syncronize-issues-to-jira Syncronize issues to Jira
Copy link

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/PEK-1462.

This message was autogenerated

@hector-cao hector-cao self-assigned this Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hector-cao hector-cao

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@syambabu8k @hector-cao