Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use prctl when ptrace is restricted #979

Open
vinser52 opened this issue Dec 8, 2024 · 4 comments
Open

Use prctl when ptrace is restricted #979

vinser52 opened this issue Dec 8, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@vinser52
Copy link
Contributor

vinser52 commented Dec 8, 2024

Rationale

Today, most UMF's memory providers (all except CUDA provider) on Linux use the pidfd_getfd(2) system call to obtain a duplicate of another process's file descriptor. The ptrace governs permission to duplicate another process's file descriptor.
Today UMF requires the /proc/sys/kernel/yama/ptrace_scope to be 0. Otherwise, the pidfd_getfd fails. It is a significant limitation because on Ubuntu the /proc/sys/kernel/yama/ptrace_scope is 1 ("restricted ptrace") by default and some UMF users might have no permission to change it.

Description

The PR_SET_PTRACER allows other processes to ptrace(2) the calling process (see man page for details). The idea is that the process that creates the IPC handle (calls umfGetIPCHandle) can call the int prctl(PR_SET_PTRACER, long pid); syscall to allow other processes to ptrace the current process when the /proc/sys/kernel/yama/ptrace_scope is 1.
The oneCCL already employs that for the L0 backend, Intel MPI will introduce it soon. We should use this mechanism as well.

API Changes

No API changes. Only implementation should be updated

Implementation details

  1. The int prctl(PR_SET_PTRACER, long pid); syscall should called once per process. We can do it in the UMF init or we can do it lazily at first access to IPC functionality.
  2. By default, oneCCL and Intel MPI use the parent process ID (retrieved using getppid()) to allow only processes with the same parent (MPI/oneCCL ranks have the same parent: mpirun) to ptrace the current process. UMF can use the same approach and provide an environment variable to tweak that and allow all processes to ptrace the current process.

How to test

Set the /proc/sys/kernel/yama/ptrace_scope to 1 and run IPC tests.

Meta

PR_SET_PTRACER

@vinser52 vinser52 added the enhancement New feature or request label Dec 8, 2024
@bratpiorka
Copy link
Contributor

looks great

@ldorau
Copy link
Contributor

ldorau commented Dec 9, 2024

Today, most UMF's memory providers (all except CUDA provider) on Linux use the pidfd_getfd(2) system call to obtain a duplicate of another process's file descriptor.

AFAIK pidfd_getfd(2) is used only in the utils_duplicate_fd() function, which is used only in 2 providers: OS and L0.

@vinser52
Copy link
Contributor Author

vinser52 commented Dec 9, 2024

AFAIK pidfd_getfd(2) is used only in the utils_duplicate_fd() function, which is used only in 2 providers: OS and L0.

What about file and devdax providers? Do they use just a file path?

Anyway, we have to implement it to improve the situation on Ubuntu for OS and L0 providers. The MPI and oneCCL are mostly interesting for L0 flow.

@vinser52
Copy link
Contributor Author

vinser52 commented Dec 9, 2024

In case of any questions regarding implementation, @nikitaxgusev might be a good contact to answer questions as he already implemented it in oneCCL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants