You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Today, most UMF's memory providers (all except CUDA provider) on Linux use the pidfd_getfd(2) system call to obtain a duplicate of another process's file descriptor. The ptrace governs permission to duplicate another process's file descriptor.
Today UMF requires the /proc/sys/kernel/yama/ptrace_scope to be 0. Otherwise, the pidfd_getfd fails. It is a significant limitation because on Ubuntu the /proc/sys/kernel/yama/ptrace_scope is 1 ("restricted ptrace") by default and some UMF users might have no permission to change it.
Description
The PR_SET_PTRACER allows other processes to ptrace(2) the calling process (see man page for details). The idea is that the process that creates the IPC handle (calls umfGetIPCHandle) can call the int prctl(PR_SET_PTRACER, long pid); syscall to allow other processes to ptrace the current process when the /proc/sys/kernel/yama/ptrace_scope is 1.
The oneCCL already employs that for the L0 backend, Intel MPI will introduce it soon. We should use this mechanism as well.
API Changes
No API changes. Only implementation should be updated
Implementation details
The int prctl(PR_SET_PTRACER, long pid); syscall should called once per process. We can do it in the UMF init or we can do it lazily at first access to IPC functionality.
By default, oneCCL and Intel MPI use the parent process ID (retrieved using getppid()) to allow only processes with the same parent (MPI/oneCCL ranks have the same parent: mpirun) to ptrace the current process. UMF can use the same approach and provide an environment variable to tweak that and allow all processes to ptrace the current process.
How to test
Set the /proc/sys/kernel/yama/ptrace_scope to 1 and run IPC tests.
Today, most UMF's memory providers (all except CUDA provider) on Linux use the pidfd_getfd(2) system call to obtain a duplicate of another process's file descriptor.
AFAIK pidfd_getfd(2) is used only in the utils_duplicate_fd() function, which is used only in 2 providers: OS and L0.
Rationale
Today, most UMF's memory providers (all except CUDA provider) on Linux use the
pidfd_getfd(2)
system call to obtain a duplicate of another process's file descriptor. The ptrace governs permission to duplicate another process's file descriptor.Today UMF requires the
/proc/sys/kernel/yama/ptrace_scope
to be0
. Otherwise, thepidfd_getfd
fails. It is a significant limitation because on Ubuntu the/proc/sys/kernel/yama/ptrace_scope
is1
("restricted ptrace") by default and some UMF users might have no permission to change it.Description
The
PR_SET_PTRACER
allows other processes to ptrace(2) the calling process (see man page for details). The idea is that the process that creates the IPC handle (callsumfGetIPCHandle
) can call theint prctl(PR_SET_PTRACER, long pid);
syscall to allow other processes toptrace
the current process when the/proc/sys/kernel/yama/ptrace_scope
is1
.The oneCCL already employs that for the L0 backend, Intel MPI will introduce it soon. We should use this mechanism as well.
API Changes
No API changes. Only implementation should be updated
Implementation details
int prctl(PR_SET_PTRACER, long pid);
syscall should called once per process. We can do it in the UMF init or we can do it lazily at first access to IPC functionality.getppid()
) to allow only processes with the same parent (MPI/oneCCL ranks have the same parent:mpirun
) to ptrace the current process. UMF can use the same approach and provide an environment variable to tweak that and allow all processes to ptrace the current process.How to test
Set the
/proc/sys/kernel/yama/ptrace_scope
to1
and run IPC tests.Meta
PR_SET_PTRACER
The text was updated successfully, but these errors were encountered: