QM:podman run is failing in Fedora 41

[Yarboa]

During the work on #660 fedora QM podman run is failing with the following error while spawning rootful container inside qm

Error: crun: setrlimit `RLIMIT_NOFILE`: Operation not permitted: OCI permission denied
[ FAILED ] Error: Command podman exec qm podman run alpine echo Hello QM failed with exit code: 126

https://artifacts.dev.testing-farm.io/0b3cd818-fbdb-40a1-bcc8-d6d7e81ead39/

TODO:
Add update with extra details

uname -r
6.11.8-300.fc41.x86_64

podman infos
podman-qm-info.json
podman-host-info.json

Running podman logs, attached reveal this, podman-err.log

[conmon:d]: failed to write to /proc/self/oom_score_adj: Permission denied

No selinux errors

ausearch -m AVC  -ts recent
<no matches>

Note when replacing quadlet, see link, Podman args with --privileged container is forked successfully
https://github.com/containers/qm/blob/main/qm.container

Additional Info
Podman inspect log

               "Ulimits": [
                    {
                         "Name": "RLIMIT_NOFILE",
                         "Soft": 1048576,
                         "Hard": 1048576
                    }, 
                    { 
                         "Name": "RLIMIT_NPROC",
                         "Soft": 1048576,
                         "Hard": 1048576
                    }
               ],

podman-inspect.log

Ulimits

[root@505c2268-5b27-4bd8-819f-e58aa67cd028 ~]# ulimit -a -S
real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) unlimited
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 31594
max locked memory           (kbytes, -l) 8192
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1024
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) 31594
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited
[root@505c2268-5b27-4bd8-819f-e58aa67cd028 ~]# ulimit -a -H
real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) unlimited
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 31594
max locked memory           (kbytes, -l) 8192
max memory size             (kbytes, -m) unlimited
open files                          (-n) 524288
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) unlimited
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) 31594
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited

[Yarboa]

@giuseppe @rhatdan Can you please take a look?

[giuseppe]

can you please try with upstream Podman? Could be fixed by containers/podman#24547

Do you have a Podman command line that shows the error? Are you running as root? Is it inside a user namespace?

[Yarboa]

can you please try with upstream Podman? Could be fixed by containers/podman#24547

Do you have a Podman command line that shows the error? Are you running as root? Is it inside a user namespace?

the command goes as following

[root@505c2268-5b27-4bd8-819f-e58aa67cd028 ~]# podman exec qm podman run ubi9-minimal pwd
Error: crun: setrlimit `RLIMIT_NOFILE`: Operation not permitted: OCI permission denied

Yes it is run as root.

Taking a look at pr24547
Do you refer this ?
https://podman.io/docs/installation#fedora-1

repo updates-testing?

[giuseppe]

thanks, that is indeed nested podman, so I think the PR I have mentioned fixes your problem

[Yarboa]

thanks, that is indeed nested podman, so I think the PR I have mentioned fixes your problem

same problem with this

dnf -y install --use-host-config   podman
Updating and loading repositories:
Repositories loaded.
Package "podman-5:5.3.1-1.fc41.x86_64" is already installed.

dnf -y install --use-host-config  --installroot /usr/lib/qm/rootfs/ podman
Updating and loading repositories:
 Copr repo for podman-next owned by rhcontainerbot    100% | 281.5 KiB/s | 139.6 KiB |  00m00s 
Repositories loaded.
Package "podman-5:5.3.1-1.fc41.x86_64" is already installed

still failing

podman exec qm podman run ubi9-minimal pwd
Error: crun: setrlimit `RLIMIT_NOFILE`: Operation not permitted: OCI permission denied
[root@475e2329-93c7-4fe6-b4bf-d7b4ea581832 ~]# 

With --log-level debug

time="2024-11-26T13:40:14Z" level=info msg="Running conmon under slice machine.slice and unitName libpod-conmon-4f17acf681a4407247b456479b905e240e2435e877deb4e9d366700de3f9bc79.scope"
[conmon:d]: failed to write to /proc/self/oom_score_adj: Permission denied

[giuseppe]

have you updated the podman inside the image?

[Yarboa]

have you updated the podman inside the image?

yes, it is partitioned file-system in qm repo not an image

[root@475e2329-93c7-4fe6-b4bf-d7b4ea581832 ~]# podman exec qm rpm -q podman
podman-5.3.1-1.fc41.x86_64

[giuseppe]

do you prefer if we move this issue to podman or do you create a new one?

Please provide a reproducer using just podman (both the external container, and the nested one). Please specify how the external container was created.

[Yarboa]

Sure thanks I will do that @giuseppe thanks,

[Yarboa]

@giuseppe thanks
containers/podman#24692