Not sure if relevant, but this could be a kernel (hidepid?) hardening or empty mount? [pid 360] 1654493537.688190 getppid() = 355 [pid 360] 1654493537.688437 openat(AT_FDCWD, "/proc/355/cmdline", O_RDONLY) = -1 ENOENT (No such file or directory) Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: linux-lvm im Auftrag von Abhishek Agarwal Gesendet: Monday, June 6, 2022 7:49:47 AM An: LVM general discussion and development Betreff: Re: [linux-lvm] lvm commands hanging when run from inside a kubernetes pod 1. Yes, use_lvmetad is 0, and its systemd units for it are stopped/disabled. 2. Yes, everything on the host machine i.e(/proc, /sys etc) are getting mounted on the pod. ubuntu@ip-172-31-89-47:~$ kubectl exec -it openebs-lvm-node-v6jrb -c openebs-lvm-plugin -n kube-system -- sh # ls bin boot dev etc home host lib lib32 lib64 libx32 media mnt opt plugin proc root run sbin srv sys tmp usr var # cd /host # ls bin boot dev etc home lib lib32 lib64 libx32 lost+found media mnt opt proc root run sbin snap srv sys tmp usr var # 3. The detail output of `strace -f -ttt` command: https://pastebin.com/raw/VFyXLNaC On Fri, 3 Jun 2022 at 12:48, Roger Heflin > wrote: Random thoughts. Make sure use_lvmetad is 0, and its systemd units for it are stopped/disabled. Are you mounting /proc and /sys and /dev into the /host chroot? /run may also be needed. you might add a "-ttt" to the strace command to give timing data. On Thu, Jun 2, 2022 at 1:41 AM Abhishek Agarwal > wrote: These are not different LVM processes. The container process is using the LVM binary that the node itself has. We have achieved this by using scripts that point to the same lvm binary that is used by the node. Configmap(~shell script) used for the same has the following contents where `/host` refers to the root directory of the node: get_bin_path: | #!/bin/sh bin_name=$1 if [ -x /host/bin/which ]; then echo $(chroot /host /bin/which $bin_name | cut -d ' ' -f 1) elif [ -x /host/usr/bin/which ]; then echo $(chroot /host /usr/bin/which $bin_name | cut -d ' ' -f 1) else $(chroot /host which $bin_name | cut -d ' ' -f 1) fi lvcreate: | #!/bin/sh path=$(/sbin/lvm-eg/get_bin_path "lvcreate") chroot /host $path "$@" Also, the above logs in the pastebin link have errors because the vg lock has not been acquired and hence creation commands will fail. Once the lock is acquired, the `strace -f` command gives the following output being stuck. Check out this link for full details -> https://pastebin.com/raw/DwQfdmr8 P.S: We at OpenEBS are trying to provide lvm storage to cloud native workloads with the help of kubernetes CSI drivers and since all these drivers run as pods and help dynamic provisioning of kubernetes volumes(storage) for the application, the lvm commands needs to be run from inside the pod. Reference -> https://github.com/openebs/lvm-localpv Regards On Wed, 1 Jun 2022 at 13:06, Demi Marie Obenour > wrote: On Wed, Jun 01, 2022 at 12:20:32AM +0530, Abhishek Agarwal wrote: > Hi Roger. Thanks for your reply. I have rerun the command with `strace -f` > as you suggested. Here is the pastebin link containing the detailed output > of the command: https://pastebin.com/raw/VRuBbHBc Even if you can get LVM “working”, it is still likely to cause data corruption at some point, as there is no guarantee that different LVM processes in different namespaces will see each others’ locks. Why do you need to run LVM in a container? What are you trying to accomplish? -- Sincerely, Demi Marie Obenour (she/her/hers) Invisible Things Lab _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/