* Firmware assisted dump support in dracut @ 2012-11-28 16:12 Mahesh J Salgaonkar [not found] ` <20121128161234.GA8991-xthvdsQ13ZrQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 19+ messages in thread From: Mahesh J Salgaonkar @ 2012-11-28 16:12 UTC (permalink / raw) To: Initramfs Dracut, Harald Hoyer Cc: Vivek Goyal, Ananth Narayan, Steven F Best, Haren Myneni Hi Harald, Last year I worked on adding Firmware assisted dump (fadump) support in PowerLinux (ppc64). This feature is now accepted upstream Linux kernel and the documentation is available at http://lwn.net/Articles/488132/ Like kdump, fadump also exports the memory dump through /proc/vmcore in ELF format. This enables us to reuse the existing kdump infrastructure for dump capture and filtering. However, unlike kdump, fadump does not use kexec-based approach, instead it depends on Power firmware to preserve the memory dump and reboot into new kernel. This is what happens in fadump after crash: 1. At the crash, kernel informs power firmware that kernel has crashed. 2. Firmware takes the control and reboots the entire system preserving only the memory (resets all other devices). 3. The reboot follows the normal booting process (non-kexec). 4. The boot loader loads the default kernel and initrd from /boot I am working on integrating fadump with existing kdump infrastructure. The current kdump infrastructure builds a separate initrd (whenever there is a change detected in kdump config file /etc/kdump.conf) which then gets loaded into memory by kexec tool for use by kdump kernel. But, in the fadump approach, the second kernel (after crash) always use the default (OS built) initramfs. Hence, to support fadump, change is required to introduce dump capturing steps in default initramfs itself. Hence the possible approaches I am looking into are: 1. Rebuild the default initramfs every time when there is a change detected in kdump config file (/etc/kdump.conf) This approach would modify existing initramfs in place. I did work on this approach by enhancing mkdumprd (tool from kexec-tools package) to extract and rebuild default initramfs with dump capturing steps. After discussing with Vivek Goyal, he suggested that better approach to add code to dracut for rebuilding boot initramfs instead of enhancing mkdumprd. This means introducing a new dracut module that will be responsible for introducing dump capture steps inside the rebuilt initramfs by pulling required modules e.g. ssh, nfs etc. depending on /etc/kdump.conf Now the question is whether it is possible for dracut to rebuild boot initramfs in place? if Yes, is there any issues in rebuilding of boot initramfs everytime when there is a change in /etc/kdump.conf? 2. Make dracut tool fadump aware and it will build fadump aware initrd (default OS initrd) during kernel install. Once built, this initrd should never be rebuilt again. Which means the default initramfs will contain vmcore capture steps. This approach would pack all possible dracut modules (nfs, ssh, bonding etc.) so that the default initrd will be capable of supporting all possible dump target. Hence this approach will bloat the initramfs. On my Power test system, the size of the default initrd generated without any changes is 16M. With the above approach this size jumps to 27M. Since both the above approaches would require changes to dracut package, I would like to know your views on above approaches. Thanks, -Mahesh. -- Mahesh J Salgaonkar ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <20121128161234.GA8991-xthvdsQ13ZrQT0dZR+AlfA@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <20121128161234.GA8991-xthvdsQ13ZrQT0dZR+AlfA@public.gmane.org> @ 2012-11-29 10:14 ` Harald Hoyer [not found] ` <50B73598.804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 19+ messages in thread From: Harald Hoyer @ 2012-11-29 10:14 UTC (permalink / raw) To: mahesh-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8 Cc: Initramfs Dracut, Vivek Goyal, Ananth Narayan, Steven F Best, Haren Myneni Am 28.11.2012 17:12, schrieb Mahesh J Salgaonkar: > Hi Harald, > > Last year I worked on adding Firmware assisted dump (fadump) support in > PowerLinux (ppc64). This feature is now accepted upstream Linux kernel and the > documentation is available at http://lwn.net/Articles/488132/ > > Like kdump, fadump also exports the memory dump through /proc/vmcore in ELF > format. This enables us to reuse the existing kdump infrastructure for dump > capture and filtering. However, unlike kdump, fadump does not use kexec-based > approach, instead it depends on Power firmware to preserve the memory dump and > reboot into new kernel. This is what happens in fadump after crash: > > 1. At the crash, kernel informs power firmware that kernel has crashed. > 2. Firmware takes the control and reboots the entire system preserving only the > memory (resets all other devices). > 3. The reboot follows the normal booting process (non-kexec). > 4. The boot loader loads the default kernel and initrd from /boot > > I am working on integrating fadump with existing kdump infrastructure. The > current kdump infrastructure builds a separate initrd (whenever there is a > change detected in kdump config file /etc/kdump.conf) which then gets loaded > into memory by kexec tool for use by kdump kernel. But, in the fadump approach, > the second kernel (after crash) always use the default (OS built) initramfs. > Hence, to support fadump, change is required to introduce dump capturing steps > in default initramfs itself. Hence the possible approaches I am looking into > are: > > 1. Rebuild the default initramfs every time when there is a change detected in > kdump config file (/etc/kdump.conf) > This approach would modify existing initramfs in place. I did work on this > approach by enhancing mkdumprd (tool from kexec-tools package) to extract > and rebuild default initramfs with dump capturing steps. After discussing > with Vivek Goyal, he suggested that better approach to add code to dracut > for rebuilding boot initramfs instead of enhancing mkdumprd. This means > introducing a new dracut module that will be responsible for introducing > dump capture steps inside the rebuilt initramfs by pulling required modules > e.g. ssh, nfs etc. depending on /etc/kdump.conf > > Now the question is whether it is possible for dracut to rebuild boot > initramfs in place? if Yes, is there any issues in rebuilding of boot > initramfs everytime when there is a change in /etc/kdump.conf? dracut is called by /sbin/new-kernel-package. dracut is not a service, which is run on every boot. So, no, dracut can't watch files and build an initramfs on itsself. Of course such a service could be created and create the initramfs on shutdown, if any of the files, which should be watched, have changed. This of course is a dangerous automatism, which might as well lead to a unbootable system. Further steps (with grubby) would have to make sure, the last bootable entry is still there, along with the last kernel version. I was already thinking about creating such a monster, but this would involve to reinvent the current /sbin/new-kernel-package / grubby infrastructure, which has to be done anyway in the next years. The first step was to create a common configuration file format for all the bootloaders, which they parse and display. http://harald.fedorapeople.org/downloads/boot-unification.pdf A simple boot manager parsing the config layout as a reference implementation: http://freedesktop.org/wiki/Software/gummiboot Next step is to patch grub2 to parse those config files and display the menu entries. Next step is to enhance /sbin/new-kernel-package and obsolete grubby. Then, I think we are ready to create such a service. > > 2. Make dracut tool fadump aware and it will build fadump aware initrd (default > OS initrd) during kernel install. Once built, this initrd should never be > rebuilt again. Which means the default initramfs will contain vmcore capture > steps. This approach would pack all possible dracut modules (nfs, ssh, bonding > etc.) so that the default initrd will be capable of supporting all possible > dump target. Hence this approach will bloat the initramfs. On my Power test > system, the size of the default initrd generated without any changes is 16M. > With the above approach this size jumps to 27M. Yeah, this is crazy. > > Since both the above approaches would require changes to dracut package, I would > like to know your views on above approaches. > > Thanks, > -Mahesh. > ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <50B73598.804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <50B73598.804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2012-11-30 15:31 ` Vivek Goyal 2012-12-06 6:21 ` Mahesh Jagannath Salgaonkar 1 sibling, 0 replies; 19+ messages in thread From: Vivek Goyal @ 2012-11-30 15:31 UTC (permalink / raw) To: Harald Hoyer Cc: mahesh-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni On Thu, Nov 29, 2012 at 11:14:48AM +0100, Harald Hoyer wrote: [..] > > Now the question is whether it is possible for dracut to rebuild boot > > initramfs in place? if Yes, is there any issues in rebuilding of boot > > initramfs everytime when there is a change in /etc/kdump.conf? > > dracut is called by /sbin/new-kernel-package. dracut is not a service, which is > run on every boot. So, no, dracut can't watch files and build an initramfs on > itsself. > > Of course such a service could be created and create the initramfs on shutdown, > if any of the files, which should be watched, have changed. > > This of course is a dangerous automatism, which might as well lead to a > unbootable system. Further steps (with grubby) would have to make sure, the last > bootable entry is still there, along with the last kernel version. > > I was already thinking about creating such a monster, but this would involve to > reinvent the current /sbin/new-kernel-package / grubby infrastructure, which has > to be done anyway in the next years. The first step was to create a common > configuration file format for all the bootloaders, which they parse and display. > > http://harald.fedorapeople.org/downloads/boot-unification.pdf > A simple boot manager parsing the config layout as a reference implementation: > http://freedesktop.org/wiki/Software/gummiboot > > Next step is to patch grub2 to parse those config files and display the menu > entries. > > Next step is to enhance /sbin/new-kernel-package and obsolete grubby. > > Then, I think we are ready to create such a service. Once we create the infrastructure where we can regenerate the initramfs, then kdump service should be able to detect change in files and call relevant hook (new-kernel-package or whatever). So important point here seems to be that yes we will regenrate initramfs but we need to do it in such a way so that we retain the old bootable kernel/initramfs around in case new initramfs fails to boot. So this is like installing new kernel execpt the fact that there is no new kernel. It is old kernel but new initramfs. > > > > > 2. Make dracut tool fadump aware and it will build fadump aware initrd (default > > OS initrd) during kernel install. Once built, this initrd should never be > > rebuilt again. Which means the default initramfs will contain vmcore capture > > steps. This approach would pack all possible dracut modules (nfs, ssh, bonding > > etc.) so that the default initrd will be capable of supporting all possible > > dump target. Hence this approach will bloat the initramfs. On my Power test > > system, the size of the default initrd generated without any changes is 16M. > > With the above approach this size jumps to 27M. > > Yeah, this is crazy. In the long term being able to regenerate initramfs in response to kdump configuration changes should prove to be a better way of doing things. Thanks Vivek ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Firmware assisted dump support in dracut [not found] ` <50B73598.804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2012-11-30 15:31 ` Vivek Goyal @ 2012-12-06 6:21 ` Mahesh Jagannath Salgaonkar [not found] ` <50C03971.4050104-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 1 sibling, 1 reply; 19+ messages in thread From: Mahesh Jagannath Salgaonkar @ 2012-12-06 6:21 UTC (permalink / raw) To: Harald Hoyer Cc: Initramfs Dracut, Vivek Goyal, Ananth Narayan, Steven F Best, Haren Myneni On 11/29/2012 03:44 PM, Harald Hoyer wrote: > Am 28.11.2012 17:12, schrieb Mahesh J Salgaonkar: >> Hi Harald, >> >> Last year I worked on adding Firmware assisted dump (fadump) support in >> PowerLinux (ppc64). This feature is now accepted upstream Linux kernel and the >> documentation is available at http://lwn.net/Articles/488132/ >> >> Like kdump, fadump also exports the memory dump through /proc/vmcore in ELF >> format. This enables us to reuse the existing kdump infrastructure for dump >> capture and filtering. However, unlike kdump, fadump does not use kexec-based >> approach, instead it depends on Power firmware to preserve the memory dump and >> reboot into new kernel. This is what happens in fadump after crash: >> >> 1. At the crash, kernel informs power firmware that kernel has crashed. >> 2. Firmware takes the control and reboots the entire system preserving only the >> memory (resets all other devices). >> 3. The reboot follows the normal booting process (non-kexec). >> 4. The boot loader loads the default kernel and initrd from /boot >> >> I am working on integrating fadump with existing kdump infrastructure. The >> current kdump infrastructure builds a separate initrd (whenever there is a >> change detected in kdump config file /etc/kdump.conf) which then gets loaded >> into memory by kexec tool for use by kdump kernel. But, in the fadump approach, >> the second kernel (after crash) always use the default (OS built) initramfs. >> Hence, to support fadump, change is required to introduce dump capturing steps >> in default initramfs itself. Hence the possible approaches I am looking into >> are: >> >> 1. Rebuild the default initramfs every time when there is a change detected in >> kdump config file (/etc/kdump.conf) >> This approach would modify existing initramfs in place. I did work on this >> approach by enhancing mkdumprd (tool from kexec-tools package) to extract >> and rebuild default initramfs with dump capturing steps. After discussing >> with Vivek Goyal, he suggested that better approach to add code to dracut >> for rebuilding boot initramfs instead of enhancing mkdumprd. This means >> introducing a new dracut module that will be responsible for introducing >> dump capture steps inside the rebuilt initramfs by pulling required modules >> e.g. ssh, nfs etc. depending on /etc/kdump.conf >> >> Now the question is whether it is possible for dracut to rebuild boot >> initramfs in place? if Yes, is there any issues in rebuilding of boot >> initramfs everytime when there is a change in /etc/kdump.conf? > > dracut is called by /sbin/new-kernel-package. dracut is not a service, which is > run on every boot. So, no, dracut can't watch files and build an initramfs on > itsself. > Normally user would restart kdump service after changing kdump config file. Idea is to invoke dracut through kdump service, as it already watches and detect changes in kdump config files. I was wondering if dracut allow us to insert/install additional dracut modules to already existing initramfs? OR does it rebuild initramfs all over again? > Of course such a service could be created and create the initramfs on shutdown, > if any of the files, which should be watched, have changed. > > This of course is a dangerous automatism, which might as well lead to a > unbootable system. Further steps (with grubby) would have to make sure, the last > bootable entry is still there, along with the last kernel version. > > I was already thinking about creating such a monster, but this would involve to > reinvent the current /sbin/new-kernel-package / grubby infrastructure, which has > to be done anyway in the next years. The first step was to create a common > configuration file format for all the bootloaders, which they parse and display. > > http://harald.fedorapeople.org/downloads/boot-unification.pdf > A simple boot manager parsing the config layout as a reference implementation: > http://freedesktop.org/wiki/Software/gummiboot > > Next step is to patch grub2 to parse those config files and display the menu > entries. I think once such a infrastructure exists, things would be much easier. Also, would it be possible to make such a infrastructure OS crash aware, so that it can load different initramfs in crash scenario and default initramfs in a normal boot process. I am not sure how practical this idea would be to implement. > > Next step is to enhance /sbin/new-kernel-package and obsolete grubby. > > Then, I think we are ready to create such a service. > >> >> 2. Make dracut tool fadump aware and it will build fadump aware initrd (default >> OS initrd) during kernel install. Once built, this initrd should never be >> rebuilt again. Which means the default initramfs will contain vmcore capture >> steps. This approach would pack all possible dracut modules (nfs, ssh, bonding >> etc.) so that the default initrd will be capable of supporting all possible >> dump target. Hence this approach will bloat the initramfs. On my Power test >> system, the size of the default initrd generated without any changes is 16M. >> With the above approach this size jumps to 27M. > > Yeah, this is crazy. > >> >> Since both the above approaches would require changes to dracut package, I would >> like to know your views on above approaches. >> >> Thanks, >> -Mahesh. >> > > -- > To unsubscribe from this list: send the line "unsubscribe initramfs" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <50C03971.4050104-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <50C03971.4050104-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> @ 2013-05-30 5:17 ` Mahesh Jagannath Salgaonkar [not found] ` <51A6E0E4.1000502-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 0 siblings, 1 reply; 19+ messages in thread From: Mahesh Jagannath Salgaonkar @ 2013-05-30 5:17 UTC (permalink / raw) To: Harald Hoyer Cc: Initramfs Dracut, Vivek Goyal, Ananth Narayan, Steven F Best, Haren Myneni On 12/06/2012 11:51 AM, Mahesh Jagannath Salgaonkar wrote: > On 11/29/2012 03:44 PM, Harald Hoyer wrote: >> Am 28.11.2012 17:12, schrieb Mahesh J Salgaonkar: >>> Hi Harald, >>> >>> Last year I worked on adding Firmware assisted dump (fadump) support in >>> PowerLinux (ppc64). This feature is now accepted upstream Linux kernel and the >>> documentation is available at http://lwn.net/Articles/488132/ >>> >>> Like kdump, fadump also exports the memory dump through /proc/vmcore in ELF >>> format. This enables us to reuse the existing kdump infrastructure for dump >>> capture and filtering. However, unlike kdump, fadump does not use kexec-based >>> approach, instead it depends on Power firmware to preserve the memory dump and >>> reboot into new kernel. This is what happens in fadump after crash: >>> >>> 1. At the crash, kernel informs power firmware that kernel has crashed. >>> 2. Firmware takes the control and reboots the entire system preserving only the >>> memory (resets all other devices). >>> 3. The reboot follows the normal booting process (non-kexec). >>> 4. The boot loader loads the default kernel and initrd from /boot >>> >>> I am working on integrating fadump with existing kdump infrastructure. The >>> current kdump infrastructure builds a separate initrd (whenever there is a >>> change detected in kdump config file /etc/kdump.conf) which then gets loaded >>> into memory by kexec tool for use by kdump kernel. But, in the fadump approach, >>> the second kernel (after crash) always use the default (OS built) initramfs. >>> Hence, to support fadump, change is required to introduce dump capturing steps >>> in default initramfs itself. Hence the possible approaches I am looking into >>> are: >>> >>> 1. Rebuild the default initramfs every time when there is a change detected in >>> kdump config file (/etc/kdump.conf) >>> This approach would modify existing initramfs in place. I did work on this >>> approach by enhancing mkdumprd (tool from kexec-tools package) to extract >>> and rebuild default initramfs with dump capturing steps. After discussing >>> with Vivek Goyal, he suggested that better approach to add code to dracut >>> for rebuilding boot initramfs instead of enhancing mkdumprd. This means >>> introducing a new dracut module that will be responsible for introducing >>> dump capture steps inside the rebuilt initramfs by pulling required modules >>> e.g. ssh, nfs etc. depending on /etc/kdump.conf >>> >>> Now the question is whether it is possible for dracut to rebuild boot >>> initramfs in place? if Yes, is there any issues in rebuilding of boot >>> initramfs everytime when there is a change in /etc/kdump.conf? >> >> dracut is called by /sbin/new-kernel-package. dracut is not a service, which is >> run on every boot. So, no, dracut can't watch files and build an initramfs on >> itsself. >> > > Normally user would restart kdump service after changing kdump config > file. Idea is to invoke dracut through kdump service, as it already > watches and detect changes in kdump config files. > > I was wondering if dracut allow us to insert/install additional dracut > modules to already existing initramfs? OR does it rebuild initramfs all > over again? Sorry for restarting this discussion very late. I would like to know how safe is to rebuild kernel's default (boot) initramfs for an already installed kernel ? Also, Does dracut provides any of following mechanism? a) Mechanism where dracut can detect what options were used during first build for a given (exsisting) initramfs. (This mechanism may help one to regenerate similar initramfs with additional dracut modules.) OR b) Add additional modules to an existing initramfs instead of regenerating again. Is this possible at all? Thanks, -Mahesh. > >> Of course such a service could be created and create the initramfs on shutdown, >> if any of the files, which should be watched, have changed. >> >> This of course is a dangerous automatism, which might as well lead to a >> unbootable system. Further steps (with grubby) would have to make sure, the last >> bootable entry is still there, along with the last kernel version. >> >> I was already thinking about creating such a monster, but this would involve to >> reinvent the current /sbin/new-kernel-package / grubby infrastructure, which has >> to be done anyway in the next years. The first step was to create a common >> configuration file format for all the bootloaders, which they parse and display. >> >> http://harald.fedorapeople.org/downloads/boot-unification.pdf >> A simple boot manager parsing the config layout as a reference implementation: >> http://freedesktop.org/wiki/Software/gummiboot >> >> Next step is to patch grub2 to parse those config files and display the menu >> entries. > > I think once such a infrastructure exists, things would be much easier. > Also, would it be possible to make such a infrastructure OS crash aware, > so that it can load different initramfs in crash scenario and default > initramfs in a normal boot process. I am not sure how practical this > idea would be to implement. > >> >> Next step is to enhance /sbin/new-kernel-package and obsolete grubby. >> >> Then, I think we are ready to create such a service. >> >>> >>> 2. Make dracut tool fadump aware and it will build fadump aware initrd (default >>> OS initrd) during kernel install. Once built, this initrd should never be >>> rebuilt again. Which means the default initramfs will contain vmcore capture >>> steps. This approach would pack all possible dracut modules (nfs, ssh, bonding >>> etc.) so that the default initrd will be capable of supporting all possible >>> dump target. Hence this approach will bloat the initramfs. On my Power test >>> system, the size of the default initrd generated without any changes is 16M. >>> With the above approach this size jumps to 27M. >> >> Yeah, this is crazy. >> >>> >>> Since both the above approaches would require changes to dracut package, I would >>> like to know your views on above approaches. >>> >>> Thanks, >>> -Mahesh. >>> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe initramfs" in >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > -- > To unsubscribe from this list: send the line "unsubscribe initramfs" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <51A6E0E4.1000502-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <51A6E0E4.1000502-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> @ 2013-05-30 6:49 ` Harald Hoyer [not found] ` <51A6F663.5030804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 19+ messages in thread From: Harald Hoyer @ 2013-05-30 6:49 UTC (permalink / raw) To: Mahesh Jagannath Salgaonkar Cc: Initramfs Dracut, Vivek Goyal, Ananth Narayan, Steven F Best, Haren Myneni On 05/30/2013 07:17 AM, Mahesh Jagannath Salgaonkar wrote: > On 12/06/2012 11:51 AM, Mahesh Jagannath Salgaonkar wrote: >> On 11/29/2012 03:44 PM, Harald Hoyer wrote: >>> Am 28.11.2012 17:12, schrieb Mahesh J Salgaonkar: >>>> Hi Harald, >>>> >>>> Last year I worked on adding Firmware assisted dump (fadump) support in >>>> PowerLinux (ppc64). This feature is now accepted upstream Linux kernel and the >>>> documentation is available at http://lwn.net/Articles/488132/ >>>> >>>> Like kdump, fadump also exports the memory dump through /proc/vmcore in ELF >>>> format. This enables us to reuse the existing kdump infrastructure for dump >>>> capture and filtering. However, unlike kdump, fadump does not use kexec-based >>>> approach, instead it depends on Power firmware to preserve the memory dump and >>>> reboot into new kernel. This is what happens in fadump after crash: >>>> >>>> 1. At the crash, kernel informs power firmware that kernel has crashed. >>>> 2. Firmware takes the control and reboots the entire system preserving only the >>>> memory (resets all other devices). >>>> 3. The reboot follows the normal booting process (non-kexec). >>>> 4. The boot loader loads the default kernel and initrd from /boot >>>> >>>> I am working on integrating fadump with existing kdump infrastructure. The >>>> current kdump infrastructure builds a separate initrd (whenever there is a >>>> change detected in kdump config file /etc/kdump.conf) which then gets loaded >>>> into memory by kexec tool for use by kdump kernel. But, in the fadump approach, >>>> the second kernel (after crash) always use the default (OS built) initramfs. >>>> Hence, to support fadump, change is required to introduce dump capturing steps >>>> in default initramfs itself. Hence the possible approaches I am looking into >>>> are: >>>> >>>> 1. Rebuild the default initramfs every time when there is a change detected in >>>> kdump config file (/etc/kdump.conf) >>>> This approach would modify existing initramfs in place. I did work on this >>>> approach by enhancing mkdumprd (tool from kexec-tools package) to extract >>>> and rebuild default initramfs with dump capturing steps. After discussing >>>> with Vivek Goyal, he suggested that better approach to add code to dracut >>>> for rebuilding boot initramfs instead of enhancing mkdumprd. This means >>>> introducing a new dracut module that will be responsible for introducing >>>> dump capture steps inside the rebuilt initramfs by pulling required modules >>>> e.g. ssh, nfs etc. depending on /etc/kdump.conf >>>> >>>> Now the question is whether it is possible for dracut to rebuild boot >>>> initramfs in place? if Yes, is there any issues in rebuilding of boot >>>> initramfs everytime when there is a change in /etc/kdump.conf? >>> >>> dracut is called by /sbin/new-kernel-package. dracut is not a service, which is >>> run on every boot. So, no, dracut can't watch files and build an initramfs on >>> itsself. >>> >> >> Normally user would restart kdump service after changing kdump config >> file. Idea is to invoke dracut through kdump service, as it already >> watches and detect changes in kdump config files. >> >> I was wondering if dracut allow us to insert/install additional dracut >> modules to already existing initramfs? OR does it rebuild initramfs all >> over again? > > Sorry for restarting this discussion very late. I would like to know how > safe is to rebuild kernel's default (boot) initramfs for an already > installed kernel ? > > Also, Does dracut provides any of following mechanism? > a) Mechanism where dracut can detect what options were used during first > build for a given (exsisting) initramfs. (This mechanism may help one to > regenerate similar initramfs with additional dracut modules.) currently dracut only stores which modules were used to generate the image in usr/lib/dracut/modules.txt But yes, you are right. Would be nice to save all the options and have a mechanism to regenerate it with those. > > OR > > b) Add additional modules to an existing initramfs instead of > regenerating again. Is this possible at all? You can always generate a second cpio image with additional files and either concatenate both images or specify those two images in the bootloader. OR unpack the initramfs, add the file, (maybe run depmod, if kernel driver added), and repack. ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <51A6F663.5030804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <51A6F663.5030804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2013-05-30 14:23 ` Vivek Goyal [not found] ` <20130530142318.GC2864-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-04 8:34 ` Mahesh Jagannath Salgaonkar 1 sibling, 1 reply; 19+ messages in thread From: Vivek Goyal @ 2013-05-30 14:23 UTC (permalink / raw) To: Harald Hoyer Cc: Mahesh Jagannath Salgaonkar, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: [..] > > Sorry for restarting this discussion very late. I would like to know how > > safe is to rebuild kernel's default (boot) initramfs for an already > > installed kernel ? > > > > Also, Does dracut provides any of following mechanism? > > a) Mechanism where dracut can detect what options were used during first > > build for a given (exsisting) initramfs. (This mechanism may help one to > > regenerate similar initramfs with additional dracut modules.) > > currently dracut only stores which modules were used to generate the image in > usr/lib/dracut/modules.txt > > But yes, you are right. Would be nice to save all the options and have a > mechanism to regenerate it with those. > I am not sure how well it will work in the context of kdump. kdump options and original options might conflict. kdump might drop some of its own cmdline options in initramfs which might not make any sense in regular boot. Kdump will specify additional mount points. IIUC, then we will try to bring up those targets in regular boot too and mount at user specified mount point. And later init services might get confused when respective daemon tries to bring up same targets again. Thanks Vivek ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <20130530142318.GC2864-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <20130530142318.GC2864-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2013-06-04 8:20 ` Mahesh Jagannath Salgaonkar [not found] ` <51ADA343.2070100-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 0 siblings, 1 reply; 19+ messages in thread From: Mahesh Jagannath Salgaonkar @ 2013-06-04 8:20 UTC (permalink / raw) To: Vivek Goyal Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni On 05/30/2013 07:53 PM, Vivek Goyal wrote: > On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: > > [..] >>> Sorry for restarting this discussion very late. I would like to know how >>> safe is to rebuild kernel's default (boot) initramfs for an already >>> installed kernel ? >>> >>> Also, Does dracut provides any of following mechanism? >>> a) Mechanism where dracut can detect what options were used during first >>> build for a given (exsisting) initramfs. (This mechanism may help one to >>> regenerate similar initramfs with additional dracut modules.) >> >> currently dracut only stores which modules were used to generate the image in >> usr/lib/dracut/modules.txt >> >> But yes, you are right. Would be nice to save all the options and have a >> mechanism to regenerate it with those. >> > > I am not sure how well it will work in the context of kdump. kdump > options and original options might conflict. kdump might drop some of > its own cmdline options in initramfs which might not make any sense > in regular boot. > > Kdump will specify additional mount points. IIUC, then we will try > to bring up those targets in regular boot too and mount at user > specified mount point. And later init services might get confused > when respective daemon tries to bring up same targets again. I think we should be able differentiate between regular boot and boot after crash by checking existence of '/proc/vmcore' file, and do the kdump specific configs/mount if this file exists. In regular boot we can mute the kdump code path. > > Thanks > Vivek > ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <51ADA343.2070100-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <51ADA343.2070100-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> @ 2013-06-04 13:57 ` Vivek Goyal 2013-06-04 14:14 ` Vivek Goyal 1 sibling, 0 replies; 19+ messages in thread From: Vivek Goyal @ 2013-06-04 13:57 UTC (permalink / raw) To: Mahesh Jagannath Salgaonkar Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni On Tue, Jun 04, 2013 at 01:50:19PM +0530, Mahesh Jagannath Salgaonkar wrote: > On 05/30/2013 07:53 PM, Vivek Goyal wrote: > > On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: > > > > [..] > >>> Sorry for restarting this discussion very late. I would like to know how > >>> safe is to rebuild kernel's default (boot) initramfs for an already > >>> installed kernel ? > >>> > >>> Also, Does dracut provides any of following mechanism? > >>> a) Mechanism where dracut can detect what options were used during first > >>> build for a given (exsisting) initramfs. (This mechanism may help one to > >>> regenerate similar initramfs with additional dracut modules.) > >> > >> currently dracut only stores which modules were used to generate the image in > >> usr/lib/dracut/modules.txt > >> > >> But yes, you are right. Would be nice to save all the options and have a > >> mechanism to regenerate it with those. > >> > > > > I am not sure how well it will work in the context of kdump. kdump > > options and original options might conflict. kdump might drop some of > > its own cmdline options in initramfs which might not make any sense > > in regular boot. > > > > Kdump will specify additional mount points. IIUC, then we will try > > to bring up those targets in regular boot too and mount at user > > specified mount point. And later init services might get confused > > when respective daemon tries to bring up same targets again. > > I think we should be able differentiate between regular boot and boot > after crash by checking existence of '/proc/vmcore' file, and do the > kdump specific configs/mount if this file exists. In regular boot we can > mute the kdump code path. dracut takes --mount option and that option doesn not differentiate between environment. One can always create new options to mount certain things only in kdump environment. I am not sure how much sense does it make to try to special case everything related to kdump in dracut. Thanks Vivek ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Firmware assisted dump support in dracut [not found] ` <51ADA343.2070100-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 2013-06-04 13:57 ` Vivek Goyal @ 2013-06-04 14:14 ` Vivek Goyal [not found] ` <20130604141403.GG4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 1 sibling, 1 reply; 19+ messages in thread From: Vivek Goyal @ 2013-06-04 14:14 UTC (permalink / raw) To: Mahesh Jagannath Salgaonkar Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni, holzheu-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8 On Tue, Jun 04, 2013 at 01:50:19PM +0530, Mahesh Jagannath Salgaonkar wrote: > On 05/30/2013 07:53 PM, Vivek Goyal wrote: > > On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: > > > > [..] > >>> Sorry for restarting this discussion very late. I would like to know how > >>> safe is to rebuild kernel's default (boot) initramfs for an already > >>> installed kernel ? > >>> > >>> Also, Does dracut provides any of following mechanism? > >>> a) Mechanism where dracut can detect what options were used during first > >>> build for a given (exsisting) initramfs. (This mechanism may help one to > >>> regenerate similar initramfs with additional dracut modules.) > >> > >> currently dracut only stores which modules were used to generate the image in > >> usr/lib/dracut/modules.txt > >> > >> But yes, you are right. Would be nice to save all the options and have a > >> mechanism to regenerate it with those. > >> > > > > I am not sure how well it will work in the context of kdump. kdump > > options and original options might conflict. kdump might drop some of > > its own cmdline options in initramfs which might not make any sense > > in regular boot. > > > > Kdump will specify additional mount points. IIUC, then we will try > > to bring up those targets in regular boot too and mount at user > > specified mount point. And later init services might get confused > > when respective daemon tries to bring up same targets again. > > I think we should be able differentiate between regular boot and boot > after crash by checking existence of '/proc/vmcore' file, and do the > kdump specific configs/mount if this file exists. In regular boot we can > mute the kdump code path. Also what about kernel command line? Kernel command line can be very specific to kdump environment. It is atleast in x86. I am hoping that's not the case with firmware assisted dump. Also can you point me to some writeup of fadump again. I want to refresh what fadump is doing and how firmware is assisting and how does it make better than regular kdump. I suspect it is very similar to s390 firmware dump (zfcpdump). Does s390 kdump or s390 zfcpdump reboot after saving dump or they have mechanism to continue to boot and hotplug memory after saving dump. I am wondering how do they do it. CCing michael. Vivek ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <20130604141403.GG4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <20130604141403.GG4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2013-06-04 14:21 ` Vivek Goyal [not found] ` <20130604142123.GH4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-04 15:19 ` Michael Holzheu 2013-06-05 9:21 ` Mahesh Jagannath Salgaonkar 2 siblings, 1 reply; 19+ messages in thread From: Vivek Goyal @ 2013-06-04 14:21 UTC (permalink / raw) To: Mahesh Jagannath Salgaonkar Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni, holzheu-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8 On Tue, Jun 04, 2013 at 10:14:03AM -0400, Vivek Goyal wrote: > On Tue, Jun 04, 2013 at 01:50:19PM +0530, Mahesh Jagannath Salgaonkar wrote: > > On 05/30/2013 07:53 PM, Vivek Goyal wrote: > > > On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: > > > > > > [..] > > >>> Sorry for restarting this discussion very late. I would like to know how > > >>> safe is to rebuild kernel's default (boot) initramfs for an already > > >>> installed kernel ? > > >>> > > >>> Also, Does dracut provides any of following mechanism? > > >>> a) Mechanism where dracut can detect what options were used during first > > >>> build for a given (exsisting) initramfs. (This mechanism may help one to > > >>> regenerate similar initramfs with additional dracut modules.) > > >> > > >> currently dracut only stores which modules were used to generate the image in > > >> usr/lib/dracut/modules.txt > > >> > > >> But yes, you are right. Would be nice to save all the options and have a > > >> mechanism to regenerate it with those. > > >> > > > > > > I am not sure how well it will work in the context of kdump. kdump > > > options and original options might conflict. kdump might drop some of > > > its own cmdline options in initramfs which might not make any sense > > > in regular boot. > > > > > > Kdump will specify additional mount points. IIUC, then we will try > > > to bring up those targets in regular boot too and mount at user > > > specified mount point. And later init services might get confused > > > when respective daemon tries to bring up same targets again. > > > > I think we should be able differentiate between regular boot and boot > > after crash by checking existence of '/proc/vmcore' file, and do the > > kdump specific configs/mount if this file exists. In regular boot we can > > mute the kdump code path. > > Also what about kernel command line? Kernel command line can be very > specific to kdump environment. It is atleast in x86. I am hoping that's > not the case with firmware assisted dump. If first kernel and kdump kernel command line are same in fadump and boot environment is same, I think it is worth exploring the option of saving kdump from kdump service when system boots back. That will do away with any need to tweak initramfs. It just requires more memory to be reserved so that system can boot , mount root, run services and then save dump. I am hoping that firmware can set aside enough memory to (1G?), to copy the memory contents and boot the kernel in that 1G. Also does fadump work well with new mmap() interface /proc/vmcore is going to support. Patches are in -mm now. s390 zfcpdump is going to face some issues and michael is working on resolving these. I suspect that fadump will run into similar issues if firmware is copying memory contents somewhere else. Thanks Vivek ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <20130604142123.GH4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <20130604142123.GH4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2013-06-05 13:44 ` Mahesh Jagannath Salgaonkar [not found] ` <51AF40BB.5010703-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 0 siblings, 1 reply; 19+ messages in thread From: Mahesh Jagannath Salgaonkar @ 2013-06-05 13:44 UTC (permalink / raw) To: Vivek Goyal Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni, holzheu-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8 On 06/04/2013 07:51 PM, Vivek Goyal wrote: > On Tue, Jun 04, 2013 at 10:14:03AM -0400, Vivek Goyal wrote: >> On Tue, Jun 04, 2013 at 01:50:19PM +0530, Mahesh Jagannath Salgaonkar wrote: >>> On 05/30/2013 07:53 PM, Vivek Goyal wrote: >>>> On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: >>>> >>>> [..] >>>>>> Sorry for restarting this discussion very late. I would like to know how >>>>>> safe is to rebuild kernel's default (boot) initramfs for an already >>>>>> installed kernel ? >>>>>> >>>>>> Also, Does dracut provides any of following mechanism? >>>>>> a) Mechanism where dracut can detect what options were used during first >>>>>> build for a given (exsisting) initramfs. (This mechanism may help one to >>>>>> regenerate similar initramfs with additional dracut modules.) >>>>> >>>>> currently dracut only stores which modules were used to generate the image in >>>>> usr/lib/dracut/modules.txt >>>>> >>>>> But yes, you are right. Would be nice to save all the options and have a >>>>> mechanism to regenerate it with those. >>>>> >>>> >>>> I am not sure how well it will work in the context of kdump. kdump >>>> options and original options might conflict. kdump might drop some of >>>> its own cmdline options in initramfs which might not make any sense >>>> in regular boot. >>>> >>>> Kdump will specify additional mount points. IIUC, then we will try >>>> to bring up those targets in regular boot too and mount at user >>>> specified mount point. And later init services might get confused >>>> when respective daemon tries to bring up same targets again. >>> >>> I think we should be able differentiate between regular boot and boot >>> after crash by checking existence of '/proc/vmcore' file, and do the >>> kdump specific configs/mount if this file exists. In regular boot we can >>> mute the kdump code path. >> >> Also what about kernel command line? Kernel command line can be very >> specific to kdump environment. It is atleast in x86. I am hoping that's >> not the case with firmware assisted dump. > > If first kernel and kdump kernel command line are same in fadump and > boot environment is same, I think it is worth exploring the option > of saving kdump from kdump service when system boots back. That will > do away with any need to tweak initramfs. This needs larger memory to be reserved. We may need to make sure that kdump service is invoked at very early stage before other services kicks in which may start requesting for memory and run into OOM issue. If we can make sure that no other services are invoked when kdump service is saving dump then it will help to fix the size of reserved memory. The systemd infrastructure provides aggressive parallelization capabilities. I am not expert on systemd but will need to explore whether with systemd is this possible. > > It just requires more memory to be reserved so that system can boot > , mount root, run services and then save dump. I am hoping that firmware > can set aside enough memory to (1G?), to copy the memory contents and > boot the kernel in that 1G. It's the OS who reserves the memory from the available system RAM and provides it to firmware to use. This means production kernel will have set aside this memory for firmware to use. > > Also does fadump work well with new mmap() interface /proc/vmcore is > going to support. Patches are in -mm now. s390 zfcpdump is going to > face some issues and michael is working on resolving these. I suspect > that fadump will run into similar issues if firmware is copying memory > contents somewhere else. Will need to check this. > > Thanks > Vivek > ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <51AF40BB.5010703-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <51AF40BB.5010703-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> @ 2013-06-05 14:55 ` Vivek Goyal [not found] ` <20130605145504.GE16339-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 19+ messages in thread From: Vivek Goyal @ 2013-06-05 14:55 UTC (permalink / raw) To: Mahesh Jagannath Salgaonkar Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni, holzheu-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8 On Wed, Jun 05, 2013 at 07:14:27PM +0530, Mahesh Jagannath Salgaonkar wrote: > On 06/04/2013 07:51 PM, Vivek Goyal wrote: > > On Tue, Jun 04, 2013 at 10:14:03AM -0400, Vivek Goyal wrote: > >> On Tue, Jun 04, 2013 at 01:50:19PM +0530, Mahesh Jagannath Salgaonkar wrote: > >>> On 05/30/2013 07:53 PM, Vivek Goyal wrote: > >>>> On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: > >>>> > >>>> [..] > >>>>>> Sorry for restarting this discussion very late. I would like to know how > >>>>>> safe is to rebuild kernel's default (boot) initramfs for an already > >>>>>> installed kernel ? > >>>>>> > >>>>>> Also, Does dracut provides any of following mechanism? > >>>>>> a) Mechanism where dracut can detect what options were used during first > >>>>>> build for a given (exsisting) initramfs. (This mechanism may help one to > >>>>>> regenerate similar initramfs with additional dracut modules.) > >>>>> > >>>>> currently dracut only stores which modules were used to generate the image in > >>>>> usr/lib/dracut/modules.txt > >>>>> > >>>>> But yes, you are right. Would be nice to save all the options and have a > >>>>> mechanism to regenerate it with those. > >>>>> > >>>> > >>>> I am not sure how well it will work in the context of kdump. kdump > >>>> options and original options might conflict. kdump might drop some of > >>>> its own cmdline options in initramfs which might not make any sense > >>>> in regular boot. > >>>> > >>>> Kdump will specify additional mount points. IIUC, then we will try > >>>> to bring up those targets in regular boot too and mount at user > >>>> specified mount point. And later init services might get confused > >>>> when respective daemon tries to bring up same targets again. > >>> > >>> I think we should be able differentiate between regular boot and boot > >>> after crash by checking existence of '/proc/vmcore' file, and do the > >>> kdump specific configs/mount if this file exists. In regular boot we can > >>> mute the kdump code path. > >> > >> Also what about kernel command line? Kernel command line can be very > >> specific to kdump environment. It is atleast in x86. I am hoping that's > >> not the case with firmware assisted dump. > > > > If first kernel and kdump kernel command line are same in fadump and > > boot environment is same, I think it is worth exploring the option > > of saving kdump from kdump service when system boots back. That will > > do away with any need to tweak initramfs. > > This needs larger memory to be reserved. We may need to make sure that > kdump service is invoked at very early stage before other services kicks > in which may start requesting for memory and run into OOM issue. If we > can make sure that no other services are invoked when kdump service is > saving dump then it will help to fix the size of reserved memory. The > systemd infrastructure provides aggressive parallelization capabilities. > I am not expert on systemd but will need to explore whether with systemd > is this possible. I think we can create a new service for saving dump and make some of the systemd services dependent on start of this service. Can't integrate with existing kdump service as that one can start later and can take time in regenerating initramfs and holding back other services for that duration will not make sense. By creating a seprate service, delay will be introduced only when there s dump to be saved. Before this we will have bring up all networking and storage in the system so that we can save dump to intended target. I think same service can be used to save dump from inside initramfs too. (As systemd is now started inside initramfs too). Other appraoch is dumping through initramfs and if you do things conditionally (if dump is available), then any dracut parameter used by kdump will have to have a conditional version too. Something like mount= mount.kdump_only= mount.kdump_only will kick in only dump is available. Also we need to make sure that effects of all kdump_only options are undone before root is mounted or we create nice handover mechanism to systemd. I think there handover mechanism already between initramfs and systemd for complex storage targets like multipathd. I will let harald comment on what does he think of kdump_only variant options. Thanks Vivek ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <20130605145504.GE16339-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <20130605145504.GE16339-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2013-06-05 15:53 ` Vivek Goyal [not found] ` <20130605155316.GI16339-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 0 siblings, 1 reply; 19+ messages in thread From: Vivek Goyal @ 2013-06-05 15:53 UTC (permalink / raw) To: Mahesh Jagannath Salgaonkar Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni, holzheu-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8 On Wed, Jun 05, 2013 at 10:55:04AM -0400, Vivek Goyal wrote: [..] > > > If first kernel and kdump kernel command line are same in fadump and > > > boot environment is same, I think it is worth exploring the option > > > of saving kdump from kdump service when system boots back. That will > > > do away with any need to tweak initramfs. > > > > This needs larger memory to be reserved. We may need to make sure that > > kdump service is invoked at very early stage before other services kicks > > in which may start requesting for memory and run into OOM issue. If we > > can make sure that no other services are invoked when kdump service is > > saving dump then it will help to fix the size of reserved memory. The > > systemd infrastructure provides aggressive parallelization capabilities. > > I am not expert on systemd but will need to explore whether with systemd > > is this possible. > > I think we can create a new service for saving dump and make some of > the systemd services dependent on start of this service. > > Can't integrate with existing kdump service as that one can start > later and can take time in regenerating initramfs and holding back other > services for that duration will not make sense. > > By creating a seprate service, delay will be introduced only when there > s dump to be saved. Before this we will have bring up all networking > and storage in the system so that we can save dump to intended target. > > I think same service can be used to save dump from inside initramfs > too. (As systemd is now started inside initramfs too). > > Other appraoch is dumping through initramfs and if you do things > conditionally (if dump is available), then any dracut parameter > used by kdump will have to have a conditional version too. Something > like > > mount= > mount.kdump_only= > > mount.kdump_only will kick in only dump is available. Also we need to > make sure that effects of all kdump_only options are undone before > root is mounted or we create nice handover mechanism to systemd. I think > there handover mechanism already between initramfs and systemd for complex > storage targets like multipathd. > > I will let harald comment on what does he think of kdump_only variant > options. Thinking more about it, I think from kdump point of view, it is better to be able to dump from initrafs. We don't want different mechanism for different architecture which will result in more confusion and possibly code duplication. Thanks Vivek ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <20130605155316.GI16339-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <20130605155316.GI16339-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> @ 2013-06-06 8:34 ` Mahesh Jagannath Salgaonkar 0 siblings, 0 replies; 19+ messages in thread From: Mahesh Jagannath Salgaonkar @ 2013-06-06 8:34 UTC (permalink / raw) To: Vivek Goyal Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni, holzheu-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8 On 06/05/2013 09:23 PM, Vivek Goyal wrote: > On Wed, Jun 05, 2013 at 10:55:04AM -0400, Vivek Goyal wrote: > > [..] >>>> If first kernel and kdump kernel command line are same in fadump and >>>> boot environment is same, I think it is worth exploring the option >>>> of saving kdump from kdump service when system boots back. That will >>>> do away with any need to tweak initramfs. >>> >>> This needs larger memory to be reserved. We may need to make sure that >>> kdump service is invoked at very early stage before other services kicks >>> in which may start requesting for memory and run into OOM issue. If we >>> can make sure that no other services are invoked when kdump service is >>> saving dump then it will help to fix the size of reserved memory. The >>> systemd infrastructure provides aggressive parallelization capabilities. >>> I am not expert on systemd but will need to explore whether with systemd >>> is this possible. >> >> I think we can create a new service for saving dump and make some of >> the systemd services dependent on start of this service. >> >> Can't integrate with existing kdump service as that one can start >> later and can take time in regenerating initramfs and holding back other >> services for that duration will not make sense. >> >> By creating a seprate service, delay will be introduced only when there >> s dump to be saved. Before this we will have bring up all networking >> and storage in the system so that we can save dump to intended target. >> >> I think same service can be used to save dump from inside initramfs >> too. (As systemd is now started inside initramfs too). >> >> Other appraoch is dumping through initramfs and if you do things >> conditionally (if dump is available), then any dracut parameter >> used by kdump will have to have a conditional version too. Something >> like >> >> mount= >> mount.kdump_only= Agree. The kdump_only options can be used to store the kdump related required configs (e.g. lvm configs, network configs etc.) in a separate directory and add a pre-udev hook that can check if dump is available and move these configs to their original paths so that the normal initramfs flow will do its job as per configs. >> >> mount.kdump_only will kick in only dump is available. Also we need to >> make sure that effects of all kdump_only options are undone before >> root is mounted or we create nice handover mechanism to systemd. I think Agree. >> there handover mechanism already between initramfs and systemd for complex >> storage targets like multipathd. >> >> I will let harald comment on what does he think of kdump_only variant >> options. > > Thinking more about it, I think from kdump point of view, it is better > to be able to dump from initrafs. We don't want different mechanism > for different architecture which will result in more confusion and > possibly code duplication. > Agree. And if we rely on dracut to rebuild the initramfs (with original dracut arguments) we can be sure that we wont run into issues like unbootable system. Thanks, -Mahesh. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Firmware assisted dump support in dracut [not found] ` <20130604141403.GG4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-04 14:21 ` Vivek Goyal @ 2013-06-04 15:19 ` Michael Holzheu 2013-06-05 9:21 ` Mahesh Jagannath Salgaonkar 2 siblings, 0 replies; 19+ messages in thread From: Michael Holzheu @ 2013-06-04 15:19 UTC (permalink / raw) To: Vivek Goyal Cc: Mahesh Jagannath Salgaonkar, Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni On Tue, 4 Jun 2013 10:14:03 -0400 Vivek Goyal <vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote: > On Tue, Jun 04, 2013 at 01:50:19PM +0530, Mahesh Jagannath Salgaonkar > wrote: > > On 05/30/2013 07:53 PM, Vivek Goyal wrote: > > > On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: > > > > > > [..] > > >>> Sorry for restarting this discussion very late. I would like to > > >>> know how safe is to rebuild kernel's default (boot) initramfs > > >>> for an already installed kernel ? > > >>> > > >>> Also, Does dracut provides any of following mechanism? > > >>> a) Mechanism where dracut can detect what options were used > > >>> during first build for a given (exsisting) initramfs. (This > > >>> mechanism may help one to regenerate similar initramfs with > > >>> additional dracut modules.) > > >> > > >> currently dracut only stores which modules were used to generate > > >> the image in usr/lib/dracut/modules.txt > > >> > > >> But yes, you are right. Would be nice to save all the options > > >> and have a mechanism to regenerate it with those. > > >> > > > > > > I am not sure how well it will work in the context of kdump. > > > kdump options and original options might conflict. kdump might > > > drop some of its own cmdline options in initramfs which might not > > > make any sense in regular boot. > > > > > > Kdump will specify additional mount points. IIUC, then we will try > > > to bring up those targets in regular boot too and mount at user > > > specified mount point. And later init services might get confused > > > when respective daemon tries to bring up same targets again. > > > > I think we should be able differentiate between regular boot and > > boot after crash by checking existence of '/proc/vmcore' file, and > > do the kdump specific configs/mount if this file exists. In regular > > boot we can mute the kdump code path. > > Also what about kernel command line? Kernel command line can be very > specific to kdump environment. It is atleast in x86. I am hoping > that's not the case with firmware assisted dump. > > Also can you point me to some writeup of fadump again. I want to > refresh what fadump is doing and how firmware is assisting and how > does it make better than regular kdump. > > I suspect it is very similar to s390 firmware dump (zfcpdump). Does > s390 kdump or s390 zfcpdump reboot after saving dump or they have > mechanism to continue to boot and hotplug memory after saving dump. I > am wondering how do they do it. Hello Vivek, Currently s390 zfcpdump and kdump both reboot the production system after dump. But we also had discussed the option to write the dump in the background while the production system already starts. Best Regards, Michael ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Firmware assisted dump support in dracut [not found] ` <20130604141403.GG4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-04 14:21 ` Vivek Goyal 2013-06-04 15:19 ` Michael Holzheu @ 2013-06-05 9:21 ` Mahesh Jagannath Salgaonkar 2 siblings, 0 replies; 19+ messages in thread From: Mahesh Jagannath Salgaonkar @ 2013-06-05 9:21 UTC (permalink / raw) To: Vivek Goyal Cc: Harald Hoyer, Initramfs Dracut, Ananth Narayan, Steven F Best, Haren Myneni, holzheu-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8 On 06/04/2013 07:44 PM, Vivek Goyal wrote: > On Tue, Jun 04, 2013 at 01:50:19PM +0530, Mahesh Jagannath Salgaonkar wrote: >> On 05/30/2013 07:53 PM, Vivek Goyal wrote: >>> On Thu, May 30, 2013 at 08:49:07AM +0200, Harald Hoyer wrote: >>> >>> [..] >>>>> Sorry for restarting this discussion very late. I would like to know how >>>>> safe is to rebuild kernel's default (boot) initramfs for an already >>>>> installed kernel ? >>>>> >>>>> Also, Does dracut provides any of following mechanism? >>>>> a) Mechanism where dracut can detect what options were used during first >>>>> build for a given (exsisting) initramfs. (This mechanism may help one to >>>>> regenerate similar initramfs with additional dracut modules.) >>>> >>>> currently dracut only stores which modules were used to generate the image in >>>> usr/lib/dracut/modules.txt >>>> >>>> But yes, you are right. Would be nice to save all the options and have a >>>> mechanism to regenerate it with those. >>>> >>> >>> I am not sure how well it will work in the context of kdump. kdump >>> options and original options might conflict. kdump might drop some of >>> its own cmdline options in initramfs which might not make any sense >>> in regular boot. >>> >>> Kdump will specify additional mount points. IIUC, then we will try >>> to bring up those targets in regular boot too and mount at user >>> specified mount point. And later init services might get confused >>> when respective daemon tries to bring up same targets again. >> >> I think we should be able differentiate between regular boot and boot >> after crash by checking existence of '/proc/vmcore' file, and do the >> kdump specific configs/mount if this file exists. In regular boot we can >> mute the kdump code path > > Also what about kernel command line? Kernel command line can be very > specific to kdump environment. It is atleast in x86. I am hoping that's > not the case with firmware assisted dump. In fadump case, kernel command line is same as first kernel. > > Also can you point me to some writeup of fadump again. I want to refresh > what fadump is doing and how firmware is assisting and how does it make > better than regular kdump. It's at Documentation/powerpc/firmware-assisted-dump.txt in upstream kernel. > > I suspect it is very similar to s390 firmware dump (zfcpdump). Does > s390 kdump or s390 zfcpdump reboot after saving dump or they have > mechanism to continue to boot and hotplug memory after saving dump. I > am wondering how do they do it. > > CCing michael. > > Vivek > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Firmware assisted dump support in dracut [not found] ` <51A6F663.5030804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-05-30 14:23 ` Vivek Goyal @ 2013-06-04 8:34 ` Mahesh Jagannath Salgaonkar [not found] ` <51ADA6AB.5010503-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 1 sibling, 1 reply; 19+ messages in thread From: Mahesh Jagannath Salgaonkar @ 2013-06-04 8:34 UTC (permalink / raw) To: Harald Hoyer Cc: Initramfs Dracut, Vivek Goyal, Ananth Narayan, Steven F Best, Haren Myneni On 05/30/2013 12:19 PM, Harald Hoyer wrote: > On 05/30/2013 07:17 AM, Mahesh Jagannath Salgaonkar wrote: >> On 12/06/2012 11:51 AM, Mahesh Jagannath Salgaonkar wrote: >>> On 11/29/2012 03:44 PM, Harald Hoyer wrote: >>>> Am 28.11.2012 17:12, schrieb Mahesh J Salgaonkar: >>>>> Hi Harald, >>>>> >>>>> Last year I worked on adding Firmware assisted dump (fadump) support in >>>>> PowerLinux (ppc64). This feature is now accepted upstream Linux kernel and the >>>>> documentation is available at http://lwn.net/Articles/488132/ >>>>> >>>>> Like kdump, fadump also exports the memory dump through /proc/vmcore in ELF >>>>> format. This enables us to reuse the existing kdump infrastructure for dump >>>>> capture and filtering. However, unlike kdump, fadump does not use kexec-based >>>>> approach, instead it depends on Power firmware to preserve the memory dump and >>>>> reboot into new kernel. This is what happens in fadump after crash: >>>>> >>>>> 1. At the crash, kernel informs power firmware that kernel has crashed. >>>>> 2. Firmware takes the control and reboots the entire system preserving only the >>>>> memory (resets all other devices). >>>>> 3. The reboot follows the normal booting process (non-kexec). >>>>> 4. The boot loader loads the default kernel and initrd from /boot >>>>> >>>>> I am working on integrating fadump with existing kdump infrastructure. The >>>>> current kdump infrastructure builds a separate initrd (whenever there is a >>>>> change detected in kdump config file /etc/kdump.conf) which then gets loaded >>>>> into memory by kexec tool for use by kdump kernel. But, in the fadump approach, >>>>> the second kernel (after crash) always use the default (OS built) initramfs. >>>>> Hence, to support fadump, change is required to introduce dump capturing steps >>>>> in default initramfs itself. Hence the possible approaches I am looking into >>>>> are: >>>>> >>>>> 1. Rebuild the default initramfs every time when there is a change detected in >>>>> kdump config file (/etc/kdump.conf) >>>>> This approach would modify existing initramfs in place. I did work on this >>>>> approach by enhancing mkdumprd (tool from kexec-tools package) to extract >>>>> and rebuild default initramfs with dump capturing steps. After discussing >>>>> with Vivek Goyal, he suggested that better approach to add code to dracut >>>>> for rebuilding boot initramfs instead of enhancing mkdumprd. This means >>>>> introducing a new dracut module that will be responsible for introducing >>>>> dump capture steps inside the rebuilt initramfs by pulling required modules >>>>> e.g. ssh, nfs etc. depending on /etc/kdump.conf >>>>> >>>>> Now the question is whether it is possible for dracut to rebuild boot >>>>> initramfs in place? if Yes, is there any issues in rebuilding of boot >>>>> initramfs everytime when there is a change in /etc/kdump.conf? >>>> >>>> dracut is called by /sbin/new-kernel-package. dracut is not a service, which is >>>> run on every boot. So, no, dracut can't watch files and build an initramfs on >>>> itsself. >>>> >>> >>> Normally user would restart kdump service after changing kdump config >>> file. Idea is to invoke dracut through kdump service, as it already >>> watches and detect changes in kdump config files. >>> >>> I was wondering if dracut allow us to insert/install additional dracut >>> modules to already existing initramfs? OR does it rebuild initramfs all >>> over again? >> >> Sorry for restarting this discussion very late. I would like to know how >> safe is to rebuild kernel's default (boot) initramfs for an already >> installed kernel ? >> >> Also, Does dracut provides any of following mechanism? >> a) Mechanism where dracut can detect what options were used during first >> build for a given (exsisting) initramfs. (This mechanism may help one to >> regenerate similar initramfs with additional dracut modules.) > > currently dracut only stores which modules were used to generate the image in > usr/lib/dracut/modules.txt > > But yes, you are right. Would be nice to save all the options and have a > mechanism to regenerate it with those. > >> >> OR >> >> b) Add additional modules to an existing initramfs instead of >> regenerating again. Is this possible at all? > > You can always generate a second cpio image with additional files and either > concatenate both images or specify those two images in the bootloader. Is it possible to specify two images to bootloader for a single kernel entry? Does grub2 support it? > > OR > > unpack the initramfs, add the file, (maybe run depmod, if kernel driver added), > and repack. > ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <51ADA6AB.5010503-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>]
* Re: Firmware assisted dump support in dracut [not found] ` <51ADA6AB.5010503-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> @ 2013-06-04 18:42 ` Andrey Borzenkov 0 siblings, 0 replies; 19+ messages in thread From: Andrey Borzenkov @ 2013-06-04 18:42 UTC (permalink / raw) To: Mahesh Jagannath Salgaonkar Cc: Harald Hoyer, Initramfs Dracut, Vivek Goyal, Ananth Narayan, Steven F Best, Haren Myneni В Tue, 04 Jun 2013 14:04:51 +0530 Mahesh Jagannath Salgaonkar <mahesh-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> пишет: > On 05/30/2013 12:19 PM, Harald Hoyer wrote: > > On 05/30/2013 07:17 AM, Mahesh Jagannath Salgaonkar wrote: > >> On 12/06/2012 11:51 AM, Mahesh Jagannath Salgaonkar wrote: > >>> On 11/29/2012 03:44 PM, Harald Hoyer wrote: > >>>> Am 28.11.2012 17:12, schrieb Mahesh J Salgaonkar: > >>>>> Hi Harald, > >>>>> > >>>>> Last year I worked on adding Firmware assisted dump (fadump) support in > >>>>> PowerLinux (ppc64). This feature is now accepted upstream Linux kernel and the > >>>>> documentation is available at http://lwn.net/Articles/488132/ > >>>>> > >>>>> Like kdump, fadump also exports the memory dump through /proc/vmcore in ELF > >>>>> format. This enables us to reuse the existing kdump infrastructure for dump > >>>>> capture and filtering. However, unlike kdump, fadump does not use kexec-based > >>>>> approach, instead it depends on Power firmware to preserve the memory dump and > >>>>> reboot into new kernel. This is what happens in fadump after crash: > >>>>> > >>>>> 1. At the crash, kernel informs power firmware that kernel has crashed. > >>>>> 2. Firmware takes the control and reboots the entire system preserving only the > >>>>> memory (resets all other devices). > >>>>> 3. The reboot follows the normal booting process (non-kexec). > >>>>> 4. The boot loader loads the default kernel and initrd from /boot > >>>>> > >>>>> I am working on integrating fadump with existing kdump infrastructure. The > >>>>> current kdump infrastructure builds a separate initrd (whenever there is a > >>>>> change detected in kdump config file /etc/kdump.conf) which then gets loaded > >>>>> into memory by kexec tool for use by kdump kernel. But, in the fadump approach, > >>>>> the second kernel (after crash) always use the default (OS built) initramfs. > >>>>> Hence, to support fadump, change is required to introduce dump capturing steps > >>>>> in default initramfs itself. Hence the possible approaches I am looking into > >>>>> are: > >>>>> > >>>>> 1. Rebuild the default initramfs every time when there is a change detected in > >>>>> kdump config file (/etc/kdump.conf) > >>>>> This approach would modify existing initramfs in place. I did work on this > >>>>> approach by enhancing mkdumprd (tool from kexec-tools package) to extract > >>>>> and rebuild default initramfs with dump capturing steps. After discussing > >>>>> with Vivek Goyal, he suggested that better approach to add code to dracut > >>>>> for rebuilding boot initramfs instead of enhancing mkdumprd. This means > >>>>> introducing a new dracut module that will be responsible for introducing > >>>>> dump capture steps inside the rebuilt initramfs by pulling required modules > >>>>> e.g. ssh, nfs etc. depending on /etc/kdump.conf > >>>>> > >>>>> Now the question is whether it is possible for dracut to rebuild boot > >>>>> initramfs in place? if Yes, is there any issues in rebuilding of boot > >>>>> initramfs everytime when there is a change in /etc/kdump.conf? > >>>> > >>>> dracut is called by /sbin/new-kernel-package. dracut is not a service, which is > >>>> run on every boot. So, no, dracut can't watch files and build an initramfs on > >>>> itsself. > >>>> > >>> > >>> Normally user would restart kdump service after changing kdump config > >>> file. Idea is to invoke dracut through kdump service, as it already > >>> watches and detect changes in kdump config files. > >>> > >>> I was wondering if dracut allow us to insert/install additional dracut > >>> modules to already existing initramfs? OR does it rebuild initramfs all > >>> over again? > >> > >> Sorry for restarting this discussion very late. I would like to know how > >> safe is to rebuild kernel's default (boot) initramfs for an already > >> installed kernel ? > >> > >> Also, Does dracut provides any of following mechanism? > >> a) Mechanism where dracut can detect what options were used during first > >> build for a given (exsisting) initramfs. (This mechanism may help one to > >> regenerate similar initramfs with additional dracut modules.) > > > > currently dracut only stores which modules were used to generate the image in > > usr/lib/dracut/modules.txt > > > > But yes, you are right. Would be nice to save all the options and have a > > mechanism to regenerate it with those. > > > >> > >> OR > >> > >> b) Add additional modules to an existing initramfs instead of > >> regenerating again. Is this possible at all? > > > > You can always generate a second cpio image with additional files and either > > concatenate both images or specify those two images in the bootloader. > > Is it possible to specify two images to bootloader for a single kernel > entry? Does grub2 support it? > Yes, it does. You can use initrd file1 file2 ... And it will concatenate files. It aligns files on 4 bytes boundary, not sure whether it can be a problem. > > > > OR > > > > unpack the initramfs, add the file, (maybe run depmod, if kernel driver added), > > and repack. > > > > -- > To unsubscribe from this list: send the line "unsubscribe initramfs" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2013-06-06 8:34 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-11-28 16:12 Firmware assisted dump support in dracut Mahesh J Salgaonkar [not found] ` <20121128161234.GA8991-xthvdsQ13ZrQT0dZR+AlfA@public.gmane.org> 2012-11-29 10:14 ` Harald Hoyer [not found] ` <50B73598.804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2012-11-30 15:31 ` Vivek Goyal 2012-12-06 6:21 ` Mahesh Jagannath Salgaonkar [not found] ` <50C03971.4050104-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 2013-05-30 5:17 ` Mahesh Jagannath Salgaonkar [not found] ` <51A6E0E4.1000502-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 2013-05-30 6:49 ` Harald Hoyer [not found] ` <51A6F663.5030804-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-05-30 14:23 ` Vivek Goyal [not found] ` <20130530142318.GC2864-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-04 8:20 ` Mahesh Jagannath Salgaonkar [not found] ` <51ADA343.2070100-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 2013-06-04 13:57 ` Vivek Goyal 2013-06-04 14:14 ` Vivek Goyal [not found] ` <20130604141403.GG4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-04 14:21 ` Vivek Goyal [not found] ` <20130604142123.GH4799-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-05 13:44 ` Mahesh Jagannath Salgaonkar [not found] ` <51AF40BB.5010703-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 2013-06-05 14:55 ` Vivek Goyal [not found] ` <20130605145504.GE16339-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-05 15:53 ` Vivek Goyal [not found] ` <20130605155316.GI16339-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> 2013-06-06 8:34 ` Mahesh Jagannath Salgaonkar 2013-06-04 15:19 ` Michael Holzheu 2013-06-05 9:21 ` Mahesh Jagannath Salgaonkar 2013-06-04 8:34 ` Mahesh Jagannath Salgaonkar [not found] ` <51ADA6AB.5010503-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> 2013-06-04 18:42 ` Andrey Borzenkov
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.