[PATCH] scripts: add extract-vmlinux

[PATCH] scripts: add extract-vmlinux

[Corentin Chary]

This script can be used to extract vmlinux from a compressed
kernel image (bzImage, etc..). It's inspired from (a subset of)
extract-ikconfig.

I don't know if this scripts really have this place in the kernel
tree, but it's something a lot of people have been looking for (mainly
people with xen < 4 that doesn't support bzImages at all).

Signed-off-by: Corentin Chary <corentin.chary@gmail.com>
---
 scripts/extract-vmlinux |   60 +++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 60 insertions(+), 0 deletions(-)
 create mode 100755 scripts/extract-vmlinux

diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
new file mode 100755
index 0000000..444f404
--- /dev/null
+++ b/scripts/extract-vmlinux
@@ -0,0 +1,60 @@
+#!/bin/sh
+# ----------------------------------------------------------------------
+# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
+#
+# The obscure use of the "tr" filter is to work around older versions of
+# "grep" that report the byte offset of the line instead of the pattern.
+#
+# Inspired from extract-ikconfig
+# (c) 2009,2010 Dick Streefland <dick@streefland.net>
+#
+# (c) 2011      Corentin Chary <corentin.chary@gmail.com>
+# Licensed under the terms of the GNU General Public License.
+# ----------------------------------------------------------------------
+
+check_elf()
+{
+	# Use readelf to check if it's a valid ELF
+	# There is probably a better way to do that...
+	readelf -h $1 > /dev/null 2>&1 || return 1
+
+	cat $1
+	exit 0
+}
+
+try_decompress()
+{
+	for	pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
+	do
+		pos=${pos%%:*}
+		tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
+		check_elf $tmp
+	done
+}
+
+# Check invocation:
+me=${0##*/}
+img=$1
+if	[ $# -ne 1 -o ! -s "$img" ]
+then
+	echo "Usage: $me <kernel-image>" >&2
+	exit 2
+fi
+
+# Prepare temp files:
+tmp=$(mktemp /tmp/vmlinux-XXX)
+trap "rm -f $tmp" 0
+
+# Initial attempt for uncompressed images or objects:
+check_elf $img
+
+# That didn't work, so retry after decompression.
+try_decompress '\037\213\010' xy    gunzip
+try_decompress '\3757zXZ\000' abcde unxz
+try_decompress 'BZh'          xy    bunzip2
+try_decompress '\135\0\0\0'   xxx   unlzma
+try_decompress '\211\114\132' xy    'lzop -d'
+
+# Bail out:
+echo "$me: Cannot find vmlinux." >&2
+exit 1
-- 
1.7.3.4

Re: [PATCH] scripts: add extract-vmlinux

[Michal Marek]

On 4.8.2011 15:52, Corentin Chary wrote:
> This script can be used to extract vmlinux from a compressed
> kernel image (bzImage, etc..). It's inspired from (a subset of)
> extract-ikconfig.

Nice.


> I don't know if this scripts really have this place in the kernel
> tree, but it's something a lot of people have been looking for (mainly
> people with xen < 4 that doesn't support bzImages at all).

I think it makes sense to have it in the tree.


> Signed-off-by: Corentin Chary <corentin.chary@gmail.com>
> ---
>  scripts/extract-vmlinux |   60 +++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 60 insertions(+), 0 deletions(-)
>  create mode 100755 scripts/extract-vmlinux
> 
> diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
> new file mode 100755
> index 0000000..444f404
> --- /dev/null
> +++ b/scripts/extract-vmlinux
> @@ -0,0 +1,60 @@
> +#!/bin/sh
> +# ----------------------------------------------------------------------
> +# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
> +#
> +# The obscure use of the "tr" filter is to work around older versions of
> +# "grep" that report the byte offset of the line instead of the pattern.
> +#
> +# Inspired from extract-ikconfig
> +# (c) 2009,2010 Dick Streefland <dick@streefland.net>
> +#
> +# (c) 2011      Corentin Chary <corentin.chary@gmail.com>
> +# Licensed under the terms of the GNU General Public License.

I guess you want to use the version 2 of the GPL?

Michal

Re: [PATCH] scripts: add extract-vmlinux

[Corentin Chary]

2011/8/10 Michal Marek <mmarek@suse.cz>:
> On 4.8.2011 15:52, Corentin Chary wrote:
>> This script can be used to extract vmlinux from a compressed
>> kernel image (bzImage, etc..). It's inspired from (a subset of)
>> extract-ikconfig.
>
> Nice.
>
>
>> I don't know if this scripts really have this place in the kernel
>> tree, but it's something a lot of people have been looking for (mainly
>> people with xen < 4 that doesn't support bzImages at all).
>
> I think it makes sense to have it in the tree.
>
>
>> Signed-off-by: Corentin Chary <corentin.chary@gmail.com>
>> ---
>>  scripts/extract-vmlinux |   60 +++++++++++++++++++++++++++++++++++++++++++++++
>>  1 files changed, 60 insertions(+), 0 deletions(-)
>>  create mode 100755 scripts/extract-vmlinux
>>
>> diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
>> new file mode 100755
>> index 0000000..444f404
>> --- /dev/null
>> +++ b/scripts/extract-vmlinux
>> @@ -0,0 +1,60 @@
>> +#!/bin/sh
>> +# ----------------------------------------------------------------------
>> +# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
>> +#
>> +# The obscure use of the "tr" filter is to work around older versions of
>> +# "grep" that report the byte offset of the line instead of the pattern.
>> +#
>> +# Inspired from extract-ikconfig
>> +# (c) 2009,2010 Dick Streefland <dick@streefland.net>
>> +#
>> +# (c) 2011      Corentin Chary <corentin.chary@gmail.com>
>> +# Licensed under the terms of the GNU General Public License.
>
> I guess you want to use the version 2 of the GPL?

Since it's heavily based on extract-ikconfig I'm not sure I can change
the license.
CCing previous authors of extract-ikconfig to see if they are ok with that.

Thanks,

-- 
Corentin Chary
http://xf.iksaif.net

Re: [PATCH] scripts: add extract-vmlinux

[Dick Streefland]

On Thursday 2011-08-11 09:21, Corentin Chary wrote:
| 2011/8/10 Michal Marek <mmarek@suse.cz>:
| > I guess you want to use the version 2 of the GPL?
| 
| Since it's heavily based on extract-ikconfig I'm not sure I can change
| the license.
| CCing previous authors of extract-ikconfig to see if they are ok with that.

GPL v2 is fine with me. I think that is the default version for the
kernel anyway.

-- 
Dick

Re: [PATCH] scripts: add extract-vmlinux

[Américo Wang]

On Thu, Aug 11, 2011 at 3:21 PM, Corentin Chary
<corentin.chary@gmail.com> wrote:
> 2011/8/10 Michal Marek <mmarek@suse.cz>:
>>
>> I guess you want to use the version 2 of the GPL?
>
> Since it's heavily based on extract-ikconfig I'm not sure I can change
> the license.
> CCing previous authors of extract-ikconfig to see if they are ok with that.
>

I don't remember I wrote any code of extract-ikconfig, I did review
the original patch. I am fine with this patch too.

Thanks!

[PATCH] scripts: add extract-vmlinux

[Corentin Chary]

This script can be used to extract vmlinux from a compressed
kernel image (bzImage, etc..). It's inspired from (a subset of)
extract-ikconfig.

It's something a lot of people have been looking for (mainly
people with xen < 4 that doesn't support bzImages at all).

Signed-off-by: Corentin Chary <corentin.chary@gmail.com>
---
 scripts/extract-vmlinux |   60 +++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 60 insertions(+), 0 deletions(-)
 create mode 100755 scripts/extract-vmlinux

diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
new file mode 100755
index 0000000..615d31a
--- /dev/null
+++ b/scripts/extract-vmlinux
@@ -0,0 +1,60 @@
+#!/bin/sh
+# ----------------------------------------------------------------------
+# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
+#
+# The obscure use of the "tr" filter is to work around older versions of
+# "grep" that report the byte offset of the line instead of the pattern.
+#
+# Inspired from extract-ikconfig
+# (c) 2009,2010 Dick Streefland <dick@streefland.net>
+#
+# (c) 2011      Corentin Chary <corentin.chary@gmail.com>
+#
+# Licensed under the GNU General Public License, version 2 (GPLv2).
+# ----------------------------------------------------------------------
+
+check_elf()
+{
+	# Use readelf to check if it's a valid ELF
+	# There is probably a better way to do that...
+	readelf -h $1 > /dev/null 2>&1 || return 1
+
+	cat $1
+	exit 0
+}
+
+try_decompress()
+{
+	for	pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
+	do
+		pos=${pos%%:*}
+		tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
+		check_elf $tmp
+	done
+}
+
+# Check invocation:
+me=${0##*/}
+img=$1
+if	[ $# -ne 1 -o ! -s "$img" ]
+then
+	echo "Usage: $me <kernel-image>" >&2
+	exit 2
+fi
+
+# Prepare temp files:
+tmp=$(mktemp /tmp/vmlinux-XXX)
+trap "rm -f $tmp" 0
+
+# Initial attempt for uncompressed images or objects:
+check_elf $img
+
+# That didn't work, so retry after decompression.
+try_decompress '\037\213\010' xy    gunzip
+try_decompress '\3757zXZ\000' abcde unxz
+try_decompress 'BZh'          xy    bunzip2
+try_decompress '\135\0\0\0'   xxx   unlzma
+try_decompress '\211\114\132' xy    'lzop -d'
+
+# Bail out:
+echo "$me: Cannot find vmlinux." >&2
-- 
1.7.3.4

Re: [PATCH] scripts: add extract-vmlinux

[Paul Bolle]

On Thu, 2011-08-11 at 10:53 +0200, Corentin Chary wrote:
> This script can be used to extract vmlinux from a compressed
> kernel image (bzImage, etc..). It's inspired from (a subset of)
> extract-ikconfig.

This is more specific than what this scripts actually does, isn't it? At
least when I tried to read this script my impression is that it does two
things:
- check whether the input file is a valid ELF file;
- if not; try to find a compressed ELF file somewhere in the input file.

There's no checking whether the input file is a kernel image and there's
no checking whether the found ELF file actually is was a, well, vmlinux.
Both checks are perhaps far from trivial. Anyhow, if that's correct this
should be made more clear. Perhaps the script should even be called
something like extract-elf.

> It's something a lot of people have been looking for (mainly
> people with xen < 4 that doesn't support bzImages at all).
> 
> Signed-off-by: Corentin Chary <corentin.chary@gmail.com>
> [...]
> +try_decompress()
> +{
> +	for	pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
> +	do
> +		pos=${pos%%:*}
> +		tail -c+$pos "$img" | $3 > $tmp 2> /dev/null

Perhaps a few comments on the above lines would be nice. Without those
comments I must guess you're finding compressed data somewhere in the
input file. It also seems you're looping through the entire input file.
Or are (sequences of) commands like the above considered obvious?

> +		check_elf $tmp
> +	done
> +}
> +
> [...]
> +# Initial attempt for uncompressed images or objects:
> +check_elf $img
> +
> +# That didn't work, so retry after decompression.
> +try_decompress '\037\213\010' xy    gunzip
> +try_decompress '\3757zXZ\000' abcde unxz
> +try_decompress 'BZh'          xy    bunzip2
> +try_decompress '\135\0\0\0'   xxx   unlzma
> +try_decompress '\211\114\132' xy    'lzop -d'

Perhaps you could first test whether these commands are available before
running try_decompress() with them?


Paul Bolle

Re: [PATCH] scripts: add extract-vmlinux

[Corentin Chary]

On Thu, Aug 11, 2011 at 1:28 PM, Paul Bolle <pebolle@tiscali.nl> wrote:
> On Thu, 2011-08-11 at 10:53 +0200, Corentin Chary wrote:
>> This script can be used to extract vmlinux from a compressed
>> kernel image (bzImage, etc..). It's inspired from (a subset of)
>> extract-ikconfig.
>
> This is more specific than what this scripts actually does, isn't it? At
> least when I tried to read this script my impression is that it does two
> things:
> - check whether the input file is a valid ELF file;
> - if not; try to find a compressed ELF file somewhere in the input file.
>
> There's no checking whether the input file is a kernel image and there's
> no checking whether the found ELF file actually is was a, well, vmlinux.
> Both checks are perhaps far from trivial. Anyhow, if that's correct this
> should be made more clear. Perhaps the script should even be called
> something like extract-elf.

Yep, I didn't found a quick way to check that the file is a valid
vmlinux and the function check_elf() has a configusing name.

Is that one better ?

check_vmlinux()
{
        # Use readelf to check if it's a valid ELF
        # TODO: find a better to way to check that it's really vmlinux
        #       and not just another elf
        readelf -h $1 > /dev/null 2>&1 || return 1

        cat $1
        exit 0
}

If you have a good way to check that an elf is a valid vmlinux, I'd be
happy to implement it.

>> It's something a lot of people have been looking for (mainly
>> people with xen < 4 that doesn't support bzImages at all).
>>
>> Signed-off-by: Corentin Chary <corentin.chary@gmail.com>
>> [...]
>> +try_decompress()
>> +{
>> +     for     pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
>> +     do
>> +             pos=${pos%%:*}
>> +             tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
>
> Perhaps a few comments on the above lines would be nice. Without those
> comments I must guess you're finding compressed data somewhere in the
> input file. It also seems you're looping through the entire input file.
> Or are (sequences of) commands like the above considered obvious?

It's copied from extract-ikconfig, there was no comment in it, so I
assumed it's obvious.
Basically it uses brute force (tm) to find well known headers and try
to decompress from here.

>> +             check_elf $tmp
>> +     done
>> +}
>> +
>> [...]
>> +# Initial attempt for uncompressed images or objects:
>> +check_elf $img
>> +
>> +# That didn't work, so retry after decompression.
>> +try_decompress '\037\213\010' xy    gunzip
>> +try_decompress '\3757zXZ\000' abcde unxz
>> +try_decompress 'BZh'          xy    bunzip2
>> +try_decompress '\135\0\0\0'   xxx   unlzma
>> +try_decompress '\211\114\132' xy    'lzop -d'
>
> Perhaps you could first test whether these commands are available before
> running try_decompress() with them?

Same as before, if it's ok for extract-ikconfig not to check the
command, then it's ok for me.
If it's not, then we should patch both.

Personnaly I think that the script is small enought so that someone
who try to use it and can't make it work will understand what's
happening.

Note that extract-ikconfig is used in the build system and is far more
"critical" that extract-vmlinux, and nobody complained before :).

-- 
Corentin Chary
http://xf.iksaif.net

Re: [PATCH] scripts: add extract-vmlinux

[Michal Marek]

On 11.8.2011 15:08, Corentin Chary wrote:
> On Thu, Aug 11, 2011 at 1:28 PM, Paul Bolle <pebolle@tiscali.nl> wrote:
>> On Thu, 2011-08-11 at 10:53 +0200, Corentin Chary wrote:
>>> +try_decompress()
>>> +{
>>> +     for     pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
>>> +     do
>>> +             pos=${pos%%:*}
>>> +             tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
>>
>> Perhaps a few comments on the above lines would be nice. Without those
>> comments I must guess you're finding compressed data somewhere in the
>> input file. It also seems you're looping through the entire input file.
>> Or are (sequences of) commands like the above considered obvious?
> 
> It's copied from extract-ikconfig, there was no comment in it, so I
> assumed it's obvious.
> Basically it uses brute force (tm) to find well known headers and try
> to decompress from here.

And explanation of the "tr" trick would not hurt though. It replaces the
binary magic with an ascii sequence on a new line (for older grep
versions to get it right) and tries all occurrences of that sequence, am
I right? But I wouldn't call it "obvious" :).


>>> +# That didn't work, so retry after decompression.
>>> +try_decompress '\037\213\010' xy    gunzip
>>> +try_decompress '\3757zXZ\000' abcde unxz
>>> +try_decompress 'BZh'          xy    bunzip2
>>> +try_decompress '\135\0\0\0'   xxx   unlzma
>>> +try_decompress '\211\114\132' xy    'lzop -d'
>>
>> Perhaps you could first test whether these commands are available before
>> running try_decompress() with them?
> 
> Same as before, if it's ok for extract-ikconfig not to check the
> command, then it's ok for me.
> If it's not, then we should patch both.

Also, if you have lzma to build a lzma-compressed kernel, then you very
likely also have unlzma.

Michal

Re: [PATCH] scripts: add extract-vmlinux

[Corentin Chary]

On Thu, Aug 11, 2011 at 3:33 PM, Michal Marek <mmarek@suse.cz> wrote:
> On 11.8.2011 15:08, Corentin Chary wrote:
>> On Thu, Aug 11, 2011 at 1:28 PM, Paul Bolle <pebolle@tiscali.nl> wrote:
>>> On Thu, 2011-08-11 at 10:53 +0200, Corentin Chary wrote:
>>>> +try_decompress()
>>>> +{
>>>> +     for     pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
>>>> +     do
>>>> +             pos=${pos%%:*}
>>>> +             tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
>>>
>>> Perhaps a few comments on the above lines would be nice. Without those
>>> comments I must guess you're finding compressed data somewhere in the
>>> input file. It also seems you're looping through the entire input file.
>>> Or are (sequences of) commands like the above considered obvious?
>>
>> It's copied from extract-ikconfig, there was no comment in it, so I
>> assumed it's obvious.
>> Basically it uses brute force (tm) to find well known headers and try
>> to decompress from here.
>
> And explanation of the "tr" trick would not hurt though. It replaces the
> binary magic with an ascii sequence on a new line (for older grep
> versions to get it right) and tries all occurrences of that sequence, am
> I right? But I wouldn't call it "obvious" :).

It's that commented in the header of both scripts ? (I can move down
the command if wanted)

>
>>>> +# That didn't work, so retry after decompression.
>>>> +try_decompress '\037\213\010' xy    gunzip
>>>> +try_decompress '\3757zXZ\000' abcde unxz
>>>> +try_decompress 'BZh'          xy    bunzip2
>>>> +try_decompress '\135\0\0\0'   xxx   unlzma
>>>> +try_decompress '\211\114\132' xy    'lzop -d'
>>>
>>> Perhaps you could first test whether these commands are available before
>>> running try_decompress() with them?
>>
>> Same as before, if it's ok for extract-ikconfig not to check the
>> command, then it's ok for me.
>> If it's not, then we should patch both.
>
> Also, if you have lzma to build a lzma-compressed kernel, then you very
> likely also have unlzma.
>
> Michal
>



-- 
Corentin Chary
http://xf.iksaif.net

Re: [PATCH] scripts: add extract-vmlinux

[Corentin Chary]

> It's that commented in the header of both scripts ? (I can move down
> the command if wanted)

s/It's/Isn't/


-- 
Corentin Chary
http://xf.iksaif.net

[PATCH v3] scripts: add extract-vmlinux

[Corentin Chary]

This script can be used to extract vmlinux from a compressed
kernel image (bzImage, etc..). It's inspired from (a subset of)
extract-ikconfig.

It's something a lot of people have been looking for (mainly
people with xen < 4 that doesn't support bzImages at all).

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
---

Since v1:
* update license

Since v2:
* add some comments

 scripts/extract-vmlinux |   62 +++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 62 insertions(+), 0 deletions(-)
 create mode 100755 scripts/extract-vmlinux

diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
new file mode 100755
index 0000000..4ab8517
--- /dev/null
+++ b/scripts/extract-vmlinux
@@ -0,0 +1,62 @@
+#!/bin/sh
+# ----------------------------------------------------------------------
+# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
+#
+# Inspired from extract-ikconfig
+# (c) 2009,2010 Dick Streefland <dick@streefland.net>
+#
+# (c) 2011      Corentin Chary <corentin.chary@gmail.com>
+#
+# Licensed under the GNU General Public License, version 2 (GPLv2).
+# ----------------------------------------------------------------------
+
+check_vmlinux()
+{
+	# Use readelf to check if it's a valid ELF
+	# TODO: find a better to way to check that it's really vmlinux
+	#       and not just an elf
+	readelf -h $1 > /dev/null 2>&1 || return 1
+
+	cat $1
+	exit 0
+}
+
+try_decompress()
+{
+	# The obscure use of the "tr" filter is to work around older versions of
+	# "grep" that report the byte offset of the line instead of the pattern.
+
+	# Try to find the header ($1) and decompress from here
+	for	pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
+	do
+		pos=${pos%%:*}
+		tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
+		check_vmlinux $tmp
+	done
+}
+
+# Check invocation:
+me=${0##*/}
+img=$1
+if	[ $# -ne 1 -o ! -s "$img" ]
+then
+	echo "Usage: $me <kernel-image>" >&2
+	exit 2
+fi
+
+# Prepare temp files:
+tmp=$(mktemp /tmp/vmlinux-XXX)
+trap "rm -f $tmp" 0
+
+# Initial attempt for uncompressed images or objects:
+check_vmlinux $img
+
+# That didn't work, so retry after decompression.
+try_decompress '\037\213\010' xy    gunzip
+try_decompress '\3757zXZ\000' abcde unxz
+try_decompress 'BZh'          xy    bunzip2
+try_decompress '\135\0\0\0'   xxx   unlzma
+try_decompress '\211\114\132' xy    'lzop -d'
+
+# Bail out:
+echo "$me: Cannot find vmlinux." >&2
-- 
1.7.3.4

Re: [PATCH v3] scripts: add extract-vmlinux

[Américo Wang]

Ping...

I don't know any way to check vmlinux, we can only check if it is ELF,
so this patch looks good.

Michal?

On Tue, Aug 16, 2011 at 4:46 PM, Corentin Chary <corentincj@iksaif.net> wrote:
> This script can be used to extract vmlinux from a compressed
> kernel image (bzImage, etc..). It's inspired from (a subset of)
> extract-ikconfig.
>
> It's something a lot of people have been looking for (mainly
> people with xen < 4 that doesn't support bzImages at all).
>
> Signed-off-by: Corentin Chary <corentincj@iksaif.net>
> ---
>
> Since v1:
> * update license
>
> Since v2:
> * add some comments
>
>  scripts/extract-vmlinux |   62 +++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 62 insertions(+), 0 deletions(-)
>  create mode 100755 scripts/extract-vmlinux
>
> diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
> new file mode 100755
> index 0000000..4ab8517
> --- /dev/null
> +++ b/scripts/extract-vmlinux
> @@ -0,0 +1,62 @@
> +#!/bin/sh
> +# ----------------------------------------------------------------------
> +# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
> +#
> +# Inspired from extract-ikconfig
> +# (c) 2009,2010 Dick Streefland <dick@streefland.net>
> +#
> +# (c) 2011      Corentin Chary <corentin.chary@gmail.com>
> +#
> +# Licensed under the GNU General Public License, version 2 (GPLv2).
> +# ----------------------------------------------------------------------
> +
> +check_vmlinux()
> +{
> +       # Use readelf to check if it's a valid ELF
> +       # TODO: find a better to way to check that it's really vmlinux
> +       #       and not just an elf
> +       readelf -h $1 > /dev/null 2>&1 || return 1
> +
> +       cat $1
> +       exit 0
> +}
> +
> +try_decompress()
> +{
> +       # The obscure use of the "tr" filter is to work around older versions of
> +       # "grep" that report the byte offset of the line instead of the pattern.
> +
> +       # Try to find the header ($1) and decompress from here
> +       for     pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
> +       do
> +               pos=${pos%%:*}
> +               tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
> +               check_vmlinux $tmp
> +       done
> +}
> +
> +# Check invocation:
> +me=${0##*/}
> +img=$1
> +if     [ $# -ne 1 -o ! -s "$img" ]
> +then
> +       echo "Usage: $me <kernel-image>" >&2
> +       exit 2
> +fi
> +
> +# Prepare temp files:
> +tmp=$(mktemp /tmp/vmlinux-XXX)
> +trap "rm -f $tmp" 0
> +
> +# Initial attempt for uncompressed images or objects:
> +check_vmlinux $img
> +
> +# That didn't work, so retry after decompression.
> +try_decompress '\037\213\010' xy    gunzip
> +try_decompress '\3757zXZ\000' abcde unxz
> +try_decompress 'BZh'          xy    bunzip2
> +try_decompress '\135\0\0\0'   xxx   unlzma
> +try_decompress '\211\114\132' xy    'lzop -d'
> +
> +# Bail out:
> +echo "$me: Cannot find vmlinux." >&2
> --
> 1.7.3.4
>
>

Re: [PATCH v3] scripts: add extract-vmlinux

[Corentin Chary]

On Thu, Aug 18, 2011 at 12:09 PM, Américo Wang <xiyou.wangcong@gmail.com> wrote:
> Ping...
>
> I don't know any way to check vmlinux, we can only check if it is ELF,
> so this patch looks good.
>
> Michal?

Ping ?

-- 
Corentin Chary
http://xf.iksaif.net

Re: [PATCH v3] scripts: add extract-vmlinux

[Ian Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 886 bytes --]

On Tue, 2011-08-16 at 10:46 +0200, Corentin Chary wrote:
> This script can be used to extract vmlinux from a compressed
> kernel image (bzImage, etc..). It's inspired from (a subset of)
> extract-ikconfig.

FWIW I wrote the attached way back when, it uses the payload_* fields in
the bzImage to find the payload rather than scanning (these are present
in bzImages from somewhere in the mid 2.6.2x range). I'd be happy to
license it under the GPLv2 if that is helpful.

> It's something a lot of people have been looking for (mainly
> people with xen < 4 that doesn't support bzImages at all).

xen 3.4 does, doesn't it (at least the tip of 3.4-testing.hg does)? And
I thought e.g. RHEL5 (which uses an older base version) had it
backported. Possibly what is missing is support for all the various
compression options.

Ian.
-- 
Ian Campbell


How do I get HOME?

[-- Attachment #1.2: bzexplode.c --]
[-- Type: text/x-csrc, Size: 1309 bytes --]

#include <stdio.h>
#include <fcntl.h>
#include <stdint.h>
#include <unistd.h>

#include <inttypes.h>

#include <err.h>

#include <sys/types.h>
#include <sys/mman.h>
#include <sys/stat.h>

int main(int argc, char **argv)
{
	int fd;
	struct stat sb;
	void *p;
	uint8_t *hdr;
	int setup_sectors;
	uint32_t compressed_payload_offset;
	uint32_t compressed_payload_length;

	if (argc != 2)
		errx(1, "usage: bzexplode <bzImage>");

	fd = open(argv[1], O_RDONLY);
	if (fd < 0)
		err(1, "open");

	if (fstat(fd, &sb) < 0)
		err(1, "fstat");

	p = mmap(0, sb.st_size, PROT_READ, MAP_SHARED, fd, 0);
	if (p == MAP_FAILED)
		err(1, "mmap");

	hdr = p;
	setup_sectors = hdr[0x1f1];
	compressed_payload_offset = *(uint32_t*)&hdr[0x248];

	fprintf(stderr, "setup sectors %d\n", setup_sectors);

	compressed_payload_offset += (setup_sectors+1) * 512;

	//compressed_payload_length = *(uint32_t*)(p + compressed_payload_offset - 4);
	compressed_payload_length = *(uint32_t*)&hdr[0x24c];

	fprintf(stderr, "compressed_payload_offset %"PRIx32" (abs)\n",
		compressed_payload_offset);
	fprintf(stderr, "compressed_payload_length %"PRIx32"\n",
		compressed_payload_length);

	write(1,
	      p + compressed_payload_offset,
	      compressed_payload_length);
	return 0;
}

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v3] scripts: add extract-vmlinux

[Corentin Chary]

On Fri, Aug 19, 2011 at 8:51 AM, Ian Campbell <ijc@hellion.org.uk> wrote:
> On Tue, 2011-08-16 at 10:46 +0200, Corentin Chary wrote:
>> This script can be used to extract vmlinux from a compressed
>> kernel image (bzImage, etc..). It's inspired from (a subset of)
>> extract-ikconfig.
>
> FWIW I wrote the attached way back when, it uses the payload_* fields in
> the bzImage to find the payload rather than scanning (these are present
> in bzImages from somewhere in the mid 2.6.2x range). I'd be happy to
> license it under the GPLv2 if that is helpful.

I think it's good to know that this code exist, but it's better to
keep the current code that's both simple and stupid because I'd really
want to avoid implemeting a specific method for bzImages if the
generic one is solid enought. The current method has the great
advantage of being esasy to port to other formats.

>> It's something a lot of people have been looking for (mainly
>> people with xen < 4 that doesn't support bzImages at all).
>
> xen 3.4 does, doesn't it (at least the tip of 3.4-testing.hg does)? And
> I thought e.g. RHEL5 (which uses an older base version) had it
> backported. Possibly what is missing is support for all the various
> compression options.

Well I don't know about RHEL5, but I know that it doesn't work on
ubuntu 8.04 (and unfortunatly I'm stuck with one of these servers).
But this script can also be used for debuging purpose, or anything
else needing the raw vmlinux.

Thanks for the tips anyway :).

-- 
Corentin Chary
http://xf.iksaif.net

Re: [PATCH v3] scripts: add extract-vmlinux

[Michal Marek]

On Tue, Aug 16, 2011 at 10:46:05AM +0200, Corentin Chary wrote:
> This script can be used to extract vmlinux from a compressed
> kernel image (bzImage, etc..). It's inspired from (a subset of)
> extract-ikconfig.
> 
> It's something a lot of people have been looking for (mainly
> people with xen < 4 that doesn't support bzImages at all).
> 
> Signed-off-by: Corentin Chary <corentincj@iksaif.net>
> ---
> 
> Since v1:
> * update license
> 
> Since v2:
> * add some comments
> 
>  scripts/extract-vmlinux |   62 +++++++++++++++++++++++++++++++++++++++++++++++

Applied to kbuild-2.6.git#misc, thanks.

Michal