From: Atsushi Kumagai <ats-kumagai@wm.jp.nec.com>
To: "zhouwj-fnst@cn.fujitsu.com" <zhouwj-fnst@cn.fujitsu.com>
Cc: "kexec@lists.infradead.org" <kexec@lists.infradead.org>
Subject: RE: [PATCH v2 00/10] makedumpfile: parallel processing
Date: Fri, 26 Jun 2015 07:07:13 +0000 [thread overview]
Message-ID: <0910DD04CBD6DE4193FCF86B9C00BE9701DC9C14@BPXM01GP.gisp.nec.co.jp> (raw)
In-Reply-To: <558B6696.4020906@cn.fujitsu.com>
Hello Zhou,
>Hello Atsushi Kumagai,
>
>I test this patch set in several machines and the following is the benchmark.
Thanks for your report, it looks good as before.
I also did simple test on kernel 3.19 on a 5GB Virtual Machine,
but I can't get such good result as below:
/ # time makedumpfile -c --num-threads 0 /proc/vmcore /mnt/dumpfile
Copying data : [100.0 %] /
The dumpfile is saved to /mnt/dumpfile.
makedumpfile Completed.
real 0m 44.40s
user 0m 43.62s
sys 0m 0.69s
/ #
/ # time makedumpfile -c --num-threads 4 /proc/vmcore /mnt/dumpfile
Copying data : [100.0 %] -
Copying data : [100.0 %] /
The dumpfile is saved to /mnt/dumpfile.
makedumpfile Completed.
real 5m 29.54s
user 6m 8.18s
sys 16m 33.25s
/ #
There is a big performance degradation.
Do you have any ideas why this happens ?
Thanks
Atsushi Kumagai
>These tables show the time that makedumpfile spends. And the unit is second.
>
>"core-data" in the table means the context in the vmcore.
> For example:
> core-data's value is 256. It means that in the vmcore, 256 * 8 bits of each page
> are set to 1.
>
>threads-num in the table means how many threads are used. 0 means original single thread
>implementation.
>
>"-l" in the table means producing lzo format vmcore
>
>"-c" in the table means producing kdump-compressed format vmcore
>
>###################################
>- System: PRIMERGY RX300 S6
>- CPU: Intel(R) Xeon(R) CPU x5660
>- memory: 16GB
>###################################
>************ makedumpfile -d 0 ******************
> core-data 0 256 512 768 1024 1280 1536 1792 2048 2304
>2560 2816 3072 3328 3584 3840
> threads-num
>-l
> 0 11 112 163 168 167 167 167 167 170 169 167
>166 167 168 169 113
> 4 5 111 158 166 167 167 167 166 169 170 166
>166 167 167 158 111
> 8 5 111 158 167 169 170 167 169 169 170 167
>168 168 168 159 111
> 12 6 111 158 168 167 167 169 168 170 169 167
>168 167 167 161 114
>-c
> 0 54 544 643 666 589 517 468 405 408 429 491
>528 592 676 654 527
> 4 60 179 210 216 206 220 209 214 211 222 220
>208 209 210 245 177
> 8 43 113 146 169 179 215 195 211 199 195 216
>197 185 165 158 109
> 12 44 106 142 162 169 171 168 173 174 171 200
>173 177 165 139 107
>
>************ makedumpfile -d 1 ******************
> core-data 0 256 512 768 1024 1280 1536 1792 2048 2304
>2560 2816 3072 3328 3584 3840
> threads-num
>-l
> 0 10 137 163 167 167 167 167 167 171 170 166
>166 166 168 169 138
> 4 5 111 158 166 167 166 167 166 170 171 166
>167 166 166 159 111
> 8 5 114 159 167 167 169 167 167 169 169 169
>167 168 167 160 111
> 12 6 113 159 168 167 168 167 168 169 170 168
>168 167 168 159 112
>-c
> 0 53 522 645 663 591 512 470 401 412 413 479
>532 587 666 648 524
> 4 57 205 244 216 208 217 216 225 221 225 223
>217 213 209 246 174
> 8 41 118 154 176 191 213 219 223 212 210 222
>207 198 176 164 118
> 12 43 111 148 174 189 194 191 182 178 173 204
>196 194 170 150 112
>
>************ makedumpfile -d 7 ******************
> core-data 0 256 512 768 1024 1280 1536 1792 2048 2304
>2560 2816 3072 3328 3584 3840
> threads-num
>-l
> 0 10 144 199 168 167 167 167 167 171 169 167
>166 166 168 169 144
> 4 5 110 158 167 167 167 166 166 170 169 166
>166 166 166 159 110
> 8 5 111 159 167 166 170 167 166 169 169 168
>167 167 167 159 110
> 12 6 111 159 167 167 166 168 167 169 169 168
>167 167 167 160 111
>-c
> 0 52 523 642 659 592 511 490 402 410 422 464
>525 591 666 647 529
> 4 58 177 210 213 208 224 217 228 223 229 227
>216 210 212 215 175
> 8 41 116 152 178 191 213 219 225 216 211 221
>211 196 180 158 116
> 12 41 111 151 176 192 187 192 190 174 175 218
>186 189 172 151 112
>
>************ makedumpfile -d 31 ******************
> core-data 0 256 512 768 1024 1280 1536 1792 2048 2304
>2560 2816 3072 3328 3584 3840
> threads-num
>-l
> 0 0 0 0 0 0 0 0 0 0 0 0
>0 0 0 0 0
> 4 2 2 2 3 3 3 2 3 3 3 3
>2 3 3 3 3
> 8 2 3 3 2 3 3 3 3 3 3 3
>3 3 3 3 3
> 12 2 3 3 3 3 3 3 3 3 3 3
>3 3 3 3 3
>-c
> 0 3 3 3 3 3 3 3 3 3 3 3
>3 3 3 3 3
> 4 4 5 5 4 5 5 5 5 4 5 5
>5 4 4 5 5
> 8 4 4 4 3 4 4 4 4 4 4 4
>4 4 4 4 4
> 12 4 4 4 4 4 4 4 4 4 4 4
>4 4 4 4 4
>
>###################################
>- System: PRIMERGY RX300 S7
>- CPU: Intel(R) Xeon(R) CPU E5-2620
>- memory: 32GB
>###################################
>************ makedumpfile -d 0 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 23 211 246 255 252 252 253 257
> 4 14 168 234 250 253 251 251 252
> 12 17 165 237 250 251 251 253 253
>-c
> 0 117 1210 1620 1699 1443 1294 1235 1051
> 4 161 438 525 543 506 465 484 415
> 12 131 211 245 250 246 248 260 265
>
>************ makedumpfile -d 7 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 24 215 253 256 255 254 257 255
> 4 14 165 235 247 249 250 252 253
> 12 16 169 237 251 255 253 252 250
>-c
> 0 120 1202 1625 1698 1434 1274 1223 1040
> 4 157 438 533 542 507 461 479 400
> 12 132 204 242 248 242 244 257 263
>
>###################################
>- System: PRIMEQUEST 1800E
>- CPU: Intel(R) Xeon(R) CPU E7540
>- memory: 32GB
>###################################
>************ makedumpfile -d 0 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 34 282 245 179 179 179 179 180
> 4 63 143 224 230 220 212 207 204
> 8 65 129 200 225 235 235 225 220
> 12 67 149 186 211 222 229 237 236
>-c
> 0 158 1505 2119 2129 1707 1483 1440 1273
> 4 207 589 672 673 636 564 536 514
> 8 176 327 377 387 367 336 314 291
> 12 191 272 295 306 288 259 257 240
>
>************ makedumpfile -d 7 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 34 270 248 187 188 187 187 187
> 4 63 154 186 188 189 189 190 190
> 8 64 131 220 218 197 186 187 188
> 12 65 159 198 232 229 207 190 188
>-c
> 0 154 1508 2089 2133 1792 1660 1462 1312
> 4 203 594 684 701 627 592 535 503
> 8 172 326 377 393 366 334 313 286
> 12 182 273 295 308 283 258 249 237
>
>************ makedumpfile -d 31 ******************
> core-data 0 256 512 768 1024 1280 1536 1792
> threads-num
>-l
> 0 2 1 1 2 1 2 2 2
> 4 48 48 48 48 49 48 48 49
> 8 48 49 48 49 48 47 49 48
> 12 49 50 49 49 49 48 50 49
>-c
> 0 10 9 10 10 9 10 9 9
> 4 52 53 52 52 53 52 53 52
> 8 51 51 52 52 52 51 51 52
> 12 53 52 52 53 52 51 52 52
>
>
>
>
>On 06/19/2015 04:56 PM, Zhou Wenjian wrote:
>> This patch set implements parallel processing by means of multiple threads.
>> With this patch set, it is available to use multiple threads to read
>> and compress pages. This parallel process will save time.
>> This feature only supports creating dumpfile in kdump-compressed format from
>> vmcore in kdump-compressed format or elf format. Currently, sadump and
>> xen kdump are not supported.
>>
>> Qiao Nuohan (10):
>> Add readpage_kdump_compressed_parallel
>> Add mappage_elf_parallel
>> Add readpage_elf_parallel
>> Add read_pfn_parallel
>> Add function to initial bitmap for parallel use
>> Add filter_data_buffer_parallel
>> Add write_kdump_pages_parallel to allow parallel process
>> Initial and free data used for parallel process
>> Make makedumpfile available to read and compress pages parallelly
>> Add usage and manual about multiple threads process
>>
>> Makefile | 2 +
>> erase_info.c | 29 ++-
>> erase_info.h | 2 +
>> makedumpfile.8 | 24 ++
>> makedumpfile.c | 1096 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
>> makedumpfile.h | 80 ++++
>> print_info.c | 16 +
>> 7 files changed, 1246 insertions(+), 3 deletions(-)
>>
>>
>> _______________________________________________
>> kexec mailing list
>> kexec@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec
>
>
>--
>Thanks
>Zhou Wenjian
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
next prev parent reply other threads:[~2015-06-26 7:11 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-19 8:56 [PATCH v2 00/10] makedumpfile: parallel processing Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 01/10] Add readpage_kdump_compressed_parallel Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 02/10] Add mappage_elf_parallel Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 03/10] Add readpage_elf_parallel Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 04/10] Add read_pfn_parallel Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 05/10] Add function to initial bitmap for parallel use Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 06/10] Add filter_data_buffer_parallel Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 07/10] Add write_kdump_pages_parallel to allow parallel process Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 08/10] Initial and free data used for " Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 09/10] Make makedumpfile available to read and compress pages parallelly Zhou Wenjian
2015-06-19 8:56 ` [PATCH v2 10/10] Add usage and manual about multiple threads process Zhou Wenjian
2015-06-25 2:25 ` [PATCH v2 00/10] makedumpfile: parallel processing "Zhou, Wenjian/周文剑"
2015-06-26 7:07 ` Atsushi Kumagai [this message]
2015-06-26 7:27 ` "Zhou, Wenjian/周文剑"
2015-06-26 7:49 ` Atsushi Kumagai
2015-06-29 6:19 ` "Zhou, Wenjian/周文剑"
2015-06-30 9:06 ` Atsushi Kumagai
2015-07-06 13:19 ` "Zhou, Wenjian/周文剑"
2015-07-08 8:10 ` Atsushi Kumagai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0910DD04CBD6DE4193FCF86B9C00BE9701DC9C14@BPXM01GP.gisp.nec.co.jp \
--to=ats-kumagai@wm.jp.nec.com \
--cc=kexec@lists.infradead.org \
--cc=zhouwj-fnst@cn.fujitsu.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.