All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Carsten Schiers" <carsten@schiers.de>
To: "Konrad Rzeszutek Wilk" <konrad@darnok.org>,
	"Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
Cc: xen-devel <xen-devel@lists.xensource.com>,
	"lersek@redhat.com" <lersek@redhat.com>,
	"Ian Campbell" <Ian.Campbell@citrix.com>,
	"zhenzhong.duan@oracle.com" <zhenzhong.duan@oracle.com>
Subject: Re: Load increase after memory upgrade (part2)
Date: Sun, 4 Dec 2011 13:09:28 +0100	[thread overview]
Message-ID: <zarafa.4edb62f8.6417.1a54a5c25b8f0fd7@uhura.space.zz> (raw)
In-Reply-To: <20111202152339.GA2687@andromeda.dapyr.net>

Here with two cards enabled and creating a bit "work" by watching TV with one oft hem:

[   23.842720] Starting SWIOTLB debug thread.
[   23.842750] swiotlb_start_thread: Go!
[   23.842838] xen_swiotlb_start_thread: Go!
[   28.841451] 0 [budget_av 0000:00:01.0] bounce: from:435596(slow:0)to:0 map:658 unmap:0 sync:435596
[   28.841592] SWIOTLB is 4% full
[   33.840147] 0 [budget_av 0000:00:01.0] bounce: from:127652(slow:0)to:0 map:0 unmap:0 sync:127652
[   33.840283] SWIOTLB is 4% full
[   33.844222] 0 budget_av 0000:00:01.0 alloc coherent: 8, free: 0
[   38.840227] 0 [budget_av 0000:00:01.0] bounce: from:128310(slow:0)to:0 map:0 unmap:0 sync:128310
[   38.840361] SWIOTLB is 4% full
[   43.840182] 0 [budget_av 0000:00:01.0] bounce: from:128310(slow:0)to:0 map:0 unmap:0 sync:128310
[   43.840323] SWIOTLB is 4% full
[   48.840094] 0 [budget_av 0000:00:01.0] bounce: from:127652(slow:0)to:0 map:0 unmap:0 sync:127652
[   48.840154] SWIOTLB is 4% full
[   53.840160] 0 [budget_av 0000:00:01.0] bounce: from:119756(slow:0)to:0 map:0 unmap:0 sync:119756
[   53.840301] SWIOTLB is 4% full
[   58.840202] 0 [budget_av 0000:00:01.0] bounce: from:128310(slow:0)to:0 map:0 unmap:0 sync:128310
[   58.840339] SWIOTLB is 4% full
[   63.840626] 0 [budget_av 0000:00:01.0] bounce: from:128310(slow:0)to:0 map:0 unmap:0 sync:128310
[   63.840686] SWIOTLB is 4% full
[   68.840122] 0 [budget_av 0000:00:01.0] bounce: from:127323(slow:0)to:0 map:0 unmap:0 sync:127323
[   68.840180] SWIOTLB is 4% full
[   73.840647] 0 [budget_av 0000:00:01.0] bounce: from:211547(slow:0)to:0 map:0 unmap:0 sync:211547
[   73.840784] SWIOTLB is 4% full
[   78.840204] 0 [budget_av 0000:00:01.0] bounce: from:255962(slow:0)to:0 map:0 unmap:0 sync:255962
[   78.840344] SWIOTLB is 4% full
[   83.840114] 0 [budget_av 0000:00:01.0] bounce: from:255304(slow:0)to:0 map:0 unmap:0 sync:255304
[   83.840178] SWIOTLB is 4% full
[   88.840158] 0 [budget_av 0000:00:01.0] bounce: from:256620(slow:0)to:0 map:0 unmap:0 sync:256620
[   88.840302] SWIOTLB is 4% full
[   93.840185] 0 [budget_av 0000:00:00.0] bounce: from:250040(slow:0)to:0 map:0 unmap:0 sync:250040
[   93.840319] SWIOTLB is 4% full
[   98.840181] 0 [budget_av 0000:00:00.0] bounce: from:255962(slow:0)to:0 map:0 unmap:0 sync:255962
[   98.841563] SWIOTLB is 4% full
[  103.841221] 0 [budget_av 0000:00:00.0] bounce: from:255962(slow:0)to:0 map:0 unmap:0 sync:255962
[  103.841361] SWIOTLB is 4% full
[  108.840247] 0 [budget_av 0000:00:00.0] bounce: from:255962(slow:0)to:0 map:0 unmap:0 sync:255962
[  108.840389] SWIOTLB is 4% full
[  113.840157] 0 [budget_av 0000:00:00.0] bounce: from:261555(slow:0)to:0 map:0 unmap:0 sync:261555
[  113.840298] SWIOTLB is 4% full
[  118.840119] 0 [budget_av 0000:00:00.0] bounce: from:295442(slow:0)to:0 map:0 unmap:0 sync:295442
[  118.840259] SWIOTLB is 4% full
[  123.841025] 0 [budget_av 0000:00:00.0] bounce: from:295113(slow:0)to:0 map:0 unmap:0 sync:295113
[  123.841164] SWIOTLB is 4% full
[  128.840175] 0 [budget_av 0000:00:00.0] bounce: from:294784(slow:0)to:0 map:0 unmap:0 sync:294784
[  128.840310] SWIOTLB is 4% full
[  133.840194] 0 [budget_av 0000:00:00.0] bounce: from:293797(slow:0)to:0 map:0 unmap:0 sync:293797
[  133.840330] SWIOTLB is 4% full
[  138.840498] 0 [budget_av 0000:00:00.0] bounce: from:341502(slow:0)to:0 map:0 unmap:0 sync:341502
[  138.840637] SWIOTLB is 4% full
[  143.840173] 0 [budget_av 0000:00:00.0] bounce: from:341502(slow:0)to:0 map:0 unmap:0 sync:341502
[  143.840313] SWIOTLB is 4% full
[  148.840215] 0 [budget_av 0000:00:00.0] bounce: from:341831(slow:0)to:0 map:0 unmap:0 sync:341831
[  148.840355] SWIOTLB is 4% full
[  153.840205] 0 [budget_av 0000:00:01.0] bounce: from:329658(slow:0)to:0 map:0 unmap:0 sync:329658
[  153.840341] SWIOTLB is 4% full
[  158.840137] 0 [budget_av 0000:00:00.0] bounce: from:342160(slow:0)to:0 map:0 unmap:0 sync:342160
[  158.840277] SWIOTLB is 4% full
[  163.841288] 0 [budget_av 0000:00:00.0] bounce: from:341502(slow:0)to:0 map:0 unmap:0 sync:341502
[  163.841424] SWIOTLB is 4% full
[  168.840198] 0 [budget_av 0000:00:00.0] bounce: from:341502(slow:0)to:0 map:0 unmap:0 sync:341502
[  168.840339] SWIOTLB is 4% full
[  173.840167] 0 [budget_av 0000:00:00.0] bounce: from:341502(slow:0)to:0 map:0 unmap:0 sync:341502
[  173.840304] SWIOTLB is 4% full
[  178.840184] 0 [budget_av 0000:00:00.0] bounce: from:328013(slow:0)to:0 map:0 unmap:0 sync:328013
[  178.840324] SWIOTLB is 4% full
[  183.840129] 0 [budget_av 0000:00:00.0] bounce: from:341831(slow:0)to:0 map:0 unmap:0 sync:341831
[  183.840269] SWIOTLB is 4% full
[  188.840123] 0 [budget_av 0000:00:01.0] bounce: from:340515(slow:0)to:0 map:0 unmap:0 sync:340515
[  188.841647] SWIOTLB is 4% full
[  193.840192] 0 [budget_av 0000:00:00.0] bounce: from:338541(slow:0)to:0 map:0 unmap:0 sync:338541
[  193.840329] SWIOTLB is 4% full
[  198.840148] 0 [budget_av 0000:00:01.0] bounce: from:330316(slow:0)to:0 map:0 unmap:0 sync:330316
[  198.840230] SWIOTLB is 4% full
[  203.840860] 0 [budget_av 0000:00:00.0] bounce: from:341831(slow:0)to:0 map:0 unmap:0 sync:341831
[  203.841000] SWIOTLB is 4% full
[  208.840562] 0 [budget_av 0000:00:01.0] bounce: from:337883(slow:0)to:0 map:0 unmap:0 sync:337883
[  208.840698] SWIOTLB is 4% full
[  213.840171] 0 [budget_av 0000:00:00.0] bounce: from:341502(slow:0)to:0 map:0 unmap:0 sync:341502
[  213.840311] SWIOTLB is 4% full
[  218.840214] 0 [budget_av 0000:00:01.0] bounce: from:320117(slow:0)to:0 map:0 unmap:0 sync:320117
[  218.840354] SWIOTLB is 4% full
[  223.840238] 0 [budget_av 0000:00:01.0] bounce: from:299390(slow:0)to:0 map:0 unmap:0 sync:299390
[  223.840373] SWIOTLB is 4% full
[  228.841415] 0 [budget_av 0000:00:01.0] bounce: from:298732(slow:0)to:0 map:0 unmap:0 sync:298732
[  228.841560] SWIOTLB is 4% full
[  233.840705] 0 [budget_av 0000:00:00.0] bounce: from:299061(slow:0)to:0 map:0 unmap:0 sync:299061
[  233.840844] SWIOTLB is 4% full
[  238.840145] 0 [budget_av 0000:00:01.0] bounce: from:293468(slow:0)to:0 map:0 unmap:0 sync:293468
[  238.840280] SWIOTLB is 4% full

-----Ursprüngliche Nachricht-----
Von: Konrad Rzeszutek Wilk [mailto:konrad@darnok.org] 
Gesendet: Freitag, 2. Dezember 2011 16:24
An: Konrad Rzeszutek Wilk
Cc: Ian Campbell; xen-devel; Carsten Schiers; zhenzhong.duan@oracle.com; lersek@redhat.com
Betreff: Re: [Xen-devel] Load increase after memory upgrade (part2)

> > > > > That is a puzzle. It should not. The code is very much the same - both
> > > > > use the generic SWIOTLB which has not changed for years.
> > > > 
> > > > The swiotlb-xen used by classic-xen kernels (which I assume is what
> > > > Carsten means by "Xenified") isn't exactly the same as the stuff in
> > > > mainline Linux, it's been heavily refactored for one thing. It's not
> > > > impossible that mainline is bouncing something it doesn't really need
> > > > to.
> > > 
> > > The usage, at least with 'pci_alloc_coherent' is that there is no bouncing
> > > being done. The alloc_coherent will allocate a nice page, underneath the 4GB
> > > mark and give it to the driver. The driver can use it as it wishes and there
> > > is no need to bounce buffer.
> > 
> > Oh, I didn't realise dma_alloc_coherent was part of swiotlb now. Only a
> > subset of swiotlb is in use then, all the bouncing stuff _should_ be
> > idle/unused -- but has that been confirmed?
> 
> Nope. I hope that the diagnostic patch I have in mind will prove/disprove that.
> Now I just need to find a moment to write it :-)

Done!

Carsten, can you please patch your kernel with this hacky patch and
when you have booted the new kernel, just do

modprobe dump_swiotlb

it should give an idea of how many bounces are happening, coherent
allocations, syncs, and so on.. along with the last driver that
did those operations.

  parent reply	other threads:[~2011-12-04 12:09 UTC|newest]

Thread overview: 66+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-11-24 12:28 Load increase after memory upgrade (part2) Carsten Schiers
2011-11-25 18:42 ` Konrad Rzeszutek Wilk
2011-11-25 22:11   ` Carsten Schiers
2011-11-28 15:28     ` Konrad Rzeszutek Wilk
2011-11-28 15:40       ` Ian Campbell
2011-11-28 16:45         ` Konrad Rzeszutek Wilk
2011-11-29  8:31           ` Jan Beulich
2011-11-29  9:31             ` Carsten Schiers
2011-11-29  9:46           ` Carsten Schiers
2011-11-29 10:23           ` Ian Campbell
2011-11-29 15:33             ` Konrad Rzeszutek Wilk
2011-12-02 15:23               ` Konrad Rzeszutek Wilk
2011-12-04 11:59                 ` Carsten Schiers
2011-12-04 12:09                 ` Carsten Schiers [this message]
2011-12-06  3:26                   ` Konrad Rzeszutek Wilk
2011-12-14 20:23                     ` Konrad Rzeszutek Wilk
2011-12-14 22:07                       ` Konrad Rzeszutek Wilk
2011-12-15 14:52                         ` Carsten Schiers
2011-12-16 14:56                         ` Carsten Schiers
2011-12-16 15:04                           ` Konrad Rzeszutek Wilk
2011-12-16 15:51                             ` Carsten Schiers
2011-12-16 16:19                               ` Konrad Rzeszutek Wilk
2011-12-17 22:12                                 ` Carsten Schiers
2011-12-18  0:19                                   ` Sander Eikelenboom
2011-12-19 14:56                                     ` Konrad Rzeszutek Wilk
2012-01-10 21:55                                       ` Konrad Rzeszutek Wilk
2012-01-12 22:06                                         ` Sander Eikelenboom
2012-01-13  8:12                                           ` Jan Beulich
2012-01-13 15:13                                           ` Konrad Rzeszutek Wilk
2012-01-15 11:32                                             ` Sander Eikelenboom
2012-01-17 21:02                                               ` Konrad Rzeszutek Wilk
2012-01-18 11:28                                                 ` Pasi Kärkkäinen
2012-01-18 11:39                                                   ` Jan Beulich
2012-01-18 11:35                                                 ` Jan Beulich
2012-01-18 14:29                                                   ` Konrad Rzeszutek Wilk
2012-01-23 22:32                                                     ` Konrad Rzeszutek Wilk
2012-01-24  8:58                                                       ` Jan Beulich
2012-01-24 14:17                                                         ` Konrad Rzeszutek Wilk
2012-01-24 21:32                                                       ` Carsten Schiers
2012-01-25 12:02                                                       ` Carsten Schiers
2012-01-25 19:06                                                       ` Carsten Schiers
2012-01-25 21:02                                                         ` Konrad Rzeszutek Wilk
2012-02-15 19:28                                                         ` Konrad Rzeszutek Wilk
2012-02-16  8:56                                                           ` Jan Beulich
2012-02-17 15:07                                                             ` Konrad Rzeszutek Wilk
2012-02-28 14:35                                                               ` Carsten Schiers
2012-02-29 12:10                                                                 ` Carsten Schiers
2012-02-29 12:56                                                                   ` Carsten Schiers
2012-05-11  9:39                                                                     ` Carsten Schiers
2012-05-11 19:41                                                                       ` Konrad Rzeszutek Wilk
2012-06-13 16:55                                                                         ` Konrad Rzeszutek Wilk
2012-06-14  7:07                                                                           ` Jan Beulich
2012-06-14 18:33                                                                             ` Konrad Rzeszutek Wilk
2012-06-14 18:43                                                                             ` Carsten Schiers
2012-06-14  8:38                                                                           ` David Vrabel
2012-06-14 18:31                                                                             ` Konrad Rzeszutek Wilk
2012-06-14 18:40                                                                           ` Carsten Schiers
2012-06-14 19:16                                                                             ` Carsten Schiers
2011-12-19 14:54                                   ` Konrad Rzeszutek Wilk
2011-12-04 12:18                 ` Carsten Schiers
2011-11-28 16:58         ` Laszlo Ersek
2011-11-29  9:37         ` Carsten Schiers
2011-11-28 15:52       ` Carsten Schiers
2011-11-26  9:14   ` Carsten Schiers
2011-11-28 15:30     ` Konrad Rzeszutek Wilk
2011-11-29  9:42       ` Carsten Schiers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=zarafa.4edb62f8.6417.1a54a5c25b8f0fd7@uhura.space.zz \
    --to=carsten@schiers.de \
    --cc=Ian.Campbell@citrix.com \
    --cc=konrad.wilk@oracle.com \
    --cc=konrad@darnok.org \
    --cc=lersek@redhat.com \
    --cc=xen-devel@lists.xensource.com \
    --cc=zhenzhong.duan@oracle.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.