* agcount 33 by default for a single HDD?
@ 2018-08-01 1:32 Chris Murphy
2018-08-01 2:01 ` Chris Murphy
2018-08-01 4:12 ` Dave Chinner
0 siblings, 2 replies; 9+ messages in thread
From: Chris Murphy @ 2018-08-01 1:32 UTC (permalink / raw)
To: xfs list
This seems suboptimal. Basically this is a 750G thin volume. I don't
have a plain partition on a device handy to try this out but I'm
pretty certain the default is 4 AG's in that case, so I'm confused why
by default 33 AGs are created on a thin volume. The LVM volume group
is on a dmcrypt PV.
xfsprogs-4.15.1-1.fc28.x86_64
lvm2-2.02.177-5.fc28.x86_64
kernel-4.17.11-200.fc28.x86_64
[chris@f28s ~]$ sudo lvdisplay 4vg/nukeit
--- Logical volume ---
LV Path /dev/4vg/nukeit
LV Name nukeit
VG Name 4vg
LV UUID T21akN-RZeV-CfpZ-br70-6A1e-IkFI-VYOhNj
LV Write Access read/write
LV Creation host, time f28s.local, 2018-07-31 19:28:21 -0600
LV Pool name 4pool
LV Status available
# open 0
LV Size 750.00 GiB
Mapped size 0.00%
Current LE 192000
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:10
[chris@f28s ~]$ sudo mkfs.xfs /dev/mapper/4vg-nukeit
meta-data=/dev/mapper/4vg-nukeit isize=512 agcount=33, agsize=6143872 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0,
rmapbt=0, reflink=0
data = bsize=4096 blocks=196608000, imaxpct=25
= sunit=128 swidth=1024 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=96000, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
[chris@f28s ~]$
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: agcount 33 by default for a single HDD?
2018-08-01 1:32 agcount 33 by default for a single HDD? Chris Murphy
@ 2018-08-01 2:01 ` Chris Murphy
2018-08-01 4:12 ` Dave Chinner
1 sibling, 0 replies; 9+ messages in thread
From: Chris Murphy @ 2018-08-01 2:01 UTC (permalink / raw)
To: Chris Murphy; +Cc: xfs list
On Tue, Jul 31, 2018 at 7:32 PM, Chris Murphy <lists@colorremedies.com> wrote:
> This seems suboptimal. Basically this is a 750G thin volume. I don't
> have a plain partition on a device handy to try this out but I'm
> pretty certain the default is 4 AG's in that case, so I'm confused why
> by default 33 AGs are created on a thin volume. The LVM volume group
> is on a dmcrypt PV.
>
>
> xfsprogs-4.15.1-1.fc28.x86_64
> lvm2-2.02.177-5.fc28.x86_64
> kernel-4.17.11-200.fc28.x86_64
>
>
> = sunit=128 swidth=1024 blks
This also seems screwy for a single non-raid device. I wonder if
there's something goofy happening with device mapper that's giving the
wrong information to XFS.
I just built xfsprogs 4.17 from git and I get different unexpected results:
[chris@f28s mkfs]$ sudo ./mkfs.xfs -V
mkfs.xfs version 4.17.0
[chris@f28s mkfs]$ sudo ./mkfs.xfs /dev/mapper/4vg-nukeit
mkfs.xfs: Use the -f option to force overwrite.
[chris@f28s mkfs]$ sudo ./mkfs.xfs -f /dev/mapper/4vg-nukeit
meta-data=/dev/mapper/4vg-nukeit isize=512 agcount=4, agsize=49152000 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=0
data = bsize=4096 blocks=196608000, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=96000, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
[chris@f28s mkfs]$
The agcount, sunit and swidth are what I expect, but sectsz=512 is
not. The device has a 4096 byte physical sector. And this is being
reported by 'lsblk' as a 4096 byte physical sector with 512 byte
logical sector (it is a 512e drive). The 4.15.0 Fedora built xfsprogs
got this right, but 4.17.0 as built is not.
:shrug:
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: agcount 33 by default for a single HDD?
2018-08-01 1:32 agcount 33 by default for a single HDD? Chris Murphy
2018-08-01 2:01 ` Chris Murphy
@ 2018-08-01 4:12 ` Dave Chinner
2018-08-01 4:38 ` Chris Murphy
1 sibling, 1 reply; 9+ messages in thread
From: Dave Chinner @ 2018-08-01 4:12 UTC (permalink / raw)
To: Chris Murphy; +Cc: xfs list
On Tue, Jul 31, 2018 at 07:32:50PM -0600, Chris Murphy wrote:
> This seems suboptimal.
It's actually a very useful optimisation to make on thin devices.
> Basically this is a 750G thin volume. I don't
> have a plain partition on a device handy to try this out but I'm
> pretty certain the default is 4 AG's in that case, so I'm confused why
> by default 33 AGs are created on a thin volume. The LVM volume group
> is on a dmcrypt PV.
It's a thin volume, therefore it advertises an optimal IO size and
alignment setting (i.e. the thin volume allocation chunk size).
Hence mkfs.xfs treats it as a "multi-disk device" and sets up
alignment and AG count appropriately.
This is actually the right optimisation to make for sparse devices -
more AGs increase filesystem concurrency but we normally restrict it
on single spindles because each AG adds more seeks into typical
workloads and slows them down. However, the thin volume already adds
that penalty to the storage stack for us because they don't have a
linear LBA-to-physical location characteristic. Hence we can
increase filesystem concurrency without addition performance
penalties being incurred.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: agcount 33 by default for a single HDD?
2018-08-01 4:12 ` Dave Chinner
@ 2018-08-01 4:38 ` Chris Murphy
2018-08-01 4:41 ` Eric Sandeen
0 siblings, 1 reply; 9+ messages in thread
From: Chris Murphy @ 2018-08-01 4:38 UTC (permalink / raw)
To: Dave Chinner; +Cc: Chris Murphy, xfs list
On Tue, Jul 31, 2018 at 10:12 PM, Dave Chinner <david@fromorbit.com> wrote:
> On Tue, Jul 31, 2018 at 07:32:50PM -0600, Chris Murphy wrote:
>> This seems suboptimal.
>
> It's actually a very useful optimisation to make on thin devices.
>
>> Basically this is a 750G thin volume. I don't
>> have a plain partition on a device handy to try this out but I'm
>> pretty certain the default is 4 AG's in that case, so I'm confused why
>> by default 33 AGs are created on a thin volume. The LVM volume group
>> is on a dmcrypt PV.
>
> It's a thin volume, therefore it advertises an optimal IO size and
> alignment setting (i.e. the thin volume allocation chunk size).
> Hence mkfs.xfs treats it as a "multi-disk device" and sets up
> alignment and AG count appropriately.
>
> This is actually the right optimisation to make for sparse devices -
> more AGs increase filesystem concurrency but we normally restrict it
> on single spindles because each AG adds more seeks into typical
> workloads and slows them down. However, the thin volume already adds
> that penalty to the storage stack for us because they don't have a
> linear LBA-to-physical location characteristic. Hence we can
> increase filesystem concurrency without addition performance
> penalties being incurred.
OK so why 33 AG's with xfsprogs 4.15, but 4 AG's with xfsprogs 4.17,
when directed to the same thin LV? And also the difference in sunit
and swidth?
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: agcount 33 by default for a single HDD?
2018-08-01 4:38 ` Chris Murphy
@ 2018-08-01 4:41 ` Eric Sandeen
2018-08-01 5:04 ` Chris Murphy
0 siblings, 1 reply; 9+ messages in thread
From: Eric Sandeen @ 2018-08-01 4:41 UTC (permalink / raw)
To: Chris Murphy, Dave Chinner; +Cc: xfs list
On 7/31/18 11:38 PM, Chris Murphy wrote:
> On Tue, Jul 31, 2018 at 10:12 PM, Dave Chinner <david@fromorbit.com> wrote:
>> On Tue, Jul 31, 2018 at 07:32:50PM -0600, Chris Murphy wrote:
>>> This seems suboptimal.
>>
>> It's actually a very useful optimisation to make on thin devices.
>>
>>> Basically this is a 750G thin volume. I don't
>>> have a plain partition on a device handy to try this out but I'm
>>> pretty certain the default is 4 AG's in that case, so I'm confused why
>>> by default 33 AGs are created on a thin volume. The LVM volume group
>>> is on a dmcrypt PV.
>>
>> It's a thin volume, therefore it advertises an optimal IO size and
>> alignment setting (i.e. the thin volume allocation chunk size).
>> Hence mkfs.xfs treats it as a "multi-disk device" and sets up
>> alignment and AG count appropriately.
>>
>> This is actually the right optimisation to make for sparse devices -
>> more AGs increase filesystem concurrency but we normally restrict it
>> on single spindles because each AG adds more seeks into typical
>> workloads and slows them down. However, the thin volume already adds
>> that penalty to the storage stack for us because they don't have a
>> linear LBA-to-physical location characteristic. Hence we can
>> increase filesystem concurrency without addition performance
>> penalties being incurred.
>
> OK so why 33 AG's with xfsprogs 4.15, but 4 AG's with xfsprogs 4.17,
> when directed to the same thin LV? And also the difference in sunit
> and swidth?
>
Did you build 4.17 with --disable-blkid?
-Eric
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: agcount 33 by default for a single HDD?
2018-08-01 4:41 ` Eric Sandeen
@ 2018-08-01 5:04 ` Chris Murphy
2018-08-01 5:05 ` Eric Sandeen
2018-08-01 11:45 ` Carlos Maiolino
0 siblings, 2 replies; 9+ messages in thread
From: Chris Murphy @ 2018-08-01 5:04 UTC (permalink / raw)
To: Eric Sandeen; +Cc: Chris Murphy, Dave Chinner, xfs list
On Tue, Jul 31, 2018 at 10:41 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
>
>
> On 7/31/18 11:38 PM, Chris Murphy wrote:
>> On Tue, Jul 31, 2018 at 10:12 PM, Dave Chinner <david@fromorbit.com> wrote:
>>> On Tue, Jul 31, 2018 at 07:32:50PM -0600, Chris Murphy wrote:
>>>> This seems suboptimal.
>>>
>>> It's actually a very useful optimisation to make on thin devices.
>>>
>>>> Basically this is a 750G thin volume. I don't
>>>> have a plain partition on a device handy to try this out but I'm
>>>> pretty certain the default is 4 AG's in that case, so I'm confused why
>>>> by default 33 AGs are created on a thin volume. The LVM volume group
>>>> is on a dmcrypt PV.
>>>
>>> It's a thin volume, therefore it advertises an optimal IO size and
>>> alignment setting (i.e. the thin volume allocation chunk size).
>>> Hence mkfs.xfs treats it as a "multi-disk device" and sets up
>>> alignment and AG count appropriately.
>>>
>>> This is actually the right optimisation to make for sparse devices -
>>> more AGs increase filesystem concurrency but we normally restrict it
>>> on single spindles because each AG adds more seeks into typical
>>> workloads and slows them down. However, the thin volume already adds
>>> that penalty to the storage stack for us because they don't have a
>>> linear LBA-to-physical location characteristic. Hence we can
>>> increase filesystem concurrency without addition performance
>>> penalties being incurred.
>>
>> OK so why 33 AG's with xfsprogs 4.15, but 4 AG's with xfsprogs 4.17,
>> when directed to the same thin LV? And also the difference in sunit
>> and swidth?
>>
>
> Did you build 4.17 with --disable-blkid?
Yep, that's what doc/INSTALL recommends. Removing that and rebuilding,
sure enough 33 AGs, sunit 128, swidth 1024, and sectsz 4096. I do get
a build time warning.
Building mkfs
[CC] proto.o
[CC] xfs_mkfs.o
In function ‘finish_superblock_setup’,
inlined from ‘main’ at xfs_mkfs.c:3937:2:
xfs_mkfs.c:3188:3: warning: ‘strncpy’ specified bound 12 equals
destination size [-Wstringop-truncation]
strncpy(sbp->sb_fname, cfg->label, sizeof(sbp->sb_fname));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[LD] mkfs.xfs
--
Chris Murphy
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: agcount 33 by default for a single HDD?
2018-08-01 5:04 ` Chris Murphy
@ 2018-08-01 5:05 ` Eric Sandeen
2018-08-01 11:45 ` Carlos Maiolino
1 sibling, 0 replies; 9+ messages in thread
From: Eric Sandeen @ 2018-08-01 5:05 UTC (permalink / raw)
To: Chris Murphy; +Cc: Dave Chinner, xfs list
On 8/1/18 12:04 AM, Chris Murphy wrote:
> On Tue, Jul 31, 2018 at 10:41 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
>>
>>
>> On 7/31/18 11:38 PM, Chris Murphy wrote:
>>> On Tue, Jul 31, 2018 at 10:12 PM, Dave Chinner <david@fromorbit.com> wrote:
>>>> On Tue, Jul 31, 2018 at 07:32:50PM -0600, Chris Murphy wrote:
>>>>> This seems suboptimal.
>>>>
>>>> It's actually a very useful optimisation to make on thin devices.
>>>>
>>>>> Basically this is a 750G thin volume. I don't
>>>>> have a plain partition on a device handy to try this out but I'm
>>>>> pretty certain the default is 4 AG's in that case, so I'm confused why
>>>>> by default 33 AGs are created on a thin volume. The LVM volume group
>>>>> is on a dmcrypt PV.
>>>>
>>>> It's a thin volume, therefore it advertises an optimal IO size and
>>>> alignment setting (i.e. the thin volume allocation chunk size).
>>>> Hence mkfs.xfs treats it as a "multi-disk device" and sets up
>>>> alignment and AG count appropriately.
>>>>
>>>> This is actually the right optimisation to make for sparse devices -
>>>> more AGs increase filesystem concurrency but we normally restrict it
>>>> on single spindles because each AG adds more seeks into typical
>>>> workloads and slows them down. However, the thin volume already adds
>>>> that penalty to the storage stack for us because they don't have a
>>>> linear LBA-to-physical location characteristic. Hence we can
>>>> increase filesystem concurrency without addition performance
>>>> penalties being incurred.
>>>
>>> OK so why 33 AG's with xfsprogs 4.15, but 4 AG's with xfsprogs 4.17,
>>> when directed to the same thin LV? And also the difference in sunit
>>> and swidth?
>>>
>>
>> Did you build 4.17 with --disable-blkid?
>
> Yep, that's what doc/INSTALL recommends. Removing that and rebuilding,
> sure enough 33 AGs, sunit 128, swidth 1024, and sectsz 4096. I do get
> a build time warning.
>
> Building mkfs
> [CC] proto.o
> [CC] xfs_mkfs.o
> In function ‘finish_superblock_setup’,
> inlined from ‘main’ at xfs_mkfs.c:3937:2:
> xfs_mkfs.c:3188:3: warning: ‘strncpy’ specified bound 12 equals
> destination size [-Wstringop-truncation]
> strncpy(sbp->sb_fname, cfg->label, sizeof(sbp->sb_fname));
yeah, the on-disk label is only 12 bytes and is not null-terminated.
I guess we could change it to a memcpy or something. o_O
-Eric
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: agcount 33 by default for a single HDD?
2018-08-01 5:04 ` Chris Murphy
2018-08-01 5:05 ` Eric Sandeen
@ 2018-08-01 11:45 ` Carlos Maiolino
2018-08-01 18:50 ` Chris Murphy
1 sibling, 1 reply; 9+ messages in thread
From: Carlos Maiolino @ 2018-08-01 11:45 UTC (permalink / raw)
To: Chris Murphy; +Cc: xfs list
On Tue, Jul 31, 2018 at 11:04:05PM -0600, Chris Murphy wrote:
> On Tue, Jul 31, 2018 at 10:41 PM, Eric Sandeen <sandeen@sandeen.net> wrote:
> >
> >
> > On 7/31/18 11:38 PM, Chris Murphy wrote:
> >> On Tue, Jul 31, 2018 at 10:12 PM, Dave Chinner <david@fromorbit.com> wrote:
> >>> On Tue, Jul 31, 2018 at 07:32:50PM -0600, Chris Murphy wrote:
> >>>> This seems suboptimal.
> >>>
> >>> It's actually a very useful optimisation to make on thin devices.
> >>>
> >>>> Basically this is a 750G thin volume. I don't
> >>>> have a plain partition on a device handy to try this out but I'm
> >>>> pretty certain the default is 4 AG's in that case, so I'm confused why
> >>>> by default 33 AGs are created on a thin volume. The LVM volume group
> >>>> is on a dmcrypt PV.
> >>>
> >>> It's a thin volume, therefore it advertises an optimal IO size and
> >>> alignment setting (i.e. the thin volume allocation chunk size).
> >>> Hence mkfs.xfs treats it as a "multi-disk device" and sets up
> >>> alignment and AG count appropriately.
> >>>
> >>> This is actually the right optimisation to make for sparse devices -
> >>> more AGs increase filesystem concurrency but we normally restrict it
> >>> on single spindles because each AG adds more seeks into typical
> >>> workloads and slows them down. However, the thin volume already adds
> >>> that penalty to the storage stack for us because they don't have a
> >>> linear LBA-to-physical location characteristic. Hence we can
> >>> increase filesystem concurrency without addition performance
> >>> penalties being incurred.
> >>
> >> OK so why 33 AG's with xfsprogs 4.15, but 4 AG's with xfsprogs 4.17,
> >> when directed to the same thin LV? And also the difference in sunit
> >> and swidth?
> >>
> >
> > Did you build 4.17 with --disable-blkid?
>
> Yep, that's what doc/INSTALL recommends. Removing that and rebuilding,
> sure enough 33 AGs, sunit 128, swidth 1024, and sectsz 4096. I do get
> a build time warning.
Where did you see that --disable-blkid is recommended? Are you using MacOS to
build it?
>
> Building mkfs
> [CC] proto.o
> [CC] xfs_mkfs.o
> In function ‘finish_superblock_setup’,
> inlined from ‘main’ at xfs_mkfs.c:3937:2:
> xfs_mkfs.c:3188:3: warning: ‘strncpy’ specified bound 12 equals
> destination size [-Wstringop-truncation]
> strncpy(sbp->sb_fname, cfg->label, sizeof(sbp->sb_fname));
> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [LD] mkfs.xfs
>
>
>
> --
> Chris Murphy
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Carlos
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: agcount 33 by default for a single HDD?
2018-08-01 11:45 ` Carlos Maiolino
@ 2018-08-01 18:50 ` Chris Murphy
0 siblings, 0 replies; 9+ messages in thread
From: Chris Murphy @ 2018-08-01 18:50 UTC (permalink / raw)
To: Chris Murphy, xfs list
[-- Attachment #1: Type: text/plain, Size: 2415 bytes --]
On Wed, Aug 1, 2018, 5:45 AM Carlos Maiolino <cmaiolino@redhat.com> wrote:
> On Tue, Jul 31, 2018 at 11:04:05PM -0600, Chris Murphy wrote:
> > On Tue, Jul 31, 2018 at 10:41 PM, Eric Sandeen <sandeen@sandeen.net>
> wrote:
> > >
> > >
> > > On 7/31/18 11:38 PM, Chris Murphy wrote:
> > >> On Tue, Jul 31, 2018 at 10:12 PM, Dave Chinner <david@fromorbit.com>
> wrote:
> > >>> On Tue, Jul 31, 2018 at 07:32:50PM -0600, Chris Murphy wrote:
> > >>>> This seems suboptimal.
> > >>>
> > >>> It's actually a very useful optimisation to make on thin devices.
> > >>>
> > >>>> Basically this is a 750G thin volume. I don't
> > >>>> have a plain partition on a device handy to try this out but I'm
> > >>>> pretty certain the default is 4 AG's in that case, so I'm confused
> why
> > >>>> by default 33 AGs are created on a thin volume. The LVM volume group
> > >>>> is on a dmcrypt PV.
> > >>>
> > >>> It's a thin volume, therefore it advertises an optimal IO size and
> > >>> alignment setting (i.e. the thin volume allocation chunk size).
> > >>> Hence mkfs.xfs treats it as a "multi-disk device" and sets up
> > >>> alignment and AG count appropriately.
> > >>>
> > >>> This is actually the right optimisation to make for sparse devices -
> > >>> more AGs increase filesystem concurrency but we normally restrict it
> > >>> on single spindles because each AG adds more seeks into typical
> > >>> workloads and slows them down. However, the thin volume already adds
> > >>> that penalty to the storage stack for us because they don't have a
> > >>> linear LBA-to-physical location characteristic. Hence we can
> > >>> increase filesystem concurrency without addition performance
> > >>> penalties being incurred.
> > >>
> > >> OK so why 33 AG's with xfsprogs 4.15, but 4 AG's with xfsprogs 4.17,
> > >> when directed to the same thin LV? And also the difference in sunit
> > >> and swidth?
> > >>
> > >
> > > Did you build 4.17 with --disable-blkid?
> >
> > Yep, that's what doc/INSTALL recommends. Removing that and rebuilding,
> > sure enough 33 AGs, sunit 128, swidth 1024, and sectsz 4096. I do get
> > a build time warning.
>
> Where did you see that --disable-blkid is recommended? Are you using MacOS
> to
> build it?
>
In the install instructions under the Mac OS X section. The Linux and Mac
OS X sections are obviously titled, nevertheless I totally missed those
titles.
Sorry for the noise.
Chris Murphy
>
[-- Attachment #2: Type: text/html, Size: 3577 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2018-08-01 20:38 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-01 1:32 agcount 33 by default for a single HDD? Chris Murphy
2018-08-01 2:01 ` Chris Murphy
2018-08-01 4:12 ` Dave Chinner
2018-08-01 4:38 ` Chris Murphy
2018-08-01 4:41 ` Eric Sandeen
2018-08-01 5:04 ` Chris Murphy
2018-08-01 5:05 ` Eric Sandeen
2018-08-01 11:45 ` Carlos Maiolino
2018-08-01 18:50 ` Chris Murphy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.