Modern uses of CONFIG_XFS_RT

Modern uses of CONFIG_XFS_RT

[Luis Chamberlain]

I hear some folks still use CONFIG_XFS_RT, I was curious what was the
actual modern typical use case for it. I thought this was somewhat
realted to DAX use but upon a quick code inspection I see direct
realtionship.

  Luis

Re: Modern uses of CONFIG_XFS_RT

[Carlos Maiolino]

On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> actual modern typical use case for it. I thought this was somewhat
> realted to DAX use but upon a quick code inspection I see direct
> realtionship.

Hm, not sure if there is any other use other than it's original purpose of
reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
thing. But anyway, I don't have much experience using XFS_RT by myself, and I
probably raised more questions than answers to yours :P

Cheers

> 
>   Luis
> 

-- 
Carlos

Re: Modern uses of CONFIG_XFS_RT

[Luis Chamberlain]

On Wed, Feb 19, 2020 at 03:32:27PM +0100, Carlos Maiolino wrote:
> On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > actual modern typical use case for it. I thought this was somewhat
> > realted to DAX use but upon a quick code inspection I see direct
> > realtionship.
> 
> Hm, not sure if there is any other use other than it's original purpose of
> reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
> thing. But anyway, I don't have much experience using XFS_RT by myself, and I
> probably raised more questions than answers to yours :P

What about another question, this would certainly drive the users out of
the corners: can we remove it upstream?

  Luis

Re: Modern uses of CONFIG_XFS_RT

[Darrick J. Wong]

On Wed, Feb 19, 2020 at 02:38:24PM +0000, Luis Chamberlain wrote:
> On Wed, Feb 19, 2020 at 03:32:27PM +0100, Carlos Maiolino wrote:
> > On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > > actual modern typical use case for it. I thought this was somewhat
> > > realted to DAX use but upon a quick code inspection I see direct
> > > realtionship.
> > 
> > Hm, not sure if there is any other use other than it's original purpose of
> > reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
> > thing. But anyway, I don't have much experience using XFS_RT by myself, and I
> > probably raised more questions than answers to yours :P
> 
> What about another question, this would certainly drive the users out of
> the corners: can we remove it upstream?

My DVR and TV still use it to record video data.

I've also been pushing the realtime volume for persistent memory devices
because you can guarantee that all the expensive pmem gets used for data
storage, that the extents will always be perfectly aligned to large page
sizes, and that fs metadata will never defeat that alignment guarantee.

(Granted now they're arguing that having a separate storage device for
metadata will inflate the BOM cost unacceptably, and wouldn't it be
cheaper if we just redesigned XFS to have 2MB blocksize, but I'm not
buying that because the next thing they'll want when pmem becomes cheap
is 1GB blocksize for Big Data applications. :P)

--D

>   Luis

Re: Modern uses of CONFIG_XFS_RT

[Luis Chamberlain]

On Wed, Feb 19, 2020 at 09:09:45AM -0800, Darrick J. Wong wrote:
> On Wed, Feb 19, 2020 at 02:38:24PM +0000, Luis Chamberlain wrote:
> > On Wed, Feb 19, 2020 at 03:32:27PM +0100, Carlos Maiolino wrote:
> > > On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > > > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > > > actual modern typical use case for it. I thought this was somewhat
> > > > realted to DAX use but upon a quick code inspection I see direct
> > > > realtionship.
> > > 
> > > Hm, not sure if there is any other use other than it's original purpose of
> > > reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
> > > thing. But anyway, I don't have much experience using XFS_RT by myself, and I
> > > probably raised more questions than answers to yours :P
> > 
> > What about another question, this would certainly drive the users out of
> > the corners: can we remove it upstream?
> 
> My DVR and TV still use it to record video data.

Is anyone productizing on that though?

I was curious since most distros are disabling CONFIG_XFS_RT so I was
curious who was actually testing this stuff or caring about it.

> I've also been pushing the realtime volume for persistent memory devices
> because you can guarantee that all the expensive pmem gets used for data
> storage, that the extents will always be perfectly aligned to large page
> sizes, and that fs metadata will never defeat that alignment guarantee.

For those that *are* using XFS in production with realtime volume with dax...
I wonder whatcha doing about all these tests on fstests which we don't
have a proper way to know if the test succeeded / failed [0] when an
external logdev is used, this then applies to regular external log dev
users as well [1].

Which makes me also wonder then, what are the typical big users of the
regular external log device?

Reviewing a way to address this on fstests has been on my TODO for
a while, but it begs the question of how much do we really care first.
And that's what I was really trying to figure out.

Can / should we phase out external logdev / realtime dev? Who really is
caring about this code these days?

[0] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_realtimedev.txt
[1] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_logdev.txt

  Luis

Re: Modern uses of CONFIG_XFS_RT

[Darrick J. Wong]

On Wed, Feb 19, 2020 at 05:55:02PM +0000, Luis Chamberlain wrote:
> On Wed, Feb 19, 2020 at 09:09:45AM -0800, Darrick J. Wong wrote:
> > On Wed, Feb 19, 2020 at 02:38:24PM +0000, Luis Chamberlain wrote:
> > > On Wed, Feb 19, 2020 at 03:32:27PM +0100, Carlos Maiolino wrote:
> > > > On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > > > > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > > > > actual modern typical use case for it. I thought this was somewhat
> > > > > realted to DAX use but upon a quick code inspection I see direct
> > > > > realtionship.
> > > > 
> > > > Hm, not sure if there is any other use other than it's original purpose of
> > > > reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
> > > > thing. But anyway, I don't have much experience using XFS_RT by myself, and I
> > > > probably raised more questions than answers to yours :P
> > > 
> > > What about another question, this would certainly drive the users out of
> > > the corners: can we remove it upstream?
> > 
> > My DVR and TV still use it to record video data.
> 
> Is anyone productizing on that though?
> 
> I was curious since most distros are disabling CONFIG_XFS_RT so I was
> curious who was actually testing this stuff or caring about it.

Most != All.  We enabled it here, for development of future products.

> > I've also been pushing the realtime volume for persistent memory devices
> > because you can guarantee that all the expensive pmem gets used for data
> > storage, that the extents will always be perfectly aligned to large page
> > sizes, and that fs metadata will never defeat that alignment guarantee.
> 
> For those that *are* using XFS in production with realtime volume with dax...
> I wonder whatcha doing about all these tests on fstests which we don't
> have a proper way to know if the test succeeded / failed [0] when an
> external logdev is used, this then applies to regular external log dev
> users as well [1].

Huh?  How did we jump from realtime devices to external log files?

> Which makes me also wonder then, what are the typical big users of the
> regular external log device?
> 
> Reviewing a way to address this on fstests has been on my TODO for
> a while, but it begs the question of how much do we really care first.
> And that's what I was really trying to figure out.
> 
> Can / should we phase out external logdev / realtime dev? Who really is
> caring about this code these days?

Not many, I guess. :/

There seem to be a lot more tests these days that use dmflakey on the
data device to simulate a temporary disk failure... but those aren't
going to work for external log devices because they seem to assume that
what we call the data device is also the log device.

--D

> [0] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_realtimedev.txt
> [1] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_logdev.txt
> 
>   Luis

Re: Modern uses of CONFIG_XFS_RT

[Luis Chamberlain]

On Wed, Feb 19, 2020 at 02:01:04PM -0800, Darrick J. Wong wrote:
> On Wed, Feb 19, 2020 at 05:55:02PM +0000, Luis Chamberlain wrote:
> > On Wed, Feb 19, 2020 at 09:09:45AM -0800, Darrick J. Wong wrote:
> > > On Wed, Feb 19, 2020 at 02:38:24PM +0000, Luis Chamberlain wrote:
> > > > On Wed, Feb 19, 2020 at 03:32:27PM +0100, Carlos Maiolino wrote:
> > > > > On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > > > > > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > > > > > actual modern typical use case for it. I thought this was somewhat
> > > > > > realted to DAX use but upon a quick code inspection I see direct
> > > > > > realtionship.
> > > > > 
> > > > > Hm, not sure if there is any other use other than it's original purpose of
> > > > > reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
> > > > > thing. But anyway, I don't have much experience using XFS_RT by myself, and I
> > > > > probably raised more questions than answers to yours :P
> > > > 
> > > > What about another question, this would certainly drive the users out of
> > > > the corners: can we remove it upstream?
> > > 
> > > My DVR and TV still use it to record video data.
> > 
> > Is anyone productizing on that though?
> > 
> > I was curious since most distros are disabling CONFIG_XFS_RT so I was
> > curious who was actually testing this stuff or caring about it.
> 
> Most != All.  We enabled it here, for development of future products.

Ah great to know, thanks!

> > > I've also been pushing the realtime volume for persistent memory devices
> > > because you can guarantee that all the expensive pmem gets used for data
> > > storage, that the extents will always be perfectly aligned to large page
> > > sizes, and that fs metadata will never defeat that alignment guarantee.
> > 
> > For those that *are* using XFS in production with realtime volume with dax...
> > I wonder whatcha doing about all these tests on fstests which we don't
> > have a proper way to know if the test succeeded / failed [0] when an
> > external logdev is used, this then applies to regular external log dev
> > users as well [1].
> 
> Huh?  How did we jump from realtime devices to external log files?

They share the same problem with fstests when using an alternative log
device, which I pointed out on [0] and [1].

[0] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_realtimedev.txt
[1] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_logdev.txt

> > Which makes me also wonder then, what are the typical big users of the
> > regular external log device?
> > 
> > Reviewing a way to address this on fstests has been on my TODO for
> > a while, but it begs the question of how much do we really care first.
> > And that's what I was really trying to figure out.
> > 
> > Can / should we phase out external logdev / realtime dev? Who really is
> > caring about this code these days?
> 
> Not many, I guess. :/
> 
> There seem to be a lot more tests these days that use dmflakey on the
> data device to simulate a temporary disk failure... but those aren't
> going to work for external log devices because they seem to assume that
> what we call the data device is also the log device.

That goes to show that the fstests assumption on a shared data/log device was
not only a thing of the past, its still present, and unless we address
soon, the gap will only get bigger.

OK thanks for the feedback. The situation in terms of testing rtdev or
external logs seems actually worse than I expected given the outlook for
the future and no one seeming to really care too much right now. If the
dax folks didn't care, then the code will likely just bit rot even more.
Is it too nutty for us to consider removing it as a future goal?

  Luis

Re: Modern uses of CONFIG_XFS_RT

[Darrick J. Wong]

On Thu, Feb 20, 2020 at 12:17:29AM +0000, Luis Chamberlain wrote:
> On Wed, Feb 19, 2020 at 02:01:04PM -0800, Darrick J. Wong wrote:
> > On Wed, Feb 19, 2020 at 05:55:02PM +0000, Luis Chamberlain wrote:
> > > On Wed, Feb 19, 2020 at 09:09:45AM -0800, Darrick J. Wong wrote:
> > > > On Wed, Feb 19, 2020 at 02:38:24PM +0000, Luis Chamberlain wrote:
> > > > > On Wed, Feb 19, 2020 at 03:32:27PM +0100, Carlos Maiolino wrote:
> > > > > > On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > > > > > > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > > > > > > actual modern typical use case for it. I thought this was somewhat
> > > > > > > realted to DAX use but upon a quick code inspection I see direct
> > > > > > > realtionship.
> > > > > > 
> > > > > > Hm, not sure if there is any other use other than it's original purpose of
> > > > > > reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
> > > > > > thing. But anyway, I don't have much experience using XFS_RT by myself, and I
> > > > > > probably raised more questions than answers to yours :P
> > > > > 
> > > > > What about another question, this would certainly drive the users out of
> > > > > the corners: can we remove it upstream?
> > > > 
> > > > My DVR and TV still use it to record video data.
> > > 
> > > Is anyone productizing on that though?
> > > 
> > > I was curious since most distros are disabling CONFIG_XFS_RT so I was
> > > curious who was actually testing this stuff or caring about it.
> > 
> > Most != All.  We enabled it here, for development of future products.
> 
> Ah great to know, thanks!
> 
> > > > I've also been pushing the realtime volume for persistent memory devices
> > > > because you can guarantee that all the expensive pmem gets used for data
> > > > storage, that the extents will always be perfectly aligned to large page
> > > > sizes, and that fs metadata will never defeat that alignment guarantee.
> > > 
> > > For those that *are* using XFS in production with realtime volume with dax...
> > > I wonder whatcha doing about all these tests on fstests which we don't
> > > have a proper way to know if the test succeeded / failed [0] when an
> > > external logdev is used, this then applies to regular external log dev
> > > users as well [1].
> > 
> > Huh?  How did we jump from realtime devices to external log files?
> 
> They share the same problem with fstests when using an alternative log
> device, which I pointed out on [0] and [1].
> 
> [0] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_realtimedev.txt
> [1] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_logdev.txt
> 
> > > Which makes me also wonder then, what are the typical big users of the
> > > regular external log device?
> > > 
> > > Reviewing a way to address this on fstests has been on my TODO for
> > > a while, but it begs the question of how much do we really care first.
> > > And that's what I was really trying to figure out.
> > > 
> > > Can / should we phase out external logdev / realtime dev? Who really is
> > > caring about this code these days?
> > 
> > Not many, I guess. :/
> > 
> > There seem to be a lot more tests these days that use dmflakey on the
> > data device to simulate a temporary disk failure... but those aren't
> > going to work for external log devices because they seem to assume that
> > what we call the data device is also the log device.
> 
> That goes to show that the fstests assumption on a shared data/log device was
> not only a thing of the past, its still present, and unless we address
> soon, the gap will only get bigger.
> 
> OK thanks for the feedback. The situation in terms of testing rtdev or
> external logs seems actually worse than I expected given the outlook for
> the future and no one seeming to really care too much right now. If the
> dax folks didn't care, then the code will likely just bit rot even more.
> Is it too nutty for us to consider removing it as a future goal?

Yes.  You just jumped from "I haven't had time to triage these failing
tests" straight to ripping out code.  Analyze the failures, fix the
tests that weren't designed flexibly enough to handle, and we'll discuss
the ones that aren't easily changed.

--D

>   Luis

Re: Modern uses of CONFIG_XFS_RT

[Eric Sandeen]

On 2/19/20 6:17 PM, Luis Chamberlain wrote:
> On Wed, Feb 19, 2020 at 02:01:04PM -0800, Darrick J. Wong wrote:
>> On Wed, Feb 19, 2020 at 05:55:02PM +0000, Luis Chamberlain wrote:
>>> On Wed, Feb 19, 2020 at 09:09:45AM -0800, Darrick J. Wong wrote:
>>>> On Wed, Feb 19, 2020 at 02:38:24PM +0000, Luis Chamberlain wrote:
>>>>> On Wed, Feb 19, 2020 at 03:32:27PM +0100, Carlos Maiolino wrote:
>>>>>> On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
>>>>>>> I hear some folks still use CONFIG_XFS_RT, I was curious what was the
>>>>>>> actual modern typical use case for it. I thought this was somewhat
>>>>>>> realted to DAX use but upon a quick code inspection I see direct
>>>>>>> realtionship.
>>>>>>
>>>>>> Hm, not sure if there is any other use other than it's original purpose of
>>>>>> reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
>>>>>> thing. But anyway, I don't have much experience using XFS_RT by myself, and I
>>>>>> probably raised more questions than answers to yours :P
>>>>>
>>>>> What about another question, this would certainly drive the users out of
>>>>> the corners: can we remove it upstream?
>>>>
>>>> My DVR and TV still use it to record video data.
>>>
>>> Is anyone productizing on that though?
>>>
>>> I was curious since most distros are disabling CONFIG_XFS_RT so I was
>>> curious who was actually testing this stuff or caring about it.
>>
>> Most != All.  We enabled it here, for development of future products.
> 
> Ah great to know, thanks!
> 
>>>> I've also been pushing the realtime volume for persistent memory devices
>>>> because you can guarantee that all the expensive pmem gets used for data
>>>> storage, that the extents will always be perfectly aligned to large page
>>>> sizes, and that fs metadata will never defeat that alignment guarantee.
>>>
>>> For those that *are* using XFS in production with realtime volume with dax...
>>> I wonder whatcha doing about all these tests on fstests which we don't
>>> have a proper way to know if the test succeeded / failed [0] when an
>>> external logdev is used, this then applies to regular external log dev
>>> users as well [1].
>>
>> Huh?  How did we jump from realtime devices to external log files?
> 
> They share the same problem with fstests when using an alternative log
> device, which I pointed out on [0] and [1].
> 
> [0] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_realtimedev.txt
> [1] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_logdev.txt
> 
>>> Which makes me also wonder then, what are the typical big users of the
>>> regular external log device?
>>>
>>> Reviewing a way to address this on fstests has been on my TODO for
>>> a while, but it begs the question of how much do we really care first.
>>> And that's what I was really trying to figure out.
>>>
>>> Can / should we phase out external logdev / realtime dev? Who really is
>>> caring about this code these days?
>>
>> Not many, I guess. :/
>>
>> There seem to be a lot more tests these days that use dmflakey on the
>> data device to simulate a temporary disk failure... but those aren't
>> going to work for external log devices because they seem to assume that
>> what we call the data device is also the log device.
> 
> That goes to show that the fstests assumption on a shared data/log device was
> not only a thing of the past, its still present, and unless we address
> soon, the gap will only get bigger.
> 
> OK thanks for the feedback. The situation in terms of testing rtdev or
> external logs seems actually worse than I expected given the outlook for
> the future and no one seeming to really care too much right now. If the
> dax folks didn't care, then the code will likely just bit rot even more.
> Is it too nutty for us to consider removing it as a future goal?

Less nutty would be to analyze the failures and fix the tests.

Here's a start, I'll send this one to fstests.

diff --git a/common/repair b/common/repair
index 5a9097f4..cf69dde9 100644
--- a/common/repair
+++ b/common/repair
@@ -9,8 +9,12 @@ _zero_position()
 	value=$1
 	struct="$2"
 
+	SCRATCH_OPTIONS=""
+	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+		SCRATCH_OPTIONS="-l$SCRATCH_LOGDEV"
+
 	# set values for off/len variables provided by db
-	eval `xfs_db -r -c "$struct" -c stack $SCRATCH_DEV | perl -ne '
+	eval `xfs_db -r -c "$struct" -c stack $SCRATCH_OPTIONS $SCRATCH_DEV | perl -ne '
 		if (/byte offset (\d+), length (\d+)/) {
 			print "offset=$1\nlength=$2\n"; exit
 		}'`
diff --git a/tests/xfs/030 b/tests/xfs/030
index efdb6a18..e1cc32ef 100755
--- a/tests/xfs/030
+++ b/tests/xfs/030
@@ -77,7 +77,10 @@ else
 	_scratch_unmount
 fi
 clear=""
-eval `xfs_db -r -c "sb 1" -c stack $SCRATCH_DEV | perl -ne '
+SCRATCH_OPTIONS=""
+[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
+	SCRATCH_OPTIONS="-l$SCRATCH_LOGDEV"
+eval `xfs_db -r -c "sb 1" -c stack $SCRATCH_OPTIONS $SCRATCH_DEV | perl -ne '
 	if (/byte offset (\d+), length (\d+)/) {
 		print "clear=", $1 / 512, "\n"; exit
 	}'`

Re: Modern uses of CONFIG_XFS_RT

[Darrick J. Wong]

On Wed, Feb 19, 2020 at 08:12:23PM -0600, Eric Sandeen wrote:
> 
> 
> On 2/19/20 6:17 PM, Luis Chamberlain wrote:
> > On Wed, Feb 19, 2020 at 02:01:04PM -0800, Darrick J. Wong wrote:
> >> On Wed, Feb 19, 2020 at 05:55:02PM +0000, Luis Chamberlain wrote:
> >>> On Wed, Feb 19, 2020 at 09:09:45AM -0800, Darrick J. Wong wrote:
> >>>> On Wed, Feb 19, 2020 at 02:38:24PM +0000, Luis Chamberlain wrote:
> >>>>> On Wed, Feb 19, 2020 at 03:32:27PM +0100, Carlos Maiolino wrote:
> >>>>>> On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> >>>>>>> I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> >>>>>>> actual modern typical use case for it. I thought this was somewhat
> >>>>>>> realted to DAX use but upon a quick code inspection I see direct
> >>>>>>> realtionship.
> >>>>>>
> >>>>>> Hm, not sure if there is any other use other than it's original purpose of
> >>>>>> reducing latency jitters. Also XFS_RT dates way back from the day DAX was even a
> >>>>>> thing. But anyway, I don't have much experience using XFS_RT by myself, and I
> >>>>>> probably raised more questions than answers to yours :P
> >>>>>
> >>>>> What about another question, this would certainly drive the users out of
> >>>>> the corners: can we remove it upstream?
> >>>>
> >>>> My DVR and TV still use it to record video data.
> >>>
> >>> Is anyone productizing on that though?
> >>>
> >>> I was curious since most distros are disabling CONFIG_XFS_RT so I was
> >>> curious who was actually testing this stuff or caring about it.
> >>
> >> Most != All.  We enabled it here, for development of future products.
> > 
> > Ah great to know, thanks!
> > 
> >>>> I've also been pushing the realtime volume for persistent memory devices
> >>>> because you can guarantee that all the expensive pmem gets used for data
> >>>> storage, that the extents will always be perfectly aligned to large page
> >>>> sizes, and that fs metadata will never defeat that alignment guarantee.
> >>>
> >>> For those that *are* using XFS in production with realtime volume with dax...
> >>> I wonder whatcha doing about all these tests on fstests which we don't
> >>> have a proper way to know if the test succeeded / failed [0] when an
> >>> external logdev is used, this then applies to regular external log dev
> >>> users as well [1].
> >>
> >> Huh?  How did we jump from realtime devices to external log files?
> > 
> > They share the same problem with fstests when using an alternative log
> > device, which I pointed out on [0] and [1].
> > 
> > [0] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_realtimedev.txt
> > [1] https://github.com/mcgrof/oscheck/blob/master/expunges/linux-next-xfs/xfs/unassigned/xfs_logdev.txt
> > 
> >>> Which makes me also wonder then, what are the typical big users of the
> >>> regular external log device?
> >>>
> >>> Reviewing a way to address this on fstests has been on my TODO for
> >>> a while, but it begs the question of how much do we really care first.
> >>> And that's what I was really trying to figure out.
> >>>
> >>> Can / should we phase out external logdev / realtime dev? Who really is
> >>> caring about this code these days?
> >>
> >> Not many, I guess. :/
> >>
> >> There seem to be a lot more tests these days that use dmflakey on the
> >> data device to simulate a temporary disk failure... but those aren't
> >> going to work for external log devices because they seem to assume that
> >> what we call the data device is also the log device.
> > 
> > That goes to show that the fstests assumption on a shared data/log device was
> > not only a thing of the past, its still present, and unless we address
> > soon, the gap will only get bigger.
> > 
> > OK thanks for the feedback. The situation in terms of testing rtdev or
> > external logs seems actually worse than I expected given the outlook for
> > the future and no one seeming to really care too much right now. If the
> > dax folks didn't care, then the code will likely just bit rot even more.
> > Is it too nutty for us to consider removing it as a future goal?
> 
> Less nutty would be to analyze the failures and fix the tests.
> 
> Here's a start, I'll send this one to fstests.
> 
> diff --git a/common/repair b/common/repair
> index 5a9097f4..cf69dde9 100644
> --- a/common/repair
> +++ b/common/repair
> @@ -9,8 +9,12 @@ _zero_position()
>  	value=$1
>  	struct="$2"
>  
> +	SCRATCH_OPTIONS=""
> +	[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> +		SCRATCH_OPTIONS="-l$SCRATCH_LOGDEV"
> +
>  	# set values for off/len variables provided by db
> -	eval `xfs_db -r -c "$struct" -c stack $SCRATCH_DEV | perl -ne '
> +	eval `xfs_db -r -c "$struct" -c stack $SCRATCH_OPTIONS $SCRATCH_DEV | perl -ne '
>  		if (/byte offset (\d+), length (\d+)/) {
>  			print "offset=$1\nlength=$2\n"; exit
>  		}'`
> diff --git a/tests/xfs/030 b/tests/xfs/030
> index efdb6a18..e1cc32ef 100755
> --- a/tests/xfs/030
> +++ b/tests/xfs/030
> @@ -77,7 +77,10 @@ else
>  	_scratch_unmount
>  fi
>  clear=""
> -eval `xfs_db -r -c "sb 1" -c stack $SCRATCH_DEV | perl -ne '
> +SCRATCH_OPTIONS=""
> +[ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> +	SCRATCH_OPTIONS="-l$SCRATCH_LOGDEV"
> +eval `xfs_db -r -c "sb 1" -c stack $SCRATCH_OPTIONS $SCRATCH_DEV | perl -ne '

_scratch_xfs_db

--D

>  	if (/byte offset (\d+), length (\d+)/) {
>  		print "clear=", $1 / 512, "\n"; exit
>  	}'`
> 
> 
> 

Re: Modern uses of CONFIG_XFS_RT

[Dave Chinner]

On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> actual modern typical use case for it. I thought this was somewhat
> realted to DAX use but upon a quick code inspection I see direct
> realtionship.

Facebook use it in production systems to separate large file data
from metadata and small files. i.e. they use a small SSD based
partition for the filesytem metadata and a spinning disk for
the large scale data storage. Essentially simple teired storage.

It's also commonly still used in multi-stream DVRs (think
multi-camera security systems), and other similar sequential access
data applications...

That's just a couple off the top of my head...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Modern uses of CONFIG_XFS_RT

[Brian Foster]

On Thu, Feb 20, 2020 at 02:41:06PM +1100, Dave Chinner wrote:
> On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > actual modern typical use case for it. I thought this was somewhat
> > realted to DAX use but upon a quick code inspection I see direct
> > realtionship.
> 
> Facebook use it in production systems to separate large file data
> from metadata and small files. i.e. they use a small SSD based
> partition for the filesytem metadata and a spinning disk for
> the large scale data storage. Essentially simple teired storage.
> 

Didn't this involve custom functionality? I thought they had posted
something at one point that wasn't seen through to merge, but I could be
misremembering (or maybe that was something else RT related). It doesn't
matter that much as there are probably other users out there, but I'm
not sure this serves as a great example use case if it did require
downstream customizations that aren't going to be generalized/supported
for the community.. Richard..?

Brian

> It's also commonly still used in multi-stream DVRs (think
> multi-camera security systems), and other similar sequential access
> data applications...
> 
> That's just a couple off the top of my head...
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: Modern uses of CONFIG_XFS_RT

[Dave Chinner]

On Thu, Feb 20, 2020 at 09:25:20AM -0500, Brian Foster wrote:
> On Thu, Feb 20, 2020 at 02:41:06PM +1100, Dave Chinner wrote:
> > On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > > actual modern typical use case for it. I thought this was somewhat
> > > realted to DAX use but upon a quick code inspection I see direct
> > > realtionship.
> > 
> > Facebook use it in production systems to separate large file data
> > from metadata and small files. i.e. they use a small SSD based
> > partition for the filesytem metadata and a spinning disk for
> > the large scale data storage. Essentially simple teired storage.
> > 
> 
> Didn't this involve custom functionality? I thought they had posted
> something at one point that wasn't seen through to merge, but I could be
> misremembering (or maybe that was something else RT related). It doesn't

Yes, but that is largely irrelevant. It requires the RT device to
function, and the RT device functionality is entirely unchanged. All
that changed was the initial data allocation policy to select
whether the RT or data device would be used, and that really isn't
that controversial as we've always suggested this is a potential use
of the RT device (fast and slow storage in the one filesystem
namespace).

> matter that much as there are probably other users out there, but I'm
> not sure this serves as a great example use case if it did require
> downstream customizations

There are almost always downstream modifications in private cloud
storage kernels, even if it is just bug fixes. They aren't shipping
the code to anyone, so they don't have to publish those changes.
However, the presence of downstream changes doesn't mean the
upstreram functionality should be considered unused and can be
removed....

> that aren't going to be generalized/supported
> for the community.. Richard..?

IIRC, we were simply waiting on an updated patchset to address
review comments...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Modern uses of CONFIG_XFS_RT

[Brian Foster]

On Fri, Feb 21, 2020 at 09:06:52AM +1100, Dave Chinner wrote:
> On Thu, Feb 20, 2020 at 09:25:20AM -0500, Brian Foster wrote:
> > On Thu, Feb 20, 2020 at 02:41:06PM +1100, Dave Chinner wrote:
> > > On Wed, Feb 19, 2020 at 01:57:15PM +0000, Luis Chamberlain wrote:
> > > > I hear some folks still use CONFIG_XFS_RT, I was curious what was the
> > > > actual modern typical use case for it. I thought this was somewhat
> > > > realted to DAX use but upon a quick code inspection I see direct
> > > > realtionship.
> > > 
> > > Facebook use it in production systems to separate large file data
> > > from metadata and small files. i.e. they use a small SSD based
> > > partition for the filesytem metadata and a spinning disk for
> > > the large scale data storage. Essentially simple teired storage.
> > > 
> > 
> > Didn't this involve custom functionality? I thought they had posted
> > something at one point that wasn't seen through to merge, but I could be
> > misremembering (or maybe that was something else RT related). It doesn't
> 
> Yes, but that is largely irrelevant. It requires the RT device to
> function, and the RT device functionality is entirely unchanged. All
> that changed was the initial data allocation policy to select
> whether the RT or data device would be used, and that really isn't
> that controversial as we've always suggested this is a potential use
> of the RT device (fast and slow storage in the one filesystem
> namespace).
> 

Ok, then we should be able to get those changes upstream.

> > matter that much as there are probably other users out there, but I'm
> > not sure this serves as a great example use case if it did require
> > downstream customizations
> 
> There are almost always downstream modifications in private cloud
> storage kernels, even if it is just bug fixes. They aren't shipping
> the code to anyone, so they don't have to publish those changes.
> However, the presence of downstream changes doesn't mean the
> upstreram functionality should be considered unused and can be
> removed....
> 

Well that's not what I said. ;P I'm pointing out that as of right now
this is a downstream only use case. I know there was upstream
communication and patches posted, etc., but that was a while ago and it
wasn't clear to me if there was still intent to get things merged
upstream. If not, then the only real outcome here for anybody outside of
FB is bitrot.

> > that aren't going to be generalized/supported
> > for the community.. Richard..?
> 
> IIRC, we were simply waiting on an updated patchset to address
> review comments...
> 

I'm not sure if I'm digging out the right mails... it appears there was
decent upstream activity at first (up to a v7 [1] across a period of
months). Since then, I don't see any updates in over two years..

Brian

[1] https://lore.kernel.org/linux-xfs/20171128215527.2510350-1-rwareing@fb.com/

> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
> 

Re: Modern uses of CONFIG_XFS_RT

[Emmanuel Florac]

[-- Attachment #1: Type: text/plain, Size: 1497 bytes --]

Le Fri, 21 Feb 2020 07:15:09 -0500
Brian Foster <bfoster@redhat.com> écrivait:

> > There are almost always downstream modifications in private cloud
> > storage kernels, even if it is just bug fixes. They aren't shipping
> > the code to anyone, so they don't have to publish those changes.
> > However, the presence of downstream changes doesn't mean the
> > upstreram functionality should be considered unused and can be
> > removed....
> >   
> 
> Well that's not what I said. ;P I'm pointing out that as of right now
> this is a downstream only use case. I know there was upstream
> communication and patches posted, etc., but that was a while ago and
> it wasn't clear to me if there was still intent to get things merged
> upstream. If not, then the only real outcome here for anybody outside
> of FB is bitrot.

Maybe, maybe not. Storage tiering is a pretty hot subject, simply
shedding some light on this capability may give it more use. I didn't
know of any actual use case for RT in XFS since IRIX times, but if
there's a way to use it for tiering, this is indeed a very promising
area of development for me (and no doubt many others).

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

[-- Attachment #2: Signature digitale OpenPGP --]
[-- Type: application/pgp-signature, Size: 163 bytes --]