understanding speculative preallocation

understanding speculative preallocation

[jbr]

Hello,

I'm looking for general documentation/help with the speculative
preallocation feature in xfs.  So far, I haven't really been able to find
any definitive, up to date documentation on it.

I'm wondering how I can find out definitively which version of xfs I am
using, and what the preallocation scheme in use is.

We are running apache kafka on our servers, and kafka uses sequential io to
write data log files.  Kafka uses, by default, a maximum log file size of
1Gb.  However, most of the log files end up being 2Gb, and thus the disk
fills up twice as fast as it should.

We are using xfs on CentOS 2.6.32-358.  Is there a way I can know which
version of xfs is built into this version of the kernel?  What preallocation
schedule does it use?  If I do a xfs_info -V, it reports 3.1.1.

We are using xfs (mounted with no allocsize specified).  I've seen varying
info suggesting this means it either defaults to an allocsize of 64K (which
doesn't seem to match my observations), or that it will use dynamic
preallocation.

I've also seen hints (but no actual canonical documentation) suggesting that
the dynamic preallocation works by progressively doubling the current file
size (which does match my observations).

What I'm not clear on, is the scheduling for the preallocation. At what
point does it decide to preallocate the next doubling of space.  Is it when
the current preallocated space is used up, or does it happen when the
current space is used up within some threshold.  What I'd like to do, is
keep the doubling behavior in tact, but have it capped so it never increases
the file beyond 1Gb.  Is there a way to do that?  Can I trick the
preallocation to not do a final doubling, if I cap my kafka log files at
say, 900Mb (or some percentage under 1Gb)?

There are numerous references to an allocation schedule like this:

freespace       max prealloc size
  >5%             full extent (8GB)
  4-5%             2GB (8GB >> 2)
  3-4%             1GB (8GB >> 3)
  2-3%           512MB (8GB >> 4)
  1-2%           256MB (8GB >> 5)
  <1%            128MB (8GB >> 6)

I'm just not sure I understand what this is telling me.  It seems to tell me
what the max prealloc size is, with being reduced if the disk is nearly
full.  But it doesn't tell me about the progressive doubling in
preallocation (I assume up to a max of 8Gb).  Is any of this configurable? 
Can we specify a max prealloc size somewhere?

The other issue seems to be that after the files are closed (from within the
java jvm), they still don't seem to have their pre-allocated space
reclaimed.  Are there known issues with closing the files in java not
properly causing a flush of the preallocated space?

Any help pointing me to any documentation/user guides which accurately
describes this would be appreciated!

Thanks,

Jason



--
View this message in context: http://xfs.9218.n7.nabble.com/understanding-speculative-preallocation-tp35002.html
Sent from the Xfs - General mailing list archive at Nabble.com.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Dave Chinner]

On Fri, Jul 26, 2013 at 12:23:40AM -0700, jbr wrote:
> Hello,
> 
> I'm looking for general documentation/help with the speculative
> preallocation feature in xfs.  So far, I haven't really been able to find
> any definitive, up to date documentation on it.

Read the code - it's documented in the comments. ;)

Or ask questions here, because the code changes and the only up to
date reference is the code and/or the developers that work on it...

> I'm wondering how I can find out definitively which version of xfs I am
> using, and what the preallocation scheme in use is.

Look at the kernel version, then look at the corresponding source
code.

> We are running apache kafka on our servers, and kafka uses sequential io to
> write data log files.  Kafka uses, by default, a maximum log file size of
> 1Gb.  However, most of the log files end up being 2Gb, and thus the disk
> fills up twice as fast as it should.
> 
> We are using xfs on CentOS 2.6.32-358.  Is there a way I can know which
> version of xfs is built into this version of the kernel?

The XFS code is part of the kernel, so look at the kernel source
code that CentOS ships.

> We are using xfs (mounted with no allocsize specified).  I've seen varying
> info suggesting this means it either defaults to an allocsize of 64K (which
> doesn't seem to match my observations), or that it will use dynamic
> preallocation.
> 
> I've also seen hints (but no actual canonical documentation) suggesting that
> the dynamic preallocation works by progressively doubling the current file
> size (which does match my observations).

Well, it started off that way, but it has been refined since to
handle many different cases where this behaviour is sub-optimal.

> What I'm not clear on, is the scheduling for the preallocation. At what
> point does it decide to preallocate the next doubling of space.

Depends on the type of IO being done.

> Is it when
> the current preallocated space is used up,

Usually.

> or does it happen when the
> current space is used up within some threshold.

No.

> What I'd like to do, is
> keep the doubling behavior in tact, but have it capped so it never increases
> the file beyond 1Gb.  Is there a way to do that?

No.

> Can I trick the
> preallocation to not do a final doubling, if I cap my kafka log files at
> say, 900Mb (or some percentage under 1Gb)?

No.

> There are numerous references to an allocation schedule like this:
> 
> freespace       max prealloc size
>   >5%             full extent (8GB)
>   4-5%             2GB (8GB >> 2)
>   3-4%             1GB (8GB >> 3)
>   2-3%           512MB (8GB >> 4)
>   1-2%           256MB (8GB >> 5)
>   <1%            128MB (8GB >> 6)
> 
> I'm just not sure I understand what this is telling me.  It seems to tell me
> what the max prealloc size is, with being reduced if the disk is nearly
> full.

Yes, that's correct. Mainline also does this for quota exhaustion,
too.

> But it doesn't tell me about the progressive doubling in
> preallocation (I assume up to a max of 8Gb).  Is any of this configurable? 

No.

> Can we specify a max prealloc size somewhere?

Use the allocsize mount option. It turns off dynamic behaviour and
fixes the pre-allocation size.

> The other issue seems to be that after the files are closed (from within the
> java jvm), they still don't seem to have their pre-allocated space
> reclaimed.  Are there known issues with closing the files in java not
> properly causing a flush of the preallocated space?

Possibly. There's a heuristic that turns of truncation at close - if
your applicatin keeps doing "open-write-close" it will not truncate
preallocation. Log files typically see this IO pattern from
applications, and hence triggering that "no truncate" heuristic is
exactly what you want to have happen to avoid severe fragmentation
of the log files.

> Any help pointing me to any documentation/user guides which accurately
> describes this would be appreciated!

The mechanism is not documented outside the code as it changes from
kernel release to kernel release and supposed to be transparent to
userspace. It's being refined and optimisaed as issues are reported.
Indeed, I suspect that all your problems would disappear on mainline
due to the background removal of preallocation that is no longer
needed, and Centos doesn't have that...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Jason Rosenberg]

[-- Attachment #1.1: Type: text/plain, Size: 6488 bytes --]

Hi Dave,

Thanks for your responses.  I'm a bit confused, as I didn't see your
responses on the actual forum (only in my email inbox).

Anyway, I'm surprised that you don't have some list or other way to
correlate version history of XFS, with os release versions.  I'm guessing
the version I have is not using the latest/greatest.  We actually have
another system that uses an older version of the kernel (2.6.32-279), and
it behaves differently (it still preallocates space beyond what will ever
be used, but not by quite as much).  When we rolled out our newer machines
to 2.6.32-358, we started seeing a marked increase in disk full problems.

If, say you tell me, the mainline xfs code has improved behavior, it would
be nice to have a way to know which version of CentOS might include that?
 Telling me to read source code across multiple kernel versions sounds like
an interesting endeavor, but not something that is the most efficient use
of my time, unless there truly is no one who can easily tell me anything
about xfs version history.

Do you have any plans to have some sort of improved documentation story
around this?  This speculative preallocation behavior is truly unexpected
and not transparent to the user.   I can see that it's probably a great
performance boost (especially for something like kafka), but kafka does
have predictable log file rotation capped at fixed sizes, so it would be
great if that could be factored in.

I suppose using the allocsize setting might work in the short term.  But I
probably don't want to set allocsize to 1Gb, since that would mean every
single file created would start with that size, is that right?  Does the
allocsize setting basically work by always keeping the file size ahead of
consumed space by the allocsize amount?

Thanks,

Jason


On Fri, Jul 26, 2013 at 7:50 AM, Dave Chinner <david@fromorbit.com> wrote:

> On Fri, Jul 26, 2013 at 12:23:40AM -0700, jbr wrote:
> > Hello,
> >
> > I'm looking for general documentation/help with the speculative
> > preallocation feature in xfs.  So far, I haven't really been able to find
> > any definitive, up to date documentation on it.
>
> Read the code - it's documented in the comments. ;)
>
> Or ask questions here, because the code changes and the only up to
> date reference is the code and/or the developers that work on it...
>
> > I'm wondering how I can find out definitively which version of xfs I am
> > using, and what the preallocation scheme in use is.
>
> Look at the kernel version, then look at the corresponding source
> code.
>
> > We are running apache kafka on our servers, and kafka uses sequential io
> to
> > write data log files.  Kafka uses, by default, a maximum log file size of
> > 1Gb.  However, most of the log files end up being 2Gb, and thus the disk
> > fills up twice as fast as it should.
> >
> > We are using xfs on CentOS 2.6.32-358.  Is there a way I can know which
> > version of xfs is built into this version of the kernel?
>
> The XFS code is part of the kernel, so look at the kernel source
> code that CentOS ships.
>
> > We are using xfs (mounted with no allocsize specified).  I've seen
> varying
> > info suggesting this means it either defaults to an allocsize of 64K
> (which
> > doesn't seem to match my observations), or that it will use dynamic
> > preallocation.
> >
> > I've also seen hints (but no actual canonical documentation) suggesting
> that
> > the dynamic preallocation works by progressively doubling the current
> file
> > size (which does match my observations).
>
> Well, it started off that way, but it has been refined since to
> handle many different cases where this behaviour is sub-optimal.
>
> > What I'm not clear on, is the scheduling for the preallocation. At what
> > point does it decide to preallocate the next doubling of space.
>
> Depends on the type of IO being done.
>
> > Is it when
> > the current preallocated space is used up,
>
> Usually.
>
> > or does it happen when the
> > current space is used up within some threshold.
>
> No.
>
> > What I'd like to do, is
> > keep the doubling behavior in tact, but have it capped so it never
> increases
> > the file beyond 1Gb.  Is there a way to do that?
>
> No.
>
> > Can I trick the
> > preallocation to not do a final doubling, if I cap my kafka log files at
> > say, 900Mb (or some percentage under 1Gb)?
>
> No.
>
> > There are numerous references to an allocation schedule like this:
> >
> > freespace       max prealloc size
> >   >5%             full extent (8GB)
> >   4-5%             2GB (8GB >> 2)
> >   3-4%             1GB (8GB >> 3)
> >   2-3%           512MB (8GB >> 4)
> >   1-2%           256MB (8GB >> 5)
> >   <1%            128MB (8GB >> 6)
> >
> > I'm just not sure I understand what this is telling me.  It seems to
> tell me
> > what the max prealloc size is, with being reduced if the disk is nearly
> > full.
>
> Yes, that's correct. Mainline also does this for quota exhaustion,
> too.
>
> > But it doesn't tell me about the progressive doubling in
> > preallocation (I assume up to a max of 8Gb).  Is any of this
> configurable?
>
> No.
>
> > Can we specify a max prealloc size somewhere?
>
> Use the allocsize mount option. It turns off dynamic behaviour and
> fixes the pre-allocation size.
>
> > The other issue seems to be that after the files are closed (from within
> the
> > java jvm), they still don't seem to have their pre-allocated space
> > reclaimed.  Are there known issues with closing the files in java not
> > properly causing a flush of the preallocated space?
>
> Possibly. There's a heuristic that turns of truncation at close - if
> your applicatin keeps doing "open-write-close" it will not truncate
> preallocation. Log files typically see this IO pattern from
> applications, and hence triggering that "no truncate" heuristic is
> exactly what you want to have happen to avoid severe fragmentation
> of the log files.
>
> > Any help pointing me to any documentation/user guides which accurately
> > describes this would be appreciated!
>
> The mechanism is not documented outside the code as it changes from
> kernel release to kernel release and supposed to be transparent to
> userspace. It's being refined and optimisaed as issues are reported.
> Indeed, I suspect that all your problems would disappear on mainline
> due to the background removal of preallocation that is no longer
> needed, and Centos doesn't have that...
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
>

[-- Attachment #1.2: Type: text/html, Size: 7756 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Stan Hoeppner]

On 7/26/2013 12:40 PM, Jason Rosenberg wrote:

> Anyway, I'm surprised that you don't have some list or other way to
> correlate version history of XFS, with os release versions.  I'm guessing
> the version I have is not using the latest/greatest.  We actually have
> another system that uses an older version of the kernel (2.6.32-279), and

2.6.32-279  - this is not a mainline kernel version.  This is a Red Hat
specific string describing their internal kernel release.  It has zero
correlation to any version number of anything else in the world of
mainline Linux.

> If, say you tell me, the mainline xfs code has improved behavior, it would
> be nice to have a way to know which version of CentOS might include that?

IMNSHO, CentOS is a free proprietary chrome plated dog turd.  It's
flashy on the outside and claims it's "ENTERPRISE", "just like RHEL!".
Then you crack it open and find nothing but crap inside.  So you take it
back to the store that gave it away for free and the doors are barred,
the place out of business.  The chrome has peeled off and you're stuck
with crap that difficult to use.  Every time you touch it you get dirty
in some fashion.

RHEL is a proprietary solid chrome turd you pay for.  You can't get to
the inside, but if you find a scratch and 'return' it Red Hat will say
"we can help you fix that".

If you avoid the flashy turds altogether while still plunking down no
cash, and use a distro based entirely on mainline Linux and GNU user
space source, you can get help directly from the folks who wrote the
code you're running because they know what is where.  Whether it be
Linux proper, the XFS subsystem, NFS, Samba, Postix, etc.  Such
distributions are too numerous to mention.  None of them are chrome
plated, none claim to be "just like ENTERPRISE distro X".  I tell all
users of RHEL knock offs every time I see a situation like this:

Either pay for and receive the support that's required for the
proprietary distribution you're running, or use a completely open source
distro based on mainline kernel source and GNU user space.  By using a
RHEL knock off, you're simply locking yourself into an outdated
proprietary code base for which there is no viable support option,
because so few people in the community understand the packaging of the
constituent parts of the RHEL kernels.  This is entirely intentional on
the part of Red Hat, specifically to make the life of CentOS users
painful, and rightfully so.

FYI, many of the folks on the XFS list are Red Hat employees, including
Dave.  They'll gladly assist RHEL customers here if needed.  However, to
support CentOS users, especially in your situation, they'd have to use
Red Hat Inc resources to hunt down the information related to the CentOS
kernel you have that correlates to the RHEL kernel it is copied from.
So they've just consumed Red Hat Inc resources to directly assist a free
competitor who copied their distro.

Thus there's not much incentive to assist CentOS users as they'd in
essence be taking money out of their employer's pocket.  Taken to the
extreme this results in pay cuts, possibly furloughs or pink slips, etc.

Surely this can't be the first time you've run into a free community
support issue with the CentOS kernel.  Surely what I've written isn't
news to you.  Pay Red Hat for RHEL, or switch to Debian, Ubuntu, Open
Suse, etc.  Either way you'll be able to get much better support.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "Stan Hoeppner" <stan@hardwarefreak.com>

> IMNSHO, CentOS is a free proprietary chrome plated dog turd. It's
> flashy on the outside and claims it's "ENTERPRISE", "just like RHEL!".
> Then you crack it open and find nothing but crap inside. So you take it
> back to the store that gave it away for free and the doors are barred,
> the place out of business. The chrome has peeled off and you're stuck
> with crap that difficult to use. Every time you touch it you get dirty
> in some fashion.

Don't hold back, Stan; tell us how you *really* feel.  :-)

> FYI, many of the folks on the XFS list are Red Hat employees, including
> Dave. They'll gladly assist RHEL customers here if needed. However, to
> support CentOS users, especially in your situation, they'd have to use
> Red Hat Inc resources to hunt down the information related to the CentOS
> kernel you have that correlates to the RHEL kernel it is copied from.
> So they've just consumed Red Hat Inc resources to directly assist a free
> competitor who copied their distro.

At this point, Stan, though, I believe you've forgotten the Social Contract
of Free/Open Source software: *I didn't built [most of] that*.  And 
participating in the FOSS ecosystem is the price I pay to not have had to
build that.

Red Hat, admittedly, is a much larger contributor to both the kernel, and user
space, than some other people.  And CentOS is, admittedly, much more of a 
knockoff than most other distributions.

But it would also be good to remember, here, the reason why it looked necessary
to create CentOS in the first place: Red Hat went into the commercial 
software business with RHEL, and left behind Fedora which -- I've tried --
is not really a practical solution for business use, the way, say, RH 7 was,
or SuSE 10/11 were.

There was an amazingly large amount of commerce that was big enough to
use Linux, but not big enough to pay $1200/machine/year for the privilege,
which ought to be no surprise to anyone who does this for a living.

I am myself, in fact, about to switch to CentOS, because the Release 
Configuration Manager for openSUSE has gone insane, not only replacing
sysVinit with Mr Poettering's $EXPLETIVE, but *dropping support throughout
the packaging system for sysV compatible initscripts on boot-start apps*.

Without proper notice of deprecation.

It isn't always just the code.

But to address your specific point, I don't believe that RH is going to 
base its opinion on what time its employees spend supporting the remainder
of the FOSS ecosystem on whether the relevant distro was copied from theirs,
or not -- and I will be disappointed, publicly, and vocally, if they do.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Stan Hoeppner]

On 7/26/2013 2:43 PM, Jay Ashworth wrote:

> But to address your specific point, I don't believe that RH is going to 
> base its opinion on what time its employees spend supporting the remainder
> of the FOSS ecosystem on whether the relevant distro was copied from theirs,
> or not -- and I will be disappointed, publicly, and vocally, if they do.

On 7/26/2013 8:26 PM, Dave Chinner wrote:
>
> Disclaimer: I'm the primary RHEL XFS developer, employed by Red Hat.
>
> CentOS is a rebadged RHEL product that is released for free. If you
> want bugs fixed in CentOS, then generally you are on your own. If
> you want paid support where people will fix problems you have, you
> need to pay for RHEL.
>
> You get what you pay for.


Jay, I'm awaiting your public and vocal disappointment.  BTW, Dave has
stated this position regarding CentOS many times.  I'm guessing you're
new to the XFS list (July 13 maybe) or simply missed such prior posts if
you've been lurking much longer.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "Stan Hoeppner" <stan@hardwarefreak.com>

> On 7/26/2013 2:43 PM, Jay Ashworth wrote:
> 
> > But to address your specific point, I don't believe that RH is going
> > to
> > base its opinion on what time its employees spend supporting the
> > remainder
> > of the FOSS ecosystem on whether the relevant distro was copied from
> > theirs,
> > or not -- and I will be disappointed, publicly, and vocally, if they
> > do.
> 
> On 7/26/2013 8:26 PM, Dave Chinner wrote:

In a message which, for some reason, I do not see here in my list folder;
(it wasn't in my inbox either; this seems a running thing)

> > Disclaimer: I'm the primary RHEL XFS developer, employed by Red Hat.
> >
> > CentOS is a rebadged RHEL product that is released for free. If you
> > want bugs fixed in CentOS, then generally you are on your own. If
> > you want paid support where people will fix problems you have, you
> > need to pay for RHEL.
> >
> > You get what you pay for.
> 
> Jay, I'm awaiting your public and vocal disappointment. BTW, Dave has
> stated this position regarding CentOS many times. I'm guessing you're
> new to the XFS list (July 13 maybe) or simply missed such prior posts
> if you've been lurking much longer.

I am new to the list, yes.  But -- and I can only reply to what you 
quoted here; apologies to DC if it's less than what he wrote -- I am
in fact disappointed: in Dave for the strawman reply.  No one, that I 
saw, *reported bugs or asked for support* with CentOS.

The question -- and it was coming from someone who sounded like
they had read their Raymond, Moen, and Tatham, was "how do I 
determine *which version of the XFS drivers* is in the kernel
I'm running, so that I can helpfully report my problems to the list".

If Dave thinks *that* question is out of bounds, then when I do my 
rotating reformat to recover from my recent power supply induced 
crash, XFS will not be the file system I pick off the pulldown menu
to reformat with.

As I've just had to explain to the openSUSE release configuration 
people over sysVinit and packaging, the policies of the developers
underneath the code are *much* more important to me than the code
is, or what the code does.

If the implication of Dave's comment above is that he's not interested
in bug reports from people who aren't paid customers of RH, I'm fine
with that as well; his call.  Clearly, I won't be one of those,
either; my sister isn't interested in paying $1200/yr for the OS on
her DVR.

Perhaps I'm misinterpreting one of you.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[aurfalien]

On Jul 27, 2013, at 2:00 PM, Jay Ashworth wrote:

> ----- Original Message -----
>> From: "Stan Hoeppner" <stan@hardwarefreak.com>
> 
>> On 7/26/2013 2:43 PM, Jay Ashworth wrote:
>> 
>>> But to address your specific point, I don't believe that RH is going
>>> to
>>> base its opinion on what time its employees spend supporting the
>>> remainder
>>> of the FOSS ecosystem on whether the relevant distro was copied from
>>> theirs,
>>> or not -- and I will be disappointed, publicly, and vocally, if they
>>> do.
>> 
>> On 7/26/2013 8:26 PM, Dave Chinner wrote:
> 
> In a message which, for some reason, I do not see here in my list folder;
> (it wasn't in my inbox either; this seems a running thing)
> 
>>> Disclaimer: I'm the primary RHEL XFS developer, employed by Red Hat.
>>> 
>>> CentOS is a rebadged RHEL product that is released for free. If you
>>> want bugs fixed in CentOS, then generally you are on your own. If
>>> you want paid support where people will fix problems you have, you
>>> need to pay for RHEL.
>>> 
>>> You get what you pay for.
>> 
>> Jay, I'm awaiting your public and vocal disappointment. BTW, Dave has
>> stated this position regarding CentOS many times. I'm guessing you're
>> new to the XFS list (July 13 maybe) or simply missed such prior posts
>> if you've been lurking much longer.
> 
> I am new to the list, yes.  But -- and I can only reply to what you 
> quoted here; apologies to DC if it's less than what he wrote -- I am
> in fact disappointed: in Dave for the strawman reply.  No one, that I 
> saw, *reported bugs or asked for support* with CentOS.
> 
> The question -- and it was coming from someone who sounded like
> they had read their Raymond, Moen, and Tatham, was "how do I 
> determine *which version of the XFS drivers* is in the kernel
> I'm running, so that I can helpfully report my problems to the list".
> 
> If Dave thinks *that* question is out of bounds, then when I do my 
> rotating reformat to recover from my recent power supply induced 
> crash, XFS will not be the file system I pick off the pulldown menu
> to reformat with.

Well, you'll be missing out.  Who cares what any one thinks in terms of Free vs Fee paradigms, XFS is simply a rockin FS.

Been using it since the Irix days and never really got into any optional flags during creation or mounting etc until now.

Actually, Dave and others who reply to my posts helped me understand what I've been blindly doing for years and that knowledge carries over to other FS's.

I was really intrigued by his and Linus's back and forth not to long ago, some great knowledge for sure.

This list is great so filter whats applicable and toss the rest in a bit bucket.

- aurf
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "aurfalien" <aurfalien@gmail.com>

> > If Dave thinks *that* question is out of bounds, then when I do my
> > rotating reformat to recover from my recent power supply induced
> > crash, XFS will not be the file system I pick off the pulldown menu
> > to reformat with.
> 
> Well, you'll be missing out. Who cares what any one thinks in terms of
> Free vs Fee paradigms, XFS is simply a rockin FS.

You bet; we've put dozens of TB on and off it in 9 years of running
MythTV.  And in the 2 instances where it's blown its brains out, 
Dave, in particular, has been very helpful in putting it back together;
it's a little unnerving, actually, to look the handle up after an 
IRC chat and discover you had the lead dev on the phone.

> I was really intrigued by his and Linus's back and forth not to long
> ago, some great knowledge for sure.

I was watching that; I didn't see it end.  Did it?

> This list is great so filter whats applicable and toss the rest in a
> bit bucket.

As it happens, though, My Sister's DVR is *not* the only machine with
Really Big Filesystems on it*, and it really does matter to me what the
development policies are on critical subsystems like this; hence my 
query above.

I really really *love* SuSE, after 8 years, but they've made a critical 
change I simply can't tolerate for commercial use, and I'm going to have
to pick a new distro.  (And anyone who says "well, just switch to SLES" 
either forgets that that's $1200/server/year, or forgets where Linux came
from in the first place...)

CentOS is the only thing that's anywhere close.

* ... for which I'm responsible ...

Cheers
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[aurfalien]

On Jul 27, 2013, at 6:50 PM, Jay Ashworth wrote:

> ----- Original Message -----
>> From: "aurfalien" <aurfalien@gmail.com>
> 
>>> If Dave thinks *that* question is out of bounds, then when I do my
>>> rotating reformat to recover from my recent power supply induced
>>> crash, XFS will not be the file system I pick off the pulldown menu
>>> to reformat with.
>> 
>> Well, you'll be missing out. Who cares what any one thinks in terms of
>> Free vs Fee paradigms, XFS is simply a rockin FS.
> 
> You bet; we've put dozens of TB on and off it in 9 years of running
> MythTV.  And in the 2 instances where it's blown its brains out, 
> Dave, in particular, has been very helpful in putting it back together;
> it's a little unnerving, actually, to look the handle up after an 
> IRC chat and discover you had the lead dev on the phone.
> 
>> I was really intrigued by his and Linus's back and forth not to long
>> ago, some great knowledge for sure.
> 
> I was watching that; I didn't see it end.  Did it?

Not really but I'm waiting :)

>> This list is great so filter whats applicable and toss the rest in a
>> bit bucket.
> 
> As it happens, though, My Sister's DVR is *not* the only machine with
> Really Big Filesystems on it*, and it really does matter to me what the
> development policies are on critical subsystems like this; hence my 
> query above.
> 
> I really really *love* SuSE, after 8 years, but they've made a critical 
> change I simply can't tolerate for commercial use, and I'm going to have
> to pick a new distro.  (And anyone who says "well, just switch to SLES" 
> either forgets that that's $1200/server/year, or forgets where Linux came
> from in the first place...)
> 
> CentOS is the only thing that's anywhere close.

If it RHEL/CentOS really is the only option, then perhaps get one box on RHEL and submit bugs for it.

- aurf
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "aurfalien" <aurfalien@gmail.com>

> If it RHEL/CentOS really is the only option, then perhaps get one box
> on RHEL and submit bugs for it.

Sure, but "submit bugs" was a straw man the entire time  The OP wasn't talking
about how and where to submit bugs, and neither was I.

The OP was trying to determine how he got from Distro/Kernel Version to
XFS checkout ID, so that he could *talk to the XFS devs* about his problem.

And no one had a straight answer for him, which straight answer *is*:

"You have to talk to whomever builds the kernel packages for the distro 
you're working on; they should be able to tell you which XFS {version|
release|checkout} is built into that particular kernel RPM."

>From there, we veered *very* close to "we don't talk to people unless 
they're paying Red Had customers", and I'm relatively certain that was
*not* the takeaway Dave was aiming for with us.

Those kernel packagers *should* (be able to) tell the XFS people those answers
off hand.  If I were a developer on a major kernel subsystem project where
that actually mattered, I think I'd probably get with the kernel builders
at at least the major distros and kernel.org, and get them to tell me...

but that is a pretty far cry from my even *suggesting* that the XFS devs do
so, much less -- as it feels like Stan believes I already have -- saying that
I *expect* they should do so.

To any event that I "expect" that -- I expect it as a professional system
administrator with 30 years experience and lots of practice dealing with
systems at both the micro and macro level.  

Not as any kind of "customer".

*I* *have* read my Raymond and Moen, for all the good it's doing me. :-}

But I still do have those professional SA reflexes, and they still inform
the way that I make strategic decisions and choices about distros and 
their various components.  And not all of those decisions are strictly
technological.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Purpose of the XFS list -- was: A short digression on FOSS (Re: understanding speculative preallocation)

[Stan Hoeppner]

On 7/27/2013 9:21 PM, Jay Ashworth wrote:
...
> From there, we veered *very* close to "we don't talk to people unless 
> they're paying Red Had customers"...

Not even close.  We don't typically talk to RHEL customers on this
mailing list Jay.  RHEL customers get their XFS support via Red Hat
Customer Support.  Let's cut through all the BS, educate you, and end
this OT thread.  Pay attention.

The XFS mailing list exists for _mainline Linux_ XFS development,
discussion, collaboration, and user support.  The list is hosted by SGI,
as well as the primary list archive, the master branch repo, etc.  Red
Hat now employs, has for some years, most/all of the core XFS
developers, many of whom previously worked for SGI.  Their participation
on the XFS list is the same as the purpose of the XFS list itself, as
stated above.  The work they do for the RHEL kernel isn't discussed here
and the RHEL kernel isn't generally supported here because it is not a
mainline kernel, and because those users should be assisted by Red Hat
Support--after all they pay for that.

CentOS is a direct clone of RHEL.  It uses the RHEL kernel.  Thus XFS in
the CentOS kernel is not generally supported on this mailing list.
CentOS (and RHEL) userland XFS tools yes, kernel no.  If there were some
other custom distro kernel out there that was non mainline, it wouldn't
get support here either.

The reason "we" (and I'm not a Red Hat employee, nor a RHEL user)
suggest to CentOS users that they become paying RHEL customers is
obvious:  they already use the software and are familiar and presumably
comfortable with it and prefer it.  If they pay they get Red Hat support
for XFS in the kernel, along with anything else kernel related.  If they
want to continue to use a "no cost" OS and get support on this mailing
list, they must have a distro with a mainline kernel.  So the other
obvious option is move to a distro that uses a mainline kernel.  They
may also be able to install a mainline kernel on CentOS, as someone
suggested previously.

Nobody here is aiming guns at CentOS users heads.  They have plenty of
support options.  It just so happens that using the stock CentOS (RHEL)
kernel isn't one of them.

-- 
Stan



_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Purpose of the XFS list -- was: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "Stan Hoeppner" <stan@hardwarefreak.com>

> CentOS is a direct clone of RHEL. It uses the RHEL kernel. Thus XFS in
> the CentOS kernel is not generally supported on this mailing list.
> CentOS (and RHEL) userland XFS tools yes, kernel no. If there were some
> other custom distro kernel out there that was non mainline, it wouldn't
> get support here either.

Like, um, SuSE.  Which hasn't used "mainline kernels" in years, if ever.

What, exactly, Stan, *is* a mainline kernel?  kernel.org kernels?

No distribution worth its oats is using those; it isn't just CentOS.

> The reason "we" (and I'm not a Red Hat employee, nor a RHEL user)
> suggest to CentOS users that they become paying RHEL customers is
> obvious: they already use the software and are familiar and presumably
> comfortable with it and prefer it. If they pay they get Red Hat support
> for XFS in the kernel, along with anything else kernel related. If they
> want to continue to use a "no cost" OS and get support on this mailing
> list, they must have a distro with a mainline kernel. So the other
> obvious option is move to a distro that uses a mainline kernel. They
> may also be able to install a mainline kernel on CentOS, as someone
> suggested previously.

Or, they could use any other distro that isn't RHEL.

Or, well, SuSE, if your assertion is actually accurate, which I think it's
not, clearly.

> Nobody here is aiming guns at CentOS users heads. They have plenty of
> support options. It just so happens that using the stock CentOS (RHEL)
> kernel isn't one of them.

Ok.  Got it.  Then, since CentOS is the only realistic non-$1200/server/yr
distro available, I won't be using XFS anymore.

Note that you are pretty wildly violating the spirit of FOSS here.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Purpose of the XFS list -- was: A short digression on FOSS (Re: understanding speculative preallocation)

[Emmanuel Florac]

Le Sun, 28 Jul 2013 11:45:54 -0400 (EDT) vous écriviez:

> No distribution worth its oats is using those; it isn't just CentOS.
> 

Slackware does. Arch does. 

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Stefan Ring]

> "You have to talk to whomever builds the kernel packages for the distro
> you're working on; they should be able to tell you which XFS {version|
> release|checkout} is built into that particular kernel RPM."

For what it's worth, you can try to pry this info out of Oracle's
RedPatch <https://oss.oracle.com/projects/RedPatch/>, which actually
doesn't look all that active anymore recently, but given that it's
Oracle, one cannot seriously be surprised about anything.

The thing is, that RHEL XFS and mainline XFS seem to have drifted
apart a good deal, and as result, for an outsider, it is very
difficult to correlate patches between the two branches.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

> The thing is, that RHEL XFS and mainline XFS seem to have drifted
> apart a good deal, and as result, for an outsider, it is very
> difficult to correlate patches between the two branches.

Exactly why I recommended the OP talk to the appropriate kernel packagers.

How my clariyfing that turned into my being an entitled snobby jerk is
entirely unclear to me.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Dave Chinner]

On Sun, Jul 28, 2013 at 09:18:52AM +0200, Stefan Ring wrote:
> > "You have to talk to whomever builds the kernel packages for the distro
> > you're working on; they should be able to tell you which XFS {version|
> > release|checkout} is built into that particular kernel RPM."
> 
> For what it's worth, you can try to pry this info out of Oracle's
> RedPatch <https://oss.oracle.com/projects/RedPatch/>, which actually
> doesn't look all that active anymore recently, but given that it's
> Oracle, one cannot seriously be surprised about anything.

Oh, that thing. It hasn't ben maintained because it didn't work
properly, needed lots of hand holding, and nobody actually cared.

> The thing is, that RHEL XFS and mainline XFS seem to have drifted
> apart a good deal, and as result, for an outsider, it is very
> difficult to correlate patches between the two branches.

Sure, they have diverged signficant;y as we've backported various
bits of XFs fetaures and bug fixes back into RHEL6. As have every
other distro that doesn't track mainline directly. That's why you
need to look at the kernel source package to know what code the
distro is running.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "Dave Chinner" <david@fromorbit.com>

> Sure, they have diverged signficant;y as we've backported various
> bits of XFs fetaures and bug fixes back into RHEL6. As have every
> other distro that doesn't track mainline directly. That's why you
> need to look at the kernel source package to know what code the
> distro is running.

And... all the way back to Jason's original question:

The way that you get the information out of a kernel/kernel RPM to 
determine which XFS version it's running... is?  Cause it's clearly
not obvious to either him or me.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Dave Chinner]

On Sun, Jul 28, 2013 at 08:06:32PM -0400, Jay Ashworth wrote:
> ----- Original Message -----
> > From: "Dave Chinner" <david@fromorbit.com>
> 
> > Sure, they have diverged signficant;y as we've backported various
> > bits of XFs fetaures and bug fixes back into RHEL6. As have every
> > other distro that doesn't track mainline directly. That's why you
> > need to look at the kernel source package to know what code the
> > distro is running.
> 
> And... all the way back to Jason's original question:
> 
> The way that you get the information out of a kernel/kernel RPM to 
> determine which XFS version it's running... is?  Cause it's clearly
> not obvious to either him or me.

The "version" of XFS that you are running is that of the
kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Eric Sandeen]

On 7/28/13 9:41 PM, Dave Chinner wrote:
> On Sun, Jul 28, 2013 at 08:06:32PM -0400, Jay Ashworth wrote:
>> ----- Original Message -----
>>> From: "Dave Chinner" <david@fromorbit.com>
>>
>>> Sure, they have diverged signficant;y as we've backported various
>>> bits of XFs fetaures and bug fixes back into RHEL6. As have every
>>> other distro that doesn't track mainline directly. That's why you
>>> need to look at the kernel source package to know what code the
>>> distro is running.
>>
>> And... all the way back to Jason's original question:
>>
>> The way that you get the information out of a kernel/kernel RPM to 
>> determine which XFS version it's running... is?  Cause it's clearly
>> not obvious to either him or me.
> 
> The "version" of XFS that you are running is that of the
> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.

IOWS:  There is no spoon.  ;)

-Eric

> Cheers,
> 
> Dave.
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Stan Hoeppner]

On 7/28/2013 10:12 PM, Eric Sandeen wrote:
> On 7/28/13 9:41 PM, Dave Chinner wrote:
>> On Sun, Jul 28, 2013 at 08:06:32PM -0400, Jay Ashworth wrote:
>>> ----- Original Message -----
>>>> From: "Dave Chinner" <david@fromorbit.com>
>>>
>>>> Sure, they have diverged signficant;y as we've backported various
>>>> bits of XFs fetaures and bug fixes back into RHEL6. As have every
>>>> other distro that doesn't track mainline directly. That's why you
>>>> need to look at the kernel source package to know what code the
>>>> distro is running.
>>>
>>> And... all the way back to Jason's original question:
>>>
>>> The way that you get the information out of a kernel/kernel RPM to 
>>> determine which XFS version it's running... is?  Cause it's clearly
>>> not obvious to either him or me.
>>
>> The "version" of XFS that you are running is that of the
>> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.
> 
> IOWS:  There is no spoon.  ;)

Excellent quote Eric, love the Matrix movies.

I think the problem here is a lack of understanding of the kernel
development and commit process.  AFAIK there's not a single Linux
subsystem that has a standalone version number or versioning scheme.
Not the other filesystems, not the USB subsystem, SCSI, ATA, etc.  The
version of each is simply the version of the kernel they are included in.

I've been wondering throughout this thread where anyone got the idea
that there is an XFS version that can be traced/tracked.  Maybe because
xfsprogs have a version number?  The way to track XFS changes from
kernel to kernel is to look at the commits, as others have already stated.

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "Stan Hoeppner" <stan@hardwarefreak.com>

> >>> And... all the way back to Jason's original question:
> >>>
> >>> The way that you get the information out of a kernel/kernel RPM to
> >>> determine which XFS version it's running... is? Cause it's clearly
> >>> not obvious to either him or me.
> >>
> >> The "version" of XFS that you are running is that of the
> >> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.
> >
> > IOWS: There is no spoon. ;)
> 
> Excellent quote Eric, love the Matrix movies.
> 
> I think the problem here is a lack of understanding of the kernel
> development and commit process. AFAIK there's not a single Linux
> subsystem that has a standalone version number or versioning scheme.
> Not the other filesystems, not the USB subsystem, SCSI, ATA, etc. The
> version of each is simply the version of the kernel they are included
> in.
> 
> I've been wondering throughout this thread where anyone got the idea
> that there is an XFS version that can be traced/tracked. Maybe because
> xfsprogs have a version number? The way to track XFS changes from
> kernel to kernel is to look at the commits, as others have already
> stated.

I got it from the fact that Jason was asked -- and when I was working with
Dave a couple weeks ago, *I* was asked -- "which XFS code we had".

Filesystems are one of the few kernel subsystems which "leave tracks", and
for it, therefore, *matters* which version of the code you're running,
and that's why the issue comes up.

There *is* an "XFS version", by inspection of these conversations, and
by general knowledge of how filesystem drivers work.  That it's backwards
compatible isn't really germane to the point that *there are different
versions of the code*.  If those versions matter, than that's why we
version-number things: so the people having problems and the people
providing support can have reasonable, cogent conversations where they
know what the hell they're talking about.

And yes, I *don't* know kernel build practice, cause *I don't work with
those people*.  We're sorta hoping you guys do, and therefore can tell us
*which quantity we're supposed to be trying to find*, so that we can 
answer the questions you ask us while you're trying to help us fix problems
in your code, for free.

If the question is "which GIT version was checked out to build the kernel SRPM
that made the kernel RPM I'm running from", just say that, and if necessary,
we'll go off and look. 

But let's not try to pretend that "there is no version" of the XFS kernel code,
because clearly there is.

If there's no "version number"... well, that's not something I would be bragging
about.

I really do apologize for sounding so combative about this stuff; I'm not really
trying to be, but "things have version numbers" has been best practice for 3 
decades for a reason, and I think it's a pretty good one: avoiding fights like
this.

Cheers,
-- jra

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Dave Howorth]

Jay Ashworth wrote:
> I really do apologize for sounding so combative about this stuff; I'm not really
> trying to be, but "things have version numbers" has been best practice for 3 
> decades for a reason, and I think it's a pretty good one: avoiding fights like
> this.

I think you're missing a fundamental point. If you go back and look at
what you've quoted, you've been asked e.g. for which kernel you're
running, rather than what kernel version, because a kernel doesn't
really have a version either. As you yourself have pointed out there are
package versions. There are multiple trees and within each tree somebody
may have chosen to apply or not particular patches.

So the easiest way for a non-expert to describe the kernel they're using
is most likely to name a distro and release, plus whatever updates have
been applied. A distro-expert can translate that into the general age of
the code and the commit numbers of the exact patches that have been
applied if necessary. And that's why the FAQ

http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F

asks people to report the uname -a amongst a lot of other stuff.

So arguing about semantics of whether or not there are or should be
version numbers for various things is a bit pointless. And as well as
being combative, it's also insulting to the good folks that are trying
to help you, IMHO. They've set up a reporting system to give them the
best chance of accurately diagnosing problems with minimal wasted time.
So most people who want to solve problems are happy to go along with
that procedure, instead of going out of their way to challenge it.

If you really care about this issue and believe that it is significant,
then I think you need to go away and do a bunch of quiet research until
you understand it deeply, and then come back with specific, detailed
proposals to improve it. Otherwise, it's better to let it rest, IMHO.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Keith Keller]

On 2013-07-29, Dave Chinner <david@fromorbit.com> wrote:
>
> The "version" of XFS that you are running is that of the
> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.

Can't you fairly easily replace a kernel module with a different version
using something like DKMS?  I feel like I did that with xfs a few months
ago.  If accurate, then it's theoretically possible to run a different
version of the xfs kernel module than the running kernel version.  (And
if I'm misremembering then there really is no spoon!)

--keith

-- 
kkeller@wombat.san-francisco.ca.us


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Eric Sandeen]

On Jul 28, 2013, at 10:38 PM, Keith Keller <kkeller@wombat.san-francisco.ca.us> wrote:

> On 2013-07-29, Dave Chinner <david@fromorbit.com> wrote:
>> 
>> The "version" of XFS that you are running is that of the
>> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.
> 
> Can't you fairly easily replace a kernel module with a different version
> using something like DKMS?  I feel like I did that with xfs a few months
> ago.  If accurate, then it's theoretically possible to run a different
> version of the xfs kernel module than the running kernel version.  (And
> if I'm misremembering then there really is no spoon!)
> 
In general, no.  There are a lot of moving parts that interface with the filesystem - one does not simply drop fs/xfs from, say, kernel 3.2 into a 2.6.32 kernel.

Ninjas like Dave can do it for an enterprise distro kernel if it benefits customers, but the back porting and testing is very much a nontrivial task.

-Eric


> --keith
> 
> -- 
> kkeller@wombat.san-francisco.ca.us
> 
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Keith Keller]

On 2013-07-29, Eric Sandeen <sandeen@sandeen.net> wrote:
> 
> In general, no.  There are a lot of moving parts that interface with the filesystem - one does not simply drop fs/xfs from, say, kernel 3.2 into a 2.6.32 kernel.

I apologize for the confusion, this was not what I was implying was
possible.  Let me try to be more explicit.  Unfortunately, I no longer
have a history of what I did, because I ultimately abandoned it, so my
example will be hypothetical.

The current stable kernel is 3.10.4.  Let's suppose that 3.10.5 comes
out tomorrow with some interesting patches to fs/xfs.  Is it possible
using dkms to build the 3.10.5 version of the xfs module for a running
3.10.4 kernel?  And if so, is there a way for the module to report its
own version?  There should (in theory) be much less wizardry involved in
this scenario than in the difficult scenario of porting 3.10's xfs back
to 2.6, and is more along the lines of what I remember doing a short
time back).  (To be specific, IIRC what I did was took a proposed patch
against my running kernel version, which had not yet been incorporated
in the distro kernel, and tested it by replacing the distro kernel's
module with one I built via DKMS.  But as I mentioned, I have no docs on
this, so I could be misremembering the process.)

I am not intentionally trying to be difficult.  :)  I am genuinely
just curious about the answer.  If it's "no" (or perhaps, in this
specific scenario, it's "use the dkms tools"), it still provides me with
valuable information I did not previously have.

--keith

-- 
kkeller@wombat.san-francisco.ca.us


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Eric Sandeen]

On 7/28/13 11:57 PM, Keith Keller wrote:
> On 2013-07-29, Eric Sandeen <sandeen@sandeen.net> wrote:
>>
>> In general, no. There are a lot of moving parts that interface with
>> the filesystem - one does not simply drop fs/xfs from, say, kernel
>> 3.2 into a 2.6.32 kernel.
> 
> I apologize for the confusion, this was not what I was implying was
> possible.  Let me try to be more explicit.  Unfortunately, I no longer
> have a history of what I did, because I ultimately abandoned it, so my
> example will be hypothetical.
> 
> The current stable kernel is 3.10.4.  Let's suppose that 3.10.5 comes
> out tomorrow with some interesting patches to fs/xfs.  Is it possible
> using dkms to build the 3.10.5 version of the xfs module for a running
> 3.10.4 kernel?  

"Probably / Maybe"

It really depends on what changed from 3.10.4 to 3.10.5, but odds are,
kernel interfaces did not change, so - probably fine.  If not, you
get to keep all the pieces, etc.

> And if so, is there a way for the module to report its
> own version?

Say it with me: there is no xfs module version.  :)

The "module version" is inherited from the kernel it's built against.

$ modinfo xfs
...
vermagic:       2.6.32-279.22.1.el6.x86_64 SMP mod_unload modversions 

> There should (in theory) be much less wizardry involved in
> this scenario than in the difficult scenario of porting 3.10's xfs back
> to 2.6, and is more along the lines of what I remember doing a short
> time back).  (To be specific, IIRC what I did was took a proposed patch
> against my running kernel version, which had not yet been incorporated
> in the distro kernel, and tested it by replacing the distro kernel's
> module with one I built via DKMS.  But as I mentioned, I have no docs on
> this, so I could be misremembering the process.)

Yeah, short version hops are more likely to be ok.

And taking kernel version X's xfs, and applying a bugfix patch, and
rebuilding it against the same kernel headers should be fine.  Still
a little wizardry, but not bad for a kernel-savvy person.

> I am not intentionally trying to be difficult.  :)  I am genuinely
> just curious about the answer.  If it's "no" (or perhaps, in this
> specific scenario, it's "use the dkms tools"), it still provides me with
> valuable information I did not previously have.

Sure, I don't think you're being difficult.

The further you go off the reservation, the less tested things are, and
the less likely they are to work.  Building a tweaked, same-era module
against a slightly different kernel is likely to be fine; it's when
you get more & more changed / moving parts that it becomes trickier.

But you need to know enough to know what you're changing and/or what
has changed in the kernel, to know if what you're doing is completely
safe, probably safe, or unlikely to be safe...

-Eric

> --keith
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Keith Keller]

On 2013-07-29, Eric Sandeen <sandeen@sandeen.net> wrote:
> On 7/28/13 11:57 PM, Keith Keller wrote:
>> 
>> The current stable kernel is 3.10.4.  Let's suppose that 3.10.5 comes
>> out tomorrow with some interesting patches to fs/xfs.  Is it possible
>> using dkms to build the 3.10.5 version of the xfs module for a running
>> 3.10.4 kernel?  
>
> "Probably / Maybe"
>
> It really depends on what changed from 3.10.4 to 3.10.5, but odds are,
> kernel interfaces did not change, so - probably fine.  If not, you
> get to keep all the pieces, etc.

Sure.  :)

>> And if so, is there a way for the module to report its
>> own version?
>
> Say it with me: there is no xfs module version.  :)

Well, wouldn't it be the same as the original kernel from which the code
was ripped?  So in the above hypothetical, one could say that the xfs
"version" is 3.10.5.  It's not *exactly*, of course, but if I were to
say "I'm running kernel 3.10.4 with xfs built from 3.10.5 using DKMS"
you'd probably know what I meant.  You might then choose not to answer
questions because I've gone too far off the reservation.

Or, maybe, a different way of saying this is that the XFS version always
matches the kernel version it comes from.  Then if you use code from one
kernel version in a different version it's at your own risk; one of the
consequences is that you need to document this discrepancy yourself.

> And taking kernel version X's xfs, and applying a bugfix patch, and
> rebuilding it against the same kernel headers should be fine.  Still
> a little wizardry, but not bad for a kernel-savvy person.

Is that all it takes to be considered "wizardry"?  Back In My Day (TM)
you had to mess with .config in order to get anything done!  ;-)

--keith

-- 
kkeller@wombat.san-francisco.ca.us


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "Dave Chinner" <david@fromorbit.com>

> The "version" of XFS that you are running is that of the
> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.

Those aren't kernel versions; those are kernel *package* versions.

Kernel versions are w.x.y or w.x.y.z.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "Jay Ashworth" <jra@baylink.com>

> > From: "Dave Chinner" <david@fromorbit.com>
> 
> > The "version" of XFS that you are running is that of the
> > kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.
> 
> Those aren't kernel versions; those are kernel *package* versions.
> 
> Kernel versions are w.x.y or w.x.y.z.

And as a further followup: it matters which kernel tree you're talking about
too.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Eric Sandeen]

On 7/29/13 9:24 AM, Jay Ashworth wrote:
> ----- Original Message -----
>> From: "Dave Chinner" <david@fromorbit.com>
> 
>> The "version" of XFS that you are running is that of the
>> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.
> 
> Those aren't kernel versions; those are kernel *package* versions.

Those are RHEL kernel version numbers, which 100% uniquely identify
the code contained in those kernels.

> Kernel versions are w.x.y or w.x.y.z.

^Upstream.

-Eric

> Cheers,
> -- jra
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Jay Ashworth]

----- Original Message -----
> From: "Eric Sandeen" <sandeen@sandeen.net>

> >> From: "Dave Chinner" <david@fromorbit.com>
> >
> >> The "version" of XFS that you are running is that of the
> >> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.
> >
> > Those aren't kernel versions; those are kernel *package* versions.
> 
> Those are RHEL kernel version numbers, which 100% uniquely identify
> the code contained in those kernels.

So how, Eric, would that help, say, SuSE users -- which the XFS website makes 
special note to point out that there's a specific agreement in place to 
support.  Even SLES users vice openSUSE, though the XFS.org website doesn't 
make that distinction.

I'm sticking with "those are kernel package versions", and I was yelled
at the other day because I was interested in things that weren't "mainline
kernel versions".  RHEL kernels are the *best available example* of "not
a mainline kernel version", so I find these conflicting reports most
conflicting.

> > Kernel versions are w.x.y or w.x.y.z.
> 
> ^Upstream.

"Mainline".  Which was not my choice of term.

Cheers,
-- jra
-- 
Jay R. Ashworth                  Baylink                       jra@baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover DII
St Petersburg FL USA               #natog                      +1 727 647 1274

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Eric Sandeen]

On 7/29/13 10:30 AM, Jay Ashworth wrote:
> ----- Original Message -----
>> From: "Eric Sandeen" <sandeen@sandeen.net>
> 
>>>> From: "Dave Chinner" <david@fromorbit.com>
>>>
>>>> The "version" of XFS that you are running is that of the
>>>> kernel you are running. i.e. 2.6.32-279.x.y or 2.6.32-358.x.y.
>>>
>>> Those aren't kernel versions; those are kernel *package* versions.
>>
>> Those are RHEL kernel version numbers, which 100% uniquely identify
>> the code contained in those kernels.
> 
> So how, Eric, would that help, say, SuSE users -- 

Well, it doesn't... SuSE helps SuSE users.  And I'm not being glib,
that's just how it works.

> which the XFS website makes 
> special note to point out that there's a specific agreement in place to 
> support.  Even SLES users vice openSUSE, though the XFS.org website doesn't 
> make that distinction.

xfs.org is a public wiki run by a 3rd party, so I can't speak to the accuracy
of statements made about support agreements between SGI & SuSE, if any.

So I guess I have no useful insight there.

> I'm sticking with "those are kernel package versions", and I was yelled
> at the other day because I was interested in things that weren't "mainline
> kernel versions".  RHEL kernels are the *best available example* of "not
> a mainline kernel version", so I find these conflicting reports most
> conflicting.

You're welcome to be interested all you want, but availability of support
from either SGI (not related to CentOS at all), upstream (focused on new work,
not older distros), or Red Hat (who sells support, vs. gives it away, in
order to pay the people to keep maintaining the distro that CentOS is
more than welcome to pick up, rebuild, & ship, and support or not as *they* 
see fit).

You'll find that you usually get very good support for XFS issues & questions
related to near-current mainline kernels.  But people who can spend time
& effort supporting free distros containing custom codebases are going
to be few & far between.

The closer you get to mainline, the more willing & able the developers
will be to help you out.  The further you get into custom distro kernels,
the more it becomes the responsibility of that distro - CentOS, in your case,
and they pretty explicitly tell you what you can expect for support from
them.

I'm not trying to be flippant or glib or antagonistic here, that's just
the way things are structured; you chose CentOS, and "CentOS is designed for
people who need an enterprise class OS without the cost or support of
the prominent North American Enterprise Linux vendor."

>>> Kernel versions are w.x.y or w.x.y.z.
>>
>> ^Upstream.
> 
> "Mainline".  Which was not my choice of term.

(Ok, mainline; I was just differentiating between RHEL & Linux.)

-Eric


> Cheers,
> -- jra
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Dave Chinner]

On Sat, Jul 27, 2013 at 10:21:34PM -0400, Jay Ashworth wrote:
> ----- Original Message -----
> > From: "aurfalien" <aurfalien@gmail.com>
> 
> > If it RHEL/CentOS really is the only option, then perhaps get one box
> > on RHEL and submit bugs for it.
> 
> Sure, but "submit bugs" was a straw man the entire time  The OP wasn't talking
> about how and where to submit bugs, and neither was I.
> 
> The OP was trying to determine how he got from Distro/Kernel Version to
> XFS checkout ID, so that he could *talk to the XFS devs* about his problem.
> 
> And no one had a straight answer for him, which straight answer *is*:
> 
> "You have to talk to whomever builds the kernel packages for the distro 
> you're working on; they should be able to tell you which XFS {version|
> release|checkout} is built into that particular kernel RPM."

I gave a straight answer: you have to look at the kernel source
package that ships with the distro.

> From there, we veered *very* close to "we don't talk to people unless 
> they're paying Red Had customers", and I'm relatively certain that was
> *not* the takeaway Dave was aiming for with us.

Yup, nothing quite like geting quoted out of context, is there?
I said CentOS is downstream of RHEL, and so the only way to
influence it's content in terms of bug fixes and documentation is to
get changes into RHEL. And getting changes and documentation into
RHEL requires $$$.....

Fundamentally, I don't care what distro you use. If you've got a
bug, I'll try to find it and fix it. That's my responsibility to the
wider XFS community. If you want me to implement or document
something for you, then you need to provide some incentive for me to
do that. Red Hat provides a lot of incentive for me - they pay my
bills.

But asking me to document something already documented in code and
git, for multiple different code bases? What's my incentive? There
isn't any from a RH POV, nor is there any from a monetary POV. And
it]ll be boring work, so there's no intellectual incentive, either.
So, tell me, why should I do what you ask?

That's the down side to open-source software - if nobody wants to
scratch the same itch as you need scratched, then only you can
scratch it. Hell, if you really wanted to be useful, you'd go and
document what you know already and propose it for inclusion into the
kernel documentation. Then we'd review it, you'd correct it, and you
would have improved the state of the code for everyone.

That's how open source software works. Not by demanding other people
do something for you, but doing stuff yourself and providing the
fruits of your labour to the wider benefit of the community.

> If I were a developer on a major kernel subsystem project where
> that actually mattered, I think I'd probably get with the kernel builders
> at at least the major distros and kernel.org, and get them to tell me...

I think you'll find in that case most kernel developers say "please
reproduce on a current, unmodified mainline kernel" and so don't
have to care one bit about what is in the distro kernel. Then they
find and fix the bug in the mainline kernel - getting that fix onto
the user's machines is the user's problem....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: A short digression on FOSS (Re: understanding speculative preallocation)

[Michael L. Semon]

On 07/27/2013 05:00 PM, Jay Ashworth wrote:
> ----- Original Message -----
>> From: "Stan Hoeppner" <stan@hardwarefreak.com>
> 
>> On 7/26/2013 2:43 PM, Jay Ashworth wrote:
>>
>>> But to address your specific point, I don't believe that RH is going
>>> to
>>> base its opinion on what time its employees spend supporting the
>>> remainder
>>> of the FOSS ecosystem on whether the relevant distro was copied from
>>> theirs,
>>> or not -- and I will be disappointed, publicly, and vocally, if they
>>> do.
>>
>> On 7/26/2013 8:26 PM, Dave Chinner wrote:
> 
> In a message which, for some reason, I do not see here in my list folder;
> (it wasn't in my inbox either; this seems a running thing)
> 
>>> Disclaimer: I'm the primary RHEL XFS developer, employed by Red Hat.
>>>
>>> CentOS is a rebadged RHEL product that is released for free. If you
>>> want bugs fixed in CentOS, then generally you are on your own. If
>>> you want paid support where people will fix problems you have, you
>>> need to pay for RHEL.
>>>
>>> You get what you pay for.
>>
>> Jay, I'm awaiting your public and vocal disappointment. BTW, Dave has
>> stated this position regarding CentOS many times. I'm guessing you're
>> new to the XFS list (July 13 maybe) or simply missed such prior posts
>> if you've been lurking much longer.
> 
> I am new to the list, yes.  But -- and I can only reply to what you 
> quoted here; apologies to DC if it's less than what he wrote -- I am
> in fact disappointed: in Dave for the strawman reply.  No one, that I 
> saw, *reported bugs or asked for support* with CentOS.
> 
> The question -- and it was coming from someone who sounded like
> they had read their Raymond, Moen, and Tatham, was "how do I 
> determine *which version of the XFS drivers* is in the kernel
> I'm running, so that I can helpfully report my problems to the list".
> 
> If Dave thinks *that* question is out of bounds, then when I do my 
> rotating reformat to recover from my recent power supply induced 
> crash, XFS will not be the file system I pick off the pulldown menu
> to reformat with.
> 
> As I've just had to explain to the openSUSE release configuration 
> people over sysVinit and packaging, the policies of the developers
> underneath the code are *much* more important to me than the code
> is, or what the code does.
> 
> If the implication of Dave's comment above is that he's not interested
> in bug reports from people who aren't paid customers of RH, I'm fine
> with that as well; his call.  Clearly, I won't be one of those,
> either; my sister isn't interested in paying $1200/yr for the OS on
> her DVR.
> 
> Perhaps I'm misinterpreting one of you.
> 
> Cheers,
> -- jra

Dave's always been perfectly nice to me and handled my bug reports 
professionally, even though I use Slackware and not RHEL.  Basically, 
don't be a selfish, entitled, snobby jerk and then expect Dave to fix 
something for you at no charge.  For him to do so would be slavery, and 
that is against the founding principles of free software.

Michael

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Jason Rosenberg]

[-- Attachment #1.1: Type: text/plain, Size: 4143 bytes --]

Hi Stan,

Thanks for the info (most of it was, in fact, news to me).  I'm an
application developer trying to debug a disk space problem, that's all.  So
far, I've tracked it down to being an XFS issue.

So you are saying there's no public information that can correlate XFS
versioning to CentOS (or RHEL) versioning?

Sad state of affairs.

If anyone can volunteer this info (if available to you) I'd be much
appreciative.

Regardless, is there a version history for XFS vis-a-via mainline Linux?

Thanks,

Jason


On Fri, Jul 26, 2013 at 3:27 PM, Stan Hoeppner <stan@hardwarefreak.com>wrote:

> On 7/26/2013 12:40 PM, Jason Rosenberg wrote:
>
> > Anyway, I'm surprised that you don't have some list or other way to
> > correlate version history of XFS, with os release versions.  I'm guessing
> > the version I have is not using the latest/greatest.  We actually have
> > another system that uses an older version of the kernel (2.6.32-279), and
>
> 2.6.32-279  - this is not a mainline kernel version.  This is a Red Hat
> specific string describing their internal kernel release.  It has zero
> correlation to any version number of anything else in the world of
> mainline Linux.
>
> > If, say you tell me, the mainline xfs code has improved behavior, it
> would
> > be nice to have a way to know which version of CentOS might include that?
>
> IMNSHO, CentOS is a free proprietary chrome plated dog turd.  It's
> flashy on the outside and claims it's "ENTERPRISE", "just like RHEL!".
> Then you crack it open and find nothing but crap inside.  So you take it
> back to the store that gave it away for free and the doors are barred,
> the place out of business.  The chrome has peeled off and you're stuck
> with crap that difficult to use.  Every time you touch it you get dirty
> in some fashion.
>
> RHEL is a proprietary solid chrome turd you pay for.  You can't get to
> the inside, but if you find a scratch and 'return' it Red Hat will say
> "we can help you fix that".
>
> If you avoid the flashy turds altogether while still plunking down no
> cash, and use a distro based entirely on mainline Linux and GNU user
> space source, you can get help directly from the folks who wrote the
> code you're running because they know what is where.  Whether it be
> Linux proper, the XFS subsystem, NFS, Samba, Postix, etc.  Such
> distributions are too numerous to mention.  None of them are chrome
> plated, none claim to be "just like ENTERPRISE distro X".  I tell all
> users of RHEL knock offs every time I see a situation like this:
>
> Either pay for and receive the support that's required for the
> proprietary distribution you're running, or use a completely open source
> distro based on mainline kernel source and GNU user space.  By using a
> RHEL knock off, you're simply locking yourself into an outdated
> proprietary code base for which there is no viable support option,
> because so few people in the community understand the packaging of the
> constituent parts of the RHEL kernels.  This is entirely intentional on
> the part of Red Hat, specifically to make the life of CentOS users
> painful, and rightfully so.
>
> FYI, many of the folks on the XFS list are Red Hat employees, including
> Dave.  They'll gladly assist RHEL customers here if needed.  However, to
> support CentOS users, especially in your situation, they'd have to use
> Red Hat Inc resources to hunt down the information related to the CentOS
> kernel you have that correlates to the RHEL kernel it is copied from.
> So they've just consumed Red Hat Inc resources to directly assist a free
> competitor who copied their distro.
>
> Thus there's not much incentive to assist CentOS users as they'd in
> essence be taking money out of their employer's pocket.  Taken to the
> extreme this results in pay cuts, possibly furloughs or pink slips, etc.
>
> Surely this can't be the first time you've run into a free community
> support issue with the CentOS kernel.  Surely what I've written isn't
> news to you.  Pay Red Hat for RHEL, or switch to Debian, Ubuntu, Open
> Suse, etc.  Either way you'll be able to get much better support.
>
> --
> Stan
>
>

[-- Attachment #1.2: Type: text/html, Size: 5123 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Ben Myers]

Hi Jason,

On Fri, Jul 26, 2013 at 04:38:21PM -0400, Jason Rosenberg wrote:
> Thanks for the info (most of it was, in fact, news to me).  I'm an
> application developer trying to debug a disk space problem, that's all.  So
> far, I've tracked it down to being an XFS issue.

The speculative block reservations have been an issue for awhile.  You are not
the first person to take issue with it.

> Regardless, is there a version history for XFS vis-a-via mainline Linux?

You can find a full version history for XFS back to 2.6.12 or so here:
http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/xfs.git;a=summary

If you're interested in going older than that look here:
http://oss.sgi.com/cgi-bin/gitweb.cgi?p=archive/xfs-import.git;a=summary

The function you'll most likely want to track is xfs_iomap_write_delay, which
calls xfs_iomap_eof_want_preallocate, both of which are in fs/xfs/xfs_iomap.c.

Recently Brian Foster added a scanner to remove the speculative block
reservations on a timer which may give you some relief.  See
xfs_queue_eofblocks in fs/xfs/xfs_icache.c

Regards,
	Ben

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Jason Rosenberg]

[-- Attachment #1.1: Type: text/plain, Size: 1245 bytes --]

Thanks Ben,

This is helpful.


On Fri, Jul 26, 2013 at 4:50 PM, Ben Myers <bpm@sgi.com> wrote:

> Hi Jason,
>
> On Fri, Jul 26, 2013 at 04:38:21PM -0400, Jason Rosenberg wrote:
> > Thanks for the info (most of it was, in fact, news to me).  I'm an
> > application developer trying to debug a disk space problem, that's all.
>  So
> > far, I've tracked it down to being an XFS issue.
>
> The speculative block reservations have been an issue for awhile.  You are
> not
> the first person to take issue with it.
>
> > Regardless, is there a version history for XFS vis-a-via mainline Linux?
>
> You can find a full version history for XFS back to 2.6.12 or so here:
> http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/xfs.git;a=summary
>
> If you're interested in going older than that look here:
> http://oss.sgi.com/cgi-bin/gitweb.cgi?p=archive/xfs-import.git;a=summary
>
> The function you'll most likely want to track is xfs_iomap_write_delay,
> which
> calls xfs_iomap_eof_want_preallocate, both of which are in
> fs/xfs/xfs_iomap.c.
>
> Recently Brian Foster added a scanner to remove the speculative block
> reservations on a timer which may give you some relief.  See
> xfs_queue_eofblocks in fs/xfs/xfs_icache.c
>
> Regards,
>         Ben
>
>

[-- Attachment #1.2: Type: text/html, Size: 1914 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Jason Rosenberg]

[-- Attachment #1.1: Type: text/plain, Size: 1972 bytes --]

Is it safe to say that speculative preallocation will not be used if a file
is opened read-only?

It turns out that the kafka server does indeed write lots of log files, and
rotate them after they reach a max size, but never closes the files until
the app exits, or until it deletes the files.  This is because it needs to
make them available for reading, etc.   So, an obvious change for kafka
might be to close each log file after rotating, and then re-open it
read-only for consumers of the data.  Does that sound like a solution that
would pro-actively release pre-allocated storage?

Thanks,

Jason


On Fri, Jul 26, 2013 at 5:04 PM, Jason Rosenberg <jbr@squareup.com> wrote:

> Thanks Ben,
>
> This is helpful.
>
>
> On Fri, Jul 26, 2013 at 4:50 PM, Ben Myers <bpm@sgi.com> wrote:
>
>> Hi Jason,
>>
>> On Fri, Jul 26, 2013 at 04:38:21PM -0400, Jason Rosenberg wrote:
>> > Thanks for the info (most of it was, in fact, news to me).  I'm an
>> > application developer trying to debug a disk space problem, that's all.
>>  So
>> > far, I've tracked it down to being an XFS issue.
>>
>> The speculative block reservations have been an issue for awhile.  You
>> are not
>> the first person to take issue with it.
>>
>> > Regardless, is there a version history for XFS vis-a-via mainline Linux?
>>
>> You can find a full version history for XFS back to 2.6.12 or so here:
>> http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/xfs.git;a=summary
>>
>> If you're interested in going older than that look here:
>> http://oss.sgi.com/cgi-bin/gitweb.cgi?p=archive/xfs-import.git;a=summary
>>
>> The function you'll most likely want to track is xfs_iomap_write_delay,
>> which
>> calls xfs_iomap_eof_want_preallocate, both of which are in
>> fs/xfs/xfs_iomap.c.
>>
>> Recently Brian Foster added a scanner to remove the speculative block
>> reservations on a timer which may give you some relief.  See
>> xfs_queue_eofblocks in fs/xfs/xfs_icache.c
>>
>> Regards,
>>         Ben
>>
>>
>

[-- Attachment #1.2: Type: text/html, Size: 2994 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Ben Myers]

Hi Jason,

On Fri, Jul 26, 2013 at 05:11:55PM -0400, Jason Rosenberg wrote:
> Is it safe to say that speculative preallocation will not be used if a file
> is opened read-only?

The blocks will only be reserved on an appending write.

> It turns out that the kafka server does indeed write lots of log files, and
> rotate them after they reach a max size, but never closes the files until
> the app exits, or until it deletes the files.  This is because it needs to
> make them available for reading, etc.   So, an obvious change for kafka
> might be to close each log file after rotating, and then re-open it
> read-only for consumers of the data.  Does that sound like a solution that
> would pro-actively release pre-allocated storage?

An interesting idea, and I'm not quite sure.  The blocks past EOF are freed in
xfs_release on close in some circumstances, and it looks like you have a chance
to call xfs_free_eofblocks (at least in the most uptodate codebase) if you did
not use explicit preallocation (e.g. fallocate or an xfs ioctl) and did not
open it append-only.  You could reopen with read-write flags and it wouldn't
make a difference vs read-only, so long as you don't do an appending write.

Seems like it's worth a try.  Another possibility is to look into what would
happen if you do a truncate up to i_size when you're ready to stop appending to
the file.  I haven't checked that out though.

Regards,
	Ben

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Dave Chinner]

On Fri, Jul 26, 2013 at 05:11:55PM -0400, Jason Rosenberg wrote:
> Is it safe to say that speculative preallocation will not be used if a file
> is opened read-only?
> 
> It turns out that the kafka server does indeed write lots of log files, and
> rotate them after they reach a max size, but never closes the files until
> the app exits, or until it deletes the files.  This is because it needs to
> make them available for reading, etc.   So, an obvious change for kafka
> might be to close each log file after rotating, and then re-open it
> read-only for consumers of the data.  Does that sound like a solution that
> would pro-actively release pre-allocated storage?

No need - the mainline code that has a periodic background scan that
stops buildup of unused prealocation. i.e. if the file is clean for
5 minutes, then the prealloc will be removed. Hence it doesn't
matter what the application does with it - if it holds it open and
doesn't write to the file, then the prealloc will get removed. More
will be added the next time the file is written, but until then it
won't use excessive space.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Jason Rosenberg]

[-- Attachment #1.1: Type: text/plain, Size: 1948 bytes --]

Thanks Dave,

The automatic prealloc removal, if no new writes after 5 minutes, sounds
perfect for my use case.  But realistically, I'm not likely to get our org
to push/find an os update just for this purpose too easily.

So, in the meantime, the question remains, assuming I have the version I
have currently (dynamic preallocation, persists indefinitely until the file
is closed/app quits, etc.), will this idea work (e.g. close the file after
writing, then re-open read-only?).  Currently, the app does keep the files
open indefinitely long after writing has stopped, and this is of course
resulting in the preallocation persisting indefinitely.

Jason


On Fri, Jul 26, 2013 at 9:30 PM, Dave Chinner <david@fromorbit.com> wrote:

> On Fri, Jul 26, 2013 at 05:11:55PM -0400, Jason Rosenberg wrote:
> > Is it safe to say that speculative preallocation will not be used if a
> file
> > is opened read-only?
> >
> > It turns out that the kafka server does indeed write lots of log files,
> and
> > rotate them after they reach a max size, but never closes the files until
> > the app exits, or until it deletes the files.  This is because it needs
> to
> > make them available for reading, etc.   So, an obvious change for kafka
> > might be to close each log file after rotating, and then re-open it
> > read-only for consumers of the data.  Does that sound like a solution
> that
> > would pro-actively release pre-allocated storage?
>
> No need - the mainline code that has a periodic background scan that
> stops buildup of unused prealocation. i.e. if the file is clean for
> 5 minutes, then the prealloc will be removed. Hence it doesn't
> matter what the application does with it - if it holds it open and
> doesn't write to the file, then the prealloc will get removed. More
> will be added the next time the file is written, but until then it
> won't use excessive space.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
>

[-- Attachment #1.2: Type: text/html, Size: 2573 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Dave Chinner]

On Sat, Jul 27, 2013 at 10:19:17PM -0400, Jason Rosenberg wrote:
> Thanks Dave,
> 
> The automatic prealloc removal, if no new writes after 5 minutes, sounds
> perfect for my use case.  But realistically, I'm not likely to get our org
> to push/find an os update just for this purpose too easily.
> 
> So, in the meantime, the question remains, assuming I have the version I
> have currently (dynamic preallocation, persists indefinitely until the file
> is closed/app quits, etc.), will this idea work (e.g. close the file after
> writing, then re-open read-only?). 

Maybe, maybe not. depends on whether the "don't remove prealloc"
heuristic was triggered at any time....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Eric Sandeen]

On 7/26/13 3:38 PM, Jason Rosenberg wrote:
> Hi Stan,
> 
> Thanks for the info (most of it was, in fact, news to me). I'm an
> application developer trying to debug a disk space problem, that's
> all. So far, I've tracked it down to being an XFS issue.
> 
> So you are saying there's no public information that can correlate
> XFS versioning to CentOS (or RHEL) versioning?
> 
> Sad state of affairs.
> 
> If anyone can volunteer this info (if available to you) I'd be much
> appreciative.
> 
> Regardless, is there a version history for XFS vis-a-via mainline
> Linux?

There is no exact version history per se, ie. no "XFS version 2.51"

Instead, the best point of reference upstream is the kernel release number,
i.e. kernel 2.6.32, kernel 3.2, etc.  Ben pointed you at changelogs
for that, which you can peruse...

Once you get into RHEL, you're into a land of backports - originally
2.6.32, but various & sundry updates & backports, to the point where
it is a bit of a special snowflake, based on the requirements of RHEL
customers and the RHEL maintainers (who, incidentally, are also major
upstream XFS developers).

Your best bet for a distro kernel is to look at i.e. the kernel RPM
changelog to see what's been going on.  But you won't be able
to correlate that exactly with any one upstream version, unless maybe
you see a "rebase $SUBSYSTEM to $KERNEL_VERSION" code type changelog.

Back to your original problem, you may find that just setting a fixed
allocsize as a mount option has more pros than cons for your usecase.

HTH,

-Eric

> Thanks,
> 
> Jason
> 
> 
> On Fri, Jul 26, 2013 at 3:27 PM, Stan Hoeppner <stan@hardwarefreak.com <mailto:stan@hardwarefreak.com>> wrote:
> 
>     On 7/26/2013 12:40 PM, Jason Rosenberg wrote:
> 
>     > Anyway, I'm surprised that you don't have some list or other way to
>     > correlate version history of XFS, with os release versions.  I'm guessing
>     > the version I have is not using the latest/greatest.  We actually have
>     > another system that uses an older version of the kernel (2.6.32-279), and
> 
>     2.6.32-279  - this is not a mainline kernel version.  This is a Red Hat
>     specific string describing their internal kernel release.  It has zero
>     correlation to any version number of anything else in the world of
>     mainline Linux.
> 
>     > If, say you tell me, the mainline xfs code has improved behavior, it would
>     > be nice to have a way to know which version of CentOS might include that?
> 
>     IMNSHO, CentOS is a free proprietary chrome plated dog turd.  It's
>     flashy on the outside and claims it's "ENTERPRISE", "just like RHEL!".
>     Then you crack it open and find nothing but crap inside.  So you take it
>     back to the store that gave it away for free and the doors are barred,
>     the place out of business.  The chrome has peeled off and you're stuck
>     with crap that difficult to use.  Every time you touch it you get dirty
>     in some fashion.
> 
>     RHEL is a proprietary solid chrome turd you pay for.  You can't get to
>     the inside, but if you find a scratch and 'return' it Red Hat will say
>     "we can help you fix that".
> 
>     If you avoid the flashy turds altogether while still plunking down no
>     cash, and use a distro based entirely on mainline Linux and GNU user
>     space source, you can get help directly from the folks who wrote the
>     code you're running because they know what is where.  Whether it be
>     Linux proper, the XFS subsystem, NFS, Samba, Postix, etc.  Such
>     distributions are too numerous to mention.  None of them are chrome
>     plated, none claim to be "just like ENTERPRISE distro X".  I tell all
>     users of RHEL knock offs every time I see a situation like this:
> 
>     Either pay for and receive the support that's required for the
>     proprietary distribution you're running, or use a completely open source
>     distro based on mainline kernel source and GNU user space.  By using a
>     RHEL knock off, you're simply locking yourself into an outdated
>     proprietary code base for which there is no viable support option,
>     because so few people in the community understand the packaging of the
>     constituent parts of the RHEL kernels.  This is entirely intentional on
>     the part of Red Hat, specifically to make the life of CentOS users
>     painful, and rightfully so.
> 
>     FYI, many of the folks on the XFS list are Red Hat employees, including
>     Dave.  They'll gladly assist RHEL customers here if needed.  However, to
>     support CentOS users, especially in your situation, they'd have to use
>     Red Hat Inc resources to hunt down the information related to the CentOS
>     kernel you have that correlates to the RHEL kernel it is copied from.
>     So they've just consumed Red Hat Inc resources to directly assist a free
>     competitor who copied their distro.
> 
>     Thus there's not much incentive to assist CentOS users as they'd in
>     essence be taking money out of their employer's pocket.  Taken to the
>     extreme this results in pay cuts, possibly furloughs or pink slips, etc.
> 
>     Surely this can't be the first time you've run into a free community
>     support issue with the CentOS kernel.  Surely what I've written isn't
>     news to you.  Pay Red Hat for RHEL, or switch to Debian, Ubuntu, Open
>     Suse, etc.  Either way you'll be able to get much better support.
> 
>     --
>     Stan
> 
> 
> 
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Keith Keller]

On 2013-07-26, Stan Hoeppner <stan@hardwarefreak.com> wrote:
> Either pay for and receive the support that's required for the
> proprietary distribution you're running, or use a completely open source
> distro based on mainline kernel source and GNU user space.

Couldn't you simply run a mainline kernel on CentOS?  elrepo supports
both the mainline kernel and the longterm kernel.

http://elrepo.org/tiki/kernel-ml
http://elrepo.org/tiki/kernel-lt

It's not perfect, of course, but at least it's closer to what the XFS
developers (RH employees and others) are working on.

--keith

-- 
kkeller@wombat.san-francisco.ca.us


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: understanding speculative preallocation

[Dave Chinner]

On Fri, Jul 26, 2013 at 01:40:16PM -0400, Jason Rosenberg wrote:
> Hi Dave,
> 
> Thanks for your responses.  I'm a bit confused, as I didn't see your
> responses on the actual forum (only in my email inbox).

I'm replying from the list ;)

[ If you have a dup filter like I do, then I only get one copy of
anything that is sent to me when there are multiple cc's. My
procmail rules determine where it gets stored.... ]

> Anyway, I'm surprised that you don't have some list or other way to
> correlate version history of XFS, with os release versions.

Of course we do.

> I'm guessing
> the version I have is not using the latest/greatest.  We actually have
> another system that uses an older version of the kernel (2.6.32-279), and
> it behaves differently (it still preallocates space beyond what will ever
> be used, but not by quite as much).  When we rolled out our newer machines
> to 2.6.32-358, we started seeing a marked increase in disk full problems.

Disclaimer: I'm the primary RHEL XFS developer, employed by Red Hat.

CentOS is a rebadged RHEL product that is released for free. If you
want bugs fixed in CentOS, then generally you are on your own. If
you want paid support where people will fix problems you have, you
need to pay for RHEL.

You get what you pay for.

> If, say you tell me, the mainline xfs code has improved behavior, it would
> be nice to have a way to know which version of CentOS might include that?

Well, that depends on what Red Hat do with RHEL, because CentOS
simply rebuild what Red Hat releases.

>  Telling me to read source code across multiple kernel versions sounds like
> an interesting endeavor, but not something that is the most efficient use
> of my time, unless there truly is no one who can easily tell me anything
> about xfs version history.

So, you want me to read it for you instead, then document it for
you? i.e. spend a day not fixing bugs and developing new code to
document the history of something that you can find out by looking
in git and rpms yourself? Unless you are offering to pay someone to
do it, I don't see it happening...

Open source != free lunch.

> Do you have any plans to have some sort of improved documentation story
> around this?

No.

> This speculative preallocation behavior is truly unexpected
> and not transparent to the user.

Which is why we've been fixing it. The problems you are reporting
are already fixed in the mainline kernel.

> I can see that it's probably a great
> performance boost (especially for something like kafka), but kafka does
> have predictable log file rotation capped at fixed sizes, so it would be
> great if that could be factored in.

Mainline already does handle this, in a much more generic manner. If
any file with speculative prealloc beyond EOF remains clean for 5
minutes, then the specualtive prealloc is removed. Hence 5 minutes
after you log file is rotated, it will have the excess space
removed.

> I suppose using the allocsize setting might work in the short term.  But I
> probably don't want to set allocsize to 1Gb, since that would mean every
> single file created would start with that size, is that right?  Does the
> allocsize setting basically work by always keeping the file size ahead of
> consumed space by the allocsize amount?

Effectively.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

understanding speculative preallocation

[jbr]

Hello, 

I'm looking for general documentation/help with the speculative
preallocation feature in xfs.  So far, I haven't really been able to find
any definitive, up to date documentation on it. 

I'm wondering how I can find out definitively which version of xfs I am
using, and what the preallocation scheme in use is. 

We are running apache kafka on our servers, and kafka uses sequential io to
write data log files.  Kafka uses, by default, a maximum log file size of
1Gb.  However, most of the log files end up being 2Gb, and thus the disk
fills up twice as fast as it should. 

We are using xfs on CentOS 2.6.32-358.  Is there a way I can know which
version of xfs is built into this version of the kernel?  What preallocation
schedule does it use?  If I do a xfs_info -V, it reports 3.1.1. 

We are using xfs (mounted with no allocsize specified).  I've seen varying
info suggesting this means it either defaults to an allocsize of 64K (which
doesn't seem to match my observations), or that it will use dynamic
preallocation. 

I've also seen hints (but no actual canonical documentation) suggesting that
the dynamic preallocation works by progressively doubling the current file
size (which does match my observations). 

What I'm not clear on, is the scheduling for the preallocation. At what
point does it decide to preallocate the next doubling of space.  Is it when
the current preallocated space is used up, or does it happen when the
current space is used up within some threshold.  What I'd like to do, is
keep the doubling behavior in tact, but have it capped so it never increases
the file beyond 1Gb.  Is there a way to do that?  Can I trick the
preallocation to not do a final doubling, if I cap my kafka log files at
say, 900Mb (or some percentage under 1Gb)? 

There are numerous references to an allocation schedule like this: 

freespace       max prealloc size 
  >5%             full extent (8GB) 
  4-5%             2GB (8GB >> 2) 
  3-4%             1GB (8GB >> 3) 
  2-3%           512MB (8GB >> 4) 
  1-2%           256MB (8GB >> 5) 
  <1%            128MB (8GB >> 6) 

I'm just not sure I understand what this is telling me.  It seems to tell me
what the max prealloc size is, with being reduced if the disk is nearly
full.  But it doesn't tell me about the progressive doubling in
preallocation (I assume up to a max of 8Gb).  Is any of this configurable? 
Can we specify a max prealloc size somewhere? 

The other issue seems to be that after the files are closed (from within the
java jvm), they still don't seem to have their pre-allocated space
reclaimed.  Are there known issues with closing the files in java not
properly causing a flush of the preallocated space? 

Any help pointing me to any documentation/user guides which accurately
describes this would be appreciated! 

Thanks, 

Jason



--
View this message in context: http://xfs.9218.n7.nabble.com/understanding-speculative-preallocation-tp35003.html
Sent from the linux-xfs mailing list archive at Nabble.com.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs