linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [GIT PULL] afs: Fixes
@ 2017-11-24 14:22 David Howells
  2017-11-25 18:05 ` Linus Torvalds
  2017-11-25 22:35 ` David Howells
  0 siblings, 2 replies; 21+ messages in thread
From: David Howells @ 2017-11-24 14:22 UTC (permalink / raw)
  To: torvalds; +Cc: dhowells, linux-afs, linux-fsdevel, linux-kernel

Hi Linus,

Can you pull these fixes for the AFS filesystem?  They include:

 (1) Make AFS file locking work again.

 (2) Don't write to a page that's being written out, but wait for it to
     complete.

 (3) Do d_drop() and d_add() in the right places.

 (4) Put keys on error paths.

 (5) Remove some redundant code.

Thanks,
David
---
The following changes since commit cf9b0772f2e410645fece13b749bd56505b998b8:

  Merge tag 'armsoc-drivers' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc (2017-11-16 16:05:01 -0800)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20171124

for you to fetch changes up to 43dd388b21c722616413781bd434522376e117cc:

  afs: remove redundant assignment of dvnode to itself (2017-11-24 13:55:46 +0000)

----------------------------------------------------------------
AFS fixes

----------------------------------------------------------------
Colin Ian King (1):
      afs: remove redundant assignment of dvnode to itself

David Howells (4):
      afs: Fix file locking
      afs: Make afs_write_begin() avoid writing to a page that's being stored
      afs: Fix some dentry handling in dir ops and missing key_puts
      afs: Fix signal handling in some file ops

Gustavo A. R. Silva (1):
      afs: cell: Remove unnecessary code in afs_lookup_cell

 fs/afs/cell.c        |   7 +-
 fs/afs/dir.c         |  25 +--
 fs/afs/flock.c       | 548 +++++++++++++++++++++++++++++----------------------
 fs/afs/internal.h    |  23 ++-
 fs/afs/rotate.c      |  70 +++++--
 fs/afs/security.c    |   4 +-
 fs/afs/server_list.c |   2 +-
 fs/afs/write.c       |   5 +
 8 files changed, 405 insertions(+), 279 deletions(-)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs: Fixes
  2017-11-24 14:22 [GIT PULL] afs: Fixes David Howells
@ 2017-11-25 18:05 ` Linus Torvalds
  2017-11-25 22:35 ` David Howells
  1 sibling, 0 replies; 21+ messages in thread
From: Linus Torvalds @ 2017-11-25 18:05 UTC (permalink / raw)
  To: David Howells; +Cc: linux-afs, linux-fsdevel, Linux Kernel Mailing List

On Fri, Nov 24, 2017 at 4:22 AM, David Howells <dhowells@redhat.com> wrote:
>
>  (2) Don't write to a page that's being written out, but wait for it to
>      complete.

So I see in the commit message why afs needs to do this, but it's
worth pointing out that it's

 (a) impossible to avoid the "inconsistent data" case for writable mmap'ed pages

 (b) can cause some really nasty latency issues

So the "page->private" issue does mean that write-vs-writeback clearly
needs that serialization (and that's not an issue for the mapped page
being changed at the same time), but in general filesystem people
should be aware that data under writeback may still be further dirtied
while the writeback is active (and the page marked dirty again). It
can obviously screw with things like checksums etc, and make for
temporarily inconsistent state if the filesystem does those kinds of
things.

            Linus

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs: Fixes
  2017-11-24 14:22 [GIT PULL] afs: Fixes David Howells
  2017-11-25 18:05 ` Linus Torvalds
@ 2017-11-25 22:35 ` David Howells
  2017-11-25 22:48   ` Dave Chinner
                     ` (2 more replies)
  1 sibling, 3 replies; 21+ messages in thread
From: David Howells @ 2017-11-25 22:35 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: dhowells, linux-afs, linux-fsdevel, Linux Kernel Mailing List

Linus Torvalds <torvalds@linux-foundation.org> wrote:

> So I see in the commit message why afs needs to do this, but it's
> worth pointing out that it's
> 
>  (a) impossible to avoid the "inconsistent data" case for writable mmap'ed
>  pages

Doesn't clear_page_dirty_for_io() write-protect the PTE for the page to be
written out, in which case ->page_mkwrite() will get called again before the
page is redirtied?

>  (b) can cause some really nasty latency issues

True, but I think the most common case is a file being opened, written start
to finish and then closed.  Actually, the worst-handled thing I've seen is a
shell script appending a bunch of things to a file because ->flush() syncs the
file each time it is closed:-/

What would you recommend instead?  I'm currently trying and keep track of what
needs to be written so that I only write what's changed to the server, rather
than writing only whole pages.

David

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs: Fixes
  2017-11-25 22:35 ` David Howells
@ 2017-11-25 22:48   ` Dave Chinner
  2017-11-25 22:55   ` Linus Torvalds
  2017-11-25 23:19   ` David Howells
  2 siblings, 0 replies; 21+ messages in thread
From: Dave Chinner @ 2017-11-25 22:48 UTC (permalink / raw)
  To: David Howells
  Cc: Linus Torvalds, linux-afs, linux-fsdevel, Linux Kernel Mailing List

On Sat, Nov 25, 2017 at 10:35:43PM +0000, David Howells wrote:
> Linus Torvalds <torvalds@linux-foundation.org> wrote:
> 
> > So I see in the commit message why afs needs to do this, but it's
> > worth pointing out that it's
> > 
> >  (a) impossible to avoid the "inconsistent data" case for writable mmap'ed
> >  pages
> 
> Doesn't clear_page_dirty_for_io() write-protect the PTE for the page to be
> written out, in which case ->page_mkwrite() will get called again before the
> page is redirtied?

Yes, but page_mkwrite will only block on writeback in progress is if
the backing device says it needs stable pages.  See
wait_for_stable_page().  e.g. stable pages are required if RAID is
in use, otherwise modification during IO can result in broken
on-disk parity/mirroring....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs: Fixes
  2017-11-25 22:35 ` David Howells
  2017-11-25 22:48   ` Dave Chinner
@ 2017-11-25 22:55   ` Linus Torvalds
  2017-11-25 23:19   ` David Howells
  2 siblings, 0 replies; 21+ messages in thread
From: Linus Torvalds @ 2017-11-25 22:55 UTC (permalink / raw)
  To: David Howells; +Cc: linux-afs, linux-fsdevel, Linux Kernel Mailing List

On Sat, Nov 25, 2017 at 12:35 PM, David Howells <dhowells@redhat.com> wrote:
>
> Doesn't clear_page_dirty_for_io() write-protect the PTE for the page to be
> written out, in which case ->page_mkwrite() will get called again before the
> page is redirtied?

No, it literally just sets the dirty bit (and does accounting).

But I think you may be right that we always write-protect he page when
we move the dirty bit from the page tables to the 'struct page'
(page_mkclean_one()).

However, even when you do that, the page can be writable in other
mappings. At least fork(), for example, only clears the dirty bit,
doesn't mark it write-protected.

So there is some rate-limiting of dirty pages, but I do not believe
that we've ever really *serialized* writes.

>>  (b) can cause some really nasty latency issues
>
> True, but I think the most common case is a file being opened, written start
> to finish and then closed.  Actually, the worst-handled thing I've seen is a
> shell script appending a bunch of things to a file because ->flush() syncs the
> file each time it is closed:-/
>
> What would you recommend instead?  I'm currently trying and keep track of what
> needs to be written so that I only write what's changed to the server, rather
> than writing only whole pages.

I don't think that what you are doing is necessarily wrong, I'm just
pointing out that you may still see mmap'ed pages being modified
concurrently with the actual IO, and that this will potentially mean
(for example) that things like checksums won't be reliably unless you
do the checksum as you copy the data to a network packet or something.

Of course, if that kind of inconsistency happens, a later write-back
will also happen, and eventually fix it. So the server may see
temporarily 'wrong' data, but it won't be final.

I just hope that the inconsistency isn't fatal to the afs client or
server code. For example, if you retry writes forever when a checksum
were to not match the data, that would be bad.

And yes, this can be

 (a) really hard  to trigger in practice

 (b) very hard to debug due to a combination of very specific timing
and behavior.

so I just wanted to bring this up as a potential issue, not
necessarily as a big problem.

             Linus

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs: Fixes
  2017-11-25 22:35 ` David Howells
  2017-11-25 22:48   ` Dave Chinner
  2017-11-25 22:55   ` Linus Torvalds
@ 2017-11-25 23:19   ` David Howells
  2 siblings, 0 replies; 21+ messages in thread
From: David Howells @ 2017-11-25 23:19 UTC (permalink / raw)
  To: Linus Torvalds
  Cc: dhowells, linux-afs, linux-fsdevel, Linux Kernel Mailing List

Linus Torvalds <torvalds@linux-foundation.org> wrote:

> However, even when you do that, the page can be writable in other
> mappings. At least fork(), for example, only clears the dirty bit,
> doesn't mark it write-protected.

I assumed the rmap walk done by page_mkclean() would take care of that but I'm
not really clear on what the code does.

> I just hope that the inconsistency isn't fatal to the afs client or
> server code. For example, if you retry writes forever when a checksum
> were to not match the data, that would be bad.

Shouldn't be a problem for the the in-Linux client.  Data is copied into
sk_bufs preparatory to doing further things to it like checksumming,
encryption or transmission (actually, in future, I would like to use the
encryption process to save on the copy, but this shouldn't bother that
either).

AFAIK, the servers are all userspace jobs that don't let anyone else touch
their storage so that they can maintain correctness on the data version number
of each vnode.

> so I just wanted to bring this up as a potential issue, not
> necessarily as a big problem.

Thanks.

David

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs fixes
  2020-10-29 14:07 [GIT PULL] afs fixes David Howells
@ 2020-10-29 17:31 ` pr-tracker-bot
  0 siblings, 0 replies; 21+ messages in thread
From: pr-tracker-bot @ 2020-10-29 17:31 UTC (permalink / raw)
  To: David Howells
  Cc: torvalds, Matthew Wilcox (Oracle),
	Christoph Hellwig, Colin Ian King, Dan Carpenter, Nick Piggin,
	dhowells, kernel test robot, linux-afs, linux-fsdevel,
	linux-kernel

The pull request you sent on Thu, 29 Oct 2020 14:07:26 +0000:

> git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20201029

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/598a597636f8618a0520fd3ccefedaed9e4709b0

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [GIT PULL] afs fixes
@ 2020-10-29 14:07 David Howells
  2020-10-29 17:31 ` pr-tracker-bot
  0 siblings, 1 reply; 21+ messages in thread
From: David Howells @ 2020-10-29 14:07 UTC (permalink / raw)
  To: torvalds
  Cc: Matthew Wilcox (Oracle),
	Christoph Hellwig, Colin Ian King, Dan Carpenter, Nick Piggin,
	dhowells, kernel test robot, linux-afs, linux-fsdevel,
	linux-kernel

Hi Linus,

Could you pull these afs fixes, please?  They include the following:

 (1) Fix copy_file_range() to an afs file now returning EINVAL if the
     splice_write file op isn't supplied.

 (2) Fix a deref-before-check in afs_unuse_cell().

 (3) Fix a use-after-free in afs_xattr_get_acl().

 (4) Fix afs to not try to clear PG_writeback when laundering a page.

 (5) Fix afs to take a ref on a page that it sets PG_private on and to drop
     that ref when clearing PG_private.  This is done through recently
     added helpers.

 (6) Fix a page leak if write_begin() fails.

 (7) Fix afs_write_begin() to not alter the dirty region info stored in
     page->private, but rather do this in afs_write_end() instead when we
     know what we actually changed.

 (8) Fix afs_invalidatepage() to alter the dirty region info on a page when
     partial page invalidation occurs so that we don't inadvertantly
     include a span of zeros that will get written back if a page gets
     laundered due to a remote 3rd-party induced invalidation.

     We mustn't, however, reduce the dirty region if the page has been seen
     to be mapped (ie. we got called through the page_mkwrite vector) as
     the page might still be mapped and we might lose data if the file is
     extended again.

 (9) Fix the dirty region info to have a lower resolution if the size of
     the page is too large for this to be encoded (e.g. powerpc32 with 64K
     pages).

     Note that this might not be the ideal way to handle this, since it may
     allow some leakage of undirtied zero bytes to the server's copy in the
     case of a 3rd-party conflict.

To aid (8) and (9), two additional patches are included:

 (*) Wrap the manipulations of the dirty region info stored in
     page->private into helper functions.

 (*) Alter the encoding of the dirty region so that the region bounds can
     be stored with one fewer bit, making a bit available for the
     indication of mappedness.

Thanks,
David
---
The following changes since commit 3650b228f83adda7e5ee532e2b90429c03f7b9ec:

  Linux 5.10-rc1 (2020-10-25 15:14:11 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20201029

for you to fetch changes up to 2d9900f26ad61e63a34f239bc76c80d2f8a6ff41:

  afs: Fix dirty-region encoding on ppc32 with 64K pages (2020-10-29 13:53:04 +0000)

----------------------------------------------------------------
AFS fixes

----------------------------------------------------------------
Dan Carpenter (1):
      afs: Fix a use after free in afs_xattr_get_acl()

David Howells (10):
      afs: Fix copy_file_range()
      afs: Fix tracing deref-before-check
      afs: Fix afs_launder_page to not clear PG_writeback
      afs: Fix to take ref on page when PG_private is set
      afs: Fix page leak on afs_write_begin() failure
      afs: Fix where page->private is set during write
      afs: Wrap page->private manipulations in inline functions
      afs: Alter dirty range encoding in page->private
      afs: Fix afs_invalidatepage to adjust the dirty region
      afs: Fix dirty-region encoding on ppc32 with 64K pages

 fs/afs/cell.c              |   3 +-
 fs/afs/dir.c               |  12 ++----
 fs/afs/dir_edit.c          |   6 +--
 fs/afs/file.c              |  78 ++++++++++++++++++++++++++-------
 fs/afs/internal.h          |  57 ++++++++++++++++++++++++
 fs/afs/write.c             | 105 ++++++++++++++++++++++++---------------------
 fs/afs/xattr.c             |   2 +-
 include/trace/events/afs.h |  20 ++-------
 8 files changed, 188 insertions(+), 95 deletions(-)


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs: Fixes
  2020-04-13 14:50 [GIT PULL] afs: Fixes David Howells
  2020-04-13 15:13 ` David Howells
@ 2020-04-14 19:05 ` pr-tracker-bot
  1 sibling, 0 replies; 21+ messages in thread
From: pr-tracker-bot @ 2020-04-14 19:05 UTC (permalink / raw)
  To: David Howells; +Cc: torvalds, dhowells, linux-afs, linux-fsdevel, linux-kernel

The pull request you sent on Mon, 13 Apr 2020 15:50:15 +0100:

> git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20200413

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/f4cd66682b2728734b2fc44f5f1e83a5c740b5cf

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs: Fixes
  2020-04-13 14:50 [GIT PULL] afs: Fixes David Howells
@ 2020-04-13 15:13 ` David Howells
  2020-04-14 19:05 ` pr-tracker-bot
  1 sibling, 0 replies; 21+ messages in thread
From: David Howells @ 2020-04-13 15:13 UTC (permalink / raw)
  Cc: dhowells, torvalds, linux-afs, linux-fsdevel, linux-kernel

David Howells <dhowells@redhat.com> wrote:

>  (1) Fix the decoding of fetched file status records do that in advances
>      the xdr pointer under all circumstances.

As Willy points out, this isn't very English.  Let me try that again.

David


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [GIT PULL] afs: Fixes
@ 2020-04-13 14:50 David Howells
  2020-04-13 15:13 ` David Howells
  2020-04-14 19:05 ` pr-tracker-bot
  0 siblings, 2 replies; 21+ messages in thread
From: David Howells @ 2020-04-13 14:50 UTC (permalink / raw)
  To: torvalds; +Cc: dhowells, linux-afs, linux-fsdevel, linux-kernel

Hi Linus,

Here are some fixes for the afs filesystem:

 (1) Fix the decoding of fetched file status records do that in advances
     the xdr pointer under all circumstances.

 (2) Fix the decoding of a fetched file status record that indicates an
     inline abort code (ie. an error) so that it sets the flag saying that
     it got an error.

 (3) Fix the decoding of the result of the rename operation so that it
     doesn't skip the decoding of the second fetched file status (ie. that
     of the dest dir) in the case that the source and dest dirs were the
     same as this causes the xdr pointer not to be advanced, leading to
     incorrect decoding of subsequent parts of the reply.

 (4) Fix the dump of a bad YFSFetchStatus record to dump the full length.

 (5) Fix a race between local editing of directory contents and accessing
     the dir for reading or d_revalidate by using the same lock in both.

 (6) Fix afs_d_revalidate() to not accidentally reverse the version on a
     dentry when it's meant to be bringing it forward.

David
---
The following changes since commit 8f3d9f354286745c751374f5f1fcafee6b3f3136:

  Linux 5.7-rc1 (2020-04-12 12:35:55 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20200413

for you to fetch changes up to 40fc81027f892284ce31f8b6de1e497f5b47e71f:

  afs: Fix afs_d_validate() to set the right directory version (2020-04-13 15:09:01 +0100)

----------------------------------------------------------------
AFS fixes

----------------------------------------------------------------
David Howells (6):
      afs: Fix missing XDR advance in xdr_decode_{AFS,YFS}FSFetchStatus()
      afs: Fix decoding of inline abort codes from version 1 status records
      afs: Fix rename operation status delivery
      afs: Fix length of dump of bad YFSFetchStatus record
      afs: Fix race between post-modification dir edit and readdir/d_revalidate
      afs: Fix afs_d_validate() to set the right directory version

 fs/afs/dir.c       | 108 +++++++++++++++++++++++++++++++++--------------------
 fs/afs/dir_silly.c |  22 +++++++----
 fs/afs/fsclient.c  |  27 ++++++++------
 fs/afs/yfsclient.c |  26 +++++++------
 4 files changed, 112 insertions(+), 71 deletions(-)


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] afs: Fixes
  2019-08-14 14:18 David Howells
@ 2019-08-14 22:35 ` pr-tracker-bot
  0 siblings, 0 replies; 21+ messages in thread
From: pr-tracker-bot @ 2019-08-14 22:35 UTC (permalink / raw)
  To: David Howells
  Cc: torvalds, dhowells, marc.dionne, baijiaju1990, linux-afs,
	linux-fsdevel, linux-kernel

The pull request you sent on Wed, 14 Aug 2019 15:18:40 +0100:

> git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20190814

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/e22a97a2a85d2a0bdfb134cbbc7ff856ae67edba

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [GIT PULL] afs: Fixes
@ 2019-08-14 14:18 David Howells
  2019-08-14 22:35 ` pr-tracker-bot
  0 siblings, 1 reply; 21+ messages in thread
From: David Howells @ 2019-08-14 14:18 UTC (permalink / raw)
  To: torvalds
  Cc: dhowells, marc.dionne, baijiaju1990, linux-afs, linux-fsdevel,
	linux-kernel

Hi Linus,

Can you pull these afs fixes please?

 (1) Fix the CB.ProbeUuid handler to generate its reply correctly.

 (2) Fix a mix up in indices when parsing a Volume Location entry record.

 (3) Fix a potential NULL-pointer deref when cleaning up a read request.

 (4) Fix the expected data version of the destination directory in
     afs_rename().

 (5) Fix afs_d_revalidate() to only update d_fsdata if it's not the same as
     the directory data version to reduce the likelihood of overwriting the
     result of a competing operation.  (d_fsdata carries the directory DV
     or the least-significant word thereof).

 (6) Fix the tracking of the data-version on a directory and make sure that
     dentry objects get properly initialised, updated and revalidated.

     Also fix rename to update d_fsdata to match the new directory's DV if
     the dentry gets moved over and unhash the dentry to stop
     afs_d_revalidate() from interfering.

David
---
The following changes since commit 2a11c76e5301dddefcb618dac04f74e6314df6bc:

  Merge tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost (2019-07-29 11:34:12 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20190814

for you to fetch changes up to 9dd0b82ef530cdfe805c9f7079c99e104be59a14:

  afs: Fix missing dentry data version updating (2019-07-30 14:38:52 +0100)

----------------------------------------------------------------
AFS Fixes

Reviewed-by: Marc Dionne <marc.dionne@auristor.com>

----------------------------------------------------------------
David Howells (4):
      afs: Fix the CB.ProbeUuid service handler to reply correctly
      afs: Fix off-by-one in afs_rename() expected data version calculation
      afs: Only update d_fsdata if different in afs_d_revalidate()
      afs: Fix missing dentry data version updating

Jia-Ju Bai (1):
      fs: afs: Fix a possible null-pointer dereference in afs_put_read()

Marc Dionne (1):
      afs: Fix loop index mixup in afs_deliver_vl_get_entry_by_name_u()

 fs/afs/cmservice.c | 10 ++----
 fs/afs/dir.c       | 89 ++++++++++++++++++++++++++++++++++++++++++++----------
 fs/afs/file.c      | 12 +++++---
 fs/afs/vlclient.c  | 11 ++++---
 4 files changed, 89 insertions(+), 33 deletions(-)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [GIT PULL] afs: Fixes
@ 2019-08-14 14:15 David Howells
  0 siblings, 0 replies; 21+ messages in thread
From: David Howells @ 2019-08-14 14:15 UTC (permalink / raw)
  To: torvalds
  Cc: dhowells, marc.dionne, baijiaju1990, linux-afs, linux-fsdevel,
	linux-kernel

Hi Linus,

Can you pull these afs fixes please?

 (1) Fix the CB.ProbeUuid handler to generate its reply correctly.

 (2) Fix a mix up in indices when parsing a Volume Location entry record.

 (3) Fix a potential NULL-pointer deref when cleaning up a read request.

 (4) Fix the expected data version of the destination directory in
     afs_rename().

 (5) Fix afs_d_revalidate() to only update d_fsdata if it's not the same as
     the directory data version to reduce the likelihood of overwriting the
     result of a competing operation.  (d_fsdata carries the directory DV
     or the least-significant word thereof).

 (6) Fix the tracking of the data-version on a directory and make sure that
     dentry objects get properly initialised, updated and revalidated.

     Also fix rename to update d_fsdata to match the new directory's DV if
     the dentry gets moved over and unhash the dentry to stop
     afs_d_revalidate() from interfering.

David
---
The following changes since commit 2a11c76e5301dddefcb618dac04f74e6314df6bc:

  Merge tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost (2019-07-29 11:34:12 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20190814

for you to fetch changes up to 9dd0b82ef530cdfe805c9f7079c99e104be59a14:

  afs: Fix missing dentry data version updating (2019-07-30 14:38:52 +0100)

----------------------------------------------------------------
AFS Fixes

Reviewed-by: Marc Dionne <marc.dionne@auristor.com>

----------------------------------------------------------------
David Howells (4):
      afs: Fix the CB.ProbeUuid service handler to reply correctly
      afs: Fix off-by-one in afs_rename() expected data version calculation
      afs: Only update d_fsdata if different in afs_d_revalidate()
      afs: Fix missing dentry data version updating

Jia-Ju Bai (1):
      fs: afs: Fix a possible null-pointer dereference in afs_put_read()

Marc Dionne (1):
      afs: Fix loop index mixup in afs_deliver_vl_get_entry_by_name_u()

 fs/afs/cmservice.c | 10 ++----
 fs/afs/dir.c       | 89 ++++++++++++++++++++++++++++++++++++++++++++----------
 fs/afs/file.c      | 12 +++++---
 fs/afs/vlclient.c  | 11 ++++---
 4 files changed, 89 insertions(+), 33 deletions(-)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] AFS fixes
  2019-06-26 13:39 [GIT PULL] AFS fixes David Howells
@ 2019-06-28  0:45 ` pr-tracker-bot
  0 siblings, 0 replies; 21+ messages in thread
From: pr-tracker-bot @ 2019-06-28  0:45 UTC (permalink / raw)
  To: David Howells
  Cc: torvalds, dhowells, iwienand, linux-afs, linux-fsdevel,
	linux-kernel, stable

The pull request you sent on Wed, 26 Jun 2019 14:39:53 +0100:

> git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20190620

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/cd0f3aaebc5b17e0ccb1b9ef9ae43042d075d767

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.wiki.kernel.org/userdoc/prtracker

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [GIT PULL] AFS fixes
@ 2019-06-26 13:39 David Howells
  2019-06-28  0:45 ` pr-tracker-bot
  0 siblings, 1 reply; 21+ messages in thread
From: David Howells @ 2019-06-26 13:39 UTC (permalink / raw)
  To: torvalds
  Cc: dhowells, iwienand, linux-afs, linux-fsdevel, linux-kernel, stable

Hi Linus,

Could you pull this please?

There are four patches:

 (1) Fix the printing of the "vnode modified" warning to exclude checks on
     files for which we don't have a callback promise from the server (and
     so don't expect the server to tell us when it changes).

     Without this, for every file or directory for which we still have an
     in-core inode that gets changed on the server, we may get a message
     logged when we next look at it.  This can happen in bulk if, for
     instance, someone does "vos release" to update a R/O volume from a R/W
     volume and a whole set of files are all changed together.

     We only really want to log a message if the file changed and the
     server didn't tell us about it or we failed to track the state
     internally.

 (2) Fix accidental corruption of either afs_vlserver struct objects or the
     the following memory locations (which could hold anything).  The issue
     is caused by a union that points to two different structs in struct
     afs_call (to save space in the struct).  The call cleanup code assumes
     that it can simply call the cleanup for one of those structs if not
     NULL - when it might be actually pointing to the other struct.

     This means that every Volume Location RPC op is going to corrupt
     something.

 (3) Fix an uninitialised spinlock.  This isn't too bad, it just causes a
     one-off warning if lockdep is enabled when "vos release" is called,
     but the spinlock still behaves correctly.

 (4) Fix the setting of i_block in the inode.  This causes du, for example,
     to produce incorrect results, but otherwise should not be dangerous to
     the kernel.

The in-kernel AFS client has been undergoing testing on opendev.org on one
of their mirror machines.  They are using AFS to hold data that is then
served via apache, and Ian Wienand had reported seeing oopses, spontaneous
machine reboots and updates to volumes going missing.  This patch series
appears to have fixed the problem, very probably due to patch (2), but it's
not 100% certain.

Reviewed-by: Jeffrey Altman <jaltman@auristor.com>
Tested-by: Marc Dionne <marc.dionne@auristor.com>
Tested-by: Ian Wienand <iwienand@redhat.com>

---
The following changes since commit a188339ca5a396acc588e5851ed7e19f66b0ebd9:

  Linux 5.2-rc1 (2019-05-19 15:47:09 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20190620

for you to fetch changes up to 2cd42d19cffa0ec3dfb57b1b3e1a07a9bf4ed80a:

  afs: Fix setting of i_blocks (2019-06-20 18:12:02 +0100)

----------------------------------------------------------------
AFS fixes

----------------------------------------------------------------
David Howells (4):
      afs: Fix over zealous "vnode modified" warnings
      afs: Fix vlserver record corruption
      afs: Fix uninitialised spinlock afs_volume::cb_break_lock
      afs: Fix setting of i_blocks

 fs/afs/callback.c |  4 ++--
 fs/afs/inode.c    | 31 +++++++++++++++++++------------
 fs/afs/internal.h |  8 +++-----
 fs/afs/volume.c   |  1 +
 4 files changed, 25 insertions(+), 19 deletions(-)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] AFS fixes
  2019-06-26  8:50 David Howells
@ 2019-06-26 13:38 ` David Howells
  0 siblings, 0 replies; 21+ messages in thread
From: David Howells @ 2019-06-26 13:38 UTC (permalink / raw)
  To: torvalds; +Cc: dhowells, iwienand, linux-afs, linux-fsdevel, linux-kernel

Oops... I forgot to include the pull request bit.  Will resend.

David

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [GIT PULL] AFS fixes
@ 2019-06-26  8:50 David Howells
  2019-06-26 13:38 ` David Howells
  0 siblings, 1 reply; 21+ messages in thread
From: David Howells @ 2019-06-26  8:50 UTC (permalink / raw)
  To: torvalds; +Cc: dhowells, iwienand, linux-afs, linux-fsdevel, linux-kernel

Hi Linus,

Could you pull this please?

There are four patches:

 (1) Fix the printing of the "vnode modified" warning to exclude checks on
     files for which we don't have a callback promise from the server (and
     so don't expect the server to tell us when it changes).

     Without this, for every file or directory for which we still have an
     in-core inode that gets changed on the server, we may get a message
     logged when we next look at it.  This can happen in bulk if, for
     instance, someone does "vos release" to update a R/O volume from a R/W
     volume and a whole set of files are all changed together.

     We only really want to log a message if the file changed and the
     server didn't tell us about it or we failed to track the state
     internally.

 (2) Fix accidental corruption of either afs_vlserver struct objects or the
     the following memory locations (which could hold anything).  The issue
     is caused by a union that points to two different structs in struct
     afs_call (to save space in the struct).  The call cleanup code assumes
     that it can simply call the cleanup for one of those structs if not
     NULL - when it might be actually pointing to the other struct.

     This means that every Volume Location RPC op is going to corrupt
     something.

 (3) Fix an uninitialised spinlock.  This isn't too bad, it just causes a
     one-off warning if lockdep is enabled when "vos release" is called,
     but the spinlock still behaves correctly.

 (4) Fix the setting of i_block in the inode.  This causes du, for example,
     to produce incorrect results, but otherwise should not be dangerous to
     the kernel.

The in-kernel AFS client has been undergoing testing on opendev.org on one
of their mirror machines.  They are using AFS to hold data that is then
served via apache, and Ian Wienand had reported seeing oopses, spontaneous
machine reboots and updates to volumes going missing.  This patch series
appears to have fixed the problem, very probably due to patch (2), but it's
not 100% certain.

Reviewed-by: Jeffrey Altman <jaltman@auristor.com>
Tested-by: Marc Dionne <marc.dionne@auristor.com>
Tested-by: Ian Wienand <iwienand@redhat.com>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [GIT PULL] afs: Fixes
@ 2018-05-14 21:25 David Howells
  0 siblings, 0 replies; 21+ messages in thread
From: David Howells @ 2018-05-14 21:25 UTC (permalink / raw)
  To: torvalds; +Cc: dhowells, linux-afs, linux-fsdevel, linux-kernel

Hi Linus,

Here's a set of patches that fix a number of bugs in the in-kernel AFS
client, including:

 (1) Fix directory locking to not use individual page locks for directory
     reading/scanning but rather to use a semaphore on the afs_vnode struct
     as the directory contents must be read in a single blob and data from
     different reads must not be mixed as the entire contents may be
     shuffled about between reads.

 (2) Fix address list parsing to handle port specifiers correctly.

 (3) Only give up callback records on a server if we actually talked to
     that server (we might not be able to access a server).

 (4) Fix some callback handling bugs, including refcounting, whole-volume
     callbacks and when callbacks actually get broken in response to a
     CB.CallBack op.

 (5) Fix some server/address rotation bugs, including giving up if we can't
     probe a server; giving up if a server says it doesn't have a volume,
     but there are more servers to try.

 (6) Fix the decoding of fetched statuses to be OpenAFS compatible.

 (7) Fix the handling of server lookups in Cache Manager ops (such as
     CB.InitCallBackState3) to use a UUID if possible and to handle no
     server being found.

 (8) Fix a bug in server lookup where not all addresses are compared.

 (9) Fix the non-encryption of calls that prevents some servers from being
     accessed (this also requires an AF_RXRPC patch that has already gone
     in through the net tree).

There's also a patch that adds tracepoints to log Cache Manager ops that
don't find a matching server, either by UUID or by address.

Thanks,
David
---
The following changes since commit f0ab773f5c96c29a5227234c4b5a820f5591b74d:

  Merge branch 'akpm' (patches from Andrew) (2018-05-11 18:04:12 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-fixes-20180514

for you to fetch changes up to 4776cab43fd3111618112737a257dc3ef368eddd:

  afs: Fix the non-encryption of calls (2018-05-14 15:15:19 +0100)

----------------------------------------------------------------
AFS fixes

----------------------------------------------------------------
David Howells (13):
      afs: Fix directory page locking
      afs: Fix address list parsing
      afs: Fix giving up callbacks on server destruction
      afs: Fix refcounting in callback registration
      afs: Fix server rotation's handling of fileserver probe failure
      afs: Fix AFSFetchStatus decoder to provide OpenAFS compatibility
      afs: Fix VNOVOL handling in address rotation
      afs: Fix the handling of CB.InitCallBackState3 to find the server by UUID
      afs: Add a tracepoint to record callbacks from unlisted servers
      afs: Fix the handling of an unfound server in CM operations
      afs: Fix whole-volume callback handling
      afs: Fix CB.CallBack handling
      afs: Fix the non-encryption of calls

Marc Dionne (1):
      afs: Fix afs_find_server search loop

 fs/afs/addr_list.c         | 25 ++++++++------
 fs/afs/callback.c          | 84 +++++++++++++++++++++++++++++++++-------------
 fs/afs/cmservice.c         | 67 ++++++++++++------------------------
 fs/afs/dir.c               | 54 ++++++++++++++---------------
 fs/afs/file.c              |  2 +-
 fs/afs/flock.c             |  6 ++--
 fs/afs/fsclient.c          | 31 ++++++++++++-----
 fs/afs/inode.c             | 19 ++++++-----
 fs/afs/internal.h          | 25 ++++++++++++--
 fs/afs/rotate.c            | 20 +++++++----
 fs/afs/rxrpc.c             | 18 ++++++----
 fs/afs/security.c          |  7 ++--
 fs/afs/server.c            | 21 +++---------
 fs/afs/server_list.c       |  7 ++--
 fs/afs/super.c             |  4 +--
 fs/afs/write.c             |  2 +-
 include/trace/events/afs.h | 42 +++++++++++++++++++++++
 17 files changed, 266 insertions(+), 168 deletions(-)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [GIT PULL] AFS fixes
  2017-03-17 15:29 [GIT PULL] AFS fixes David Howells
@ 2017-03-17 19:23 ` Linus Torvalds
  0 siblings, 0 replies; 21+ messages in thread
From: Linus Torvalds @ 2017-03-17 19:23 UTC (permalink / raw)
  To: David Howells; +Cc: linux-afs, linux-fsdevel, Linux Kernel Mailing List

On Fri, Mar 17, 2017 at 8:29 AM, David Howells <dhowells@redhat.com> wrote:
>
> Could you pull these fixes to the AFS filesystem in the kernel please?

I took this, but honestly, most of the patches look like "merge
window" patches rather than -rc patches.

Exactly _one_ of the patches was marked for stable. The rest look like
"yes, they are technically bugs, but this is general afs development".

                Linus

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [GIT PULL] AFS fixes
@ 2017-03-17 15:29 David Howells
  2017-03-17 19:23 ` Linus Torvalds
  0 siblings, 1 reply; 21+ messages in thread
From: David Howells @ 2017-03-17 15:29 UTC (permalink / raw)
  To: torvalds; +Cc: dhowells, linux-afs, linux-fsdevel, linux-kernel

Hi Linus,

Could you pull these fixes to the AFS filesystem in the kernel please?

They fix a variety of bugs.  These include some issues fixed for
consistency with other AFS implementations:

 (*) Handle AFS mode bits better.

 (*) Use the client mtime rather than the server mtime in the protocol.

 (*) Handle the server returning more or less data than was requested in a
     FetchData call.

 (*) Distinguish mountpoints from symlinks based on the mode bits rather
     than preemptively reading every symlink to find out what it actually
     represents.

One other notable change for the user is that files are now flushed on
close analogously with other network filesystems.

Thanks,
David
---
The following changes since commit 69eea5a4ab9c705496e912b55a9d312325de19e6:

  Merge branch 'for-linus' of git://git.kernel.dk/linux-block (2017-03-15 16:54:58 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-20170316

for you to fetch changes up to c5051c7bc777dffa5661569dec5997f432b9a34a:

  afs: Don't wait for page writeback with the page lock held (2017-03-16 16:29:30 +0000)

----------------------------------------------------------------
kAFS fixes

----------------------------------------------------------------
Andreea-Cristina Bernat (2):
      afs: inode: Replace rcu_assign_pointer() with RCU_INIT_POINTER()
      afs: security: Replace rcu_assign_pointer() with RCU_INIT_POINTER()

David Howells (20):
      afs: Fix missing put_page()
      afs: Fix page overput in afs_fill_page()
      afs: Handle better the server returning excess or short data
      afs: Kill struct afs_read::pg_offset
      afs: Handle a short write to an AFS page
      afs: Flush outstanding writes when an fd is closed
      afs: Distinguish mountpoints from symlinks by file mode alone
      afs: Fix AFS read bug
      afs: Make struct afs_read::remain 64-bit
      afs: Use a bvec rather than a kvec in afs_send_pages()
      afs: Fix the maths in afs_fs_store_data()
      afs: Invalid op ID should abort with RXGEN_OPCODE
      afs: Better abort and net error handling
      afs: Don't set PG_error on local EINTR or ENOMEM when filling a page
      afs: Fix page leak in afs_write_begin()
      afs: Fix afs_kill_pages()
      afs: Fix an off-by-one error in afs_send_pages()
      afs: Fix abort on signal while waiting for call completion
      afs: ->writepage() shouldn't call clear_page_dirty_for_io()
      afs: Don't wait for page writeback with the page lock held

Marc Dionne (4):
      afs: Populate group ID from vnode status
      afs: Adjust mode bits processing
      afs: Deal with an empty callback array
      afs: Populate and use client modification time

Tina Ruchandani (2):
      afs: Migrate vlocation fields to 64-bit
      afs: Prevent callback expiry timer overflow

 fs/afs/callback.c  |   7 +--
 fs/afs/cmservice.c |  11 ++--
 fs/afs/file.c      |  20 +++++--
 fs/afs/fsclient.c  |  77 +++++++++++++++++----------
 fs/afs/inode.c     |  42 ++++++++-------
 fs/afs/internal.h  |  23 +++++----
 fs/afs/misc.c      |   2 +
 fs/afs/mntpt.c     |  53 -------------------
 fs/afs/rxrpc.c     | 149 ++++++++++++++++++++++++++++++-----------------------
 fs/afs/security.c  |   9 +++-
 fs/afs/server.c    |   6 +--
 fs/afs/vlocation.c |  16 +++---
 fs/afs/write.c     |  76 +++++++++++++++++++--------
 13 files changed, 269 insertions(+), 222 deletions(-)

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2020-10-29 17:31 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-24 14:22 [GIT PULL] afs: Fixes David Howells
2017-11-25 18:05 ` Linus Torvalds
2017-11-25 22:35 ` David Howells
2017-11-25 22:48   ` Dave Chinner
2017-11-25 22:55   ` Linus Torvalds
2017-11-25 23:19   ` David Howells
  -- strict thread matches above, loose matches on Subject: below --
2020-10-29 14:07 [GIT PULL] afs fixes David Howells
2020-10-29 17:31 ` pr-tracker-bot
2020-04-13 14:50 [GIT PULL] afs: Fixes David Howells
2020-04-13 15:13 ` David Howells
2020-04-14 19:05 ` pr-tracker-bot
2019-08-14 14:18 David Howells
2019-08-14 22:35 ` pr-tracker-bot
2019-08-14 14:15 David Howells
2019-06-26 13:39 [GIT PULL] AFS fixes David Howells
2019-06-28  0:45 ` pr-tracker-bot
2019-06-26  8:50 David Howells
2019-06-26 13:38 ` David Howells
2018-05-14 21:25 [GIT PULL] afs: Fixes David Howells
2017-03-17 15:29 [GIT PULL] AFS fixes David Howells
2017-03-17 19:23 ` Linus Torvalds

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).