* [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code
@ 2019-08-26 21:21 Darrick J. Wong
2019-08-26 21:21 ` [PATCH 1/4] libxfs: fix uncached buffer refcounting Darrick J. Wong
` (3 more replies)
0 siblings, 4 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-08-26 21:21 UTC (permalink / raw)
To: sandeen, darrick.wong; +Cc: linux-xfs
Hi all,
In this series, we start by adapting the libxfs AG construction code to
be aware that an internal log can be placed with an AG that is being
initialized. This is necessary to refactor mkfs to use the AG
construction set instead of its own open-coded initialization work.
In userspace, the next thing we have to do is to fix the uncached buffer
code so that libxfs_putbuf won't try to suck them into the buffer cache;
and then fix delwri_{queue,submit} so that IO errors are returned by the
submit function.
The final patch in the xfsprogs series replaces all of mkfs' AG
initialization functions with a single call to the functions in libxfs.
If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy!
Comments and questions are, as always, welcome.
--D
kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=mkfs-refactor
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=mkfs-refactor
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/4] libxfs: fix uncached buffer refcounting
2019-08-26 21:21 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
@ 2019-08-26 21:21 ` Darrick J. Wong
2019-08-26 21:21 ` [PATCH 2/4] libxfs: fix buffer refcounting in delwri_queue Darrick J. Wong
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-08-26 21:21 UTC (permalink / raw)
To: sandeen, darrick.wong; +Cc: linux-xfs, Christoph Hellwig
From: Darrick J. Wong <darrick.wong@oracle.com>
Currently, uncached buffers in userspace are created with zero refcount
and are fed to cache_node_put when they're released. This is totally
broken -- the refcount should be 1 (because the caller now holds a
reference) and we should never be dumping uncached buffers into the
cache. Fix both of these problems.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
libxfs/libxfs_io.h | 18 ++++++++++++++++++
libxfs/libxfs_priv.h | 2 --
libxfs/rdwr.c | 5 ++++-
3 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/libxfs/libxfs_io.h b/libxfs/libxfs_io.h
index 0b8d9774..7dcb4bff 100644
--- a/libxfs/libxfs_io.h
+++ b/libxfs/libxfs_io.h
@@ -227,4 +227,22 @@ xfs_buf_associate_memory(struct xfs_buf *bp, void *mem, size_t len)
return 0;
}
+/*
+ * Allocate an uncached buffer that points nowhere. The refcount will be 1,
+ * and the cache node hash list will be empty to indicate that it's uncached.
+ */
+static inline struct xfs_buf *
+xfs_buf_get_uncached(struct xfs_buftarg *targ, size_t bblen, int flags)
+{
+ struct xfs_buf *bp;
+
+ bp = libxfs_getbufr(targ, XFS_BUF_DADDR_NULL, bblen);
+ if (!bp)
+ return NULL;
+
+ INIT_LIST_HEAD(&bp->b_node.cn_hash);
+ bp->b_node.cn_count = 1;
+ return bp;
+}
+
#endif /* __LIBXFS_IO_H__ */
diff --git a/libxfs/libxfs_priv.h b/libxfs/libxfs_priv.h
index 8b3de170..99cfac1f 100644
--- a/libxfs/libxfs_priv.h
+++ b/libxfs/libxfs_priv.h
@@ -379,8 +379,6 @@ roundup_64(uint64_t x, uint32_t y)
(len) = __bar; /* no set-but-unused warning */ \
NULL; \
})
-#define xfs_buf_get_uncached(t,n,f) \
- libxfs_getbufr((t), XFS_BUF_DADDR_NULL, (n));
#define xfs_buf_relse(bp) libxfs_putbuf(bp)
#define xfs_buf_get(devp,blkno,len) (libxfs_getbuf((devp), (blkno), (len)))
#define xfs_bwrite(bp) libxfs_writebuf((bp), 0)
diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c
index 1f2c129b..3282f6de 100644
--- a/libxfs/rdwr.c
+++ b/libxfs/rdwr.c
@@ -867,7 +867,10 @@ libxfs_putbuf(xfs_buf_t *bp)
}
}
- cache_node_put(libxfs_bcache, (struct cache_node *)bp);
+ if (!list_empty(&bp->b_node.cn_hash))
+ cache_node_put(libxfs_bcache, (struct cache_node *)bp);
+ else if (--bp->b_node.cn_count == 0)
+ libxfs_putbufr(bp);
}
void
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/4] libxfs: fix buffer refcounting in delwri_queue
2019-08-26 21:21 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
2019-08-26 21:21 ` [PATCH 1/4] libxfs: fix uncached buffer refcounting Darrick J. Wong
@ 2019-08-26 21:21 ` Darrick J. Wong
2019-08-26 21:21 ` [PATCH 3/4] libxfs: make xfs_buf_delwri_submit actually do something Darrick J. Wong
2019-08-26 21:22 ` [PATCH 4/4] mkfs: use libxfs to write out new AGs Darrick J. Wong
3 siblings, 0 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-08-26 21:21 UTC (permalink / raw)
To: sandeen, darrick.wong; +Cc: linux-xfs, Christoph Hellwig
From: Darrick J. Wong <darrick.wong@oracle.com>
In the kernel, xfs_buf_delwri_queue increments the buffer reference
count before putting the buffer on the buffer list, and the refcount is
decremented after the io completes for a net refcount change of zero.
In userspace, delwri_queue calls libxfs_writebuf, which puts the buffer.
delwri_queue is a no-op, for a net refcount change of -1. This creates
problems for any callers that expect a net change of zero, so increment
the buffer refcount before calling writebuf.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
libxfs/libxfs_io.h | 7 +++++++
libxfs/libxfs_priv.h | 1 -
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/libxfs/libxfs_io.h b/libxfs/libxfs_io.h
index 7dcb4bff..09ed043b 100644
--- a/libxfs/libxfs_io.h
+++ b/libxfs/libxfs_io.h
@@ -245,4 +245,11 @@ xfs_buf_get_uncached(struct xfs_buftarg *targ, size_t bblen, int flags)
return bp;
}
+static inline void
+xfs_buf_delwri_queue(struct xfs_buf *bp, struct list_head *buffer_list)
+{
+ bp->b_node.cn_count++;
+ libxfs_writebuf(bp, 0);
+}
+
#endif /* __LIBXFS_IO_H__ */
diff --git a/libxfs/libxfs_priv.h b/libxfs/libxfs_priv.h
index 99cfac1f..ed2d665a 100644
--- a/libxfs/libxfs_priv.h
+++ b/libxfs/libxfs_priv.h
@@ -382,7 +382,6 @@ roundup_64(uint64_t x, uint32_t y)
#define xfs_buf_relse(bp) libxfs_putbuf(bp)
#define xfs_buf_get(devp,blkno,len) (libxfs_getbuf((devp), (blkno), (len)))
#define xfs_bwrite(bp) libxfs_writebuf((bp), 0)
-#define xfs_buf_delwri_queue(bp, bl) libxfs_writebuf((bp), 0)
#define xfs_buf_delwri_submit(bl) (0)
#define xfs_buf_oneshot(bp) ((void) 0)
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/4] libxfs: make xfs_buf_delwri_submit actually do something
2019-08-26 21:21 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
2019-08-26 21:21 ` [PATCH 1/4] libxfs: fix uncached buffer refcounting Darrick J. Wong
2019-08-26 21:21 ` [PATCH 2/4] libxfs: fix buffer refcounting in delwri_queue Darrick J. Wong
@ 2019-08-26 21:21 ` Darrick J. Wong
2019-08-26 21:22 ` [PATCH 4/4] mkfs: use libxfs to write out new AGs Darrick J. Wong
3 siblings, 0 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-08-26 21:21 UTC (permalink / raw)
To: sandeen, darrick.wong; +Cc: linux-xfs, Christoph Hellwig
From: Darrick J. Wong <darrick.wong@oracle.com>
xfs_buf_delwri_queue doesn't report errors, which means that if the
buffer write fails we have no way of knowing that something bad
happened. In the kernel we queue and then submit buffers, and the
submit call communicates errors to callers. Do the same here since
we're going to start using the AG header initialization functions, which
use delwri_{queue,submit} heavily.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
libxfs/libxfs_io.h | 6 +++++-
libxfs/libxfs_priv.h | 1 -
libxfs/rdwr.c | 25 +++++++++++++++++++++++++
3 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/libxfs/libxfs_io.h b/libxfs/libxfs_io.h
index 09ed043b..579df52b 100644
--- a/libxfs/libxfs_io.h
+++ b/libxfs/libxfs_io.h
@@ -71,6 +71,7 @@ typedef struct xfs_buf {
struct xfs_buf_map *b_maps;
struct xfs_buf_map __b_map;
int b_nmaps;
+ struct list_head b_list;
#ifdef XFS_BUF_TRACING
struct list_head b_lock_list;
const char *b_func;
@@ -245,11 +246,14 @@ xfs_buf_get_uncached(struct xfs_buftarg *targ, size_t bblen, int flags)
return bp;
}
+/* Push a single buffer on a delwri queue. */
static inline void
xfs_buf_delwri_queue(struct xfs_buf *bp, struct list_head *buffer_list)
{
bp->b_node.cn_count++;
- libxfs_writebuf(bp, 0);
+ list_add_tail(&bp->b_list, buffer_list);
}
+int xfs_buf_delwri_submit(struct list_head *buffer_list);
+
#endif /* __LIBXFS_IO_H__ */
diff --git a/libxfs/libxfs_priv.h b/libxfs/libxfs_priv.h
index ed2d665a..57b58642 100644
--- a/libxfs/libxfs_priv.h
+++ b/libxfs/libxfs_priv.h
@@ -382,7 +382,6 @@ roundup_64(uint64_t x, uint32_t y)
#define xfs_buf_relse(bp) libxfs_putbuf(bp)
#define xfs_buf_get(devp,blkno,len) (libxfs_getbuf((devp), (blkno), (len)))
#define xfs_bwrite(bp) libxfs_writebuf((bp), 0)
-#define xfs_buf_delwri_submit(bl) (0)
#define xfs_buf_oneshot(bp) ((void) 0)
#define XBRW_READ LIBXFS_BREAD
diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c
index 3282f6de..0d3e6089 100644
--- a/libxfs/rdwr.c
+++ b/libxfs/rdwr.c
@@ -1473,3 +1473,28 @@ libxfs_irele(
libxfs_idestroy(ip);
kmem_zone_free(xfs_inode_zone, ip);
}
+
+/*
+ * Write out a buffer list synchronously.
+ *
+ * This will take the @buffer_list, write all buffers out and wait for I/O
+ * completion on all of the buffers. @buffer_list is consumed by the function,
+ * so callers must have some other way of tracking buffers if they require such
+ * functionality.
+ */
+int
+xfs_buf_delwri_submit(
+ struct list_head *buffer_list)
+{
+ struct xfs_buf *bp, *n;
+ int error = 0, error2;
+
+ list_for_each_entry_safe(bp, n, buffer_list, b_list) {
+ list_del_init(&bp->b_list);
+ error2 = libxfs_writebuf(bp, 0);
+ if (!error)
+ error = error2;
+ }
+
+ return error;
+}
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 4/4] mkfs: use libxfs to write out new AGs
2019-08-26 21:21 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
` (2 preceding siblings ...)
2019-08-26 21:21 ` [PATCH 3/4] libxfs: make xfs_buf_delwri_submit actually do something Darrick J. Wong
@ 2019-08-26 21:22 ` Darrick J. Wong
3 siblings, 0 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-08-26 21:22 UTC (permalink / raw)
To: sandeen, darrick.wong; +Cc: linux-xfs, Christoph Hellwig
From: Darrick J. Wong <darrick.wong@oracle.com>
Use the libxfs AG initialization functions to write out the new
filesystem instead of open-coding everything.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
include/libxfs.h | 1
libxfs/libxfs_api_defs.h | 3
libxfs/util.c | 1
mkfs/xfs_mkfs.c | 371 ++++------------------------------------------
4 files changed, 41 insertions(+), 335 deletions(-)
diff --git a/include/libxfs.h b/include/libxfs.h
index dd5fe542..3bf7feab 100644
--- a/include/libxfs.h
+++ b/include/libxfs.h
@@ -63,6 +63,7 @@ extern uint32_t crc32c_le(uint32_t crc, unsigned char const *p, size_t len);
#include "xfs_bmap.h"
#include "xfs_trace.h"
#include "xfs_trans.h"
+#include "xfs_ag.h"
#include "xfs_rmap_btree.h"
#include "xfs_rmap.h"
#include "xfs_refcount_btree.h"
diff --git a/libxfs/libxfs_api_defs.h b/libxfs/libxfs_api_defs.h
index 0ae21318..645c9b1b 100644
--- a/libxfs/libxfs_api_defs.h
+++ b/libxfs/libxfs_api_defs.h
@@ -154,4 +154,7 @@
#define LIBXFS_ATTR_CREATE ATTR_CREATE
#define LIBXFS_ATTR_REPLACE ATTR_REPLACE
+#define xfs_ag_init_headers libxfs_ag_init_headers
+#define xfs_buf_delwri_submit libxfs_buf_delwri_submit
+
#endif /* __LIBXFS_API_DEFS_H__ */
diff --git a/libxfs/util.c b/libxfs/util.c
index 95054094..885dd42b 100644
--- a/libxfs/util.c
+++ b/libxfs/util.c
@@ -762,3 +762,4 @@ xfs_fs_mark_healthy(
spin_unlock(&mp->m_sb_lock);
}
+void xfs_ag_geom_health(struct xfs_perag *pag, struct xfs_ag_geometry *ageo) { }
diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c
index d05a6898..9cca4f11 100644
--- a/mkfs/xfs_mkfs.c
+++ b/mkfs/xfs_mkfs.c
@@ -3430,349 +3430,30 @@ initialise_ag_headers(
struct xfs_mount *mp,
struct xfs_sb *sbp,
xfs_agnumber_t agno,
- int *worst_freelist)
+ int *worst_freelist,
+ struct list_head *buffer_list)
{
+ struct aghdr_init_data id = {
+ .agno = agno,
+ .agsize = cfg->agsize,
+ };
struct xfs_perag *pag = libxfs_perag_get(mp, agno);
- struct xfs_agfl *agfl;
- struct xfs_agf *agf;
- struct xfs_agi *agi;
- struct xfs_buf *buf;
- struct xfs_btree_block *block;
- struct xfs_alloc_rec *arec;
- struct xfs_alloc_rec *nrec;
- int bucket;
- uint64_t agsize = cfg->agsize;
- xfs_agblock_t agblocks;
- bool is_log_ag = false;
- int c;
-
- if (cfg->loginternal && agno == cfg->logagno)
- is_log_ag = true;
-
- /*
- * Superblock.
- */
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AG_DADDR(mp, agno, XFS_SB_DADDR),
- XFS_FSS_TO_BB(mp, 1));
- buf->b_ops = &xfs_sb_buf_ops;
- memset(buf->b_addr, 0, cfg->sectorsize);
- libxfs_sb_to_disk(buf->b_addr, sbp);
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
+ int error;
- /*
- * AG header block: freespace
- */
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AG_DADDR(mp, agno, XFS_AGF_DADDR(mp)),
- XFS_FSS_TO_BB(mp, 1));
- buf->b_ops = &xfs_agf_buf_ops;
- agf = XFS_BUF_TO_AGF(buf);
- memset(agf, 0, cfg->sectorsize);
if (agno == cfg->agcount - 1)
- agsize = cfg->dblocks - (xfs_rfsblock_t)(agno * agsize);
- agf->agf_magicnum = cpu_to_be32(XFS_AGF_MAGIC);
- agf->agf_versionnum = cpu_to_be32(XFS_AGF_VERSION);
- agf->agf_seqno = cpu_to_be32(agno);
- agf->agf_length = cpu_to_be32(agsize);
- agf->agf_roots[XFS_BTNUM_BNOi] = cpu_to_be32(XFS_BNO_BLOCK(mp));
- agf->agf_roots[XFS_BTNUM_CNTi] = cpu_to_be32(XFS_CNT_BLOCK(mp));
- agf->agf_levels[XFS_BTNUM_BNOi] = cpu_to_be32(1);
- agf->agf_levels[XFS_BTNUM_CNTi] = cpu_to_be32(1);
- pag->pagf_levels[XFS_BTNUM_BNOi] = 1;
- pag->pagf_levels[XFS_BTNUM_CNTi] = 1;
-
- if (xfs_sb_version_hasrmapbt(sbp)) {
- agf->agf_roots[XFS_BTNUM_RMAPi] = cpu_to_be32(XFS_RMAP_BLOCK(mp));
- agf->agf_levels[XFS_BTNUM_RMAPi] = cpu_to_be32(1);
- agf->agf_rmap_blocks = cpu_to_be32(1);
- }
+ id.agsize = cfg->dblocks - (xfs_rfsblock_t)(agno * cfg->agsize);
- if (xfs_sb_version_hasreflink(sbp)) {
- agf->agf_refcount_root = cpu_to_be32(libxfs_refc_block(mp));
- agf->agf_refcount_level = cpu_to_be32(1);
- agf->agf_refcount_blocks = cpu_to_be32(1);
+ INIT_LIST_HEAD(&id.buffer_list);
+ error = -libxfs_ag_init_headers(mp, &id);
+ if (error) {
+ fprintf(stderr, _("AG header init failed, error %d\n"), error);
+ exit(1);
}
- agf->agf_flfirst = 0;
- agf->agf_fllast = cpu_to_be32(libxfs_agfl_size(mp) - 1);
- agf->agf_flcount = 0;
- agblocks = (xfs_agblock_t)(agsize - libxfs_prealloc_blocks(mp));
- agf->agf_freeblks = cpu_to_be32(agblocks);
- agf->agf_longest = cpu_to_be32(agblocks);
+ list_splice_tail_init(&id.buffer_list, buffer_list);
- if (xfs_sb_version_hascrc(sbp))
- platform_uuid_copy(&agf->agf_uuid, &sbp->sb_uuid);
-
- if (is_log_ag) {
- be32_add_cpu(&agf->agf_freeblks, -(int64_t)cfg->logblocks);
- agf->agf_longest = cpu_to_be32(agsize -
- XFS_FSB_TO_AGBNO(mp, cfg->logstart) - cfg->logblocks);
- }
if (libxfs_alloc_min_freelist(mp, pag) > *worst_freelist)
*worst_freelist = libxfs_alloc_min_freelist(mp, pag);
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
- /*
- * AG freelist header block
- */
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AG_DADDR(mp, agno, XFS_AGFL_DADDR(mp)),
- XFS_FSS_TO_BB(mp, 1));
- buf->b_ops = &xfs_agfl_buf_ops;
- agfl = XFS_BUF_TO_AGFL(buf);
- /* setting to 0xff results in initialisation to NULLAGBLOCK */
- memset(agfl, 0xff, cfg->sectorsize);
- if (xfs_sb_version_hascrc(sbp)) {
- agfl->agfl_magicnum = cpu_to_be32(XFS_AGFL_MAGIC);
- agfl->agfl_seqno = cpu_to_be32(agno);
- platform_uuid_copy(&agfl->agfl_uuid, &sbp->sb_uuid);
- for (bucket = 0; bucket < libxfs_agfl_size(mp); bucket++)
- agfl->agfl_bno[bucket] = cpu_to_be32(NULLAGBLOCK);
- }
-
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
- /*
- * AG header block: inodes
- */
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AG_DADDR(mp, agno, XFS_AGI_DADDR(mp)),
- XFS_FSS_TO_BB(mp, 1));
- agi = XFS_BUF_TO_AGI(buf);
- buf->b_ops = &xfs_agi_buf_ops;
- memset(agi, 0, cfg->sectorsize);
- agi->agi_magicnum = cpu_to_be32(XFS_AGI_MAGIC);
- agi->agi_versionnum = cpu_to_be32(XFS_AGI_VERSION);
- agi->agi_seqno = cpu_to_be32(agno);
- agi->agi_length = cpu_to_be32(agsize);
- agi->agi_count = 0;
- agi->agi_root = cpu_to_be32(XFS_IBT_BLOCK(mp));
- agi->agi_level = cpu_to_be32(1);
- if (xfs_sb_version_hasfinobt(sbp)) {
- agi->agi_free_root = cpu_to_be32(XFS_FIBT_BLOCK(mp));
- agi->agi_free_level = cpu_to_be32(1);
- }
- agi->agi_freecount = 0;
- agi->agi_newino = cpu_to_be32(NULLAGINO);
- agi->agi_dirino = cpu_to_be32(NULLAGINO);
- if (xfs_sb_version_hascrc(sbp))
- platform_uuid_copy(&agi->agi_uuid, &sbp->sb_uuid);
- for (c = 0; c < XFS_AGI_UNLINKED_BUCKETS; c++)
- agi->agi_unlinked[c] = cpu_to_be32(NULLAGINO);
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
- /*
- * BNO btree root block
- */
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AGB_TO_DADDR(mp, agno, XFS_BNO_BLOCK(mp)),
- BTOBB(cfg->blocksize));
- buf->b_ops = &xfs_bnobt_buf_ops;
- block = XFS_BUF_TO_BLOCK(buf);
- memset(block, 0, cfg->blocksize);
- libxfs_btree_init_block(mp, buf, XFS_BTNUM_BNO, 0, 1, agno);
-
- arec = XFS_ALLOC_REC_ADDR(mp, block, 1);
- arec->ar_startblock = cpu_to_be32(libxfs_prealloc_blocks(mp));
- if (is_log_ag) {
- xfs_agblock_t start = XFS_FSB_TO_AGBNO(mp, cfg->logstart);
-
- ASSERT(start >= libxfs_prealloc_blocks(mp));
- if (start != libxfs_prealloc_blocks(mp)) {
- /*
- * Modify first record to pad stripe align of log
- */
- arec->ar_blockcount = cpu_to_be32(start -
- libxfs_prealloc_blocks(mp));
- nrec = arec + 1;
- /*
- * Insert second record at start of internal log
- * which then gets trimmed.
- */
- nrec->ar_startblock = cpu_to_be32(
- be32_to_cpu(arec->ar_startblock) +
- be32_to_cpu(arec->ar_blockcount));
- arec = nrec;
- be16_add_cpu(&block->bb_numrecs, 1);
- }
- /*
- * Change record start to after the internal log
- */
- be32_add_cpu(&arec->ar_startblock, cfg->logblocks);
- }
- /*
- * Calculate the record block count and check for the case where
- * the log might have consumed all available space in the AG. If
- * so, reset the record count to 0 to avoid exposure of an invalid
- * record start block.
- */
- arec->ar_blockcount = cpu_to_be32(agsize -
- be32_to_cpu(arec->ar_startblock));
- if (!arec->ar_blockcount)
- block->bb_numrecs = 0;
-
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
- /*
- * CNT btree root block
- */
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AGB_TO_DADDR(mp, agno, XFS_CNT_BLOCK(mp)),
- BTOBB(cfg->blocksize));
- buf->b_ops = &xfs_cntbt_buf_ops;
- block = XFS_BUF_TO_BLOCK(buf);
- memset(block, 0, cfg->blocksize);
- libxfs_btree_init_block(mp, buf, XFS_BTNUM_CNT, 0, 1, agno);
-
- arec = XFS_ALLOC_REC_ADDR(mp, block, 1);
- arec->ar_startblock = cpu_to_be32(libxfs_prealloc_blocks(mp));
- if (is_log_ag) {
- xfs_agblock_t start = XFS_FSB_TO_AGBNO(mp, cfg->logstart);
-
- ASSERT(start >= libxfs_prealloc_blocks(mp));
- if (start != libxfs_prealloc_blocks(mp)) {
- arec->ar_blockcount = cpu_to_be32(start -
- libxfs_prealloc_blocks(mp));
- nrec = arec + 1;
- nrec->ar_startblock = cpu_to_be32(
- be32_to_cpu(arec->ar_startblock) +
- be32_to_cpu(arec->ar_blockcount));
- arec = nrec;
- be16_add_cpu(&block->bb_numrecs, 1);
- }
- be32_add_cpu(&arec->ar_startblock, cfg->logblocks);
- }
- /*
- * Calculate the record block count and check for the case where
- * the log might have consumed all available space in the AG. If
- * so, reset the record count to 0 to avoid exposure of an invalid
- * record start block.
- */
- arec->ar_blockcount = cpu_to_be32(agsize -
- be32_to_cpu(arec->ar_startblock));
- if (!arec->ar_blockcount)
- block->bb_numrecs = 0;
-
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
- /*
- * refcount btree root block
- */
- if (xfs_sb_version_hasreflink(sbp)) {
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AGB_TO_DADDR(mp, agno, libxfs_refc_block(mp)),
- BTOBB(cfg->blocksize));
- buf->b_ops = &xfs_refcountbt_buf_ops;
-
- block = XFS_BUF_TO_BLOCK(buf);
- memset(block, 0, cfg->blocksize);
- libxfs_btree_init_block(mp, buf, XFS_BTNUM_REFC, 0, 0, agno);
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
- }
-
- /*
- * INO btree root block
- */
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AGB_TO_DADDR(mp, agno, XFS_IBT_BLOCK(mp)),
- BTOBB(cfg->blocksize));
- buf->b_ops = &xfs_inobt_buf_ops;
- block = XFS_BUF_TO_BLOCK(buf);
- memset(block, 0, cfg->blocksize);
- libxfs_btree_init_block(mp, buf, XFS_BTNUM_INO, 0, 0, agno);
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
-
- /*
- * Free INO btree root block
- */
- if (xfs_sb_version_hasfinobt(sbp)) {
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AGB_TO_DADDR(mp, agno, XFS_FIBT_BLOCK(mp)),
- BTOBB(cfg->blocksize));
- buf->b_ops = &xfs_finobt_buf_ops;
- block = XFS_BUF_TO_BLOCK(buf);
- memset(block, 0, cfg->blocksize);
- libxfs_btree_init_block(mp, buf, XFS_BTNUM_FINO, 0, 0, agno);
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
- }
-
- /* RMAP btree root block */
- if (xfs_sb_version_hasrmapbt(sbp)) {
- struct xfs_rmap_rec *rrec;
-
- buf = libxfs_getbuf(mp->m_ddev_targp,
- XFS_AGB_TO_DADDR(mp, agno, XFS_RMAP_BLOCK(mp)),
- BTOBB(cfg->blocksize));
- buf->b_ops = &xfs_rmapbt_buf_ops;
- block = XFS_BUF_TO_BLOCK(buf);
- memset(block, 0, cfg->blocksize);
-
- libxfs_btree_init_block(mp, buf, XFS_BTNUM_RMAP, 0, 0, agno);
-
- /*
- * mark the AG header regions as static metadata
- * The BNO btree block is the first block after the
- * headers, so it's location defines the size of region
- * the static metadata consumes.
- */
- rrec = XFS_RMAP_REC_ADDR(block, 1);
- rrec->rm_startblock = 0;
- rrec->rm_blockcount = cpu_to_be32(XFS_BNO_BLOCK(mp));
- rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_FS);
- rrec->rm_offset = 0;
- be16_add_cpu(&block->bb_numrecs, 1);
-
- /* account freespace btree root blocks */
- rrec = XFS_RMAP_REC_ADDR(block, 2);
- rrec->rm_startblock = cpu_to_be32(XFS_BNO_BLOCK(mp));
- rrec->rm_blockcount = cpu_to_be32(2);
- rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_AG);
- rrec->rm_offset = 0;
- be16_add_cpu(&block->bb_numrecs, 1);
-
- /* account inode btree root blocks */
- rrec = XFS_RMAP_REC_ADDR(block, 3);
- rrec->rm_startblock = cpu_to_be32(XFS_IBT_BLOCK(mp));
- rrec->rm_blockcount = cpu_to_be32(XFS_RMAP_BLOCK(mp) -
- XFS_IBT_BLOCK(mp));
- rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_INOBT);
- rrec->rm_offset = 0;
- be16_add_cpu(&block->bb_numrecs, 1);
-
- /* account for rmap btree root */
- rrec = XFS_RMAP_REC_ADDR(block, 4);
- rrec->rm_startblock = cpu_to_be32(XFS_RMAP_BLOCK(mp));
- rrec->rm_blockcount = cpu_to_be32(1);
- rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_AG);
- rrec->rm_offset = 0;
- be16_add_cpu(&block->bb_numrecs, 1);
-
- /* account for refcount btree root */
- if (xfs_sb_version_hasreflink(sbp)) {
- rrec = XFS_RMAP_REC_ADDR(block, 5);
- rrec->rm_startblock = cpu_to_be32(libxfs_refc_block(mp));
- rrec->rm_blockcount = cpu_to_be32(1);
- rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_REFC);
- rrec->rm_offset = 0;
- be16_add_cpu(&block->bb_numrecs, 1);
- }
-
- /* account for the log space */
- if (is_log_ag) {
- rrec = XFS_RMAP_REC_ADDR(block,
- be16_to_cpu(block->bb_numrecs) + 1);
- rrec->rm_startblock = cpu_to_be32(
- XFS_FSB_TO_AGBNO(mp, cfg->logstart));
- rrec->rm_blockcount = cpu_to_be32(cfg->logblocks);
- rrec->rm_owner = cpu_to_be64(XFS_RMAP_OWN_LOG);
- rrec->rm_offset = 0;
- be16_add_cpu(&block->bb_numrecs, 1);
- }
-
- libxfs_writebuf(buf, LIBXFS_EXIT_ON_FAILURE);
- }
-
libxfs_perag_put(pag);
}
@@ -3895,6 +3576,8 @@ main(
},
};
+ struct list_head buffer_list;
+
platform_uuid_generate(&cli.uuid);
progname = basename(argv[0]);
setlocale(LC_ALL, "");
@@ -4086,8 +3769,26 @@ main(
/*
* Initialise all the static on disk metadata.
*/
- for (agno = 0; agno < cfg.agcount; agno++)
- initialise_ag_headers(&cfg, mp, sbp, agno, &worst_freelist);
+ INIT_LIST_HEAD(&buffer_list);
+ for (agno = 0; agno < cfg.agcount; agno++) {
+ initialise_ag_headers(&cfg, mp, sbp, agno, &worst_freelist,
+ &buffer_list);
+
+ if (agno % 16)
+ continue;
+
+ if (libxfs_buf_delwri_submit(&buffer_list)) {
+ fprintf(stderr, _("%s: writing AG headers failed\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ if (libxfs_buf_delwri_submit(&buffer_list)) {
+ fprintf(stderr, _("%s: writing AG headers failed\n"),
+ progname);
+ exit(1);
+ }
/*
* Initialise the freespace freelists (i.e. AGFLs) in each AG.
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 1/4] libxfs: fix uncached buffer refcounting
2019-09-25 21:32 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
@ 2019-09-25 21:32 ` Darrick J. Wong
0 siblings, 0 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-09-25 21:32 UTC (permalink / raw)
To: sandeen, darrick.wong; +Cc: linux-xfs, Christoph Hellwig
From: Darrick J. Wong <darrick.wong@oracle.com>
Currently, uncached buffers in userspace are created with zero refcount
and are fed to cache_node_put when they're released. This is totally
broken -- the refcount should be 1 (because the caller now holds a
reference) and we should never be dumping uncached buffers into the
cache. Fix both of these problems.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
libxfs/libxfs_io.h | 18 ++++++++++++++++++
libxfs/libxfs_priv.h | 2 --
libxfs/rdwr.c | 5 ++++-
3 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/libxfs/libxfs_io.h b/libxfs/libxfs_io.h
index 0b8d9774..7dcb4bff 100644
--- a/libxfs/libxfs_io.h
+++ b/libxfs/libxfs_io.h
@@ -227,4 +227,22 @@ xfs_buf_associate_memory(struct xfs_buf *bp, void *mem, size_t len)
return 0;
}
+/*
+ * Allocate an uncached buffer that points nowhere. The refcount will be 1,
+ * and the cache node hash list will be empty to indicate that it's uncached.
+ */
+static inline struct xfs_buf *
+xfs_buf_get_uncached(struct xfs_buftarg *targ, size_t bblen, int flags)
+{
+ struct xfs_buf *bp;
+
+ bp = libxfs_getbufr(targ, XFS_BUF_DADDR_NULL, bblen);
+ if (!bp)
+ return NULL;
+
+ INIT_LIST_HEAD(&bp->b_node.cn_hash);
+ bp->b_node.cn_count = 1;
+ return bp;
+}
+
#endif /* __LIBXFS_IO_H__ */
diff --git a/libxfs/libxfs_priv.h b/libxfs/libxfs_priv.h
index 96d74bfa..b05e082a 100644
--- a/libxfs/libxfs_priv.h
+++ b/libxfs/libxfs_priv.h
@@ -379,8 +379,6 @@ roundup_64(uint64_t x, uint32_t y)
(len) = __bar; /* no set-but-unused warning */ \
NULL; \
})
-#define xfs_buf_get_uncached(t,n,f) \
- libxfs_getbufr((t), XFS_BUF_DADDR_NULL, (n));
#define xfs_buf_relse(bp) libxfs_putbuf(bp)
#define xfs_buf_get(devp,blkno,len) (libxfs_getbuf((devp), (blkno), (len)))
#define xfs_bwrite(bp) libxfs_writebuf((bp), 0)
diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c
index 1f2c129b..3282f6de 100644
--- a/libxfs/rdwr.c
+++ b/libxfs/rdwr.c
@@ -867,7 +867,10 @@ libxfs_putbuf(xfs_buf_t *bp)
}
}
- cache_node_put(libxfs_bcache, (struct cache_node *)bp);
+ if (!list_empty(&bp->b_node.cn_hash))
+ cache_node_put(libxfs_bcache, (struct cache_node *)bp);
+ else if (--bp->b_node.cn_count == 0)
+ libxfs_putbufr(bp);
}
void
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 1/4] libxfs: fix uncached buffer refcounting
2019-09-06 3:34 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
@ 2019-09-06 3:34 ` Darrick J. Wong
0 siblings, 0 replies; 7+ messages in thread
From: Darrick J. Wong @ 2019-09-06 3:34 UTC (permalink / raw)
To: sandeen, darrick.wong; +Cc: linux-xfs, Christoph Hellwig
From: Darrick J. Wong <darrick.wong@oracle.com>
Currently, uncached buffers in userspace are created with zero refcount
and are fed to cache_node_put when they're released. This is totally
broken -- the refcount should be 1 (because the caller now holds a
reference) and we should never be dumping uncached buffers into the
cache. Fix both of these problems.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
libxfs/libxfs_io.h | 18 ++++++++++++++++++
libxfs/libxfs_priv.h | 2 --
libxfs/rdwr.c | 5 ++++-
3 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/libxfs/libxfs_io.h b/libxfs/libxfs_io.h
index 0b8d9774..7dcb4bff 100644
--- a/libxfs/libxfs_io.h
+++ b/libxfs/libxfs_io.h
@@ -227,4 +227,22 @@ xfs_buf_associate_memory(struct xfs_buf *bp, void *mem, size_t len)
return 0;
}
+/*
+ * Allocate an uncached buffer that points nowhere. The refcount will be 1,
+ * and the cache node hash list will be empty to indicate that it's uncached.
+ */
+static inline struct xfs_buf *
+xfs_buf_get_uncached(struct xfs_buftarg *targ, size_t bblen, int flags)
+{
+ struct xfs_buf *bp;
+
+ bp = libxfs_getbufr(targ, XFS_BUF_DADDR_NULL, bblen);
+ if (!bp)
+ return NULL;
+
+ INIT_LIST_HEAD(&bp->b_node.cn_hash);
+ bp->b_node.cn_count = 1;
+ return bp;
+}
+
#endif /* __LIBXFS_IO_H__ */
diff --git a/libxfs/libxfs_priv.h b/libxfs/libxfs_priv.h
index 96d74bfa..b05e082a 100644
--- a/libxfs/libxfs_priv.h
+++ b/libxfs/libxfs_priv.h
@@ -379,8 +379,6 @@ roundup_64(uint64_t x, uint32_t y)
(len) = __bar; /* no set-but-unused warning */ \
NULL; \
})
-#define xfs_buf_get_uncached(t,n,f) \
- libxfs_getbufr((t), XFS_BUF_DADDR_NULL, (n));
#define xfs_buf_relse(bp) libxfs_putbuf(bp)
#define xfs_buf_get(devp,blkno,len) (libxfs_getbuf((devp), (blkno), (len)))
#define xfs_bwrite(bp) libxfs_writebuf((bp), 0)
diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c
index 1f2c129b..3282f6de 100644
--- a/libxfs/rdwr.c
+++ b/libxfs/rdwr.c
@@ -867,7 +867,10 @@ libxfs_putbuf(xfs_buf_t *bp)
}
}
- cache_node_put(libxfs_bcache, (struct cache_node *)bp);
+ if (!list_empty(&bp->b_node.cn_hash))
+ cache_node_put(libxfs_bcache, (struct cache_node *)bp);
+ else if (--bp->b_node.cn_count == 0)
+ libxfs_putbufr(bp);
}
void
^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-09-25 21:32 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-26 21:21 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
2019-08-26 21:21 ` [PATCH 1/4] libxfs: fix uncached buffer refcounting Darrick J. Wong
2019-08-26 21:21 ` [PATCH 2/4] libxfs: fix buffer refcounting in delwri_queue Darrick J. Wong
2019-08-26 21:21 ` [PATCH 3/4] libxfs: make xfs_buf_delwri_submit actually do something Darrick J. Wong
2019-08-26 21:22 ` [PATCH 4/4] mkfs: use libxfs to write out new AGs Darrick J. Wong
2019-09-06 3:34 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
2019-09-06 3:34 ` [PATCH 1/4] libxfs: fix uncached buffer refcounting Darrick J. Wong
2019-09-25 21:32 [PATCH 0/4] xfsprogs: help mkfs shed its AG initialization code Darrick J. Wong
2019-09-25 21:32 ` [PATCH 1/4] libxfs: fix uncached buffer refcounting Darrick J. Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).