* [PATCH v3 0/4] xfsrestore: fix inventory unpacking
@ 2022-10-13 3:15 Donald Douwsma
2022-10-13 3:15 ` [PATCH 1/4] xfsrestore: fix on-media inventory media unpacking Donald Douwsma
` (3 more replies)
0 siblings, 4 replies; 13+ messages in thread
From: Donald Douwsma @ 2022-10-13 3:15 UTC (permalink / raw)
To: linux-xfs; +Cc: Donald Douwsma
When xfsrestore reads its inventory from tape it fails to convert the media
record on bigendian systems, if the online inventory is unavailable this results
in invalid data being writen to the online inventory and failure to restore
non-directory files.
The series fixes the converstion and related issues.
---
v2
- Seperate out cleanup and content.c changes, fix whitespace.
- Show a full reproducer in the first patch.
V3
- Fix whitespace and Signed-off-by.
- Make for loop formatting consistent.
- Rename patches to make their intent clearer.
- Add xfsdump: fix on-media inventory stream packing.
- Add descriptions that reproduce the stream problems.
Donald Douwsma (4):
xfsrestore: fix on-media inventory media unpacking
xfsrestore: fix on-media inventory stream unpacking
xfsdump: fix on-media inventory stream packing
xfsrestore: untangle inventory unpacking logic
inventory/inv_stobj.c | 42 +++++++++++++++---------------------------
restore/content.c | 13 +++++--------
2 files changed, 20 insertions(+), 35 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH 1/4] xfsrestore: fix on-media inventory media unpacking 2022-10-13 3:15 [PATCH v3 0/4] xfsrestore: fix inventory unpacking Donald Douwsma @ 2022-10-13 3:15 ` Donald Douwsma 2022-10-13 3:15 ` [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking Donald Douwsma ` (2 subsequent siblings) 3 siblings, 0 replies; 13+ messages in thread From: Donald Douwsma @ 2022-10-13 3:15 UTC (permalink / raw) To: linux-xfs; +Cc: Donald Douwsma, Darrick J . Wong When xfsrestore reads the inventory from tape media it fails to convert media file records from bigendian. If the xfsdump inventory is not available xfsrestore will write this invalid record to the on-line inventory. [root@rhel8 ~]# xfsdump -L Test1 -M "" -f /dev/st0 /boot > /dev/null [root@rhel8 ~]# xfsdump -L Test2 -M "" -f /dev/st0 /boot > /dev/null [root@rhel8 ~]# rm -rf /var/lib/xfsdump/inventory/ [root@rhel8 ~]# mt -f /dev/nst0 asf 2 [root@rhel8 ~]# xfsrestore -f /dev/nst0 -L Test2 /tmp/test2 xfsrestore: using scsi tape (drive_scsitape) strategy xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status and control xfsrestore: searching media for dump xfsrestore: preparing drive xfsrestore: examining media file 3 xfsrestore: found dump matching specified label: xfsrestore: hostname: rhel8 xfsrestore: mount point: /boot xfsrestore: volume: /dev/sda1 xfsrestore: session time: Tue Sep 27 16:05:28 2022 xfsrestore: level: 0 xfsrestore: session label: "Test2" xfsrestore: media label: "" xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 xfsrestore: session id: 62402423-7ae0-49ed-8ecb-9e5bc7642b91 xfsrestore: media id: 47ba45ca-3417-4006-ab10-3dc6419b83e2 xfsrestore: incorporating on-media session inventory into online inventory xfsrestore: /var/lib/xfsdump/inventory created xfsrestore: using on-media session inventory xfsrestore: searching media for directory dump xfsrestore: rewinding xfsrestore: examining media file 0 xfsrestore: inventory session uuid (62402423-7ae0-49ed-8ecb-9e5bc7642b91) does not match the media header's session uuid (1771d9e8-a1ba-4e87-a61e-f6be97e41b45) xfsrestore: examining media file 1 xfsrestore: inventory session uuid (62402423-7ae0-49ed-8ecb-9e5bc7642b91) does not match the media header's session uuid (1771d9e8-a1ba-4e87-a61e-f6be97e41b45) xfsrestore: examining media file 2 xfsrestore: reading directories xfsrestore: 9 directories and 320 entries processed xfsrestore: directory post-processing xfsrestore: restore complete: 0 seconds elapsed xfsrestore: Restore Summary: xfsrestore: stream 0 /dev/nst0 OK (success) xfsrestore: Restore Status: SUCCESS [root@rhel8 ~]# xfsdump -I file system 0: fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 session 0: mount point: rhel8:/boot device: rhel8:/dev/sda1 time: Tue Sep 27 16:05:28 2022 session label: "Test2" session id: 62402423-7ae0-49ed-8ecb-9e5bc7642b91 level: 0 resumed: NO subtree: NO streams: 1 stream 0: pathname: /dev/st0 start: ino 133 offset 0 end: ino 1572997 offset 0 interrupted: YES media files: 1 media file 0: mfile index: 33554432 mfile type: data mfile size: 211187836911616 mfile start: ino 9583660007044415488 offset 0 mfile end: ino 9583686395323482112 offset 0 media label: "" media id: 47ba45ca-3417-4006-ab10-3dc6419b83e2 xfsdump: Dump Status: SUCCESS [root@rhel8 ~]# [root@rhel8 ~]# ls /tmp/test2 efi grub2 loader The invalid start and end inode information cause xfsrestore to consider that non-directory files do not reside in the current media and will fail to restore them. The behaviour of an initial restore may succeed if the position of the tape is such that the data file is encountered before the inventory file, or if there is only one dump session on tape, xfsrestore is somewhat inconsistent in this regard. Subsequent restores will use the invalid on-line inventory and fail to restore files. Fix this by correctly unpacking the inventory data. Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> --- inventory/inv_stobj.c | 27 +++++++-------------------- 1 file changed, 7 insertions(+), 20 deletions(-) diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c index c20e71c..b461666 100644 --- a/inventory/inv_stobj.c +++ b/inventory/inv_stobj.c @@ -1008,7 +1008,7 @@ stobj_unpack_sessinfo( size_t bufsz, invt_sessinfo_t *s) { - uint i; + uint i, j; char *tmpbuf; char *p = (char *)bufp; @@ -1087,26 +1087,13 @@ stobj_unpack_sessinfo( /* all the media files */ s->mfiles = (invt_mediafile_t *)p; - -#ifdef INVT_DELETION - { - int tmpfd = open("moids", O_RDWR | O_CREAT, S_IRUSR|S_IWUSR); - uint j; - invt_mediafile_t *mmf = s->mfiles; - for (i=0; i< s->ses->s_cur_nstreams; i++) { - for (j=0; j< s->strms[i].st_nmediafiles; - j++, mmf++) - xlate_invt_mediafile((invt_mediafile_t *)mmf, (invt_mediafile_t *)tmpbuf, 1); - bcopy(tmpbuf, mmf, sizeof(invt_mediafile_t)); - put_invtrecord(tmpfd, &mmf->mf_moid, - sizeof(uuid_t), 0, SEEK_END, 0); - } - close(tmpfd); - } -#endif for (i = 0; i < s->ses->s_cur_nstreams; i++) { - p += (size_t) (s->strms[i].st_nmediafiles) - * sizeof(invt_mediafile_t); + for(j = 0; j < s->strms[i].st_nmediafiles; j++) { + xlate_invt_mediafile((invt_mediafile_t *)p, + (invt_mediafile_t *)tmpbuf, 1); + bcopy(tmpbuf, p, sizeof(invt_mediafile_t)); + p += sizeof(invt_mediafile_t); + } } /* sanity check the size of the buffer given to us vs. the size it -- 2.31.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking 2022-10-13 3:15 [PATCH v3 0/4] xfsrestore: fix inventory unpacking Donald Douwsma 2022-10-13 3:15 ` [PATCH 1/4] xfsrestore: fix on-media inventory media unpacking Donald Douwsma @ 2022-10-13 3:15 ` Donald Douwsma 2022-10-13 19:05 ` Darrick J. Wong 2022-10-13 3:15 ` [PATCH 3/4] xfsdump: fix on-media inventory stream packing Donald Douwsma 2022-10-13 3:15 ` [PATCH 4/4] xfsrestore: untangle inventory unpacking logic Donald Douwsma 3 siblings, 1 reply; 13+ messages in thread From: Donald Douwsma @ 2022-10-13 3:15 UTC (permalink / raw) To: linux-xfs; +Cc: Donald Douwsma xfsdump can create multiple streams, when restoring the online inventory with multiple streams we fail to process these and assert when the inventory buffer is not fully decoded. [root@rhel8 ~]# xfsdump -L "Test1" -f /dev/nst0 -M tape1 -f /dev/nst1 -M tape2 /boot xfsdump: using scsi tape (drive_scsitape) strategy xfsdump: using scsi tape (drive_scsitape) strategy xfsdump: version 3.1.8 (dump format 3.0) - type ^C for status and control xfsdump: level 0 dump of rhel8:/boot xfsdump: dump date: Thu Oct 6 13:50:45 2022 xfsdump: session id: aa25fa48-4493-45c7-9027-61e53e486445 xfsdump: session label: "Test1" xfsdump: ino map phase 1: constructing initial dump list xfsdump: ino map phase 2: skipping (no pruning necessary) xfsdump: ino map phase 3: identifying stream starting points xfsdump: stream 0: ino 133 offset 0 to ino 28839 offset 0 xfsdump: stream 1: ino 28839 offset 0 to end xfsdump: ino map construction complete xfsdump: estimated dump size: 328720704 bytes xfsdump: estimated dump size per stream: 164375728 bytes xfsdump: /var/lib/xfsdump/inventory created xfsdump: drive 0: preparing drive xfsdump: drive 1: preparing drive xfsdump: drive 1: creating dump session media file 0 (media 0, file 0) xfsdump: drive 1: dumping ino map xfsdump: drive 1: dumping non-directory files xfsdump: drive 0: creating dump session media file 0 (media 0, file 0) xfsdump: drive 0: dumping ino map xfsdump: drive 0: dumping directories xfsdump: drive 0: dumping non-directory files xfsdump: drive 1: ending media file xfsdump: drive 1: media file size 166723584 bytes xfsdump: drive 1: waiting for synchronized session inventory dump xfsdump: drive 0: ending media file xfsdump: drive 0: media file size 165675008 bytes xfsdump: drive 0: waiting for synchronized session inventory dump xfsdump: drive 0: dumping session inventory xfsdump: drive 0: beginning inventory media file xfsdump: drive 0: media file 1 (media 0, file 1) xfsdump: drive 0: ending inventory media file xfsdump: drive 0: inventory media file size 2097152 bytes xfsdump: drive 0: writing stream terminator xfsdump: drive 0: beginning media stream terminator xfsdump: drive 0: media file 2 (media 0, file 2) xfsdump: drive 0: ending media stream terminator xfsdump: drive 0: media stream terminator size 1048576 bytes xfsdump: drive 1: dumping session inventory xfsdump: drive 1: beginning inventory media file xfsdump: drive 1: media file 1 (media 0, file 1) xfsdump: drive 1: ending inventory media file xfsdump: drive 1: inventory media file size 2097152 bytes xfsdump: drive 1: writing stream terminator xfsdump: drive 1: beginning media stream terminator xfsdump: drive 1: media file 2 (media 0, file 2) xfsdump: drive 1: ending media stream terminator xfsdump: drive 1: media stream terminator size 1048576 bytes xfsdump: dump size (non-dir files) : 328189016 bytes xfsdump: dump complete: 4 seconds elapsed xfsdump: Dump Summary: xfsdump: stream 0 /dev/nst0 OK (success) xfsdump: stream 1 /dev/nst1 OK (success) xfsdump: Dump Status: SUCCESS [root@rhel8 ~]# xfsdump -I file system 0: fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 session 0: mount point: rhel8:/boot device: rhel8:/dev/sda1 time: Thu Oct 6 13:50:45 2022 session label: "Test1" session id: aa25fa48-4493-45c7-9027-61e53e486445 level: 0 resumed: NO subtree: NO streams: 2 stream 0: pathname: /dev/nst0 start: ino 133 offset 0 end: ino 28839 offset 0 interrupted: NO media files: 2 media file 0: mfile index: 0 mfile type: data mfile size: 165675008 mfile start: ino 133 offset 0 mfile end: ino 28839 offset 0 media label: "tape1" media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 media file 1: mfile index: 1 mfile type: inventory mfile size: 2097152 media label: "tape1" media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 stream 1: pathname: /dev/nst1 start: ino 28839 offset 0 end: ino 1572997 offset 0 interrupted: NO media files: 2 media file 0: mfile index: 0 mfile type: data mfile size: 166723584 mfile start: ino 28839 offset 0 mfile end: ino 1572997 offset 0 media label: "tape2" media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 media file 1: mfile index: 1 mfile type: inventory mfile size: 2097152 media label: "tape2" media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 xfsdump: Dump Status: SUCCESS [root@rhel8 ~]# mv /var/lib/xfsdump/inventory /var/lib/xfsdump/inventory_two_sessions [root@rhel8 ~]# xfsdump -I xfsdump: Dump Status: SUCCESS [root@rhel8 ~]# xfsrestore -L Test1 -f /dev/nst0 /tmp/test1/ xfsrestore: using scsi tape (drive_scsitape) strategy xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status and control xfsrestore: searching media for dump xfsrestore: preparing drive xfsrestore: examining media file 2 xfsrestore: found dump matching specified label: xfsrestore: hostname: rhel8 xfsrestore: mount point: /boot xfsrestore: volume: /dev/sda1 xfsrestore: session time: Thu Oct 6 13:50:45 2022 xfsrestore: level: 0 xfsrestore: session label: "Test1" xfsrestore: media label: "tape1" xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 xfsrestore: session id: aa25fa48-4493-45c7-9027-61e53e486445 xfsrestore: media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 xfsrestore: searching media for directory dump xfsrestore: rewinding xfsrestore: examining media file 0 xfsrestore: reading directories xfsrestore: 9 directories and 320 entries processed xfsrestore: directory post-processing xfsrestore: restoring non-directory files xfsrestore: examining media file 1 xfsrestore: inv_stobj.c:1119: stobj_unpack_sessinfo: Assertion `(size_t) ( p - (char *) bufp ) == bufsz' failed. Aborted (core dumped) Make sure we unpack multiple streams when restoring the online inventory from media. Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> --- inventory/inv_stobj.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c index b461666..025d431 100644 --- a/inventory/inv_stobj.c +++ b/inventory/inv_stobj.c @@ -1065,25 +1065,26 @@ stobj_unpack_sessinfo( return BOOL_FALSE; } + /* get the seshdr and then, the remainder of the session */ xlate_invt_seshdr((invt_seshdr_t *)p, (invt_seshdr_t *)tmpbuf, 1); bcopy(tmpbuf, p, sizeof(invt_seshdr_t)); - - /* get the seshdr and then, the remainder of the session */ s->seshdr = (invt_seshdr_t *)p; s->seshdr->sh_sess_off = -1; p += sizeof(invt_seshdr_t); - xlate_invt_session((invt_session_t *)p, (invt_session_t *)tmpbuf, 1); bcopy (tmpbuf, p, sizeof(invt_session_t)); s->ses = (invt_session_t *)p; p += sizeof(invt_session_t); /* the array of all the streams belonging to this session */ - xlate_invt_stream((invt_stream_t *)p, (invt_stream_t *)tmpbuf, 1); - bcopy(tmpbuf, p, sizeof(invt_stream_t)); s->strms = (invt_stream_t *)p; - p += s->ses->s_cur_nstreams * sizeof(invt_stream_t); + for (i = 0; i < s->ses->s_cur_nstreams; i++) { + xlate_invt_stream((invt_stream_t *)p, + (invt_stream_t *)tmpbuf, 1); + bcopy(tmpbuf, p, sizeof(invt_stream_t)); + p += sizeof(invt_stream_t); + } /* all the media files */ s->mfiles = (invt_mediafile_t *)p; -- 2.31.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking 2022-10-13 3:15 ` [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking Donald Douwsma @ 2022-10-13 19:05 ` Darrick J. Wong 2022-10-14 3:37 ` Donald Douwsma 0 siblings, 1 reply; 13+ messages in thread From: Darrick J. Wong @ 2022-10-13 19:05 UTC (permalink / raw) To: Donald Douwsma; +Cc: linux-xfs On Thu, Oct 13, 2022 at 02:15:16PM +1100, Donald Douwsma wrote: > xfsdump can create multiple streams, when restoring the online inventory > with multiple streams we fail to process these and assert when the > inventory buffer is not fully decoded. > > [root@rhel8 ~]# xfsdump -L "Test1" -f /dev/nst0 -M tape1 -f /dev/nst1 -M tape2 /boot > xfsdump: using scsi tape (drive_scsitape) strategy > xfsdump: using scsi tape (drive_scsitape) strategy > xfsdump: version 3.1.8 (dump format 3.0) - type ^C for status and control > xfsdump: level 0 dump of rhel8:/boot > xfsdump: dump date: Thu Oct 6 13:50:45 2022 > xfsdump: session id: aa25fa48-4493-45c7-9027-61e53e486445 > xfsdump: session label: "Test1" > xfsdump: ino map phase 1: constructing initial dump list > xfsdump: ino map phase 2: skipping (no pruning necessary) > xfsdump: ino map phase 3: identifying stream starting points > xfsdump: stream 0: ino 133 offset 0 to ino 28839 offset 0 > xfsdump: stream 1: ino 28839 offset 0 to end > xfsdump: ino map construction complete > xfsdump: estimated dump size: 328720704 bytes > xfsdump: estimated dump size per stream: 164375728 bytes > xfsdump: /var/lib/xfsdump/inventory created > xfsdump: drive 0: preparing drive > xfsdump: drive 1: preparing drive > xfsdump: drive 1: creating dump session media file 0 (media 0, file 0) > xfsdump: drive 1: dumping ino map > xfsdump: drive 1: dumping non-directory files > xfsdump: drive 0: creating dump session media file 0 (media 0, file 0) > xfsdump: drive 0: dumping ino map > xfsdump: drive 0: dumping directories > xfsdump: drive 0: dumping non-directory files > xfsdump: drive 1: ending media file > xfsdump: drive 1: media file size 166723584 bytes > xfsdump: drive 1: waiting for synchronized session inventory dump > xfsdump: drive 0: ending media file > xfsdump: drive 0: media file size 165675008 bytes > xfsdump: drive 0: waiting for synchronized session inventory dump > xfsdump: drive 0: dumping session inventory > xfsdump: drive 0: beginning inventory media file > xfsdump: drive 0: media file 1 (media 0, file 1) > xfsdump: drive 0: ending inventory media file > xfsdump: drive 0: inventory media file size 2097152 bytes > xfsdump: drive 0: writing stream terminator > xfsdump: drive 0: beginning media stream terminator > xfsdump: drive 0: media file 2 (media 0, file 2) > xfsdump: drive 0: ending media stream terminator > xfsdump: drive 0: media stream terminator size 1048576 bytes > xfsdump: drive 1: dumping session inventory > xfsdump: drive 1: beginning inventory media file > xfsdump: drive 1: media file 1 (media 0, file 1) > xfsdump: drive 1: ending inventory media file > xfsdump: drive 1: inventory media file size 2097152 bytes > xfsdump: drive 1: writing stream terminator > xfsdump: drive 1: beginning media stream terminator > xfsdump: drive 1: media file 2 (media 0, file 2) > xfsdump: drive 1: ending media stream terminator > xfsdump: drive 1: media stream terminator size 1048576 bytes > xfsdump: dump size (non-dir files) : 328189016 bytes > xfsdump: dump complete: 4 seconds elapsed > xfsdump: Dump Summary: > xfsdump: stream 0 /dev/nst0 OK (success) > xfsdump: stream 1 /dev/nst1 OK (success) > xfsdump: Dump Status: SUCCESS > [root@rhel8 ~]# xfsdump -I > file system 0: > fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 > session 0: > mount point: rhel8:/boot > device: rhel8:/dev/sda1 > time: Thu Oct 6 13:50:45 2022 > session label: "Test1" > session id: aa25fa48-4493-45c7-9027-61e53e486445 > level: 0 > resumed: NO > subtree: NO > streams: 2 > stream 0: > pathname: /dev/nst0 > start: ino 133 offset 0 > end: ino 28839 offset 0 > interrupted: NO > media files: 2 > media file 0: > mfile index: 0 > mfile type: data > mfile size: 165675008 > mfile start: ino 133 offset 0 > mfile end: ino 28839 offset 0 > media label: "tape1" > media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > media file 1: > mfile index: 1 > mfile type: inventory > mfile size: 2097152 > media label: "tape1" > media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > stream 1: > pathname: /dev/nst1 > start: ino 28839 offset 0 > end: ino 1572997 offset 0 > interrupted: NO > media files: 2 > media file 0: > mfile index: 0 > mfile type: data > mfile size: 166723584 > mfile start: ino 28839 offset 0 > mfile end: ino 1572997 offset 0 > media label: "tape2" > media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 > media file 1: > mfile index: 1 > mfile type: inventory > mfile size: 2097152 > media label: "tape2" > media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 > xfsdump: Dump Status: SUCCESS > [root@rhel8 ~]# mv /var/lib/xfsdump/inventory /var/lib/xfsdump/inventory_two_sessions > [root@rhel8 ~]# xfsdump -I > xfsdump: Dump Status: SUCCESS > > [root@rhel8 ~]# xfsrestore -L Test1 -f /dev/nst0 /tmp/test1/ > xfsrestore: using scsi tape (drive_scsitape) strategy > xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status and control > xfsrestore: searching media for dump > xfsrestore: preparing drive > xfsrestore: examining media file 2 > xfsrestore: found dump matching specified label: > xfsrestore: hostname: rhel8 > xfsrestore: mount point: /boot > xfsrestore: volume: /dev/sda1 > xfsrestore: session time: Thu Oct 6 13:50:45 2022 > xfsrestore: level: 0 > xfsrestore: session label: "Test1" > xfsrestore: media label: "tape1" > xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 > xfsrestore: session id: aa25fa48-4493-45c7-9027-61e53e486445 > xfsrestore: media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > xfsrestore: searching media for directory dump > xfsrestore: rewinding > xfsrestore: examining media file 0 > xfsrestore: reading directories > xfsrestore: 9 directories and 320 entries processed > xfsrestore: directory post-processing > xfsrestore: restoring non-directory files > xfsrestore: examining media file 1 > xfsrestore: inv_stobj.c:1119: stobj_unpack_sessinfo: Assertion `(size_t) ( p - (char *) bufp ) == bufsz' failed. > Aborted (core dumped) > > Make sure we unpack multiple streams when restoring the online > inventory from media. > > Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> Much better now, though I hope there's an fstest coming to make sure that multistream restore/dump work properly. Reviewed-by: Darrick J. Wong <djwong@kernel.org> --D > --- > inventory/inv_stobj.c | 13 +++++++------ > 1 file changed, 7 insertions(+), 6 deletions(-) > > diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c > index b461666..025d431 100644 > --- a/inventory/inv_stobj.c > +++ b/inventory/inv_stobj.c > @@ -1065,25 +1065,26 @@ stobj_unpack_sessinfo( > return BOOL_FALSE; > } > > + /* get the seshdr and then, the remainder of the session */ > xlate_invt_seshdr((invt_seshdr_t *)p, (invt_seshdr_t *)tmpbuf, 1); > bcopy(tmpbuf, p, sizeof(invt_seshdr_t)); > - > - /* get the seshdr and then, the remainder of the session */ > s->seshdr = (invt_seshdr_t *)p; > s->seshdr->sh_sess_off = -1; > p += sizeof(invt_seshdr_t); > > - > xlate_invt_session((invt_session_t *)p, (invt_session_t *)tmpbuf, 1); > bcopy (tmpbuf, p, sizeof(invt_session_t)); > s->ses = (invt_session_t *)p; > p += sizeof(invt_session_t); > > /* the array of all the streams belonging to this session */ > - xlate_invt_stream((invt_stream_t *)p, (invt_stream_t *)tmpbuf, 1); > - bcopy(tmpbuf, p, sizeof(invt_stream_t)); > s->strms = (invt_stream_t *)p; > - p += s->ses->s_cur_nstreams * sizeof(invt_stream_t); > + for (i = 0; i < s->ses->s_cur_nstreams; i++) { > + xlate_invt_stream((invt_stream_t *)p, > + (invt_stream_t *)tmpbuf, 1); > + bcopy(tmpbuf, p, sizeof(invt_stream_t)); > + p += sizeof(invt_stream_t); > + } > > /* all the media files */ > s->mfiles = (invt_mediafile_t *)p; > -- > 2.31.1 > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking 2022-10-13 19:05 ` Darrick J. Wong @ 2022-10-14 3:37 ` Donald Douwsma 2022-10-14 4:15 ` Donald Douwsma 0 siblings, 1 reply; 13+ messages in thread From: Donald Douwsma @ 2022-10-14 3:37 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs On 14/10/2022 06:05, Darrick J. Wong wrote: > On Thu, Oct 13, 2022 at 02:15:16PM +1100, Donald Douwsma wrote: >> xfsdump can create multiple streams, when restoring the online inventory >> with multiple streams we fail to process these and assert when the >> inventory buffer is not fully decoded. >> >> [root@rhel8 ~]# xfsdump -L "Test1" -f /dev/nst0 -M tape1 -f /dev/nst1 -M tape2 /boot >> xfsdump: using scsi tape (drive_scsitape) strategy >> xfsdump: using scsi tape (drive_scsitape) strategy >> xfsdump: version 3.1.8 (dump format 3.0) - type ^C for status and control >> xfsdump: level 0 dump of rhel8:/boot >> xfsdump: dump date: Thu Oct 6 13:50:45 2022 >> xfsdump: session id: aa25fa48-4493-45c7-9027-61e53e486445 >> xfsdump: session label: "Test1" >> xfsdump: ino map phase 1: constructing initial dump list >> xfsdump: ino map phase 2: skipping (no pruning necessary) >> xfsdump: ino map phase 3: identifying stream starting points >> xfsdump: stream 0: ino 133 offset 0 to ino 28839 offset 0 >> xfsdump: stream 1: ino 28839 offset 0 to end >> xfsdump: ino map construction complete >> xfsdump: estimated dump size: 328720704 bytes >> xfsdump: estimated dump size per stream: 164375728 bytes >> xfsdump: /var/lib/xfsdump/inventory created >> xfsdump: drive 0: preparing drive >> xfsdump: drive 1: preparing drive >> xfsdump: drive 1: creating dump session media file 0 (media 0, file 0) >> xfsdump: drive 1: dumping ino map >> xfsdump: drive 1: dumping non-directory files >> xfsdump: drive 0: creating dump session media file 0 (media 0, file 0) >> xfsdump: drive 0: dumping ino map >> xfsdump: drive 0: dumping directories >> xfsdump: drive 0: dumping non-directory files >> xfsdump: drive 1: ending media file >> xfsdump: drive 1: media file size 166723584 bytes >> xfsdump: drive 1: waiting for synchronized session inventory dump >> xfsdump: drive 0: ending media file >> xfsdump: drive 0: media file size 165675008 bytes >> xfsdump: drive 0: waiting for synchronized session inventory dump >> xfsdump: drive 0: dumping session inventory >> xfsdump: drive 0: beginning inventory media file >> xfsdump: drive 0: media file 1 (media 0, file 1) >> xfsdump: drive 0: ending inventory media file >> xfsdump: drive 0: inventory media file size 2097152 bytes >> xfsdump: drive 0: writing stream terminator >> xfsdump: drive 0: beginning media stream terminator >> xfsdump: drive 0: media file 2 (media 0, file 2) >> xfsdump: drive 0: ending media stream terminator >> xfsdump: drive 0: media stream terminator size 1048576 bytes >> xfsdump: drive 1: dumping session inventory >> xfsdump: drive 1: beginning inventory media file >> xfsdump: drive 1: media file 1 (media 0, file 1) >> xfsdump: drive 1: ending inventory media file >> xfsdump: drive 1: inventory media file size 2097152 bytes >> xfsdump: drive 1: writing stream terminator >> xfsdump: drive 1: beginning media stream terminator >> xfsdump: drive 1: media file 2 (media 0, file 2) >> xfsdump: drive 1: ending media stream terminator >> xfsdump: drive 1: media stream terminator size 1048576 bytes >> xfsdump: dump size (non-dir files) : 328189016 bytes >> xfsdump: dump complete: 4 seconds elapsed >> xfsdump: Dump Summary: >> xfsdump: stream 0 /dev/nst0 OK (success) >> xfsdump: stream 1 /dev/nst1 OK (success) >> xfsdump: Dump Status: SUCCESS >> [root@rhel8 ~]# xfsdump -I >> file system 0: >> fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 >> session 0: >> mount point: rhel8:/boot >> device: rhel8:/dev/sda1 >> time: Thu Oct 6 13:50:45 2022 >> session label: "Test1" >> session id: aa25fa48-4493-45c7-9027-61e53e486445 >> level: 0 >> resumed: NO >> subtree: NO >> streams: 2 >> stream 0: >> pathname: /dev/nst0 >> start: ino 133 offset 0 >> end: ino 28839 offset 0 >> interrupted: NO >> media files: 2 >> media file 0: >> mfile index: 0 >> mfile type: data >> mfile size: 165675008 >> mfile start: ino 133 offset 0 >> mfile end: ino 28839 offset 0 >> media label: "tape1" >> media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >> media file 1: >> mfile index: 1 >> mfile type: inventory >> mfile size: 2097152 >> media label: "tape1" >> media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >> stream 1: >> pathname: /dev/nst1 >> start: ino 28839 offset 0 >> end: ino 1572997 offset 0 >> interrupted: NO >> media files: 2 >> media file 0: >> mfile index: 0 >> mfile type: data >> mfile size: 166723584 >> mfile start: ino 28839 offset 0 >> mfile end: ino 1572997 offset 0 >> media label: "tape2" >> media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 >> media file 1: >> mfile index: 1 >> mfile type: inventory >> mfile size: 2097152 >> media label: "tape2" >> media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 >> xfsdump: Dump Status: SUCCESS >> [root@rhel8 ~]# mv /var/lib/xfsdump/inventory /var/lib/xfsdump/inventory_two_sessions >> [root@rhel8 ~]# xfsdump -I >> xfsdump: Dump Status: SUCCESS >> >> [root@rhel8 ~]# xfsrestore -L Test1 -f /dev/nst0 /tmp/test1/ >> xfsrestore: using scsi tape (drive_scsitape) strategy >> xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status and control >> xfsrestore: searching media for dump >> xfsrestore: preparing drive >> xfsrestore: examining media file 2 >> xfsrestore: found dump matching specified label: >> xfsrestore: hostname: rhel8 >> xfsrestore: mount point: /boot >> xfsrestore: volume: /dev/sda1 >> xfsrestore: session time: Thu Oct 6 13:50:45 2022 >> xfsrestore: level: 0 >> xfsrestore: session label: "Test1" >> xfsrestore: media label: "tape1" >> xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 >> xfsrestore: session id: aa25fa48-4493-45c7-9027-61e53e486445 >> xfsrestore: media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >> xfsrestore: searching media for directory dump >> xfsrestore: rewinding >> xfsrestore: examining media file 0 >> xfsrestore: reading directories >> xfsrestore: 9 directories and 320 entries processed >> xfsrestore: directory post-processing >> xfsrestore: restoring non-directory files >> xfsrestore: examining media file 1 >> xfsrestore: inv_stobj.c:1119: stobj_unpack_sessinfo: Assertion `(size_t) ( p - (char *) bufp ) == bufsz' failed. >> Aborted (core dumped) >> >> Make sure we unpack multiple streams when restoring the online >> inventory from media. >> >> Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> > > Much better now, though I hope there's an fstest coming to make sure > that multistream restore/dump work properly. That could be a little tricky. xfstests currently only supports one TAPE_DEV (and a remote RMT_TAPE_DEV). So we'd need to add support for multiple tapes, something like TAPE_DEV1=/dev/nst0 TAPE_DEV2=/dev/nst1 Then it can be hard to get access to a tape drive, let alone two. Probably the easiest way to address that is to provide instructions on how to setup mhvtl, https://github.com/markh794/mhvtl and load the tapes before a test run with [root@rhel8 ~]# systemctl start mhvtl.target [root@rhel8 ~]# mtx -f /dev/sch0 load 1 0; mtx -f /dev/sch load 2 1 Loading media from Storage Element 1 into drive 0...done Loading media from Storage Element 2 into drive 1...done [root@rhel8 ~]# ./check -g xfs/tape Not sure there's many people who test the tape group, so this may be generally useful. mhvtl requires a kernel module to be built, not sure if that would bother QE folk when doing general tests, but for xfsdump userspace testing it should be fine. Don > Reviewed-by: Darrick J. Wong <djwong@kernel.org> > > --D > >> --- >> inventory/inv_stobj.c | 13 +++++++------ >> 1 file changed, 7 insertions(+), 6 deletions(-) >> >> diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c >> index b461666..025d431 100644 >> --- a/inventory/inv_stobj.c >> +++ b/inventory/inv_stobj.c >> @@ -1065,25 +1065,26 @@ stobj_unpack_sessinfo( >> return BOOL_FALSE; >> } >> >> + /* get the seshdr and then, the remainder of the session */ >> xlate_invt_seshdr((invt_seshdr_t *)p, (invt_seshdr_t *)tmpbuf, 1); >> bcopy(tmpbuf, p, sizeof(invt_seshdr_t)); >> - >> - /* get the seshdr and then, the remainder of the session */ >> s->seshdr = (invt_seshdr_t *)p; >> s->seshdr->sh_sess_off = -1; >> p += sizeof(invt_seshdr_t); >> >> - >> xlate_invt_session((invt_session_t *)p, (invt_session_t *)tmpbuf, 1); >> bcopy (tmpbuf, p, sizeof(invt_session_t)); >> s->ses = (invt_session_t *)p; >> p += sizeof(invt_session_t); >> >> /* the array of all the streams belonging to this session */ >> - xlate_invt_stream((invt_stream_t *)p, (invt_stream_t *)tmpbuf, 1); >> - bcopy(tmpbuf, p, sizeof(invt_stream_t)); >> s->strms = (invt_stream_t *)p; >> - p += s->ses->s_cur_nstreams * sizeof(invt_stream_t); >> + for (i = 0; i < s->ses->s_cur_nstreams; i++) { >> + xlate_invt_stream((invt_stream_t *)p, >> + (invt_stream_t *)tmpbuf, 1); >> + bcopy(tmpbuf, p, sizeof(invt_stream_t)); >> + p += sizeof(invt_stream_t); >> + } >> >> /* all the media files */ >> s->mfiles = (invt_mediafile_t *)p; >> -- >> 2.31.1 >> > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking 2022-10-14 3:37 ` Donald Douwsma @ 2022-10-14 4:15 ` Donald Douwsma 2022-10-14 15:46 ` Darrick J. Wong 0 siblings, 1 reply; 13+ messages in thread From: Donald Douwsma @ 2022-10-14 4:15 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs On 14/10/2022 14:37, Donald Douwsma wrote: > > > On 14/10/2022 06:05, Darrick J. Wong wrote: >> On Thu, Oct 13, 2022 at 02:15:16PM +1100, Donald Douwsma wrote: >>> xfsdump can create multiple streams, when restoring the online inventory >>> with multiple streams we fail to process these and assert when the >>> inventory buffer is not fully decoded. >>> >>> [root@rhel8 ~]# xfsdump -L "Test1" -f /dev/nst0 -M tape1 -f /dev/nst1 >>> -M tape2 /boot >>> xfsdump: using scsi tape (drive_scsitape) strategy >>> xfsdump: using scsi tape (drive_scsitape) strategy >>> xfsdump: version 3.1.8 (dump format 3.0) - type ^C for status and >>> control >>> xfsdump: level 0 dump of rhel8:/boot >>> xfsdump: dump date: Thu Oct 6 13:50:45 2022 >>> xfsdump: session id: aa25fa48-4493-45c7-9027-61e53e486445 >>> xfsdump: session label: "Test1" >>> xfsdump: ino map phase 1: constructing initial dump list >>> xfsdump: ino map phase 2: skipping (no pruning necessary) >>> xfsdump: ino map phase 3: identifying stream starting points >>> xfsdump: stream 0: ino 133 offset 0 to ino 28839 offset 0 >>> xfsdump: stream 1: ino 28839 offset 0 to end >>> xfsdump: ino map construction complete >>> xfsdump: estimated dump size: 328720704 bytes >>> xfsdump: estimated dump size per stream: 164375728 bytes >>> xfsdump: /var/lib/xfsdump/inventory created >>> xfsdump: drive 0: preparing drive >>> xfsdump: drive 1: preparing drive >>> xfsdump: drive 1: creating dump session media file 0 (media 0, file 0) >>> xfsdump: drive 1: dumping ino map >>> xfsdump: drive 1: dumping non-directory files >>> xfsdump: drive 0: creating dump session media file 0 (media 0, file 0) >>> xfsdump: drive 0: dumping ino map >>> xfsdump: drive 0: dumping directories >>> xfsdump: drive 0: dumping non-directory files >>> xfsdump: drive 1: ending media file >>> xfsdump: drive 1: media file size 166723584 bytes >>> xfsdump: drive 1: waiting for synchronized session inventory dump >>> xfsdump: drive 0: ending media file >>> xfsdump: drive 0: media file size 165675008 bytes >>> xfsdump: drive 0: waiting for synchronized session inventory dump >>> xfsdump: drive 0: dumping session inventory >>> xfsdump: drive 0: beginning inventory media file >>> xfsdump: drive 0: media file 1 (media 0, file 1) >>> xfsdump: drive 0: ending inventory media file >>> xfsdump: drive 0: inventory media file size 2097152 bytes >>> xfsdump: drive 0: writing stream terminator >>> xfsdump: drive 0: beginning media stream terminator >>> xfsdump: drive 0: media file 2 (media 0, file 2) >>> xfsdump: drive 0: ending media stream terminator >>> xfsdump: drive 0: media stream terminator size 1048576 bytes >>> xfsdump: drive 1: dumping session inventory >>> xfsdump: drive 1: beginning inventory media file >>> xfsdump: drive 1: media file 1 (media 0, file 1) >>> xfsdump: drive 1: ending inventory media file >>> xfsdump: drive 1: inventory media file size 2097152 bytes >>> xfsdump: drive 1: writing stream terminator >>> xfsdump: drive 1: beginning media stream terminator >>> xfsdump: drive 1: media file 2 (media 0, file 2) >>> xfsdump: drive 1: ending media stream terminator >>> xfsdump: drive 1: media stream terminator size 1048576 bytes >>> xfsdump: dump size (non-dir files) : 328189016 bytes >>> xfsdump: dump complete: 4 seconds elapsed >>> xfsdump: Dump Summary: >>> xfsdump: stream 0 /dev/nst0 OK (success) >>> xfsdump: stream 1 /dev/nst1 OK (success) >>> xfsdump: Dump Status: SUCCESS >>> [root@rhel8 ~]# xfsdump -I >>> file system 0: >>> fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 >>> session 0: >>> mount point: rhel8:/boot >>> device: rhel8:/dev/sda1 >>> time: Thu Oct 6 13:50:45 2022 >>> session label: "Test1" >>> session id: aa25fa48-4493-45c7-9027-61e53e486445 >>> level: 0 >>> resumed: NO >>> subtree: NO >>> streams: 2 >>> stream 0: >>> pathname: /dev/nst0 >>> start: ino 133 offset 0 >>> end: ino 28839 offset 0 >>> interrupted: NO >>> media files: 2 >>> media file 0: >>> mfile index: 0 >>> mfile type: data >>> mfile size: 165675008 >>> mfile start: ino 133 offset 0 >>> mfile end: ino 28839 offset 0 >>> media label: "tape1" >>> media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >>> media file 1: >>> mfile index: 1 >>> mfile type: inventory >>> mfile size: 2097152 >>> media label: "tape1" >>> media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >>> stream 1: >>> pathname: /dev/nst1 >>> start: ino 28839 offset 0 >>> end: ino 1572997 offset 0 >>> interrupted: NO >>> media files: 2 >>> media file 0: >>> mfile index: 0 >>> mfile type: data >>> mfile size: 166723584 >>> mfile start: ino 28839 offset 0 >>> mfile end: ino 1572997 offset 0 >>> media label: "tape2" >>> media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 >>> media file 1: >>> mfile index: 1 >>> mfile type: inventory >>> mfile size: 2097152 >>> media label: "tape2" >>> media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 >>> xfsdump: Dump Status: SUCCESS >>> [root@rhel8 ~]# mv /var/lib/xfsdump/inventory >>> /var/lib/xfsdump/inventory_two_sessions >>> [root@rhel8 ~]# xfsdump -I >>> xfsdump: Dump Status: SUCCESS >>> >>> [root@rhel8 ~]# xfsrestore -L Test1 -f /dev/nst0 /tmp/test1/ >>> xfsrestore: using scsi tape (drive_scsitape) strategy >>> xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status and >>> control >>> xfsrestore: searching media for dump >>> xfsrestore: preparing drive >>> xfsrestore: examining media file 2 >>> xfsrestore: found dump matching specified label: >>> xfsrestore: hostname: rhel8 >>> xfsrestore: mount point: /boot >>> xfsrestore: volume: /dev/sda1 >>> xfsrestore: session time: Thu Oct 6 13:50:45 2022 >>> xfsrestore: level: 0 >>> xfsrestore: session label: "Test1" >>> xfsrestore: media label: "tape1" >>> xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 >>> xfsrestore: session id: aa25fa48-4493-45c7-9027-61e53e486445 >>> xfsrestore: media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >>> xfsrestore: searching media for directory dump >>> xfsrestore: rewinding >>> xfsrestore: examining media file 0 >>> xfsrestore: reading directories >>> xfsrestore: 9 directories and 320 entries processed >>> xfsrestore: directory post-processing >>> xfsrestore: restoring non-directory files >>> xfsrestore: examining media file 1 >>> xfsrestore: inv_stobj.c:1119: stobj_unpack_sessinfo: Assertion >>> `(size_t) ( p - (char *) bufp ) == bufsz' failed. >>> Aborted (core dumped) >>> >>> Make sure we unpack multiple streams when restoring the online >>> inventory from media. >>> >>> Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> >> >> Much better now, though I hope there's an fstest coming to make sure >> that multistream restore/dump work properly. > > That could be a little tricky. > > xfstests currently only supports one TAPE_DEV (and a remote > RMT_TAPE_DEV). So we'd need to add support for multiple tapes, > something like Thinking about this some more RMT_TAPE_DEV may be ok. If streaming to multiple targets is meant to enable an offsite dump it could be done via remote i.e. # xfsdump -L "Test3" -f /dev/nst0 -M "tape1" -f backup.example.com:/dev/nst0 -M "tape2" / The existing remote tests xfs/037 and xfs/038 just use the device for a simple backup. Or I could be inventing a never useful use case. Don > > TAPE_DEV1=/dev/nst0 > TAPE_DEV2=/dev/nst1 > > Then it can be hard to get access to a tape drive, let alone two. > > Probably the easiest way to address that is to provide instructions on > how to setup mhvtl, https://github.com/markh794/mhvtl and load the > tapes before a test run with > > [root@rhel8 ~]# systemctl start mhvtl.target > [root@rhel8 ~]# mtx -f /dev/sch0 load 1 0; mtx -f /dev/sch load 2 1 > Loading media from Storage Element 1 into drive 0...done > Loading media from Storage Element 2 into drive 1...done > [root@rhel8 ~]# ./check -g xfs/tape > > Not sure there's many people who test the tape group, so this may be > generally useful. > > mhvtl requires a kernel module to be built, not sure if that would > bother QE folk when doing general tests, but for xfsdump userspace > testing it should be fine. > > Don > > >> Reviewed-by: Darrick J. Wong <djwong@kernel.org> >> >> --D >> >>> --- >>> inventory/inv_stobj.c | 13 +++++++------ >>> 1 file changed, 7 insertions(+), 6 deletions(-) >>> >>> diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c >>> index b461666..025d431 100644 >>> --- a/inventory/inv_stobj.c >>> +++ b/inventory/inv_stobj.c >>> @@ -1065,25 +1065,26 @@ stobj_unpack_sessinfo( >>> return BOOL_FALSE; >>> } >>> + /* get the seshdr and then, the remainder of the session */ >>> xlate_invt_seshdr((invt_seshdr_t *)p, (invt_seshdr_t *)tmpbuf, 1); >>> bcopy(tmpbuf, p, sizeof(invt_seshdr_t)); >>> - >>> - /* get the seshdr and then, the remainder of the session */ >>> s->seshdr = (invt_seshdr_t *)p; >>> s->seshdr->sh_sess_off = -1; >>> p += sizeof(invt_seshdr_t); >>> - >>> xlate_invt_session((invt_session_t *)p, (invt_session_t >>> *)tmpbuf, 1); >>> bcopy (tmpbuf, p, sizeof(invt_session_t)); >>> s->ses = (invt_session_t *)p; >>> p += sizeof(invt_session_t); >>> /* the array of all the streams belonging to this session */ >>> - xlate_invt_stream((invt_stream_t *)p, (invt_stream_t *)tmpbuf, 1); >>> - bcopy(tmpbuf, p, sizeof(invt_stream_t)); >>> s->strms = (invt_stream_t *)p; >>> - p += s->ses->s_cur_nstreams * sizeof(invt_stream_t); >>> + for (i = 0; i < s->ses->s_cur_nstreams; i++) { >>> + xlate_invt_stream((invt_stream_t *)p, >>> + (invt_stream_t *)tmpbuf, 1); >>> + bcopy(tmpbuf, p, sizeof(invt_stream_t)); >>> + p += sizeof(invt_stream_t); >>> + } >>> /* all the media files */ >>> s->mfiles = (invt_mediafile_t *)p; >>> -- >>> 2.31.1 >>> >> ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking 2022-10-14 4:15 ` Donald Douwsma @ 2022-10-14 15:46 ` Darrick J. Wong 2022-10-16 21:54 ` Donald Douwsma 0 siblings, 1 reply; 13+ messages in thread From: Darrick J. Wong @ 2022-10-14 15:46 UTC (permalink / raw) To: Donald Douwsma; +Cc: linux-xfs On Fri, Oct 14, 2022 at 03:15:33PM +1100, Donald Douwsma wrote: > > > On 14/10/2022 14:37, Donald Douwsma wrote: > > > > > > On 14/10/2022 06:05, Darrick J. Wong wrote: > > > On Thu, Oct 13, 2022 at 02:15:16PM +1100, Donald Douwsma wrote: > > > > xfsdump can create multiple streams, when restoring the online inventory > > > > with multiple streams we fail to process these and assert when the > > > > inventory buffer is not fully decoded. > > > > > > > > [root@rhel8 ~]# xfsdump -L "Test1" -f /dev/nst0 -M tape1 -f > > > > /dev/nst1 -M tape2 /boot > > > > xfsdump: using scsi tape (drive_scsitape) strategy > > > > xfsdump: using scsi tape (drive_scsitape) strategy > > > > xfsdump: version 3.1.8 (dump format 3.0) - type ^C for status > > > > and control > > > > xfsdump: level 0 dump of rhel8:/boot > > > > xfsdump: dump date: Thu Oct 6 13:50:45 2022 > > > > xfsdump: session id: aa25fa48-4493-45c7-9027-61e53e486445 > > > > xfsdump: session label: "Test1" > > > > xfsdump: ino map phase 1: constructing initial dump list > > > > xfsdump: ino map phase 2: skipping (no pruning necessary) > > > > xfsdump: ino map phase 3: identifying stream starting points > > > > xfsdump: stream 0: ino 133 offset 0 to ino 28839 offset 0 > > > > xfsdump: stream 1: ino 28839 offset 0 to end > > > > xfsdump: ino map construction complete > > > > xfsdump: estimated dump size: 328720704 bytes > > > > xfsdump: estimated dump size per stream: 164375728 bytes > > > > xfsdump: /var/lib/xfsdump/inventory created > > > > xfsdump: drive 0: preparing drive > > > > xfsdump: drive 1: preparing drive > > > > xfsdump: drive 1: creating dump session media file 0 (media 0, file 0) > > > > xfsdump: drive 1: dumping ino map > > > > xfsdump: drive 1: dumping non-directory files > > > > xfsdump: drive 0: creating dump session media file 0 (media 0, file 0) > > > > xfsdump: drive 0: dumping ino map > > > > xfsdump: drive 0: dumping directories > > > > xfsdump: drive 0: dumping non-directory files > > > > xfsdump: drive 1: ending media file > > > > xfsdump: drive 1: media file size 166723584 bytes > > > > xfsdump: drive 1: waiting for synchronized session inventory dump > > > > xfsdump: drive 0: ending media file > > > > xfsdump: drive 0: media file size 165675008 bytes > > > > xfsdump: drive 0: waiting for synchronized session inventory dump > > > > xfsdump: drive 0: dumping session inventory > > > > xfsdump: drive 0: beginning inventory media file > > > > xfsdump: drive 0: media file 1 (media 0, file 1) > > > > xfsdump: drive 0: ending inventory media file > > > > xfsdump: drive 0: inventory media file size 2097152 bytes > > > > xfsdump: drive 0: writing stream terminator > > > > xfsdump: drive 0: beginning media stream terminator > > > > xfsdump: drive 0: media file 2 (media 0, file 2) > > > > xfsdump: drive 0: ending media stream terminator > > > > xfsdump: drive 0: media stream terminator size 1048576 bytes > > > > xfsdump: drive 1: dumping session inventory > > > > xfsdump: drive 1: beginning inventory media file > > > > xfsdump: drive 1: media file 1 (media 0, file 1) > > > > xfsdump: drive 1: ending inventory media file > > > > xfsdump: drive 1: inventory media file size 2097152 bytes > > > > xfsdump: drive 1: writing stream terminator > > > > xfsdump: drive 1: beginning media stream terminator > > > > xfsdump: drive 1: media file 2 (media 0, file 2) > > > > xfsdump: drive 1: ending media stream terminator > > > > xfsdump: drive 1: media stream terminator size 1048576 bytes > > > > xfsdump: dump size (non-dir files) : 328189016 bytes > > > > xfsdump: dump complete: 4 seconds elapsed > > > > xfsdump: Dump Summary: > > > > xfsdump: stream 0 /dev/nst0 OK (success) > > > > xfsdump: stream 1 /dev/nst1 OK (success) > > > > xfsdump: Dump Status: SUCCESS > > > > [root@rhel8 ~]# xfsdump -I > > > > file system 0: > > > > fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 > > > > session 0: > > > > mount point: rhel8:/boot > > > > device: rhel8:/dev/sda1 > > > > time: Thu Oct 6 13:50:45 2022 > > > > session label: "Test1" > > > > session id: aa25fa48-4493-45c7-9027-61e53e486445 > > > > level: 0 > > > > resumed: NO > > > > subtree: NO > > > > streams: 2 > > > > stream 0: > > > > pathname: /dev/nst0 > > > > start: ino 133 offset 0 > > > > end: ino 28839 offset 0 > > > > interrupted: NO > > > > media files: 2 > > > > media file 0: > > > > mfile index: 0 > > > > mfile type: data > > > > mfile size: 165675008 > > > > mfile start: ino 133 offset 0 > > > > mfile end: ino 28839 offset 0 > > > > media label: "tape1" > > > > media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > > > > media file 1: > > > > mfile index: 1 > > > > mfile type: inventory > > > > mfile size: 2097152 > > > > media label: "tape1" > > > > media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > > > > stream 1: > > > > pathname: /dev/nst1 > > > > start: ino 28839 offset 0 > > > > end: ino 1572997 offset 0 > > > > interrupted: NO > > > > media files: 2 > > > > media file 0: > > > > mfile index: 0 > > > > mfile type: data > > > > mfile size: 166723584 > > > > mfile start: ino 28839 offset 0 > > > > mfile end: ino 1572997 offset 0 > > > > media label: "tape2" > > > > media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 > > > > media file 1: > > > > mfile index: 1 > > > > mfile type: inventory > > > > mfile size: 2097152 > > > > media label: "tape2" > > > > media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 > > > > xfsdump: Dump Status: SUCCESS > > > > [root@rhel8 ~]# mv /var/lib/xfsdump/inventory > > > > /var/lib/xfsdump/inventory_two_sessions > > > > [root@rhel8 ~]# xfsdump -I > > > > xfsdump: Dump Status: SUCCESS > > > > > > > > [root@rhel8 ~]# xfsrestore -L Test1 -f /dev/nst0 /tmp/test1/ > > > > xfsrestore: using scsi tape (drive_scsitape) strategy > > > > xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status > > > > and control > > > > xfsrestore: searching media for dump > > > > xfsrestore: preparing drive > > > > xfsrestore: examining media file 2 > > > > xfsrestore: found dump matching specified label: > > > > xfsrestore: hostname: rhel8 > > > > xfsrestore: mount point: /boot > > > > xfsrestore: volume: /dev/sda1 > > > > xfsrestore: session time: Thu Oct 6 13:50:45 2022 > > > > xfsrestore: level: 0 > > > > xfsrestore: session label: "Test1" > > > > xfsrestore: media label: "tape1" > > > > xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 > > > > xfsrestore: session id: aa25fa48-4493-45c7-9027-61e53e486445 > > > > xfsrestore: media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 > > > > xfsrestore: searching media for directory dump > > > > xfsrestore: rewinding > > > > xfsrestore: examining media file 0 > > > > xfsrestore: reading directories > > > > xfsrestore: 9 directories and 320 entries processed > > > > xfsrestore: directory post-processing > > > > xfsrestore: restoring non-directory files > > > > xfsrestore: examining media file 1 > > > > xfsrestore: inv_stobj.c:1119: stobj_unpack_sessinfo: Assertion > > > > `(size_t) ( p - (char *) bufp ) == bufsz' failed. > > > > Aborted (core dumped) > > > > > > > > Make sure we unpack multiple streams when restoring the online > > > > inventory from media. > > > > > > > > Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> > > > > > > Much better now, though I hope there's an fstest coming to make sure > > > that multistream restore/dump work properly. > > > > That could be a little tricky. > > > > xfstests currently only supports one TAPE_DEV (and a remote > > RMT_TAPE_DEV). So we'd need to add support for multiple tapes, > > something like > > Thinking about this some more RMT_TAPE_DEV may be ok. If streaming to > multiple targets is meant to enable an offsite dump it could be done > via remote i.e. > > # xfsdump -L "Test3" -f /dev/nst0 -M "tape1" -f backup.example.com:/dev/nst0 > -M "tape2" / > > The existing remote tests xfs/037 and xfs/038 just use the device for a > simple backup. > > Or I could be inventing a never useful use case. I wonder, does the "remote" tape device have to be remote, or is xfsdump flexible enough to let you set up a dummy file to dump into? I bet any fstest that actually requires a real tape device hasn't been run in years, except maybe by the RH QA department. :/ --D > Don > > > > > TAPE_DEV1=/dev/nst0 > > TAPE_DEV2=/dev/nst1 > > > > Then it can be hard to get access to a tape drive, let alone two. > > > > Probably the easiest way to address that is to provide instructions on > > how to setup mhvtl, https://github.com/markh794/mhvtl and load the > > tapes before a test run with > > > > [root@rhel8 ~]# systemctl start mhvtl.target > > [root@rhel8 ~]# mtx -f /dev/sch0 load 1 0; mtx -f /dev/sch load 2 1 > > Loading media from Storage Element 1 into drive 0...done > > Loading media from Storage Element 2 into drive 1...done > > [root@rhel8 ~]# ./check -g xfs/tape > > > > Not sure there's many people who test the tape group, so this may be > > generally useful. > > > > mhvtl requires a kernel module to be built, not sure if that would > > bother QE folk when doing general tests, but for xfsdump userspace > > testing it should be fine. > > > > Don > > > > > > > Reviewed-by: Darrick J. Wong <djwong@kernel.org> > > > > > > --D > > > > > > > --- > > > > inventory/inv_stobj.c | 13 +++++++------ > > > > 1 file changed, 7 insertions(+), 6 deletions(-) > > > > > > > > diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c > > > > index b461666..025d431 100644 > > > > --- a/inventory/inv_stobj.c > > > > +++ b/inventory/inv_stobj.c > > > > @@ -1065,25 +1065,26 @@ stobj_unpack_sessinfo( > > > > return BOOL_FALSE; > > > > } > > > > + /* get the seshdr and then, the remainder of the session */ > > > > xlate_invt_seshdr((invt_seshdr_t *)p, (invt_seshdr_t *)tmpbuf, 1); > > > > bcopy(tmpbuf, p, sizeof(invt_seshdr_t)); > > > > - > > > > - /* get the seshdr and then, the remainder of the session */ > > > > s->seshdr = (invt_seshdr_t *)p; > > > > s->seshdr->sh_sess_off = -1; > > > > p += sizeof(invt_seshdr_t); > > > > - > > > > xlate_invt_session((invt_session_t *)p, (invt_session_t > > > > *)tmpbuf, 1); > > > > bcopy (tmpbuf, p, sizeof(invt_session_t)); > > > > s->ses = (invt_session_t *)p; > > > > p += sizeof(invt_session_t); > > > > /* the array of all the streams belonging to this session */ > > > > - xlate_invt_stream((invt_stream_t *)p, (invt_stream_t *)tmpbuf, 1); > > > > - bcopy(tmpbuf, p, sizeof(invt_stream_t)); > > > > s->strms = (invt_stream_t *)p; > > > > - p += s->ses->s_cur_nstreams * sizeof(invt_stream_t); > > > > + for (i = 0; i < s->ses->s_cur_nstreams; i++) { > > > > + xlate_invt_stream((invt_stream_t *)p, > > > > + (invt_stream_t *)tmpbuf, 1); > > > > + bcopy(tmpbuf, p, sizeof(invt_stream_t)); > > > > + p += sizeof(invt_stream_t); > > > > + } > > > > /* all the media files */ > > > > s->mfiles = (invt_mediafile_t *)p; > > > > -- > > > > 2.31.1 > > > > > > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking 2022-10-14 15:46 ` Darrick J. Wong @ 2022-10-16 21:54 ` Donald Douwsma 0 siblings, 0 replies; 13+ messages in thread From: Donald Douwsma @ 2022-10-16 21:54 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs On 15/10/2022 02:46, Darrick J. Wong wrote: > On Fri, Oct 14, 2022 at 03:15:33PM +1100, Donald Douwsma wrote: >> >> >> On 14/10/2022 14:37, Donald Douwsma wrote: >>> >>> >>> On 14/10/2022 06:05, Darrick J. Wong wrote: >>>> On Thu, Oct 13, 2022 at 02:15:16PM +1100, Donald Douwsma wrote: >>>>> xfsdump can create multiple streams, when restoring the online inventory >>>>> with multiple streams we fail to process these and assert when the >>>>> inventory buffer is not fully decoded. >>>>> >>>>> [root@rhel8 ~]# xfsdump -L "Test1" -f /dev/nst0 -M tape1 -f >>>>> /dev/nst1 -M tape2 /boot >>>>> xfsdump: using scsi tape (drive_scsitape) strategy >>>>> xfsdump: using scsi tape (drive_scsitape) strategy >>>>> xfsdump: version 3.1.8 (dump format 3.0) - type ^C for status >>>>> and control >>>>> xfsdump: level 0 dump of rhel8:/boot >>>>> xfsdump: dump date: Thu Oct 6 13:50:45 2022 >>>>> xfsdump: session id: aa25fa48-4493-45c7-9027-61e53e486445 >>>>> xfsdump: session label: "Test1" >>>>> xfsdump: ino map phase 1: constructing initial dump list >>>>> xfsdump: ino map phase 2: skipping (no pruning necessary) >>>>> xfsdump: ino map phase 3: identifying stream starting points >>>>> xfsdump: stream 0: ino 133 offset 0 to ino 28839 offset 0 >>>>> xfsdump: stream 1: ino 28839 offset 0 to end >>>>> xfsdump: ino map construction complete >>>>> xfsdump: estimated dump size: 328720704 bytes >>>>> xfsdump: estimated dump size per stream: 164375728 bytes >>>>> xfsdump: /var/lib/xfsdump/inventory created >>>>> xfsdump: drive 0: preparing drive >>>>> xfsdump: drive 1: preparing drive >>>>> xfsdump: drive 1: creating dump session media file 0 (media 0, file 0) >>>>> xfsdump: drive 1: dumping ino map >>>>> xfsdump: drive 1: dumping non-directory files >>>>> xfsdump: drive 0: creating dump session media file 0 (media 0, file 0) >>>>> xfsdump: drive 0: dumping ino map >>>>> xfsdump: drive 0: dumping directories >>>>> xfsdump: drive 0: dumping non-directory files >>>>> xfsdump: drive 1: ending media file >>>>> xfsdump: drive 1: media file size 166723584 bytes >>>>> xfsdump: drive 1: waiting for synchronized session inventory dump >>>>> xfsdump: drive 0: ending media file >>>>> xfsdump: drive 0: media file size 165675008 bytes >>>>> xfsdump: drive 0: waiting for synchronized session inventory dump >>>>> xfsdump: drive 0: dumping session inventory >>>>> xfsdump: drive 0: beginning inventory media file >>>>> xfsdump: drive 0: media file 1 (media 0, file 1) >>>>> xfsdump: drive 0: ending inventory media file >>>>> xfsdump: drive 0: inventory media file size 2097152 bytes >>>>> xfsdump: drive 0: writing stream terminator >>>>> xfsdump: drive 0: beginning media stream terminator >>>>> xfsdump: drive 0: media file 2 (media 0, file 2) >>>>> xfsdump: drive 0: ending media stream terminator >>>>> xfsdump: drive 0: media stream terminator size 1048576 bytes >>>>> xfsdump: drive 1: dumping session inventory >>>>> xfsdump: drive 1: beginning inventory media file >>>>> xfsdump: drive 1: media file 1 (media 0, file 1) >>>>> xfsdump: drive 1: ending inventory media file >>>>> xfsdump: drive 1: inventory media file size 2097152 bytes >>>>> xfsdump: drive 1: writing stream terminator >>>>> xfsdump: drive 1: beginning media stream terminator >>>>> xfsdump: drive 1: media file 2 (media 0, file 2) >>>>> xfsdump: drive 1: ending media stream terminator >>>>> xfsdump: drive 1: media stream terminator size 1048576 bytes >>>>> xfsdump: dump size (non-dir files) : 328189016 bytes >>>>> xfsdump: dump complete: 4 seconds elapsed >>>>> xfsdump: Dump Summary: >>>>> xfsdump: stream 0 /dev/nst0 OK (success) >>>>> xfsdump: stream 1 /dev/nst1 OK (success) >>>>> xfsdump: Dump Status: SUCCESS >>>>> [root@rhel8 ~]# xfsdump -I >>>>> file system 0: >>>>> fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 >>>>> session 0: >>>>> mount point: rhel8:/boot >>>>> device: rhel8:/dev/sda1 >>>>> time: Thu Oct 6 13:50:45 2022 >>>>> session label: "Test1" >>>>> session id: aa25fa48-4493-45c7-9027-61e53e486445 >>>>> level: 0 >>>>> resumed: NO >>>>> subtree: NO >>>>> streams: 2 >>>>> stream 0: >>>>> pathname: /dev/nst0 >>>>> start: ino 133 offset 0 >>>>> end: ino 28839 offset 0 >>>>> interrupted: NO >>>>> media files: 2 >>>>> media file 0: >>>>> mfile index: 0 >>>>> mfile type: data >>>>> mfile size: 165675008 >>>>> mfile start: ino 133 offset 0 >>>>> mfile end: ino 28839 offset 0 >>>>> media label: "tape1" >>>>> media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >>>>> media file 1: >>>>> mfile index: 1 >>>>> mfile type: inventory >>>>> mfile size: 2097152 >>>>> media label: "tape1" >>>>> media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >>>>> stream 1: >>>>> pathname: /dev/nst1 >>>>> start: ino 28839 offset 0 >>>>> end: ino 1572997 offset 0 >>>>> interrupted: NO >>>>> media files: 2 >>>>> media file 0: >>>>> mfile index: 0 >>>>> mfile type: data >>>>> mfile size: 166723584 >>>>> mfile start: ino 28839 offset 0 >>>>> mfile end: ino 1572997 offset 0 >>>>> media label: "tape2" >>>>> media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 >>>>> media file 1: >>>>> mfile index: 1 >>>>> mfile type: inventory >>>>> mfile size: 2097152 >>>>> media label: "tape2" >>>>> media id: 22224f02-b6c7-47d5-ad61-a61ba071c8a8 >>>>> xfsdump: Dump Status: SUCCESS >>>>> [root@rhel8 ~]# mv /var/lib/xfsdump/inventory >>>>> /var/lib/xfsdump/inventory_two_sessions >>>>> [root@rhel8 ~]# xfsdump -I >>>>> xfsdump: Dump Status: SUCCESS >>>>> >>>>> [root@rhel8 ~]# xfsrestore -L Test1 -f /dev/nst0 /tmp/test1/ >>>>> xfsrestore: using scsi tape (drive_scsitape) strategy >>>>> xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status >>>>> and control >>>>> xfsrestore: searching media for dump >>>>> xfsrestore: preparing drive >>>>> xfsrestore: examining media file 2 >>>>> xfsrestore: found dump matching specified label: >>>>> xfsrestore: hostname: rhel8 >>>>> xfsrestore: mount point: /boot >>>>> xfsrestore: volume: /dev/sda1 >>>>> xfsrestore: session time: Thu Oct 6 13:50:45 2022 >>>>> xfsrestore: level: 0 >>>>> xfsrestore: session label: "Test1" >>>>> xfsrestore: media label: "tape1" >>>>> xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 >>>>> xfsrestore: session id: aa25fa48-4493-45c7-9027-61e53e486445 >>>>> xfsrestore: media id: adb31f2a-f026-4597-a20a-326f28ecbaf1 >>>>> xfsrestore: searching media for directory dump >>>>> xfsrestore: rewinding >>>>> xfsrestore: examining media file 0 >>>>> xfsrestore: reading directories >>>>> xfsrestore: 9 directories and 320 entries processed >>>>> xfsrestore: directory post-processing >>>>> xfsrestore: restoring non-directory files >>>>> xfsrestore: examining media file 1 >>>>> xfsrestore: inv_stobj.c:1119: stobj_unpack_sessinfo: Assertion >>>>> `(size_t) ( p - (char *) bufp ) == bufsz' failed. >>>>> Aborted (core dumped) >>>>> >>>>> Make sure we unpack multiple streams when restoring the online >>>>> inventory from media. >>>>> >>>>> Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> >>>> >>>> Much better now, though I hope there's an fstest coming to make sure >>>> that multistream restore/dump work properly. >>> >>> That could be a little tricky. >>> >>> xfstests currently only supports one TAPE_DEV (and a remote >>> RMT_TAPE_DEV). So we'd need to add support for multiple tapes, >>> something like >> >> Thinking about this some more RMT_TAPE_DEV may be ok. If streaming to >> multiple targets is meant to enable an offsite dump it could be done >> via remote i.e. >> >> # xfsdump -L "Test3" -f /dev/nst0 -M "tape1" -f backup.example.com:/dev/nst0 >> -M "tape2" / >> >> The existing remote tests xfs/037 and xfs/038 just use the device for a >> simple backup. >> >> Or I could be inventing a never useful use case. > > I wonder, does the "remote" tape device have to be remote, or is xfsdump > flexible enough to let you set up a dummy file to dump into? xfsdump identifies a remote device by looking for a : and will chose which drive target to use based on this. I cant see anywhere that xfstests checks for the format of RMT_TAPE_DEV, so should work. > I bet any fstest that actually requires a real tape device hasn't been > run in years, except maybe by the RH QA department. :/ I dont know if anyone is running it regularly. I know some folk at sgi tested with it in 2008, and I've been running it a bit while trying to track this problem down. All the tests that are run still pass (apart from a couple of Irix tests that we cant run). Don > >> Don >> >>> >>> TAPE_DEV1=/dev/nst0 >>> TAPE_DEV2=/dev/nst1 >>> >>> Then it can be hard to get access to a tape drive, let alone two. >>> >>> Probably the easiest way to address that is to provide instructions on >>> how to setup mhvtl, https://github.com/markh794/mhvtl and load the >>> tapes before a test run with >>> >>> [root@rhel8 ~]# systemctl start mhvtl.target >>> [root@rhel8 ~]# mtx -f /dev/sch0 load 1 0; mtx -f /dev/sch load 2 1 >>> Loading media from Storage Element 1 into drive 0...done >>> Loading media from Storage Element 2 into drive 1...done >>> [root@rhel8 ~]# ./check -g xfs/tape >>> >>> Not sure there's many people who test the tape group, so this may be >>> generally useful. >>> >>> mhvtl requires a kernel module to be built, not sure if that would >>> bother QE folk when doing general tests, but for xfsdump userspace >>> testing it should be fine. >>> >>> Don >>> >>> >>>> Reviewed-by: Darrick J. Wong <djwong@kernel.org> >>>> >>>> --D >>>> >>>>> --- >>>>> inventory/inv_stobj.c | 13 +++++++------ >>>>> 1 file changed, 7 insertions(+), 6 deletions(-) >>>>> >>>>> diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c >>>>> index b461666..025d431 100644 >>>>> --- a/inventory/inv_stobj.c >>>>> +++ b/inventory/inv_stobj.c >>>>> @@ -1065,25 +1065,26 @@ stobj_unpack_sessinfo( >>>>> return BOOL_FALSE; >>>>> } >>>>> + /* get the seshdr and then, the remainder of the session */ >>>>> xlate_invt_seshdr((invt_seshdr_t *)p, (invt_seshdr_t *)tmpbuf, 1); >>>>> bcopy(tmpbuf, p, sizeof(invt_seshdr_t)); >>>>> - >>>>> - /* get the seshdr and then, the remainder of the session */ >>>>> s->seshdr = (invt_seshdr_t *)p; >>>>> s->seshdr->sh_sess_off = -1; >>>>> p += sizeof(invt_seshdr_t); >>>>> - >>>>> xlate_invt_session((invt_session_t *)p, (invt_session_t >>>>> *)tmpbuf, 1); >>>>> bcopy (tmpbuf, p, sizeof(invt_session_t)); >>>>> s->ses = (invt_session_t *)p; >>>>> p += sizeof(invt_session_t); >>>>> /* the array of all the streams belonging to this session */ >>>>> - xlate_invt_stream((invt_stream_t *)p, (invt_stream_t *)tmpbuf, 1); >>>>> - bcopy(tmpbuf, p, sizeof(invt_stream_t)); >>>>> s->strms = (invt_stream_t *)p; >>>>> - p += s->ses->s_cur_nstreams * sizeof(invt_stream_t); >>>>> + for (i = 0; i < s->ses->s_cur_nstreams; i++) { >>>>> + xlate_invt_stream((invt_stream_t *)p, >>>>> + (invt_stream_t *)tmpbuf, 1); >>>>> + bcopy(tmpbuf, p, sizeof(invt_stream_t)); >>>>> + p += sizeof(invt_stream_t); >>>>> + } >>>>> /* all the media files */ >>>>> s->mfiles = (invt_mediafile_t *)p; >>>>> -- >>>>> 2.31.1 >>>>> >>>> >> > ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 3/4] xfsdump: fix on-media inventory stream packing 2022-10-13 3:15 [PATCH v3 0/4] xfsrestore: fix inventory unpacking Donald Douwsma 2022-10-13 3:15 ` [PATCH 1/4] xfsrestore: fix on-media inventory media unpacking Donald Douwsma 2022-10-13 3:15 ` [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking Donald Douwsma @ 2022-10-13 3:15 ` Donald Douwsma 2022-10-13 19:10 ` Darrick J. Wong 2022-10-13 3:15 ` [PATCH 4/4] xfsrestore: untangle inventory unpacking logic Donald Douwsma 3 siblings, 1 reply; 13+ messages in thread From: Donald Douwsma @ 2022-10-13 3:15 UTC (permalink / raw) To: linux-xfs; +Cc: Donald Douwsma, Darrick J . Wong With the on-media inventory now being restored for multiple streams we can see that the restored streams both claim to be for /dev/nst0. [root@rhel8 xfsdump-dev]# xfsdump -L "Test2" -f /dev/nst0 -M "tape1" -f /dev/nst1 -M "tape2" /boot ... [root@rhel8 xfsdump-dev]# rm -rf /var/lib/xfsdump/inventory /tmp/test1/* [root@rhel8 xfsdump-dev]# restore/xfsrestore -L Test2 -f /dev/nst0 -f /dev/nst1 /tmp/test2 restore/xfsrestore: using scsi tape (drive_scsitape) strategy restore/xfsrestore: using scsi tape (drive_scsitape) strategy restore/xfsrestore: version 3.1.10 (dump format 3.0) - type ^C for status and control ... restore/xfsrestore: Restore Summary: restore/xfsrestore: stream 0 /dev/nst0 OK (success) restore/xfsrestore: stream 1 /dev/nst1 ALREADY_DONE (another stream completed the operation) restore/xfsrestore: Restore Status: SUCCESS [root@rhel8 xfsdump-dev]# xfsdump -I file system 0: fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 session 0: mount point: rhel8:/boot device: rhel8:/dev/sda1 time: Wed Oct 12 15:36:55 2022 session label: "Test2" session id: 50be3b17-d9f9-414d-885b-ababf660e189 level: 0 resumed: NO subtree: NO streams: 2 stream 0: pathname: /dev/nst0 start: ino 133 offset 0 end: ino 28839 offset 0 interrupted: YES media files: 1 media file 0: mfile index: 2 mfile type: data mfile size: 165675008 mfile start: ino 133 offset 0 mfile end: ino 28839 offset 0 media label: "test" media id: e2e6978d-5546-4f1f-8c9e-307168071889 stream 1: pathname: /dev/nst0 start: ino 133 offset 0 end: ino 28839 offset 0 interrupted: YES media files: 1 media file 0: mfile index: 0 mfile type: data mfile size: 166723584 mfile start: ino 28839 offset 0 mfile end: ino 1572997 offset 0 media label: "tape2" media id: 1ad6d919-7159-42fb-a20f-5a2c4e3e24b1 xfsdump: Dump Status: SUCCESS [root@rhel8 xfsdump-dev]# Fix this by indexing the stream being packed for the on-media inventory. Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> Suggested-by: Darrick J. Wong <djwong@kernel.org> --- inventory/inv_stobj.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c index 025d431..fb4d93a 100644 --- a/inventory/inv_stobj.c +++ b/inventory/inv_stobj.c @@ -798,7 +798,7 @@ stobj_pack_sessinfo(int fd, invt_session_t *ses, invt_seshdr_t *hdr, sesbuf += sizeof(invt_session_t); for (i = 0; i < ses->s_cur_nstreams; i++) { - xlate_invt_stream(strms, (invt_stream_t *)sesbuf, 1); + xlate_invt_stream(&strms[i], (invt_stream_t *)sesbuf, 1); sesbuf += sizeof(invt_stream_t); } -- 2.31.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 3/4] xfsdump: fix on-media inventory stream packing 2022-10-13 3:15 ` [PATCH 3/4] xfsdump: fix on-media inventory stream packing Donald Douwsma @ 2022-10-13 19:10 ` Darrick J. Wong 2022-10-14 1:15 ` Donald Douwsma 0 siblings, 1 reply; 13+ messages in thread From: Darrick J. Wong @ 2022-10-13 19:10 UTC (permalink / raw) To: Donald Douwsma; +Cc: linux-xfs On Thu, Oct 13, 2022 at 02:15:17PM +1100, Donald Douwsma wrote: > With the on-media inventory now being restored for multiple streams we > can see that the restored streams both claim to be for /dev/nst0. > > [root@rhel8 xfsdump-dev]# xfsdump -L "Test2" -f /dev/nst0 -M "tape1" -f /dev/nst1 -M "tape2" /boot > ... > [root@rhel8 xfsdump-dev]# rm -rf /var/lib/xfsdump/inventory /tmp/test1/* > [root@rhel8 xfsdump-dev]# restore/xfsrestore -L Test2 -f /dev/nst0 -f /dev/nst1 /tmp/test2 > restore/xfsrestore: using scsi tape (drive_scsitape) strategy > restore/xfsrestore: using scsi tape (drive_scsitape) strategy > restore/xfsrestore: version 3.1.10 (dump format 3.0) - type ^C for status and control > ... > restore/xfsrestore: Restore Summary: > restore/xfsrestore: stream 0 /dev/nst0 OK (success) > restore/xfsrestore: stream 1 /dev/nst1 ALREADY_DONE (another stream completed the operation) > restore/xfsrestore: Restore Status: SUCCESS > [root@rhel8 xfsdump-dev]# xfsdump -I > file system 0: > fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 > session 0: > mount point: rhel8:/boot > device: rhel8:/dev/sda1 > time: Wed Oct 12 15:36:55 2022 > session label: "Test2" > session id: 50be3b17-d9f9-414d-885b-ababf660e189 > level: 0 > resumed: NO > subtree: NO > streams: 2 > stream 0: > pathname: /dev/nst0 > start: ino 133 offset 0 > end: ino 28839 offset 0 > interrupted: YES > media files: 1 > media file 0: > mfile index: 2 > mfile type: data > mfile size: 165675008 > mfile start: ino 133 offset 0 > mfile end: ino 28839 offset 0 > media label: "test" It's odd that you have -M tape1 above but this ends up labelled "test"? If that isn't just a munged patch message, then that might need fixing (separate patch) as well. The code change looks correct though. Thanks for fixing dump. Reviewed-by: Darrick J. Wong <djwong@kernel.org> --D > media id: e2e6978d-5546-4f1f-8c9e-307168071889 > stream 1: > pathname: /dev/nst0 > start: ino 133 offset 0 > end: ino 28839 offset 0 > interrupted: YES > media files: 1 > media file 0: > mfile index: 0 > mfile type: data > mfile size: 166723584 > mfile start: ino 28839 offset 0 > mfile end: ino 1572997 offset 0 > media label: "tape2" > media id: 1ad6d919-7159-42fb-a20f-5a2c4e3e24b1 > xfsdump: Dump Status: SUCCESS > [root@rhel8 xfsdump-dev]# > > Fix this by indexing the stream being packed for the on-media inventory. > > Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> > Suggested-by: Darrick J. Wong <djwong@kernel.org> > --- > inventory/inv_stobj.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c > index 025d431..fb4d93a 100644 > --- a/inventory/inv_stobj.c > +++ b/inventory/inv_stobj.c > @@ -798,7 +798,7 @@ stobj_pack_sessinfo(int fd, invt_session_t *ses, invt_seshdr_t *hdr, > sesbuf += sizeof(invt_session_t); > > for (i = 0; i < ses->s_cur_nstreams; i++) { > - xlate_invt_stream(strms, (invt_stream_t *)sesbuf, 1); > + xlate_invt_stream(&strms[i], (invt_stream_t *)sesbuf, 1); > sesbuf += sizeof(invt_stream_t); > } > > -- > 2.31.1 > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 3/4] xfsdump: fix on-media inventory stream packing 2022-10-13 19:10 ` Darrick J. Wong @ 2022-10-14 1:15 ` Donald Douwsma 0 siblings, 0 replies; 13+ messages in thread From: Donald Douwsma @ 2022-10-14 1:15 UTC (permalink / raw) To: Darrick J. Wong; +Cc: linux-xfs On 14/10/2022 06:10, Darrick J. Wong wrote: > On Thu, Oct 13, 2022 at 02:15:17PM +1100, Donald Douwsma wrote: >> With the on-media inventory now being restored for multiple streams we >> can see that the restored streams both claim to be for /dev/nst0. >> >> [root@rhel8 xfsdump-dev]# xfsdump -L "Test2" -f /dev/nst0 -M "tape1" -f /dev/nst1 -M "tape2" /boot >> ... >> [root@rhel8 xfsdump-dev]# rm -rf /var/lib/xfsdump/inventory /tmp/test1/* >> [root@rhel8 xfsdump-dev]# restore/xfsrestore -L Test2 -f /dev/nst0 -f /dev/nst1 /tmp/test2 >> restore/xfsrestore: using scsi tape (drive_scsitape) strategy >> restore/xfsrestore: using scsi tape (drive_scsitape) strategy >> restore/xfsrestore: version 3.1.10 (dump format 3.0) - type ^C for status and control >> ... >> restore/xfsrestore: Restore Summary: >> restore/xfsrestore: stream 0 /dev/nst0 OK (success) >> restore/xfsrestore: stream 1 /dev/nst1 ALREADY_DONE (another stream completed the operation) >> restore/xfsrestore: Restore Status: SUCCESS >> [root@rhel8 xfsdump-dev]# xfsdump -I >> file system 0: >> fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 >> session 0: >> mount point: rhel8:/boot >> device: rhel8:/dev/sda1 >> time: Wed Oct 12 15:36:55 2022 >> session label: "Test2" >> session id: 50be3b17-d9f9-414d-885b-ababf660e189 >> level: 0 >> resumed: NO >> subtree: NO >> streams: 2 >> stream 0: >> pathname: /dev/nst0 >> start: ino 133 offset 0 >> end: ino 28839 offset 0 >> interrupted: YES >> media files: 1 >> media file 0: >> mfile index: 2 >> mfile type: data >> mfile size: 165675008 >> mfile start: ino 133 offset 0 >> mfile end: ino 28839 offset 0 >> media label: "test" > > It's odd that you have -M tape1 above but this ends up labelled "test"? > If that isn't just a munged patch message, then that might need fixing > (separate patch) as well. I looked back over the terminals where I tested this and they're all tape1, I think this was an earlier cut n pasta mistake when I prepared the commit message. Its probably worth fixing the message because not everyone will want to setup a virtual tape library to look at this. Thanks for the reviews, Don > The code change looks correct though. Thanks for fixing dump. > > Reviewed-by: Darrick J. Wong <djwong@kernel.org> > > --D > >> media id: e2e6978d-5546-4f1f-8c9e-307168071889 >> stream 1: >> pathname: /dev/nst0 >> start: ino 133 offset 0 >> end: ino 28839 offset 0 >> interrupted: YES >> media files: 1 >> media file 0: >> mfile index: 0 >> mfile type: data >> mfile size: 166723584 >> mfile start: ino 28839 offset 0 >> mfile end: ino 1572997 offset 0 >> media label: "tape2" >> media id: 1ad6d919-7159-42fb-a20f-5a2c4e3e24b1 >> xfsdump: Dump Status: SUCCESS >> [root@rhel8 xfsdump-dev]# >> >> Fix this by indexing the stream being packed for the on-media inventory. >> >> Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> >> Suggested-by: Darrick J. Wong <djwong@kernel.org> >> --- >> inventory/inv_stobj.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c >> index 025d431..fb4d93a 100644 >> --- a/inventory/inv_stobj.c >> +++ b/inventory/inv_stobj.c >> @@ -798,7 +798,7 @@ stobj_pack_sessinfo(int fd, invt_session_t *ses, invt_seshdr_t *hdr, >> sesbuf += sizeof(invt_session_t); >> >> for (i = 0; i < ses->s_cur_nstreams; i++) { >> - xlate_invt_stream(strms, (invt_stream_t *)sesbuf, 1); >> + xlate_invt_stream(&strms[i], (invt_stream_t *)sesbuf, 1); >> sesbuf += sizeof(invt_stream_t); >> } >> >> -- >> 2.31.1 >> > ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 4/4] xfsrestore: untangle inventory unpacking logic 2022-10-13 3:15 [PATCH v3 0/4] xfsrestore: fix inventory unpacking Donald Douwsma ` (2 preceding siblings ...) 2022-10-13 3:15 ` [PATCH 3/4] xfsdump: fix on-media inventory stream packing Donald Douwsma @ 2022-10-13 3:15 ` Donald Douwsma 3 siblings, 0 replies; 13+ messages in thread From: Donald Douwsma @ 2022-10-13 3:15 UTC (permalink / raw) To: linux-xfs; +Cc: Donald Douwsma, Darrick J . Wong stobj_unpack_sessinfo returns bool_t, fix logic in pi_addfile so errors can be properly reported. Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> --- restore/content.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/restore/content.c b/restore/content.c index b3999f9..04b6f81 100644 --- a/restore/content.c +++ b/restore/content.c @@ -5463,17 +5463,14 @@ pi_addfile(Media_t *Mediap, * desc. */ sessp = 0; - if (!buflen) { - ok = BOOL_FALSE; - } else { - /* extract the session information from the buffer */ - if (stobj_unpack_sessinfo(bufp, buflen, &sessinfo)<0) { - ok = BOOL_FALSE; - } else { + ok = BOOL_FALSE; + /* extract the session information from the buffer */ + if (buflen && + stobj_unpack_sessinfo(bufp, buflen, &sessinfo)) { stobj_convert_sessinfo(&sessp, &sessinfo); ok = BOOL_TRUE; - } } + if (!ok || !sessp) { mlog(MLOG_DEBUG | MLOG_WARNING | MLOG_MEDIA, _( "on-media session " -- 2.31.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v4 0/4] xfsrestore: fix inventory unpacking @ 2022-10-14 7:58 Donald Douwsma 2022-10-14 7:58 ` [PATCH 1/4] xfsrestore: fix on-media inventory media unpacking Donald Douwsma 0 siblings, 1 reply; 13+ messages in thread From: Donald Douwsma @ 2022-10-14 7:58 UTC (permalink / raw) To: linux-xfs; +Cc: Donald Douwsma When xfsrestore reads its inventory from tape it fails to convert the media record on bigendian systems, if the online inventory is unavailable this results in invalid data being writen to the online inventory and failure to restore non-directory files. The series fixes the converstion and related issues. --- v2 - Seperate out cleanup and content.c changes, fix whitespace. - Show a full reproducer in the first patch. V3 - Fix whitespace and Signed-off-by. - Make for loop formatting consistent. - Rename patches to make their intent clearer. - Add xfsdump: fix on-media inventory stream packing. - Add descriptions that reproduce the stream problems. V4 - Rerun testcase in "xfsdump: fix on-media inventory stream packing" description to avoid confusion. Donald Douwsma (4): xfsrestore: fix on-media inventory media unpacking xfsrestore: fix on-media inventory stream unpacking xfsdump: fix on-media inventory stream packing xfsrestore: untangle inventory unpacking logic inventory/inv_stobj.c | 42 +++++++++++++++--------------------------- restore/content.c | 13 +++++-------- 2 files changed, 20 insertions(+), 35 deletions(-) -- 2.31.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 1/4] xfsrestore: fix on-media inventory media unpacking 2022-10-14 7:58 [PATCH v4 0/4] xfsrestore: fix inventory unpacking Donald Douwsma @ 2022-10-14 7:58 ` Donald Douwsma 0 siblings, 0 replies; 13+ messages in thread From: Donald Douwsma @ 2022-10-14 7:58 UTC (permalink / raw) To: linux-xfs; +Cc: Donald Douwsma, Darrick J . Wong When xfsrestore reads the inventory from tape media it fails to convert media file records from bigendian. If the xfsdump inventory is not available xfsrestore will write this invalid record to the on-line inventory. [root@rhel8 ~]# xfsdump -L Test1 -M "" -f /dev/st0 /boot > /dev/null [root@rhel8 ~]# xfsdump -L Test2 -M "" -f /dev/st0 /boot > /dev/null [root@rhel8 ~]# rm -rf /var/lib/xfsdump/inventory/ [root@rhel8 ~]# mt -f /dev/nst0 asf 2 [root@rhel8 ~]# xfsrestore -f /dev/nst0 -L Test2 /tmp/test2 xfsrestore: using scsi tape (drive_scsitape) strategy xfsrestore: version 3.1.8 (dump format 3.0) - type ^C for status and control xfsrestore: searching media for dump xfsrestore: preparing drive xfsrestore: examining media file 3 xfsrestore: found dump matching specified label: xfsrestore: hostname: rhel8 xfsrestore: mount point: /boot xfsrestore: volume: /dev/sda1 xfsrestore: session time: Tue Sep 27 16:05:28 2022 xfsrestore: level: 0 xfsrestore: session label: "Test2" xfsrestore: media label: "" xfsrestore: file system id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 xfsrestore: session id: 62402423-7ae0-49ed-8ecb-9e5bc7642b91 xfsrestore: media id: 47ba45ca-3417-4006-ab10-3dc6419b83e2 xfsrestore: incorporating on-media session inventory into online inventory xfsrestore: /var/lib/xfsdump/inventory created xfsrestore: using on-media session inventory xfsrestore: searching media for directory dump xfsrestore: rewinding xfsrestore: examining media file 0 xfsrestore: inventory session uuid (62402423-7ae0-49ed-8ecb-9e5bc7642b91) does not match the media header's session uuid (1771d9e8-a1ba-4e87-a61e-f6be97e41b45) xfsrestore: examining media file 1 xfsrestore: inventory session uuid (62402423-7ae0-49ed-8ecb-9e5bc7642b91) does not match the media header's session uuid (1771d9e8-a1ba-4e87-a61e-f6be97e41b45) xfsrestore: examining media file 2 xfsrestore: reading directories xfsrestore: 9 directories and 320 entries processed xfsrestore: directory post-processing xfsrestore: restore complete: 0 seconds elapsed xfsrestore: Restore Summary: xfsrestore: stream 0 /dev/nst0 OK (success) xfsrestore: Restore Status: SUCCESS [root@rhel8 ~]# xfsdump -I file system 0: fs id: 26dd5aa0-b901-4cf5-9b68-0c5753cb3ab8 session 0: mount point: rhel8:/boot device: rhel8:/dev/sda1 time: Tue Sep 27 16:05:28 2022 session label: "Test2" session id: 62402423-7ae0-49ed-8ecb-9e5bc7642b91 level: 0 resumed: NO subtree: NO streams: 1 stream 0: pathname: /dev/st0 start: ino 133 offset 0 end: ino 1572997 offset 0 interrupted: YES media files: 1 media file 0: mfile index: 33554432 mfile type: data mfile size: 211187836911616 mfile start: ino 9583660007044415488 offset 0 mfile end: ino 9583686395323482112 offset 0 media label: "" media id: 47ba45ca-3417-4006-ab10-3dc6419b83e2 xfsdump: Dump Status: SUCCESS [root@rhel8 ~]# [root@rhel8 ~]# ls /tmp/test2 efi grub2 loader The invalid start and end inode information cause xfsrestore to consider that non-directory files do not reside in the current media and will fail to restore them. The behaviour of an initial restore may succeed if the position of the tape is such that the data file is encountered before the inventory file, or if there is only one dump session on tape, xfsrestore is somewhat inconsistent in this regard. Subsequent restores will use the invalid on-line inventory and fail to restore files. Fix this by correctly unpacking the inventory data. Signed-off-by: Donald Douwsma <ddouwsma@redhat.com> Reviewed-by: Darrick J. Wong <djwong@kernel.org> --- inventory/inv_stobj.c | 27 +++++++-------------------- 1 file changed, 7 insertions(+), 20 deletions(-) diff --git a/inventory/inv_stobj.c b/inventory/inv_stobj.c index c20e71c..b461666 100644 --- a/inventory/inv_stobj.c +++ b/inventory/inv_stobj.c @@ -1008,7 +1008,7 @@ stobj_unpack_sessinfo( size_t bufsz, invt_sessinfo_t *s) { - uint i; + uint i, j; char *tmpbuf; char *p = (char *)bufp; @@ -1087,26 +1087,13 @@ stobj_unpack_sessinfo( /* all the media files */ s->mfiles = (invt_mediafile_t *)p; - -#ifdef INVT_DELETION - { - int tmpfd = open("moids", O_RDWR | O_CREAT, S_IRUSR|S_IWUSR); - uint j; - invt_mediafile_t *mmf = s->mfiles; - for (i=0; i< s->ses->s_cur_nstreams; i++) { - for (j=0; j< s->strms[i].st_nmediafiles; - j++, mmf++) - xlate_invt_mediafile((invt_mediafile_t *)mmf, (invt_mediafile_t *)tmpbuf, 1); - bcopy(tmpbuf, mmf, sizeof(invt_mediafile_t)); - put_invtrecord(tmpfd, &mmf->mf_moid, - sizeof(uuid_t), 0, SEEK_END, 0); - } - close(tmpfd); - } -#endif for (i = 0; i < s->ses->s_cur_nstreams; i++) { - p += (size_t) (s->strms[i].st_nmediafiles) - * sizeof(invt_mediafile_t); + for(j = 0; j < s->strms[i].st_nmediafiles; j++) { + xlate_invt_mediafile((invt_mediafile_t *)p, + (invt_mediafile_t *)tmpbuf, 1); + bcopy(tmpbuf, p, sizeof(invt_mediafile_t)); + p += sizeof(invt_mediafile_t); + } } /* sanity check the size of the buffer given to us vs. the size it -- 2.31.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
end of thread, other threads:[~2022-10-16 21:54 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-10-13 3:15 [PATCH v3 0/4] xfsrestore: fix inventory unpacking Donald Douwsma 2022-10-13 3:15 ` [PATCH 1/4] xfsrestore: fix on-media inventory media unpacking Donald Douwsma 2022-10-13 3:15 ` [PATCH 2/4] xfsrestore: fix on-media inventory stream unpacking Donald Douwsma 2022-10-13 19:05 ` Darrick J. Wong 2022-10-14 3:37 ` Donald Douwsma 2022-10-14 4:15 ` Donald Douwsma 2022-10-14 15:46 ` Darrick J. Wong 2022-10-16 21:54 ` Donald Douwsma 2022-10-13 3:15 ` [PATCH 3/4] xfsdump: fix on-media inventory stream packing Donald Douwsma 2022-10-13 19:10 ` Darrick J. Wong 2022-10-14 1:15 ` Donald Douwsma 2022-10-13 3:15 ` [PATCH 4/4] xfsrestore: untangle inventory unpacking logic Donald Douwsma 2022-10-14 7:58 [PATCH v4 0/4] xfsrestore: fix inventory unpacking Donald Douwsma 2022-10-14 7:58 ` [PATCH 1/4] xfsrestore: fix on-media inventory media unpacking Donald Douwsma
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).