From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dennis Yang Subject: Re: Possible data corruption with dm-thin Date: Mon, 27 Jun 2016 17:32:35 +0800 Message-ID: References: <20160624135510.GA13946@roamer.bos.redhat.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5457158760609248397==" Return-path: In-Reply-To: <20160624135510.GA13946@roamer.bos.redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com To: Dennis Yang , device-mapper development List-Id: dm-devel.ids --===============5457158760609248397== Content-Type: multipart/alternative; boundary=047d7bb049c67f3d0405363f342e --047d7bb049c67f3d0405363f342e Content-Type: text/plain; charset=UTF-8 Hi Joe, 2016-06-24 21:55 GMT+08:00 Edward Thornber : > Hi Dennis, > > On Tue, Jun 21, 2016 at 03:56:26PM +0800, Dennis Yang wrote: > > So my question is, does dm-thin have any mechanism to eliminate the race > > when > > discarded block is reused right away by another device? > > I'll try and recreate your scenario. The transaction-manager will not > reallocate a block that's been freed within a transaction so that we > can always rollback. So as long as there hasn't been a commit, > reallocation shouldn't be possible. This is what normally guards > allocation, but in your case I think you may be onto something and we > may have to hold onto the data cell until the passdown discards are > complete. > > - Joe > In my experience, this issue is pretty hard to reproduce solely by a thin-pool which is built on top of a regular hard disk or RAID, since I rarely observe the DISCARD and WRITE coming from different thin devices get reordered in the lower level. In my system, the thin-pool is built on top of another device-mapper device providing data tiering functionality which could delay the DISCARD request a little bit and lead to a request reorder situation. Based on the log I had, I suspect that there was metadata commit before another thin device reused the discarded block while the DISCARD request was still being processed by the lower level stacks. In this case, holding the data cell until the passdown discards are complete seems to only protect the discarded block from being reallocated by the same thin device which allocates the block in the first place, because only those write I/Os that are going to the same device will be prisoned. In my opinion, I think maybe the to-be-discard block should only be freed after the passdown discards have been complete. I had only written a patch in this way and applied it on my site since last week, and the issue had not been seen on my site for 72 hours. I am not very confident about whether this patch will cause any side effect, I would be highly appreciated if you can share your concern with me. Thanks for your reply, Dennis -- Dennis Yang QNAP Systems, Inc. Skype: qnap.dennis.yang Email: dennisyang@qnap.com Tel: (+886)-2-2393-5152 ext. 15018 Address: 13F., No.56, Sec. 1, Xinsheng S. Rd., Zhongzheng Dist., Taipei City, Taiwan --047d7bb049c67f3d0405363f342e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Joe,
request a little bit and lead to a request reorder situation.=C2=A0