From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756480Ab2JQK55 (ORCPT ); Wed, 17 Oct 2012 06:57:57 -0400 Received: from mail-wg0-f44.google.com ([74.125.82.44]:51091 "EHLO mail-wg0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751132Ab2JQK5z convert rfc822-to-8bit (ORCPT ); Wed, 17 Oct 2012 06:57:55 -0400 MIME-Version: 1.0 In-Reply-To: <87txtu206u.fsf@devron.myhome.or.jp> References: <1350138661-2454-1-git-send-email-linkinjeon@gmail.com> <87lif958lb.fsf@devron.myhome.or.jp> <87txtu206u.fsf@devron.myhome.or.jp> Date: Wed, 17 Oct 2012 19:57:53 +0900 Message-ID: Subject: Re: [PATCH v2] fat: editions to support fat_fallocate() From: Namjae Jeon To: OGAWA Hirofumi Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Ravishankar N , Amit Sahrawat , linux-fsdevel Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 2012/10/16, OGAWA Hirofumi : > Namjae Jeon writes: > >>> What is real usage pattern of persistent across remounts on FAT? >> Yes, like a TORRENT FILE -> it reserves space in advance >> even though the system can be rebooted/disk unmounted and remount >> but the space still remains there - as long as the torrent exists >> Or if Torrent case does not matches currently >> Then, Consider a case for a TV series to be recorded >> Since – we want all the parts to be recorded on the same file (i.e., >> APPEND write) – and in such cases there are chances of TV shutdown, >> device unmount-mount again. So, we need to have the space to be remain >> available in such cases. > > The expectation of fallocate() is just for space reservation? If it was > just for space reservation, I'm not sure, why TV applications can't > reserve in userland without any kernel help (I wonder who interrupts TV > application). I feel a bit, it may be more lightweight than fallocate(), > and more reliable than out of spec fallocate(). > > I'm still not sure why apps really want fallocate() on FAT. Yes, it is for user space reservation. >>From the application perspective it is needed to realize in advance how much space is needed for that file write – so the requirement is precisely that the space reserved is entirely for me and no other I/O operation in that time should consume the space. Of course, as you said, space can be pre-allocated from user space by doing expanding truncate. Main drawbacks for reserving through USER space: 1) If we need to allocate 1GB space -> seek (1GB) and write -> it will ZEROUT the 1GB area (which is very time consuming) just for reserving space. 2) The Application must always be aware of the SEEK OFFSET - otherwise the APPEND WRITE will never occur and file is closed/opened (Append mode) again for writing – it will instead start writing from the end of file which is past the reserved space. So, this will also result in losing space in such case – if application is not keeping track of OFFSET 3) If suppose from user space we are doing expanding truncate of 1GB and suppose it fails after 256MB of allocation - in that case it did allocation of 256MB blocks - did ZEROUT for all these blocks and then returned failure - which is not optimal for just allocation of space. While if we make use of FALLOCATE 1) It allows reserving the space in advance without any delay. 2) Since, the space is reserved in advance. So, if suppose space is reserved for 1hour TV recording than any other application in the background cannot cause recording to fail in case of “no free space” left as it already pre-allocated space. Only other Applications will close. 3) It allows for APPEND write to continue smoothly without actually keeping track of the file state, offset. 4) Initially when the disk is not fragmented. It allows the possibility to get contiguous blocks and thus reducing fragmentation for that file. Thanks. > >>> If once device was unmounted, we can't know the state of FS anymore, >>> there are >>> many implementations of FAT. And preallocation is not in the spec. >> I agree, As you said before, we can make fat fallocate feature as >> configurable – so this is entirely in the hands of USER. >>> >>> I worry to break something. And I guess the freeing preallocation on >>> last close may fix the issue for usage. >> Okay, we can avoid most of your concerns except suddenly unplugging usb >> device. >> But fallocate behavior will be different with other filesystem. >> >> How about to make fat fallocate with configuration to be used by users >> is having needs? > > Hmm... I'm not still convinced to add makes really apps happy. Maybe, > I'm sill not understanding your usage. I think the out of spec feature > wouldn't be added if it was just a "better than nothing". > > Thanks. > -- > OGAWA Hirofumi > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Namjae Jeon Subject: Re: [PATCH v2] fat: editions to support fat_fallocate() Date: Wed, 17 Oct 2012 19:57:53 +0900 Message-ID: References: <1350138661-2454-1-git-send-email-linkinjeon@gmail.com> <87lif958lb.fsf@devron.myhome.or.jp> <87txtu206u.fsf@devron.myhome.or.jp> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Ravishankar N , Amit Sahrawat , linux-fsdevel To: OGAWA Hirofumi Return-path: In-Reply-To: <87txtu206u.fsf@devron.myhome.or.jp> Sender: linux-kernel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org 2012/10/16, OGAWA Hirofumi : > Namjae Jeon writes: > >>> What is real usage pattern of persistent across remounts on FAT? >> Yes, like a TORRENT FILE -> it reserves space in advance >> even though the system can be rebooted/disk unmounted and remount >> but the space still remains there - as long as the torrent exists >> Or if Torrent case does not matches currently >> Then, Consider a case for a TV series to be recorded >> Since =E2=80=93 we want all the parts to be recorded on the same fil= e (i.e., >> APPEND write) =E2=80=93 and in such cases there are chances of TV sh= utdown, >> device unmount-mount again. So, we need to have the space to be rema= in >> available in such cases. > > The expectation of fallocate() is just for space reservation? If it w= as > just for space reservation, I'm not sure, why TV applications can't > reserve in userland without any kernel help (I wonder who interrupts = TV > application). I feel a bit, it may be more lightweight than fallocate= (), > and more reliable than out of spec fallocate(). > > I'm still not sure why apps really want fallocate() on FAT. Yes, it is for user space reservation. =46rom the application perspective it is needed to realize in advance how much space is needed for that file write =E2=80=93 so the requireme= nt is precisely that the space reserved is entirely for me and no other I/O operation in that time should consume the space. Of course, as you said, space can be pre-allocated from user space by doing expanding truncate. Main drawbacks for reserving through USER space: 1) If we need to allocate 1GB space -> seek (1GB) and write -> it will ZEROUT the 1GB area (which is very time consuming) just for reserving space. 2) The Application must always be aware of the SEEK OFFSET - otherwise the APPEND WRITE will never occur and file is closed/opened (Append mode) again for writing =E2=80=93 it will instead start writing from th= e end of file which is past the reserved space. So, this will also result in losing space in such case =E2=80=93 if application is not keeping track= of OFFSET 3) If suppose from user space we are doing expanding truncate of 1GB and suppose it fails after 256MB of allocation - in that case it did allocation of 256MB blocks - did ZEROUT for all these blocks and then returned failure - which is not optimal for just allocation of space. While if we make use of FALLOCATE 1) It allows reserving the space in advance without any delay. 2) Since, the space is reserved in advance. So, if suppose space is reserved for 1hour TV recording than any other application in the background cannot cause recording to fail in case of =E2=80=9Cno free s= pace=E2=80=9D left as it already pre-allocated space. Only other Applications will close. 3) It allows for APPEND write to continue smoothly without actually keeping track of the file state, offset. 4) Initially when the disk is not fragmented. It allows the possibility to get contiguous blocks and thus reducing fragmentation for that file. Thanks. > >>> If once device was unmounted, we can't know the state of FS anymore= , >>> there are >>> many implementations of FAT. And preallocation is not in the spec. >> I agree, As you said before, we can make fat fallocate feature as >> configurable =E2=80=93 so this is entirely in the hands of USER. >>> >>> I worry to break something. And I guess the freeing preallocation o= n >>> last close may fix the issue for usage. >> Okay, we can avoid most of your concerns except suddenly unplugging = usb >> device. >> But fallocate behavior will be different with other filesystem. >> >> How about to make fat fallocate with configuration to be used by use= rs >> is having needs? > > Hmm... I'm not still convinced to add makes really apps happy. Maybe, > I'm sill not understanding your usage. I think the out of spec featur= e > wouldn't be added if it was just a "better than nothing". > > Thanks. > -- > OGAWA Hirofumi >