From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752587AbdHPS2h (ORCPT ); Wed, 16 Aug 2017 14:28:37 -0400 Received: from relmlor4.renesas.com ([210.160.252.174]:14686 "EHLO relmlie3.idc.renesas.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752555AbdHPS2e (ORCPT ); Wed, 16 Aug 2017 14:28:34 -0400 X-IronPort-AV: E=Sophos;i="5.41,383,1498489200"; d="scan'208";a="254759681" From: Chris Brandt To: Nicolas Pitre , Alexander Viro CC: "linux-fsdevel@vger.kernel.org" , "linux-embedded@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: RE: [PATCH v2 3/5] cramfs: implement uncompressed and arbitrary data block positioning Thread-Topic: [PATCH v2 3/5] cramfs: implement uncompressed and arbitrary data block positioning Thread-Index: AQHTFrYi4m2htrBmkEK3h732ZuqAvaKHSvYw Date: Wed, 16 Aug 2017 18:28:29 +0000 Message-ID: References: <20170816173536.1879-1-nicolas.pitre@linaro.org> <20170816173536.1879-4-nicolas.pitre@linaro.org> In-Reply-To: <20170816173536.1879-4-nicolas.pitre@linaro.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Chris.Brandt@renesas.com; x-originating-ip: [4.59.13.106] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;SG2PR06MB1744;20:IH2qlJoOcLNY2YrKiphzMqTZK6+wHy1knXOl1Dhg8yU46o4cstptHRflLWUUs1LlzPFBbTlqmRN0cD84JNATV92/xzCZrb45cN1Q12/QddC47YWYafGXEE1CdWH18P2FT+erJqCc4BZJvHA5GNB+C2f9+OcwhJy8LQybDSY4xiE= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 6b4e33ff-a290-44d9-d78f-08d4e4d497e0 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(48565401081)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:SG2PR06MB1744; x-ms-traffictypediagnostic: SG2PR06MB1744: x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(6055026)(6041248)(20161123555025)(20161123564025)(20161123558100)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:SG2PR06MB1744;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:SG2PR06MB1744; x-forefront-prvs: 0401647B7F x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(39860400002)(189002)(24454002)(199003)(9686003)(3846002)(5660300001)(55016002)(54906002)(5250100002)(14454004)(2900100001)(6506006)(53936002)(6436002)(72206003)(99286003)(66066001)(6116002)(4326008)(97736004)(7696004)(3280700002)(478600001)(3660700001)(25786009)(105586002)(102836003)(6246003)(2906002)(305945005)(106356001)(68736007)(229853002)(101416001)(74316002)(8936002)(575784001)(86362001)(189998001)(7736002)(2950100002)(33656002)(50986999)(54356999)(81166006)(76176999)(81156014)(8676002);DIR:OUT;SFP:1102;SCL:1;SRVR:SG2PR06MB1744;H:SG2PR06MB1165.apcprd06.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 X-OriginatorOrg: renesas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Aug 2017 18:28:29.6814 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 53d82571-da19-47e4-9cb4-625a166a4a2a X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2PR06MB1744 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by nfs id v7GIShAm032695 On Wednesday, August 16, 2017, Nicolas Pitre wrote: > Two new capabilities are introduced here: > > - The ability to store some blocks uncompressed. > > - The ability to locate blocks anywhere. > > Those capabilities can be used independently, but the combination > opens the possibility for execute-in-place (XIP) of program text segments > that must remain uncompressed, and in the MMU case, must have a specific > alignment. It is even possible to still have the writable data segments > from the same file compressed as they have to be copied into RAM anyway. > > This is achieved by giving special meanings to some unused block pointer > bits while remaining compatible with legacy cramfs images. > > Signed-off-by: Nicolas Pitre > --- > fs/cramfs/README | 31 ++++++++++++++- > fs/cramfs/inode.c | 87 +++++++++++++++++++++++++++++++++---- > ----- > include/uapi/linux/cramfs_fs.h | 20 +++++++++- > 3 files changed, 118 insertions(+), 20 deletions(-) > > diff --git a/fs/cramfs/README b/fs/cramfs/README > index 9d4e7ea311..d71b27e0ff 100644 > --- a/fs/cramfs/README > +++ b/fs/cramfs/README > @@ -49,17 +49,46 @@ same as the start of the (i+1)'th if there is > one). The first > immediately follows the last for the file. > s are each 32 bits long. > > +When the CRAMFS_FLAG_EXT_BLOCK_POINTERS capability bit is set, each > +'s top bits may contain special flags as follows: > + > +CRAMFS_BLK_FLAG_UNCOMPRESSED (bit 31): > + The block data is not compressed and should be copied verbatim. > + > +CRAMFS_BLK_FLAG_DIRECT_PTR (bit 30): > + The stores the actual block start offset and not > + its end, shifted right by 2 bits. The block must therefore be > + aligned to a 4-byte boundary. The block size is either blksize > + if CRAMFS_BLK_FLAG_UNCOMPRESSED is also specified, otherwise > + the compressed data length is included in the first 2 bytes of > + the block data. This is used to allow discontiguous data layout > + and specific data block alignments e.g. for XIP applications. > + > + > The order of 's is a depth-first descent of the directory > tree, i.e. the same order as `find -size +0 \( -type f -o -type l \) > -print'. > > > : The i'th is the output of zlib's compress function > -applied to the i'th blksize-sized chunk of the input data. > +applied to the i'th blksize-sized chunk of the input data if the > +corresponding CRAMFS_BLK_FLAG_UNCOMPRESSED bit is not set, > +otherwise it is the input data directly. > (For the last of the file, the input may of course be smaller.) > Each may be a different size. (See above.) > + > s are merely byte-aligned, not generally u32-aligned. > > +When CRAMFS_BLK_FLAG_DIRECT_PTR is specified then the corresponding > + may be located anywhere and not necessarily contiguous with > +the previous/next blocks. In that case it is minimally u32-aligned. > +If CRAMFS_BLK_FLAG_UNCOMPRESSED is also specified then the size is always > +blksize except for the last block which is limited by the file length. > +If CRAMFS_BLK_FLAG_DIRECT_PTR is set and CRAMFS_BLK_FLAG_UNCOMPRESSED > +is not set then the first 2 bytes of the block contains the size of the > +remaining block data as this cannot be determined from the placement of > +logically adjacent blocks. > + > > Holes > ----- > diff --git a/fs/cramfs/inode.c b/fs/cramfs/inode.c > index 393eb27ef4..b825ae162c 100644 > --- a/fs/cramfs/inode.c > +++ b/fs/cramfs/inode.c > @@ -636,33 +636,84 @@ static int cramfs_readpage(struct file *file, struct > page *page) > if (page->index < maxblock) { > struct super_block *sb = inode->i_sb; > u32 blkptr_offset = OFFSET(inode) + page->index*4; > - u32 start_offset, compr_len; > + u32 block_ptr, block_start, block_len; > + bool uncompressed, direct; > > - start_offset = OFFSET(inode) + maxblock*4; > mutex_lock(&read_mutex); > - if (page->index) > - start_offset = *(u32 *) cramfs_read(sb, blkptr_offset-4, > - 4); > - compr_len = (*(u32 *) cramfs_read(sb, blkptr_offset, 4) - > - start_offset); > - mutex_unlock(&read_mutex); > + block_ptr = *(u32 *) cramfs_read(sb, blkptr_offset, 4); > + uncompressed = (block_ptr & CRAMFS_BLK_FLAG_UNCOMPRESSED); > + direct = (block_ptr & CRAMFS_BLK_FLAG_DIRECT_PTR); > + block_ptr &= ~CRAMFS_BLK_FLAGS; > + > + if (direct) { > + /* > + * The block pointer is an absolute start pointer, > + * shifted by 2 bits. The size is included in the > + * first 2 bytes of the data block when compressed, > + * or PAGE_SIZE otherwise. > + */ > + block_start = block_ptr << 2; > + if (uncompressed) { > + block_len = PAGE_SIZE; > + /* if last block: cap to file length */ > + if (page->index == maxblock - 1) > + block_len = offset_in_page(inode->i_size); > + } else { > + block_len = *(u16 *) > + cramfs_read(sb, block_start, 2); > + block_start += 2; > + } > + } else { > + /* > + * The block pointer indicates one past the end of > + * the current block (start of next block). If this > + * is the first block then it starts where the block > + * pointer table ends, otherwise its start comes > + * from the previous block's pointer. > + */ > + block_start = OFFSET(inode) + maxblock*4; > + if (page->index) > + block_start = *(u32 *) > + cramfs_read(sb, blkptr_offset-4, 4); > + /* Beware... previous ptr might be a direct ptr */ > + if (unlikely(block_start & CRAMFS_BLK_FLAG_DIRECT_PTR)) > { > + /* See comments on earlier code. */ > + u32 prev_start = block_start; > + block_start = prev_start & ~CRAMFS_BLK_FLAGS; > + block_start <<= 2; > + if (prev_start & CRAMFS_BLK_FLAG_UNCOMPRESSED) { > + block_start += PAGE_SIZE; > + } else { > + block_len = *(u16 *) > + cramfs_read(sb, block_start, 2); > + block_start += 2 + block_len; > + } > + } > + block_start &= ~CRAMFS_BLK_FLAGS; > + block_len = block_ptr - block_start; > + } > > - if (compr_len == 0) > + if (block_len == 0) > ; /* hole */ > - else if (unlikely(compr_len > (PAGE_SIZE << 1))) { > - pr_err("bad compressed blocksize %u\n", > - compr_len); > + else if (unlikely(block_len > 2*PAGE_SIZE || > + (uncompressed && block_len > PAGE_SIZE))) { > + mutex_unlock(&read_mutex); > + pr_err("bad data blocksize %u\n", block_len); > goto err; > + } else if (uncompressed) { > + memcpy(pgdata, > + cramfs_read(sb, block_start, block_len), > + block_len); > + bytes_filled = block_len; > } else { > - mutex_lock(&read_mutex); > bytes_filled = cramfs_uncompress_block(pgdata, > PAGE_SIZE, > - cramfs_read(sb, start_offset, compr_len), > - compr_len); > - mutex_unlock(&read_mutex); > - if (unlikely(bytes_filled < 0)) > - goto err; > + cramfs_read(sb, block_start, block_len), > + block_len); > } > + mutex_unlock(&read_mutex); > + if (unlikely(bytes_filled < 0)) > + goto err; > } > > memset(pgdata + bytes_filled, 0, PAGE_SIZE - bytes_filled); > diff --git a/include/uapi/linux/cramfs_fs.h > b/include/uapi/linux/cramfs_fs.h > index e4611a9b92..ed250aa372 100644 > --- a/include/uapi/linux/cramfs_fs.h > +++ b/include/uapi/linux/cramfs_fs.h > @@ -73,6 +73,7 @@ struct cramfs_super { > #define CRAMFS_FLAG_HOLES 0x00000100 /* support for holes */ > #define CRAMFS_FLAG_WRONG_SIGNATURE 0x00000200 /* reserved */ > #define CRAMFS_FLAG_SHIFTED_ROOT_OFFSET 0x00000400 /* shifted root fs > */ > +#define CRAMFS_FLAG_EXT_BLOCK_POINTERS 0x00000800 /* block pointer > extensions */ > > /* > * Valid values in super.flags. Currently we refuse to mount > @@ -82,7 +83,24 @@ struct cramfs_super { > #define CRAMFS_SUPPORTED_FLAGS ( 0x000000ff \ > | CRAMFS_FLAG_HOLES \ > | CRAMFS_FLAG_WRONG_SIGNATURE \ > - | CRAMFS_FLAG_SHIFTED_ROOT_OFFSET ) > + | CRAMFS_FLAG_SHIFTED_ROOT_OFFSET \ > + | CRAMFS_FLAG_EXT_BLOCK_POINTERS ) > > +/* > + * Block pointer flags > + * > + * The maximum block offset that needs to be represented is roughly: > + * trailing whitespace -Chris