From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1mKxwv-0000YF-B5 for mharc-grub-devel@gnu.org; Tue, 31 Aug 2021 03:12:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44442) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mKxwr-0000WH-JE for grub-devel@gnu.org; Tue, 31 Aug 2021 03:12:45 -0400 Received: from de-smtp-delivery-102.mimecast.com ([194.104.111.102]:25665) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mKxwo-00022D-Cr for grub-devel@gnu.org; Tue, 31 Aug 2021 03:12:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1630393958; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Eg1khg2FGchwu2tx2Mz5804Ku2BaIB1zQ6AzQzINd7M=; b=k20usQGq8HhaU6YmqMFjI8rWu6T+AunHe+3/iYnRx4pDNdDPsN3jg982uoHWuTs6s4OuOa eAgbFouvSXL2Dvu4EXamWcAs0kks4si9sMmDf4eg9Z3bVOz2mo5DNNGYmfuB8nTje2A3TA ZqLo968u/ROBlPkKEzBn3QP8aAiUGP8= Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52]) (Using TLS) by relay.mimecast.com with ESMTP id de-mta-16-t19KIzBQPpaeSTSlKCO1Eg-1; Tue, 31 Aug 2021 09:12:37 +0200 X-MC-Unique: t19KIzBQPpaeSTSlKCO1Eg-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UIzr4UdNOQFLnemV48m7BJmT/jq/3sueBWD9fxa7zw3Iz4RUOwNTHnD8o2PGGjOOO3yC1oOrHxsDouWM0CU3p1ugMYQxpfirkWuGOWf9ytFGMjrN944z4pW6qSZb/CUsVrWBwFB7V3DOOqV0lhjJaslQHv7awQd18xr0a+xgCdV12loXlYNM0zTV0/pHqDGDMkvp+jNzHL0eXEwGZdAgkxXhyPYQzFEA1cLg+RomQa96b6mw2ZH0NU4d+qRUj92vFVsJHoQOlpjzPlwqi7v1JA+YZ8f+DTDZ/nAoLtnGFUSvBGP/Y+1GRzcSrYVOW8zPDClHNUJMRf4Rks0i/DM+nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=au91c8/j+bmWwThP5kCLdvcNkt8udKAim5mwvffr6wA=; b=nLaPEAknS3RjviHnHUoLc6+p/J39N4K+fnhffVAgeHUuhZrqYELQGugABpPcDwMlhEjR6gxUKhyPf4VAU8KacTNZ0hXhhxqt1Qf01ZJyNTM4XeCTz/wePnGIdcMCowmImGktJd/Bs5SWI8Y5E0DAeYA3E4EHyCp+SbU6l+LkW4UZrioFzTFnhD7UYpNq7AZdRaikwM2oscBXf2mPhCraS+SnL0K+w3tCupQT6zy6pBkUsMKF9Z33uUM+QHEZE4KJsAUSmIV+5391ksJrt1yb8XBKbb1kBDupzoYI2WVtMqT0bN0TEYU/PvWXQ6xvV9wj9wzb931x5hSufNe/d6kq0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: gnu.org; dkim=none (message not signed) header.d=none;gnu.org; dmarc=none action=none header.from=suse.com; Received: from DU2PR04MB8648.eurprd04.prod.outlook.com (2603:10a6:10:2df::21) by DB9PR04MB8493.eurprd04.prod.outlook.com (2603:10a6:10:2c5::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17; Tue, 31 Aug 2021 07:12:35 +0000 Received: from DU2PR04MB8648.eurprd04.prod.outlook.com ([fe80::5586:d75d:b656:b85]) by DU2PR04MB8648.eurprd04.prod.outlook.com ([fe80::5586:d75d:b656:b85%5]) with mapi id 15.20.4457.024; Tue, 31 Aug 2021 07:12:35 +0000 From: Michael Chang To: The development of GNU GRUB Subject: [PATCH] i386-pc: build btrfs zstd support into separate module Date: Tue, 31 Aug 2021 15:12:28 +0800 Message-ID: <20210831071228.8282-1-mchang@suse.com> X-Mailer: git-send-email 2.31.1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain X-ClientProxiedBy: HK2PR04CA0084.apcprd04.prod.outlook.com (2603:1096:202:15::28) To DU2PR04MB8648.eurprd04.prod.outlook.com (2603:10a6:10:2df::21) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost (2001:b011:30d0:3976:6600:6aff:fe77:a7be) by HK2PR04CA0084.apcprd04.prod.outlook.com (2603:1096:202:15::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.19 via Frontend Transport; Tue, 31 Aug 2021 07:12:34 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5ceb5f5b-fe87-4801-0e90-08d96c4eb507 X-MS-TrafficTypeDiagnostic: DB9PR04MB8493: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:549; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MVqTRylrBjsRkP+mBVFYlQwT0PAjNBBExf3nLgfrv1Ha6qIUw+ql/p3N5rUVGNlVK86veFAZdMvZGUxw93Z8YdVZXP93WkSWuNIIO0KwN8v+9cEIhvYurjRjqPiRHIzySRAD195+ntmQfdYfp2kobeC12YPZQor7PKx8XFSnBXHeUbw46MJFyWpewJY5RBXSxdHJq5q5LoZApTapF3EdujpjqMukr7ZAjuz7o37QQ/oOLHhT/8iZMYqzhEaZCEg5lj4uGTT8tWcbxvFhbg/aofmgMEvyCDFJGQp4lvuh7LF9AQgrgt2kVGpWTiGLJMgQJjXloqnPiGYVoINiHjr0Ktsj341js/HozzhkVZi1WtlY+X3YPKT7PG098izH6m8Vn3QWQqvnE3zagD6PeiwHmUfsoqquTTtCFxG5fSF/JrNH9sRf5sfsVD9WDB/M669kxFLZY1Ul5ycv6ysfPkcda/EZIB9vzQ8zsDJMZOc4Kkm3n2X4a+FofUvPflpw/2ADlBg03j3BSH44ofYYrUcpxWWmIM8shYnce3eS1aaWR0PZV+NX8w1+rYivUGUDKHrMl/ZXGfd9f/fwPWS2FZX6brdSQhE/JLob24xGKGcPQeBp2JQ+cSAbGi5mZ3eQbY1rM7vHAfVMnO4PAzzgPQbnMp/ANa9owmFk6j8f4/snPfUUVr9CGuSeSprDVIO2Tmxzi5yR3d5uSa2yKKX2RGaPdogmh1aNsQmOylJxKbi8YEDiEZZfpVmezeisSGEcNRvFW54feyJ0kD+KgNOmdEjHaw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DU2PR04MB8648.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(396003)(376002)(366004)(136003)(346002)(86362001)(2616005)(6496006)(36756003)(5660300002)(66556008)(66476007)(66946007)(316002)(186003)(6666004)(8936002)(30864003)(83380400001)(6486002)(38100700002)(8676002)(478600001)(2906002)(1076003)(6916009)(2004002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?9J57XXeZmozG3zG3NkMEuk+Q8Z9Pakr2DO6kDg0u7FfG43VMnZJpr9d/rs4P?= =?us-ascii?Q?Z7kfP7SKA47YsnNIM+yErHjkT0/Gsf0H29W0LkN+0sNrHtYkTBKSJnJXF/ML?= =?us-ascii?Q?p0UrSAmhbwOgPnpVcNO4Jf922sXvkvzzVFTMrp/OvZBU6qkAf07eBnWY9fuG?= =?us-ascii?Q?nwc3HuTyMYNxDud/QWKlaFMmmLqXlhNrj0vNwjzPxFHbznpPAEkC5T8eWGGe?= =?us-ascii?Q?wWyk/ncKLuH84D+liHVPFh1CXtrwZdfIOZ97Fs9Ojo28S0mGUbTMBiG6yIHP?= =?us-ascii?Q?+6zN0aTqLUurYB1ll+EF/TZXyeH4R5tA5uNWg9JD0apqt/ZCNn1h2esj9Jlr?= =?us-ascii?Q?vWoPxUgXmg1AKsKSeqM3nNrMJtDemMgefBSTps0iTzSJc+MvsJIBNgW0sgpn?= =?us-ascii?Q?Sqogzf+2AicOQCtPuRU9V5jkddveg6tSIo5jOClmtzk7rDnFROuKE3hJHH9O?= =?us-ascii?Q?cm71ZgGwx9QwzGLyxHwNgW1NufilbDiO1eWElDbl+Hn18rXTIMCsCH/0rYxh?= =?us-ascii?Q?iEOmBEsMIz6+euCi+dmwPvlI1yJrz3eo/KLjvbbqzdVToBoy+TaNVUwmiR+f?= =?us-ascii?Q?08zoQOM4E6y50gdserAG0K8IpQVtsVhnqc46gY9BMIg0L3oVafrkCwnh5xMU?= =?us-ascii?Q?4Fqbi5RMymDSPQuz1UXJx1GLBO+8RCVBf5QGzW3I6lz5l0B+elQlmeijK8oH?= =?us-ascii?Q?ryk/uvs1171fB+1CQ04nBuWq1hcZhQt87qfUTeEirZRphB8bPxGupHBKWBy4?= =?us-ascii?Q?UnyhJsQoANOZX4JT+KKWVzo5ywbmIPiDW9BlJyCxhKSsj7WAxLoRKf82Ls0O?= =?us-ascii?Q?FmkEsvn6NrKJSrn/nbGH6lpkN/gx1iZ3LmQTuTFt0ArBzTdw5TrjRdlZJeaI?= =?us-ascii?Q?MDQY2SCRQLyNxmxIauGxYETBiHCrtoXmysgjKcJevmf6LM0+3IJSjsIqZeNY?= =?us-ascii?Q?gxdxlvG9Ws51Lw6uKGUQLSvep+Yo4Y4CkOuvnXeMK5eUAIrS7MFqX/V+KxeI?= =?us-ascii?Q?hYwho7rWJ6ZgXLmBWVPyivRHyFtTGPKEMp9wswNQGFMHmqxsetF1tztEdu5+?= =?us-ascii?Q?5D4cAacv0x3oTEuXKnyWqjnqFM4M7Df+ZVpyBRH3VfrB2En1gELqU6YZj9RJ?= =?us-ascii?Q?WEteume1LebM3FThB8dlq4VSyfYV4XTx9pyC1WW/m+ZxLZsS5OqyzP94rw/w?= =?us-ascii?Q?cZnrLhwOpcHeeWvhMDaFWKW2ihBidTbizrqEDvbyFzUh9Po4/OQTnaIC6bk6?= =?us-ascii?Q?NXETWwi/T+vGosBXOvwGFOt2r0/t0Y5csU5A0MX7b+MaX65LzkBEXlQ6KFy2?= =?us-ascii?Q?cKP35jZVQVjOoxNsdDXCU8j4cMjY3oEjfzPCYcNDp7PjpvKYKRaXUU21Ob77?= =?us-ascii?Q?Zyv2ECF2NNLWqY2R26fGRkPukpqu?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5ceb5f5b-fe87-4801-0e90-08d96c4eb507 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8648.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Aug 2021 07:12:35.1745 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lNu0BVQjdyX61vcs7itiRyX2o1+wybmtCFuhp3hD+td1VhhEJ2DXne2k8R5XmWpH X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR04MB8493 Received-SPF: pass client-ip=194.104.111.102; envelope-from=mchang@suse.com; helo=de-smtp-delivery-102.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Aug 2021 07:12:45 -0000 The zstd support in btrfs brings significant size increment to the on-disk image that it can no longer fit into btrfs bootloader area and short mbr gap. In order to support grub update on outstanding i386-pc setup with these size constraints remain in place, here we build the zstd suppprt of btrfs into a separate module, named btrfs_zstd, to alleviate the size change. Please note this only makes it's way to i386-pc, other architecture is not affected. Therefore if the system has enough space of embedding area for grub then zstd support for btrfs will be enabled automatically in the process of running grub-install through inserting btrfs_zstd module to the on-disk image, otherwise a warning will be logged on screen to indicate user that zstd support for btrfs is disabled due to the size limit. Signed-off-by: Michael Chang --- Makefile.util.def | 1 + grub-core/Makefile.core.def | 11 ++++ grub-core/fs/btrfs.c | 114 +++++--------------------------- grub-core/fs/btrfs_zstd.c | 36 +++++++++++ grub-core/lib/zstd.c | 126 ++++++++++++++++++++++++++++++++++++ include/grub/btrfs.h | 6 ++ include/grub/lib/zstd.h | 26 ++++++++ include/grub/util/install.h | 2 + util/grub-install.c | 20 ++++++ util/setup.c | 43 ++++++++++++ 10 files changed, 288 insertions(+), 97 deletions(-) create mode 100644 grub-core/fs/btrfs_zstd.c create mode 100644 grub-core/lib/zstd.c create mode 100644 include/grub/lib/zstd.h diff --git a/Makefile.util.def b/Makefile.util.def index 2f2881cb7..ac2b6aab1 100644 --- a/Makefile.util.def +++ b/Makefile.util.def @@ -176,6 +176,7 @@ library =3D { common =3D grub-core/lib/zstd/xxhash.c; common =3D grub-core/lib/zstd/zstd_common.c; common =3D grub-core/lib/zstd/zstd_decompress.c; + common =3D grub-core/lib/zstd.c; }; =20 program =3D { diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def index dd1777bd3..b5328d7a0 100644 --- a/grub-core/Makefile.core.def +++ b/grub-core/Makefile.core.def @@ -1370,10 +1370,21 @@ module =3D { name =3D btrfs; common =3D fs/btrfs.c; common =3D lib/crc.c; + nopc =3D lib/zstd.c; cflags =3D '$(CFLAGS_POSIX) -Wno-undef'; + i386_pc_cppflags =3D '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/minilzo= -DMINILZO_HAVE_CONFIG_H'; cppflags =3D '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/minilzo -I$(src= dir)/lib/zstd -DMINILZO_HAVE_CONFIG_H'; }; =20 +module =3D { + name =3D btrfs_zstd; + common =3D fs/btrfs_zstd.c; + common =3D lib/zstd.c; + cflags =3D '$(CFLAGS_POSIX) -Wno-undef'; + cppflags =3D '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/zstd'; + enable =3D i386_pc; +}; + module =3D { name =3D archelp; common =3D fs/archelp.c; diff --git a/grub-core/fs/btrfs.c b/grub-core/fs/btrfs.c index c908b460f..9625bdf16 100644 --- a/grub-core/fs/btrfs.c +++ b/grub-core/fs/btrfs.c @@ -17,14 +17,6 @@ * along with GRUB. If not, see . */ =20 -/* - * Tell zstd to expose functions that aren't part of the stable API, which - * aren't safe to use when linking against a dynamic library. We vendor in= a - * specific zstd version, so we know what we're getting. We need these uns= table - * functions to provide our own allocator, which uses grub_malloc(), to zs= td. - */ -#define ZSTD_STATIC_LINKING_ONLY - #include #include #include @@ -35,7 +27,9 @@ #include #include #include -#include +#ifndef GRUB_MACHINE_PCBIOS +#include +#endif #include #include #include @@ -61,12 +55,15 @@ GRUB_MOD_LICENSE ("GPLv3+"); #define GRUB_BTRFS_LZO_BLOCK_MAX_CSIZE (GRUB_BTRFS_LZO_BLOCK_SIZE + \ (GRUB_BTRFS_LZO_BLOCK_SIZE / 16) + 64 + 3) =20 -#define ZSTD_BTRFS_MAX_WINDOWLOG 17 -#define ZSTD_BTRFS_MAX_INPUT (1 << ZSTD_BTRFS_MAX_WINDOWLOG) - typedef grub_uint8_t grub_btrfs_checksum_t[0x20]; typedef grub_uint16_t grub_btrfs_uuid_t[8]; =20 +#ifdef GRUB_MACHINE_PCBIOS +grub_ssize_t (*grub_btrfs_zstd_decompress_func) (char *ibuf, + grub_size_t isize, grub_off_t off, + char *obuf, grub_size_t osize) =3D NULL; +#endif + struct grub_btrfs_device { grub_uint64_t device_id; @@ -1392,94 +1389,17 @@ grub_btrfs_read_inode (struct grub_btrfs_data *data= , return grub_btrfs_read_logical (data, elemaddr, inode, sizeof (*inode), = 0); } =20 -static void *grub_zstd_malloc (void *state __attribute__((unused)), size_t= size) -{ - return grub_malloc (size); -} - -static void grub_zstd_free (void *state __attribute__((unused)), void *add= ress) -{ - return grub_free (address); -} - -static ZSTD_customMem grub_zstd_allocator (void) -{ - ZSTD_customMem allocator; - - allocator.customAlloc =3D &grub_zstd_malloc; - allocator.customFree =3D &grub_zstd_free; - allocator.opaque =3D NULL; - - return allocator; -} - -static grub_ssize_t +static inline grub_ssize_t grub_btrfs_zstd_decompress (char *ibuf, grub_size_t isize, grub_off_t off, char *obuf, grub_size_t osize) { - void *allocated =3D NULL; - char *otmpbuf =3D obuf; - grub_size_t otmpsize =3D osize; - ZSTD_DCtx *dctx =3D NULL; - grub_size_t zstd_ret; - grub_ssize_t ret =3D -1; - - /* - * Zstd will fail if it can't fit the entire output in the destination - * buffer, so if osize isn't large enough, allocate a temporary buffer. - */ - if (otmpsize < ZSTD_BTRFS_MAX_INPUT) - { - allocated =3D grub_malloc (ZSTD_BTRFS_MAX_INPUT); - if (!allocated) - { - grub_error (GRUB_ERR_OUT_OF_MEMORY, "failed allocate a zstd buffer"); - goto err; - } - otmpbuf =3D (char *) allocated; - otmpsize =3D ZSTD_BTRFS_MAX_INPUT; - } - - /* Create the ZSTD_DCtx. */ - dctx =3D ZSTD_createDCtx_advanced (grub_zstd_allocator ()); - if (!dctx) - { - /* ZSTD_createDCtx_advanced() only fails if it is out of memory. */ - grub_error (GRUB_ERR_OUT_OF_MEMORY, "failed to create a zstd context= "); - goto err; - } - - /* - * Get the real input size, there may be junk at the - * end of the frame. - */ - isize =3D ZSTD_findFrameCompressedSize (ibuf, isize); - if (ZSTD_isError (isize)) - { - grub_error (GRUB_ERR_BAD_COMPRESSED_DATA, "zstd data corrupted"); - goto err; - } - - /* Decompress and check for errors. */ - zstd_ret =3D ZSTD_decompressDCtx (dctx, otmpbuf, otmpsize, ibuf, isize); - if (ZSTD_isError (zstd_ret)) - { - grub_error (GRUB_ERR_BAD_COMPRESSED_DATA, "zstd data corrupted"); - goto err; - } - - /* - * Move the requested data into the obuf. obuf may be equal - * to otmpbuf, which is why grub_memmove() is required. - */ - grub_memmove (obuf, otmpbuf + off, osize); - ret =3D osize; - -err: - grub_free (allocated); - ZSTD_freeDCtx (dctx); - - return ret; +#ifdef GRUB_MACHINE_PCBIOS + if (!grub_btrfs_zstd_decompress_func) + return -1; + return grub_btrfs_zstd_decompress_func (ibuf, isize, off, obuf, osize); +#else + return grub_zstd_decompress (ibuf, isize, off, obuf, osize); +#endif } =20 static grub_ssize_t diff --git a/grub-core/fs/btrfs_zstd.c b/grub-core/fs/btrfs_zstd.c new file mode 100644 index 000000000..d5d1e013c --- /dev/null +++ b/grub-core/fs/btrfs_zstd.c @@ -0,0 +1,36 @@ +/* + * GRUB -- GRand Unified Bootloader + * Copyright (C) 2008 Free Software Foundation, Inc. + * + * GRUB is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * GRUB is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GRUB. If not, see . + */ + +#include +#include +/* For NULL. */ +#include +#include +#include + +GRUB_MOD_LICENSE ("GPLv3+"); + +GRUB_MOD_INIT (btrfs_zstd) +{ + grub_btrfs_zstd_decompress_func =3D grub_zstd_decompress; +} + +GRUB_MOD_FINI (btrfs_zstd) +{ + grub_btrfs_zstd_decompress_func =3D NULL; +} diff --git a/grub-core/lib/zstd.c b/grub-core/lib/zstd.c new file mode 100644 index 000000000..643e90d84 --- /dev/null +++ b/grub-core/lib/zstd.c @@ -0,0 +1,126 @@ +/* + * GRUB -- GRand Unified Bootloader + * Copyright (C) 2008 Free Software Foundation, Inc. + * + * GRUB is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * GRUB is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GRUB. If not, see . + */ + +/* + * Tell zstd to expose functions that aren't part of the stable API, which + * aren't safe to use when linking against a dynamic library. We vendor in= a + * specific zstd version, so we know what we're getting. We need these uns= table + * functions to provide our own allocator, which uses grub_malloc(), to zs= td. + */ +#define ZSTD_STATIC_LINKING_ONLY + +#include +#include +#include +#include +#include +#include + +#define ZSTD_MAX_WINDOWLOG 17 +#define ZSTD_MAX_INPUT (1 << ZSTD_MAX_WINDOWLOG) + +static void *grub_zstd_malloc (void *state __attribute__((unused)), size_t= size) +{ + return grub_malloc (size); +} + +static void grub_zstd_free (void *state __attribute__((unused)), void *add= ress) +{ + return grub_free (address); +} + +static ZSTD_customMem grub_zstd_allocator (void) +{ + ZSTD_customMem allocator; + + allocator.customAlloc =3D &grub_zstd_malloc; + allocator.customFree =3D &grub_zstd_free; + allocator.opaque =3D NULL; + + return allocator; +} + +grub_ssize_t +grub_zstd_decompress (char *ibuf, grub_size_t isize, grub_off_t off, + char *obuf, grub_size_t osize) +{ + void *allocated =3D NULL; + char *otmpbuf =3D obuf; + grub_size_t otmpsize =3D osize; + ZSTD_DCtx *dctx =3D NULL; + grub_size_t zstd_ret; + grub_ssize_t ret =3D -1; + + /* + * Zstd will fail if it can't fit the entire output in the destination + * buffer, so if osize isn't large enough, allocate a temporary buffer. + */ + if (otmpsize < ZSTD_MAX_INPUT) + { + allocated =3D grub_malloc (ZSTD_MAX_INPUT); + if (!allocated) + { + grub_error (GRUB_ERR_OUT_OF_MEMORY, "failed allocate a zstd buffer"); + goto err; + } + otmpbuf =3D (char *) allocated; + otmpsize =3D ZSTD_MAX_INPUT; + } + + /* Create the ZSTD_DCtx. */ + dctx =3D ZSTD_createDCtx_advanced (grub_zstd_allocator ()); + if (!dctx) + { + /* ZSTD_createDCtx_advanced() only fails if it is out of memory. */ + grub_error (GRUB_ERR_OUT_OF_MEMORY, "failed to create a zstd context= "); + goto err; + } + + /* + * Get the real input size, there may be junk at the + * end of the frame. + */ + isize =3D ZSTD_findFrameCompressedSize (ibuf, isize); + if (ZSTD_isError (isize)) + { + grub_error (GRUB_ERR_BAD_COMPRESSED_DATA, "zstd data corrupted"); + goto err; + } + + /* Decompress and check for errors. */ + zstd_ret =3D ZSTD_decompressDCtx (dctx, otmpbuf, otmpsize, ibuf, isize); + if (ZSTD_isError (zstd_ret)) + { + grub_error (GRUB_ERR_BAD_COMPRESSED_DATA, "zstd data corrupted"); + goto err; + } + + /* + * Move the requested data into the obuf. obuf may be equal + * to otmpbuf, which is why grub_memmove() is required. + */ + grub_memmove (obuf, otmpbuf + off, osize); + ret =3D osize; + +err: + grub_free (allocated); + ZSTD_freeDCtx (dctx); + + return ret; +} + diff --git a/include/grub/btrfs.h b/include/grub/btrfs.h index 234ad9767..f8e551982 100644 --- a/include/grub/btrfs.h +++ b/include/grub/btrfs.h @@ -69,4 +69,10 @@ struct grub_btrfs_inode_ref char name[0]; }; =20 +#ifdef GRUB_MACHINE_PCBIOS +extern grub_ssize_t (*EXPORT_VAR (grub_btrfs_zstd_decompress_func)) (char = *ibuf, + grub_size_t isize, grub_off_t off, + char *obuf, grub_size_t osize); +#endif + #endif diff --git a/include/grub/lib/zstd.h b/include/grub/lib/zstd.h new file mode 100644 index 000000000..0867b0c34 --- /dev/null +++ b/include/grub/lib/zstd.h @@ -0,0 +1,26 @@ +/* + * GRUB -- GRand Unified Bootloader + * Copyright (C) 2008 Free Software Foundation, Inc. + * + * GRUB is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * GRUB is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with GRUB. If not, see . + */ + +#ifndef GRUB_ZSTD_HEADER +#define GRUB_ZSTD_HEADER 1 + +grub_ssize_t +grub_zstd_decompress (char *ibuf, grub_size_t isize, grub_off_t off, + char *obuf, grub_size_t osize); + +#endif /* ! GRUB_ZSTD_HEADER */ diff --git a/include/grub/util/install.h b/include/grub/util/install.h index faf9ff1ab..9e83e1339 100644 --- a/include/grub/util/install.h +++ b/include/grub/util/install.h @@ -208,6 +208,8 @@ grub_util_sparc_setup (const char *dir, const char *dest, int force, int fs_probe, int allow_floppy, int add_rs_codes, int warn_short_mbr_gap); +int +grub_util_try_partmap_embed (const char *dest, unsigned int *nsec); =20 char * grub_install_get_image_targets_string (void); diff --git a/util/grub-install.c b/util/grub-install.c index 891e3ced8..a2286b3dd 100644 --- a/util/grub-install.c +++ b/util/grub-install.c @@ -1482,6 +1482,26 @@ main (int argc, char *argv[]) } } =20 + if (install_drive + && platform =3D=3D GRUB_INSTALL_PLATFORM_I386_PC + && grub_strcmp (grub_fs->name, "btrfs") =3D=3D 0) + { + char *mod; + mod =3D grub_util_path_concat (2, grub_install_source_directory, "bt= rfs_zstd.mod"); + if (grub_util_is_regular (mod)) + { + unsigned int nsec =3D GRUB_MIN_RECOMMENDED_MBR_GAP; + int ret =3D grub_util_try_partmap_embed (install_drive, &nsec); + if (ret =3D=3D 0) + grub_install_push_module ("btrfs_zstd"); + else if (ret =3D=3D 1) + grub_util_warn ("%s", _("btrfs zstd compression is disabled, please c= hange install device to disk")); + else + grub_util_warn ("%s", _("btrfs zstd compression is disabled due to no= t enough space to embed image")); + } + grub_free (mod); + } + if (!have_abstractions) { if ((disk_module && grub_strcmp (disk_module, "biosdisk") !=3D 0) diff --git a/util/setup.c b/util/setup.c index 980e74374..6604d86bb 100644 --- a/util/setup.c +++ b/util/setup.c @@ -241,6 +241,49 @@ identify_partmap (grub_disk_t disk __attribute__ ((unu= sed)), return 1; } =20 +#ifdef GRUB_SETUP_BIOS +int +grub_util_try_partmap_embed (const char *dest, unsigned int *nsec) +{ + grub_device_t dest_dev; + + dest_dev =3D grub_device_open (dest); + if (! dest_dev) + grub_util_error ("%s", grub_errmsg); + + struct identify_partmap_ctx ctx =3D { + .dest_partmap =3D NULL, + .container =3D dest_dev->disk->partition, + .multiple_partmaps =3D 0 + }; + + grub_partition_iterate (dest_dev->disk, identify_partmap, &ctx); + + if (ctx.dest_partmap && ctx.dest_partmap->embed) + { + grub_err_t err; + + grub_disk_addr_t *sectors =3D NULL; + + err =3D ctx.dest_partmap->embed (dest_dev->disk, nsec, *nsec, + GRUB_EMBED_PCBIOS, §ors, 0); + if (err) + { + grub_errno =3D GRUB_ERR_NONE; + goto no_embed; + } + grub_free (sectors); + return 0; + } + +no_embed: + grub_device_close (dest_dev); + if (ctx.container) + return 1; + return 2; +} +#endif + #ifdef GRUB_SETUP_BIOS #define SETUP grub_util_bios_setup #elif GRUB_SETUP_SPARC64 --=20 2.31.1