From: Damien Le Moal <Damien.LeMoal@wdc.com>
To: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
"linux-xfs@vger.kernel.org" <linux-xfs@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Johannes Thumshirn <jth@kernel.org>,
Naohiro Aota <Naohiro.Aota@wdc.com>,
Hannes Reinecke <hare@suse.de>
Subject: Re: [PATCH v2 1/2] fs: New zonefs file system
Date: Wed, 25 Dec 2019 05:02:28 +0000 [thread overview]
Message-ID: <BYAPR04MB5816C2F5625DC5B534BF050EE7280@BYAPR04MB5816.namprd04.prod.outlook.com> (raw)
In-Reply-To: 20191224042557.GW7489@magnolia
Darrick,
On 2019/12/24 13:28, Darrick J. Wong wrote:
>> [...]
>>>> +
>>>> +static int zonefs_get_zone_info(struct zonefs_zone_data *zd)
>>>> +{
>>>> + struct block_device *bdev = zd->sb->s_bdev;
>>>> + int ret;
>>>> +
>>>> + zd->zones = kvcalloc(blkdev_nr_zones(bdev->bd_disk),
>>>> + sizeof(struct blk_zone), GFP_KERNEL);
>>>
>>> Hmm, so one 64-byte blk_zone structure for each zone on the disk?
>>>
>>> I have a 14TB SMR disk with ~459,000x 32M zones on it. That's going to
>>> require a contiguous 30MB memory allocation to hold all the zone
>>> information. Even your 15T drive from the commit message will need a
>>> contiguous 3.8MB memory allocation for all the zone info.
>>>
>>> I wonder if each zone should really be allocated separately and then
>>> indexed with an xarray or something like that to reduce the chance of
>>> failure when memory is fragmented or tight.
>>>
>>> That could be subsequent work though, since in the meantime that just
>>> makes zonefs mounts more likely to run out of memory and fail. I
>>> suppose you don't hang on to the huge allocation for very long.
>>
>> No, this memory allocation is only for mount. It is dropped as soon as
>> all the zone file inodes are created. Furthermore, this allocation is a
>> kvalloc, not a kmalloc. So there is no memory continuity requirement.
>> This is only an array of structures and that is not used to do IOs for
>> the report zone itself.
>>
>> I debated trying to optimize (I mean reducing the mount temporary memory
>> use) by processing mount in small chunks of zones instead of all zones
>> in one go. I kept simple, but rather brutal, approach to keep the code
>> simple. This can be rewritten and optimized at any time if we see
>> problems appearing.
>
> <nod> vmalloc space is quite limited on 32-bit platforms, so that's the
> most likely place you'll get complaints.
Yes, agreed. But the main use case for host-managed zoned drives (HDDs
or SSDs) being enterprise servers, 32-bits arch are unlikely to be an
issue. So for now, if there is no strong opposition, I would like to
keep the initialization as it is and revisit later if problems are reported.
--
Damien Le Moal
Western Digital Research
next prev parent reply other threads:[~2019-12-25 5:02 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-20 6:55 [PATCH v2 0/2] New zonefs file system Damien Le Moal
2019-12-20 6:55 ` [PATCH v2 1/2] fs: " Damien Le Moal
2019-12-20 22:36 ` Darrick J. Wong
2019-12-23 1:33 ` Damien Le Moal
2019-12-24 4:25 ` Darrick J. Wong
2019-12-25 5:02 ` Damien Le Moal [this message]
2019-12-20 6:55 ` [PATCH v2 2/2] zonefs: Add documentation Damien Le Moal
2019-12-20 7:03 ` Hannes Reinecke
2019-12-20 22:39 ` Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BYAPR04MB5816C2F5625DC5B534BF050EE7280@BYAPR04MB5816.namprd04.prod.outlook.com \
--to=damien.lemoal@wdc.com \
--cc=Naohiro.Aota@wdc.com \
--cc=darrick.wong@oracle.com \
--cc=hare@suse.de \
--cc=jth@kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).