[PATCH 0/2] lib/kstrtox: introduce kstrtoull_suffix() helper

[PATCH 0/2] lib/kstrtox: introduce kstrtoull_suffix() helper

[Qu Wenruo]

Recently David Sterba <dsterba@suse.cz> exposed some weird behavior of
btrfs sysfs interface, which is utilize memparse().

One example is using "echo 25e >
/sys/fs/btrfs/<uuid>/devinfo/scrub_speed_max".

The result value is not 0x25e, because there is no "0x" prefix provided.
Nor (25 << 60), as that would overflow.

All these are the caveats of memparse(), which lacks:

- Overflow check
- Reasonable suffix selection
  I know there may be some niche cases for memory layout, but for most
  callers, they just want a kstrtoull() with reasonable suffix
  selection.
  And I don't think exabytes is a reasonable suffix.

So here we introduce a new helper, "kstrtoull_suffix", which has some
the following two abilities added:

- Allow caller to select the suffix they want to enable
  The default list is "KkMmGgTtPp", no unreasonable "Ee" ones.

- Allow suffix parsing with overflow detection.
  The int part detection is already done by _kstrtoull(), and with the
  extra left shift, do the check again before returning the value.

Unfortunately this new helper is still not a drop-in replacement for
memparse():

- No @retptr support
  It's possible to add, but I'm not sure how many call sites need that
  for extra separators.

- Need to check the failure properly
  Which memparse() callers never seems to check anyway.

Thus the existing memparse() callers need to opt-in.
For now, btrfs usage of memparse() can be easily converted to
kstrtoull_suffix(), in the 2nd patch.

Qu Wenruo (2):
  lib/strtox: introduce kstrtoull_suffix() helper
  btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

 fs/btrfs/sysfs.c        |  20 +++----
 include/linux/kstrtox.h |   7 +++
 lib/kstrtox.c           | 113 ++++++++++++++++++++++++++++++++++++++--
 3 files changed, 123 insertions(+), 17 deletions(-)

-- 
2.43.0

[PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

[Qu Wenruo]

Just as mentioned in the comment of memparse(), the simple_stroull()
usage can lead to overflow all by itself.

Furthermore, the suffix calculation is also super overflow prone because
that some suffix like "E" itself would eat 60bits, leaving only 4 bits
available.

And that suffix "E" can also lead to confusion since it's using the same
char of hex Ox'E'.

One simple example to expose all the problem is to use memparse() on
"25E".
The correct value should be 28823037615171174400, but the suffix E makes
it super simple to overflow, resulting the incorrect value
10376293541461622784 (9E).

So here we introduce a new helper to address the problem,
kstrtoull_suffix():

- Enhance _kstrtoull()
  This allow _kstrtoull() to return even if it hits an invalid char, as
  long as the optional parameter @retptr is provided.

  If @retptr is provided, _kstrtoull() would try its best to parse the
  valid part, and leave the remaining to be handled by the caller.

  If @retptr is not provided, the behavior is not altered.

- New kstrtoull_suffix() helper
  This new helper utilize the new @retptr capability of _kstrtoull(),
  and provides 2 new ability:

  * Allow certain suffixes to be chosen
    The recommended suffix list is "KkMmGgTtPp", excluding the overflow
    prone "Ee". Undermost cases there is really no need to use "E" suffix
    anyway.
    And for those who really need that exabytes suffix, they can enable
    that suffix pretty easily.

  * Add overflow checks for the suffixes
    If the original number string is fine, but with the extra left
    shift overflow happens, then -EOVERFLOW is returned.

Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 include/linux/kstrtox.h |   7 +++
 lib/kstrtox.c           | 113 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 115 insertions(+), 5 deletions(-)

diff --git a/include/linux/kstrtox.h b/include/linux/kstrtox.h
index 7fcf29a4e0de..12c754152c15 100644
--- a/include/linux/kstrtox.h
+++ b/include/linux/kstrtox.h
@@ -9,6 +9,13 @@
 int __must_check _kstrtoul(const char *s, unsigned int base, unsigned long *res);
 int __must_check _kstrtol(const char *s, unsigned int base, long *res);
 
+/*
+ * The default suffix list would not include "E" since it's too easy to overflow
+ * and not much real world usage.
+ */
+#define KSTRTOULL_SUFFIX_DEFAULT		("KkMmGgTtPp")
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     const char *suffixes);
 int __must_check kstrtoull(const char *s, unsigned int base, unsigned long long *res);
 int __must_check kstrtoll(const char *s, unsigned int base, long long *res);
 
diff --git a/lib/kstrtox.c b/lib/kstrtox.c
index d586e6af5e5a..63831207dfdd 100644
--- a/lib/kstrtox.c
+++ b/lib/kstrtox.c
@@ -93,7 +93,8 @@ unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long
 	return _parse_integer_limit(s, base, p, INT_MAX);
 }
 
-static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
+static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res,
+		      char **retptr)
 {
 	unsigned long long _res;
 	unsigned int rv;
@@ -105,11 +106,19 @@ static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
 	if (rv == 0)
 		return -EINVAL;
 	s += rv;
-	if (*s == '\n')
+
+	/*
+	 * If @retptr is provided, caller is responsible to detect
+	 * the extra chars, otherwise we can skip one newline.
+	 */
+	if (!retptr && *s == '\n')
 		s++;
-	if (*s)
+	if (!retptr && *s)
 		return -EINVAL;
+
 	*res = _res;
+	if (retptr)
+		*retptr = (char *)s;
 	return 0;
 }
 
@@ -133,10 +142,104 @@ int kstrtoull(const char *s, unsigned int base, unsigned long long *res)
 {
 	if (s[0] == '+')
 		s++;
-	return _kstrtoull(s, base, res);
+	return _kstrtoull(s, base, res, NULL);
 }
 EXPORT_SYMBOL(kstrtoull);
 
+/**
+ * kstrtoull_suffix - convert a string to ull with suffixes support
+ * @s: The start of the string. The string must be null-terminated, and may also
+ *  include a single newline before its terminating null.
+ * @base: The number base to use. The maximum supported base is 16. If base is
+ *  given as 0, then the base of the string is automatically detected with the
+ *  conventional semantics - If it begins with 0x the number will be parsed as a
+ *  hexadecimal (case insensitive), if it otherwise begins with 0, it will be
+ *  parsed as an octal number. Otherwise it will be parsed as a decimal.
+ * @res: Where to write the result of the conversion on success.
+ * @suffixes: A string of acceptable suffixes, must be provided. Or caller
+ *  should use kstrtoull() directly.
+ *
+ *
+ * Return 0 on success.
+ *
+ * Return -ERANGE on overflow or -EINVAL if invalid chars found.
+ * Return value must be checked.
+ */
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     const char *suffixes)
+{
+	unsigned long long init_value;
+	unsigned long long final_value;
+	char *endptr;
+	int ret;
+
+	ret = _kstrtoull(s, base, &init_value, &endptr);
+	/* Either already overflow or no number string at all. */
+	if (ret < 0)
+		return ret;
+	final_value = init_value;
+	/* No suffixes. */
+	if (!*endptr)
+		goto done;
+
+	switch (*endptr) {
+	case 'K':
+	case 'k':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 10;
+		endptr++;
+		break;
+	case 'M':
+	case 'm':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 20;
+		endptr++;
+		break;
+	case 'G':
+	case 'g':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 30;
+		endptr++;
+		break;
+	case 'T':
+	case 't':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 40;
+		endptr++;
+		break;
+	case 'P':
+	case 'p':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 50;
+		endptr++;
+		break;
+	case 'E':
+	case 'e':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 60;
+		endptr++;
+		break;
+	}
+	if (*endptr == '\n')
+		endptr++;
+	if (*endptr)
+		return -EINVAL;
+
+	/* Overflow check. */
+	if (final_value < init_value)
+		return -EOVERFLOW;
+done:
+	*res = final_value;
+	return 0;
+}
+EXPORT_SYMBOL(kstrtoull_suffix);
+
 /**
  * kstrtoll - convert a string to a long long
  * @s: The start of the string. The string must be null-terminated, and may also
@@ -159,7 +262,7 @@ int kstrtoll(const char *s, unsigned int base, long long *res)
 	int rv;
 
 	if (s[0] == '-') {
-		rv = _kstrtoull(s + 1, base, &tmp);
+		rv = _kstrtoull(s + 1, base, &tmp, NULL);
 		if (rv < 0)
 			return rv;
 		if ((long long)-tmp > 0)
-- 
2.43.0

[PATCH 2/2] btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

[Qu Wenruo]

Since memparse() itself can not handle overflow at all, use
memparse_ull() to be extra safe.

Now overflow values can be properly detected.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 fs/btrfs/sysfs.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
index 84c05246ffd8..089c3fc123fe 100644
--- a/fs/btrfs/sysfs.c
+++ b/fs/btrfs/sysfs.c
@@ -760,7 +760,7 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
 {
 	struct btrfs_space_info *space_info = to_space_info(kobj);
 	struct btrfs_fs_info *fs_info = to_fs_info(get_btrfs_kobj(kobj));
-	char *retptr;
+	int ret;
 	u64 val;
 
 	if (!capable(CAP_SYS_ADMIN))
@@ -776,11 +776,9 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
 	if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM)
 		return -EPERM;
 
-	val = memparse(buf, &retptr);
-	/* There could be trailing '\n', also catch any typos after the value */
-	retptr = skip_spaces(retptr);
-	if (*retptr != 0 || val == 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &val, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 
 	val = min(val, BTRFS_MAX_DATA_CHUNK_SIZE);
 
@@ -1779,14 +1777,12 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
 {
 	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
 						   devid_kobj);
-	char *endptr;
 	unsigned long long limit;
+	int ret;
 
-	limit = memparse(buf, &endptr);
-	/* There could be trailing '\n', also catch any typos after the value. */
-	endptr = skip_spaces(endptr);
-	if (*endptr != 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &limit, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 	WRITE_ONCE(device->scrub_speed_max, limit);
 	return len;
 }
-- 
2.43.0

Re: [PATCH 2/2] btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

[David Disseldorp]

Hi Qu,

On Fri, 15 Dec 2023 19:09:24 +1030, Qu Wenruo wrote:

> Since memparse() itself can not handle overflow at all, use
> memparse_ull() to be extra safe.

s/memparse_ull/kstrtoull_suffix/

> Now overflow values can be properly detected.

Please document how the sysfs API changes with this, in addition to
overflow handling:
- support for 'E' / 'e' suffixes dropped
- only one trailing '\n' accepted, instead of many isspace()

The latter might break a few scripts.

Cheers, David

> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>  fs/btrfs/sysfs.c | 20 ++++++++------------
>  1 file changed, 8 insertions(+), 12 deletions(-)
> 
> diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
> index 84c05246ffd8..089c3fc123fe 100644
> --- a/fs/btrfs/sysfs.c
> +++ b/fs/btrfs/sysfs.c
> @@ -760,7 +760,7 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
>  {
>  	struct btrfs_space_info *space_info = to_space_info(kobj);
>  	struct btrfs_fs_info *fs_info = to_fs_info(get_btrfs_kobj(kobj));
> -	char *retptr;
> +	int ret;
>  	u64 val;
>  
>  	if (!capable(CAP_SYS_ADMIN))
> @@ -776,11 +776,9 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
>  	if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM)
>  		return -EPERM;
>  
> -	val = memparse(buf, &retptr);
> -	/* There could be trailing '\n', also catch any typos after the value */
> -	retptr = skip_spaces(retptr);
> -	if (*retptr != 0 || val == 0)
> -		return -EINVAL;
> +	ret = kstrtoull_suffix(buf, 0, &val, KSTRTOULL_SUFFIX_DEFAULT);
> +	if (ret < 0)
> +		return ret;
>  
>  	val = min(val, BTRFS_MAX_DATA_CHUNK_SIZE);
>  
> @@ -1779,14 +1777,12 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
>  {
>  	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
>  						   devid_kobj);
> -	char *endptr;
>  	unsigned long long limit;
> +	int ret;
>  
> -	limit = memparse(buf, &endptr);
> -	/* There could be trailing '\n', also catch any typos after the value. */
> -	endptr = skip_spaces(endptr);
> -	if (*endptr != 0)
> -		return -EINVAL;
> +	ret = kstrtoull_suffix(buf, 0, &limit, KSTRTOULL_SUFFIX_DEFAULT);
> +	if (ret < 0)
> +		return ret;
>  	WRITE_ONCE(device->scrub_speed_max, limit);
>  	return len;
>  }

Re: [PATCH 2/2] btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

[Qu Wenruo]

[-- Attachment #1.1.1: Type: text/plain, Size: 2757 bytes --]



On 2023/12/18 18:19, David Disseldorp wrote:
> Hi Qu,
> 
> On Fri, 15 Dec 2023 19:09:24 +1030, Qu Wenruo wrote:
> 
>> Since memparse() itself can not handle overflow at all, use
>> memparse_ull() to be extra safe.
> 
> s/memparse_ull/kstrtoull_suffix/
> 
>> Now overflow values can be properly detected.
> 
> Please document how the sysfs API changes with this, in addition to
> overflow handling:
> - support for 'E' / 'e' suffixes dropped
> - only one trailing '\n' accepted, instead of many isspace()

Well, multiple spaces are already an abuse, and I don't believe sane 
scripts should go multiple spaces/newlines.

As all the other call sites are going kstrtox, which only accept one 
newline.

Although the change is indeed worthy a document update.

Thanks,
Qu
> 
> The latter might break a few scripts.
> 
> Cheers, David
> 
>>
>> Signed-off-by: Qu Wenruo <wqu@suse.com>
>> ---
>>   fs/btrfs/sysfs.c | 20 ++++++++------------
>>   1 file changed, 8 insertions(+), 12 deletions(-)
>>
>> diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c
>> index 84c05246ffd8..089c3fc123fe 100644
>> --- a/fs/btrfs/sysfs.c
>> +++ b/fs/btrfs/sysfs.c
>> @@ -760,7 +760,7 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
>>   {
>>   	struct btrfs_space_info *space_info = to_space_info(kobj);
>>   	struct btrfs_fs_info *fs_info = to_fs_info(get_btrfs_kobj(kobj));
>> -	char *retptr;
>> +	int ret;
>>   	u64 val;
>>   
>>   	if (!capable(CAP_SYS_ADMIN))
>> @@ -776,11 +776,9 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
>>   	if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM)
>>   		return -EPERM;
>>   
>> -	val = memparse(buf, &retptr);
>> -	/* There could be trailing '\n', also catch any typos after the value */
>> -	retptr = skip_spaces(retptr);
>> -	if (*retptr != 0 || val == 0)
>> -		return -EINVAL;
>> +	ret = kstrtoull_suffix(buf, 0, &val, KSTRTOULL_SUFFIX_DEFAULT);
>> +	if (ret < 0)
>> +		return ret;
>>   
>>   	val = min(val, BTRFS_MAX_DATA_CHUNK_SIZE);
>>   
>> @@ -1779,14 +1777,12 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
>>   {
>>   	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
>>   						   devid_kobj);
>> -	char *endptr;
>>   	unsigned long long limit;
>> +	int ret;
>>   
>> -	limit = memparse(buf, &endptr);
>> -	/* There could be trailing '\n', also catch any typos after the value. */
>> -	endptr = skip_spaces(endptr);
>> -	if (*endptr != 0)
>> -		return -EINVAL;
>> +	ret = kstrtoull_suffix(buf, 0, &limit, KSTRTOULL_SUFFIX_DEFAULT);
>> +	if (ret < 0)
>> +		return ret;
>>   	WRITE_ONCE(device->scrub_speed_max, limit);
>>   	return len;
>>   }
> 
> 
> 
> 

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7027 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

[David Disseldorp]

Hi Qu,

On Fri, 15 Dec 2023 19:09:23 +1030, Qu Wenruo wrote:

> Just as mentioned in the comment of memparse(), the simple_stroull()
> usage can lead to overflow all by itself.
> 
> Furthermore, the suffix calculation is also super overflow prone because
> that some suffix like "E" itself would eat 60bits, leaving only 4 bits
> available.
> 
> And that suffix "E" can also lead to confusion since it's using the same
> char of hex Ox'E'.
> 
> One simple example to expose all the problem is to use memparse() on
> "25E".
> The correct value should be 28823037615171174400, but the suffix E makes
> it super simple to overflow, resulting the incorrect value
> 10376293541461622784 (9E).
> 
> So here we introduce a new helper to address the problem,
> kstrtoull_suffix():
> 
> - Enhance _kstrtoull()
>   This allow _kstrtoull() to return even if it hits an invalid char, as
>   long as the optional parameter @retptr is provided.
> 
>   If @retptr is provided, _kstrtoull() would try its best to parse the
>   valid part, and leave the remaining to be handled by the caller.
> 
>   If @retptr is not provided, the behavior is not altered.
> 
> - New kstrtoull_suffix() helper
>   This new helper utilize the new @retptr capability of _kstrtoull(),
>   and provides 2 new ability:
> 
>   * Allow certain suffixes to be chosen
>     The recommended suffix list is "KkMmGgTtPp", excluding the overflow
>     prone "Ee". Undermost cases there is really no need to use "E" suffix
>     anyway.
>     And for those who really need that exabytes suffix, they can enable
>     that suffix pretty easily.
> 
>   * Add overflow checks for the suffixes
>     If the original number string is fine, but with the extra left
>     shift overflow happens, then -EOVERFLOW is returned.
> 
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> Cc: linux-kernel@vger.kernel.org
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>  include/linux/kstrtox.h |   7 +++
>  lib/kstrtox.c           | 113 ++++++++++++++++++++++++++++++++++++++--
>  2 files changed, 115 insertions(+), 5 deletions(-)
> 
> diff --git a/include/linux/kstrtox.h b/include/linux/kstrtox.h
> index 7fcf29a4e0de..12c754152c15 100644
> --- a/include/linux/kstrtox.h
> +++ b/include/linux/kstrtox.h
> @@ -9,6 +9,13 @@
>  int __must_check _kstrtoul(const char *s, unsigned int base, unsigned long *res);
>  int __must_check _kstrtol(const char *s, unsigned int base, long *res);
>  
> +/*
> + * The default suffix list would not include "E" since it's too easy to overflow
> + * and not much real world usage.
> + */
> +#define KSTRTOULL_SUFFIX_DEFAULT		("KkMmGgTtPp")
> +int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
> +		     const char *suffixes);
>  int __must_check kstrtoull(const char *s, unsigned int base, unsigned long long *res);
>  int __must_check kstrtoll(const char *s, unsigned int base, long long *res);
>  
> diff --git a/lib/kstrtox.c b/lib/kstrtox.c
> index d586e6af5e5a..63831207dfdd 100644
> --- a/lib/kstrtox.c
> +++ b/lib/kstrtox.c
> @@ -93,7 +93,8 @@ unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long
>  	return _parse_integer_limit(s, base, p, INT_MAX);
>  }
>  
> -static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
> +static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res,
> +		      char **retptr)
>  {
>  	unsigned long long _res;
>  	unsigned int rv;
> @@ -105,11 +106,19 @@ static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
>  	if (rv == 0)
>  		return -EINVAL;
>  	s += rv;
> -	if (*s == '\n')
> +
> +	/*
> +	 * If @retptr is provided, caller is responsible to detect
> +	 * the extra chars, otherwise we can skip one newline.
> +	 */
> +	if (!retptr && *s == '\n')
>  		s++;
> -	if (*s)
> +	if (!retptr && *s)
>  		return -EINVAL;
> +
>  	*res = _res;
> +	if (retptr)
> +		*retptr = (char *)s;
>  	return 0;
>  }
>  
> @@ -133,10 +142,104 @@ int kstrtoull(const char *s, unsigned int base, unsigned long long *res)
>  {
>  	if (s[0] == '+')
>  		s++;
> -	return _kstrtoull(s, base, res);
> +	return _kstrtoull(s, base, res, NULL);
>  }
>  EXPORT_SYMBOL(kstrtoull);
>  
> +/**
> + * kstrtoull_suffix - convert a string to ull with suffixes support
> + * @s: The start of the string. The string must be null-terminated, and may also
> + *  include a single newline before its terminating null.
> + * @base: The number base to use. The maximum supported base is 16. If base is
> + *  given as 0, then the base of the string is automatically detected with the
> + *  conventional semantics - If it begins with 0x the number will be parsed as a
> + *  hexadecimal (case insensitive), if it otherwise begins with 0, it will be
> + *  parsed as an octal number. Otherwise it will be parsed as a decimal.
> + * @res: Where to write the result of the conversion on success.
> + * @suffixes: A string of acceptable suffixes, must be provided. Or caller
> + *  should use kstrtoull() directly.

The suffixes parameter seems a bit cumbersome; callers need to provide
both upper and lower cases, and unsupported characters aren't checked
for. However, I can't think of any better suggestions at this stage.

> + *
> + *
> + * Return 0 on success.
> + *
> + * Return -ERANGE on overflow or -EINVAL if invalid chars found.
> + * Return value must be checked.
> + */
> +int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
> +		     const char *suffixes)
> +{
> +	unsigned long long init_value;
> +	unsigned long long final_value;
> +	char *endptr;
> +	int ret;
> +
> +	ret = _kstrtoull(s, base, &init_value, &endptr);
> +	/* Either already overflow or no number string at all. */
> +	if (ret < 0)
> +		return ret;
> +	final_value = init_value;
> +	/* No suffixes. */
> +	if (!*endptr)
> +		goto done;
> +
> +	switch (*endptr) {
> +	case 'K':
> +	case 'k':
> +		if (!strchr(suffixes, *endptr))
> +			return -EINVAL;
> +		final_value <<= 10;
> +		endptr++;
> +		break;
> +	case 'M':
> +	case 'm':
> +		if (!strchr(suffixes, *endptr))
> +			return -EINVAL;
> +		final_value <<= 20;
> +		endptr++;
> +		break;
> +	case 'G':
> +	case 'g':
> +		if (!strchr(suffixes, *endptr))
> +			return -EINVAL;
> +		final_value <<= 30;
> +		endptr++;
> +		break;
> +	case 'T':
> +	case 't':
> +		if (!strchr(suffixes, *endptr))
> +			return -EINVAL;
> +		final_value <<= 40;
> +		endptr++;
> +		break;
> +	case 'P':
> +	case 'p':
> +		if (!strchr(suffixes, *endptr))
> +			return -EINVAL;
> +		final_value <<= 50;
> +		endptr++;
> +		break;
> +	case 'E':
> +	case 'e':
> +		if (!strchr(suffixes, *endptr))
> +			return -EINVAL;
> +		final_value <<= 60;
> +		endptr++;
> +		break;
> +	}
> +	if (*endptr == '\n')

Nit: the per-case logic could be simplified to a single "shift_val = X"
if you initialise and handle !shift_val.

> +		endptr++;
> +	if (*endptr)
> +		return -EINVAL;
> +
> +	/* Overflow check. */
> +	if (final_value < init_value)
> +		return -EOVERFLOW;
> +done:
> +	*res = final_value;
> +	return 0;
> +}
> +EXPORT_SYMBOL(kstrtoull_suffix);
> +
>  /**
>   * kstrtoll - convert a string to a long long
>   * @s: The start of the string. The string must be null-terminated, and may also
> @@ -159,7 +262,7 @@ int kstrtoll(const char *s, unsigned int base, long long *res)
>  	int rv;
>  
>  	if (s[0] == '-') {
> -		rv = _kstrtoull(s + 1, base, &tmp);
> +		rv = _kstrtoull(s + 1, base, &tmp, NULL);
>  		if (rv < 0)
>  			return rv;
>  		if ((long long)-tmp > 0)

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

[Qu Wenruo]

On 2023/12/18 23:29, David Disseldorp wrote:
> Hi Qu,
>
> On Fri, 15 Dec 2023 19:09:23 +1030, Qu Wenruo wrote:
>
[...]
>> +/**
>> + * kstrtoull_suffix - convert a string to ull with suffixes support
>> + * @s: The start of the string. The string must be null-terminated, and may also
>> + *  include a single newline before its terminating null.
>> + * @base: The number base to use. The maximum supported base is 16. If base is
>> + *  given as 0, then the base of the string is automatically detected with the
>> + *  conventional semantics - If it begins with 0x the number will be parsed as a
>> + *  hexadecimal (case insensitive), if it otherwise begins with 0, it will be
>> + *  parsed as an octal number. Otherwise it will be parsed as a decimal.
>> + * @res: Where to write the result of the conversion on success.
>> + * @suffixes: A string of acceptable suffixes, must be provided. Or caller
>> + *  should use kstrtoull() directly.
>
> The suffixes parameter seems a bit cumbersome; callers need to provide
> both upper and lower cases, and unsupported characters aren't checked
> for. However, I can't think of any better suggestions at this stage.
>

Initially I went bitmap for the prefixes, but it's not any better.

Firstly where the bitmap should start. If we go bit 0 for "K", then the
code would introduce some difference between the bit number and left
shift (bit 0, left shift 10), which may be a little confusing.

If we go bit 1 for "K", the bit and left shift it much better, but bit 0
behavior would be left untouched.

Finally the bitmap itself is not that straightforward.

The limitation of providing both upper and lower case is due to the fact
that we don't have a case insensitive version of strchr().
But I think it's not that to fix, just convert them all to lower or
upper case, then do the strchr().

Would accepting both cases for the suffixes be good enough?

>> + *
>> + *
>> + * Return 0 on success.
>> + *
>> + * Return -ERANGE on overflow or -EINVAL if invalid chars found.
>> + * Return value must be checked.
>> + */
>> +int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
>> +		     const char *suffixes)
>> +{
>> +	unsigned long long init_value;
>> +	unsigned long long final_value;
>> +	char *endptr;
>> +	int ret;
>> +
>> +	ret = _kstrtoull(s, base, &init_value, &endptr);
>> +	/* Either already overflow or no number string at all. */
>> +	if (ret < 0)
>> +		return ret;
>> +	final_value = init_value;
>> +	/* No suffixes. */
>> +	if (!*endptr)
>> +		goto done;
>> +
>> +	switch (*endptr) {
>> +	case 'K':
>> +	case 'k':
>> +		if (!strchr(suffixes, *endptr))
>> +			return -EINVAL;
>> +		final_value <<= 10;
>> +		endptr++;
>> +		break;
>> +	case 'M':
>> +	case 'm':
>> +		if (!strchr(suffixes, *endptr))
>> +			return -EINVAL;
>> +		final_value <<= 20;
>> +		endptr++;
>> +		break;
>> +	case 'G':
>> +	case 'g':
>> +		if (!strchr(suffixes, *endptr))
>> +			return -EINVAL;
>> +		final_value <<= 30;
>> +		endptr++;
>> +		break;
>> +	case 'T':
>> +	case 't':
>> +		if (!strchr(suffixes, *endptr))
>> +			return -EINVAL;
>> +		final_value <<= 40;
>> +		endptr++;
>> +		break;
>> +	case 'P':
>> +	case 'p':
>> +		if (!strchr(suffixes, *endptr))
>> +			return -EINVAL;
>> +		final_value <<= 50;
>> +		endptr++;
>> +		break;
>> +	case 'E':
>> +	case 'e':
>> +		if (!strchr(suffixes, *endptr))
>> +			return -EINVAL;
>> +		final_value <<= 60;
>> +		endptr++;
>> +		break;
>> +	}
>> +	if (*endptr == '\n')
>
> Nit: the per-case logic could be simplified to a single "shift_val = X"
> if you initialise and handle !shift_val.

Indeed, thanks for the hint!

Thanks,
Qu
>
>> +		endptr++;
>> +	if (*endptr)
>> +		return -EINVAL;
>> +
>> +	/* Overflow check. */
>> +	if (final_value < init_value)
>> +		return -EOVERFLOW;
>> +done:
>> +	*res = final_value;
>> +	return 0;
>> +}
>> +EXPORT_SYMBOL(kstrtoull_suffix);
>> +
>>   /**
>>    * kstrtoll - convert a string to a long long
>>    * @s: The start of the string. The string must be null-terminated, and may also
>> @@ -159,7 +262,7 @@ int kstrtoll(const char *s, unsigned int base, long long *res)
>>   	int rv;
>>
>>   	if (s[0] == '-') {
>> -		rv = _kstrtoull(s + 1, base, &tmp);
>> +		rv = _kstrtoull(s + 1, base, &tmp, NULL);
>>   		if (rv < 0)
>>   			return rv;
>>   		if ((long long)-tmp > 0)
>
>
>

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

[David Disseldorp]

On Tue, 19 Dec 2023 06:22:42 +1030, Qu Wenruo wrote:

> On 2023/12/18 23:29, David Disseldorp wrote:
> > Hi Qu,
> >
> > On Fri, 15 Dec 2023 19:09:23 +1030, Qu Wenruo wrote:
> >  
> [...]
> >> +/**
> >> + * kstrtoull_suffix - convert a string to ull with suffixes support
> >> + * @s: The start of the string. The string must be null-terminated, and may also
> >> + *  include a single newline before its terminating null.
> >> + * @base: The number base to use. The maximum supported base is 16. If base is
> >> + *  given as 0, then the base of the string is automatically detected with the
> >> + *  conventional semantics - If it begins with 0x the number will be parsed as a
> >> + *  hexadecimal (case insensitive), if it otherwise begins with 0, it will be
> >> + *  parsed as an octal number. Otherwise it will be parsed as a decimal.
> >> + * @res: Where to write the result of the conversion on success.
> >> + * @suffixes: A string of acceptable suffixes, must be provided. Or caller
> >> + *  should use kstrtoull() directly.  
> >
> > The suffixes parameter seems a bit cumbersome; callers need to provide
> > both upper and lower cases, and unsupported characters aren't checked
> > for. However, I can't think of any better suggestions at this stage.
> >  
> 
> Initially I went bitmap for the prefixes, but it's not any better.
> 
> Firstly where the bitmap should start. If we go bit 0 for "K", then the
> code would introduce some difference between the bit number and left
> shift (bit 0, left shift 10), which may be a little confusing.
> 
> If we go bit 1 for "K", the bit and left shift it much better, but bit 0
> behavior would be left untouched.
> 
> Finally the bitmap itself is not that straightforward.

One benefit from a bitmap would be that unsupported @suffixes are easier
to detect (instead of ignored), but I think if you rename the function
kstrtoull_unit_suffix() then it should be pretty clear what's supported.

> The limitation of providing both upper and lower case is due to the fact
> that we don't have a case insensitive version of strchr().
> But I think it's not that to fix, just convert them all to lower or
> upper case, then do the strchr().
> 
> Would accepting both cases for the suffixes be good enough?

I think so.

Cheers, David

RE: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

[David Laight]

From: David Disseldorp
> Sent: 18 December 2023 13:00
> 
> On Fri, 15 Dec 2023 19:09:23 +1030, Qu Wenruo wrote:
> 
> > Just as mentioned in the comment of memparse(), the simple_stroull()
> > usage can lead to overflow all by itself.
> >
> > Furthermore, the suffix calculation is also super overflow prone because
> > that some suffix like "E" itself would eat 60bits, leaving only 4 bits
> > available.
> >
> > And that suffix "E" can also lead to confusion since it's using the same
> > char of hex Ox'E'.
> >
> > One simple example to expose all the problem is to use memparse() on
> > "25E".
> > The correct value should be 28823037615171174400, but the suffix E makes
> > it super simple to overflow, resulting the incorrect value
> > 10376293541461622784 (9E).

Some more bikeshed paint :-)
...
> > +	ret = _kstrtoull(s, base, &init_value, &endptr);
> > +	/* Either already overflow or no number string at all. */
> > +	if (ret < 0)
> > +		return ret;
> > +	final_value = init_value;
> > +	/* No suffixes. */
> > +	if (!*endptr)
> > +		goto done;

How about:
	suffix = *endptr;
	if (!strchr(suffixes, suffix))
		return -ENIVAL;
	shift = strcspn("KkMmGgTtPp", suffix)/2 * 10 + 10;
	if (shift > 50)
		return -EINVAL;
	if (value >> (64 - shift))
		return -EOVERFLOW;
	value <<= shift;

Although purists might want to multiply by 1000 not 1024.
And SI multipliers are all upper-case - except k.
	
...
> > +	/* Overflow check. */
> > +	if (final_value < init_value)
> > +		return -EOVERFLOW;

That is just plain wrong.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

[Qu Wenruo]

On 2023/12/20 03:12, David Laight wrote:
> From: David Disseldorp
>> Sent: 18 December 2023 13:00
>>
>> On Fri, 15 Dec 2023 19:09:23 +1030, Qu Wenruo wrote:
>>
>>> Just as mentioned in the comment of memparse(), the simple_stroull()
>>> usage can lead to overflow all by itself.
>>>
>>> Furthermore, the suffix calculation is also super overflow prone because
>>> that some suffix like "E" itself would eat 60bits, leaving only 4 bits
>>> available.
>>>
>>> And that suffix "E" can also lead to confusion since it's using the same
>>> char of hex Ox'E'.
>>>
>>> One simple example to expose all the problem is to use memparse() on
>>> "25E".
>>> The correct value should be 28823037615171174400, but the suffix E makes
>>> it super simple to overflow, resulting the incorrect value
>>> 10376293541461622784 (9E).
>
> Some more bikeshed paint :-)
> ...
>>> +	ret = _kstrtoull(s, base, &init_value, &endptr);
>>> +	/* Either already overflow or no number string at all. */
>>> +	if (ret < 0)
>>> +		return ret;
>>> +	final_value = init_value;
>>> +	/* No suffixes. */
>>> +	if (!*endptr)
>>> +		goto done;
>
> How about:
> 	suffix = *endptr;
> 	if (!strchr(suffixes, suffix))
> 		return -ENIVAL;
> 	shift = strcspn("KkMmGgTtPp", suffix)/2 * 10 + 10;

This means the caller has to provide the suffix string in this
particular order.
For default suffix list it's not that hard as it's already defined as a
macro.

But for those call sites which needs "E", wrongly located "Ee" can screw
up the whole process.

> 	if (shift > 50)
> 		return -EINVAL;
> 	if (value >> (64 - shift))
> 		return -EOVERFLOW;
> 	value <<= shift;
>
> Although purists might want to multiply by 1000 not 1024.
> And SI multipliers are all upper-case - except k.
>
> ...
>>> +	/* Overflow check. */
>>> +	if (final_value < init_value)
>>> +		return -EOVERFLOW;
>
> That is just plain wrong.

Indeed, I just found a very simple example to prove it wrong, 4 bit
binary 0110, left shift 2, result is 1000, still larger than the
original one.

Thanks,
Qu
>
> 	David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
>
>

RE: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

[David Laight]

From: Qu Wenruo
> Sent: 19 December 2023 21:18
...
> > How about:
> > 	suffix = *endptr;
> > 	if (!strchr(suffixes, suffix))
> > 		return -ENIVAL;
> > 	shift = strcspn("KkMmGgTtPp", suffix)/2 * 10 + 10;
> 
> This means the caller has to provide the suffix string in this
> particular order.

No, The strchr() checks that the suffix is one the caller wanted.
The strcspn() is against a fixed list - so the order can be
selected to make the code shorter.

Actually strcspn() isn't the function you need.
There might be a function like strchr() that returns a count
but I can't remember its name and it may not be in kernel.
You might have to write:
	shift = 0;
	for (const char *sfp = "KkMmGgTtPp"; suffix != *sfp; sfp++, shift++) {
		if (!*sfp)
			return -EINVAL;
	}
	shift = shift/2 + 1 * 10;

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

[Qu Wenruo]

On 2023/12/20 19:01, David Laight wrote:
> From: Qu Wenruo
>> Sent: 19 December 2023 21:18
> ...
>>> How about:
>>> 	suffix = *endptr;
>>> 	if (!strchr(suffixes, suffix))
>>> 		return -ENIVAL;
>>> 	shift = strcspn("KkMmGgTtPp", suffix)/2 * 10 + 10;
>>
>> This means the caller has to provide the suffix string in this
>> particular order.
>
> No, The strchr() checks that the suffix is one the caller wanted.
> The strcspn() is against a fixed list - so the order can be
> selected to make the code shorter.

Ah, got it.
Although in that case, the 1st parameter should be ("KkMmGgTtPpEe"), as
we still support "Ee", just not by default.

Thanks,
Qu
>
> Actually strcspn() isn't the function you need.
> There might be a function like strchr() that returns a count
> but I can't remember its name and it may not be in kernel.
> You might have to write:
> 	shift = 0;
> 	for (const char *sfp = "KkMmGgTtPp"; suffix != *sfp; sfp++, shift++) {
> 		if (!*sfp)
> 			return -EINVAL;
> 	}
> 	shift = shift/2 + 1 * 10;
>
> 	David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)