* libuuid related tasks for next v2.39
@ 2022-04-01 9:21 Karel Zak
2022-04-11 13:36 ` Stanislav Brabec
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Karel Zak @ 2022-04-01 9:21 UTC (permalink / raw)
To: util-linux; +Cc: Stanislav Brabec, Trapp, Michael
Hi,
I'd like to invest time to libuuid and uuidd in the next v2.39.
Stanislav had same ideas and a lot of research in libuuid code to
make things more stable, and I'd like to improve testability of the
all UUID stack we have in util-linux.
TODO:
* uuid_generate_* functions refactoring to make things readable and
to separate ways how UUIDs are generated
* clean up uuidd to generate UUID only by the most robust UUIDs way
without any fallback (use uuid_generate_time_sync_by_file())
* report all issues on uuidd stderr
* optionally keep clockfile (/var/lib/libuuid/clock.txt) in uuidd hands
to make sure the file is not shared with other users if the system
is only-uuidd based; for example add --clock-file uuidd option to
overwrite libuuid default.
* invite our SAP friends to the upstream development to share ideas
and requirement :-)
* rewrite misc-utils/test_uuidd.c to something commonly usable
(uuidinfo tool?) and use it in util-linux regression tests,
features:
- detect if uuidd is running
- add statistics to the uuidd (number of generated UUIDs,
clients, errors, ...)
- add new UUIDD_OP_* to the uuidd to get additional information
about the daemon (paths, statistics, etc.)
- stress test (like the current test_uuidd.c)
* anything else?
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: libuuid related tasks for next v2.39
2022-04-01 9:21 libuuid related tasks for next v2.39 Karel Zak
@ 2022-04-11 13:36 ` Stanislav Brabec
2022-04-12 11:36 ` Karel Zak
[not found] ` <AM9PR02MB70447FD1C9D326ABAF7ABFE8F9E69@AM9PR02MB7044.eurprd02.prod.outlook.com>
[not found] ` <CAA0A08XAF2eWScVKQ1Jd0vUPfHzZw8PAtiS-jRbom0EvRQ+Zsg@mail.gmail.com>
2 siblings, 1 reply; 6+ messages in thread
From: Stanislav Brabec @ 2022-04-11 13:36 UTC (permalink / raw)
To: Karel Zak, util-linux; +Cc: Trapp, Michael
Dne 01. 04. 22 v 11:21 Karel Zak napsal(a):
>
> Hi,
>
> I'd like to invest time to libuuid and uuidd in the next v2.39.
>
> Stanislav had same ideas and a lot of research in libuuid code to
> make things more stable, and I'd like to improve testability of the
> all UUID stack we have in util-linux.
>
> TODO:
>
> * uuid_generate_* functions refactoring to make things readable and
> to separate ways how UUIDs are generated
Yes, it makes sense.
> * clean up uuidd to generate UUID only by the most robust UUIDs way
> without any fallback (use uuid_generate_time_sync_by_file())
Yes.
> * report all issues on uuidd stderr
Yes. Or even exit with error on any (potentially permanent) issue.
But more important:
All issues should generate error to the libuuid. Otherwise the
application gets downgraded unsafe UUIDs without properly detecting it.
> * optionally keep clockfile (/var/lib/libuuid/clock.txt) in uuidd hands
> to make sure the file is not shared with other users if the system
> is only-uuidd based; for example add --clock-file uuidd option to
> overwrite libuuid default.
I guess it is mandatory, otherwise we will have a problem of the root
stealing of the lock file.
The proposed default was ~/.libuuid_clock.txt
There should be an internal function that sets the path to the clock
file. It probably makes sense to make the path configurable in the
config file, but the function should not be public:
- Only uuidd and the configuration read will use the function.
- Applications on systems systems without writable home could configure
it e. g. to /run.
- Using different paths for different applications would cause bad
things; private function makes it impossible.
- It will ensure that nothing else than uuidd will use
/var/lib/libuuid/clock.txt.
I have applied a temporary work-around for the root stealing problem:
uuidd.service:
ExecStartPre=+-/usr/bin/chown uuidd:uuidd /var/lib/libuuid/clock.txt
But it is ugly and it cures just a symptoms and does not work with older version of systemd.
> * invite our SAP friends to the upstream development to share ideas
> and requirement :-)
>
> * rewrite misc-utils/test_uuidd.c to something commonly usable
> (uuidinfo tool?) and use it in util-linux regression tests,
> features:
>
> - detect if uuidd is running
> - add statistics to the uuidd (number of generated UUIDs,
> clients, errors, ...)
> - add new UUIDD_OP_* to the uuidd to get additional information
> about the daemon (paths, statistics, etc.)
> - stress test (like the current test_uuidd.c)
>
>
> * anything else?
>
Add support for three levels of safe state for uuid_generate_time_safe():
- Absolutely safe (uuidd was used, and there was no error in uuidd). The
UUID is guaranteed to be unique.
- Safe for the current UID (clock file was used, but no uuidd). The UUID
is guaranteed to be unique for the current UID, but there is no
guarantee that other UIDs.
- Unsafe (clock file use failed, uuidd use failed). The UUID is not
guaranteed to be unique at all. Just a random.
It opens a discussion whether the new return value should be integrated
to the current functions, or add a new function with three return
values. Or even change the current function to the macro, so the newly
compiled code will stop to use the old function name.
If we will change the API, then no code will fail. But the result will
depend on the check implementation:
If the return value will be compared by == -1, it will catch only the
unsafe return.
If the return value will be compared by != 0, it will catch both unsafe
and current-UID-safe.
--
Best Regards / S pozdravem,
Stanislav Brabec
software developer
---------------------------------------------------------------------
SUSE LINUX, s. r. o. e-mail: sbrabec@suse.com
Křižíkova 148/34 (Corso IIa) tel: +420 284 084 060
186 00 Praha 8-Karlín fax: +420 284 084 001
Czech Republic http://www.suse.cz/
PGP: 830B 40D5 9E05 35D8 5E27 6FA3 717C 209F A04F CD76
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: libuuid related tasks for next v2.39
[not found] ` <AM9PR02MB70447FD1C9D326ABAF7ABFE8F9E69@AM9PR02MB7044.eurprd02.prod.outlook.com>
@ 2022-04-11 13:45 ` Stanislav Brabec
0 siblings, 0 replies; 6+ messages in thread
From: Stanislav Brabec @ 2022-04-11 13:45 UTC (permalink / raw)
To: Trapp, Michael, util-linux; +Cc: Karel Zak
Dne 07. 04. 22 v 10:23 Trapp, Michael napsal(a):
> For uuidd this would result in:
> -no need for a lock on LIBUUID_CLOCK_FILE, what also might help to
> solve some authorization issues.
This should be a benefit.
The only possible downside would be losing of context on uuidd restart.
That happens e. g. during a security update.Could this be a problem?
Really fast machines restart in a small fraction of second.
--
Best Regards / S pozdravem,
Stanislav Brabec
software developer
---------------------------------------------------------------------
SUSE LINUX, s. r. o. e-mail: sbrabec@suse.com
Křižíkova 148/34 (Corso IIa) tel: +420 284 084 060
186 00 Praha 8-Karlín fax: +420 284 084 001
Czech Republic http://www.suse.cz/
PGP: 830B 40D5 9E05 35D8 5E27 6FA3 717C 209F A04F CD76
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: libuuid related tasks for next v2.39
[not found] ` <CAA0A08XAF2eWScVKQ1Jd0vUPfHzZw8PAtiS-jRbom0EvRQ+Zsg@mail.gmail.com>
@ 2022-04-12 10:34 ` Karel Zak
[not found] ` <CAA0A08VmzKTzFCEqnFP9tw50zgx4iPsif_BktuRSAvfm3hH52w@mail.gmail.com>
0 siblings, 1 reply; 6+ messages in thread
From: Karel Zak @ 2022-04-12 10:34 UTC (permalink / raw)
To: Aurélien Lajoie; +Cc: Util-Linux, Stanislav Brabec, Trapp, Michael
On Fri, Apr 01, 2022 at 11:49:17AM +0200, Aurélien Lajoie wrote:
> What about nice helpers to get fast access to the different part of the
> uuid?
I'm not sure if I understand. What do you mean with the helpers?
> We need to keep an eye on performance, php uuid code was more perf than the
> previous version of our lib.
Yes, printf sucks. It's fixed now. Thanks to you :-)
> Perhaps add some test to catch big impact on serialization and
> deserialization
Good point, maybe it's something we can add to (suggested)
uuidinfo command.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: libuuid related tasks for next v2.39
2022-04-11 13:36 ` Stanislav Brabec
@ 2022-04-12 11:36 ` Karel Zak
0 siblings, 0 replies; 6+ messages in thread
From: Karel Zak @ 2022-04-12 11:36 UTC (permalink / raw)
To: Stanislav Brabec; +Cc: util-linux, Trapp, Michael
On Mon, Apr 11, 2022 at 03:36:01PM +0200, Stanislav Brabec wrote:
> Dne 01. 04. 22 v 11:21 Karel Zak napsal(a):
> > * uuid_generate_* functions refactoring to make things readable and
> > to separate ways how UUIDs are generated
> Yes, it makes sense.
> > * clean up uuidd to generate UUID only by the most robust UUIDs way
> > without any fallback (use uuid_generate_time_sync_by_file())
> Yes.
> > * report all issues on uuidd stderr
> Yes. Or even exit with error on any (potentially permanent) issue.
>
> But more important:
>
> All issues should generate error to the libuuid. Otherwise the application
> gets downgraded unsafe UUIDs without properly detecting it.
Yes.
> > * optionally keep clockfile (/var/lib/libuuid/clock.txt) in uuidd hands
> > to make sure the file is not shared with other users if the system
> > is only-uuidd based; for example add --clock-file uuidd option to
> > overwrite libuuid default.
> I guess it is mandatory, otherwise we will have a problem of the root
> stealing of the lock file.
>
> The proposed default was ~/.libuuid_clock.txt
>
> There should be an internal function that sets the path to the clock file.
> It probably makes sense to make the path configurable in the config file,
> but the function should not be public:
>
> - Only uuidd and the configuration read will use the function.
> - Applications on systems systems without writable home could configure it
> e. g. to /run.
> - Using different paths for different applications would cause bad things;
> private function makes it impossible.
I can imagine this as public :-) The majority of users will use basic
simple API (just give me UUID). The advanced minority has to read docs
:-)
I can imagine use-case where you have more independent users who need
time based UUIDs, but they don't care about collisions with the other
users (for example two web applications with independent DBs, etc.)/
The default should be safe as much as possible, but we should not try
to be more smart than admins/developers as we have no clue about their
use-cases ;-)
But this is not so important for now, we can keep it as uuidd only
in UUIDD_PRIVATE namespace in libuuid/src/libuuid.sym.
> - It will ensure that nothing else than uuidd will use
> /var/lib/libuuid/clock.txt.
>
> I have applied a temporary work-around for the root stealing problem:
>
> uuidd.service:
>
> ExecStartPre=+-/usr/bin/chown uuidd:uuidd /var/lib/libuuid/clock.txt
>
> But it is ugly and it cures just a symptoms and does not work with older version of systemd.
>
> > * invite our SAP friends to the upstream development to share ideas
> > and requirement :-)
> >
> > * rewrite misc-utils/test_uuidd.c to something commonly usable
> > (uuidinfo tool?) and use it in util-linux regression tests,
> > features:
> >
> > - detect if uuidd is running
> > - add statistics to the uuidd (number of generated UUIDs,
> > clients, errors, ...)
> > - add new UUIDD_OP_* to the uuidd to get additional information
> > about the daemon (paths, statistics, etc.)
> > - stress test (like the current test_uuidd.c)
> >
> >
> > * anything else?
> >
> Add support for three levels of safe state for uuid_generate_time_safe():
> - Absolutely safe (uuidd was used, and there was no error in uuidd). The
> UUID is guaranteed to be unique.
> - Safe for the current UID (clock file was used, but no uuidd). The UUID is
> guaranteed to be unique for the current UID, but there is no guarantee that
> other UIDs.
> - Unsafe (clock file use failed, uuidd use failed). The UUID is not
> guaranteed to be unique at all. Just a random.
>
> It opens a discussion whether the new return value should be integrated to
> the current functions, or add a new function with three return values. Or
> even change the current function to the macro, so the newly compiled code
> will stop to use the old function name.
>
> If we will change the API, then no code will fail. But the result will
> depend on the check implementation:
> If the return value will be compared by == -1, it will catch only the unsafe
> return.
> If the return value will be compared by != 0, it will catch both unsafe and
> current-UID-safe.
If we decide to change the API (introduce a new functions) then I'd like to
see something based on handler/context. IMHO only context/handler based libraries
are extendible and easy to maintain and ready for relatively complex tasks.
libuuid_context *cxt = uuid_new_context();
uuid_context_set_clockfile(cxt, "/somewhere/clock.txt");
uuid_context_generate_time(cxt, &uuid, &state);
where @state is UUID_STATE_{SAFE,UNSAFE,UUIDD,...}
we can also add
uuid_context_force_state(cxt, UUID_STATE_ABSOLUTE);
to make sure uuid_context_generate_time() returns only requested
quality or error, etc.
We can also for uuidd make locking optional:
uuid_context_disable_lock(cxt, TRUE);
etc.
We can also use the context for communication with uuidd
uuid_uuidd *uuidd = uuid_new_uuidd();
uuid_uuidd_get_state()
uuid_uuidd_get_pid()
uuid_uuidd_get_statistic()
uuid_uuidd_get_clockfile()
uuid_uuidd_get_error()
uuid_uuidd_get_uuid()
....
Anyway, the first step we need uuid_generate_* functions
refactoring to clean up the spaghetti stuff.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: libuuid related tasks for next v2.39
[not found] ` <CAA0A08VmzKTzFCEqnFP9tw50zgx4iPsif_BktuRSAvfm3hH52w@mail.gmail.com>
@ 2022-04-12 11:44 ` Karel Zak
0 siblings, 0 replies; 6+ messages in thread
From: Karel Zak @ 2022-04-12 11:44 UTC (permalink / raw)
To: Aurélien Lajoie; +Cc: Util-Linux, Stanislav Brabec, Trapp, Michael
On Tue, Apr 12, 2022 at 01:15:56PM +0200, Aurélien Lajoie wrote:
> On Tue, Apr 12, 2022 at 12:35 PM Karel Zak <kzak@redhat.com> wrote:
> >
> > On Fri, Apr 01, 2022 at 11:49:17AM +0200, Aurélien Lajoie wrote:
> > > What about nice helpers to get fast access to the different part of the
> > > uuid?
> >
> > I'm not sure if I understand. What do you mean with the helpers?
>
> Accessing the different part of the uuid
> version
> time_low
> time_mid
> time_hi_and_version
> clock_seq_hi_and_res clock_seq_low
> node
This is obvious, I'm trying to understand "Accessing". Do you mean
unparse the specified parts or update/set it? For what use-case?
We have something like this for command line (misc-utils/uuidparse.c).
$ uuidparse $(uuidgen --time)
UUID VARIANT TYPE TIME
970f197e-ba55-11ec-aa93-43a3097bca4a DCE time-based 2022-04-12 13:42:11,097843+02:00
Do you mean anything like;
uuid_time_get_version(uuidstr);
uuid_time_get_clock_seq()
...
or the same but for already parsed uuid (uuid_t)?
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-04-12 12:28 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-01 9:21 libuuid related tasks for next v2.39 Karel Zak
2022-04-11 13:36 ` Stanislav Brabec
2022-04-12 11:36 ` Karel Zak
[not found] ` <AM9PR02MB70447FD1C9D326ABAF7ABFE8F9E69@AM9PR02MB7044.eurprd02.prod.outlook.com>
2022-04-11 13:45 ` Stanislav Brabec
[not found] ` <CAA0A08XAF2eWScVKQ1Jd0vUPfHzZw8PAtiS-jRbom0EvRQ+Zsg@mail.gmail.com>
2022-04-12 10:34 ` Karel Zak
[not found] ` <CAA0A08VmzKTzFCEqnFP9tw50zgx4iPsif_BktuRSAvfm3hH52w@mail.gmail.com>
2022-04-12 11:44 ` Karel Zak
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).