[PATCH v2] libsemanage: genhomedircon - improve handling large groups

[Petr Lautrbach <plautrba@redhat.com>]

getgrnam_r() uses a preallocated buffer to store a structure containing
the broken-out fields of the record in the group database. The size of
this buffer is usually sysconf(_SC_GETGR_R_SIZE_MAX) == 1024 and it is
not enough for groups with a large number of users.  In these cases,
getgrnam_r() returns -1 and sets errno to ERANGE and the caller can
retry with a larger buffer.

Fixes:
$ semanage login -a -s user_u -r s0-s0:c1.c2 '%largegroup'
libsemanage.semanage_direct_commit: semanage_genhomedircon returned error code -1. (Numerical result out of range).
OSError: Numerical result out of range

Signed-off-by: Petr Lautrbach <plautrba@redhat.com>
---
 libsemanage/src/genhomedircon.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/libsemanage/src/genhomedircon.c b/libsemanage/src/genhomedircon.c
index 591941fb..e5f8d371 100644
--- a/libsemanage/src/genhomedircon.c
+++ b/libsemanage/src/genhomedircon.c
@@ -1077,10 +1077,24 @@ static int get_group_users(genhomedircon_settings_t * s,
 
 	const char *grname = selogin + 1;
 
-	if (getgrnam_r(grname, &grstorage, grbuf,
-			(size_t) grbuflen, &group) != 0) {
-		goto cleanup;
+	errno = 0;
+	while (
+		(retval = getgrnam_r(grname, &grstorage, grbuf, (size_t) grbuflen, &group)) != 0 &&
+		errno == ERANGE
+	) {
+		char *new_grbuf;
+		grbuflen *= 2;
+		if (grbuflen < 0)
+			/* the member list could exceed 2Gb on a system with a 32-bit CPU (where
+			 * sizeof(long) = 4) - if this ever happened, the loop would become infinite. */
+			goto cleanup;
+		new_grbuf = realloc(grbuf, grbuflen);
+		if (new_grbuf == NULL)
+			goto cleanup;
+		grbuf = new_grbuf;
 	}
+	if (retval != 0)
+		goto cleanup;
 
 	if (group == NULL) {
 		ERR(s->h_semanage, "Can't find group named %s\n", grname);
-- 
2.20.1