xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: Pawel Wieczorkiewicz <wipawel@amazon.de>
To: <xen-devel@lists.xen.org>, <xen-devel@lists.xenproject.org>
Cc: wipawel@amazon.com, Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wl@xen.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	George Dunlap <George.Dunlap@eu.citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Ross Lagerwall <ross.lagerwall@citrix.com>,
	Ian Jackson <ian.jackson@eu.citrix.com>,
	mpohlack@amazon.com, Tim Deegan <tim@xen.org>,
	Pawel Wieczorkiewicz <wipawel@amazon.de>,
	Julien Grall <julien.grall@arm.com>,
	Jan Beulich <jbeulich@suse.com>
Subject: [Xen-devel] [PATCH 01/14] livepatch: Always check hypervisor build ID upon hotpatch upload
Date: Wed, 21 Aug 2019 08:19:18 +0000	[thread overview]
Message-ID: <20190821081931.90887-2-wipawel@amazon.de> (raw)
In-Reply-To: <20190821081931.90887-1-wipawel@amazon.de>

This change is part of a independant stacked hotpatch modules
feature. This feature allows to bypass dependencies between modules
upon loading, but still verifies Xen build ID matching.

In order to prevent (up)loading any hotpatches built for different
hypervisor version as indicated by the Xen Build ID, add checking for
the payload's vs Xen's build id match.

To achieve that embed into every hotpatch another section with a
dedicated hypervisor build id in it. After the payload is loaded and
the .livepatch.xen_depends section becomes available, perform the
check and reject the payload if there is no match.

Signed-off-by: Pawel Wieczorkiewicz <wipawel@amazon.de>
Reviewed-by: Andra-Irina Paraschiv <andraprs@amazon.com>
Reviewed-by: Bjoern Doebel <doebel@amazon.de>
Reviewed-by: Eslam Elnikety <elnikety@amazon.de>
Reviewed-by: Martin Pohlack <mpohlack@amazon.de>
---
 .gitignore                  |  1 +
 docs/misc/livepatch.pandoc  | 28 +++++++++++++++++++--------
 xen/common/livepatch.c      | 47 +++++++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/livepatch.h |  7 ++++---
 xen/test/livepatch/Makefile | 31 +++++++++++++++++++++++++-----
 5 files changed, 98 insertions(+), 16 deletions(-)

diff --git a/.gitignore b/.gitignore
index 3c947ac948..6f83fc8728 100644
--- a/.gitignore
+++ b/.gitignore
@@ -312,6 +312,7 @@ xen/test/livepatch/xen_bye_world.livepatch
 xen/test/livepatch/xen_hello_world.livepatch
 xen/test/livepatch/xen_nop.livepatch
 xen/test/livepatch/xen_replace_world.livepatch
+xen/test/livepatch/xen_no_xen_buildid.livepatch
 xen/tools/kconfig/.tmp_gtkcheck
 xen/tools/kconfig/.tmp_qtcheck
 xen/tools/symbols
diff --git a/docs/misc/livepatch.pandoc b/docs/misc/livepatch.pandoc
index 6d9f72f49b..fd1f5d0126 100644
--- a/docs/misc/livepatch.pandoc
+++ b/docs/misc/livepatch.pandoc
@@ -270,6 +270,8 @@ like what the Linux kernel module loader does.
 The payload contains at least three sections:
 
  * `.livepatch.funcs` - which is an array of livepatch_func structures.
+ * `.livepatch.xen_depends` - which is an ELF Note that describes what Xen
+    build-id the payload depends on. **MUST** have one.
  * `.livepatch.depends` - which is an ELF Note that describes what the payload
     depends on. **MUST** have one.
  *  `.note.gnu.build-id` - the build-id of this payload. **MUST** have one.
@@ -383,16 +385,16 @@ The type definition of the function are as follow:
     typedef void (*livepatch_loadcall_t)(void);
     typedef void (*livepatch_unloadcall_t)(void);
 
-### .livepatch.depends and .note.gnu.build-id
+### .livepatch.xen_depends, .livepatch.depends and .note.gnu.build-id
 
 To support dependencies checking and safe loading (to load the
 appropiate payload against the right hypervisor) there is a need
 to embbed an build-id dependency.
 
-This is done by the payload containing an section `.livepatch.depends`
-which follows the format of an ELF Note. The contents of this
-(name, and description) are specific to the linker utilized to
-build the hypevisor and payload.
+This is done by the payload containing sections `.livepatch.xen_depends`
+and `.livepatch.depends` which follow the format of an ELF Note.
+The contents of these (name, and description) are specific to the linker
+utilized to build the hypevisor and payload.
 
 If GNU linker is used then the name is `GNU` and the description
 is a NT_GNU_BUILD_ID type ID. The description can be an SHA1
@@ -400,6 +402,13 @@ checksum, MD5 checksum or any unique value.
 
 The size of these structures varies with the `--build-id` linker option.
 
+There are two kinds of build-id dependencies:
+
+ * Xen build-id dependency (.livepatch.xen_depends section)
+ * previous payload build-id dependency (.livepatch.depends section)
+
+See "Live patch interdependencies" for more information.
+
 ## Hypercalls
 
 We will employ the sub operations of the system management hypercall (sysctl).
@@ -894,13 +903,16 @@ but is more complex to implement.
 The second option which requires an build-id of the hypervisor
 is implemented in the Xen hypervisor.
 
-Specifically each payload has two build-id ELF notes:
+Specifically each payload has three build-id ELF notes:
  * The build-id of the payload itself (generated via --build-id).
+ * The build-id of the Xen hypervisor it depends on (extracted from the
+   hypervisor during build time).
  * The build-id of the payload it depends on (extracted from the
    the previous payload or hypervisor during build time).
 
-This means that the very first payload depends on the hypervisor
-build-id.
+This means that every payload depends on the hypervisor build-id and on
+the build-id of the previous payload in the stack.
+The very first payload depends on the hypervisor build-id only.
 
 # Not Yet Done
 
diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c
index d6eaae6d3b..6a4af6ce57 100644
--- a/xen/common/livepatch.c
+++ b/xen/common/livepatch.c
@@ -74,6 +74,7 @@ struct payload {
     unsigned int nsyms;                  /* Nr of entries in .strtab and symbols. */
     struct livepatch_build_id id;        /* ELFNOTE_DESC(.note.gnu.build-id) of the payload. */
     struct livepatch_build_id dep;       /* ELFNOTE_DESC(.livepatch.depends). */
+    struct livepatch_build_id xen_dep;   /* ELFNOTE_DESC(.livepatch.xen_depends). */
     livepatch_loadcall_t *const *load_funcs;   /* The array of funcs to call after */
     livepatch_unloadcall_t *const *unload_funcs;/* load and unload of the payload. */
     unsigned int n_load_funcs;           /* Nr of the funcs to load and execute. */
@@ -476,11 +477,34 @@ static bool section_ok(const struct livepatch_elf *elf,
     return true;
 }
 
+static int check_xen_build_id(const struct payload *payload)
+{
+    const void *id = NULL;
+    unsigned int len = 0;
+    int rc;
+
+    ASSERT(payload->xen_dep.len);
+    ASSERT(payload->xen_dep.p);
+
+    rc = xen_build_id(&id, &len);
+    if ( rc )
+        return rc;
+
+    if ( payload->xen_dep.len != len || memcmp(id, payload->xen_dep.p, len) ) {
+        dprintk(XENLOG_ERR, "%s%s: check against hypervisor build-id failed!\n",
+                LIVEPATCH, payload->name);
+        return -EINVAL;
+    }
+
+    return 0;
+}
+
 static int check_special_sections(const struct livepatch_elf *elf)
 {
     unsigned int i;
     static const char *const names[] = { ELF_LIVEPATCH_FUNC,
                                          ELF_LIVEPATCH_DEPENDS,
+                                         ELF_LIVEPATCH_XEN_DEPENDS,
                                          ELF_BUILD_ID_NOTE};
     DECLARE_BITMAP(found, ARRAY_SIZE(names)) = { 0 };
 
@@ -632,6 +656,22 @@ static int prepare_payload(struct payload *payload,
             return -EINVAL;
     }
 
+    sec = livepatch_elf_sec_by_name(elf, ELF_LIVEPATCH_XEN_DEPENDS);
+    if ( sec )
+    {
+        n = sec->load_addr;
+
+        if ( sec->sec->sh_size <= sizeof(*n) )
+            return -EINVAL;
+
+        if ( xen_build_id_check(n, sec->sec->sh_size,
+                                &payload->xen_dep.p, &payload->xen_dep.len) )
+            return -EINVAL;
+
+        if ( !payload->xen_dep.len || !payload->xen_dep.p )
+            return -EINVAL;
+    }
+
     /* Setup the virtual region with proper data. */
     region = &payload->region;
 
@@ -882,6 +922,10 @@ static int load_payload_data(struct payload *payload, void *raw, size_t len)
     if ( rc )
         goto out;
 
+    rc = check_xen_build_id(payload);
+    if ( rc )
+        goto out;
+
     rc = build_symbol_table(payload, &elf);
     if ( rc )
         goto out;
@@ -1655,6 +1699,9 @@ static void livepatch_printall(unsigned char key)
 
         if ( data->dep.len )
             printk("depend-on=%*phN\n", data->dep.len, data->dep.p);
+
+        if ( data->xen_dep.len )
+            printk("depend-on-xen=%*phN\n", data->xen_dep.len, data->xen_dep.p);
     }
 
     spin_unlock(&payload_lock);
diff --git a/xen/include/xen/livepatch.h b/xen/include/xen/livepatch.h
index 1b1817ca0d..ed997aa4cc 100644
--- a/xen/include/xen/livepatch.h
+++ b/xen/include/xen/livepatch.h
@@ -29,9 +29,10 @@ struct xen_sysctl_livepatch_op;
 /* Convenience define for printk. */
 #define LIVEPATCH             "livepatch: "
 /* ELF payload special section names. */
-#define ELF_LIVEPATCH_FUNC    ".livepatch.funcs"
-#define ELF_LIVEPATCH_DEPENDS ".livepatch.depends"
-#define ELF_BUILD_ID_NOTE      ".note.gnu.build-id"
+#define ELF_LIVEPATCH_FUNC        ".livepatch.funcs"
+#define ELF_LIVEPATCH_DEPENDS     ".livepatch.depends"
+#define ELF_LIVEPATCH_XEN_DEPENDS ".livepatch.xen_depends"
+#define ELF_BUILD_ID_NOTE         ".note.gnu.build-id"
 /* Arbitrary limit for payload size and .bss section size. */
 #define LIVEPATCH_MAX_SIZE     MB(2)
 
diff --git a/xen/test/livepatch/Makefile b/xen/test/livepatch/Makefile
index 6831383db1..fdb82782d2 100644
--- a/xen/test/livepatch/Makefile
+++ b/xen/test/livepatch/Makefile
@@ -19,11 +19,13 @@ LIVEPATCH := xen_hello_world.livepatch
 LIVEPATCH_BYE := xen_bye_world.livepatch
 LIVEPATCH_REPLACE := xen_replace_world.livepatch
 LIVEPATCH_NOP := xen_nop.livepatch
+LIVEPATCH_NO_XEN_BUILDID := xen_no_xen_buildid.livepatch
 
 LIVEPATCHES += $(LIVEPATCH)
 LIVEPATCHES += $(LIVEPATCH_BYE)
 LIVEPATCHES += $(LIVEPATCH_REPLACE)
 LIVEPATCHES += $(LIVEPATCH_NOP)
+LIVEPATCHES += $(LIVEPATCH_NO_XEN_BUILDID)
 
 LIVEPATCH_DEBUG_DIR ?= $(DEBUG_DIR)/xen-livepatch
 
@@ -59,7 +61,7 @@ config.h: xen_hello_world_func.o
 xen_hello_world.o: config.h
 
 .PHONY: $(LIVEPATCH)
-$(LIVEPATCH): xen_hello_world_func.o xen_hello_world.o note.o
+$(LIVEPATCH): xen_hello_world_func.o xen_hello_world.o note.o xen_note.o
 	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH) $^
 
 #
@@ -78,6 +80,17 @@ note.o:
 		   --rename-section=.data=.livepatch.depends,alloc,load,readonly,data,contents -S $@.bin $@
 	rm -f $@.bin
 
+#
+# Append .livepatch.xen_depends section
+# with Xen build-id derived from xen-syms.
+#
+.PHONY: xen_note.o
+xen_note.o:
+	$(OBJCOPY) -O binary --only-section=.note.gnu.build-id $(BASEDIR)/xen-syms $@.bin
+	$(OBJCOPY) $(OBJCOPY_MAGIC) \
+		   --rename-section=.data=.livepatch.xen_depends,alloc,load,readonly,data,contents -S $@.bin $@
+	rm -f $@.bin
+
 #
 # Extract the build-id of the xen_hello_world.livepatch
 # (which xen_bye_world will depend on).
@@ -92,20 +105,28 @@ hello_world_note.o: $(LIVEPATCH)
 xen_bye_world.o: config.h
 
 .PHONY: $(LIVEPATCH_BYE)
-$(LIVEPATCH_BYE): xen_bye_world_func.o xen_bye_world.o hello_world_note.o
+$(LIVEPATCH_BYE): xen_bye_world_func.o xen_bye_world.o hello_world_note.o xen_note.o
 	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_BYE) $^
 
 xen_replace_world.o: config.h
 
 .PHONY: $(LIVEPATCH_REPLACE)
-$(LIVEPATCH_REPLACE): xen_replace_world_func.o xen_replace_world.o note.o
+$(LIVEPATCH_REPLACE): xen_replace_world_func.o xen_replace_world.o note.o xen_note.o
 	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_REPLACE) $^
 
 xen_nop.o: config.h
 
 .PHONY: $(LIVEPATCH_NOP)
-$(LIVEPATCH_NOP): xen_nop.o note.o
+$(LIVEPATCH_NOP): xen_nop.o note.o xen_note.o
 	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_NOP) $^
 
+# This one always fails upon upload, because it deliberetely
+# does not have a .livepatch.xen_depends (xen_note.o) section.
+xen_no_xen_buildid.o: config.h
+
+.PHONY: $(LIVEPATCH_NO_XEN_BUILDID)
+$(LIVEPATCH_NO_XEN_BUILDID): xen_nop.o note.o
+	$(LD) $(LDFLAGS) $(build_id_linker) -r -o $(LIVEPATCH_NO_XEN_BUILDID) $^
+
 .PHONY: livepatch
-livepatch: $(LIVEPATCH) $(LIVEPATCH_BYE) $(LIVEPATCH_REPLACE) $(LIVEPATCH_NOP)
+livepatch: $(LIVEPATCH) $(LIVEPATCH_BYE) $(LIVEPATCH_REPLACE) $(LIVEPATCH_NOP) $(LIVEPATCH_NO_XEN_BUILDID)
-- 
2.16.5




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

  reply	other threads:[~2019-08-21  8:20 UTC|newest]

Thread overview: 66+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-21  8:19 [Xen-devel] [PATCH 00/14] livepatch: new features and fixes Pawel Wieczorkiewicz
2019-08-21  8:19 ` Pawel Wieczorkiewicz [this message]
2019-08-21 18:16   ` [Xen-devel] [PATCH 01/14] livepatch: Always check hypervisor build ID upon hotpatch upload Konrad Rzeszutek Wilk
2019-08-21  8:19 ` [Xen-devel] [PATCH 02/14] livepatch: Allow to override inter-modules buildid dependency Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 03/14] python: Add XC binding for Xen build ID Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 04/14] livepatch: Export payload structure via livepatch_payload.h Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 05/14] livepatch: Implement pre-|post- apply|revert hooks Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 06/14] livepatch: Add support for apply|revert action replacement hooks Pawel Wieczorkiewicz
2019-08-21 18:31   ` Konrad Rzeszutek Wilk
2019-08-21 19:06     ` Wieczorkiewicz, Pawel
2019-08-26 14:30       ` Konrad Rzeszutek Wilk
2019-08-21  8:19 ` [Xen-devel] [PATCH 07/14] livepatch: Do not enforce ELF_LIVEPATCH_FUNC section presence Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 08/14] livepatch: always print XENLOG_ERR information Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 09/14] livepatch: Add per-function applied/reverted state tracking marker Pawel Wieczorkiewicz
2019-08-21 18:28   ` Konrad Rzeszutek Wilk
2019-08-21 19:00     ` Wieczorkiewicz, Pawel
2019-08-21 21:34   ` Julien Grall
2019-08-22  7:44     ` Wieczorkiewicz, Pawel
2019-08-22 10:07       ` Julien Grall
2019-08-22 10:20         ` Wieczorkiewicz, Pawel
2019-08-22 10:43           ` Julien Grall
2019-08-22 11:15             ` Wieczorkiewicz, Pawel
2019-08-22 15:02               ` Julien Grall
2019-08-22 10:29   ` Julien Grall
2019-08-22 11:02     ` Wieczorkiewicz, Pawel
2019-08-22 15:30       ` Julien Grall
2019-08-22 15:42         ` Wieczorkiewicz, Pawel
2019-08-21  8:19 ` [Xen-devel] [PATCH 10/14] livepatch: Add support for inline asm hotpatching expectations Pawel Wieczorkiewicz
2019-08-21 18:30   ` Konrad Rzeszutek Wilk
2019-08-21 19:02     ` Wieczorkiewicz, Pawel
2019-08-22 10:31   ` Julien Grall
2019-08-22 11:03     ` Wieczorkiewicz, Pawel
2019-08-21  8:19 ` [Xen-devel] [PATCH 11/14] livepatch: Add support for modules .modinfo section metadata Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 12/14] livepatch: Handle arbitrary size names with the list operation Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 13/14] livepatch: Add metadata runtime retrieval mechanism Pawel Wieczorkiewicz
2019-08-21  8:19 ` [Xen-devel] [PATCH 14/14] livepatch: Add python bindings for livepatch operations Pawel Wieczorkiewicz
2019-08-22 21:55   ` Marek Marczykowski-Górecki
2019-08-27  8:46 ` [Xen-devel] [PATCH v2 00/12] livepatch: new features and fixes Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 01/12] livepatch: Always check hypervisor build ID upon hotpatch upload Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 02/12] livepatch: Allow to override inter-modules buildid dependency Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 03/12] livepatch: Export payload structure via livepatch_payload.h Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 04/12] livepatch: Implement pre-|post- apply|revert hooks Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 05/12] livepatch: Add support for apply|revert action replacement hooks Pawel Wieczorkiewicz
2019-08-27 16:58     ` Konrad Rzeszutek Wilk
2019-08-28  7:37       ` Wieczorkiewicz, Pawel
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 06/12] livepatch: Do not enforce ELF_LIVEPATCH_FUNC section presence Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 07/12] livepatch: Add per-function applied/reverted state tracking marker Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 08/12] livepatch: Add support for inline asm hotpatching expectations Pawel Wieczorkiewicz
2019-08-29 14:34     ` Konrad Rzeszutek Wilk
2019-08-29 15:29       ` Wieczorkiewicz, Pawel
2019-08-29 15:58     ` Konrad Rzeszutek Wilk
2019-08-29 16:16       ` Wieczorkiewicz, Pawel
2019-08-29 17:49         ` Konrad Rzeszutek Wilk
2019-08-29 19:07           ` Wieczorkiewicz, Pawel
2019-08-29 20:48             ` Konrad Rzeszutek Wilk
2019-09-05 18:05     ` Konrad Rzeszutek Wilk
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 09/12] livepatch: Add support for modules .modinfo section metadata Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 10/12] livepatch: Handle arbitrary size names with the list operation Pawel Wieczorkiewicz
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 11/12] livepatch: Add metadata runtime retrieval mechanism Pawel Wieczorkiewicz
2019-08-29 20:48     ` Konrad Rzeszutek Wilk
2019-08-27  8:46   ` [Xen-devel] [PATCH v2 12/12] livepatch: Add python bindings for livepatch operations Pawel Wieczorkiewicz
2019-08-28 13:21     ` Marek Marczykowski-Górecki
2019-08-29 19:23   ` [Xen-devel] [PATCH v2 00/12] livepatch: new features and fixes Konrad Rzeszutek Wilk
2019-09-05 19:13   ` Konrad Rzeszutek Wilk
2019-09-06 22:52     ` Julien Grall
2019-09-06 22:42   ` Julien Grall

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190821081931.90887-2-wipawel@amazon.de \
    --to=wipawel@amazon.de \
    --cc=George.Dunlap@eu.citrix.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=ian.jackson@eu.citrix.com \
    --cc=jbeulich@suse.com \
    --cc=julien.grall@arm.com \
    --cc=konrad.wilk@oracle.com \
    --cc=mpohlack@amazon.com \
    --cc=ross.lagerwall@citrix.com \
    --cc=sstabellini@kernel.org \
    --cc=tim@xen.org \
    --cc=wipawel@amazon.com \
    --cc=wl@xen.org \
    --cc=xen-devel@lists.xen.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).