linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Sean Christopherson <seanjc@google.com>
To: Brijesh Singh <brijesh.singh@amd.com>
Cc: Dave Hansen <dave.hansen@intel.com>,
	Peter Gonda <pgonda@google.com>,
	x86@kernel.org, linux-kernel@vger.kernel.org,
	kvm@vger.kernel.org, linux-coco@lists.linux.dev,
	linux-mm@kvack.org, linux-crypto@vger.kernel.org,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Joerg Roedel <jroedel@suse.de>,
	Tom Lendacky <Thomas.Lendacky@amd.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Ard Biesheuvel <ardb@kernel.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>,
	Andy Lutomirski <luto@kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Sergio Lopez <slp@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>,
	David Rientjes <rientjes@google.com>,
	Dov Murik <dovmurik@linux.ibm.com>,
	Tobin Feldman-Fitzthum <tobin@ibm.com>,
	Borislav Petkov <bp@alien8.de>,
	Michael Roth <michael.roth@amd.com>,
	Vlastimil Babka <vbabka@suse.cz>,
	"Kirill A . Shutemov" <kirill@shutemov.name>,
	Andi Kleen <ak@linux.intel.com>,
	tony.luck@intel.com, marcorr@google.com,
	sathyanarayanan.kuppuswamy@linux.intel.com
Subject: Re: [PATCH Part2 v5 00/45] Add AMD Secure Nested Paging (SEV-SNP) Hypervisor Support
Date: Mon, 22 Nov 2021 21:34:06 +0000	[thread overview]
Message-ID: <YZwMzpx8422JiJTS@google.com> (raw)
In-Reply-To: <d83e6668-bec4-8d1f-7f8a-085829146846@amd.com>

On Mon, Nov 22, 2021, Brijesh Singh wrote:
> 
> On 11/22/21 1:14 PM, Dave Hansen wrote:
> > On 11/22/21 11:06 AM, Brijesh Singh wrote:
> > > > 3. Kernel accesses guest private memory via a kernel mapping.  This one
> > > >     is tricky.  These probably *do* result in a panic() today, but
> > > >     ideally shouldn't.
> > > KVM has defined some helper functions to maps and unmap the guest pages.
> > > Those helper functions do the GPA to PFN lookup before calling the
> > > kmap(). Those helpers are enhanced such that it check the RMP table
> > > before the kmap() and acquire a lock to prevent a page state change
> > > until the kunmap() is called. So, in the current implementation, we
> > > should *not* see a panic() unless there is a KVM driver bug that didn't
> > > use the helper functions or a bug in the helper function itself.
> > 
> > I don't think this is really KVM specific.
> > 
> > Think of a remote process doing ptrace(PTRACE_POKEUSER) or pretty much
> > any generic get_user_pages() instance.  As long as the memory is mapped
> > into the page tables, you're exposed to users that walk the page tables.
> > 
> > How do we, for example, prevent ptrace() from inducing a panic()?
> > 
> 
> In the current approach, this access will induce a panic(). In general,
> supporting the ptrace() for the encrypted VM region is going to be
> difficult.

But ptrace() is just an example, any path in the kernel that accesses a gup'd
page through a kernel mapping will explode if handed a guest private page.

> The upcoming TDX work to unmap the guest memory region from the current process
> page table can easily extend for the SNP to cover the current limitations.

That represents an ABI change though.  If KVM allows userspace to create SNP guests
without any guarantees that userspace cannot coerce the kernel into accessing guest
private memory, then we are stuck supporting that behavior even if KVM later gains
the ability to provide such guarantees through new APIs.

If allowing this behavior was only a matter of the system admin opting into a
dangerous configuration, I would probably be ok merging SNP with it buried behind
EXPERT or something scarier, but this impacts KVM's ABI as well as kernel internals,
e.g. the hooks in kvm_vcpu_map() and friends are unnecessary if KVM can differentiate
between shared and private gfns in its memslots, as gfn_to_pfn() will either fail or
point at memory that is guaranteed to be in the shared state.

  reply	other threads:[~2021-11-22 21:34 UTC|newest]

Thread overview: 239+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-20 15:58 [PATCH Part2 v5 00/45] Add AMD Secure Nested Paging (SEV-SNP) Hypervisor Support Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 01/45] x86/cpufeatures: Add SEV-SNP CPU feature Brijesh Singh
2021-09-16 16:56   ` Borislav Petkov
2021-09-16 17:35     ` Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 02/45] iommu/amd: Introduce function to check SEV-SNP support Brijesh Singh
2021-09-16 17:26   ` Borislav Petkov
2021-08-20 15:58 ` [PATCH Part2 v5 03/45] x86/sev: Add the host SEV-SNP initialization support Brijesh Singh
2021-09-24  8:58   ` Borislav Petkov
2021-08-20 15:58 ` [PATCH Part2 v5 04/45] x86/sev: Add RMP entry lookup helpers Brijesh Singh
2021-09-24  9:49   ` Borislav Petkov
2021-09-27 16:01     ` Brijesh Singh
2021-09-27 16:04       ` Brijesh Singh
2021-09-29 12:56         ` Borislav Petkov
2022-06-02 11:57   ` Jarkko Sakkinen
2021-08-20 15:58 ` [PATCH Part2 v5 05/45] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction Brijesh Singh
2021-09-24 14:04   ` Borislav Petkov
2021-09-27 16:06     ` Brijesh Singh
2021-10-15 18:05   ` Sean Christopherson
2021-10-15 20:18     ` Brijesh Singh
2021-10-15 20:27       ` Sean Christopherson
2021-10-15 20:36         ` Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 06/45] x86/sev: Invalid pages from direct map when adding it to RMP table Brijesh Singh
2021-09-29 14:34   ` Borislav Petkov
2021-09-30 16:19     ` Brijesh Singh
2021-10-01 11:06       ` Borislav Petkov
2021-08-20 15:58 ` [PATCH Part2 v5 07/45] x86/traps: Define RMP violation #PF error code Brijesh Singh
2021-09-29 17:25   ` Borislav Petkov
2021-08-20 15:58 ` [PATCH Part2 v5 08/45] x86/fault: Add support to handle the RMP fault for user address Brijesh Singh
2021-08-23 14:20   ` Dave Hansen
2021-08-23 14:36     ` Brijesh Singh
2021-08-23 14:50       ` Dave Hansen
2021-08-24 16:42         ` Joerg Roedel
2021-08-25  9:16           ` Vlastimil Babka
2021-08-25 13:50             ` Tom Lendacky
2021-09-29 18:19   ` Borislav Petkov
2021-08-20 15:58 ` [PATCH Part2 v5 09/45] x86/fault: Add support to dump RMP entry on fault Brijesh Singh
2021-09-29 18:38   ` Borislav Petkov
2021-08-20 15:58 ` [PATCH Part2 v5 10/45] crypto: ccp: shutdown SEV firmware on kexec Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 11/45] crypto:ccp: Define the SEV-SNP commands Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 12/45] crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 13/45] crypto:ccp: Provide APIs to issue SEV-SNP commands Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 14/45] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled Brijesh Singh
2022-02-25 18:03   ` Alper Gun
2022-03-01 14:12     ` Brijesh Singh
2022-06-14  0:10   ` Alper Gun
2021-08-20 15:58 ` [PATCH Part2 v5 15/45] crypto: ccp: Handle the legacy SEV command " Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 16/45] crypto: ccp: Add the SNP_PLATFORM_STATUS command Brijesh Singh
2021-09-10  3:18   ` Marc Orr
2021-09-13 11:17     ` Brijesh Singh
2021-09-22 17:35   ` Dr. David Alan Gilbert
2021-09-23 18:01     ` Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 17/45] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command Brijesh Singh
2021-09-01 21:02   ` Connor Kuehl
2021-09-01 23:06     ` Brijesh Singh
2021-09-10  3:27   ` Marc Orr
2021-09-13 11:29     ` Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 18/45] crypto: ccp: Provide APIs to query extended attestation report Brijesh Singh
2021-09-10  3:30   ` Marc Orr
2021-09-12  7:46     ` Dov Murik
2021-08-20 15:58 ` [PATCH Part2 v5 19/45] KVM: SVM: Add support to handle AP reset MSR protocol Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 20/45] KVM: SVM: Provide the Hypervisor Feature support VMGEXIT Brijesh Singh
2021-10-12 20:38   ` Sean Christopherson
2021-08-20 15:58 ` [PATCH Part2 v5 21/45] KVM: SVM: Make AVIC backing, VMSA and VMCB memory allocation SNP safe Brijesh Singh
2021-09-22 18:55   ` Dr. David Alan Gilbert
2021-09-23 18:09     ` Brijesh Singh
2021-09-23 18:39       ` Dr. David Alan Gilbert
2021-09-23 22:23         ` Brijesh Singh
2021-09-23 19:17       ` Marc Orr
2021-09-23 20:44         ` Brijesh Singh
2021-09-23 20:55           ` Marc Orr
2021-10-12 20:44   ` Sean Christopherson
2021-08-20 15:58 ` [PATCH Part2 v5 22/45] KVM: SVM: Add initial SEV-SNP support Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 23/45] KVM: SVM: Add KVM_SNP_INIT command Brijesh Singh
2021-09-05  6:56   ` Dov Murik
2021-09-05 13:59     ` Brijesh Singh
2021-09-10  3:32   ` Marc Orr
2021-09-13 11:32     ` Brijesh Singh
2021-09-16 15:50   ` Peter Gonda
2022-06-13 20:58   ` Alper Gun
2022-06-13 23:15     ` Ashish Kalra
2022-06-13 23:33       ` Alper Gun
2022-06-14  0:21         ` Ashish Kalra
2022-06-14 15:37           ` Peter Gonda
2022-06-14 16:11             ` Kalra, Ashish
2022-06-14 16:30               ` Peter Gonda
2022-06-14 17:16                 ` Kalra, Ashish
2022-06-14 18:58                   ` Alper Gun
2022-06-14 20:23                     ` Kalra, Ashish
2022-06-14 20:29                       ` Peter Gonda
2022-06-14 20:39                         ` Kalra, Ashish
2021-08-20 15:58 ` [PATCH Part2 v5 24/45] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command Brijesh Singh
2021-08-20 15:58 ` [PATCH Part2 v5 25/45] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command Brijesh Singh
2021-09-27 16:43   ` Peter Gonda
2021-09-27 19:33     ` Brijesh Singh
2021-10-05 15:01       ` Peter Gonda
2021-08-20 15:58 ` [PATCH Part2 v5 26/45] KVM: SVM: Mark the private vma unmerable for SEV-SNP guests Brijesh Singh
2021-09-23 17:18   ` Dr. David Alan Gilbert
2021-10-12 18:46   ` Sean Christopherson
2021-10-13 12:39     ` Brijesh Singh
2021-10-13 14:34       ` Sean Christopherson
2021-10-13 14:51         ` Brijesh Singh
2021-10-13 15:33           ` Sean Christopherson
2021-08-20 15:59 ` [PATCH Part2 v5 27/45] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command Brijesh Singh
2022-05-18 20:21   ` Marc Orr
2022-05-18 20:35     ` Kalra, Ashish
2021-08-20 15:59 ` [PATCH Part2 v5 28/45] KVM: X86: Keep the NPT and RMP page level in sync Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 29/45] KVM: x86/mmu: Move 'pfn' variable to caller of direct_page_fault() Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 30/45] KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX and SNP Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 31/45] KVM: x86: Introduce kvm_mmu_get_tdp_walk() for SEV-SNP use Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 32/45] KVM: x86: Define RMP page fault error bits for #NPF Brijesh Singh
2021-09-30 23:41   ` Marc Orr
2021-10-01 13:03     ` Borislav Petkov
2021-08-20 15:59 ` [PATCH Part2 v5 33/45] KVM: x86: Update page-fault trace to log full 64-bit error code Brijesh Singh
2021-10-13 21:23   ` Sean Christopherson
2021-08-20 15:59 ` [PATCH Part2 v5 34/45] KVM: SVM: Do not use long-lived GHCB map while setting scratch area Brijesh Singh
2021-10-13 21:20   ` Sean Christopherson
2021-10-15 16:11     ` Brijesh Singh
2021-10-15 16:44       ` Sean Christopherson
2021-08-20 15:59 ` [PATCH Part2 v5 35/45] KVM: SVM: Remove the long-lived GHCB host map Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 36/45] KVM: SVM: Add support to handle GHCB GPA register VMGEXIT Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 37/45] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT Brijesh Singh
2021-09-28  9:56   ` Dr. David Alan Gilbert
2021-10-12 21:48   ` Sean Christopherson
2021-10-13 17:04     ` Sean Christopherson
2021-10-13 17:05     ` Brijesh Singh
2021-10-13 17:24       ` Sean Christopherson
2021-10-13 17:49         ` Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 38/45] KVM: SVM: Add support to handle " Brijesh Singh
2021-09-28 10:17   ` Dr. David Alan Gilbert
2021-09-28 23:20     ` Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 39/45] KVM: SVM: Introduce ops for the post gfn map and unmap Brijesh Singh
2021-10-13  0:23   ` Sean Christopherson
2021-10-13 18:10     ` Brijesh Singh
2021-10-13 20:10       ` Sean Christopherson
2021-10-13 21:49         ` Brijesh Singh
2021-10-13 22:10           ` Sean Christopherson
2021-10-13 22:31             ` Brijesh Singh
2021-10-13 20:16     ` Sean Christopherson
2021-10-15 16:31       ` Brijesh Singh
2021-10-15 17:16         ` Sean Christopherson
2022-09-08 21:21           ` Michael Roth
2022-09-08 22:28             ` Michael Roth
2022-09-14  8:05             ` Sean Christopherson
2022-09-14 11:02               ` Marc Orr
2022-09-14 16:15                 ` Sean Christopherson
2022-09-14 16:32                   ` Marc Orr
2022-09-14 16:39                     ` Marc Orr
2022-09-19 17:56               ` Michael Roth
2021-08-20 15:59 ` [PATCH Part2 v5 40/45] KVM: x86: Export the kvm_zap_gfn_range() for the SNP use Brijesh Singh
2021-08-20 15:59 ` [PATCH Part2 v5 41/45] KVM: SVM: Add support to handle the RMP nested page fault Brijesh Singh
2021-09-29 12:24   ` Dr. David Alan Gilbert
2021-10-13 17:57   ` Sean Christopherson
2021-08-20 15:59 ` [PATCH Part2 v5 42/45] KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event Brijesh Singh
2021-09-29 21:33   ` Peter Gonda
2021-09-29 22:00   ` Peter Gonda
2021-08-20 15:59 ` [PATCH Part2 v5 43/45] KVM: SVM: Use a VMSA physical address variable for populating VMCB Brijesh Singh
2021-10-15 18:58   ` Sean Christopherson
2021-08-20 15:59 ` [PATCH Part2 v5 44/45] KVM: SVM: Support SEV-SNP AP Creation NAE event Brijesh Singh
2021-10-15 19:50   ` Sean Christopherson
2021-10-20 21:48     ` Brijesh Singh
2021-10-20 23:01       ` Sean Christopherson
2021-08-20 15:59 ` [PATCH Part2 v5 45/45] KVM: SVM: Add module parameter to enable the SEV-SNP Brijesh Singh
2021-11-12 15:43 ` [PATCH Part2 v5 00/45] Add AMD Secure Nested Paging (SEV-SNP) Hypervisor Support Peter Gonda
2021-11-12 17:59   ` Dave Hansen
2021-11-12 18:35     ` Borislav Petkov
2021-11-12 19:48       ` Sean Christopherson
2021-11-12 20:04         ` Borislav Petkov
2021-11-12 20:37           ` Sean Christopherson
2021-11-12 20:53             ` Borislav Petkov
2021-11-12 21:12               ` Peter Gonda
2021-11-12 21:20                 ` Andy Lutomirski
2021-11-12 22:04                   ` Borislav Petkov
2021-11-12 22:52                   ` Peter Gonda
2021-11-13  0:00                 ` Sean Christopherson
2021-11-13  0:10                   ` Marc Orr
2021-11-13 18:34                     ` Sean Christopherson
2021-11-14  7:54                       ` Marc Orr
2021-11-15 17:16                         ` Sean Christopherson
2021-11-15 16:36                       ` Joerg Roedel
2021-11-15 17:25                         ` Sean Christopherson
2021-11-12 21:30             ` Marc Orr
2021-11-12 21:37               ` Dave Hansen
2021-11-12 21:40                 ` Marc Orr
2021-11-12 21:39               ` Andy Lutomirski
2021-11-12 21:43                 ` Marc Orr
2021-11-12 22:54                   ` Peter Gonda
2021-11-13  0:53                     ` Sean Christopherson
2021-11-13  1:04                       ` Marc Orr
2021-11-13 18:28                         ` Sean Christopherson
2021-11-14  7:41                           ` Marc Orr
2021-11-15 18:17                             ` Sean Christopherson
2021-11-15 16:52                           ` Joerg Roedel
2021-11-15 16:18             ` Brijesh Singh
2021-11-15 18:44               ` Sean Christopherson
2021-11-15 18:58                 ` Brijesh Singh
2021-11-12 21:16         ` Marc Orr
2021-11-12 21:23           ` Andy Lutomirski
2021-11-12 21:35             ` Borislav Petkov
2021-11-15 12:30         ` Dr. David Alan Gilbert
2021-11-15 14:42           ` Joerg Roedel
2021-11-15 15:33             ` Dr. David Alan Gilbert
2021-11-15 16:20               ` Joerg Roedel
2021-11-15 16:32                 ` Dr. David Alan Gilbert
2021-11-15 18:26           ` Sean Christopherson
2021-11-15 18:41             ` Marc Orr
2021-11-15 19:15               ` Sean Christopherson
2021-11-16  3:07                 ` Marc Orr
2021-11-16  5:14                   ` Andy Lutomirski
2021-11-16 13:21                     ` Joerg Roedel
2021-11-16 18:26                       ` Sean Christopherson
2021-11-16 18:39                         ` Peter Gonda
2021-11-16 13:30                 ` Joerg Roedel
2021-11-16  5:00               ` Andy Lutomirski
2021-11-16 13:02             ` Joerg Roedel
2021-11-16 20:08               ` Sean Christopherson
2021-11-15 16:16         ` Joerg Roedel
2021-11-22 15:23   ` Brijesh Singh
2021-11-22 17:03     ` Vlastimil Babka
2021-11-22 18:01       ` Brijesh Singh
2021-11-22 18:30     ` Dave Hansen
2021-11-22 19:06       ` Brijesh Singh
2021-11-22 19:14         ` Dave Hansen
2021-11-22 20:33           ` Brijesh Singh
2021-11-22 21:34             ` Sean Christopherson [this message]
2021-11-22 22:51             ` Dave Hansen
2021-11-23  5:15               ` Luck, Tony
2021-11-23  7:18               ` Borislav Petkov
2021-11-23 15:36                 ` Sean Christopherson
2021-11-23 16:26                   ` Borislav Petkov
2021-11-23  8:55               ` Vlastimil Babka
2021-11-24 16:03               ` Joerg Roedel
2021-11-24 17:48                 ` Dave Hansen
2021-11-24 19:34                   ` Vlastimil Babka
2021-11-25 10:05                   ` Joerg Roedel
2021-11-29 14:44                     ` Brijesh Singh
2021-11-29 14:58                       ` Vlastimil Babka
2021-11-29 16:13                         ` Brijesh Singh
2021-11-30 19:40                           ` Vlastimil Babka
2021-11-29 16:41                     ` Dave Hansen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YZwMzpx8422JiJTS@google.com \
    --to=seanjc@google.com \
    --cc=Thomas.Lendacky@amd.com \
    --cc=ak@linux.intel.com \
    --cc=ardb@kernel.org \
    --cc=bp@alien8.de \
    --cc=brijesh.singh@amd.com \
    --cc=dave.hansen@intel.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=dovmurik@linux.ibm.com \
    --cc=hpa@zytor.com \
    --cc=jmattson@google.com \
    --cc=jroedel@suse.de \
    --cc=kirill@shutemov.name \
    --cc=kvm@vger.kernel.org \
    --cc=linux-coco@lists.linux.dev \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=luto@kernel.org \
    --cc=marcorr@google.com \
    --cc=michael.roth@amd.com \
    --cc=mingo@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=peterz@infradead.org \
    --cc=pgonda@google.com \
    --cc=rientjes@google.com \
    --cc=sathyanarayanan.kuppuswamy@linux.intel.com \
    --cc=slp@redhat.com \
    --cc=srinivas.pandruvada@linux.intel.com \
    --cc=tglx@linutronix.de \
    --cc=tobin@ibm.com \
    --cc=tony.luck@intel.com \
    --cc=vbabka@suse.cz \
    --cc=vkuznets@redhat.com \
    --cc=wanpengli@tencent.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).