From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AABEC4320A for ; Fri, 20 Aug 2021 06:36:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68BC360BD3 for ; Fri, 20 Aug 2021 06:36:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238573AbhHTGgg (ORCPT ); Fri, 20 Aug 2021 02:36:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238318AbhHTGge (ORCPT ); Fri, 20 Aug 2021 02:36:34 -0400 Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6DC1DC061575 for ; Thu, 19 Aug 2021 23:35:57 -0700 (PDT) Received: by mail-qt1-x834.google.com with SMTP id e3so509489qth.9 for ; Thu, 19 Aug 2021 23:35:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=x4BUx2XF2g+eANj8K3DiizaXKH8rES+7+IBJDMT/M84=; b=fEKmCkmPW2WRSTH0UOBZIggTO3VzSvZceyA2rDxv0GHM/eIW9xyTtwRGG/IG/RrvHQ RBDQ3PojrK85BTJLS06A3fsNsYc5EVQtSPEq9AzUDWDIvX6a0h16icO9BtnIDDmt3BP3 U/6YCDEnnYBoCiSP8ZOF5TFvwLEzE38eCQ2Srvh3Xtik7z8B+nPZUESn81VTYA1NWtyh cf/Ps0dsmzJ3hwsXf1UUDYyOP13BVC5tGK59tTydrAvmHV+93tpyzOPdKT9yYT3UpP6a Kq2TMZY1/YwVAivFFABqh2kp+D4Bvqecwmlhx2/MFzKwkba0wXFURlimYWn0vDLbsTIL ALNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=x4BUx2XF2g+eANj8K3DiizaXKH8rES+7+IBJDMT/M84=; b=WOJqTY34YdxGPGcDGEbCYIBxsy90uZE0Q1qsD6pGcBCUIXO98iYU7Au2g6Ru5SBORH DEm6j6ju199D3lzZFS41RBvQ3lXpRw8X4PtNALu7UHH4amephqI2MuWwMzUJ3ffYqur5 dmdmJF69iR+WM2vJoVlO5f5sTzr8zbaYEcs69mDuEEkc8rHi+fK//wDCmsyn3qZkGLrC Z6f53YO7P4ORXGBmrjEr8dM4O4goVF6FcEGUWVZXOL3x5W5MRp+/uylTtpon8zSq8I1q z9sYiygaQZp6HeAdHOs1dblN/GLcAys00R0y5W15QFp5MiJ3T5kUxbIOnWyUILmbAQnS R7IQ== X-Gm-Message-State: AOAM531D9gWcrHpCGkj87ktmtd4J7mMVknSTrbPXZNHkJsIaH9LEP07e BmImlE+kY+atp2EjHcq8+BRVpwBK9u7GIc1Sj6pTaw== X-Google-Smtp-Source: ABdhPJzszjsxOcKw23gqj3JYe++tO8Il5Wrqncm/dBgLF58wQcrlK8qhJO/51V6bt9Fg98a0gcw497JTpdQHs5q+b14= X-Received: by 2002:ac8:7154:: with SMTP id h20mr16540010qtp.251.1629441356449; Thu, 19 Aug 2021 23:35:56 -0700 (PDT) MIME-Version: 1.0 References: <20210819154910.1064090-1-pgonda@google.com> <20210819154910.1064090-2-pgonda@google.com> In-Reply-To: From: Marc Orr Date: Thu, 19 Aug 2021 23:35:45 -0700 Message-ID: Subject: Re: [PATCH 1/2 V4] KVM, SEV: Add support for SEV intra host migration To: Sean Christopherson Cc: Peter Gonda , kvm list , Paolo Bonzini , David Rientjes , "Dr . David Alan Gilbert" , Brijesh Singh , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Aug 19, 2021 at 3:58 PM Sean Christopherson wrote: > > On Thu, Aug 19, 2021, Peter Gonda wrote: > > > > > > > > +static int svm_sev_lock_for_migration(struct kvm *kvm) > > > > +{ > > > > + struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; > > > > + int ret; > > > > + > > > > + /* > > > > + * Bail if this VM is already involved in a migration to avoid deadlock > > > > + * between two VMs trying to migrate to/from each other. > > > > + */ > > > > + spin_lock(&sev->migration_lock); > > > > + if (sev->migration_in_progress) > > > > + ret = -EBUSY; > > > > + else { > > > > + /* > > > > + * Otherwise indicate VM is migrating and take the KVM lock. > > > > + */ > > > > + sev->migration_in_progress = true; > > > > + mutex_lock(&kvm->lock); > > Deadlock aside, mutex_lock() can sleep, which is not allowed while holding a > spinlock, i.e. this patch does not work. That's my suggestion did the crazy > dance of "acquiring" a flag. > > What I don't know is why on earth I suggested a global spinlock, a simple atomic > should work, e.g. > > if (atomic_cmpxchg_acquire(&sev->migration_in_progress, 0, 1)) > return -EBUSY; > > mutex_lock(&kvm->lock); > > and on the backend... > > mutex_unlock(&kvm->lock); > > atomic_set_release(&sev->migration_in_progress, 0); > > > > > + ret = 0; > > > > + } > > > > + spin_unlock(&sev->migration_lock); > > > > + > > > > + return ret; > > > > +} > > > > + > > > > +static void svm_unlock_after_migration(struct kvm *kvm) > > > > +{ > > > > + struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info; > > > > + > > > > + mutex_unlock(&kvm->lock); > > > > + WRITE_ONCE(sev->migration_in_progress, false); > > > > +} > > > > + > > > > > > This entire locking scheme seems over-complicated to me. Can we simply > > > rely on `migration_lock` and get rid of `migration_in_progress`? I was > > > chatting about these patches with Peter, while he worked on this new > > > version. But he mentioned that this locking scheme had been suggested > > > by Sean in a previous review. Sean: what do you think? My rationale > > > was that this is called via a VM-level ioctl. So serializing the > > > entire code path on `migration_lock` seems fine. But maybe I'm missing > > > something? > > > > > > Marc I think that only having the spin lock could result in > > deadlocking. If userspace double migrated 2 VMs, A and B for > > discussion, A could grab VM_A.spin_lock then VM_A.kvm_mutex. Meanwhile > > B could grab VM_B.spin_lock and VM_B.kvm_mutex. Then A attempts to > > grab VM_B.spin_lock and we have a deadlock. If the same happens with > > the proposed scheme when A attempts to lock B, VM_B.spin_lock will be > > open but the bool will mark the VM under migration so A will unlock > > and bail. Sean originally proposed a global spin lock but I thought a > > per kvm_sev_info struct would also be safe. > > Close. The issue is taking kvm->lock from both VM_A and VM_B. If userspace > double migrates we'll end up with lock ordering A->B and B-A, so we need a way > to guarantee one of those wins. My proposed solution is to use a flag as a sort > of one-off "try lock" to detect a mean userspace. Got it now. Thanks to you both, for the explanation. By the way, just to make sure I completely follow, I assume that if a "double migration" occurs, then user space is mis-behaving -- correct? But presumably, we need to reason about how to respond to such mis-behavior so that buggy or malicious user-space code cannot stumble over/exploit this scenario?