From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C799C433E9 for ; Mon, 8 Feb 2021 20:38:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EBA3964E8C for ; Mon, 8 Feb 2021 20:38:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233766AbhBHUiE (ORCPT ); Mon, 8 Feb 2021 15:38:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235102AbhBHTTq (ORCPT ); Mon, 8 Feb 2021 14:19:46 -0500 Received: from mail-il1-x132.google.com (mail-il1-x132.google.com [IPv6:2607:f8b0:4864:20::132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FBC8C061786 for ; Mon, 8 Feb 2021 11:19:05 -0800 (PST) Received: by mail-il1-x132.google.com with SMTP id p15so13819843ilq.8 for ; Mon, 08 Feb 2021 11:19:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juliacomputing.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qCfnLupVibojf6p+QXa/fy+jh7kBw6RuWeVeUdTLXwE=; b=HlSddigbrczVGSDywCR6eQKGHQukIBzWukt9DJWNVuShmOKJuD68leLbzUJr8L6wyv sAL8cMNVIwsZUy4iZy8L3FZNPfaYU73Vap1Zf9+1cI6yC0DIrJSlHYvBwEOZCG3BH36F KBfePGV09pLgg1LKYkyWfvVxYcbhv5asBuUvtDJp9Lrs4LmuQX2GJ/TlmyPkq7cK6gTk F+n9rgD6IqbMOqM/2ywib2WehuBY8PMwMB47VotdKDNjYl+x6YtLt5h3pgO9fJ2TzXRQ RVKe44UQIjtJGGMn3x6+2p66HCRvuueCyWjLr5DFqahiGg1IEnOLaKHV9jovAZnuSJ55 fWvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qCfnLupVibojf6p+QXa/fy+jh7kBw6RuWeVeUdTLXwE=; b=c2BIN6/QkLUMVY2WVj129lMsCydi86536LQ1jDadAWNq3c9SOMH9BUWqko7kkwxxdv vP88+RazIt4e/nzI19+xvq7iqev2Wh1AvLg0bsotRFQqYHghQn0A+sGFLSV8dC6J9EnN oQ2h3oJgJVTpBncyVo0yi1BsjqKLPDlLdNR5s3wxhMv7l0WNs2bCj/tlTdP3y+NMHOtP 6IC1tbSdBx0zt0epTrjwpUZNsjmjHD32eZU5K0qJ/Sp/YK3D2FAM9iwWvWTTCP/p3qSd iJE6vQjAE/orWfZR17+jhjLW5j4wXiEPpljU+Q9j5wU3jePbWm8xs7xSTmRN7r2gFD7o 4ZtA== X-Gm-Message-State: AOAM530683WqrCTswwRLXu9feXZs4J7eLhXlUY/89RuSPqfQPtr89YqU E3z8WA6XAebMsF9TpYjvQ1hX4o0CAsOUkh55wy9VrYF7KJAxBg== X-Google-Smtp-Source: ABdhPJxNZLD9lYS+fpPxInJSo2bM0a/KDWCemZds9jS3Kq78Mj/APwEqEMeSlaGMApriaXkdxnHlDCFBH8vE3iC8G4w= X-Received: by 2002:a05:6e02:1be6:: with SMTP id y6mr16174254ilv.145.1612811944425; Mon, 08 Feb 2021 11:19:04 -0800 (PST) MIME-Version: 1.0 References: <20210201194012.524831-1-avagin@gmail.com> <20210208183752.GB559391@gmail.com> In-Reply-To: <20210208183752.GB559391@gmail.com> From: Keno Fischer Date: Mon, 8 Feb 2021 14:18:28 -0500 Message-ID: Subject: Re: [PATCH 0/3 v2] arm64/ptrace: allow to get all registers on syscall traps To: Andrei Vagin Cc: Will Deacon , Catalin Marinas , Oleg Nesterov , linux-arm-kernel@lists.infradead.org, Linux Kernel Mailing List , linux-api@vger.kernel.org, Anthony Steinhauser , Dave Martin , Kyle Huey , "Robert O'Callahan" Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > > Could you describe the problem in more details? I wonder whether we have > the same thing in CRIU... > Sure, basically the issue is that orig_x0 gets recorded at the start of the syscall entry, but is not otherwise part of the ptrace state. It used to be primarily used for resetting the argument back to its original value during syscall restarts, but I see it's been expanded slightly with the user dispatch mechanism (though as far as I can tell not in a way that interacts with ptrace). Basically the problem is that if you change the value of x0 during either the ptrace entry stop or a seccomp stop and then incur a syscall restart, the syscall will restart with the original x0 rather than with the modified x0, which may be unexpected. Of course, relatedly, if you're doing CRIU-like things you can end up in situations where the future behavior will depend on the orig_x0 value, which isn't restore-able at the moment. It's possible to work around all of this by keeping a local copy of orig_x0 and being very careful with the ptrace traps around restarts, but getting the logic right is extremely tricky. My suggestion for what I thought would be reasonable behavior was: 1. Expose orig_x0 to ptrace 2. Set orig_x0 to x0 and set x0 to -ENOSYS at the start of the syscall dispatcher 3. Use orig_x0 for syscall arguments/seccomp/restarts That's basically how rax works on x86_64 and it doesn't seem to cause major problems (though of course I may be biased by having x86_64 already work when I started the aarch64 port). Just the first item would be sufficient of course for getting rid of most of the bookkeeping. I should also say that, for us, the ptrace getregs call can be the throughput limiting operation, so it would be nice if getting the entire basic register set would only require one syscall. I won't insist on it, since we do have a solution in place that kinda works (and only requires the one syscall), but I thought I'd mention it. While we're on this topic, and in case it's helpful to anybody, I should also point out that the order of the ptrace-signal-stop, vs setup for the syscall restart differs between x86 and aarch64 (aarch64 sets up the restart first then delivers the ptrace trap/signal - x86 the other way around). I actually think the aarch64 behavior is saner here, but I figured I'd leave this breadcrumb for anybody who's writing a ptracer and stumbles across this. Keno From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4011FC433E0 for ; Mon, 8 Feb 2021 19:20:22 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CE4A564E74 for ; Mon, 8 Feb 2021 19:20:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE4A564E74 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=juliacomputing.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vZtzVtDjt9MCBu+UA7n6gVMFpnS4m1kzXUUKe4Y84fA=; b=imtv+bwE7LsL9KWLkOsqFzb+n XS7BEyRz3g/qGcVUnb5FiS5Rh+gww/MXo89jgy3/wxvkkBddG9l5Rq5FVK02L7epNULUcSLbrd+QS YXxW6ZQvNYVhmNQgTPNbPv/MI/AyepStU4C5U4mH4G8iqYOaqvvPzwBikPkogElvCwYqCxXQe7l+w n+AndWkoPnFCpXQBuW4RabTM3GyghCH+xwiyyoXGREht6a839wMAe3scBVVc4A0A7UCKjYckxIEXf Y91RkubbEYV+Wi8oqblpCj2HuJWF8hBpmGmx04uRC4ldY/28eghfecBsZ5wr6C4OkjPy/4zV7ilI5 sJIpFse1g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l9C3y-0005Uy-R7; Mon, 08 Feb 2021 19:19:10 +0000 Received: from mail-il1-x12d.google.com ([2607:f8b0:4864:20::12d]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l9C3v-0005Sq-SK for linux-arm-kernel@lists.infradead.org; Mon, 08 Feb 2021 19:19:08 +0000 Received: by mail-il1-x12d.google.com with SMTP id y15so13821889ilj.11 for ; Mon, 08 Feb 2021 11:19:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juliacomputing.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qCfnLupVibojf6p+QXa/fy+jh7kBw6RuWeVeUdTLXwE=; b=HlSddigbrczVGSDywCR6eQKGHQukIBzWukt9DJWNVuShmOKJuD68leLbzUJr8L6wyv sAL8cMNVIwsZUy4iZy8L3FZNPfaYU73Vap1Zf9+1cI6yC0DIrJSlHYvBwEOZCG3BH36F KBfePGV09pLgg1LKYkyWfvVxYcbhv5asBuUvtDJp9Lrs4LmuQX2GJ/TlmyPkq7cK6gTk F+n9rgD6IqbMOqM/2ywib2WehuBY8PMwMB47VotdKDNjYl+x6YtLt5h3pgO9fJ2TzXRQ RVKe44UQIjtJGGMn3x6+2p66HCRvuueCyWjLr5DFqahiGg1IEnOLaKHV9jovAZnuSJ55 fWvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qCfnLupVibojf6p+QXa/fy+jh7kBw6RuWeVeUdTLXwE=; b=BNPeP5GsnbNLlYxES4gR3+v0nDoIU65ZpXdW2Nr3qgdu8ghA7/OHBLNmTWL1yjlsLg xMI8Rx/0kI7kpQdzSnR8ZKN+sGompGVGHbtjzbWJrxzvfXT1ut7yqGvwjYn+FlcQW6v2 Un2l3I7290B9PGrwES20RgRV3Be1z+eODrMP+n4q12V6K84kA0yhCgcDksOuG6Byc5t6 XlY69ebtUk0cOliJKZKIlJ6JI8jKaqB5NVER0psevGqCJO2HeZMME6c4RGY/HZhRni+w fAjmU3R471fUf33RGfCAZCQPsAd5MU6IsR8RpNSEN0rlq8lSl4sC8yFVfjtgqOGOoYr6 T1fw== X-Gm-Message-State: AOAM533CFsglYZp2onCp5oWGmeBLh7wWf4AMq8HwXcBPvlazgDsQjSRD 5+VSi0904UwiqH7w7p25TIq0DY7Xkmy5cdh87NB7WQ== X-Google-Smtp-Source: ABdhPJxNZLD9lYS+fpPxInJSo2bM0a/KDWCemZds9jS3Kq78Mj/APwEqEMeSlaGMApriaXkdxnHlDCFBH8vE3iC8G4w= X-Received: by 2002:a05:6e02:1be6:: with SMTP id y6mr16174254ilv.145.1612811944425; Mon, 08 Feb 2021 11:19:04 -0800 (PST) MIME-Version: 1.0 References: <20210201194012.524831-1-avagin@gmail.com> <20210208183752.GB559391@gmail.com> In-Reply-To: <20210208183752.GB559391@gmail.com> From: Keno Fischer Date: Mon, 8 Feb 2021 14:18:28 -0500 Message-ID: Subject: Re: [PATCH 0/3 v2] arm64/ptrace: allow to get all registers on syscall traps To: Andrei Vagin X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210208_141908_020969_0506562A X-CRM114-Status: GOOD ( 18.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kyle Huey , Anthony Steinhauser , Catalin Marinas , Oleg Nesterov , Linux Kernel Mailing List , Robert O'Callahan , linux-api@vger.kernel.org, Will Deacon , Dave Martin , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > > Could you describe the problem in more details? I wonder whether we have > the same thing in CRIU... > Sure, basically the issue is that orig_x0 gets recorded at the start of the syscall entry, but is not otherwise part of the ptrace state. It used to be primarily used for resetting the argument back to its original value during syscall restarts, but I see it's been expanded slightly with the user dispatch mechanism (though as far as I can tell not in a way that interacts with ptrace). Basically the problem is that if you change the value of x0 during either the ptrace entry stop or a seccomp stop and then incur a syscall restart, the syscall will restart with the original x0 rather than with the modified x0, which may be unexpected. Of course, relatedly, if you're doing CRIU-like things you can end up in situations where the future behavior will depend on the orig_x0 value, which isn't restore-able at the moment. It's possible to work around all of this by keeping a local copy of orig_x0 and being very careful with the ptrace traps around restarts, but getting the logic right is extremely tricky. My suggestion for what I thought would be reasonable behavior was: 1. Expose orig_x0 to ptrace 2. Set orig_x0 to x0 and set x0 to -ENOSYS at the start of the syscall dispatcher 3. Use orig_x0 for syscall arguments/seccomp/restarts That's basically how rax works on x86_64 and it doesn't seem to cause major problems (though of course I may be biased by having x86_64 already work when I started the aarch64 port). Just the first item would be sufficient of course for getting rid of most of the bookkeeping. I should also say that, for us, the ptrace getregs call can be the throughput limiting operation, so it would be nice if getting the entire basic register set would only require one syscall. I won't insist on it, since we do have a solution in place that kinda works (and only requires the one syscall), but I thought I'd mention it. While we're on this topic, and in case it's helpful to anybody, I should also point out that the order of the ptrace-signal-stop, vs setup for the syscall restart differs between x86 and aarch64 (aarch64 sets up the restart first then delivers the ptrace trap/signal - x86 the other way around). I actually think the aarch64 behavior is saner here, but I figured I'd leave this breadcrumb for anybody who's writing a ptracer and stumbles across this. Keno _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel