From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AC13C433FE for ; Mon, 21 Nov 2022 13:59:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231602AbiKUN7b (ORCPT ); Mon, 21 Nov 2022 08:59:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231462AbiKUN7F (ORCPT ); Mon, 21 Nov 2022 08:59:05 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCB9517071 for ; Mon, 21 Nov 2022 05:57:43 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id s5so2771207wru.1 for ; Mon, 21 Nov 2022 05:57:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=iF/t8apUyWuhDIryZyYBVw69J04guPwa/Q1m0PGtgj4=; b=NTH1gFTQhD35MwNnKxHma+Dne19TTBr9hsuIMfgWyipYlc+hQpXP0YHxMy7h8Gc9/P 0f25YPA/9D37uxZ1XTs2rIwDzIlXHO27c7qFKIEdA4pT2otaDRECGXrh0ynhMkm/7lQI NlLhp7csumAx4/4Dt0zetWAyTt+6qhR2oVrm9cmLg85/agpLojhCQA+M/Sybnh5wdfSE 5Bo0BIxNpf6aGi0bv2Q+oF14l67NmrgFMkRtwD81OXBwkpAC819yFFFUwy2dIZlilt47 VlmoWs5ZGzAxPsLy958luCizhuz0UKuMOh7xZynKuj11VHOVWbrDj+uwE4LRi8DQewCd VmUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=iF/t8apUyWuhDIryZyYBVw69J04guPwa/Q1m0PGtgj4=; b=HOzp712R1Gj3tIsOhY1QBNlsaOJz8GY3LIkmdIrFUSHZK9ynGP05GMLeI9cGDMINWQ LXSlhcojyYKjuD6lKoPPnocQ4vPjrPMY3olPezDPDCkGou2Mmbn/oOz3xUI7xLZqLDa9 sj546HNswD3uyZ2N9ZS6di+GkATXcYt6hIqx2rD1iS667ujmj1vSf0SKKmjab+Uu+061 Rb+grkuZBph7Tk4riefAbcz37E4l/2yvYDfFxGgZK+6cPLwMuJ6LppU7JeDh1OAIG3Om wUvsKNvmZTo0s5ShYDqfz3fkc3JQ1HbiZMa6Sl67WpPUbbVEmPWZRMLNHvlEI8BiwEF9 2DGA== X-Gm-Message-State: ANoB5pmDfkCKkkh6tyH8p7tDTXj3bFAjOeZgp5gDEntPD5HhHTeKJmlx iY7jLgkYRZyFqF+iFtHr48AWQv4s8HhTfmMMPVQJcw== X-Google-Smtp-Source: AA0mqf61vjT7gCdQcTsAcZAYE1/8Ro4EWqaK2HE4/5KcS+UqmBGZszawiFOhM+JSqSfT37FAFwbLoGXKdjqDAzCV9K0= X-Received: by 2002:a5d:4247:0:b0:241:a82b:5dee with SMTP id s7-20020a5d4247000000b00241a82b5deemr533360wrr.425.1669039062122; Mon, 21 Nov 2022 05:57:42 -0800 (PST) MIME-Version: 1.0 References: <3b2f6267-e7a0-4266-867d-b0109d5a7cb4@acm.org> In-Reply-To: From: Haris Iqbal Date: Mon, 21 Nov 2022 14:57:31 +0100 Message-ID: Subject: Re: [RFC] Reliable Multicast on top of RTRS To: Bart Van Assche , linux-rdma@vger.kernel.org, linux-block@vger.kernel.org, Aleksei Marov Cc: danil.kipnis@posteo.net, Jinpu Wang Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Hey there, We've got a prototype version working in-house. We've also implemented a block device client and server. Another client and server could probably be rbd and dm-thin. Will update as soon as we have a github link. Some technical details - Ability to perform sync across storage nodes without the involvement of the client. - This helps performing sync, extending/adding legs/members without the help of the client. Candidate users - We've implemented a stand-alone replicating block device. The client is similar to the rnbd-clt and our own corresponding "store"/server does linear mapping on the server side. - rbd could be a client of rmr-clt. The rmr-srv (store) would talk to lvm. rbd would provide the block device (over multiple objects) on the client side. Lvm would function as the store on the server side. One object would be stored on one dm-thin volume. rmr would provide for the replication in the network. Setups: RMR vs MD-RAID vs DRBD - Active-active - RMR as means of replication over network differs from the md-raid configuration because sync traffic goes directly between servers. The difference to the drbd setup is that the IO traffic goes to both legs in a single hop. How does RMR solve the activity log issue - Synchronous replication; much like Protocol C of DRBD. - RMR tracks all successful queue_depth (max number of IOs that can be inflight at any moment) worth of last IOs on each storage node. Best, Haris Signed-off: alexv.marov@gmail.com Reviewed-by: danil.kipnis@posteo.net On Sun, Nov 22, 2020 at 5:20 PM Danil Kipnis wrote: > > On Fri, Sep 4, 2020 at 5:33 PM Bart Van Assche wrote: > > > > On 2020-09-04 04:35, Danil Kipnis wrote: > > > On Thu, Sep 3, 2020 at 1:07 AM Bart Van Assche wrote: > > >> How will it be guaranteed that the resulting software does > > >> not suffer from the problems that have been solved by the introduction > > >> of the DRBD activity log > > >> (https://www.linbit.com/drbd-user-guide/users-guide-drbd-8-4/#s-activity-log)? > > > > > > The above would require some kind of activity log also, I'm afraid. > > > > How about collaborating with the DRBD team? My concern is that otherwise > > we will end up with two drivers in the kernel that implement block device > > replication between servers connected over a network. > > Will take a closer look at drbd, > > Thank you, > Danil. > > > > > Thanks, > > > > Bart.