All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Christian König" <ckoenig.leichtzumerken@gmail.com>
To: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
	Daniel Vetter <daniel@ffwll.ch>
Cc: linaro-mm-sig@lists.linaro.org, dri-devel@lists.freedesktop.org,
	linux-media@vger.kernel.org, intel-gfx@lists.freedesktop.org
Subject: Re: [Intel-gfx] [PATCH 01/26] dma-buf: add dma_resv_for_each_fence_unlocked v2
Date: Mon, 20 Sep 2021 12:09:32 +0200	[thread overview]
Message-ID: <e4aa2e1e-753e-655c-423f-93a0bb853b9d@gmail.com> (raw)
In-Reply-To: <8268d2e8-8a37-0ff1-7065-c8aaf5c8672b@linux.intel.com>

Am 20.09.21 um 10:43 schrieb Tvrtko Ursulin:
> On 17/09/2021 14:23, Daniel Vetter wrote:
>> On Fri, Sep 17, 2021 at 02:34:48PM +0200, Christian König wrote:
>>> Abstract the complexity of iterating over all the fences
>>> in a dma_resv object.
>>>
>>> The new loop handles the whole RCU and retry dance and
>>> returns only fences where we can be sure we grabbed the
>>> right one.
>>>
>>> v2: fix accessing the shared fences while they might be freed,
>>>      improve kerneldoc, rename _cursor to _iter, add
>>>      dma_resv_iter_is_exclusive, add dma_resv_iter_begin/end
>>>
>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>> ---
>>>   drivers/dma-buf/dma-resv.c | 61 +++++++++++++++++++++++++++
>>>   include/linux/dma-resv.h   | 84 
>>> ++++++++++++++++++++++++++++++++++++++
>>>   2 files changed, 145 insertions(+)
>>>
>>> diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
>>> index 84fbe60629e3..3e77cad2c9d4 100644
>>> --- a/drivers/dma-buf/dma-resv.c
>>> +++ b/drivers/dma-buf/dma-resv.c
>>> @@ -323,6 +323,67 @@ void dma_resv_add_excl_fence(struct dma_resv 
>>> *obj, struct dma_fence *fence)
>>>   }
>>>   EXPORT_SYMBOL(dma_resv_add_excl_fence);
>>>   +/**
>>> + * dma_resv_iter_walk_unlocked - walk over fences in a dma_resv obj
>>> + * @cursor: cursor to record the current position
>>> + * @first: if we should start over
>>> + *
>>> + * Return all the fences in the dma_resv object which are not yet 
>>> signaled.
>>> + * The returned fence has an extra local reference so will stay alive.
>>> + * If a concurrent modify is detected the whole iterration is 
>>> started over again.
>>> + */
>>> +struct dma_fence *dma_resv_iter_walk_unlocked(struct dma_resv_iter 
>>> *cursor,
>>
>> Bit ocd, but I'd still just call that iter_next.
>>
>>> +                          bool first)
>>
>> Hm I'd put all the init code into iter_begin ...
>
> @Christian:
>
> Could you engineer something in here which would, at least in debug 
> builds, catch failures to call "iter begin" before using the iterator 
> macro?

Yeah, I've already played with the thought of somehow teaching lockdep 
that. But then abandoned this as abusive of lockdep.

>
>>
>>> +{
>>> +    struct dma_resv *obj = cursor->obj;
>>
>> Aren't we missing rcu_read_lock() around the entire thing here?
>>
>>> +
>>> +    first |= read_seqcount_retry(&obj->seq, cursor->seq);
>>> +    do {
>>> +        /* Drop the reference from the previous round */
>>> +        dma_fence_put(cursor->fence);
>>> +
>>> +        cursor->is_first = first;
>>> +        if (first) {
>>> +            cursor->seq = read_seqcount_begin(&obj->seq);
>>> +            cursor->index = -1;
>>> +            cursor->fences = dma_resv_shared_list(obj);
>>
>> And then also call iter_begin from here. That way we guarantee that
>> read_seqcount_begin is always called before _retry(). It's not a problem
>> with the seqcount implementation (I think at least), but it definitely
>> looks funny.
>>
>> Calling iter_begin here also makes it clear that we're essentially
>> restarting.
>>
>>> +
>>> +            cursor->fence = dma_resv_excl_fence(obj);
>>> +            if (cursor->fence &&
>>> +                test_bit(DMA_FENCE_FLAG_SIGNALED_BIT,
>>
>> Please use the right dma_fence wrapper here for this and don't look 
>> at the
>> bits/flags outside of dma_fence.[hc] code. I just realized that we don't
>> have the right amount of barriers in there for the fastpath, i.e. if we
>> have:
>>
>> x = 0; /* static initializer */
>>
>> thread a
>>     x = 1;
>>     dma_fence_signal(fence);
>>
>>
>> thread b;
>>     if (dma_fence_is_signalled(fence))
>>         printk("%i\n", x);
>>
>> Then you might actually be able to observe x == 0 in thread b. Which is
>> not what we want at all.
>
> @Daniel:
>
> What do you mean here - in terms of if 'x' is "external" (not part of 
> dma-fence), then are you suggesting dma-fence code should serialise it 
> by using barriers?
>
> That would sound incorrect to me, or in other words, I think it's fine 
> if x == 0 is observed in your example thread B since that code is 
> mixing external data with dma-fence.

No, Daniel is right. The problem is that on architectures other than x86 
barriers are per memory address (or rather cache line in practice).

So you need to be really careful that you see the fully consistent state 
and not just one variable but others in the old state.

But this was buggy before as well. I'm just pulling the existing test 
into the new iterator.

>
> Hm also, there is that annoying bit where by using 
> dma_fence_is_signaled any code becomes a fence signaling critical 
> path, which I never bought into. There should be a way to test the 
> signaled status without actually doing the signaling. Or I am 
> misunderstanding something so badly that is really really has to be 
> like this?

You are mixing things up. Testing is unproblematic, signaling is the 
problematic one.

>
>> So no open-coding of dma_fence flag bits code outside of drm_fence.[hc]
>> please. And yes i915-gem code is unfortunately a disaster.
>
> Don't even miss an opportunity for some good trashing no? :D
>
> But yes, deconstructed dma_fence_signal I thought we were supposed to 
> add to core. Or at least propose, don't exactly remember how that went.

The problem is that you need to grab a reference to call 
dma_fence_signal while testing the flag works without one.

Regards,
Christian.

>
>>
>>> + &cursor->fence->flags))
>>> +                cursor->fence = NULL;
>>> +        } else {
>>> +            cursor->fence = NULL;
>>> +        }
>>> +
>>> +        if (cursor->fence) {
>>> +            cursor->fence = dma_fence_get_rcu(cursor->fence);
>>> +        } else if (cursor->all_fences && cursor->fences) {
>>> +            struct dma_resv_list *fences = cursor->fences;
>>> +
>>> +            while (++cursor->index < fences->shared_count) {
>>> +                cursor->fence = rcu_dereference(
>>> +                    fences->shared[cursor->index]);
>>> +                if (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT,
>>> +                          &cursor->fence->flags))
>>> +                    break;
>>> +            }
>>> +            if (cursor->index < fences->shared_count)
>>> +                cursor->fence =
>>> +                    dma_fence_get_rcu(cursor->fence);
>>> +            else
>>> +                cursor->fence = NULL;
>>> +        }
>>
>> The control flow here is very hairy, but I'm not sure how to best do 
>> this.
>> With my suggestion to move the read_seqcount_begin into iter_begin maybe
>> something like this:
>>
>> iter_next()
>> {
>>     do {
>>         dma_fence_put(cursor->fence)
>>         cursor->fence = NULL;
>>
>>         if (cursor->index == -1) { /* reset by iter_begin()
>>             cursor->fence = get_exclusive();
>>             cusor->index++;
>>         } else {
>>             cursor->fence = shared_fences[++cursor->index]
>>         }
>>
>>         if (!dma_fence_is_signalled(cursor->fence))
>>             continue; /* just grab the next fence. */
>>
>>         cursor->fence =  dma_fence_get_rcu(cursor->fence);
>>
>>         if (!cursor->fence || read_seqcount_retry()) {
>>             /* we lost the race, restart completely */
>>             iter_begin(); /* ->fence will be cleaned up at beginning 
>> of the loop */
>>             continue;
>>         }
>>
>>         return cursor->fence;
>>     } while (true);
>> }
>>
>> Maybe I missed something, but that avoids the duplication of all the
>> tricky code, i.e. checking for signalling, rcu protected conditional
>> fence_get, and the retry is also nicely at the end.
>>> +
>>> +        /* For the eventually next round */
>>> +        first = true;
>>> +    } while (read_seqcount_retry(&obj->seq, cursor->seq));
>>> +
>>> +    return cursor->fence;
>>> +}
>>> +EXPORT_SYMBOL_GPL(dma_resv_iter_walk_unlocked);
>>> +
>>>   /**
>>>    * dma_resv_copy_fences - Copy all fences from src to dst.
>>>    * @dst: the destination reservation object
>>> diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
>>> index 9100dd3dc21f..693d16117153 100644
>>> --- a/include/linux/dma-resv.h
>>> +++ b/include/linux/dma-resv.h
>>> @@ -149,6 +149,90 @@ struct dma_resv {
>>>       struct dma_resv_list __rcu *fence;
>>>   };
>>>   +/**
>>> + * struct dma_resv_iter - current position into the dma_resv fences
>>> + *
>>> + * Don't touch this directly in the driver, use the accessor 
>>> function instead.
>>> + */
>>> +struct dma_resv_iter {
>>> +    /** @obj: The dma_resv object we iterate over */
>>> +    struct dma_resv *obj;
>>> +
>>> +    /** @all_fences: If all fences should be returned */
>>> +    bool all_fences;
>>> +
>>> +    /** @fence: the currently handled fence */
>>> +    struct dma_fence *fence;
>>> +
>>> +    /** @seq: sequence number to check for modifications */
>>> +    unsigned int seq;
>>> +
>>> +    /** @index: index into the shared fences */
>>
>> If you go with my suggestion (assuming it works): Please add "-1 
>> indicates
>> to pick the exclusive fence instead."
>>
>>> +    unsigned int index;
>>> +
>>> +    /** @fences: the shared fences */
>>> +    struct dma_resv_list *fences;
>>> +
>>> +    /** @is_first: true if this is the first returned fence */
>>> +    bool is_first;
>>
>> I think if we just rely on -1 == exclusive fence/is_first we don't need
>> this one here?
>>
>>> +};
>>> +
>>> +struct dma_fence *dma_resv_iter_walk_unlocked(struct dma_resv_iter 
>>> *cursor,
>>> +                          bool first);
>>> +
>>> +/**
>>> + * dma_resv_iter_begin - initialize a dma_resv_iter object
>>> + * @cursor: The dma_resv_iter object to initialize
>>> + * @obj: The dma_resv object which we want to iterator over
>>> + * @all_fences: If all fences should be returned or just the 
>>> exclusive one
>>
>> Please add: "Callers must clean up the iterator with 
>> dma_resv_iter_end()."
>>
>>> + */
>>> +static inline void dma_resv_iter_begin(struct dma_resv_iter *cursor,
>>> +                    struct dma_resv *obj,
>>> +                    bool all_fences)
>>> +{
>>> +    cursor->obj = obj;
>>> +    cursor->all_fences = all_fences;
>>> +    cursor->fence = NULL;
>>> +}
>>> +
>>> +/**
>>> + * dma_resv_iter_end - cleanup a dma_resv_iter object
>>> + * @cursor: the dma_resv_iter object which should be cleaned up
>>> + *
>>> + * Make sure that the reference to the fence in the cursor is properly
>>> + * dropped.
>>
>> Please add:
>>
>> "This function must be called every time dma_resv_iter_begin() was 
>> called
>> to clean up any references."
>>> + */
>>> +static inline void dma_resv_iter_end(struct dma_resv_iter *cursor)
>>> +{
>>> +    dma_fence_put(cursor->fence);
>>> +}
>>> +
>>> +/**
>>> + * dma_resv_iter_is_exclusive - test if the current fence is the 
>>> exclusive one
>>> + * @cursor: the cursor of the current position
>>> + *
>>> + * Returns true if the currently returned fence is the exclusive one.
>>> + */
>>> +static inline bool dma_resv_iter_is_exclusive(struct dma_resv_iter 
>>> *cursor)
>>> +{
>>> +    return cursor->index == -1;
>>> +}
>>> +
>>> +/**
>>> + * dma_resv_for_each_fence_unlocked - unlocked fence iterator
>>> + * @cursor: a struct dma_resv_iter pointer
>>> + * @fence: the current fence
>>> + *
>>> + * Iterate over the fences in a struct dma_resv object without 
>>> holding the
>>> + * dma_resv::lock. The RCU read side lock must be hold when using 
>>> this, but can
>>> + * be dropped and re-taken as necessary inside the loop. The cursor 
>>> needs to be
>>> + * initialized with dma_resv_iter_begin_unlocked() and cleaned up with
>>
>> We don't have an _unlocked version?
>
> @Christian:
>
> I'd also mention that the fence reference is held during the walk so 
> someone is less likely to grab extra ones.
>
>>
>>> + * dma_resv_iter_end_unlocked().
>>> + */
>>> +#define dma_resv_for_each_fence_unlocked(cursor, fence)            \
>>> +    for (fence = dma_resv_iter_walk_unlocked(cursor, true);        \
>>> +         fence; fence = dma_resv_iter_walk_unlocked(cursor, false))
>>> +
>>>   #define dma_resv_held(obj) lockdep_is_held(&(obj)->lock.base)
>>>   #define dma_resv_assert_held(obj) 
>>> lockdep_assert_held(&(obj)->lock.base)
>>>   --
>>> 2.25.1
>>>
>>
>
> Regards,
>
> Tvrtko
>


  reply	other threads:[~2021-09-20 10:09 UTC|newest]

Thread overview: 116+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-17 12:34 Deploying new iterator interface for dma-buf Christian König
2021-09-17 12:34 ` [Intel-gfx] " Christian König
2021-09-17 12:34 ` [PATCH 01/26] dma-buf: add dma_resv_for_each_fence_unlocked v2 Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 13:23   ` Daniel Vetter
2021-09-17 13:23     ` [Intel-gfx] " Daniel Vetter
2021-09-20  8:43     ` Tvrtko Ursulin
2021-09-20 10:09       ` Christian König [this message]
2021-09-20 10:26         ` Tvrtko Ursulin
2021-09-17 12:34 ` [PATCH 02/26] dma-buf: add dma_resv_for_each_fence Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 13:27   ` Daniel Vetter
2021-09-17 13:27     ` [Intel-gfx] " Daniel Vetter
2021-09-17 14:30     ` Daniel Vetter
2021-09-17 14:30       ` Daniel Vetter
2021-09-17 12:34 ` [PATCH 03/26] dma-buf: use new iterator in dma_resv_copy_fences Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 14:35   ` Daniel Vetter
2021-09-17 14:35     ` [Intel-gfx] " Daniel Vetter
2021-09-20  7:23     ` Christian König
2021-09-20  7:23       ` [Intel-gfx] " Christian König
2021-09-17 12:34 ` [PATCH 04/26] dma-buf: use new iterator in dma_resv_get_fences v2 Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 14:39   ` Daniel Vetter
2021-09-17 14:39     ` [Intel-gfx] " Daniel Vetter
2021-09-17 12:34 ` [PATCH 05/26] dma-buf: use new iterator in dma_resv_wait_timeout Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 14:43   ` Daniel Vetter
2021-09-17 14:43     ` [Intel-gfx] " Daniel Vetter
2021-09-20  7:27     ` Christian König
2021-09-20  7:27       ` [Intel-gfx] " Christian König
2021-09-17 12:34 ` [PATCH 06/26] dma-buf: use new iterator in dma_resv_test_signaled Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 14:45   ` Daniel Vetter
2021-09-17 14:45     ` [Intel-gfx] " Daniel Vetter
2021-09-17 12:34 ` [PATCH 07/26] drm/ttm: use the new iterator in ttm_bo_flush_all_fences Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 14:50   ` Daniel Vetter
2021-09-17 14:50     ` [Intel-gfx] " Daniel Vetter
2021-09-17 12:34 ` [PATCH 08/26] drm/amdgpu: use the new iterator in amdgpu_sync_resv Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 12:34 ` [PATCH 09/26] drm/amdgpu: use new iterator in amdgpu_ttm_bo_eviction_valuable Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 12:34 ` [PATCH 10/26] drm/msm: use new iterator in msm_gem_describe Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 12:34 ` [PATCH 11/26] drm/radeon: use new iterator in radeon_sync_resv Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 12:34 ` [PATCH 12/26] drm/scheduler: use new iterator in drm_sched_job_add_implicit_dependencies v2 Christian König
2021-09-17 12:34   ` [Intel-gfx] " Christian König
2021-09-17 14:52   ` Daniel Vetter
2021-09-17 14:52     ` [Intel-gfx] " Daniel Vetter
2021-11-15 14:03   ` Sascha Hauer
2021-11-15 14:03     ` Sascha Hauer
2021-11-15 14:03     ` [Intel-gfx] " Sascha Hauer
2021-11-15 14:08     ` Daniel Vetter
2021-11-15 14:08       ` [Intel-gfx] " Daniel Vetter
2021-11-15 14:08       ` Daniel Vetter
2021-11-15 20:32       ` Christian König
2021-11-15 20:32         ` Christian König
2021-11-15 20:32         ` [Intel-gfx] " Christian König
2021-11-16  7:56       ` Sascha Hauer
2021-11-16  7:56         ` [Intel-gfx] " Sascha Hauer
2021-11-16  7:56         ` Sascha Hauer
2021-09-17 12:35 ` [PATCH 13/26] drm/i915: use the new iterator in i915_gem_busy_ioctl Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-20  8:45   ` Tvrtko Ursulin
2021-09-20 10:13     ` Christian König
2021-09-20 10:33       ` Tvrtko Ursulin
2021-09-21  9:41         ` Christian König
2021-09-21 13:10           ` Tvrtko Ursulin
2021-09-17 12:35 ` [PATCH 14/26] drm/i915: use the new iterator in i915_sw_fence_await_reservation v3 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-20  8:45   ` Tvrtko Ursulin
2021-09-20  8:47     ` Tvrtko Ursulin
2021-09-20 10:14       ` Christian König
2021-09-17 12:35 ` [PATCH 15/26] drm/i915: use the new iterator in i915_request_await_object v2 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 16/26] drm/i915: use new iterator in i915_gem_object_wait_reservation v2 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-20 10:00   ` Tvrtko Ursulin
2021-09-21 17:35     ` Christian König
2021-09-17 12:35 ` [PATCH 17/26] drm/i915: use new iterator in i915_gem_object_wait_priority v2 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 18/26] drm/i915: use new iterator in i915_gem_object_last_write_engine v2 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 19/26] drm/i915: use new cursor in intel_prepare_plane_fb v2 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 20/26] drm: use new iterator in drm_gem_fence_array_add_implicit v2 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 14:53   ` Daniel Vetter
2021-09-17 14:53     ` [Intel-gfx] " Daniel Vetter
2021-09-20  7:31     ` Christian König
2021-09-20  7:31       ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 21/26] drm: use new iterator in drm_gem_plane_helper_prepare_fb v2 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 14:55   ` Daniel Vetter
2021-09-17 14:55     ` [Intel-gfx] " Daniel Vetter
2021-09-20  7:35     ` Christian König
2021-09-20  7:35       ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 22/26] drm/nouveau: use the new iterator in nouveau_fence_sync Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 23/26] drm/nouveau: use the new interator in nv50_wndw_prepare_fb v2 Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 24/26] drm/etnaviv: use new iterator in etnaviv_gem_describe Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 25/26] drm/etnaviv: replace dma_resv_get_excl_unlocked Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 12:35 ` [PATCH 26/26] dma-buf: nuke dma_resv_get_excl_unlocked Christian König
2021-09-17 12:35   ` [Intel-gfx] " Christian König
2021-09-17 14:56   ` Daniel Vetter
2021-09-17 14:56     ` [Intel-gfx] " Daniel Vetter
2021-09-17 14:01 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for series starting with [01/26] dma-buf: add dma_resv_for_each_fence_unlocked v2 Patchwork
2021-09-17 14:29 ` [Intel-gfx] ✓ Fi.CI.BAT: success " Patchwork
2021-09-17 15:43 ` [Intel-gfx] ✗ Fi.CI.IGT: failure " Patchwork
  -- strict thread matches above, loose matches on Subject: below --
2021-09-16 11:30 Deploying new iterator interface for dma-buf Christian König
2021-09-16 11:30 ` [Intel-gfx] [PATCH 01/26] dma-buf: add dma_resv_for_each_fence_unlocked v2 Christian König
2021-09-16 12:15   ` Daniel Vetter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e4aa2e1e-753e-655c-423f-93a0bb853b9d@gmail.com \
    --to=ckoenig.leichtzumerken@gmail.com \
    --cc=daniel@ffwll.ch \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=intel-gfx@lists.freedesktop.org \
    --cc=linaro-mm-sig@lists.linaro.org \
    --cc=linux-media@vger.kernel.org \
    --cc=tvrtko.ursulin@linux.intel.com \
    --subject='Re: [Intel-gfx] [PATCH 01/26] dma-buf: add dma_resv_for_each_fence_unlocked v2' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.