From: Paolo Bonzini <pbonzini@redhat.com>
To: Greg KH <gregkh@linuxfoundation.org>,
Jing Zhang <jingzhangos@google.com>
Cc: KVM <kvm@vger.kernel.org>, KVMARM <kvmarm@lists.cs.columbia.edu>,
LinuxMIPS <linux-mips@vger.kernel.org>,
KVMPPC <kvm-ppc@vger.kernel.org>,
LinuxS390 <linux-s390@vger.kernel.org>,
Linuxkselftest <linux-kselftest@vger.kernel.org>,
Marc Zyngier <maz@kernel.org>, James Morse <james.morse@arm.com>,
Julien Thierry <julien.thierry.kdev@gmail.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Will Deacon <will@kernel.org>,
Huacai Chen <chenhuacai@kernel.org>,
Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>,
Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
Paul Mackerras <paulus@ozlabs.org>,
Christian Borntraeger <borntraeger@de.ibm.com>,
Janosch Frank <frankja@linux.ibm.com>,
David Hildenbrand <david@redhat.com>,
Cornelia Huck <cohuck@redhat.com>,
Claudio Imbrenda <imbrenda@linux.ibm.com>,
Sean Christopherson <seanjc@google.com>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Jim Mattson <jmattson@google.com>,
Peter Shier <pshier@google.com>, Oliver Upton <oupton@google.com>,
David Rientjes <rientjes@google.com>,
Emanuele Giuseppe Esposito <eesposit@redhat.com>,
David Matlack <dmatlack@google.com>,
Ricardo Koller <ricarkol@google.com>,
Krish Sadhukhan <krish.sadhukhan@oracle.com>,
Fuad Tabba <tabba@google.com>
Subject: Re: [PATCH v9 3/5] KVM: stats: Add documentation for statistics data binary interface
Date: Wed, 16 Jun 2021 18:59:15 +0200 [thread overview]
Message-ID: <9b9a951d-d020-5599-5c4f-e154b40522b9@redhat.com> (raw)
In-Reply-To: <YMoW7SBqO9EPgCXw@kroah.com>
On 16/06/21 17:21, Greg KH wrote:
> I forgot to comment on this one, sorry for the delay.
>
> Why are you "inventing" your own schema format here for this? Why not
> use a well-known or at least well-designed/implemented one that we have
> in userspace already?
>
> There are a few that I would love to see in the kernel, varlink being
> the best example. We have kernel examples of this and I would consider
> using that as a transport for sysfs-like data in the future, but never
> got around to it.
Thanks, that's a good observation, and it's a problem that the rationale
and the design process didn't end up in either the documentation or the
commit message (only the outcome did). In order to fix that, this is
going to be quite a long message.
Varlink comprises both a schema and an encoding, and each has its own
problems. For the encoding of the data, varlink is really just using
JSON and it is really just the wrong tool here. The first few problems
that come to mind are:
- varlink structs are encoded as JSON dictionaries. Therefore, every
time userspace reads the fields, the kernel has to include the field
names as JSON dictionary keys. This means that a lot of time is spent
writing buffers, and on the receiving side parsing them.
- because numeric data has to be converted to ASCII the output doesn't
have fixed offsets, so it is not possible to make an efficient
implementation of pread.
- even though Varlink specifies that int is "usually int64", a
little-known gem is that JSON behavior for numbers not representable as
a double (i.e. exceeding 2^53) is implementation-defined;
implementations can and will mess up values outside that range. This
problem is not specific to this KVM stats usecase; varlink's schema
specification really is at odds with its encoding specification.
For the schema, there are some specific problems with varlink, but also
a more generic issue. The specific problems are:
- the schema doesn't include the length of arrays. This makes it hard
to compute in advance lengths and offsets of fields (even ignoring the
fact that data is not binary, which I'll get to later)
- the schema also is not extensible with user annotations. In our case
that would be mostly the unit in which the value is expressed.
The main problem with the various available serialization formats is
that their schema languages are designed to be compiled with a code
generator. Here instead the schema is transmitted from the kernel to
userspace. Userspace is not really supposed to know what a value means
or even that if it exists. Userspace takes care of collecting the data
from the kernel, but ultimately there will be a human that knows what
e.g. "io_exits" or "req_event" means, and they will ask for the current
value, or a plot over time, of a specific statistic.
Now, unlike most other schema languages, varlink does not require
precompiling the schema in its C bindings. However, this was mostly
just the authors not bothering to write a C code generator---they did
write one for Rust, for example. When using the "official" varlink C
bindings, you hardly use the schema at all.
Something similar to the above issues is quite common in other formats.
For example, FlatBuffers[2]'s schema language[1] does have annotations
(which it calls metadata), but then we would still have to invent some
standard annotations and teach programs about them. However, the deal
breaker is that again there is no way to transmit the schema from the
server to the client, and that users are supposed to precompile the
schema using a code generator.
All that said, what we _could_ do is serialize the schema as JSON
instead of using a binary format, like so:
[{
"name": "exits",
"kind": "cumulative",
"count": 1
}, {
"name": "halt_poll_fail_ns",
"kind": "cumulative",
"unit": "seconds",
"scale": {"base": "pow2", exponent: -9},
"count": 1
}, ...]
while keeping the actual statistics as an array of u64 values just like
in these patches. The JSON representation of the schema would be always
the same, so it could be treated as fixed-size and an efficient
implementation of pread would be possible. And once the schema is JSON,
there could be a *meta*-schema expressed using the varlink language:
# compare to struct kvm_stats_desc, included after my sig
# for reference
type StatsDescriptor {
name: string,
kind: (cumulative, instant, peak),
unit: ?(bytes, seconds, cycles),
scale: ?(base: (pow2, pow10), exponent: int),
count: int
}
type StatsSchema {
name: string,
stats: []StatsDescriptor
}
Varlink would *not* be used in the kernel, because Varlink isreally
just JSON. The above ten lines (plus comments) would be dropped in
Documentation/ as a .varlink file; they would be a convenient way to
describe (to either a human or a program) how to parse the JSON schema.
There would then be another problem, namely how to include the varlink
schema in the generated documentation, since right now there are for
example no sphinx bindings for varlink. Nevertheless, if you think it's
better to have the schema as JSON instead of binary, according to the
above varlink meta-schema, then we can look into it.
Thanks,
Paolo
>> + struct kvm_stats_desc {
>> + __u32 flags;
>> + __s16 exponent;
>> + __u16 size;
>> + __u32 offset;
>> + __u32 unused;
>> + char name[0];
>> + };
WARNING: multiple messages have this Message-ID (diff)
From: Paolo Bonzini <pbonzini@redhat.com>
To: Greg KH <gregkh@linuxfoundation.org>,
Jing Zhang <jingzhangos@google.com>
Cc: KVM <kvm@vger.kernel.org>, David Hildenbrand <david@redhat.com>,
Paul Mackerras <paulus@ozlabs.org>,
Linuxkselftest <linux-kselftest@vger.kernel.org>,
Claudio Imbrenda <imbrenda@linux.ibm.com>,
Will Deacon <will@kernel.org>,
KVMARM <kvmarm@lists.cs.columbia.edu>,
Emanuele Giuseppe Esposito <eesposit@redhat.com>,
LinuxS390 <linux-s390@vger.kernel.org>,
Janosch Frank <frankja@linux.ibm.com>,
Marc Zyngier <maz@kernel.org>,
Huacai Chen <chenhuacai@kernel.org>,
Christian Borntraeger <borntraeger@de.ibm.com>,
Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>,
David Rientjes <rientjes@google.com>,
KVMPPC <kvm-ppc@vger.kernel.org>,
Krish Sadhukhan <krish.sadhukhan@oracle.com>,
David Matlack <dmatlack@google.com>,
Jim Mattson <jmattson@google.com>,
Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
Sean Christopherson <seanjc@google.com>,
Cornelia Huck <cohuck@redhat.com>,
Peter Shier <pshier@google.com>,
LinuxMIPS <linux-mips@vger.kernel.org>,
Vitaly Kuznetsov <vkuznets@redhat.com>
Subject: Re: [PATCH v9 3/5] KVM: stats: Add documentation for statistics data binary interface
Date: Wed, 16 Jun 2021 18:59:15 +0200 [thread overview]
Message-ID: <9b9a951d-d020-5599-5c4f-e154b40522b9@redhat.com> (raw)
In-Reply-To: <YMoW7SBqO9EPgCXw@kroah.com>
On 16/06/21 17:21, Greg KH wrote:
> I forgot to comment on this one, sorry for the delay.
>
> Why are you "inventing" your own schema format here for this? Why not
> use a well-known or at least well-designed/implemented one that we have
> in userspace already?
>
> There are a few that I would love to see in the kernel, varlink being
> the best example. We have kernel examples of this and I would consider
> using that as a transport for sysfs-like data in the future, but never
> got around to it.
Thanks, that's a good observation, and it's a problem that the rationale
and the design process didn't end up in either the documentation or the
commit message (only the outcome did). In order to fix that, this is
going to be quite a long message.
Varlink comprises both a schema and an encoding, and each has its own
problems. For the encoding of the data, varlink is really just using
JSON and it is really just the wrong tool here. The first few problems
that come to mind are:
- varlink structs are encoded as JSON dictionaries. Therefore, every
time userspace reads the fields, the kernel has to include the field
names as JSON dictionary keys. This means that a lot of time is spent
writing buffers, and on the receiving side parsing them.
- because numeric data has to be converted to ASCII the output doesn't
have fixed offsets, so it is not possible to make an efficient
implementation of pread.
- even though Varlink specifies that int is "usually int64", a
little-known gem is that JSON behavior for numbers not representable as
a double (i.e. exceeding 2^53) is implementation-defined;
implementations can and will mess up values outside that range. This
problem is not specific to this KVM stats usecase; varlink's schema
specification really is at odds with its encoding specification.
For the schema, there are some specific problems with varlink, but also
a more generic issue. The specific problems are:
- the schema doesn't include the length of arrays. This makes it hard
to compute in advance lengths and offsets of fields (even ignoring the
fact that data is not binary, which I'll get to later)
- the schema also is not extensible with user annotations. In our case
that would be mostly the unit in which the value is expressed.
The main problem with the various available serialization formats is
that their schema languages are designed to be compiled with a code
generator. Here instead the schema is transmitted from the kernel to
userspace. Userspace is not really supposed to know what a value means
or even that if it exists. Userspace takes care of collecting the data
from the kernel, but ultimately there will be a human that knows what
e.g. "io_exits" or "req_event" means, and they will ask for the current
value, or a plot over time, of a specific statistic.
Now, unlike most other schema languages, varlink does not require
precompiling the schema in its C bindings. However, this was mostly
just the authors not bothering to write a C code generator---they did
write one for Rust, for example. When using the "official" varlink C
bindings, you hardly use the schema at all.
Something similar to the above issues is quite common in other formats.
For example, FlatBuffers[2]'s schema language[1] does have annotations
(which it calls metadata), but then we would still have to invent some
standard annotations and teach programs about them. However, the deal
breaker is that again there is no way to transmit the schema from the
server to the client, and that users are supposed to precompile the
schema using a code generator.
All that said, what we _could_ do is serialize the schema as JSON
instead of using a binary format, like so:
[{
"name": "exits",
"kind": "cumulative",
"count": 1
}, {
"name": "halt_poll_fail_ns",
"kind": "cumulative",
"unit": "seconds",
"scale": {"base": "pow2", exponent: -9},
"count": 1
}, ...]
while keeping the actual statistics as an array of u64 values just like
in these patches. The JSON representation of the schema would be always
the same, so it could be treated as fixed-size and an efficient
implementation of pread would be possible. And once the schema is JSON,
there could be a *meta*-schema expressed using the varlink language:
# compare to struct kvm_stats_desc, included after my sig
# for reference
type StatsDescriptor {
name: string,
kind: (cumulative, instant, peak),
unit: ?(bytes, seconds, cycles),
scale: ?(base: (pow2, pow10), exponent: int),
count: int
}
type StatsSchema {
name: string,
stats: []StatsDescriptor
}
Varlink would *not* be used in the kernel, because Varlink isreally
just JSON. The above ten lines (plus comments) would be dropped in
Documentation/ as a .varlink file; they would be a convenient way to
describe (to either a human or a program) how to parse the JSON schema.
There would then be another problem, namely how to include the varlink
schema in the generated documentation, since right now there are for
example no sphinx bindings for varlink. Nevertheless, if you think it's
better to have the schema as JSON instead of binary, according to the
above varlink meta-schema, then we can look into it.
Thanks,
Paolo
>> + struct kvm_stats_desc {
>> + __u32 flags;
>> + __s16 exponent;
>> + __u16 size;
>> + __u32 offset;
>> + __u32 unused;
>> + char name[0];
>> + };
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
WARNING: multiple messages have this Message-ID (diff)
From: Paolo Bonzini <pbonzini@redhat.com>
To: Greg KH <gregkh@linuxfoundation.org>,
Jing Zhang <jingzhangos@google.com>
Cc: KVM <kvm@vger.kernel.org>, KVMARM <kvmarm@lists.cs.columbia.edu>,
LinuxMIPS <linux-mips@vger.kernel.org>,
KVMPPC <kvm-ppc@vger.kernel.org>,
LinuxS390 <linux-s390@vger.kernel.org>,
Linuxkselftest <linux-kselftest@vger.kernel.org>,
Marc Zyngier <maz@kernel.org>, James Morse <james.morse@arm.com>,
Julien Thierry <julien.thierry.kdev@gmail.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Will Deacon <will@kernel.org>,
Huacai Chen <chenhuacai@kernel.org>,
Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>,
Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
Paul Mackerras <paulus@ozlabs.org>,
Christian Borntraeger <borntraeger@de.ibm.com>,
Janosch Frank <frankja@linux.ibm.com>,
David Hildenbrand <david@redhat.com>,
Cornelia Huck <cohuck@redhat.com>,
Claudio Imbrenda <imbrenda@linux.ibm.com>,
Sean Christopherson <seanjc@google.com>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Jim Mattson <jmattson@google.com>,
Peter Shier <pshier@google.com>, Oliver Upton <oupton@google.com>,
David Rientjes <rientjes@google.com>,
Emanuele Giuseppe Esposito <eesposit@redhat.com>,
David Matlack <dmatlack@google.com>,
Ricardo Koller <ricarkol@google.com>,
Krish Sadhukhan <krish.sadhukhan@oracle.com>,
Fuad Tabba <tabba@google.com>
Subject: Re: [PATCH v9 3/5] KVM: stats: Add documentation for statistics data binary interface
Date: Wed, 16 Jun 2021 16:59:15 +0000 [thread overview]
Message-ID: <9b9a951d-d020-5599-5c4f-e154b40522b9@redhat.com> (raw)
In-Reply-To: <YMoW7SBqO9EPgCXw@kroah.com>
On 16/06/21 17:21, Greg KH wrote:
> I forgot to comment on this one, sorry for the delay.
>
> Why are you "inventing" your own schema format here for this? Why not
> use a well-known or at least well-designed/implemented one that we have
> in userspace already?
>
> There are a few that I would love to see in the kernel, varlink being
> the best example. We have kernel examples of this and I would consider
> using that as a transport for sysfs-like data in the future, but never
> got around to it.
Thanks, that's a good observation, and it's a problem that the rationale
and the design process didn't end up in either the documentation or the
commit message (only the outcome did). In order to fix that, this is
going to be quite a long message.
Varlink comprises both a schema and an encoding, and each has its own
problems. For the encoding of the data, varlink is really just using
JSON and it is really just the wrong tool here. The first few problems
that come to mind are:
- varlink structs are encoded as JSON dictionaries. Therefore, every
time userspace reads the fields, the kernel has to include the field
names as JSON dictionary keys. This means that a lot of time is spent
writing buffers, and on the receiving side parsing them.
- because numeric data has to be converted to ASCII the output doesn't
have fixed offsets, so it is not possible to make an efficient
implementation of pread.
- even though Varlink specifies that int is "usually int64", a
little-known gem is that JSON behavior for numbers not representable as
a double (i.e. exceeding 2^53) is implementation-defined;
implementations can and will mess up values outside that range. This
problem is not specific to this KVM stats usecase; varlink's schema
specification really is at odds with its encoding specification.
For the schema, there are some specific problems with varlink, but also
a more generic issue. The specific problems are:
- the schema doesn't include the length of arrays. This makes it hard
to compute in advance lengths and offsets of fields (even ignoring the
fact that data is not binary, which I'll get to later)
- the schema also is not extensible with user annotations. In our case
that would be mostly the unit in which the value is expressed.
The main problem with the various available serialization formats is
that their schema languages are designed to be compiled with a code
generator. Here instead the schema is transmitted from the kernel to
userspace. Userspace is not really supposed to know what a value means
or even that if it exists. Userspace takes care of collecting the data
from the kernel, but ultimately there will be a human that knows what
e.g. "io_exits" or "req_event" means, and they will ask for the current
value, or a plot over time, of a specific statistic.
Now, unlike most other schema languages, varlink does not require
precompiling the schema in its C bindings. However, this was mostly
just the authors not bothering to write a C code generator---they did
write one for Rust, for example. When using the "official" varlink C
bindings, you hardly use the schema at all.
Something similar to the above issues is quite common in other formats.
For example, FlatBuffers[2]'s schema language[1] does have annotations
(which it calls metadata), but then we would still have to invent some
standard annotations and teach programs about them. However, the deal
breaker is that again there is no way to transmit the schema from the
server to the client, and that users are supposed to precompile the
schema using a code generator.
All that said, what we _could_ do is serialize the schema as JSON
instead of using a binary format, like so:
[{
"name": "exits",
"kind": "cumulative",
"count": 1
}, {
"name": "halt_poll_fail_ns",
"kind": "cumulative",
"unit": "seconds",
"scale": {"base": "pow2", exponent: -9},
"count": 1
}, ...]
while keeping the actual statistics as an array of u64 values just like
in these patches. The JSON representation of the schema would be always
the same, so it could be treated as fixed-size and an efficient
implementation of pread would be possible. And once the schema is JSON,
there could be a *meta*-schema expressed using the varlink language:
# compare to struct kvm_stats_desc, included after my sig
# for reference
type StatsDescriptor {
name: string,
kind: (cumulative, instant, peak),
unit: ?(bytes, seconds, cycles),
scale: ?(base: (pow2, pow10), exponent: int),
count: int
}
type StatsSchema {
name: string,
stats: []StatsDescriptor
}
Varlink would *not* be used in the kernel, because Varlink isreally
just JSON. The above ten lines (plus comments) would be dropped in
Documentation/ as a .varlink file; they would be a convenient way to
describe (to either a human or a program) how to parse the JSON schema.
There would then be another problem, namely how to include the varlink
schema in the generated documentation, since right now there are for
example no sphinx bindings for varlink. Nevertheless, if you think it's
better to have the schema as JSON instead of binary, according to the
above varlink meta-schema, then we can look into it.
Thanks,
Paolo
>> + struct kvm_stats_desc {
>> + __u32 flags;
>> + __s16 exponent;
>> + __u16 size;
>> + __u32 offset;
>> + __u32 unused;
>> + char name[0];
>> + };
next prev parent reply other threads:[~2021-06-16 16:59 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-14 21:21 [PATCH v9 0/5] KVM statistics data fd-based binary interface Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-14 21:21 ` [PATCH v9 1/5] KVM: stats: Separate generic stats from architecture specific ones Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-14 21:21 ` [PATCH v9 2/5] KVM: stats: Add fd-based API to read binary stats data Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-16 17:12 ` Paolo Bonzini
2021-06-16 17:12 ` Paolo Bonzini
2021-06-16 17:12 ` Paolo Bonzini
2021-06-16 18:04 ` Jing Zhang
2021-06-16 18:04 ` Jing Zhang
2021-06-16 18:04 ` Jing Zhang
2021-06-14 21:21 ` [PATCH v9 3/5] KVM: stats: Add documentation for statistics data binary interface Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-16 15:21 ` Greg KH
2021-06-16 15:21 ` Greg KH
2021-06-16 15:21 ` Greg KH
2021-06-16 16:59 ` Paolo Bonzini [this message]
2021-06-16 16:59 ` Paolo Bonzini
2021-06-16 16:59 ` Paolo Bonzini
2021-06-16 18:18 ` Greg KH
2021-06-16 18:18 ` Greg KH
2021-06-16 18:18 ` Greg KH
2021-06-16 19:45 ` Paolo Bonzini
2021-06-16 19:45 ` Paolo Bonzini
2021-06-16 19:45 ` Paolo Bonzini
2021-06-14 21:21 ` [PATCH v9 4/5] KVM: selftests: Add selftest for KVM " Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-15 8:03 ` Fuad Tabba
2021-06-15 8:03 ` Fuad Tabba
2021-06-15 8:03 ` Fuad Tabba
2021-06-16 21:35 ` Jing Zhang
2021-06-16 21:35 ` Jing Zhang
2021-06-16 21:35 ` Jing Zhang
2021-06-14 21:21 ` [PATCH v9 5/5] KVM: stats: Remove code duplication for binary and debugfs stats Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-14 21:21 ` Jing Zhang
2021-06-15 5:25 ` [PATCH v9 0/5] KVM statistics data fd-based binary interface Leon Romanovsky
2021-06-15 5:25 ` Leon Romanovsky
2021-06-15 5:25 ` Leon Romanovsky
2021-06-15 7:06 ` Paolo Bonzini
2021-06-15 7:06 ` Paolo Bonzini
2021-06-15 7:06 ` Paolo Bonzini
2021-06-15 7:53 ` Leon Romanovsky
2021-06-15 7:53 ` Leon Romanovsky
2021-06-15 7:53 ` Leon Romanovsky
2021-06-15 11:03 ` Paolo Bonzini
2021-06-15 11:03 ` Paolo Bonzini
2021-06-15 11:03 ` Paolo Bonzini
2021-06-15 13:34 ` Leon Romanovsky
2021-06-15 13:34 ` Leon Romanovsky
2021-06-15 13:34 ` Leon Romanovsky
2021-06-15 8:37 ` Enrico Weigelt, metux IT consult
2021-06-15 8:37 ` Enrico Weigelt, metux IT consult
2021-06-15 8:37 ` Enrico Weigelt, metux IT consult
2021-06-15 9:21 ` Greg KH
2021-06-15 9:21 ` Greg KH
2021-06-15 9:21 ` Greg KH
2021-06-15 11:31 ` Paolo Bonzini
2021-06-15 11:31 ` Paolo Bonzini
2021-06-15 11:31 ` Paolo Bonzini
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9b9a951d-d020-5599-5c4f-e154b40522b9@redhat.com \
--to=pbonzini@redhat.com \
--cc=aleksandar.qemu.devel@gmail.com \
--cc=borntraeger@de.ibm.com \
--cc=chenhuacai@kernel.org \
--cc=cohuck@redhat.com \
--cc=david@redhat.com \
--cc=dmatlack@google.com \
--cc=eesposit@redhat.com \
--cc=frankja@linux.ibm.com \
--cc=gregkh@linuxfoundation.org \
--cc=imbrenda@linux.ibm.com \
--cc=james.morse@arm.com \
--cc=jingzhangos@google.com \
--cc=jmattson@google.com \
--cc=julien.thierry.kdev@gmail.com \
--cc=krish.sadhukhan@oracle.com \
--cc=kvm-ppc@vger.kernel.org \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=maz@kernel.org \
--cc=oupton@google.com \
--cc=paulus@ozlabs.org \
--cc=pshier@google.com \
--cc=ricarkol@google.com \
--cc=rientjes@google.com \
--cc=seanjc@google.com \
--cc=suzuki.poulose@arm.com \
--cc=tabba@google.com \
--cc=tsbogend@alpha.franken.de \
--cc=vkuznets@redhat.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.