Regarding TX/RX rings

Regarding TX/RX rings

[Mahendra Kutare]

[-- Attachment #1.1: Type: text/plain, Size: 265 bytes --]

Hi All,

I have to monitor the rate of data transfer over a time window for TX/RX
rings in DOM0 for all DOMUs.

Is there a good place to start with in terms of code ? Any pointers of
sample implementation already existing for this will be useful.

Regards
Mahendra

[-- Attachment #1.2: Type: text/html, Size: 299 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Regarding TX/RX rings

[James Harper]

[-- Attachment #1: Type: text/plain, Size: 1427 bytes --]

> Hi All,
> 
> I have to monitor the rate of data transfer over a time window for TX/RX rings
> in DOM0 for all DOMUs.
> 
> Is there a good place to start with in terms of code ? Any pointers of sample
> implementation already existing for this will be useful.
> 

For what purpose?

Why monitor the rings directly rather than monitoring throughput at a level that already allows such things (disk io / network io etc)?

Would you be monitoring rings slots used or actual data?

There appears to be data available in /sys/devices/vbd-<domid>-<devid> that might tell you what you want for vbd's (at least there is in 2.6.31.12 - your kernel might be different). That may correlate directly to rings slots or may be an abstraction. Vbd only has one ring containing both read and write requests but the stats seem to break it down.

For network you can find throughput information in the /sys/devices/vif-<domid>-<devid>/net:vif<domid>.<devid>/statistics (maybe a few ways to get there) but that seems more packet oriented, and a single packet doesn't correlate directly to a ring slot - you could have 1 packet taking several rings slots and for GSO you could have a 'large' packet in one ring slot that would be broken up into several MTU sized packets. You'd probably need to implement a similar statistics reporting function in vif like vbd to tell you what you want to know about the actual rings.

James

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Regarding TX/RX rings

[George Shuklin]

Well...

I'd like to pointed to this again: HVM machine without PV network
dirvers lack of network statistics.

В Сбт, 23/10/2010 в 12:03 +1100, James Harper пишет:
> > Hi All,
> > 
> > I have to monitor the rate of data transfer over a time window for TX/RX rings
> > in DOM0 for all DOMUs.
> > 
> > Is there a good place to start with in terms of code ? Any pointers of sample
> > implementation already existing for this will be useful.
> > 
> 
> For what purpose?
> 
> Why monitor the rings directly rather than monitoring throughput at a level that already allows such things (disk io / network io etc)?
> 
> Would you be monitoring rings slots used or actual data?
> 
> There appears to be data available in /sys/devices/vbd-<domid>-<devid> that might tell you what you want for vbd's (at least there is in 2.6.31.12 - your kernel might be different). That may correlate directly to rings slots or may be an abstraction. Vbd only has one ring containing both read and write requests but the stats seem to break it down.
> 
> For network you can find throughput information in the /sys/devices/vif-<domid>-<devid>/net:vif<domid>.<devid>/statistics (maybe a few ways to get there) but that seems more packet oriented, and a single packet doesn't correlate directly to a ring slot - you could have 1 packet taking several rings slots and for GSO you could have a 'large' packet in one ring slot that would be broken up into several MTU sized packets. You'd probably need to implement a similar statistics reporting function in vif like vbd to tell you what you want to know about the actual rings.
> 
> James
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Regarding TX/RX rings

[Mahendra Kutare]

[-- Attachment #1.1: Type: text/plain, Size: 2397 bytes --]

On Fri, Oct 22, 2010 at 9:03 PM, James Harper <james.harper@bendigoit.com.au
> wrote:

> > Hi All,
> >
> > I have to monitor the rate of data transfer over a time window for TX/RX
> rings
> > in DOM0 for all DOMUs.
> >
> > Is there a good place to start with in terms of code ? Any pointers of
> sample
> > implementation already existing for this will be useful.
> >
>
> For what purpose?
>
> Why monitor the rings directly rather than monitoring throughput at a level
> that already allows such things (disk io / network io etc)?
>
> Would you be monitoring rings slots used or actual data?
>
> There appears to be data available in /sys/devices/vbd-<domid>-<devid> that
> might tell you what you want for vbd's (at least there is in 2.6.31.12 -
> your kernel might be different). That may correlate directly to rings slots
> or may be an abstraction. Vbd only has one ring containing both read and
> write requests but the stats seem to break it down.
>
> For network you can find throughput information in the
> /sys/devices/vif-<domid>-<devid>/net:vif<domid>.<devid>/statistics (maybe a
> few ways to get there) but that seems more packet oriented, and a single
> packet doesn't correlate directly to a ring slot - you could have 1 packet
> taking several rings slots and for GSO you could have a 'large' packet in
> one ring slot that would be broken up into several MTU sized packets. You'd
> probably need to implement a similar statistics reporting function in vif
> like vbd to tell you what you want to know about the actual rings.
>
> James
>

The reason I am looking to monitor these rings is to find the variability of
the data transfer through the TX/RX rings i.e. inbound/outbound network
traffic. I am doing variability analysis for various combination of VMs
running in Xen. I am looking at both CPU and Network level metrics. The goal
is to infer characteristics looking at these lower level metrics about the
VMs running.

On my current installation of Xen 3.4.3 with 2.6.18.8 kernel ,  the data
mentioned in /sys/devices/vbd- or /sys/devices/vif is not available. I need
to upgrade my kernel.

I have seen that if I monitor the network traffic input and output
throughput data (through say xentop running application - RUBiS) it does not
really vary a lot. So I want to monitor the lower levels to see if there
exists variability that eventually evens out.


Regards
Mahendra

[-- Attachment #1.2: Type: text/html, Size: 2902 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Regarding TX/RX rings

[James Harper]

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]

> 
> The reason I am looking to monitor these rings is to find the variability of
> the data transfer through the TX/RX rings i.e. inbound/outbound network
> traffic. I am doing variability analysis for various combination of VMs
> running in Xen. I am looking at both CPU and Network level metrics. The goal
> is to infer characteristics looking at these lower level metrics about the VMs
> running.
> 
> On my current installation of Xen 3.4.3 with 2.6.18.8 kernel ,  the data
> mentioned in /sys/devices/vbd- or /sys/devices/vif is not available. I need to
> upgrade my kernel.
> 
> I have seen that if I monitor the network traffic input and output throughput
> data (through say xentop running application - RUBiS) it does not really vary
> a lot. So I want to monitor the lower levels to see if there exists
> variability that eventually evens out.
> 

Monitoring just the rings will not tell you about tx-full or rx-starvation conditions. You would also need to measure when DomU wants to transmit a packet but there is no space left on the tx ring, or when Dom0 has a packet for DomU but Dom0 has used all of the rx buffers given by DomU. I would find those measurements useful.

James

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Regarding TX/RX rings

[Mahendra Kutare]

[-- Attachment #1.1: Type: text/plain, Size: 1465 bytes --]

Can you point me to the code files which would be good starting point for
the monitoring mentioned below ?

Regards
Mahendra

On Sat, Oct 23, 2010 at 8:35 AM, James Harper <james.harper@bendigoit.com.au
> wrote:

> >
> > The reason I am looking to monitor these rings is to find the variability
> of
> > the data transfer through the TX/RX rings i.e. inbound/outbound network
> > traffic. I am doing variability analysis for various combination of VMs
> > running in Xen. I am looking at both CPU and Network level metrics. The
> goal
> > is to infer characteristics looking at these lower level metrics about
> the VMs
> > running.
> >
> > On my current installation of Xen 3.4.3 with 2.6.18.8 kernel ,  the data
> > mentioned in /sys/devices/vbd- or /sys/devices/vif is not available. I
> need to
> > upgrade my kernel.
> >
> > I have seen that if I monitor the network traffic input and output
> throughput
> > data (through say xentop running application - RUBiS) it does not really
> vary
> > a lot. So I want to monitor the lower levels to see if there exists
> > variability that eventually evens out.
> >
>
> Monitoring just the rings will not tell you about tx-full or rx-starvation
> conditions. You would also need to measure when DomU wants to transmit a
> packet but there is no space left on the tx ring, or when Dom0 has a packet
> for DomU but Dom0 has used all of the rx buffers given by DomU. I would find
> those measurements useful.
>
> James
>

[-- Attachment #1.2: Type: text/html, Size: 1894 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Regarding TX/RX rings

[James Harper]

[-- Attachment #1: Type: text/plain, Size: 646 bytes --]

> 
> Can you point me to the code files which would be good starting point for the
> monitoring mentioned below ?
> 

drivers/xen/netback or something like that. I think netback would be a good keyword to search for. I know they've been moved around a bit over time.

You can detect a 'no free rx buffers' condition in Dom0, but only DomU knows if it wanted to put more information on the ring but couldn't because it was full. From Dom0 you can probably make the assumption that if the tx ring was full when you processed it then DomU is probably waiting for more room, but you can't tell how much extra there was to put on it.

James

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel