Little help with blk ring

Little help with blk ring

[Daniel Castro]

Hello List,

I have a small problem with the ring when transferring blocks the id
on the response is different from the request.
This is the boot up read, count 0.
The guest requests block 0, it has to be located at 7c00.

I go ahead and create a REQUEST with this data:
ring_req = RING_GET_REQUEST(priv,priv->req_prod_pvt);
ring_req->id = 9;
ring_req->nr_segments=1;
ring_req->operation = BLKIF_OP_READ;
ring_req->sector_number = (int)op->lba; //sector to be read
ring_req->seg[0].gref = (bi->buffer_gref); //this should be get_free_gref();
ring_req->seg[0].first_sect = 0;//op->lba;
ring_req->seg[0].last_sect = 7;//op->lba + op->count;

RING_PUSH_REQUESTS_AND_CHECK_NOTIFY((priv),notify);  //return notify=0
if(notify){
		dprintf(1,"Start notify procedure\n");
		evtchn_send_t send;
		send.port = (bi->port);
		dprintf(1,"In notify before hypercall port is %d\n",send.port);
		//hypercall_event_channel_op(EVTCHNOP_send, &send);
		dprintf(1,"HYPERCALL read operation notify res %d\n",
hypercall_event_channel_op(EVTCHNOP_send,&send));
}
ring_res = RING_GET_RESPONSE((priv),(temp->rsp_prod));
Then I get:
FAIL RING RESPONSE 0x0009a040 id:256 status:9 operation 0
this is the line: 		
dprintf(1,"FAIL RING RESPONSE %p id:%d status:%d operation %d\n",
ring_res,ring_res->id,ring_res->status,ring_res->operation);


The id should be the same on both the request and response, there are
bi ither requests on the fly.

Any clue?=

Thanks,

Daniel


-- 
+-=====---------------------------+
| +---------------------------------+ | This space intentionally blank
for notetaking.
| |   | Daniel Castro,                |
| |   | Consultant/Programmer.|
| |   | U Andes                         |
+-------------------------------------+

Re: Little help with blk ring

[Jan Beulich]

>>> On 07.05.12 at 08:23, Daniel Castro <evil.dani@gmail.com> wrote:
> Hello List,
> 
> I have a small problem with the ring when transferring blocks the id
> on the response is different from the request.
> This is the boot up read, count 0.
> The guest requests block 0, it has to be located at 7c00.
> 
> I go ahead and create a REQUEST with this data:
> ring_req = RING_GET_REQUEST(priv,priv->req_prod_pvt);
> ring_req->id = 9;
> ring_req->nr_segments=1;
> ring_req->operation = BLKIF_OP_READ;
> ring_req->sector_number = (int)op->lba; //sector to be read
> ring_req->seg[0].gref = (bi->buffer_gref); //this should be get_free_gref();
> ring_req->seg[0].first_sect = 0;//op->lba;
> ring_req->seg[0].last_sect = 7;//op->lba + op->count;
> 
> RING_PUSH_REQUESTS_AND_CHECK_NOTIFY((priv),notify);  //return notify=0
> if(notify){
> 		dprintf(1,"Start notify procedure\n");
> 		evtchn_send_t send;
> 		send.port = (bi->port);
> 		dprintf(1,"In notify before hypercall port is %d\n",send.port);
> 		//hypercall_event_channel_op(EVTCHNOP_send, &send);
> 		dprintf(1,"HYPERCALL read operation notify res %d\n",
> hypercall_event_channel_op(EVTCHNOP_send,&send));
> }
> ring_res = RING_GET_RESPONSE((priv),(temp->rsp_prod));
> Then I get:
> FAIL RING RESPONSE 0x0009a040 id:256 status:9 operation 0
> this is the line: 		
> dprintf(1,"FAIL RING RESPONSE %p id:%d status:%d operation %d\n",
> ring_res,ring_res->id,ring_res->status,ring_res->operation);
> 
> 
> The id should be the same on both the request and response, there are
> bi ither requests on the fly.
> 
> Any clue?=

With status happening to be 9 when id should be, is this perhaps a
broken response structure definition?

Jan

Re: Little help with blk ring

[James Harper]

> 
> >>> On 07.05.12 at 08:23, Daniel Castro <evil.dani@gmail.com> wrote:
> > Hello List,
> >
> > I have a small problem with the ring when transferring blocks the id
> > on the response is different from the request.
> > This is the boot up read, count 0.
> > The guest requests block 0, it has to be located at 7c00.
> >
> > I go ahead and create a REQUEST with this data:
> > ring_req = RING_GET_REQUEST(priv,priv->req_prod_pvt);
> > ring_req->id = 9;
> > ring_req->nr_segments=1;
> > ring_req->operation = BLKIF_OP_READ;
> > ring_req->sector_number = (int)op->lba; //sector to be read
> > ring_req->seg[0].gref = (bi->buffer_gref); //this should be
> > get_free_gref(); ring_req->seg[0].first_sect = 0;//op->lba;
> > ring_req->seg[0].last_sect = 7;//op->lba + op->count;
> >
> > RING_PUSH_REQUESTS_AND_CHECK_NOTIFY((priv),notify);  //return
> notify=0
> > if(notify){
> > 		dprintf(1,"Start notify procedure\n");
> > 		evtchn_send_t send;
> > 		send.port = (bi->port);
> > 		dprintf(1,"In notify before hypercall port is %d\n",send.port);
> > 		//hypercall_event_channel_op(EVTCHNOP_send, &send);
> > 		dprintf(1,"HYPERCALL read operation notify res %d\n",
> > hypercall_event_channel_op(EVTCHNOP_send,&send));
> > }
> > ring_res = RING_GET_RESPONSE((priv),(temp->rsp_prod));
> > Then I get:
> > FAIL RING RESPONSE 0x0009a040 id:256 status:9 operation 0
> > this is the line:
> > dprintf(1,"FAIL RING RESPONSE %p id:%d status:%d operation %d\n",
> > ring_res,ring_res->id,ring_res->status,ring_res->operation);
> >
> >
> > The id should be the same on both the request and response, there are
> > bi ither requests on the fly.
> >
> > Any clue?=
> 
> With status happening to be 9 when id should be, is this perhaps a broken
> response structure definition?
> 

That would be my guess too. The structure aligns differently under 64 and 32 bits so I'd guess the OP is talking between the two.

James

Re: Little help with blk ring

[James Harper]

> >
> > >>> On 07.05.12 at 08:23, Daniel Castro <evil.dani@gmail.com> wrote:
> > > Hello List,
> > >
> > > I have a small problem with the ring when transferring blocks the id
> > > on the response is different from the request.
> > > This is the boot up read, count 0.
> > > The guest requests block 0, it has to be located at 7c00.
> > >
> > > I go ahead and create a REQUEST with this data:
> > > ring_req = RING_GET_REQUEST(priv,priv->req_prod_pvt);
> > > ring_req->id = 9;
> > > ring_req->nr_segments=1;
> > > ring_req->operation = BLKIF_OP_READ; ring_req->sector_number =
> > > (int)op->lba; //sector to be read ring_req->seg[0].gref =
> > > (bi->buffer_gref); //this should be get_free_gref();
> > > ring_req->seg[0].first_sect = 0;//op->lba;
> > > ring_req->seg[0].last_sect = 7;//op->lba + op->count;
> > >
> > > RING_PUSH_REQUESTS_AND_CHECK_NOTIFY((priv),notify);  //return
> > notify=0
> > > if(notify){
> > > 		dprintf(1,"Start notify procedure\n");
> > > 		evtchn_send_t send;
> > > 		send.port = (bi->port);
> > > 		dprintf(1,"In notify before hypercall port is %d\n",send.port);
> > > 		//hypercall_event_channel_op(EVTCHNOP_send, &send);
> > > 		dprintf(1,"HYPERCALL read operation notify res %d\n",
> > > hypercall_event_channel_op(EVTCHNOP_send,&send));
> > > }
> > > ring_res = RING_GET_RESPONSE((priv),(temp->rsp_prod));
> > > Then I get:
> > > FAIL RING RESPONSE 0x0009a040 id:256 status:9 operation 0 this is
> > > the line:
> > > dprintf(1,"FAIL RING RESPONSE %p id:%d status:%d operation %d\n",
> > > ring_res,ring_res->id,ring_res->status,ring_res->operation);
> > >
> > >
> > > The id should be the same on both the request and response, there
> > > are bi ither requests on the fly.
> > >
> > > Any clue?=
> >
> > With status happening to be 9 when id should be, is this perhaps a
> > broken response structure definition?
> >
> 
> That would be my guess too. The structure aligns differently under 64 and 32
> bits so I'd guess the OP is talking between the two.
> 

Further to this, Dom0 sets the "protocol" entry in the frontend xenstore (/local/domain/<domu id>/device/vbd/<vbd id>/protocol) to tell it what alignment it is using. I wrote GPLPV before this setting existed so I solved the problem by sending 2 invalid requests down the ring and checking the response. If the fields have shifted around I assume that dom0 is not the same bit width as the domu and switch structure definitions on the fly.

Are you writing a new frontend driver?

James

Re: Little help with blk ring

[Daniel Castro]

I am writing the PV Driver front end in Seabios.

Could you explain your method in a little more detail please?

Thanks,

Daniel

On Mon, May 7, 2012 at 5:53 PM, James Harper
<james.harper@bendigoit.com.au> wrote:
>> >
>> > >>> On 07.05.12 at 08:23, Daniel Castro <evil.dani@gmail.com> wrote:
>> > > Hello List,
>> > >
>> > > I have a small problem with the ring when transferring blocks the id
>> > > on the response is different from the request.
>> > > This is the boot up read, count 0.
>> > > The guest requests block 0, it has to be located at 7c00.
>> > >
>> > > I go ahead and create a REQUEST with this data:
>> > > ring_req = RING_GET_REQUEST(priv,priv->req_prod_pvt);
>> > > ring_req->id = 9;
>> > > ring_req->nr_segments=1;
>> > > ring_req->operation = BLKIF_OP_READ; ring_req->sector_number =
>> > > (int)op->lba; //sector to be read ring_req->seg[0].gref =
>> > > (bi->buffer_gref); //this should be get_free_gref();
>> > > ring_req->seg[0].first_sect = 0;//op->lba;
>> > > ring_req->seg[0].last_sect = 7;//op->lba + op->count;
>> > >
>> > > RING_PUSH_REQUESTS_AND_CHECK_NOTIFY((priv),notify);  //return
>> > notify=0
>> > > if(notify){
>> > >           dprintf(1,"Start notify procedure\n");
>> > >           evtchn_send_t send;
>> > >           send.port = (bi->port);
>> > >           dprintf(1,"In notify before hypercall port is %d\n",send.port);
>> > >           //hypercall_event_channel_op(EVTCHNOP_send, &send);
>> > >           dprintf(1,"HYPERCALL read operation notify res %d\n",
>> > > hypercall_event_channel_op(EVTCHNOP_send,&send));
>> > > }
>> > > ring_res = RING_GET_RESPONSE((priv),(temp->rsp_prod));
>> > > Then I get:
>> > > FAIL RING RESPONSE 0x0009a040 id:256 status:9 operation 0 this is
>> > > the line:
>> > > dprintf(1,"FAIL RING RESPONSE %p id:%d status:%d operation %d\n",
>> > > ring_res,ring_res->id,ring_res->status,ring_res->operation);
>> > >
>> > >
>> > > The id should be the same on both the request and response, there
>> > > are bi ither requests on the fly.
>> > >
>> > > Any clue?=
>> >
>> > With status happening to be 9 when id should be, is this perhaps a
>> > broken response structure definition?
>> >
>>
>> That would be my guess too. The structure aligns differently under 64 and 32
>> bits so I'd guess the OP is talking between the two.
>>
>
> Further to this, Dom0 sets the "protocol" entry in the frontend xenstore (/local/domain/<domu id>/device/vbd/<vbd id>/protocol) to tell it what alignment it is using. I wrote GPLPV before this setting existed so I solved the problem by sending 2 invalid requests down the ring and checking the response. If the fields have shifted around I assume that dom0 is not the same bit width as the domu and switch structure definitions on the fly.
>
> Are you writing a new frontend driver?
>
> James
>



-- 
+-=====---------------------------+
| +---------------------------------+ | This space intentionally blank
for notetaking.
| |   | Daniel Castro,                |
| |   | Consultant/Programmer.|
| |   | U Andes                         |
+-------------------------------------+

Re: Little help with blk ring

[James Harper]

> 
> I am writing the PV Driver front end in Seabios.
> 
> Could you explain your method in a little more detail please?
> 

I'm not sure that my way is the best way. The existing linux pv drivers should do what you want - have a look at the source. If you really want to look at my code you can get it from hg and have a look. It's in the xenvbd driver.

And I think I got it backwards in a previous email. It seems that the frontend writes the protocol into the xenstore, eg "x86_64-abi" for 64 bit. You should probably just make sure your structures are correctly laid out for a 32 bit system and then write the correct protocol string into the frontend when you set up communications and that should ensure that it will work on all but the most ancient Xen implementations.

James

Re: Little help with blk ring

[Ian Campbell]

On Wed, 2012-05-09 at 08:54 +0100, James Harper wrote:
> > 
> > I am writing the PV Driver front end in Seabios.
> > 
> > Could you explain your method in a little more detail please?
> > 
> 
> I'm not sure that my way is the best way. The existing linux pv
> drivers should do what you want - have a look at the source. If you
> really want to look at my code you can get it from hg and have a look.
> It's in the xenvbd driver.
> 
> And I think I got it backwards in a previous email. It seems that the
> frontend writes the protocol into the xenstore, eg "x86_64-abi" for 64
> bit.

That's right, xen/include/public/io/protocols.h defines the valid
protocols, your frontend should publish the one it implements (probably
x86_32-abi for a SeaBIOS driver).

>  You should probably just make sure your structures are correctly laid
> out for a 32 bit system and then write the correct protocol string
> into the frontend when you set up communications and that should
> ensure that it will work on all but the most ancient Xen
> implementations.
> 
> James
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Little help with blk ring

[Daniel Castro]

On Wed, May 9, 2012 at 5:18 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Wed, 2012-05-09 at 08:54 +0100, James Harper wrote:
>> >
>> > I am writing the PV Driver front end in Seabios.
>> >
>> > Could you explain your method in a little more detail please?
>> >
>>
>> I'm not sure that my way is the best way. The existing linux pv
>> drivers should do what you want - have a look at the source. If you
>> really want to look at my code you can get it from hg and have a look.
>> It's in the xenvbd driver.
>>
>> And I think I got it backwards in a previous email. It seems that the
>> frontend writes the protocol into the xenstore, eg "x86_64-abi" for 64
>> bit.
I was not doing this, now it is written in xenstore, as Ian´s
suggestion 32-abi, yet the same problem persists...
After RING RESPONSE 0x0009a040
status:9
operation 0
id:256
This above is the content of the response. The id should be 9. If I
change the id the response status will change to the same number. Any
ideas?

Thanks,

Daniel

>
> That's right, xen/include/public/io/protocols.h defines the valid
> protocols, your frontend should publish the one it implements (probably
> x86_32-abi for a SeaBIOS driver).
>
>>  You should probably just make sure your structures are correctly laid
>> out for a 32 bit system and then write the correct protocol string
>> into the frontend when you set up communications and that should
>> ensure that it will work on all but the most ancient Xen
>> implementations.
>>
>> James
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>
>



-- 
+-=====---------------------------+
| +---------------------------------+ | This space intentionally blank
for notetaking.
| |   | Daniel Castro,                |
| |   | Consultant/Programmer.|
| |   | U Andes                         |
+-------------------------------------+

Re: Little help with blk ring

[Ian Campbell]

On Tue, 2012-05-15 at 12:09 +0100, Daniel Castro wrote:
> On Wed, May 9, 2012 at 5:18 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > On Wed, 2012-05-09 at 08:54 +0100, James Harper wrote:
> >> >
> >> > I am writing the PV Driver front end in Seabios.
> >> >
> >> > Could you explain your method in a little more detail please?
> >> >
> >>
> >> I'm not sure that my way is the best way. The existing linux pv
> >> drivers should do what you want - have a look at the source. If you
> >> really want to look at my code you can get it from hg and have a look.
> >> It's in the xenvbd driver.
> >>
> >> And I think I got it backwards in a previous email. It seems that the
> >> frontend writes the protocol into the xenstore, eg "x86_64-abi" for 64
> >> bit.
> I was not doing this, now it is written in xenstore, as Ian´s
> suggestion 32-abi, yet the same problem persists...
> After RING RESPONSE 0x0009a040
> status:9
> operation 0
> id:256
> This above is the content of the response. The id should be 9. If I
> change the id the response status will change to the same number. Any
> ideas?

Compare the layout and data sizes of the members of your request and
response structures very carefully with the ones in xen/include/public.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Little help with blk ring

[Daniel Castro]

On Tue, May 15, 2012 at 10:00 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Tue, 2012-05-15 at 12:09 +0100, Daniel Castro wrote:
>> On Wed, May 9, 2012 at 5:18 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> > On Wed, 2012-05-09 at 08:54 +0100, James Harper wrote:
>> >> >
>> >> > I am writing the PV Driver front end in Seabios.
>> >> >
>> >> > Could you explain your method in a little more detail please?
>> >> >
>> >>
>> >> I'm not sure that my way is the best way. The existing linux pv
>> >> drivers should do what you want - have a look at the source. If you
>> >> really want to look at my code you can get it from hg and have a look.
>> >> It's in the xenvbd driver.
>> >>
>> >> And I think I got it backwards in a previous email. It seems that the
>> >> frontend writes the protocol into the xenstore, eg "x86_64-abi" for 64
>> >> bit.
>> I was not doing this, now it is written in xenstore, as Ian´s
>> suggestion 32-abi, yet the same problem persists...
>> After RING RESPONSE 0x0009a040
>> status:9
>> operation 0
>> id:256
>> This above is the content of the response. The id should be 9. If I
>> change the id the response status will change to the same number. Any
>> ideas?
>
> Compare the layout and data sizes of the members of your request and
> response structures very carefully with the ones in xen/include/public.

I have done this, and they are in fact identical. Here are the
addresses of the fields...
RING at 0x0009a040
0x0009a04a status:0
0x0009a048 operation 0
0x0009a040 id:256

What else could it be?

>
> Ian.
>
>



-- 
+-=====---------------------------+
| +---------------------------------+ | This space intentionally blank
for notetaking.
| |   | Daniel Castro,                |
| |   | Consultant/Programmer.|
| |   | U Andes                         |
+-------------------------------------+

Re: Little help with blk ring

[James Harper]

> I have done this, and they are in fact identical. Here are the addresses of the
> fields...
> RING at 0x0009a040
> 0x0009a04a status:0
> 0x0009a048 operation 0
> 0x0009a040 id:256
> 
> What else could it be?
> 

Is sizeof() what you expect it to be? That would cause the first request to be (mostly) okay, but subsequent requests to be further and further out of alignment.

James

Re: Little help with blk ring

[Daniel Castro]

On Tue, May 22, 2012 at 8:53 AM, James Harper
<james.harper@bendigoit.com.au> wrote:
>> I have done this, and they are in fact identical. Here are the addresses of the
>> fields...
>> RING at 0x0009a040
>> 0x0009a04a status:0
>> 0x0009a048 operation 0
>> 0x0009a040 id:256
>>
>> What else could it be?
>>
>
> Is sizeof() what you expect it to be? That would cause the first request to be (mostly) okay, but subsequent requests to be further and further out of alignment.

Right on the spot!
Here is the output of sizeof for the struct and for each member:
SIZEOF ring_res:12 id:8 operation:1 status:2
There is seems to be an alignment problem, them sum of members is 11,
but the whole struct is 12, the question now is, how can I tackle this
problem?

Daniel
>
> James



-- 
+-=====---------------------------+
| +---------------------------------+ | This space intentionally blank
for notetaking.
| |   | Daniel Castro,                |
| |   | Consultant/Programmer.|
| |   | U Andes                         |
+-------------------------------------+

Re: Little help with blk ring

[Jan Beulich]

>>> On 23.05.12 at 01:43, Daniel Castro <evil.dani@gmail.com> wrote:
> On Tue, May 22, 2012 at 8:53 AM, James Harper <james.harper@bendigoit.com.au> wrote:
>> Is sizeof() what you expect it to be? That would cause the first request to 
> be (mostly) okay, but subsequent requests to be further and further out of 
> alignment.
> 
> Right on the spot!
> Here is the output of sizeof for the struct and for each member:
> SIZEOF ring_res:12 id:8 operation:1 status:2
> There is seems to be an alignment problem, them sum of members is 11,
> but the whole struct is 12, the question now is, how can I tackle this
> problem?

But sizeof is 12, as expected. There should be no place in your
code where you (directly or indirectly) sum up the individual
members' sizes.

Jan

Re: Little help with blk ring

[Daniel Castro]

On Wed, May 23, 2012 at 5:20 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 23.05.12 at 01:43, Daniel Castro <evil.dani@gmail.com> wrote:
>> On Tue, May 22, 2012 at 8:53 AM, James Harper <james.harper@bendigoit.com.au> wrote:
>>> Is sizeof() what you expect it to be? That would cause the first request to
>> be (mostly) okay, but subsequent requests to be further and further out of
>> alignment.
>>
>> Right on the spot!
>> Here is the output of sizeof for the struct and for each member:
>> SIZEOF ring_res:12 id:8 operation:1 status:2
>> There is seems to be an alignment problem, them sum of members is 11,
>> but the whole struct is 12, the question now is, how can I tackle this
>> problem?
>
> But sizeof is 12, as expected. There should be no place in your
> code where you (directly or indirectly) sum up the individual
> members' sizes.

But if you add the members it is id:8 + operation:1 + status:2 = 11.
Am I missing something?
Jan, the problem is that I am getting id=256 and operation=real_id, so
James suggested to see if there was an alignment problem. In the back
driver there is no place in the code that would assign id=

Daniel

>
> Jan
>



-- 
+-=====---------------------------+
| +---------------------------------+ | This space intentionally blank
for notetaking.
| |   | Daniel Castro,                |
| |   | Consultant/Programmer.|
| |   | U Andes                         |
+-------------------------------------+

Re: Little help with blk ring

[Jan Beulich]

>>> On 23.05.12 at 10:54, Daniel Castro <evil.dani@gmail.com> wrote:
> On Wed, May 23, 2012 at 5:20 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 23.05.12 at 01:43, Daniel Castro <evil.dani@gmail.com> wrote:
>>> On Tue, May 22, 2012 at 8:53 AM, James Harper <james.harper@bendigoit.com.au> 
> wrote:
>>>> Is sizeof() what you expect it to be? That would cause the first request to
>>> be (mostly) okay, but subsequent requests to be further and further out of
>>> alignment.
>>>
>>> Right on the spot!
>>> Here is the output of sizeof for the struct and for each member:
>>> SIZEOF ring_res:12 id:8 operation:1 status:2
>>> There is seems to be an alignment problem, them sum of members is 11,
>>> but the whole struct is 12, the question now is, how can I tackle this
>>> problem?
>>
>> But sizeof is 12, as expected. There should be no place in your
>> code where you (directly or indirectly) sum up the individual
>> members' sizes.
> 
> But if you add the members it is id:8 + operation:1 + status:2 = 11.
> Am I missing something?

As said - the sum of the members' sizes should not be used
(and hence not matter) in your code. I'm not clear why you're
insisting on this calculation having any relevance.

Jan