All of lore.kernel.org
 help / color / mirror / Atom feed
* radosgw Segmentation fault on obj copy
@ 2013-12-02 12:19 Dominik Mostowiec
       [not found] ` <CAMNMNTzEVkom0cBe_jepaE95hzVGys+8y+-PA+2dSsTqNwg=tA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-02 12:19 UTC (permalink / raw)
  To: ceph-users-idqoXFIVOFJgJs9I8MT0rw, ceph-devel

Hi,
I have strange problem.
Obj copy (0 size) killing radosgw.

Head for this file:
Content-Type: application/octet-stream
Server: Apache/2.2.22 (Ubuntu)
ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
Last-Modified: 2013-12-01T10:37:15Z

rgw log.
2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
req=0x2be6fa0 =====
2013-12-02 08:18:59.196709 7f5308ff1700  2 req 237:0.000058::::initializing
2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>> HTTP_X_AMZ_ACL=public-read
2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
x-amz-copy-source:/testbucket/testfile.xml
2013-12-02 08:18:59.196773 7f5308ff1700 10 x>> x-amz-metadata-directive:COPY
2013-12-02 08:18:59.196786 7f5308ff1700 10
s->object=/testbucket/new_testfile.ini s->bucket=testbucket
2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
/testbucket/new_testfile.ini::getting op
2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
/testbucket/new_testfile.ini:copy_obj:authorizing
2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
dest=/testbucket/new_testfile.ini
2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
PUT
1B2M2Y8AsgTpgAmY7PhCfg==
application/octet-stream
Mon, 02 Dec 2013 07:18:55 GMT
x-amz-acl:public-read
x-amz-copy-source:/testbucket/testfile.xml
x-amz-metadata-directive:COPY
/testbucket/new_testfile.ini
2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
/testbucket/new_testfile.ini:copy_obj:reading permissions
2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
/testbucket/new_testfile.ini:copy_obj:verifying op permissions
2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
uid=0 mask=1
2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
(type)=1, policy perm=1, user_perm_mask=15, acl perm=1
2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
uid=0 mask=2
2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
(type)=2, policy perm=2, user_perm_mask=15, acl perm=2
2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
/testbucket/new_testfile.ini:copy_obj:verifying op params
2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
/testbucket/new_testfile.ini:copy_obj:executing
2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
x-amz-copy-source:/testbucket/testfile.xml
2013-12-02 08:18:59.207709 7f5308ff1700 10 x>> x-amz-metadata-directive:COPY
2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
(Segmentation fault) **
 in thread 7f5308ff1700


-- 
Regards
Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found] ` <CAMNMNTzEVkom0cBe_jepaE95hzVGys+8y+-PA+2dSsTqNwg=tA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-02 12:53   ` Dominik Mostowiec
       [not found]     ` <CAMNMNTzx_CHmYRXXE-KhbEwNZqLAAcTBp-5SZks7UiQ+OS1XqA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-02 12:53 UTC (permalink / raw)
  To: ceph-users-idqoXFIVOFJgJs9I8MT0rw, ceph-devel

Hi,
I found that issue is related with "ETag: ....-0" (ends -0)
This is known bug ?

--
Regards
Dominik

2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> Hi,
> I have strange problem.
> Obj copy (0 size) killing radosgw.
>
> Head for this file:
> Content-Type: application/octet-stream
> Server: Apache/2.2.22 (Ubuntu)
> ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
> Last-Modified: 2013-12-01T10:37:15Z
>
> rgw log.
> 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
> req=0x2be6fa0 =====
> 2013-12-02 08:18:59.196709 7f5308ff1700  2 req 237:0.000058::::initializing
> 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>> HTTP_X_AMZ_ACL=public-read
> 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
> HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
> 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
> HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
> 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
> 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
> x-amz-copy-source:/testbucket/testfile.xml
> 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>> x-amz-metadata-directive:COPY
> 2013-12-02 08:18:59.196786 7f5308ff1700 10
> s->object=/testbucket/new_testfile.ini s->bucket=testbucket
> 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
> /testbucket/new_testfile.ini::getting op
> 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
> /testbucket/new_testfile.ini:copy_obj:authorizing
> 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
> dest=/testbucket/new_testfile.ini
> 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
> PUT
> 1B2M2Y8AsgTpgAmY7PhCfg==
> application/octet-stream
> Mon, 02 Dec 2013 07:18:55 GMT
> x-amz-acl:public-read
> x-amz-copy-source:/testbucket/testfile.xml
> x-amz-metadata-directive:COPY
> /testbucket/new_testfile.ini
> 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
> /testbucket/new_testfile.ini:copy_obj:reading permissions
> 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
> /testbucket/new_testfile.ini:copy_obj:verifying op permissions
> 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
> 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
> uid=0 mask=1
> 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
> 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
> (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
> 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
> uid=0 mask=2
> 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
> 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
> (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
> 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
> /testbucket/new_testfile.ini:copy_obj:verifying op params
> 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
> /testbucket/new_testfile.ini:copy_obj:executing
> 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
> 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
> x-amz-copy-source:/testbucket/testfile.xml
> 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>> x-amz-metadata-directive:COPY
> 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
> testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
> testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
> 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
> (Segmentation fault) **
>  in thread 7f5308ff1700
>
>
> --
> Regards
> Dominik



-- 
Pozdrawiam
Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]     ` <CAMNMNTzx_CHmYRXXE-KhbEwNZqLAAcTBp-5SZks7UiQ+OS1XqA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-02 17:14       ` Yehuda Sadeh
       [not found]         ` <CABBk=J-HAPxi4eg0AW_vD0T+QoqRzBMchpovM9s+EhzzC2ZZmg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Yehuda Sadeh @ 2013-12-02 17:14 UTC (permalink / raw)
  To: Dominik Mostowiec; +Cc: ceph-devel, ceph-users


[-- Attachment #1.1: Type: text/plain, Size: 4190 bytes --]

That's unknown bug. I have a guess as to how the original object was
created. Can you read the original object, but only copy fails?
On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
wrote:

> Hi,
> I found that issue is related with "ETag: ....-0" (ends -0)
> This is known bug ?
>
> --
> Regards
> Dominik
>
> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> > Hi,
> > I have strange problem.
> > Obj copy (0 size) killing radosgw.
> >
> > Head for this file:
> > Content-Type: application/octet-stream
> > Server: Apache/2.2.22 (Ubuntu)
> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
> > Last-Modified: 2013-12-01T10:37:15Z
> >
> > rgw log.
> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
> > req=0x2be6fa0 =====
> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
> 237:0.000058::::initializing
> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
> HTTP_X_AMZ_ACL=public-read
> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
> > x-amz-copy-source:/testbucket/testfile.xml
> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
> x-amz-metadata-directive:COPY
> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
> > /testbucket/new_testfile.ini::getting op
> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
> > /testbucket/new_testfile.ini:copy_obj:authorizing
> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
> > dest=/testbucket/new_testfile.ini
> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
> > PUT
> > 1B2M2Y8AsgTpgAmY7PhCfg==
> > application/octet-stream
> > Mon, 02 Dec 2013 07:18:55 GMT
> > x-amz-acl:public-read
> > x-amz-copy-source:/testbucket/testfile.xml
> > x-amz-metadata-directive:COPY
> > /testbucket/new_testfile.ini
> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
> > /testbucket/new_testfile.ini:copy_obj:reading permissions
> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
> > uid=0 mask=1
> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
> > uid=0 mask=2
> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
> > /testbucket/new_testfile.ini:copy_obj:verifying op params
> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
> > /testbucket/new_testfile.ini:copy_obj:executing
> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
> > x-amz-copy-source:/testbucket/testfile.xml
> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
> x-amz-metadata-directive:COPY
> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
> > (Segmentation fault) **
> >  in thread 7f5308ff1700
> >
> >
> > --
> > Regards
> > Dominik
>
>
>
> --
> Pozdrawiam
> Dominik
> _______________________________________________
> ceph-users mailing list
> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>

[-- Attachment #1.2: Type: text/html, Size: 5257 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]         ` <CABBk=J-HAPxi4eg0AW_vD0T+QoqRzBMchpovM9s+EhzzC2ZZmg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-02 18:07           ` Dominik Mostowiec
  2013-12-02 20:08           ` Dominik Mostowiec
  1 sibling, 0 replies; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-02 18:07 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-users-idqoXFIVOFJgJs9I8MT0rw, ceph-devel


[-- Attachment #1.1: Type: text/plain, Size: 4459 bytes --]

Yes I can read it.  Oryginal object is 0 size.

Regards
Dominik
On Dec 2, 2013 6:14 PM, "Yehuda Sadeh" <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org> wrote:

> That's unknown bug. I have a guess as to how the original object was
> created. Can you read the original object, but only copy fails?
> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> wrote:
>
>> Hi,
>> I found that issue is related with "ETag: ....-0" (ends -0)
>> This is known bug ?
>>
>> --
>> Regards
>> Dominik
>>
>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>> > Hi,
>> > I have strange problem.
>> > Obj copy (0 size) killing radosgw.
>> >
>> > Head for this file:
>> > Content-Type: application/octet-stream
>> > Server: Apache/2.2.22 (Ubuntu)
>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>> > Last-Modified: 2013-12-01T10:37:15Z
>> >
>> > rgw log.
>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
>> > req=0x2be6fa0 =====
>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>> 237:0.000058::::initializing
>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>> HTTP_X_AMZ_ACL=public-read
>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>> > x-amz-copy-source:/testbucket/testfile.xml
>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>> x-amz-metadata-directive:COPY
>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
>> > /testbucket/new_testfile.ini::getting op
>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
>> > dest=/testbucket/new_testfile.ini
>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>> > PUT
>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>> > application/octet-stream
>> > Mon, 02 Dec 2013 07:18:55 GMT
>> > x-amz-acl:public-read
>> > x-amz-copy-source:/testbucket/testfile.xml
>> > x-amz-metadata-directive:COPY
>> > /testbucket/new_testfile.ini
>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
>> > uid=0 mask=1
>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
>> > uid=0 mask=2
>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:executing
>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>> > x-amz-copy-source:/testbucket/testfile.xml
>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>> x-amz-metadata-directive:COPY
>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>> > (Segmentation fault) **
>> >  in thread 7f5308ff1700
>> >
>> >
>> > --
>> > Regards
>> > Dominik
>>
>>
>>
>> --
>> Pozdrawiam
>> Dominik
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>

[-- Attachment #1.2: Type: text/html, Size: 5767 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]         ` <CABBk=J-HAPxi4eg0AW_vD0T+QoqRzBMchpovM9s+EhzzC2ZZmg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  2013-12-02 18:07           ` Dominik Mostowiec
@ 2013-12-02 20:08           ` Dominik Mostowiec
       [not found]             ` <CAMNMNTzMvb-udVW5LrOhJLCSWWP5KVMHHyLrHwsY=bv3ckXyGA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  1 sibling, 1 reply; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-02 20:08 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-devel, ceph-users

Hi,
Another file with the same problems:

2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new request
req=0x25406d0 =====
2013-12-01 11:37:15.556739 7f7891fd3700  2 req 1314:0.000052::::initializing
2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
s->bucket=testbucket
2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
/testbucket/files/192.txt::getting op
2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
/testbucket/files/192.txt:complete_multipart:authorizing
2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
POST

application/xml
Sun, 01 Dec 2013 10:37:10 GMT
/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
/testbucket/files/192.txt:complete_multipart:reading permissions
2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
/testbucket/files/192.txt:complete_multipart:verifying op permissions
2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
uid=0 mask=2
2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
(type)=2, policy perm=2, user_perm_mask=2, acl perm=2
2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
/testbucket/files/192.txt:complete_multipart:verifying op params
2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
/testbucket/files/192.txt:complete_multipart:executing
2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
d41d8cd98f00b204e9800998ecf8427e-0
2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
testbucket:files/192.txt to shadow object, tag/shadow_obj haven't been
set
2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
/testbucket/files/192.txt:complete_multipart:http status=200
2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
req=0x25406d0 http_status=200 ======

Yes, I can read oryginal object.

--
Regards
Dominik

2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> That's unknown bug. I have a guess as to how the original object was
> created. Can you read the original object, but only copy fails?
>
> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> wrote:
>>
>> Hi,
>> I found that issue is related with "ETag: ....-0" (ends -0)
>> This is known bug ?
>>
>> --
>> Regards
>> Dominik
>>
>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>> > Hi,
>> > I have strange problem.
>> > Obj copy (0 size) killing radosgw.
>> >
>> > Head for this file:
>> > Content-Type: application/octet-stream
>> > Server: Apache/2.2.22 (Ubuntu)
>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>> > Last-Modified: 2013-12-01T10:37:15Z
>> >
>> > rgw log.
>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
>> > req=0x2be6fa0 =====
>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>> > 237:0.000058::::initializing
>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>> > HTTP_X_AMZ_ACL=public-read
>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>> > x-amz-copy-source:/testbucket/testfile.xml
>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>> > x-amz-metadata-directive:COPY
>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
>> > /testbucket/new_testfile.ini::getting op
>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
>> > dest=/testbucket/new_testfile.ini
>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>> > PUT
>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>> > application/octet-stream
>> > Mon, 02 Dec 2013 07:18:55 GMT
>> > x-amz-acl:public-read
>> > x-amz-copy-source:/testbucket/testfile.xml
>> > x-amz-metadata-directive:COPY
>> > /testbucket/new_testfile.ini
>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
>> > uid=0 mask=1
>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
>> > uid=0 mask=2
>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
>> > /testbucket/new_testfile.ini:copy_obj:executing
>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>> > x-amz-copy-source:/testbucket/testfile.xml
>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>> > x-amz-metadata-directive:COPY
>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>> > (Segmentation fault) **
>> >  in thread 7f5308ff1700
>> >
>> >
>> > --
>> > Regards
>> > Dominik
>>
>>
>>
>> --
>> Pozdrawiam
>> Dominik
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



-- 
Pozdrawiam
Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]             ` <CAMNMNTzMvb-udVW5LrOhJLCSWWP5KVMHHyLrHwsY=bv3ckXyGA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-02 20:27               ` Yehuda Sadeh
       [not found]                 ` <CABBk=J-xw8G5Uuj1HGonP-B_Xzqn=A0myE-C28sWMMpnH9tbmA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Yehuda Sadeh @ 2013-12-02 20:27 UTC (permalink / raw)
  To: Dominik Mostowiec; +Cc: ceph-devel, ceph-users

By any chance are you uploading empty objects through the multipart upload api?

On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
<dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hi,
> Another file with the same problems:
>
> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new request
> req=0x25406d0 =====
> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req 1314:0.000052::::initializing
> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
> s->bucket=testbucket
> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
> /testbucket/files/192.txt::getting op
> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
> /testbucket/files/192.txt:complete_multipart:authorizing
> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
> POST
>
> application/xml
> Sun, 01 Dec 2013 10:37:10 GMT
> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
> /testbucket/files/192.txt:complete_multipart:reading permissions
> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
> /testbucket/files/192.txt:complete_multipart:verifying op permissions
> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
> uid=0 mask=2
> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
> /testbucket/files/192.txt:complete_multipart:verifying op params
> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
> /testbucket/files/192.txt:complete_multipart:executing
> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
> d41d8cd98f00b204e9800998ecf8427e-0
> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't been
> set
> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
> /testbucket/files/192.txt:complete_multipart:http status=200
> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
> req=0x25406d0 http_status=200 ======
>
> Yes, I can read oryginal object.
>
> --
> Regards
> Dominik
>
> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> That's unknown bug. I have a guess as to how the original object was
>> created. Can you read the original object, but only copy fails?
>>
>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>> wrote:
>>>
>>> Hi,
>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>> This is known bug ?
>>>
>>> --
>>> Regards
>>> Dominik
>>>
>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>>> > Hi,
>>> > I have strange problem.
>>> > Obj copy (0 size) killing radosgw.
>>> >
>>> > Head for this file:
>>> > Content-Type: application/octet-stream
>>> > Server: Apache/2.2.22 (Ubuntu)
>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>> > Last-Modified: 2013-12-01T10:37:15Z
>>> >
>>> > rgw log.
>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
>>> > req=0x2be6fa0 =====
>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>> > 237:0.000058::::initializing
>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>> > HTTP_X_AMZ_ACL=public-read
>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>> > x-amz-metadata-directive:COPY
>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
>>> > /testbucket/new_testfile.ini::getting op
>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
>>> > dest=/testbucket/new_testfile.ini
>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>> > PUT
>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>> > application/octet-stream
>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>> > x-amz-acl:public-read
>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> > x-amz-metadata-directive:COPY
>>> > /testbucket/new_testfile.ini
>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
>>> > uid=0 mask=1
>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
>>> > uid=0 mask=2
>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>> > x-amz-metadata-directive:COPY
>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>>> > (Segmentation fault) **
>>> >  in thread 7f5308ff1700
>>> >
>>> >
>>> > --
>>> > Regards
>>> > Dominik
>>>
>>>
>>>
>>> --
>>> Pozdrawiam
>>> Dominik
>>> _______________________________________________
>>> ceph-users mailing list
>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
>
> --
> Pozdrawiam
> Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]                 ` <CABBk=J-xw8G5Uuj1HGonP-B_Xzqn=A0myE-C28sWMMpnH9tbmA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-02 20:32                   ` Dominik Mostowiec
       [not found]                     ` <CAMNMNTxYE=nqg04X7Wa0U2g3iW-HfkDBCEY_fv75D+nE7=xz4w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-02 20:32 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-devel, ceph-users

Yes, this is probably upload empty file.
This is the problem?

--
Regards
Dominik


2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> By any chance are you uploading empty objects through the multipart upload api?
>
> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> Hi,
>> Another file with the same problems:
>>
>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new request
>> req=0x25406d0 =====
>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req 1314:0.000052::::initializing
>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
>> s->bucket=testbucket
>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
>> /testbucket/files/192.txt::getting op
>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
>> /testbucket/files/192.txt:complete_multipart:authorizing
>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>> POST
>>
>> application/xml
>> Sun, 01 Dec 2013 10:37:10 GMT
>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
>> /testbucket/files/192.txt:complete_multipart:reading permissions
>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
>> /testbucket/files/192.txt:complete_multipart:verifying op permissions
>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
>> uid=0 mask=2
>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
>> /testbucket/files/192.txt:complete_multipart:verifying op params
>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
>> /testbucket/files/192.txt:complete_multipart:executing
>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>> d41d8cd98f00b204e9800998ecf8427e-0
>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't been
>> set
>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
>> /testbucket/files/192.txt:complete_multipart:http status=200
>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>> req=0x25406d0 http_status=200 ======
>>
>> Yes, I can read oryginal object.
>>
>> --
>> Regards
>> Dominik
>>
>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>> That's unknown bug. I have a guess as to how the original object was
>>> created. Can you read the original object, but only copy fails?
>>>
>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>> wrote:
>>>>
>>>> Hi,
>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>>> This is known bug ?
>>>>
>>>> --
>>>> Regards
>>>> Dominik
>>>>
>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>>>> > Hi,
>>>> > I have strange problem.
>>>> > Obj copy (0 size) killing radosgw.
>>>> >
>>>> > Head for this file:
>>>> > Content-Type: application/octet-stream
>>>> > Server: Apache/2.2.22 (Ubuntu)
>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>>> > Last-Modified: 2013-12-01T10:37:15Z
>>>> >
>>>> > rgw log.
>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
>>>> > req=0x2be6fa0 =====
>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>>> > 237:0.000058::::initializing
>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>>> > HTTP_X_AMZ_ACL=public-read
>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>>> > x-amz-metadata-directive:COPY
>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
>>>> > /testbucket/new_testfile.ini::getting op
>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
>>>> > dest=/testbucket/new_testfile.ini
>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>>> > PUT
>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>>> > application/octet-stream
>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>>> > x-amz-acl:public-read
>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>> > x-amz-metadata-directive:COPY
>>>> > /testbucket/new_testfile.ini
>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
>>>> > uid=0 mask=1
>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
>>>> > uid=0 mask=2
>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>>> > x-amz-metadata-directive:COPY
>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>>>> > (Segmentation fault) **
>>>> >  in thread 7f5308ff1700
>>>> >
>>>> >
>>>> > --
>>>> > Regards
>>>> > Dominik
>>>>
>>>>
>>>>
>>>> --
>>>> Pozdrawiam
>>>> Dominik
>>>> _______________________________________________
>>>> ceph-users mailing list
>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>>
>>
>> --
>> Pozdrawiam
>> Dominik



-- 
Pozdrawiam
Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]                     ` <CAMNMNTxYE=nqg04X7Wa0U2g3iW-HfkDBCEY_fv75D+nE7=xz4w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-02 20:45                       ` Yehuda Sadeh
  2013-12-02 20:54                         ` [ceph-users] " Dominik Mostowiec
  0 siblings, 1 reply; 19+ messages in thread
From: Yehuda Sadeh @ 2013-12-02 20:45 UTC (permalink / raw)
  To: Dominik Mostowiec; +Cc: ceph-devel, ceph-users

Looks like it. There should be a guard against it (mulitpart upload
minimum is 5M).

On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
<dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Yes, this is probably upload empty file.
> This is the problem?
>
> --
> Regards
> Dominik
>
>
> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> By any chance are you uploading empty objects through the multipart upload api?
>>
>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> Hi,
>>> Another file with the same problems:
>>>
>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new request
>>> req=0x25406d0 =====
>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req 1314:0.000052::::initializing
>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
>>> s->bucket=testbucket
>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
>>> /testbucket/files/192.txt::getting op
>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
>>> /testbucket/files/192.txt:complete_multipart:authorizing
>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>>> POST
>>>
>>> application/xml
>>> Sun, 01 Dec 2013 10:37:10 GMT
>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
>>> /testbucket/files/192.txt:complete_multipart:reading permissions
>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
>>> /testbucket/files/192.txt:complete_multipart:verifying op permissions
>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
>>> uid=0 mask=2
>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
>>> /testbucket/files/192.txt:complete_multipart:verifying op params
>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
>>> /testbucket/files/192.txt:complete_multipart:executing
>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>>> d41d8cd98f00b204e9800998ecf8427e-0
>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't been
>>> set
>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
>>> /testbucket/files/192.txt:complete_multipart:http status=200
>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>>> req=0x25406d0 http_status=200 ======
>>>
>>> Yes, I can read oryginal object.
>>>
>>> --
>>> Regards
>>> Dominik
>>>
>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>>> That's unknown bug. I have a guess as to how the original object was
>>>> created. Can you read the original object, but only copy fails?
>>>>
>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>>>> This is known bug ?
>>>>>
>>>>> --
>>>>> Regards
>>>>> Dominik
>>>>>
>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>>>>> > Hi,
>>>>> > I have strange problem.
>>>>> > Obj copy (0 size) killing radosgw.
>>>>> >
>>>>> > Head for this file:
>>>>> > Content-Type: application/octet-stream
>>>>> > Server: Apache/2.2.22 (Ubuntu)
>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>>>>> >
>>>>> > rgw log.
>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
>>>>> > req=0x2be6fa0 =====
>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>>>> > 237:0.000058::::initializing
>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>>>> > HTTP_X_AMZ_ACL=public-read
>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>>>> > x-amz-metadata-directive:COPY
>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
>>>>> > /testbucket/new_testfile.ini::getting op
>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
>>>>> > dest=/testbucket/new_testfile.ini
>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>>>> > PUT
>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>>>> > application/octet-stream
>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>>>> > x-amz-acl:public-read
>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>> > x-amz-metadata-directive:COPY
>>>>> > /testbucket/new_testfile.ini
>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
>>>>> > uid=0 mask=1
>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
>>>>> > uid=0 mask=2
>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>>>> > x-amz-metadata-directive:COPY
>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>>>>> > (Segmentation fault) **
>>>>> >  in thread 7f5308ff1700
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Regards
>>>>> > Dominik
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Pozdrawiam
>>>>> Dominik
>>>>> _______________________________________________
>>>>> ceph-users mailing list
>>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>>>
>>>
>>> --
>>> Pozdrawiam
>>> Dominik
>
>
>
> --
> Pozdrawiam
> Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [ceph-users] radosgw Segmentation fault on obj copy
  2013-12-02 20:45                       ` Yehuda Sadeh
@ 2013-12-02 20:54                         ` Dominik Mostowiec
  2013-12-02 22:16                           ` Yehuda Sadeh
  0 siblings, 1 reply; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-02 20:54 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-devel, ceph-users

You're right.

S3 api doc: http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
"Err:EntityTooSmall
Your proposed upload is smaller than the minimum allowed object size.
Each part must be at least 5 MB in size, except the last part."

Thanks.

This error should be triggered from radosgw also.

--
Regards
Dominik

2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
> Looks like it. There should be a guard against it (mulitpart upload
> minimum is 5M).
>
> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
> <dominikmostowiec@gmail.com> wrote:
>> Yes, this is probably upload empty file.
>> This is the problem?
>>
>> --
>> Regards
>> Dominik
>>
>>
>> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>> By any chance are you uploading empty objects through the multipart upload api?
>>>
>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>>> <dominikmostowiec@gmail.com> wrote:
>>>> Hi,
>>>> Another file with the same problems:
>>>>
>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new request
>>>> req=0x25406d0 =====
>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req 1314:0.000052::::initializing
>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
>>>> s->bucket=testbucket
>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
>>>> /testbucket/files/192.txt::getting op
>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
>>>> /testbucket/files/192.txt:complete_multipart:authorizing
>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>>>> POST
>>>>
>>>> application/xml
>>>> Sun, 01 Dec 2013 10:37:10 GMT
>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
>>>> /testbucket/files/192.txt:complete_multipart:reading permissions
>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
>>>> /testbucket/files/192.txt:complete_multipart:verifying op permissions
>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
>>>> uid=0 mask=2
>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
>>>> /testbucket/files/192.txt:complete_multipart:verifying op params
>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
>>>> /testbucket/files/192.txt:complete_multipart:executing
>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>>>> d41d8cd98f00b204e9800998ecf8427e-0
>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't been
>>>> set
>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
>>>> /testbucket/files/192.txt:complete_multipart:http status=200
>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>>>> req=0x25406d0 http_status=200 ======
>>>>
>>>> Yes, I can read oryginal object.
>>>>
>>>> --
>>>> Regards
>>>> Dominik
>>>>
>>>> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>>>> That's unknown bug. I have a guess as to how the original object was
>>>>> created. Can you read the original object, but only copy fails?
>>>>>
>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>>>>> This is known bug ?
>>>>>>
>>>>>> --
>>>>>> Regards
>>>>>> Dominik
>>>>>>
>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec@gmail.com>:
>>>>>> > Hi,
>>>>>> > I have strange problem.
>>>>>> > Obj copy (0 size) killing radosgw.
>>>>>> >
>>>>>> > Head for this file:
>>>>>> > Content-Type: application/octet-stream
>>>>>> > Server: Apache/2.2.22 (Ubuntu)
>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>>>>>> >
>>>>>> > rgw log.
>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
>>>>>> > req=0x2be6fa0 =====
>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>>>>> > 237:0.000058::::initializing
>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>>>>> > HTTP_X_AMZ_ACL=public-read
>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>>>>> > x-amz-metadata-directive:COPY
>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
>>>>>> > /testbucket/new_testfile.ini::getting op
>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
>>>>>> > dest=/testbucket/new_testfile.ini
>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>>>>> > PUT
>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>>>>> > application/octet-stream
>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>>>>> > x-amz-acl:public-read
>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>> > x-amz-metadata-directive:COPY
>>>>>> > /testbucket/new_testfile.ini
>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
>>>>>> > uid=0 mask=1
>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
>>>>>> > uid=0 mask=2
>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>>>>> > x-amz-metadata-directive:COPY
>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>>>>>> > (Segmentation fault) **
>>>>>> >  in thread 7f5308ff1700
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Regards
>>>>>> > Dominik
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Pozdrawiam
>>>>>> Dominik
>>>>>> _______________________________________________
>>>>>> ceph-users mailing list
>>>>>> ceph-users@lists.ceph.com
>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>>
>>>>
>>>>
>>>> --
>>>> Pozdrawiam
>>>> Dominik
>>
>>
>>
>> --
>> Pozdrawiam
>> Dominik



-- 
Pozdrawiam
Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: [ceph-users] radosgw Segmentation fault on obj copy
  2013-12-02 20:54                         ` [ceph-users] " Dominik Mostowiec
@ 2013-12-02 22:16                           ` Yehuda Sadeh
       [not found]                             ` <CABBk=J8GeCckYaxFu+_8jwagKtt2vOkmMsQ=1vii+MR7-erYjQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Yehuda Sadeh @ 2013-12-02 22:16 UTC (permalink / raw)
  To: Dominik Mostowiec; +Cc: ceph-devel, ceph-users

Actually, I read that differently. It only says that if there's more
than 1 part, all parts except for the last one need to be > 5M. Which
means that for uploads that are smaller than 5M there should be zero
or one parts.

On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
<dominikmostowiec@gmail.com> wrote:
> You're right.
>
> S3 api doc: http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
> "Err:EntityTooSmall
> Your proposed upload is smaller than the minimum allowed object size.
> Each part must be at least 5 MB in size, except the last part."
>
> Thanks.
>
> This error should be triggered from radosgw also.
>
> --
> Regards
> Dominik
>
> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>> Looks like it. There should be a guard against it (mulitpart upload
>> minimum is 5M).
>>
>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
>> <dominikmostowiec@gmail.com> wrote:
>>> Yes, this is probably upload empty file.
>>> This is the problem?
>>>
>>> --
>>> Regards
>>> Dominik
>>>
>>>
>>> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>>> By any chance are you uploading empty objects through the multipart upload api?
>>>>
>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>>>> <dominikmostowiec@gmail.com> wrote:
>>>>> Hi,
>>>>> Another file with the same problems:
>>>>>
>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new request
>>>>> req=0x25406d0 =====
>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req 1314:0.000052::::initializing
>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
>>>>> s->bucket=testbucket
>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
>>>>> /testbucket/files/192.txt::getting op
>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
>>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>>>>> POST
>>>>>
>>>>> application/xml
>>>>> Sun, 01 Dec 2013 10:37:10 GMT
>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
>>>>> /testbucket/files/192.txt:complete_multipart:reading permissions
>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
>>>>> /testbucket/files/192.txt:complete_multipart:verifying op permissions
>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
>>>>> uid=0 mask=2
>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
>>>>> /testbucket/files/192.txt:complete_multipart:verifying op params
>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
>>>>> /testbucket/files/192.txt:complete_multipart:executing
>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>>>>> d41d8cd98f00b204e9800998ecf8427e-0
>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't been
>>>>> set
>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>>>>> req=0x25406d0 http_status=200 ======
>>>>>
>>>>> Yes, I can read oryginal object.
>>>>>
>>>>> --
>>>>> Regards
>>>>> Dominik
>>>>>
>>>>> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>>>>> That's unknown bug. I have a guess as to how the original object was
>>>>>> created. Can you read the original object, but only copy fails?
>>>>>>
>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>>>>>> This is known bug ?
>>>>>>>
>>>>>>> --
>>>>>>> Regards
>>>>>>> Dominik
>>>>>>>
>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec@gmail.com>:
>>>>>>> > Hi,
>>>>>>> > I have strange problem.
>>>>>>> > Obj copy (0 size) killing radosgw.
>>>>>>> >
>>>>>>> > Head for this file:
>>>>>>> > Content-Type: application/octet-stream
>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>>>>>>> >
>>>>>>> > rgw log.
>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
>>>>>>> > req=0x2be6fa0 =====
>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>>>>>> > 237:0.000058::::initializing
>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>>>>>> > HTTP_X_AMZ_ACL=public-read
>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>>>>>> > x-amz-metadata-directive:COPY
>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
>>>>>>> > /testbucket/new_testfile.ini::getting op
>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
>>>>>>> > dest=/testbucket/new_testfile.ini
>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>>>>>> > PUT
>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>>>>>> > application/octet-stream
>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>>>>>> > x-amz-acl:public-read
>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>>> > x-amz-metadata-directive:COPY
>>>>>>> > /testbucket/new_testfile.ini
>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
>>>>>>> > uid=0 mask=1
>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
>>>>>>> > uid=0 mask=2
>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>>>>>> > x-amz-metadata-directive:COPY
>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>>>>>>> > (Segmentation fault) **
>>>>>>> >  in thread 7f5308ff1700
>>>>>>> >
>>>>>>> >
>>>>>>> > --
>>>>>>> > Regards
>>>>>>> > Dominik
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Pozdrawiam
>>>>>>> Dominik
>>>>>>> _______________________________________________
>>>>>>> ceph-users mailing list
>>>>>>> ceph-users@lists.ceph.com
>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Pozdrawiam
>>>>> Dominik
>>>
>>>
>>>
>>> --
>>> Pozdrawiam
>>> Dominik
>
>
>
> --
> Pozdrawiam
> Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]                             ` <CABBk=J8GeCckYaxFu+_8jwagKtt2vOkmMsQ=1vii+MR7-erYjQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-02 22:30                               ` Yehuda Sadeh
       [not found]                                 ` <CABBk=J9a3M6yAZj40+XZKLfmoHj7xsYj3GTfs34Be8twSb19UQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Yehuda Sadeh @ 2013-12-02 22:30 UTC (permalink / raw)
  To: Dominik Mostowiec; +Cc: ceph-devel, ceph-users

I'm having trouble reproducing the issue. What version are you using?

Thanks,
Yehuda

On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org> wrote:
> Actually, I read that differently. It only says that if there's more
> than 1 part, all parts except for the last one need to be > 5M. Which
> means that for uploads that are smaller than 5M there should be zero
> or one parts.
>
> On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> You're right.
>>
>> S3 api doc: http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
>> "Err:EntityTooSmall
>> Your proposed upload is smaller than the minimum allowed object size.
>> Each part must be at least 5 MB in size, except the last part."
>>
>> Thanks.
>>
>> This error should be triggered from radosgw also.
>>
>> --
>> Regards
>> Dominik
>>
>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>> Looks like it. There should be a guard against it (mulitpart upload
>>> minimum is 5M).
>>>
>>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>> Yes, this is probably upload empty file.
>>>> This is the problem?
>>>>
>>>> --
>>>> Regards
>>>> Dominik
>>>>
>>>>
>>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>>>> By any chance are you uploading empty objects through the multipart upload api?
>>>>>
>>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>>>> Hi,
>>>>>> Another file with the same problems:
>>>>>>
>>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new request
>>>>>> req=0x25406d0 =====
>>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req 1314:0.000052::::initializing
>>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
>>>>>> s->bucket=testbucket
>>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
>>>>>> /testbucket/files/192.txt::getting op
>>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
>>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
>>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
>>>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>>>>>> POST
>>>>>>
>>>>>> application/xml
>>>>>> Sun, 01 Dec 2013 10:37:10 GMT
>>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
>>>>>> /testbucket/files/192.txt:complete_multipart:reading permissions
>>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
>>>>>> /testbucket/files/192.txt:complete_multipart:verifying op permissions
>>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
>>>>>> uid=0 mask=2
>>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
>>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
>>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
>>>>>> /testbucket/files/192.txt:complete_multipart:verifying op params
>>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
>>>>>> /testbucket/files/192.txt:complete_multipart:executing
>>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>>>>>> d41d8cd98f00b204e9800998ecf8427e-0
>>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't been
>>>>>> set
>>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
>>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
>>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>>>>>> req=0x25406d0 http_status=200 ======
>>>>>>
>>>>>> Yes, I can read oryginal object.
>>>>>>
>>>>>> --
>>>>>> Regards
>>>>>> Dominik
>>>>>>
>>>>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>>>>>> That's unknown bug. I have a guess as to how the original object was
>>>>>>> created. Can you read the original object, but only copy fails?
>>>>>>>
>>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>>>>>>> This is known bug ?
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards
>>>>>>>> Dominik
>>>>>>>>
>>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>>>>>>>> > Hi,
>>>>>>>> > I have strange problem.
>>>>>>>> > Obj copy (0 size) killing radosgw.
>>>>>>>> >
>>>>>>>> > Head for this file:
>>>>>>>> > Content-Type: application/octet-stream
>>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
>>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>>>>>>>> >
>>>>>>>> > rgw log.
>>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new request
>>>>>>>> > req=0x2be6fa0 =====
>>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>>>>>>> > 237:0.000058::::initializing
>>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>>>>>>> > HTTP_X_AMZ_ACL=public-read
>>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>>>>>>> > x-amz-metadata-directive:COPY
>>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req 237:0.000141:s3:PUT
>>>>>>>> > /testbucket/new_testfile.ini::getting op
>>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req 237:0.000146:s3:PUT
>>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
>>>>>>>> > dest=/testbucket/new_testfile.ini
>>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>>>>>>> > PUT
>>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>>>>>>> > application/octet-stream
>>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>>>>>>> > x-amz-acl:public-read
>>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>>>> > x-amz-metadata-directive:COPY
>>>>>>>> > /testbucket/new_testfile.ini
>>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req 237:0.004066:s3:PUT
>>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req 237:0.006679:s3:PUT
>>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size = 0
>>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching permissions for
>>>>>>>> > uid=0 mask=1
>>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
>>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching permissions for
>>>>>>>> > uid=0 mask=2
>>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
>>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req 237:0.011030:s3:PUT
>>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req 237:0.011035:s3:PUT
>>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> x-amz-acl:public-read
>>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>>>>>>> > x-amz-metadata-directive:COPY
>>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>>>>>>>> > (Segmentation fault) **
>>>>>>>> >  in thread 7f5308ff1700
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > --
>>>>>>>> > Regards
>>>>>>>> > Dominik
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Pozdrawiam
>>>>>>>> Dominik
>>>>>>>> _______________________________________________
>>>>>>>> ceph-users mailing list
>>>>>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Pozdrawiam
>>>>>> Dominik
>>>>
>>>>
>>>>
>>>> --
>>>> Pozdrawiam
>>>> Dominik
>>
>>
>>
>> --
>> Pozdrawiam
>> Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]                                 ` <CABBk=J9a3M6yAZj40+XZKLfmoHj7xsYj3GTfs34Be8twSb19UQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-03  2:13                                   ` Dominik Mostowiec
  2013-12-03  2:19                                   ` Dominik Mostowiec
  1 sibling, 0 replies; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-03  2:13 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-users-idqoXFIVOFJgJs9I8MT0rw, ceph-devel


[-- Attachment #1.1: Type: text/plain, Size: 9971 bytes --]

0.56.7
On Dec 2, 2013 11:30 PM, "Yehuda Sadeh" <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org> wrote:

> I'm having trouble reproducing the issue. What version are you using?
>
> Thanks,
> Yehuda
>
> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org> wrote:
> > Actually, I read that differently. It only says that if there's more
> > than 1 part, all parts except for the last one need to be > 5M. Which
> > means that for uploads that are smaller than 5M there should be zero
> > or one parts.
> >
> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
> > <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >> You're right.
> >>
> >> S3 api doc:
> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
> >> "Err:EntityTooSmall
> >> Your proposed upload is smaller than the minimum allowed object size.
> >> Each part must be at least 5 MB in size, except the last part."
> >>
> >> Thanks.
> >>
> >> This error should be triggered from radosgw also.
> >>
> >> --
> >> Regards
> >> Dominik
> >>
> >> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> >>> Looks like it. There should be a guard against it (mulitpart upload
> >>> minimum is 5M).
> >>>
> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
> >>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >>>> Yes, this is probably upload empty file.
> >>>> This is the problem?
> >>>>
> >>>> --
> >>>> Regards
> >>>> Dominik
> >>>>
> >>>>
> >>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> >>>>> By any chance are you uploading empty objects through the multipart
> upload api?
> >>>>>
> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
> >>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >>>>>> Hi,
> >>>>>> Another file with the same problems:
> >>>>>>
> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new
> request
> >>>>>> req=0x25406d0 =====
> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req
> 1314:0.000052::::initializing
> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
> >>>>>> s->bucket=testbucket
> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
> >>>>>> /testbucket/files/192.txt::getting op
> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
> >>>>>>
> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
> >>>>>> POST
> >>>>>>
> >>>>>> application/xml
> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT
> >>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:reading permissions
> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
> permissions
> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
> >>>>>> uid=0 mask=2
> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op params
> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:executing
> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0
> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't
> been
> >>>>>> set
> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
> >>>>>> req=0x25406d0 http_status=200 ======
> >>>>>>
> >>>>>> Yes, I can read oryginal object.
> >>>>>>
> >>>>>> --
> >>>>>> Regards
> >>>>>> Dominik
> >>>>>>
> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> >>>>>>> That's unknown bug. I have a guess as to how the original object
> was
> >>>>>>> created. Can you read the original object, but only copy fails?
> >>>>>>>
> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <
> dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
> >>>>>>>> This is known bug ?
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Regards
> >>>>>>>> Dominik
> >>>>>>>>
> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> >>>>>>>> > Hi,
> >>>>>>>> > I have strange problem.
> >>>>>>>> > Obj copy (0 size) killing radosgw.
> >>>>>>>> >
> >>>>>>>> > Head for this file:
> >>>>>>>> > Content-Type: application/octet-stream
> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
> >>>>>>>> >
> >>>>>>>> > rgw log.
> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new
> request
> >>>>>>>> > req=0x2be6fa0 =====
> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
> >>>>>>>> > 237:0.000058::::initializing
> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
> >>>>>>>> > HTTP_X_AMZ_ACL=public-read
> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>>
> x-amz-acl:public-read
> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
> >>>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req
> 237:0.000141:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini::getting op
> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req
> 237:0.000146:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
> >>>>>>>> > dest=/testbucket/new_testfile.ini
> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
> >>>>>>>> > PUT
> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
> >>>>>>>> > application/octet-stream
> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
> >>>>>>>> > x-amz-acl:public-read
> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>>>>>> > /testbucket/new_testfile.ini
> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req
> 237:0.004066:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req
> 237:0.006679:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size
> = 0
> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching
> permissions for
> >>>>>>>> > uid=0 mask=1
> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching
> permissions for
> >>>>>>>> > uid=0 mask=2
> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req
> 237:0.011030:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req
> 237:0.011035:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>>
> x-amz-acl:public-read
> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
> >>>>>>>> > (Segmentation fault) **
> >>>>>>>> >  in thread 7f5308ff1700
> >>>>>>>> >
> >>>>>>>> >
> >>>>>>>> > --
> >>>>>>>> > Regards
> >>>>>>>> > Dominik
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Pozdrawiam
> >>>>>>>> Dominik
> >>>>>>>> _______________________________________________
> >>>>>>>> ceph-users mailing list
> >>>>>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Pozdrawiam
> >>>>>> Dominik
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Pozdrawiam
> >>>> Dominik
> >>
> >>
> >>
> >> --
> >> Pozdrawiam
> >> Dominik
>

[-- Attachment #1.2: Type: text/html, Size: 15999 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]                                 ` <CABBk=J9a3M6yAZj40+XZKLfmoHj7xsYj3GTfs34Be8twSb19UQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  2013-12-03  2:13                                   ` Dominik Mostowiec
@ 2013-12-03  2:19                                   ` Dominik Mostowiec
       [not found]                                     ` <CAMNMNTwk=q0aStrqJACqnB64AhGSH13LwNGzb3hC1DQMeZtwOg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  1 sibling, 1 reply; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-03  2:19 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-devel, ceph-users


[-- Attachment #1.1: Type: text/plain, Size: 10376 bytes --]

0.56.7

W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał:

> I'm having trouble reproducing the issue. What version are you using?
>
> Thanks,
> Yehuda
>
> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org<javascript:;>>
> wrote:
> > Actually, I read that differently. It only says that if there's more
> > than 1 part, all parts except for the last one need to be > 5M. Which
> > means that for uploads that are smaller than 5M there should be zero
> > or one parts.
> >
> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
> > <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:;>> wrote:
> >> You're right.
> >>
> >> S3 api doc:
> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
> >> "Err:EntityTooSmall
> >> Your proposed upload is smaller than the minimum allowed object size.
> >> Each part must be at least 5 MB in size, except the last part."
> >>
> >> Thanks.
> >>
> >> This error should be triggered from radosgw also.
> >>
> >> --
> >> Regards
> >> Dominik
> >>
> >> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org <javascript:;>>:
> >>> Looks like it. There should be a guard against it (mulitpart upload
> >>> minimum is 5M).
> >>>
> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
> >>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:;>> wrote:
> >>>> Yes, this is probably upload empty file.
> >>>> This is the problem?
> >>>>
> >>>> --
> >>>> Regards
> >>>> Dominik
> >>>>
> >>>>
> >>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org <javascript:;>>:
> >>>>> By any chance are you uploading empty objects through the multipart
> upload api?
> >>>>>
> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
> >>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:;>> wrote:
> >>>>>> Hi,
> >>>>>> Another file with the same problems:
> >>>>>>
> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new
> request
> >>>>>> req=0x25406d0 =====
> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req
> 1314:0.000052::::initializing
> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
> >>>>>> s->bucket=testbucket
> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req 1314:0.000112:s3:POST
> >>>>>> /testbucket/files/192.txt::getting op
> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req 1314:0.000118:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
> >>>>>>
> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
> >>>>>> POST
> >>>>>>
> >>>>>> application/xml
> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT
> >>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req 1314:0.003399:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:reading permissions
> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req 1314:0.005670:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
> permissions
> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions for
> >>>>>> uid=0 mask=2
> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req 1314:0.005695:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op params
> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req 1314:0.005698:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:executing
> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0
> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't
> been
> >>>>>> set
> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req 1314:0.122286:s3:POST
> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
> >>>>>> req=0x25406d0 http_status=200 ======
> >>>>>>
> >>>>>> Yes, I can read oryginal object.
> >>>>>>
> >>>>>> --
> >>>>>> Regards
> >>>>>> Dominik
> >>>>>>
> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org <javascript:;>>:
> >>>>>>> That's unknown bug. I have a guess as to how the original object
> was
> >>>>>>> created. Can you read the original object, but only copy fails?
> >>>>>>>
> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" <
> dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:;>>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
> >>>>>>>> This is known bug ?
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Regards
> >>>>>>>> Dominik
> >>>>>>>>
> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org<javascript:;>
> >:
> >>>>>>>> > Hi,
> >>>>>>>> > I have strange problem.
> >>>>>>>> > Obj copy (0 size) killing radosgw.
> >>>>>>>> >
> >>>>>>>> > Head for this file:
> >>>>>>>> > Content-Type: application/octet-stream
> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
> >>>>>>>> >
> >>>>>>>> > rgw log.
> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new
> request
> >>>>>>>> > req=0x2be6fa0 =====
> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
> >>>>>>>> > 237:0.000058::::initializing
> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
> >>>>>>>> > HTTP_X_AMZ_ACL=public-read
> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>>
> x-amz-acl:public-read
> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
> >>>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req
> 237:0.000141:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini::getting op
> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req
> 237:0.000146:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 get_canon_resource():
> >>>>>>>> > dest=/testbucket/new_testfile.ini
> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
> >>>>>>>> > PUT
> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
> >>>>>>>> > application/octet-stream
> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
> >>>>>>>> > x-amz-acl:public-read
> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>>>>>> > /testbucket/new_testfile.ini
> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req
> 237:0.004066:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req
> 237:0.006679:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size
> = 0
> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching
> permissions for
> >>>>>>>> > uid=0 mask=1
> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested perm
> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching
> permissions for
> >>>>>>>> > uid=0 mask=2
> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested perm
> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req
> 237:0.011030:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req
> 237:0.011035:s3:PUT
> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>>
> x-amz-acl:public-read
> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
> >>>>>>>> > (Segmentation fault) **
> >>>>>>>> >  in thread 7f5308ff1700
> >>>>>>>> >
> >>>>>>>> >
> >>>>>>>> > --
> >>>>>>>> > Regards
> >>>>>>>> > Dominik
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Pozdrawiam
> >>>>>>>> Dominik
> >>>>>>>> _______________________________________________
> >>>>>>>> ceph-users mailing list
> >>>>>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org <javascript:;>
> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Pozdrawiam
> >>>>>> Dominik
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Pozdrawiam
> >>>> Dominik
> >>
> >>
> >>
> >> --
> >> Pozdrawiam
> >> Dominik
>


-- 
Pozdrawiam
Dominik

[-- Attachment #1.2: Type: text/html, Size: 16374 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]                                     ` <CAMNMNTwk=q0aStrqJACqnB64AhGSH13LwNGzb3hC1DQMeZtwOg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-03  2:46                                       ` Yehuda Sadeh
  2013-12-03  4:39                                         ` Dominik Mostowiec
  0 siblings, 1 reply; 19+ messages in thread
From: Yehuda Sadeh @ 2013-12-03  2:46 UTC (permalink / raw)
  To: Dominik Mostowiec; +Cc: ceph-devel, ceph-users

I see. Do you have backtrace for the crash?

On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec
<dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> 0.56.7
>
> W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał:
>
>> I'm having trouble reproducing the issue. What version are you using?
>>
>> Thanks,
>> Yehuda
>>
>> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org> wrote:
>> > Actually, I read that differently. It only says that if there's more
>> > than 1 part, all parts except for the last one need to be > 5M. Which
>> > means that for uploads that are smaller than 5M there should be zero
>> > or one parts.
>> >
>> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
>> > <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >> You're right.
>> >>
>> >> S3 api doc:
>> >> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
>> >> "Err:EntityTooSmall
>> >> Your proposed upload is smaller than the minimum allowed object size.
>> >> Each part must be at least 5 MB in size, except the last part."
>> >>
>> >> Thanks.
>> >>
>> >> This error should be triggered from radosgw also.
>> >>
>> >> --
>> >> Regards
>> >> Dominik
>> >>
>> >> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> >>> Looks like it. There should be a guard against it (mulitpart upload
>> >>> minimum is 5M).
>> >>>
>> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
>> >>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >>>> Yes, this is probably upload empty file.
>> >>>> This is the problem?
>> >>>>
>> >>>> --
>> >>>> Regards
>> >>>> Dominik
>> >>>>
>> >>>>
>> >>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> >>>>> By any chance are you uploading empty objects through the multipart
>> >>>>> upload api?
>> >>>>>
>> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>> >>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >>>>>> Hi,
>> >>>>>> Another file with the same problems:
>> >>>>>>
>> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new
>> >>>>>> request
>> >>>>>> req=0x25406d0 =====
>> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req
>> >>>>>> 1314:0.000052::::initializing
>> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
>> >>>>>> s->bucket=testbucket
>> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req
>> >>>>>> 1314:0.000112:s3:POST
>> >>>>>> /testbucket/files/192.txt::getting op
>> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req
>> >>>>>> 1314:0.000118:s3:POST
>> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
>> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
>> >>>>>>
>> >>>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>> >>>>>> POST
>> >>>>>>
>> >>>>>> application/xml
>> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT
>> >>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req
>> >>>>>> 1314:0.003399:s3:POST
>> >>>>>> /testbucket/files/192.txt:complete_multipart:reading permissions
>> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req
>> >>>>>> 1314:0.005670:s3:POST
>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
>> >>>>>> permissions
>> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions
>> >>>>>> for
>> >>>>>> uid=0 mask=2
>> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
>> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
>> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req
>> >>>>>> 1314:0.005695:s3:POST
>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op params
>> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req
>> >>>>>> 1314:0.005698:s3:POST
>> >>>>>> /testbucket/files/192.txt:complete_multipart:executing
>> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0
>> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't
>> >>>>>> been
>> >>>>>> set
>> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req
>> >>>>>> 1314:0.122286:s3:POST
>> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
>> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>> >>>>>> req=0x25406d0 http_status=200 ======
>> >>>>>>
>> >>>>>> Yes, I can read oryginal object.
>> >>>>>>
>> >>>>>> --
>> >>>>>> Regards
>> >>>>>> Dominik
>> >>>>>>
>> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> >>>>>>> That's unknown bug. I have a guess as to how the original object
>> >>>>>>> was
>> >>>>>>> created. Can you read the original object, but only copy fails?
>> >>>>>>>
>> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec"
>> >>>>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi,
>> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>> >>>>>>>> This is known bug ?
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Regards
>> >>>>>>>> Dominik
>> >>>>>>>>
>> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>> >>>>>>>> > Hi,
>> >>>>>>>> > I have strange problem.
>> >>>>>>>> > Obj copy (0 size) killing radosgw.
>> >>>>>>>> >
>> >>>>>>>> > Head for this file:
>> >>>>>>>> > Content-Type: application/octet-stream
>> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
>> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>> >>>>>>>> >
>> >>>>>>>> > rgw log.
>> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new
>> >>>>>>>> > request
>> >>>>>>>> > req=0x2be6fa0 =====
>> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>> >>>>>>>> > 237:0.000058::::initializing
>> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>> >>>>>>>> > HTTP_X_AMZ_ACL=public-read
>> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>>
>> >>>>>>>> > x-amz-acl:public-read
>> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>> >>>>>>>> > x-amz-metadata-directive:COPY
>> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>> >>>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req
>> >>>>>>>> > 237:0.000141:s3:PUT
>> >>>>>>>> > /testbucket/new_testfile.ini::getting op
>> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req
>> >>>>>>>> > 237:0.000146:s3:PUT
>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10
>> >>>>>>>> > get_canon_resource():
>> >>>>>>>> > dest=/testbucket/new_testfile.ini
>> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>> >>>>>>>> > PUT
>> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>> >>>>>>>> > application/octet-stream
>> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>> >>>>>>>> > x-amz-acl:public-read
>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>> >>>>>>>> > x-amz-metadata-directive:COPY
>> >>>>>>>> > /testbucket/new_testfile.ini
>> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req
>> >>>>>>>> > 237:0.004066:s3:PUT
>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req
>> >>>>>>>> > 237:0.006679:s3:PUT
>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size
>> >>>>>>>> > = 0
>> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching
>> >>>>>>>> > permissions for
>> >>>>>>>> > uid=0 mask=1
>> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested
>> >>>>>>>> > perm
>> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching
>> >>>>>>>> > permissions for
>> >>>>>>>> > uid=0 mask=2
>> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested
>> >>>>>>>> > perm
>> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req
>> >>>>>>>> > 237:0.011030:s3:PUT
>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req
>> >>>>>>>> > 237:0.011035:s3:PUT
>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>>
>> >>>>>>>> > x-amz-acl:public-read
>> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>> >>>>>>>> > x-amz-metadata-directive:COPY
>> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>> >>>>>>>> > (Segmentation fault) **
>> >>>>>>>> >  in thread 7f5308ff1700
>> >>>>>>>> >
>> >>>>>>>> >
>> >>>>>>>> > --
>> >>>>>>>> > Regards
>> >>>>>>>> > Dominik
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Pozdrawiam
>> >>>>>>>> Dominik
>> >>>>>>>> _______________________________________________
>> >>>>>>>> ceph-users mailing list
>> >>>>>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Pozdrawiam
>> >>>>>> Dominik
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Pozdrawiam
>> >>>> Dominik
>> >>
>> >>
>> >>
>> >> --
>> >> Pozdrawiam
>> >> Dominik
>
>
>
> --
> Pozdrawiam
> Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
  2013-12-03  2:46                                       ` Yehuda Sadeh
@ 2013-12-03  4:39                                         ` Dominik Mostowiec
  2013-12-03  5:43                                           ` Yehuda Sadeh
  0 siblings, 1 reply; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-03  4:39 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-devel, ceph-users

for another object.
http://pastebin.com/VkVAYgwn


2013/12/3 Yehuda Sadeh <yehuda@inktank.com>:
> I see. Do you have backtrace for the crash?
>
> On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec
> <dominikmostowiec@gmail.com> wrote:
>> 0.56.7
>>
>> W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał:
>>
>>> I'm having trouble reproducing the issue. What version are you using?
>>>
>>> Thanks,
>>> Yehuda
>>>
>>> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
>>> > Actually, I read that differently. It only says that if there's more
>>> > than 1 part, all parts except for the last one need to be > 5M. Which
>>> > means that for uploads that are smaller than 5M there should be zero
>>> > or one parts.
>>> >
>>> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
>>> > <dominikmostowiec@gmail.com> wrote:
>>> >> You're right.
>>> >>
>>> >> S3 api doc:
>>> >> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
>>> >> "Err:EntityTooSmall
>>> >> Your proposed upload is smaller than the minimum allowed object size.
>>> >> Each part must be at least 5 MB in size, except the last part."
>>> >>
>>> >> Thanks.
>>> >>
>>> >> This error should be triggered from radosgw also.
>>> >>
>>> >> --
>>> >> Regards
>>> >> Dominik
>>> >>
>>> >> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>> >>> Looks like it. There should be a guard against it (mulitpart upload
>>> >>> minimum is 5M).
>>> >>>
>>> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
>>> >>> <dominikmostowiec@gmail.com> wrote:
>>> >>>> Yes, this is probably upload empty file.
>>> >>>> This is the problem?
>>> >>>>
>>> >>>> --
>>> >>>> Regards
>>> >>>> Dominik
>>> >>>>
>>> >>>>
>>> >>>> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>> >>>>> By any chance are you uploading empty objects through the multipart
>>> >>>>> upload api?
>>> >>>>>
>>> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>>> >>>>> <dominikmostowiec@gmail.com> wrote:
>>> >>>>>> Hi,
>>> >>>>>> Another file with the same problems:
>>> >>>>>>
>>> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new
>>> >>>>>> request
>>> >>>>>> req=0x25406d0 =====
>>> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req
>>> >>>>>> 1314:0.000052::::initializing
>>> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
>>> >>>>>> s->bucket=testbucket
>>> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req
>>> >>>>>> 1314:0.000112:s3:POST
>>> >>>>>> /testbucket/files/192.txt::getting op
>>> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req
>>> >>>>>> 1314:0.000118:s3:POST
>>> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
>>> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
>>> >>>>>>
>>> >>>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>>> >>>>>> POST
>>> >>>>>>
>>> >>>>>> application/xml
>>> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT
>>> >>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req
>>> >>>>>> 1314:0.003399:s3:POST
>>> >>>>>> /testbucket/files/192.txt:complete_multipart:reading permissions
>>> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req
>>> >>>>>> 1314:0.005670:s3:POST
>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
>>> >>>>>> permissions
>>> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions
>>> >>>>>> for
>>> >>>>>> uid=0 mask=2
>>> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
>>> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
>>> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>>> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req
>>> >>>>>> 1314:0.005695:s3:POST
>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op params
>>> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req
>>> >>>>>> 1314:0.005698:s3:POST
>>> >>>>>> /testbucket/files/192.txt:complete_multipart:executing
>>> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>>> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0
>>> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>>> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't
>>> >>>>>> been
>>> >>>>>> set
>>> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>>> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>>> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req
>>> >>>>>> 1314:0.122286:s3:POST
>>> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
>>> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>>> >>>>>> req=0x25406d0 http_status=200 ======
>>> >>>>>>
>>> >>>>>> Yes, I can read oryginal object.
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Regards
>>> >>>>>> Dominik
>>> >>>>>>
>>> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>> >>>>>>> That's unknown bug. I have a guess as to how the original object
>>> >>>>>>> was
>>> >>>>>>> created. Can you read the original object, but only copy fails?
>>> >>>>>>>
>>> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec"
>>> >>>>>>> <dominikmostowiec@gmail.com>
>>> >>>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>> Hi,
>>> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>> >>>>>>>> This is known bug ?
>>> >>>>>>>>
>>> >>>>>>>> --
>>> >>>>>>>> Regards
>>> >>>>>>>> Dominik
>>> >>>>>>>>
>>> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec@gmail.com>:
>>> >>>>>>>> > Hi,
>>> >>>>>>>> > I have strange problem.
>>> >>>>>>>> > Obj copy (0 size) killing radosgw.
>>> >>>>>>>> >
>>> >>>>>>>> > Head for this file:
>>> >>>>>>>> > Content-Type: application/octet-stream
>>> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
>>> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>>> >>>>>>>> >
>>> >>>>>>>> > rgw log.
>>> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new
>>> >>>>>>>> > request
>>> >>>>>>>> > req=0x2be6fa0 =====
>>> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>> >>>>>>>> > 237:0.000058::::initializing
>>> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>> >>>>>>>> > HTTP_X_AMZ_ACL=public-read
>>> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>>
>>> >>>>>>>> > x-amz-acl:public-read
>>> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>> >>>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>>> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req
>>> >>>>>>>> > 237:0.000141:s3:PUT
>>> >>>>>>>> > /testbucket/new_testfile.ini::getting op
>>> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req
>>> >>>>>>>> > 237:0.000146:s3:PUT
>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10
>>> >>>>>>>> > get_canon_resource():
>>> >>>>>>>> > dest=/testbucket/new_testfile.ini
>>> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>> >>>>>>>> > PUT
>>> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>> >>>>>>>> > application/octet-stream
>>> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>> >>>>>>>> > x-amz-acl:public-read
>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>> >>>>>>>> > /testbucket/new_testfile.ini
>>> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req
>>> >>>>>>>> > 237:0.004066:s3:PUT
>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req
>>> >>>>>>>> > 237:0.006679:s3:PUT
>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>>> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size
>>> >>>>>>>> > = 0
>>> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching
>>> >>>>>>>> > permissions for
>>> >>>>>>>> > uid=0 mask=1
>>> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>>> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested
>>> >>>>>>>> > perm
>>> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching
>>> >>>>>>>> > permissions for
>>> >>>>>>>> > uid=0 mask=2
>>> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>>> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested
>>> >>>>>>>> > perm
>>> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req
>>> >>>>>>>> > 237:0.011030:s3:PUT
>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req
>>> >>>>>>>> > 237:0.011035:s3:PUT
>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>>
>>> >>>>>>>> > x-amz-acl:public-read
>>> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>>> >>>>>>>> > (Segmentation fault) **
>>> >>>>>>>> >  in thread 7f5308ff1700
>>> >>>>>>>> >
>>> >>>>>>>> >
>>> >>>>>>>> > --
>>> >>>>>>>> > Regards
>>> >>>>>>>> > Dominik
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> --
>>> >>>>>>>> Pozdrawiam
>>> >>>>>>>> Dominik
>>> >>>>>>>> _______________________________________________
>>> >>>>>>>> ceph-users mailing list
>>> >>>>>>>> ceph-users@lists.ceph.com
>>> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Pozdrawiam
>>> >>>>>> Dominik
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> Pozdrawiam
>>> >>>> Dominik
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Pozdrawiam
>>> >> Dominik
>>
>>
>>
>> --
>> Pozdrawiam
>> Dominik



-- 
Pozdrawiam
Dominik
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
  2013-12-03  4:39                                         ` Dominik Mostowiec
@ 2013-12-03  5:43                                           ` Yehuda Sadeh
       [not found]                                             ` <CABBk=J8E7eiViy3KMcTmrZgb5rUnXQigYKGkD+kRhbAH2gP2Bw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Yehuda Sadeh @ 2013-12-03  5:43 UTC (permalink / raw)
  To: Dominik Mostowiec; +Cc: ceph-devel, ceph-users

I created earlier an issue (6919) and updated it with the relevant
issue. This has been fixed in dumpling, although I don't remember
hitting the scenario that you did. Was probably hitting it as part of
the development work that was done then.
In any case I created a branch with the relevant fixes in it (wip-6919).

Thanks,
Yehuda

On Mon, Dec 2, 2013 at 8:39 PM, Dominik Mostowiec
<dominikmostowiec@gmail.com> wrote:
> for another object.
> http://pastebin.com/VkVAYgwn
>
>
> 2013/12/3 Yehuda Sadeh <yehuda@inktank.com>:
>> I see. Do you have backtrace for the crash?
>>
>> On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec
>> <dominikmostowiec@gmail.com> wrote:
>>> 0.56.7
>>>
>>> W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał:
>>>
>>>> I'm having trouble reproducing the issue. What version are you using?
>>>>
>>>> Thanks,
>>>> Yehuda
>>>>
>>>> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda@inktank.com> wrote:
>>>> > Actually, I read that differently. It only says that if there's more
>>>> > than 1 part, all parts except for the last one need to be > 5M. Which
>>>> > means that for uploads that are smaller than 5M there should be zero
>>>> > or one parts.
>>>> >
>>>> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
>>>> > <dominikmostowiec@gmail.com> wrote:
>>>> >> You're right.
>>>> >>
>>>> >> S3 api doc:
>>>> >> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
>>>> >> "Err:EntityTooSmall
>>>> >> Your proposed upload is smaller than the minimum allowed object size.
>>>> >> Each part must be at least 5 MB in size, except the last part."
>>>> >>
>>>> >> Thanks.
>>>> >>
>>>> >> This error should be triggered from radosgw also.
>>>> >>
>>>> >> --
>>>> >> Regards
>>>> >> Dominik
>>>> >>
>>>> >> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>>> >>> Looks like it. There should be a guard against it (mulitpart upload
>>>> >>> minimum is 5M).
>>>> >>>
>>>> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
>>>> >>> <dominikmostowiec@gmail.com> wrote:
>>>> >>>> Yes, this is probably upload empty file.
>>>> >>>> This is the problem?
>>>> >>>>
>>>> >>>> --
>>>> >>>> Regards
>>>> >>>> Dominik
>>>> >>>>
>>>> >>>>
>>>> >>>> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>>> >>>>> By any chance are you uploading empty objects through the multipart
>>>> >>>>> upload api?
>>>> >>>>>
>>>> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>>>> >>>>> <dominikmostowiec@gmail.com> wrote:
>>>> >>>>>> Hi,
>>>> >>>>>> Another file with the same problems:
>>>> >>>>>>
>>>> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new
>>>> >>>>>> request
>>>> >>>>>> req=0x25406d0 =====
>>>> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req
>>>> >>>>>> 1314:0.000052::::initializing
>>>> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 s->object=files/192.txt
>>>> >>>>>> s->bucket=testbucket
>>>> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req
>>>> >>>>>> 1314:0.000112:s3:POST
>>>> >>>>>> /testbucket/files/192.txt::getting op
>>>> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req
>>>> >>>>>> 1314:0.000118:s3:POST
>>>> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
>>>> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 get_canon_resource():
>>>> >>>>>>
>>>> >>>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>>> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>>>> >>>>>> POST
>>>> >>>>>>
>>>> >>>>>> application/xml
>>>> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT
>>>> >>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>>> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req
>>>> >>>>>> 1314:0.003399:s3:POST
>>>> >>>>>> /testbucket/files/192.txt:complete_multipart:reading permissions
>>>> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req
>>>> >>>>>> 1314:0.005670:s3:POST
>>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
>>>> >>>>>> permissions
>>>> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching permissions
>>>> >>>>>> for
>>>> >>>>>> uid=0 mask=2
>>>> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
>>>> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested perm
>>>> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>>>> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req
>>>> >>>>>> 1314:0.005695:s3:POST
>>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op params
>>>> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req
>>>> >>>>>> 1314:0.005698:s3:POST
>>>> >>>>>> /testbucket/files/192.txt:complete_multipart:executing
>>>> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>>>> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0
>>>> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>>>> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj haven't
>>>> >>>>>> been
>>>> >>>>>> set
>>>> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>>>> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>>>> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req
>>>> >>>>>> 1314:0.122286:s3:POST
>>>> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
>>>> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>>>> >>>>>> req=0x25406d0 http_status=200 ======
>>>> >>>>>>
>>>> >>>>>> Yes, I can read oryginal object.
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> Regards
>>>> >>>>>> Dominik
>>>> >>>>>>
>>>> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda@inktank.com>:
>>>> >>>>>>> That's unknown bug. I have a guess as to how the original object
>>>> >>>>>>> was
>>>> >>>>>>> created. Can you read the original object, but only copy fails?
>>>> >>>>>>>
>>>> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec"
>>>> >>>>>>> <dominikmostowiec@gmail.com>
>>>> >>>>>>> wrote:
>>>> >>>>>>>>
>>>> >>>>>>>> Hi,
>>>> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>>> >>>>>>>> This is known bug ?
>>>> >>>>>>>>
>>>> >>>>>>>> --
>>>> >>>>>>>> Regards
>>>> >>>>>>>> Dominik
>>>> >>>>>>>>
>>>> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec@gmail.com>:
>>>> >>>>>>>> > Hi,
>>>> >>>>>>>> > I have strange problem.
>>>> >>>>>>>> > Obj copy (0 size) killing radosgw.
>>>> >>>>>>>> >
>>>> >>>>>>>> > Head for this file:
>>>> >>>>>>>> > Content-Type: application/octet-stream
>>>> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
>>>> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>>> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>>>> >>>>>>>> >
>>>> >>>>>>>> > rgw log.
>>>> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting new
>>>> >>>>>>>> > request
>>>> >>>>>>>> > req=0x2be6fa0 =====
>>>> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>>> >>>>>>>> > 237:0.000058::::initializing
>>>> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>>> >>>>>>>> > HTTP_X_AMZ_ACL=public-read
>>>> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>>> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>>> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>>> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>>> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>>
>>>> >>>>>>>> > x-amz-acl:public-read
>>>> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>>> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>>> >>>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
>>>> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req
>>>> >>>>>>>> > 237:0.000141:s3:PUT
>>>> >>>>>>>> > /testbucket/new_testfile.ini::getting op
>>>> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req
>>>> >>>>>>>> > 237:0.000146:s3:PUT
>>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>>> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10
>>>> >>>>>>>> > get_canon_resource():
>>>> >>>>>>>> > dest=/testbucket/new_testfile.ini
>>>> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>>> >>>>>>>> > PUT
>>>> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>>> >>>>>>>> > application/octet-stream
>>>> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>>> >>>>>>>> > x-amz-acl:public-read
>>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>>> >>>>>>>> > /testbucket/new_testfile.ini
>>>> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req
>>>> >>>>>>>> > 237:0.004066:s3:PUT
>>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>>> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req
>>>> >>>>>>>> > 237:0.006679:s3:PUT
>>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op permissions
>>>> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: total_size
>>>> >>>>>>>> > = 0
>>>> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching
>>>> >>>>>>>> > permissions for
>>>> >>>>>>>> > uid=0 mask=1
>>>> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found permission: 15
>>>> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested
>>>> >>>>>>>> > perm
>>>> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>>> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching
>>>> >>>>>>>> > permissions for
>>>> >>>>>>>> > uid=0 mask=2
>>>> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found permission: 15
>>>> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested
>>>> >>>>>>>> > perm
>>>> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>>> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req
>>>> >>>>>>>> > 237:0.011030:s3:PUT
>>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>>> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req
>>>> >>>>>>>> > 237:0.011035:s3:PUT
>>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>>> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>>
>>>> >>>>>>>> > x-amz-acl:public-read
>>>> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>>> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>>> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>>> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught signal
>>>> >>>>>>>> > (Segmentation fault) **
>>>> >>>>>>>> >  in thread 7f5308ff1700
>>>> >>>>>>>> >
>>>> >>>>>>>> >
>>>> >>>>>>>> > --
>>>> >>>>>>>> > Regards
>>>> >>>>>>>> > Dominik
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> --
>>>> >>>>>>>> Pozdrawiam
>>>> >>>>>>>> Dominik
>>>> >>>>>>>> _______________________________________________
>>>> >>>>>>>> ceph-users mailing list
>>>> >>>>>>>> ceph-users@lists.ceph.com
>>>> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> Pozdrawiam
>>>> >>>>>> Dominik
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> Pozdrawiam
>>>> >>>> Dominik
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Pozdrawiam
>>>> >> Dominik
>>>
>>>
>>>
>>> --
>>> Pozdrawiam
>>> Dominik
>
>
>
> --
> Pozdrawiam
> Dominik
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]                                             ` <CABBk=J8E7eiViy3KMcTmrZgb5rUnXQigYKGkD+kRhbAH2gP2Bw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-03  6:24                                               ` Dominik Mostowiec
       [not found]                                                 ` <CAMNMNTyaUy_Tr6V9TDn2QHMf1oR_MXyPzwy5f8cg_=js9ZxCUg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-03  6:24 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-users-idqoXFIVOFJgJs9I8MT0rw, ceph-devel


[-- Attachment #1.1: Type: text/plain, Size: 12943 bytes --]

Thanks
Workaround, don't use multipart when obj size == 0 ?
On Dec 3, 2013 6:43 AM, "Yehuda Sadeh" <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org> wrote:

> I created earlier an issue (6919) and updated it with the relevant
> issue. This has been fixed in dumpling, although I don't remember
> hitting the scenario that you did. Was probably hitting it as part of
> the development work that was done then.
> In any case I created a branch with the relevant fixes in it (wip-6919).
>
> Thanks,
> Yehuda
>
> On Mon, Dec 2, 2013 at 8:39 PM, Dominik Mostowiec
> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > for another object.
> > http://pastebin.com/VkVAYgwn
> >
> >
> > 2013/12/3 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> >> I see. Do you have backtrace for the crash?
> >>
> >> On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec
> >> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >>> 0.56.7
> >>>
> >>> W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał:
> >>>
> >>>> I'm having trouble reproducing the issue. What version are you using?
> >>>>
> >>>> Thanks,
> >>>> Yehuda
> >>>>
> >>>> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>
> wrote:
> >>>> > Actually, I read that differently. It only says that if there's more
> >>>> > than 1 part, all parts except for the last one need to be > 5M.
> Which
> >>>> > means that for uploads that are smaller than 5M there should be zero
> >>>> > or one parts.
> >>>> >
> >>>> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
> >>>> > <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >>>> >> You're right.
> >>>> >>
> >>>> >> S3 api doc:
> >>>> >>
> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
> >>>> >> "Err:EntityTooSmall
> >>>> >> Your proposed upload is smaller than the minimum allowed object
> size.
> >>>> >> Each part must be at least 5 MB in size, except the last part."
> >>>> >>
> >>>> >> Thanks.
> >>>> >>
> >>>> >> This error should be triggered from radosgw also.
> >>>> >>
> >>>> >> --
> >>>> >> Regards
> >>>> >> Dominik
> >>>> >>
> >>>> >> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> >>>> >>> Looks like it. There should be a guard against it (mulitpart
> upload
> >>>> >>> minimum is 5M).
> >>>> >>>
> >>>> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
> >>>> >>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >>>> >>>> Yes, this is probably upload empty file.
> >>>> >>>> This is the problem?
> >>>> >>>>
> >>>> >>>> --
> >>>> >>>> Regards
> >>>> >>>> Dominik
> >>>> >>>>
> >>>> >>>>
> >>>> >>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> >>>> >>>>> By any chance are you uploading empty objects through the
> multipart
> >>>> >>>>> upload api?
> >>>> >>>>>
> >>>> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
> >>>> >>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >>>> >>>>>> Hi,
> >>>> >>>>>> Another file with the same problems:
> >>>> >>>>>>
> >>>> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new
> >>>> >>>>>> request
> >>>> >>>>>> req=0x25406d0 =====
> >>>> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req
> >>>> >>>>>> 1314:0.000052::::initializing
> >>>> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10
> s->object=files/192.txt
> >>>> >>>>>> s->bucket=testbucket
> >>>> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req
> >>>> >>>>>> 1314:0.000112:s3:POST
> >>>> >>>>>> /testbucket/files/192.txt::getting op
> >>>> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req
> >>>> >>>>>> 1314:0.000118:s3:POST
> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
> >>>> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10
> get_canon_resource():
> >>>> >>>>>>
> >>>> >>>>>>
> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
> >>>> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
> >>>> >>>>>> POST
> >>>> >>>>>>
> >>>> >>>>>> application/xml
> >>>> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT
> >>>> >>>>>>
> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
> >>>> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req
> >>>> >>>>>> 1314:0.003399:s3:POST
> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:reading
> permissions
> >>>> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req
> >>>> >>>>>> 1314:0.005670:s3:POST
> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
> >>>> >>>>>> permissions
> >>>> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching
> permissions
> >>>> >>>>>> for
> >>>> >>>>>> uid=0 mask=2
> >>>> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission: 15
> >>>> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested
> perm
> >>>> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
> >>>> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req
> >>>> >>>>>> 1314:0.005695:s3:POST
> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
> params
> >>>> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req
> >>>> >>>>>> 1314:0.005698:s3:POST
> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:executing
> >>>> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
> >>>> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0
> >>>> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
> >>>> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj
> haven't
> >>>> >>>>>> been
> >>>> >>>>>> set
> >>>> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
> >>>> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
> >>>> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req
> >>>> >>>>>> 1314:0.122286:s3:POST
> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
> >>>> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
> >>>> >>>>>> req=0x25406d0 http_status=200 ======
> >>>> >>>>>>
> >>>> >>>>>> Yes, I can read oryginal object.
> >>>> >>>>>>
> >>>> >>>>>> --
> >>>> >>>>>> Regards
> >>>> >>>>>> Dominik
> >>>> >>>>>>
> >>>> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> >>>> >>>>>>> That's unknown bug. I have a guess as to how the original
> object
> >>>> >>>>>>> was
> >>>> >>>>>>> created. Can you read the original object, but only copy
> fails?
> >>>> >>>>>>>
> >>>> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec"
> >>>> >>>>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> >>>> >>>>>>> wrote:
> >>>> >>>>>>>>
> >>>> >>>>>>>> Hi,
> >>>> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
> >>>> >>>>>>>> This is known bug ?
> >>>> >>>>>>>>
> >>>> >>>>>>>> --
> >>>> >>>>>>>> Regards
> >>>> >>>>>>>> Dominik
> >>>> >>>>>>>>
> >>>> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> >>>> >>>>>>>> > Hi,
> >>>> >>>>>>>> > I have strange problem.
> >>>> >>>>>>>> > Obj copy (0 size) killing radosgw.
> >>>> >>>>>>>> >
> >>>> >>>>>>>> > Head for this file:
> >>>> >>>>>>>> > Content-Type: application/octet-stream
> >>>> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
> >>>> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
> >>>> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
> >>>> >>>>>>>> >
> >>>> >>>>>>>> > rgw log.
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting
> new
> >>>> >>>>>>>> > request
> >>>> >>>>>>>> > req=0x2be6fa0 =====
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
> >>>> >>>>>>>> > 237:0.000058::::initializing
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
> >>>> >>>>>>>> > HTTP_X_AMZ_ACL=public-read
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
> >>>> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
> >>>> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>>
> >>>> >>>>>>>> > x-amz-acl:public-read
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
> >>>> >>>>>>>> > s->object=/testbucket/new_testfile.ini s->bucket=testbucket
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req
> >>>> >>>>>>>> > 237:0.000141:s3:PUT
> >>>> >>>>>>>> > /testbucket/new_testfile.ini::getting op
> >>>> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req
> >>>> >>>>>>>> > 237:0.000146:s3:PUT
> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
> >>>> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10
> >>>> >>>>>>>> > get_canon_resource():
> >>>> >>>>>>>> > dest=/testbucket/new_testfile.ini
> >>>> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
> >>>> >>>>>>>> > PUT
> >>>> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
> >>>> >>>>>>>> > application/octet-stream
> >>>> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
> >>>> >>>>>>>> > x-amz-acl:public-read
> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>> >>>>>>>> > /testbucket/new_testfile.ini
> >>>> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req
> >>>> >>>>>>>> > 237:0.004066:s3:PUT
> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
> >>>> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req
> >>>> >>>>>>>> > 237:0.006679:s3:PUT
> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op
> permissions
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest:
> total_size
> >>>> >>>>>>>> > = 0
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching
> >>>> >>>>>>>> > permissions for
> >>>> >>>>>>>> > uid=0 mask=1
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found
> permission: 15
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0 requested
> >>>> >>>>>>>> > perm
> >>>> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching
> >>>> >>>>>>>> > permissions for
> >>>> >>>>>>>> > uid=0 mask=2
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found
> permission: 15
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0 requested
> >>>> >>>>>>>> > perm
> >>>> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req
> >>>> >>>>>>>> > 237:0.011030:s3:PUT
> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req
> >>>> >>>>>>>> > 237:0.011035:s3:PUT
> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>>
> >>>> >>>>>>>> > x-amz-acl:public-read
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
> >>>> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
> >>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
> >>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
> >>>> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught
> signal
> >>>> >>>>>>>> > (Segmentation fault) **
> >>>> >>>>>>>> >  in thread 7f5308ff1700
> >>>> >>>>>>>> >
> >>>> >>>>>>>> >
> >>>> >>>>>>>> > --
> >>>> >>>>>>>> > Regards
> >>>> >>>>>>>> > Dominik
> >>>> >>>>>>>>
> >>>> >>>>>>>>
> >>>> >>>>>>>>
> >>>> >>>>>>>> --
> >>>> >>>>>>>> Pozdrawiam
> >>>> >>>>>>>> Dominik
> >>>> >>>>>>>> _______________________________________________
> >>>> >>>>>>>> ceph-users mailing list
> >>>> >>>>>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
> >>>> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> >>>>>>
> >>>> >>>>>> --
> >>>> >>>>>> Pozdrawiam
> >>>> >>>>>> Dominik
> >>>> >>>>
> >>>> >>>>
> >>>> >>>>
> >>>> >>>> --
> >>>> >>>> Pozdrawiam
> >>>> >>>> Dominik
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> --
> >>>> >> Pozdrawiam
> >>>> >> Dominik
> >>>
> >>>
> >>>
> >>> --
> >>> Pozdrawiam
> >>> Dominik
> >
> >
> >
> > --
> > Pozdrawiam
> > Dominik
>

[-- Attachment #1.2: Type: text/html, Size: 23028 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
       [not found]                                                 ` <CAMNMNTyaUy_Tr6V9TDn2QHMf1oR_MXyPzwy5f8cg_=js9ZxCUg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2013-12-03  6:56                                                   ` Yehuda Sadeh
  2013-12-03  8:34                                                     ` Dominik Mostowiec
  0 siblings, 1 reply; 19+ messages in thread
From: Yehuda Sadeh @ 2013-12-03  6:56 UTC (permalink / raw)
  To: Dominik Mostowiec; +Cc: ceph-users-idqoXFIVOFJgJs9I8MT0rw, ceph-devel

For bobtail at this point yes. You can try the unofficial version with
that fix off the gitbuilder. Another option is to upgrade everything
to dumpling.

Yehuda

On Mon, Dec 2, 2013 at 10:24 PM, Dominik Mostowiec
<dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Thanks
> Workaround, don't use multipart when obj size == 0 ?
>
> On Dec 3, 2013 6:43 AM, "Yehuda Sadeh" <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org> wrote:
>>
>> I created earlier an issue (6919) and updated it with the relevant
>> issue. This has been fixed in dumpling, although I don't remember
>> hitting the scenario that you did. Was probably hitting it as part of
>> the development work that was done then.
>> In any case I created a branch with the relevant fixes in it (wip-6919).
>>
>> Thanks,
>> Yehuda
>>
>> On Mon, Dec 2, 2013 at 8:39 PM, Dominik Mostowiec
>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> > for another object.
>> > http://pastebin.com/VkVAYgwn
>> >
>> >
>> > 2013/12/3 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> >> I see. Do you have backtrace for the crash?
>> >>
>> >> On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec
>> >> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >>> 0.56.7
>> >>>
>> >>> W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał:
>> >>>
>> >>>> I'm having trouble reproducing the issue. What version are you using?
>> >>>>
>> >>>> Thanks,
>> >>>> Yehuda
>> >>>>
>> >>>> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>
>> >>>> wrote:
>> >>>> > Actually, I read that differently. It only says that if there's
>> >>>> > more
>> >>>> > than 1 part, all parts except for the last one need to be > 5M.
>> >>>> > Which
>> >>>> > means that for uploads that are smaller than 5M there should be
>> >>>> > zero
>> >>>> > or one parts.
>> >>>> >
>> >>>> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
>> >>>> > <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >>>> >> You're right.
>> >>>> >>
>> >>>> >> S3 api doc:
>> >>>> >>
>> >>>> >> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
>> >>>> >> "Err:EntityTooSmall
>> >>>> >> Your proposed upload is smaller than the minimum allowed object
>> >>>> >> size.
>> >>>> >> Each part must be at least 5 MB in size, except the last part."
>> >>>> >>
>> >>>> >> Thanks.
>> >>>> >>
>> >>>> >> This error should be triggered from radosgw also.
>> >>>> >>
>> >>>> >> --
>> >>>> >> Regards
>> >>>> >> Dominik
>> >>>> >>
>> >>>> >> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> >>>> >>> Looks like it. There should be a guard against it (mulitpart
>> >>>> >>> upload
>> >>>> >>> minimum is 5M).
>> >>>> >>>
>> >>>> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
>> >>>> >>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >>>> >>>> Yes, this is probably upload empty file.
>> >>>> >>>> This is the problem?
>> >>>> >>>>
>> >>>> >>>> --
>> >>>> >>>> Regards
>> >>>> >>>> Dominik
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> >>>> >>>>> By any chance are you uploading empty objects through the
>> >>>> >>>>> multipart
>> >>>> >>>>> upload api?
>> >>>> >>>>>
>> >>>> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>> >>>> >>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> >>>> >>>>>> Hi,
>> >>>> >>>>>> Another file with the same problems:
>> >>>> >>>>>>
>> >>>> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new
>> >>>> >>>>>> request
>> >>>> >>>>>> req=0x25406d0 =====
>> >>>> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req
>> >>>> >>>>>> 1314:0.000052::::initializing
>> >>>> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10
>> >>>> >>>>>> s->object=files/192.txt
>> >>>> >>>>>> s->bucket=testbucket
>> >>>> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req
>> >>>> >>>>>> 1314:0.000112:s3:POST
>> >>>> >>>>>> /testbucket/files/192.txt::getting op
>> >>>> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req
>> >>>> >>>>>> 1314:0.000118:s3:POST
>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
>> >>>> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10
>> >>>> >>>>>> get_canon_resource():
>> >>>> >>>>>>
>> >>>> >>>>>>
>> >>>> >>>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>> >>>> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>> >>>> >>>>>> POST
>> >>>> >>>>>>
>> >>>> >>>>>> application/xml
>> >>>> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT
>> >>>> >>>>>>
>> >>>> >>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>> >>>> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req
>> >>>> >>>>>> 1314:0.003399:s3:POST
>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:reading
>> >>>> >>>>>> permissions
>> >>>> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req
>> >>>> >>>>>> 1314:0.005670:s3:POST
>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
>> >>>> >>>>>> permissions
>> >>>> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching
>> >>>> >>>>>> permissions
>> >>>> >>>>>> for
>> >>>> >>>>>> uid=0 mask=2
>> >>>> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission:
>> >>>> >>>>>> 15
>> >>>> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested
>> >>>> >>>>>> perm
>> >>>> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>> >>>> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req
>> >>>> >>>>>> 1314:0.005695:s3:POST
>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
>> >>>> >>>>>> params
>> >>>> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req
>> >>>> >>>>>> 1314:0.005698:s3:POST
>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:executing
>> >>>> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>> >>>> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0
>> >>>> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>> >>>> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj
>> >>>> >>>>>> haven't
>> >>>> >>>>>> been
>> >>>> >>>>>> set
>> >>>> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>> >>>> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>> >>>> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req
>> >>>> >>>>>> 1314:0.122286:s3:POST
>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
>> >>>> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>> >>>> >>>>>> req=0x25406d0 http_status=200 ======
>> >>>> >>>>>>
>> >>>> >>>>>> Yes, I can read oryginal object.
>> >>>> >>>>>>
>> >>>> >>>>>> --
>> >>>> >>>>>> Regards
>> >>>> >>>>>> Dominik
>> >>>> >>>>>>
>> >>>> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>> >>>> >>>>>>> That's unknown bug. I have a guess as to how the original
>> >>>> >>>>>>> object
>> >>>> >>>>>>> was
>> >>>> >>>>>>> created. Can you read the original object, but only copy
>> >>>> >>>>>>> fails?
>> >>>> >>>>>>>
>> >>>> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec"
>> >>>> >>>>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>> >>>> >>>>>>> wrote:
>> >>>> >>>>>>>>
>> >>>> >>>>>>>> Hi,
>> >>>> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>> >>>> >>>>>>>> This is known bug ?
>> >>>> >>>>>>>>
>> >>>> >>>>>>>> --
>> >>>> >>>>>>>> Regards
>> >>>> >>>>>>>> Dominik
>> >>>> >>>>>>>>
>> >>>> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>> >>>> >>>>>>>> > Hi,
>> >>>> >>>>>>>> > I have strange problem.
>> >>>> >>>>>>>> > Obj copy (0 size) killing radosgw.
>> >>>> >>>>>>>> >
>> >>>> >>>>>>>> > Head for this file:
>> >>>> >>>>>>>> > Content-Type: application/octet-stream
>> >>>> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
>> >>>> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>> >>>> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>> >>>> >>>>>>>> >
>> >>>> >>>>>>>> > rgw log.
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting
>> >>>> >>>>>>>> > new
>> >>>> >>>>>>>> > request
>> >>>> >>>>>>>> > req=0x2be6fa0 =====
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>> >>>> >>>>>>>> > 237:0.000058::::initializing
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>> >>>> >>>>>>>> > HTTP_X_AMZ_ACL=public-read
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>> >>>> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>> >>>> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>>
>> >>>> >>>>>>>> > x-amz-acl:public-read
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>> >>>> >>>>>>>> > s->object=/testbucket/new_testfile.ini
>> >>>> >>>>>>>> > s->bucket=testbucket
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req
>> >>>> >>>>>>>> > 237:0.000141:s3:PUT
>> >>>> >>>>>>>> > /testbucket/new_testfile.ini::getting op
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req
>> >>>> >>>>>>>> > 237:0.000146:s3:PUT
>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10
>> >>>> >>>>>>>> > get_canon_resource():
>> >>>> >>>>>>>> > dest=/testbucket/new_testfile.ini
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>> >>>> >>>>>>>> > PUT
>> >>>> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>> >>>> >>>>>>>> > application/octet-stream
>> >>>> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>> >>>> >>>>>>>> > x-amz-acl:public-read
>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
>> >>>> >>>>>>>> > /testbucket/new_testfile.ini
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req
>> >>>> >>>>>>>> > 237:0.004066:s3:PUT
>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req
>> >>>> >>>>>>>> > 237:0.006679:s3:PUT
>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op
>> >>>> >>>>>>>> > permissions
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest:
>> >>>> >>>>>>>> > total_size
>> >>>> >>>>>>>> > = 0
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching
>> >>>> >>>>>>>> > permissions for
>> >>>> >>>>>>>> > uid=0 mask=1
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found
>> >>>> >>>>>>>> > permission: 15
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0
>> >>>> >>>>>>>> > requested
>> >>>> >>>>>>>> > perm
>> >>>> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching
>> >>>> >>>>>>>> > permissions for
>> >>>> >>>>>>>> > uid=0 mask=2
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found
>> >>>> >>>>>>>> > permission: 15
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0
>> >>>> >>>>>>>> > requested
>> >>>> >>>>>>>> > perm
>> >>>> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req
>> >>>> >>>>>>>> > 237:0.011030:s3:PUT
>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req
>> >>>> >>>>>>>> > 237:0.011035:s3:PUT
>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>>
>> >>>> >>>>>>>> > x-amz-acl:public-read
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>> >>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>> >>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>> >>>> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught
>> >>>> >>>>>>>> > signal
>> >>>> >>>>>>>> > (Segmentation fault) **
>> >>>> >>>>>>>> >  in thread 7f5308ff1700
>> >>>> >>>>>>>> >
>> >>>> >>>>>>>> >
>> >>>> >>>>>>>> > --
>> >>>> >>>>>>>> > Regards
>> >>>> >>>>>>>> > Dominik
>> >>>> >>>>>>>>
>> >>>> >>>>>>>>
>> >>>> >>>>>>>>
>> >>>> >>>>>>>> --
>> >>>> >>>>>>>> Pozdrawiam
>> >>>> >>>>>>>> Dominik
>> >>>> >>>>>>>> _______________________________________________
>> >>>> >>>>>>>> ceph-users mailing list
>> >>>> >>>>>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>> >>>> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> >>>> >>>>>>
>> >>>> >>>>>>
>> >>>> >>>>>>
>> >>>> >>>>>> --
>> >>>> >>>>>> Pozdrawiam
>> >>>> >>>>>> Dominik
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>>
>> >>>> >>>> --
>> >>>> >>>> Pozdrawiam
>> >>>> >>>> Dominik
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> --
>> >>>> >> Pozdrawiam
>> >>>> >> Dominik
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Pozdrawiam
>> >>> Dominik
>> >
>> >
>> >
>> > --
>> > Pozdrawiam
>> > Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
* Re: radosgw Segmentation fault on obj copy
  2013-12-03  6:56                                                   ` Yehuda Sadeh
@ 2013-12-03  8:34                                                     ` Dominik Mostowiec
  0 siblings, 0 replies; 19+ messages in thread
From: Dominik Mostowiec @ 2013-12-03  8:34 UTC (permalink / raw)
  To: Yehuda Sadeh; +Cc: ceph-users-idqoXFIVOFJgJs9I8MT0rw, ceph-devel

Thanks.

--
Regards
Dominik

2013/12/3 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
> For bobtail at this point yes. You can try the unofficial version with
> that fix off the gitbuilder. Another option is to upgrade everything
> to dumpling.
>
> Yehuda
>
> On Mon, Dec 2, 2013 at 10:24 PM, Dominik Mostowiec
> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> Thanks
>> Workaround, don't use multipart when obj size == 0 ?
>>
>> On Dec 3, 2013 6:43 AM, "Yehuda Sadeh" <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org> wrote:
>>>
>>> I created earlier an issue (6919) and updated it with the relevant
>>> issue. This has been fixed in dumpling, although I don't remember
>>> hitting the scenario that you did. Was probably hitting it as part of
>>> the development work that was done then.
>>> In any case I created a branch with the relevant fixes in it (wip-6919).
>>>
>>> Thanks,
>>> Yehuda
>>>
>>> On Mon, Dec 2, 2013 at 8:39 PM, Dominik Mostowiec
>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> > for another object.
>>> > http://pastebin.com/VkVAYgwn
>>> >
>>> >
>>> > 2013/12/3 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>> >> I see. Do you have backtrace for the crash?
>>> >>
>>> >> On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec
>>> >> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> >>> 0.56.7
>>> >>>
>>> >>> W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał:
>>> >>>
>>> >>>> I'm having trouble reproducing the issue. What version are you using?
>>> >>>>
>>> >>>> Thanks,
>>> >>>> Yehuda
>>> >>>>
>>> >>>> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>
>>> >>>> wrote:
>>> >>>> > Actually, I read that differently. It only says that if there's
>>> >>>> > more
>>> >>>> > than 1 part, all parts except for the last one need to be > 5M.
>>> >>>> > Which
>>> >>>> > means that for uploads that are smaller than 5M there should be
>>> >>>> > zero
>>> >>>> > or one parts.
>>> >>>> >
>>> >>>> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec
>>> >>>> > <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> >>>> >> You're right.
>>> >>>> >>
>>> >>>> >> S3 api doc:
>>> >>>> >>
>>> >>>> >> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
>>> >>>> >> "Err:EntityTooSmall
>>> >>>> >> Your proposed upload is smaller than the minimum allowed object
>>> >>>> >> size.
>>> >>>> >> Each part must be at least 5 MB in size, except the last part."
>>> >>>> >>
>>> >>>> >> Thanks.
>>> >>>> >>
>>> >>>> >> This error should be triggered from radosgw also.
>>> >>>> >>
>>> >>>> >> --
>>> >>>> >> Regards
>>> >>>> >> Dominik
>>> >>>> >>
>>> >>>> >> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>> >>>> >>> Looks like it. There should be a guard against it (mulitpart
>>> >>>> >>> upload
>>> >>>> >>> minimum is 5M).
>>> >>>> >>>
>>> >>>> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec
>>> >>>> >>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> >>>> >>>> Yes, this is probably upload empty file.
>>> >>>> >>>> This is the problem?
>>> >>>> >>>>
>>> >>>> >>>> --
>>> >>>> >>>> Regards
>>> >>>> >>>> Dominik
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>> >>>> >>>>> By any chance are you uploading empty objects through the
>>> >>>> >>>>> multipart
>>> >>>> >>>>> upload api?
>>> >>>> >>>>>
>>> >>>> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec
>>> >>>> >>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> >>>> >>>>>> Hi,
>>> >>>> >>>>>> Another file with the same problems:
>>> >>>> >>>>>>
>>> >>>> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700  1 ====== starting new
>>> >>>> >>>>>> request
>>> >>>> >>>>>> req=0x25406d0 =====
>>> >>>> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700  2 req
>>> >>>> >>>>>> 1314:0.000052::::initializing
>>> >>>> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10
>>> >>>> >>>>>> s->object=files/192.txt
>>> >>>> >>>>>> s->bucket=testbucket
>>> >>>> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700  2 req
>>> >>>> >>>>>> 1314:0.000112:s3:POST
>>> >>>> >>>>>> /testbucket/files/192.txt::getting op
>>> >>>> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700  2 req
>>> >>>> >>>>>> 1314:0.000118:s3:POST
>>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing
>>> >>>> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10
>>> >>>> >>>>>> get_canon_resource():
>>> >>>> >>>>>>
>>> >>>> >>>>>>
>>> >>>> >>>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>> >>>> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr:
>>> >>>> >>>>>> POST
>>> >>>> >>>>>>
>>> >>>> >>>>>> application/xml
>>> >>>> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT
>>> >>>> >>>>>>
>>> >>>> >>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC
>>> >>>> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700  2 req
>>> >>>> >>>>>> 1314:0.003399:s3:POST
>>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:reading
>>> >>>> >>>>>> permissions
>>> >>>> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700  2 req
>>> >>>> >>>>>> 1314:0.005670:s3:POST
>>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
>>> >>>> >>>>>> permissions
>>> >>>> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700  5 Searching
>>> >>>> >>>>>> permissions
>>> >>>> >>>>>> for
>>> >>>> >>>>>> uid=0 mask=2
>>> >>>> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700  5 Found permission:
>>> >>>> >>>>>> 15
>>> >>>> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10  uid=0 requested
>>> >>>> >>>>>> perm
>>> >>>> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2
>>> >>>> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700  2 req
>>> >>>> >>>>>> 1314:0.005695:s3:POST
>>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op
>>> >>>> >>>>>> params
>>> >>>> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700  2 req
>>> >>>> >>>>>> 1314:0.005698:s3:POST
>>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:executing
>>> >>>> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag:
>>> >>>> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0
>>> >>>> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object
>>> >>>> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj
>>> >>>> >>>>>> haven't
>>> >>>> >>>>>> been
>>> >>>> >>>>>> set
>>> >>>> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700  0 setting object
>>> >>>> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP
>>> >>>> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700  2 req
>>> >>>> >>>>>> 1314:0.122286:s3:POST
>>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200
>>> >>>> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700  1 ====== req done
>>> >>>> >>>>>> req=0x25406d0 http_status=200 ======
>>> >>>> >>>>>>
>>> >>>> >>>>>> Yes, I can read oryginal object.
>>> >>>> >>>>>>
>>> >>>> >>>>>> --
>>> >>>> >>>>>> Regards
>>> >>>> >>>>>> Dominik
>>> >>>> >>>>>>
>>> >>>> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda-4GqslpFJ+cxBDgjK7y7TUQ@public.gmane.org>:
>>> >>>> >>>>>>> That's unknown bug. I have a guess as to how the original
>>> >>>> >>>>>>> object
>>> >>>> >>>>>>> was
>>> >>>> >>>>>>> created. Can you read the original object, but only copy
>>> >>>> >>>>>>> fails?
>>> >>>> >>>>>>>
>>> >>>> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec"
>>> >>>> >>>>>>> <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>> >>>> >>>>>>> wrote:
>>> >>>> >>>>>>>>
>>> >>>> >>>>>>>> Hi,
>>> >>>> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0)
>>> >>>> >>>>>>>> This is known bug ?
>>> >>>> >>>>>>>>
>>> >>>> >>>>>>>> --
>>> >>>> >>>>>>>> Regards
>>> >>>> >>>>>>>> Dominik
>>> >>>> >>>>>>>>
>>> >>>> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
>>> >>>> >>>>>>>> > Hi,
>>> >>>> >>>>>>>> > I have strange problem.
>>> >>>> >>>>>>>> > Obj copy (0 size) killing radosgw.
>>> >>>> >>>>>>>> >
>>> >>>> >>>>>>>> > Head for this file:
>>> >>>> >>>>>>>> > Content-Type: application/octet-stream
>>> >>>> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu)
>>> >>>> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0"
>>> >>>> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z
>>> >>>> >>>>>>>> >
>>> >>>> >>>>>>>> > rgw log.
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700  1 ====== starting
>>> >>>> >>>>>>>> > new
>>> >>>> >>>>>>>> > request
>>> >>>> >>>>>>>> > req=0x2be6fa0 =====
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700  2 req
>>> >>>> >>>>>>>> > 237:0.000058::::initializing
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>>
>>> >>>> >>>>>>>> > HTTP_X_AMZ_ACL=public-read
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>>
>>> >>>> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>>
>>> >>>> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>>
>>> >>>> >>>>>>>> > x-amz-acl:public-read
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>>
>>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>>
>>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10
>>> >>>> >>>>>>>> > s->object=/testbucket/new_testfile.ini
>>> >>>> >>>>>>>> > s->bucket=testbucket
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700  2 req
>>> >>>> >>>>>>>> > 237:0.000141:s3:PUT
>>> >>>> >>>>>>>> > /testbucket/new_testfile.ini::getting op
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700  2 req
>>> >>>> >>>>>>>> > 237:0.000146:s3:PUT
>>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10
>>> >>>> >>>>>>>> > get_canon_resource():
>>> >>>> >>>>>>>> > dest=/testbucket/new_testfile.ini
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr:
>>> >>>> >>>>>>>> > PUT
>>> >>>> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg==
>>> >>>> >>>>>>>> > application/octet-stream
>>> >>>> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT
>>> >>>> >>>>>>>> > x-amz-acl:public-read
>>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>> >>>> >>>>>>>> > /testbucket/new_testfile.ini
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700  2 req
>>> >>>> >>>>>>>> > 237:0.004066:s3:PUT
>>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700  2 req
>>> >>>> >>>>>>>> > 237:0.006679:s3:PUT
>>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op
>>> >>>> >>>>>>>> > permissions
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest:
>>> >>>> >>>>>>>> > total_size
>>> >>>> >>>>>>>> > = 0
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700  5 Searching
>>> >>>> >>>>>>>> > permissions for
>>> >>>> >>>>>>>> > uid=0 mask=1
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700  5 Found
>>> >>>> >>>>>>>> > permission: 15
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10  uid=0
>>> >>>> >>>>>>>> > requested
>>> >>>> >>>>>>>> > perm
>>> >>>> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700  5 Searching
>>> >>>> >>>>>>>> > permissions for
>>> >>>> >>>>>>>> > uid=0 mask=2
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700  5 Found
>>> >>>> >>>>>>>> > permission: 15
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10  uid=0
>>> >>>> >>>>>>>> > requested
>>> >>>> >>>>>>>> > perm
>>> >>>> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700  2 req
>>> >>>> >>>>>>>> > 237:0.011030:s3:PUT
>>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700  2 req
>>> >>>> >>>>>>>> > 237:0.011035:s3:PUT
>>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>>
>>> >>>> >>>>>>>> > x-amz-acl:public-read
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>>
>>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>>
>>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700  5 Copy object
>>> >>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini =>
>>> >>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini
>>> >>>> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught
>>> >>>> >>>>>>>> > signal
>>> >>>> >>>>>>>> > (Segmentation fault) **
>>> >>>> >>>>>>>> >  in thread 7f5308ff1700
>>> >>>> >>>>>>>> >
>>> >>>> >>>>>>>> >
>>> >>>> >>>>>>>> > --
>>> >>>> >>>>>>>> > Regards
>>> >>>> >>>>>>>> > Dominik
>>> >>>> >>>>>>>>
>>> >>>> >>>>>>>>
>>> >>>> >>>>>>>>
>>> >>>> >>>>>>>> --
>>> >>>> >>>>>>>> Pozdrawiam
>>> >>>> >>>>>>>> Dominik
>>> >>>> >>>>>>>> _______________________________________________
>>> >>>> >>>>>>>> ceph-users mailing list
>>> >>>> >>>>>>>> ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
>>> >>>> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>> >>>> >>>>>>
>>> >>>> >>>>>>
>>> >>>> >>>>>>
>>> >>>> >>>>>> --
>>> >>>> >>>>>> Pozdrawiam
>>> >>>> >>>>>> Dominik
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>>
>>> >>>> >>>> --
>>> >>>> >>>> Pozdrawiam
>>> >>>> >>>> Dominik
>>> >>>> >>
>>> >>>> >>
>>> >>>> >>
>>> >>>> >> --
>>> >>>> >> Pozdrawiam
>>> >>>> >> Dominik
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Pozdrawiam
>>> >>> Dominik
>>> >
>>> >
>>> >
>>> > --
>>> > Pozdrawiam
>>> > Dominik



-- 
Pozdrawiam
Dominik

^ permalink raw reply	[flat|nested] 19+ messages in thread
end of thread, other threads:[~2013-12-03  8:34 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-12-02 12:19 radosgw Segmentation fault on obj copy Dominik Mostowiec
     [not found] ` <CAMNMNTzEVkom0cBe_jepaE95hzVGys+8y+-PA+2dSsTqNwg=tA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-02 12:53   ` Dominik Mostowiec
     [not found]     ` <CAMNMNTzx_CHmYRXXE-KhbEwNZqLAAcTBp-5SZks7UiQ+OS1XqA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-02 17:14       ` Yehuda Sadeh
     [not found]         ` <CABBk=J-HAPxi4eg0AW_vD0T+QoqRzBMchpovM9s+EhzzC2ZZmg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-02 18:07           ` Dominik Mostowiec
2013-12-02 20:08           ` Dominik Mostowiec
     [not found]             ` <CAMNMNTzMvb-udVW5LrOhJLCSWWP5KVMHHyLrHwsY=bv3ckXyGA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-02 20:27               ` Yehuda Sadeh
     [not found]                 ` <CABBk=J-xw8G5Uuj1HGonP-B_Xzqn=A0myE-C28sWMMpnH9tbmA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-02 20:32                   ` Dominik Mostowiec
     [not found]                     ` <CAMNMNTxYE=nqg04X7Wa0U2g3iW-HfkDBCEY_fv75D+nE7=xz4w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-02 20:45                       ` Yehuda Sadeh
2013-12-02 20:54                         ` [ceph-users] " Dominik Mostowiec
2013-12-02 22:16                           ` Yehuda Sadeh
     [not found]                             ` <CABBk=J8GeCckYaxFu+_8jwagKtt2vOkmMsQ=1vii+MR7-erYjQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-02 22:30                               ` Yehuda Sadeh
     [not found]                                 ` <CABBk=J9a3M6yAZj40+XZKLfmoHj7xsYj3GTfs34Be8twSb19UQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-03  2:13                                   ` Dominik Mostowiec
2013-12-03  2:19                                   ` Dominik Mostowiec
     [not found]                                     ` <CAMNMNTwk=q0aStrqJACqnB64AhGSH13LwNGzb3hC1DQMeZtwOg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-03  2:46                                       ` Yehuda Sadeh
2013-12-03  4:39                                         ` Dominik Mostowiec
2013-12-03  5:43                                           ` Yehuda Sadeh
     [not found]                                             ` <CABBk=J8E7eiViy3KMcTmrZgb5rUnXQigYKGkD+kRhbAH2gP2Bw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-03  6:24                                               ` Dominik Mostowiec
     [not found]                                                 ` <CAMNMNTyaUy_Tr6V9TDn2QHMf1oR_MXyPzwy5f8cg_=js9ZxCUg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-12-03  6:56                                                   ` Yehuda Sadeh
2013-12-03  8:34                                                     ` Dominik Mostowiec

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.