RGW S3 vs S3 signature

RGW S3 vs S3 signature

[Zhou, Yuan]

Hello,

Trying to some tests with Hadoop 2.9 + Ceph S3(Jewel, 10.2.10) and ran into a multi-part uploading issue. It looks like the new S3a driver(aws-java-sdk-bundle-1.11.199.jar, v4 signature) in Hadoop is not compatible with Ceph S3(Jewel). 

I find there's a doc on S3 API support status, but lack of the corresponding S3 version info there. Is there any info on the detail supported version, v2 only?  
http://docs.ceph.com/docs/jewel/radosgw/s3/

I also find some commit to enable the V4 support here, I think this should be ready in Luminous?  
https://github.com/ceph/ceph/pull/15965/commits/82a7aa9e3eabfde4a82046d78aad2dadbfc0d8e1


Any comments would be appreciated!

Thanks, -yuan 

Re: RGW S3 vs S3 signature

[Matt Benjamin]

Hi Yuan,

AWS v4 signatures are supported in Jewel.  Could you create a tracker
issue with RGW debug logs to help track down the issue you hit?

thanks,

Matt

On Thu, Jan 18, 2018 at 9:39 AM, Zhou, Yuan <yuan.zhou@intel.com> wrote:
> Hello,
>
> Trying to some tests with Hadoop 2.9 + Ceph S3(Jewel, 10.2.10) and ran into a multi-part uploading issue. It looks like the new S3a driver(aws-java-sdk-bundle-1.11.199.jar, v4 signature) in Hadoop is not compatible with Ceph S3(Jewel).
>
> I find there's a doc on S3 API support status, but lack of the corresponding S3 version info there. Is there any info on the detail supported version, v2 only?
> http://docs.ceph.com/docs/jewel/radosgw/s3/
>
> I also find some commit to enable the V4 support here, I think this should be ready in Luminous?
> https://github.com/ceph/ceph/pull/15965/commits/82a7aa9e3eabfde4a82046d78aad2dadbfc0d8e1
>
>
> Any comments would be appreciated!
>
> Thanks, -yuan
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 

Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-821-5101
fax.  734-769-8938
cel.  734-216-5309