* Resending: fio testing on S3 object storage
@ 2018-09-26 15:52 eliezer
0 siblings, 0 replies; 5+ messages in thread
From: eliezer @ 2018-09-26 15:52 UTC (permalink / raw)
To: fio
Greetings,
First of all, my apologies for re-sending this email. I've now sent it to
the correct maillist this time and in the proper mail format.
How does one run fio against an S3 object storage device?
Here�s the fio parameters I�m using but I�m getting the below error message:
# fio --ioengine=http --iodepth=1 --http_mode=s3 --http_s3_region=us-geo
--http_s3_key=123mYkEyInfo --http_s3_keyid=123mYkEyiDinfO --bs=16k
--size=100MB --http_host=s3-api.us-geo.objectstorage.inthecloud.net
--https=on --http_verbose=2
fio: engine https not loadable
fio: failed to load engine
Am I missing a kernel module required by fio �or am I missing a few more fio
parameters from the command line?�� If so, what do I need to do to enable
that module?
Here�s my server info:
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)
# fio --version
fio-3.1
# uname -a
Linux mytestserver.testfio.com 3.10.0-862.11.6.el7.x86_64 #1 SMP Fri Aug 10
16:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Many thanks in advance for your attention and your response(s).
Regards,
Eliezer
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Resending: fio testing on S3 object storage
2018-09-27 16:43 ` eliezer
@ 2018-09-27 17:01 ` Samuel Shapiro
0 siblings, 0 replies; 5+ messages in thread
From: Samuel Shapiro @ 2018-09-27 17:01 UTC (permalink / raw)
To: eliezer; +Cc: Sitsofe Wheeler, fio
Well, I'm just investigating this engine myself as well, but you can
start playing with that workload and modify it accordingly:
https://github.com/axboe/fio/blob/master/examples/http-s3.fio
On Thu, Sep 27, 2018 at 7:43 PM <eliezer@integritech.solutions> wrote:
>
> Thank you Sitsofe and Samuel for the response.
>
> Per your suggestion, I installed the following packages on my RH 7.5:
>
> openssl
> openssl-devel
> curl-devel
>
> Then I downloaded the latest fio source and compiled it.
>
> Now, I'm seeing the "http" engine listed:
>
> # fio --enghelp
> Available IO engines:
> cpuio
> mmap
> sync
> psync
> vsync
> pvsync
> pvsync2
> null
> net
> netsplice
> ftruncate
> filecreate
> posixaio
> falloc
> e4defrag
> splice
> >> http <<
> mtd
> sg
>
>
> Many thanks!
>
> Do you have some examples of fio benchmark params for object storage that you could share?
>
>
> Regards,
> Eliezer
>
>
> -----Original Message-----
> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On Behalf Of Sitsofe Wheeler
> Sent: Thursday, September 27, 2018 8:18 AM
> To: eliezer@integritech.solutions
> Cc: fio
> Subject: Re: Resending: fio testing on S3 object storage
>
> Hi,
>
> You
> On Wed, 26 Sep 2018 at 17:54, <eliezer@integritech.solutions> wrote:
> >
> > It looks like I don’t have the "http" ioengine on my server:
> >
> > # fio --enghelp
> > Available IO engines:
> > cpuio
> > mmap
> > sync
> > psync
> > vsync
> > pvsync
> > pvsync2
> > null
> > net
> > netsplice
> > ftruncate
> > libaio
> > rdma
> > posixaio
> > falloc
> > e4defrag
> > splice
> > rbd
> > mtd
> > pmemblk
> > dev-dax
> > sg
> > binject
>
> You need to have the appropriate dependencies available when you go to
> compile fio (you'll find that ./configure says that http is enabled).
> I believe for the http engine you need the development libraries for
> curl and openssl installed...
>
> --
> Sitsofe | http://sucs.org/~sits/
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: Resending: fio testing on S3 object storage
2018-09-27 13:18 ` Sitsofe Wheeler
@ 2018-09-27 16:43 ` eliezer
2018-09-27 17:01 ` Samuel Shapiro
0 siblings, 1 reply; 5+ messages in thread
From: eliezer @ 2018-09-27 16:43 UTC (permalink / raw)
To: 'Sitsofe Wheeler', 'Samuel Shapiro'; +Cc: 'fio'
Thank you Sitsofe and Samuel for the response.
Per your suggestion, I installed the following packages on my RH 7.5:
openssl
openssl-devel
curl-devel
Then I downloaded the latest fio source and compiled it.
Now, I'm seeing the "http" engine listed:
# fio --enghelp
Available IO engines:
cpuio
mmap
sync
psync
vsync
pvsync
pvsync2
null
net
netsplice
ftruncate
filecreate
posixaio
falloc
e4defrag
splice
>> http <<
mtd
sg
Many thanks!
Do you have some examples of fio benchmark params for object storage that you could share?
Regards,
Eliezer
-----Original Message-----
From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On Behalf Of Sitsofe Wheeler
Sent: Thursday, September 27, 2018 8:18 AM
To: eliezer@integritech.solutions
Cc: fio
Subject: Re: Resending: fio testing on S3 object storage
Hi,
You
On Wed, 26 Sep 2018 at 17:54, <eliezer@integritech.solutions> wrote:
>
> It looks like I don’t have the "http" ioengine on my server:
>
> # fio --enghelp
> Available IO engines:
> cpuio
> mmap
> sync
> psync
> vsync
> pvsync
> pvsync2
> null
> net
> netsplice
> ftruncate
> libaio
> rdma
> posixaio
> falloc
> e4defrag
> splice
> rbd
> mtd
> pmemblk
> dev-dax
> sg
> binject
You need to have the appropriate dependencies available when you go to
compile fio (you'll find that ./configure says that http is enabled).
I believe for the http engine you need the development libraries for
curl and openssl installed...
--
Sitsofe | http://sucs.org/~sits/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Resending: fio testing on S3 object storage
2018-09-26 16:53 eliezer
@ 2018-09-27 13:18 ` Sitsofe Wheeler
2018-09-27 16:43 ` eliezer
0 siblings, 1 reply; 5+ messages in thread
From: Sitsofe Wheeler @ 2018-09-27 13:18 UTC (permalink / raw)
To: eliezer; +Cc: fio
Hi,
You
On Wed, 26 Sep 2018 at 17:54, <eliezer@integritech.solutions> wrote:
>
> It looks like I don’t have the "http" ioengine on my server:
>
> # fio --enghelp
> Available IO engines:
> cpuio
> mmap
> sync
> psync
> vsync
> pvsync
> pvsync2
> null
> net
> netsplice
> ftruncate
> libaio
> rdma
> posixaio
> falloc
> e4defrag
> splice
> rbd
> mtd
> pmemblk
> dev-dax
> sg
> binject
You need to have the appropriate dependencies available when you go to
compile fio (you'll find that ./configure says that http is enabled).
I believe for the http engine you need the development libraries for
curl and openssl installed...
--
Sitsofe | http://sucs.org/~sits/
^ permalink raw reply [flat|nested] 5+ messages in thread
* RE: Resending: fio testing on S3 object storage
@ 2018-09-26 16:53 eliezer
2018-09-27 13:18 ` Sitsofe Wheeler
0 siblings, 1 reply; 5+ messages in thread
From: eliezer @ 2018-09-26 16:53 UTC (permalink / raw)
To: fio
It looks like I don�t have the "http" ioengine on my server:
# fio --enghelp
Available IO engines:
cpuio
mmap
sync
psync
vsync
pvsync
pvsync2
null
net
netsplice
ftruncate
libaio
rdma
posixaio
falloc
e4defrag
splice
rbd
mtd
pmemblk
dev-dax
sg
binject
How would I go about adding the "http" ioengine on my server?
Thank you in advance.
Regards,
Eliezer
-----Original Message-----
From: Eliezer Nebab (eliezer@integritech.solutions)
[mailto:eliezer@integritech.solutions]
Sent: Wednesday, September 26, 2018 10:52 AM
To: 'fio@vger.kernel.org'
Subject: Resending: fio testing on S3 object storage
Greetings,
First of all, my apologies for re-sending this email. I've now sent it to
the correct maillist this time and in the proper mail format.
How does one run fio against an S3 object storage device?
Here�s the fio parameters I�m using but I�m getting the below error message:
# fio --ioengine=http --iodepth=1 --http_mode=s3 --http_s3_region=us-geo
--http_s3_key=123mYkEyInfo --http_s3_keyid=123mYkEyiDinfO --bs=16k
--size=100MB --http_host=s3-api.us-geo.objectstorage.inthecloud.net
--https=on --http_verbose=2
fio: engine https not loadable
fio: failed to load engine
Am I missing a kernel module required by fio �or am I missing a few more fio
parameters from the command line?�� If so, what do I need to do to enable
that module?
Here�s my server info:
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.5 (Maipo)
# fio --version
fio-3.1
# uname -a
Linux mytestserver.testfio.com 3.10.0-862.11.6.el7.x86_64 #1 SMP Fri Aug 10
16:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Many thanks in advance for your attention and your response(s).
Regards,
Eliezer
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2018-09-27 23:20 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-26 15:52 Resending: fio testing on S3 object storage eliezer
2018-09-26 16:53 eliezer
2018-09-27 13:18 ` Sitsofe Wheeler
2018-09-27 16:43 ` eliezer
2018-09-27 17:01 ` Samuel Shapiro
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.