HTTP Engine of fio

HTTP Engine of fio

[Marcel Bielcik]

Hello,



I wanted to execute some S3 performance tests on my storage. If I am
execute fio with the HTTP engine, I always receive the following
output:


fio: engine http not loadable

fio: failed to load engine

Bad option <http_verbose=0>

Bad option <https=off>

Did you mean loops?

Bad option <http_mode=s3>

Did you mean zonemode?

Bad option <http_s3_key=someKey

Bad option <http_s3_keyid=someKey>

Bad option <http_host=http://…. >

Bad option <http_s3_region=eu-central-1>

fio: job global dropped

”



If I running the command “./configure”, I can see that the http engine
is not available.


Operating system              Linux

CPU                           x86_64

Big endian                    no

Compiler                      gcc

Cross compile                 no



Static build                  no

Wordsize                      64

zlib                          yes

Linux AIO support             no

Linux AIO support rw flags    no

POSIX AIO support             yes

POSIX AIO support needs -lrt  yes

POSIX AIO fsync               yes

POSIX pshared support         yes

pthread_condattr_setclock()   yes

pthread_sigmask()             yes

Solaris AIO support           no

__sync_fetch_and_add          yes

__sync_synchronize            yes

__sync_val_compare_and_swap   yes

libverbs                      no

rdmacm                        no

asprintf()                    yes

vasprintf()                   yes

Linux fallocate               yes

POSIX fadvise                 yes

POSIX fallocate               yes

sched_setaffinity(3 arg)      yes

sched_setaffinity(2 arg)      no

clock_gettime                 yes

CLOCK_MONOTONIC               yes

clockid_t                     yes

gettimeofday                  yes

fdatasync                     yes

pipe()                        yes

pipe2()                       yes

pread()                       yes

sync_file_range               yes

EXT4 move extent              yes

Linux splice(2)               yes

libnuma                       no

strsep                        yes

strcasestr                    yes

strlcat                       no

getopt_long_only()            yes

inet_aton                     yes

socklen_t                     yes

__thread                      yes

RUSAGE_THREAD                 yes

SCHED_IDLE                    yes

TCP_NODELAY                   yes

Net engine window_size        yes

TCP_MAXSEG                    yes

RLIMIT_MEMLOCK                yes

pwritev/preadv                yes

pwritev2/preadv2              yes

IPv6 helpers                  yes

http engine                   no

Rados engine                  no

Rados Block Device engine     no

setvbuf                       yes

Gluster API engine            no

s390_z196_facilities          no

HDFS engine                   no

MTD                           yes

libpmem                       no

libpmem1_5                    no

libpmemblk                    no

PMDK pmemblk engine           no

PMDK dev-dax engine           no

PMDK libpmem engine           no

DDN's Infinite Memory Engine  no

iscsi engine                  no

NBD engine                    no

lex/yacc for arithmetic       no

getmntent                     yes

getmntinfo                    no

Static Assert                 yes

bool                          yes

strndup                       yes

Valgrind headers              no

Zoned block device support    yes

Zoned block device capacity   no

libzbc engine                 no

march_armv8_a_crc_crypto      no

cuda                          no

libcufile                     no

Build march=native            yes

CUnit                         no

__kernel_rwf_t                yes

-Wimplicit-fallthrough=2      yes

MADV_HUGEPAGE                 yes

gettid                        yes

statx(2)/libc                 yes

statx(2)/syscall              yes

Windows PDB generation        no

timerfd_create                yes

Lib-based ioengines dynamic   no

TCMalloc support              no

“

I am working with the current master.

Environment:

Operating System: Ubuntu 20.04.2 LTS

Kernel: Linux 5.4.0-62-generic

Architecture: x86-64



I hope someone can help me.


Kind Regards

Marcel

Re: HTTP Engine of fio

[Sitsofe Wheeler]

Hi,

On Thu, 4 Feb 2021 at 16:05, Marcel Bielcik <bielcik1995@gmail.com> wrote:
>
> I wanted to execute some S3 performance tests on my storage. If I am
> execute fio with the HTTP engine, I always receive the following
> output:
>
>
> fio: engine http not loadable
>
> fio: failed to load engine

^^^ The fio you are running was compiled without the HTTP ioengine
enabled (so anything to do with it won't be available to you). You
would either have to go to whoever provided you that binary and ask
them to build you an fio with that ioengine enabled or you have to
compile an fio of your own with the necessary packages installed that
allow that ioengine to be built like the user over in
https://github.com/axboe/fio/issues/1178#issuecomment-774471547 did.

-- 
Sitsofe