From: Haomai Wang <haomaiwang@gmail.com>
To: Casey Bodley <cbodley@redhat.com>
Cc: Casey Bodley <cbodley@gmail.com>,
"Matt W. Benjamin" <matt@cohortfs.com>,
"James (Fei) Liu-SSI" <james.liu@ssi.samsung.com>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: About Fio backend with ObjectStore API
Date: Thu, 23 Jul 2015 09:07:04 +0800 [thread overview]
Message-ID: <CACJqLyY9t_t7omJfb9vknwqBQ2V7csfLPqU-jvUtvDj2fhuyPw@mail.gmail.com> (raw)
In-Reply-To: <327699266.948920.1437598600219.JavaMail.zimbra@redhat.com>
no special
[global]
#logging
#write_iops_log=write_iops_log
#write_bw_log=write_bw_log
#write_lat_log=write_lat_log
ioengine=./ceph-int/src/.libs/libfio_ceph_objectstore.so
invalidate=0 # mandatory
rw=write
#bs=4k
[filestore]
iodepth=1
# create a journaled filestore
objectstore=filestore
directory=./osd/
filestore_journal=./osd/journal
On Thu, Jul 23, 2015 at 4:56 AM, Casey Bodley <cbodley@redhat.com> wrote:
> Hi Haomai,
>
> Sorry for the late response, I was out of the office. I'm afraid I haven't run into that segfault. The io_ops should be set at the very beginning when it calls get_ioengine(). All I can suggest is that you verify that your job file is pointing to the correct fio_ceph_objectstore.so. If you've made any other interesting changes to the job file, could you share it here?
>
> Casey
>
> ----- Original Message -----
> From: "Haomai Wang" <haomaiwang@gmail.com>
> To: "Casey Bodley" <cbodley@gmail.com>
> Cc: "Matt W. Benjamin" <matt@cohortfs.com>, "James (Fei) Liu-SSI" <james.liu@ssi.samsung.com>, ceph-devel@vger.kernel.org
> Sent: Tuesday, July 21, 2015 7:50:32 AM
> Subject: Re: About Fio backend with ObjectStore API
>
> Hi Casey,
>
> I check your commits and know what you fixed. I cherry-picked your new
> commits but I still met the same problem.
>
> """
> It's strange that it alwasys hit segment fault when entering
> "_fio_setup_ceph_filestore_data", gdb tells "td->io_ops" is NULL but
> when I up the stack, the "td->io_ops" is not null. Maybe it's related
> to dlopen?
> """
>
> Do you have any hint about this?
>
> On Thu, Jul 16, 2015 at 5:23 AM, Casey Bodley <cbodley@gmail.com> wrote:
>> Hi Haomai,
>>
>> I was able to run this after a couple changes to the filestore.fio job
>> file. Two of the config options were using the wrong names. I pushed a
>> fix for the job file, as well as a patch that renames everything from
>> filestore to objectstore (thanks James), to
>> https://github.com/linuxbox2/linuxbox-ceph/commits/fio-objectstore.
>>
>> I found that the read support doesn't appear to work anymore, so give
>> "rw=write" a try. And because it does a mkfs(), make sure you're
>> pointing it to an empty xfs directory with the "directory=" option.
>>
>> Casey
>>
>> On Tue, Jul 14, 2015 at 2:45 AM, Haomai Wang <haomaiwang@gmail.com> wrote:
>>> Anyone who have successfully ran the fio with this external io engine
>>> ceph_objectstore?
>>>
>>> It's strange that it alwasys hit segment fault when entering
>>> "_fio_setup_ceph_filestore_data", gdb tells "td->io_ops" is NULL but
>>> when I up the stack, the "td->io_ops" is not null. Maybe it's related
>>> to dlopen?
>>>
>>> On Fri, Jul 10, 2015 at 3:51 PM, Haomai Wang <haomaiwang@gmail.com> wrote:
>>>> I have rebased the branch with master, and push it to ceph upstream
>>>> repo. https://github.com/ceph/ceph/compare/fio-objectstore?expand=1
>>>>
>>>> Plz let me know if who is working on this. Otherwise, I would like to
>>>> improve this to be merge ready.
>>>>
>>>> On Fri, Jul 10, 2015 at 4:26 AM, Matt W. Benjamin <matt@cohortfs.com> wrote:
>>>>> That makes sense.
>>>>>
>>>>> Matt
>>>>>
>>>>> ----- "James (Fei) Liu-SSI" <james.liu@ssi.samsung.com> wrote:
>>>>>
>>>>>> Hi Casey,
>>>>>> Got it. I was directed to the old code base. By the way, Since the
>>>>>> testing case was used to exercise all of object stores. Strongly
>>>>>> recommend to change the name from fio_ceph_filestore.cc to
>>>>>> fio_ceph_objectstore.cc . And the code in fio_ceph_filestore.cc should
>>>>>> be refactored to reflect that the whole objectstore will be supported
>>>>>> by fio_ceph_objectstore.cc. what you think?
>>>>>>
>>>>>> Let me know if you need any help from my side.
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> James
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Casey Bodley [mailto:cbodley@gmail.com]
>>>>>> Sent: Thursday, July 09, 2015 12:32 PM
>>>>>> To: James (Fei) Liu-SSI
>>>>>> Cc: Haomai Wang; ceph-devel@vger.kernel.org
>>>>>> Subject: Re: About Fio backend with ObjectStore API
>>>>>>
>>>>>> Hi James,
>>>>>>
>>>>>> Are you looking at the code from
>>>>>> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore? It
>>>>>> uses ObjectStore::create() instead of new FileStore(). This allows us
>>>>>> to exercise all of the object stores with the same code.
>>>>>>
>>>>>> Casey
>>>>>>
>>>>>> On Thu, Jul 9, 2015 at 2:01 PM, James (Fei) Liu-SSI
>>>>>> <james.liu@ssi.samsung.com> wrote:
>>>>>> > Hi Casey,
>>>>>> > Here is the code in the fio_ceph_filestore.cc. Basically, it
>>>>>> creates a filestore as backend engine for IO exercises. If we got to
>>>>>> send IO commands to KeyValue Store or Newstore, we got to change the
>>>>>> code accordingly, right? I did not see any other files like
>>>>>> fio_ceph_keyvaluestore.cc or fio_ceph_newstore.cc. In my humble
>>>>>> opinion, we might need to create other two fio engines for
>>>>>> keyvaluestore and newstore if we want to exercise these two, right?
>>>>>> >
>>>>>> > Regards,
>>>>>> > James
>>>>>> >
>>>>>> > static int fio_ceph_filestore_init(struct thread_data *td)
>>>>>> > 209 {
>>>>>> > 210 vector<const char*> args;
>>>>>> > 211 struct ceph_filestore_data *ceph_filestore_data = (struct
>>>>>> ceph_filestore_data *) td->io_ops->data;
>>>>>> > 212 ObjectStore::Transaction ft;
>>>>>> > 213
>>>>>> >
>>>>>> > 214 global_init(NULL, args, CEPH_ENTITY_TYPE_OSD,
>>>>>> CODE_ENVIRONMENT_UTILITY, 0);
>>>>>> > 215 //g_conf->journal_dio = false;
>>>>>> > 216 common_init_finish(g_ceph_context);
>>>>>> > 217 //g_ceph_context->_conf->set_val("debug_filestore", "20");
>>>>>> > 218 //g_ceph_context->_conf->set_val("debug_throttle", "20");
>>>>>> > 219 g_ceph_context->_conf->apply_changes(NULL);
>>>>>> > 220
>>>>>> >
>>>>>> > 221 ceph_filestore_data->osd_path =
>>>>>> strdup("/mnt/fio_ceph_filestore.XXXXXXX");
>>>>>> > 222 ceph_filestore_data->journal_path =
>>>>>> strdup("/var/lib/ceph/osd/journal-ram/fio_ceph_filestore.XXXXXXX");
>>>>>> > 223
>>>>>> >
>>>>>> > 224 if (!mkdtemp(ceph_filestore_data->osd_path)) {
>>>>>> > 225 cout << "mkdtemp failed: " << strerror(errno) <<
>>>>>> std::endl;
>>>>>> > 226 return 1;
>>>>>> > 227 }
>>>>>> > 228 //mktemp(ceph_filestore_data->journal_path); // NOSPC issue
>>>>>> > 229
>>>>>> >
>>>>>> > 230 ObjectStore *fs = new
>>>>>> FileStore(ceph_filestore_data->osd_path,
>>>>>> ceph_filestore_data->journal_path);
>>>>>> > 231 ceph_filestore_data->fs = fs;
>>>>>> > 232
>>>>>> >
>>>>>> > 233 if (fs->mkfs() < 0) {
>>>>>> > 234 cout << "mkfs failed" << std::endl;
>>>>>> > 235 goto failed;
>>>>>> > 236 }
>>>>>> > 237
>>>>>> > 238 if (fs->mount() < 0) {
>>>>>> > 239 cout << "mount failed" << std::endl;
>>>>>> > 240 goto failed;
>>>>>> > 241 }
>>>>>> > 242
>>>>>> >
>>>>>> > 243 ft.create_collection(coll_t());
>>>>>> > 244 fs->apply_transaction(ft);
>>>>>> > 245
>>>>>> >
>>>>>> > 246
>>>>>> >
>>>>>> > 247 return 0;
>>>>>> > 248
>>>>>> >
>>>>>> > 249 failed:
>>>>>> > 250 return 1;
>>>>>> > 251
>>>>>> >
>>>>>> > 252 }
>>>>>> > -----Original Message-----
>>>>>> > From: Casey Bodley [mailto:cbodley@gmail.com]
>>>>>> > Sent: Thursday, July 09, 2015 9:19 AM
>>>>>> > To: James (Fei) Liu-SSI
>>>>>> > Cc: Haomai Wang; ceph-devel@vger.kernel.org
>>>>>> > Subject: Re: About Fio backend with ObjectStore API
>>>>>> >
>>>>>> > Hi James,
>>>>>> >
>>>>>> > In the job file src/test/filestore.fio, you can modify the line
>>>>>> > "objectstore=filestore" to use any objectstore type supported by
>>>>>> the
>>>>>> > ObjectStore::create() factory.
>>>>>> >
>>>>>> > Casey
>>>>>> >
>>>>>> > On Wed, Jul 8, 2015 at 8:02 PM, James (Fei) Liu-SSI
>>>>>> <james.liu@ssi.samsung.com> wrote:
>>>>>> >> Hi Casey,
>>>>>> >> Quick questions, The code in the trunk only cover the test for
>>>>>> filestore. I was wondering do you have any plan to cover the test for
>>>>>> kvstore and newstore?
>>>>>> >>
>>>>>> >> Thanks,
>>>>>> >> James
>>>>>> >>
>>>>>> >> -----Original Message-----
>>>>>> >> From: ceph-devel-owner@vger.kernel.org
>>>>>> >> [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of James (Fei)
>>>>>>
>>>>>> >> Liu-SSI
>>>>>> >> Sent: Tuesday, June 30, 2015 2:19 PM
>>>>>> >> To: Casey Bodley
>>>>>> >> Cc: Haomai Wang; ceph-devel@vger.kernel.org
>>>>>> >> Subject: RE: About Fio backend with ObjectStore API
>>>>>> >>
>>>>>> >> Hi Casey,
>>>>>> >>
>>>>>> >> Thanks a lot.
>>>>>> >>
>>>>>> >> Regards,
>>>>>> >> James
>>>>>> >>
>>>>>> >> -----Original Message-----
>>>>>> >> From: Casey Bodley [mailto:cbodley@gmail.com]
>>>>>> >> Sent: Tuesday, June 30, 2015 2:16 PM
>>>>>> >> To: James (Fei) Liu-SSI
>>>>>> >> Cc: Haomai Wang; ceph-devel@vger.kernel.org
>>>>>> >> Subject: Re: About Fio backend with ObjectStore API
>>>>>> >>
>>>>>> >> Hi,
>>>>>> >>
>>>>>> >> When Danny Al-Gaaf & Daniel Gollub published "Ceph Performance
>>>>>> >> Analysis: fio and RBD" at
>>>>>> >>
>>>>>> https://telekomcloud.github.io/ceph/2014/02/26/ceph-performance-analy
>>>>>> >> s is_fio_rbd.html, they also mentioned a fio engine that linked
>>>>>> >> directly into ceph's FileStore. I was able to find Daniel's branch
>>>>>> on
>>>>>> >> github at https://github.com/gollub/ceph/tree/fio_filestore_v2, and
>>>>>> did some more work on it at the time.
>>>>>> >>
>>>>>> >> I just rebased that work onto the latest ceph master branch, and
>>>>>> pushed to our github at
>>>>>> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore. You
>>>>>> can find the source in src/test/fio_ceph_filestore.cc, and run fio
>>>>>> with the provided example fio job file in src/test/filestore.fio.
>>>>>> >>
>>>>>> >> I didn't have a chance to confirm that it builds with automake, but
>>>>>>
>>>>>> >> the cmake version built for me. I'm happy to help if you run into
>>>>>> >> problems, Casey
>>>>>> >>
>>>>>> >> On Tue, Jun 30, 2015 at 2:31 PM, James (Fei) Liu-SSI
>>>>>> <james.liu@ssi.samsung.com> wrote:
>>>>>> >>> Hi Haomai,
>>>>>> >>> What are you trying to ask is to benchmark local
>>>>>> objectstore(like kvstore/filestore/newstore) locally with
>>>>>> FIO(ObjectStore engine)? You want to purely compare the performance
>>>>>> locally for these objectstores, right?
>>>>>> >>>
>>>>>> >>> Regards,
>>>>>> >>> James
>>>>>> >>>
>>>>>> >>> -----Original Message-----
>>>>>> >>> From: ceph-devel-owner@vger.kernel.org
>>>>>> >>> [mailto:ceph-devel-owner@vger.kernel.org] On Behalf Of Haomai
>>>>>> Wang
>>>>>> >>> Sent: Tuesday, June 30, 2015 9:06 AM
>>>>>> >>> To: ceph-devel@vger.kernel.org
>>>>>> >>> Subject: About Fio backend with ObjectStore API
>>>>>> >>>
>>>>>> >>> Hi all,
>>>>>> >>>
>>>>>> >>> Long long ago, is there someone said about fio backend with Ceph
>>>>>> ObjectStore API? So we could use the existing mature fio facility to
>>>>>> benchmark ceph objectstore.
>>>>>> >>>
>>>>>> >>> --
>>>>>> >>> Best Regards,
>>>>>> >>>
>>>>>> >>> Wheat
>>>>>> >>> --
>>>>>> >>> To unsubscribe from this list: send the line "unsubscribe
>>>>>> ceph-devel"
>>>>>> >>> in the body of a message to majordomo@vger.kernel.org More
>>>>>> majordomo
>>>>>> >>> info at http://vger.kernel.org/majordomo-info.html
>>>>>> >> {.n + +% lzwm b 맲 r yǩ ׯzX ܨ} Ơz &j:+v zZ+
>>>>>> +zf h ~ i z w ? & )ߢ f
>>>>>> N�����r��y���b�X��ǧv�^�){.n�+���z�]z���{ay�ʇڙ�,j��f���h���z��w������j:+v���w�j�m��������zZ+��ݢj"��
>>>>>
>>>>> --
>>>>> Matt Benjamin
>>>>> CohortFS, LLC.
>>>>> 315 West Huron Street, Suite 140A
>>>>> Ann Arbor, Michigan 48103
>>>>>
>>>>> http://cohortfs.com
>>>>>
>>>>> tel. 734-761-4689
>>>>> fax. 734-769-8938
>>>>> cel. 734-216-5309
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>>
>>>> Wheat
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>>
>>> Wheat
>
>
>
> --
> Best Regards,
>
> Wheat
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Best Regards,
Wheat
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2015-07-23 1:07 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-30 16:05 About Fio backend with ObjectStore API Haomai Wang
2015-06-30 18:31 ` James (Fei) Liu-SSI
2015-06-30 21:15 ` Casey Bodley
2015-06-30 21:19 ` James (Fei) Liu-SSI
2015-06-30 22:38 ` Mark Nelson
2015-06-30 22:57 ` Jens Axboe
2015-06-30 23:18 ` Josh Durgin
2015-06-30 23:21 ` Josh Durgin
[not found] ` <CAHAfqgPMyfVtYrAp8ox11uA=q+zEDKP7Zoc2WveuYat=0hD8zg@mail.gmail.com>
2015-07-07 8:17 ` Haomai Wang
2015-07-07 17:25 ` James (Fei) Liu-SSI
2015-07-09 0:02 ` James (Fei) Liu-SSI
2015-07-09 16:19 ` Casey Bodley
2015-07-09 18:01 ` James (Fei) Liu-SSI
2015-07-09 19:32 ` Casey Bodley
2015-07-09 20:21 ` James (Fei) Liu-SSI
2015-07-09 20:26 ` Matt W. Benjamin
2015-07-10 7:51 ` Haomai Wang
2015-07-14 6:45 ` Haomai Wang
2015-07-15 21:23 ` Casey Bodley
2015-07-21 11:50 ` Haomai Wang
2015-07-22 20:56 ` Casey Bodley
2015-07-23 1:07 ` Haomai Wang [this message]
2015-09-03 0:06 ` James (Fei) Liu-SSI
2015-09-03 17:43 ` Casey Bodley
2015-09-04 20:29 ` James (Fei) Liu-SSI
2015-09-05 3:43 ` Haomai Wang
2015-09-11 0:08 ` James (Fei) Liu-SSI
2015-09-11 14:28 ` Casey Bodley
2015-09-11 14:35 ` Casey Bodley
2015-09-11 17:18 ` James (Fei) Liu-SSI
2015-09-11 20:08 ` Casey Bodley
2015-09-12 9:04 ` Haomai Wang
2015-09-12 9:16 ` Haomai Wang
2015-09-15 0:02 ` James (Fei) Liu-SSI
2015-09-12 23:23 ` Matt Benjamin
[not found] ` <99767EA2E27DD44DB4E9F9B9ACA458C03BD7AC7A@SSIEXCH-MB3.ssi.samsung.com>
[not found] ` <CACJqLyapDJ+H54fAeOVaWNh4DQHJkMVXbRXrzys08QNuCcXskw@mail.gmail.com>
2015-09-09 21:48 ` James (Fei) Liu-SSI
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACJqLyY9t_t7omJfb9vknwqBQ2V7csfLPqU-jvUtvDj2fhuyPw@mail.gmail.com \
--to=haomaiwang@gmail.com \
--cc=cbodley@gmail.com \
--cc=cbodley@redhat.com \
--cc=ceph-devel@vger.kernel.org \
--cc=james.liu@ssi.samsung.com \
--cc=matt@cohortfs.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.