* Log spam in master with leveldb messages
@ 2016-02-03 0:03 Somnath Roy
2016-02-03 1:59 ` Sage Weil
0 siblings, 1 reply; 6+ messages in thread
From: Somnath Roy @ 2016-02-03 0:03 UTC (permalink / raw)
To: ceph-devel
Hi,
I saw osd log is spamming with following leveldb messages..
2016-02-02 15:47:01.559086 7f6565fff700 1 leveldb: Compacting 2@0 + 6@1 files
2016-02-02 15:47:01.601971 7f6565fff700 1 leveldb: Generated table #45588: 57785 keys, 2130651 bytes
2016-02-02 15:47:01.645375 7f6565fff700 1 leveldb: Generated table #45589: 58653 keys, 2131228 bytes
2016-02-02 15:47:01.692566 7f6565fff700 1 leveldb: Generated table #45590: 57729 keys, 2131490 bytes
2016-02-02 15:47:01.696688 7f6565fff700 1 leveldb: Generated table #45591: 4131 keys, 167758 bytes
2016-02-02 15:47:01.735986 7f6565fff700 1 leveldb: Generated table #45592: 44341 keys, 1684603 bytes
2016-02-02 15:47:01.783394 7f6565fff700 1 leveldb: Generated table #45593: 57672 keys, 2130488 bytes
2016-02-02 15:47:01.792750 7f6565fff700 1 leveldb: Generated table #45594: 11073 keys, 408429 bytes
2016-02-02 15:47:01.792759 7f6565fff700 1 leveldb: Compacted 2@0 + 6@1 files => 10784647 bytes
2016-02-02 15:47:01.793017 7f6565fff700 1 leveldb: compacted to: files[ 0 7 25 0 0 0 0 ]
2016-02-02 15:47:01.793089 7f6565fff700 1 leveldb: Compacting 1@1 + 4@2 files
2016-02-02 15:47:01.830144 7f6565fff700 1 leveldb: Generated table #45595: 32249 keys, 2130919 bytes
2016-02-02 15:47:01.867178 7f6565fff700 1 leveldb: Generated table #45596: 32228 keys, 2131274 bytes
2016-02-02 15:47:01.908135 7f6565fff700 1 leveldb: Generated table #45597: 32246 keys, 2130944 bytes
2016-02-02 15:47:01.949642 7f6565fff700 1 leveldb: Generated table #45598: 32211 keys, 2130740 bytes
2016-02-02 15:47:01.950102 7f6565fff700 1 leveldb: Generated table #45599: 60 keys, 4062 bytes
2016-02-02 15:47:01.950110 7f6565fff700 1 leveldb: Compacted 1@1 + 4@2 files => 8527939 bytes
2016-02-02 15:47:01.950356 7f6565fff700 1 leveldb: compacted to: files[ 0 6 26 0 0 0 0 ]
2016-02-02 15:47:01.950566 7f6565fff700 1 leveldb: Delete type=2 #45577
2016-02-02 15:47:01.951127 7f6565fff700 1 leveldb: Delete type=2 #45578
I found out now we are redirecting leveldb logs to ceph log based on config option "leveldb_log_to_ceph_log" which is by default true.
If there is no strong reason I think it should be by default false.
Thanks & Regards
Somnath
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Log spam in master with leveldb messages
2016-02-03 0:03 Log spam in master with leveldb messages Somnath Roy
@ 2016-02-03 1:59 ` Sage Weil
2016-02-03 2:39 ` Somnath Roy
2016-02-03 9:13 ` Dan van der Ster
0 siblings, 2 replies; 6+ messages in thread
From: Sage Weil @ 2016-02-03 1:59 UTC (permalink / raw)
To: Somnath Roy; +Cc: ceph-devel
On Wed, 3 Feb 2016, Somnath Roy wrote:
> Hi,
> I saw osd log is spamming with following leveldb messages..
>
> 2016-02-02 15:47:01.559086 7f6565fff700 1 leveldb: Compacting 2@0 + 6@1 files
> 2016-02-02 15:47:01.601971 7f6565fff700 1 leveldb: Generated table #45588: 57785 keys, 2130651 bytes
> 2016-02-02 15:47:01.645375 7f6565fff700 1 leveldb: Generated table #45589: 58653 keys, 2131228 bytes
> 2016-02-02 15:47:01.692566 7f6565fff700 1 leveldb: Generated table #45590: 57729 keys, 2131490 bytes
> 2016-02-02 15:47:01.696688 7f6565fff700 1 leveldb: Generated table #45591: 4131 keys, 167758 bytes
> 2016-02-02 15:47:01.735986 7f6565fff700 1 leveldb: Generated table #45592: 44341 keys, 1684603 bytes
> 2016-02-02 15:47:01.783394 7f6565fff700 1 leveldb: Generated table #45593: 57672 keys, 2130488 bytes
> 2016-02-02 15:47:01.792750 7f6565fff700 1 leveldb: Generated table #45594: 11073 keys, 408429 bytes
> 2016-02-02 15:47:01.792759 7f6565fff700 1 leveldb: Compacted 2@0 + 6@1 files => 10784647 bytes
> 2016-02-02 15:47:01.793017 7f6565fff700 1 leveldb: compacted to: files[ 0 7 25 0 0 0 0 ]
> 2016-02-02 15:47:01.793089 7f6565fff700 1 leveldb: Compacting 1@1 + 4@2 files
> 2016-02-02 15:47:01.830144 7f6565fff700 1 leveldb: Generated table #45595: 32249 keys, 2130919 bytes
> 2016-02-02 15:47:01.867178 7f6565fff700 1 leveldb: Generated table #45596: 32228 keys, 2131274 bytes
> 2016-02-02 15:47:01.908135 7f6565fff700 1 leveldb: Generated table #45597: 32246 keys, 2130944 bytes
> 2016-02-02 15:47:01.949642 7f6565fff700 1 leveldb: Generated table #45598: 32211 keys, 2130740 bytes
> 2016-02-02 15:47:01.950102 7f6565fff700 1 leveldb: Generated table #45599: 60 keys, 4062 bytes
> 2016-02-02 15:47:01.950110 7f6565fff700 1 leveldb: Compacted 1@1 + 4@2 files => 8527939 bytes
> 2016-02-02 15:47:01.950356 7f6565fff700 1 leveldb: compacted to: files[ 0 6 26 0 0 0 0 ]
> 2016-02-02 15:47:01.950566 7f6565fff700 1 leveldb: Delete type=2 #45577
>
> 2016-02-02 15:47:01.951127 7f6565fff700 1 leveldb: Delete type=2 #45578
>
> I found out now we are redirecting leveldb logs to ceph log based on config option "leveldb_log_to_ceph_log" which is by default true.
>
> If there is no strong reason I think it should be by default false.
The problem is that otherwise the log goes to the db directory and is
never rotated. Or, we set it to /dev/null and we get no logs at all.
Is the quantity of logging too much? In rocksdb we can limit by log level
(warn/info/debug etc), but not so in leveldb. :/
sage
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: Log spam in master with leveldb messages
2016-02-03 1:59 ` Sage Weil
@ 2016-02-03 2:39 ` Somnath Roy
2016-02-03 3:50 ` Sage Weil
2016-02-03 9:13 ` Dan van der Ster
1 sibling, 1 reply; 6+ messages in thread
From: Somnath Roy @ 2016-02-03 2:39 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
In my single osd test, every one sec (~) it's dumping the below chunk..
In past, was the leveldb log helpful, helped debugging in any environment ?
If so, we have to keep it I guess :-(..
Thanks & Regards
Somnath
-----Original Message-----
From: Sage Weil [mailto:sage@newdream.net]
Sent: Tuesday, February 02, 2016 6:00 PM
To: Somnath Roy
Cc: ceph-devel@vger.kernel.org
Subject: Re: Log spam in master with leveldb messages
On Wed, 3 Feb 2016, Somnath Roy wrote:
> Hi,
> I saw osd log is spamming with following leveldb messages..
>
> 2016-02-02 15:47:01.559086 7f6565fff700 1 leveldb: Compacting 2@0 +
> 6@1 files
> 2016-02-02 15:47:01.601971 7f6565fff700 1 leveldb: Generated table
> #45588: 57785 keys, 2130651 bytes
> 2016-02-02 15:47:01.645375 7f6565fff700 1 leveldb: Generated table
> #45589: 58653 keys, 2131228 bytes
> 2016-02-02 15:47:01.692566 7f6565fff700 1 leveldb: Generated table
> #45590: 57729 keys, 2131490 bytes
> 2016-02-02 15:47:01.696688 7f6565fff700 1 leveldb: Generated table
> #45591: 4131 keys, 167758 bytes
> 2016-02-02 15:47:01.735986 7f6565fff700 1 leveldb: Generated table
> #45592: 44341 keys, 1684603 bytes
> 2016-02-02 15:47:01.783394 7f6565fff700 1 leveldb: Generated table
> #45593: 57672 keys, 2130488 bytes
> 2016-02-02 15:47:01.792750 7f6565fff700 1 leveldb: Generated table
> #45594: 11073 keys, 408429 bytes
> 2016-02-02 15:47:01.792759 7f6565fff700 1 leveldb: Compacted 2@0 +
> 6@1 files => 10784647 bytes
> 2016-02-02 15:47:01.793017 7f6565fff700 1 leveldb: compacted to:
> files[ 0 7 25 0 0 0 0 ]
> 2016-02-02 15:47:01.793089 7f6565fff700 1 leveldb: Compacting 1@1 +
> 4@2 files
> 2016-02-02 15:47:01.830144 7f6565fff700 1 leveldb: Generated table
> #45595: 32249 keys, 2130919 bytes
> 2016-02-02 15:47:01.867178 7f6565fff700 1 leveldb: Generated table
> #45596: 32228 keys, 2131274 bytes
> 2016-02-02 15:47:01.908135 7f6565fff700 1 leveldb: Generated table
> #45597: 32246 keys, 2130944 bytes
> 2016-02-02 15:47:01.949642 7f6565fff700 1 leveldb: Generated table
> #45598: 32211 keys, 2130740 bytes
> 2016-02-02 15:47:01.950102 7f6565fff700 1 leveldb: Generated table
> #45599: 60 keys, 4062 bytes
> 2016-02-02 15:47:01.950110 7f6565fff700 1 leveldb: Compacted 1@1 +
> 4@2 files => 8527939 bytes
> 2016-02-02 15:47:01.950356 7f6565fff700 1 leveldb: compacted to:
> files[ 0 6 26 0 0 0 0 ]
> 2016-02-02 15:47:01.950566 7f6565fff700 1 leveldb: Delete type=2
> #45577
>
> 2016-02-02 15:47:01.951127 7f6565fff700 1 leveldb: Delete type=2
> #45578
>
> I found out now we are redirecting leveldb logs to ceph log based on config option "leveldb_log_to_ceph_log" which is by default true.
>
> If there is no strong reason I think it should be by default false.
The problem is that otherwise the log goes to the db directory and is never rotated. Or, we set it to /dev/null and we get no logs at all.
Is the quantity of logging too much? In rocksdb we can limit by log level (warn/info/debug etc), but not so in leveldb. :/
sage
PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: Log spam in master with leveldb messages
2016-02-03 2:39 ` Somnath Roy
@ 2016-02-03 3:50 ` Sage Weil
2016-02-03 3:56 ` Somnath Roy
0 siblings, 1 reply; 6+ messages in thread
From: Sage Weil @ 2016-02-03 3:50 UTC (permalink / raw)
To: Somnath Roy; +Cc: ceph-devel
On Wed, 3 Feb 2016, Somnath Roy wrote:
> In my single osd test, every one sec (~) it's dumping the below chunk..
> In past, was the leveldb log helpful, helped debugging in any environment ?
> If so, we have to keep it I guess :-(..
Hrm, yeah that's a lot. It makes me nervous to just send it all to
/dev/null, though, too.
Perhaps what we should really do is move the monitor kv store to rocksdb,
which lets you adjust the logging level. This will avoid our annoying
distro-dependent leveldb issues since it's linked statically. (And get the
distro folks up in arms, no doubt...)
sage
>
> Thanks & Regards
> Somnath
>
> -----Original Message-----
> From: Sage Weil [mailto:sage@newdream.net]
> Sent: Tuesday, February 02, 2016 6:00 PM
> To: Somnath Roy
> Cc: ceph-devel@vger.kernel.org
> Subject: Re: Log spam in master with leveldb messages
>
> On Wed, 3 Feb 2016, Somnath Roy wrote:
> > Hi,
> > I saw osd log is spamming with following leveldb messages..
> >
> > 2016-02-02 15:47:01.559086 7f6565fff700 1 leveldb: Compacting 2@0 +
> > 6@1 files
> > 2016-02-02 15:47:01.601971 7f6565fff700 1 leveldb: Generated table
> > #45588: 57785 keys, 2130651 bytes
> > 2016-02-02 15:47:01.645375 7f6565fff700 1 leveldb: Generated table
> > #45589: 58653 keys, 2131228 bytes
> > 2016-02-02 15:47:01.692566 7f6565fff700 1 leveldb: Generated table
> > #45590: 57729 keys, 2131490 bytes
> > 2016-02-02 15:47:01.696688 7f6565fff700 1 leveldb: Generated table
> > #45591: 4131 keys, 167758 bytes
> > 2016-02-02 15:47:01.735986 7f6565fff700 1 leveldb: Generated table
> > #45592: 44341 keys, 1684603 bytes
> > 2016-02-02 15:47:01.783394 7f6565fff700 1 leveldb: Generated table
> > #45593: 57672 keys, 2130488 bytes
> > 2016-02-02 15:47:01.792750 7f6565fff700 1 leveldb: Generated table
> > #45594: 11073 keys, 408429 bytes
> > 2016-02-02 15:47:01.792759 7f6565fff700 1 leveldb: Compacted 2@0 +
> > 6@1 files => 10784647 bytes
> > 2016-02-02 15:47:01.793017 7f6565fff700 1 leveldb: compacted to:
> > files[ 0 7 25 0 0 0 0 ]
> > 2016-02-02 15:47:01.793089 7f6565fff700 1 leveldb: Compacting 1@1 +
> > 4@2 files
> > 2016-02-02 15:47:01.830144 7f6565fff700 1 leveldb: Generated table
> > #45595: 32249 keys, 2130919 bytes
> > 2016-02-02 15:47:01.867178 7f6565fff700 1 leveldb: Generated table
> > #45596: 32228 keys, 2131274 bytes
> > 2016-02-02 15:47:01.908135 7f6565fff700 1 leveldb: Generated table
> > #45597: 32246 keys, 2130944 bytes
> > 2016-02-02 15:47:01.949642 7f6565fff700 1 leveldb: Generated table
> > #45598: 32211 keys, 2130740 bytes
> > 2016-02-02 15:47:01.950102 7f6565fff700 1 leveldb: Generated table
> > #45599: 60 keys, 4062 bytes
> > 2016-02-02 15:47:01.950110 7f6565fff700 1 leveldb: Compacted 1@1 +
> > 4@2 files => 8527939 bytes
> > 2016-02-02 15:47:01.950356 7f6565fff700 1 leveldb: compacted to:
> > files[ 0 6 26 0 0 0 0 ]
> > 2016-02-02 15:47:01.950566 7f6565fff700 1 leveldb: Delete type=2
> > #45577
> >
> > 2016-02-02 15:47:01.951127 7f6565fff700 1 leveldb: Delete type=2
> > #45578
> >
> > I found out now we are redirecting leveldb logs to ceph log based on config option "leveldb_log_to_ceph_log" which is by default true.
> >
> > If there is no strong reason I think it should be by default false.
>
> The problem is that otherwise the log goes to the db directory and is never rotated. Or, we set it to /dev/null and we get no logs at all.
>
> Is the quantity of logging too much? In rocksdb we can limit by log level (warn/info/debug etc), but not so in leveldb. :/
>
> sage
> PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* RE: Log spam in master with leveldb messages
2016-02-03 3:50 ` Sage Weil
@ 2016-02-03 3:56 ` Somnath Roy
0 siblings, 0 replies; 6+ messages in thread
From: Somnath Roy @ 2016-02-03 3:56 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
That would be ideal , BTW, it is the filestore kv store which is dumping this log for me not the monitor kv store..
-----Original Message-----
From: Sage Weil [mailto:sage@newdream.net]
Sent: Tuesday, February 02, 2016 7:51 PM
To: Somnath Roy
Cc: ceph-devel@vger.kernel.org
Subject: RE: Log spam in master with leveldb messages
On Wed, 3 Feb 2016, Somnath Roy wrote:
> In my single osd test, every one sec (~) it's dumping the below chunk..
> In past, was the leveldb log helpful, helped debugging in any environment ?
> If so, we have to keep it I guess :-(..
Hrm, yeah that's a lot. It makes me nervous to just send it all to /dev/null, though, too.
Perhaps what we should really do is move the monitor kv store to rocksdb, which lets you adjust the logging level. This will avoid our annoying distro-dependent leveldb issues since it's linked statically. (And get the distro folks up in arms, no doubt...)
sage
>
> Thanks & Regards
> Somnath
>
> -----Original Message-----
> From: Sage Weil [mailto:sage@newdream.net]
> Sent: Tuesday, February 02, 2016 6:00 PM
> To: Somnath Roy
> Cc: ceph-devel@vger.kernel.org
> Subject: Re: Log spam in master with leveldb messages
>
> On Wed, 3 Feb 2016, Somnath Roy wrote:
> > Hi,
> > I saw osd log is spamming with following leveldb messages..
> >
> > 2016-02-02 15:47:01.559086 7f6565fff700 1 leveldb: Compacting 2@0 +
> > 6@1 files
> > 2016-02-02 15:47:01.601971 7f6565fff700 1 leveldb: Generated table
> > #45588: 57785 keys, 2130651 bytes
> > 2016-02-02 15:47:01.645375 7f6565fff700 1 leveldb: Generated table
> > #45589: 58653 keys, 2131228 bytes
> > 2016-02-02 15:47:01.692566 7f6565fff700 1 leveldb: Generated table
> > #45590: 57729 keys, 2131490 bytes
> > 2016-02-02 15:47:01.696688 7f6565fff700 1 leveldb: Generated table
> > #45591: 4131 keys, 167758 bytes
> > 2016-02-02 15:47:01.735986 7f6565fff700 1 leveldb: Generated table
> > #45592: 44341 keys, 1684603 bytes
> > 2016-02-02 15:47:01.783394 7f6565fff700 1 leveldb: Generated table
> > #45593: 57672 keys, 2130488 bytes
> > 2016-02-02 15:47:01.792750 7f6565fff700 1 leveldb: Generated table
> > #45594: 11073 keys, 408429 bytes
> > 2016-02-02 15:47:01.792759 7f6565fff700 1 leveldb: Compacted 2@0 +
> > 6@1 files => 10784647 bytes
> > 2016-02-02 15:47:01.793017 7f6565fff700 1 leveldb: compacted to:
> > files[ 0 7 25 0 0 0 0 ]
> > 2016-02-02 15:47:01.793089 7f6565fff700 1 leveldb: Compacting 1@1 +
> > 4@2 files
> > 2016-02-02 15:47:01.830144 7f6565fff700 1 leveldb: Generated table
> > #45595: 32249 keys, 2130919 bytes
> > 2016-02-02 15:47:01.867178 7f6565fff700 1 leveldb: Generated table
> > #45596: 32228 keys, 2131274 bytes
> > 2016-02-02 15:47:01.908135 7f6565fff700 1 leveldb: Generated table
> > #45597: 32246 keys, 2130944 bytes
> > 2016-02-02 15:47:01.949642 7f6565fff700 1 leveldb: Generated table
> > #45598: 32211 keys, 2130740 bytes
> > 2016-02-02 15:47:01.950102 7f6565fff700 1 leveldb: Generated table
> > #45599: 60 keys, 4062 bytes
> > 2016-02-02 15:47:01.950110 7f6565fff700 1 leveldb: Compacted 1@1 +
> > 4@2 files => 8527939 bytes
> > 2016-02-02 15:47:01.950356 7f6565fff700 1 leveldb: compacted to:
> > files[ 0 6 26 0 0 0 0 ]
> > 2016-02-02 15:47:01.950566 7f6565fff700 1 leveldb: Delete type=2
> > #45577
> >
> > 2016-02-02 15:47:01.951127 7f6565fff700 1 leveldb: Delete type=2
> > #45578
> >
> > I found out now we are redirecting leveldb logs to ceph log based on config option "leveldb_log_to_ceph_log" which is by default true.
> >
> > If there is no strong reason I think it should be by default false.
>
> The problem is that otherwise the log goes to the db directory and is never rotated. Or, we set it to /dev/null and we get no logs at all.
>
> Is the quantity of logging too much? In rocksdb we can limit by log
> level (warn/info/debug etc), but not so in leveldb. :/
>
> sage
> PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies).
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in the body of a message to majordomo@vger.kernel.org More majordomo
> info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Log spam in master with leveldb messages
2016-02-03 1:59 ` Sage Weil
2016-02-03 2:39 ` Somnath Roy
@ 2016-02-03 9:13 ` Dan van der Ster
1 sibling, 0 replies; 6+ messages in thread
From: Dan van der Ster @ 2016-02-03 9:13 UTC (permalink / raw)
To: Sage Weil; +Cc: Somnath Roy, ceph-devel
On Wed, Feb 3, 2016 at 2:59 AM, Sage Weil <sage@newdream.net> wrote:
> On Wed, 3 Feb 2016, Somnath Roy wrote:
>> Hi,
>> I saw osd log is spamming with following leveldb messages..
>>
>> 2016-02-02 15:47:01.559086 7f6565fff700 1 leveldb: Compacting 2@0 + 6@1 files
>> 2016-02-02 15:47:01.601971 7f6565fff700 1 leveldb: Generated table #45588: 57785 keys, 2130651 bytes
>> 2016-02-02 15:47:01.645375 7f6565fff700 1 leveldb: Generated table #45589: 58653 keys, 2131228 bytes
>> 2016-02-02 15:47:01.692566 7f6565fff700 1 leveldb: Generated table #45590: 57729 keys, 2131490 bytes
>> 2016-02-02 15:47:01.696688 7f6565fff700 1 leveldb: Generated table #45591: 4131 keys, 167758 bytes
>> 2016-02-02 15:47:01.735986 7f6565fff700 1 leveldb: Generated table #45592: 44341 keys, 1684603 bytes
>> 2016-02-02 15:47:01.783394 7f6565fff700 1 leveldb: Generated table #45593: 57672 keys, 2130488 bytes
>> 2016-02-02 15:47:01.792750 7f6565fff700 1 leveldb: Generated table #45594: 11073 keys, 408429 bytes
>> 2016-02-02 15:47:01.792759 7f6565fff700 1 leveldb: Compacted 2@0 + 6@1 files => 10784647 bytes
>> 2016-02-02 15:47:01.793017 7f6565fff700 1 leveldb: compacted to: files[ 0 7 25 0 0 0 0 ]
>> 2016-02-02 15:47:01.793089 7f6565fff700 1 leveldb: Compacting 1@1 + 4@2 files
>> 2016-02-02 15:47:01.830144 7f6565fff700 1 leveldb: Generated table #45595: 32249 keys, 2130919 bytes
>> 2016-02-02 15:47:01.867178 7f6565fff700 1 leveldb: Generated table #45596: 32228 keys, 2131274 bytes
>> 2016-02-02 15:47:01.908135 7f6565fff700 1 leveldb: Generated table #45597: 32246 keys, 2130944 bytes
>> 2016-02-02 15:47:01.949642 7f6565fff700 1 leveldb: Generated table #45598: 32211 keys, 2130740 bytes
>> 2016-02-02 15:47:01.950102 7f6565fff700 1 leveldb: Generated table #45599: 60 keys, 4062 bytes
>> 2016-02-02 15:47:01.950110 7f6565fff700 1 leveldb: Compacted 1@1 + 4@2 files => 8527939 bytes
>> 2016-02-02 15:47:01.950356 7f6565fff700 1 leveldb: compacted to: files[ 0 6 26 0 0 0 0 ]
>> 2016-02-02 15:47:01.950566 7f6565fff700 1 leveldb: Delete type=2 #45577
>>
>> 2016-02-02 15:47:01.951127 7f6565fff700 1 leveldb: Delete type=2 #45578
>>
>> I found out now we are redirecting leveldb logs to ceph log based on config option "leveldb_log_to_ceph_log" which is by default true.
>>
>> If there is no strong reason I think it should be by default false.
>
> The problem is that otherwise the log goes to the db directory and is
> never rotated. Or, we set it to /dev/null and we get no logs at all.
>
What about writing them to a separate file in /var/log/ceph:
leveldb log = /var/log/ceph/ceph-osd.$id.leveldb.log
(we used to do this on the mons in the firefly days -- I only just
noticed today that the config option changed in hammer).
--
dan
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-02-03 9:13 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-03 0:03 Log spam in master with leveldb messages Somnath Roy
2016-02-03 1:59 ` Sage Weil
2016-02-03 2:39 ` Somnath Roy
2016-02-03 3:50 ` Sage Weil
2016-02-03 3:56 ` Somnath Roy
2016-02-03 9:13 ` Dan van der Ster
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.