rwg/civetweb log verbosity level

rwg/civetweb log verbosity level

[zyn赵亚楠]

[-- Attachment #1.1: Type: text/plain, Size: 1588 bytes --]

Hi there,

I have a question about rgw/civetweb log settings.

Currently, rgw/civetweb prints 3 lines of logs with loglevel 1 (high priority) for each HTTP request, like following:

$ tail /var/log/ceph/ceph-client.rgw.node-1.log
2018-11-28 11:52:45.339229 7fbf2d693700  1 ====== starting new request req=0x7fbf2d68d190 =====
2018-11-28 11:52:45.341961 7fbf2d693700  1 ====== req done req=0x7fbf2d68d190 op status=0 http_status=200 ======
2018-11-28 11:52:45.341993 7fbf2d693700  1 civetweb: 0x558f04330000: 127.0.0.1 - - [28/Nov/2018:11:48:10 +0800] "HEAD /swift/v1/images.xxx.com/8801234/BFAB307D-F5FE-4BC6-9449-E854944A460F_160_180.jpg HTTP/1.1" 1 0 - goswift/1.0

The above 3 lines occupies roughly 0.5KB space on average, varying a little with the lengths of bucket names and object names.

Now the problem is, when requests are intensive, it will consume a huge mount of space. For example, 4 million requests (on a single RGW node) will result to 2GB, which takes only ~6 hours to happen in our cluster node in busy period (a large part may be HEAD requests).

When trouble shooting, I usually need to turn the loglevel to 5, 10 or even bigger to check the detailed logs, but most of the log space is occupied by the above access logs (level 1), which doesn’t provide much information.

My question is, is there a way to configure Ceph skip those logs? E.g. only print logs with verbosity in a specified range (NOT support, according to my investigation).
Or, are there any suggested ways for turning on more logs for debugging?

Best Regards
Arthur Chiao


[-- Attachment #1.2: Type: text/html, Size: 4845 bytes --]

[-- Attachment #2: Type: text/plain, Size: 178 bytes --]

_______________________________________________
ceph-users mailing list
ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: rwg/civetweb log verbosity level

[Casey Bodley]

This stuff is logged under the 'civetweb' subsystem, so can be turned 
off with 'debug_civetweb = 0'. You can configure 'debug_rgw' separately.

On 11/28/18 1:03 AM, zyn赵亚楠 wrote:
>
> Hi there,
>
> I have a question about rgw/civetweb log settings.
>
> Currently, rgw/civetweb prints 3 lines of logs with loglevel 1 (high 
> priority) for each HTTP request, like following:
>
> $ tail /var/log/ceph/ceph-client.rgw.node-1.log
>
> 2018-11-28 11:52:45.339229 7fbf2d693700  1 ====== starting new request 
> req=0x7fbf2d68d190 =====
>
> 2018-11-28 11:52:45.341961 7fbf2d693700  1 ====== req done 
> req=0x7fbf2d68d190 op status=0 http_status=200 ======
>
> 2018-11-28 11:52:45.341993 7fbf2d693700  1 civetweb: 0x558f04330000: 
> 127.0.0.1 - - [28/Nov/2018:11:48:10 +0800] "HEAD 
> /swift/v1/images.xxx.com/8801234/BFAB307D-F5FE-4BC6-9449-E854944A460F_160_180.jpg 
> HTTP/1.1" 1 0 - goswift/1.0
>
> The above 3 lines occupies roughly 0.5KB space on average, varying a 
> little with the lengths of bucket names and object names.
>
> Now the problem is, when requests are intensive, it will consume a 
> huge mount of space. For example, 4 million requests (on a single RGW 
> node) will result to 2GB, which takes only ~6 hours to happen in our 
> cluster node in busy period (a large part may be HEAD requests).
>
> When trouble shooting, I usually need to turn the loglevel to 5, 10 or 
> even bigger to check the detailed logs, but most of the log space is 
> occupied by the above access logs (level 1), which doesn’t provide 
> much information.
>
> My question is, is there a way to configure Ceph skip those logs? E.g. 
> only print logs with verbosity in a specified range (NOT support, 
> according to my investigation).
>
> Or, are there any suggested ways for turning on more logs for debugging?
>
> Best Regards
>
> Arthur Chiao
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com