Logging

Logging

[Zenon Panoussis]

Question: what's the "do not log at all" syntax in ceph.conf? I tried
putting everything on "debug xyz = 0", but ceph (osd in this case) is
still logging plenty. According to this, I think it shouldn't:

# grep debug /etc/ceph/ceph.conf
	; logging, for debugging monitor crashes, in order of
	debug ms = 0
	debug mon = 0
	debug paxos = 0
	debug auth = 0
	; mds logging to debug issues.
	debug ms = 0
	debug mds = 0
	debug mds balancer = 0
	debug mds log = 0
	debug mds_migrator = 0
	debug monc = 0
	; osd logging to debug osd issues, in order of likelihood of being
	debug ms = 0
	debug osd = 0
	debug filestore = 0
	debug journal = 0

Log example:

2011-04-18 01:28:56.787046 7f7976fd8700 osd0 5 pg[1.3d( v 5'186 (5'184,5'186] n=1 ec=2 les=5 3/3/2) [0,1] r=0 mlcod 5'184 active+clean]  sending commit on repgather(0x7f794827fb70 applying 5'186 rep_tid=49196 wfack=0 wfdisk= op=osd_op(mds0.1:27067 200.00000058 [write 3241176~1615] 1.fbbd) v2) 0x7f794811b630

Z

Re: Logging

[Colin McCabe]

On Sun, Apr 17, 2011 at 4:30 PM, Zenon Panoussis <oracle@provocation.net> wrote:
>
> Question: what's the "do not log at all" syntax in ceph.conf? I tried
> putting everything on "debug xyz = 0", but ceph (osd in this case) is
> still logging plenty. According to this, I think it shouldn't:

Hi Zenon,

Messages that are output at log level 0 or -1 will always be
displayed. I'm not sure if this message belongs at that log level or
not-- I haven't dealt with the OSD repop code in a while.

In general, I don't think that anyone will ever want to turn off
logging completely. That would be a little bit like running your
server without syslog. It just doesn't happen in the real world. Also,
Ceph is not stable yet, and logs are the main (only?) diagnostic tool
we have for most problems. I guess sometimes there is a core file if
you've set up your system for that.

That being said, we do need to be a lot less verbose in some places.
If you find any log messages you think are mis-prioritized, patches
are welcome!

Colin


>
> # grep debug /etc/ceph/ceph.conf
>        ; logging, for debugging monitor crashes, in order of
>        debug ms = 0
>        debug mon = 0
>        debug paxos = 0
>        debug auth = 0
>        ; mds logging to debug issues.
>        debug ms = 0
>        debug mds = 0
>        debug mds balancer = 0
>        debug mds log = 0
>        debug mds_migrator = 0
>        debug monc = 0
>        ; osd logging to debug osd issues, in order of likelihood of being
>        debug ms = 0
>        debug osd = 0
>        debug filestore = 0
>        debug journal = 0
>
> Log example:
>
> 2011-04-18 01:28:56.787046 7f7976fd8700 osd0 5 pg[1.3d( v 5'186 (5'184,5'186] n=1 ec=2 les=5 3/3/2) [0,1] r=0 mlcod 5'184 active+clean]  sending commit on repgather(0x7f794827fb70 applying 5'186 rep_tid=49196 wfack=0 wfdisk= op=osd_op(mds0.1:27067 200.00000058 [write 3241176~1615] 1.fbbd) v2) 0x7f794811b630
>
> Z
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Logging

[Zenon Panoussis]

On 04/18/2011 02:53 AM, Colin McCabe wrote:

Hi

>> Question: what's the "do not log at all" syntax in ceph.conf? I tried
>> putting everything on "debug xyz = 0", but ceph (osd in this case) is
>> still logging plenty. According to this, I think it shouldn't:

> Messages that are output at log level 0 or -1 will always be
> displayed. I'm not sure if this message belongs at that log level or
> not-- I haven't dealt with the OSD repop code in a while.

Wido mentioned "log file = " in a recent posting; I'll try it.

> In general, I don't think that anyone will ever want to turn off
> logging completely. That would be a little bit like running your
> server without syslog. It just doesn't happen in the real world. 

Of course you're right in general, but right now I'm trying to figure why
my cluster is sluggy and logging is one potential cause that I'd like to
test eliminating. I've seen something like this once before (in openldap),
inexplicable slowness that I spent a long time trying to understand by
reading logs, until I found out that it went away completely as soon as
I switched off logging.

Z

Re: Logging

[Colin McCabe]

On Mon, Apr 18, 2011 at 3:13 AM, Zenon Panoussis <oracle@provocation.net> wrote:
>
>
> On 04/18/2011 02:53 AM, Colin McCabe wrote:
>
> Hi
>
>>> Question: what's the "do not log at all" syntax in ceph.conf? I tried
>>> putting everything on "debug xyz = 0", but ceph (osd in this case) is
>>> still logging plenty. According to this, I think it shouldn't:
>
>> Messages that are output at log level 0 or -1 will always be
>> displayed. I'm not sure if this message belongs at that log level or
>> not-- I haven't dealt with the OSD repop code in a while.
>
> Wido mentioned "log file = " in a recent posting; I'll try it.
>
>> In general, I don't think that anyone will ever want to turn off
>> logging completely. That would be a little bit like running your
>> server without syslog. It just doesn't happen in the real world.
>
> Of course you're right in general, but right now I'm trying to figure why
> my cluster is sluggy and logging is one potential cause that I'd like to
> test eliminating. I've seen something like this once before (in openldap),
> inexplicable slowness that I spent a long time trying to understand by
> reading logs, until I found out that it went away completely as soon as
> I switched off logging.

Hi Zenon,

If you'd like to measure performance without logging, first set all
the debug levels to 0. You should be able to do this, at least in
head-of-line, with this:

debug mds = 0
debug mds balancer = 0
debug mds log = 0
debug mds log expire = 0
debug mds migrator = 0
debug filestore = 0
debug journal = 0
debug bdev = 0
debug mon = 0
debug auth = 0
debug finisher = 0

Then disable all the log sinks with this:

log file =
log to stderr = 0
log to syslog = false
clog to monitors = false
clog to syslog = false

cheers,
Colin

>
> Z
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Logging

[Zenon Panoussis]

On 04/18/2011 07:16 PM, Colin McCabe wrote:

> If you'd like to measure performance without logging, first set all
> the debug levels to 0. You should be able to do this, at least in
> head-of-line, with this:

> Then disable all the log sinks with this:

Thanks, that's the part I was missing.

In the meanwhile I found a major bottleneck in my cluster, namely the fact
that I had mounted the ext4 partitions that hold the stores with atime
enabled. I should have thought of it earlier...

Z

Re: Logging

[Colin McCabe]

I added "Mount the partition you are logging to using noatime" as a
"tip" on http://ceph.newdream.net/wiki/Cluster_configuration

cheers,
Colin


On Mon, Apr 18, 2011 at 11:17 AM, Zenon Panoussis
<oracle@provocation.net> wrote:
>
>
> On 04/18/2011 07:16 PM, Colin McCabe wrote:
>
>> If you'd like to measure performance without logging, first set all
>> the debug levels to 0. You should be able to do this, at least in
>> head-of-line, with this:
>
>> Then disable all the log sinks with this:
>
> Thanks, that's the part I was missing.
>
> In the meanwhile I found a major bottleneck in my cluster, namely the fact
> that I had mounted the ext4 partitions that hold the stores with atime
> enabled. I should have thought of it earlier...
>
> Z
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Logging

[Zenon Panoussis]

On 04/18/2011 08:41 PM, Colin McCabe wrote:

> I added "Mount the partition you are logging to using noatime" as a
> "tip" on http://ceph.newdream.net/wiki/Cluster_configuration

Aha, and I had put a word about noatime for the OSD partition in
http://ceph.newdream.net/wiki/Mounting_the_file_system . Obviously it doesn't
make much of a difference on writes, but it makes a HUGE difference on reads.

Z

Re: Logging

[Colin McCabe]

On Mon, Apr 18, 2011 at 1:56 PM, Zenon Panoussis <oracle@provocation.net> wrote:
>
> On 04/18/2011 08:41 PM, Colin McCabe wrote:
>
>> I added "Mount the partition you are logging to using noatime" as a
>> "tip" on http://ceph.newdream.net/wiki/Cluster_configuration
>
> Aha, and I had put a word about noatime for the OSD partition in
> http://ceph.newdream.net/wiki/Mounting_the_file_system . Obviously it doesn't
> make much of a difference on writes, but it makes a HUGE difference on reads.
>
> Z

Oh yeah. mtime and ctime are the relevant things for writing (i.e.
logging.) Well, I feel dumb now.

I'm sure there are other optimizations you could do for the logging partition.

I'm curious how much you saved by disabling logging. Also, how are you
measuring performance?

sincerely,
Colin


>
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Logging

[Zenon Panoussis]

On 04/19/2011 12:25 AM, Colin McCabe wrote:

> I'm curious how much you saved by disabling logging. 

To be honest, I have no idea.

> Also, how are you measuring performance?

My measuring is very primitive and un-scientific (which is why I couldn't
answer your previous question). I have a two-node cluster with rule data
{ min_size 2 } and rule metadata { min_size 2 }. My ceph.conf says
osd data = /mnt/osd and the latter is an ext4 partition of its own on
each node. One monitor is running on one of the nodes and one client is
running on my workstation.

So what I do is mount the monitor, run 'watch "df -m; echo; uptime"' on both
nodes and start copying files to ceph with rsync. I have a news spool of just
over a million small (1-3 KB) files which is perfect for the task. The rsync
command is

 rsync -vva --progress --bwlimit=N newsspool 127.0.0.1:/mnt/ceph/

where N is a number in KBps ranging from max 500 all the way down to 5. This
gives me something like

===
Every 2.0s: df -m; echo; uptime                        Tue Apr 19 01:19:12 2011

Filesystem           1M-blocks      Used Available Use% Mounted on
/dev/sda1                15019      3768     10489  27% /
tmpfs                      993         0       993   0% /dev/shm
/dev/sda2                 7510      1342      5787  19% /var
/dev/sda5                15019      1777     12480  13% /var/log
/dev/mapper/sda6        232003      5571    213174   3% /mnt/osd

 01:19:12 up 1 day,  2:25,  1 user,  load average: 0.91, 1.12, 1.75
===

on each node and a fair idea of what the client is doing at the same time.
The reason for using 127.0.0.1 as the target is that delta transfers and
bwlimit do not apply to local tagets, so

rsync -vva --progress --bwlimit=N newsspool /mnt/ceph/

which in principle is equivalent, would just run full speed and re-copy
everything that's not already on the target.

The load is the prime measure. When I see load 16 on a dual-CPU system,
I know that things are getting out of hand before anything even breaks.
Then I lower bwlimit and start again.

The relation between OSD partitions (/dev/mapper/sda6 in the example above)
is another interesting factor. As long as the load is under 100%, the
partitions on both nodes grow in almost perfect sync. When the load exceeds
100%, one node starts lagging behind the other. If that continues long enough,
the lagging node falls out completely while the other node keeps growing.
I've seen differences of almost 300% (11 GB on node02 and 4 GB on node01)
which won't go away until a full ceph restart followed by a few hours of
replaying patience.

Interestingly, the node with the monitor that actually receives the data
(node01) is the one unable to write them to its own disk, while the other
node (node02) grows way ahead. I'm not sure yet, but the likely cause of
this is that the network speed of node01 and the disk speed of node02 both
exceed the disk speed of node01. In any case, a load above 100% is not a
reason for concern all and by itself, but the node storages growing out of
sync is. If that goes far enough, the nodes are unable to synchronise again
without an unmount and full restart of ceph and in the worst case scenario
the data is corrupted.

Now, having run rsync for a while, I kill it and start it again. This will
cause it to compare all files that are already on the target, skip those
that have identical size and near-identical mtime, and only copy files that
are either missing or different. Assuming for example that I successfully
copied 200.000 files in the first run, the second run will compare 200.000
files and start copying again from the 200.001:st file on.

This is where I could really see the difference between atime and noatime on
the OSD underlying partition. bwlimit is meaningless on size and mtime reads,
so the client would read file attributes at full blast. With atime on the
underlying partition this caused the nodes to jump to loads of 5 or more. With
noatime, they displayed a load of 0.1 or so and the whole compare opration
completed in a small fraction of the time previously required.

Z

Re: Logging

[Sage Weil]

On Tue, 19 Apr 2011, Zenon Panoussis wrote:
> The relation between OSD partitions (/dev/mapper/sda6 in the example above)
> is another interesting factor. As long as the load is under 100%, the
> partitions on both nodes grow in almost perfect sync. When the load exceeds
> 100%, one node starts lagging behind the other. If that continues long enough,
> the lagging node falls out completely while the other node keeps growing.
> I've seen differences of almost 300% (11 GB on node02 and 4 GB on node01)
> which won't go away until a full ceph restart followed by a few hours of
> replaying patience.

This is really interesting.  This is on the partitions that have _just_ 
the OSD data?  Do you see any OSD flapping (down/up cycles) during this 
period?

It's possible that the MDS is getting ahead of the OSDs, as there isn't 
currently any throttling of metadata request processing when the 
journaling is slow.  (We should fix this.)  I don't see how that would 
explain the variance in disk usage, though, unless you are also seeing the 
difference in disk usage reflected in the cosd memory usage on the 
less-disk-used node?

> Interestingly, the node with the monitor that actually receives the data
> (node01) is the one unable to write them to its own disk, while the other
> node (node02) grows way ahead. I'm not sure yet, but the likely cause of
> this is that the network speed of node01 and the disk speed of node02 both
> exceed the disk speed of node01. In any case, a load above 100% is not a
> reason for concern all and by itself, but the node storages growing out of
> sync is. If that goes far enough, the nodes are unable to synchronise again
> without an unmount and full restart of ceph and in the worst case scenario
> the data is corrupted.

I don't think it is the monitors, although you could verify that with a du 
on the mon data directory on the two nodes. 

In any case, none of these things should be corrupting data... :/

> Now, having run rsync for a while, I kill it and start it again. This will
> cause it to compare all files that are already on the target, skip those
> that have identical size and near-identical mtime, and only copy files that
> are either missing or different. Assuming for example that I successfully
> copied 200.000 files in the first run, the second run will compare 200.000
> files and start copying again from the 200.001:st file on.
> 
> This is where I could really see the difference between atime and noatime on
> the OSD underlying partition. bwlimit is meaningless on size and mtime reads,
> so the client would read file attributes at full blast. With atime on the
> underlying partition this caused the nodes to jump to loads of 5 or more. With
> noatime, they displayed a load of 0.1 or so and the whole compare opration
> completed in a small fraction of the time previously required.

This is good to keep in mind.  If the 'btrfs devs' option is defined and 
the init script mounts the data volume for you, the default 'btrfs opts' 
mount options include noatime.  Anyone overriding that or mounting the osd 
data themselves would be wise to keep noatime in the option set.

sage

Re: Logging

[Zenon Panoussis]

On 04/19/2011 07:02 AM, Sage Weil wrote:

>> The relation between OSD partitions (/dev/mapper/sda6 in the example above)
>> is another interesting factor. As long as the load is under 100%, the
>> partitions on both nodes grow in almost perfect sync. When the load exceeds
>> 100%, one node starts lagging behind the other. If that continues long enough,
>> the lagging node falls out completely while the other node keeps growing.

> This is really interesting.  This is on the partitions that have _just_ 
> the OSD data? 

Yes, with a couple of extra layers. node01 keeps its OSD data on an ext4
filesystem on top of a dm-crypt encrypted native disk partition. node02
on the other hand has an mdadm RAID0 of two partitions on separate disks
with dm-crypt and ext4 on top of that. This layering - in particular the
encryption - consumes CPU and can slow down things, but for the rest it's
rock-solid; I've been running systems with these setups for years and
never had a problem with them even once.

Here's an example from this morning:

node01:
/dev/mapper/sda6        232003      5914    212830   3% /mnt/osd

node02:
/dev/mapper/md4         225716      5704    207112   3% /mnt/osd

client:
192.168.178.100:6789:/
                        232002      5913    212829   3% /mnt/n01

You can see that the total space on the client corresponds to that of node01,
so the osd of node02 has gone belly up. The load on node01 is creeping upwards
of 200% while rsync on the client keeps smiling and pushing data.

node01 top:
PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
24793 root      20   0 1679m 510m 1756 S  9.2 25.7  29:23.11 cosd
30235 root      20   0     0    0    0 S  1.0  0.0   0:01.10 kworker/0:1
  637 root      20   0     0    0    0 S  0.7  0.0   4:56.26 jbd2/sda2-8
30468 root      20   0 14988 1152  864 R  0.7  0.1   0:00.14 top
21748 root      20   0  104m  796  504 S  0.3  0.0   1:04.27 watch
29418 root      20   0     0    0    0 S  0.3  0.0   0:02.12 kworker/0:2

node01 iotop:
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
24933 be/4 root      109.97 K/s    7.12 K/s  0.00 % 95.49 % cosd -i 0 -c ~ceph/ceph.conf
24934 be/4 root       94.15 K/s    7.12 K/s  0.00 % 92.45 % cosd -i 0 -c ~ceph/ceph.conf
24830 be/4 root        0.00 B/s   36.39 K/s  0.00 % 81.10 % cosd -i 0 -c ~ceph/ceph.conf
  637 be/3 root        0.00 B/s    0.00 B/s  0.00 % 80.27 % [jbd2/sda2-8]
  256 be/3 root        0.00 B/s    2.37 K/s  0.00 % 72.93 % [jbd2/sda1-8]
24831 be/4 root        0.00 B/s    0.00 B/s  0.00 % 27.85 % cosd -i 0 -c ~ceph/ceph.conf
24826 be/4 root        0.00 B/s  272.94 K/s  0.00 % 19.28 % cosd -i 0 -c ~ceph/ceph.conf
24829 be/4 root        0.00 B/s   45.89 K/s  0.00 % 18.03 % cosd -i 0 -c ~ceph/ceph.conf
24632 be/4 root        0.00 B/s   26.90 K/s  0.00 %  5.99 % cmon -i 0 -c ~ceph/ceph.conf
24556 be/3 root        0.00 B/s    5.54 K/s  0.00 %  2.95 % [jbd2/dm-0-8]
  639 be/3 root        0.00 B/s    0.00 B/s  0.00 %  2.32 % [jbd2/sda5-8]
24833 be/4 root        0.00 B/s   10.28 K/s  0.00 %  0.00 % cosd -i 0 -c ~ceph/ceph.conf


At this point I unmounted ceph on the client and restarted ceph. A few minutes
later I see this:

node01:
/dev/mapper/sda6        232003      5907    212837   3% /mnt/osd

node02:
/dev/mapper/md4         225716      5626    207190   3% /mnt/osd

Note how disk usage went down on both nodes, considerably on node02.

Then they start exchanging data and an hour later or so they're back in sync:

node01:
/dev/mapper/sda6        232003      5906    212838   3% /mnt/osd

node02:
/dev/mapper/md4         225716      5906    206910   3% /mnt/osd


> Do you see any OSD flapping (down/up cycles) during this 
> period?

I've been running without logs since yesterday, but my experience is that
they don't flap; once an OSD goes down it stays down until ceph is restarted.

> It's possible that the MDS is getting ahead of the OSDs, as there isn't 
> currently any throttling of metadata request processing when the 
> journaling is slow.  (We should fix this.)  I don't see how that would 
> explain the variance in disk usage, though, unless you are also seeing the 
> difference in disk usage reflected in the cosd memory usage on the 
> less-disk-used node?

I didn't pay attention to memory usage, but I think I can rule this out
anyway. node01 has 2 GB RAM and 2 GB swap, node02 has 4 GB RAM and no
swap. Since I saw 11 GB on the node02 OSD the other day and 4 GB on the
node01 OSD, the difference could not have been in memory.

Z

Re: Logging

[Sage Weil]

> > This is really interesting.  This is on the partitions that have _just_ 
> > the OSD data? 
> 
> Yes, with a couple of extra layers. node01 keeps its OSD data on an ext4
> filesystem on top of a dm-crypt encrypted native disk partition. node02
> on the other hand has an mdadm RAID0 of two partitions on separate disks
> with dm-crypt and ext4 on top of that. This layering - in particular the
> encryption - consumes CPU and can slow down things, but for the rest it's
> rock-solid; I've been running systems with these setups for years and
> never had a problem with them even once.
> 
> Here's an example from this morning:
> 
> node01:
> /dev/mapper/sda6        232003      5914    212830   3% /mnt/osd
> 
> node02:
> /dev/mapper/md4         225716      5704    207112   3% /mnt/osd
> 
> client:
> 192.168.178.100:6789:/
>                         232002      5913    212829   3% /mnt/n01

Oh... I suspect that only one of the OSDs is active.  The ceph client's 
df/statfs result is really just a sum over the statfs results on all of 
the OSDs.  The fact that is corresponds to node01 suggests that node02 
isn't part of the cluster.  Can you post the output from

	ceph osd dump -o -

> At this point I unmounted ceph on the client and restarted ceph. A few minutes
> later I see this:
> 
> node01:
> /dev/mapper/sda6        232003      5907    212837   3% /mnt/osd
> 
> node02:
> /dev/mapper/md4         225716      5626    207190   3% /mnt/osd
> 
> Note how disk usage went down on both nodes, considerably on node02.
> 
> Then they start exchanging data and an hour later or so they're back in sync:
> 
> node01:
> /dev/mapper/sda6        232003      5906    212838   3% /mnt/osd
> 
> node02:
> /dev/mapper/md4         225716      5906    206910   3% /mnt/osd

I wouldn't read into the disk utilizations too closely.  There is logging 
going on at a couple of different levels that can make the utilization 
fluctuate depending on the timing of trimming.

> > Do you see any OSD flapping (down/up cycles) during this 
> > period?
> 
> I've been running without logs since yesterday, but my experience is that
> they don't flap; once an OSD goes down it stays down until ceph is restarted.

Also, one thing you should do during these tests is keep a 

	ceph -w

running to monitor changes in the cluster state (to see, for example, if 
it's marking either osd down).

Thanks!
sage

Re: Logging

[Luis Ressel]

On Wed, Mar 18, 2020 at 09:14:42AM +0100, J.R. Oldroyd wrote:
> First, I should point out that the whole purpose of syslog(3) is
> to do the flexible directing of different daemons' logs to different
> places, including in chroots.

By design, syslog funnels all logs through a single socket. Separating
them again requires matching the contents of log messages, which is
inefficient and error-prone. Getting syslog to work in chroots can be
annoying, since it requires opening the logging socket before chrooting
(which requires support by the daemon), or providing a /dev/log socket
inside the chroot.

That said, I'm aware that syslog is more convenient in some setups, so
offering both stderr and syslog logging sounds reasonable to me.

Cheers,
Luis

Re: Logging

[J.R. Oldroyd]

On Tue, 17 Mar 2020 18:12:05 +0000 Luis Ressel <aranea@aixah.de> wrote:
>
> If you're adding logging support, I'd prefer logs on stderr over a
> centralized legacy mechanism such as syslog. That's much more flexible;
> in particular, it makes it much easier to direct logs of different
> daemons to different places, or run daemons in chroots.
> 

First, I should point out that the whole purpose of syslog(3) is
to do the flexible directing of different daemons' logs to different
places, including in chroots.

That said, adding logging to stderr is also very trivial, given the
current code.  I have updated the two files on my website [1] to include
this.  Just set environment variable WG_LOG_DEST to "stderr" and then
start wireguard-go.

Please test.  If this works as desired, I will update the git patch
that I sent in yesterday.

	-jr

[1] optional syslog logging for wireguard-go
    http://opal.com/jr/wireguard/logger.go
    http://opal.com/jr/wireguard/logger_syslog.go
	(both files are needed)

Re: Logging

[Luis Ressel]

On Tue, Mar 17, 2020 at 08:37:17AM +0100, J.R. Oldroyd wrote:
> Since adding syslog support is so trivial, given the existing code
> is already designed around logging levels and given Go's clean support
> of syslog, why not just build it in so that wireguard's logging is done
> consistently with all other UNIX daemons?  Piping stdout to logger
> is non-standard and also loses the ability to filter the different
> log levels to different log destinations.

If you're adding logging support, I'd prefer logs on stderr over a
centralized legacy mechanism such as syslog. That's much more flexible;
in particular, it makes it much easier to direct logs of different
daemons to different places, or run daemons in chroots.

Cheers,
Luis

Re: Logging

[J.R. Oldroyd]

On Mon, 16 Mar 2020 13:30:17 -0600 "Jason A. Donenfeld" <Jason@zx2c4.com> wrote:
>
> Adding direct syslog support might make sense. I'll look into
> integrating those files you sent, though, perhaps it'd be better if
> you submitted those as a patch to the mailing list with a proper
> Signed-off-by line? (Or even to Github?)
>
Will do.
 
> I'm curious to know: is there a reason why you prefer this to something like:
> 
> `LOG_LEVEL=debug wireguard-go -f wg0 2>&1 | logger &`
> 
Since adding syslog support is so trivial, given the existing code
is already designed around logging levels and given Go's clean support
of syslog, why not just build it in so that wireguard's logging is done
consistently with all other UNIX daemons?  Piping stdout to logger
is non-standard and also loses the ability to filter the different
log levels to different log destinations.

	-jr

Re: Logging

[Jason A. Donenfeld]

Hi JR,

Adding direct syslog support might make sense. I'll look into
integrating those files you sent, though, perhaps it'd be better if
you submitted those as a patch to the mailing list with a proper
Signed-off-by line? (Or even to Github?)

I'm curious to know: is there a reason why you prefer this to something like:

`LOG_LEVEL=debug wireguard-go -f wg0 2>&1 | logger &`

Jason

Re: Logging

[Arti Zirk]

On P, 2020-03-15 at 14:16 +0100, J.R. Oldroyd wrote:
> New here.  Apologies if I am re-hashing something discussed before.
> I did read back a few months of this list and didn't see any relevant
> discussion.

Quite a lot of information can also be obtained via Linux Wireguard
module. wg(8) man page has a section on that. 

https://git.zx2c4.com/wireguard-tools/about/src/man/wg.8#DEBUGGING%20INFORMATION

Logging

[J.R. Oldroyd]

Hi all,

New here.  Apologies if I am re-hashing something discussed before.
I did read back a few months of this list and didn't see any relevant
discussion.

Unlike many here who are providing anonymous VPN services and who
don't want logging at all, I am helping set up Wireguard in a corporate
VPN environment.  The logging requirements here are very different.

Specifically, there is a need for permanent logs.  And, the logs should
ideally include:

	- when a client connects
	- when a client disconnects
	- ideally also how much data was transferred in and out
	  during a session

So this is precisely the opposite logging requirement from those
who are managing anonymous VPNs.  That's understood, and my proposed
changes maintain current default no-logging behavior for those that
need no logging.

I have made replacements [1] for device/logger.go that allow syslog
to be used for logging if available.

There are two files, both are needed.  There are OS-dependent
compilation directives so that the syslog version is not used on
Windows or Plan9, which do not have syslog.

In both, the behavior is exactly the same as now by default.  If no
other config is used, logging is still at info level to stdout as is
current practice.  I.e., you can drop these in and nothing will change
for current users, even on systems that support syslog.

To use the new syslog logging, environment variables must be set.
Logging level and logging destination are controlled by the existing
variable LOG_LEVEL and the new WG_LOG_DEST and WG_LOG_FACILITY:

	LOG_LEVEL	"debug", "info", "error", "silent"
				(default is still "info")

	WG_LOG_DEST	"stdout", "syslog"
				(default is "stdout")

	WG_LOG_FACILITY
			any syslog facility, e.g., "daemon", "local0",
			"local1", etc (also "log_", prefix OK and
			either lower- or upper-case)
				(default is "daemon")

Note that when using syslog, your syslog.conf needs to be configured
to send messages from your chosen facility.level to somewhere useful.

Also, it's worth saying that wireguard-go's logging includes some
UTF-8 characters.  Certain OSs' syslogd don't handle 8-bit data
very well.  E.g., FreeBSD.  A patch for FreeBSD's syslogd is at [2].

Since this is backwards compatible, it would be great to see this
logger.go and logger_syslog.go replace the current logger.go.

I have also been playing with some patches to add the session start and
end log messages.  I realize that this is a stateless protocol and that
the idea of a session isn't really there.  While my current placement of
these session log messages in receive.go and in timers.go is close, it
isn't perfect.  Right now I am logging "session start" in the initial
handshake code and "session end" in the handshake timeout code.  Neither
are perfect but, as I said, close.  I need to look more at the peer
state information that is currently maintained in order to see if
there's a better place to put these log messages.  I realize also that
these session log messages must not be logged for those who don't want
any logging.  These session log patches are NOT part of the current
proposed logger.go changes.  If anyone wants to see these, I'll send
them along separately.

	-jr

[1] optional syslog logging for wireguard-go
    http://opal.com/jr/wireguard/logger.go
    http://opal.com/jr/wireguard/logger_syslog.go
	(both files are needed)

[2] Patch for FreeBSD's syslogd to support UTF-8 chars in messages
    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244226

Re: Logging

[Grant Taylor]

Svenne Krap wrote:
> Hi.
> 
> I am currently working on a not so simple firewall setup on a modern 
> machine (Xeon, Gigs of memory, SCSI subsystem).
> 
> As part of it, I would like to know various "event" statistics.Questions 
> I would like to answer is "How many hits on port 1433 have i got, and 
> how is that distributed amongst the machines". Think pivot table data.
> 
> Is there some way to get netfilter to collect rule hits (like with no -j 
> clause) for a each port/ip-address individually within a range ?
> Other than creating thousands of lines of rules and add them to my 
> "firewall-startup" script (which is currently slightly less than 80 rules).
> 
> I have thought of just logging all traffic and running it through a 
> userspace program via syslog-ng, but frankly I worry about performance 
> (the firewall should be able to filter at least the 100Mbps connection, 
> it currently sits on) under flooding.
> 
> Your thoughs are apprieciated :)
> 
> Svenne
> 


You might want to take a look at the ACCOUNT match (http://www.netfilter.org/patch-o-matic/pom-extra.html#pom-extra-ACCOUNT).



Grant. . . .

Re: Logging

[Chris Brenton]

On Thu, 2005-08-11 at 11:49, Svenne Krap wrote:
>
> Is there some way to get netfilter to collect rule hits (like with no -j 
> clause) for a each port/ip-address individually within a range ?
> Other than creating thousands of lines of rules and add them to my 
> "firewall-startup" script (which is currently slightly less than 80 rules).

Have LogWatch process the /var/log/message file and produce a medium
level detail report. You'll get output similar to the following:

Dropped 603 packets on interface eth3
  From 4.78.20.2 - 12 packets to udp(53)
  From 12.120.1.21 - 10 packets to
tcp(4355,10045,12579,17520,18552,36906,53249,54319,58702,62703)
  From 12.120.1.22 - 9 packets to
tcp(4063,11107,13063,30538,37001,40758,45575,48153,57370)
  From 12.130.62.16 - 24 packets to udp(53)
  From 60.26.129.15 - 8 packets to
tcp(5554,9898,5554,9898,5554,9898,5554,9898)
  From 61.152.167.59 - 4 packets to tcp(22,22,22,22)
  From 61.221.58.212 - 4 packets to tcp(22,22,22,22)
  From 62.105.6.52 - 1 packet to icmp(0)

You can then further parse it as needed. You don't need a unique log
rule for each port and/or IP. LogWatch will sort it all out for you. 

HTH,
Chris

Logging

[Svenne Krap]

Hi.

I am currently working on a not so simple firewall setup on a modern 
machine (Xeon, Gigs of memory, SCSI subsystem).

As part of it, I would like to know various "event" statistics.Questions 
I would like to answer is "How many hits on port 1433 have i got, and 
how is that distributed amongst the machines". Think pivot table data.

Is there some way to get netfilter to collect rule hits (like with no -j 
clause) for a each port/ip-address individually within a range ?
Other than creating thousands of lines of rules and add them to my 
"firewall-startup" script (which is currently slightly less than 80 rules).

I have thought of just logging all traffic and running it through a 
userspace program via syslog-ng, but frankly I worry about performance 
(the firewall should be able to filter at least the 100Mbps connection, 
it currently sits on) under flooding.

Your thoughs are apprieciated :)

Svenne

Re: logging

[Antony Stone]

On Sunday 25 April 2004 4:32 pm, IT Clown wrote:

> Hi All
>
> Does any one know what the local0 - local7 ( locally
> defined levels ) would log in syslog.

The name is a bit of a clue: "locally defined levels" are locally defined - 
ie: you can use them on your system for whatever you want - there's no 
convention or standard way of using these - they're provided so that you can 
do things with them which aren't convenient to do using the other pre-defined 
levels.

Regards,

Antony.

-- 
90% of networking problems are routing problems.
9 of the remaining 10% are routing problems in the other direction.
The remaining 1% might be something else, but check the routing anyway.

                                                     Please reply to the list;
                                                           please don't CC me.

logging

[IT Clown]

Hi All
                                                            
                                                            
   
Does any one know what the local0 - local7 ( locally
defined levels ) would log in syslog. I know that when i
use the following:
daemon.*
local2.*
                                                            
                                                            
   
i log ppp. What do the rest log ( local0 - local7 )?
                                                            
                                                            
   
Regards
______________________________________________________________
Herbalife Independent Distributor http://www.healthiest.co.za

logging

[ip tables]

hello all,

I need to get a hint on how to log traffic on a
firewall using iptables. I have setup some rules but i
don't know how to log certain things such as port scan
attempts or other break-in attempts. Do i need to have
an IDS for it?

Thanks

__________________________________
Do you Yahoo!?
Yahoo! Finance Tax Center - File online. File on time.
http://taxes.yahoo.com/filing.html

Re: logging

[D. Prima Prayudi]

If you have difficulties with that I suggest you use ulogd daemon since
with syslogd the log from the kernel and iptables will be mixed-up.

Regards, D. Prima Prayudi

IPv6-enabled
ARC-ITB 2001:d30:3:160::2/64

On Thu, 1 Apr 2004, IT Clown wrote:

> Hi
>
> Thanks i googled abit and found something else but its
> logging to terminal and to a log file how do i get rid of
> the terminal logging? Apparently you have to remove
> anything like: /dev/console, "root" or '8' in syslog.conf
> and there is nothing like that in it.I added the following
> to syslog.conf:
>
> kern.* /var/log/iptables
>
> Regards
> __________________________________________________________________________
> http://www.webmail.co.za/dialup Webmail ISP - Cool Connection, Cool Price
>
>

logging

[IT Clown]

Hi

Thanks i googled abit and found something else but its
logging to terminal and to a log file how do i get rid of
the terminal logging? Apparently you have to remove
anything like: /dev/console, "root" or '8' in syslog.conf
and there is nothing like that in it.I added the following
to syslog.conf:

kern.* /var/log/iptables

Regards
__________________________________________________________________________
http://www.webmail.co.za/dialup Webmail ISP - Cool Connection, Cool Price

Re: logging

[Mark Page]

C'mon Clown, you can do it ;-) - there is so much stuff out on the net
regarding  iptables/netfilter - I presume you have a browser?

This is a really general question which you could get answered in ten
minutes by reading a few articles.

try this one: 

https://lists.balabit.hu/pipermail/syslog-ng/2003-April/004990.html

On Wed, 2004-03-31 at 10:18, IT Clown wrote: 
> Hi
> 
> How do you log your iptables activity to a file because all
> the activity is being displayed in my terminal while im
> busy working.I would like to log it to
> /var/log/iptables.
>  Do i need to configure syslog.conf? how would i need to
> configure it?
> 
> Regards
> __________________________________________________________________________
> http://www.webmail.co.za/dialup Webmail ISP - Cool Connection, Cool Price
> 

logging

[IT Clown]

Hi

How do you log your iptables activity to a file because all
the activity is being displayed in my terminal while im
busy working.I would like to log it to
/var/log/iptables.
 Do i need to configure syslog.conf? how would i need to
configure it?

Regards
__________________________________________________________________________
http://www.webmail.co.za/dialup Webmail ISP - Cool Connection, Cool Price

Re: Logging

[forum]

> I have done some googling however did not manage to find out if there is
> a way to define a file where iptables writes its logs.

In UNIX, the 'syslog' interface uses two parameters for any information 
logged:

The logging facility
The logging level

I believe the facility is fixed to kernel, but the level can be changed. 
Corresponding to that is the configuration in syslog.conf

Another useful thing is --log-prefix in the iptables command, which lets 
you output a descriptive log prefix. You can then script your own log 
placement, using grep or something.

-- 
forum@users.pc9.org

Re: Logging

[Frank Gruellich]

* Mario Udina <m.udina@libero.it> 15. Mar 04:
> Hello!

Hi,

> As I read in the man the file where the logs are written is the kernel
> log i.e. /var/log/messages

Maybe, by default.

> is there a way to send this output to another file in order to keep the
> logs clean?

This is not possible with iptables or netfilter.  You want to read the
man page of your sysklogd, especially of syslog.conf.  You can specify
priority in a rule.   Maybe you want to use another syslogger (syslog-ng
provides file redirect on patterns, AFAIK, would be usefull with the
--log-prefix option.).

HTH,
 regards, Frank.
-- 
Sigmentation fault

Re: Logging

[Antony Stone]

On Monday 15 March 2004 3:51 pm, Mario Udina wrote:

> Hello!
>
> I have done some googling however did not manage to find out if there
> is a way to define a file where iptables writes its logs.
>
> As I read in the man the file where the logs are written is the kernel
> log i.e. /var/log/messages
>
> is there a way to send this output to another file in order to keep the
> logs clean?

netfilter logs via syslogd, therefore by changing the --log-level option to 
the LOG target you can get messages sent to some other file as specified in 
your syslog.conf (however you may not be able to keep them *completely* 
separate from other messages, depending on how you arrange syslog.conf).

If you want to change the syslog facility as well as the level, which is the 
ideal solution, you'll have to hack the netfilter logging source code :(

Antony.

-- 
The first fifty percent of an engineering project takes ninety percent of the 
time, and the remaining fifty percent takes another ninety percent of the 
time.

                                                     Please reply to the list;
                                                           please don't CC me.

Re: Logging

[Frederic de Villamil]

On Mon, 15 Mar 2004, Mario Udina wrote:

> Hello!
>
> I have done some googling however did not manage to find out if there
> is a way to define a file where iptables writes its logs.
>
> As I read in the man the file where the logs are written is the kernel
> log i.e. /var/log/messages
>
> is there a way to send this output to another file in order to keep the
> logs clean?
>
> regards,

Hi Mario,
maybe what you're looking for is in /etc/syslog.conf.

--
< Ylli> lol je rigole neuro jte prend pa pr un pervers ms un president
et pere de famille respectable :s
http://www.seclab.jp

RE: Logging

[Hurley, Michael]

Try something like this:

set logging to, for example,
iptables -A FORWARD -j LOG --log-level 7 --log-prefix "FORWARD: "

Then in /etc/syslog.conf:
kern.7	-/var/log/firewall



-----Original Message-----
From: Mario Udina [mailto:m.udina@libero.it]
Sent: Monday, March 15, 2004 10:51 AM
To: netfilter@lists.netfilter.org
Subject: Logging


Hello!

I have done some googling however did not manage to find out if there
is a way to define a file where iptables writes its logs.

As I read in the man the file where the logs are written is the kernel
log i.e. /var/log/messages

is there a way to send this output to another file in order to keep the
logs clean?

regards,
	Mario Udina
-- 
i would like to die as my gradfather did, quietly in his sleep
and not in panic and disbelief as the passengers of his bus.

Logging

[Mario Udina]

Hello!

I have done some googling however did not manage to find out if there
is a way to define a file where iptables writes its logs.

As I read in the man the file where the logs are written is the kernel
log i.e. /var/log/messages

is there a way to send this output to another file in order to keep the
logs clean?

regards,
	Mario Udina
-- 
i would like to die as my gradfather did, quietly in his sleep
and not in panic and disbelief as the passengers of his bus.

Re: logging

[Ray Olszewski]

At 05:43 PM 12/29/2003 -0500, John T. Williams wrote:
>I was wondering if there is a good method for logging all attempts to
>connect to a port on my computer.  Basically, I was looking for
>something that logged the port and ip and the destination port of
>attempted connections.
>
>I'm running Mandrake 9.1

Presumably a 2.4.x kernel then.

iptables itself (the kernel's built-in firewalling) will do this; LOG is a 
predefined target that does not terminate (in the way that ACCEPT, REJECT, 
and DENY do) but does log each matching packet before passing it on to the 
next rule in the chain.

The man page for iptables will at least get you started. If you need help 
with specifics after reading it, don't hesitate to ask here.





-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: logging

[caszonyi]

On Tue, 29 Dec 2003, John T. Williams wrote:

> I was wondering if there is a good method for logging all attempts to
> connect to a port on my computer.  Basically, I was looking for
> something that logged the port and ip and the destination port of
> attempted connections.
>
> I'm running Mandrake 9.1
>

iptables has a logging option but I can't tell you more.
http://lartc.org and look for HOWTO on that page

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

logging

[John T. Williams]

I was wondering if there is a good method for logging all attempts to
connect to a port on my computer.  Basically, I was looking for
something that logged the port and ip and the destination port of
attempted connections.

I'm running Mandrake 9.1




-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

logging

[Philippe Dhont (Sea-ro)]

hi all,

i use fwbuilder to configure a firewall script
This seems to work but all my logging comes into my command prompt and in my
/var/log/messages
Now, i only want them in my /var/log/messages and NOT on my screen (very
annoying)
what do i have to do for that ?

regards,

Philippe Dhont 

Re: Logging

[Dharmendra.T]

On Tue, 2003-01-14 at 23:53, Subba Rao wrote:
> Hi
> 
> I have the following rules to allow traffic on SMTP port.
> 
> --------------------
> iptables -A INPUT -i $EXTERNAL_IF -p tcp --destination-port 25 -j LOG \
>                 --log-level 4 --log-prefix "Incoming Mail Traffic "
> iptables -A INPUT -i $EXTERNAL_IF -p tcp --destination-port 25 -j ACCEPT
> --------------------
> 
> The default policy is on the INPUT chain is to drop the packets.  How do I capture
> what is being dropped?
> 
> My goal is to log the inbound traffic and my syslog has the following
> configuration:
> 
> --------------------
> *.=info;*.=notice;mail.none                     /usr/adm/messages
> *.=alert                                        /usr/adm/messages
> *.=crit                                         /usr/adm/debug
> mail.*                                          /var/log/mail-log
> kern.*                                          /var/log/messages
> user.*                                          /var/log/messages
> syslog.*                                        /var/log/messages
> auth.*                                          /var/log/messages
> authpriv.*                                      /var/log/messages
> --------------------
> 
> Is this configuration sufficient to capture the inbound connections?
> 
> -- 
> 
> Subba Rao
> subba9@cablespeed.com
> 

Hi Rao,

 Just add a rule after the default policy. This should give you all the
packets which are dropped.

iptables -A INPUT -j LOG
iptables -A OUTPUT -j LOG
iptables -A FORWARD -j LOG
-- 
Dharmendra.T
Linux Enthu

Re: Logging

[Athan]

[-- Attachment #1: Type: text/plain, Size: 1634 bytes --]

On Tue, Jan 14, 2003 at 01:23:10PM -0500, Subba Rao wrote:
> --------------------
> iptables -A INPUT -i $EXTERNAL_IF -p tcp --destination-port 25 -j LOG \
>                 --log-level 4 --log-prefix "Incoming Mail Traffic "
> iptables -A INPUT -i $EXTERNAL_IF -p tcp --destination-port 25 -j ACCEPT
> --------------------
> 
> The default policy is on the INPUT chain is to drop the packets.  How do I capture
> what is being dropped?

  iptables -A INPUT -i $EXTERNAL_IF -j LOG --log-level 4 \
  	--log-prefix "Dropped Incoming "

Remember a LOG target simply LOGs then returns, so it'll still drop
through to the policy after this.

> My goal is to log the inbound traffic and my syslog has the following
> configuration:
> 
> --------------------
> *.=info;*.=notice;mail.none                     /usr/adm/messages
> *.=alert                                        /usr/adm/messages
> *.=crit                                         /usr/adm/debug
> mail.*                                          /var/log/mail-log
> kern.*                                          /var/log/messages

   All iptables -j LOG goes to kern.*, so you should see it in your
/var/log/messages.  You could also use --log-level 7 (DEBUG) instead
and:

kern.=debug			/var/log/iptables

Note you _CAN_ also do --log-level debug to make things clearer.

HTH,

-Ath
-- 
- Athanasius = Athanasius(at)miggy.org / http://www.miggy.org/
                  Finger athan(at)fysh.org for PGP key
	   "And it's me who is my enemy. Me who beats me up.
Me who makes the monsters. Me who strips my confidence." Paula Cole - ME

[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Logging

[Subba Rao]

Hi

I have the following rules to allow traffic on SMTP port.

--------------------
iptables -A INPUT -i $EXTERNAL_IF -p tcp --destination-port 25 -j LOG \
                --log-level 4 --log-prefix "Incoming Mail Traffic "
iptables -A INPUT -i $EXTERNAL_IF -p tcp --destination-port 25 -j ACCEPT
--------------------

The default policy is on the INPUT chain is to drop the packets.  How do I capture
what is being dropped?

My goal is to log the inbound traffic and my syslog has the following
configuration:

--------------------
*.=info;*.=notice;mail.none                     /usr/adm/messages
*.=alert                                        /usr/adm/messages
*.=crit                                         /usr/adm/debug
mail.*                                          /var/log/mail-log
kern.*                                          /var/log/messages
user.*                                          /var/log/messages
syslog.*                                        /var/log/messages
auth.*                                          /var/log/messages
authpriv.*                                      /var/log/messages
--------------------

Is this configuration sufficient to capture the inbound connections?

-- 

Subba Rao
subba9@cablespeed.com

Re: Logging

[Antony Stone]

On Monday 21 October 2002 10:04 pm, Tasha Smith wrote:

> I was wandering if you can help me with my logging rules. Ill give you my
> rules exactly what they look like now (without typos) :)  And i scanned it
> with nmap from a machine NOT ROUTED by the firewall machine! And i get no
> LOG file.

No typos, heh :-) ?

Anyway, can you explain those last two comments - where are you scanning from 
(I don't understand your routing comment) - and "you get no log file" - do 
you mean you never get any logs at all, or you get log entries on the screen 
but not in a file, or you get logs sometimes, but not when you're doing a 
particular type of scan ?

> I even added this to my syslog.conf file......
>
>  kern.warn                            /var/log/fwlog

Does this successfully log anything at all ?  I mean, if you insert a rule 
right at the start of your INPUT chain:
iptables -I INPUT -j LOG --log-prefix "fwlog: "

Does anything go into /var/log/fwlog ?

(I would expect you to have to add the option "--log-level=warn" to match the 
entry in your syslog.conf file.)

>  How can i get this machine to log STEALTH port scans and stuff???

Explain what you mean by a Stealth port scan ?   If yu;re using nmap, what 
options are you using ?

> iptables           --flush
> iptables -t -nat   --flush
> iptables -t mangle --flush
>
> iptables -A INPUT  -i lo -j ACCEPT
> iptables -A OUTPUT -o lo -j ACCEPT
>
> iptables --policy INPUT DROP
> iptables --policy FORWARD DROP
> iptables --policy OUTPUT ACCEPT
>
> iptables -A INPUT      -p tcp --tcp-flags ALL NONE -j DROP
> iptables -A FORWARD    -p tcp --tcp-flags ALL NONE -j DROP

Do you get the same result from your port scan (and your logs) if you remove 
these two rules ?

> iptables -A INPUT    -m state --state ESTABLISHED,RELATED -j ACCEPT
> iptables -A OUTPUT   -m state --state ESTABLISHED,RELATED -j ACCEPT
> iptables -A FOWWARD  -m state --state ESTABLISHED,RELATED -j ACCEPT
>
> iptables -A INPUT -i eth0 -p udp \
>          -s ISP.DHCP  --sport 67 \
>          --dport 68 -j ACCEPT
> iptables -A OUTPUT -o eth0 -p udp \
>          -s eth0 --sport 68 \
>          -d ISP.DHCP --dport 67 -j ACCEPT
>
> iptables -A FORWARD -i eth1 -o eth0 -s 192.168.0.0/24 -j ACCEPT
>
> iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
>
> iptables -A INPUT -i eth0 -p tcp \
>          --dport 22,25,111,1024,1025 -j LOG --log-prefix "Log-test: "

Okay, so this LOGging rule is last in your INPUT chain, just before the 
default DROP policy.

I assume you are scanning the Firewall address itself ?

By the way, what result do you get from the scan ?   Does it suggest you have 
closed ports, open ones, nothing accessible, what ?

What happens if you simply ssh to the Firewall, or telnet to port 25 ?   Do 
you see a log entry then ?

Antony.

-- 

Having been asked to provide a reference for this man,
I can confidently state that you will be very lucky indeed
if you can get him to work for you.

Re: Logging

[Richard Adams]

On Wednesday 10 April 2002 06:36, Chris Rose wrote:

> >Why would one want to ue a fancy program to read a simple text file, the
> >program less allows the use of a search engine, hit the '/' key and a
> > slash will appier at the bottom left of the screen, type a word to look
> > for, then hit enter, to repeat hit the slash again and press enter or
> > simply hit the space bar, the found word is highlighted.
>
> i think you're missing the point - what i'm looking for is not, per se, the
> mechanism used to read the log, so much as i'm looking for the means to
> make sense of what i'm seeing, and also which log files/settings of the
> daemon will provide me with stats on uploads/downloads from my ftp site.

I dont think i am, i meant what i said, what i can tell you futher is that 
proftp.log does not really reveal much infomation at all, its "xferlog" in 
/var/log that tells all, 'man xferlog' will explain all there is to be known.

Just in case you may read my mail as being prudent here is what profftp log 
shows.
Aug 05 15:16:02 unix.pa3gcu proftpd[3839]

Date and time i am sure we all now what that is, unix.pa3gcu = the server 
name, proftpd[389] is the process number as would be shown by 'ps ax' when 
the connection was presant. Further on the same line in the log one would see;

unix.pa3gcu.ampr.org (192.168.1.160[192.168.1.160]): USER pa3gcu: Login 
successful

unix.pa3gcu.ampr.org is the fullservername (192.168.1.160[192.168.1.160]): is 
the IP# of the remote computer who just opened the ftp connect,
USER: pa3gcu = the user who logged in, that could be ftp or guest when 
anoymous is used, Login succesful means a valid passwd was used.
Any discrepancy's would be logged as well as the last entry on the line.
No more info is given, you now would refer to xferlog to get details of what 
was done in that connection.
See 'man xferlog' to get all the details on those entries.

I trust i have not missed the point in your eyes this time...

>
>
> Chris Rose
> ==========
> 1952 - 2001 = 42

-- 
Regards Richard
pa3gcu@zeelandnet.nl
http://people.zeelandnet.nl/pa3gcu/

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: Logging

[Chris Rose]

At 06:08 AM 10/04/2002 +0000, Richard Adams wrote:
>On Wednesday 10 April 2002 05:46, Chris Rose wrote:
> > Can anyone on the list suggest some tools for log reading?  i'm trying to
> > audit my ftp server (proftpd) and i have no idea how to configure its
> > logging, nor how best to view what it puts out.  i have Webmin, but the
> > logging interface in there is pretty unclear to someone who's got nothing
> > to start from, knowledge-wise.
>
>Why would one want to ue a fancy program to read a simple text file, the
>program less allows the use of a search engine, hit the '/' key and a slash
>will appier at the bottom left of the screen, type a word to look for, then
>hit enter, to repeat hit the slash again and press enter or simply hit the
>space bar, the found word is highlighted.

i think you're missing the point - what i'm looking for is not, per se, the 
mechanism used to read the log, so much as i'm looking for the means to 
make sense of what i'm seeing, and also which log files/settings of the 
daemon will provide me with stats on uploads/downloads from my ftp site.


Chris Rose
==========
1952 - 2001 = 42


-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: Logging

[Richard Adams]

On Wednesday 10 April 2002 05:46, Chris Rose wrote:
> Can anyone on the list suggest some tools for log reading?  i'm trying to
> audit my ftp server (proftpd) and i have no idea how to configure its
> logging, nor how best to view what it puts out.  i have Webmin, but the
> logging interface in there is pretty unclear to someone who's got nothing
> to start from, knowledge-wise.

Why would one want to ue a fancy program to read a simple text file, the 
program less allows the use of a search engine, hit the '/' key and a slash 
will appier at the bottom left of the screen, type a word to look for, then 
hit enter, to repeat hit the slash again and press enter or simply hit the 
space bar, the found word is highlighted.

>
> Chris Rose
> ==========
> 1952 - 2001 = 42

-- 
Regards Richard
pa3gcu@zeelandnet.nl
http://people.zeelandnet.nl/pa3gcu/

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Logging

[Chris Rose]

Can anyone on the list suggest some tools for log reading?  i'm trying to 
audit my ftp server (proftpd) and i have no idea how to configure its 
logging, nor how best to view what it puts out.  i have Webmin, but the 
logging interface in there is pretty unclear to someone who's got nothing 
to start from, knowledge-wise.


Chris Rose
==========
1952 - 2001 = 42


-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs