ulogd extension

ulogd extension

[Fred Leeflang]

Hi,

For vuurmuur I would like to write an ulogd extension that writes out
a log somewhat comparable to syslog. Currently vuurmuur parses syslog
line generated with the LOG target, we are considering changing this
to having ulogd do this for us through the ULOG target and said
extension. What's the best way to go about doing this to keep the
extension maintainable?

-Fred

Re: ulogd extension

[Pierre Chifflier]

On Fri, Oct 30, 2009 at 09:17:26PM +0100, Fred Leeflang wrote:
> Hi,
> 
> For vuurmuur I would like to write an ulogd extension that writes out
> a log somewhat comparable to syslog. Currently vuurmuur parses syslog
> line generated with the LOG target, we are considering changing this
> to having ulogd do this for us through the ULOG target and said
> extension. What's the best way to go about doing this to keep the
> extension maintainable?
> 

Hi Fred,

Have you looked at the SYSLOG or LOGEMU output plugins ? Writing a
similar plugin should be fairly easy.
Writing external plugins is not supported. It's very easy, yet Pablo
prefers [1] to keep things in mainline when possible, which is of course
a good thing [2].

Pierre

[1] http://marc.info/?t=123996560400005&r=1&w=2
[2] external plugins would require us to create, document, and
maintain a stable API, for ex. Not that it can't be done, but I'm not
sure it was part of the original plans :)

Re: ulogd extension

[Fred Leeflang]

2009/10/30 Pierre Chifflier <chifflier@inl.fr>:
> On Fri, Oct 30, 2009 at 09:17:26PM +0100, Fred Leeflang wrote:
>> Hi,
>>
>> For vuurmuur I would like to write an ulogd extension that writes out
>> a log somewhat comparable to syslog. Currently vuurmuur parses syslog
>> line generated with the LOG target, we are considering changing this
>> to having ulogd do this for us through the ULOG target and said
>> extension. What's the best way to go about doing this to keep the
>> extension maintainable?
>>
>
> Hi Fred,
>
> Have you looked at the SYSLOG or LOGEMU output plugins ? Writing a
> similar plugin should be fairly easy.
> Writing external plugins is not supported. It's very easy, yet Pablo
> prefers [1] to keep things in mainline when possible, which is of course
> a good thing [2].
>
> Pierre
>
> [1] http://marc.info/?t=123996560400005&r=1&w=2
> [2] external plugins would require us to create, document, and
> maintain a stable API, for ex. Not that it can't be done, but I'm not
> sure it was part of the original plans :)
>



Hi Pierre,

Yes I was looking at those although I looked at the ulogd 1 source and
only very briefly at ulogd 2, looking at those actually gave somebody
else in #vuurmuur the idea to check into this way. It's far easier for
us to make such an extension and require ulogd to run and parse
packets for us than it is to parse a syslog text file. So if I were to
write an extension that would do what we want it to do, could that
extension be part of the builltin extensions of ulogd[1|2] ? I'm more
than happy to hack something together and let the list have a look at
it.

-Fred

Re: ulogd extension

[Pierre Chifflier]

On Sat, Oct 31, 2009 at 12:11:05AM +0100, Fred Leeflang wrote:
> Yes I was looking at those although I looked at the ulogd 1 source and
> only very briefly at ulogd 2, looking at those actually gave somebody
> else in #vuurmuur the idea to check into this way. It's far easier for

Hi Fred,

I'd suggest to look only for ulogd2 if possible. The architecture is
very different, and I think adding a plugin will be far easier.

> us to make such an extension and require ulogd to run and parse
> packets for us than it is to parse a syslog text file. So if I were to
> write an extension that would do what we want it to do, could that
> extension be part of the builltin extensions of ulogd[1|2] ? I'm more
> than happy to hack something together and let the list have a look at
> it.

If the plugin is generic, it should not be a problem. Ulogd2 has 3 types
of plugins: input, filter, and output. So, from what you are telling,
I suppose it would be an input plugin ?

BTW, I don't know if it can help you, but I have written an input plugin
(UNIXSOCK) which allows to send data to ulogd2 through a unix socket,
with a script to read pcap files.
I was about to send the patches.

Regards,
Pierre

Re: ulogd extension

[Fred Leeflang]

2009/11/1 Pierre Chifflier <pchifflier@edenwall.com>:
> On Sat, Oct 31, 2009 at 12:11:05AM +0100, Fred Leeflang wrote:
>> Yes I was looking at those although I looked at the ulogd 1 source and
>> only very briefly at ulogd 2, looking at those actually gave somebody
>> else in #vuurmuur the idea to check into this way. It's far easier for
>
> Hi Fred,
>
> I'd suggest to look only for ulogd2 if possible. The architecture is
> very different, and I think adding a plugin will be far easier.
>

I had already decided to do so and checked out
git://git.netfilter.org/ulogd2.git (couldn't get to your personal git
repo at the time) so I have a pretty decent idea what I'm looking at
already.

>> us to make such an extension and require ulogd to run and parse
>> packets for us than it is to parse a syslog text file. So if I were to
>> write an extension that would do what we want it to do, could that
>> extension be part of the builltin extensions of ulogd[1|2] ? I'm more
>> than happy to hack something together and let the list have a look at
>> it.
>
> If the plugin is generic, it should not be a problem. Ulogd2 has 3 types
> of plugins: input, filter, and output. So, from what you are telling,
> I suppose it would be an input plugin ?
>

I suppose it would be an output plugin :) It should essentially do
similar stuff from what I could see, kinda like the
ulogd_output_SYSLOG.c. It would not do syslogging however but instead
write out a logfile for vuurmuur UI displaying.

I just discussed that this morning with the other developers and the
project leader feels that we might be better off using
libnetfilter_log directly from libvuurmuur as there's apparently a
bigger chunk of library code in libvuurmuur that would have to be
either ported to the plugin or we'd create a dependency on libvuurmuur
in ulogd2, which he was pretty sure you wouldn't like :)

So for now we'll stick to the approach of using libnetfilter_log from
within our own library.

Thanks,
-Fred Leeflang

Re: ulogd extension

[Fred Leeflang]

Pierre,

I've hacked support for creating nflog targets into vuurmuur. When experimenting
with this I've found that it was real easy to have ulogd2's syslogemu
output module write a syslog file that vuurmuur could use to do our
syslog parsing. This may be
more useful even than writing our own extension for ulogd2 as our project leader
decided not to want to obsolete the log parsing daemon.

So, we could simply add a requirement to vuurmuur (or at least a recommended)
of ulogd. Does ulogd1 also support nflog tgts? If not, when's ulogd2
scheduled to be
released? Are any distro packagers working on it already?

Thanks,
Fred


2009/11/1 Fred Leeflang <fredl@dutchie.org>:
> 2009/11/1 Pierre Chifflier <pchifflier@edenwall.com>:
>> On Sat, Oct 31, 2009 at 12:11:05AM +0100, Fred Leeflang wrote:
>>> Yes I was looking at those although I looked at the ulogd 1 source and
>>> only very briefly at ulogd 2, looking at those actually gave somebody
>>> else in #vuurmuur the idea to check into this way. It's far easier for
>>
>> Hi Fred,
>>
>> I'd suggest to look only for ulogd2 if possible. The architecture is
>> very different, and I think adding a plugin will be far easier.
>>
>
> I had already decided to do so and checked out
> git://git.netfilter.org/ulogd2.git (couldn't get to your personal git
> repo at the time) so I have a pretty decent idea what I'm looking at
> already.
>
>>> us to make such an extension and require ulogd to run and parse
>>> packets for us than it is to parse a syslog text file. So if I were to
>>> write an extension that would do what we want it to do, could that
>>> extension be part of the builltin extensions of ulogd[1|2] ? I'm more
>>> than happy to hack something together and let the list have a look at
>>> it.
>>
>> If the plugin is generic, it should not be a problem. Ulogd2 has 3 types
>> of plugins: input, filter, and output. So, from what you are telling,
>> I suppose it would be an input plugin ?
>>
>
> I suppose it would be an output plugin :) It should essentially do
> similar stuff from what I could see, kinda like the
> ulogd_output_SYSLOG.c. It would not do syslogging however but instead
> write out a logfile for vuurmuur UI displaying.
>
> I just discussed that this morning with the other developers and the
> project leader feels that we might be better off using
> libnetfilter_log directly from libvuurmuur as there's apparently a
> bigger chunk of library code in libvuurmuur that would have to be
> either ported to the plugin or we'd create a dependency on libvuurmuur
> in ulogd2, which he was pretty sure you wouldn't like :)
>
> So for now we'll stick to the approach of using libnetfilter_log from
> within our own library.
>
> Thanks,
> -Fred Leeflang
>

Re: ulogd extension

[Pierre Chifflier]

On Fri, Nov 06, 2009 at 04:36:58PM +0100, Fred Leeflang wrote:
> Pierre,
> 
> I've hacked support for creating nflog targets into vuurmuur. When experimenting
> with this I've found that it was real easy to have ulogd2's syslogemu
> output module write a syslog file that vuurmuur could use to do our
> syslog parsing. This may be
> more useful even than writing our own extension for ulogd2 as our project leader
> decided not to want to obsolete the log parsing daemon.
> 
> So, we could simply add a requirement to vuurmuur (or at least a recommended)
> of ulogd. Does ulogd1 also support nflog tgts? If not, when's ulogd2
> scheduled to be
> released? Are any distro packagers working on it already?
> 

Hi Fred,

AFAICT ulogd1 does not support NFLOG, only ulogd2. For the release date,
I have no idea, though as it is in beta stage it should not be long.
On the packages, I have working packages for Debian. In fact, I will
upload them in experimental as soon as I have some time.

Regards,
Pierre