* ausearch on the fly @ 2019-12-20 13:33 MAUPERTUIS, PHILIPPE 2019-12-20 19:23 ` Steve Grubb 0 siblings, 1 reply; 9+ messages in thread From: MAUPERTUIS, PHILIPPE @ 2019-12-20 13:33 UTC (permalink / raw) To: linux-audit [-- Attachment #1.1: Type: text/plain, Size: 1254 bytes --] Hi, We are centralizing the audit logs with rsyslog. The SIEM behind the central log server is unable to process the raw logs. We would like to push the ausearch result in CSV format in real time or near real time. Is there a way to have ausearch working from a pipe and and waiting when no logs are received Regards Philippe equensWorldline is a registered trade mark and trading name owned by the Worldline Group through its holding company. This e-mail and the documents attached are confidential and intended solely for the addressee. If you receive this e-mail in error, you are not authorized to copy, disclose, use or retain it. Please notify the sender immediately and delete this email from your systems. As emails may be intercepted, amended or lost, they are not secure. EquensWorldline and the Worldline Group therefore can accept no liability for any errors or their content. Although equensWorldline and the Worldline Group endeavours to maintain a virus-free network, we do not warrant that this transmission is virus-free and can accept no liability for any damages resulting from any virus transmitted. The risks are deemed to be accepted by everyone who communicates with equensWorldline and the Worldline Group by email [-- Attachment #1.2: Type: text/html, Size: 6274 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: ausearch on the fly 2019-12-20 13:33 ausearch on the fly MAUPERTUIS, PHILIPPE @ 2019-12-20 19:23 ` Steve Grubb 2019-12-20 19:26 ` MAUPERTUIS, PHILIPPE 2020-02-07 8:13 ` MAUPERTUIS, PHILIPPE 0 siblings, 2 replies; 9+ messages in thread From: Steve Grubb @ 2019-12-20 19:23 UTC (permalink / raw) To: linux-audit; +Cc: MAUPERTUIS, PHILIPPE On Friday, December 20, 2019 8:33:11 AM EST MAUPERTUIS, PHILIPPE wrote: > We are centralizing the audit logs with rsyslog. > The SIEM behind the central log server is unable to process the raw logs. > We would like to push the ausearch result in CSV format in real time or > near real time. Is there a way to have ausearch working from a pipe and > and waiting when no logs are received I think that I've seen others who setup a cron job and use the checkpointing feature so that they do not miss anything. You can pipe its output into logger. You probably also want to cut the first line which has the column headers. ausearch --start today --checkpoint /root/last-ausearch .chpt --format csv | tail -n +2 | logger Also, the latest syslog plugin can now do interpretation. I think its in alpha-9 which dates back to Nov 04, 2019. It really shouldn't be hard to copy and paste the code from ausearch into the syslog plugin to log directly in that format. I wonder if anyone else would find that useful? -Steve ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: ausearch on the fly 2019-12-20 19:23 ` Steve Grubb @ 2019-12-20 19:26 ` MAUPERTUIS, PHILIPPE 2019-12-24 1:15 ` warron.french 2020-02-07 8:13 ` MAUPERTUIS, PHILIPPE 1 sibling, 1 reply; 9+ messages in thread From: MAUPERTUIS, PHILIPPE @ 2019-12-20 19:26 UTC (permalink / raw) To: Steve Grubb, linux-audit Thank you steve, I will have a look at it. Philippe -----Message d'origine----- De : Steve Grubb [mailto:sgrubb@redhat.com] Envoyé : vendredi 20 décembre 2019 20:24 À : linux-audit@redhat.com Cc : MAUPERTUIS, PHILIPPE Objet : Re: ausearch on the fly On Friday, December 20, 2019 8:33:11 AM EST MAUPERTUIS, PHILIPPE wrote: > We are centralizing the audit logs with rsyslog. > The SIEM behind the central log server is unable to process the raw logs. > We would like to push the ausearch result in CSV format in real time or > near real time. Is there a way to have ausearch working from a pipe and > and waiting when no logs are received I think that I've seen others who setup a cron job and use the checkpointing feature so that they do not miss anything. You can pipe its output into logger. You probably also want to cut the first line which has the column headers. ausearch --start today --checkpoint /root/last-ausearch .chpt --format csv | tail -n +2 | logger Also, the latest syslog plugin can now do interpretation. I think its in alpha-9 which dates back to Nov 04, 2019. It really shouldn't be hard to copy and paste the code from ausearch into the syslog plugin to log directly in that format. I wonder if anyone else would find that useful? -Steve equensWorldline is a registered trade mark and trading name owned by the Worldline Group through its holding company. This e-mail and the documents attached are confidential and intended solely for the addressee. If you receive this e-mail in error, you are not authorized to copy, disclose, use or retain it. Please notify the sender immediately and delete this email from your systems. As emails may be intercepted, amended or lost, they are not secure. EquensWorldline and the Worldline Group therefore can accept no liability for any errors or their content. Although equensWorldline and the Worldline Group endeavours to maintain a virus-free network, we do not warrant that this transmission is virus-free and can accept no liability for any damages resulting from any virus transmitted. The risks are deemed to be accepted by everyone who communicates with equensWorldline and the Worldline Group by email ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: ausearch on the fly 2019-12-20 19:26 ` MAUPERTUIS, PHILIPPE @ 2019-12-24 1:15 ` warron.french 0 siblings, 0 replies; 9+ messages in thread From: warron.french @ 2019-12-24 1:15 UTC (permalink / raw) To: MAUPERTUIS, PHILIPPE; +Cc: linux-audit [-- Attachment #1.1: Type: text/plain, Size: 2691 bytes --] I have seen it done in exactly this manner too. Where I work we do this. -------------------------- Warron French On Fri, Dec 20, 2019 at 2:26 PM MAUPERTUIS, PHILIPPE < philippe.maupertuis@equensworldline.com> wrote: > Thank you steve, > I will have a look at it. > Philippe > > -----Message d'origine----- > De : Steve Grubb [mailto:sgrubb@redhat.com] > Envoyé : vendredi 20 décembre 2019 20:24 > À : linux-audit@redhat.com > Cc : MAUPERTUIS, PHILIPPE > Objet : Re: ausearch on the fly > > On Friday, December 20, 2019 8:33:11 AM EST MAUPERTUIS, PHILIPPE wrote: > > We are centralizing the audit logs with rsyslog. > > The SIEM behind the central log server is unable to process the raw logs. > > We would like to push the ausearch result in CSV format in real time or > > near real time. Is there a way to have ausearch working from a pipe and > > and waiting when no logs are received > > I think that I've seen others who setup a cron job and use the > checkpointing > feature so that they do not miss anything. You can pipe its output into > logger. You probably also want to cut the first line which has the column > headers. > > ausearch --start today --checkpoint /root/last-ausearch .chpt --format csv > | tail -n +2 | logger > > Also, the latest syslog plugin can now do interpretation. I think its in > alpha-9 which dates back to Nov 04, 2019. > > It really shouldn't be hard to copy and paste the code from ausearch into > the > syslog plugin to log directly in that format. I wonder if anyone else would > find that useful? > > -Steve > > > equensWorldline is a registered trade mark and trading name owned by the > Worldline Group through its holding company. > This e-mail and the documents attached are confidential and intended > solely for the addressee. If you receive this e-mail in error, you are not > authorized to copy, disclose, use or retain it. Please notify the sender > immediately and delete this email from your systems. As emails may be > intercepted, amended or lost, they are not secure. EquensWorldline and the > Worldline Group therefore can accept no liability for any errors or their > content. Although equensWorldline and the Worldline Group endeavours to > maintain a virus-free network, we do not warrant that this transmission is > virus-free and can accept no liability for any damages resulting from any > virus transmitted. The risks are deemed to be accepted by everyone who > communicates with equensWorldline and the Worldline Group by email > > > -- > Linux-audit mailing list > Linux-audit@redhat.com > https://www.redhat.com/mailman/listinfo/linux-audit > > [-- Attachment #1.2: Type: text/html, Size: 3590 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: ausearch on the fly 2019-12-20 19:23 ` Steve Grubb 2019-12-20 19:26 ` MAUPERTUIS, PHILIPPE @ 2020-02-07 8:13 ` MAUPERTUIS, PHILIPPE 2020-02-07 11:49 ` Burn Alting 1 sibling, 1 reply; 9+ messages in thread From: MAUPERTUIS, PHILIPPE @ 2020-02-07 8:13 UTC (permalink / raw) To: Steve Grubb, linux-audit > On Friday, December 20, 2019 8:33:11 AM EST MAUPERTUIS, PHILIPPE wrote: > > We are centralizing the audit logs with rsyslog. > > The SIEM behind the central log server is unable to process the raw logs. > > We would like to push the ausearch result in CSV format in real time or > > near real time. Is there a way to have ausearch working from a pipe and > > and waiting when no logs are received > > I think that I've seen others who setup a cron job and use the checkpointing > feature so that they do not miss anything. You can pipe its output into > logger. You probably also want to cut the first line which has the column > headers. > > ausearch --start today --checkpoint /root/last-ausearch .chpt --format csv | > tail -n +2 | logger On a central log server the input file can grow very big and very fast. Probably logrotate is needed to keep it in check. What happen to the checkpointing feature when the file is rotated ? How not to miss the last events from the old file and get the new events from the new file ? > > Also, the latest syslog plugin can now do interpretation. I think its in > alpha-9 which dates back to Nov 04, 2019. > > It really shouldn't be hard to copy and paste the code from ausearch into the > syslog plugin to log directly in that format. I wonder if anyone else would > find that useful? > It would be very useful indeed Philippe equensWorldline is a registered trade mark and trading name owned by the Worldline Group through its holding company. This e-mail and the documents attached are confidential and intended solely for the addressee. If you receive this e-mail in error, you are not authorized to copy, disclose, use or retain it. Please notify the sender immediately and delete this email from your systems. As emails may be intercepted, amended or lost, they are not secure. EquensWorldline and the Worldline Group therefore can accept no liability for any errors or their content. Although equensWorldline and the Worldline Group endeavours to maintain a virus-free network, we do not warrant that this transmission is virus-free and can accept no liability for any damages resulting from any virus transmitted. The risks are deemed to be accepted by everyone who communicates with equensWorldline and the Worldline Group by email ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: ausearch on the fly 2020-02-07 8:13 ` MAUPERTUIS, PHILIPPE @ 2020-02-07 11:49 ` Burn Alting 2020-02-07 12:44 ` MAUPERTUIS, PHILIPPE 0 siblings, 1 reply; 9+ messages in thread From: Burn Alting @ 2020-02-07 11:49 UTC (permalink / raw) To: MAUPERTUIS, PHILIPPE, Steve Grubb, linux-audit [-- Attachment #1.1: Type: text/plain, Size: 2363 bytes --] Philippe, On Fri, 2020-02-07 at 08:13 +0000, MAUPERTUIS, PHILIPPE wrote: > > On Friday, December 20, 2019 8:33:11 AM EST MAUPERTUIS, PHILIPPE wrote: > > > We are centralizing the audit logs with rsyslog.The SIEM behind the central > > > log server is unable to process the raw logs.We would like to push the > > > ausearch result in CSV format in real time ornear real time. Is there a way > > > to have ausearch working from a pipe andand waiting when no logs are received > > > > I think that I've seen others who setup a cron job and use the > > checkpointingfeature so that they do not miss anything. You can pipe its output > > intologger. You probably also want to cut the first line which has the > > columnheaders. > > ausearch --start today --checkpoint /root/last-ausearch .chpt --format csv |tail > > -n +2 | logger > > On a central log server the input file can grow very big and very fast.Probably > logrotate is needed to keep it in check.What happen to the checkpointing feature > when the file is rotated ?How not to miss the last events from the old file and > get the new events from the new file ? The above performs a checkpoint on the local machine and then sends it's output to the local syslog service via the logger program. Ausearchis independent of the syslog service. The checkpoint function of ausearch takes into account the audit.log log file roll-over feature built into auditd so,providing your auditd log file rotation is set appropriately, checkpoint works no matter how many audit.log files are in the audit log directory.For information, a 9 file 32MB per log file configuration works well for a very heavy processing host where exec's are logged. Further, if the generation of logsis such that the checkpoint does miss logs, then the checkpoint documentation shows one how to address this. If this is noted, then include the size of or numberof local log files. > > Also, the latest syslog plugin can now do interpretation. I think its inalpha-9 > > which dates back to Nov 04, 2019. > > It really shouldn't be hard to copy and paste the code from ausearch into > > thesyslog plugin to log directly in that format. I wonder if anyone else > > wouldfind that useful? > > > --Linux-audit mailing listLinux-audit@redhat.com > https://www.redhat.com/mailman/listinfo/linux-audit > [-- Attachment #1.2: Type: text/html, Size: 3598 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: ausearch on the fly 2020-02-07 11:49 ` Burn Alting @ 2020-02-07 12:44 ` MAUPERTUIS, PHILIPPE 2020-02-08 0:39 ` Burn Alting 0 siblings, 1 reply; 9+ messages in thread From: MAUPERTUIS, PHILIPPE @ 2020-02-07 12:44 UTC (permalink / raw) To: Burn Alting, Steve Grubb, linux-audit [-- Attachment #1.1: Type: text/plain, Size: 3850 bytes --] Hi, I should have been more precise in my question. I want to run this ausearch on the central log server, not on the original server. Very likely i need to specify the input file in that. And this input file would need to be rotated outside auditd Philippe Envoyé depuis mon mobile Orange ------ Message d'origine------ De: Burn Alting Date: ven. 7 févr. 2020 12:49 À: MAUPERTUIS, PHILIPPE;Steve Grubb;linux-audit@redhat.com; Cc: Objet :Re: ausearch on the fly Philippe, On Fri, 2020-02-07 at 08:13 +0000, MAUPERTUIS, PHILIPPE wrote: On Friday, December 20, 2019 8:33:11 AM EST MAUPERTUIS, PHILIPPE wrote: We are centralizing the audit logs with rsyslog. The SIEM behind the central log server is unable to process the raw logs. We would like to push the ausearch result in CSV format in real time or near real time. Is there a way to have ausearch working from a pipe and and waiting when no logs are received I think that I've seen others who setup a cron job and use the checkpointing feature so that they do not miss anything. You can pipe its output into logger. You probably also want to cut the first line which has the column headers. ausearch --start today --checkpoint /root/last-ausearch .chpt --format csv | tail -n +2 | logger On a central log server the input file can grow very big and very fast. Probably logrotate is needed to keep it in check. What happen to the checkpointing feature when the file is rotated ? How not to miss the last events from the old file and get the new events from the new file ? The above performs a checkpoint on the local machine and then sends it's output to the local syslog service via the logger program. Ausearch is independent of the syslog service. The checkpoint function of ausearch takes into account the audit.log log file roll-over feature built into auditd so, providing your auditd log file rotation is set appropriately, checkpoint works no matter how many audit.log files are in the audit log directory. For information, a 9 file 32MB per log file configuration works well for a very heavy processing host where exec's are logged. Further, if the generation of logs is such that the checkpoint does miss logs, then the checkpoint documentation shows one how to address this. If this is noted, then include the size of or number of local log files. Also, the latest syslog plugin can now do interpretation. I think its in alpha-9 which dates back to Nov 04, 2019. It really shouldn't be hard to copy and paste the code from ausearch into the syslog plugin to log directly in that format. I wonder if anyone else would find that useful? -- Linux-audit mailing list <mailto:Linux-audit@redhat.com> Linux-audit@redhat.com <https://www.redhat.com/mailman/listinfo/linux-audit> https://www.redhat.com/mailman/listinfo/linux-audit equensWorldline is a registered trade mark and trading name owned by the Worldline Group through its holding company. This e-mail and the documents attached are confidential and intended solely for the addressee. If you receive this e-mail in error, you are not authorized to copy, disclose, use or retain it. Please notify the sender immediately and delete this email from your systems. As emails may be intercepted, amended or lost, they are not secure. EquensWorldline and the Worldline Group therefore can accept no liability for any errors or their content. Although equensWorldline and the Worldline Group endeavours to maintain a virus-free network, we do not warrant that this transmission is virus-free and can accept no liability for any damages resulting from any virus transmitted. The risks are deemed to be accepted by everyone who communicates with equensWorldline and the Worldline Group by email [-- Attachment #1.2: Type: text/html, Size: 5772 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: ausearch on the fly 2020-02-07 12:44 ` MAUPERTUIS, PHILIPPE @ 2020-02-08 0:39 ` Burn Alting 2020-02-10 13:08 ` MAUPERTUIS, PHILIPPE 0 siblings, 1 reply; 9+ messages in thread From: Burn Alting @ 2020-02-08 0:39 UTC (permalink / raw) To: MAUPERTUIS, PHILIPPE, Steve Grubb, linux-audit [-- Attachment #1.1: Type: text/plain, Size: 1051 bytes --] Phillipe, To close this off, yes, you will need to investigate making use of the --input option. You may also want to consider having a central syslog directory structure that allows you to simulate the local directory structure (/var/log/audit) by having a revolving set of audit.log, audit.log.1, etc files and age them off to your raw archive once you have processed them. My preference is to enrich the logs via ausearch on the local host as this mitigates a number of risks (lost logs as per this discussion, local uids) and then send the results to a central log management capability. Regards Burn On Fri, 2020-02-07 at 12:44 +0000, MAUPERTUIS, PHILIPPE wrote: > Hi, > I should have been more precise in my question. > I want to run this ausearch on the central log server, not on the original > server. > Very likely i need to specify the input file in that. > And this input file would need to be rotated outside auditd > Philippe > > > > > Envoyé depuis mon mobile Orange > > > > > Snip [-- Attachment #1.2: Type: text/html, Size: 1747 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: ausearch on the fly 2020-02-08 0:39 ` Burn Alting @ 2020-02-10 13:08 ` MAUPERTUIS, PHILIPPE 0 siblings, 0 replies; 9+ messages in thread From: MAUPERTUIS, PHILIPPE @ 2020-02-10 13:08 UTC (permalink / raw) To: burn, Steve Grubb, linux-audit [-- Attachment #1.1: Type: text/plain, Size: 2301 bytes --] Hi, I should have read the man more carefully. I missed the point that --input can be a directory and not only a single file as I wrongly assumed. That solves any problems I had. Philippe De : Burn Alting [mailto:burn.alting@iinet.net.au] Envoyé : samedi 8 février 2020 01:39 À : MAUPERTUIS, PHILIPPE; Steve Grubb; linux-audit@redhat.com Objet : Re: ausearch on the fly Phillipe, To close this off, yes, you will need to investigate making use of the --input option. You may also want to consider having a central syslog directory structure that allows you to simulate the local directory structure (/var/log/audit) by having a revolving set of audit.log, audit.log.1, etc files and age them off to your raw archive once you have processed them. My preference is to enrich the logs via ausearch on the local host as this mitigates a number of risks (lost logs as per this discussion, local uids) and then send the results to a central log management capability. Regards Burn On Fri, 2020-02-07 at 12:44 +0000, MAUPERTUIS, PHILIPPE wrote: Hi, I should have been more precise in my question. I want to run this ausearch on the central log server, not on the original server. Very likely i need to specify the input file in that. And this input file would need to be rotated outside auditd Philippe Envoyé depuis mon mobile Orange Snip equensWorldline is a registered trade mark and trading name owned by the Worldline Group through its holding company. This e-mail and the documents attached are confidential and intended solely for the addressee. If you receive this e-mail in error, you are not authorized to copy, disclose, use or retain it. Please notify the sender immediately and delete this email from your systems. As emails may be intercepted, amended or lost, they are not secure. EquensWorldline and the Worldline Group therefore can accept no liability for any errors or their content. Although equensWorldline and the Worldline Group endeavours to maintain a virus-free network, we do not warrant that this transmission is virus-free and can accept no liability for any damages resulting from any virus transmitted. The risks are deemed to be accepted by everyone who communicates with equensWorldline and the Worldline Group by email [-- Attachment #1.2: Type: text/html, Size: 7304 bytes --] [-- Attachment #2: Type: text/plain, Size: 0 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-02-10 13:09 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-12-20 13:33 ausearch on the fly MAUPERTUIS, PHILIPPE 2019-12-20 19:23 ` Steve Grubb 2019-12-20 19:26 ` MAUPERTUIS, PHILIPPE 2019-12-24 1:15 ` warron.french 2020-02-07 8:13 ` MAUPERTUIS, PHILIPPE 2020-02-07 11:49 ` Burn Alting 2020-02-07 12:44 ` MAUPERTUIS, PHILIPPE 2020-02-08 0:39 ` Burn Alting 2020-02-10 13:08 ` MAUPERTUIS, PHILIPPE
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.