* [PATCH ulogd2 v2 v2 01/34] ulogd: fix parse-error check
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 02/34] filter: fix buffer sizes in filter plug-ins Jeremy Sowden
` (33 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
If `config_parse_file` returns `-ERRTOOLONG`, `config_errce` may
be NULL. However, the calling function was checking whether
`config_errce->key` was NULL instead.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
src/ulogd.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/ulogd.c b/src/ulogd.c
index b02f2602a895..8ea9793ec0fb 100644
--- a/src/ulogd.c
+++ b/src/ulogd.c
@@ -1134,7 +1134,7 @@ static int parse_conffile(const char *section, struct config_keyset *ce)
"section \"%s\" not found\n", section);
break;
case -ERRTOOLONG:
- if (config_errce->key)
+ if (config_errce)
ulogd_log(ULOGD_ERROR,
"string value too long for key \"%s\"\n",
config_errce->key);
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 02/34] filter: fix buffer sizes in filter plug-ins
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 01/34] ulogd: fix parse-error check Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 03/34] output: JSON: remove incorrect config value check Jeremy Sowden
` (32 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Three of the filter plug-ins define arrays to hold output key values.
The arrays are sized based on the values of enums. For example:
enum output_keys {
KEY_MAC_TYPE,
KEY_MAC_PROTOCOL,
KEY_MAC_SADDR,
START_KEY = KEY_MAC_SADDR,
KEY_MAC_DADDR,
KEY_MAC_ADDR,
MAX_KEY = KEY_MAC_ADDR,
};
static char hwmac_str[MAX_KEY - START_KEY][HWADDR_LENGTH];
The arrays are indexed by subtracting `START_KEY` from the enum value of
the key currently being processed: `hwmac_str[okey - START_KEY]`.
However, this means that the last key (`KEY_MAC_ADDR` in this example)
will run off the end of the array. Increase the size of the arrays.
In the case of `IP2BIN` and `IP2HBIN`, there is no overrun, but only
because they use the wrong upper bound when looping over the keys, and
thus don't assign a value to the last key. Correct the bound.
Also some small white-space tweaks.
Link: https://bugzilla.netfilter.org/show_bug.cgi?id=890
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
filter/ulogd_filter_HWHDR.c | 4 ++--
filter/ulogd_filter_IP2BIN.c | 14 +++++++-------
filter/ulogd_filter_IP2HBIN.c | 2 +-
filter/ulogd_filter_IP2STR.c | 6 +++---
4 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/filter/ulogd_filter_HWHDR.c b/filter/ulogd_filter_HWHDR.c
index bbca5e9b92f2..a5ee60dea44b 100644
--- a/filter/ulogd_filter_HWHDR.c
+++ b/filter/ulogd_filter_HWHDR.c
@@ -109,7 +109,7 @@ static struct ulogd_key mac2str_keys[] = {
},
};
-static char hwmac_str[MAX_KEY - START_KEY][HWADDR_LENGTH];
+static char hwmac_str[MAX_KEY - START_KEY + 1][HWADDR_LENGTH];
static int parse_mac2str(struct ulogd_key *ret, unsigned char *mac,
int okey, int len)
@@ -126,7 +126,7 @@ static int parse_mac2str(struct ulogd_key *ret, unsigned char *mac,
buf_cur = hwmac_str[okey - START_KEY];
for (i = 0; i < len; i++)
buf_cur += sprintf(buf_cur, "%02x%c", mac[i],
- i == len - 1 ? 0 : ':');
+ i == len - 1 ? 0 : ':');
okey_set_ptr(&ret[okey], hwmac_str[okey - START_KEY]);
diff --git a/filter/ulogd_filter_IP2BIN.c b/filter/ulogd_filter_IP2BIN.c
index 2172d93506d5..42bcd7c15f1b 100644
--- a/filter/ulogd_filter_IP2BIN.c
+++ b/filter/ulogd_filter_IP2BIN.c
@@ -114,7 +114,7 @@ static struct ulogd_key ip2bin_keys[] = {
};
-static char ipbin_array[MAX_KEY-START_KEY][IPADDR_LENGTH];
+static char ipbin_array[MAX_KEY - START_KEY + 1][IPADDR_LENGTH];
/**
* Convert IPv4 address (as 32-bit unsigned integer) to IPv6 address:
@@ -128,7 +128,7 @@ static inline void uint32_to_ipv6(const uint32_t ipv4, struct in6_addr *ipv6)
ipv6->s6_addr32[3] = ipv4;
}
-static int ip2bin(struct ulogd_key* inp, int index, int oindex)
+static int ip2bin(struct ulogd_key *inp, int index, int oindex)
{
char family = ikey_get_u8(&inp[KEY_OOB_FAMILY]);
char convfamily = family;
@@ -184,7 +184,7 @@ static int ip2bin(struct ulogd_key* inp, int index, int oindex)
addr8 = &addr->s6_addr[0];
for (i = 0; i < 4; i++) {
written = sprintf(buffer, "%02x%02x%02x%02x",
- addr8[0], addr8[1], addr8[2], addr8[3]);
+ addr8[0], addr8[1], addr8[2], addr8[3]);
if (written != 2 * 4) {
buffer[0] = 0;
return ULOGD_IRET_ERR;
@@ -205,13 +205,13 @@ static int interp_ip2bin(struct ulogd_pluginstance *pi)
int fret;
/* Iter on all addr fields */
- for(i = START_KEY; i < MAX_KEY; i++) {
+ for(i = START_KEY; i <= MAX_KEY; i++) {
if (pp_is_valid(inp, i)) {
- fret = ip2bin(inp, i, i-START_KEY);
+ fret = ip2bin(inp, i, i - START_KEY);
if (fret != ULOGD_IRET_OK)
return fret;
- okey_set_ptr(&ret[i-START_KEY],
- ipbin_array[i-START_KEY]);
+ okey_set_ptr(&ret[i - START_KEY],
+ ipbin_array[i - START_KEY]);
}
}
diff --git a/filter/ulogd_filter_IP2HBIN.c b/filter/ulogd_filter_IP2HBIN.c
index 087e824ae94b..2711f9c3e12a 100644
--- a/filter/ulogd_filter_IP2HBIN.c
+++ b/filter/ulogd_filter_IP2HBIN.c
@@ -153,7 +153,7 @@ static int interp_ip2hbin(struct ulogd_pluginstance *pi)
}
/* Iter on all addr fields */
- for(i = START_KEY; i < MAX_KEY; i++) {
+ for(i = START_KEY; i <= MAX_KEY; i++) {
if (pp_is_valid(inp, i)) {
switch (convfamily) {
case AF_INET:
diff --git a/filter/ulogd_filter_IP2STR.c b/filter/ulogd_filter_IP2STR.c
index 66324b0b3b22..4d0536817b6c 100644
--- a/filter/ulogd_filter_IP2STR.c
+++ b/filter/ulogd_filter_IP2STR.c
@@ -137,7 +137,7 @@ static struct ulogd_key ip2str_keys[] = {
},
};
-static char ipstr_array[MAX_KEY-START_KEY][IPADDR_LENGTH];
+static char ipstr_array[MAX_KEY - START_KEY + 1][IPADDR_LENGTH];
static int ip2str(struct ulogd_key *inp, int index, int oindex)
{
@@ -197,10 +197,10 @@ static int interp_ip2str(struct ulogd_pluginstance *pi)
/* Iter on all addr fields */
for (i = START_KEY; i <= MAX_KEY; i++) {
if (pp_is_valid(inp, i)) {
- fret = ip2str(inp, i, i-START_KEY);
+ fret = ip2str(inp, i, i - START_KEY);
if (fret != ULOGD_IRET_OK)
return fret;
- okey_set_ptr(&ret[i-START_KEY],
+ okey_set_ptr(&ret[i - START_KEY],
ipstr_array[i-START_KEY]);
}
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 03/34] output: JSON: remove incorrect config value check
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 01/34] ulogd: fix parse-error check Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 02/34] filter: fix buffer sizes in filter plug-ins Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 04/34] db: fix back-log capacity checks Jeremy Sowden
` (31 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
The `u.string` member of a config entry is an array, and so never NULL.
Output the device string unconditionally.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/ulogd_output_JSON.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/output/ulogd_output_JSON.c b/output/ulogd_output_JSON.c
index bbc3dba5d41a..700abc25e5ea 100644
--- a/output/ulogd_output_JSON.c
+++ b/output/ulogd_output_JSON.c
@@ -277,7 +277,7 @@ static int json_interp(struct ulogd_pluginstance *upi)
{
struct json_priv *opi = (struct json_priv *) &upi->private;
unsigned int i;
- char *buf, *tmp;
+ char *dvc, *buf, *tmp;
size_t buflen;
json_t *msg;
@@ -335,10 +335,8 @@ static int json_interp(struct ulogd_pluginstance *upi)
json_object_set_new(msg, "timestamp", json_string(timestr));
}
- if (upi->config_kset->ces[JSON_CONF_DEVICE].u.string) {
- char *dvc = upi->config_kset->ces[JSON_CONF_DEVICE].u.string;
- json_object_set_new(msg, "dvc", json_string(dvc));
- }
+ dvc = upi->config_kset->ces[JSON_CONF_DEVICE].u.string;
+ json_object_set_new(msg, "dvc", json_string(dvc));
for (i = 0; i < upi->input.num_keys; i++) {
struct ulogd_key *key = upi->input.keys[i].u.source;
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 04/34] db: fix back-log capacity checks
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (2 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 03/34] output: JSON: remove incorrect config value check Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 05/34] build: add checks to configure.ac Jeremy Sowden
` (30 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Previously, when adding queries to the back-log, the memory usage is
incremented and decremented by the size of the query structure and the
length of the SQL statement, `sizeof(struct db_stmt) + len`. However,
when checking whether there was available capacity to add a new query,
the struct size was ignored. Amend the check to include the struct
size, and also account for the NUL that terminates the SQL.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
util/db.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/util/db.c b/util/db.c
index c1d24365239f..ebd9f152ed83 100644
--- a/util/db.c
+++ b/util/db.c
@@ -404,14 +404,17 @@ static void __format_query_db(struct ulogd_pluginstance *upi, char *start)
static int __add_to_backlog(struct ulogd_pluginstance *upi, const char *stmt, unsigned int len)
{
struct db_instance *di = (struct db_instance *) &upi->private;
+ unsigned int query_size;
struct db_stmt *query;
/* check if we are using backlog */
if (di->backlog_memcap == 0)
return 0;
+ query_size = sizeof(*query) + len + 1;
+
/* check len against backlog */
- if (len + di->backlog_memusage > di->backlog_memcap) {
+ if (query_size + di->backlog_memcap - di->backlog_memusage) {
if (di->backlog_full == 0)
ulogd_log(ULOGD_ERROR,
"Backlog is full starting to reject events.\n");
@@ -419,7 +422,7 @@ static int __add_to_backlog(struct ulogd_pluginstance *upi, const char *stmt, un
return -1;
}
- query = malloc(sizeof(struct db_stmt));
+ query = malloc(sizeof(*query));
if (query == NULL)
return -1;
@@ -431,7 +434,7 @@ static int __add_to_backlog(struct ulogd_pluginstance *upi, const char *stmt, un
return -1;
}
- di->backlog_memusage += len + sizeof(struct db_stmt);
+ di->backlog_memusage += query_size;
di->backlog_full = 0;
llist_add_tail(&query->list, &di->backlog);
@@ -489,7 +492,7 @@ static int __treat_backlog(struct ulogd_pluginstance *upi)
di->driver->close_db(upi);
return _init_reconnect(upi);
} else {
- di->backlog_memusage -= query->len + sizeof(struct db_stmt);
+ di->backlog_memusage -= sizeof(*query) + query->len + 1;
llist_del(&query->list);
free(query->stmt);
free(query);
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 05/34] build: add checks to configure.ac
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (3 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 04/34] db: fix back-log capacity checks Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-30 10:04 ` Jan Engelhardt
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 06/34] src: remove some trailing white space Jeremy Sowden
` (29 subsequent siblings)
34 siblings, 1 reply; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Autoscan complains about a number of missing function, header and type
checks.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
configure.ac | 47 ++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 44 insertions(+), 3 deletions(-)
diff --git a/configure.ac b/configure.ac
index 6ee29ce321d0..07c45cfb7b27 100644
--- a/configure.ac
+++ b/configure.ac
@@ -22,17 +22,58 @@ AC_SUBST([libdl_LIBS])
dnl Checks for header files.
AC_HEADER_DIRENT
-AC_CHECK_HEADERS([fcntl.h unistd.h])
+AC_CHECK_HEADER_STDBOOL
+AC_CHECK_HEADERS([arpa/inet.h \
+ fcntl.h \
+ inttypes.h \
+ netdb.h \
+ netinet/in.h \
+ stdint.h \
+ sys/param.h \
+ sys/socket.h \
+ sys/time.h \
+ syslog.h \
+ unistd.h])
dnl Checks for typedefs, structures, and compiler characteristics.
AC_C_CONST
-AC_TYPE_SIZE_T
+AC_C_INLINE
AC_STRUCT_TM
AC_SYS_LARGEFILE
+AC_TYPE_INT8_T
+AC_TYPE_INT16_T
+AC_TYPE_INT32_T
+AC_TYPE_INT64_T
+AC_TYPE_PID_T
+AC_TYPE_SIZE_T
+AC_TYPE_SSIZE_T
+AC_TYPE_UID_T
+AC_TYPE_UINT8_T
+AC_TYPE_UINT16_T
+AC_TYPE_UINT32_T
+AC_TYPE_UINT64_T
dnl Checks for library functions.
+AC_FUNC_CHOWN
+AC_FUNC_MALLOC
+AC_FUNC_REALLOC
AC_FUNC_VPRINTF
-AC_CHECK_FUNCS([socket strerror])
+AC_CHECK_FUNCS([alarm \
+ ftruncate \
+ gethostname \
+ gettimeofday \
+ localtime_r \
+ memmove \
+ memset \
+ select \
+ socket \
+ strcasecmp \
+ strchr \
+ strdup \
+ strerror \
+ strncasecmp \
+ strndup \
+ strtoul])
AC_SEARCH_LIBS([pthread_create], [pthread], [libpthread_LIBS="$LIBS"; LIBS=""])
AC_SUBST([libpthread_LIBS])
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 06/34] src: remove some trailing white space
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (4 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 05/34] build: add checks to configure.ac Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 07/34] src: remove zero-valued config-key fields Jeremy Sowden
` (28 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/dbi/ulogd_output_DBI.c | 26 +++++-----
output/mysql/ulogd_output_MYSQL.c | 24 ++++-----
output/pcap/ulogd_output_PCAP.c | 18 +++----
output/pgsql/ulogd_output_PGSQL.c | 34 ++++++------
output/sqlite3/ulogd_output_SQLITE3.c | 18 +++----
src/ulogd.c | 74 +++++++++++++--------------
util/db.c | 8 +--
7 files changed, 101 insertions(+), 101 deletions(-)
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 88af7a1473c3..1a623e14c41a 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -4,8 +4,8 @@
* layer
*
* (C) 2000-2008 by Pierre Chifflier <chifflier@inl.fr>
- * This software is distributed under the terms of GNU GPL
- *
+ * This software is distributed under the terms of GNU GPL
+ *
* This plugin is based on the PostgreSQL plugin made by Harald Welte.
*
*/
@@ -44,23 +44,23 @@ static struct config_keyset dbi_kset = {
.num_ces = DB_CE_NUM + 7,
.ces = {
DB_CES,
- {
- .key = "db",
+ {
+ .key = "db",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
{
- .key = "host",
+ .key = "host",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_NONE,
},
- {
- .key = "user",
+ {
+ .key = "user",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
{
- .key = "pass",
+ .key = "pass",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_NONE,
},
@@ -70,13 +70,13 @@ static struct config_keyset dbi_kset = {
.options = CONFIG_OPT_NONE,
},
{
- .key = "schema",
+ .key = "schema",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_NONE,
.u.string = "public",
},
- {
- .key = "dbtype",
+ {
+ .key = "dbtype",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
@@ -291,8 +291,8 @@ static int configure_dbi(struct ulogd_pluginstance *upi,
return ulogd_db_configure(upi, stack);
}
-static struct ulogd_plugin dbi_plugin = {
- .name = "DBI",
+static struct ulogd_plugin dbi_plugin = {
+ .name = "DBI",
.input = {
.keys = NULL,
.num_keys = 0,
diff --git a/output/mysql/ulogd_output_MYSQL.c b/output/mysql/ulogd_output_MYSQL.c
index 9727e303f2c5..dc49a2ae4e5b 100644
--- a/output/mysql/ulogd_output_MYSQL.c
+++ b/output/mysql/ulogd_output_MYSQL.c
@@ -5,7 +5,7 @@
* (C) 2000-2005 by Harald Welte <laforge@gnumonks.org>
*
* This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
+ * it under the terms of the GNU General Public License version 2
* as published by the Free Software Foundation
*
* This program is distributed in the hope that it will be useful,
@@ -64,22 +64,22 @@ static struct config_keyset kset_mysql = {
.ces = {
DB_CES,
{
- .key = "db",
+ .key = "db",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
{
- .key = "host",
+ .key = "host",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
{
- .key = "user",
+ .key = "user",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
{
- .key = "pass",
+ .key = "pass",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
@@ -107,7 +107,7 @@ static int get_columns_mysql(struct ulogd_pluginstance *upi)
return -1;
}
- result = mysql_list_fields(mi->dbh,
+ result = mysql_list_fields(mi->dbh,
table_ce(upi->config_kset).u.string, NULL);
if (!result) {
ulogd_log(ULOGD_ERROR, "error in list_fields(): %s\n",
@@ -176,7 +176,7 @@ static int open_db_mysql(struct ulogd_pluginstance *upi)
char *db = db_ce(upi->config_kset).u.string;
#ifdef MYSQL_OPT_RECONNECT
my_bool trueval = 1;
-#endif
+#endif
mi->dbh = mysql_init(NULL);
if (!mi->dbh) {
@@ -185,20 +185,20 @@ static int open_db_mysql(struct ulogd_pluginstance *upi)
}
if (connect_timeout)
- mysql_options(mi->dbh, MYSQL_OPT_CONNECT_TIMEOUT,
+ mysql_options(mi->dbh, MYSQL_OPT_CONNECT_TIMEOUT,
(const char *) &connect_timeout);
#ifdef MYSQL_OPT_RECONNECT
# if defined(MYSQL_VERSION_ID) && (MYSQL_VERSION_ID >= 50019)
mysql_options(mi->dbh, MYSQL_OPT_RECONNECT, &trueval);
# endif
-#endif
+#endif
if (!mysql_real_connect(mi->dbh, server, user, pass, db, port, NULL, 0)) {
ulogd_log(ULOGD_ERROR, "can't connect to db: %s\n",
mysql_error(mi->dbh));
return -1;
}
-
+
#ifdef MYSQL_OPT_RECONNECT
# if defined(MYSQL_VERSION_ID) && (MYSQL_VERSION_ID < 50019)
mysql_options(mi->dbh, MYSQL_OPT_RECONNECT, &trueval);
@@ -263,7 +263,7 @@ static struct ulogd_plugin plugin_mysql = {
.input = {
.keys = NULL,
.num_keys = 0,
- .type = ULOGD_DTYPE_PACKET | ULOGD_DTYPE_FLOW,
+ .type = ULOGD_DTYPE_PACKET | ULOGD_DTYPE_FLOW,
},
.output = {
.type = ULOGD_DTYPE_SINK,
@@ -280,7 +280,7 @@ static struct ulogd_plugin plugin_mysql = {
void __attribute__ ((constructor)) init(void);
-void init(void)
+void init(void)
{
ulogd_register_plugin(&plugin_mysql);
}
diff --git a/output/pcap/ulogd_output_PCAP.c b/output/pcap/ulogd_output_PCAP.c
index e7798f20c8fc..6640087f55a5 100644
--- a/output/pcap/ulogd_output_PCAP.c
+++ b/output/pcap/ulogd_output_PCAP.c
@@ -5,7 +5,7 @@
* (C) 2002-2005 by Harald Welte <laforge@gnumonks.org>
*
* This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
+ * it under the terms of the GNU General Public License version 2
* as published by the Free Software Foundation
*
* This program is distributed in the hope that it will be useful,
@@ -91,14 +91,14 @@ struct pcap_sf_pkthdr {
static struct config_keyset pcap_kset = {
.num_ces = 2,
.ces = {
- {
- .key = "file",
- .type = CONFIG_TYPE_STRING,
+ {
+ .key = "file",
+ .type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_NONE,
.u = { .string = ULOGD_PCAP_DEFAULT },
},
- {
- .key = "sync",
+ {
+ .key = "sync",
.type = CONFIG_TYPE_INT,
.options = CONFIG_OPT_NONE,
.u = { .value = ULOGD_PCAP_SYNC_DEFAULT },
@@ -112,7 +112,7 @@ struct pcap_instance {
struct intr_id {
char* name;
- unsigned int id;
+ unsigned int id;
};
#define INTR_IDS 7
@@ -242,11 +242,11 @@ static int append_create_outfile(struct ulogd_pluginstance *upi)
} else {
pi->of = fopen(filename, "a");
if (!pi->of) {
- ulogd_log(ULOGD_ERROR, "can't open pcap file %s: %s\n",
+ ulogd_log(ULOGD_ERROR, "can't open pcap file %s: %s\n",
filename,
strerror(errno));
return -EPERM;
- }
+ }
}
return 0;
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index 04c266510a40..6f3cde61a312 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -2,9 +2,9 @@
*
* ulogd output plugin for logging to a PGSQL database
*
- * (C) 2000-2005 by Harald Welte <laforge@gnumonks.org>
- * This software is distributed under the terms of GNU GPL
- *
+ * (C) 2000-2005 by Harald Welte <laforge@gnumonks.org>
+ * This software is distributed under the terms of GNU GPL
+ *
* This plugin is based on the MySQL plugin made by Harald Welte.
* The support PostgreSQL were made by Jakab Laszlo.
*
@@ -41,23 +41,23 @@ static struct config_keyset pgsql_kset = {
.num_ces = DB_CE_NUM + 7,
.ces = {
DB_CES,
- {
- .key = "db",
+ {
+ .key = "db",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
{
- .key = "host",
+ .key = "host",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_NONE,
},
- {
- .key = "user",
+ {
+ .key = "user",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
{
- .key = "pass",
+ .key = "pass",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_NONE,
},
@@ -67,7 +67,7 @@ static struct config_keyset pgsql_kset = {
.options = CONFIG_OPT_NONE,
},
{
- .key = "schema",
+ .key = "schema",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_NONE,
.u.string = "public",
@@ -94,7 +94,7 @@ static struct config_keyset pgsql_kset = {
static int pgsql_namespace(struct ulogd_pluginstance *upi)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
- char pgbuf[strlen(PGSQL_HAVE_NAMESPACE_TEMPLATE) +
+ char pgbuf[strlen(PGSQL_HAVE_NAMESPACE_TEMPLATE) +
strlen(schema_ce(upi->config_kset).u.string) + 1];
if (!pi->dbh)
@@ -103,7 +103,7 @@ static int pgsql_namespace(struct ulogd_pluginstance *upi)
sprintf(pgbuf, PGSQL_HAVE_NAMESPACE_TEMPLATE,
schema_ce(upi->config_kset).u.string);
ulogd_log(ULOGD_DEBUG, "%s\n", pgbuf);
-
+
pi->pgres = PQexec(pi->dbh, pgbuf);
if (!pi->pgres) {
ulogd_log(ULOGD_DEBUG, "\n result false");
@@ -126,7 +126,7 @@ static int pgsql_namespace(struct ulogd_pluginstance *upi)
}
PQclear(pi->pgres);
-
+
return 0;
}
@@ -141,7 +141,7 @@ static int get_columns_pgsql(struct ulogd_pluginstance *upi)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
char pgbuf[strlen(PGSQL_GETCOLUMN_TEMPLATE_SCHEMA)
- + strlen(table_ce(upi->config_kset).u.string)
+ + strlen(table_ce(upi->config_kset).u.string)
+ strlen(pi->db_inst.schema) + 2];
int i;
@@ -303,7 +303,7 @@ static int open_db_pgsql(struct ulogd_pluginstance *upi)
static int escape_string_pgsql(struct ulogd_pluginstance *upi,
char *dst, const char *src, unsigned int len)
{
- return PQescapeString(dst, src, strlen(src));
+ return PQescapeString(dst, src, strlen(src));
}
static int execute_pgsql(struct ulogd_pluginstance *upi,
@@ -342,8 +342,8 @@ static int configure_pgsql(struct ulogd_pluginstance *upi,
return ulogd_db_configure(upi, stack);
}
-static struct ulogd_plugin pgsql_plugin = {
- .name = "PGSQL",
+static struct ulogd_plugin pgsql_plugin = {
+ .name = "PGSQL",
.input = {
.keys = NULL,
.num_keys = 0,
diff --git a/output/sqlite3/ulogd_output_SQLITE3.c b/output/sqlite3/ulogd_output_SQLITE3.c
index 0a9ad67edcff..8dd7cec586cf 100644
--- a/output/sqlite3/ulogd_output_SQLITE3.c
+++ b/output/sqlite3/ulogd_output_SQLITE3.c
@@ -4,7 +4,7 @@
* (C) 2005 by Ben La Monica <ben.lamonica@gmail.com>
*
* This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
+ * it under the terms of the GNU General Public License version 2
* as published by the Free Software Foundation
*
* This program is distributed in the hope that it will be useful,
@@ -15,7 +15,7 @@
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
+ *
* This module has been adapted from the ulogd_MYSQL.c written by
* Harald Welte <laforge@gnumonks.org>
* Alex Janssen <alex@ynfonatic.de>
@@ -24,7 +24,7 @@
* at http://www.pojo.us/ulogd/
*
* 2005-02-09 Harald Welte <laforge@gnumonks.org>:
- * - port to ulogd-1.20
+ * - port to ulogd-1.20
*
* 2006-10-09 Holger Eitzenberger <holger@my-eitzenberger.de>
* - port to ulogd-2.00
@@ -160,11 +160,11 @@ sqlite3_interp(struct ulogd_pluginstance *pi)
case ULOGD_RET_INT64:
ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.i64);
break;
-
+
case ULOGD_RET_UINT8:
ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.ui8);
break;
-
+
case ULOGD_RET_UINT16:
ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.ui16);
break;
@@ -205,7 +205,7 @@ sqlite3_interp(struct ulogd_pluginstance *pi)
err_bind:
ulogd_log(ULOGD_ERROR, "SQLITE: bind: %s\n", sqlite3_errmsg(priv->dbh));
-
+
return ULOGD_IRET_ERR;
}
@@ -418,8 +418,8 @@ sqlite3_stop(struct ulogd_pluginstance *pi)
return 0;
}
-static struct ulogd_plugin sqlite3_plugin = {
- .name = "SQLITE3",
+static struct ulogd_plugin sqlite3_plugin = {
+ .name = "SQLITE3",
.input = {
.type = ULOGD_DTYPE_PACKET | ULOGD_DTYPE_FLOW,
},
@@ -438,7 +438,7 @@ static struct ulogd_plugin sqlite3_plugin = {
static void init(void) __attribute__((constructor));
static void
-init(void)
+init(void)
{
ulogd_register_plugin(&sqlite3_plugin);
}
diff --git a/src/ulogd.c b/src/ulogd.c
index 8ea9793ec0fb..cdb5c689ab36 100644
--- a/src/ulogd.c
+++ b/src/ulogd.c
@@ -7,7 +7,7 @@
* (C) 2013 Chris Boot <bootc@bootc.net>
*
* This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
+ * it under the terms of the GNU General Public License version 2
* as published by the Free Software Foundation
*
* This program is distributed in the hope that it will be useful,
@@ -71,7 +71,7 @@
#ifdef DEBUG
#define DEBUGP(format, args...) fprintf(stderr, format, ## args)
#else
-#define DEBUGP(format, args...)
+#define DEBUGP(format, args...)
#endif
#define COPYRIGHT \
@@ -143,7 +143,7 @@ static struct config_keyset ulogd_kset = {
.u.parser = &load_plugin,
},
{
- .key = "loglevel",
+ .key = "loglevel",
.type = CONFIG_TYPE_INT,
.options = CONFIG_OPT_NONE,
.u.value = ULOGD_NOTICE,
@@ -253,7 +253,7 @@ int ulogd_wildcard_inputkeys(struct ulogd_pluginstance *upi)
/***********************************************************************
- * PLUGIN MANAGEMENT
+ * PLUGIN MANAGEMENT
***********************************************************************/
/* try to lookup a registered plugin for a given name */
@@ -325,7 +325,7 @@ void get_plugin_infos(struct ulogd_plugin *me)
switch (me->config_kset->ces[i].type) {
case CONFIG_TYPE_STRING:
printf("String");
- printf(", Default: %s",
+ printf(", Default: %s",
me->config_kset->ces[i].u.string);
break;
case CONFIG_TYPE_INT:
@@ -340,7 +340,7 @@ void get_plugin_infos(struct ulogd_plugin *me)
printf("Unknown");
break;
}
- if (me->config_kset->ces[i].options ==
+ if (me->config_kset->ces[i].options ==
CONFIG_OPT_MANDATORY) {
printf(", Mandatory");
}
@@ -353,7 +353,7 @@ void get_plugin_infos(struct ulogd_plugin *me)
printf("\tNo statically defined keys\n");
} else {
for(i = 0; i < me->input.num_keys; i++) {
- char *tstring =
+ char *tstring =
type_to_string(me->input.keys[i].type);
printf("\tKey: %s (%s",
me->input.keys[i].name,
@@ -391,8 +391,8 @@ void get_plugin_infos(struct ulogd_plugin *me)
/* the function called by all plugins for registering themselves */
void ulogd_register_plugin(struct ulogd_plugin *me)
{
- if (strcmp(me->version, VERSION)) {
- ulogd_log(ULOGD_NOTICE,
+ if (strcmp(me->version, VERSION)) {
+ ulogd_log(ULOGD_NOTICE,
"plugin `%s' has incompatible version %s\n",
me->name,
me->version);
@@ -588,7 +588,7 @@ static void ulogd_clean_results(struct ulogd_pluginstance *pi)
/* iterate through plugin stack */
llist_for_each_entry(cur, &pi->stack->list, list) {
unsigned int i;
-
+
/* iterate through input keys of pluginstance */
for (i = 0; i < cur->output.num_keys; i++) {
struct ulogd_key *key = &cur->output.keys[i];
@@ -614,7 +614,7 @@ void ulogd_propagate_results(struct ulogd_pluginstance *pi)
/* iterate over remaining plugin stack */
llist_for_each_entry_continue(cur, &pi->stack->list, list) {
int ret;
-
+
ret = cur->plugin->interp(cur);
switch (ret) {
case ULOGD_IRET_ERR:
@@ -656,7 +656,7 @@ pluginstance_alloc_init(struct ulogd_plugin *pl, char *pi_id,
if (pl->config_kset) {
size += sizeof(struct config_keyset);
if (pl->config_kset->num_ces)
- size += pl->config_kset->num_ces *
+ size += pl->config_kset->num_ces *
sizeof(struct config_entry);
}
size += pl->input.num_keys * sizeof(struct ulogd_key);
@@ -681,9 +681,9 @@ pluginstance_alloc_init(struct ulogd_plugin *pl, char *pi_id,
ptr += sizeof(struct config_keyset);
pi->config_kset->num_ces = pl->config_kset->num_ces;
if (pi->config_kset->num_ces) {
- ptr += pi->config_kset->num_ces
+ ptr += pi->config_kset->num_ces
* sizeof(struct config_entry);
- memcpy(pi->config_kset->ces, pl->config_kset->ces,
+ memcpy(pi->config_kset->ces, pl->config_kset->ces,
pi->config_kset->num_ces
* sizeof(struct config_entry));
}
@@ -694,16 +694,16 @@ pluginstance_alloc_init(struct ulogd_plugin *pl, char *pi_id,
if (pl->input.num_keys) {
pi->input.num_keys = pl->input.num_keys;
pi->input.keys = ptr;
- memcpy(pi->input.keys, pl->input.keys,
+ memcpy(pi->input.keys, pl->input.keys,
pl->input.num_keys * sizeof(struct ulogd_key));
ptr += pl->input.num_keys * sizeof(struct ulogd_key);
}
-
+
/* copy input keys */
if (pl->output.num_keys) {
pi->output.num_keys = pl->output.num_keys;
pi->output.keys = ptr;
- memcpy(pi->output.keys, pl->output.keys,
+ memcpy(pi->output.keys, pl->output.keys,
pl->output.num_keys * sizeof(struct ulogd_key));
}
@@ -799,12 +799,12 @@ create_stack_resolve_keys(struct ulogd_pluginstance_stack *stack)
/* pre-configuration pass */
llist_for_each_entry_reverse(pi_cur, &stack->list, list) {
- ulogd_log(ULOGD_DEBUG, "traversing plugin `%s'\n",
+ ulogd_log(ULOGD_DEBUG, "traversing plugin `%s'\n",
pi_cur->plugin->name);
/* call plugin to tell us which keys it requires in
* given configuration */
if (pi_cur->plugin->configure) {
- int ret = pi_cur->plugin->configure(pi_cur,
+ int ret = pi_cur->plugin->configure(pi_cur,
stack);
if (ret < 0) {
ulogd_log(ULOGD_ERROR, "error during "
@@ -838,7 +838,7 @@ create_stack_resolve_keys(struct ulogd_pluginstance_stack *stack)
if (&pi_prev->list == &stack->list) {
/* this is the last one in the stack */
- if (!(pi_cur->plugin->input.type
+ if (!(pi_cur->plugin->input.type
& ULOGD_DTYPE_SOURCE)) {
ulogd_log(ULOGD_ERROR, "first plugin in stack "
"has to be source plugin\n");
@@ -856,7 +856,7 @@ create_stack_resolve_keys(struct ulogd_pluginstance_stack *stack)
pi_cur->plugin->name,
pi_prev->plugin->name);
}
-
+
for (j = 0; j < pi_cur->input.num_keys; j++) {
struct ulogd_key *okey;
struct ulogd_key *ikey = &pi_cur->input.keys[j];
@@ -866,7 +866,7 @@ create_stack_resolve_keys(struct ulogd_pluginstance_stack *stack)
if (ikey->flags & ULOGD_KEYF_INACTIVE)
continue;
- if (ikey->u.source) {
+ if (ikey->u.source) {
ulogd_log(ULOGD_ERROR, "input key `%s' "
"already has source\n",
ikey->name);
@@ -874,7 +874,7 @@ create_stack_resolve_keys(struct ulogd_pluginstance_stack *stack)
return -EINVAL;
}
- okey = find_okey_in_stack(ikey->name,
+ okey = find_okey_in_stack(ikey->name,
stack, pi_cur);
if (!okey) {
if (ikey->flags & ULOGD_KEYF_OPTIONAL)
@@ -942,7 +942,7 @@ static int create_stack_start_instances(struct ulogd_pluginstance_stack *stack)
if (!pluginstance_started(pi)) {
ret = pi->plugin->start(pi);
if (ret < 0) {
- ulogd_log(ULOGD_ERROR,
+ ulogd_log(ULOGD_ERROR,
"error starting `%s'\n",
pi->id);
return ret;
@@ -998,7 +998,7 @@ static int create_stack(const char *option)
strncpy(pi_id, tok, ULOGD_MAX_KEYLEN-1);
pi_id[equals-tok] = '\0';
plname = equals+1;
-
+
/* find matching plugin */
pl = find_plugin(plname);
if (!pl) {
@@ -1012,16 +1012,16 @@ static int create_stack(const char *option)
/* allocate */
pi = pluginstance_alloc_init(pl, pi_id, stack);
if (!pi) {
- ulogd_log(ULOGD_ERROR,
+ ulogd_log(ULOGD_ERROR,
"unable to allocate pluginstance for %s\n",
pi_id);
ret = -ENOMEM;
goto out;
}
-
+
/* FIXME: call constructor routine from end to beginning,
* fix up input/output keys */
-
+
ulogd_log(ULOGD_DEBUG, "pushing `%s' on stack\n", pl->name);
llist_add_tail(&pi->list, &stack->list);
}
@@ -1052,7 +1052,7 @@ out_stack:
out_buf:
return ret;
}
-
+
static void ulogd_main_loop(void)
{
@@ -1089,7 +1089,7 @@ static int logfile_open(const char *name)
} else {
logfile = fopen(ulogd_logfile, "a");
if (!logfile) {
- fprintf(stderr, "ERROR: can't open logfile '%s': %s\n",
+ fprintf(stderr, "ERROR: can't open logfile '%s': %s\n",
name, strerror(errno));
exit(2);
}
@@ -1410,7 +1410,7 @@ static void sigterm_handler_task(int signal)
static void signal_handler_task(int signal)
{
ulogd_log(ULOGD_NOTICE, "signal received, calling pluginstances\n");
-
+
switch (signal) {
case SIGHUP:
/* reopen logfile */
@@ -1418,12 +1418,12 @@ static void signal_handler_task(int signal)
fclose(logfile);
logfile = fopen(ulogd_logfile, "a");
if (!logfile) {
- fprintf(stderr,
- "ERROR: can't open logfile %s: %s\n",
+ fprintf(stderr,
+ "ERROR: can't open logfile %s: %s\n",
ulogd_logfile, strerror(errno));
sigterm_handler_task(signal);
}
-
+
}
break;
default:
@@ -1494,7 +1494,7 @@ int main(int argc, char* argv[])
default:
case '?':
if (isprint(optopt))
- fprintf(stderr, "Unknown option `-%c'.\n",
+ fprintf(stderr, "Unknown option `-%c'.\n",
optopt);
else
fprintf(stderr, "Unknown option character "
@@ -1591,7 +1591,7 @@ int main(int argc, char* argv[])
}
if (llist_empty(&ulogd_pi_stacks)) {
- ulogd_log(ULOGD_FATAL,
+ ulogd_log(ULOGD_FATAL,
"not even a single working plugin stack\n");
warn_and_exit(daemonize);
}
@@ -1642,7 +1642,7 @@ int main(int argc, char* argv[])
signal(SIGUSR2, &signal_handler);
set_scheduler();
- ulogd_log(ULOGD_INFO,
+ ulogd_log(ULOGD_INFO,
"initialization finished, entering main loop\n");
ulogd_main_loop();
diff --git a/util/db.c b/util/db.c
index ebd9f152ed83..dab66216e07d 100644
--- a/util/db.c
+++ b/util/db.c
@@ -10,7 +10,7 @@
* (C) 2008,2013 Eric Leblond <eric@regit.org>
*
* This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2
+ * it under the terms of the GNU General Public License version 2
* as published by the Free Software Foundation
*
* This program is distributed in the hope that it will be useful,
@@ -21,7 +21,7 @@
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
+ *
*/
#include <unistd.h>
@@ -170,7 +170,7 @@ int ulogd_db_configure(struct ulogd_pluginstance *upi,
ret = di->driver->get_columns(upi);
if (ret < 0)
ulogd_log(ULOGD_ERROR, "error in get_columns\n");
-
+
/* Close database, since ulogd core could just call configure
* but abort during input key resolving routines. configure
* doesn't have a destructor... */
@@ -315,7 +315,7 @@ static int _init_reconnect(struct ulogd_pluginstance *upi)
/* Disable plugin permanently */
ulogd_log(ULOGD_ERROR, "permanently disabling plugin\n");
di->interp = &disabled_interp_db;
-
+
return 0;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 07/34] src: remove zero-valued config-key fields
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (5 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 06/34] src: remove some trailing white space Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-30 10:21 ` Jan Engelhardt
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 08/34] src: parenthesize config-entry macro arguments Jeremy Sowden
` (27 subsequent siblings)
34 siblings, 1 reply; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Struct members are zero-initialized as a matter of course.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
filter/ulogd_filter_MARK.c | 3 --
include/ulogd/db.h | 1 -
input/flow/ulogd_inpflow_NFCT.c | 14 ---------
input/packet/ulogd_inppkt_NFLOG.c | 19 ------------
input/packet/ulogd_inppkt_ULOG.c | 44 ++++++++++++----------------
input/packet/ulogd_inppkt_UNIXSOCK.c | 5 ----
input/sum/ulogd_inpflow_NFACCT.c | 5 ----
output/dbi/ulogd_output_DBI.c | 4 ---
output/ipfix/ulogd_output_IPFIX.c | 2 --
output/pcap/ulogd_output_PCAP.c | 2 --
output/pgsql/ulogd_output_PGSQL.c | 5 ----
output/ulogd_output_GPRINT.c | 7 +----
output/ulogd_output_GRAPHITE.c | 3 --
output/ulogd_output_JSON.c | 13 +-------
output/ulogd_output_LOGEMU.c | 2 --
output/ulogd_output_NACCT.c | 5 +---
output/ulogd_output_OPRINT.c | 5 +---
output/ulogd_output_SYSLOG.c | 16 +++++-----
output/ulogd_output_XML.c | 5 ----
src/ulogd.c | 2 --
20 files changed, 30 insertions(+), 132 deletions(-)
diff --git a/filter/ulogd_filter_MARK.c b/filter/ulogd_filter_MARK.c
index 149725d92574..94343a2b6b5b 100644
--- a/filter/ulogd_filter_MARK.c
+++ b/filter/ulogd_filter_MARK.c
@@ -35,13 +35,10 @@ static struct config_keyset libulog_kset = {
[MARK_MARK] = {
.key = "mark",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
[MARK_MASK] = {
.key = "mask",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = 0xffffffff,
},
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index 9f9e96d6e077..bf62f236d772 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -90,7 +90,6 @@ struct db_instance {
{ \
.key = "backlog_memcap", \
.type = CONFIG_TYPE_INT, \
- .u.value = 0, \
}, \
{ \
.key = "backlog_oneshot_requests", \
diff --git a/input/flow/ulogd_inpflow_NFCT.c b/input/flow/ulogd_inpflow_NFCT.c
index 899b7e3b8039..76250f2a9982 100644
--- a/input/flow/ulogd_inpflow_NFCT.c
+++ b/input/flow/ulogd_inpflow_NFCT.c
@@ -83,71 +83,57 @@ static struct config_keyset nfct_kset = {
{
.key = "pollinterval",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "hash_enable",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = 1,
},
{
.key = "hash_buckets",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = HTABLE_SIZE,
},
{
.key = "hash_max_entries",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = MAX_ENTRIES,
},
{
.key = "event_mask",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = EVENT_MASK,
},
{
.key = "netlink_socket_buffer_size",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = 0,
},
{
.key = "netlink_socket_buffer_maxsize",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = 0,
},
{
.key = "netlink_resync_timeout",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = 60,
},
{
.key = "reliable",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "accept_src_filter",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "accept_dst_filter",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "accept_proto_filter",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
},
};
diff --git a/input/packet/ulogd_inppkt_NFLOG.c b/input/packet/ulogd_inppkt_NFLOG.c
index 4fdeb12886cc..e197a705cb4d 100644
--- a/input/packet/ulogd_inppkt_NFLOG.c
+++ b/input/packet/ulogd_inppkt_NFLOG.c
@@ -45,69 +45,50 @@ static struct config_keyset libulog_kset = {
{
.key = "bufsize",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = NFLOG_BUFSIZE_DEFAULT,
},
{
.key = "group",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = NFLOG_GROUP_DEFAULT,
},
{
.key = "unbind",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = 1,
},
{
.key = "bind",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "seq_local",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "seq_global",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "numeric_label",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "netlink_socket_buffer_size",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "netlink_socket_buffer_maxsize",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "netlink_qthreshold",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "netlink_qtimeout",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "attach_conntrack",
diff --git a/input/packet/ulogd_inppkt_ULOG.c b/input/packet/ulogd_inppkt_ULOG.c
index 45ffc8b57ac4..bc271dbfd0b1 100644
--- a/input/packet/ulogd_inppkt_ULOG.c
+++ b/input/packet/ulogd_inppkt_ULOG.c
@@ -37,31 +37,25 @@ struct ulog_input {
static struct config_keyset libulog_kset = {
.num_ces = 4,
.ces = {
- {
- .key = "bufsize",
- .type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = ULOGD_BUFSIZE_DEFAULT,
- },
- {
- .key = "nlgroup",
- .type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = ULOGD_NLGROUP_DEFAULT,
- },
- {
- .key = "rmem",
- .type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = ULOGD_RMEM_DEFAULT,
- },
- {
- .key = "numeric_label",
- .type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
- },
-
+ {
+ .key = "bufsize",
+ .type = CONFIG_TYPE_INT,
+ .u.value = ULOGD_BUFSIZE_DEFAULT,
+ },
+ {
+ .key = "nlgroup",
+ .type = CONFIG_TYPE_INT,
+ .u.value = ULOGD_NLGROUP_DEFAULT,
+ },
+ {
+ .key = "rmem",
+ .type = CONFIG_TYPE_INT,
+ .u.value = ULOGD_RMEM_DEFAULT,
+ },
+ {
+ .key = "numeric_label",
+ .type = CONFIG_TYPE_INT,
+ },
}
};
enum ulog_keys {
diff --git a/input/packet/ulogd_inppkt_UNIXSOCK.c b/input/packet/ulogd_inppkt_UNIXSOCK.c
index f1d15348ccb3..0ce599bd9b84 100644
--- a/input/packet/ulogd_inppkt_UNIXSOCK.c
+++ b/input/packet/ulogd_inppkt_UNIXSOCK.c
@@ -290,30 +290,25 @@ static struct config_keyset libunixsock_kset = {
{
.key = "socket_path",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u.string = UNIXSOCK_UNIXPATH_DEFAULT,
},
{
.key = "bufsize",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = UNIXSOCK_BUFSIZE_DEFAULT,
},
{
.key = "perms",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = UNIXSOCK_PERMS_DEFAULT,
},
{
.key = "owner",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "group",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
},
};
diff --git a/input/sum/ulogd_inpflow_NFACCT.c b/input/sum/ulogd_inpflow_NFACCT.c
index b022e6373406..539fb67f5863 100644
--- a/input/sum/ulogd_inpflow_NFACCT.c
+++ b/input/sum/ulogd_inpflow_NFACCT.c
@@ -38,20 +38,15 @@ static struct config_keyset nfacct_kset = {
{
.key = "pollinterval",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
},
{
.key = "zerocounter",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = 1,
},
{
.key = "timestamp",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u.value = 0,
}
},
.num_ces = 3,
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 1a623e14c41a..5639125dde1f 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -52,7 +52,6 @@ static struct config_keyset dbi_kset = {
{
.key = "host",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "user",
@@ -62,17 +61,14 @@ static struct config_keyset dbi_kset = {
{
.key = "pass",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "port",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
},
{
.key = "schema",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u.string = "public",
},
{
diff --git a/output/ipfix/ulogd_output_IPFIX.c b/output/ipfix/ulogd_output_IPFIX.c
index 4863d008562e..13d170108375 100644
--- a/output/ipfix/ulogd_output_IPFIX.c
+++ b/output/ipfix/ulogd_output_IPFIX.c
@@ -53,12 +53,10 @@ static const struct config_keyset ipfix_kset = {
{
.key = "oid",
.type = CONFIG_TYPE_INT,
- .u.value = 0
},
{
.key = "host",
.type = CONFIG_TYPE_STRING,
- .u.string = ""
},
{
.key = "port",
diff --git a/output/pcap/ulogd_output_PCAP.c b/output/pcap/ulogd_output_PCAP.c
index 6640087f55a5..c125ffcd2a8f 100644
--- a/output/pcap/ulogd_output_PCAP.c
+++ b/output/pcap/ulogd_output_PCAP.c
@@ -94,13 +94,11 @@ static struct config_keyset pcap_kset = {
{
.key = "file",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u = { .string = ULOGD_PCAP_DEFAULT },
},
{
.key = "sync",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u = { .value = ULOGD_PCAP_SYNC_DEFAULT },
},
},
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index 6f3cde61a312..1890cb574a95 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -49,7 +49,6 @@ static struct config_keyset pgsql_kset = {
{
.key = "host",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "user",
@@ -59,23 +58,19 @@ static struct config_keyset pgsql_kset = {
{
.key = "pass",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "port",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
},
{
.key = "schema",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u.string = "public",
},
{
.key = "connstring",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
},
};
diff --git a/output/ulogd_output_GPRINT.c b/output/ulogd_output_GPRINT.c
index aedd08e980f7..72365dd87cc1 100644
--- a/output/ulogd_output_GPRINT.c
+++ b/output/ulogd_output_GPRINT.c
@@ -51,20 +51,15 @@ static struct config_keyset gprint_kset = {
[GPRINT_CONF_FILENAME] = {
.key = "file",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
- .u = {.string = ULOGD_GPRINT_DEFAULT },
+ .u = { .string = ULOGD_GPRINT_DEFAULT },
},
[GPRINT_CONF_SYNC] = {
.key = "sync",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
[GPRINT_CONF_TIMESTAMP] = {
.key = "timestamp",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
},
};
diff --git a/output/ulogd_output_GRAPHITE.c b/output/ulogd_output_GRAPHITE.c
index 5328f8e95572..28280b200544 100644
--- a/output/ulogd_output_GRAPHITE.c
+++ b/output/ulogd_output_GRAPHITE.c
@@ -67,17 +67,14 @@ static struct config_keyset graphite_kset = {
{
.key = "host",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "port",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
{
.key = "prefix",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
},
},
};
diff --git a/output/ulogd_output_JSON.c b/output/ulogd_output_JSON.c
index 700abc25e5ea..59aab305e545 100644
--- a/output/ulogd_output_JSON.c
+++ b/output/ulogd_output_JSON.c
@@ -87,26 +87,20 @@ static struct config_keyset json_kset = {
[JSON_CONF_FILENAME] = {
.key = "file",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
- .u = {.string = ULOGD_JSON_DEFAULT },
+ .u = { .string = ULOGD_JSON_DEFAULT },
},
[JSON_CONF_SYNC] = {
.key = "sync",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
[JSON_CONF_TIMESTAMP] = {
.key = "timestamp",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u = { .value = 1 },
},
[JSON_CONF_EVENTV1] = {
.key = "eventv1",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
[JSON_CONF_DEVICE] = {
.key = "device",
@@ -117,25 +111,20 @@ static struct config_keyset json_kset = {
[JSON_CONF_BOOLEAN_LABEL] = {
.key = "boolean_label",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
[JSON_CONF_MODE] = {
.key = "mode",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u = { .string = "file" },
},
[JSON_CONF_HOST] = {
.key = "host",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u = { .string = "127.0.0.1" },
},
[JSON_CONF_PORT] = {
.key = "port",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u = { .string = "12345" },
},
},
diff --git a/output/ulogd_output_LOGEMU.c b/output/ulogd_output_LOGEMU.c
index cfcfab734746..c20ba1852be8 100644
--- a/output/ulogd_output_LOGEMU.c
+++ b/output/ulogd_output_LOGEMU.c
@@ -64,13 +64,11 @@ static struct config_keyset logemu_kset = {
{
.key = "file",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u = { .string = ULOGD_LOGEMU_DEFAULT },
},
{
.key = "sync",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u = { .value = ULOGD_LOGEMU_SYNC_DEFAULT },
},
},
diff --git a/output/ulogd_output_NACCT.c b/output/ulogd_output_NACCT.c
index d369c7a87315..7f5d1c5ecbf2 100644
--- a/output/ulogd_output_NACCT.c
+++ b/output/ulogd_output_NACCT.c
@@ -163,14 +163,11 @@ static struct config_keyset nacct_kset = {
{
.key = "file",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
- .u = {.string = NACCT_FILE_DEFAULT },
+ .u = { .string = NACCT_FILE_DEFAULT },
},
{
.key = "sync",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
},
};
diff --git a/output/ulogd_output_OPRINT.c b/output/ulogd_output_OPRINT.c
index 6fde445ed1e4..265103a04719 100644
--- a/output/ulogd_output_OPRINT.c
+++ b/output/ulogd_output_OPRINT.c
@@ -105,14 +105,11 @@ static struct config_keyset oprint_kset = {
{
.key = "file",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
- .u = {.string = ULOGD_OPRINT_DEFAULT },
+ .u = { .string = ULOGD_OPRINT_DEFAULT },
},
{
.key = "sync",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
},
};
diff --git a/output/ulogd_output_SYSLOG.c b/output/ulogd_output_SYSLOG.c
index 9777f0fe9185..675db7daf16d 100644
--- a/output/ulogd_output_SYSLOG.c
+++ b/output/ulogd_output_SYSLOG.c
@@ -49,16 +49,14 @@ static struct config_keyset syslog_kset = {
.num_ces = 2,
.ces = {
{
- .key = "facility",
- .type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
- .u = { .string = SYSLOG_FACILITY_DEFAULT }
+ .key = "facility",
+ .type = CONFIG_TYPE_STRING,
+ .u = { .string = SYSLOG_FACILITY_DEFAULT }
},
- {
- .key = "level",
- .type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
- .u = { .string = SYSLOG_LEVEL_DEFAULT }
+ {
+ .key = "level",
+ .type = CONFIG_TYPE_STRING,
+ .u = { .string = SYSLOG_LEVEL_DEFAULT }
},
},
};
diff --git a/output/ulogd_output_XML.c b/output/ulogd_output_XML.c
index 44af596dc2bc..512293c506ae 100644
--- a/output/ulogd_output_XML.c
+++ b/output/ulogd_output_XML.c
@@ -75,20 +75,15 @@ static struct config_keyset xml_kset = {
[CFG_XML_DIR] = {
.key = "directory",
.type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_NONE,
.u = { .string = ULOGD_XML_DEFAULT_DIR },
},
[CFG_XML_SYNC] = {
.key = "sync",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
[CFG_XML_STDOUT] = {
.key = "stdout",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
- .u = { .value = 0 },
},
},
};
diff --git a/src/ulogd.c b/src/ulogd.c
index cdb5c689ab36..ec0745e63169 100644
--- a/src/ulogd.c
+++ b/src/ulogd.c
@@ -133,7 +133,6 @@ static struct config_keyset ulogd_kset = {
{
.key = "logfile",
.type = CONFIG_TYPE_CALLBACK,
- .options = CONFIG_OPT_NONE,
.u.parser = &logfile_open,
},
{
@@ -145,7 +144,6 @@ static struct config_keyset ulogd_kset = {
{
.key = "loglevel",
.type = CONFIG_TYPE_INT,
- .options = CONFIG_OPT_NONE,
.u.value = ULOGD_NOTICE,
},
{
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* Re: [PATCH ulogd2 v2 v2 07/34] src: remove zero-valued config-key fields
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 07/34] src: remove zero-valued config-key fields Jeremy Sowden
@ 2022-11-30 10:21 ` Jan Engelhardt
0 siblings, 0 replies; 40+ messages in thread
From: Jan Engelhardt @ 2022-11-30 10:21 UTC (permalink / raw)
To: Jeremy Sowden; +Cc: Netfilter Devel
On Tuesday 2022-11-29 22:47, Jeremy Sowden wrote:
>Struct members are zero-initialized as a matter of course.
>
>@@ -35,13 +35,10 @@ static struct config_keyset libulog_kset = {
> [MARK_MARK] = {
> .key = "mark",
> .type = CONFIG_TYPE_INT,
>- .options = CONFIG_OPT_NONE,
>- .u.value = 0,
> },
The struct is *aggregate-initialized*, which means two things:
- unspecified elements are *empty-initialized*
- since the cost of initialization has already been paid,
one might as well be explicit about .options
and not rely on CONFIG_OPT_NONE being 0.
NB: these structs could use a "const" qualifier.
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH ulogd2 v2 v2 08/34] src: parenthesize config-entry macro arguments
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (6 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 07/34] src: remove zero-valued config-key fields Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 09/34] src: define constructors and destructors consistently Jeremy Sowden
` (26 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
It's avoids problems with operator precedence. For example, with no
parentheses:
#define blah_ce(x) (x->ces[0])
blah_ce(&whatever)
yields:
(&(whatever->ces[0]))
instead of:
((&whatever)->ces[0])
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 14 +++++++-------
input/flow/ulogd_inpflow_NFCT.c | 18 +++++++++---------
input/packet/ulogd_inppkt_NFLOG.c | 24 ++++++++++++------------
input/sum/ulogd_inpflow_NFACCT.c | 6 +++---
output/dbi/ulogd_output_DBI.c | 14 +++++++-------
output/ipfix/ulogd_output_IPFIX.c | 12 ++++++------
output/mysql/ulogd_output_MYSQL.c | 10 +++++-----
output/pgsql/ulogd_output_PGSQL.c | 14 +++++++-------
output/ulogd_output_GRAPHITE.c | 6 +++---
output/ulogd_output_JSON.c | 8 ++++----
10 files changed, 63 insertions(+), 63 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index bf62f236d772..50925a69f240 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -103,13 +103,13 @@ struct db_instance {
}
#define DB_CE_NUM 7
-#define table_ce(x) (x->ces[0])
-#define reconnect_ce(x) (x->ces[1])
-#define timeout_ce(x) (x->ces[2])
-#define procedure_ce(x) (x->ces[3])
-#define backlog_memcap_ce(x) (x->ces[4])
-#define backlog_oneshot_ce(x) (x->ces[5])
-#define ringsize_ce(x) (x->ces[6])
+#define table_ce(x) ((x)->ces[0])
+#define reconnect_ce(x) ((x)->ces[1])
+#define timeout_ce(x) ((x)->ces[2])
+#define procedure_ce(x) ((x)->ces[3])
+#define backlog_memcap_ce(x) ((x)->ces[4])
+#define backlog_oneshot_ce(x) ((x)->ces[5])
+#define ringsize_ce(x) ((x)->ces[6])
void ulogd_db_signal(struct ulogd_pluginstance *upi, int signal);
int ulogd_db_start(struct ulogd_pluginstance *upi);
diff --git a/input/flow/ulogd_inpflow_NFCT.c b/input/flow/ulogd_inpflow_NFCT.c
index 76250f2a9982..04910601fb8c 100644
--- a/input/flow/ulogd_inpflow_NFCT.c
+++ b/input/flow/ulogd_inpflow_NFCT.c
@@ -137,15 +137,15 @@ static struct config_keyset nfct_kset = {
},
},
};
-#define pollint_ce(x) (x->ces[0])
-#define usehash_ce(x) (x->ces[1])
-#define buckets_ce(x) (x->ces[2])
-#define maxentries_ce(x) (x->ces[3])
-#define eventmask_ce(x) (x->ces[4])
-#define nlsockbufsize_ce(x) (x->ces[5])
-#define nlsockbufmaxsize_ce(x) (x->ces[6])
-#define nlresynctimeout_ce(x) (x->ces[7])
-#define reliable_ce(x) (x->ces[8])
+#define pollint_ce(x) ((x)->ces[0])
+#define usehash_ce(x) ((x)->ces[1])
+#define buckets_ce(x) ((x)->ces[2])
+#define maxentries_ce(x) ((x)->ces[3])
+#define eventmask_ce(x) ((x)->ces[4])
+#define nlsockbufsize_ce(x) ((x)->ces[5])
+#define nlsockbufmaxsize_ce(x) ((x)->ces[6])
+#define nlresynctimeout_ce(x) ((x)->ces[7])
+#define reliable_ce(x) ((x)->ces[8])
#define src_filter_ce(x) ((x)->ces[9])
#define dst_filter_ce(x) ((x)->ces[10])
#define proto_filter_ce(x) ((x)->ces[11])
diff --git a/input/packet/ulogd_inppkt_NFLOG.c b/input/packet/ulogd_inppkt_NFLOG.c
index e197a705cb4d..4cbae69fee20 100644
--- a/input/packet/ulogd_inppkt_NFLOG.c
+++ b/input/packet/ulogd_inppkt_NFLOG.c
@@ -99,18 +99,18 @@ static struct config_keyset libulog_kset = {
}
};
-#define bufsiz_ce(x) (x->ces[0])
-#define group_ce(x) (x->ces[1])
-#define unbind_ce(x) (x->ces[2])
-#define bind_ce(x) (x->ces[3])
-#define seq_ce(x) (x->ces[4])
-#define seq_global_ce(x) (x->ces[5])
-#define label_ce(x) (x->ces[6])
-#define nlsockbufsize_ce(x) (x->ces[7])
-#define nlsockbufmaxsize_ce(x) (x->ces[8])
-#define nlthreshold_ce(x) (x->ces[9])
-#define nltimeout_ce(x) (x->ces[10])
-#define attach_conntrack_ce(x) (x->ces[11])
+#define bufsiz_ce(x) ((x)->ces[0])
+#define group_ce(x) ((x)->ces[1])
+#define unbind_ce(x) ((x)->ces[2])
+#define bind_ce(x) ((x)->ces[3])
+#define seq_ce(x) ((x)->ces[4])
+#define seq_global_ce(x) ((x)->ces[5])
+#define label_ce(x) ((x)->ces[6])
+#define nlsockbufsize_ce(x) ((x)->ces[7])
+#define nlsockbufmaxsize_ce(x) ((x)->ces[8])
+#define nlthreshold_ce(x) ((x)->ces[9])
+#define nltimeout_ce(x) ((x)->ces[10])
+#define attach_conntrack_ce(x) ((x)->ces[11])
enum nflog_keys {
NFLOG_KEY_RAW_MAC = 0,
diff --git a/input/sum/ulogd_inpflow_NFACCT.c b/input/sum/ulogd_inpflow_NFACCT.c
index 539fb67f5863..c680a52a80e8 100644
--- a/input/sum/ulogd_inpflow_NFACCT.c
+++ b/input/sum/ulogd_inpflow_NFACCT.c
@@ -51,9 +51,9 @@ static struct config_keyset nfacct_kset = {
},
.num_ces = 3,
};
-#define pollint_ce(x) (x->ces[0])
-#define zerocounter_ce(x) (x->ces[1])
-#define timestamp_ce(x) (x->ces[2])
+#define pollint_ce(x) ((x)->ces[0])
+#define zerocounter_ce(x) ((x)->ces[1])
+#define timestamp_ce(x) ((x)->ces[2])
enum ulogd_nfacct_keys {
ULOGD_NFACCT_NAME,
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 5639125dde1f..95bd35789c08 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -78,13 +78,13 @@ static struct config_keyset dbi_kset = {
},
},
};
-#define db_ce(x) (x->ces[DB_CE_NUM+0])
-#define host_ce(x) (x->ces[DB_CE_NUM+1])
-#define user_ce(x) (x->ces[DB_CE_NUM+2])
-#define pass_ce(x) (x->ces[DB_CE_NUM+3])
-#define port_ce(x) (x->ces[DB_CE_NUM+4])
-#define schema_ce(x) (x->ces[DB_CE_NUM+5])
-#define dbtype_ce(x) (x->ces[DB_CE_NUM+6])
+#define db_ce(x) ((x)->ces[DB_CE_NUM + 0])
+#define host_ce(x) ((x)->ces[DB_CE_NUM + 1])
+#define user_ce(x) ((x)->ces[DB_CE_NUM + 2])
+#define pass_ce(x) ((x)->ces[DB_CE_NUM + 3])
+#define port_ce(x) ((x)->ces[DB_CE_NUM + 4])
+#define schema_ce(x) ((x)->ces[DB_CE_NUM + 5])
+#define dbtype_ce(x) ((x)->ces[DB_CE_NUM + 6])
/* find out which columns the table has */
diff --git a/output/ipfix/ulogd_output_IPFIX.c b/output/ipfix/ulogd_output_IPFIX.c
index 13d170108375..745d30a163b2 100644
--- a/output/ipfix/ulogd_output_IPFIX.c
+++ b/output/ipfix/ulogd_output_IPFIX.c
@@ -40,12 +40,12 @@ enum {
SEND_TEMPLATE_CE
};
-#define oid_ce(x) (x->ces[OID_CE])
-#define host_ce(x) (x->ces[HOST_CE])
-#define port_ce(x) (x->ces[PORT_CE])
-#define proto_ce(x) (x->ces[PROTO_CE])
-#define mtu_ce(x) (x->ces[MTU_CE])
-#define send_template_ce(x) (x->ces[SEND_TEMPLATE_CE])
+#define oid_ce(x) ((x)->ces[OID_CE])
+#define host_ce(x) ((x)->ces[HOST_CE])
+#define port_ce(x) ((x)->ces[PORT_CE])
+#define proto_ce(x) ((x)->ces[PROTO_CE])
+#define mtu_ce(x) ((x)->ces[MTU_CE])
+#define send_template_ce(x) ((x)->ces[SEND_TEMPLATE_CE])
static const struct config_keyset ipfix_kset = {
.num_ces = 6,
diff --git a/output/mysql/ulogd_output_MYSQL.c b/output/mysql/ulogd_output_MYSQL.c
index dc49a2ae4e5b..0b79a9cd270c 100644
--- a/output/mysql/ulogd_output_MYSQL.c
+++ b/output/mysql/ulogd_output_MYSQL.c
@@ -89,11 +89,11 @@ static struct config_keyset kset_mysql = {
},
},
};
-#define db_ce(x) (x->ces[DB_CE_NUM+0])
-#define host_ce(x) (x->ces[DB_CE_NUM+1])
-#define user_ce(x) (x->ces[DB_CE_NUM+2])
-#define pass_ce(x) (x->ces[DB_CE_NUM+3])
-#define port_ce(x) (x->ces[DB_CE_NUM+4])
+#define db_ce(x) ((x)->ces[DB_CE_NUM + 0])
+#define host_ce(x) ((x)->ces[DB_CE_NUM + 1])
+#define user_ce(x) ((x)->ces[DB_CE_NUM + 2])
+#define pass_ce(x) ((x)->ces[DB_CE_NUM + 3])
+#define port_ce(x) ((x)->ces[DB_CE_NUM + 4])
/* find out which columns the table has */
static int get_columns_mysql(struct ulogd_pluginstance *upi)
{
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index 1890cb574a95..7f215f0fee4c 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -74,13 +74,13 @@ static struct config_keyset pgsql_kset = {
},
},
};
-#define db_ce(x) (x->ces[DB_CE_NUM+0])
-#define host_ce(x) (x->ces[DB_CE_NUM+1])
-#define user_ce(x) (x->ces[DB_CE_NUM+2])
-#define pass_ce(x) (x->ces[DB_CE_NUM+3])
-#define port_ce(x) (x->ces[DB_CE_NUM+4])
-#define schema_ce(x) (x->ces[DB_CE_NUM+5])
-#define connstr_ce(x) (x->ces[DB_CE_NUM+6])
+#define db_ce(x) ((x)->ces[DB_CE_NUM + 0])
+#define host_ce(x) ((x)->ces[DB_CE_NUM + 1])
+#define user_ce(x) ((x)->ces[DB_CE_NUM + 2])
+#define pass_ce(x) ((x)->ces[DB_CE_NUM + 3])
+#define port_ce(x) ((x)->ces[DB_CE_NUM + 4])
+#define schema_ce(x) ((x)->ces[DB_CE_NUM + 5])
+#define connstr_ce(x) ((x)->ces[DB_CE_NUM + 6])
#define PGSQL_HAVE_NAMESPACE_TEMPLATE \
"SELECT nspname FROM pg_namespace n WHERE n.nspname='%s'"
diff --git a/output/ulogd_output_GRAPHITE.c b/output/ulogd_output_GRAPHITE.c
index 28280b200544..22434707bcf9 100644
--- a/output/ulogd_output_GRAPHITE.c
+++ b/output/ulogd_output_GRAPHITE.c
@@ -79,9 +79,9 @@ static struct config_keyset graphite_kset = {
},
};
-#define host_ce(x) (x->ces[0])
-#define port_ce(x) (x->ces[1])
-#define prefix_ce(x) (x->ces[2])
+#define host_ce(x) ((x)->ces[0])
+#define port_ce(x) ((x)->ces[1])
+#define prefix_ce(x) ((x)->ces[2])
struct graphite_instance {
int sck;
diff --git a/output/ulogd_output_JSON.c b/output/ulogd_output_JSON.c
index 59aab305e545..76b51b2afaa7 100644
--- a/output/ulogd_output_JSON.c
+++ b/output/ulogd_output_JSON.c
@@ -45,10 +45,10 @@
#define ULOGD_JSON_DEFAULT_DEVICE "Netfilter"
#endif
-#define host_ce(x) (x->ces[JSON_CONF_HOST])
-#define port_ce(x) (x->ces[JSON_CONF_PORT])
-#define mode_ce(x) (x->ces[JSON_CONF_MODE])
-#define file_ce(x) (x->ces[JSON_CONF_FILENAME])
+#define host_ce(x) ((x)->ces[JSON_CONF_HOST])
+#define port_ce(x) ((x)->ces[JSON_CONF_PORT])
+#define mode_ce(x) ((x)->ces[JSON_CONF_MODE])
+#define file_ce(x) ((x)->ces[JSON_CONF_FILENAME])
#define unlikely(x) __builtin_expect((x),0)
struct json_priv {
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 09/34] src: define constructors and destructors consistently
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (7 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 08/34] src: parenthesize config-entry macro arguments Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 10/34] src: remove `TIME_ERR` macro Jeremy Sowden
` (25 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Remove separate declarations and declare the functions static.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
filter/raw2packet/ulogd_raw2packet_BASE.c | 4 +---
filter/ulogd_filter_HWHDR.c | 4 +---
filter/ulogd_filter_IFINDEX.c | 4 +---
filter/ulogd_filter_IP2BIN.c | 4 +---
filter/ulogd_filter_IP2HBIN.c | 4 +---
filter/ulogd_filter_IP2STR.c | 4 +---
filter/ulogd_filter_MARK.c | 4 +---
filter/ulogd_filter_PRINTFLOW.c | 4 +---
filter/ulogd_filter_PRINTPKT.c | 4 +---
filter/ulogd_filter_PWSNIFF.c | 2 +-
input/flow/ulogd_inpflow_NFCT.c | 5 +----
input/packet/ulogd_inppkt_NFLOG.c | 4 +---
input/packet/ulogd_inppkt_ULOG.c | 11 ++++++-----
input/sum/ulogd_inpflow_NFACCT.c | 4 +---
output/dbi/ulogd_output_DBI.c | 4 ++--
output/ipfix/ulogd_output_IPFIX.c | 4 +---
output/mysql/ulogd_output_MYSQL.c | 4 +---
output/pcap/ulogd_output_PCAP.c | 4 +---
output/pgsql/ulogd_output_PGSQL.c | 4 +---
output/sqlite3/ulogd_output_SQLITE3.c | 5 +----
20 files changed, 26 insertions(+), 61 deletions(-)
diff --git a/filter/raw2packet/ulogd_raw2packet_BASE.c b/filter/raw2packet/ulogd_raw2packet_BASE.c
index 9117d27da09a..71ef006c90fb 100644
--- a/filter/raw2packet/ulogd_raw2packet_BASE.c
+++ b/filter/raw2packet/ulogd_raw2packet_BASE.c
@@ -1002,9 +1002,7 @@ static struct ulogd_plugin base_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&base_plugin);
}
diff --git a/filter/ulogd_filter_HWHDR.c b/filter/ulogd_filter_HWHDR.c
index a5ee60dea44b..08a84fc49d75 100644
--- a/filter/ulogd_filter_HWHDR.c
+++ b/filter/ulogd_filter_HWHDR.c
@@ -236,9 +236,7 @@ static struct ulogd_plugin mac2str_pluging = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static __attribute__ ((constructor)) void init(void)
{
ulogd_register_plugin(&mac2str_pluging);
}
diff --git a/filter/ulogd_filter_IFINDEX.c b/filter/ulogd_filter_IFINDEX.c
index 3ca57955190c..2205e0025ca1 100644
--- a/filter/ulogd_filter_IFINDEX.c
+++ b/filter/ulogd_filter_IFINDEX.c
@@ -149,9 +149,7 @@ static struct ulogd_plugin ifindex_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static __attribute__ ((constructor)) void init(void)
{
ulogd_register_plugin(&ifindex_plugin);
}
diff --git a/filter/ulogd_filter_IP2BIN.c b/filter/ulogd_filter_IP2BIN.c
index 42bcd7c15f1b..ca6d3abae884 100644
--- a/filter/ulogd_filter_IP2BIN.c
+++ b/filter/ulogd_filter_IP2BIN.c
@@ -234,9 +234,7 @@ static struct ulogd_plugin ip2bin_pluging = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static __attribute__ ((constructor)) void init(void)
{
ulogd_register_plugin(&ip2bin_pluging);
}
diff --git a/filter/ulogd_filter_IP2HBIN.c b/filter/ulogd_filter_IP2HBIN.c
index 2711f9c3e12a..2d003a6335db 100644
--- a/filter/ulogd_filter_IP2HBIN.c
+++ b/filter/ulogd_filter_IP2HBIN.c
@@ -190,9 +190,7 @@ static struct ulogd_plugin ip2hbin_pluging = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static __attribute__ ((constructor)) void init(void)
{
ulogd_register_plugin(&ip2hbin_pluging);
}
diff --git a/filter/ulogd_filter_IP2STR.c b/filter/ulogd_filter_IP2STR.c
index 4d0536817b6c..177107fa8707 100644
--- a/filter/ulogd_filter_IP2STR.c
+++ b/filter/ulogd_filter_IP2STR.c
@@ -224,9 +224,7 @@ static struct ulogd_plugin ip2str_pluging = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static __attribute__ ((constructor)) void init(void)
{
ulogd_register_plugin(&ip2str_pluging);
}
diff --git a/filter/ulogd_filter_MARK.c b/filter/ulogd_filter_MARK.c
index 94343a2b6b5b..61b4d9a7eaea 100644
--- a/filter/ulogd_filter_MARK.c
+++ b/filter/ulogd_filter_MARK.c
@@ -111,9 +111,7 @@ static struct ulogd_plugin mark_pluging = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static __attribute__ ((constructor)) void init(void)
{
ulogd_register_plugin(&mark_pluging);
}
diff --git a/filter/ulogd_filter_PRINTFLOW.c b/filter/ulogd_filter_PRINTFLOW.c
index 95351f881c54..618b18b445c5 100644
--- a/filter/ulogd_filter_PRINTFLOW.c
+++ b/filter/ulogd_filter_PRINTFLOW.c
@@ -57,9 +57,7 @@ static struct ulogd_plugin printflow_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static __attribute__ ((constructor)) void init(void)
{
ulogd_register_plugin(&printflow_plugin);
}
diff --git a/filter/ulogd_filter_PRINTPKT.c b/filter/ulogd_filter_PRINTPKT.c
index de5c4688a4f1..7199e3cada07 100644
--- a/filter/ulogd_filter_PRINTPKT.c
+++ b/filter/ulogd_filter_PRINTPKT.c
@@ -57,9 +57,7 @@ static struct ulogd_plugin printpkt_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static __attribute__ ((constructor)) void init(void)
{
ulogd_register_plugin(&printpkt_plugin);
}
diff --git a/filter/ulogd_filter_PWSNIFF.c b/filter/ulogd_filter_PWSNIFF.c
index ef9e02115d84..6f4a04fc8329 100644
--- a/filter/ulogd_filter_PWSNIFF.c
+++ b/filter/ulogd_filter_PWSNIFF.c
@@ -170,7 +170,7 @@ static struct ulogd_plugin pwsniff_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&pwsniff_plugin);
}
diff --git a/input/flow/ulogd_inpflow_NFCT.c b/input/flow/ulogd_inpflow_NFCT.c
index 04910601fb8c..6209cdbd5dc1 100644
--- a/input/flow/ulogd_inpflow_NFCT.c
+++ b/input/flow/ulogd_inpflow_NFCT.c
@@ -1545,10 +1545,7 @@ static struct ulogd_plugin nfct_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&nfct_plugin);
}
-
diff --git a/input/packet/ulogd_inppkt_NFLOG.c b/input/packet/ulogd_inppkt_NFLOG.c
index 4cbae69fee20..191716b6e2d7 100644
--- a/input/packet/ulogd_inppkt_NFLOG.c
+++ b/input/packet/ulogd_inppkt_NFLOG.c
@@ -712,9 +712,7 @@ struct ulogd_plugin libulog_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&libulog_plugin);
}
diff --git a/input/packet/ulogd_inppkt_ULOG.c b/input/packet/ulogd_inppkt_ULOG.c
index bc271dbfd0b1..fd4f5693cab2 100644
--- a/input/packet/ulogd_inppkt_ULOG.c
+++ b/input/packet/ulogd_inppkt_ULOG.c
@@ -265,7 +265,8 @@ static int configure(struct ulogd_pluginstance *upi,
{
return config_parse_file(upi->id, upi->config_kset);
}
-static int init(struct ulogd_pluginstance *upi)
+
+static int start(struct ulogd_pluginstance *upi)
{
struct ulog_input *ui = (struct ulog_input *) &upi->private;
@@ -298,7 +299,7 @@ out_buf:
return -1;
}
-static int fini(struct ulogd_pluginstance *pi)
+static int stop(struct ulogd_pluginstance *pi)
{
struct ulog_input *ui = (struct ulog_input *)pi->private;
@@ -320,13 +321,13 @@ struct ulogd_plugin libulog_plugin = {
.num_keys = ARRAY_SIZE(output_keys),
},
.configure = &configure,
- .start = &init,
- .stop = &fini,
+ .start = &start,
+ .stop = &stop,
.config_kset = &libulog_kset,
.version = VERSION,
};
-void __attribute__ ((constructor)) initializer(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&libulog_plugin);
}
diff --git a/input/sum/ulogd_inpflow_NFACCT.c b/input/sum/ulogd_inpflow_NFACCT.c
index c680a52a80e8..9a1991e7aef7 100644
--- a/input/sum/ulogd_inpflow_NFACCT.c
+++ b/input/sum/ulogd_inpflow_NFACCT.c
@@ -300,9 +300,7 @@ static struct ulogd_plugin nfacct_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&nfacct_plugin);
}
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 95bd35789c08..7f42c08efc2b 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -307,14 +307,14 @@ static struct ulogd_plugin dbi_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void)
+static void __attribute__ ((constructor)) init(void)
{
dbi_initialize_r(NULL, &libdbi_instance);
ulogd_register_plugin(&dbi_plugin);
}
-void __attribute__ ((destructor)) fini(void)
+static void __attribute__ ((destructor)) fini(void)
{
dbi_shutdown_r(libdbi_instance);
}
diff --git a/output/ipfix/ulogd_output_IPFIX.c b/output/ipfix/ulogd_output_IPFIX.c
index 745d30a163b2..45fd36bc271c 100644
--- a/output/ipfix/ulogd_output_IPFIX.c
+++ b/output/ipfix/ulogd_output_IPFIX.c
@@ -504,9 +504,7 @@ static struct ulogd_plugin ipfix_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&ipfix_plugin);
}
diff --git a/output/mysql/ulogd_output_MYSQL.c b/output/mysql/ulogd_output_MYSQL.c
index 0b79a9cd270c..c98b98284748 100644
--- a/output/mysql/ulogd_output_MYSQL.c
+++ b/output/mysql/ulogd_output_MYSQL.c
@@ -278,9 +278,7 @@ static struct ulogd_plugin plugin_mysql = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&plugin_mysql);
}
diff --git a/output/pcap/ulogd_output_PCAP.c b/output/pcap/ulogd_output_PCAP.c
index c125ffcd2a8f..5f9fde3c48ed 100644
--- a/output/pcap/ulogd_output_PCAP.c
+++ b/output/pcap/ulogd_output_PCAP.c
@@ -307,9 +307,7 @@ static struct ulogd_plugin pcap_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&pcap_plugin);
}
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index 7f215f0fee4c..a508f9cf75a1 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -357,9 +357,7 @@ static struct ulogd_plugin pgsql_plugin = {
.version = VERSION,
};
-void __attribute__ ((constructor)) init(void);
-
-void init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&pgsql_plugin);
}
diff --git a/output/sqlite3/ulogd_output_SQLITE3.c b/output/sqlite3/ulogd_output_SQLITE3.c
index 8dd7cec586cf..40d99ca3b249 100644
--- a/output/sqlite3/ulogd_output_SQLITE3.c
+++ b/output/sqlite3/ulogd_output_SQLITE3.c
@@ -435,10 +435,7 @@ static struct ulogd_plugin sqlite3_plugin = {
.version = VERSION,
};
-static void init(void) __attribute__((constructor));
-
-static void
-init(void)
+static void __attribute__ ((constructor)) init(void)
{
ulogd_register_plugin(&sqlite3_plugin);
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 10/34] src: remove `TIME_ERR` macro
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (8 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 09/34] src: define constructors and destructors consistently Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 11/34] src: remove superfluous casts Jeremy Sowden
` (24 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
The `TIME_ERR` macro is used to check the return value of time(2).
However, two of the definitions of it are never used. The third is used
to check the return value of `time(NULL)`. However, time(2) with a
`NULL` argument can never fail, so we don't need to perform the check.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 2 +-
output/dbi/ulogd_output_DBI.c | 1 -
output/pgsql/ulogd_output_PGSQL.c | 1 -
util/db.c | 14 ++++++--------
4 files changed, 7 insertions(+), 11 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index 50925a69f240..a6fd25b4c043 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -62,7 +62,7 @@ struct db_instance {
unsigned char backlog_full;
struct llist_head backlog;
};
-#define TIME_ERR ((time_t)-1) /* Be paranoid */
+
#define RECONNECT_DEFAULT 2
#define MAX_ONESHOT_REQUEST 10
#define RING_BUFFER_DEFAULT_SIZE 0
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 7f42c08efc2b..88b530ead034 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -37,7 +37,6 @@ struct dbi_instance {
dbi_conn dbh;
dbi_result result;
};
-#define TIME_ERR ((time_t)-1)
/* our configuration directives */
static struct config_keyset dbi_kset = {
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index a508f9cf75a1..b798555b5ade 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -34,7 +34,6 @@ struct pgsql_instance {
PGresult *pgres;
unsigned char pgsql_have_schemas;
};
-#define TIME_ERR ((time_t)-1)
/* our configuration directives */
static struct config_keyset pgsql_kset = {
diff --git a/util/db.c b/util/db.c
index dab66216e07d..fb41266648d5 100644
--- a/util/db.c
+++ b/util/db.c
@@ -302,14 +302,12 @@ static int _init_reconnect(struct ulogd_pluginstance *upi)
if (time(NULL) < di->reconnect)
return -1;
di->reconnect = time(NULL);
- if (di->reconnect != TIME_ERR) {
- ulogd_log(ULOGD_ERROR, "no connection to database, "
- "attempting to reconnect after %u seconds\n",
- reconnect_ce(upi->config_kset).u.value);
- di->reconnect += reconnect_ce(upi->config_kset).u.value;
- di->interp = &_init_db;
- return -1;
- }
+ ulogd_log(ULOGD_ERROR,
+ "no connection to database, attempting to reconnect after %u seconds\n",
+ reconnect_ce(upi->config_kset).u.value);
+ di->reconnect += reconnect_ce(upi->config_kset).u.value;
+ di->interp = &_init_db;
+ return -1;
}
/* Disable plugin permanently */
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 11/34] src: remove superfluous casts
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (9 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 10/34] src: remove `TIME_ERR` macro Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 12/34] conffile: replace malloc+strcpy with strdup Jeremy Sowden
` (23 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
We're not writing C++, we don't need to cast void pointers.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
cftest/cftest.c | 2 +-
libipulog/libipulog.c | 2 +-
libipulog/ulog_test.c | 2 +-
src/conffile.c | 2 +-
src/hash.c | 4 ++--
src/ulogd.c | 2 +-
util/db.c | 4 ++--
7 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/cftest/cftest.c b/cftest/cftest.c
index b99882be5840..b16e75a5a451 100644
--- a/cftest/cftest.c
+++ b/cftest/cftest.c
@@ -19,7 +19,7 @@ int main()
strcpy(f.key, "spalte");
f.type = CONFIG_TYPE_STRING;
f.options |= CONFIG_OPT_MANDATORY;
- f.u.str.string = (char *) malloc(100);
+ f.u.str.string = malloc(100);
f.u.str.maxlen = 99;
config_register_key(&f);
diff --git a/libipulog/libipulog.c b/libipulog/libipulog.c
index b49f7f2cbd79..371c0d7622d6 100644
--- a/libipulog/libipulog.c
+++ b/libipulog/libipulog.c
@@ -129,7 +129,7 @@ struct ipulog_handle *ipulog_create_handle(uint32_t gmask,
struct ipulog_handle *h;
int status;
- h = (struct ipulog_handle *) malloc(sizeof(struct ipulog_handle));
+ h = malloc(sizeof(*h));
if (h == NULL)
{
ipulog_errno = IPULOG_ERR_HANDLE;
diff --git a/libipulog/ulog_test.c b/libipulog/ulog_test.c
index 06657172a0cd..6b796d064992 100644
--- a/libipulog/ulog_test.c
+++ b/libipulog/ulog_test.c
@@ -51,7 +51,7 @@ int main(int argc, char *argv[])
}
/* allocate a receive buffer */
- buf = (unsigned char *) malloc(MYBUFSIZ);
+ buf = malloc(MYBUFSIZ);
/* create ipulog handle */
h = ipulog_create_handle(ipulog_group2gmask(atoi(argv[2])));
diff --git a/src/conffile.c b/src/conffile.c
index 66769decc93c..8a208d6d8cfe 100644
--- a/src/conffile.c
+++ b/src/conffile.c
@@ -104,7 +104,7 @@ int config_register_file(const char *file)
pr_debug("%s: registered config file '%s'\n", __func__, file);
- fname = (char *) malloc(strlen(file)+1);
+ fname = malloc(strlen(file)+1);
if (!fname)
return -ERROOM;
diff --git a/src/hash.c b/src/hash.c
index 1d991309734f..2f7f5deebece 100644
--- a/src/hash.c
+++ b/src/hash.c
@@ -34,9 +34,9 @@ hashtable_create(int hashsize, int limit,
int i;
struct hashtable *h;
int size = sizeof(struct hashtable)
- + hashsize * sizeof(struct llist_head);
+ + sizeof(struct llist_head) * hashsize;
- h = (struct hashtable *) calloc(size, 1);
+ h = calloc(size, 1);
if (h == NULL) {
errno = ENOMEM;
return NULL;
diff --git a/src/ulogd.c b/src/ulogd.c
index ec0745e63169..82f936168ebc 100644
--- a/src/ulogd.c
+++ b/src/ulogd.c
@@ -720,7 +720,7 @@ static int load_plugin(const char *file)
return -1;
}
- ph = (struct ulogd_plugin_handle *) calloc(1, sizeof(*ph));
+ ph = calloc(1, sizeof(*ph));
ph->handle = handle;
llist_add(&ph->list, &ulogd_plugins_handle);
return 0;
diff --git a/util/db.c b/util/db.c
index fb41266648d5..6749079697dc 100644
--- a/util/db.c
+++ b/util/db.c
@@ -85,7 +85,7 @@ static int sql_createstmt(struct ulogd_pluginstance *upi)
ulogd_log(ULOGD_DEBUG, "allocating %u bytes for statement\n", size);
- mi->stmt = (char *) malloc(size);
+ mi->stmt = malloc(size);
if (!mi->stmt) {
ulogd_log(ULOGD_ERROR, "OOM!\n");
return -ENOMEM;
@@ -575,7 +575,7 @@ static int __loop_reconnect_db(struct ulogd_pluginstance * upi) {
static void *__inject_thread(void *gdi)
{
- struct ulogd_pluginstance *upi = (struct ulogd_pluginstance *) gdi;
+ struct ulogd_pluginstance *upi = gdi;
struct db_instance *di = (struct db_instance *) &upi->private;
char *wr_place;
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 12/34] conffile: replace malloc+strcpy with strdup
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (10 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 11/34] src: remove superfluous casts Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 13/34] output: remove zero-initialized `struct ulogd_plugin` members Jeremy Sowden
` (22 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
src/conffile.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/src/conffile.c b/src/conffile.c
index 8a208d6d8cfe..e58f54778e41 100644
--- a/src/conffile.c
+++ b/src/conffile.c
@@ -104,12 +104,10 @@ int config_register_file(const char *file)
pr_debug("%s: registered config file '%s'\n", __func__, file);
- fname = malloc(strlen(file)+1);
+ fname = strdup(file);
if (!fname)
return -ERROOM;
- strcpy(fname, file);
-
return 0;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 13/34] output: remove zero-initialized `struct ulogd_plugin` members
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (11 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 12/34] conffile: replace malloc+strcpy with strdup Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-30 10:26 ` Jan Engelhardt
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 14/34] output: de-duplicate allocation of input keys Jeremy Sowden
` (21 subsequent siblings)
34 siblings, 1 reply; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Struct members are zero-initialized as a matter of course.
Use consistent formatting.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/dbi/ulogd_output_DBI.c | 10 ++++------
output/ipfix/ulogd_output_IPFIX.c | 20 ++++++++++----------
output/mysql/ulogd_output_MYSQL.c | 22 ++++++++++------------
output/pcap/ulogd_output_PCAP.c | 6 +++---
output/pgsql/ulogd_output_PGSQL.c | 10 ++++------
output/sqlite3/ulogd_output_SQLITE3.c | 18 +++++++++---------
6 files changed, 40 insertions(+), 46 deletions(-)
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 88b530ead034..6312ac1649e2 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -287,16 +287,14 @@ static int configure_dbi(struct ulogd_pluginstance *upi,
}
static struct ulogd_plugin dbi_plugin = {
- .name = "DBI",
- .input = {
- .keys = NULL,
- .num_keys = 0,
+ .name = "DBI",
+ .input = {
.type = ULOGD_DTYPE_PACKET | ULOGD_DTYPE_FLOW,
},
- .output = {
+ .output = {
.type = ULOGD_DTYPE_SINK,
},
- .config_kset = &dbi_kset,
+ .config_kset = &dbi_kset,
.priv_size = sizeof(struct dbi_instance),
.configure = &configure_dbi,
.start = &ulogd_db_start,
diff --git a/output/ipfix/ulogd_output_IPFIX.c b/output/ipfix/ulogd_output_IPFIX.c
index 45fd36bc271c..f252ec72b5f6 100644
--- a/output/ipfix/ulogd_output_IPFIX.c
+++ b/output/ipfix/ulogd_output_IPFIX.c
@@ -486,22 +486,22 @@ again:
}
static struct ulogd_plugin ipfix_plugin = {
- .name = "IPFIX",
- .input = {
+ .name = "IPFIX",
+ .input = {
.keys = ipfix_in_keys,
.num_keys = ARRAY_SIZE(ipfix_in_keys),
.type = ULOGD_DTYPE_PACKET | ULOGD_DTYPE_FLOW | ULOGD_DTYPE_SUM
},
- .output = {
+ .output = {
.type = ULOGD_DTYPE_SINK
},
- .config_kset = (struct config_keyset *) &ipfix_kset,
- .priv_size = sizeof(struct ipfix_priv),
- .configure = ipfix_configure,
- .start = ipfix_start,
- .stop = ipfix_stop,
- .interp = ipfix_interp,
- .version = VERSION,
+ .config_kset = (struct config_keyset *) &ipfix_kset,
+ .priv_size = sizeof(struct ipfix_priv),
+ .configure = ipfix_configure,
+ .start = ipfix_start,
+ .stop = ipfix_stop,
+ .interp = ipfix_interp,
+ .version = VERSION,
};
static void __attribute__ ((constructor)) init(void)
diff --git a/output/mysql/ulogd_output_MYSQL.c b/output/mysql/ulogd_output_MYSQL.c
index c98b98284748..5891207d5990 100644
--- a/output/mysql/ulogd_output_MYSQL.c
+++ b/output/mysql/ulogd_output_MYSQL.c
@@ -259,23 +259,21 @@ static int configure_mysql(struct ulogd_pluginstance *upi,
}
static struct ulogd_plugin plugin_mysql = {
- .name = "MYSQL",
- .input = {
- .keys = NULL,
- .num_keys = 0,
+ .name = "MYSQL",
+ .input = {
.type = ULOGD_DTYPE_PACKET | ULOGD_DTYPE_FLOW,
},
- .output = {
+ .output = {
.type = ULOGD_DTYPE_SINK,
},
.config_kset = &kset_mysql,
- .priv_size = sizeof(struct mysql_instance),
- .configure = &configure_mysql,
- .start = &ulogd_db_start,
- .stop = &ulogd_db_stop,
- .signal = &ulogd_db_signal,
- .interp = &ulogd_db_interp,
- .version = VERSION,
+ .priv_size = sizeof(struct mysql_instance),
+ .configure = &configure_mysql,
+ .start = &ulogd_db_start,
+ .stop = &ulogd_db_stop,
+ .signal = &ulogd_db_signal,
+ .interp = &ulogd_db_interp,
+ .version = VERSION,
};
static void __attribute__ ((constructor)) init(void)
diff --git a/output/pcap/ulogd_output_PCAP.c b/output/pcap/ulogd_output_PCAP.c
index 5f9fde3c48ed..5336f88e539d 100644
--- a/output/pcap/ulogd_output_PCAP.c
+++ b/output/pcap/ulogd_output_PCAP.c
@@ -287,13 +287,13 @@ static int stop_pcap(struct ulogd_pluginstance *upi)
}
static struct ulogd_plugin pcap_plugin = {
- .name = "PCAP",
- .input = {
+ .name = "PCAP",
+ .input = {
.keys = pcap_keys,
.num_keys = ARRAY_SIZE(pcap_keys),
.type = ULOGD_DTYPE_PACKET,
},
- .output = {
+ .output = {
.type = ULOGD_DTYPE_SINK,
},
.config_kset = &pcap_kset,
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index b798555b5ade..bc0eae7558b3 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -337,16 +337,14 @@ static int configure_pgsql(struct ulogd_pluginstance *upi,
}
static struct ulogd_plugin pgsql_plugin = {
- .name = "PGSQL",
- .input = {
- .keys = NULL,
- .num_keys = 0,
+ .name = "PGSQL",
+ .input = {
.type = ULOGD_DTYPE_PACKET | ULOGD_DTYPE_FLOW,
},
- .output = {
+ .output = {
.type = ULOGD_DTYPE_SINK,
},
- .config_kset = &pgsql_kset,
+ .config_kset = &pgsql_kset,
.priv_size = sizeof(struct pgsql_instance),
.configure = &configure_pgsql,
.start = &ulogd_db_start,
diff --git a/output/sqlite3/ulogd_output_SQLITE3.c b/output/sqlite3/ulogd_output_SQLITE3.c
index 40d99ca3b249..32459dd6c4c5 100644
--- a/output/sqlite3/ulogd_output_SQLITE3.c
+++ b/output/sqlite3/ulogd_output_SQLITE3.c
@@ -419,20 +419,20 @@ sqlite3_stop(struct ulogd_pluginstance *pi)
}
static struct ulogd_plugin sqlite3_plugin = {
- .name = "SQLITE3",
- .input = {
+ .name = "SQLITE3",
+ .input = {
.type = ULOGD_DTYPE_PACKET | ULOGD_DTYPE_FLOW,
},
- .output = {
+ .output = {
.type = ULOGD_DTYPE_SINK,
},
.config_kset = &sqlite3_kset,
- .priv_size = sizeof(struct sqlite3_priv),
- .configure = sqlite3_configure,
- .start = sqlite3_start,
- .stop = sqlite3_stop,
- .interp = sqlite3_interp,
- .version = VERSION,
+ .priv_size = sizeof(struct sqlite3_priv),
+ .configure = sqlite3_configure,
+ .start = sqlite3_start,
+ .stop = sqlite3_stop,
+ .interp = sqlite3_interp,
+ .version = VERSION,
};
static void __attribute__ ((constructor)) init(void)
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 14/34] output: de-duplicate allocation of input keys
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (12 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 13/34] output: remove zero-initialized `struct ulogd_plugin` members Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 15/34] db: reorganize source Jeremy Sowden
` (20 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
The three DB output plug-ins which use the ulogd_db API all derive the
names of their input keys from DB column names, and do so in almost
identical fashion. Create a common ulogd_db implementation and update
them to use it.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 11 ++++-
output/dbi/ulogd_output_DBI.c | 67 ++++++++++++++-----------------
output/mysql/ulogd_output_MYSQL.c | 46 ++++++++-------------
output/pgsql/ulogd_output_PGSQL.c | 43 ++++++--------------
util/db.c | 48 ++++++++++++++++++++++
5 files changed, 116 insertions(+), 99 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index a6fd25b4c043..7c0649583f1d 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -12,13 +12,19 @@
#include <ulogd/ulogd.h>
struct db_driver {
+
int (*get_columns)(struct ulogd_pluginstance *upi);
+ const char *(*get_column)(void *, unsigned int);
+ char *(*format_key)(char *);
+
int (*open_db)(struct ulogd_pluginstance *upi);
int (*close_db)(struct ulogd_pluginstance *upi);
+
int (*escape_string)(struct ulogd_pluginstance *upi,
char *dst, const char *src, unsigned int len);
int (*execute)(struct ulogd_pluginstance *upi,
const char *stmt, unsigned int len);
+
};
enum {
@@ -116,7 +122,8 @@ int ulogd_db_start(struct ulogd_pluginstance *upi);
int ulogd_db_stop(struct ulogd_pluginstance *upi);
int ulogd_db_interp(struct ulogd_pluginstance *upi);
int ulogd_db_configure(struct ulogd_pluginstance *upi,
- struct ulogd_pluginstance_stack *stack);
-
+ struct ulogd_pluginstance_stack *stack);
+int ulogd_db_alloc_input_keys(struct ulogd_pluginstance *upi,
+ unsigned int num_keys, void *arg);
#endif
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 6312ac1649e2..5c10c787fb6a 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -92,7 +92,7 @@ static int get_columns_dbi(struct ulogd_pluginstance *upi)
struct dbi_instance *pi = (struct dbi_instance *) upi->private;
char *table = table_ce(upi->config_kset).u.string;
char query[256];
- unsigned int ui;
+ int rv;
if (!pi->dbh) {
ulogd_log(ULOGD_ERROR, "no database handle\n");
@@ -111,48 +111,39 @@ static int get_columns_dbi(struct ulogd_pluginstance *upi)
return -1;
}
- if (upi->input.keys)
- free(upi->input.keys);
+ rv = ulogd_db_alloc_input_keys(upi,
+ dbi_result_get_numfields(pi->result),
+ pi->result);
- upi->input.num_keys = dbi_result_get_numfields(pi->result);
- ulogd_log(ULOGD_DEBUG, "%u fields in table\n", upi->input.num_keys);
-
- upi->input.keys = calloc(upi->input.num_keys, sizeof(*upi->input.keys));
- if (!upi->input.keys) {
- upi->input.num_keys = 0;
- ulogd_log(ULOGD_ERROR, "ENOMEM\n");
- dbi_result_free(pi->result);
- return -ENOMEM;
- }
-
- for (ui=1; ui<=upi->input.num_keys; ui++) {
- const char *field_name = dbi_result_get_field_name(pi->result, ui);
- char *cp;
+ /* ID is a sequence */
+ if (!rv)
+ upi->input.keys[0].flags |= ULOGD_KEYF_INACTIVE;
- if (!field_name)
- break;
+ dbi_result_free(pi->result);
+ return rv;
+}
- snprintf(upi->input.keys[ui - 1].name,
- sizeof(upi->input.keys[ui - 1].name),
- "%s", field_name);
+static const char *
+get_column_dbi(void *vp, unsigned int i)
+{
+ dbi_result result = vp;
- /* down-case and replace all underscores with dots */
- for (cp = upi->input.keys[ui - 1].name; *cp; cp++) {
- if (*cp == '_')
- *cp = '.';
- else
- *cp = tolower(*cp);
- }
+ return dbi_result_get_field_name(result, i + 1);
+}
- DEBUGP("field '%s' found: ", upi->input.keys[ui - 1].name);
+static char *
+format_key_dbi(char *key)
+{
+ char *cp;
+
+ /* down-case and replace all underscores with dots */
+ for (cp = key; *cp; cp++) {
+ if (*cp == '_')
+ *cp = '.';
+ else
+ *cp = tolower(*cp);
}
-
- /* ID is a sequence */
- upi->input.keys[0].flags |= ULOGD_KEYF_INACTIVE;
-
- dbi_result_free(pi->result);
-
- return 0;
+ return key;
}
static int close_db_dbi(struct ulogd_pluginstance *upi)
@@ -270,6 +261,8 @@ static int execute_dbi(struct ulogd_pluginstance *upi,
static struct db_driver db_driver_dbi = {
.get_columns = &get_columns_dbi,
+ .get_column = &get_column_dbi,
+ .format_key = &format_key_dbi,
.open_db = &open_db_dbi,
.close_db = &close_db_dbi,
.escape_string = &escape_string_dbi,
diff --git a/output/mysql/ulogd_output_MYSQL.c b/output/mysql/ulogd_output_MYSQL.c
index 5891207d5990..55059f5c189e 100644
--- a/output/mysql/ulogd_output_MYSQL.c
+++ b/output/mysql/ulogd_output_MYSQL.c
@@ -94,13 +94,13 @@ static struct config_keyset kset_mysql = {
#define user_ce(x) ((x)->ces[DB_CE_NUM + 2])
#define pass_ce(x) ((x)->ces[DB_CE_NUM + 3])
#define port_ce(x) ((x)->ces[DB_CE_NUM + 4])
+
/* find out which columns the table has */
static int get_columns_mysql(struct ulogd_pluginstance *upi)
{
struct mysql_instance *mi = (struct mysql_instance *) upi->private;
MYSQL_RES *result;
- MYSQL_FIELD *field;
- int i;
+ int rv;
if (!mi->dbh) {
ulogd_log(ULOGD_ERROR, "no database handle\n");
@@ -121,38 +121,23 @@ static int get_columns_mysql(struct ulogd_pluginstance *upi)
* in case the core just calls ->configure() and then aborts (and thus
* never free()s the memory we allocate here. FIXME. */
- /* Cleanup before reconnect */
- if (upi->input.keys)
- free(upi->input.keys);
-
- upi->input.num_keys = mysql_num_fields(result);
- ulogd_log(ULOGD_DEBUG, "%u fields in table\n", upi->input.num_keys);
- upi->input.keys = calloc(upi->input.num_keys, sizeof(*upi->input.keys));
- if (!upi->input.keys) {
- upi->input.num_keys = 0;
- ulogd_log(ULOGD_ERROR, "ENOMEM\n");
- return -ENOMEM;
- }
+ rv = ulogd_db_alloc_input_keys(upi, mysql_num_fields(result), result);
- for (i = 0; (field = mysql_fetch_field(result)); i++) {
- char *underscore;
+ /* MySQL Auto increment ... ID :) */
+ if (!rv)
+ upi->input.keys[0].flags |= ULOGD_KEYF_INACTIVE;
- snprintf(upi->input.keys[i].name,
- sizeof(upi->input.keys[i].name),
- "%s", field->name);
+ mysql_free_result(result);
+ return rv;
+}
- /* replace all underscores with dots */
- for (underscore = upi->input.keys[i].name;
- (underscore = strchr(underscore, '_')); )
- *underscore = '.';
+static const char *
+get_column_mysql(void *vp, unsigned int i __attribute__ ((unused)))
+{
+ MYSQL_RES *result = vp;
+ MYSQL_FIELD *field = mysql_fetch_field(result);
- DEBUGP("field '%s' found\n", upi->input.keys[i].name);
- }
- /* MySQL Auto increment ... ID :) */
- upi->input.keys[0].flags |= ULOGD_KEYF_INACTIVE;
-
- mysql_free_result(result);
- return 0;
+ return field->name;
}
static int close_db_mysql(struct ulogd_pluginstance *upi)
@@ -243,6 +228,7 @@ static int execute_mysql(struct ulogd_pluginstance *upi,
static struct db_driver db_driver_mysql = {
.get_columns = &get_columns_mysql,
+ .get_column = &get_column_mysql,
.open_db = &open_db_mysql,
.close_db = &close_db_mysql,
.escape_string = &escape_string_mysql,
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index bc0eae7558b3..c5bbc966d66d 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -137,7 +137,7 @@ static int get_columns_pgsql(struct ulogd_pluginstance *upi)
char pgbuf[strlen(PGSQL_GETCOLUMN_TEMPLATE_SCHEMA)
+ strlen(table_ce(upi->config_kset).u.string)
+ strlen(pi->db_inst.schema) + 2];
- int i;
+ int rv;
if (!pi->dbh) {
ulogd_log(ULOGD_ERROR, "no database handle\n");
@@ -170,40 +170,22 @@ static int get_columns_pgsql(struct ulogd_pluginstance *upi)
return -1;
}
- if (upi->input.keys)
- free(upi->input.keys);
-
- upi->input.num_keys = PQntuples(pi->pgres);
- ulogd_log(ULOGD_DEBUG, "%u fields in table\n", upi->input.num_keys);
- upi->input.keys = calloc(upi->input.num_keys, sizeof(*upi->input.keys));
- if (!upi->input.keys) {
- upi->input.num_keys = 0;
- ulogd_log(ULOGD_ERROR, "ENOMEM\n");
- PQclear(pi->pgres);
- return -ENOMEM;
- }
-
- for (i = 0; i < PQntuples(pi->pgres); i++) {
- char *underscore;
-
- snprintf(upi->input.keys[i].name,
- sizeof(upi->input.keys[i].name),
- "%s", PQgetvalue(pi->pgres, i, 0));
-
- /* replace all underscores with dots */
- for (underscore = upi->input.keys[i].name;
- (underscore = strchr(underscore, '_')); )
- *underscore = '.';
-
- DEBUGP("field '%s' found\n", upi->input.keys[i].name);
- }
+ rv = ulogd_db_alloc_input_keys(upi, PQntuples(pi->pgres), pi->pgres);
/* ID (starting by '.') is a sequence */
- if (upi->input.keys[0].name[0] == '.')
+ if (!rv && upi->input.keys[0].name[0] == '.')
upi->input.keys[0].flags |= ULOGD_KEYF_INACTIVE;
PQclear(pi->pgres);
- return 0;
+ return rv;
+}
+
+static const char *
+get_column_pgsql(void *vp, unsigned int i)
+{
+ PGresult *pgres = vp;
+
+ return PQgetvalue(pgres, i, 0);
}
static int close_db_pgsql(struct ulogd_pluginstance *upi)
@@ -320,6 +302,7 @@ static int execute_pgsql(struct ulogd_pluginstance *upi,
static struct db_driver db_driver_pgsql = {
.get_columns = &get_columns_pgsql,
+ .get_column = &get_column_pgsql,
.open_db = &open_db_pgsql,
.close_db = &close_db_pgsql,
.escape_string = &escape_string_pgsql,
diff --git a/util/db.c b/util/db.c
index 6749079697dc..4ec0af2ea38a 100644
--- a/util/db.c
+++ b/util/db.c
@@ -644,3 +644,51 @@ void ulogd_db_signal(struct ulogd_pluginstance *upi, int signal)
break;
}
}
+
+static char *
+_format_key(char *key)
+{
+ char *cp = key;
+
+ /* replace all underscores with dots */
+ while ((cp = strchr(cp, '_')))
+ *cp = '.';
+
+ return key;
+}
+
+int
+ulogd_db_alloc_input_keys(struct ulogd_pluginstance *upi,
+ unsigned int num_keys, void *arg)
+{
+ struct db_instance *di = (struct db_instance *) &upi->private;
+ char *(*format_key)(char *) = di->driver->format_key ? : _format_key;
+ unsigned int i;
+
+ if (upi->input.keys)
+ free(upi->input.keys);
+
+ upi->input.num_keys = num_keys;
+ ulogd_log(ULOGD_DEBUG, "%u fields in table\n", upi->input.num_keys);
+ upi->input.keys = calloc(upi->input.num_keys, sizeof(upi->input.keys[0]));
+ if (!upi->input.keys) {
+ upi->input.num_keys = 0;
+ ulogd_log(ULOGD_ERROR, "ENOMEM\n");
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < num_keys; i++) {
+ const char *col = di->driver->get_column(arg, i);
+
+ if (!col)
+ break;
+
+ snprintf(upi->input.keys[i].name,
+ sizeof(upi->input.keys[i].name),
+ "%s", col);
+
+ format_key(upi->input.keys[i].name);
+ }
+
+ return 0;
+}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 15/34] db: reorganize source
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (13 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 14/34] output: de-duplicate allocation of input keys Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 16/34] db: use consistent integer return values to indicate errors Jeremy Sowden
` (19 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
The functions are rather haphazardly ordered and inconsistently named.
Group all function declarations first, followed by the public functions
(those with extern linkage) next, and then the static ones, grouping
those related to the back-log and ring-buffer separately. Extern
functions are all prefixed `ulogd_db_`, static ones are all prefixed
with a single underscore.
Rename a couple of `struct db_instance` variables for consistency.
No functional changes.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
util/db.c | 623 ++++++++++++++++++++++++++++--------------------------
1 file changed, 328 insertions(+), 295 deletions(-)
diff --git a/util/db.c b/util/db.c
index 4ec0af2ea38a..7a6401f73609 100644
--- a/util/db.c
+++ b/util/db.c
@@ -37,115 +37,30 @@
#include <ulogd/ulogd.h>
#include <ulogd/db.h>
-
/* generic db layer */
-static int __interp_db(struct ulogd_pluginstance *upi);
-
-/* this is a wrapper that just calls the current real
- * interp function */
-int ulogd_db_interp(struct ulogd_pluginstance *upi)
-{
- struct db_instance *dbi = (struct db_instance *) &upi->private;
- return dbi->interp(upi);
-}
-
-/* no connection, plugin disabled */
-static int disabled_interp_db(struct ulogd_pluginstance *upi)
-{
- return 0;
-}
-
-#define SQL_INSERTTEMPL "SELECT P(Y)"
-#define SQL_VALSIZE 100
-
-/* create the static part of our insert statement */
-static int sql_createstmt(struct ulogd_pluginstance *upi)
-{
- struct db_instance *mi = (struct db_instance *) upi->private;
- unsigned int size;
- unsigned int i;
- char *table = table_ce(upi->config_kset).u.string;
- char *procedure = procedure_ce(upi->config_kset).u.string;
-
- if (mi->stmt)
- free(mi->stmt);
-
- /* caclulate the size for the insert statement */
- size = strlen(SQL_INSERTTEMPL) + strlen(table);
-
- for (i = 0; i < upi->input.num_keys; i++) {
- if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
- continue;
- /* we need space for the key and a comma, as well as
- * enough space for the values */
- size += strlen(upi->input.keys[i].name) + 1 + SQL_VALSIZE;
- }
- size += strlen(procedure);
-
- ulogd_log(ULOGD_DEBUG, "allocating %u bytes for statement\n", size);
-
- mi->stmt = malloc(size);
- if (!mi->stmt) {
- ulogd_log(ULOGD_ERROR, "OOM!\n");
- return -ENOMEM;
- }
- mi->ring.length = size + 1;
-
- if (strncasecmp(procedure,"INSERT", strlen("INSERT")) == 0 &&
- (procedure[strlen("INSERT")] == '\0' ||
- procedure[strlen("INSERT")] == ' ')) {
- char *stmt_val = mi->stmt;
-
- if(procedure[6] == '\0') {
- /* procedure == "INSERT" */
- if (mi->schema)
- stmt_val += sprintf(stmt_val,
- "insert into %s.%s (",
- mi->schema, table);
- else
- stmt_val += sprintf(stmt_val,
- "insert into %s (", table);
- } else
- stmt_val += sprintf(stmt_val, "%s (", procedure);
-
- for (i = 0; i < upi->input.num_keys; i++) {
- char *underscore;
-
- if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
- continue;
-
- underscore = stmt_val;
-
- stmt_val += sprintf(stmt_val, "%s,",
- upi->input.keys[i].name);
-
- while ((underscore = strchr(underscore, '.')))
- *underscore = '_';
- }
- *(stmt_val - 1) = ')';
-
- sprintf(stmt_val, " values (");
- } else if (strncasecmp(procedure,"CALL", strlen("CALL")) == 0) {
- sprintf(mi->stmt, "CALL %s(", procedure);
- } else {
- sprintf(mi->stmt, "SELECT %s(", procedure);
-
- }
+static int _interp_db_init(struct ulogd_pluginstance *upi);
+static int _interp_db_main(struct ulogd_pluginstance *upi);
+static int _interp_db_disabled(struct ulogd_pluginstance *upi);
+static int _reconnect_db(struct ulogd_pluginstance *upi);
+static int _stop_db(struct ulogd_pluginstance *upi);
- mi->stmt_offset = strlen(mi->stmt);
+static char *_format_key(char *key);
+static int _create_sql_stmt(struct ulogd_pluginstance *upi);
+static void _bind_sql_stmt(struct ulogd_pluginstance *upi,
+ char *stmt);
- ulogd_log(ULOGD_DEBUG, "stmt='%s'\n", mi->stmt);
+static int _add_to_backlog(struct ulogd_pluginstance *upi,
+ const char *stmt, unsigned int len);
+static int _process_backlog(struct ulogd_pluginstance *upi);
- return 0;
-}
-
-static int _init_db(struct ulogd_pluginstance *upi);
-
-static void *__inject_thread(void *gdi);
+static int _add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di);
+static void *_process_ring(void *arg);
+static int _loop_reconnect_db(struct ulogd_pluginstance *upi);
-int ulogd_db_configure(struct ulogd_pluginstance *upi,
- struct ulogd_pluginstance_stack *stack)
+int
+ulogd_db_configure(struct ulogd_pluginstance *upi,
+ struct ulogd_pluginstance_stack *stack)
{
struct db_instance *di = (struct db_instance *) upi->private;
int ret;
@@ -199,7 +114,44 @@ int ulogd_db_configure(struct ulogd_pluginstance *upi,
return ret;
}
-int ulogd_db_start(struct ulogd_pluginstance *upi)
+int
+ulogd_db_alloc_input_keys(struct ulogd_pluginstance *upi,
+ unsigned int num_keys, void *arg)
+{
+ struct db_instance *di = (struct db_instance *) &upi->private;
+ char *(*format_key)(char *) = di->driver->format_key ? : _format_key;
+ unsigned int i;
+
+ if (upi->input.keys)
+ free(upi->input.keys);
+
+ upi->input.num_keys = num_keys;
+ ulogd_log(ULOGD_DEBUG, "%u fields in table\n", upi->input.num_keys);
+ upi->input.keys = calloc(upi->input.num_keys, sizeof(upi->input.keys[0]));
+ if (!upi->input.keys) {
+ upi->input.num_keys = 0;
+ ulogd_log(ULOGD_ERROR, "ENOMEM\n");
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < num_keys; i++) {
+ const char *col = di->driver->get_column(arg, i);
+
+ if (!col)
+ break;
+
+ snprintf(upi->input.keys[i].name,
+ sizeof(upi->input.keys[i].name),
+ "%s", col);
+
+ format_key(upi->input.keys[i].name);
+ }
+
+ return 0;
+}
+
+int
+ulogd_db_start(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) upi->private;
int ret;
@@ -211,7 +163,7 @@ int ulogd_db_start(struct ulogd_pluginstance *upi)
if (ret < 0)
return ret;
- ret = sql_createstmt(upi);
+ ret = _create_sql_stmt(upi);
if (ret < 0)
goto db_error;
@@ -239,12 +191,12 @@ int ulogd_db_start(struct ulogd_pluginstance *upi)
if (ret != 0)
goto cond_error;
/* create thread */
- ret = pthread_create(&di->db_thread_id, NULL, __inject_thread, upi);
+ ret = pthread_create(&di->db_thread_id, NULL, _process_ring, upi);
if (ret != 0)
goto mutex_error;
}
- di->interp = &_init_db;
+ di->interp = &_interp_db_init;
return ret;
@@ -259,30 +211,55 @@ db_error:
return ret;
}
-static int ulogd_db_instance_stop(struct ulogd_pluginstance *upi)
+/* this is a wrapper that just calls the current real interp function */
+int
+ulogd_db_interp(struct ulogd_pluginstance *upi)
{
- struct db_instance *di = (struct db_instance *) upi->private;
- ulogd_log(ULOGD_NOTICE, "stopping\n");
- di->driver->close_db(upi);
+ struct db_instance *di = (struct db_instance *) &upi->private;
+ return di->interp(upi);
+}
- /* try to free the buffer for insert statement */
- if (di->stmt) {
- free(di->stmt);
- di->stmt = NULL;
- }
- if (di->ring.size > 0) {
- pthread_cancel(di->db_thread_id);
- free(di->ring.ring);
- pthread_cond_destroy(&di->ring.cond);
- pthread_mutex_destroy(&di->ring.mutex);
- di->ring.ring = NULL;
+void
+ulogd_db_signal(struct ulogd_pluginstance *upi, int signal)
+{
+ struct db_instance *di = (struct db_instance *) &upi->private;
+
+ switch (signal) {
+ case SIGHUP:
+ if (!di->ring.size) {
+ /* reopen database connection */
+ _stop_db(upi);
+ ulogd_db_start(upi);
+ } else
+ ulogd_log(ULOGD_ERROR,
+ "No SIGHUP handling if ring buffer is used\n");
+ break;
+ case SIGTERM:
+ case SIGINT:
+ if (di->ring.size) {
+ int s = pthread_cancel(di->db_thread_id);
+ if (s != 0) {
+ ulogd_log(ULOGD_ERROR,
+ "Can't cancel injection thread\n");
+ break;
+ }
+ s = pthread_join(di->db_thread_id, NULL);
+ if (s != 0) {
+ ulogd_log(ULOGD_ERROR,
+ "Error waiting for injection thread"
+ "cancelation\n");
+ }
+ }
+ break;
+ default:
+ break;
}
- return 0;
}
-int ulogd_db_stop(struct ulogd_pluginstance *upi)
+int
+ulogd_db_stop(struct ulogd_pluginstance *upi)
{
- ulogd_db_instance_stop(upi);
+ _stop_db(upi);
/* try to free our dynamically allocated input key array */
if (upi->input.keys) {
@@ -293,8 +270,85 @@ int ulogd_db_stop(struct ulogd_pluginstance *upi)
return 0;
}
+/******************************************************************************/
-static int _init_reconnect(struct ulogd_pluginstance *upi)
+static int
+_interp_db_init(struct ulogd_pluginstance *upi)
+{
+ struct db_instance *di = (struct db_instance *) upi->private;
+
+ if (di->reconnect && di->reconnect > time(NULL)) {
+ /* store entry to backlog if it is active */
+ if (di->backlog_memcap && !di->backlog_full) {
+ _bind_sql_stmt(upi, di->stmt);
+ _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ }
+ return 0;
+ }
+
+ if (di->driver->open_db(upi)) {
+ ulogd_log(ULOGD_ERROR, "can't establish database connection\n");
+ if (di->backlog_memcap && !di->backlog_full) {
+ _bind_sql_stmt(upi, di->stmt);
+ _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ }
+ return _reconnect_db(upi);
+ }
+
+ /* enable 'real' logging */
+ di->interp = &_interp_db_main;
+
+ di->reconnect = 0;
+
+ /* call the interpreter function to actually write the
+ * log line that we wanted to write */
+ return _interp_db_main(upi);
+}
+
+/* our main output function, called by ulogd */
+static int
+_interp_db_main(struct ulogd_pluginstance *upi)
+{
+ struct db_instance *di = (struct db_instance *) &upi->private;
+
+ if (di->ring.size)
+ return _add_to_ring(upi, di);
+
+ _bind_sql_stmt(upi, di->stmt);
+
+ /* if backup log is not empty we add current query to it */
+ if (!llist_empty(&di->backlog)) {
+ int ret = _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ if (ret == 0)
+ return _process_backlog(upi);
+ else {
+ ret = _process_backlog(upi);
+ if (ret)
+ return ret;
+ /* try adding once the data to backlog */
+ return _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ }
+ }
+
+ if (di->driver->execute(upi, di->stmt, strlen(di->stmt)) < 0) {
+ _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ /* error occur, database connexion need to be closed */
+ di->driver->close_db(upi);
+ return _reconnect_db(upi);
+ }
+
+ return 0;
+}
+
+/* no connection, plugin disabled */
+static int
+_interp_db_disabled(struct ulogd_pluginstance *upi)
+{
+ return 0;
+}
+
+static int
+_reconnect_db(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) upi->private;
@@ -306,18 +360,137 @@ static int _init_reconnect(struct ulogd_pluginstance *upi)
"no connection to database, attempting to reconnect after %u seconds\n",
reconnect_ce(upi->config_kset).u.value);
di->reconnect += reconnect_ce(upi->config_kset).u.value;
- di->interp = &_init_db;
+ di->interp = &_interp_db_init;
return -1;
}
/* Disable plugin permanently */
ulogd_log(ULOGD_ERROR, "permanently disabling plugin\n");
- di->interp = &disabled_interp_db;
+ di->interp = &_interp_db_disabled;
return 0;
}
-static void __format_query_db(struct ulogd_pluginstance *upi, char *start)
+static int
+_stop_db(struct ulogd_pluginstance *upi)
+{
+ struct db_instance *di = (struct db_instance *) upi->private;
+ ulogd_log(ULOGD_NOTICE, "stopping\n");
+ di->driver->close_db(upi);
+
+ /* try to free the buffer for insert statement */
+ if (di->stmt) {
+ free(di->stmt);
+ di->stmt = NULL;
+ }
+ if (di->ring.size > 0) {
+ pthread_cancel(di->db_thread_id);
+ free(di->ring.ring);
+ pthread_cond_destroy(&di->ring.cond);
+ pthread_mutex_destroy(&di->ring.mutex);
+ di->ring.ring = NULL;
+ }
+ return 0;
+}
+
+static char *
+_format_key(char *key)
+{
+ char *cp = key;
+
+ /* replace all underscores with dots */
+ while ((cp = strchr(cp, '_')))
+ *cp = '.';
+
+ return key;
+}
+
+#define SQL_INSERTTEMPL "SELECT P(Y)"
+#define SQL_VALSIZE 100
+
+/* create the static part of our insert statement */
+static int
+_create_sql_stmt(struct ulogd_pluginstance *upi)
+{
+ struct db_instance *di = (struct db_instance *) upi->private;
+ unsigned int size;
+ unsigned int i;
+ char *table = table_ce(upi->config_kset).u.string;
+ char *procedure = procedure_ce(upi->config_kset).u.string;
+
+ if (di->stmt)
+ free(di->stmt);
+
+ /* caclulate the size for the insert statement */
+ size = strlen(SQL_INSERTTEMPL) + strlen(table);
+
+ for (i = 0; i < upi->input.num_keys; i++) {
+ if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
+ continue;
+ /* we need space for the key and a comma, as well as
+ * enough space for the values */
+ size += strlen(upi->input.keys[i].name) + 1 + SQL_VALSIZE;
+ }
+ size += strlen(procedure);
+
+ ulogd_log(ULOGD_DEBUG, "allocating %u bytes for statement\n", size);
+
+ di->stmt = malloc(size);
+ if (!di->stmt) {
+ ulogd_log(ULOGD_ERROR, "OOM!\n");
+ return -ENOMEM;
+ }
+ di->ring.length = size + 1;
+
+ if (strncasecmp(procedure,"INSERT", strlen("INSERT")) == 0 &&
+ (procedure[strlen("INSERT")] == '\0' ||
+ procedure[strlen("INSERT")] == ' ')) {
+ char *stmt_val = di->stmt;
+
+ if(procedure[6] == '\0') {
+ /* procedure == "INSERT" */
+ if (di->schema)
+ stmt_val += sprintf(stmt_val,
+ "insert into %s.%s (",
+ di->schema, table);
+ else
+ stmt_val += sprintf(stmt_val,
+ "insert into %s (", table);
+ } else
+ stmt_val += sprintf(stmt_val, "%s (", procedure);
+
+ for (i = 0; i < upi->input.num_keys; i++) {
+ char *underscore;
+
+ if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
+ continue;
+
+ underscore = stmt_val;
+
+ stmt_val += sprintf(stmt_val, "%s,",
+ upi->input.keys[i].name);
+
+ while ((underscore = strchr(underscore, '.')))
+ *underscore = '_';
+ }
+ *(stmt_val - 1) = ')';
+
+ sprintf(stmt_val, " values (");
+ } else if (strncasecmp(procedure,"CALL", strlen("CALL")) == 0) {
+ sprintf(di->stmt, "CALL %s(", procedure);
+ } else {
+ sprintf(di->stmt, "SELECT %s(", procedure);
+ }
+
+ di->stmt_offset = strlen(di->stmt);
+
+ ulogd_log(ULOGD_DEBUG, "stmt='%s'\n", di->stmt);
+
+ return 0;
+}
+
+static void
+_bind_sql_stmt(struct ulogd_pluginstance *upi, char *start)
{
struct db_instance *di = (struct db_instance *) &upi->private;
@@ -399,7 +572,11 @@ static void __format_query_db(struct ulogd_pluginstance *upi, char *start)
*(stmt_ins - 1) = ')';
}
-static int __add_to_backlog(struct ulogd_pluginstance *upi, const char *stmt, unsigned int len)
+/******************************************************************************/
+
+static int
+_add_to_backlog(struct ulogd_pluginstance *upi,
+ const char *stmt, unsigned int len)
{
struct db_instance *di = (struct db_instance *) &upi->private;
unsigned int query_size;
@@ -440,40 +617,8 @@ static int __add_to_backlog(struct ulogd_pluginstance *upi, const char *stmt, un
return 0;
}
-static int _init_db(struct ulogd_pluginstance *upi)
-{
- struct db_instance *di = (struct db_instance *) upi->private;
-
- if (di->reconnect && di->reconnect > time(NULL)) {
- /* store entry to backlog if it is active */
- if (di->backlog_memcap && !di->backlog_full) {
- __format_query_db(upi, di->stmt);
- __add_to_backlog(upi, di->stmt,
- strlen(di->stmt));
- }
- return 0;
- }
-
- if (di->driver->open_db(upi)) {
- ulogd_log(ULOGD_ERROR, "can't establish database connection\n");
- if (di->backlog_memcap && !di->backlog_full) {
- __format_query_db(upi, di->stmt);
- __add_to_backlog(upi, di->stmt, strlen(di->stmt));
- }
- return _init_reconnect(upi);
- }
-
- /* enable 'real' logging */
- di->interp = &__interp_db;
-
- di->reconnect = 0;
-
- /* call the interpreter function to actually write the
- * log line that we wanted to write */
- return __interp_db(upi);
-}
-
-static int __treat_backlog(struct ulogd_pluginstance *upi)
+static int
+_process_backlog(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
int i = di->backlog_oneshot;
@@ -488,7 +633,7 @@ static int __treat_backlog(struct ulogd_pluginstance *upi)
if (di->driver->execute(upi, query->stmt, query->len) < 0) {
/* error occur, database connexion need to be closed */
di->driver->close_db(upi);
- return _init_reconnect(upi);
+ return _reconnect_db(upi);
} else {
di->backlog_memusage -= sizeof(*query) + query->len + 1;
llist_del(&query->list);
@@ -501,7 +646,10 @@ static int __treat_backlog(struct ulogd_pluginstance *upi)
return 0;
}
-static int __add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di)
+/******************************************************************************/
+
+static int
+_add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di)
{
if (*di->ring.wr_place == RING_QUERY_READY) {
if (di->ring.full == 0) {
@@ -513,7 +661,7 @@ static int __add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di)
ulogd_log(ULOGD_NOTICE, "Recovered some place in ring\n");
di->ring.full = 0;
}
- __format_query_db(upi, di->ring.wr_place + 1);
+ _bind_sql_stmt(upi, di->ring.wr_place + 1);
*di->ring.wr_place = RING_QUERY_READY;
pthread_cond_signal(&di->ring.cond);
di->ring.wr_item ++;
@@ -525,55 +673,8 @@ static int __add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di)
return ULOGD_IRET_OK;
}
-/* our main output function, called by ulogd */
-static int __interp_db(struct ulogd_pluginstance *upi)
-{
- struct db_instance *di = (struct db_instance *) &upi->private;
-
- if (di->ring.size)
- return __add_to_ring(upi, di);
-
- __format_query_db(upi, di->stmt);
-
- /* if backup log is not empty we add current query to it */
- if (!llist_empty(&di->backlog)) {
- int ret = __add_to_backlog(upi, di->stmt, strlen(di->stmt));
- if (ret == 0)
- return __treat_backlog(upi);
- else {
- ret = __treat_backlog(upi);
- if (ret)
- return ret;
- /* try adding once the data to backlog */
- return __add_to_backlog(upi, di->stmt, strlen(di->stmt));
- }
- }
-
- if (di->driver->execute(upi, di->stmt, strlen(di->stmt)) < 0) {
- __add_to_backlog(upi, di->stmt, strlen(di->stmt));
- /* error occur, database connexion need to be closed */
- di->driver->close_db(upi);
- return _init_reconnect(upi);
- }
-
- return 0;
-}
-
-static int __loop_reconnect_db(struct ulogd_pluginstance * upi) {
- struct db_instance *di = (struct db_instance *) &upi->private;
-
- di->driver->close_db(upi);
- while (1) {
- if (di->driver->open_db(upi)) {
- sleep(1);
- } else {
- return 0;
- }
- }
- return 0;
-}
-
-static void *__inject_thread(void *gdi)
+static void *
+_process_ring(void *gdi)
{
struct ulogd_pluginstance *upi = gdi;
struct db_instance *di = (struct db_instance *) &upi->private;
@@ -587,11 +688,11 @@ static void *__inject_thread(void *gdi)
while (*wr_place == RING_QUERY_READY) {
if (di->driver->execute(upi, wr_place + 1,
strlen(wr_place + 1)) < 0) {
- if (__loop_reconnect_db(upi) != 0) {
+ if (_loop_reconnect_db(upi) != 0) {
/* loop has failed on unrecoverable error */
ulogd_log(ULOGD_ERROR,
"permanently disabling plugin\n");
- di->interp = &disabled_interp_db;
+ di->interp = &_interp_db_disabled;
return NULL;
} else /* try to re run query */
continue;
@@ -609,86 +710,18 @@ static void *__inject_thread(void *gdi)
return NULL;
}
-
-void ulogd_db_signal(struct ulogd_pluginstance *upi, int signal)
+static int
+_loop_reconnect_db(struct ulogd_pluginstance * upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
- switch (signal) {
- case SIGHUP:
- if (!di->ring.size) {
- /* reopen database connection */
- ulogd_db_instance_stop(upi);
- ulogd_db_start(upi);
- } else
- ulogd_log(ULOGD_ERROR,
- "No SIGHUP handling if ring buffer is used\n");
- break;
- case SIGTERM:
- case SIGINT:
- if (di->ring.size) {
- int s = pthread_cancel(di->db_thread_id);
- if (s != 0) {
- ulogd_log(ULOGD_ERROR,
- "Can't cancel injection thread\n");
- break;
- }
- s = pthread_join(di->db_thread_id, NULL);
- if (s != 0) {
- ulogd_log(ULOGD_ERROR,
- "Error waiting for injection thread"
- "cancelation\n");
- }
- }
- break;
- default:
- break;
- }
-}
-
-static char *
-_format_key(char *key)
-{
- char *cp = key;
-
- /* replace all underscores with dots */
- while ((cp = strchr(cp, '_')))
- *cp = '.';
-
- return key;
-}
-
-int
-ulogd_db_alloc_input_keys(struct ulogd_pluginstance *upi,
- unsigned int num_keys, void *arg)
-{
- struct db_instance *di = (struct db_instance *) &upi->private;
- char *(*format_key)(char *) = di->driver->format_key ? : _format_key;
- unsigned int i;
-
- if (upi->input.keys)
- free(upi->input.keys);
- upi->input.num_keys = num_keys;
- ulogd_log(ULOGD_DEBUG, "%u fields in table\n", upi->input.num_keys);
- upi->input.keys = calloc(upi->input.num_keys, sizeof(upi->input.keys[0]));
- if (!upi->input.keys) {
- upi->input.num_keys = 0;
- ulogd_log(ULOGD_ERROR, "ENOMEM\n");
- return -ENOMEM;
- }
-
- for (i = 0; i < num_keys; i++) {
- const char *col = di->driver->get_column(arg, i);
-
- if (!col)
- break;
-
- snprintf(upi->input.keys[i].name,
- sizeof(upi->input.keys[i].name),
- "%s", col);
-
- format_key(upi->input.keys[i].name);
+ di->driver->close_db(upi);
+ while (1) {
+ if (di->driver->open_db(upi)) {
+ sleep(1);
+ } else {
+ return 0;
+ }
}
-
return 0;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 16/34] db: use consistent integer return values to indicate errors
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (14 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 15/34] db: reorganize source Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 17/34] db: change return type of two functions to `void` Jeremy Sowden
` (18 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Internally, we use `-1` for error and `0` for success. The `interp`
functions that return to `ulogd_propagate_results` return
`ULOGD_IRET_ERR` and `ULOGD_IRET_OK`.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/dbi/ulogd_output_DBI.c | 2 +-
output/pgsql/ulogd_output_PGSQL.c | 8 ++---
util/db.c | 59 +++++++++++++++++--------------
3 files changed, 38 insertions(+), 31 deletions(-)
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 5c10c787fb6a..3b8d8c325131 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -96,7 +96,7 @@ static int get_columns_dbi(struct ulogd_pluginstance *upi)
if (!pi->dbh) {
ulogd_log(ULOGD_ERROR, "no database handle\n");
- return 1;
+ return -1;
}
snprintf(query, 256, "SELECT * FROM %s", table);
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index c5bbc966d66d..70a4d48459ff 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -92,7 +92,7 @@ static int pgsql_namespace(struct ulogd_pluginstance *upi)
strlen(schema_ce(upi->config_kset).u.string) + 1];
if (!pi->dbh)
- return 1;
+ return -1;
sprintf(pgbuf, PGSQL_HAVE_NAMESPACE_TEMPLATE,
schema_ce(upi->config_kset).u.string);
@@ -101,7 +101,7 @@ static int pgsql_namespace(struct ulogd_pluginstance *upi)
pi->pgres = PQexec(pi->dbh, pgbuf);
if (!pi->pgres) {
ulogd_log(ULOGD_DEBUG, "\n result false");
- return 1;
+ return -1;
}
if (PQresultStatus(pi->pgres) == PGRES_TUPLES_OK) {
@@ -141,7 +141,7 @@ static int get_columns_pgsql(struct ulogd_pluginstance *upi)
if (!pi->dbh) {
ulogd_log(ULOGD_ERROR, "no database handle\n");
- return 1;
+ return -1;
}
if (pi->db_inst.schema) {
@@ -228,7 +228,7 @@ static int open_db_pgsql(struct ulogd_pluginstance *upi)
cp = connstr = malloc(len);
if (!connstr)
- return -ENOMEM;
+ return -1;
if (server[0])
cp += sprintf(cp, "host=%s ", server);
diff --git a/util/db.c b/util/db.c
index 7a6401f73609..cd9ebef077dd 100644
--- a/util/db.c
+++ b/util/db.c
@@ -111,7 +111,7 @@ ulogd_db_configure(struct ulogd_pluginstance *upi,
di->backlog_full = 0;
}
- return ret;
+ return 0;
}
int
@@ -131,7 +131,7 @@ ulogd_db_alloc_input_keys(struct ulogd_pluginstance *upi,
if (!upi->input.keys) {
upi->input.num_keys = 0;
ulogd_log(ULOGD_ERROR, "ENOMEM\n");
- return -ENOMEM;
+ return -1;
}
for (i = 0; i < num_keys; i++) {
@@ -170,10 +170,8 @@ ulogd_db_start(struct ulogd_pluginstance *upi)
if (di->ring.size > 0) {
/* allocate */
di->ring.ring = calloc(di->ring.size, sizeof(char) * di->ring.length);
- if (di->ring.ring == NULL) {
- ret = -1;
+ if (di->ring.ring == NULL)
goto db_error;
- }
di->ring.wr_place = di->ring.ring;
ulogd_log(ULOGD_NOTICE,
"Allocating %d elements of size %d for ring\n",
@@ -198,7 +196,7 @@ ulogd_db_start(struct ulogd_pluginstance *upi)
di->interp = &_interp_db_init;
- return ret;
+ return 0;
mutex_error:
pthread_mutex_destroy(&di->ring.mutex);
@@ -208,7 +206,7 @@ alloc_error:
free(di->ring.ring);
db_error:
di->driver->close_db(upi);
- return ret;
+ return -1;
}
/* this is a wrapper that just calls the current real interp function */
@@ -283,16 +281,18 @@ _interp_db_init(struct ulogd_pluginstance *upi)
_bind_sql_stmt(upi, di->stmt);
_add_to_backlog(upi, di->stmt, strlen(di->stmt));
}
- return 0;
+ return ULOGD_IRET_OK;
}
- if (di->driver->open_db(upi)) {
+ if (di->driver->open_db(upi) < 0) {
ulogd_log(ULOGD_ERROR, "can't establish database connection\n");
if (di->backlog_memcap && !di->backlog_full) {
_bind_sql_stmt(upi, di->stmt);
_add_to_backlog(upi, di->stmt, strlen(di->stmt));
}
- return _reconnect_db(upi);
+ if (_reconnect_db(upi) < 0)
+ return ULOGD_IRET_ERR;
+ return ULOGD_IRET_OK;
}
/* enable 'real' logging */
@@ -311,40 +311,48 @@ _interp_db_main(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
- if (di->ring.size)
- return _add_to_ring(upi, di);
+ if (di->ring.size) {
+ if (_add_to_ring(upi, di) < 0)
+ return ULOGD_IRET_ERR;
+ return ULOGD_IRET_OK;
+ }
_bind_sql_stmt(upi, di->stmt);
/* if backup log is not empty we add current query to it */
if (!llist_empty(&di->backlog)) {
int ret = _add_to_backlog(upi, di->stmt, strlen(di->stmt));
- if (ret == 0)
- return _process_backlog(upi);
- else {
- ret = _process_backlog(upi);
- if (ret)
- return ret;
- /* try adding once the data to backlog */
- return _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ if (ret == 0) {
+ if (_process_backlog(upi) < 0)
+ return ULOGD_IRET_ERR;
+ return ULOGD_IRET_OK;
}
+ ret = _process_backlog(upi);
+ if (ret < 0)
+ return ULOGD_IRET_ERR;
+ /* try adding once the data to backlog */
+ if (_add_to_backlog(upi, di->stmt, strlen(di->stmt)) < 0)
+ return ULOGD_IRET_ERR;
+ return ULOGD_IRET_OK;
}
if (di->driver->execute(upi, di->stmt, strlen(di->stmt)) < 0) {
_add_to_backlog(upi, di->stmt, strlen(di->stmt));
/* error occur, database connexion need to be closed */
di->driver->close_db(upi);
- return _reconnect_db(upi);
+ if (_reconnect_db(upi) < 0)
+ return ULOGD_IRET_ERR;
+ return ULOGD_IRET_OK;
}
- return 0;
+ return ULOGD_IRET_OK;
}
/* no connection, plugin disabled */
static int
_interp_db_disabled(struct ulogd_pluginstance *upi)
{
- return 0;
+ return ULOGD_IRET_OK;
}
static int
@@ -367,7 +375,6 @@ _reconnect_db(struct ulogd_pluginstance *upi)
/* Disable plugin permanently */
ulogd_log(ULOGD_ERROR, "permanently disabling plugin\n");
di->interp = &_interp_db_disabled;
-
return 0;
}
@@ -438,7 +445,7 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
di->stmt = malloc(size);
if (!di->stmt) {
ulogd_log(ULOGD_ERROR, "OOM!\n");
- return -ENOMEM;
+ return -1;
}
di->ring.length = size + 1;
@@ -717,7 +724,7 @@ _loop_reconnect_db(struct ulogd_pluginstance * upi)
di->driver->close_db(upi);
while (1) {
- if (di->driver->open_db(upi)) {
+ if (di->driver->open_db(upi) < 0) {
sleep(1);
} else {
return 0;
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 17/34] db: change return type of two functions to `void`
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (15 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 16/34] db: use consistent integer return values to indicate errors Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 18/34] db: open-code `_loop_reconnect_db` Jeremy Sowden
` (17 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
`_stop_db` and `_loop_reconnect_db` always return successfully. Change
the return types and remove error-checking from callers.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
util/db.c | 24 +++++++++---------------
1 file changed, 9 insertions(+), 15 deletions(-)
diff --git a/util/db.c b/util/db.c
index cd9ebef077dd..0514a87e0759 100644
--- a/util/db.c
+++ b/util/db.c
@@ -43,7 +43,7 @@ static int _interp_db_init(struct ulogd_pluginstance *upi);
static int _interp_db_main(struct ulogd_pluginstance *upi);
static int _interp_db_disabled(struct ulogd_pluginstance *upi);
static int _reconnect_db(struct ulogd_pluginstance *upi);
-static int _stop_db(struct ulogd_pluginstance *upi);
+static void _stop_db(struct ulogd_pluginstance *upi);
static char *_format_key(char *key);
static int _create_sql_stmt(struct ulogd_pluginstance *upi);
@@ -56,7 +56,7 @@ static int _process_backlog(struct ulogd_pluginstance *upi);
static int _add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di);
static void *_process_ring(void *arg);
-static int _loop_reconnect_db(struct ulogd_pluginstance *upi);
+static void _loop_reconnect_db(struct ulogd_pluginstance *upi);
int
ulogd_db_configure(struct ulogd_pluginstance *upi,
@@ -378,7 +378,7 @@ _reconnect_db(struct ulogd_pluginstance *upi)
return 0;
}
-static int
+static void
_stop_db(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) upi->private;
@@ -397,7 +397,6 @@ _stop_db(struct ulogd_pluginstance *upi)
pthread_mutex_destroy(&di->ring.mutex);
di->ring.ring = NULL;
}
- return 0;
}
static char *
@@ -695,14 +694,10 @@ _process_ring(void *gdi)
while (*wr_place == RING_QUERY_READY) {
if (di->driver->execute(upi, wr_place + 1,
strlen(wr_place + 1)) < 0) {
- if (_loop_reconnect_db(upi) != 0) {
- /* loop has failed on unrecoverable error */
- ulogd_log(ULOGD_ERROR,
- "permanently disabling plugin\n");
- di->interp = &_interp_db_disabled;
- return NULL;
- } else /* try to re run query */
- continue;
+ _loop_reconnect_db(upi);
+ /* try to re run query */
+ continue;
+
}
*wr_place = RING_NO_QUERY;
di->ring.rd_item++;
@@ -717,7 +712,7 @@ _process_ring(void *gdi)
return NULL;
}
-static int
+static void
_loop_reconnect_db(struct ulogd_pluginstance * upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
@@ -727,8 +722,7 @@ _loop_reconnect_db(struct ulogd_pluginstance * upi)
if (di->driver->open_db(upi) < 0) {
sleep(1);
} else {
- return 0;
+ return;
}
}
- return 0;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 18/34] db: open-code `_loop_reconnect_db`
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (16 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 17/34] db: change return type of two functions to `void` Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 19/34] db: improve calculation of sql statement length Jeremy Sowden
` (16 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Now that `_loop_reconnect_db` returns `void`, it can be reduced to
`while (open_db fails) sleep 1 second`, so remove the function altogther
and open code the simplified while-loop.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
util/db.c | 21 +++------------------
1 file changed, 3 insertions(+), 18 deletions(-)
diff --git a/util/db.c b/util/db.c
index 0514a87e0759..afa86a3f137b 100644
--- a/util/db.c
+++ b/util/db.c
@@ -56,7 +56,6 @@ static int _process_backlog(struct ulogd_pluginstance *upi);
static int _add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di);
static void *_process_ring(void *arg);
-static void _loop_reconnect_db(struct ulogd_pluginstance *upi);
int
ulogd_db_configure(struct ulogd_pluginstance *upi,
@@ -694,10 +693,11 @@ _process_ring(void *gdi)
while (*wr_place == RING_QUERY_READY) {
if (di->driver->execute(upi, wr_place + 1,
strlen(wr_place + 1)) < 0) {
- _loop_reconnect_db(upi);
+ di->driver->close_db(upi);
+ while (di->driver->open_db(upi) < 0)
+ sleep(1);
/* try to re run query */
continue;
-
}
*wr_place = RING_NO_QUERY;
di->ring.rd_item++;
@@ -711,18 +711,3 @@ _process_ring(void *gdi)
return NULL;
}
-
-static void
-_loop_reconnect_db(struct ulogd_pluginstance * upi)
-{
- struct db_instance *di = (struct db_instance *) &upi->private;
-
- di->driver->close_db(upi);
- while (1) {
- if (di->driver->open_db(upi) < 0) {
- sleep(1);
- } else {
- return;
- }
- }
-}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 19/34] db: improve calculation of sql statement length
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (17 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 18/34] db: open-code `_loop_reconnect_db` Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 20/34] db: refactor configuration Jeremy Sowden
` (15 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Currently, we calculate the space required from a prefix which doesn't
correspond to anything we use, space for each key, although the SQL
statement may not contain the keys, and a fixed amount for each value.
However, we can use snprintf to tighten up the estimate by using
`snprintf(NULL, 0, ...)` to tell us how much room we actually need for
the parts we know in advance.
Rename a couple of variables and replace `strlen` with `sizeof` where
appropriate.
Sort included headers.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
util/db.c | 149 ++++++++++++++++++++++++++++++++++++++----------------
1 file changed, 105 insertions(+), 44 deletions(-)
diff --git a/util/db.c b/util/db.c
index afa86a3f137b..ce1273638ae0 100644
--- a/util/db.c
+++ b/util/db.c
@@ -24,15 +24,16 @@
*
*/
-#include <unistd.h>
-#include <stdlib.h>
-#include <string.h>
#include <errno.h>
-#include <netinet/in.h>
-#include <arpa/inet.h>
-#include <time.h>
#include <inttypes.h>
#include <pthread.h>
+#include <stdbool.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+#include <arpa/inet.h>
+#include <netinet/in.h>
#include <ulogd/ulogd.h>
#include <ulogd/db.h>
@@ -47,6 +48,10 @@ static void _stop_db(struct ulogd_pluginstance *upi);
static char *_format_key(char *key);
static int _create_sql_stmt(struct ulogd_pluginstance *upi);
+static unsigned int _calc_sql_stmt_size(const char *procedure,
+ const char *schema, const char *table,
+ struct ulogd_key *keys,
+ unsigned int num_keys);
static void _bind_sql_stmt(struct ulogd_pluginstance *upi,
char *stmt);
@@ -410,33 +415,23 @@ _format_key(char *key)
return key;
}
-#define SQL_INSERTTEMPL "SELECT P(Y)"
-#define SQL_VALSIZE 100
-
/* create the static part of our insert statement */
static int
_create_sql_stmt(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) upi->private;
+ char *procedure = procedure_ce(upi->config_kset).u.string;
+ char *table = table_ce(upi->config_kset).u.string;
unsigned int size;
unsigned int i;
- char *table = table_ce(upi->config_kset).u.string;
- char *procedure = procedure_ce(upi->config_kset).u.string;
+ char *stmtp;
if (di->stmt)
free(di->stmt);
- /* caclulate the size for the insert statement */
- size = strlen(SQL_INSERTTEMPL) + strlen(table);
-
- for (i = 0; i < upi->input.num_keys; i++) {
- if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
- continue;
- /* we need space for the key and a comma, as well as
- * enough space for the values */
- size += strlen(upi->input.keys[i].name) + 1 + SQL_VALSIZE;
- }
- size += strlen(procedure);
+ /* calculate the size for the insert statement */
+ size = _calc_sql_stmt_size(procedure, di->schema, table,
+ upi->input.keys, upi->input.num_keys);
ulogd_log(ULOGD_DEBUG, "allocating %u bytes for statement\n", size);
@@ -447,22 +442,22 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
}
di->ring.length = size + 1;
- if (strncasecmp(procedure,"INSERT", strlen("INSERT")) == 0 &&
- (procedure[strlen("INSERT")] == '\0' ||
- procedure[strlen("INSERT")] == ' ')) {
- char *stmt_val = di->stmt;
+ stmtp = di->stmt;
+
+ if (strncasecmp(procedure, "INSERT", sizeof("INSERT") - 1) == 0 &&
+ (procedure[sizeof("INSERT") - 1] == '\0' ||
+ procedure[sizeof("INSERT") - 1] == ' ')) {
- if(procedure[6] == '\0') {
+ if(procedure[sizeof("INSERT") - 1] == '\0') {
/* procedure == "INSERT" */
if (di->schema)
- stmt_val += sprintf(stmt_val,
- "insert into %s.%s (",
- di->schema, table);
+ stmtp += sprintf(stmtp, "insert into %s.%s (",
+ di->schema, table);
else
- stmt_val += sprintf(stmt_val,
- "insert into %s (", table);
+ stmtp += sprintf(stmtp, "insert into %s (",
+ table);
} else
- stmt_val += sprintf(stmt_val, "%s (", procedure);
+ stmtp += sprintf(stmtp, "%s (", procedure);
for (i = 0; i < upi->input.num_keys; i++) {
char *underscore;
@@ -470,30 +465,96 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
continue;
- underscore = stmt_val;
+ underscore = stmtp;
- stmt_val += sprintf(stmt_val, "%s,",
- upi->input.keys[i].name);
+ stmtp += sprintf(stmtp, "%s,",
+ upi->input.keys[i].name);
while ((underscore = strchr(underscore, '.')))
*underscore = '_';
}
- *(stmt_val - 1) = ')';
+ stmtp --;
- sprintf(stmt_val, " values (");
- } else if (strncasecmp(procedure,"CALL", strlen("CALL")) == 0) {
- sprintf(di->stmt, "CALL %s(", procedure);
- } else {
- sprintf(di->stmt, "SELECT %s(", procedure);
- }
+ stmtp += sprintf(stmtp, ") values (");
+
+ } else if (strncasecmp(procedure, "CALL", sizeof("CALL") - 1) == 0)
+ stmtp += sprintf(stmtp, "CALL %s(", procedure);
+ else
+ stmtp += sprintf(stmtp, "SELECT %s(", procedure);
- di->stmt_offset = strlen(di->stmt);
+ di->stmt_offset = stmtp - di->stmt;
ulogd_log(ULOGD_DEBUG, "stmt='%s'\n", di->stmt);
return 0;
}
+#define SQL_VALSIZE 100
+
+static unsigned int
+_calc_sql_stmt_size(const char *procedure,
+ const char *schema, const char *table,
+ struct ulogd_key *keys, unsigned int num_keys)
+{
+ unsigned int i, size = 0;
+ bool include_keys;
+
+ /* Fixed size bit */
+
+ if (strncasecmp(procedure, "INSERT", sizeof("INSERT") - 1) == 0 &&
+ (procedure[sizeof("INSERT") - 1] == '\0' ||
+ procedure[sizeof("INSERT") - 1] == ' ')) {
+
+ /* insert into t (k0, k1, ...) values (v0, v1, ...) */
+
+ if(procedure[sizeof("INSERT") - 1] == '\0') {
+ /* procedure == "INSERT" */
+ if (schema)
+ size += snprintf(NULL, 0,
+ "insert into %s.%s (",
+ schema, table);
+ else
+ size += snprintf(NULL, 0,
+ "insert into %s (", table);
+ } else
+ size += snprintf(NULL, 0, "%s (", procedure);
+
+ size += snprintf(NULL, 0, ") values (");
+
+ include_keys = true;
+
+ } else {
+
+ /* `call p(v0, v1, ...)` or `select p(v0, v1, ...)` */
+
+ if (strncasecmp(procedure, "CALL", sizeof("CALL") - 1) == 0)
+ size += snprintf(NULL, 0, "CALL %s(", procedure);
+ else
+ size += snprintf(NULL, 0, "SELECT %s(", procedure);
+
+ include_keys = false;
+
+ }
+
+ /* Per-field bits.
+ *
+ * We need space for the value and a comma or the closing parenthesis.
+ * We may also need space for the key and a comma.
+ */
+
+ for (i = 0; i < num_keys; i++) {
+ if (keys[i].flags & ULOGD_KEYF_INACTIVE)
+ continue;
+ if (include_keys)
+ size += strlen(keys[i].name) + 1;
+ size += SQL_VALSIZE + 1;
+ }
+
+ size++; /* Allow for the final NUL */
+
+ return size;
+}
+
static void
_bind_sql_stmt(struct ulogd_pluginstance *upi, char *start)
{
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 20/34] db: refactor configuration
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (18 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 19/34] db: improve calculation of sql statement length Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 21/34] db: refactor ring-buffer initialization Jeremy Sowden
` (14 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Move DB, back-log and ring-buffer config code into separate functions.
No functional changes.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
util/db.c | 118 ++++++++++++++++++++++++++++++++++++++----------------
1 file changed, 83 insertions(+), 35 deletions(-)
diff --git a/util/db.c b/util/db.c
index ce1273638ae0..a633257b5929 100644
--- a/util/db.c
+++ b/util/db.c
@@ -40,6 +40,7 @@
/* generic db layer */
+static int _configure_db(struct ulogd_pluginstance *upi);
static int _interp_db_init(struct ulogd_pluginstance *upi);
static int _interp_db_main(struct ulogd_pluginstance *upi);
static int _interp_db_disabled(struct ulogd_pluginstance *upi);
@@ -55,10 +56,12 @@ static unsigned int _calc_sql_stmt_size(const char *procedure,
static void _bind_sql_stmt(struct ulogd_pluginstance *upi,
char *stmt);
+static int _configure_backlog(struct ulogd_pluginstance *upi);
static int _add_to_backlog(struct ulogd_pluginstance *upi,
const char *stmt, unsigned int len);
static int _process_backlog(struct ulogd_pluginstance *upi);
+static int _configure_ring(struct ulogd_pluginstance *upi);
static int _add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di);
static void *_process_ring(void *arg);
@@ -66,53 +69,23 @@ int
ulogd_db_configure(struct ulogd_pluginstance *upi,
struct ulogd_pluginstance_stack *stack)
{
- struct db_instance *di = (struct db_instance *) upi->private;
int ret;
ulogd_log(ULOGD_NOTICE, "(re)configuring\n");
- /* First: Parse configuration file section for this instance */
- ret = config_parse_file(upi->id, upi->config_kset);
+ ret = _configure_db(upi);
if (ret < 0) {
- ulogd_log(ULOGD_ERROR, "error parsing config file\n");
return ret;
}
- /* Second: Open Database */
- ret = di->driver->open_db(upi);
+ ret = _configure_backlog(upi);
if (ret < 0) {
- ulogd_log(ULOGD_ERROR, "error in open_db\n");
return ret;
}
- /* Third: Determine required input keys for given table */
- ret = di->driver->get_columns(upi);
- if (ret < 0)
- ulogd_log(ULOGD_ERROR, "error in get_columns\n");
-
- /* Close database, since ulogd core could just call configure
- * but abort during input key resolving routines. configure
- * doesn't have a destructor... */
- di->driver->close_db(upi);
-
- INIT_LLIST_HEAD(&di->backlog);
- di->backlog_memusage = 0;
-
- di->ring.size = ringsize_ce(upi->config_kset).u.value;
- di->backlog_memcap = backlog_memcap_ce(upi->config_kset).u.value;
-
- if (di->ring.size && di->backlog_memcap) {
- ulogd_log(ULOGD_ERROR, "Ring buffer has precedence over backlog\n");
- di->backlog_memcap = 0;
- } else if (di->backlog_memcap > 0) {
- di->backlog_oneshot = backlog_oneshot_ce(upi->config_kset).u.value;
- if (di->backlog_oneshot <= 2) {
- ulogd_log(ULOGD_ERROR,
- "backlog_oneshot_requests must be > 2 to hope"
- " cleaning. Setting it to 3.\n");
- di->backlog_oneshot = 3;
- }
- di->backlog_full = 0;
+ ret = _configure_ring(upi);
+ if (ret < 0) {
+ return ret;
}
return 0;
@@ -274,6 +247,40 @@ ulogd_db_stop(struct ulogd_pluginstance *upi)
/******************************************************************************/
+static int
+_configure_db(struct ulogd_pluginstance *upi)
+{
+ struct db_instance *di = (struct db_instance *) upi->private;
+ int ret;
+
+ /* First: Parse configuration file section for this instance */
+ ret = config_parse_file(upi->id, upi->config_kset);
+ if (ret < 0) {
+ ulogd_log(ULOGD_ERROR, "error parsing config file\n");
+ return ret;
+ }
+
+ /* Second: Open Database */
+ ret = di->driver->open_db(upi);
+ if (ret < 0) {
+ ulogd_log(ULOGD_ERROR, "error in open_db\n");
+ return ret;
+ }
+
+ /* Third: Determine required input keys for given table */
+ ret = di->driver->get_columns(upi);
+ if (ret < 0)
+ ulogd_log(ULOGD_ERROR, "error in get_columns\n");
+
+ /* Close database, since ulogd core could just call configure
+ * but abort during input key resolving routines. configure
+ * doesn't have a destructor... */
+ di->driver->close_db(upi);
+
+ return ret;
+
+}
+
static int
_interp_db_init(struct ulogd_pluginstance *upi)
{
@@ -640,6 +647,37 @@ _bind_sql_stmt(struct ulogd_pluginstance *upi, char *start)
/******************************************************************************/
+static int
+_configure_backlog(struct ulogd_pluginstance *upi)
+{
+ struct db_instance *di = (struct db_instance *) &upi->private;
+
+ INIT_LLIST_HEAD(&di->backlog);
+
+ di->backlog_memusage = 0;
+ di->backlog_memcap = backlog_memcap_ce(upi->config_kset).u.value;
+ di->backlog_full = 0;
+
+ if (di->backlog_memcap == 0)
+ return 0;
+
+ if (ringsize_ce(upi->config_kset).u.value) {
+ ulogd_log(ULOGD_ERROR,
+ "Ring buffer has precedence over backlog\n");
+ di->backlog_memcap = 0;
+ return 0;
+ }
+
+ di->backlog_oneshot = backlog_oneshot_ce(upi->config_kset).u.value;
+ if (di->backlog_oneshot <= 2) {
+ ulogd_log(ULOGD_ERROR,
+ "backlog_oneshot_requests must be > 2 to be effective. Setting it to 3.\n");
+ di->backlog_oneshot = 3;
+ }
+
+ return 0;
+}
+
static int
_add_to_backlog(struct ulogd_pluginstance *upi,
const char *stmt, unsigned int len)
@@ -714,6 +752,16 @@ _process_backlog(struct ulogd_pluginstance *upi)
/******************************************************************************/
+static int
+_configure_ring(struct ulogd_pluginstance *upi)
+{
+ struct db_instance *di = (struct db_instance *) &upi->private;
+
+ di->ring.size = ringsize_ce(upi->config_kset).u.value;
+
+ return 0;
+}
+
static int
_add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di)
{
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 21/34] db: refactor ring-buffer initialization
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (19 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 20/34] db: refactor configuration Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 22/34] db: refactor ring-buffer Jeremy Sowden
` (13 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Move the code to initialize and start the thread that manages the ring-buffer
into a separate function.
No functional changes.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
util/db.c | 87 +++++++++++++++++++++++++++++++++----------------------
1 file changed, 53 insertions(+), 34 deletions(-)
diff --git a/util/db.c b/util/db.c
index a633257b5929..ee6dfb6b5a2a 100644
--- a/util/db.c
+++ b/util/db.c
@@ -62,6 +62,7 @@ static int _add_to_backlog(struct ulogd_pluginstance *upi,
static int _process_backlog(struct ulogd_pluginstance *upi);
static int _configure_ring(struct ulogd_pluginstance *upi);
+static int _start_ring(struct ulogd_pluginstance *upi);
static int _add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di);
static void *_process_ring(void *arg);
@@ -132,7 +133,6 @@ ulogd_db_start(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) upi->private;
int ret;
- unsigned int i;
ulogd_log(ULOGD_NOTICE, "starting\n");
@@ -144,43 +144,13 @@ ulogd_db_start(struct ulogd_pluginstance *upi)
if (ret < 0)
goto db_error;
- if (di->ring.size > 0) {
- /* allocate */
- di->ring.ring = calloc(di->ring.size, sizeof(char) * di->ring.length);
- if (di->ring.ring == NULL)
- goto db_error;
- di->ring.wr_place = di->ring.ring;
- ulogd_log(ULOGD_NOTICE,
- "Allocating %d elements of size %d for ring\n",
- di->ring.size, di->ring.length);
- /* init start of query for each element */
- for(i = 0; i < di->ring.size; i++) {
- strcpy(di->ring.ring + di->ring.length * i + 1,
- di->stmt);
- }
- /* init cond & mutex */
- ret = pthread_cond_init(&di->ring.cond, NULL);
- if (ret != 0)
- goto alloc_error;
- ret = pthread_mutex_init(&di->ring.mutex, NULL);
- if (ret != 0)
- goto cond_error;
- /* create thread */
- ret = pthread_create(&di->db_thread_id, NULL, _process_ring, upi);
- if (ret != 0)
- goto mutex_error;
- }
+ ret = _start_ring(upi);
+ if (ret < 0)
+ goto db_error;
di->interp = &_interp_db_init;
-
return 0;
-mutex_error:
- pthread_mutex_destroy(&di->ring.mutex);
-cond_error:
- pthread_cond_destroy(&di->ring.cond);
-alloc_error:
- free(di->ring.ring);
db_error:
di->driver->close_db(upi);
return -1;
@@ -762,6 +732,55 @@ _configure_ring(struct ulogd_pluginstance *upi)
return 0;
}
+static int
+_start_ring(struct ulogd_pluginstance *upi)
+{
+ struct db_instance *di = (struct db_instance *) &upi->private;
+ unsigned int i;
+ int ret;
+
+ if (di->ring.size == 0)
+ return 0;
+
+ /* allocate */
+ di->ring.ring = calloc(di->ring.size, sizeof(char) * di->ring.length);
+ if (di->ring.ring == NULL)
+ return -1;
+ di->ring.wr_place = di->ring.ring;
+ ulogd_log(ULOGD_NOTICE,
+ "Allocating %d elements of size %d for ring\n",
+ di->ring.size, di->ring.length);
+
+ /* init start of query for each element */
+ for(i = 0; i < di->ring.size; i++)
+ strcpy(di->ring.ring + di->ring.length * i + 1,
+ di->stmt);
+
+ /* init cond & mutex */
+ ret = pthread_cond_init(&di->ring.cond, NULL);
+ if (ret != 0)
+ goto alloc_error;
+ ret = pthread_mutex_init(&di->ring.mutex, NULL);
+ if (ret != 0)
+ goto cond_error;
+
+ /* create thread */
+ ret = pthread_create(&di->db_thread_id, NULL, _process_ring, upi);
+ if (ret != 0)
+ goto mutex_error;
+
+ return 0;
+
+mutex_error:
+ pthread_mutex_destroy(&di->ring.mutex);
+cond_error:
+ pthread_cond_destroy(&di->ring.cond);
+alloc_error:
+ free(di->ring.ring);
+
+ return -1;
+}
+
static int
_add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di)
{
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 22/34] db: refactor ring-buffer
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (20 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 21/34] db: refactor ring-buffer initialization Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 23/34] db: refactor backlog Jeremy Sowden
` (12 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
* Rename some fields.
* Use `uint32_t` consistently for all sizes and indices.
* Move thread ID into the ring structure.
* Replace status flag with a count of in-use elements.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 25 ++++++------
util/db.c | 100 +++++++++++++++++++++++++--------------------
2 files changed, 68 insertions(+), 57 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index 7c0649583f1d..ebf4f42917c3 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -27,22 +27,22 @@ struct db_driver {
};
-enum {
- RING_NO_QUERY,
- RING_QUERY_READY,
-};
-
struct db_stmt_ring {
- /* Ring buffer: 1 status byte + string */
- char *ring; /* pointer to the ring */
- uint32_t size; /* size of ring buffer in element */
- int length; /* length of one ring buffer element */
- uint32_t wr_item; /* write item in ring buffer */
- uint32_t rd_item; /* read item in ring buffer */
- char *wr_place;
+
+ char *elems; /* Buffer containing `size` strings of `length` bytes */
+
+ uint32_t size; /* No. of elements in ring buffer */
+ uint32_t used; /* No. of elements in ring buffer in use */
+ uint32_t length; /* Length of one element in ring buffer */
+ uint32_t wr_idx; /* Index of next element to write in ring buffer */
+ uint32_t rd_idx; /* Index of next element to read in ring buffer */
+
+ pthread_t thread_id;
pthread_cond_t cond;
pthread_mutex_t mutex;
+
int full;
+
};
struct db_stmt {
@@ -60,7 +60,6 @@ struct db_instance {
struct db_driver *driver;
/* DB ring buffer */
struct db_stmt_ring ring;
- pthread_t db_thread_id;
/* Backlog system */
unsigned int backlog_memcap;
unsigned int backlog_memusage;
diff --git a/util/db.c b/util/db.c
index ee6dfb6b5a2a..8a870846332b 100644
--- a/util/db.c
+++ b/util/db.c
@@ -63,8 +63,10 @@ static int _process_backlog(struct ulogd_pluginstance *upi);
static int _configure_ring(struct ulogd_pluginstance *upi);
static int _start_ring(struct ulogd_pluginstance *upi);
-static int _add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di);
+static int _add_to_ring(struct ulogd_pluginstance *upi);
static void *_process_ring(void *arg);
+static char *_get_ring_elem(struct db_stmt_ring *r, uint32_t i);
+static void _incr_ring_used(struct db_stmt_ring *r, int incr);
int
ulogd_db_configure(struct ulogd_pluginstance *upi,
@@ -182,17 +184,16 @@ ulogd_db_signal(struct ulogd_pluginstance *upi, int signal)
case SIGTERM:
case SIGINT:
if (di->ring.size) {
- int s = pthread_cancel(di->db_thread_id);
+ int s = pthread_cancel(di->ring.thread_id);
if (s != 0) {
ulogd_log(ULOGD_ERROR,
- "Can't cancel injection thread\n");
+ "Can't cancel ring-processing thread\n");
break;
}
- s = pthread_join(di->db_thread_id, NULL);
+ s = pthread_join(di->ring.thread_id, NULL);
if (s != 0) {
ulogd_log(ULOGD_ERROR,
- "Error waiting for injection thread"
- "cancelation\n");
+ "Error waiting for ring-processing thread cancellation\n");
}
}
break;
@@ -293,7 +294,7 @@ _interp_db_main(struct ulogd_pluginstance *upi)
struct db_instance *di = (struct db_instance *) &upi->private;
if (di->ring.size) {
- if (_add_to_ring(upi, di) < 0)
+ if (_add_to_ring(upi) < 0)
return ULOGD_IRET_ERR;
return ULOGD_IRET_OK;
}
@@ -372,11 +373,11 @@ _stop_db(struct ulogd_pluginstance *upi)
di->stmt = NULL;
}
if (di->ring.size > 0) {
- pthread_cancel(di->db_thread_id);
- free(di->ring.ring);
+ pthread_cancel(di->ring.thread_id);
pthread_cond_destroy(&di->ring.cond);
pthread_mutex_destroy(&di->ring.mutex);
- di->ring.ring = NULL;
+ free(di->ring.elems);
+ di->ring.elems = NULL;
}
}
@@ -743,18 +744,17 @@ _start_ring(struct ulogd_pluginstance *upi)
return 0;
/* allocate */
- di->ring.ring = calloc(di->ring.size, sizeof(char) * di->ring.length);
- if (di->ring.ring == NULL)
+ di->ring.elems = calloc(di->ring.size, di->ring.length);
+ if (di->ring.elems == NULL)
return -1;
- di->ring.wr_place = di->ring.ring;
+ di->ring.wr_idx = di->ring.rd_idx = di->ring.used = 0;
ulogd_log(ULOGD_NOTICE,
- "Allocating %d elements of size %d for ring\n",
+ "Allocating %" PRIu32 " elements of size %" PRIu32 " for ring\n",
di->ring.size, di->ring.length);
/* init start of query for each element */
for(i = 0; i < di->ring.size; i++)
- strcpy(di->ring.ring + di->ring.length * i + 1,
- di->stmt);
+ strcpy(_get_ring_elem(&di->ring, i), di->stmt);
/* init cond & mutex */
ret = pthread_cond_init(&di->ring.cond, NULL);
@@ -765,7 +765,7 @@ _start_ring(struct ulogd_pluginstance *upi)
goto cond_error;
/* create thread */
- ret = pthread_create(&di->db_thread_id, NULL, _process_ring, upi);
+ ret = pthread_create(&di->ring.thread_id, NULL, _process_ring, upi);
if (ret != 0)
goto mutex_error;
@@ -776,66 +776,78 @@ mutex_error:
cond_error:
pthread_cond_destroy(&di->ring.cond);
alloc_error:
- free(di->ring.ring);
+ free(di->ring.elems);
return -1;
}
static int
-_add_to_ring(struct ulogd_pluginstance *upi, struct db_instance *di)
+_add_to_ring(struct ulogd_pluginstance *upi)
{
- if (*di->ring.wr_place == RING_QUERY_READY) {
- if (di->ring.full == 0) {
+ struct db_instance *di = (struct db_instance *) &upi->private;
+
+ if (di->ring.used == di->ring.size) {
+ if (!di->ring.full) {
ulogd_log(ULOGD_ERROR, "No place left in ring\n");
di->ring.full = 1;
}
return ULOGD_IRET_OK;
- } else if (di->ring.full) {
+ }
+
+ if (di->ring.full) {
ulogd_log(ULOGD_NOTICE, "Recovered some place in ring\n");
di->ring.full = 0;
}
- _bind_sql_stmt(upi, di->ring.wr_place + 1);
- *di->ring.wr_place = RING_QUERY_READY;
+
+ _bind_sql_stmt(upi, _get_ring_elem(&di->ring, di->ring.wr_idx));
+ _incr_ring_used(&di->ring, 1);
+
pthread_cond_signal(&di->ring.cond);
- di->ring.wr_item ++;
- di->ring.wr_place += di->ring.length;
- if (di->ring.wr_item == di->ring.size) {
- di->ring.wr_item = 0;
- di->ring.wr_place = di->ring.ring;
- }
return ULOGD_IRET_OK;
}
static void *
-_process_ring(void *gdi)
+_process_ring(void *arg)
{
- struct ulogd_pluginstance *upi = gdi;
+ struct ulogd_pluginstance *upi = arg;
struct db_instance *di = (struct db_instance *) &upi->private;
- char *wr_place;
- wr_place = di->ring.ring;
pthread_mutex_lock(&di->ring.mutex);
while(1) {
/* wait cond */
pthread_cond_wait(&di->ring.cond, &di->ring.mutex);
- while (*wr_place == RING_QUERY_READY) {
- if (di->driver->execute(upi, wr_place + 1,
- strlen(wr_place + 1)) < 0) {
+ while (di->ring.used > 0) {
+ char *stmt = _get_ring_elem(&di->ring, di->ring.rd_idx);
+
+ if (di->driver->execute(upi, stmt,
+ strlen(stmt)) < 0) {
+
di->driver->close_db(upi);
while (di->driver->open_db(upi) < 0)
sleep(1);
/* try to re run query */
continue;
}
- *wr_place = RING_NO_QUERY;
- di->ring.rd_item++;
- if (di->ring.rd_item == di->ring.size) {
- di->ring.rd_item = 0;
- wr_place = di->ring.ring;
- } else
- wr_place += di->ring.length;
+
+ _incr_ring_used(&di->ring, -1);
}
}
return NULL;
}
+
+static char *
+_get_ring_elem(struct db_stmt_ring *r, uint32_t i)
+{
+ return &r->elems[i * r->length];
+}
+
+static void
+_incr_ring_used(struct db_stmt_ring *r, int incr)
+{
+ uint32_t *idx = incr > 0 ? &r->wr_idx : &r->rd_idx;
+
+ *idx = (*idx + 1) % r->size;
+
+ r->used += incr;
+}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 23/34] db: refactor backlog
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (21 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 22/34] db: refactor ring-buffer Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 24/34] db: use `struct db_stmt` objects more widely Jeremy Sowden
` (11 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Move the backlog fields into a separate structure along the same lines
as the ring-buffer.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 18 +++++++++++++-----
util/db.c | 44 ++++++++++++++++++++++----------------------
2 files changed, 35 insertions(+), 27 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index ebf4f42917c3..fc3b15ef0e0f 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -45,6 +45,18 @@ struct db_stmt_ring {
};
+struct db_stmt_backlog {
+
+ struct llist_head items;
+
+ unsigned int memcap;
+ unsigned int memusage;
+ unsigned int oneshot;
+
+ int full;
+
+};
+
struct db_stmt {
char *stmt;
int len;
@@ -61,11 +73,7 @@ struct db_instance {
/* DB ring buffer */
struct db_stmt_ring ring;
/* Backlog system */
- unsigned int backlog_memcap;
- unsigned int backlog_memusage;
- unsigned int backlog_oneshot;
- unsigned char backlog_full;
- struct llist_head backlog;
+ struct db_stmt_backlog backlog;
};
#define RECONNECT_DEFAULT 2
diff --git a/util/db.c b/util/db.c
index 8a870846332b..89c81d8d1dc5 100644
--- a/util/db.c
+++ b/util/db.c
@@ -259,7 +259,7 @@ _interp_db_init(struct ulogd_pluginstance *upi)
if (di->reconnect && di->reconnect > time(NULL)) {
/* store entry to backlog if it is active */
- if (di->backlog_memcap && !di->backlog_full) {
+ if (di->backlog.memcap && !di->backlog.full) {
_bind_sql_stmt(upi, di->stmt);
_add_to_backlog(upi, di->stmt, strlen(di->stmt));
}
@@ -268,7 +268,7 @@ _interp_db_init(struct ulogd_pluginstance *upi)
if (di->driver->open_db(upi) < 0) {
ulogd_log(ULOGD_ERROR, "can't establish database connection\n");
- if (di->backlog_memcap && !di->backlog_full) {
+ if (di->backlog.memcap && !di->backlog.full) {
_bind_sql_stmt(upi, di->stmt);
_add_to_backlog(upi, di->stmt, strlen(di->stmt));
}
@@ -302,7 +302,7 @@ _interp_db_main(struct ulogd_pluginstance *upi)
_bind_sql_stmt(upi, di->stmt);
/* if backup log is not empty we add current query to it */
- if (!llist_empty(&di->backlog)) {
+ if (!llist_empty(&di->backlog.items)) {
int ret = _add_to_backlog(upi, di->stmt, strlen(di->stmt));
if (ret == 0) {
if (_process_backlog(upi) < 0)
@@ -623,27 +623,27 @@ _configure_backlog(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
- INIT_LLIST_HEAD(&di->backlog);
+ INIT_LLIST_HEAD(&di->backlog.items);
- di->backlog_memusage = 0;
- di->backlog_memcap = backlog_memcap_ce(upi->config_kset).u.value;
- di->backlog_full = 0;
+ di->backlog.memusage = 0;
+ di->backlog.memcap = backlog_memcap_ce(upi->config_kset).u.value;
+ di->backlog.full = 0;
- if (di->backlog_memcap == 0)
+ if (di->backlog.memcap == 0)
return 0;
if (ringsize_ce(upi->config_kset).u.value) {
ulogd_log(ULOGD_ERROR,
"Ring buffer has precedence over backlog\n");
- di->backlog_memcap = 0;
+ di->backlog.memcap = 0;
return 0;
}
- di->backlog_oneshot = backlog_oneshot_ce(upi->config_kset).u.value;
- if (di->backlog_oneshot <= 2) {
+ di->backlog.oneshot = backlog_oneshot_ce(upi->config_kset).u.value;
+ if (di->backlog.oneshot <= 2) {
ulogd_log(ULOGD_ERROR,
"backlog_oneshot_requests must be > 2 to be effective. Setting it to 3.\n");
- di->backlog_oneshot = 3;
+ di->backlog.oneshot = 3;
}
return 0;
@@ -658,17 +658,17 @@ _add_to_backlog(struct ulogd_pluginstance *upi,
struct db_stmt *query;
/* check if we are using backlog */
- if (di->backlog_memcap == 0)
+ if (di->backlog.memcap == 0)
return 0;
query_size = sizeof(*query) + len + 1;
/* check len against backlog */
- if (query_size + di->backlog_memcap - di->backlog_memusage) {
- if (di->backlog_full == 0)
+ if (query_size + di->backlog.memcap - di->backlog.memusage) {
+ if (!di->backlog.full)
ulogd_log(ULOGD_ERROR,
"Backlog is full starting to reject events.\n");
- di->backlog_full = 1;
+ di->backlog.full = 1;
return -1;
}
@@ -684,10 +684,10 @@ _add_to_backlog(struct ulogd_pluginstance *upi,
return -1;
}
- di->backlog_memusage += query_size;
- di->backlog_full = 0;
+ di->backlog.memusage += query_size;
+ di->backlog.full = 0;
- llist_add_tail(&query->list, &di->backlog);
+ llist_add_tail(&query->list, &di->backlog.items);
return 0;
}
@@ -696,7 +696,7 @@ static int
_process_backlog(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
- int i = di->backlog_oneshot;
+ int i = di->backlog.oneshot;
struct db_stmt *query;
struct db_stmt *nquery;
@@ -704,13 +704,13 @@ _process_backlog(struct ulogd_pluginstance *upi)
if (di->reconnect && di->reconnect > time(NULL))
return 0;
- llist_for_each_entry_safe(query, nquery, &di->backlog, list) {
+ llist_for_each_entry_safe(query, nquery, &di->backlog.items, list) {
if (di->driver->execute(upi, query->stmt, query->len) < 0) {
/* error occur, database connexion need to be closed */
di->driver->close_db(upi);
return _reconnect_db(upi);
} else {
- di->backlog_memusage -= sizeof(*query) + query->len + 1;
+ di->backlog.memusage -= sizeof(*query) + query->len + 1;
llist_del(&query->list);
free(query->stmt);
free(query);
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 24/34] db: use `struct db_stmt` objects more widely
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (22 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 23/34] db: refactor backlog Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 25/34] db: synchronize access to ring-buffer Jeremy Sowden
` (10 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
The back-log API uses a struct to encapsulate SQL statements. Elsewhere
we pass pairs of string & length arguments. Extend `struct db_stmt` and
reuse where it is useful.
Change references to "queries" to "statements" (we don't run any queries).
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 18 ++-
output/dbi/ulogd_output_DBI.c | 6 +-
output/mysql/ulogd_output_MYSQL.c | 4 +-
output/pgsql/ulogd_output_PGSQL.c | 4 +-
util/db.c | 185 +++++++++++++++---------------
5 files changed, 110 insertions(+), 107 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index fc3b15ef0e0f..6c2e3d07f463 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -11,6 +11,8 @@
#include <ulogd/ulogd.h>
+struct db_stmt;
+
struct db_driver {
int (*get_columns)(struct ulogd_pluginstance *upi);
@@ -23,13 +25,14 @@ struct db_driver {
int (*escape_string)(struct ulogd_pluginstance *upi,
char *dst, const char *src, unsigned int len);
int (*execute)(struct ulogd_pluginstance *upi,
- const char *stmt, unsigned int len);
+ const struct db_stmt *stmt);
};
struct db_stmt_ring {
- char *elems; /* Buffer containing `size` strings of `length` bytes */
+ struct db_stmt *elems; /* Buffer containing `size` statements of
+ * `length` bytes */
uint32_t size; /* No. of elements in ring buffer */
uint32_t used; /* No. of elements in ring buffer in use */
@@ -58,14 +61,17 @@ struct db_stmt_backlog {
};
struct db_stmt {
- char *stmt;
- int len;
+ unsigned int len, size;
+ char sql[];
+};
+
+struct db_stmt_item {
struct llist_head list;
+ struct db_stmt stmt;
};
struct db_instance {
- char *stmt; /* buffer for our insert statement */
- int stmt_offset; /* offset to the beginning of the "VALUES" part */
+ struct db_stmt *stmt; /* buffer for our insert statement */
char *schema;
time_t reconnect;
int (*interp)(struct ulogd_pluginstance *upi);
diff --git a/output/dbi/ulogd_output_DBI.c b/output/dbi/ulogd_output_DBI.c
index 3b8d8c325131..49c2d45cc992 100644
--- a/output/dbi/ulogd_output_DBI.c
+++ b/output/dbi/ulogd_output_DBI.c
@@ -239,18 +239,18 @@ static int escape_string_dbi(struct ulogd_pluginstance *upi,
}
static int execute_dbi(struct ulogd_pluginstance *upi,
- const char *stmt, unsigned int len)
+ const struct db_stmt *stmt)
{
struct dbi_instance *pi = (struct dbi_instance *) upi->private;
- pi->result = dbi_conn_query(pi->dbh,stmt);
+ pi->result = dbi_conn_query(pi->dbh, stmt->sql);
if (!pi->result) {
const char *errptr;
dbi_conn_error(pi->dbh, &errptr);
ulogd_log(ULOGD_ERROR, "execute failed (%s)\n",
errptr);
ulogd_log(ULOGD_DEBUG, "failed query: [%s]\n",
- stmt);
+ stmt->sql);
return -1;
}
diff --git a/output/mysql/ulogd_output_MYSQL.c b/output/mysql/ulogd_output_MYSQL.c
index 55059f5c189e..bed1d7488019 100644
--- a/output/mysql/ulogd_output_MYSQL.c
+++ b/output/mysql/ulogd_output_MYSQL.c
@@ -206,13 +206,13 @@ static int escape_string_mysql(struct ulogd_pluginstance *upi,
}
static int execute_mysql(struct ulogd_pluginstance *upi,
- const char *stmt, unsigned int len)
+ const struct db_stmt *stmt)
{
struct mysql_instance *mi = (struct mysql_instance *) upi->private;
int ret;
MYSQL_RES * result;
- ret = mysql_real_query(mi->dbh, stmt, len);
+ ret = mysql_real_query(mi->dbh, stmt->sql, stmt->len);
if (ret) {
ulogd_log(ULOGD_ERROR, "execute failed (%s)\n",
mysql_error(mi->dbh));
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index 70a4d48459ff..9dfef7946775 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -283,11 +283,11 @@ static int escape_string_pgsql(struct ulogd_pluginstance *upi,
}
static int execute_pgsql(struct ulogd_pluginstance *upi,
- const char *stmt, unsigned int len)
+ const struct db_stmt *stmt)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
- pi->pgres = PQexec(pi->dbh, stmt);
+ pi->pgres = PQexec(pi->dbh, stmt->sql);
if (!(pi->pgres && ((PQresultStatus(pi->pgres) == PGRES_COMMAND_OK)
|| (PQresultStatus(pi->pgres) == PGRES_TUPLES_OK)))) {
ulogd_log(ULOGD_ERROR, "execute failed (%s)\n",
diff --git a/util/db.c b/util/db.c
index 89c81d8d1dc5..487eaed26153 100644
--- a/util/db.c
+++ b/util/db.c
@@ -54,18 +54,18 @@ static unsigned int _calc_sql_stmt_size(const char *procedure,
struct ulogd_key *keys,
unsigned int num_keys);
static void _bind_sql_stmt(struct ulogd_pluginstance *upi,
- char *stmt);
+ struct db_stmt *stmt);
static int _configure_backlog(struct ulogd_pluginstance *upi);
static int _add_to_backlog(struct ulogd_pluginstance *upi,
- const char *stmt, unsigned int len);
+ const struct db_stmt *stmt);
static int _process_backlog(struct ulogd_pluginstance *upi);
static int _configure_ring(struct ulogd_pluginstance *upi);
static int _start_ring(struct ulogd_pluginstance *upi);
static int _add_to_ring(struct ulogd_pluginstance *upi);
static void *_process_ring(void *arg);
-static char *_get_ring_elem(struct db_stmt_ring *r, uint32_t i);
+static struct db_stmt *_get_ring_elem(struct db_stmt_ring *r, uint32_t i);
static void _incr_ring_used(struct db_stmt_ring *r, int incr);
int
@@ -261,7 +261,7 @@ _interp_db_init(struct ulogd_pluginstance *upi)
/* store entry to backlog if it is active */
if (di->backlog.memcap && !di->backlog.full) {
_bind_sql_stmt(upi, di->stmt);
- _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ _add_to_backlog(upi, di->stmt);
}
return ULOGD_IRET_OK;
}
@@ -270,7 +270,7 @@ _interp_db_init(struct ulogd_pluginstance *upi)
ulogd_log(ULOGD_ERROR, "can't establish database connection\n");
if (di->backlog.memcap && !di->backlog.full) {
_bind_sql_stmt(upi, di->stmt);
- _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ _add_to_backlog(upi, di->stmt);
}
if (_reconnect_db(upi) < 0)
return ULOGD_IRET_ERR;
@@ -301,25 +301,23 @@ _interp_db_main(struct ulogd_pluginstance *upi)
_bind_sql_stmt(upi, di->stmt);
- /* if backup log is not empty we add current query to it */
+ /* if backup log is not empty we add current statement to it */
if (!llist_empty(&di->backlog.items)) {
- int ret = _add_to_backlog(upi, di->stmt, strlen(di->stmt));
- if (ret == 0) {
- if (_process_backlog(upi) < 0)
- return ULOGD_IRET_ERR;
- return ULOGD_IRET_OK;
- }
- ret = _process_backlog(upi);
- if (ret < 0)
- return ULOGD_IRET_ERR;
- /* try adding once the data to backlog */
- if (_add_to_backlog(upi, di->stmt, strlen(di->stmt)) < 0)
+ int ret = _add_to_backlog(upi, di->stmt);
+
+ if (_process_backlog(upi) < 0)
return ULOGD_IRET_ERR;
- return ULOGD_IRET_OK;
+
+ /* If the initial attempt to add the new item failed, try again
+ */
+ if (ret < 0)
+ ret = _add_to_backlog(upi, di->stmt);
+
+ return ret < 0 ? ULOGD_IRET_ERR : ULOGD_IRET_OK;
}
- if (di->driver->execute(upi, di->stmt, strlen(di->stmt)) < 0) {
- _add_to_backlog(upi, di->stmt, strlen(di->stmt));
+ if (di->driver->execute(upi, di->stmt) < 0) {
+ _add_to_backlog(upi, di->stmt);
/* error occur, database connexion need to be closed */
di->driver->close_db(upi);
if (_reconnect_db(upi) < 0)
@@ -402,7 +400,7 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
char *table = table_ce(upi->config_kset).u.string;
unsigned int size;
unsigned int i;
- char *stmtp;
+ char *sqlp;
if (di->stmt)
free(di->stmt);
@@ -413,14 +411,14 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
ulogd_log(ULOGD_DEBUG, "allocating %u bytes for statement\n", size);
- di->stmt = malloc(size);
+ di->stmt = malloc(sizeof(*di->stmt) + size);
if (!di->stmt) {
ulogd_log(ULOGD_ERROR, "OOM!\n");
return -1;
}
- di->ring.length = size + 1;
+ di->stmt->size = size;
- stmtp = di->stmt;
+ sqlp = di->stmt->sql;
if (strncasecmp(procedure, "INSERT", sizeof("INSERT") - 1) == 0 &&
(procedure[sizeof("INSERT") - 1] == '\0' ||
@@ -429,13 +427,13 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
if(procedure[sizeof("INSERT") - 1] == '\0') {
/* procedure == "INSERT" */
if (di->schema)
- stmtp += sprintf(stmtp, "insert into %s.%s (",
+ sqlp += sprintf(sqlp, "insert into %s.%s (",
di->schema, table);
else
- stmtp += sprintf(stmtp, "insert into %s (",
+ sqlp += sprintf(sqlp, "insert into %s (",
table);
} else
- stmtp += sprintf(stmtp, "%s (", procedure);
+ sqlp += sprintf(sqlp, "%s (", procedure);
for (i = 0; i < upi->input.num_keys; i++) {
char *underscore;
@@ -443,26 +441,26 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
continue;
- underscore = stmtp;
+ underscore = sqlp;
- stmtp += sprintf(stmtp, "%s,",
+ sqlp += sprintf(sqlp, "%s,",
upi->input.keys[i].name);
while ((underscore = strchr(underscore, '.')))
*underscore = '_';
}
- stmtp --;
+ sqlp --;
- stmtp += sprintf(stmtp, ") values (");
+ sqlp += sprintf(sqlp, ") values (");
} else if (strncasecmp(procedure, "CALL", sizeof("CALL") - 1) == 0)
- stmtp += sprintf(stmtp, "CALL %s(", procedure);
+ sqlp += sprintf(sqlp, "CALL %s(", procedure);
else
- stmtp += sprintf(stmtp, "SELECT %s(", procedure);
+ sqlp += sprintf(sqlp, "SELECT %s(", procedure);
- di->stmt_offset = stmtp - di->stmt;
+ di->stmt->len = sqlp - di->stmt->sql;
- ulogd_log(ULOGD_DEBUG, "stmt='%s'\n", di->stmt);
+ ulogd_log(ULOGD_DEBUG, "stmt='%s'\n", di->stmt->sql);
return 0;
}
@@ -534,14 +532,12 @@ _calc_sql_stmt_size(const char *procedure,
}
static void
-_bind_sql_stmt(struct ulogd_pluginstance *upi, char *start)
+_bind_sql_stmt(struct ulogd_pluginstance *upi, struct db_stmt *stmt)
{
struct db_instance *di = (struct db_instance *) &upi->private;
-
+ char *sqlp = stmt->sql + stmt->len;
unsigned int i;
- char *stmt_ins = start + di->stmt_offset;
-
for (i = 0; i < upi->input.num_keys; i++) {
struct ulogd_key *res = upi->input.keys[i].u.source;
@@ -554,52 +550,54 @@ _bind_sql_stmt(struct ulogd_pluginstance *upi, char *start)
if (!res || !IS_VALID(*res)) {
/* no result, we have to fake something */
- stmt_ins += sprintf(stmt_ins, "NULL,");
+ sqlp += sprintf(sqlp, "NULL,");
continue;
}
switch (res->type) {
case ULOGD_RET_INT8:
- sprintf(stmt_ins, "%d,", res->u.value.i8);
+ sqlp += sprintf(sqlp, "%d,", res->u.value.i8);
break;
case ULOGD_RET_INT16:
- sprintf(stmt_ins, "%d,", res->u.value.i16);
+ sqlp += sprintf(sqlp, "%d,", res->u.value.i16);
break;
case ULOGD_RET_INT32:
- sprintf(stmt_ins, "%d,", res->u.value.i32);
+ sqlp += sprintf(sqlp, "%d,", res->u.value.i32);
break;
case ULOGD_RET_INT64:
- sprintf(stmt_ins, "%" PRId64 ",", res->u.value.i64);
+ sqlp += sprintf(sqlp, "%" PRId64 ",", res->u.value.i64);
break;
case ULOGD_RET_UINT8:
- sprintf(stmt_ins, "%u,", res->u.value.ui8);
+ sqlp += sprintf(sqlp, "%u,", res->u.value.ui8);
break;
case ULOGD_RET_UINT16:
- sprintf(stmt_ins, "%u,", res->u.value.ui16);
+ sqlp += sprintf(sqlp, "%u,", res->u.value.ui16);
break;
case ULOGD_RET_IPADDR:
/* fallthrough when logging IP as uint32_t */
case ULOGD_RET_UINT32:
- sprintf(stmt_ins, "%u,", res->u.value.ui32);
+ sqlp += sprintf(sqlp, "%u,", res->u.value.ui32);
break;
case ULOGD_RET_UINT64:
- sprintf(stmt_ins, "%" PRIu64 ",", res->u.value.ui64);
+ sqlp += sprintf(sqlp, "%" PRIu64 ",",
+ res->u.value.ui64);
break;
case ULOGD_RET_BOOL:
- sprintf(stmt_ins, "'%d',", res->u.value.b);
+ sqlp += sprintf(sqlp, "'%d',", res->u.value.b);
break;
case ULOGD_RET_STRING:
- *(stmt_ins++) = '\'';
+ *sqlp++ = '\'';
if (res->u.value.ptr) {
- stmt_ins +=
- di->driver->escape_string(upi, stmt_ins,
- res->u.value.ptr,
- strlen(res->u.value.ptr));
+ char *str = res->u.value.ptr;
+ size_t len = strlen(str);
+
+ sqlp += di->driver->escape_string(upi, sqlp,
+ str, len);
}
- sprintf(stmt_ins, "',");
+ sqlp += sprintf(sqlp, "',");
break;
case ULOGD_RET_RAWSTR:
- sprintf(stmt_ins, "%s,", (char *) res->u.value.ptr);
+ sqlp += sprintf(sqlp, "%s,", (char *) res->u.value.ptr);
break;
case ULOGD_RET_RAW:
ulogd_log(ULOGD_NOTICE,
@@ -611,9 +609,10 @@ _bind_sql_stmt(struct ulogd_pluginstance *upi, char *start)
res->type, upi->input.keys[i].name);
break;
}
- stmt_ins = start + strlen(start);
}
- *(stmt_ins - 1) = ')';
+ *(sqlp - 1) = ')';
+
+ stmt->len = sqlp - stmt->sql;
}
/******************************************************************************/
@@ -650,21 +649,20 @@ _configure_backlog(struct ulogd_pluginstance *upi)
}
static int
-_add_to_backlog(struct ulogd_pluginstance *upi,
- const char *stmt, unsigned int len)
+_add_to_backlog(struct ulogd_pluginstance *upi, const struct db_stmt *stmt)
{
struct db_instance *di = (struct db_instance *) &upi->private;
- unsigned int query_size;
- struct db_stmt *query;
+ struct db_stmt_item *item;
+ unsigned int item_size;
/* check if we are using backlog */
if (di->backlog.memcap == 0)
return 0;
- query_size = sizeof(*query) + len + 1;
+ item_size = sizeof(*item) + stmt->size;
/* check len against backlog */
- if (query_size + di->backlog.memcap - di->backlog.memusage) {
+ if (item_size + di->backlog.memusage > di->backlog.memcap) {
if (!di->backlog.full)
ulogd_log(ULOGD_ERROR,
"Backlog is full starting to reject events.\n");
@@ -672,23 +670,16 @@ _add_to_backlog(struct ulogd_pluginstance *upi,
return -1;
}
- query = malloc(sizeof(*query));
- if (query == NULL)
+ item = malloc(item_size);
+ if (item == NULL)
return -1;
- query->stmt = strndup(stmt, len);
- query->len = len;
-
- if (query->stmt == NULL) {
- free(query);
- return -1;
- }
+ memcpy(&item->stmt, stmt, sizeof(*stmt) + stmt->size);
+ llist_add_tail(&item->list, &di->backlog.items);
- di->backlog.memusage += query_size;
+ di->backlog.memusage += item_size;
di->backlog.full = 0;
- llist_add_tail(&query->list, &di->backlog.items);
-
return 0;
}
@@ -696,27 +687,28 @@ static int
_process_backlog(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
+ struct db_stmt_item *item, *nitem;
int i = di->backlog.oneshot;
- struct db_stmt *query;
- struct db_stmt *nquery;
/* Don't try reconnect before timeout */
if (di->reconnect && di->reconnect > time(NULL))
return 0;
- llist_for_each_entry_safe(query, nquery, &di->backlog.items, list) {
- if (di->driver->execute(upi, query->stmt, query->len) < 0) {
+ llist_for_each_entry_safe(item, nitem, &di->backlog.items, list) {
+
+ if (di->driver->execute(upi, &item->stmt) < 0) {
/* error occur, database connexion need to be closed */
di->driver->close_db(upi);
return _reconnect_db(upi);
- } else {
- di->backlog.memusage -= sizeof(*query) + query->len + 1;
- llist_del(&query->list);
- free(query->stmt);
- free(query);
}
+
+ di->backlog.memusage -= sizeof(*item) + item->stmt.size;
+ llist_del(&item->list);
+ free(item);
+
if (--i < 0)
break;
+
}
return 0;
}
@@ -737,6 +729,7 @@ static int
_start_ring(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
+ size_t stmt_size, stmt_align;
unsigned int i;
int ret;
@@ -744,6 +737,11 @@ _start_ring(struct ulogd_pluginstance *upi)
return 0;
/* allocate */
+ stmt_size = sizeof(*di->stmt) + di->stmt->size;
+ stmt_align = __alignof__(*di->stmt);
+ di->ring.length = stmt_size % stmt_align != 0
+ ? (1 + stmt_size / stmt_align) * stmt_align
+ : stmt_size;
di->ring.elems = calloc(di->ring.size, di->ring.length);
if (di->ring.elems == NULL)
return -1;
@@ -752,9 +750,9 @@ _start_ring(struct ulogd_pluginstance *upi)
"Allocating %" PRIu32 " elements of size %" PRIu32 " for ring\n",
di->ring.size, di->ring.length);
- /* init start of query for each element */
+ /* init start of statement for each element */
for(i = 0; i < di->ring.size; i++)
- strcpy(_get_ring_elem(&di->ring, i), di->stmt);
+ memcpy(_get_ring_elem(&di->ring, i), di->stmt, stmt_size);
/* init cond & mutex */
ret = pthread_cond_init(&di->ring.cond, NULL);
@@ -817,15 +815,14 @@ _process_ring(void *arg)
/* wait cond */
pthread_cond_wait(&di->ring.cond, &di->ring.mutex);
while (di->ring.used > 0) {
- char *stmt = _get_ring_elem(&di->ring, di->ring.rd_idx);
-
- if (di->driver->execute(upi, stmt,
- strlen(stmt)) < 0) {
+ struct db_stmt *stmt = _get_ring_elem(&di->ring,
+ di->ring.rd_idx);
+ if (di->driver->execute(upi, stmt) < 0) {
di->driver->close_db(upi);
while (di->driver->open_db(upi) < 0)
sleep(1);
- /* try to re run query */
+ /* try to re-run statement */
continue;
}
@@ -836,10 +833,10 @@ _process_ring(void *arg)
return NULL;
}
-static char *
+static struct db_stmt *
_get_ring_elem(struct db_stmt_ring *r, uint32_t i)
{
- return &r->elems[i * r->length];
+ return (struct db_stmt *) ((char *) r->elems + i * r->length);
}
static void
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 25/34] db: synchronize access to ring-buffer
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (23 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 24/34] db: use `struct db_stmt` objects more widely Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 26/34] db: avoid cancelling ring-buffer thread Jeremy Sowden
` (9 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
There are a mutex and condition-variable associated with the
ring-buffer. However, they are not used to synchronize access to the
buffer, but only to wake the thread that processes the buffer when new
statements are added to it. Thus there is nothing to prevent concurrent
modifications of the buffer.
Instead, acquire the mutex before adding to the buffer, and, in the
processing thread, copy the statement we're about to execute out of the
buffer and release the mutex while processing it.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 2 ++
util/db.c | 63 +++++++++++++++++++++++++++++++++++++---------
2 files changed, 53 insertions(+), 12 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index 6c2e3d07f463..bf4a19dea150 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -34,6 +34,8 @@ struct db_stmt_ring {
struct db_stmt *elems; /* Buffer containing `size` statements of
* `length` bytes */
+ struct db_stmt *stmt; /* Currently executing statement */
+
uint32_t size; /* No. of elements in ring buffer */
uint32_t used; /* No. of elements in ring buffer in use */
uint32_t length; /* Length of one element in ring buffer */
diff --git a/util/db.c b/util/db.c
index 487eaed26153..6cfbcbc16791 100644
--- a/util/db.c
+++ b/util/db.c
@@ -372,10 +372,15 @@ _stop_db(struct ulogd_pluginstance *upi)
}
if (di->ring.size > 0) {
pthread_cancel(di->ring.thread_id);
+
pthread_cond_destroy(&di->ring.cond);
pthread_mutex_destroy(&di->ring.mutex);
+
free(di->ring.elems);
+ free(di->ring.stmt);
+
di->ring.elems = NULL;
+ di->ring.stmt = NULL;
}
}
@@ -737,14 +742,22 @@ _start_ring(struct ulogd_pluginstance *upi)
return 0;
/* allocate */
+
stmt_size = sizeof(*di->stmt) + di->stmt->size;
stmt_align = __alignof__(*di->stmt);
di->ring.length = stmt_size % stmt_align != 0
? (1 + stmt_size / stmt_align) * stmt_align
: stmt_size;
+
+ di->ring.stmt = malloc(di->ring.length);
+ if (di->ring.stmt == NULL)
+ return -1;
+
di->ring.elems = calloc(di->ring.size, di->ring.length);
- if (di->ring.elems == NULL)
+ if (di->ring.elems == NULL) {
+ free(di->ring.stmt);
return -1;
+ }
di->ring.wr_idx = di->ring.rd_idx = di->ring.used = 0;
ulogd_log(ULOGD_NOTICE,
"Allocating %" PRIu32 " elements of size %" PRIu32 " for ring\n",
@@ -775,6 +788,7 @@ cond_error:
pthread_cond_destroy(&di->ring.cond);
alloc_error:
free(di->ring.elems);
+ free(di->ring.stmt);
return -1;
}
@@ -784,12 +798,14 @@ _add_to_ring(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
+ pthread_mutex_lock(&di->ring.mutex);
+
if (di->ring.used == di->ring.size) {
if (!di->ring.full) {
ulogd_log(ULOGD_ERROR, "No place left in ring\n");
di->ring.full = 1;
}
- return ULOGD_IRET_OK;
+ goto unlock_mutex;
}
if (di->ring.full) {
@@ -801,6 +817,9 @@ _add_to_ring(struct ulogd_pluginstance *upi)
_incr_ring_used(&di->ring, 1);
pthread_cond_signal(&di->ring.cond);
+unlock_mutex:
+ pthread_mutex_unlock(&di->ring.mutex);
+
return ULOGD_IRET_OK;
}
@@ -809,27 +828,47 @@ _process_ring(void *arg)
{
struct ulogd_pluginstance *upi = arg;
struct db_instance *di = (struct db_instance *) &upi->private;
+ struct db_stmt *stmt = di->ring.stmt;
pthread_mutex_lock(&di->ring.mutex);
+
while(1) {
- /* wait cond */
+
pthread_cond_wait(&di->ring.cond, &di->ring.mutex);
+
while (di->ring.used > 0) {
- struct db_stmt *stmt = _get_ring_elem(&di->ring,
- di->ring.rd_idx);
-
- if (di->driver->execute(upi, stmt) < 0) {
- di->driver->close_db(upi);
- while (di->driver->open_db(upi) < 0)
- sleep(1);
- /* try to re-run statement */
+
+ memcpy(stmt, _get_ring_elem(&di->ring, di->ring.rd_idx),
+ di->ring.length);
+
+ pthread_mutex_unlock(&di->ring.mutex);
+
+exec_stmt:
+ if (di->driver->execute(upi, stmt) == 0) {
+
+ pthread_mutex_lock(&di->ring.mutex);
+
+ _incr_ring_used(&di->ring, -1);
+
continue;
+
}
- _incr_ring_used(&di->ring, -1);
+ /* If the exec fails, close the DB connexion and try to
+ * open it again. Once the connexion is made, retry the
+ * statement.
+ */
+ di->driver->close_db(upi);
+ while (di->driver->open_db(upi) < 0)
+ sleep(1);
+ goto exec_stmt;
+
}
+
}
+ pthread_mutex_unlock(&di->ring.mutex);
+
return NULL;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 26/34] db: avoid cancelling ring-buffer thread
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (24 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 25/34] db: synchronize access to ring-buffer Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 27/34] db, IP2BIN: defer formatting of raw strings Jeremy Sowden
` (8 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Using `pthread_cancel` can leave the DB driver in an inconsistent state
and may lead to undefined behaviour when cleaning up the plug-in.
Instead of cancelling the thread, set a flag and signal the condition
variable, thereby giving the thread a chance to exit in good order.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 1 +
util/db.c | 30 +++++++++++++-----------------
2 files changed, 14 insertions(+), 17 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index bf4a19dea150..17eaa7cf60db 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -47,6 +47,7 @@ struct db_stmt_ring {
pthread_mutex_t mutex;
int full;
+ volatile sig_atomic_t shut_down;
};
diff --git a/util/db.c b/util/db.c
index 6cfbcbc16791..42b59cc6284c 100644
--- a/util/db.c
+++ b/util/db.c
@@ -184,17 +184,11 @@ ulogd_db_signal(struct ulogd_pluginstance *upi, int signal)
case SIGTERM:
case SIGINT:
if (di->ring.size) {
- int s = pthread_cancel(di->ring.thread_id);
- if (s != 0) {
+ di->ring.shut_down = 1;
+ pthread_cond_signal(&di->ring.cond);
+ if (pthread_join(di->ring.thread_id, NULL) != 0)
ulogd_log(ULOGD_ERROR,
- "Can't cancel ring-processing thread\n");
- break;
- }
- s = pthread_join(di->ring.thread_id, NULL);
- if (s != 0) {
- ulogd_log(ULOGD_ERROR,
- "Error waiting for ring-processing thread cancellation\n");
- }
+ "Error waiting for ring-processing thread exit\n");
}
break;
default:
@@ -371,8 +365,6 @@ _stop_db(struct ulogd_pluginstance *upi)
di->stmt = NULL;
}
if (di->ring.size > 0) {
- pthread_cancel(di->ring.thread_id);
-
pthread_cond_destroy(&di->ring.cond);
pthread_mutex_destroy(&di->ring.mutex);
@@ -832,11 +824,11 @@ _process_ring(void *arg)
pthread_mutex_lock(&di->ring.mutex);
- while(1) {
+ while(!di->ring.shut_down) {
pthread_cond_wait(&di->ring.cond, &di->ring.mutex);
- while (di->ring.used > 0) {
+ while (!di->ring.shut_down && di->ring.used > 0) {
memcpy(stmt, _get_ring_elem(&di->ring, di->ring.rd_idx),
di->ring.length);
@@ -859,9 +851,13 @@ exec_stmt:
* statement.
*/
di->driver->close_db(upi);
- while (di->driver->open_db(upi) < 0)
- sleep(1);
- goto exec_stmt;
+ while (!di->ring.shut_down) {
+ if (di->driver->open_db(upi) < 0) {
+ sleep(1);
+ continue;
+ }
+ goto exec_stmt;
+ }
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 27/34] db, IP2BIN: defer formatting of raw strings
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (25 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 26/34] db: avoid cancelling ring-buffer thread Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 28/34] db: add prep & exec support Jeremy Sowden
` (7 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Currently, the only use of the output of IP2BIN is embedding in literal
SQL strings, so it converts the binary data to a suitably formatted
hexadecimal string and propagates that. However, we will shortly
introduce prep & exec support to the DB API, at which point we may want
the raw binary data in the output plug-ins. Therefore, in order to
avoid having to decode the hex back to binary, defer the hex encoding to
where it is used.
At the same time, resize the buffers. IPv6 addresses are 128 *bits*, so the
buffers do not need to be 128 *bytes*.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
filter/ulogd_filter_IP2BIN.c | 61 +++++++++++++-----------------------
util/db.c | 10 +++++-
2 files changed, 31 insertions(+), 40 deletions(-)
diff --git a/filter/ulogd_filter_IP2BIN.c b/filter/ulogd_filter_IP2BIN.c
index ca6d3abae884..6a4e10e4fb58 100644
--- a/filter/ulogd_filter_IP2BIN.c
+++ b/filter/ulogd_filter_IP2BIN.c
@@ -28,8 +28,6 @@
#include <ulogd/ulogd.h>
#include <netinet/if_ether.h>
-#define IPADDR_LENGTH 128
-
enum input_keys {
KEY_OOB_FAMILY,
KEY_OOB_PROTOCOL,
@@ -89,32 +87,38 @@ static struct ulogd_key ip2bin_inp[] = {
static struct ulogd_key ip2bin_keys[] = {
{
.type = ULOGD_RET_RAWSTR,
+ .len = sizeof(struct in6_addr),
.name = "ip.saddr.bin",
},
{
.type = ULOGD_RET_RAWSTR,
+ .len = sizeof(struct in6_addr),
.name = "ip.daddr.bin",
},
{
.type = ULOGD_RET_RAWSTR,
+ .len = sizeof(struct in6_addr),
.name = "orig.ip.saddr.bin",
},
{
.type = ULOGD_RET_RAWSTR,
+ .len = sizeof(struct in6_addr),
.name = "orig.ip.daddr.bin",
},
{
.type = ULOGD_RET_RAWSTR,
+ .len = sizeof(struct in6_addr),
.name = "reply.ip.saddr.bin",
},
{
.type = ULOGD_RET_RAWSTR,
+ .len = sizeof(struct in6_addr),
.name = "reply.ip.daddr.bin",
},
};
-static char ipbin_array[MAX_KEY - START_KEY + 1][IPADDR_LENGTH];
+static unsigned char ipbin_array[MAX_KEY - START_KEY + 1][sizeof(struct in6_addr)];
/**
* Convert IPv4 address (as 32-bit unsigned integer) to IPv6 address:
@@ -130,13 +134,8 @@ static inline void uint32_to_ipv6(const uint32_t ipv4, struct in6_addr *ipv6)
static int ip2bin(struct ulogd_key *inp, int index, int oindex)
{
- char family = ikey_get_u8(&inp[KEY_OOB_FAMILY]);
- char convfamily = family;
- unsigned char *addr8;
- struct in6_addr *addr;
- struct in6_addr ip4_addr;
- char *buffer;
- int i, written;
+ char family = ikey_get_u8(&inp[KEY_OOB_FAMILY]), convfamily = family;
+ struct in6_addr *addr, ip4_addr;
if (family == AF_BRIDGE) {
if (!pp_is_valid(inp, KEY_OOB_PROTOCOL)) {
@@ -162,37 +161,21 @@ static int ip2bin(struct ulogd_key *inp, int index, int oindex)
}
switch (convfamily) {
- case AF_INET6:
- addr = (struct in6_addr *)ikey_get_u128(&inp[index]);
- break;
- case AF_INET:
- /* Convert IPv4 to IPv4 in IPv6 */
- addr = &ip4_addr;
- uint32_to_ipv6(ikey_get_u32(&inp[index]), addr);
- break;
- default:
- /* TODO handle error */
- ulogd_log(ULOGD_NOTICE, "Unknown protocol family\n");
- return ULOGD_IRET_ERR;
+ case AF_INET6:
+ addr = (struct in6_addr *)ikey_get_u128(&inp[index]);
+ break;
+ case AF_INET:
+ /* Convert IPv4 to IPv4 in IPv6 */
+ addr = &ip4_addr;
+ uint32_to_ipv6(ikey_get_u32(&inp[index]), addr);
+ break;
+ default:
+ /* TODO handle error */
+ ulogd_log(ULOGD_NOTICE, "Unknown protocol family\n");
+ return ULOGD_IRET_ERR;
}
- buffer = ipbin_array[oindex];
- /* format IPv6 to BINARY(16) as "0x..." */
- buffer[0] = '0';
- buffer[1] = 'x';
- buffer += 2;
- addr8 = &addr->s6_addr[0];
- for (i = 0; i < 4; i++) {
- written = sprintf(buffer, "%02x%02x%02x%02x",
- addr8[0], addr8[1], addr8[2], addr8[3]);
- if (written != 2 * 4) {
- buffer[0] = 0;
- return ULOGD_IRET_ERR;
- }
- buffer += written;
- addr8 += 4;
- }
- buffer[0] = 0;
+ memcpy(ipbin_array[oindex], &addr->s6_addr, sizeof(addr->s6_addr));
return ULOGD_IRET_OK;
}
diff --git a/util/db.c b/util/db.c
index 42b59cc6284c..271cd25efeaf 100644
--- a/util/db.c
+++ b/util/db.c
@@ -594,7 +594,15 @@ _bind_sql_stmt(struct ulogd_pluginstance *upi, struct db_stmt *stmt)
sqlp += sprintf(sqlp, "',");
break;
case ULOGD_RET_RAWSTR:
- sqlp += sprintf(sqlp, "%s,", (char *) res->u.value.ptr);
+ strcpy(sqlp, "0x");
+ sqlp += 2;
+ {
+ unsigned char *cp = res->u.value.ptr;
+ unsigned int j;
+ for (j = 0; j < res->len; ++j)
+ sqlp += sprintf(sqlp, "%02x", cp[j]);
+ }
+ *sqlp++ = ',';
break;
case ULOGD_RET_RAW:
ulogd_log(ULOGD_NOTICE,
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 28/34] db: add prep & exec support
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (26 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 27/34] db, IP2BIN: defer formatting of raw strings Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 29/34] output: mysql: " Jeremy Sowden
` (6 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Currently, the common DB API pre-allocates the prefix of the SQL
statement, e.g.:
insert into t (col1, col2, col3, col4) values (
and concatenates the literal values for each packet before executing the
statement. However, this requires escaping these values, which is not
supported by all DB API's. Furthermore, most data-bases offer the
ability to prepare statements with place-holders and bind values before
executing them since this avoids the need to analyse and optimize the
statement on every execution, and can, therefore, be more efficient.
Add the infrastructure to allow output plug-ins to prep & exec if the
data-base supports it.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
include/ulogd/db.h | 33 ++++++
util/db.c | 286 ++++++++++++++++++++++++++++++++++++++++-----
2 files changed, 287 insertions(+), 32 deletions(-)
diff --git a/include/ulogd/db.h b/include/ulogd/db.h
index 17eaa7cf60db..602ab25dc1b2 100644
--- a/include/ulogd/db.h
+++ b/include/ulogd/db.h
@@ -9,6 +9,7 @@
#ifndef _ULOGD_DB_H
#define _ULOGD_DB_H
+#include <stdbool.h>
#include <ulogd/ulogd.h>
struct db_stmt;
@@ -18,12 +19,15 @@ struct db_driver {
int (*get_columns)(struct ulogd_pluginstance *upi);
const char *(*get_column)(void *, unsigned int);
char *(*format_key)(char *);
+ unsigned int (*format_param)(char *, unsigned int, unsigned int);
int (*open_db)(struct ulogd_pluginstance *upi);
int (*close_db)(struct ulogd_pluginstance *upi);
int (*escape_string)(struct ulogd_pluginstance *upi,
char *dst, const char *src, unsigned int len);
+ int (*prepare)(struct ulogd_pluginstance *upi,
+ const struct db_stmt *stmt);
int (*execute)(struct ulogd_pluginstance *upi,
const struct db_stmt *stmt);
@@ -63,9 +67,34 @@ struct db_stmt_backlog {
};
+union db_stmt_arg_value {
+ uint8_t b;
+ uint8_t ui8;
+ uint16_t ui16;
+ uint32_t ui32;
+ uint64_t ui64;
+ int8_t i8;
+ int16_t i16;
+ int32_t i32;
+ int64_t i64;
+ void *ptr;
+};
+
+struct db_stmt_arg {
+ uint32_t len;
+ uint16_t type;
+ bool null;
+ union db_stmt_arg_value value;
+};
+
struct db_stmt {
+
+ struct db_stmt_arg *args;
+ unsigned int nr_params;
+
unsigned int len, size;
char sql[];
+
};
struct db_stmt_item {
@@ -74,15 +103,19 @@ struct db_stmt_item {
};
struct db_instance {
+
struct db_stmt *stmt; /* buffer for our insert statement */
char *schema;
time_t reconnect;
int (*interp)(struct ulogd_pluginstance *upi);
struct db_driver *driver;
+
/* DB ring buffer */
struct db_stmt_ring ring;
+
/* Backlog system */
struct db_stmt_backlog backlog;
+
};
#define RECONNECT_DEFAULT 2
diff --git a/util/db.c b/util/db.c
index 271cd25efeaf..dafa9aa621a4 100644
--- a/util/db.c
+++ b/util/db.c
@@ -52,9 +52,21 @@ static int _create_sql_stmt(struct ulogd_pluginstance *upi);
static unsigned int _calc_sql_stmt_size(const char *procedure,
const char *schema, const char *table,
struct ulogd_key *keys,
- unsigned int num_keys);
-static void _bind_sql_stmt(struct ulogd_pluginstance *upi,
- struct db_stmt *stmt);
+ unsigned int num_keys,
+ unsigned int (*format_param)(char *,
+ unsigned int,
+ unsigned int));
+static bool _key_supported(const struct ulogd_key *key);
+static bool _key_type_supported(uint16_t key_type);
+static unsigned int _format_param(char *buf, unsigned int size,
+ unsigned int idx);
+static int _bind_sql_stmt(struct ulogd_pluginstance *upi,
+ struct db_stmt *stmt);
+static int _copy_sql_stmt_args(struct ulogd_pluginstance *upi,
+ struct db_stmt *stmt);
+static void _concat_sql_stmt_args(struct ulogd_pluginstance *upi,
+ struct db_stmt *stmt);
+static void _free_sql_stmt_args(struct db_stmt *stmt);
static int _configure_backlog(struct ulogd_pluginstance *upi);
static int _add_to_backlog(struct ulogd_pluginstance *upi,
@@ -254,7 +266,8 @@ _interp_db_init(struct ulogd_pluginstance *upi)
if (di->reconnect && di->reconnect > time(NULL)) {
/* store entry to backlog if it is active */
if (di->backlog.memcap && !di->backlog.full) {
- _bind_sql_stmt(upi, di->stmt);
+ if (_bind_sql_stmt(upi, di->stmt) < 0)
+ return ULOGD_IRET_ERR;
_add_to_backlog(upi, di->stmt);
}
return ULOGD_IRET_OK;
@@ -263,14 +276,19 @@ _interp_db_init(struct ulogd_pluginstance *upi)
if (di->driver->open_db(upi) < 0) {
ulogd_log(ULOGD_ERROR, "can't establish database connection\n");
if (di->backlog.memcap && !di->backlog.full) {
- _bind_sql_stmt(upi, di->stmt);
- _add_to_backlog(upi, di->stmt);
+ if (_bind_sql_stmt(upi, di->stmt) == 0)
+ _add_to_backlog(upi, di->stmt);
}
if (_reconnect_db(upi) < 0)
return ULOGD_IRET_ERR;
return ULOGD_IRET_OK;
}
+ if (di->driver->prepare) {
+ if (di->driver->prepare(upi, di->stmt) < 0)
+ return ULOGD_IRET_ERR;
+ }
+
/* enable 'real' logging */
di->interp = &_interp_db_main;
@@ -293,7 +311,8 @@ _interp_db_main(struct ulogd_pluginstance *upi)
return ULOGD_IRET_OK;
}
- _bind_sql_stmt(upi, di->stmt);
+ if (_bind_sql_stmt(upi, di->stmt) < 0)
+ return ULOGD_IRET_ERR;
/* if backup log is not empty we add current statement to it */
if (!llist_empty(&di->backlog.items)) {
@@ -393,6 +412,10 @@ static int
_create_sql_stmt(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) upi->private;
+ unsigned int (*format_param)(char *, unsigned int, unsigned int)
+ = di->driver->prepare != NULL
+ ? di->driver->format_param ? : _format_param
+ : NULL;
char *procedure = procedure_ce(upi->config_kset).u.string;
char *table = table_ce(upi->config_kset).u.string;
unsigned int size;
@@ -404,7 +427,8 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
/* calculate the size for the insert statement */
size = _calc_sql_stmt_size(procedure, di->schema, table,
- upi->input.keys, upi->input.num_keys);
+ upi->input.keys, upi->input.num_keys,
+ format_param);
ulogd_log(ULOGD_DEBUG, "allocating %u bytes for statement\n", size);
@@ -433,15 +457,15 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
sqlp += sprintf(sqlp, "%s (", procedure);
for (i = 0; i < upi->input.num_keys; i++) {
+ struct ulogd_key *key = &upi->input.keys[i];
char *underscore;
- if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
+ if (!_key_supported(key))
continue;
underscore = sqlp;
- sqlp += sprintf(sqlp, "%s,",
- upi->input.keys[i].name);
+ sqlp += sprintf(sqlp, "%s,", key->name);
while ((underscore = strchr(underscore, '.')))
*underscore = '_';
@@ -457,6 +481,29 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
di->stmt->len = sqlp - di->stmt->sql;
+ if (di->driver->prepare) {
+ unsigned int size, nr_params = 0;
+
+ sqlp = di->stmt->sql + di->stmt->len;
+ size = di->stmt->size - di->stmt->len;
+
+ for (i = 0; i < upi->input.num_keys; i++) {
+ unsigned int used;
+
+ if (!_key_supported(&upi->input.keys[i]))
+ continue;
+
+ used = format_param(sqlp, size, nr_params++);
+
+ sqlp += used;
+ size -= used;
+ }
+ strcpy(sqlp - 1, ")");
+
+ di->stmt->len = sqlp - di->stmt->sql;
+ di->stmt->nr_params = nr_params;
+ }
+
ulogd_log(ULOGD_DEBUG, "stmt='%s'\n", di->stmt->sql);
return 0;
@@ -467,7 +514,9 @@ _create_sql_stmt(struct ulogd_pluginstance *upi)
static unsigned int
_calc_sql_stmt_size(const char *procedure,
const char *schema, const char *table,
- struct ulogd_key *keys, unsigned int num_keys)
+ struct ulogd_key *keys, unsigned int num_keys,
+ unsigned int (*format_param)(char *, unsigned int,
+ unsigned int))
{
unsigned int i, size = 0;
bool include_keys;
@@ -516,11 +565,32 @@ _calc_sql_stmt_size(const char *procedure,
*/
for (i = 0; i < num_keys; i++) {
- if (keys[i].flags & ULOGD_KEYF_INACTIVE)
+ struct ulogd_key *key = &keys[i], *res = key->u.source;
+
+ if (key->flags & ULOGD_KEYF_INACTIVE)
+ continue;
+
+ if (!res)
+ continue;
+
+ if (!_key_type_supported(res->type)) {
+ if (res->type == ULOGD_RET_RAW)
+ ulogd_log(ULOGD_NOTICE,
+ "RAW type is unsupported in SQL output\n");
+ else
+ ulogd_log(ULOGD_NOTICE,
+ "Unknown type %d for %s\n",
+ res->type, key->name);
continue;
+ }
+
if (include_keys)
- size += strlen(keys[i].name) + 1;
- size += SQL_VALSIZE + 1;
+ size += strlen(key->name) + 1;
+
+ if (format_param)
+ size += format_param(NULL, 0, i);
+ else
+ size += SQL_VALSIZE + 1;
}
size++; /* Allow for the final NUL */
@@ -528,24 +598,149 @@ _calc_sql_stmt_size(const char *procedure,
return size;
}
-static void
+static bool
+_key_supported(const struct ulogd_key *key)
+{
+ if (key->flags & ULOGD_KEYF_INACTIVE)
+ return false;
+
+ if (!key->u.source)
+ return false;
+
+ return _key_type_supported(key->u.source->type);
+}
+
+static bool
+_key_type_supported(uint16_t key_type)
+{
+ switch (key_type) {
+ case ULOGD_RET_BOOL:
+ case ULOGD_RET_UINT8:
+ case ULOGD_RET_UINT16:
+ case ULOGD_RET_IPADDR:
+ case ULOGD_RET_UINT32:
+ case ULOGD_RET_UINT64:
+ case ULOGD_RET_INT8:
+ case ULOGD_RET_INT16:
+ case ULOGD_RET_INT32:
+ case ULOGD_RET_INT64:
+ case ULOGD_RET_STRING:
+ case ULOGD_RET_RAWSTR:
+ return true;
+ default:
+ return false;;
+ }
+}
+
+static unsigned int
+_format_param(char *buf, unsigned int size,
+ unsigned int idx __attribute__ ((unused)))
+{
+ return snprintf(buf, size, "?,");
+}
+
+static int
_bind_sql_stmt(struct ulogd_pluginstance *upi, struct db_stmt *stmt)
+{
+ struct db_instance *di = (struct db_instance *) &upi->private;
+
+ if (di->driver->prepare)
+ return _copy_sql_stmt_args(upi, stmt);
+
+ _concat_sql_stmt_args(upi, stmt);
+ return 0;
+}
+
+static int
+_copy_sql_stmt_args(struct ulogd_pluginstance *upi, struct db_stmt *stmt)
+{
+ struct db_stmt_arg *stmt_args;
+ unsigned int i, j;
+
+ stmt_args = calloc(stmt->nr_params, sizeof(*stmt_args));
+ if (!stmt_args)
+ return -1;
+
+ for (i = 0, j = 0; i < upi->input.num_keys; i++) {
+ struct ulogd_key *key = &upi->input.keys[i],
+ *res = key->u.source;
+ struct db_stmt_arg *arg = &stmt_args[j];
+ union db_stmt_arg_value *val = &arg->value;
+
+ if (!_key_supported(key))
+ continue;
+
+ if (!IS_VALID(*res))
+ arg->null = true;
+ else {
+ arg->type = res->type;
+
+ switch (res->type) {
+ case ULOGD_RET_BOOL:
+ case ULOGD_RET_UINT8:
+ case ULOGD_RET_UINT16:
+ case ULOGD_RET_IPADDR:
+ case ULOGD_RET_UINT32:
+ case ULOGD_RET_UINT64:
+ case ULOGD_RET_INT8:
+ case ULOGD_RET_INT16:
+ case ULOGD_RET_INT32:
+ case ULOGD_RET_INT64:
+ memcpy(val, &res->u.value, sizeof(*val));
+ break;
+ case ULOGD_RET_STRING:
+ if (!res->u.value.ptr)
+ continue;
+ arg->len = strlen(res->u.value.ptr);
+ val->ptr = strdup(res->u.value.ptr);
+ if (!val->ptr)
+ goto err_free_args;
+ break;
+ case ULOGD_RET_RAWSTR:
+ if (!res->u.value.ptr)
+ continue;
+ arg->len = res->len;
+ val->ptr = malloc(res->len);
+ if (!val->ptr)
+ goto err_free_args;
+ memcpy(val->ptr, res->u.value.ptr, res->len);
+ break;
+ }
+ }
+
+ j++;
+ }
+
+ stmt->args = stmt_args;
+ return 0;
+
+err_free_args:
+ while (i-- > 0)
+ switch(stmt_args[i].type) {
+ case ULOGD_RET_STRING:
+ case ULOGD_RET_RAWSTR:
+ free (stmt_args[i].value.ptr);
+ break;
+ }
+ free (stmt_args);
+ return -1;
+}
+
+static void
+_concat_sql_stmt_args(struct ulogd_pluginstance *upi, struct db_stmt *stmt)
{
struct db_instance *di = (struct db_instance *) &upi->private;
char *sqlp = stmt->sql + stmt->len;
unsigned int i;
for (i = 0; i < upi->input.num_keys; i++) {
- struct ulogd_key *res = upi->input.keys[i].u.source;
+ struct ulogd_key *key = &upi->input.keys[i],
+ *res = key->u.source;
- if (upi->input.keys[i].flags & ULOGD_KEYF_INACTIVE)
+ if (!_key_supported(key))
continue;
- if (!res)
- ulogd_log(ULOGD_NOTICE, "no source for `%s' ?!?\n",
- upi->input.keys[i].name);
-
- if (!res || !IS_VALID(*res)) {
+ if (!IS_VALID(*res)) {
/* no result, we have to fake something */
sqlp += sprintf(sqlp, "NULL,");
continue;
@@ -604,14 +799,7 @@ _bind_sql_stmt(struct ulogd_pluginstance *upi, struct db_stmt *stmt)
}
*sqlp++ = ',';
break;
- case ULOGD_RET_RAW:
- ulogd_log(ULOGD_NOTICE,
- "Unsupported RAW type is unsupported in SQL output");
- break;
default:
- ulogd_log(ULOGD_NOTICE,
- "unknown type %d for %s\n",
- res->type, upi->input.keys[i].name);
break;
}
}
@@ -620,6 +808,28 @@ _bind_sql_stmt(struct ulogd_pluginstance *upi, struct db_stmt *stmt)
stmt->len = sqlp - stmt->sql;
}
+static void
+_free_sql_stmt_args(struct db_stmt *stmt)
+{
+ unsigned int i;
+
+ if (stmt->nr_params == 0)
+ return;
+
+ if (!stmt->args)
+ return;
+
+ for(i = 0; i < stmt->nr_params; ++i)
+ switch(stmt->args[i].type) {
+ case ULOGD_RET_STRING:
+ case ULOGD_RET_RAWSTR:
+ free(stmt->args[i].value.ptr);
+ break;
+ }
+ free(stmt->args);
+ stmt->args = NULL;
+}
+
/******************************************************************************/
static int
@@ -708,6 +918,7 @@ _process_backlog(struct ulogd_pluginstance *upi)
}
di->backlog.memusage -= sizeof(*item) + item->stmt.size;
+ _free_sql_stmt_args(&item->stmt);
llist_del(&item->list);
free(item);
@@ -797,6 +1008,7 @@ static int
_add_to_ring(struct ulogd_pluginstance *upi)
{
struct db_instance *di = (struct db_instance *) &upi->private;
+ int rv = ULOGD_IRET_OK;
pthread_mutex_lock(&di->ring.mutex);
@@ -813,14 +1025,18 @@ _add_to_ring(struct ulogd_pluginstance *upi)
di->ring.full = 0;
}
- _bind_sql_stmt(upi, _get_ring_elem(&di->ring, di->ring.wr_idx));
+ if (_bind_sql_stmt(upi, _get_ring_elem(&di->ring,
+ di->ring.wr_idx)) < 0) {
+ rv = ULOGD_IRET_ERR;
+ goto unlock_mutex;
+ }
_incr_ring_used(&di->ring, 1);
pthread_cond_signal(&di->ring.cond);
unlock_mutex:
pthread_mutex_unlock(&di->ring.mutex);
- return ULOGD_IRET_OK;
+ return rv;
}
static void *
@@ -848,6 +1064,7 @@ exec_stmt:
pthread_mutex_lock(&di->ring.mutex);
+ _free_sql_stmt_args(stmt);
_incr_ring_used(&di->ring, -1);
continue;
@@ -858,12 +1075,17 @@ exec_stmt:
* open it again. Once the connexion is made, retry the
* statement.
*/
+close_db:
di->driver->close_db(upi);
while (!di->ring.shut_down) {
if (di->driver->open_db(upi) < 0) {
sleep(1);
continue;
}
+ if (di->driver->prepare) {
+ if (di->driver->prepare(upi, stmt) < 0)
+ goto close_db;
+ }
goto exec_stmt;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 29/34] output: mysql: add prep & exec support
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (27 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 28/34] db: add prep & exec support Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 30/34] output: pgsql: remove a couple of struct members Jeremy Sowden
` (5 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Now that the common DB API offers prep & exec support, update the MySQL
plug-in to use it.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/mysql/ulogd_output_MYSQL.c | 163 ++++++++++++++++++++++++------
1 file changed, 131 insertions(+), 32 deletions(-)
diff --git a/output/mysql/ulogd_output_MYSQL.c b/output/mysql/ulogd_output_MYSQL.c
index bed1d7488019..d64da85a418a 100644
--- a/output/mysql/ulogd_output_MYSQL.c
+++ b/output/mysql/ulogd_output_MYSQL.c
@@ -37,11 +37,12 @@
* Port to ulogd2 (@ 0sec conference, Bern, Suisse)
*/
+#include <assert.h>
+#ifdef DEBUG_MYSQL
+#include <stdio.h>
+#endif
#include <stdlib.h>
#include <string.h>
-#include <errno.h>
-#include <time.h>
-#include <arpa/inet.h>
#include <mysql/mysql.h>
#include <ulogd/ulogd.h>
#include <ulogd/conffile.h>
@@ -56,6 +57,12 @@
struct mysql_instance {
struct db_instance db_inst;
MYSQL *dbh; /* the database handle we are using */
+ MYSQL_STMT *sth;
+};
+
+struct mysql_buffer {
+ unsigned long length;
+ my_bool is_null;
};
/* our configuration directives */
@@ -193,46 +200,138 @@ static int open_db_mysql(struct ulogd_pluginstance *upi)
return 0;
}
-static int escape_string_mysql(struct ulogd_pluginstance *upi,
- char *dst, const char *src, unsigned int len)
+static int prepare_mysql(struct ulogd_pluginstance *upi,
+ const struct db_stmt *stmt)
{
struct mysql_instance *mi = (struct mysql_instance *) upi->private;
-#ifdef OLD_MYSQL
- return mysql_escape_string(dst, src, len);
-#else
- return mysql_real_escape_string(mi->dbh, dst, src, len);
-#endif /* OLD_MYSQL */
+ if (mi->sth)
+ mysql_stmt_close(mi->sth);
+
+ if (!(mi->sth = mysql_stmt_init(mi->dbh))) {
+ ulogd_log(ULOGD_ERROR,
+ "Failed to initialize statement: %s\n",
+ mysql_error(mi->dbh));
+ return -1;
+ }
+
+ if (mysql_stmt_prepare(mi->sth, stmt->sql, stmt->len) != 0) {
+ ulogd_log(ULOGD_ERROR,
+ "Failed to prepare statement: %s\n",
+ mysql_stmt_error(mi->sth));
+ return -1;
+ }
+
+ return 0;
}
static int execute_mysql(struct ulogd_pluginstance *upi,
const struct db_stmt *stmt)
{
struct mysql_instance *mi = (struct mysql_instance *) upi->private;
- int ret;
- MYSQL_RES * result;
+ struct mysql_buffer *buffers;
+ MYSQL_BIND *bindings;
+ unsigned int i;
+ int rv = -1;
- ret = mysql_real_query(mi->dbh, stmt->sql, stmt->len);
- if (ret) {
- ulogd_log(ULOGD_ERROR, "execute failed (%s)\n",
- mysql_error(mi->dbh));
- return -1;
+ if (!(bindings = calloc(stmt->nr_params, sizeof(*bindings))))
+ goto err_return;
+
+ if (!(buffers = calloc(stmt->nr_params, sizeof(*buffers))))
+ goto err_free_bindings;
+
+ for (i = 0; i < stmt->nr_params; i++) {
+ struct db_stmt_arg *arg = &stmt->args[i];
+ union db_stmt_arg_value *val = &arg->value;
+
+ if (arg->null) {
+ bindings[i++].buffer_type = MYSQL_TYPE_NULL;
+ continue;
+ }
+
+ switch (arg->type) {
+ case ULOGD_RET_BOOL:
+ case ULOGD_RET_INT8:
+ case ULOGD_RET_UINT8:
+ bindings[i].buffer_type = MYSQL_TYPE_TINY;
+ bindings[i].buffer = val;
+ bindings[i].is_unsigned = arg->type != ULOGD_RET_INT8;
+ break;
+ case ULOGD_RET_INT16:
+ case ULOGD_RET_UINT16:
+ bindings[i].buffer_type = MYSQL_TYPE_SHORT;
+ bindings[i].buffer = val;
+ bindings[i].is_unsigned = arg->type != ULOGD_RET_INT16;
+ break;
+ case ULOGD_RET_INT32:
+ case ULOGD_RET_UINT32:
+ case ULOGD_RET_IPADDR:
+ bindings[i].buffer_type = MYSQL_TYPE_LONG;
+ bindings[i].buffer = val;
+ bindings[i].is_unsigned = arg->type != ULOGD_RET_INT32;
+ break;
+ case ULOGD_RET_INT64:
+ case ULOGD_RET_UINT64:
+ bindings[i].buffer_type = MYSQL_TYPE_LONGLONG;
+ bindings[i].buffer = val;
+ bindings[i].is_unsigned = arg->type != ULOGD_RET_INT64;
+ break;
+ case ULOGD_RET_STRING:
+ bindings[i].buffer_type = MYSQL_TYPE_STRING;
+ bindings[i].is_null = &buffers[i].is_null;
+ bindings[i].length = &buffers[i].length;
+ if (val->ptr) {
+ bindings[i].buffer = val->ptr;
+ bindings[i].buffer_length = arg->len + 1;
+ buffers[i].length = arg->len;
+ } else
+ buffers[i].is_null = 1;
+ break;
+ case ULOGD_RET_RAWSTR:
+ bindings[i].buffer_type = MYSQL_TYPE_BLOB;
+ bindings[i].is_null = &buffers[i].is_null;
+ bindings[i].length = &buffers[i].length;
+ if (val->ptr) {
+ bindings[i].buffer = val->ptr;
+ bindings[i].buffer_length = arg->len;
+ buffers[i].length = arg->len;
+ } else
+ buffers[i].is_null = 1;
+ break;
+ }
}
- result = mysql_use_result(mi->dbh);
- if (result) {
- mysql_free_result(result);
+
+ if (mysql_stmt_bind_param(mi->sth, bindings) != 0) {
+ ulogd_log(ULOGD_ERROR,
+ "Failed to bind statement parameters: %s\n",
+ mysql_stmt_error(mi->sth));
+ goto err_free_buffers;
}
- return 0;
+ if (mysql_stmt_execute(mi->sth) != 0) {
+ ulogd_log(ULOGD_ERROR,
+ "Failed to execute statement: %s\n",
+ mysql_stmt_error(mi->sth));
+ goto err_free_buffers;
+ }
+
+ rv = 0;
+
+err_free_buffers:
+ free (buffers);
+err_free_bindings:
+ free (bindings);
+err_return:
+ return rv;
}
static struct db_driver db_driver_mysql = {
- .get_columns = &get_columns_mysql,
- .get_column = &get_column_mysql,
- .open_db = &open_db_mysql,
- .close_db = &close_db_mysql,
- .escape_string = &escape_string_mysql,
- .execute = &execute_mysql,
+ .get_columns = get_columns_mysql,
+ .get_column = get_column_mysql,
+ .open_db = open_db_mysql,
+ .close_db = close_db_mysql,
+ .prepare = prepare_mysql,
+ .execute = execute_mysql,
};
static int configure_mysql(struct ulogd_pluginstance *upi,
@@ -254,11 +353,11 @@ static struct ulogd_plugin plugin_mysql = {
},
.config_kset = &kset_mysql,
.priv_size = sizeof(struct mysql_instance),
- .configure = &configure_mysql,
- .start = &ulogd_db_start,
- .stop = &ulogd_db_stop,
- .signal = &ulogd_db_signal,
- .interp = &ulogd_db_interp,
+ .configure = configure_mysql,
+ .start = ulogd_db_start,
+ .stop = ulogd_db_stop,
+ .signal = ulogd_db_signal,
+ .interp = ulogd_db_interp,
.version = VERSION,
};
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 30/34] output: pgsql: remove a couple of struct members
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (28 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 29/34] output: mysql: " Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 31/34] output: pgsql: remove variable-length arrays Jeremy Sowden
` (4 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
The `pgsql_have_schemas` of `struct pgsql_instance` is never used. The
`pgres` member is always assigned and cleared in the same function.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/pgsql/ulogd_output_PGSQL.c | 46 ++++++++++++++++---------------
1 file changed, 24 insertions(+), 22 deletions(-)
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index 9dfef7946775..70bfbee7f565 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -31,8 +31,6 @@ struct pgsql_instance {
struct db_instance db_inst;
PGconn *dbh;
- PGresult *pgres;
- unsigned char pgsql_have_schemas;
};
/* our configuration directives */
@@ -90,6 +88,7 @@ static int pgsql_namespace(struct ulogd_pluginstance *upi)
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
char pgbuf[strlen(PGSQL_HAVE_NAMESPACE_TEMPLATE) +
strlen(schema_ce(upi->config_kset).u.string) + 1];
+ PGresult *pgres;
if (!pi->dbh)
return -1;
@@ -98,28 +97,28 @@ static int pgsql_namespace(struct ulogd_pluginstance *upi)
schema_ce(upi->config_kset).u.string);
ulogd_log(ULOGD_DEBUG, "%s\n", pgbuf);
- pi->pgres = PQexec(pi->dbh, pgbuf);
- if (!pi->pgres) {
+ pgres = PQexec(pi->dbh, pgbuf);
+ if (!pgres) {
ulogd_log(ULOGD_DEBUG, "\n result false");
return -1;
}
- if (PQresultStatus(pi->pgres) == PGRES_TUPLES_OK) {
- if (PQntuples(pi->pgres)) {
+ if (PQresultStatus(pgres) == PGRES_TUPLES_OK) {
+ if (PQntuples(pgres)) {
ulogd_log(ULOGD_DEBUG, "using schema %s\n",
schema_ce(upi->config_kset).u.string);
pi->db_inst.schema = schema_ce(upi->config_kset).u.string;
} else {
ulogd_log(ULOGD_ERROR, "schema %s not found: %s\n",
schema_ce(upi->config_kset).u.string, PQerrorMessage(pi->dbh));
- PQclear(pi->pgres);
+ PQclear(pgres);
return -1;
}
} else {
pi->db_inst.schema = NULL;
}
- PQclear(pi->pgres);
+ PQclear(pgres);
return 0;
}
@@ -137,6 +136,7 @@ static int get_columns_pgsql(struct ulogd_pluginstance *upi)
char pgbuf[strlen(PGSQL_GETCOLUMN_TEMPLATE_SCHEMA)
+ strlen(table_ce(upi->config_kset).u.string)
+ strlen(pi->db_inst.schema) + 2];
+ PGresult *pgres;
int rv;
if (!pi->dbh) {
@@ -156,27 +156,27 @@ static int get_columns_pgsql(struct ulogd_pluginstance *upi)
ulogd_log(ULOGD_DEBUG, "%s\n", pgbuf);
- pi->pgres = PQexec(pi->dbh, pgbuf);
- if (!pi->pgres) {
+ pgres = PQexec(pi->dbh, pgbuf);
+ if (!pgres) {
ulogd_log(ULOGD_DEBUG, "result false (%s)",
PQerrorMessage(pi->dbh));
return -1;
}
- if (PQresultStatus(pi->pgres) != PGRES_TUPLES_OK) {
+ if (PQresultStatus(pgres) != PGRES_TUPLES_OK) {
ulogd_log(ULOGD_DEBUG, "pres_command_not_ok (%s)",
PQerrorMessage(pi->dbh));
- PQclear(pi->pgres);
+ PQclear(pgres);
return -1;
}
- rv = ulogd_db_alloc_input_keys(upi, PQntuples(pi->pgres), pi->pgres);
+ rv = ulogd_db_alloc_input_keys(upi, PQntuples(pgres), pgres);
/* ID (starting by '.') is a sequence */
if (!rv && upi->input.keys[0].name[0] == '.')
upi->input.keys[0].flags |= ULOGD_KEYF_INACTIVE;
- PQclear(pi->pgres);
+ PQclear(pgres);
return rv;
}
@@ -207,6 +207,7 @@ static int open_db_pgsql(struct ulogd_pluginstance *upi)
char *connstr = connstr_ce(upi->config_kset).u.string;
char *schema = NULL;
char pgbuf[128];
+ PGresult *pgres;
if (!connstr[0]) {
char *server = host_ce(upi->config_kset).u.string;
@@ -260,13 +261,13 @@ static int open_db_pgsql(struct ulogd_pluginstance *upi)
if (!(schema == NULL) && (strcmp(schema,"public"))) {
snprintf(pgbuf, 128, "SET search_path='%.63s', \"$user\", 'public'", schema);
- pi->pgres = PQexec(pi->dbh, pgbuf);
- if ((PQresultStatus(pi->pgres) == PGRES_COMMAND_OK)) {
- PQclear(pi->pgres);
+ pgres = PQexec(pi->dbh, pgbuf);
+ if ((PQresultStatus(pgres) == PGRES_COMMAND_OK)) {
+ PQclear(pgres);
} else {
ulogd_log(ULOGD_ERROR, "could not set search path to (%s): %s\n",
schema, PQerrorMessage(pi->dbh));
- PQclear(pi->pgres);
+ PQclear(pgres);
close_db_pgsql(upi);
return -1;
}
@@ -286,16 +287,17 @@ static int execute_pgsql(struct ulogd_pluginstance *upi,
const struct db_stmt *stmt)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
+ PGresult *pgres;
- pi->pgres = PQexec(pi->dbh, stmt->sql);
- if (!(pi->pgres && ((PQresultStatus(pi->pgres) == PGRES_COMMAND_OK)
- || (PQresultStatus(pi->pgres) == PGRES_TUPLES_OK)))) {
+ pgres = PQexec(pi->dbh, stmt->sql);
+ if (!(pgres && ((PQresultStatus(pgres) == PGRES_COMMAND_OK)
+ || (PQresultStatus(pgres) == PGRES_TUPLES_OK)))) {
ulogd_log(ULOGD_ERROR, "execute failed (%s)\n",
PQerrorMessage(pi->dbh));
return -1;
}
- PQclear(pi->pgres);
+ PQclear(pgres);
return 0;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 31/34] output: pgsql: remove variable-length arrays
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (29 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 30/34] output: pgsql: remove a couple of struct members Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 32/34] output: pgsql: tidy up `open_db_pgsql` and fix memory leak Jeremy Sowden
` (3 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
VLA's are deprecated.
Group all the SQL macros at the top of the file and format them.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/pgsql/ulogd_output_PGSQL.c | 95 ++++++++++++++++++++-----------
1 file changed, 62 insertions(+), 33 deletions(-)
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index 70bfbee7f565..b125674b7364 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -79,20 +79,46 @@ static struct config_keyset pgsql_kset = {
#define schema_ce(x) ((x)->ces[DB_CE_NUM + 5])
#define connstr_ce(x) ((x)->ces[DB_CE_NUM + 6])
-#define PGSQL_HAVE_NAMESPACE_TEMPLATE \
- "SELECT nspname FROM pg_namespace n WHERE n.nspname='%s'"
+#define PGSQL_HAVE_NAMESPACE_TEMPLATE \
+ "SELECT nspname " \
+ "FROM pg_namespace n " \
+ "WHERE n.nspname = '%s'"
+
+#define PGSQL_GETCOLUMN_TEMPLATE \
+ "SELECT a.attname " \
+ "FROM pg_class c, pg_attribute a " \
+ "WHERE c.relname = '%s' " \
+ "AND a.attnum > 0 " \
+ "AND a.attrelid = c.oid " \
+ "ORDER BY a.attnum"
+
+#define PGSQL_GETCOLUMN_TEMPLATE_SCHEMA \
+ "SELECT a.attname " \
+ "FROM pg_attribute a, pg_class c " \
+ "LEFT JOIN pg_namespace n ON c.relnamespace = n.oid " \
+ "WHERE c.relname = '%s' " \
+ "AND n.nspname = '%s' " \
+ "AND a.attnum > 0 " \
+ "AND a.attrelid = c.oid " \
+ "AND a.attisdropped = FALSE " \
+ "ORDER BY a.attnum"
/* Determine if server support schemas */
static int pgsql_namespace(struct ulogd_pluginstance *upi)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
- char pgbuf[strlen(PGSQL_HAVE_NAMESPACE_TEMPLATE) +
- strlen(schema_ce(upi->config_kset).u.string) + 1];
PGresult *pgres;
+ char *pgbuf;
+ int rv = -1;
if (!pi->dbh)
return -1;
+ pgbuf = malloc(sizeof(PGSQL_HAVE_NAMESPACE_TEMPLATE) +
+ strlen(schema_ce(upi->config_kset).u.string));
+ if (!pgbuf)
+ return -1;
+
sprintf(pgbuf, PGSQL_HAVE_NAMESPACE_TEMPLATE,
schema_ce(upi->config_kset).u.string);
ulogd_log(ULOGD_DEBUG, "%s\n", pgbuf);
@@ -100,7 +126,7 @@ static int pgsql_namespace(struct ulogd_pluginstance *upi)
pgres = PQexec(pi->dbh, pgbuf);
if (!pgres) {
ulogd_log(ULOGD_DEBUG, "\n result false");
- return -1;
+ goto err_free_buf;
}
if (PQresultStatus(pgres) == PGRES_TUPLES_OK) {
@@ -108,51 +134,50 @@ static int pgsql_namespace(struct ulogd_pluginstance *upi)
ulogd_log(ULOGD_DEBUG, "using schema %s\n",
schema_ce(upi->config_kset).u.string);
pi->db_inst.schema = schema_ce(upi->config_kset).u.string;
- } else {
+ rv = 0;
+ } else
ulogd_log(ULOGD_ERROR, "schema %s not found: %s\n",
- schema_ce(upi->config_kset).u.string, PQerrorMessage(pi->dbh));
- PQclear(pgres);
- return -1;
- }
+ schema_ce(upi->config_kset).u.string,
+ PQerrorMessage(pi->dbh));
} else {
pi->db_inst.schema = NULL;
+ rv = 0;
}
PQclear(pgres);
- return 0;
-}
-
-#define PGSQL_GETCOLUMN_TEMPLATE \
- "SELECT a.attname FROM pg_class c, pg_attribute a WHERE c.relname ='%s' AND a.attnum>0 AND a.attrelid=c.oid ORDER BY a.attnum"
+err_free_buf:
+ free(pgbuf);
-#define PGSQL_GETCOLUMN_TEMPLATE_SCHEMA \
- "SELECT a.attname FROM pg_attribute a, pg_class c LEFT JOIN pg_namespace n ON c.relnamespace=n.oid WHERE c.relname ='%s' AND n.nspname='%s' AND a.attnum>0 AND a.attrelid=c.oid AND a.attisdropped=FALSE ORDER BY a.attnum"
+ return rv;
+}
/* find out which columns the table has */
static int get_columns_pgsql(struct ulogd_pluginstance *upi)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
- char pgbuf[strlen(PGSQL_GETCOLUMN_TEMPLATE_SCHEMA)
- + strlen(table_ce(upi->config_kset).u.string)
- + strlen(pi->db_inst.schema) + 2];
PGresult *pgres;
- int rv;
+ char *pgbuf;
+ int rv = -1;
if (!pi->dbh) {
ulogd_log(ULOGD_ERROR, "no database handle\n");
return -1;
}
- if (pi->db_inst.schema) {
- snprintf(pgbuf, sizeof(pgbuf)-1,
- PGSQL_GETCOLUMN_TEMPLATE_SCHEMA,
- table_ce(upi->config_kset).u.string,
- pi->db_inst.schema);
- } else {
- snprintf(pgbuf, sizeof(pgbuf)-1, PGSQL_GETCOLUMN_TEMPLATE,
- table_ce(upi->config_kset).u.string);
- }
+ pgbuf = malloc(sizeof(PGSQL_GETCOLUMN_TEMPLATE_SCHEMA) +
+ strlen(table_ce(upi->config_kset).u.string) +
+ strlen(pi->db_inst.schema));
+ if (!pgbuf)
+ return -1;
+
+ if (pi->db_inst.schema)
+ sprintf(pgbuf, PGSQL_GETCOLUMN_TEMPLATE_SCHEMA,
+ table_ce(upi->config_kset).u.string,
+ pi->db_inst.schema);
+ else
+ sprintf(pgbuf, PGSQL_GETCOLUMN_TEMPLATE,
+ table_ce(upi->config_kset).u.string);
ulogd_log(ULOGD_DEBUG, "%s\n", pgbuf);
@@ -160,14 +185,13 @@ static int get_columns_pgsql(struct ulogd_pluginstance *upi)
if (!pgres) {
ulogd_log(ULOGD_DEBUG, "result false (%s)",
PQerrorMessage(pi->dbh));
- return -1;
+ goto err_free_buf;
}
if (PQresultStatus(pgres) != PGRES_TUPLES_OK) {
ulogd_log(ULOGD_DEBUG, "pres_command_not_ok (%s)",
PQerrorMessage(pi->dbh));
- PQclear(pgres);
- return -1;
+ goto err_clear_res;
}
rv = ulogd_db_alloc_input_keys(upi, PQntuples(pgres), pgres);
@@ -176,7 +200,12 @@ static int get_columns_pgsql(struct ulogd_pluginstance *upi)
if (!rv && upi->input.keys[0].name[0] == '.')
upi->input.keys[0].flags |= ULOGD_KEYF_INACTIVE;
+err_clear_res:
PQclear(pgres);
+
+err_free_buf:
+ free(pgbuf);
+
return rv;
}
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 32/34] output: pgsql: tidy up `open_db_pgsql` and fix memory leak
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (30 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 31/34] output: pgsql: remove variable-length arrays Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 33/34] output: pgsql: add prep & exec support Jeremy Sowden
` (2 subsequent siblings)
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
* Remove some excess parentheses and a superfluous local variable.
* Tighten up the scope of a couple of other variables.
* Streamline the error-handling.
* Free the connexion string.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/pgsql/ulogd_output_PGSQL.c | 57 +++++++++++++++++--------------
1 file changed, 32 insertions(+), 25 deletions(-)
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index b125674b7364..8c9aabf873ca 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -232,11 +232,9 @@ static int close_db_pgsql(struct ulogd_pluginstance *upi)
static int open_db_pgsql(struct ulogd_pluginstance *upi)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
- int len;
char *connstr = connstr_ce(upi->config_kset).u.string;
- char *schema = NULL;
- char pgbuf[128];
- PGresult *pgres;
+ PGresult *pgres = NULL;
+ int rv = -1;
if (!connstr[0]) {
char *server = host_ce(upi->config_kset).u.string;
@@ -245,10 +243,11 @@ static int open_db_pgsql(struct ulogd_pluginstance *upi)
char *pass = pass_ce(upi->config_kset).u.string;
char *db = db_ce(upi->config_kset).u.string;
char *cp;
+
/* 80 is more than what we need for the fixed parts below */
- len = 80 + strlen(user) + strlen(db);
+ size_t len = 80 + strlen(user) + strlen(db);
- /* hostname and and password are the only optionals */
+ /* hostname and and password are the only optionals */
if (server[0])
len += strlen(server);
if (pass[0])
@@ -271,39 +270,47 @@ static int open_db_pgsql(struct ulogd_pluginstance *upi)
if (pass[0])
cp += sprintf(cp, " password=%s", pass);
}
+
pi->dbh = PQconnectdb(connstr);
if (PQstatus(pi->dbh) != CONNECTION_OK) {
ulogd_log(ULOGD_ERROR, "unable to connect to db (%s): %s\n",
connstr, PQerrorMessage(pi->dbh));
- close_db_pgsql(upi);
- return -1;
+ goto err_close_db;
}
- if (pgsql_namespace(upi)) {
+ if (pgsql_namespace(upi) < 0) {
ulogd_log(ULOGD_ERROR, "problem testing for pgsql schemas\n");
- close_db_pgsql(upi);
- return -1;
+ goto err_close_db;
}
- pi=(struct pgsql_instance *)upi->private;
- schema = pi->db_inst.schema;
+ if (pi->db_inst.schema != NULL && strcmp(pi->db_inst.schema,"public") != 0) {
+ char pgbuf[128];
- if (!(schema == NULL) && (strcmp(schema,"public"))) {
- snprintf(pgbuf, 128, "SET search_path='%.63s', \"$user\", 'public'", schema);
+ snprintf(pgbuf, sizeof(pgbuf),
+ "SET search_path='%.63s', \"$user\", 'public'",
+ pi->db_inst.schema);
pgres = PQexec(pi->dbh, pgbuf);
- if ((PQresultStatus(pgres) == PGRES_COMMAND_OK)) {
- PQclear(pgres);
- } else {
- ulogd_log(ULOGD_ERROR, "could not set search path to (%s): %s\n",
- schema, PQerrorMessage(pi->dbh));
- PQclear(pgres);
- close_db_pgsql(upi);
- return -1;
+ if (PQresultStatus(pgres) != PGRES_COMMAND_OK) {
+ ulogd_log(ULOGD_ERROR,
+ "could not set search path to (%s): %s\n",
+ pi->db_inst.schema, PQerrorMessage(pi->dbh));
+ goto err_free_result;
}
-
}
- return 0;
+ rv = 0;
+
+err_free_result:
+ PQclear(pgres);
+
+err_close_db:
+ if (rv == -1)
+ close_db_pgsql(upi);
+
+ if (connstr != connstr_ce(upi->config_kset).u.string)
+ free(connstr);
+
+ return rv;
}
static int escape_string_pgsql(struct ulogd_pluginstance *upi,
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 33/34] output: pgsql: add prep & exec support
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (31 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 32/34] output: pgsql: tidy up `open_db_pgsql` and fix memory leak Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 34/34] output: sqlite3: reimplement using the common DB API Jeremy Sowden
2022-11-30 10:27 ` [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Pablo Neira Ayuso
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
Now that the common DB API offers prep-exec support, update the Postgres
plug-in to use it.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/pgsql/ulogd_output_PGSQL.c | 145 +++++++++++++++++++++++++-----
1 file changed, 123 insertions(+), 22 deletions(-)
diff --git a/output/pgsql/ulogd_output_PGSQL.c b/output/pgsql/ulogd_output_PGSQL.c
index 8c9aabf873ca..01afec1003f0 100644
--- a/output/pgsql/ulogd_output_PGSQL.c
+++ b/output/pgsql/ulogd_output_PGSQL.c
@@ -10,6 +10,7 @@
*
*/
+#include <limits.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
@@ -27,6 +28,8 @@
#define DEBUGP(x, args...)
#endif
+#define BUF_SIZE (sizeof(uint64_t) * CHAR_BIT / 3 + 1)
+
struct pgsql_instance {
struct db_instance db_inst;
@@ -217,6 +220,12 @@ get_column_pgsql(void *vp, unsigned int i)
return PQgetvalue(pgres, i, 0);
}
+static unsigned int
+format_param_pgsql(char *buf, unsigned int size, unsigned int idx)
+{
+ return snprintf(buf, size, "$%u,", idx + 1);
+}
+
static int close_db_pgsql(struct ulogd_pluginstance *upi)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
@@ -313,38 +322,130 @@ err_close_db:
return rv;
}
-static int escape_string_pgsql(struct ulogd_pluginstance *upi,
- char *dst, const char *src, unsigned int len)
+static int
+prepare_pgsql(struct ulogd_pluginstance *upi, const struct db_stmt *stmt)
{
- return PQescapeString(dst, src, strlen(src));
+ struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
+ PGresult *pgres;
+ int rv;
+
+ pgres = PQprepare(pi->dbh, procedure_ce(upi->config_kset).u.string,
+ stmt->sql, 0, NULL);
+ if (PQresultStatus(pgres) != PGRES_COMMAND_OK) {
+ ulogd_log(ULOGD_ERROR, "Failed to prepare statement: %s\n",
+ PQerrorMessage(pi->dbh));
+ rv = -1;
+ } else
+ rv = 0;
+
+ PQclear(pgres);
+ return rv;
}
-static int execute_pgsql(struct ulogd_pluginstance *upi,
- const struct db_stmt *stmt)
+static int
+execute_pgsql(struct ulogd_pluginstance *upi, const struct db_stmt *stmt)
{
struct pgsql_instance *pi = (struct pgsql_instance *) upi->private;
+ char **bindings, *buffers;
PGresult *pgres;
+ unsigned int i;
+ int rv = -1;
- pgres = PQexec(pi->dbh, stmt->sql);
- if (!(pgres && ((PQresultStatus(pgres) == PGRES_COMMAND_OK)
- || (PQresultStatus(pgres) == PGRES_TUPLES_OK)))) {
- ulogd_log(ULOGD_ERROR, "execute failed (%s)\n",
- PQerrorMessage(pi->dbh));
+ if (!(bindings = malloc(stmt->nr_params * sizeof(*bindings)))) {
+ ulogd_log(ULOGD_ERROR, "Memory allocation failure\n");
return -1;
}
+ if (!(buffers = malloc(stmt->nr_params * BUF_SIZE))) {
+ ulogd_log(ULOGD_ERROR, "Memory allocation failure\n");
+ goto err_free_bindings;
+ }
+
+ for (i = 0; i < stmt->nr_params; i++) {
+ struct db_stmt_arg *arg = &stmt->args[i];
+ union db_stmt_arg_value *val = &arg->value;
+
+ if (arg->null) {
+ bindings[i] = NULL;
+ continue;
+ }
+
+ switch (arg->type) {
+ case ULOGD_RET_STRING:
+ case ULOGD_RET_RAWSTR:
+ bindings[i] = val->ptr;
+ continue;
+ }
+
+ bindings[i] = &buffers[i * BUF_SIZE];
+
+ switch (arg->type) {
+ case ULOGD_RET_BOOL:
+ snprintf(bindings[i], BUF_SIZE, "%d", val->b);
+ break;
+ case ULOGD_RET_INT8:
+ snprintf(bindings[i], BUF_SIZE, "%" PRId8, val->i8);
+ break;
+ case ULOGD_RET_UINT8:
+ snprintf(bindings[i], BUF_SIZE, "%" PRIu8, val->ui8);
+ break;
+ case ULOGD_RET_INT16:
+ snprintf(bindings[i], BUF_SIZE, "%" PRId16, val->i16);
+ break;
+ case ULOGD_RET_UINT16:
+ snprintf(bindings[i], BUF_SIZE, "%" PRIu16, val->ui16);
+ break;
+ case ULOGD_RET_INT32:
+ snprintf(bindings[i], BUF_SIZE, "%" PRId32, val->i32);
+ break;
+ case ULOGD_RET_IPADDR:
+ case ULOGD_RET_UINT32:
+ snprintf(bindings[i], BUF_SIZE, "%" PRIu32, val->ui32);
+ break;
+ case ULOGD_RET_INT64:
+ snprintf(bindings[i], BUF_SIZE, "%" PRId64, val->i64);
+ break;
+ case ULOGD_RET_UINT64:
+ snprintf(bindings[i], BUF_SIZE, "%" PRIu64, val->ui64);
+ break;
+ }
+ }
+
+ pgres = PQexecPrepared(pi->dbh, procedure_ce(upi->config_kset).u.string,
+ stmt->nr_params, (const char **) bindings, NULL,
+ NULL, 0);
+
+ switch (PQresultStatus(pgres)) {
+ case PGRES_COMMAND_OK:
+ case PGRES_TUPLES_OK:
+ rv = 0;
+ break;
+ default:
+ ulogd_log(ULOGD_ERROR,
+ "Failed to execute statement: status = %s, error = %s\n",
+ PQresStatus(PQresultStatus(pgres)),
+ PQerrorMessage(pi->dbh));
+ break;
+ }
+
PQclear(pgres);
- return 0;
+ free (buffers);
+
+err_free_bindings:
+ free (bindings);
+
+ return rv;
}
static struct db_driver db_driver_pgsql = {
- .get_columns = &get_columns_pgsql,
- .get_column = &get_column_pgsql,
- .open_db = &open_db_pgsql,
- .close_db = &close_db_pgsql,
- .escape_string = &escape_string_pgsql,
- .execute = &execute_pgsql,
+ .get_columns = get_columns_pgsql,
+ .get_column = get_column_pgsql,
+ .format_param = format_param_pgsql,
+ .open_db = open_db_pgsql,
+ .close_db = close_db_pgsql,
+ .prepare = prepare_pgsql,
+ .execute = execute_pgsql,
};
static int configure_pgsql(struct ulogd_pluginstance *upi,
@@ -367,11 +468,11 @@ static struct ulogd_plugin pgsql_plugin = {
},
.config_kset = &pgsql_kset,
.priv_size = sizeof(struct pgsql_instance),
- .configure = &configure_pgsql,
- .start = &ulogd_db_start,
- .stop = &ulogd_db_stop,
- .signal = &ulogd_db_signal,
- .interp = &ulogd_db_interp,
+ .configure = configure_pgsql,
+ .start = ulogd_db_start,
+ .stop = ulogd_db_stop,
+ .signal = ulogd_db_signal,
+ .interp = ulogd_db_interp,
.version = VERSION,
};
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* [PATCH ulogd2 v2 v2 34/34] output: sqlite3: reimplement using the common DB API
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (32 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 33/34] output: pgsql: add prep & exec support Jeremy Sowden
@ 2022-11-29 21:47 ` Jeremy Sowden
2022-11-30 10:27 ` [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Pablo Neira Ayuso
34 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-29 21:47 UTC (permalink / raw)
To: Netfilter Devel
SQLite doesn't have support for escaping strings to embed them in SQL
statements. Therefore, the sqlite3 plug-in has not used the `util_db`
API. Now that the common API supports prep & exec, convert the sqlite3
plug-in to use it.
Signed-off-by: Jeremy Sowden <jeremy@azazel.net>
---
output/sqlite3/ulogd_output_SQLITE3.c | 459 +++++++++-----------------
1 file changed, 150 insertions(+), 309 deletions(-)
diff --git a/output/sqlite3/ulogd_output_SQLITE3.c b/output/sqlite3/ulogd_output_SQLITE3.c
index 32459dd6c4c5..2875c504d20e 100644
--- a/output/sqlite3/ulogd_output_SQLITE3.c
+++ b/output/sqlite3/ulogd_output_SQLITE3.c
@@ -30,392 +30,232 @@
* - port to ulogd-2.00
*/
+#ifdef DEBUG_SQLITE3
+#include <stdio.h>
+#endif
#include <stdlib.h>
#include <string.h>
-#include <arpa/inet.h>
+#include <sqlite3.h>
#include <ulogd/ulogd.h>
#include <ulogd/conffile.h>
-#include <sqlite3.h>
-#include <sys/queue.h>
+#include <ulogd/db.h>
-#define CFG_BUFFER_DEFAULT 10
-
-#if 0
+#ifdef DEBUG_SQLITE3
#define DEBUGP(x, args...) fprintf(stderr, x, ## args)
#else
#define DEBUGP(x, args...)
#endif
-struct field {
- TAILQ_ENTRY(field) link;
- char name[ULOGD_MAX_KEYLEN + 1];
- struct ulogd_key *key;
-};
-
-TAILQ_HEAD(field_lh, field);
-
-#define tailq_for_each(pos, head, link) \
- for (pos = (head).tqh_first; pos != NULL; pos = pos->link.tqe_next)
-
+#define SQLITE3_BUSY_TIMEOUT 300
-struct sqlite3_priv {
+struct sqlite3_instance {
+ struct db_instance db_inst;
sqlite3 *dbh; /* database handle we are using */
- struct field_lh fields;
- char *stmt;
sqlite3_stmt *p_stmt;
- struct {
- unsigned err_tbl_busy; /* "Table busy" */
- } stats;
};
-static struct config_keyset sqlite3_kset = {
- .num_ces = 3,
+static struct config_keyset kset_sqlite3 = {
+ .num_ces = DB_CE_NUM + 1,
.ces = {
+ DB_CES,
{
.key = "db",
.type = CONFIG_TYPE_STRING,
.options = CONFIG_OPT_MANDATORY,
},
- {
- .key = "table",
- .type = CONFIG_TYPE_STRING,
- .options = CONFIG_OPT_MANDATORY,
- },
},
};
-#define db_ce(pi) (pi)->config_kset->ces[0].u.string
-#define table_ce(pi) (pi)->config_kset->ces[1].u.string
-
-/* forward declarations */
-static int sqlite3_createstmt(struct ulogd_pluginstance *);
+#define db_ce(x) ((x)->ces[DB_CE_NUM + 0])
+#define SELECT_ALL_FROM "select * from "
static int
-add_row(struct ulogd_pluginstance *pi)
+get_columns_sqlite3(struct ulogd_pluginstance *upi)
{
- struct sqlite3_priv *priv = (void *)pi->private;
- int ret;
-
- ret = sqlite3_step(priv->p_stmt);
- if (ret == SQLITE_BUSY)
- priv->stats.err_tbl_busy++;
- else if (ret == SQLITE_ERROR) {
- ret = sqlite3_finalize(priv->p_stmt);
- priv->p_stmt = NULL;
-
- if (ret != SQLITE_SCHEMA) {
- ulogd_log(ULOGD_ERROR, "SQLITE3: step: %s\n",
- sqlite3_errmsg(priv->dbh));
- goto err_reset;
- }
- if (sqlite3_createstmt(pi) < 0) {
- ulogd_log(ULOGD_ERROR,
- "SQLITE3: Could not create statement.\n");
- goto err_reset;
- }
- }
+ struct sqlite3_instance *si = (struct sqlite3_instance *) upi->private;
+ char query[sizeof(SELECT_ALL_FROM) + CONFIG_VAL_STRING_LEN];
+ sqlite3_stmt *stmt;
+ int rv;
- ret = sqlite3_reset(priv->p_stmt);
+ snprintf(query, sizeof(query), SELECT_ALL_FROM "%s",
+ table_ce(upi->config_kset).u.string);
- return 0;
+ if (sqlite3_prepare_v2(si->dbh, query, -1, &stmt, NULL) != SQLITE_OK) {
+ ulogd_log(ULOGD_ERROR, "%s: prepare failed: %s\n",
+ __func__, sqlite3_errmsg(si->dbh));
+ return -1;
+ }
- err_reset:
- sqlite3_reset(priv->p_stmt);
+ rv = ulogd_db_alloc_input_keys(upi, sqlite3_column_count(stmt), stmt);
- return -1;
+ sqlite3_finalize(stmt);
+ return rv;
}
-
-/* our main output function, called by ulogd */
-static int
-sqlite3_interp(struct ulogd_pluginstance *pi)
+static const char *
+get_column_sqlite3(void *vp, unsigned int i)
{
- struct sqlite3_priv *priv = (void *)pi->private;
- struct field *f;
- int ret, i = 1;
-
- tailq_for_each(f, priv->fields, link) {
- struct ulogd_key *k_ret = f->key->u.source;
-
- if (f->key == NULL || !IS_VALID(*k_ret)) {
- sqlite3_bind_null(priv->p_stmt, i);
- i++;
- continue;
- }
-
- switch (f->key->type) {
- case ULOGD_RET_INT8:
- ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.i8);
- break;
-
- case ULOGD_RET_INT16:
- ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.i16);
- break;
-
- case ULOGD_RET_INT32:
- ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.i32);
- break;
-
- case ULOGD_RET_INT64:
- ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.i64);
- break;
+ sqlite3_stmt *schema_stmt = vp;
- case ULOGD_RET_UINT8:
- ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.ui8);
- break;
-
- case ULOGD_RET_UINT16:
- ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.ui16);
- break;
-
- case ULOGD_RET_UINT32:
- ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.ui32);
- break;
-
- case ULOGD_RET_IPADDR:
- case ULOGD_RET_UINT64:
- ret = sqlite3_bind_int64(priv->p_stmt, i, k_ret->u.value.ui64);
- break;
-
- case ULOGD_RET_BOOL:
- ret = sqlite3_bind_int(priv->p_stmt, i, k_ret->u.value.b);
- break;
-
- case ULOGD_RET_STRING:
- ret = sqlite3_bind_text(priv->p_stmt, i, k_ret->u.value.ptr,
- strlen(k_ret->u.value.ptr), SQLITE_STATIC);
- break;
-
- default:
- ret = SQLITE_OK;
- ulogd_log(ULOGD_NOTICE, "unknown type %d for %s\n",
- f->key->type, f->key->name);
- }
- if (ret != SQLITE_OK)
- goto err_bind;
-
- i++;
- }
-
- if (add_row(pi) < 0)
- return ULOGD_IRET_ERR;
-
- return ULOGD_IRET_OK;
-
- err_bind:
- ulogd_log(ULOGD_ERROR, "SQLITE: bind: %s\n", sqlite3_errmsg(priv->dbh));
-
- return ULOGD_IRET_ERR;
+ return sqlite3_column_name(schema_stmt, i);
}
-#define _SQLITE3_INSERTTEMPL "insert into X (Y) values (Z)"
-
-/* create the static part of our insert statement */
static int
-sqlite3_createstmt(struct ulogd_pluginstance *pi)
+open_db_sqlite3(struct ulogd_pluginstance *upi)
{
- struct sqlite3_priv *priv = (void *)pi->private;
- struct field *f;
- int i, cols = 0;
- char *stmt_pos;
-
- if (priv->stmt != NULL)
- free(priv->stmt);
-
- if ((priv->stmt = calloc(1, 1024)) == NULL) {
- ulogd_log(ULOGD_ERROR, "SQLITE3: out of memory\n");
- return -1;
- }
- stmt_pos = priv->stmt;
-
- stmt_pos += sprintf(stmt_pos, "insert into %s (", table_ce(pi));
-
- tailq_for_each(f, priv->fields, link) {
- stmt_pos += sprintf(stmt_pos, "%s,", f->name);
- cols++;
- }
-
- *(stmt_pos - 1) = ')';
-
- stmt_pos += sprintf(stmt_pos, " values (");
-
- for (i = 0; i < cols - 1; i++)
- stmt_pos += sprintf(stmt_pos, "?,");
-
- sprintf(stmt_pos, "?)");
- ulogd_log(ULOGD_DEBUG, "%s: stmt='%s'\n", pi->id, priv->stmt);
+ struct sqlite3_instance *si = (struct sqlite3_instance *) upi->private;
- DEBUGP("about to prepare statement.\n");
-
- sqlite3_prepare(priv->dbh, priv->stmt, -1, &priv->p_stmt, 0);
- if (priv->p_stmt == NULL) {
- ulogd_log(ULOGD_ERROR, "SQLITE3: prepare: %s\n",
- sqlite3_errmsg(priv->dbh));
+ if (sqlite3_open(db_ce(upi->config_kset).u.string,
+ &si->dbh) != SQLITE_OK) {
+ ulogd_log(ULOGD_ERROR, "%s: open failed: %s\n",
+ __func__, sqlite3_errmsg(si->dbh));
return -1;
}
- DEBUGP("statement prepared.\n");
+ /* Set the timeout so that we don't automatically fail if the table is
+ * busy.
+ */
+ sqlite3_busy_timeout(si->dbh, SQLITE3_BUSY_TIMEOUT);
return 0;
}
-
-static struct ulogd_key *
-ulogd_find_key(struct ulogd_pluginstance *pi, const char *name)
+static int
+close_db_sqlite3(struct ulogd_pluginstance *upi)
{
- char buf[ULOGD_MAX_KEYLEN + 1] = "";
- unsigned int i;
+ struct sqlite3_instance *si = (struct sqlite3_instance *) upi->private;
- /* replace all underscores with dots */
- for (i = 0; i < sizeof(buf) - 1 && name[i]; ++i)
- buf[i] = name[i] != '_' ? name[i] : '.';
+ if (si->p_stmt) {
+ sqlite3_finalize(si->p_stmt);
+ si->p_stmt = NULL;
+ }
- for (i = 0; i < pi->input.num_keys; i++) {
- if (strcmp(pi->input.keys[i].name, buf) == 0)
- return &pi->input.keys[i];
+ if (si->dbh) {
+ sqlite3_close(si->dbh);
+ si->dbh = NULL;
}
- return NULL;
+ return 0;
}
-#define SELECT_ALL_STR "select * from "
-#define SELECT_ALL_LEN sizeof(SELECT_ALL_STR)
-
static int
-db_count_cols(struct ulogd_pluginstance *pi, sqlite3_stmt **stmt)
+prepare_sqlite3(struct ulogd_pluginstance *upi, const struct db_stmt *stmt)
{
- struct sqlite3_priv *priv = (void *)pi->private;
- char query[SELECT_ALL_LEN + CONFIG_VAL_STRING_LEN] = SELECT_ALL_STR;
+ struct sqlite3_instance *si = (struct sqlite3_instance *) upi->private;
- strncat(query, table_ce(pi), sizeof(query) - strlen(query) - 1);
+ if (si->p_stmt) {
+ sqlite3_finalize(si->p_stmt);
+ si->p_stmt = NULL;
+ }
- if (sqlite3_prepare(priv->dbh, query, -1, stmt, 0) != SQLITE_OK)
+ if (sqlite3_prepare_v2(si->dbh, stmt->sql, stmt->len + 1,
+ &si->p_stmt, NULL) != SQLITE_OK) {
+ ulogd_log(ULOGD_ERROR, "%s: prepare failed: %s\n",
+ __func__, sqlite3_errmsg(si->dbh));
return -1;
+ }
- return sqlite3_column_count(*stmt);
+ return 0;
}
-/* initialize DB, possibly creating it */
static int
-sqlite3_init_db(struct ulogd_pluginstance *pi)
+execute_sqlite3(struct ulogd_pluginstance *upi, const struct db_stmt *stmt)
{
- struct sqlite3_priv *priv = (void *)pi->private;
- sqlite3_stmt *schema_stmt;
- int col, num_cols;
-
- if (priv->dbh == NULL) {
- ulogd_log(ULOGD_ERROR, "SQLITE3: No database handle.\n");
- return -1;
- }
-
- num_cols = db_count_cols(pi, &schema_stmt);
- if (num_cols <= 0) {
- ulogd_log(ULOGD_ERROR, "table `%s' is empty or missing in "
- "file `%s'. Did you created this "
- "table in the database file? Please, "
- "see ulogd2 documentation.\n",
- table_ce(pi), db_ce(pi));
- return -1;
- }
+ struct sqlite3_instance *si = (struct sqlite3_instance *) upi->private;
+ unsigned int i;
+ int rv;
- for (col = 0; col < num_cols; col++) {
- struct field *f;
+ for (i = 0; i < stmt->nr_params; i++) {
+ struct db_stmt_arg *arg = &stmt->args[i];
+ union db_stmt_arg_value *val = &arg->value;
- /* prepend it to the linked list */
- if ((f = calloc(1, sizeof(struct field))) == NULL) {
- ulogd_log(ULOGD_ERROR, "SQLITE3: out of memory\n");
- return -1;
+ if (arg->null) {
+ sqlite3_bind_null(si->p_stmt, i + 1);
+ continue;
}
- snprintf(f->name, sizeof(f->name),
- "%s", sqlite3_column_name(schema_stmt, col));
-
- DEBUGP("field '%s' found\n", f->name);
- if ((f->key = ulogd_find_key(pi, f->name)) == NULL) {
- ulogd_log(ULOGD_ERROR,
- "SQLITE3: unknown input key: %s\n", f->name);
- free(f);
- return -1;
+ switch (arg->type) {
+ case ULOGD_RET_BOOL:
+ rv = sqlite3_bind_int(si->p_stmt, i + 1, val->b);
+ break;
+ case ULOGD_RET_INT8:
+ rv = sqlite3_bind_int(si->p_stmt, i + 1, val->i8);
+ break;
+ case ULOGD_RET_INT16:
+ rv = sqlite3_bind_int(si->p_stmt, i + 1, val->i16);
+ break;
+ case ULOGD_RET_INT32:
+ rv = sqlite3_bind_int(si->p_stmt, i + 1, val->i32);
+ break;
+ case ULOGD_RET_INT64:
+ rv = sqlite3_bind_int(si->p_stmt, i + 1, val->i64);
+ break;
+ case ULOGD_RET_UINT8:
+ rv = sqlite3_bind_int(si->p_stmt, i + 1, val->ui8);
+ break;
+ case ULOGD_RET_UINT16:
+ rv = sqlite3_bind_int(si->p_stmt, i + 1, val->ui16);
+ break;
+ case ULOGD_RET_UINT32:
+ rv = sqlite3_bind_int(si->p_stmt, i + 1, val->ui32);
+ break;
+ case ULOGD_RET_IPADDR:
+ case ULOGD_RET_UINT64:
+ rv = sqlite3_bind_int64(si->p_stmt, i + 1, val->ui64);
+ break;
+ case ULOGD_RET_STRING:
+ rv = sqlite3_bind_text(si->p_stmt, i + 1, val->ptr,
+ arg->len, SQLITE_STATIC);
+ break;
+ case ULOGD_RET_RAWSTR:
+ rv = sqlite3_bind_blob(si->p_stmt, i + 1, val->ptr,
+ arg->len, SQLITE_STATIC);
+ break;
}
- TAILQ_INSERT_TAIL(&priv->fields, f, link);
+ if (rv != SQLITE_OK) {
+ ulogd_log(ULOGD_ERROR, "%s: bind %u failed: %s\n",
+ __func__, i + 1, sqlite3_errmsg(si->dbh));
+ rv = -1;
+ goto err_clear_bindings;
+ }
}
- sqlite3_finalize(schema_stmt);
-
- return 0;
-}
+ rv = sqlite3_step(si->p_stmt);
-#define SQLITE3_BUSY_TIMEOUT 300
+ if (rv != SQLITE_DONE) {
+ ulogd_log(ULOGD_ERROR, "%s: step failed: %s\n",
+ __func__, sqlite3_errmsg(si->dbh));
+ rv = -1;
+ } else
+ rv = 0;
-static int
-sqlite3_configure(struct ulogd_pluginstance *pi,
- struct ulogd_pluginstance_stack *stack)
-{
- /* struct sqlite_priv *priv = (void *)pi->private; */
+ sqlite3_reset(si->p_stmt);
- config_parse_file(pi->id, pi->config_kset);
+err_clear_bindings:
+ sqlite3_clear_bindings(si->p_stmt);
- if (ulogd_wildcard_inputkeys(pi) < 0)
- return -1;
-
- DEBUGP("%s: db='%s' table='%s'\n", pi->id, db_ce(pi), table_ce(pi));
-
- return 0;
+ return rv;
}
-static int
-sqlite3_start(struct ulogd_pluginstance *pi)
-{
- struct sqlite3_priv *priv = (void *)pi->private;
-
- TAILQ_INIT(&priv->fields);
-
- if (sqlite3_open(db_ce(pi), &priv->dbh) != SQLITE_OK) {
- ulogd_log(ULOGD_ERROR, "SQLITE3: %s\n", sqlite3_errmsg(priv->dbh));
- return -1;
- }
-
- /* set the timeout so that we don't automatically fail
- if the table is busy */
- sqlite3_busy_timeout(priv->dbh, SQLITE3_BUSY_TIMEOUT);
-
- /* read the fieldnames to know which values to insert */
- if (sqlite3_init_db(pi) < 0) {
- ulogd_log(ULOGD_ERROR, "SQLITE3: Could not read database fieldnames.\n");
- return -1;
- }
-
- /* create and prepare the actual insert statement */
- if (sqlite3_createstmt(pi) < 0) {
- ulogd_log(ULOGD_ERROR, "SQLITE3: Could not create statement.\n");
- return -1;
- }
-
- return 0;
-}
+static struct db_driver sqlite3_db_driver = {
+ .get_columns = get_columns_sqlite3,
+ .get_column = get_column_sqlite3,
+ .open_db = open_db_sqlite3,
+ .close_db = close_db_sqlite3,
+ .prepare = prepare_sqlite3,
+ .execute = execute_sqlite3,
+};
-/* give us an opportunity to close the database down properly */
static int
-sqlite3_stop(struct ulogd_pluginstance *pi)
+configure_sqlite3(struct ulogd_pluginstance *upi,
+ struct ulogd_pluginstance_stack *stack)
{
- struct sqlite3_priv *priv = (void *)pi->private;
+ struct db_instance *di = (struct db_instance *) &upi->private;
+ di->driver = &sqlite3_db_driver;
- /* free up our prepared statements so we can close the db */
- if (priv->p_stmt) {
- sqlite3_finalize(priv->p_stmt);
- DEBUGP("prepared statement finalized\n");
- }
-
- if (priv->dbh == NULL)
- return -1;
-
- sqlite3_close(priv->dbh);
-
- priv->dbh = NULL;
-
- return 0;
+ return ulogd_db_configure(upi, stack);
}
static struct ulogd_plugin sqlite3_plugin = {
@@ -426,12 +266,13 @@ static struct ulogd_plugin sqlite3_plugin = {
.output = {
.type = ULOGD_DTYPE_SINK,
},
- .config_kset = &sqlite3_kset,
- .priv_size = sizeof(struct sqlite3_priv),
- .configure = sqlite3_configure,
- .start = sqlite3_start,
- .stop = sqlite3_stop,
- .interp = sqlite3_interp,
+ .config_kset = &kset_sqlite3,
+ .priv_size = sizeof(struct sqlite3_instance),
+ .configure = configure_sqlite3,
+ .start = ulogd_db_start,
+ .stop = ulogd_db_stop,
+ .signal = ulogd_db_signal,
+ .interp = ulogd_db_interp,
.version = VERSION,
};
--
2.35.1
^ permalink raw reply related [flat|nested] 40+ messages in thread* Re: [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins
2022-11-29 21:47 [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Jeremy Sowden
` (33 preceding siblings ...)
2022-11-29 21:47 ` [PATCH ulogd2 v2 v2 34/34] output: sqlite3: reimplement using the common DB API Jeremy Sowden
@ 2022-11-30 10:27 ` Pablo Neira Ayuso
2022-11-30 16:03 ` Jeremy Sowden
34 siblings, 1 reply; 40+ messages in thread
From: Pablo Neira Ayuso @ 2022-11-30 10:27 UTC (permalink / raw)
To: Jeremy Sowden; +Cc: Netfilter Devel
On Tue, Nov 29, 2022 at 09:47:15PM +0000, Jeremy Sowden wrote:
> In his feedback to my last series of clean-up patches at the beginning
> of the year, Pablo suggested consolidating some parallel implementations
> of the same functionality in the SQL output plug-ins. I already had
> some patches in the works aimed at tidying up the DB API. This
> patch-set is the result. In addition to the suggested de-duping and
> other tidy-ups, I have added prep & exec support in order to convert the
> sqlite3 plug-in to the DB API, and updated the MySQL and PostgreSQL
> plug-ins to use it as well (DBI doesn't do prep & exec).
>
> This patch-set is structured as follows.
>
> * Patches 1-4 are bug-fixes.
> * Patches 5-13 are miscellaneous tidying.
> * Patch 14 does the consolidation Pablo suggested.
> * Patches 15-26 refactor and clean up the common DB API.
> * Patches 27-28 add prep & exec support to the common DB API.
> * Patch 29 converts the MySQL plug-in to use prep & exec.
> * Patch 30-33 tidy up and convert the PostgreSQL plug-in to use prep &
> exec.
> * Patch 34 converts the SQLite plug-in to use the common DB API.
It's great that ulogd2 is getting updates, thanks a lot.
But would it be possible to start with a smaller batch? We review
integrate and then you follow up with more updates.
I'll aim at being swift on it.
I'd suggest 10-15 patches in each round.
^ permalink raw reply [flat|nested] 40+ messages in thread* Re: [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins
2022-11-30 10:27 ` [PATCH ulogd2 v2 v2 00/34] Refactor of the DB output plug-ins Pablo Neira Ayuso
@ 2022-11-30 16:03 ` Jeremy Sowden
0 siblings, 0 replies; 40+ messages in thread
From: Jeremy Sowden @ 2022-11-30 16:03 UTC (permalink / raw)
To: Pablo Neira Ayuso; +Cc: Netfilter Devel
[-- Attachment #1: Type: text/plain, Size: 1586 bytes --]
On 2022-11-30, at 11:27:50 +0100, Pablo Neira Ayuso wrote:
> On Tue, Nov 29, 2022 at 09:47:15PM +0000, Jeremy Sowden wrote:
> > In his feedback to my last series of clean-up patches at the
> > beginning of the year, Pablo suggested consolidating some parallel
> > implementations of the same functionality in the SQL output
> > plug-ins. I already had some patches in the works aimed at tidying
> > up the DB API. This patch-set is the result. In addition to the
> > suggested de-duping and other tidy-ups, I have added prep & exec
> > support in order to convert the sqlite3 plug-in to the DB API, and
> > updated the MySQL and PostgreSQL plug-ins to use it as well (DBI
> > doesn't do prep & exec).
> >
> > This patch-set is structured as follows.
> >
> > * Patches 1-4 are bug-fixes.
> > * Patches 5-13 are miscellaneous tidying.
> > * Patch 14 does the consolidation Pablo suggested.
> > * Patches 15-26 refactor and clean up the common DB API.
> > * Patches 27-28 add prep & exec support to the common DB API.
> > * Patch 29 converts the MySQL plug-in to use prep & exec.
> > * Patch 30-33 tidy up and convert the PostgreSQL plug-in to use
> > prep & exec.
> > * Patch 34 converts the SQLite plug-in to use the common DB API.
>
> It's great that ulogd2 is getting updates, thanks a lot.
>
> But would it be possible to start with a smaller batch? We review
> integrate and then you follow up with more updates.
No problem. I didn't expect the series to end up this big. :)
> I'll aim at being swift on it.
>
> I'd suggest 10-15 patches in each round.
J.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread