All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/12] multipath-tools: improve processing efficiency for addition and deletion of multipath devices
@ 2016-12-27  8:03 tang.junhui
  2016-12-27  8:03 ` [PATCH 01/12] libmultipath: add wwid for "struct uevent" to record wwid of uevent tang.junhui
                   ` (11 more replies)
  0 siblings, 12 replies; 34+ messages in thread
From: tang.junhui @ 2016-12-27  8:03 UTC (permalink / raw)
  To: christophe.varoqui, hare, bmarzins, mwilck, bart.vanassche
  Cc: zhang.kai16, dm-devel, tang.junhui, tang.wenjun3

From: tang.junhui <tang.junhui@zte.com.cn>

Hello Christophe, Ben, Hannes, Martin, Bart, and other guys:

The twelve patches in this series are used to improve processing
efficiency for addition and deletion of multipath devices.

These patches are tested pass by ZTE multipath automatic testing syetem.
The modification reduces the system consumption(such as CPU) and shortens
the processing time obviously in scene of massive multipath devices
addition or deletion.

The main processing flow of code is:
1)uevents accumulation in uevents burst scene:
  --[pacth]libmultipath: wait one seconds for more uevents in
    uevent_listen() in uevents burst situations
2)uevents filtering and merging:
  --[pacth]multipathd: merge uevents before proccessing
  --[pacth]libmultipath: filter uevents before proccessing
3)uevents proccessing:
  --[pacth]multipathd: add need_do_map to indicate whether need calling
    domap() in ev_add_path()
  --[pacth]multipathd: add need_do_map to indicate whether need calling
    domap() in ev_remove_path()
  --[pacth]multipathd: proccess merged uevents

There are some patches providing basic struct members or maros which
would used in above pathes:
  --[patch]libmultipath: add wwid for "struct uevent" to record wwid of
    uevent
  --[patch]libmultipath: add merge_node for "struct uevent" to record
    nodes of merged uevents
  --[patch]libmultipath: add two list iteration macros

There are also some independent optimization pathes for further
improvement:
  --[patch]multipathd: move uev_discard() to uevent.c and change its name
    to uevent_can_discard()
  --[patch]multipathd: move calling filter_devnode() from uev_trigger() to
    uevent_can_discard()
  --[patch]libmultipath: use existing wwid when wwid has already been
    existed in uevent

Any comment will be welcome, and it would be appreciated if these
patches would be considered for inclusion in the upstream
multipath-tools.

Thank you all,
Tang Junhui

^ permalink raw reply	[flat|nested] 34+ messages in thread
* [PATCH 08/12] libmultipath: wait one seconds for more uevents in uevent_listen() in uevents burst situations
@ 2016-12-29  1:54 tang.junhui
  0 siblings, 0 replies; 34+ messages in thread
From: tang.junhui @ 2016-12-29  1:54 UTC (permalink / raw)
  To: mwilck, christophe.varoqui, bmarzins, hare, bart.vanassche
  Cc: zhang.kai16, dm-devel, tang.junhui, tang.wenjun3

From: tang.junhui <tang.junhui@zte.com.cn>

The more uevents are merged, the higher efficiency program will performs.
So, do not process uevents after receiving immediately in uevents burst
situations, but continue wait one seconds for more uevents except that
too much uevents (2048) have already been received or too much time
eclipse (30 seconds).

Change-Id: I763d491540e8114a81d12d603281540a81502742
Signed-off-by: tang.junhui <tang.junhui@zte.com.cn>
---
 libmultipath/uevent.c | 42 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)

diff --git a/libmultipath/uevent.c b/libmultipath/uevent.c
index cc10d65..0345e6e 100644
--- a/libmultipath/uevent.c
+++ b/libmultipath/uevent.c
@@ -39,6 +39,7 @@
 #include <linux/netlink.h>
 #include <pthread.h>
 #include <sys/mman.h>
+#include <sys/time.h>
 #include <libudev.h>
 #include <errno.h>
 
@@ -51,6 +52,10 @@
 #include "config.h"
 #include "blacklist.h"
 
+#define MAX_ACCUMULATION_COUNT 2048
+#define MAX_ACCUMULATION_TIME 30*1000
+#define MIN_BURST_SPEED 10
+
 typedef int (uev_trigger)(struct uevent *, void * trigger_data);
 
 LIST_HEAD(uevq);
@@ -490,11 +495,43 @@ struct uevent *uevent_from_udev_device(struct udev_device *dev)
 	return uev;
 }
 
+bool uevent_burst(struct timeval *start_time, int events)
+{
+	struct timeval diff_time, end_time;
+	unsigned long speed;
+	unsigned long eclipse_ms;
+
+	if(events > MAX_ACCUMULATION_COUNT) {
+		condlog(2, "burst got %u uevents, too much uevents, stopped", events);
+		return false;
+	}
+
+	gettimeofday(&end_time, NULL);
+	timersub(&end_time, start_time, &diff_time);
+
+	eclipse_ms = diff_time.tv_sec * 1000 + diff_time.tv_usec / 1000;
+
+	if (eclipse_ms == 0)
+		return true;
+
+	if (eclipse_ms > MAX_ACCUMULATION_TIME) {
+		condlog(2, "burst continued %lu ms, too long time, stopped", eclipse_ms);
+		return false;
+	}
+
+	speed = (events * 1000) / eclipse_ms;
+	if (speed > MIN_BURST_SPEED)
+		return true;
+
+	return false;
+}
+
 int uevent_listen(struct udev *udev)
 {
 	int err = 2;
 	struct udev_monitor *monitor = NULL;
 	int fd, socket_flags, events;
+	struct timeval start_time;
 	int need_failback = 1;
 	int timeout = 30;
 	LIST_HEAD(uevlisten_tmp);
@@ -548,6 +585,7 @@ int uevent_listen(struct udev *udev)
 	}
 
 	events = 0;
+	gettimeofday(&start_time, NULL);
 	while (1) {
 		struct uevent *uev;
 		struct udev_device *dev;
@@ -562,7 +600,8 @@ int uevent_listen(struct udev *udev)
 		errno = 0;
 		fdcount = poll(&ev_poll, 1, poll_timeout);
 		if (fdcount && ev_poll.revents & POLLIN) {
-			timeout = 0;
+			timeout = uevent_burst(&start_time, events + 1) ? 1 : 0;
+
 			dev = udev_monitor_receive_device(monitor);
 			if (!dev) {
 				condlog(0, "failed getting udev device");
@@ -600,6 +639,7 @@ int uevent_listen(struct udev *udev)
 			pthread_mutex_unlock(uevq_lockp);
 			events = 0;
 		}
+		gettimeofday(&start_time, NULL);
 		timeout = 30;
 	}
 	need_failback = 0;
-- 
2.8.1.windows.1

^ permalink raw reply related	[flat|nested] 34+ messages in thread
end of thread, other threads:[~2017-01-09  7:22 UTC | newest]

Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-27  8:03 [PATCH 00/12] multipath-tools: improve processing efficiency for addition and deletion of multipath devices tang.junhui
2016-12-27  8:03 ` [PATCH 01/12] libmultipath: add wwid for "struct uevent" to record wwid of uevent tang.junhui
2017-01-03 22:02   ` Benjamin Marzinski
2017-01-04  6:56     ` tang.junhui
2017-01-04 18:14       ` Benjamin Marzinski
2017-01-04 20:33         ` Martin Wilck
2017-01-05  3:00           ` Benjamin Marzinski
2017-01-05  3:10         ` tang.junhui
2017-01-05 17:36           ` Benjamin Marzinski
2017-01-06  0:59             ` tang.junhui
2017-01-06 16:02               ` Benjamin Marzinski
2017-01-09  7:22                 ` tang.junhui
2016-12-27  8:03 ` [PATCH 02/12] libmultipath: add merge_node for "struct uevent" to record nodes of merged uevents tang.junhui
2016-12-27  8:03 ` [PATCH 03/12] libmultipath: add two list iteration macros tang.junhui
2016-12-27  8:03 ` [PATCH 04/12] multipathd: add need_do_map to indicate whether need calling domap() in ev_add_path() tang.junhui
2016-12-27  8:03 ` [PATCH 05/12] multipathd: add need_do_map to indicate whether need calling domap() in ev_remove_path() tang.junhui
2016-12-27  8:03 ` [PATCH 06/12] multipathd: move uev_discard() to uevent.c and change its name to uevent_can_discard() tang.junhui
2016-12-27  8:03 ` [PATCH 07/12] multipathd: move calling filter_devnode() from uev_trigger() " tang.junhui
2016-12-27  8:03 ` [PATCH 08/12] libmultipath: wait one seconds for more uevents in uevent_listen() in uevents burst situations tang.junhui
2016-12-28 20:25   ` Martin Wilck
2016-12-29  0:48     ` tang.junhui
2017-01-03 22:31   ` Benjamin Marzinski
2017-01-04  7:32     ` tang.junhui
2016-12-27  8:03 ` [PATCH 09/12] multipathd: merge uevents before proccessing tang.junhui
2017-01-04  0:30   ` Benjamin Marzinski
2017-01-04  3:29     ` tang.junhui
2016-12-27  8:03 ` [PATCH 10/12] libmultipath: filter " tang.junhui
2017-01-04  1:21   ` Benjamin Marzinski
2017-01-04  2:03     ` tang.junhui
2016-12-27  8:03 ` [PATCH 11/12] multipathd: proccess merged uevents tang.junhui
2017-01-04  1:03   ` Benjamin Marzinski
2017-01-04  1:54     ` tang.junhui
2016-12-27  8:03 ` [PATCH 12/12] libmultipath: use existing wwid when wwid has already been existed in uevent tang.junhui
2016-12-29  1:54 [PATCH 08/12] libmultipath: wait one seconds for more uevents in uevent_listen() in uevents burst situations tang.junhui

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.