From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34530) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bf1QO-00065O-1a for qemu-devel@nongnu.org; Wed, 31 Aug 2016 05:03:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bf1QH-0005Pw-JA for qemu-devel@nongnu.org; Wed, 31 Aug 2016 05:03:11 -0400 Received: from [59.151.112.132] (port=3897 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bf1QF-0005PR-TM for qemu-devel@nongnu.org; Wed, 31 Aug 2016 05:03:05 -0400 References: <1471421428-26379-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> <1471421428-26379-3-git-send-email-zhangchen.fnst@cn.fujitsu.com> From: Zhang Chen Message-ID: Date: Wed, 31 Aug 2016 17:03:06 +0800 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [PATCH V12 02/10] colo-compare: introduce colo compare initialization List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jason Wang , qemu devel Cc: Li Zhijian , Wen Congyang , zhanghailiang , "eddie . dong" , "Dr . David Alan Gilbert" On 08/31/2016 03:53 PM, Jason Wang wrote: > > > On 2016年08月17日 16:10, Zhang Chen wrote: >> This a COLO net ascii figure: >> >> Primary qemu Secondary qemu >> +--------------------------------------------------------------+ >> +----------------------------------------------------------------+ >> | +----------------------------------------------------------+ >> | | >> +-----------------------------------------------------------+ | >> | | | >> | | | >> | | >> | | guest | >> | | | guest | | >> | | | >> | | | >> | | >> | +-------^--------------------------+-----------------------+ >> | | >> +---------------------+--------+----------------------------+ | >> | | | | >> | ^ | | >> | | | | >> | | | | >> | | +------------------------------------------------------+ >> | | | | >> |netfilter| | | | | | >> netfilter | | | >> | +----------+ +----------------------------+ | | | >> +-----------------------------------------------------------+ | >> | | | | | | out | | | >> | | | filter excute order | | >> | | | | +-----------------------------+ | | | >> | | | +-------------------> | | >> | | | | | | | | | | | >> | | | TCP | | >> | | +-----+--+-+ +-----v----+ +-----v----+ |pri +----+----+sec| >> | | | +------------+ +---+----+---v+rewriter++ +------------+ | | >> | | | | | | | | |in | |in | >> | | | | | | | | | | | | >> | | | filter | | filter | | filter +------> colo <------+ >> +--------> filter +--> adjust | adjust +--> filter | | | >> | | | mirror | |redirector| |redirector| | | compare | | | >> | | | redirector | | ack | seq | | redirector | | | >> | | | | | | | | | | | | | >> | | | | | | | | | | | >> | | +----^-----+ +----+-----+ +----------+ | +---------+ | | >> | | +------------+ +--------+--------------+ +---+--------+ | | >> | | | tx | rx rx | | | | | >> tx all | rx | | >> | | | | | | | | >> +-----------------------------------------------------------+ | >> | | | +--------------+ | | | | | | >> | | | filter excute order | | | | | | | >> | | | +----------------> | | | >> +--------------------------------------------------------+ | >> | +-----------------------------------------+ | | | >> | | | | | | >> +--------------------------------------------------------------+ >> +----------------------------------------------------------------+ >> |guest receive | guest send >> | | >> +--------+----------------------------v------------------------+ >> | | NOTE: filter direction is rx/tx/all >> | tap | rx:receive >> packets sent to the netdev >> | | tx:receive packets sent by the netdev >> +--------------------------------------------------------------+ > > It's better to add a doc under docs to explain this configuration in > detail on top of this series. > As you say, Am I add /docs/colo-proxy.txt to explain it or add this in hailiang's COLO-FT.txt after merge? >> In COLO-compare, we do packet comparing job. >> Packets coming from the primary char indev will be sent to outdev. >> Packets coming from the secondary char dev will be dropped after >> comparing. >> colo-comapre need two input chardev and one output chardev: >> primary_in=chardev1-id (source: primary send packet) >> secondary_in=chardev2-id (source: secondary send packet) >> outdev=chardev3-id >> >> usage: >> >> primary: >> -netdev >> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >> -device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >> -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> -chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >> -chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >> -chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >> -chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >> -chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >> -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >> -object >> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >> -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >> -object >> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >> >> secondary: >> -netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >> script=/etc/qemu-ifdown >> -device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >> -chardev socket,id=red0,host=3.3.3.3,port=9003 >> -chardev socket,id=red1,host=3.3.3.3,port=9004 >> -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >> -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >> >> Signed-off-by: Zhang Chen >> Signed-off-by: Li Zhijian >> Signed-off-by: Wen Congyang >> --- >> net/Makefile.objs | 1 + >> net/colo-compare.c | 284 >> +++++++++++++++++++++++++++++++++++++++++++++++++++++ >> qemu-options.hx | 39 ++++++++ >> vl.c | 3 +- >> 4 files changed, 326 insertions(+), 1 deletion(-) >> create mode 100644 net/colo-compare.c >> >> diff --git a/net/Makefile.objs b/net/Makefile.objs >> index b7c22fd..ba92f73 100644 >> --- a/net/Makefile.objs >> +++ b/net/Makefile.objs >> @@ -16,3 +16,4 @@ common-obj-$(CONFIG_NETMAP) += netmap.o >> common-obj-y += filter.o >> common-obj-y += filter-buffer.o >> common-obj-y += filter-mirror.o >> +common-obj-y += colo-compare.o >> diff --git a/net/colo-compare.c b/net/colo-compare.c >> new file mode 100644 >> index 0000000..cdc3e0e >> --- /dev/null >> +++ b/net/colo-compare.c >> @@ -0,0 +1,284 @@ >> +/* >> + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service >> (COLO) >> + * (a.k.a. Fault Tolerance or Continuous Replication) >> + * >> + * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. >> + * Copyright (c) 2016 FUJITSU LIMITED >> + * Copyright (c) 2016 Intel Corporation >> + * >> + * Author: Zhang Chen >> + * >> + * This work is licensed under the terms of the GNU GPL, version 2 or >> + * later. See the COPYING file in the top-level directory. >> + */ >> + >> +#include "qemu/osdep.h" >> +#include "qemu/error-report.h" >> +#include "qemu-common.h" >> +#include "qapi/qmp/qerror.h" >> +#include "qapi/error.h" >> +#include "net/net.h" >> +#include "net/vhost_net.h" > > Looks unnecessary. I will remove it. > >> +#include "qom/object_interfaces.h" >> +#include "qemu/iov.h" >> +#include "qom/object.h" >> +#include "qemu/typedefs.h" >> +#include "net/queue.h" >> +#include "sysemu/char.h" >> +#include "qemu/sockets.h" >> +#include "qapi-visit.h" >> + >> +#define TYPE_COLO_COMPARE "colo-compare" >> +#define COLO_COMPARE(obj) \ >> + OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) >> + >> +#define COMPARE_READ_LEN_MAX NET_BUFSIZE >> + >> +typedef struct CompareState { >> + Object parent; >> + >> + char *pri_indev; >> + char *sec_indev; >> + char *outdev; >> + CharDriverState *chr_pri_in; >> + CharDriverState *chr_sec_in; >> + CharDriverState *chr_out; >> + QTAILQ_ENTRY(CompareState) next; > > This looks not used in this series but in commit "colo-compare and > filter-rewriter work with colo-frame". We'd better delay the > introducing to that patch. OK~ I got your point. > >> + SocketReadState pri_rs; >> + SocketReadState sec_rs; >> +} CompareState; >> + >> +typedef struct CompareClass { >> + ObjectClass parent_class; >> +} CompareClass; >> + >> +typedef struct CompareChardevProps { >> + bool is_socket; >> + bool is_unix; >> +} CompareChardevProps; >> + >> +static char *compare_get_pri_indev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->pri_indev); >> +} >> + >> +static void compare_set_pri_indev(Object *obj, const char *value, >> Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->pri_indev); >> + s->pri_indev = g_strdup(value); >> +} >> + >> +static char *compare_get_sec_indev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->sec_indev); >> +} >> + >> +static void compare_set_sec_indev(Object *obj, const char *value, >> Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->sec_indev); >> + s->sec_indev = g_strdup(value); >> +} >> + >> +static char *compare_get_outdev(Object *obj, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + return g_strdup(s->outdev); >> +} >> + >> +static void compare_set_outdev(Object *obj, const char *value, Error >> **errp) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + g_free(s->outdev); >> + s->outdev = g_strdup(value); >> +} >> + >> +static void compare_pri_rs_finalize(SocketReadState *pri_rs) >> +{ >> + /* if packet_enqueue pri pkt failed we will send unsupported >> packet */ >> +} >> + >> +static void compare_sec_rs_finalize(SocketReadState *sec_rs) >> +{ >> + /* if packet_enqueue sec pkt failed we will notify trace */ >> +} >> + >> +static int compare_chardev_opts(void *opaque, >> + const char *name, const char *value, >> + Error **errp) >> +{ >> + CompareChardevProps *props = opaque; >> + >> + if (strcmp(name, "backend") == 0 && strcmp(value, "socket") == 0) { >> + props->is_socket = true; >> + } else if (strcmp(name, "host") == 0) { > > Typo? net_vhost_chardev_opts() did: > > } else if (strcmp(name, "path") == 0) { > props->is_unix = true; > } > > No, In colo-compare we use chardev like this: -chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait If we only use "path" here will trigger a error. Should I add anthor "path" here? > >> + props->is_unix = true; >> + } else if (strcmp(name, "port") == 0) { >> + } else if (strcmp(name, "server") == 0) { >> + } else if (strcmp(name, "wait") == 0) { >> + } else { >> + error_setg(errp, >> + "COLO-compare does not support a chardev with >> option %s=%s", >> + name, value); >> + return -1; >> + } >> + return 0; >> +} >> + >> +/* >> + * called from the main thread on the primary >> + * to setup colo-compare. >> + */ >> +static void colo_compare_complete(UserCreatable *uc, Error **errp) >> +{ >> + CompareState *s = COLO_COMPARE(uc); >> + CompareChardevProps props; >> + >> + if (!s->pri_indev || !s->sec_indev || !s->outdev) { >> + error_setg(errp, "colo compare needs 'primary_in' ," >> + "'secondary_in','outdev' property set"); >> + return; >> + } else if (!strcmp(s->pri_indev, s->outdev) || >> + !strcmp(s->sec_indev, s->outdev) || >> + !strcmp(s->pri_indev, s->sec_indev)) { >> + error_setg(errp, "'indev' and 'outdev' could not be same " >> + "for compare module"); >> + return; >> + } >> + >> + s->chr_pri_in = qemu_chr_find(s->pri_indev); >> + if (s->chr_pri_in == NULL) { >> + error_setg(errp, "Primary IN Device '%s' not found", >> + s->pri_indev); >> + return; >> + } >> + >> + /* inspect chardev opts */ >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_pri_in->opts, compare_chardev_opts, >> &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->pri_indev); >> + return; >> + } >> + >> + s->chr_sec_in = qemu_chr_find(s->sec_indev); >> + if (s->chr_sec_in == NULL) { >> + error_setg(errp, "Secondary IN Device '%s' not found", >> + s->sec_indev); >> + return; >> + } >> + >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_sec_in->opts, compare_chardev_opts, >> &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->sec_indev); > > I believe tcp socket is also supported? If I understand correctly, "tcp socket" in here is the "-chardev socket". I will rename "unix socket" to "tcp socket". > >> + return; >> + } >> + >> + s->chr_out = qemu_chr_find(s->outdev); >> + if (s->chr_out == NULL) { >> + error_setg(errp, "OUT Device '%s' not found", s->outdev); >> + return; >> + } >> + >> + memset(&props, 0, sizeof(props)); >> + if (qemu_opt_foreach(s->chr_out->opts, compare_chardev_opts, >> &props, errp)) { >> + return; >> + } >> + >> + if (!props.is_socket || !props.is_unix) { >> + error_setg(errp, "chardev \"%s\" is not a unix socket", >> + s->outdev); > > Ditto, and there's code duplication, please introduce a helper to do > above. I don't understand what the "helper"? In here we check each chardev, will I change to "goto error;" ? > >> + return; >> + } >> + >> + qemu_chr_fe_claim_no_fail(s->chr_pri_in); >> + >> + qemu_chr_fe_claim_no_fail(s->chr_sec_in); >> + >> + qemu_chr_fe_claim_no_fail(s->chr_out); >> + >> + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize); >> + net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize); >> + >> + return; >> +} >> + >> +static void colo_compare_class_init(ObjectClass *oc, void *data) >> +{ >> + UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); >> + >> + ucc->complete = colo_compare_complete; >> +} >> + >> +static void colo_compare_init(Object *obj) >> +{ >> + object_property_add_str(obj, "primary_in", >> + compare_get_pri_indev, >> compare_set_pri_indev, >> + NULL); >> + object_property_add_str(obj, "secondary_in", >> + compare_get_sec_indev, >> compare_set_sec_indev, >> + NULL); >> + object_property_add_str(obj, "outdev", >> + compare_get_outdev, compare_set_outdev, >> + NULL); >> +} >> + >> +static void colo_compare_finalize(Object *obj) >> +{ >> + CompareState *s = COLO_COMPARE(obj); >> + >> + if (s->chr_pri_in) { >> + qemu_chr_add_handlers(s->chr_pri_in, NULL, NULL, NULL, NULL); >> + qemu_chr_fe_release(s->chr_pri_in); >> + } >> + if (s->chr_sec_in) { >> + qemu_chr_add_handlers(s->chr_sec_in, NULL, NULL, NULL, NULL); >> + qemu_chr_fe_release(s->chr_sec_in); >> + } >> + if (s->chr_out) { >> + qemu_chr_fe_release(s->chr_out); >> + } >> + >> + g_free(s->pri_indev); >> + g_free(s->sec_indev); >> + g_free(s->outdev); >> +} >> + >> +static const TypeInfo colo_compare_info = { >> + .name = TYPE_COLO_COMPARE, >> + .parent = TYPE_OBJECT, >> + .instance_size = sizeof(CompareState), >> + .instance_init = colo_compare_init, >> + .instance_finalize = colo_compare_finalize, >> + .class_size = sizeof(CompareClass), >> + .class_init = colo_compare_class_init, >> + .interfaces = (InterfaceInfo[]) { >> + { TYPE_USER_CREATABLE }, >> + { } >> + } >> +}; >> + >> +static void register_types(void) >> +{ >> + type_register_static(&colo_compare_info); >> +} >> + >> +type_init(register_types); >> diff --git a/qemu-options.hx b/qemu-options.hx >> index 587de8f..33d5d0b 100644 >> --- a/qemu-options.hx >> +++ b/qemu-options.hx >> @@ -3866,6 +3866,45 @@ Dump the network traffic on netdev @var{dev} >> to the file specified by >> The file format is libpcap, so it can be analyzed with tools such >> as tcpdump >> or Wireshark. >> +@item -object >> colo-compare,id=@var{id},primary_in=@var{chardevid},secondary_in=@var{chardevid}, >> +outdev=@var{chardevid} >> + >> +Colo-compare gets packet from primary_in@var{chardevid} and >> secondary_in@var{chardevid}, than compare primary packet with >> +secondary packet. If the packet same, we will output primary > > s/If the packet same/If the packets are same/. OK. > >> +packet to outdev@var{chardevid}, else we will notify colo-frame >> +do checkpoint and send primary packet to outdev@var{chardevid}. >> + >> +we can use it with the help of filter-mirror and filter-redirector. > > s/we/We/ and looks like colo compare must be used with the help of > mirror and redirector? Currently yes. > >> + >> +@example >> + >> +primary: >> +-netdev >> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown >> +-device e1000,id=e0,netdev=hn0,mac=52:a4:00:12:78:66 >> +-chardev socket,id=mirror0,host=3.3.3.3,port=9003,server,nowait >> +-chardev socket,id=compare1,host=3.3.3.3,port=9004,server,nowait >> +-chardev socket,id=compare0,host=3.3.3.3,port=9001,server,nowait >> +-chardev socket,id=compare0-0,host=3.3.3.3,port=9001 >> +-chardev socket,id=compare_out,host=3.3.3.3,port=9005,server,nowait >> +-chardev socket,id=compare_out0,host=3.3.3.3,port=9005 >> +-object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 >> +-object >> filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out >> +-object >> filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0 >> +-object >> colo-compare,id=comp0,primary_in=compare0-0,secondary_in=compare1,outdev=compare_out0 >> + >> +secondary: >> +-netdev tap,id=hn0,vhost=off,script=/etc/qemu-ifup,down >> script=/etc/qemu-ifdown >> +-device e1000,netdev=hn0,mac=52:a4:00:12:78:66 >> +-chardev socket,id=red0,host=3.3.3.3,port=9003 >> +-chardev socket,id=red1,host=3.3.3.3,port=9004 >> +-object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 >> +-object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 >> + >> +@end example >> + >> +If you want to know the detail of above command line, you can read >> +the colo-compare git log. >> + >> @item -object >> secret,id=@var{id},data=@var{string},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >> @item -object >> secret,id=@var{id},file=@var{filename},format=@var{raw|base64}[,keyid=@var{secretid},iv=@var{string}] >> diff --git a/vl.c b/vl.c >> index cbe51ac..c6b9a6f 100644 >> --- a/vl.c >> +++ b/vl.c >> @@ -2865,7 +2865,8 @@ static bool object_create_initial(const char >> *type) >> if (g_str_equal(type, "filter-buffer") || >> g_str_equal(type, "filter-dump") || >> g_str_equal(type, "filter-mirror") || >> - g_str_equal(type, "filter-redirector")) { >> + g_str_equal(type, "filter-redirector") || >> + g_str_equal(type, "colo-compare")) { >> return false; >> } > > > > . > -- Thanks zhangchen