* [RFC PATCH 0/5] LWIP stack integration
@ 2023-05-05 10:25 Maxim Uvarov
2023-05-05 10:25 ` [RFC PATCH 1/5] add lwip-external submodule Maxim Uvarov
` (7 more replies)
0 siblings, 8 replies; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-05 10:25 UTC (permalink / raw)
To: u-boot
Cc: pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev, Maxim Uvarov
Greetings,
This RFC patchset is an attempt to try to use an already existing IP network stack inside U-boot.
U-Boot recently got basic TCP/IP support, implementing wget, but in order to get a full IP stack
with new features (e.g ipv6), it would be preferable to use an established embedded ip library,
instead of rewriting the code from scratch.
For this experiment LWIP network stack was selected:
https://savannah.nongnu.org/git/?group=lwip
LWIP main features include:
- Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
- DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf),
SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
- APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
- Extended features: IP forwarding over multiple network interfaces, TCP congestion control,
RTT estimation and fast recovery/fast retransmit
- Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver,
mDNS responder, MQTT client, TFTP server.
This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
TCP/IP stack designed for embedded systems for U-boot. That will allow using already
written network applications for microcontrollers.
lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
Which should be compatible with u-boot.
In the current RFC I tried to use minimal changes to better see how LWIP code can be embedded into
U-boot. Patches implement ping and wget commands work. Both commands are currently copy pasting and
reusing lwIP examples. Whether we want to add the final application in U-Boot or lwIP is up to
discussion, but the current approach was the easiest one for an RFC.
Looking for your comments,
Best regards,
Maxim.
Maxim Uvarov (5):
add lwip-external submodule
lib/lwip: compile-in core files
add doc/README.lwip
add doc/README.lwip.size
lwip: implement wget command from http_client.c example
.gitignore | 5 +
.gitmodules | 3 +
doc/README.lwip | 90 +++++
doc/README.lwip.size | 291 +++++++++++++++
include/net.h | 2 +-
lib/Kconfig | 2 +
lib/Makefile | 2 +
lib/lwip/Kconfig | 12 +
lib/lwip/Makefile | 86 +++++
lib/lwip/apps/http/lwip-wget.c | 67 ++++
lib/lwip/apps/http/rmstatic.patch | 47 +++
lib/lwip/apps/ping/lwip_ping.c | 33 ++
lib/lwip/apps/ping/lwip_ping.h | 19 +
lib/lwip/apps/ping/ping.h | 0
lib/lwip/apps/ping/rmstatic.patch | 32 ++
lib/lwip/cmd-lwip.c | 129 +++++++
lib/lwip/lwip-external | 1 +
lib/lwip/lwipopts.h | 484 +++++++++++++++++++++++++
lib/lwip/port/if.c | 256 +++++++++++++
lib/lwip/port/include/arch/cc.h | 41 +++
lib/lwip/port/include/arch/sys_arch.h | 78 ++++
lib/lwip/port/include/arch/u-sockets.h | 26 ++
lib/lwip/port/include/limits.h | 0
lib/lwip/port/sys-arch.c | 7 +
net/eth-uclass.c | 4 +-
net/net.c | 14 +
26 files changed, 1729 insertions(+), 2 deletions(-)
create mode 100644 .gitmodules
create mode 100644 doc/README.lwip
create mode 100644 doc/README.lwip.size
create mode 100644 lib/lwip/Kconfig
create mode 100644 lib/lwip/Makefile
create mode 100644 lib/lwip/apps/http/lwip-wget.c
create mode 100644 lib/lwip/apps/http/rmstatic.patch
create mode 100644 lib/lwip/apps/ping/lwip_ping.c
create mode 100644 lib/lwip/apps/ping/lwip_ping.h
create mode 100644 lib/lwip/apps/ping/ping.h
create mode 100644 lib/lwip/apps/ping/rmstatic.patch
create mode 100644 lib/lwip/cmd-lwip.c
create mode 160000 lib/lwip/lwip-external
create mode 100644 lib/lwip/lwipopts.h
create mode 100644 lib/lwip/port/if.c
create mode 100644 lib/lwip/port/include/arch/cc.h
create mode 100644 lib/lwip/port/include/arch/sys_arch.h
create mode 100644 lib/lwip/port/include/arch/u-sockets.h
create mode 100644 lib/lwip/port/include/limits.h
create mode 100644 lib/lwip/port/sys-arch.c
--
2.30.2
^ permalink raw reply [flat|nested] 37+ messages in thread
* [RFC PATCH 1/5] add lwip-external submodule
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
@ 2023-05-05 10:25 ` Maxim Uvarov
2023-05-08 14:43 ` Simon Glass
2023-05-11 13:51 ` Tom Rini
2023-05-05 10:25 ` [RFC PATCH 2/5] lib/lwip: compile-in core files Maxim Uvarov
` (6 subsequent siblings)
7 siblings, 2 replies; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-05 10:25 UTC (permalink / raw)
To: u-boot
Cc: pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev, Maxim Uvarov
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
---
.gitmodules | 3 +++
lib/lwip/lwip-external | 1 +
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 lib/lwip/lwip-external
diff --git a/.gitmodules b/.gitmodules
new file mode 100644
index 0000000000..afc709af10
--- /dev/null
+++ b/.gitmodules
@@ -0,0 +1,3 @@
+[submodule "lib/lwip/lwip-external"]
+ path = lib/lwip/lwip-external
+ url = https://git.savannah.nongnu.org/git/lwip.git
diff --git a/lib/lwip/lwip-external b/lib/lwip/lwip-external
new file mode 160000
index 0000000000..3fe8d2fc43
--- /dev/null
+++ b/lib/lwip/lwip-external
@@ -0,0 +1 @@
+Subproject commit 3fe8d2fc43a9b69f7ed28c63d44a7744f9c0def9
--
2.30.2
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 2/5] lib/lwip: compile-in core files
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
2023-05-05 10:25 ` [RFC PATCH 1/5] add lwip-external submodule Maxim Uvarov
@ 2023-05-05 10:25 ` Maxim Uvarov
2023-05-05 10:25 ` [RFC PATCH 3/5] add doc/README.lwip Maxim Uvarov
` (5 subsequent siblings)
7 siblings, 0 replies; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-05 10:25 UTC (permalink / raw)
To: u-boot
Cc: pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev, Maxim Uvarov
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
---
.gitignore | 3 +
include/net.h | 2 +-
lib/Kconfig | 2 +
lib/Makefile | 2 +
lib/lwip/Kconfig | 12 +
lib/lwip/Makefile | 78 ++++
lib/lwip/apps/ping/lwip_ping.c | 33 ++
lib/lwip/apps/ping/lwip_ping.h | 19 +
lib/lwip/apps/ping/ping.h | 0
lib/lwip/apps/ping/rmstatic.patch | 32 ++
lib/lwip/cmd-lwip.c | 98 +++++
lib/lwip/lwipopts.h | 484 +++++++++++++++++++++++++
lib/lwip/port/if.c | 256 +++++++++++++
lib/lwip/port/include/arch/cc.h | 41 +++
lib/lwip/port/include/arch/sys_arch.h | 78 ++++
lib/lwip/port/include/arch/u-sockets.h | 26 ++
lib/lwip/port/include/limits.h | 0
lib/lwip/port/sys-arch.c | 7 +
net/eth-uclass.c | 4 +-
net/net.c | 14 +
20 files changed, 1189 insertions(+), 2 deletions(-)
create mode 100644 lib/lwip/Kconfig
create mode 100644 lib/lwip/Makefile
create mode 100644 lib/lwip/apps/ping/lwip_ping.c
create mode 100644 lib/lwip/apps/ping/lwip_ping.h
create mode 100644 lib/lwip/apps/ping/ping.h
create mode 100644 lib/lwip/apps/ping/rmstatic.patch
create mode 100644 lib/lwip/cmd-lwip.c
create mode 100644 lib/lwip/lwipopts.h
create mode 100644 lib/lwip/port/if.c
create mode 100644 lib/lwip/port/include/arch/cc.h
create mode 100644 lib/lwip/port/include/arch/sys_arch.h
create mode 100644 lib/lwip/port/include/arch/u-sockets.h
create mode 100644 lib/lwip/port/include/limits.h
create mode 100644 lib/lwip/port/sys-arch.c
diff --git a/.gitignore b/.gitignore
index eb769f144c..aeaf847543 100644
--- a/.gitignore
+++ b/.gitignore
@@ -104,3 +104,6 @@ __pycache__
# pylint files
/pylint.cur
/pylint.out/
+
+/lib/lwip/lwip-external
+lib/lwip/apps/ping/ping.c
diff --git a/include/net.h b/include/net.h
index 1a99009959..8622983597 100644
--- a/include/net.h
+++ b/include/net.h
@@ -561,7 +561,7 @@ extern int net_restart_wrap; /* Tried all network devices */
enum proto_t {
BOOTP, RARP, ARP, TFTPGET, DHCP, PING, PING6, DNS, NFS, CDP, NETCONS,
- SNTP, TFTPSRV, TFTPPUT, LINKLOCAL, FASTBOOT, WOL, UDP, NCSI, WGET
+ SNTP, TFTPSRV, TFTPPUT, LINKLOCAL, FASTBOOT, WOL, UDP, NCSI, WGET, LWIP
};
extern char net_boot_file_name[1024];/* Boot File name */
diff --git a/lib/Kconfig b/lib/Kconfig
index 3c5a4ab386..7485a1f3bf 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -1031,3 +1031,5 @@ menu "FWU Multi Bank Updates"
source lib/fwu_updates/Kconfig
endmenu
+
+source lib/lwip/Kconfig
diff --git a/lib/Makefile b/lib/Makefile
index d77b33e7f4..3b80a41187 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -91,6 +91,8 @@ obj-$(CONFIG_LIBAVB) += libavb/
obj-$(CONFIG_$(SPL_TPL_)OF_LIBFDT) += libfdt/
obj-$(CONFIG_$(SPL_TPL_)OF_REAL) += fdtdec_common.o fdtdec.o
+obj-y += lwip/
+
ifdef CONFIG_SPL_BUILD
obj-$(CONFIG_SPL_YMODEM_SUPPORT) += crc16-ccitt.o
obj-$(CONFIG_$(SPL_TPL_)HASH) += crc16-ccitt.o
diff --git a/lib/lwip/Kconfig b/lib/lwip/Kconfig
new file mode 100644
index 0000000000..dd0b3d4671
--- /dev/null
+++ b/lib/lwip/Kconfig
@@ -0,0 +1,12 @@
+config LWIP_LIB
+ bool "Support LWIP library"
+ help
+ Selecting this option will enable the shared LWIP library code.
+
+config CMD_LWIP
+ bool "lwip"
+ default y
+ depends on LWIP_LIB
+ help
+ lwip networking command.
+
diff --git a/lib/lwip/Makefile b/lib/lwip/Makefile
new file mode 100644
index 0000000000..2c665dcb88
--- /dev/null
+++ b/lib/lwip/Makefile
@@ -0,0 +1,78 @@
+# SPDX-License-Identifier: GPL-2.0+
+#
+# (C) Copyright 2023 Linaro
+
+LWIPDIR=lwip-external/src
+
+ccflags-y += -I$(CURDIR)/lib/lwip/port/include
+ccflags-y += -I$(CURDIR)/lib/lwip/lwip-external/src/include -I$(CURDIR)/lib/lwip
+
+obj-y += $(LWIPDIR)/core/init.o \
+ $(LWIPDIR)/core/def.o \
+ $(LWIPDIR)/core/dns.o \
+ $(LWIPDIR)/core/inet_chksum.o \
+ $(LWIPDIR)/core/ip.o \
+ $(LWIPDIR)/core/mem.o \
+ $(LWIPDIR)/core/memp.o \
+ $(LWIPDIR)/core/netif.o \
+ $(LWIPDIR)/core/pbuf.o \
+ $(LWIPDIR)/core/raw.o \
+ $(LWIPDIR)/core/stats.o \
+ $(LWIPDIR)/core/sys.o \
+ $(LWIPDIR)/core/altcp.o \
+ $(LWIPDIR)/core/altcp_alloc.o \
+ $(LWIPDIR)/core/altcp_tcp.o \
+ $(LWIPDIR)/core/tcp.o \
+ $(LWIPDIR)/core/tcp_in.o \
+ $(LWIPDIR)/core/tcp_out.o \
+ $(LWIPDIR)/core/timeouts.o \
+ $(LWIPDIR)/core/udp.o
+
+# IPv4
+obj-y += $(LWIPDIR)/core/ipv4/acd.o \
+ $(LWIPDIR)/core/ipv4/autoip.o \
+ $(LWIPDIR)/core/ipv4/dhcp.o \
+ $(LWIPDIR)/core/ipv4/etharp.o \
+ $(LWIPDIR)/core/ipv4/icmp.o \
+ $(LWIPDIR)/core/ipv4/igmp.o \
+ $(LWIPDIR)/core/ipv4/ip4_frag.o \
+ $(LWIPDIR)/core/ipv4/ip4.o \
+ $(LWIPDIR)/core/ipv4/ip4_addr.o
+# IPv6
+obj-y += $(LWIPDIR)/core/ipv6/dhcp6.o \
+ $(LWIPDIR)/core/ipv6/ethip6.o \
+ $(LWIPDIR)/core/ipv6/icmp6.o \
+ $(LWIPDIR)/core/ipv6/inet6.o \
+ $(LWIPDIR)/core/ipv6/ip6.o \
+ $(LWIPDIR)/core/ipv6/ip6_addr.o \
+ $(LWIPDIR)/core/ipv6/ip6_frag.o \
+ $(LWIPDIR)/core/ipv6/mld6.o \
+ $(LWIPDIR)/core/ipv6/nd6.o
+# API
+obj-y += $(LWIPDIR)/api/api_lib.o \
+ $(LWIPDIR)/api/api_msg.o \
+ $(LWIPDIR)/api/err.o \
+ $(LWIPDIR)/api/if_api.o \
+ $(LWIPDIR)/api/netbuf.o \
+ $(LWIPDIR)/api/netdb.o \
+ $(LWIPDIR)/api/netifapi.o \
+ $(LWIPDIR)/api/sockets.o \
+ $(LWIPDIR)/api/tcpip.o
+
+# Netdevs
+obj-y += $(LWIPDIR)/netif/ethernet.o
+
+obj-y += port/if.o
+obj-y += port/sys-arch.o
+
+obj-$(CONFIG_CMD_LWIP) += cmd-lwip.o
+
+$(obj)/apps/ping/ping.o: $(obj)/apps/ping/ping.c
+$(obj)/apps/ping/ping.c:
+ cp ./lib/lwip/lwip-external/contrib/apps/ping/ping.c $(obj)/apps/ping/ping.c
+ patch -p1 < $(obj)/apps/ping/rmstatic.patch
+
+obj-$(CONFIG_CMD_LWIP) += apps/ping/ping.o
+obj-$(CONFIG_CMD_LWIP) += apps/ping/lwip_ping.o
+
+
diff --git a/lib/lwip/apps/ping/lwip_ping.c b/lib/lwip/apps/ping/lwip_ping.c
new file mode 100644
index 0000000000..91ce63063a
--- /dev/null
+++ b/lib/lwip/apps/ping/lwip_ping.c
@@ -0,0 +1,33 @@
+#include "lwip/opt.h"
+
+#include "lwip_ping.h"
+
+#include "lwip/mem.h"
+#include "lwip/raw.h"
+#include "lwip/icmp.h"
+#include "lwip/netif.h"
+#include "lwip/sys.h"
+#include "lwip/timeouts.h"
+#include "lwip/inet_chksum.h"
+#include "lwip/prot/ip4.h"
+
+#include "lwip/ip_addr.h"
+
+extern const ip_addr_t* ping_target;
+
+static ip_addr_t ip_target;
+
+int lwip_ping_init(char *ping_addr)
+{
+ //ipaddr_aton(ping_addr, ping_target);
+ IP4_ADDR(&ip_target, 192,168,1,2);
+ ping_target = &ip_target;
+ if (ping_target == 0) {
+ printf("%s() wrong ping addr\n", __func__);
+ return -1;
+ }
+
+ ping_raw_init();
+
+ return 0;
+}
diff --git a/lib/lwip/apps/ping/lwip_ping.h b/lib/lwip/apps/ping/lwip_ping.h
new file mode 100644
index 0000000000..047dfc2111
--- /dev/null
+++ b/lib/lwip/apps/ping/lwip_ping.h
@@ -0,0 +1,19 @@
+#ifndef LWIP_PING_H
+#define LWIP_PING_H
+
+#include <lwip/ip_addr.h>
+
+/**
+ * PING_USE_SOCKETS: Set to 1 to use sockets, otherwise the raw api is used
+ */
+#ifndef PING_USE_SOCKETS
+#define PING_USE_SOCKETS 0
+#endif
+
+//void lwip_ping_init(const ip_addr_t* ping_addr);
+int lwip_ping_init(char *ping_addr);
+
+void ping_raw_init(void);
+void lwip_ping_send_now(void);
+
+#endif /* LWIP_PING_H */
diff --git a/lib/lwip/apps/ping/ping.h b/lib/lwip/apps/ping/ping.h
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/lib/lwip/apps/ping/rmstatic.patch b/lib/lwip/apps/ping/rmstatic.patch
new file mode 100644
index 0000000000..758ce729fa
--- /dev/null
+++ b/lib/lwip/apps/ping/rmstatic.patch
@@ -0,0 +1,32 @@
+--- ./lib/lwip/lwip-external/contrib/apps/ping/ping.c 2023-05-03 13:20:14.510795833 +0000
++++ ./lib/lwip/apps/ping/ping.c 2023-05-03 14:37:05.497873883 +0000
+@@ -93,7 +93,7 @@
+ #endif
+
+ /* ping variables */
+-static const ip_addr_t* ping_target;
++const ip_addr_t* ping_target;
+ static u16_t ping_seq_num;
+ #ifdef LWIP_DEBUG
+ static u32_t ping_time;
+@@ -304,9 +304,9 @@ ping_recv(void *arg, struct raw_pcb *pcb
+ iecho = (struct icmp_echo_hdr *)p->payload;
+
+ if ((iecho->id == PING_ID) && (iecho->seqno == lwip_htons(ping_seq_num))) {
+- LWIP_DEBUGF( PING_DEBUG, ("ping: recv "));
++ printf("ping: recv ");
+ ip_addr_debug_print(PING_DEBUG, addr);
+- LWIP_DEBUGF( PING_DEBUG, (" %"U32_F" ms\n", (sys_now()-ping_time)));
++ printf(" %"U32_F" ms\n", (sys_now()-ping_time));
+
+ /* do some ping result processing */
+ PING_RESULT(1);
+@@ -361,7 +361,7 @@ ping_timeout(void *arg)
+ sys_timeout(PING_DELAY, ping_timeout, pcb);
+ }
+
+-static void
++void
+ ping_raw_init(void)
+ {
+ ping_pcb = raw_new(IP_PROTO_ICMP);
diff --git a/lib/lwip/cmd-lwip.c b/lib/lwip/cmd-lwip.c
new file mode 100644
index 0000000000..f2e25a8c29
--- /dev/null
+++ b/lib/lwip/cmd-lwip.c
@@ -0,0 +1,98 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2023
+ * Maxim Uvarov, maxim.uvarov@linaro.org
+ */
+
+#include <common.h>
+#include <command.h>
+#include <console.h>
+#include <display_options.h>
+#include <memalign.h>
+#include <net.h>
+
+#include "apps/ping/lwip_ping.h"
+
+extern int uboot_lwip_init(void);
+extern int uboot_lwip_loop_is_done(void);
+
+static int do_lwip_info(struct cmd_tbl *cmdtp, int flag, int argc,
+ char *const argv[])
+{
+ printf("TBD: %s\n", __func__);
+ return CMD_RET_SUCCESS;
+}
+
+static int do_lwip_init(struct cmd_tbl *cmdtp, int flag, int argc,
+ char *const argv[])
+{
+ if (!uboot_lwip_init())
+ return CMD_RET_SUCCESS;
+ return CMD_RET_FAILURE;
+}
+
+static int do_lwip_ping(struct cmd_tbl *cmdtp, int flag, int argc,
+ char *const argv[])
+{
+ if (argc < 2) {
+ printf("argc = %d, error\n", argc);
+ return CMD_RET_USAGE;
+ }
+
+ eth_init(); /* activate u-boot eth dev */
+
+ printf("pinging addr: %s\n", argv[1]);
+ if (lwip_ping_init(argv[2])) {
+ printf("ping init fail\n");
+ return CMD_RET_FAILURE;
+ }
+
+ ping_send_now();
+
+ lwip_loop_set();
+ if (net_loop(LWIP) < 0) {
+ printf("ping failed; host %s is not alive\n", argv[1]);
+ return CMD_RET_FAILURE;
+ }
+ return CMD_RET_SUCCESS;
+}
+
+static struct cmd_tbl cmds[] = {
+ U_BOOT_CMD_MKENT(info, 1, 0, do_lwip_info, "", ""),
+ U_BOOT_CMD_MKENT(init, 1, 0, do_lwip_init, "", ""),
+ U_BOOT_CMD_MKENT(ping, 2, 0, do_lwip_ping, "", ""),
+};
+
+static int do_ops(struct cmd_tbl *cmdtp, int flag, int argc,
+ char *const argv[])
+{
+ struct cmd_tbl *cp;
+
+ cp = find_cmd_tbl(argv[1], cmds, ARRAY_SIZE(cmds));
+
+ /* Drop the mmc command */
+ argc--;
+ argv++;
+
+ if (cp == NULL || argc > cp->maxargs)
+ return CMD_RET_USAGE;
+ if (flag == CMD_FLAG_REPEAT && !cmd_is_repeatable(cp))
+ return CMD_RET_SUCCESS;
+
+ return cp->cmd(cmdtp, flag, argc, argv);
+}
+
+U_BOOT_CMD(
+ lwip, 4, 1, do_ops,
+ "LWIP sub system",
+ "info - display info\n"
+ "init - init LWIP\n"
+ "ping addr - ping addr\n"
+ );
+
+/* Old command kept for compatibility. Same as 'mmc info' */
+U_BOOT_CMD(
+ lwipinfo, 1, 0, do_lwip_info,
+ "display LWIP info",
+ "- display LWIP stack info"
+);
diff --git a/lib/lwip/lwipopts.h b/lib/lwip/lwipopts.h
new file mode 100644
index 0000000000..581d05420a
--- /dev/null
+++ b/lib/lwip/lwipopts.h
@@ -0,0 +1,484 @@
+/**
+ * @file
+ *
+ * lwIP Options Configuration
+ */
+
+/*
+ * Copyright (c) 2001-2004 Swedish Institute of Computer Science.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without modification,
+ * are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright notice,
+ * this list of conditions and the following disclaimer in the documentation
+ * and/or other materials provided with the distribution.
+ * 3. The name of the author may not be used to endorse or promote products
+ * derived from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED
+ * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
+ * SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT
+ * OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
+ * OF SUCH DAMAGE.
+ *
+ * This file is part of the lwIP TCP/IP stack.
+ *
+ * Author: Adam Dunkels <adam@sics.se>
+ *
+ */
+#ifndef LWIP_LWIPOPTS_H
+#define LWIP_LWIPOPTS_H
+
+
+
+/*
+ * Include user defined options first. Anything not defined in these files
+ * will be set to standard values. Override anything you don't like!
+ */
+#include "lwipopts.h"
+
+#define LWIP_DEBUG 1
+#define LWIP_DBG_MIN_LEVEL LWIP_DBG_LEVEL_ALL
+#define LWIP_DBG_TYPES_ON LWIP_DBG_OFF
+#define ETHARP_DEBUG LWIP_DBG_OFF
+#define NETIF_DEBUG LWIP_DBG_OFF
+#define PBUF_DEBUG LWIP_DBG_OFF
+#define API_LIB_DEBUG LWIP_DBG_OFF
+#define API_MSG_DEBUG LWIP_DBG_OFF
+#define SOCKETS_DEBUG LWIP_DBG_OFF
+#define ICMP_DEBUG LWIP_DBG_OFF
+#define IGMP_DEBUG LWIP_DBG_OFF
+#define INET_DEBUG LWIP_DBG_OFF
+#define IP_DEBUG LWIP_DBG_OFF
+#define IP_REASS_DEBUG LWIP_DBG_OFF
+#define RAW_DEBUG LWIP_DBG_OFF
+#define MEM_DEBUG LWIP_DBG_OFF
+#define MEMP_DEBUG LWIP_DBG_OFF
+#define SYS_DEBUG LWIP_DBG_OFF
+#define TIMERS_DEBUG LWIP_DBG_OFF
+#define TCP_DEBUG LWIP_DBG_OFF
+#define TCP_INPUT_DEBUG LWIP_DBG_OFF
+#define TCP_FR_DEBUG LWIP_DBG_OFF
+#define TCP_RTO_DEBUG LWIP_DBG_OFF
+#define TCP_CWND_DEBUG LWIP_DBG_OFF
+#define TCP_WND_DEBUG LWIP_DBG_OFF
+#define TCP_OUTPUT_DEBUG LWIP_DBG_OFF
+#define TCP_RST_DEBUG LWIP_DBG_OFF
+#define TCP_QLEN_DEBUG LWIP_DBG_OFF
+#define UDP_DEBUG LWIP_DBG_OFF
+#define TCPIP_DEBUG LWIP_DBG_OFF
+#define SLIP_DEBUG LWIP_DBG_OFF
+#define DHCP_DEBUG LWIP_DBG_OFF
+#define AUTOIP_DEBUG LWIP_DBG_OFF
+#define DNS_DEBUG LWIP_DBG_OFF
+#define IP6_DEBUG LWIP_DBG_OFF
+#define DHCP6_DEBUG LWIP_DBG_OFF
+#define LWIP_TESTMODE 0
+#include "lwip/debug.h"
+
+/*
+ -----------------------------------------------
+ ---------- Platform specific locking ----------
+ -----------------------------------------------
+*/
+
+/**
+ * SYS_LIGHTWEIGHT_PROT==1: if you want inter-task protection for certain
+ * critical regions during buffer allocation, deallocation and memory
+ * allocation and deallocation.
+ */
+#define SYS_LIGHTWEIGHT_PROT 0
+
+/**
+ * NO_SYS==1: Provides VERY minimal functionality. Otherwise,
+ * use lwIP facilities.
+ */
+#define NO_SYS 0
+
+/*
+ ------------------------------------
+ ---------- Memory options ----------
+ ------------------------------------
+*/
+
+/**
+ * MEM_ALIGNMENT: should be set to the alignment of the CPU
+ * 4 byte alignment -> #define MEM_ALIGNMENT 4
+ * 2 byte alignment -> #define MEM_ALIGNMENT 2
+ */
+#define MEM_ALIGNMENT 1
+
+/**
+ * MEM_SIZE: the size of the heap memory. If the application will send
+ * a lot of data that needs to be copied, this should be set high.
+ */
+#define MEM_SIZE 1600
+
+/*
+ ------------------------------------------------
+ ---------- Internal Memory Pool Sizes ----------
+ ------------------------------------------------
+*/
+/**
+ * MEMP_NUM_PBUF: the number of memp struct pbufs (used for PBUF_ROM and PBUF_REF).
+ * If the application sends a lot of data out of ROM (or other static memory),
+ * this should be set high.
+ */
+#define MEMP_NUM_PBUF 4
+
+/**
+ * MEMP_NUM_RAW_PCB: Number of raw connection PCBs
+ * (requires the LWIP_RAW option)
+ */
+#define MEMP_NUM_RAW_PCB 2
+
+/**
+ * MEMP_NUM_UDP_PCB: the number of UDP protocol control blocks. One
+ * per active UDP "connection".
+ * (requires the LWIP_UDP option)
+ */
+#define MEMP_NUM_UDP_PCB 4
+
+/**
+ * MEMP_NUM_TCP_PCB: the number of simulatenously active TCP connections.
+ * (requires the LWIP_TCP option)
+ */
+#define MEMP_NUM_TCP_PCB 2
+
+/**
+ * MEMP_NUM_TCP_PCB_LISTEN: the number of listening TCP connections.
+ * (requires the LWIP_TCP option)
+ */
+#define MEMP_NUM_TCP_PCB_LISTEN 2
+
+/**
+ * MEMP_NUM_TCP_SEG: the number of simultaneously queued TCP segments.
+ * (requires the LWIP_TCP option)
+ */
+#define MEMP_NUM_TCP_SEG 16
+
+/**
+ * MEMP_NUM_REASSDATA: the number of simultaneously IP packets queued for
+ * reassembly (whole packets, not fragments!)
+ */
+#define MEMP_NUM_REASSDATA 1
+
+/**
+ * MEMP_NUM_ARP_QUEUE: the number of simulateously queued outgoing
+ * packets (pbufs) that are waiting for an ARP request (to resolve
+ * their destination address) to finish.
+ * (requires the ARP_QUEUEING option)
+ */
+#define MEMP_NUM_ARP_QUEUE 2
+
+/**
+ * MEMP_NUM_SYS_TIMEOUT: the number of simulateously active timeouts.
+ * (requires NO_SYS==0)
+ */
+#define MEMP_NUM_SYS_TIMEOUT 4
+
+/**
+ * MEMP_NUM_NETBUF: the number of struct netbufs.
+ * (only needed if you use the sequential API, like api_lib.c)
+ */
+#define MEMP_NUM_NETBUF 2
+
+/**
+ * MEMP_NUM_NETCONN: the number of struct netconns.
+ * (only needed if you use the sequential API, like api_lib.c)
+ */
+#define MEMP_NUM_NETCONN 32
+
+/**
+ * MEMP_NUM_TCPIP_MSG_API: the number of struct tcpip_msg, which are used
+ * for callback/timeout API communication.
+ * (only needed if you use tcpip.c)
+ */
+#define MEMP_NUM_TCPIP_MSG_API 8
+
+/**
+ * MEMP_NUM_TCPIP_MSG_INPKT: the number of struct tcpip_msg, which are used
+ * for incoming packets.
+ * (only needed if you use tcpip.c)
+ */
+#define MEMP_NUM_TCPIP_MSG_INPKT 8
+
+/**
+ * PBUF_POOL_SIZE: the number of buffers in the pbuf pool.
+ */
+#define PBUF_POOL_SIZE 8
+
+/*
+ ---------------------------------
+ ---------- ARP options ----------
+ ---------------------------------
+*/
+/**
+ * LWIP_ARP==1: Enable ARP functionality.
+ */
+#define LWIP_ARP 1
+
+/*
+ --------------------------------
+ ---------- IP options ----------
+ --------------------------------
+*/
+/**
+ * IP_FORWARD==1: Enables the ability to forward IP packets across network
+ * interfaces. If you are going to run lwIP on a device with only one network
+ * interface, define this to 0.
+ */
+#define IP_FORWARD 0
+
+/**
+ * IP_OPTIONS: Defines the behavior for IP options.
+ * IP_OPTIONS_ALLOWED==0: All packets with IP options are dropped.
+ * IP_OPTIONS_ALLOWED==1: IP options are allowed (but not parsed).
+ */
+#define IP_OPTIONS_ALLOWED 1
+
+/**
+ * IP_REASSEMBLY==1: Reassemble incoming fragmented IP packets. Note that
+ * this option does not affect outgoing packet sizes, which can be controlled
+ * via IP_FRAG.
+ */
+#define IP_REASSEMBLY 1
+
+/**
+ * IP_FRAG==1: Fragment outgoing IP packets if their size exceeds MTU. Note
+ * that this option does not affect incoming packet sizes, which can be
+ * controlled via IP_REASSEMBLY.
+ */
+#define IP_FRAG 1
+
+/**
+ * IP_REASS_MAXAGE: Maximum time (in multiples of IP_TMR_INTERVAL - so seconds, normally)
+ * a fragmented IP packet waits for all fragments to arrive. If not all fragments arrived
+ * in this time, the whole packet is discarded.
+ */
+#define IP_REASS_MAXAGE 3
+
+/**
+ * IP_REASS_MAX_PBUFS: Total maximum amount of pbufs waiting to be reassembled.
+ * Since the received pbufs are enqueued, be sure to configure
+ * PBUF_POOL_SIZE > IP_REASS_MAX_PBUFS so that the stack is still able to receive
+ * packets even if the maximum amount of fragments is enqueued for reassembly!
+ */
+#define IP_REASS_MAX_PBUFS 4
+
+/**
+ * IP_FRAG_USES_STATIC_BUF==1: Use a static MTU-sized buffer for IP
+ * fragmentation. Otherwise pbufs are allocated and reference the original
+ * packet data to be fragmented.
+*/
+#define IP_FRAG_USES_STATIC_BUF 0
+
+/**
+ * IP_DEFAULT_TTL: Default value for Time-To-Live used by transport layers.
+ */
+#define IP_DEFAULT_TTL 255
+
+/*
+ ----------------------------------
+ ---------- ICMP options ----------
+ ----------------------------------
+*/
+/**
+ * LWIP_ICMP==1: Enable ICMP module inside the IP stack.
+ * Be careful, disable that make your product non-compliant to RFC1122
+ */
+#define LWIP_ICMP 1
+
+/*
+ ---------------------------------
+ ---------- RAW options ----------
+ ---------------------------------
+*/
+/**
+ * LWIP_RAW==1: Enable application layer to hook into the IP layer itself.
+ */
+#define LWIP_RAW 1
+
+/*
+ ----------------------------------
+ ---------- DHCP options ----------
+ ----------------------------------
+*/
+/**
+ * LWIP_DHCP==1: Enable DHCP module.
+ */
+#define LWIP_DHCP 0
+
+
+/*
+ ------------------------------------
+ ---------- AUTOIP options ----------
+ ------------------------------------
+*/
+/**
+ * LWIP_AUTOIP==1: Enable AUTOIP module.
+ */
+#define LWIP_AUTOIP 0
+
+/*
+ ----------------------------------
+ ---------- SNMP options ----------
+ ----------------------------------
+*/
+/**
+ * LWIP_SNMP==1: Turn on SNMP module. UDP must be available for SNMP
+ * transport.
+ */
+#define LWIP_SNMP 0
+
+/*
+ ----------------------------------
+ ---------- IGMP options ----------
+ ----------------------------------
+*/
+/**
+ * LWIP_IGMP==1: Turn on IGMP module.
+ */
+#define LWIP_IGMP 0
+
+/*
+ ----------------------------------
+ ---------- DNS options -----------
+ ----------------------------------
+*/
+/**
+ * LWIP_DNS==1: Turn on DNS module. UDP must be available for DNS
+ * transport.
+ */
+#define LWIP_DNS 0
+
+/*
+ ---------------------------------
+ ---------- UDP options ----------
+ ---------------------------------
+*/
+/**
+ * LWIP_UDP==1: Turn on UDP.
+ */
+#define LWIP_UDP 1
+
+/*
+ ---------------------------------
+ ---------- TCP options ----------
+ ---------------------------------
+*/
+/**
+ * LWIP_TCP==1: Turn on TCP.
+ */
+#define LWIP_TCP 1
+
+#define LWIP_LISTEN_BACKLOG 0
+
+/*
+ ----------------------------------
+ ---------- Pbuf options ----------
+ ----------------------------------
+*/
+/**
+ * PBUF_LINK_HLEN: the number of bytes that should be allocated for a
+ * link level header. The default is 14, the standard value for
+ * Ethernet.
+ */
+#define PBUF_LINK_HLEN 16
+
+/**
+ * PBUF_POOL_BUFSIZE: the size of each pbuf in the pbuf pool. The default is
+ * designed to accommodate single full size TCP frame in one pbuf, including
+ * TCP_MSS, IP header, and link header.
+*
+ */
+#define PBUF_POOL_BUFSIZE LWIP_MEM_ALIGN_SIZE(TCP_MSS+40+PBUF_LINK_HLEN)
+
+/*
+ ------------------------------------
+ ---------- LOOPIF options ----------
+ ------------------------------------
+*/
+/**
+ * LWIP_HAVE_LOOPIF==1: Support loop interface (127.0.0.1) and loopif.c
+ */
+#define LWIP_HAVE_LOOPIF 0
+
+/*
+ ----------------------------------------------
+ ---------- Sequential layer options ----------
+ ----------------------------------------------
+*/
+
+/**
+ * LWIP_NETCONN==1: Enable Netconn API (require to use api_lib.c)
+ */
+#define LWIP_NETCONN 1
+
+/*
+ ------------------------------------
+ ---------- Socket options ----------
+ ------------------------------------
+*/
+/**
+ * LWIP_SOCKET==1: Enable Socket API (require to use sockets.c)
+ */
+#define LWIP_SOCKET 1
+
+/**
+ * SO_REUSE==1: Enable SO_REUSEADDR
+ */
+#define SO_REUSE 1
+
+/*
+ ----------------------------------------
+ ---------- Statistics options ----------
+ ----------------------------------------
+*/
+/**
+ * LWIP_STATS==1: Enable statistics collection in lwip_stats.
+ */
+#define LWIP_STATS 0
+/*
+ ---------------------------------
+ ---------- PPP options ----------
+ ---------------------------------
+*/
+/**
+ * PPP_SUPPORT==1: Enable PPP.
+ */
+#define PPP_SUPPORT 0
+
+
+
+/*
+ ---------------------------------------
+ ---------- Threading options ----------
+ ---------------------------------------
+*/
+
+#define LWIP_TCPIP_CORE_LOCKING 0
+
+// #if !NO_SYS
+// void sys_check_core_locking(void);
+// #define LWIP_ASSERT_CORE_LOCKED() sys_check_core_locking()
+// #endif
+
+#define LWIP_NETIF_LOOPBACK 1
+/* use malloc instead of pool */
+#define MEMP_MEM_MALLOC 1
+#define MEMP_MEM_INIT 1
+#define MEM_LIBC_MALLOC 1
+
+
+
+#endif /* LWIP_LWIPOPTS_H */
diff --git a/lib/lwip/port/if.c b/lib/lwip/port/if.c
new file mode 100644
index 0000000000..dd423eed16
--- /dev/null
+++ b/lib/lwip/port/if.c
@@ -0,0 +1,256 @@
+#include <common.h>
+#include <command.h>
+
+#include "lwip/debug.h"
+#include "lwip/arch.h"
+#include "netif/etharp.h"
+#include "lwip/stats.h"
+#include "lwip/def.h"
+#include "lwip/mem.h"
+#include "lwip/pbuf.h"
+#include "lwip/sys.h"
+#include "lwip/netif.h"
+
+#include "lwip/ip.h"
+
+#define IFNAME0 'e'
+#define IFNAME1 '0'
+
+static struct pbuf * low_level_input(struct netif *netif);
+static int uboot_net_use_lwip = 0;
+
+int lwip_enabled(void)
+{
+ return uboot_net_use_lwip;
+}
+
+static int loop_lwip;
+
+/* ret 0 - loop not done
+ * 1 - loop is done.
+ */
+int lwip_loop_is_done(void)
+{
+ return loop_lwip;
+}
+
+void lwip_loop_set(void)
+{
+ loop_lwip = 1;
+}
+
+struct uboot_lwip_if {
+};
+
+static struct netif uboot_netif;
+
+#define LWIP_PORT_INIT_IPADDR(addr) IP4_ADDR((addr), 192,168,1,200)
+#define LWIP_PORT_INIT_GW(addr) IP4_ADDR((addr), 192,168,1,1)
+#define LWIP_PORT_INIT_NETMASK(addr) IP4_ADDR((addr), 255,255,255,0)
+
+extern uchar *net_rx_packet;
+extern int net_rx_packet_len;
+
+int uboot_lwip_poll(void) {
+ struct pbuf *p;
+ int err;
+
+ //printf("call for netif_poll!!!!\n");
+ //netif_poll(&uboot_netif);
+ p = low_level_input(&uboot_netif);
+ if (NULL == p) {
+ printf("error p = low_level_input = NULL\n");
+ return 0;
+ }
+ err = ethernet_input(p, &uboot_netif);
+ if (err)
+ printf("ip4_input err %d\n", err);
+
+ return 0;
+}
+
+static struct pbuf * low_level_input(struct netif *netif)
+{
+ struct pbuf *p, *q;
+ u16_t len = net_rx_packet_len;
+ uchar *data = net_rx_packet;
+
+ //printf("%s()\n", __func__);
+#if ETH_PAD_SIZE
+ len += ETH_PAD_SIZE; /* allow room for Ethernet padding */
+#endif
+
+ /* We allocate a pbuf chain of pbufs from the pool. */
+ p = pbuf_alloc(PBUF_RAW, len, PBUF_POOL);
+ if (p != NULL) {
+#if ETH_PAD_SIZE
+ pbuf_remove_header(p, ETH_PAD_SIZE); /* drop the padding word */
+#endif
+ /* We iterate over the pbuf chain until we have read the entire
+ * packet into the pbuf. */
+ for (q = p; q != NULL; q = q->next) {
+ /* Read enough bytes to fill this pbuf in the chain. The
+ * available data in the pbuf is given by the q->len
+ * variable.
+ * This does not necessarily have to be a memcpy, you can also preallocate
+ * pbufs for a DMA-enabled MAC and after receiving truncate it to the
+ * actually received size. In this case, ensure the tot_len member of the
+ * pbuf is the sum of the chained pbuf len members.
+ */
+ MEMCPY(q->payload, data, q->len);
+ data += q->len;
+ }
+ //acknowledge that packet has been read();
+
+#if ETH_PAD_SIZE
+ pbuf_add_header(p, ETH_PAD_SIZE); /* reclaim the padding word */
+#endif
+ LINK_STATS_INC(link.recv);
+ } else {
+ //drop packet();
+ LINK_STATS_INC(link.memerr);
+ LINK_STATS_INC(link.drop);
+ }
+
+ return p;
+}
+
+/* Maxim: accoding to lwip idea this has to be separate thread in the background
+ * to poll RX packets from the network. The first we need to try to inject
+ * it directly to net/net.c eth_rx() calls. Assuming we do not implement threading
+ * and we roll in the main polling loop. Then socket operation should also be possible
+ * to call for non blocking socket calls.
+ * If lwip will not work without threading, then we to implement some scheduler with semaphores.
+ * We can base ./examples/standalone/sched.c example code for that.
+ *
+ */
+static int ethernetif_input(struct pbuf *p, struct netif *netif)
+{
+ struct ethernetif *ethernetif;
+
+ ethernetif = netif->state;
+
+ /* move received packet into a new pbuf */
+ p = low_level_input(netif);
+
+ /* if no packet could be read, silently ignore this */
+ if (p != NULL) {
+ /* pass all packets to ethernet_input, which decides what packets it supports */
+ if (netif->input(p, netif) != ERR_OK) {
+ LWIP_DEBUGF(NETIF_DEBUG, ("ethernetif_input: IP input error\n"));
+ pbuf_free(p);
+ p = NULL;
+ }
+ }
+ return 0;
+}
+
+static err_t low_level_output(struct netif *netif, struct pbuf *p)
+{
+ int err;
+
+ //printf("TX data len %d, tot_len %d\n", p->len, p->tot_len);
+ err = eth_send(p->payload, p->len);
+ if (err != 0) {
+ printf("eth_send error %d\n", err);
+ return ERR_ABRT;
+ }
+ return ERR_OK;
+}
+
+err_t uboot_lwip_if_init(struct netif *netif)
+{
+ struct uboot_lwip_if *uif = (struct uboot_lwip_if *)malloc(sizeof(struct uboot_lwip_if));
+
+ if (uif == NULL) {
+ printf("uboot_lwip_if: out of memory\n");
+ return ERR_MEM;
+ }
+ netif->state = uif;
+
+ netif->name[0] = IFNAME0;
+ netif->name[1] = IFNAME1;
+
+ netif->hwaddr_len = ETHARP_HWADDR_LEN;
+ /* set MAC hardware address */
+ netif->hwaddr[0] = 0xf6;
+ netif->hwaddr[1] = 0x11;
+ netif->hwaddr[2] = 0x1;
+ netif->hwaddr[3] = 0x2;
+ netif->hwaddr[4] = 0x3;
+ netif->hwaddr[5] = 0x4;
+
+#if LWIP_IPV4
+ netif->output = etharp_output;
+#endif /* LWIP_IPV4 */
+#if LWIP_IPV6
+ netif->output_ip6 = ethip6_output;
+#endif /* LWIP_IPV6 */
+ netif->linkoutput = low_level_output;
+ netif->mtu = 1500;
+ netif->flags = NETIF_FLAG_BROADCAST | NETIF_FLAG_ETHARP | NETIF_FLAG_LINK_UP;
+
+ eth_init(); /* activate u-boot eth dev */
+
+ printf("Initialized LWIP stack\n");
+ return ERR_OK;
+}
+
+int uboot_lwip_init()
+{
+ ip4_addr_t ipaddr, netmask, gw;
+ //int err;
+
+ if (uboot_net_use_lwip)
+ return CMD_RET_SUCCESS;
+
+ ip4_addr_set_zero(&gw);
+ ip4_addr_set_zero(&ipaddr);
+ ip4_addr_set_zero(&netmask);
+
+#if USE_DHCP
+ printf("Starting lwIP, local interface IP is dhcp-enabled\n");
+#elif USE_AUTOIP
+ printf("Starting lwIP, local interface IP is autoip-enabled\n");
+#else /* USE_DHCP */
+ LWIP_PORT_INIT_GW(&gw);
+ LWIP_PORT_INIT_IPADDR(&ipaddr);
+ LWIP_PORT_INIT_NETMASK(&netmask);
+ printf("Starting lwIP, local interface IP is %s\n", ip4addr_ntoa(&ipaddr));
+#endif /* USE_DHCP */
+
+ if (netif_add(&uboot_netif, &ipaddr, &netmask, &gw, &uboot_netif, uboot_lwip_if_init, ethernetif_input) == NULL)
+ printf("err: netif_add failed!\n");
+ netif_set_up(&uboot_netif);
+ netif_set_link_up(&uboot_netif);
+
+#if LWIP_IPV6
+ netif_create_ip6_linklocal_address(netif_default, 1);
+ printf("ip6 linklocal address: %s\n", ip6addr_ntoa(netif_ip6_addr(netif_default, 0)));
+#endif /* LWIP_IPV6 */
+
+#if LWIP_NETIF_STATUS_CALLBACK
+ netif_set_status_callback(netif_default, status_callback);
+#endif /* LWIP_NETIF_STATUS_CALLBACK */
+#if LWIP_NETIF_LINK_CALLBACK
+ netif_set_link_callback(netif_default, link_callback);
+#endif /* LWIP_NETIF_LINK_CALLBACK */
+
+#if LWIP_AUTOIP
+ autoip_set_struct(netif_default, &netif_autoip);
+#endif /* LWIP_AUTOIP */
+#if LWIP_DHCP
+ dhcp_set_struct(netif_default, &netif_dhcp);
+#endif /* LWIP_DHCP */
+ //netif_set_up(netif_default);
+#if USE_DHCP
+ err = dhcp_start(netif_default);
+ LWIP_ASSERT("dhcp_start failed", err == ERR_OK);
+#elif USE_AUTOIP
+ err = autoip_start(netif_default);
+ LWIP_ASSERT("autoip_start failed", err == ERR_OK);
+#endif /* USE_DHCP */
+ uboot_net_use_lwip = 1;
+
+ return CMD_RET_SUCCESS;
+}
diff --git a/lib/lwip/port/include/arch/cc.h b/lib/lwip/port/include/arch/cc.h
new file mode 100644
index 0000000000..06cc807e85
--- /dev/null
+++ b/lib/lwip/port/include/arch/cc.h
@@ -0,0 +1,41 @@
+#ifndef LWIP_ARCH_CC_H
+#define LWIP_ARCH_CC_H
+
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include "u-sockets.h"
+
+#define LWIP_ERRNO_INCLUDE <errno.h>
+
+#define LWIP_ERRNO_STDINCLUDE 1
+#define LWIP_NO_UNISTD_H 1
+#define LWIP_TIMEVAL_PRIVATE 1
+
+extern unsigned int lwip_port_rand(void);
+#define LWIP_RAND() (lwip_port_rand())
+
+/* different handling for unit test, normally not needed */
+#ifdef LWIP_NOASSERT_ON_ERROR
+#define LWIP_ERROR(message, expression, handler) do { if (!(expression)) { \
+ handler;}} while(0)
+#endif
+
+#define LWIP_DONT_PROVIDE_BYTEORDER_FUNCTIONS
+
+#define LWIP_PLATFORM_ASSERT(x) do {printf("Assertion \"%s\" failed at line %d in %s\n", \
+ x, __LINE__, __FILE__);} while(0)
+
+static inline int atoi(const char* str)
+{
+ int r = 0;
+ int i;
+
+ for (i = 0; str[i] != '\0'; ++i)
+ r = r * 10 + str[i] - '0';
+
+ return r;
+}
+
+#define LWIP_ERR_T int
+
+#endif /* LWIP_ARCH_CC_H */
diff --git a/lib/lwip/port/include/arch/sys_arch.h b/lib/lwip/port/include/arch/sys_arch.h
new file mode 100644
index 0000000000..929a35e12d
--- /dev/null
+++ b/lib/lwip/port/include/arch/sys_arch.h
@@ -0,0 +1,78 @@
+#ifndef LWIP_ARCH_SYS_ARCH_H
+#define LWIP_ARCH_SYS_ARCH_H
+
+#include "lwip/opt.h"
+#include "lwip/arch.h"
+#include "lwip/err.h"
+
+#define ERR_NEED_SCHED 123
+
+void sys_arch_msleep(u32_t delay_ms);
+#define sys_msleep(ms) sys_arch_msleep(ms)
+
+#if SYS_LIGHTWEIGHT_PROT
+typedef u32_t sys_prot_t;
+#endif /* SYS_LIGHTWEIGHT_PROT */
+
+#include <errno.h>
+
+#define SYS_MBOX_NULL NULL
+#define SYS_SEM_NULL NULL
+
+typedef u32_t sys_prot_t;
+
+struct sys_sem;
+typedef struct sys_sem * sys_sem_t;
+#define sys_sem_valid(sem) (((sem) != NULL) && (*(sem) != NULL))
+#define sys_sem_set_invalid(sem) do { if((sem) != NULL) { *(sem) = NULL; }}while(0)
+
+/* let sys.h use binary semaphores for mutexes */
+#define LWIP_COMPAT_MUTEX 1
+#define LWIP_COMPAT_MUTEX_ALLOWED 1
+
+struct sys_mbox;
+typedef struct sys_mbox *sys_mbox_t;
+#define sys_mbox_valid(mbox) (((mbox) != NULL) && (*(mbox) != NULL))
+#define sys_mbox_set_invalid(mbox) do { if((mbox) != NULL) { *(mbox) = NULL; }}while(0)
+
+struct sys_thread;
+typedef struct sys_thread * sys_thread_t;
+
+static inline u32_t sys_arch_sem_wait(sys_sem_t *sem, u32_t timeout)
+{
+ return 0;
+};
+
+static inline err_t sys_mbox_trypost(sys_mbox_t *mbox, void *msg)
+{
+ return 0;
+};
+
+#if 0
+#define sys_sem_new(s, c) ERR_OK
+
+#define sys_sem_wait(s)
+#define sys_sem_free(s)
+#define sys_sem_valid(s) 0
+#define sys_sem_set_invalid(s)
+#define sys_mutex_new(mu) ERR_OK
+#define sys_mutex_lock(mu)
+#define sys_mutex_unlock(mu)
+#define sys_mutex_free(mu)
+#define sys_mutex_valid(mu) 0
+#define sys_mutex_set_invalid(mu)
+#define sys_mbox_new(m, s) ERR_OK
+#define sys_mbox_fetch(m,d)
+#define sys_mbox_tryfetch(m,d)
+#define sys_mbox_post(m,d)
+
+#define sys_mbox_free(m)
+#define sys_mbox_valid(m)
+#define sys_mbox_set_invalid(m)
+#endif
+
+#define sys_sem_signal(s)
+
+//int uboot_lwip_init(void);
+
+#endif /* LWIP_ARCH_SYS_ARCH_H */
diff --git a/lib/lwip/port/include/arch/u-sockets.h b/lib/lwip/port/include/arch/u-sockets.h
new file mode 100644
index 0000000000..aef0b94ee0
--- /dev/null
+++ b/lib/lwip/port/include/arch/u-sockets.h
@@ -0,0 +1,26 @@
+#define MEMP_NUM_NETCONN 32
+#define LWIP_SOCKET_OFFSET 0
+#undef FD_SETSIZE
+/* Make FD_SETSIZE match NUM_SOCKETS in socket.c */
+#define FD_SETSIZE MEMP_NUM_NETCONN
+#define FDSETSAFESET(n, code) do { \
+ if (((n) - LWIP_SOCKET_OFFSET < MEMP_NUM_NETCONN) && (((int)(n) - LWIP_SOCKET_OFFSET) >= 0)) { \
+ code; }} while(0)
+#define FDSETSAFEGET(n, code) (((n) - LWIP_SOCKET_OFFSET < MEMP_NUM_NETCONN) && (((int)(n) - LWIP_SOCKET_OFFSET) >= 0) ?\
+ (code) : 0)
+//#define FD_SET(n, p) FDSETSAFESET(n, (p)->fd_bits[((n)-LWIP_SOCKET_OFFSET)/8] = (u8_t)((p)->fd_bits[((n)-LWIP_SOCKET_OFFSET)/8] | (1 << (((n)-LWIP_SOCKET_OFFSET) & 7))))
+#define FD_SET(n, p) FDSETSAFESET(n, (p)->fds_bits[((n)-LWIP_SOCKET_OFFSET)/8] = (u8_t)((p)->fds_bits[((n)-LWIP_SOCKET_OFFSET)/8] | (1 << (((n)-LWIP_SOCKET_OFFSET) & 7))))
+#define FD_CLR(n, p) FDSETSAFESET(n, (p)->fds_bits[((n)-LWIP_SOCKET_OFFSET)/8] = (u8_t)((p)->fds_bits[((n)-LWIP_SOCKET_OFFSET)/8] & ~(1 << (((n)-LWIP_SOCKET_OFFSET) & 7))))
+#define FD_ISSET(n,p) FDSETSAFEGET(n, (p)->fds_bits[((n)-LWIP_SOCKET_OFFSET)/8] & (1 << (((n)-LWIP_SOCKET_OFFSET) & 7)))
+#define FD_ZERO(p) memset((void*)(p), 0, sizeof(*(p)))
+
+#if 0
+typedef struct fd_set
+{
+ unsigned char fd_bits [(FD_SETSIZE+7)/8];
+} fd_set;
+#endif
+
+#if FD_SETSIZE < (LWIP_SOCKET_OFFSET + MEMP_NUM_NETCONN)
+#error "external FD_SETSIZE too small for number of sockets"
+#endif
diff --git a/lib/lwip/port/include/limits.h b/lib/lwip/port/include/limits.h
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/lib/lwip/port/sys-arch.c b/lib/lwip/port/sys-arch.c
new file mode 100644
index 0000000000..1ae521fde7
--- /dev/null
+++ b/lib/lwip/port/sys-arch.c
@@ -0,0 +1,7 @@
+#include <common.h>
+#include "lwip/opt.h"
+
+u32_t sys_now(void)
+{
+ return get_timer(0);
+}
diff --git a/net/eth-uclass.c b/net/eth-uclass.c
index f41da4b37b..6031ad805d 100644
--- a/net/eth-uclass.c
+++ b/net/eth-uclass.c
@@ -367,8 +367,10 @@ int eth_send(void *packet, int length)
if (!current)
return -ENODEV;
- if (!eth_is_active(current))
+ if (!eth_is_active(current)) {
+ printf("%s() !eth_is_active\n", __func__);
return -EINVAL;
+ }
ret = eth_get_ops(current)->send(current, packet, length);
if (ret < 0) {
diff --git a/net/net.c b/net/net.c
index 57da9bda85..301d825462 100644
--- a/net/net.c
+++ b/net/net.c
@@ -201,6 +201,10 @@ static ulong time_delta;
/* THE transmit packet */
uchar *net_tx_packet;
+extern int lwip_enabled(void);
+extern int lwip_loop_is_done(void);
+extern int uboot_lwip_poll(void);
+
static int net_check_prereq(enum proto_t protocol);
static int net_try_count;
@@ -1177,6 +1181,16 @@ void net_process_received_packet(uchar *in_packet, int len)
if (len < ETHER_HDR_SIZE)
return;
+
+#if defined(CONFIG_LWIP_LIB)
+ if (lwip_enabled()) {
+ if (lwip_loop_is_done())
+ net_set_state(NETLOOP_SUCCESS);
+ uboot_lwip_poll();
+ return;
+ }
+#endif
+
#if defined(CONFIG_API) || defined(CONFIG_EFI_LOADER)
if (push_packet) {
(*push_packet)(in_packet, len);
--
2.30.2
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 3/5] add doc/README.lwip
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
2023-05-05 10:25 ` [RFC PATCH 1/5] add lwip-external submodule Maxim Uvarov
2023-05-05 10:25 ` [RFC PATCH 2/5] lib/lwip: compile-in core files Maxim Uvarov
@ 2023-05-05 10:25 ` Maxim Uvarov
2023-05-11 13:51 ` Tom Rini
2023-05-05 10:25 ` [RFC PATCH 4/5] add doc/README.lwip.size Maxim Uvarov
` (4 subsequent siblings)
7 siblings, 1 reply; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-05 10:25 UTC (permalink / raw)
To: u-boot
Cc: pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev, Maxim Uvarov
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
---
doc/README.lwip | 56 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
create mode 100644 doc/README.lwip
diff --git a/doc/README.lwip b/doc/README.lwip
new file mode 100644
index 0000000000..df3462ca1b
--- /dev/null
+++ b/doc/README.lwip
@@ -0,0 +1,56 @@
+ RFC LWIP IP stack intergation for U-boot
+ ----------------------------------------
+
+Reliable and bug free IP stack is usually an issue when you are trying to write it
+from scratch. It looks like not, but when addinging new features it will be chelledging.
+This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
+TCP/IP stack designed for embedded systems for U-boot. That will allow using already
+written network applications for microcontrollers.
+
+LwIP license:
+lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
+
+Main features include:
+- Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
+- DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf), SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
+- APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
+- Extended features: IP forwarding over multiple network interfaces, TCP congestion control, RTT estimation and fast recovery/fast retransmit
+- Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver, mDNS responder, MQTT client, TFTP server
+
+U-boot implementation details:
+
+1. In general we can build lwIP as .a library and link it against u-boot or compile it in
+the U-boot tree in the same way as other U-boot files. There are few reasons why I selected
+the second variant: iwIP is very customizable with defines for features, memory size, types of
+allocation, some internal types and platform specific code. And it was more easy to enable/disable
+ debug which is also done with defines, and is needed periodically.
+
+2. lwIP has 2 APIs - raw mode and sequential (as lwIP names it, or socket API as we name it in Linux).
+ This RFC implements only raw API as the proof of work.
+
+Raw IP means that the call back function for RX path is registered and will be called when packet
+data passes the IP stack and is ready for the application.
+
+This RFC has an unmodified working ping example from lwip sources which registeres this callback.
+ ping_pcb = raw_new(IP_PROTO_ICMP);
+ raw_recv(ping_pcb, ping_recv, NULL); <- ping_recv is app callback.
+ raw_bind(ping_pcb, IP_ADDR_ANY)
+
+Socket API also gives nice advantages due it will be easy to port linux socket applications to u-boot.
+I.e. lwIP sockets compatible with the linux ones. But that will require RX thread running in the background.
+So that means we need some kind of scheduler, locking and threading support or find some other solution.
+
+3. Input and output
+
+RX packet path is injected to U-boot eth_rx() polling loop and TX patch is in eth_send() accordingly.
+So we do not touch any drivers code and just eat packets when they are ready.
+
+4. Testing
+
+Unmodified ping example can be used. I did ping from qemu/u-boot tap device on the host:
+
+=> lwip init
+=> lwip ping 192.168.1.2
+ping: recv 3 ms
+tcpdump on the host:
+5:09:10.925951 ARP, Request who-has 192.168.1.200 tell 192.168.1.200, length 28 15:09:12.114643 IP6 fe80::38e2:41ff:fec3:8bda > ip6-allrouters: ICMP6, router solicitation, length 16 15:09:20.370725 ARP, Request who-has 192.168.1.2 tell 192.168.1.200, length 28 15:09:20.370740 ARP, Reply 192.168.1.2 is-at 3a:e2:41:c3:8b:da (oui Unknown), length 28 15:09:20.372789 IP 192.168.1.200 > 192.168.1.2: ICMP echo request, id 44975, seq 1, length 40 15:09:20.372810 IP 192.168.1.2 > 192.168.1.200: ICMP echo reply, id 44975, seq 1, length 40 15:09:25.426636 ARP, Request who-has 192.168.1.200 tell 192.168.1.2, length 28 15:09:25.426870 ARP, Reply 192.168.1.200 is-at f6:11:01:02:03:04 (oui Unknown), length 28
--
2.30.2
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 4/5] add doc/README.lwip.size
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
` (2 preceding siblings ...)
2023-05-05 10:25 ` [RFC PATCH 3/5] add doc/README.lwip Maxim Uvarov
@ 2023-05-05 10:25 ` Maxim Uvarov
2023-05-11 13:51 ` Tom Rini
2023-05-05 10:25 ` [RFC PATCH 5/5] lwip: implement wget command from http_client.c example Maxim Uvarov
` (3 subsequent siblings)
7 siblings, 1 reply; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-05 10:25 UTC (permalink / raw)
To: u-boot
Cc: pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev, Maxim Uvarov
Add doc with size calculation accoding to original u-boot.
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
---
doc/README.lwip.size | 291 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 291 insertions(+)
create mode 100644 doc/README.lwip.size
diff --git a/doc/README.lwip.size b/doc/README.lwip.size
new file mode 100644
index 0000000000..7d3ff1e4d6
--- /dev/null
+++ b/doc/README.lwip.size
@@ -0,0 +1,291 @@
+Size with lwip stack + ping app + http wget app related
+to original U-boot is:
+
+Total: Before=546322, After=594494, chg +8.82%
+Total: Before=99215, After=100365, chg +1.16%
+Total: Before=77777, After=78227, chg +0.58%
+
+(gdb) p (594494-546322) + (100365-99215) + (78227-77777)
+$1 = 49772
+(gdb) p 49772 / 1024
+$2 = 48
+
+Size will be increased on 48Kib.
+
+Full log:
+linux/scripts/bloat-o-meter -c u-boot_orig/u-boot u-boot/u-boot
+add/remove: 187/0 grow/shrink: 2/0 up/down: 48172/0 (48172)
+Function old new delta
+tcp_input - 4364 +4364
+tcp_receive - 3444 +3444
+tcp_write - 2192 +2192
+ip4_reass - 2096 +2096
+tcp_output - 1616 +1616
+tcp_slowtmr - 1572 +1572
+httpc_tcp_recv - 1044 +1044
+udp_input - 812 +812
+ip4_frag - 748 +748
+tcp_close_shutdown - 716 +716
+ip4_input - 688 +688
+icmp_input - 672 +672
+tcp_split_unsent_seg - 660 +660
+ip4_output_if_src - 612 +612
+tcp_connect - 608 +608
+etharp_input - 568 +568
+httpc_init_connection_common.constprop - 556 +556
+etharp_query - 556 +556
+ip_reass_free_complete_datagram - 520 +520
+etharp_output - 512 +512
+pbuf_alloc - 488 +488
+tcp_alloc - 484 +484
+pbuf_copy_partial_pbuf - 484 +484
+etharp_find_entry - 476 +476
+tcp_enqueue_flags - 440 +440
+tcp_create_segment - 424 +424
+tcp_abandon - 420 +420
+netif_add - 412 +412
+etharp_raw - 408 +408
+tcp_zero_window_probe - 400 +400
+raw_sendto_if_src - 360 +360
+tcp_pcb_remove - 356 +356
+raw_input - 352 +352
+pbuf_realloc - 336 +336
+tcp_free_acked_segments.constprop - 328 +328
+pbuf_free - 328 +328
+icmp_send_response - 324 +324
+tcp_output_alloc_header_common.constprop - 316 +316
+tcp_oos_insert_segment - 312 +312
+netif_loop_output - 296 +296
+httpc_create_request_string.constprop.isra - 292 +292
+ethernet_input - 284 +284
+etharp_output_to_arp_index - 280 +280
+tcp_rexmit - 276 +276
+netif_poll - 276 +276
+ip_reass_remove_oldest_datagram - 272 +272
+tcp_rexmit_rto_prepare - 260 +260
+tcp_pbuf_prealloc - 260 +260
+ping_send - 256 +256
+tcp_parseopt - 244 +244
+ping_recv - 244 +244
+ethernet_output - 236 +236
+inet_chksum_pseudo - 216 +216
+ip4addr_ntoa_r - 212 +212
+tcp_send_fin - 208 +208
+tcp_process_refused_data - 208 +208
+pbuf_copy_partial - 208 +208
+netif_set_addr - 204 +204
+tcp_output_control_segment_netif - 200 +200
+tcp_fasttmr - 200 +200
+pbuf_cat - 200 +200
+uboot_lwip_init - 196 +196
+lwip_wget - 192 +192
+tcp_rst_common.isra - 188 +188
+ip4_route - 188 +188
+tcp_update_rcv_ann_wnd - 184 +184
+sys_timeout_abs - 184 +184
+pbuf_memcmp - 184 +184
+tcp_recved - 180 +180
+pbuf_add_header_impl - 176 +176
+lwip_ping_init - 176 +176
+netif_do_set_ipaddr.isra - 172 +172
+httpc_free_state - 172 +172
+tcp_close_shutdown_fin - 168 +168
+httpc_recv - 168 +168
+tcp_send_empty_ack - 164 +164
+tcp_output_control_segment - 164 +164
+do_ops - 164 +164
+tcp_rexmit_fast - 160 +160
+raw_sendto - 160 +160
+do_lwip_ping - 156 +156
+pbuf_remove_header - 144 +144
+uboot_lwip_if_init - 140 +140
+tcp_input_delayed_close - 140 +140
+do_lwip_wget - 140 +140
+tcp_keepalive - 136 +136
+tcp_output_alloc_header.constprop - 132 +132
+tcp_netif_ip_addr_changed - 132 +132
+inet_chksum_pbuf - 132 +132
+pbuf_memfind - 128 +128
+pbuf_alloc_reference - 128 +128
+lwip_standard_chksum - 128 +128
+tcp_pcb_purge - 124 +124
+tcp_new_port - 124 +124
+pbuf_free_header - 120 +120
+tcp_rst_netif - 116 +116
+tcp_netif_ip_addr_changed_pcblist - 116 +116
+netif_issue_reports - 116 +116
+tcpip_try_callback - 112 +112
+tcp_poll - 112 +112
+tcp_eff_send_mss_netif - 112 +112
+ip_reass_dequeue_datagram - 112 +112
+ip4_input_accept - 112 +112
+httpc_tcp_connected - 112 +112
+etharp_free_entry - 112 +112
+tcp_rst - 108 +108
+ping_timeout - 108 +108
+low_level_input.constprop - 108 +108
+tcp_seg_copy - 104 +104
+tcp_next_iss - 104 +104
+sys_timeout - 104 +104
+httpc_get_internal_addr - 104 +104
+tcp_recv_null - 96 +96
+pbuf_clone - 96 +96
+tcp_rexmit_rto - 92 +92
+ip4_addr_isbroadcast_u32 - 92 +92
+tcp_sent - 88 +88
+tcp_rexmit_rto_commit - 88 +88
+tcp_recv - 88 +88
+tcp_kill_state - 88 +88
+tcp_err - 88 +88
+raw_new - 88 +88
+lwip_ping_send_now - 84 +84
+tcp_output_segment_busy - 80 +80
+tcp_get_next_optbyte - 80 +80
+tcp_free - 80 +80
+pbuf_alloced_custom - 80 +80
+ipfrag_free_pbuf_custom - 80 +80
+httpc_close - 80 +80
+uboot_lwip_poll - 76 +76
+tcpip_tcp_timer - 76 +76
+udp_netif_ip_addr_changed - 72 +72
+raw_netif_ip_addr_changed - 72 +72
+ip_frag_free_pbuf_custom_ref - 72 +72
+tcp_timer_needed - 68 +68
+tcp_close - 68 +68
+mem_free - 68 +68
+ethernetif_input - 68 +68
+pbuf_try_get_at - 64 +64
+pbuf_ref - 60 +60
+net_process_received_packet 768 828 +60
+memp_malloc - 60 +60
+tcp_seg_free - 56 +56
+netif_get_by_index - 56 +56
+low_level_output - 56 +56
+tcp_tmr - 52 +52
+tcp_segs_free - 48 +48
+pbuf_skip_const - 48 +48
+pbuf_copy - 48 +48
+httpc_tcp_poll - 48 +48
+tcp_free_ooseq - 44 +44
+pbuf_free_ooseq_callback - 44 +44
+netif_set_up - 44 +44
+netif_set_link_up - 44 +44
+tcp_output_fill_options.constprop - 40 +40
+etharp_request - 40 +40
+do_lwip_info - 40 +40
+raw_bind - 36 +36
+pbuf_chain - 36 +36
+memp_free - 36 +36
+ip4_output_if - 36 +36
+pbuf_header_force - 32 +32
+pbuf_clen - 32 +32
+inet_chksum - 32 +32
+pbuf_get_at - 28 +28
+httpc_tcp_err - 28 +28
+do_lwip_init - 28 +28
+sys_now - 24 +24
+tcp_trigger_input_pcb_close - 20 +20
+eth_send 116 136 +20
+lwip_loop_set - 16 +16
+ip4addr_ntoa - 16 +16
+tcp_arg - 12 +12
+lwip_loop_is_done - 12 +12
+lwip_enabled - 12 +12
+icmp_time_exceeded - 12 +12
+icmp_dest_unreach - 12 +12
+tcp_new - 8 +8
+tcp_abort - 8 +8
+raw_recv - 8 +8
+pbuf_add_header - 8 +8
+netif_null_output_ip4 - 8 +8
+lwip_htons - 8 +8
+lwip_htonl - 8 +8
+httpc_tcp_sent - 8 +8
+mem_trim - 4 +4
+mem_malloc - 4 +4
+ip_chksum_pseudo - 4 +4
+httpc_init_connection - 4 +4
+Total: Before=546322, After=594494, chg +8.82%
+add/remove: 53/0 grow/shrink: 0/0 up/down: 1150/0 (1150)
+Data old new delta
+arp_table - 400 +400
+cmds - 224 +224
+uboot_netif - 96 +96
+_u_boot_list_2_cmd_2_lwipinfo - 56 +56
+_u_boot_list_2_cmd_2_lwip - 56 +56
+ip_data - 40 +40
+inseg - 32 +32
+settings - 24 +24
+str - 16 +16
+udp_pcbs - 8 +8
+tcpip_mbox - 8 +8
+tcphdr_opt2 - 8 +8
+tcphdr - 8 +8
+tcp_tw_pcbs - 8 +8
+tcp_listen_pcbs - 8 +8
+tcp_input_pcb - 8 +8
+tcp_bound_pcbs - 8 +8
+tcp_active_pcbs - 8 +8
+recv_data - 8 +8
+reassdatagrams - 8 +8
+raw_pcbs - 8 +8
+ping_target - 8 +8
+ping_pcb - 8 +8
+next_timeout - 8 +8
+netif_list - 8 +8
+netif_default - 8 +8
+daddr - 8 +8
+uboot_net_use_lwip - 4 +4
+tcpip_tcp_timer_active - 4 +4
+tcp_ticks - 4 +4
+seqno - 4 +4
+ping_time - 4 +4
+loop_lwip - 4 +4
+iss - 4 +4
+ip_target - 4 +4
+ackno - 4 +4
+tcplen - 2 +2
+tcphdr_optlen - 2 +2
+tcphdr_opt1len - 2 +2
+tcp_port - 2 +2
+tcp_optidx - 2 +2
+recv_acked - 2 +2
+ping_seq_num - 2 +2
+ip_reass_pbufcount - 2 +2
+ip_id - 2 +2
+tcp_timer_ctr - 1 +1
+tcp_timer - 1 +1
+tcp_active_pcbs_changed - 1 +1
+recv_flags - 1 +1
+pbuf_free_ooseq_pending - 1 +1
+netif_num - 1 +1
+flags - 1 +1
+etharp_cached_entry - 1 +1
+Total: Before=99215, After=100365, chg +1.16%
+add/remove: 21/0 grow/shrink: 2/0 up/down: 450/0 (450)
+RO Data old new delta
+memp_pools - 112 +112
+tcp_pcb_lists - 32 +32
+version_string 46 70 +24
+__func__ 1317 1339 +22
+memp_UDP_PCB - 16 +16
+memp_TCP_SEG - 16 +16
+memp_TCP_PCB_LISTEN - 16 +16
+memp_TCP_PCB - 16 +16
+memp_TCPIP_MSG_INPKT - 16 +16
+memp_TCPIP_MSG_API - 16 +16
+memp_SYS_TIMEOUT - 16 +16
+memp_REASSDATA - 16 +16
+memp_RAW_PCB - 16 +16
+memp_PBUF_POOL - 16 +16
+memp_PBUF - 16 +16
+memp_NETCONN - 16 +16
+memp_NETBUF - 16 +16
+memp_FRAG_PBUF - 16 +16
+tcp_backoff - 13 +13
+tcp_persist_backoff - 7 +7
+ethzero - 6 +6
+ethbroadcast - 6 +6
+ip_addr_any - 4 +4
+Total: Before=77777, After=78227, chg +0.58%
+
--
2.30.2
^ permalink raw reply related [flat|nested] 37+ messages in thread
* [RFC PATCH 5/5] lwip: implement wget command from http_client.c example
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
` (3 preceding siblings ...)
2023-05-05 10:25 ` [RFC PATCH 4/5] add doc/README.lwip.size Maxim Uvarov
@ 2023-05-05 10:25 ` Maxim Uvarov
2023-05-05 10:27 ` [RFC PATCH 0/5] LWIP stack integration Ilias Apalodimas
` (2 subsequent siblings)
7 siblings, 0 replies; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-05 10:25 UTC (permalink / raw)
To: u-boot
Cc: pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev, Maxim Uvarov
Intend of RFC is to show how we can reuse existance lwip apps
and examples inside u-boot. This commit shows how to do that with
minimal changes.
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
---
.gitignore | 2 +
doc/README.lwip | 34 ++++++++++++++++
lib/lwip/Makefile | 8 ++++
lib/lwip/apps/http/lwip-wget.c | 67 +++++++++++++++++++++++++++++++
lib/lwip/apps/http/rmstatic.patch | 47 ++++++++++++++++++++++
lib/lwip/cmd-lwip.c | 31 ++++++++++++++
6 files changed, 189 insertions(+)
create mode 100644 lib/lwip/apps/http/lwip-wget.c
create mode 100644 lib/lwip/apps/http/rmstatic.patch
diff --git a/.gitignore b/.gitignore
index aeaf847543..339692f5a7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -107,3 +107,5 @@ __pycache__
/lib/lwip/lwip-external
lib/lwip/apps/ping/ping.c
+lib/lwip/apps/http/http_client.c
+lib/lwip/apps/http/http_client.h
diff --git a/doc/README.lwip b/doc/README.lwip
index df3462ca1b..b59a1568b4 100644
--- a/doc/README.lwip
+++ b/doc/README.lwip
@@ -54,3 +54,37 @@ Unmodified ping example can be used. I did ping from qemu/u-boot tap device on t
ping: recv 3 ms
tcpdump on the host:
5:09:10.925951 ARP, Request who-has 192.168.1.200 tell 192.168.1.200, length 28 15:09:12.114643 IP6 fe80::38e2:41ff:fec3:8bda > ip6-allrouters: ICMP6, router solicitation, length 16 15:09:20.370725 ARP, Request who-has 192.168.1.2 tell 192.168.1.200, length 28 15:09:20.370740 ARP, Reply 192.168.1.2 is-at 3a:e2:41:c3:8b:da (oui Unknown), length 28 15:09:20.372789 IP 192.168.1.200 > 192.168.1.2: ICMP echo request, id 44975, seq 1, length 40 15:09:20.372810 IP 192.168.1.2 > 192.168.1.200: ICMP echo reply, id 44975, seq 1, length 40 15:09:25.426636 ARP, Request who-has 192.168.1.200 tell 192.168.1.2, length 28 15:09:25.426870 ARP, Reply 192.168.1.200 is-at f6:11:01:02:03:04 (oui Unknown), length 28
+
+
+5. Wget example
+
+Http server has 192.168.1.2 IP address. (I did not port DNS resolving yet,
+but it's also exist in lwip.) So example just downloads some file with http
+protocol:
+
+Net: eth0: virtio-net#30
+Hit any key to stop autoboot: 0
+=> lwip init
+Starting lwIP, local interface IP is 192.168.1.200
+Initialized LWIP stack
+=> lwip wget http://192.168.1.2/
+downloading http://192.168.1.2/ to addr 0x40200000
+downloaded chunk size 294, to addr 0x40200000
+downloaded chunk size 318, to addr 0x40200126
+=> md 0x40200000 0x40
+40200000: 4f44213c 50595443 74682045 0a3e6c6d <!DOCTYPE html>.
+40200010: 6d74683c 3c0a3e6c 64616568 743c0a3e <html>.<head>.<t
+40200020: 656c7469 6c65573e 656d6f63 206f7420 itle>Welcome to
+40200030: 6e69676e 2f3c2178 6c746974 3c0a3e65 nginx!</title>.<
+40200040: 6c797473 200a3e65 62202020 2079646f style>. body
+40200050: 20200a7b 20202020 69772020 3a687464 {. width:
+40200060: 65353320 200a3b6d 20202020 6d202020 35em;. m
+40200070: 69677261 30203a6e 74756120 200a3b6f argin: 0 auto;.
+40200080: 20202020 66202020 2d746e6f 696d6166 font-fami
+40200090: 203a796c 6f686154 202c616d 64726556 ly: Tahoma, Verd
+402000a0: 2c616e61 69724120 202c6c61 736e6173 ana, Arial, sans
+402000b0: 7265732d 0a3b6669 20202020 2f3c0a7d -serif;. }.</
+402000c0: 6c797473 3c0a3e65 6165682f 3c0a3e64 style>.</head>.<
+402000d0: 79646f62 683c0a3e 65573e31 6d6f636c body>.<h1>Welcom
+402000e0: 6f742065 69676e20 3c21786e 3e31682f e to nginx!</h1>
+402000f0: 3e703c0a 79206649 7320756f 74206565 .<p>If you see t
diff --git a/lib/lwip/Makefile b/lib/lwip/Makefile
index 2c665dcb88..e28cfd726e 100644
--- a/lib/lwip/Makefile
+++ b/lib/lwip/Makefile
@@ -75,4 +75,12 @@ $(obj)/apps/ping/ping.c:
obj-$(CONFIG_CMD_LWIP) += apps/ping/ping.o
obj-$(CONFIG_CMD_LWIP) += apps/ping/lwip_ping.o
+$(obj)/apps/http/http_clinet.o: $(obj)/apps/http/http_client.c
+$(obj)/apps/http/http_client.c:
+ cp ./lib/lwip/lwip-external/src/apps/http/http_client.c $(obj)/apps/http/http_client.c
+ cp ./lib/lwip/lwip-external/src/include/lwip/apps/http_client.h $(obj)/apps/http/http_client.h
+ patch -p0 < $(obj)/apps/http/rmstatic.patch
+
+obj-y += apps/http/http_client.o
+obj-y += apps/http/lwip-wget.o
diff --git a/lib/lwip/apps/http/lwip-wget.c b/lib/lwip/apps/http/lwip-wget.c
new file mode 100644
index 0000000000..f3295d67f1
--- /dev/null
+++ b/lib/lwip/apps/http/lwip-wget.c
@@ -0,0 +1,67 @@
+#include <common.h>
+#include <command.h>
+#include <console.h>
+
+#include "http_client.h"
+
+extern err_t
+httpc_init_connection(httpc_state_t **connection, const httpc_connection_t *settings, const char* server_name,
+ u16_t server_port, const char* uri, altcp_recv_fn recv_fn, void* callback_arg);
+extern err_t
+httpc_get_internal_addr(httpc_state_t* req, const ip_addr_t *ipaddr);
+extern err_t
+httpc_free_state(httpc_state_t* req);
+
+static ulong daddr;
+static httpc_connection_t settings;
+
+static err_t
+httpc_recv(void *arg, struct altcp_pcb *pcb, struct pbuf *p, err_t err)
+{
+ struct pbuf* q;
+ LWIP_UNUSED_ARG(err);
+
+ if (p == NULL) {
+ printf("httpc_tcp_recv bug!\n");
+ return ERR_BUF;
+ }
+
+ for (q = p; q != NULL; q = q->next) {
+ memcpy(daddr, q->payload, q->len);
+ printf("downloaded chunk size %d, to addr 0x%lx\n", q->len, daddr);
+ daddr += q->len;
+ }
+ altcp_recved(pcb, p->tot_len);
+ pbuf_free(p);
+ return ERR_OK;
+}
+
+int lwip_wget(ulong addr, char *url)
+{
+ httpc_state_t* req;
+ err_t err;
+ int port = 80;
+ ip4_addr_t server_addr;
+ char *server_name;
+
+ daddr = addr;
+ IP4_ADDR(&server_addr, 192,168,1,2);
+ server_name = ipaddr_ntoa(&server_addr);
+
+ memset(&settings, 0, sizeof(httpc_connection_t));
+ err = httpc_init_connection(&req, &settings, server_name, port,
+ url, httpc_recv, NULL /*&addr*/);
+ if (err != ERR_OK) {
+ printf("httpc_init_connection failed\n");
+ return err;
+ }
+
+ err = httpc_get_internal_addr(req, &server_addr);
+ if (err != ERR_OK) {
+ httpc_free_state(req);
+ printf("error httpc_get_internal_addr\n");
+ return err;
+ }
+
+ return 0;
+}
diff --git a/lib/lwip/apps/http/rmstatic.patch b/lib/lwip/apps/http/rmstatic.patch
new file mode 100644
index 0000000000..547236de52
--- /dev/null
+++ b/lib/lwip/apps/http/rmstatic.patch
@@ -0,0 +1,47 @@
+--- ./lib/lwip/lwip-external/src/apps/http/http_client.c 2023-05-03 15:26:42.038088829 +0000
++++ lib/lwip/apps/http/http_client.c 2023-05-03 15:27:25.298151160 +0000
+@@ -45,7 +48,7 @@
+ * - IPv6 support
+ */
+
+-#include "lwip/apps/http_client.h"
++#include "http_client.h"
+
+ #include "lwip/altcp_tcp.h"
+ #include "lwip/dns.h"
+@@ -153,7 +156,7 @@ typedef struct _httpc_state
+ } httpc_state_t;
+
+ /** Free http client state and deallocate all resources within */
+-static err_t
++/*static*/ err_t
+ httpc_free_state(httpc_state_t* req)
+ {
+ struct altcp_pcb* tpcb;
+@@ -415,7 +418,7 @@ httpc_tcp_connected(void *arg, struct al
+ }
+
+ /** Start the http request when the server IP addr is known */
+-static err_t
++/*static*/ err_t
+ httpc_get_internal_addr(httpc_state_t* req, const ip_addr_t *ipaddr)
+ {
+ err_t err;
+@@ -592,7 +595,7 @@ httpc_init_connection_common(httpc_state
+ /**
+ * Initialize the connection struct
+ */
+-static err_t
++/* static */ err_t
+ httpc_init_connection(httpc_state_t **connection, const httpc_connection_t *settings, const char* server_name,
+ u16_t server_port, const char* uri, altcp_recv_fn recv_fn, void* callback_arg)
+ {
+@@ -786,7 +789,7 @@ httpc_fs_result(void *arg, httpc_result_
+ }
+
+ /** tcp recv callback */
+-static err_t
++/*static*/ err_t
+ httpc_fs_tcp_recv(void *arg, struct altcp_pcb *pcb, struct pbuf *p, err_t err)
+ {
+ httpc_filestate_t *filestate = (httpc_filestate_t*)arg;
diff --git a/lib/lwip/cmd-lwip.c b/lib/lwip/cmd-lwip.c
index f2e25a8c29..797f0f2fe2 100644
--- a/lib/lwip/cmd-lwip.c
+++ b/lib/lwip/cmd-lwip.c
@@ -10,6 +10,7 @@
#include <display_options.h>
#include <memalign.h>
#include <net.h>
+#include <image.h>
#include "apps/ping/lwip_ping.h"
@@ -57,10 +58,39 @@ static int do_lwip_ping(struct cmd_tbl *cmdtp, int flag, int argc,
return CMD_RET_SUCCESS;
}
+extern int lwip_wget(ulong addr, char *url);
+
+static int do_lwip_wget(struct cmd_tbl *cmdtp, int flag, int argc,
+ char *const argv[])
+{
+ char *url;
+
+ if (argc < 2) {
+ printf("argc = %d, error\n", argc);
+ return CMD_RET_USAGE;
+ }
+
+ url = argv[1];
+ printf("downloading %s to addr 0x%lx\n", url, image_load_addr);
+
+ eth_init(); /* activate u-boot eth dev */
+
+ lwip_wget(image_load_addr, url);
+
+ lwip_loop_set();
+ if (net_loop(LWIP) < 0) {
+ printf("wget failed\n");
+ return CMD_RET_FAILURE;
+ }
+
+ return CMD_RET_SUCCESS;
+}
+
static struct cmd_tbl cmds[] = {
U_BOOT_CMD_MKENT(info, 1, 0, do_lwip_info, "", ""),
U_BOOT_CMD_MKENT(init, 1, 0, do_lwip_init, "", ""),
U_BOOT_CMD_MKENT(ping, 2, 0, do_lwip_ping, "", ""),
+ U_BOOT_CMD_MKENT(wget, 2, 0, do_lwip_wget, "", ""),
};
static int do_ops(struct cmd_tbl *cmdtp, int flag, int argc,
@@ -88,6 +118,7 @@ U_BOOT_CMD(
"info - display info\n"
"init - init LWIP\n"
"ping addr - ping addr\n"
+ "wget http://192.168.1.2/ \n"
);
/* Old command kept for compatibility. Same as 'mmc info' */
--
2.30.2
^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
` (4 preceding siblings ...)
2023-05-05 10:25 ` [RFC PATCH 5/5] lwip: implement wget command from http_client.c example Maxim Uvarov
@ 2023-05-05 10:27 ` Ilias Apalodimas
2023-05-08 21:23 ` Simon Glass
2023-05-11 13:52 ` Tom Rini
7 siblings, 0 replies; 37+ messages in thread
From: Ilias Apalodimas @ 2023-05-05 10:27 UTC (permalink / raw)
To: Maxim Uvarov
Cc: u-boot, pbrobinson, joe.hershberger, rfried.dev, Tom Rini, Simon Glass
+cc Tom and Simon at least on the cover letter so they can take a peek
On Fri, 5 May 2023 at 13:25, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>
> Greetings,
>
> This RFC patchset is an attempt to try to use an already existing IP network stack inside U-boot.
> U-Boot recently got basic TCP/IP support, implementing wget, but in order to get a full IP stack
> with new features (e.g ipv6), it would be preferable to use an established embedded ip library,
> instead of rewriting the code from scratch.
>
> For this experiment LWIP network stack was selected:
> https://savannah.nongnu.org/git/?group=lwip
>
> LWIP main features include:
> - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
> - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf),
> SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> - APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
> - Extended features: IP forwarding over multiple network interfaces, TCP congestion control,
> RTT estimation and fast recovery/fast retransmit
> - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver,
> mDNS responder, MQTT client, TFTP server.
>
> This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
> TCP/IP stack designed for embedded systems for U-boot. That will allow using already
> written network applications for microcontrollers.
>
> lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
> Which should be compatible with u-boot.
>
> In the current RFC I tried to use minimal changes to better see how LWIP code can be embedded into
> U-boot. Patches implement ping and wget commands work. Both commands are currently copy pasting and
> reusing lwIP examples. Whether we want to add the final application in U-Boot or lwIP is up to
> discussion, but the current approach was the easiest one for an RFC.
>
> Looking for your comments,
> Best regards,
> Maxim.
>
> Maxim Uvarov (5):
> add lwip-external submodule
> lib/lwip: compile-in core files
> add doc/README.lwip
> add doc/README.lwip.size
> lwip: implement wget command from http_client.c example
>
> .gitignore | 5 +
> .gitmodules | 3 +
> doc/README.lwip | 90 +++++
> doc/README.lwip.size | 291 +++++++++++++++
> include/net.h | 2 +-
> lib/Kconfig | 2 +
> lib/Makefile | 2 +
> lib/lwip/Kconfig | 12 +
> lib/lwip/Makefile | 86 +++++
> lib/lwip/apps/http/lwip-wget.c | 67 ++++
> lib/lwip/apps/http/rmstatic.patch | 47 +++
> lib/lwip/apps/ping/lwip_ping.c | 33 ++
> lib/lwip/apps/ping/lwip_ping.h | 19 +
> lib/lwip/apps/ping/ping.h | 0
> lib/lwip/apps/ping/rmstatic.patch | 32 ++
> lib/lwip/cmd-lwip.c | 129 +++++++
> lib/lwip/lwip-external | 1 +
> lib/lwip/lwipopts.h | 484 +++++++++++++++++++++++++
> lib/lwip/port/if.c | 256 +++++++++++++
> lib/lwip/port/include/arch/cc.h | 41 +++
> lib/lwip/port/include/arch/sys_arch.h | 78 ++++
> lib/lwip/port/include/arch/u-sockets.h | 26 ++
> lib/lwip/port/include/limits.h | 0
> lib/lwip/port/sys-arch.c | 7 +
> net/eth-uclass.c | 4 +-
> net/net.c | 14 +
> 26 files changed, 1729 insertions(+), 2 deletions(-)
> create mode 100644 .gitmodules
> create mode 100644 doc/README.lwip
> create mode 100644 doc/README.lwip.size
> create mode 100644 lib/lwip/Kconfig
> create mode 100644 lib/lwip/Makefile
> create mode 100644 lib/lwip/apps/http/lwip-wget.c
> create mode 100644 lib/lwip/apps/http/rmstatic.patch
> create mode 100644 lib/lwip/apps/ping/lwip_ping.c
> create mode 100644 lib/lwip/apps/ping/lwip_ping.h
> create mode 100644 lib/lwip/apps/ping/ping.h
> create mode 100644 lib/lwip/apps/ping/rmstatic.patch
> create mode 100644 lib/lwip/cmd-lwip.c
> create mode 160000 lib/lwip/lwip-external
> create mode 100644 lib/lwip/lwipopts.h
> create mode 100644 lib/lwip/port/if.c
> create mode 100644 lib/lwip/port/include/arch/cc.h
> create mode 100644 lib/lwip/port/include/arch/sys_arch.h
> create mode 100644 lib/lwip/port/include/arch/u-sockets.h
> create mode 100644 lib/lwip/port/include/limits.h
> create mode 100644 lib/lwip/port/sys-arch.c
>
> --
> 2.30.2
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 1/5] add lwip-external submodule
2023-05-05 10:25 ` [RFC PATCH 1/5] add lwip-external submodule Maxim Uvarov
@ 2023-05-08 14:43 ` Simon Glass
2023-05-10 7:40 ` Ilias Apalodimas
2023-05-11 13:51 ` Tom Rini
1 sibling, 1 reply; 37+ messages in thread
From: Simon Glass @ 2023-05-08 14:43 UTC (permalink / raw)
To: Maxim Uvarov
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
Hi Maxim,
On Fri, 5 May 2023 at 04:50, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>
> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
> ---
> .gitmodules | 3 +++
> lib/lwip/lwip-external | 1 +
> 2 files changed, 4 insertions(+)
> create mode 100644 .gitmodules
> create mode 160000 lib/lwip/lwip-external
>
Please no submodules. They are such a pain. If we want the code in
U-Boot, let's put it in U-Boot and upstream our changes as needed.
Regards,
Simon
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
` (5 preceding siblings ...)
2023-05-05 10:27 ` [RFC PATCH 0/5] LWIP stack integration Ilias Apalodimas
@ 2023-05-08 21:23 ` Simon Glass
2023-05-11 13:28 ` Maxim Uvarov
2023-05-11 13:52 ` Tom Rini
7 siblings, 1 reply; 37+ messages in thread
From: Simon Glass @ 2023-05-08 21:23 UTC (permalink / raw)
To: Maxim Uvarov
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
Hi Maxim,
On Fri, 5 May 2023 at 04:50, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>
> Greetings,
>
> This RFC patchset is an attempt to try to use an already existing IP network stack inside U-boot.
> U-Boot recently got basic TCP/IP support, implementing wget, but in order to get a full IP stack
> with new features (e.g ipv6), it would be preferable to use an established embedded ip library,
> instead of rewriting the code from scratch.
>
> For this experiment LWIP network stack was selected:
> https://savannah.nongnu.org/git/?group=lwip
>
> LWIP main features include:
> - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
> - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf),
> SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> - APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
> - Extended features: IP forwarding over multiple network interfaces, TCP congestion control,
> RTT estimation and fast recovery/fast retransmit
> - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver,
> mDNS responder, MQTT client, TFTP server.
>
> This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
> TCP/IP stack designed for embedded systems for U-boot. That will allow using already
> written network applications for microcontrollers.
>
> lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
> Which should be compatible with u-boot.
>
> In the current RFC I tried to use minimal changes to better see how LWIP code can be embedded into
> U-boot. Patches implement ping and wget commands work. Both commands are currently copy pasting and
> reusing lwIP examples. Whether we want to add the final application in U-Boot or lwIP is up to
> discussion, but the current approach was the easiest one for an RFC.
>
> Looking for your comments,
> Best regards,
> Maxim.
>
> Maxim Uvarov (5):
> add lwip-external submodule
> lib/lwip: compile-in core files
> add doc/README.lwip
> add doc/README.lwip.size
> lwip: implement wget command from http_client.c example
>
> .gitignore | 5 +
> .gitmodules | 3 +
> doc/README.lwip | 90 +++++
> doc/README.lwip.size | 291 +++++++++++++++
> include/net.h | 2 +-
> lib/Kconfig | 2 +
> lib/Makefile | 2 +
> lib/lwip/Kconfig | 12 +
> lib/lwip/Makefile | 86 +++++
> lib/lwip/apps/http/lwip-wget.c | 67 ++++
> lib/lwip/apps/http/rmstatic.patch | 47 +++
> lib/lwip/apps/ping/lwip_ping.c | 33 ++
> lib/lwip/apps/ping/lwip_ping.h | 19 +
> lib/lwip/apps/ping/ping.h | 0
> lib/lwip/apps/ping/rmstatic.patch | 32 ++
> lib/lwip/cmd-lwip.c | 129 +++++++
> lib/lwip/lwip-external | 1 +
> lib/lwip/lwipopts.h | 484 +++++++++++++++++++++++++
> lib/lwip/port/if.c | 256 +++++++++++++
> lib/lwip/port/include/arch/cc.h | 41 +++
> lib/lwip/port/include/arch/sys_arch.h | 78 ++++
> lib/lwip/port/include/arch/u-sockets.h | 26 ++
> lib/lwip/port/include/limits.h | 0
> lib/lwip/port/sys-arch.c | 7 +
> net/eth-uclass.c | 4 +-
> net/net.c | 14 +
> 26 files changed, 1729 insertions(+), 2 deletions(-)
> create mode 100644 .gitmodules
> create mode 100644 doc/README.lwip
> create mode 100644 doc/README.lwip.size
> create mode 100644 lib/lwip/Kconfig
> create mode 100644 lib/lwip/Makefile
> create mode 100644 lib/lwip/apps/http/lwip-wget.c
> create mode 100644 lib/lwip/apps/http/rmstatic.patch
> create mode 100644 lib/lwip/apps/ping/lwip_ping.c
> create mode 100644 lib/lwip/apps/ping/lwip_ping.h
> create mode 100644 lib/lwip/apps/ping/ping.h
> create mode 100644 lib/lwip/apps/ping/rmstatic.patch
> create mode 100644 lib/lwip/cmd-lwip.c
> create mode 160000 lib/lwip/lwip-external
> create mode 100644 lib/lwip/lwipopts.h
> create mode 100644 lib/lwip/port/if.c
> create mode 100644 lib/lwip/port/include/arch/cc.h
> create mode 100644 lib/lwip/port/include/arch/sys_arch.h
> create mode 100644 lib/lwip/port/include/arch/u-sockets.h
> create mode 100644 lib/lwip/port/include/limits.h
> create mode 100644 lib/lwip/port/sys-arch.c
>
> --
> 2.30.2
>
I don't know much about lwip but I certainly think we should be open
to changing the network stack, if it is better.
Regards,
Simon
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 1/5] add lwip-external submodule
2023-05-08 14:43 ` Simon Glass
@ 2023-05-10 7:40 ` Ilias Apalodimas
2023-05-10 14:46 ` Peter Robinson
0 siblings, 1 reply; 37+ messages in thread
From: Ilias Apalodimas @ 2023-05-10 7:40 UTC (permalink / raw)
To: Simon Glass; +Cc: Maxim Uvarov, u-boot, pbrobinson, joe.hershberger, rfried.dev
Hi Simon,
On Mon, May 08, 2023 at 08:43:14AM -0600, Simon Glass wrote:
> Hi Maxim,
>
> On Fri, 5 May 2023 at 04:50, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> >
> > Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
> > ---
> > .gitmodules | 3 +++
> > lib/lwip/lwip-external | 1 +
> > 2 files changed, 4 insertions(+)
> > create mode 100644 .gitmodules
> > create mode 160000 lib/lwip/lwip-external
> >
>
> Please no submodules. They are such a pain. If we want the code in
> U-Boot, let's put it in U-Boot and upstream our changes as needed.
Can you explain a bit more the pain points you are seeing in u-boot with
submodules? EDK2 does submodules for openSSL and it's quite convenient,
since you dont have to maintain any code, do backports etc. Instead we can
just use upstream projects as-is.
IMHO we should work on having it as an experimental feature in parallel
with the current TCP efforts for a while and have a Kconfig switch. If we
are happy in the long run and the code increase isn't prohibitive, we can
consider switching permanently
Regards
/Ilias
>
> Regards,
> Simon
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 1/5] add lwip-external submodule
2023-05-10 7:40 ` Ilias Apalodimas
@ 2023-05-10 14:46 ` Peter Robinson
2023-05-11 13:51 ` Tom Rini
0 siblings, 1 reply; 37+ messages in thread
From: Peter Robinson @ 2023-05-10 14:46 UTC (permalink / raw)
To: Ilias Apalodimas
Cc: Simon Glass, Maxim Uvarov, u-boot, pbrobinson, joe.hershberger,
rfried.dev
On Wed, May 10, 2023 at 8:40 AM Ilias Apalodimas
<ilias.apalodimas@linaro.org> wrote:
>
> Hi Simon,
>
> On Mon, May 08, 2023 at 08:43:14AM -0600, Simon Glass wrote:
> > Hi Maxim,
> >
> > On Fri, 5 May 2023 at 04:50, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> > >
> > > Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
> > > ---
> > > .gitmodules | 3 +++
> > > lib/lwip/lwip-external | 1 +
> > > 2 files changed, 4 insertions(+)
> > > create mode 100644 .gitmodules
> > > create mode 160000 lib/lwip/lwip-external
> > >
> >
> > Please no submodules. They are such a pain. If we want the code in
> > U-Boot, let's put it in U-Boot and upstream our changes as needed.
>
> Can you explain a bit more the pain points you are seeing in u-boot with
> submodules? EDK2 does submodules for openSSL and it's quite convenient,
> since you dont have to maintain any code, do backports etc. Instead we can
> just use upstream projects as-is.
I feel there's pros and cons for both, similarly different projects
have different projects have different policies. Tom may have a more
definite opinion.
> IMHO we should work on having it as an experimental feature in parallel
> with the current TCP efforts for a while and have a Kconfig switch. If we
> are happy in the long run and the code increase isn't prohibitive, we can
> consider switching permanently
At least for initial review of the prototype I don't see it as
blocking for people to get a general idea what is going on.
Peter
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-08 21:23 ` Simon Glass
@ 2023-05-11 13:28 ` Maxim Uvarov
0 siblings, 0 replies; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-11 13:28 UTC (permalink / raw)
To: Simon Glass
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
On Mon, 8 May 2023 at 17:23, Simon Glass <sjg@chromium.org> wrote:
>
> Hi Maxim,
>
> On Fri, 5 May 2023 at 04:50, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> >
> > Greetings,
> >
> > This RFC patchset is an attempt to try to use an already existing IP network stack inside U-boot.
> > U-Boot recently got basic TCP/IP support, implementing wget, but in order to get a full IP stack
> > with new features (e.g ipv6), it would be preferable to use an established embedded ip library,
> > instead of rewriting the code from scratch.
> >
> > For this experiment LWIP network stack was selected:
> > https://savannah.nongnu.org/git/?group=lwip
> >
> > LWIP main features include:
> > - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
> > - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf),
> > SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> > - APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
> > - Extended features: IP forwarding over multiple network interfaces, TCP congestion control,
> > RTT estimation and fast recovery/fast retransmit
> > - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver,
> > mDNS responder, MQTT client, TFTP server.
> >
> > This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
> > TCP/IP stack designed for embedded systems for U-boot. That will allow using already
> > written network applications for microcontrollers.
> >
> > lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
> > Which should be compatible with u-boot.
> >
> > In the current RFC I tried to use minimal changes to better see how LWIP code can be embedded into
> > U-boot. Patches implement ping and wget commands work. Both commands are currently copy pasting and
> > reusing lwIP examples. Whether we want to add the final application in U-Boot or lwIP is up to
> > discussion, but the current approach was the easiest one for an RFC.
> >
> > Looking for your comments,
> > Best regards,
> > Maxim.
> >
> > Maxim Uvarov (5):
> > add lwip-external submodule
> > lib/lwip: compile-in core files
> > add doc/README.lwip
> > add doc/README.lwip.size
> > lwip: implement wget command from http_client.c example
> >
> > .gitignore | 5 +
> > .gitmodules | 3 +
> > doc/README.lwip | 90 +++++
> > doc/README.lwip.size | 291 +++++++++++++++
> > include/net.h | 2 +-
> > lib/Kconfig | 2 +
> > lib/Makefile | 2 +
> > lib/lwip/Kconfig | 12 +
> > lib/lwip/Makefile | 86 +++++
> > lib/lwip/apps/http/lwip-wget.c | 67 ++++
> > lib/lwip/apps/http/rmstatic.patch | 47 +++
> > lib/lwip/apps/ping/lwip_ping.c | 33 ++
> > lib/lwip/apps/ping/lwip_ping.h | 19 +
> > lib/lwip/apps/ping/ping.h | 0
> > lib/lwip/apps/ping/rmstatic.patch | 32 ++
> > lib/lwip/cmd-lwip.c | 129 +++++++
> > lib/lwip/lwip-external | 1 +
> > lib/lwip/lwipopts.h | 484 +++++++++++++++++++++++++
> > lib/lwip/port/if.c | 256 +++++++++++++
> > lib/lwip/port/include/arch/cc.h | 41 +++
> > lib/lwip/port/include/arch/sys_arch.h | 78 ++++
> > lib/lwip/port/include/arch/u-sockets.h | 26 ++
> > lib/lwip/port/include/limits.h | 0
> > lib/lwip/port/sys-arch.c | 7 +
> > net/eth-uclass.c | 4 +-
> > net/net.c | 14 +
> > 26 files changed, 1729 insertions(+), 2 deletions(-)
> > create mode 100644 .gitmodules
> > create mode 100644 doc/README.lwip
> > create mode 100644 doc/README.lwip.size
> > create mode 100644 lib/lwip/Kconfig
> > create mode 100644 lib/lwip/Makefile
> > create mode 100644 lib/lwip/apps/http/lwip-wget.c
> > create mode 100644 lib/lwip/apps/http/rmstatic.patch
> > create mode 100644 lib/lwip/apps/ping/lwip_ping.c
> > create mode 100644 lib/lwip/apps/ping/lwip_ping.h
> > create mode 100644 lib/lwip/apps/ping/ping.h
> > create mode 100644 lib/lwip/apps/ping/rmstatic.patch
> > create mode 100644 lib/lwip/cmd-lwip.c
> > create mode 160000 lib/lwip/lwip-external
> > create mode 100644 lib/lwip/lwipopts.h
> > create mode 100644 lib/lwip/port/if.c
> > create mode 100644 lib/lwip/port/include/arch/cc.h
> > create mode 100644 lib/lwip/port/include/arch/sys_arch.h
> > create mode 100644 lib/lwip/port/include/arch/u-sockets.h
> > create mode 100644 lib/lwip/port/include/limits.h
> > create mode 100644 lib/lwip/port/sys-arch.c
> >
> > --
> > 2.30.2
> >
>
> I don't know much about lwip but I certainly think we should be open
> to changing the network stack, if it is better.
>
> Regards,
> Simon
lwip is commonly used for non linux micro controllers which should
have a wide community and a bunch of examples for applications. So
it's closer to resources which we have inside u-boot. I don't strictly
vote for lwip, it can be any IP stack which already exists. But not
implement an IP stack inside u-boot. lwip looks reasonable. So if the
RFC idea is accepted I can work on real patches.
Best regards,
Maxim.
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 1/5] add lwip-external submodule
2023-05-10 14:46 ` Peter Robinson
@ 2023-05-11 13:51 ` Tom Rini
0 siblings, 0 replies; 37+ messages in thread
From: Tom Rini @ 2023-05-11 13:51 UTC (permalink / raw)
To: Peter Robinson
Cc: Ilias Apalodimas, Simon Glass, Maxim Uvarov, u-boot, pbrobinson,
joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 1838 bytes --]
On Wed, May 10, 2023 at 03:46:31PM +0100, Peter Robinson wrote:
> On Wed, May 10, 2023 at 8:40 AM Ilias Apalodimas
> <ilias.apalodimas@linaro.org> wrote:
> >
> > Hi Simon,
> >
> > On Mon, May 08, 2023 at 08:43:14AM -0600, Simon Glass wrote:
> > > Hi Maxim,
> > >
> > > On Fri, 5 May 2023 at 04:50, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> > > >
> > > > Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
> > > > ---
> > > > .gitmodules | 3 +++
> > > > lib/lwip/lwip-external | 1 +
> > > > 2 files changed, 4 insertions(+)
> > > > create mode 100644 .gitmodules
> > > > create mode 160000 lib/lwip/lwip-external
> > > >
> > >
> > > Please no submodules. They are such a pain. If we want the code in
> > > U-Boot, let's put it in U-Boot and upstream our changes as needed.
> >
> > Can you explain a bit more the pain points you are seeing in u-boot with
> > submodules? EDK2 does submodules for openSSL and it's quite convenient,
> > since you dont have to maintain any code, do backports etc. Instead we can
> > just use upstream projects as-is.
>
> I feel there's pros and cons for both, similarly different projects
> have different projects have different policies. Tom may have a more
> definite opinion.
>
> > IMHO we should work on having it as an experimental feature in parallel
> > with the current TCP efforts for a while and have a Kconfig switch. If we
> > are happy in the long run and the code increase isn't prohibitive, we can
> > consider switching permanently
>
> At least for initial review of the prototype I don't see it as
> blocking for people to get a general idea what is going on.
Yeah, I also don't like submodules. But for the purpose of this RFC, at
this stage in the RFC, it's fine. May or may not be easier to review.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 1/5] add lwip-external submodule
2023-05-05 10:25 ` [RFC PATCH 1/5] add lwip-external submodule Maxim Uvarov
2023-05-08 14:43 ` Simon Glass
@ 2023-05-11 13:51 ` Tom Rini
1 sibling, 0 replies; 37+ messages in thread
From: Tom Rini @ 2023-05-11 13:51 UTC (permalink / raw)
To: Maxim Uvarov
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 1125 bytes --]
On Fri, May 05, 2023 at 10:25:25AM +0000, Maxim Uvarov wrote:
> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
> ---
> .gitmodules | 3 +++
> lib/lwip/lwip-external | 1 +
> 2 files changed, 4 insertions(+)
> create mode 100644 .gitmodules
> create mode 160000 lib/lwip/lwip-external
>
> diff --git a/.gitmodules b/.gitmodules
> new file mode 100644
> index 0000000000..afc709af10
> --- /dev/null
> +++ b/.gitmodules
> @@ -0,0 +1,3 @@
> +[submodule "lib/lwip/lwip-external"]
> + path = lib/lwip/lwip-external
> + url = https://git.savannah.nongnu.org/git/lwip.git
> diff --git a/lib/lwip/lwip-external b/lib/lwip/lwip-external
> new file mode 160000
> index 0000000000..3fe8d2fc43
> --- /dev/null
> +++ b/lib/lwip/lwip-external
> @@ -0,0 +1 @@
> +Subproject commit 3fe8d2fc43a9b69f7ed28c63d44a7744f9c0def9
Why are we using top of tree here, rather than say STABLE-2_1_3_RELEASE
tag? Is there some feature we need post that release? I'd be much more
inclined to track releases than top of trees for a project which does
have (from a quick look) release tags.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 3/5] add doc/README.lwip
2023-05-05 10:25 ` [RFC PATCH 3/5] add doc/README.lwip Maxim Uvarov
@ 2023-05-11 13:51 ` Tom Rini
0 siblings, 0 replies; 37+ messages in thread
From: Tom Rini @ 2023-05-11 13:51 UTC (permalink / raw)
To: Maxim Uvarov
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 317 bytes --]
On Fri, May 05, 2023 at 10:25:27AM +0000, Maxim Uvarov wrote:
> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
> ---
> doc/README.lwip | 56 +++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 56 insertions(+)
> create mode 100644 doc/README.lwip
This needs to be rST.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 4/5] add doc/README.lwip.size
2023-05-05 10:25 ` [RFC PATCH 4/5] add doc/README.lwip.size Maxim Uvarov
@ 2023-05-11 13:51 ` Tom Rini
0 siblings, 0 replies; 37+ messages in thread
From: Tom Rini @ 2023-05-11 13:51 UTC (permalink / raw)
To: Maxim Uvarov
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 508 bytes --]
On Fri, May 05, 2023 at 10:25:28AM +0000, Maxim Uvarov wrote:
> Add doc with size calculation accoding to original u-boot.
>
> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
> ---
> doc/README.lwip.size | 291 +++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 291 insertions(+)
> create mode 100644 doc/README.lwip.size
This too needs to be rST, and should just be another part of the first
doc. It should probably really instead be part of the cover letter.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
` (6 preceding siblings ...)
2023-05-08 21:23 ` Simon Glass
@ 2023-05-11 13:52 ` Tom Rini
2023-05-15 15:25 ` Maxim Uvarov
2023-05-19 13:17 ` Ilias Apalodimas
7 siblings, 2 replies; 37+ messages in thread
From: Tom Rini @ 2023-05-11 13:52 UTC (permalink / raw)
To: Maxim Uvarov
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 2505 bytes --]
On Fri, May 05, 2023 at 10:25:24AM +0000, Maxim Uvarov wrote:
> Greetings,
>
> This RFC patchset is an attempt to try to use an already existing IP network stack inside U-boot.
> U-Boot recently got basic TCP/IP support, implementing wget, but in order to get a full IP stack
> with new features (e.g ipv6), it would be preferable to use an established embedded ip library,
> instead of rewriting the code from scratch.
>
> For this experiment LWIP network stack was selected:
> https://savannah.nongnu.org/git/?group=lwip
>
> LWIP main features include:
> - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
> - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf),
> SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> - APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
> - Extended features: IP forwarding over multiple network interfaces, TCP congestion control,
> RTT estimation and fast recovery/fast retransmit
> - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver,
> mDNS responder, MQTT client, TFTP server.
>
> This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
> TCP/IP stack designed for embedded systems for U-boot. That will allow using already
> written network applications for microcontrollers.
>
> lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
> Which should be compatible with u-boot.
>
> In the current RFC I tried to use minimal changes to better see how LWIP code can be embedded into
> U-boot. Patches implement ping and wget commands work. Both commands are currently copy pasting and
> reusing lwIP examples. Whether we want to add the final application in U-Boot or lwIP is up to
> discussion, but the current approach was the easiest one for an RFC.
I'm honestly not sure this is the most useful way of doing an RFC. The
long term goal would be that we replace our existing net/ with lwIP,
yes? So what I'd see as more valuable is what it looks like to limit
yourself to either sandbox or some QEMU target, disable the current
network stack, and instead use lwIP to support just cmd/net.c so that
the scope of the conversion is visible. Then the size comparison you do
should be between platform + net + cmd/net.c (and the rest of networking
turned off) and platform + lwip + cmd/net.c converted.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-11 13:52 ` Tom Rini
@ 2023-05-15 15:25 ` Maxim Uvarov
2023-05-15 15:39 ` Tom Rini
2023-05-19 13:17 ` Ilias Apalodimas
1 sibling, 1 reply; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-15 15:25 UTC (permalink / raw)
To: Tom Rini
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
On Thu, 11 May 2023 at 09:52, Tom Rini <trini@konsulko.com> wrote:
> On Fri, May 05, 2023 at 10:25:24AM +0000, Maxim Uvarov wrote:
>
> > Greetings,
> >
> > This RFC patchset is an attempt to try to use an already existing IP
> network stack inside U-boot.
> > U-Boot recently got basic TCP/IP support, implementing wget, but in
> order to get a full IP stack
> > with new features (e.g ipv6), it would be preferable to use an
> established embedded ip library,
> > instead of rewriting the code from scratch.
> >
> > For this experiment LWIP network stack was selected:
> > https://savannah.nongnu.org/git/?group=lwip
> >
> > LWIP main features include:
> > - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
> > - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA
> (Zeroconf),
> > SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> > - APIs: specialized APIs for enhanced performance, optional
> Berkeley-alike socket API
> > - Extended features: IP forwarding over multiple network interfaces, TCP
> congestion control,
> > RTT estimation and fast recovery/fast retransmit
> > - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping,
> NetBIOS nameserver,
> > mDNS responder, MQTT client, TFTP server.
> >
> > This RFC work is a demo to enable lwIP (lightweight IP) which is a
> widely used open-source
> > TCP/IP stack designed for embedded systems for U-boot. That will allow
> using already
> > written network applications for microcontrollers.
> >
> > lwIP is licensed under a BSD-style license:
> http://lwip.wikia.com/wiki/License.
> > Which should be compatible with u-boot.
> >
> > In the current RFC I tried to use minimal changes to better see how LWIP
> code can be embedded into
> > U-boot. Patches implement ping and wget commands work. Both commands are
> currently copy pasting and
> > reusing lwIP examples. Whether we want to add the final application in
> U-Boot or lwIP is up to
> > discussion, but the current approach was the easiest one for an RFC.
>
> I'm honestly not sure this is the most useful way of doing an RFC. The
> long term goal would be that we replace our existing net/ with lwIP,
> yes? So what I'd see as more valuable is what it looks like to limit
> yourself to either sandbox or some QEMU target, disable the current
> network stack, and instead use lwIP to support just cmd/net.c so that
> the scope of the conversion is visible. Then the size comparison you do
> should be between platform + net + cmd/net.c (and the rest of networking
> turned off) and platform + lwip + cmd/net.c converted.
>
> --
> Tom
>
Is there any acceptance criteria for size? If we say that additing lwip
will add about 48kb and removing current code will also release some kbs.
How size is critical here or it's just numbers good to know?
BR,
Maxim.
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-15 15:25 ` Maxim Uvarov
@ 2023-05-15 15:39 ` Tom Rini
0 siblings, 0 replies; 37+ messages in thread
From: Tom Rini @ 2023-05-15 15:39 UTC (permalink / raw)
To: Maxim Uvarov
Cc: u-boot, pbrobinson, ilias.apalodimas, joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 3787 bytes --]
On Mon, May 15, 2023 at 11:25:58AM -0400, Maxim Uvarov wrote:
> On Thu, 11 May 2023 at 09:52, Tom Rini <trini@konsulko.com> wrote:
>
> > On Fri, May 05, 2023 at 10:25:24AM +0000, Maxim Uvarov wrote:
> >
> > > Greetings,
> > >
> > > This RFC patchset is an attempt to try to use an already existing IP
> > network stack inside U-boot.
> > > U-Boot recently got basic TCP/IP support, implementing wget, but in
> > order to get a full IP stack
> > > with new features (e.g ipv6), it would be preferable to use an
> > established embedded ip library,
> > > instead of rewriting the code from scratch.
> > >
> > > For this experiment LWIP network stack was selected:
> > > https://savannah.nongnu.org/git/?group=lwip
> > >
> > > LWIP main features include:
> > > - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
> > > - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA
> > (Zeroconf),
> > > SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> > > - APIs: specialized APIs for enhanced performance, optional
> > Berkeley-alike socket API
> > > - Extended features: IP forwarding over multiple network interfaces, TCP
> > congestion control,
> > > RTT estimation and fast recovery/fast retransmit
> > > - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping,
> > NetBIOS nameserver,
> > > mDNS responder, MQTT client, TFTP server.
> > >
> > > This RFC work is a demo to enable lwIP (lightweight IP) which is a
> > widely used open-source
> > > TCP/IP stack designed for embedded systems for U-boot. That will allow
> > using already
> > > written network applications for microcontrollers.
> > >
> > > lwIP is licensed under a BSD-style license:
> > http://lwip.wikia.com/wiki/License.
> > > Which should be compatible with u-boot.
> > >
> > > In the current RFC I tried to use minimal changes to better see how LWIP
> > code can be embedded into
> > > U-boot. Patches implement ping and wget commands work. Both commands are
> > currently copy pasting and
> > > reusing lwIP examples. Whether we want to add the final application in
> > U-Boot or lwIP is up to
> > > discussion, but the current approach was the easiest one for an RFC.
> >
> > I'm honestly not sure this is the most useful way of doing an RFC. The
> > long term goal would be that we replace our existing net/ with lwIP,
> > yes? So what I'd see as more valuable is what it looks like to limit
> > yourself to either sandbox or some QEMU target, disable the current
> > network stack, and instead use lwIP to support just cmd/net.c so that
> > the scope of the conversion is visible. Then the size comparison you do
> > should be between platform + net + cmd/net.c (and the rest of networking
> > turned off) and platform + lwip + cmd/net.c converted.
> >
> > --
> > Tom
> >
>
> Is there any acceptance criteria for size? If we say that additing lwip
> will add about 48kb and removing current code will also release some kbs.
> How size is critical here or it's just numbers good to know?
Well, the text portion of a current sandbox build (with LTO off, so
adding up sizes is easier to do quickly) net/ is 46kB. But that
includes v6 and fastboot and so on.
So, it's a matter of discussion. If replacing the network stack but
maintaining the same level of functionality causes us to grow by single
digit kilobytes, we can maybe justify it due to easier to maintain. If
it's smaller, that's great and an argument in favor of it. But if we're
growing everything by tens of kilobytes, that's a lot harder to justify
but maybe shows we need to work with upstream as perhaps some things
need to be more configurable, or otherwise something to investigate.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-11 13:52 ` Tom Rini
2023-05-15 15:25 ` Maxim Uvarov
@ 2023-05-19 13:17 ` Ilias Apalodimas
2023-05-19 13:52 ` Tom Rini
1 sibling, 1 reply; 37+ messages in thread
From: Ilias Apalodimas @ 2023-05-19 13:17 UTC (permalink / raw)
To: Tom Rini; +Cc: Maxim Uvarov, u-boot, pbrobinson, joe.hershberger, rfried.dev
Hi Tom,
Apologies for being late to the party
> On Thu, May 11, 2023 at 09:52:04AM -0400, Tom Rini wrote:
> On Fri, May 05, 2023 at 10:25:24AM +0000, Maxim Uvarov wrote:
>
> > Greetings,
> >
> > This RFC patchset is an attempt to try to use an already existing IP network stack inside U-boot.
> > U-Boot recently got basic TCP/IP support, implementing wget, but in order to get a full IP stack
> > with new features (e.g ipv6), it would be preferable to use an established embedded ip library,
> > instead of rewriting the code from scratch.
> >
> > For this experiment LWIP network stack was selected:
> > https://savannah.nongnu.org/git/?group=lwip
> >
> > LWIP main features include:
> > - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
> > - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf),
> > SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> > - APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
> > - Extended features: IP forwarding over multiple network interfaces, TCP congestion control,
> > RTT estimation and fast recovery/fast retransmit
> > - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver,
> > mDNS responder, MQTT client, TFTP server.
> >
> > This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
> > TCP/IP stack designed for embedded systems for U-boot. That will allow using already
> > written network applications for microcontrollers.
> >
> > lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
> > Which should be compatible with u-boot.
> >
> > In the current RFC I tried to use minimal changes to better see how LWIP code can be embedded into
> > U-boot. Patches implement ping and wget commands work. Both commands are currently copy pasting and
> > reusing lwIP examples. Whether we want to add the final application in U-Boot or lwIP is up to
> > discussion, but the current approach was the easiest one for an RFC.
>
> I'm honestly not sure this is the most useful way of doing an RFC. The
> long term goal would be that we replace our existing net/ with lwIP,
> yes? So what I'd see as more valuable is what it looks like to limit
> yourself to either sandbox or some QEMU target, disable the current
> network stack, and instead use lwIP to support just cmd/net.c so that
> the scope of the conversion is visible. Then the size comparison you do
> should be between platform + net + cmd/net.c (and the rest of networking
> turned off) and platform + lwip + cmd/net.c converted.
This might be a bit too much imho. How about replacing the TCP stack which
is new an under heavy devel as well. If we do that we could replace lwip
with the version Maxim proposes and check the difference between
U-boot + homegrown TCP + wget
U-Boot + LWIP (for tcp only) + new wget
That would give us an idea before trying to replace the UDP portion which
is way bigger
Regards
/Ilias
>
> --
> Tom
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-19 13:17 ` Ilias Apalodimas
@ 2023-05-19 13:52 ` Tom Rini
2023-05-22 9:01 ` Maxim Uvarov
0 siblings, 1 reply; 37+ messages in thread
From: Tom Rini @ 2023-05-19 13:52 UTC (permalink / raw)
To: Ilias Apalodimas
Cc: Maxim Uvarov, u-boot, pbrobinson, joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 3409 bytes --]
On Fri, May 19, 2023 at 04:17:06PM +0300, Ilias Apalodimas wrote:
> Hi Tom,
>
> Apologies for being late to the party
>
> > On Thu, May 11, 2023 at 09:52:04AM -0400, Tom Rini wrote:
> > On Fri, May 05, 2023 at 10:25:24AM +0000, Maxim Uvarov wrote:
> >
> > > Greetings,
> > >
> > > This RFC patchset is an attempt to try to use an already existing IP network stack inside U-boot.
> > > U-Boot recently got basic TCP/IP support, implementing wget, but in order to get a full IP stack
> > > with new features (e.g ipv6), it would be preferable to use an established embedded ip library,
> > > instead of rewriting the code from scratch.
> > >
> > > For this experiment LWIP network stack was selected:
> > > https://savannah.nongnu.org/git/?group=lwip
> > >
> > > LWIP main features include:
> > > - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
> > > - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf),
> > > SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> > > - APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
> > > - Extended features: IP forwarding over multiple network interfaces, TCP congestion control,
> > > RTT estimation and fast recovery/fast retransmit
> > > - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver,
> > > mDNS responder, MQTT client, TFTP server.
> > >
> > > This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
> > > TCP/IP stack designed for embedded systems for U-boot. That will allow using already
> > > written network applications for microcontrollers.
> > >
> > > lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
> > > Which should be compatible with u-boot.
> > >
> > > In the current RFC I tried to use minimal changes to better see how LWIP code can be embedded into
> > > U-boot. Patches implement ping and wget commands work. Both commands are currently copy pasting and
> > > reusing lwIP examples. Whether we want to add the final application in U-Boot or lwIP is up to
> > > discussion, but the current approach was the easiest one for an RFC.
> >
> > I'm honestly not sure this is the most useful way of doing an RFC. The
> > long term goal would be that we replace our existing net/ with lwIP,
> > yes? So what I'd see as more valuable is what it looks like to limit
> > yourself to either sandbox or some QEMU target, disable the current
> > network stack, and instead use lwIP to support just cmd/net.c so that
> > the scope of the conversion is visible. Then the size comparison you do
> > should be between platform + net + cmd/net.c (and the rest of networking
> > turned off) and platform + lwip + cmd/net.c converted.
>
> This might be a bit too much imho. How about replacing the TCP stack which
> is new an under heavy devel as well. If we do that we could replace lwip
> with the version Maxim proposes and check the difference between
> U-boot + homegrown TCP + wget
> U-Boot + LWIP (for tcp only) + new wget
>
> That would give us an idea before trying to replace the UDP portion which
> is way bigger
I guess we can try that as a starting point and see what things look
like. My gut feeling however is that's not going to look like a win.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-19 13:52 ` Tom Rini
@ 2023-05-22 9:01 ` Maxim Uvarov
2023-05-22 13:33 ` Ilias Apalodimas
0 siblings, 1 reply; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-22 9:01 UTC (permalink / raw)
To: Tom Rini
Cc: Ilias Apalodimas, u-boot, pbrobinson, joe.hershberger, rfried.dev
My measurements for binary after LTO looks like:
U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
870728 | 915000 | 912560 | 41832
| 4.8
BR,
Maxim.
On Fri, 19 May 2023 at 09:52, Tom Rini <trini@konsulko.com> wrote:
> On Fri, May 19, 2023 at 04:17:06PM +0300, Ilias Apalodimas wrote:
> > Hi Tom,
> >
> > Apologies for being late to the party
> >
> > > On Thu, May 11, 2023 at 09:52:04AM -0400, Tom Rini wrote:
> > > On Fri, May 05, 2023 at 10:25:24AM +0000, Maxim Uvarov wrote:
> > >
> > > > Greetings,
> > > >
> > > > This RFC patchset is an attempt to try to use an already existing IP
> network stack inside U-boot.
> > > > U-Boot recently got basic TCP/IP support, implementing wget, but in
> order to get a full IP stack
> > > > with new features (e.g ipv6), it would be preferable to use an
> established embedded ip library,
> > > > instead of rewriting the code from scratch.
> > > >
> > > > For this experiment LWIP network stack was selected:
> > > > https://savannah.nongnu.org/git/?group=lwip
> > > >
> > > > LWIP main features include:
> > > > - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS,
> PPPoE
> > > > - DHCP client, DNS client (incl. mDNS hostname resolver),
> AutoIP/APIPA (Zeroconf),
> > > > SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
> > > > - APIs: specialized APIs for enhanced performance, optional
> Berkeley-alike socket API
> > > > - Extended features: IP forwarding over multiple network interfaces,
> TCP congestion control,
> > > > RTT estimation and fast recovery/fast retransmit
> > > > - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client,
> ping, NetBIOS nameserver,
> > > > mDNS responder, MQTT client, TFTP server.
> > > >
> > > > This RFC work is a demo to enable lwIP (lightweight IP) which is a
> widely used open-source
> > > > TCP/IP stack designed for embedded systems for U-boot. That will
> allow using already
> > > > written network applications for microcontrollers.
> > > >
> > > > lwIP is licensed under a BSD-style license:
> http://lwip.wikia.com/wiki/License.
> > > > Which should be compatible with u-boot.
> > > >
> > > > In the current RFC I tried to use minimal changes to better see how
> LWIP code can be embedded into
> > > > U-boot. Patches implement ping and wget commands work. Both commands
> are currently copy pasting and
> > > > reusing lwIP examples. Whether we want to add the final application
> in U-Boot or lwIP is up to
> > > > discussion, but the current approach was the easiest one for an RFC.
> > >
> > > I'm honestly not sure this is the most useful way of doing an RFC. The
> > > long term goal would be that we replace our existing net/ with lwIP,
> > > yes? So what I'd see as more valuable is what it looks like to limit
> > > yourself to either sandbox or some QEMU target, disable the current
> > > network stack, and instead use lwIP to support just cmd/net.c so that
> > > the scope of the conversion is visible. Then the size comparison you
> do
> > > should be between platform + net + cmd/net.c (and the rest of
> networking
> > > turned off) and platform + lwip + cmd/net.c converted.
> >
> > This might be a bit too much imho. How about replacing the TCP stack
> which
> > is new an under heavy devel as well. If we do that we could replace lwip
> > with the version Maxim proposes and check the difference between
> > U-boot + homegrown TCP + wget
> > U-Boot + LWIP (for tcp only) + new wget
> >
> > That would give us an idea before trying to replace the UDP portion which
> > is way bigger
>
> I guess we can try that as a starting point and see what things look
> like. My gut feeling however is that's not going to look like a win.
>
> --
> Tom
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-22 9:01 ` Maxim Uvarov
@ 2023-05-22 13:33 ` Ilias Apalodimas
2023-05-22 14:20 ` Tom Rini
0 siblings, 1 reply; 37+ messages in thread
From: Ilias Apalodimas @ 2023-05-22 13:33 UTC (permalink / raw)
To: Maxim Uvarov; +Cc: Tom Rini, u-boot, pbrobinson, joe.hershberger, rfried.dev
Hi Maxim
On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>
> My measurements for binary after LTO looks like:
>
> U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> 870728 | 915000 | 912560 | 41832 | 4.8
I think you'll need to analyze that a bit more. First of all I don't
think the '+ping' tab is useful. What is is trying to achieve?
- How was LWIP compiled?
- Was ipv6 supported?
- Can we strip it down even further?
In general please give as much information as you can with what we
gain in functionality from LWIP with those extra bytes of code.
Thanks
/Ilias
> BR,
> Maxim.
>
> On Fri, 19 May 2023 at 09:52, Tom Rini <trini@konsulko.com> wrote:
>>
>> On Fri, May 19, 2023 at 04:17:06PM +0300, Ilias Apalodimas wrote:
>> > Hi Tom,
>> >
>> > Apologies for being late to the party
>> >
>> > > On Thu, May 11, 2023 at 09:52:04AM -0400, Tom Rini wrote:
>> > > On Fri, May 05, 2023 at 10:25:24AM +0000, Maxim Uvarov wrote:
>> > >
>> > > > Greetings,
>> > > >
>> > > > This RFC patchset is an attempt to try to use an already existing IP network stack inside U-boot.
>> > > > U-Boot recently got basic TCP/IP support, implementing wget, but in order to get a full IP stack
>> > > > with new features (e.g ipv6), it would be preferable to use an established embedded ip library,
>> > > > instead of rewriting the code from scratch.
>> > > >
>> > > > For this experiment LWIP network stack was selected:
>> > > > https://savannah.nongnu.org/git/?group=lwip
>> > > >
>> > > > LWIP main features include:
>> > > > - Protocols: IP, IPv6, ICMP, ND, MLD, UDP, TCP, IGMP, ARP, PPPoS, PPPoE
>> > > > - DHCP client, DNS client (incl. mDNS hostname resolver), AutoIP/APIPA (Zeroconf),
>> > > > SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
>> > > > - APIs: specialized APIs for enhanced performance, optional Berkeley-alike socket API
>> > > > - Extended features: IP forwarding over multiple network interfaces, TCP congestion control,
>> > > > RTT estimation and fast recovery/fast retransmit
>> > > > - Addon applications: HTTP(S) server, SNTP client, SMTP(S) client, ping, NetBIOS nameserver,
>> > > > mDNS responder, MQTT client, TFTP server.
>> > > >
>> > > > This RFC work is a demo to enable lwIP (lightweight IP) which is a widely used open-source
>> > > > TCP/IP stack designed for embedded systems for U-boot. That will allow using already
>> > > > written network applications for microcontrollers.
>> > > >
>> > > > lwIP is licensed under a BSD-style license: http://lwip.wikia.com/wiki/License.
>> > > > Which should be compatible with u-boot.
>> > > >
>> > > > In the current RFC I tried to use minimal changes to better see how LWIP code can be embedded into
>> > > > U-boot. Patches implement ping and wget commands work. Both commands are currently copy pasting and
>> > > > reusing lwIP examples. Whether we want to add the final application in U-Boot or lwIP is up to
>> > > > discussion, but the current approach was the easiest one for an RFC.
>> > >
>> > > I'm honestly not sure this is the most useful way of doing an RFC. The
>> > > long term goal would be that we replace our existing net/ with lwIP,
>> > > yes? So what I'd see as more valuable is what it looks like to limit
>> > > yourself to either sandbox or some QEMU target, disable the current
>> > > network stack, and instead use lwIP to support just cmd/net.c so that
>> > > the scope of the conversion is visible. Then the size comparison you do
>> > > should be between platform + net + cmd/net.c (and the rest of networking
>> > > turned off) and platform + lwip + cmd/net.c converted.
>> >
>> > This might be a bit too much imho. How about replacing the TCP stack which
>> > is new an under heavy devel as well. If we do that we could replace lwip
>> > with the version Maxim proposes and check the difference between
>> > U-boot + homegrown TCP + wget
>> > U-Boot + LWIP (for tcp only) + new wget
>> >
>> > That would give us an idea before trying to replace the UDP portion which
>> > is way bigger
>>
>> I guess we can try that as a starting point and see what things look
>> like. My gut feeling however is that's not going to look like a win.
>>
>> --
>> Tom
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-22 13:33 ` Ilias Apalodimas
@ 2023-05-22 14:20 ` Tom Rini
2023-05-22 16:40 ` Maxim Uvarov
0 siblings, 1 reply; 37+ messages in thread
From: Tom Rini @ 2023-05-22 14:20 UTC (permalink / raw)
To: Ilias Apalodimas
Cc: Maxim Uvarov, u-boot, pbrobinson, joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 886 bytes --]
On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> Hi Maxim
>
> On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> >
> > My measurements for binary after LTO looks like:
> >
> > U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> > 870728 | 915000 | 912560 | 41832 | 4.8
>
>
> I think you'll need to analyze that a bit more. First of all I don't
> think the '+ping' tab is useful. What is is trying to achieve?
> - How was LWIP compiled?
> - Was ipv6 supported?
> - Can we strip it down even further?
>
> In general please give as much information as you can with what we
> gain in functionality from LWIP with those extra bytes of code.
And please make sure to disable the previous support, my guess fro that
much growth is that you didn't.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-22 14:20 ` Tom Rini
@ 2023-05-22 16:40 ` Maxim Uvarov
2023-05-22 21:23 ` Tom Rini
0 siblings, 1 reply; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-22 16:40 UTC (permalink / raw)
To: Tom Rini
Cc: Ilias Apalodimas, u-boot, pbrobinson, joe.hershberger, rfried.dev
On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
> On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> > Hi Maxim
> >
> > On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
> wrote:
> > >
> > > My measurements for binary after LTO looks like:
> > >
> > > U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> > > 870728 | 915000 | 912560 |
> 41832 | 4.8
> >
> >
> > I think you'll need to analyze that a bit more. First of all I don't
> > think the '+ping' tab is useful. What is is trying to achieve?
>
To show the difference of extra bytes if we add a ping app on top.
> > - How was LWIP compiled?
>
It has a really huge configuration. I tried to turn off everything off
everything what can impact on size but still make http app work:
#define LWIP_HAVE_LOOPIF 0
#define LWIP_NETCONN 0
#define LWIP_SOCKET 0
#define SO_REUSE 0
#define LWIP_STATS 0
#define PPP_SUPPORT 0
Disabling loopback:
#define LWIP_NETIF_LOOPBACK 0
can lower to 912288 bytes.
And it's the same compilation option (optimization for size) as the main
u-boot. I will do more experiments, but I think the goal is not to turn off
everything.
> > - Was ipv6 supported?
>
No. I.e. when I sent results it was enabled on the compilation stage but
not used. I just checked that size remains the same if IPv6 is not even
compiled.
> > - Can we strip it down even further?
> >
>
There is always room for optimization. I think I tried to turn off
everything that is configurable with defines. I can play with disable IP
reassembly and things like that or figure out which functions have more
size and if it's possible to exclude them.
> > In general please give as much information as you can with what we
> > gain in functionality from LWIP with those extra bytes of code.
>
>
The main idea is to reuse a maintainable IP stack outside of U-boot. LWIP
can give a nice separation between IP stack code and network application
code. I.e. application should not take care about any TCP details (SYN,
ACK, retransmission, reassembly etc) and should open connection and use
functions similar to recv() and send() to transfer data. Data means
application data, no network packets. And LWIP allows
us to do that.
Because LWIP has an API similar to sockets, it has to be very easy to port
a linux application to LWIP. Then you can test it with a tap device. Then
copy sources to U-boot, add a small integration layer (cmd command to
call), compile and use.
So my suggestion was:
- do not maintain new network stack code in the current U-boot. Use lwip
sources as an external project. All bugs related to network stack go to
lwip project first, then sync with U-boot.
- maintain network apps code* or
-- inside U-boot. Write our own code for application and maintain it
inside U-boot.
-- inside LWIP. Add examples to LWIP which are suitable for both U-boot
and LWIP.
* Let's define a U-boot network application as a cmd command. It might be
ping, wget (http or https download), telnet, arp dns etc..
Let's consider the real use case, like HTTPS download client. We need to
enable TLS connection, validate certificates, then do http download.
Looking at the current code of wget command it's quite difficult to
implement this due to the application having some protol level things. On
the other side we can find embedTLS examples to do https download on
sockets. If LWIP socket API is ported then the only thing you need to do is
change socket() -> lwip_socket(), recv()->lwip_recv(), send()->lwip_send()
and etc, even function names are similar. If LWIP socket API is not
supported, then use callback API for recv() and send(), which are also
easy.
So yes we add extra bytes, but that will allow us to write more complex
apps, use standard debug tools, use applications with very minimal
integration changes, use help from the LWIP community to fix protocol bugs,
etc..
Bunch of things already implemented there:
- ipv6
- dhcp
- snmp
- igmp
- dns
- tcp and udp and raw.
- loopback
- netconn
- socket
- stats
- ppp
(I just followed configurable defines).
And please make sure to disable the previous support, my guess fro that
> much growth is that you didn't.
>
# CONFIG_PROT_TCP is not set
# CONFIG_PROT_UDP is not set
# CONFIG_UDP_CHECKSUM is not set
# CONFIG_UDP_FUNCTION_FASTBOOT is not set
# CONFIG_CMD_WGET is not set
BR,
Maxim.
>
> --
> Tom
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-22 16:40 ` Maxim Uvarov
@ 2023-05-22 21:23 ` Tom Rini
2023-05-24 14:05 ` Maxim Uvarov
0 siblings, 1 reply; 37+ messages in thread
From: Tom Rini @ 2023-05-22 21:23 UTC (permalink / raw)
To: Maxim Uvarov
Cc: Ilias Apalodimas, u-boot, pbrobinson, joe.hershberger, rfried.dev
[-- Attachment #1: Type: text/plain, Size: 5490 bytes --]
On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
> On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
>
> > On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> > > Hi Maxim
> > >
> > > On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
> > wrote:
> > > >
> > > > My measurements for binary after LTO looks like:
> > > >
> > > > U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> > > > 870728 | 915000 | 912560 |
> > 41832 | 4.8
> > >
> > >
> > > I think you'll need to analyze that a bit more. First of all I don't
> > > think the '+ping' tab is useful. What is is trying to achieve?
> >
>
> To show the difference of extra bytes if we add a ping app on top.
>
>
> > > - How was LWIP compiled?
> >
>
> It has a really huge configuration. I tried to turn off everything off
> everything what can impact on size but still make http app work:
> #define LWIP_HAVE_LOOPIF 0
> #define LWIP_NETCONN 0
> #define LWIP_SOCKET 0
> #define SO_REUSE 0
> #define LWIP_STATS 0
> #define PPP_SUPPORT 0
>
> Disabling loopback:
> #define LWIP_NETIF_LOOPBACK 0
> can lower to 912288 bytes.
>
> And it's the same compilation option (optimization for size) as the main
> u-boot. I will do more experiments, but I think the goal is not to turn off
> everything.
>
>
> > > - Was ipv6 supported?
> >
>
> No. I.e. when I sent results it was enabled on the compilation stage but
> not used. I just checked that size remains the same if IPv6 is not even
> compiled.
>
>
> > > - Can we strip it down even further?
> > >
> >
>
> There is always room for optimization. I think I tried to turn off
> everything that is configurable with defines. I can play with disable IP
> reassembly and things like that or figure out which functions have more
> size and if it's possible to exclude them.
>
>
> > > In general please give as much information as you can with what we
> > > gain in functionality from LWIP with those extra bytes of code.
> >
> >
> The main idea is to reuse a maintainable IP stack outside of U-boot. LWIP
> can give a nice separation between IP stack code and network application
> code. I.e. application should not take care about any TCP details (SYN,
> ACK, retransmission, reassembly etc) and should open connection and use
> functions similar to recv() and send() to transfer data. Data means
> application data, no network packets. And LWIP allows
> us to do that.
> Because LWIP has an API similar to sockets, it has to be very easy to port
> a linux application to LWIP. Then you can test it with a tap device. Then
> copy sources to U-boot, add a small integration layer (cmd command to
> call), compile and use.
>
> So my suggestion was:
> - do not maintain new network stack code in the current U-boot. Use lwip
> sources as an external project. All bugs related to network stack go to
> lwip project first, then sync with U-boot.
> - maintain network apps code* or
> -- inside U-boot. Write our own code for application and maintain it
> inside U-boot.
> -- inside LWIP. Add examples to LWIP which are suitable for both U-boot
> and LWIP.
>
> * Let's define a U-boot network application as a cmd command. It might be
> ping, wget (http or https download), telnet, arp dns etc..
>
> Let's consider the real use case, like HTTPS download client. We need to
> enable TLS connection, validate certificates, then do http download.
> Looking at the current code of wget command it's quite difficult to
> implement this due to the application having some protol level things. On
> the other side we can find embedTLS examples to do https download on
> sockets. If LWIP socket API is ported then the only thing you need to do is
> change socket() -> lwip_socket(), recv()->lwip_recv(), send()->lwip_send()
> and etc, even function names are similar. If LWIP socket API is not
> supported, then use callback API for recv() and send(), which are also
> easy.
>
> So yes we add extra bytes, but that will allow us to write more complex
> apps, use standard debug tools, use applications with very minimal
> integration changes, use help from the LWIP community to fix protocol bugs,
> etc..
> Bunch of things already implemented there:
> - ipv6
> - dhcp
> - snmp
> - igmp
> - dns
> - tcp and udp and raw.
> - loopback
> - netconn
> - socket
> - stats
> - ppp
> (I just followed configurable defines).
>
>
> And please make sure to disable the previous support, my guess fro that
> > much growth is that you didn't.
> >
>
> # CONFIG_PROT_TCP is not set
> # CONFIG_PROT_UDP is not set
> # CONFIG_UDP_CHECKSUM is not set
> # CONFIG_UDP_FUNCTION_FASTBOOT is not set
> # CONFIG_CMD_WGET is not set
I think you need to step back and figure out a better way to measure the
size change and growth.
I am not interested in a path that long term means two networking stacks
in U-Boot.
I am not interested in massively growing the overall binary size for
every platform. Given how much larger just TCP support is, that's
strongly implying a huge growth for the older use cases too.
But I also suspect given the overall reputation that LWIP enjoys,
there's something amiss here.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [RFC PATCH 0/5] LWIP stack integration
2023-05-22 21:23 ` Tom Rini
@ 2023-05-24 14:05 ` Maxim Uvarov
2023-05-24 20:18 ` [lwip-devel] " Simon Goldschmidt
0 siblings, 1 reply; 37+ messages in thread
From: Maxim Uvarov @ 2023-05-24 14:05 UTC (permalink / raw)
To: Tom Rini
Cc: Ilias Apalodimas, u-boot, pbrobinson, joe.hershberger,
rfried.dev, lwip-devel
On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
> On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
> > On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
> >
> > > On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> > > > Hi Maxim
> > > >
> > > > On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
> > > wrote:
> > > > >
> > > > > My measurements for binary after LTO looks like:
> > > > >
> > > > > U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> > > > > 870728 | 915000 | 912560 |
> > > 41832 | 4.8
> > > >
> > > >
> > > > I think you'll need to analyze that a bit more. First of all I don't
> > > > think the '+ping' tab is useful. What is is trying to achieve?
> > >
> >
> > To show the difference of extra bytes if we add a ping app on top.
> >
> >
> > > > - How was LWIP compiled?
> > >
> >
> > It has a really huge configuration. I tried to turn off everything off
> > everything what can impact on size but still make http app work:
> > #define LWIP_HAVE_LOOPIF 0
> > #define LWIP_NETCONN 0
> > #define LWIP_SOCKET 0
> > #define SO_REUSE 0
> > #define LWIP_STATS 0
> > #define PPP_SUPPORT 0
> >
> > Disabling loopback:
> > #define LWIP_NETIF_LOOPBACK 0
> > can lower to 912288 bytes.
> >
> > And it's the same compilation option (optimization for size) as the main
> > u-boot. I will do more experiments, but I think the goal is not to turn
> off
> > everything.
> >
> >
> > > > - Was ipv6 supported?
> > >
> >
> > No. I.e. when I sent results it was enabled on the compilation stage but
> > not used. I just checked that size remains the same if IPv6 is not even
> > compiled.
> >
> >
> > > > - Can we strip it down even further?
> > > >
> > >
> >
> > There is always room for optimization. I think I tried to turn off
> > everything that is configurable with defines. I can play with disable IP
> > reassembly and things like that or figure out which functions have more
> > size and if it's possible to exclude them.
> >
> >
> > > > In general please give as much information as you can with what we
> > > > gain in functionality from LWIP with those extra bytes of code.
> > >
> > >
> > The main idea is to reuse a maintainable IP stack outside of U-boot.
> LWIP
> > can give a nice separation between IP stack code and network application
> > code. I.e. application should not take care about any TCP details (SYN,
> > ACK, retransmission, reassembly etc) and should open connection and use
> > functions similar to recv() and send() to transfer data. Data means
> > application data, no network packets. And LWIP allows
> > us to do that.
> > Because LWIP has an API similar to sockets, it has to be very easy to
> port
> > a linux application to LWIP. Then you can test it with a tap device. Then
> > copy sources to U-boot, add a small integration layer (cmd command to
> > call), compile and use.
> >
> > So my suggestion was:
> > - do not maintain new network stack code in the current U-boot. Use lwip
> > sources as an external project. All bugs related to network stack go to
> > lwip project first, then sync with U-boot.
> > - maintain network apps code* or
> > -- inside U-boot. Write our own code for application and maintain it
> > inside U-boot.
> > -- inside LWIP. Add examples to LWIP which are suitable for both
> U-boot
> > and LWIP.
> >
> > * Let's define a U-boot network application as a cmd command. It might be
> > ping, wget (http or https download), telnet, arp dns etc..
> >
> > Let's consider the real use case, like HTTPS download client. We need to
> > enable TLS connection, validate certificates, then do http download.
> > Looking at the current code of wget command it's quite difficult to
> > implement this due to the application having some protol level things. On
> > the other side we can find embedTLS examples to do https download on
> > sockets. If LWIP socket API is ported then the only thing you need to do
> is
> > change socket() -> lwip_socket(), recv()->lwip_recv(),
> send()->lwip_send()
> > and etc, even function names are similar. If LWIP socket API is not
> > supported, then use callback API for recv() and send(), which are also
> > easy.
> >
> > So yes we add extra bytes, but that will allow us to write more complex
> > apps, use standard debug tools, use applications with very minimal
> > integration changes, use help from the LWIP community to fix protocol
> bugs,
> > etc..
> > Bunch of things already implemented there:
> > - ipv6
> > - dhcp
> > - snmp
> > - igmp
> > - dns
> > - tcp and udp and raw.
> > - loopback
> > - netconn
> > - socket
> > - stats
> > - ppp
> > (I just followed configurable defines).
> >
> >
> > And please make sure to disable the previous support, my guess fro that
> > > much growth is that you didn't.
> > >
> >
> > # CONFIG_PROT_TCP is not set
> > # CONFIG_PROT_UDP is not set
> > # CONFIG_UDP_CHECKSUM is not set
> > # CONFIG_UDP_FUNCTION_FASTBOOT is not set
> > # CONFIG_CMD_WGET is not set
>
> I think you need to step back and figure out a better way to measure the
> size change and growth.
>
> I am not interested in a path that long term means two networking stacks
> in U-Boot.
>
> I am not interested in massively growing the overall binary size for
> every platform. Given how much larger just TCP support is, that's
> strongly implying a huge growth for the older use cases too.
>
> But I also suspect given the overall reputation that LWIP enjoys,
> there's something amiss here.
>
> --
> Tom
>
+cc lwip-devel@ mailing list, maybe they have something to add.
My measurements say that the current U-boot IP stack + wget command adds an
additional 9 Kbytes.
The minimal configuration of LWIP with wget command is 30 Kbytes.
(compiled out all asserts, debugs, not used protocols etc.).
And the most bigger functions are tcp in/out itself:
* These functions are generally called in the order (ip_input() ->)
* tcp_input() -> * tcp_process() -> tcp_receive() (-> application).
+tcp_input - 4364 +4364
https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n118
+tcp_receive - 3444 +3444
https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n1154
+tcp_write - 2192 +2192
https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n393
+ip4_reass - 2096 +2096
https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/ipv4/ip4_frag.c#n503
+tcp_output - 1616 +1616
https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n1241
If we transfer current net commands to lwip then we can decrease the size,
because of functions reuse.
And if we turn on all features in lwip it will be about 50 Kbytes.
BR,
Maxim.
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-05-24 14:05 ` Maxim Uvarov
@ 2023-05-24 20:18 ` Simon Goldschmidt
2023-06-06 14:33 ` Maxim Uvarov
2023-06-07 20:07 ` Tom Rini
0 siblings, 2 replies; 37+ messages in thread
From: Simon Goldschmidt @ 2023-05-24 20:18 UTC (permalink / raw)
To: Maxim Uvarov, Tom Rini
Cc: Ilias Apalodimas, u-boot, pbrobinson, joe.hershberger,
rfried.dev, lwip-devel
Hi Maxim, Tom,
On 24.05.2023 16:05, Maxim Uvarov wrote:
> On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
>
>> On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
>>> On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
>>>
>>>> On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
>>>>> Hi Maxim
>>>>>
>>>>> On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
>>>> wrote:
>>>>>>
>>>>>> My measurements for binary after LTO looks like:
>>>>>>
>>>>>> U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
>>>>>> 870728 | 915000 | 912560 |
>>>> 41832 | 4.8
>>>>>
>>>>>
>>>>> I think you'll need to analyze that a bit more. First of all I don't
>>>>> think the '+ping' tab is useful. What is is trying to achieve?
>>>>
>>>
>>> To show the difference of extra bytes if we add a ping app on top.
>>>
>>>
>>>>> - How was LWIP compiled?
>>>>
>>>
>>> It has a really huge configuration. I tried to turn off everything off
>>> everything what can impact on size but still make http app work:
>>> #define LWIP_HAVE_LOOPIF 0
>>> #define LWIP_NETCONN 0
>>> #define LWIP_SOCKET 0
>>> #define SO_REUSE 0
>>> #define LWIP_STATS 0
>>> #define PPP_SUPPORT 0
>>>
>>> Disabling loopback:
>>> #define LWIP_NETIF_LOOPBACK 0
>>> can lower to 912288 bytes.
>>>
>>> And it's the same compilation option (optimization for size) as the main
>>> u-boot. I will do more experiments, but I think the goal is not to turn
>> off
>>> everything.
>>>
>>>
>>>>> - Was ipv6 supported?
>>>>
>>>
>>> No. I.e. when I sent results it was enabled on the compilation stage but
>>> not used. I just checked that size remains the same if IPv6 is not even
>>> compiled.
>>>
>>>
>>>>> - Can we strip it down even further?
>>>>>
>>>>
>>>
>>> There is always room for optimization. I think I tried to turn off
>>> everything that is configurable with defines. I can play with disable IP
>>> reassembly and things like that or figure out which functions have more
>>> size and if it's possible to exclude them.
>>>
>>>
>>>>> In general please give as much information as you can with what we
>>>>> gain in functionality from LWIP with those extra bytes of code.
>>>>
>>>>
>>> The main idea is to reuse a maintainable IP stack outside of U-boot.
>> LWIP
>>> can give a nice separation between IP stack code and network application
>>> code. I.e. application should not take care about any TCP details (SYN,
>>> ACK, retransmission, reassembly etc) and should open connection and use
>>> functions similar to recv() and send() to transfer data. Data means
>>> application data, no network packets. And LWIP allows
>>> us to do that.
>>> Because LWIP has an API similar to sockets, it has to be very easy to
>> port
>>> a linux application to LWIP. Then you can test it with a tap device. Then
>>> copy sources to U-boot, add a small integration layer (cmd command to
>>> call), compile and use.
>>>
>>> So my suggestion was:
>>> - do not maintain new network stack code in the current U-boot. Use lwip
>>> sources as an external project. All bugs related to network stack go to
>>> lwip project first, then sync with U-boot.
>>> - maintain network apps code* or
>>> -- inside U-boot. Write our own code for application and maintain it
>>> inside U-boot.
>>> -- inside LWIP. Add examples to LWIP which are suitable for both
>> U-boot
>>> and LWIP.
>>>
>>> * Let's define a U-boot network application as a cmd command. It might be
>>> ping, wget (http or https download), telnet, arp dns etc..
>>>
>>> Let's consider the real use case, like HTTPS download client. We need to
>>> enable TLS connection, validate certificates, then do http download.
>>> Looking at the current code of wget command it's quite difficult to
>>> implement this due to the application having some protol level things. On
>>> the other side we can find embedTLS examples to do https download on
>>> sockets. If LWIP socket API is ported then the only thing you need to do
>> is
>>> change socket() -> lwip_socket(), recv()->lwip_recv(),
>> send()->lwip_send()
>>> and etc, even function names are similar. If LWIP socket API is not
>>> supported, then use callback API for recv() and send(), which are also
>>> easy.
>>>
>>> So yes we add extra bytes, but that will allow us to write more complex
>>> apps, use standard debug tools, use applications with very minimal
>>> integration changes, use help from the LWIP community to fix protocol
>> bugs,
>>> etc..
>>> Bunch of things already implemented there:
>>> - ipv6
>>> - dhcp
>>> - snmp
>>> - igmp
>>> - dns
>>> - tcp and udp and raw.
>>> - loopback
>>> - netconn
>>> - socket
>>> - stats
>>> - ppp
>>> (I just followed configurable defines).
>>>
>>>
>>> And please make sure to disable the previous support, my guess fro that
>>>> much growth is that you didn't.
>>>>
>>>
>>> # CONFIG_PROT_TCP is not set
>>> # CONFIG_PROT_UDP is not set
>>> # CONFIG_UDP_CHECKSUM is not set
>>> # CONFIG_UDP_FUNCTION_FASTBOOT is not set
>>> # CONFIG_CMD_WGET is not set
>>
>> I think you need to step back and figure out a better way to measure the
>> size change and growth.
>>
>> I am not interested in a path that long term means two networking stacks
>> in U-Boot.
>>
>> I am not interested in massively growing the overall binary size for
>> every platform. Given how much larger just TCP support is, that's
>> strongly implying a huge growth for the older use cases too.
>>
>> But I also suspect given the overall reputation that LWIP enjoys,
>> there's something amiss here.
>>
>> --
>> Tom
>>
>
> +cc lwip-devel@ mailing list, maybe they have something to add.
I do think using lwIP instead of "inventing yet another IP stack" is a
good idea! However, in terms of code size, lwIP will lose against what's
in U-Boot at present. And this is only natural, as lwIP is a "full-size"
stack supporting multiple concurrently running applications while the
current IP stack in U-Boot is rather "crippled" down to just what the
implementor needed at the time of writing.
One example of this is that (if I remember correctly), U-Boot only has
one single network packet buffer, while lwIP has support for multiple
buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
of that development in U-Boot about 3 years ago), we're comparing "we
have implemented everything we need so that it just kind of works" to
"we can easily add a HTTPS client to download something over the
internet just by enabling more compile options".
Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
U-Boot TCP (at least that of some years ago) is far from complete when
compared to lwIP!
lwIP is meant to be highly configurable and we're always open to add yet
more options to leave out more code when it's not needed. However, I
think there are some design decisions that will make lwIP larger than
the current IP stack in U-Boot. To me, that's a natural result of having
a "generic code" approach vs "developed to our needs". However, while
DHCP + BOOTP and even a simple network console was rather easy to
implement, I would not recommend implementing your own HTTPS download
but rather using the existing lwIP + apps for that.
In the end, I cannot take the decision from you. In my opinion, lwIP
would be the better decision in terms of future work load and
compatibility, but in the short run, it *will* lead to bigger binaries
at least in some setups. And I do know from my past that it sometimes
has been a pain to try and stuff a new U-Boot release into the existing
space of flash or RAM, so that's not an easy decision.
If you do take the lwIP approach however, let us know if we can help!
Regards,
Simon
>
> My measurements say that the current U-boot IP stack + wget command adds an
> additional 9 Kbytes.
> The minimal configuration of LWIP with wget command is 30 Kbytes.
> (compiled out all asserts, debugs, not used protocols etc.).
>
> And the most bigger functions are tcp in/out itself:
> * These functions are generally called in the order (ip_input() ->)
> * tcp_input() -> * tcp_process() -> tcp_receive() (-> application).
>
> +tcp_input - 4364 +4364
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n118
> +tcp_receive - 3444 +3444
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n1154
> +tcp_write - 2192 +2192
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n393
> +ip4_reass - 2096 +2096
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/ipv4/ip4_frag.c#n503
> +tcp_output - 1616 +1616
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n1241
>
> If we transfer current net commands to lwip then we can decrease the size,
> because of functions reuse.
> And if we turn on all features in lwip it will be about 50 Kbytes.
>
> BR,
> Maxim.
>
>
> _______________________________________________
> lwip-devel mailing list
> lwip-devel@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-devel
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-05-24 20:18 ` [lwip-devel] " Simon Goldschmidt
@ 2023-06-06 14:33 ` Maxim Uvarov
2023-06-07 9:46 ` Ilias Apalodimas
2023-06-07 20:07 ` Tom Rini
1 sibling, 1 reply; 37+ messages in thread
From: Maxim Uvarov @ 2023-06-06 14:33 UTC (permalink / raw)
To: Simon Goldschmidt
Cc: Tom Rini, Ilias Apalodimas, u-boot, pbrobinson, joe.hershberger,
rfried.dev, lwip-devel
Greetings,
I implemented the tftp client (that was quick due to lwip has example app
for tftp), and did some more measurements.
I uploaded patches here if somebody want to do his own measurements:
https://github.com/muvarov/uboot-lwip
measure 1:
976K - total (total means lwip with all 3 commands ping, tftp, wget)
971K - total - tftp (total, but disable/minus tftp)
965K - total - tftp - wget (disable tftp and wget)
963K - total - tftp - wget - ping (disable tftp, wget, ping)
931K - no lwip
result 1: lwip tftp (+ udp protocol) protocol 976-971k = 5kb
result 2: lwip ping command 965- 963 = 2kb
result 3: lwip wget command 971- 965 = 6kb
result 4: lwip core stack with apps 976 - 931 = 45kb
measure 2:
890K - no CONFIG_NET_CMD
930K - + lwip tftp only
937K - + full lwip (ping wget tftp)
result 1: 937-890=47kb ( lwip + all 3 commands)
result 2: 937-930=7kb (ping and lwip command)
measure 3:
904K - no lwip, CMD_NET_TFTP=y
900K - no lwip, CMD_NET_TFTP=n
result 1: original u-boot tftp command 904-900=4kb
890K - no lwip, CMD_NET=n
result 2: 900-890=10k original u-boot net/IP stack.
My findings for all that measurements and lwip configuration:
1. The original u-boot net stack (packet process and up layers) is 10k vs
lwip 40k (the very minimal settings were 30k).
2. Network applications size is about the same 4kb for tftp original
command 5kb for lwip.
3. It's quite easy to reuse LWIP examples to implement the same
functionality for the U-boot.
4. I still think that there are other criterias which might have more
priority than size (bug free code, code reuse, development speed,
compatible API to posix and etc).
BR,
Maxim.
On Thu, 25 May 2023 at 02:18, Simon Goldschmidt <goldsimon@gmx.de> wrote:
> Hi Maxim, Tom,
>
> On 24.05.2023 16:05, Maxim Uvarov wrote:
> > On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
> >
> >> On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
> >>> On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
> >>>
> >>>> On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> >>>>> Hi Maxim
> >>>>>
> >>>>> On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
> >>>> wrote:
> >>>>>>
> >>>>>> My measurements for binary after LTO looks like:
> >>>>>>
> >>>>>> U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> >>>>>> 870728 | 915000 | 912560 |
> >>>> 41832 | 4.8
> >>>>>
> >>>>>
> >>>>> I think you'll need to analyze that a bit more. First of all I don't
> >>>>> think the '+ping' tab is useful. What is is trying to achieve?
> >>>>
> >>>
> >>> To show the difference of extra bytes if we add a ping app on top.
> >>>
> >>>
> >>>>> - How was LWIP compiled?
> >>>>
> >>>
> >>> It has a really huge configuration. I tried to turn off everything off
> >>> everything what can impact on size but still make http app work:
> >>> #define LWIP_HAVE_LOOPIF 0
> >>> #define LWIP_NETCONN 0
> >>> #define LWIP_SOCKET 0
> >>> #define SO_REUSE 0
> >>> #define LWIP_STATS 0
> >>> #define PPP_SUPPORT 0
> >>>
> >>> Disabling loopback:
> >>> #define LWIP_NETIF_LOOPBACK 0
> >>> can lower to 912288 bytes.
> >>>
> >>> And it's the same compilation option (optimization for size) as the
> main
> >>> u-boot. I will do more experiments, but I think the goal is not to turn
> >> off
> >>> everything.
> >>>
> >>>
> >>>>> - Was ipv6 supported?
> >>>>
> >>>
> >>> No. I.e. when I sent results it was enabled on the compilation stage
> but
> >>> not used. I just checked that size remains the same if IPv6 is not even
> >>> compiled.
> >>>
> >>>
> >>>>> - Can we strip it down even further?
> >>>>>
> >>>>
> >>>
> >>> There is always room for optimization. I think I tried to turn off
> >>> everything that is configurable with defines. I can play with disable
> IP
> >>> reassembly and things like that or figure out which functions have more
> >>> size and if it's possible to exclude them.
> >>>
> >>>
> >>>>> In general please give as much information as you can with what we
> >>>>> gain in functionality from LWIP with those extra bytes of code.
> >>>>
> >>>>
> >>> The main idea is to reuse a maintainable IP stack outside of U-boot.
> >> LWIP
> >>> can give a nice separation between IP stack code and network
> application
> >>> code. I.e. application should not take care about any TCP details
> (SYN,
> >>> ACK, retransmission, reassembly etc) and should open connection and use
> >>> functions similar to recv() and send() to transfer data. Data means
> >>> application data, no network packets. And LWIP allows
> >>> us to do that.
> >>> Because LWIP has an API similar to sockets, it has to be very easy to
> >> port
> >>> a linux application to LWIP. Then you can test it with a tap device.
> Then
> >>> copy sources to U-boot, add a small integration layer (cmd command to
> >>> call), compile and use.
> >>>
> >>> So my suggestion was:
> >>> - do not maintain new network stack code in the current U-boot. Use
> lwip
> >>> sources as an external project. All bugs related to network stack go
> to
> >>> lwip project first, then sync with U-boot.
> >>> - maintain network apps code* or
> >>> -- inside U-boot. Write our own code for application and maintain it
> >>> inside U-boot.
> >>> -- inside LWIP. Add examples to LWIP which are suitable for both
> >> U-boot
> >>> and LWIP.
> >>>
> >>> * Let's define a U-boot network application as a cmd command. It might
> be
> >>> ping, wget (http or https download), telnet, arp dns etc..
> >>>
> >>> Let's consider the real use case, like HTTPS download client. We need
> to
> >>> enable TLS connection, validate certificates, then do http download.
> >>> Looking at the current code of wget command it's quite difficult to
> >>> implement this due to the application having some protol level things.
> On
> >>> the other side we can find embedTLS examples to do https download on
> >>> sockets. If LWIP socket API is ported then the only thing you need to
> do
> >> is
> >>> change socket() -> lwip_socket(), recv()->lwip_recv(),
> >> send()->lwip_send()
> >>> and etc, even function names are similar. If LWIP socket API is not
> >>> supported, then use callback API for recv() and send(), which are also
> >>> easy.
> >>>
> >>> So yes we add extra bytes, but that will allow us to write more complex
> >>> apps, use standard debug tools, use applications with very minimal
> >>> integration changes, use help from the LWIP community to fix protocol
> >> bugs,
> >>> etc..
> >>> Bunch of things already implemented there:
> >>> - ipv6
> >>> - dhcp
> >>> - snmp
> >>> - igmp
> >>> - dns
> >>> - tcp and udp and raw.
> >>> - loopback
> >>> - netconn
> >>> - socket
> >>> - stats
> >>> - ppp
> >>> (I just followed configurable defines).
> >>>
> >>>
> >>> And please make sure to disable the previous support, my guess fro that
> >>>> much growth is that you didn't.
> >>>>
> >>>
> >>> # CONFIG_PROT_TCP is not set
> >>> # CONFIG_PROT_UDP is not set
> >>> # CONFIG_UDP_CHECKSUM is not set
> >>> # CONFIG_UDP_FUNCTION_FASTBOOT is not set
> >>> # CONFIG_CMD_WGET is not set
> >>
> >> I think you need to step back and figure out a better way to measure the
> >> size change and growth.
> >>
> >> I am not interested in a path that long term means two networking stacks
> >> in U-Boot.
> >>
> >> I am not interested in massively growing the overall binary size for
> >> every platform. Given how much larger just TCP support is, that's
> >> strongly implying a huge growth for the older use cases too.
> >>
> >> But I also suspect given the overall reputation that LWIP enjoys,
> >> there's something amiss here.
> >>
> >> --
> >> Tom
> >>
> >
> > +cc lwip-devel@ mailing list, maybe they have something to add.
>
> I do think using lwIP instead of "inventing yet another IP stack" is a
> good idea! However, in terms of code size, lwIP will lose against what's
> in U-Boot at present. And this is only natural, as lwIP is a "full-size"
> stack supporting multiple concurrently running applications while the
> current IP stack in U-Boot is rather "crippled" down to just what the
> implementor needed at the time of writing.
>
> One example of this is that (if I remember correctly), U-Boot only has
> one single network packet buffer, while lwIP has support for multiple
> buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
> of that development in U-Boot about 3 years ago), we're comparing "we
> have implemented everything we need so that it just kind of works" to
> "we can easily add a HTTPS client to download something over the
> internet just by enabling more compile options".
>
> Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
> U-Boot TCP (at least that of some years ago) is far from complete when
> compared to lwIP!
>
> lwIP is meant to be highly configurable and we're always open to add yet
> more options to leave out more code when it's not needed. However, I
> think there are some design decisions that will make lwIP larger than
> the current IP stack in U-Boot. To me, that's a natural result of having
> a "generic code" approach vs "developed to our needs". However, while
> DHCP + BOOTP and even a simple network console was rather easy to
> implement, I would not recommend implementing your own HTTPS download
> but rather using the existing lwIP + apps for that.
>
> In the end, I cannot take the decision from you. In my opinion, lwIP
> would be the better decision in terms of future work load and
> compatibility, but in the short run, it *will* lead to bigger binaries
> at least in some setups. And I do know from my past that it sometimes
> has been a pain to try and stuff a new U-Boot release into the existing
> space of flash or RAM, so that's not an easy decision.
>
> If you do take the lwIP approach however, let us know if we can help!
>
> Regards,
> Simon
>
> >
> > My measurements say that the current U-boot IP stack + wget command adds
> an
> > additional 9 Kbytes.
> > The minimal configuration of LWIP with wget command is 30 Kbytes.
> > (compiled out all asserts, debugs, not used protocols etc.).
> >
> > And the most bigger functions are tcp in/out itself:
> > * These functions are generally called in the order (ip_input() ->)
> > * tcp_input() -> * tcp_process() -> tcp_receive() (-> application).
> >
> > +tcp_input - 4364 +4364
> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n118
> > +tcp_receive - 3444 +3444
> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n1154
> > +tcp_write - 2192 +2192
> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n393
> > +ip4_reass - 2096 +2096
> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/ipv4/ip4_frag.c#n503
> > +tcp_output - 1616 +1616
> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n1241
> >
> > If we transfer current net commands to lwip then we can decrease the
> size,
> > because of functions reuse.
> > And if we turn on all features in lwip it will be about 50 Kbytes.
> >
> > BR,
> > Maxim.
> >
> >
> > _______________________________________________
> > lwip-devel mailing list
> > lwip-devel@nongnu.org
> > https://lists.nongnu.org/mailman/listinfo/lwip-devel
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-06-06 14:33 ` Maxim Uvarov
@ 2023-06-07 9:46 ` Ilias Apalodimas
2023-06-07 12:01 ` Maxim Uvarov
2023-06-11 8:24 ` Simon Glass
0 siblings, 2 replies; 37+ messages in thread
From: Ilias Apalodimas @ 2023-06-07 9:46 UTC (permalink / raw)
To: Maxim Uvarov
Cc: Simon Goldschmidt, Tom Rini, u-boot, pbrobinson, joe.hershberger,
rfried.dev, lwip-devel
Hi Maxim,
On Tue, 6 Jun 2023 at 17:33, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>
> Greetings,
>
> I implemented the tftp client (that was quick due to lwip has example app for tftp), and did some more measurements.
> I uploaded patches here if somebody want to do his own measurements:
> https://github.com/muvarov/uboot-lwip
>
> measure 1:
> 976K - total (total means lwip with all 3 commands ping, tftp, wget)
> 971K - total - tftp (total, but disable/minus tftp)
> 965K - total - tftp - wget (disable tftp and wget)
> 963K - total - tftp - wget - ping (disable tftp, wget, ping)
> 931K - no lwip
>
> result 1: lwip tftp (+ udp protocol) protocol 976-971k = 5kb
> result 2: lwip ping command 965- 963 = 2kb
> result 3: lwip wget command 971- 965 = 6kb
> result 4: lwip core stack with apps 976 - 931 = 45kb
So tftp = 5kb, wget = 6kb ping =2kb and lwip = 32kb
>
> measure 2:
> 890K - no CONFIG_NET_CMD
> 930K - + lwip tftp only
> 937K - + full lwip (ping wget tftp)
>
> result 1: 937-890=47kb ( lwip + all 3 commands)
> result 2: 937-930=7kb (ping and lwip command)
I am not sure I understand this measurement. How is this different
from measurement 1 where the entire binary was 976K?
>
> measure 3:
> 904K - no lwip, CMD_NET_TFTP=y
> 900K - no lwip, CMD_NET_TFTP=n
> result 1: original u-boot tftp command 904-900=4kb
> 890K - no lwip, CMD_NET=n
> result 2: 900-890=10k original u-boot net/IP stack.
>
> My findings for all that measurements and lwip configuration:
> 1. The original u-boot net stack (packet process and up layers) is 10k vs lwip 40k (the very minimal settings were 30k).
> 2. Network applications size is about the same 4kb for tftp original command 5kb for lwip.
> 3. It's quite easy to reuse LWIP examples to implement the same functionality for the U-boot.
> 4. I still think that there are other criterias which might have more priority than size (bug free code, code reuse, development speed, compatible API to posix and etc).
Yes, there are other criteria and certainly having a complete network
stack might be worth it in many cases, but we need to keep in mind
30kb might be a lot for some systems.
I personally think this is decent and we can optimize lwip more in the
future. Tom, Simon, how about adding lwip as 'experimental' and
making it depend on !CMD_NET or something similar?
Thanks
/Ilias
>
> BR,
> Maxim.
>
> On Thu, 25 May 2023 at 02:18, Simon Goldschmidt <goldsimon@gmx.de> wrote:
>>
>> Hi Maxim, Tom,
>>
>> On 24.05.2023 16:05, Maxim Uvarov wrote:
>> > On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
>> >
>> >> On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
>> >>> On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
>> >>>
>> >>>> On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
>> >>>>> Hi Maxim
>> >>>>>
>> >>>>> On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
>> >>>> wrote:
>> >>>>>>
>> >>>>>> My measurements for binary after LTO looks like:
>> >>>>>>
>> >>>>>> U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
>> >>>>>> 870728 | 915000 | 912560 |
>> >>>> 41832 | 4.8
>> >>>>>
>> >>>>>
>> >>>>> I think you'll need to analyze that a bit more. First of all I don't
>> >>>>> think the '+ping' tab is useful. What is is trying to achieve?
>> >>>>
>> >>>
>> >>> To show the difference of extra bytes if we add a ping app on top.
>> >>>
>> >>>
>> >>>>> - How was LWIP compiled?
>> >>>>
>> >>>
>> >>> It has a really huge configuration. I tried to turn off everything off
>> >>> everything what can impact on size but still make http app work:
>> >>> #define LWIP_HAVE_LOOPIF 0
>> >>> #define LWIP_NETCONN 0
>> >>> #define LWIP_SOCKET 0
>> >>> #define SO_REUSE 0
>> >>> #define LWIP_STATS 0
>> >>> #define PPP_SUPPORT 0
>> >>>
>> >>> Disabling loopback:
>> >>> #define LWIP_NETIF_LOOPBACK 0
>> >>> can lower to 912288 bytes.
>> >>>
>> >>> And it's the same compilation option (optimization for size) as the main
>> >>> u-boot. I will do more experiments, but I think the goal is not to turn
>> >> off
>> >>> everything.
>> >>>
>> >>>
>> >>>>> - Was ipv6 supported?
>> >>>>
>> >>>
>> >>> No. I.e. when I sent results it was enabled on the compilation stage but
>> >>> not used. I just checked that size remains the same if IPv6 is not even
>> >>> compiled.
>> >>>
>> >>>
>> >>>>> - Can we strip it down even further?
>> >>>>>
>> >>>>
>> >>>
>> >>> There is always room for optimization. I think I tried to turn off
>> >>> everything that is configurable with defines. I can play with disable IP
>> >>> reassembly and things like that or figure out which functions have more
>> >>> size and if it's possible to exclude them.
>> >>>
>> >>>
>> >>>>> In general please give as much information as you can with what we
>> >>>>> gain in functionality from LWIP with those extra bytes of code.
>> >>>>
>> >>>>
>> >>> The main idea is to reuse a maintainable IP stack outside of U-boot.
>> >> LWIP
>> >>> can give a nice separation between IP stack code and network application
>> >>> code. I.e. application should not take care about any TCP details (SYN,
>> >>> ACK, retransmission, reassembly etc) and should open connection and use
>> >>> functions similar to recv() and send() to transfer data. Data means
>> >>> application data, no network packets. And LWIP allows
>> >>> us to do that.
>> >>> Because LWIP has an API similar to sockets, it has to be very easy to
>> >> port
>> >>> a linux application to LWIP. Then you can test it with a tap device. Then
>> >>> copy sources to U-boot, add a small integration layer (cmd command to
>> >>> call), compile and use.
>> >>>
>> >>> So my suggestion was:
>> >>> - do not maintain new network stack code in the current U-boot. Use lwip
>> >>> sources as an external project. All bugs related to network stack go to
>> >>> lwip project first, then sync with U-boot.
>> >>> - maintain network apps code* or
>> >>> -- inside U-boot. Write our own code for application and maintain it
>> >>> inside U-boot.
>> >>> -- inside LWIP. Add examples to LWIP which are suitable for both
>> >> U-boot
>> >>> and LWIP.
>> >>>
>> >>> * Let's define a U-boot network application as a cmd command. It might be
>> >>> ping, wget (http or https download), telnet, arp dns etc..
>> >>>
>> >>> Let's consider the real use case, like HTTPS download client. We need to
>> >>> enable TLS connection, validate certificates, then do http download.
>> >>> Looking at the current code of wget command it's quite difficult to
>> >>> implement this due to the application having some protol level things. On
>> >>> the other side we can find embedTLS examples to do https download on
>> >>> sockets. If LWIP socket API is ported then the only thing you need to do
>> >> is
>> >>> change socket() -> lwip_socket(), recv()->lwip_recv(),
>> >> send()->lwip_send()
>> >>> and etc, even function names are similar. If LWIP socket API is not
>> >>> supported, then use callback API for recv() and send(), which are also
>> >>> easy.
>> >>>
>> >>> So yes we add extra bytes, but that will allow us to write more complex
>> >>> apps, use standard debug tools, use applications with very minimal
>> >>> integration changes, use help from the LWIP community to fix protocol
>> >> bugs,
>> >>> etc..
>> >>> Bunch of things already implemented there:
>> >>> - ipv6
>> >>> - dhcp
>> >>> - snmp
>> >>> - igmp
>> >>> - dns
>> >>> - tcp and udp and raw.
>> >>> - loopback
>> >>> - netconn
>> >>> - socket
>> >>> - stats
>> >>> - ppp
>> >>> (I just followed configurable defines).
>> >>>
>> >>>
>> >>> And please make sure to disable the previous support, my guess fro that
>> >>>> much growth is that you didn't.
>> >>>>
>> >>>
>> >>> # CONFIG_PROT_TCP is not set
>> >>> # CONFIG_PROT_UDP is not set
>> >>> # CONFIG_UDP_CHECKSUM is not set
>> >>> # CONFIG_UDP_FUNCTION_FASTBOOT is not set
>> >>> # CONFIG_CMD_WGET is not set
>> >>
>> >> I think you need to step back and figure out a better way to measure the
>> >> size change and growth.
>> >>
>> >> I am not interested in a path that long term means two networking stacks
>> >> in U-Boot.
>> >>
>> >> I am not interested in massively growing the overall binary size for
>> >> every platform. Given how much larger just TCP support is, that's
>> >> strongly implying a huge growth for the older use cases too.
>> >>
>> >> But I also suspect given the overall reputation that LWIP enjoys,
>> >> there's something amiss here.
>> >>
>> >> --
>> >> Tom
>> >>
>> >
>> > +cc lwip-devel@ mailing list, maybe they have something to add.
>>
>> I do think using lwIP instead of "inventing yet another IP stack" is a
>> good idea! However, in terms of code size, lwIP will lose against what's
>> in U-Boot at present. And this is only natural, as lwIP is a "full-size"
>> stack supporting multiple concurrently running applications while the
>> current IP stack in U-Boot is rather "crippled" down to just what the
>> implementor needed at the time of writing.
>>
>> One example of this is that (if I remember correctly), U-Boot only has
>> one single network packet buffer, while lwIP has support for multiple
>> buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
>> of that development in U-Boot about 3 years ago), we're comparing "we
>> have implemented everything we need so that it just kind of works" to
>> "we can easily add a HTTPS client to download something over the
>> internet just by enabling more compile options".
>>
>> Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
>> U-Boot TCP (at least that of some years ago) is far from complete when
>> compared to lwIP!
>>
>> lwIP is meant to be highly configurable and we're always open to add yet
>> more options to leave out more code when it's not needed. However, I
>> think there are some design decisions that will make lwIP larger than
>> the current IP stack in U-Boot. To me, that's a natural result of having
>> a "generic code" approach vs "developed to our needs". However, while
>> DHCP + BOOTP and even a simple network console was rather easy to
>> implement, I would not recommend implementing your own HTTPS download
>> but rather using the existing lwIP + apps for that.
>>
>> In the end, I cannot take the decision from you. In my opinion, lwIP
>> would be the better decision in terms of future work load and
>> compatibility, but in the short run, it *will* lead to bigger binaries
>> at least in some setups. And I do know from my past that it sometimes
>> has been a pain to try and stuff a new U-Boot release into the existing
>> space of flash or RAM, so that's not an easy decision.
>>
>> If you do take the lwIP approach however, let us know if we can help!
>>
>> Regards,
>> Simon
>>
>> >
>> > My measurements say that the current U-boot IP stack + wget command adds an
>> > additional 9 Kbytes.
>> > The minimal configuration of LWIP with wget command is 30 Kbytes.
>> > (compiled out all asserts, debugs, not used protocols etc.).
>> >
>> > And the most bigger functions are tcp in/out itself:
>> > * These functions are generally called in the order (ip_input() ->)
>> > * tcp_input() -> * tcp_process() -> tcp_receive() (-> application).
>> >
>> > +tcp_input - 4364 +4364
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n118
>> > +tcp_receive - 3444 +3444
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n1154
>> > +tcp_write - 2192 +2192
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n393
>> > +ip4_reass - 2096 +2096
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/ipv4/ip4_frag.c#n503
>> > +tcp_output - 1616 +1616
>> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n1241
>> >
>> > If we transfer current net commands to lwip then we can decrease the size,
>> > because of functions reuse.
>> > And if we turn on all features in lwip it will be about 50 Kbytes.
>> >
>> > BR,
>> > Maxim.
>> >
>> >
>> > _______________________________________________
>> > lwip-devel mailing list
>> > lwip-devel@nongnu.org
>> > https://lists.nongnu.org/mailman/listinfo/lwip-devel
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-06-07 9:46 ` Ilias Apalodimas
@ 2023-06-07 12:01 ` Maxim Uvarov
2023-06-11 8:24 ` Simon Glass
1 sibling, 0 replies; 37+ messages in thread
From: Maxim Uvarov @ 2023-06-07 12:01 UTC (permalink / raw)
To: Ilias Apalodimas
Cc: Simon Goldschmidt, Tom Rini, u-boot, pbrobinson, joe.hershberger,
rfried.dev, lwip-devel
On Wed, 7 Jun 2023 at 15:47, Ilias Apalodimas <ilias.apalodimas@linaro.org>
wrote:
> Hi Maxim,
>
> On Tue, 6 Jun 2023 at 17:33, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> >
> > Greetings,
> >
> > I implemented the tftp client (that was quick due to lwip has example
> app for tftp), and did some more measurements.
> > I uploaded patches here if somebody want to do his own measurements:
> > https://github.com/muvarov/uboot-lwip
> >
> > measure 1:
> > 976K - total (total means lwip with all 3 commands ping, tftp, wget)
> > 971K - total - tftp (total, but disable/minus tftp)
> > 965K - total - tftp - wget (disable tftp and wget)
> > 963K - total - tftp - wget - ping (disable tftp, wget, ping)
> > 931K - no lwip
> >
> > result 1: lwip tftp (+ udp protocol) protocol 976-971k = 5kb
> > result 2: lwip ping command 965- 963 = 2kb
> > result 3: lwip wget command 971- 965 = 6kb
> > result 4: lwip core stack with apps 976 - 931 = 45kb
>
> So tftp = 5kb, wget = 6kb ping =2kb and lwip = 32kb
>
>
tftp also compiles in the UDP stack. So if there will be one more UDP
application,
then this size will be lower.
> >
> > measure 2:
> > 890K - no CONFIG_NET_CMD
> > 930K - + lwip tftp only
> > 937K - + full lwip (ping wget tftp)
> >
> > result 1: 937-890=47kb ( lwip + all 3 commands)
> > result 2: 937-930=7kb (ping and lwip command)
>
> I am not sure I understand this measurement. How is this different
> from measurement 1 where the entire binary was 976K?
>
> This is when NET_CMD is off and LWIP is off also. First measurement had
NET_CMD=y.
(moved numbers to separate changes due to u-boot can not just enable only
tfpt command due
to a compilation error and had to fix it.).
>
> > measure 3:
> > 904K - no lwip, CMD_NET_TFTP=y
> > 900K - no lwip, CMD_NET_TFTP=n
> > result 1: original u-boot tftp command 904-900=4kb
> > 890K - no lwip, CMD_NET=n
> > result 2: 900-890=10k original u-boot net/IP stack.
> >
> > My findings for all that measurements and lwip configuration:
> > 1. The original u-boot net stack (packet process and up layers) is 10k
> vs lwip 40k (the very minimal settings were 30k).
> > 2. Network applications size is about the same 4kb for tftp original
> command 5kb for lwip.
> > 3. It's quite easy to reuse LWIP examples to implement the same
> functionality for the U-boot.
> > 4. I still think that there are other criterias which might have more
> priority than size (bug free code, code reuse, development speed,
> compatible API to posix and etc).
>
> Yes, there are other criteria and certainly having a complete network
> stack might be worth it in many cases, but we need to keep in mind
> 30kb might be a lot for some systems.
>
> I personally think this is decent and we can optimize lwip more in the
> future. Tom, Simon, how about adding lwip as 'experimental' and
> making it depend on !CMD_NET or something similar?
>
> Thanks
> /Ilias
> >
> > BR,
> > Maxim.
> >
> > On Thu, 25 May 2023 at 02:18, Simon Goldschmidt <goldsimon@gmx.de>
> wrote:
> >>
> >> Hi Maxim, Tom,
> >>
> >> On 24.05.2023 16:05, Maxim Uvarov wrote:
> >> > On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
> >> >
> >> >> On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
> >> >>> On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
> >> >>>
> >> >>>> On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> >> >>>>> Hi Maxim
> >> >>>>>
> >> >>>>> On Mon, 22 May 2023 at 12:01, Maxim Uvarov <
> maxim.uvarov@linaro.org>
> >> >>>> wrote:
> >> >>>>>>
> >> >>>>>> My measurements for binary after LTO looks like:
> >> >>>>>>
> >> >>>>>> U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> >> >>>>>> 870728 | 915000 | 912560
> |
> >> >>>> 41832 | 4.8
> >> >>>>>
> >> >>>>>
> >> >>>>> I think you'll need to analyze that a bit more. First of all I
> don't
> >> >>>>> think the '+ping' tab is useful. What is is trying to achieve?
> >> >>>>
> >> >>>
> >> >>> To show the difference of extra bytes if we add a ping app on top.
> >> >>>
> >> >>>
> >> >>>>> - How was LWIP compiled?
> >> >>>>
> >> >>>
> >> >>> It has a really huge configuration. I tried to turn off everything
> off
> >> >>> everything what can impact on size but still make http app work:
> >> >>> #define LWIP_HAVE_LOOPIF 0
> >> >>> #define LWIP_NETCONN 0
> >> >>> #define LWIP_SOCKET 0
> >> >>> #define SO_REUSE 0
> >> >>> #define LWIP_STATS 0
> >> >>> #define PPP_SUPPORT 0
> >> >>>
> >> >>> Disabling loopback:
> >> >>> #define LWIP_NETIF_LOOPBACK 0
> >> >>> can lower to 912288 bytes.
> >> >>>
> >> >>> And it's the same compilation option (optimization for size) as the
> main
> >> >>> u-boot. I will do more experiments, but I think the goal is not to
> turn
> >> >> off
> >> >>> everything.
> >> >>>
> >> >>>
> >> >>>>> - Was ipv6 supported?
> >> >>>>
> >> >>>
> >> >>> No. I.e. when I sent results it was enabled on the compilation
> stage but
> >> >>> not used. I just checked that size remains the same if IPv6 is not
> even
> >> >>> compiled.
> >> >>>
> >> >>>
> >> >>>>> - Can we strip it down even further?
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>> There is always room for optimization. I think I tried to turn off
> >> >>> everything that is configurable with defines. I can play with
> disable IP
> >> >>> reassembly and things like that or figure out which functions have
> more
> >> >>> size and if it's possible to exclude them.
> >> >>>
> >> >>>
> >> >>>>> In general please give as much information as you can with what
> we
> >> >>>>> gain in functionality from LWIP with those extra bytes of code.
> >> >>>>
> >> >>>>
> >> >>> The main idea is to reuse a maintainable IP stack outside of U-boot.
> >> >> LWIP
> >> >>> can give a nice separation between IP stack code and network
> application
> >> >>> code. I.e. application should not take care about any TCP details
> (SYN,
> >> >>> ACK, retransmission, reassembly etc) and should open connection and
> use
> >> >>> functions similar to recv() and send() to transfer data. Data means
> >> >>> application data, no network packets. And LWIP allows
> >> >>> us to do that.
> >> >>> Because LWIP has an API similar to sockets, it has to be very easy
> to
> >> >> port
> >> >>> a linux application to LWIP. Then you can test it with a tap
> device. Then
> >> >>> copy sources to U-boot, add a small integration layer (cmd command
> to
> >> >>> call), compile and use.
> >> >>>
> >> >>> So my suggestion was:
> >> >>> - do not maintain new network stack code in the current U-boot.
> Use lwip
> >> >>> sources as an external project. All bugs related to network stack
> go to
> >> >>> lwip project first, then sync with U-boot.
> >> >>> - maintain network apps code* or
> >> >>> -- inside U-boot. Write our own code for application and
> maintain it
> >> >>> inside U-boot.
> >> >>> -- inside LWIP. Add examples to LWIP which are suitable for both
> >> >> U-boot
> >> >>> and LWIP.
> >> >>>
> >> >>> * Let's define a U-boot network application as a cmd command. It
> might be
> >> >>> ping, wget (http or https download), telnet, arp dns etc..
> >> >>>
> >> >>> Let's consider the real use case, like HTTPS download client. We
> need to
> >> >>> enable TLS connection, validate certificates, then do http download.
> >> >>> Looking at the current code of wget command it's quite difficult to
> >> >>> implement this due to the application having some protol level
> things. On
> >> >>> the other side we can find embedTLS examples to do https download on
> >> >>> sockets. If LWIP socket API is ported then the only thing you need
> to do
> >> >> is
> >> >>> change socket() -> lwip_socket(), recv()->lwip_recv(),
> >> >> send()->lwip_send()
> >> >>> and etc, even function names are similar. If LWIP socket API is not
> >> >>> supported, then use callback API for recv() and send(), which are
> also
> >> >>> easy.
> >> >>>
> >> >>> So yes we add extra bytes, but that will allow us to write more
> complex
> >> >>> apps, use standard debug tools, use applications with very minimal
> >> >>> integration changes, use help from the LWIP community to fix
> protocol
> >> >> bugs,
> >> >>> etc..
> >> >>> Bunch of things already implemented there:
> >> >>> - ipv6
> >> >>> - dhcp
> >> >>> - snmp
> >> >>> - igmp
> >> >>> - dns
> >> >>> - tcp and udp and raw.
> >> >>> - loopback
> >> >>> - netconn
> >> >>> - socket
> >> >>> - stats
> >> >>> - ppp
> >> >>> (I just followed configurable defines).
> >> >>>
> >> >>>
> >> >>> And please make sure to disable the previous support, my guess fro
> that
> >> >>>> much growth is that you didn't.
> >> >>>>
> >> >>>
> >> >>> # CONFIG_PROT_TCP is not set
> >> >>> # CONFIG_PROT_UDP is not set
> >> >>> # CONFIG_UDP_CHECKSUM is not set
> >> >>> # CONFIG_UDP_FUNCTION_FASTBOOT is not set
> >> >>> # CONFIG_CMD_WGET is not set
> >> >>
> >> >> I think you need to step back and figure out a better way to measure
> the
> >> >> size change and growth.
> >> >>
> >> >> I am not interested in a path that long term means two networking
> stacks
> >> >> in U-Boot.
> >> >>
> >> >> I am not interested in massively growing the overall binary size for
> >> >> every platform. Given how much larger just TCP support is, that's
> >> >> strongly implying a huge growth for the older use cases too.
> >> >>
> >> >> But I also suspect given the overall reputation that LWIP enjoys,
> >> >> there's something amiss here.
> >> >>
> >> >> --
> >> >> Tom
> >> >>
> >> >
> >> > +cc lwip-devel@ mailing list, maybe they have something to add.
> >>
> >> I do think using lwIP instead of "inventing yet another IP stack" is a
> >> good idea! However, in terms of code size, lwIP will lose against what's
> >> in U-Boot at present. And this is only natural, as lwIP is a "full-size"
> >> stack supporting multiple concurrently running applications while the
> >> current IP stack in U-Boot is rather "crippled" down to just what the
> >> implementor needed at the time of writing.
> >>
> >> One example of this is that (if I remember correctly), U-Boot only has
> >> one single network packet buffer, while lwIP has support for multiple
> >> buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
> >> of that development in U-Boot about 3 years ago), we're comparing "we
> >> have implemented everything we need so that it just kind of works" to
> >> "we can easily add a HTTPS client to download something over the
> >> internet just by enabling more compile options".
> >>
> >> Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
> >> U-Boot TCP (at least that of some years ago) is far from complete when
> >> compared to lwIP!
> >>
> >> lwIP is meant to be highly configurable and we're always open to add yet
> >> more options to leave out more code when it's not needed. However, I
> >> think there are some design decisions that will make lwIP larger than
> >> the current IP stack in U-Boot. To me, that's a natural result of having
> >> a "generic code" approach vs "developed to our needs". However, while
> >> DHCP + BOOTP and even a simple network console was rather easy to
> >> implement, I would not recommend implementing your own HTTPS download
> >> but rather using the existing lwIP + apps for that.
> >>
> >> In the end, I cannot take the decision from you. In my opinion, lwIP
> >> would be the better decision in terms of future work load and
> >> compatibility, but in the short run, it *will* lead to bigger binaries
> >> at least in some setups. And I do know from my past that it sometimes
> >> has been a pain to try and stuff a new U-Boot release into the existing
> >> space of flash or RAM, so that's not an easy decision.
> >>
> >> If you do take the lwIP approach however, let us know if we can help!
> >>
> >> Regards,
> >> Simon
> >>
> >> >
> >> > My measurements say that the current U-boot IP stack + wget command
> adds an
> >> > additional 9 Kbytes.
> >> > The minimal configuration of LWIP with wget command is 30 Kbytes.
> >> > (compiled out all asserts, debugs, not used protocols etc.).
> >> >
> >> > And the most bigger functions are tcp in/out itself:
> >> > * These functions are generally called in the order (ip_input() ->)
> >> > * tcp_input() -> * tcp_process() -> tcp_receive() (-> application).
> >> >
> >> > +tcp_input - 4364 +4364
> >> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n118
> >> > +tcp_receive - 3444 +3444
> >> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n1154
> >> > +tcp_write - 2192 +2192
> >> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n393
> >> > +ip4_reass - 2096 +2096
> >> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/ipv4/ip4_frag.c#n503
> >> > +tcp_output - 1616 +1616
> >> >
> https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n1241
> >> >
> >> > If we transfer current net commands to lwip then we can decrease the
> size,
> >> > because of functions reuse.
> >> > And if we turn on all features in lwip it will be about 50 Kbytes.
> >> >
> >> > BR,
> >> > Maxim.
> >> >
> >> >
> >> > _______________________________________________
> >> > lwip-devel mailing list
> >> > lwip-devel@nongnu.org
> >> > https://lists.nongnu.org/mailman/listinfo/lwip-devel
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-05-24 20:18 ` [lwip-devel] " Simon Goldschmidt
2023-06-06 14:33 ` Maxim Uvarov
@ 2023-06-07 20:07 ` Tom Rini
2023-06-08 10:14 ` Maxim Uvarov
1 sibling, 1 reply; 37+ messages in thread
From: Tom Rini @ 2023-06-07 20:07 UTC (permalink / raw)
To: Simon Goldschmidt
Cc: Maxim Uvarov, Ilias Apalodimas, u-boot, pbrobinson,
joe.hershberger, rfried.dev, lwip-devel
[-- Attachment #1: Type: text/plain, Size: 9172 bytes --]
On Wed, May 24, 2023 at 10:18:13PM +0200, Simon Goldschmidt wrote:
> Hi Maxim, Tom,
>
> On 24.05.2023 16:05, Maxim Uvarov wrote:
> > On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
> >
> > > On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
> > > > On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > > On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> > > > > > Hi Maxim
> > > > > >
> > > > > > On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
> > > > > wrote:
> > > > > > >
> > > > > > > My measurements for binary after LTO looks like:
> > > > > > >
> > > > > > > U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> > > > > > > 870728 | 915000 | 912560 |
> > > > > 41832 | 4.8
> > > > > >
> > > > > >
> > > > > > I think you'll need to analyze that a bit more. First of all I don't
> > > > > > think the '+ping' tab is useful. What is is trying to achieve?
> > > > >
> > > >
> > > > To show the difference of extra bytes if we add a ping app on top.
> > > >
> > > >
> > > > > > - How was LWIP compiled?
> > > > >
> > > >
> > > > It has a really huge configuration. I tried to turn off everything off
> > > > everything what can impact on size but still make http app work:
> > > > #define LWIP_HAVE_LOOPIF 0
> > > > #define LWIP_NETCONN 0
> > > > #define LWIP_SOCKET 0
> > > > #define SO_REUSE 0
> > > > #define LWIP_STATS 0
> > > > #define PPP_SUPPORT 0
> > > >
> > > > Disabling loopback:
> > > > #define LWIP_NETIF_LOOPBACK 0
> > > > can lower to 912288 bytes.
> > > >
> > > > And it's the same compilation option (optimization for size) as the main
> > > > u-boot. I will do more experiments, but I think the goal is not to turn
> > > off
> > > > everything.
> > > >
> > > >
> > > > > > - Was ipv6 supported?
> > > > >
> > > >
> > > > No. I.e. when I sent results it was enabled on the compilation stage but
> > > > not used. I just checked that size remains the same if IPv6 is not even
> > > > compiled.
> > > >
> > > >
> > > > > > - Can we strip it down even further?
> > > > > >
> > > > >
> > > >
> > > > There is always room for optimization. I think I tried to turn off
> > > > everything that is configurable with defines. I can play with disable IP
> > > > reassembly and things like that or figure out which functions have more
> > > > size and if it's possible to exclude them.
> > > >
> > > >
> > > > > > In general please give as much information as you can with what we
> > > > > > gain in functionality from LWIP with those extra bytes of code.
> > > > >
> > > > >
> > > > The main idea is to reuse a maintainable IP stack outside of U-boot.
> > > LWIP
> > > > can give a nice separation between IP stack code and network application
> > > > code. I.e. application should not take care about any TCP details (SYN,
> > > > ACK, retransmission, reassembly etc) and should open connection and use
> > > > functions similar to recv() and send() to transfer data. Data means
> > > > application data, no network packets. And LWIP allows
> > > > us to do that.
> > > > Because LWIP has an API similar to sockets, it has to be very easy to
> > > port
> > > > a linux application to LWIP. Then you can test it with a tap device. Then
> > > > copy sources to U-boot, add a small integration layer (cmd command to
> > > > call), compile and use.
> > > >
> > > > So my suggestion was:
> > > > - do not maintain new network stack code in the current U-boot. Use lwip
> > > > sources as an external project. All bugs related to network stack go to
> > > > lwip project first, then sync with U-boot.
> > > > - maintain network apps code* or
> > > > -- inside U-boot. Write our own code for application and maintain it
> > > > inside U-boot.
> > > > -- inside LWIP. Add examples to LWIP which are suitable for both
> > > U-boot
> > > > and LWIP.
> > > >
> > > > * Let's define a U-boot network application as a cmd command. It might be
> > > > ping, wget (http or https download), telnet, arp dns etc..
> > > >
> > > > Let's consider the real use case, like HTTPS download client. We need to
> > > > enable TLS connection, validate certificates, then do http download.
> > > > Looking at the current code of wget command it's quite difficult to
> > > > implement this due to the application having some protol level things. On
> > > > the other side we can find embedTLS examples to do https download on
> > > > sockets. If LWIP socket API is ported then the only thing you need to do
> > > is
> > > > change socket() -> lwip_socket(), recv()->lwip_recv(),
> > > send()->lwip_send()
> > > > and etc, even function names are similar. If LWIP socket API is not
> > > > supported, then use callback API for recv() and send(), which are also
> > > > easy.
> > > >
> > > > So yes we add extra bytes, but that will allow us to write more complex
> > > > apps, use standard debug tools, use applications with very minimal
> > > > integration changes, use help from the LWIP community to fix protocol
> > > bugs,
> > > > etc..
> > > > Bunch of things already implemented there:
> > > > - ipv6
> > > > - dhcp
> > > > - snmp
> > > > - igmp
> > > > - dns
> > > > - tcp and udp and raw.
> > > > - loopback
> > > > - netconn
> > > > - socket
> > > > - stats
> > > > - ppp
> > > > (I just followed configurable defines).
> > > >
> > > >
> > > > And please make sure to disable the previous support, my guess fro that
> > > > > much growth is that you didn't.
> > > > >
> > > >
> > > > # CONFIG_PROT_TCP is not set
> > > > # CONFIG_PROT_UDP is not set
> > > > # CONFIG_UDP_CHECKSUM is not set
> > > > # CONFIG_UDP_FUNCTION_FASTBOOT is not set
> > > > # CONFIG_CMD_WGET is not set
> > >
> > > I think you need to step back and figure out a better way to measure the
> > > size change and growth.
> > >
> > > I am not interested in a path that long term means two networking stacks
> > > in U-Boot.
> > >
> > > I am not interested in massively growing the overall binary size for
> > > every platform. Given how much larger just TCP support is, that's
> > > strongly implying a huge growth for the older use cases too.
> > >
> > > But I also suspect given the overall reputation that LWIP enjoys,
> > > there's something amiss here.
> > >
> > > --
> > > Tom
> > >
> >
> > +cc lwip-devel@ mailing list, maybe they have something to add.
>
> I do think using lwIP instead of "inventing yet another IP stack" is a
> good idea! However, in terms of code size, lwIP will lose against what's
> in U-Boot at present. And this is only natural, as lwIP is a "full-size"
> stack supporting multiple concurrently running applications while the
> current IP stack in U-Boot is rather "crippled" down to just what the
> implementor needed at the time of writing.
>
> One example of this is that (if I remember correctly), U-Boot only has
> one single network packet buffer, while lwIP has support for multiple
> buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
> of that development in U-Boot about 3 years ago), we're comparing "we
> have implemented everything we need so that it just kind of works" to
> "we can easily add a HTTPS client to download something over the
> internet just by enabling more compile options".
>
> Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
> U-Boot TCP (at least that of some years ago) is far from complete when
> compared to lwIP!
>
> lwIP is meant to be highly configurable and we're always open to add yet
> more options to leave out more code when it's not needed. However, I
> think there are some design decisions that will make lwIP larger than
> the current IP stack in U-Boot. To me, that's a natural result of having
> a "generic code" approach vs "developed to our needs". However, while
> DHCP + BOOTP and even a simple network console was rather easy to
> implement, I would not recommend implementing your own HTTPS download
> but rather using the existing lwIP + apps for that.
>
> In the end, I cannot take the decision from you. In my opinion, lwIP
> would be the better decision in terms of future work load and
> compatibility, but in the short run, it *will* lead to bigger binaries
> at least in some setups. And I do know from my past that it sometimes
> has been a pain to try and stuff a new U-Boot release into the existing
> space of flash or RAM, so that's not an easy decision.
>
> If you do take the lwIP approach however, let us know if we can help!
Given Maxim's more recent experiments, I'm sure we can come up with
something that works overall. There's hopefully a place or two U-Boot
people can help introduce a tunable or two to lwIP to bring some sizes
down. But I think it's overall looking to be the right direction.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-06-07 20:07 ` Tom Rini
@ 2023-06-08 10:14 ` Maxim Uvarov
2023-06-08 17:52 ` Ilias Apalodimas
0 siblings, 1 reply; 37+ messages in thread
From: Maxim Uvarov @ 2023-06-08 10:14 UTC (permalink / raw)
To: Tom Rini
Cc: Simon Goldschmidt, Ilias Apalodimas, u-boot, pbrobinson,
joe.hershberger, rfried.dev, lwip-devel
Ilias asked to make more clear results to compare the original stack and
LWIP stack. So the difference between the current U-boot stack and the LWIP
stack with 3 network commands is:
a) 18Kb - ls -lh size
b) 15Kb - bloat-o-meter script total line report.
BOM=linux/scripts/bloat-o-meter (script)
1. 893K - U-boot CMD_NET=n
2. 928K - U-boot CMD_NET=y TFTP=y PING=y WGET=y
BOM 1-2: Total: Before=692286, After=722283, chg +4.33%
3. 940K - U-boot CMD_NET=n, LWIP_TFTP=y LWIP_PING=y LWIP_PING=y
BOM 1-3: Total: Before=692286, After=738425, chg +6.66%
BOM 2-3:
add/remove: 287/203 grow/shrink: 3/11 up/down: 43459/-27317 (16142)
Function old new delta
tcp_input - 3588 +3588
tcp_receive - 2884 +2884
ip4_reass - 1760 +1760
tcp_output - 1400 +1400
tcp_write - 1300 +1300
tcp_slowtmr - 1172 +1172
httpc_tcp_recv - 1044 +1044
tftp_recv - 888 +888
ip4_input - 700 +700
ip4_frag - 632 +632
icmp_input - 604 +604
udp_input - 596 +596
etharp_input - 512 +512
tcp_split_unsent_seg - 500 +500
ip4addr_aton - 492 +492
tcp_alloc - 484 +484
ip4_output_if_src - 476 +476
tcp_close_shutdown - 448 +448
etharp_query - 436 +436
httpc_init_connection_common.constprop - 416 +416
udp_sendto_if_src - 408 +408
etharp_output - 404 +404
arp_table - 400 +400
tcp_connect - 396 +396
pbuf_alloc - 376 +376
etharp_find_entry - 372 +372
tcp_abandon - 368 +368
tcp_zero_window_probe - 356 +356
raw_sendto_if_src - 328 +328
pbuf_copy_partial_pbuf - 328 +328
ip_reass_free_complete_datagram - 328 +328
tcp_create_segment - 300 +300
raw_input - 292 +292
uboot_lwip_init - 284 +284
ethernet_input - 284 +284
etharp_raw - 284 +284
tcp_output_alloc_header_common.constprop - 280 +280
cmds - 280 +280
udp_bind - 276 +276
tcp_oos_insert_segment - 276 +276
ip_reass_remove_oldest_datagram - 272 +272
icmp_send_response - 268 +268
netif_add - 260 +260
ping_send - 244 +244
tcp_rexmit - 232 +232
tcp_parseopt - 220 +220
tcp_free_acked_segments.constprop - 220 +220
send_request - 220 +220
inet_chksum_pseudo - 216 +216
ip4addr_ntoa_r - 212 +212
do_lwip_ping - 212 +212
tcp_enqueue_flags - 208 +208
etharp_output_to_arp_index - 208 +208
netif_set_addr - 204 +204
tcp_fasttmr - 200 +200
tcp_rexmit_rto_prepare - 196 +196
tcp_process_refused_data - 196 +196
send_data - 196 +196
lwip_wget - 192 +192
ethernet_output - 192 +192
ping_recv - 188 +188
pbuf_memcmp - 184 +184
pbuf_copy_partial - 184 +184
httpc_free_state - 180 +180
tcp_send_fin - 172 +172
httpc_recv - 168 +168
tcp_output_control_segment_netif - 164 +164
send_error.isra - 164 +164
do_ops - 164 +164
raw_sendto - 160 +160
pbuf_realloc - 160 +160
pbuf_free - 160 +160
do_lwip_wget - 160 +160
do_lwip_tftp - 160 +160
tftp_init_common - 156 +156
tcp_rst_netif - 152 +152
udp_sendto - 144 +144
tftp_tmr - 144 +144
tcp_rst - 144 +144
uboot_lwip_if_init - 140 +140
tcp_pcb_remove - 140 +140
tcp_pbuf_prealloc - 140 +140
sys_timeout_abs - 140 +140
lwip_tftp - 140 +140
netif_do_set_ipaddr.isra - 136 +136
ip4_route - 136 +136
tcp_netif_ip_addr_changed - 132 +132
resend_data.isra - 132 +132
inet_chksum_pbuf - 132 +132
tcp_output_control_segment - 128 +128
pbuf_memfind - 128 +128
lwip_standard_chksum - 128 +128
tcp_rexmit_fast - 124 +124
tcp_new_port - 124 +124
tcp_close_shutdown_fin - 124 +124
pbuf_add_header_impl - 124 +124
tcp_send_empty_ack - 120 +120
httpc_create_request_string.constprop.isra - 120 +120
tftp_get - 116 +116
tcp_recved - 116 +116
tcp_pcb_purge - 116 +116
tftp_write - 112 +112
pbuf_free_header - 112 +112
httpc_tcp_connected - 112 +112
tftp_error - 108 +108
send_ack.isra - 108 +108
low_level_input.constprop - 108 +108
tcp_input_delayed_close - 104 +104
close_handle - 100 +100
sys_untimeout - 96 +96
memp_pools - 96 +96
tcp_keepalive - 92 +92
ip4_addr_isbroadcast_u32 - 92 +92
init_packet - 92 +92
tcp_kill_state - 88 +88
raw_new - 88 +88
ping_raw_init - 88 +88
lwip_ping_init - 88 +88
udp_sendto_if - 84 +84
tcp_update_rcv_ann_wnd - 84 +84
tcp_recv_null - 84 +84
pbuf_remove_header - 84 +84
pbuf_alloc_reference - 84 +84
udp_remove - 80 +80
tcp_get_next_optbyte - 80 +80
pbuf_alloced_custom - 80 +80
ip4_input_accept - 80 +80
httpc_close - 80 +80
etharp_free_entry - 80 +80
uboot_lwip_poll - 76 +76
tcpip_tcp_timer - 76 +76
udp_netif_ip_addr_changed - 72 +72
uboot_netif - 72 +72
tcp_output_alloc_header.constprop - 72 +72
raw_netif_ip_addr_changed - 72 +72
tcpip_try_callback - 68 +68
tcp_timer_needed - 68 +68
tcp_seg_copy - 68 +68
tcp_netif_ip_addr_changed_pcblist - 68 +68
ping_timeout - 68 +68
ethernetif_input - 68 +68
udp_new - 64 +64
pbuf_try_get_at - 64 +64
sys_timeout - 60 +60
pbuf_clone - 60 +60
tcp_seg_free - 56 +56
pbuf_cat - 56 +56
netif_get_by_index - 56 +56
low_level_output - 56 +56
_u_boot_list_2_cmd_2_lwipinfo - 56 +56
_u_boot_list_2_cmd_2_lwip - 56 +56
tftp_state 4 56 +52
tcp_tmr - 52 +52
tcp_rexmit_rto - 52 +52
tcp_segs_free - 48 +48
tcp_eff_send_mss_netif - 48 +48
pbuf_skip_const - 48 +48
ipfrag_free_pbuf_custom - 48 +48
httpc_tcp_poll - 48 +48
tcp_free_ooseq - 44 +44
tcp_close - 44 +44
pbuf_free_ooseq_callback - 44 +44
netif_issue_reports - 44 +44
ip_reass_dequeue_datagram - 44 +44
httpc_get_internal_addr - 44 +44
tftp_read - 40 +40
tftp - 40 +40
ip_data - 40 +40
etharp_request - 40 +40
do_lwip_info - 40 +40
ulwip_timeout_handler - 36 +36
raw_bind - 36 +36
memp_malloc - 36 +36
ip4_output_if - 36 +36
tcp_pcb_lists - 32 +32
pbuf_header_force - 32 +32
pbuf_clen - 32 +32
netif_set_up - 32 +32
netif_set_link_up - 32 +32
inseg - 32 +32
inet_chksum - 32 +32
tcp_next_iss - 28 +28
pbuf_get_at - 28 +28
httpc_tcp_err - 28 +28
do_lwip_init - 28 +28
tcp_rexmit_rto_commit - 24 +24
sys_now - 24 +24
settings - 24 +24
pbuf_copy - 24 +24
pbuf_chain - 24 +24
memp_free - 24 +24
__func__ 1243 1266 +23
ulwip_exit - 20 +20
tcp_trigger_input_pcb_close - 20 +20
tcp_poll - 20 +20
ping_send_now - 20 +20
pbuf_ref - 20 +20
str - 16 +16
ip4addr_ntoa - 16 +16
daddr - 16 +16
tcp_backoff - 13 +13
ulwip_loop_set - 12 +12
ulwip_in_loop - 12 +12
ulwip_enabled - 12 +12
ulwip_app_get_err - 12 +12
udp_recv - 12 +12
tftp_init_client - 12 +12
tcp_sent - 12 +12
tcp_recv - 12 +12
tcp_free - 12 +12
tcp_err - 12 +12
tcp_arg - 12 +12
net_process_received_packet 800 812 +12
icmp_time_exceeded - 12 +12
icmp_dest_unreach - 12 +12
udp_pcbs - 8 +8
tftp_open - 8 +8
tftp_close - 8 +8
tcphdr_opt2 - 8 +8
tcphdr - 8 +8
tcp_tw_pcbs - 8 +8
tcp_new - 8 +8
tcp_listen_pcbs - 8 +8
tcp_input_pcb - 8 +8
tcp_bound_pcbs - 8 +8
tcp_active_pcbs - 8 +8
tcp_abort - 8 +8
recv_data - 8 +8
reassdatagrams - 8 +8
raw_recv - 8 +8
raw_pcbs - 8 +8
ping_target - 8 +8
ping_pcb - 8 +8
pbuf_add_header - 8 +8
next_timeout - 8 +8
netif_null_output_ip4 - 8 +8
netif_list - 8 +8
netif_default - 8 +8
lwip_htons - 8 +8
lwip_htonl - 8 +8
httpc_tcp_sent - 8 +8
tcp_persist_backoff - 7 +7
ethzero - 6 +6
ethbroadcast - 6 +6
ulwip_app_err - 4 +4
udp_new_ip_type - 4 +4
uboot_net_use_lwip - 4 +4
tcpip_tcp_timer_active - 4 +4
tcp_ticks - 4 +4
seqno - 4 +4
mem_trim - 4 +4
mem_malloc - 4 +4
mem_free - 4 +4
loop_lwip - 4 +4
iss - 4 +4
ip_target - 4 +4
ip_chksum_pseudo - 4 +4
ip_addr_any - 4 +4
httpc_init_connection - 4 +4
ackno - 4 +4
udp_port - 2 +2
tcplen - 2 +2
tcphdr_optlen - 2 +2
tcphdr_opt1len - 2 +2
tcp_port - 2 +2
tcp_optidx - 2 +2
recv_acked - 2 +2
ping_seq_num - 2 +2
memp_UDP_PCB - 2 +2
memp_TCP_SEG - 2 +2
memp_TCP_PCB_LISTEN - 2 +2
memp_TCP_PCB - 2 +2
memp_TCPIP_MSG_INPKT - 2 +2
memp_TCPIP_MSG_API - 2 +2
memp_SYS_TIMEOUT - 2 +2
memp_REASSDATA - 2 +2
memp_RAW_PCB - 2 +2
memp_PBUF_POOL - 2 +2
memp_PBUF - 2 +2
memp_FRAG_PBUF - 2 +2
ip_reass_pbufcount - 2 +2
ip_id - 2 +2
tcp_timer_ctr - 1 +1
tcp_timer - 1 +1
tcp_active_pcbs_changed - 1 +1
recv_flags - 1 +1
pbuf_free_ooseq_pending - 1 +1
netif_num - 1 +1
flags - 1 +1
etharp_cached_entry - 1 +1
supported_nfs_versions 1 - -1
retry_action 1 - -1
net_boot_file_name_explicit 1 - -1
dhcp_option_overload 1 - -1
tftp_windowsize 2 - -2
tftp_window_size_option 2 - -2
tftp_next_ack 2 - -2
tftp_last_nack 2 - -2
tftp_block_size_option 2 - -2
tftp_block_size 2 - -2
ping_seq_number 2 - -2
last_op 2 - -2
env_flags_vartype_rep 7 5 -2
linefeed 3 - -3
wget_timeout_count 4 - -4
wget_loop_state 4 - -4
web_server_ip 4 - -4
timeout_count_max 4 - -4
timeout_count 4 - -4
tftp_timeout_count_max 4 - -4
tftp_remote_port 4 - -4
tftp_remote_ip 4 - -4
tftp_our_port 4 - -4
saved_tftp_block_size_option 4 - -4
retry_tcp_seq_num 4 - -4
retry_tcp_ack_num 4 - -4
retry_len 4 - -4
pkt_q_idx 4 - -4
packets 4 - -4
our_port 4 - -4
nfs_timeout_count 4 - -4
nfs_state 4 - -4
nfs_server_port 4 - -4
nfs_server_mount_port 4 - -4
nfs_server_ip 4 - -4
nfs_our_port 4 - -4
nfs_offset 4 - -4
nfs_len 4 - -4
nfs_download_state 4 - -4
net_ping_ip 4 - -4
net_dns_server 4 - -4
net_boot_file_expected_size_in_blocks 4 - -4
last_reg_lo 4 - -4
last_reg_hi 4 - -4
last_mask 4 - -4
last_data 4 - -4
last_addr_lo 4 - -4
last_addr_hi 4 - -4
initial_data_seq_num 4 - -4
http_ok 4 - -4
fs_mounted 4 - -4
filefh3_length 4 - -4
eth_common_init 4 - -4
dummy_handler 8 4 -4
dhcp_state 4 - -4
dhcp_server_ip 4 - -4
dhcp_leasetime 4 - -4
current_wget_state 4 - -4
bootp_try 4 - -4
bootp_num_ids 4 - -4
http_eom 5 - -5
bootfile1 5 - -5
timeout_ms 8 - -8
time_taken_max 8 - -8
time_start 16 8 -8
tftp_prev_block 8 - -8
tftp_load_size 8 - -8
tftp_load_addr 8 - -8
tftp_cur_block 8 - -8
tftp_block_wrap_offset 8 - -8
tftp_block_wrap 8 - -8
rpc_id 8 - -8
nfs_path 8 - -8
nfs_filename 8 - -8
miiphy_is_1000base_x 8 - -8
init_sequence_r 264 256 -8
image_url 8 - -8
distro_pxe_check 8 - -8
current_mii 8 - -8
content_length 8 - -8
bootp_timeout 8 - -8
bootp_start 8 - -8
tcp_get_tcp_state 12 - -12
do_wget 12 - -12
do_tftpb 12 - -12
do_nfs 12 - -12
do_dhcp 12 - -12
do_bootp 12 - -12
default_filename 13 - -13
bootfile3 14 - -14
content_len 15 - -15
reg_2_desc_tbl 16 - -16
pkt_q 16 - -16
mii_devs 16 - -16
bootp_ids 16 - -16
miiphy_get_current_dev 20 - -20
tcp_set_tcp_handler 24 - -24
pxe_default_paths 24 - -24
net_set_udp_handler 24 - -24
net_check_prereq 256 232 -24
miiphy_init 28 - -28
ping_timeout_handler 32 - -32
net_nis_domain 32 - -32
net_hostname 32 - -32
distro_bootmeth_pxe_ids 32 - -32
dirfh 32 - -32
initr_net 36 - -36
distro_bootmeth_pxe_bind 36 - -36
ip_to_string 40 - -40
distro_bootmeth_pxe_ops 40 - -40
net_send_udp_packet 44 - -44
label_boot 1944 1900 -44
env_flags_validate 632 588 -44
reg_3_desc_tbl 48 - -48
do_get_tftp 56 - -56
cmd_net 56 - -56
_u_boot_list_2_cmd_2_wget 56 - -56
_u_boot_list_2_cmd_2_tftpboot 56 - -56
_u_boot_list_2_cmd_2_pxe 56 - -56
_u_boot_list_2_cmd_2_ping 56 - -56
_u_boot_list_2_cmd_2_nfs 56 - -56
_u_boot_list_2_cmd_2_net 56 - -56
_u_boot_list_2_cmd_2_mii 56 - -56
_u_boot_list_2_cmd_2_dhcp 56 - -56
_u_boot_list_2_cmd_2_bootp 56 - -56
net_loop 652 592 -60
net_eth_hdr_size 60 - -60
bootp_reset 60 - -60
net_root_path 64 - -64
filefh 64 - -64
do_bootvx 816 748 -68
miiphy_set_current_dev 72 - -72
basename 72 - -72
pxe_get_file_size 76 - -76
copy_filename 76 - -76
distro_pxe_getfile 80 - -80
tftp_init_load_addr 92 - -92
miiphy_read 92 - -92
extract_range 92 - -92
miiphy_write 96 - -96
miiphy_get_active_dev 96 - -96
distro_pxe_read_file 96 - -96
wget_fail 104 - -104
skip_num 104 - -104
miiphy_get_dev_by_name 104 - -104
dump_field 104 - -104
do_bdinfo 432 328 -104
bootp_timeout_handler 104 - -104
nfs_timeout_handler 108 - -108
cmd_pxe_sub 112 - -112
nfs_umountall_req 120 - -120
_u_boot_list_2_driver_2_bootmeth_pxe 120 - -120
do_ping 124 - -124
tftp_filename 128 - -128
reg_9_desc_tbl 128 - -128
reg_10_desc_tbl 128 - -128
distro_pxe_boot 128 - -128
tftp_timeout_handler 132 - -132
do_pxe 132 - -132
nfs_umountall_reply 136 - -136
lmb_get_free_size 136 - -136
format_mac_pxe 136 - -136
miiphy_listdev 144 - -144
efi_net_set_dhcp_ack 144 - -144
wget_timeout_handler 148 - -148
nfs_mount_reply 148 - -148
dhcp_packet_process_options 148 - -148
eth_validate_ethaddr_str 152 - -152
do_pxe_get 156 - -156
reg_0_desc_tbl 160 - -160
net_parse_bootfile 160 - -160
miiphy_info 160 - -160
get_pxelinux_path 160 - -160
do_net 164 - -164
net_auto_load 172 - -172
do_net_list 176 - -176
rpc_lookup_reply 180 - -180
nfs_readlink_req 184 - -184
nfs_mount_req 188 - -188
reg_5_desc_tbl 192 - -192
reg_4_desc_tbl 192 - -192
miiphy_speed 200 - -200
miiphy_duplex 200 - -200
nfs_read_req 224 - -224
do_pxe_boot 248 - -248
reg_1_desc_tbl 256 - -256
mii_reg_desc_tbl 256 - -256
nfs_send 260 - -260
wget_start 268 - -268
ping_start 276 - -276
nfs_lookup_reply 280 - -280
rpc_req 300 - -300
eth_initialize 300 - -300
distro_pxe_read_bootflow 300 - -300
nfs_readlink_reply 328 - -328
nfs_lookup_req 328 - -328
ping_receive 332 - -332
pxe_get 376 - -376
nfs_read_reply 396 - -396
wget_send_stored 444 - -444
nfs_start 468 - -468
dhcp_process_options 508 - -508
tftp_send 560 - -560
nfs_handler 580 - -580
bootp_request 612 - -612
dhcp_extended 616 - -616
netboot_common 632 - -632
default_environment 4444 3800 -644
tftp_start 912 - -912
dhcp_handler 1000 - -1000
wget_handler 1092 - -1092
tftp_handler 1304 - -1304
nfs_path_buff 2048 - -2048
do_mii 2124 - -2124
Total: Before=722283, After=738425, chg +2.23%
On Thu, 8 Jun 2023 at 02:07, Tom Rini <trini@konsulko.com> wrote:
> On Wed, May 24, 2023 at 10:18:13PM +0200, Simon Goldschmidt wrote:
> > Hi Maxim, Tom,
> >
> > On 24.05.2023 16:05, Maxim Uvarov wrote:
> > > On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > > On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
> > > > > On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > > On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> > > > > > > Hi Maxim
> > > > > > >
> > > > > > > On Mon, 22 May 2023 at 12:01, Maxim Uvarov <
> maxim.uvarov@linaro.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > My measurements for binary after LTO looks like:
> > > > > > > >
> > > > > > > > U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes|
> diff %
> > > > > > > > 870728 | 915000 | 912560
> |
> > > > > > 41832 | 4.8
> > > > > > >
> > > > > > >
> > > > > > > I think you'll need to analyze that a bit more. First of all
> I don't
> > > > > > > think the '+ping' tab is useful. What is is trying to achieve?
> > > > > >
> > > > >
> > > > > To show the difference of extra bytes if we add a ping app on top.
> > > > >
> > > > >
> > > > > > > - How was LWIP compiled?
> > > > > >
> > > > >
> > > > > It has a really huge configuration. I tried to turn off everything
> off
> > > > > everything what can impact on size but still make http app work:
> > > > > #define LWIP_HAVE_LOOPIF 0
> > > > > #define LWIP_NETCONN 0
> > > > > #define LWIP_SOCKET 0
> > > > > #define SO_REUSE 0
> > > > > #define LWIP_STATS 0
> > > > > #define PPP_SUPPORT 0
> > > > >
> > > > > Disabling loopback:
> > > > > #define LWIP_NETIF_LOOPBACK 0
> > > > > can lower to 912288 bytes.
> > > > >
> > > > > And it's the same compilation option (optimization for size) as
> the main
> > > > > u-boot. I will do more experiments, but I think the goal is not to
> turn
> > > > off
> > > > > everything.
> > > > >
> > > > >
> > > > > > > - Was ipv6 supported?
> > > > > >
> > > > >
> > > > > No. I.e. when I sent results it was enabled on the compilation
> stage but
> > > > > not used. I just checked that size remains the same if IPv6 is not
> even
> > > > > compiled.
> > > > >
> > > > >
> > > > > > > - Can we strip it down even further?
> > > > > > >
> > > > > >
> > > > >
> > > > > There is always room for optimization. I think I tried to turn off
> > > > > everything that is configurable with defines. I can play with
> disable IP
> > > > > reassembly and things like that or figure out which functions have
> more
> > > > > size and if it's possible to exclude them.
> > > > >
> > > > >
> > > > > > > In general please give as much information as you can with
> what we
> > > > > > > gain in functionality from LWIP with those extra bytes of code.
> > > > > >
> > > > > >
> > > > > The main idea is to reuse a maintainable IP stack outside of
> U-boot.
> > > > LWIP
> > > > > can give a nice separation between IP stack code and network
> application
> > > > > code. I.e. application should not take care about any TCP
> details (SYN,
> > > > > ACK, retransmission, reassembly etc) and should open connection
> and use
> > > > > functions similar to recv() and send() to transfer data. Data means
> > > > > application data, no network packets. And LWIP allows
> > > > > us to do that.
> > > > > Because LWIP has an API similar to sockets, it has to be very easy
> to
> > > > port
> > > > > a linux application to LWIP. Then you can test it with a tap
> device. Then
> > > > > copy sources to U-boot, add a small integration layer (cmd command
> to
> > > > > call), compile and use.
> > > > >
> > > > > So my suggestion was:
> > > > > - do not maintain new network stack code in the current U-boot.
> Use lwip
> > > > > sources as an external project. All bugs related to network stack
> go to
> > > > > lwip project first, then sync with U-boot.
> > > > > - maintain network apps code* or
> > > > > -- inside U-boot. Write our own code for application and
> maintain it
> > > > > inside U-boot.
> > > > > -- inside LWIP. Add examples to LWIP which are suitable for both
> > > > U-boot
> > > > > and LWIP.
> > > > >
> > > > > * Let's define a U-boot network application as a cmd command. It
> might be
> > > > > ping, wget (http or https download), telnet, arp dns etc..
> > > > >
> > > > > Let's consider the real use case, like HTTPS download client. We
> need to
> > > > > enable TLS connection, validate certificates, then do http
> download.
> > > > > Looking at the current code of wget command it's quite difficult to
> > > > > implement this due to the application having some protol level
> things. On
> > > > > the other side we can find embedTLS examples to do https download
> on
> > > > > sockets. If LWIP socket API is ported then the only thing you need
> to do
> > > > is
> > > > > change socket() -> lwip_socket(), recv()->lwip_recv(),
> > > > send()->lwip_send()
> > > > > and etc, even function names are similar. If LWIP socket API is not
> > > > > supported, then use callback API for recv() and send(), which are
> also
> > > > > easy.
> > > > >
> > > > > So yes we add extra bytes, but that will allow us to write more
> complex
> > > > > apps, use standard debug tools, use applications with very minimal
> > > > > integration changes, use help from the LWIP community to fix
> protocol
> > > > bugs,
> > > > > etc..
> > > > > Bunch of things already implemented there:
> > > > > - ipv6
> > > > > - dhcp
> > > > > - snmp
> > > > > - igmp
> > > > > - dns
> > > > > - tcp and udp and raw.
> > > > > - loopback
> > > > > - netconn
> > > > > - socket
> > > > > - stats
> > > > > - ppp
> > > > > (I just followed configurable defines).
> > > > >
> > > > >
> > > > > And please make sure to disable the previous support, my guess fro
> that
> > > > > > much growth is that you didn't.
> > > > > >
> > > > >
> > > > > # CONFIG_PROT_TCP is not set
> > > > > # CONFIG_PROT_UDP is not set
> > > > > # CONFIG_UDP_CHECKSUM is not set
> > > > > # CONFIG_UDP_FUNCTION_FASTBOOT is not set
> > > > > # CONFIG_CMD_WGET is not set
> > > >
> > > > I think you need to step back and figure out a better way to measure
> the
> > > > size change and growth.
> > > >
> > > > I am not interested in a path that long term means two networking
> stacks
> > > > in U-Boot.
> > > >
> > > > I am not interested in massively growing the overall binary size for
> > > > every platform. Given how much larger just TCP support is, that's
> > > > strongly implying a huge growth for the older use cases too.
> > > >
> > > > But I also suspect given the overall reputation that LWIP enjoys,
> > > > there's something amiss here.
> > > >
> > > > --
> > > > Tom
> > > >
> > >
> > > +cc lwip-devel@ mailing list, maybe they have something to add.
> >
> > I do think using lwIP instead of "inventing yet another IP stack" is a
> > good idea! However, in terms of code size, lwIP will lose against what's
> > in U-Boot at present. And this is only natural, as lwIP is a "full-size"
> > stack supporting multiple concurrently running applications while the
> > current IP stack in U-Boot is rather "crippled" down to just what the
> > implementor needed at the time of writing.
> >
> > One example of this is that (if I remember correctly), U-Boot only has
> > one single network packet buffer, while lwIP has support for multiple
> > buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
> > of that development in U-Boot about 3 years ago), we're comparing "we
> > have implemented everything we need so that it just kind of works" to
> > "we can easily add a HTTPS client to download something over the
> > internet just by enabling more compile options".
> >
> > Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
> > U-Boot TCP (at least that of some years ago) is far from complete when
> > compared to lwIP!
> >
> > lwIP is meant to be highly configurable and we're always open to add yet
> > more options to leave out more code when it's not needed. However, I
> > think there are some design decisions that will make lwIP larger than
> > the current IP stack in U-Boot. To me, that's a natural result of having
> > a "generic code" approach vs "developed to our needs". However, while
> > DHCP + BOOTP and even a simple network console was rather easy to
> > implement, I would not recommend implementing your own HTTPS download
> > but rather using the existing lwIP + apps for that.
> >
> > In the end, I cannot take the decision from you. In my opinion, lwIP
> > would be the better decision in terms of future work load and
> > compatibility, but in the short run, it *will* lead to bigger binaries
> > at least in some setups. And I do know from my past that it sometimes
> > has been a pain to try and stuff a new U-Boot release into the existing
> > space of flash or RAM, so that's not an easy decision.
> >
> > If you do take the lwIP approach however, let us know if we can help!
>
> Given Maxim's more recent experiments, I'm sure we can come up with
> something that works overall. There's hopefully a place or two U-Boot
> people can help introduce a tunable or two to lwIP to bring some sizes
> down. But I think it's overall looking to be the right direction.
>
> --
> Tom
>
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-06-08 10:14 ` Maxim Uvarov
@ 2023-06-08 17:52 ` Ilias Apalodimas
2023-06-09 7:37 ` Peter Robinson
0 siblings, 1 reply; 37+ messages in thread
From: Ilias Apalodimas @ 2023-06-08 17:52 UTC (permalink / raw)
To: Maxim Uvarov
Cc: Tom Rini, Simon Goldschmidt, u-boot, pbrobinson, joe.hershberger,
rfried.dev, lwip-devel
Thanks Maxim,
On Thu, 8 Jun 2023 at 13:14, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>
> Ilias asked to make more clear results to compare the original stack and LWIP stack. So the difference between the current U-boot stack and the LWIP stack with 3 network commands is:
> a) 18Kb - ls -lh size
> b) 15Kb - bloat-o-meter script total line report.
>
> BOM=linux/scripts/bloat-o-meter (script)
>
> 1. 893K - U-boot CMD_NET=n
> 2. 928K - U-boot CMD_NET=y TFTP=y PING=y WGET=y
> BOM 1-2: Total: Before=692286, After=722283, chg +4.33%
> 3. 940K - U-boot CMD_NET=n, LWIP_TFTP=y LWIP_PING=y LWIP_PING=y
> BOM 1-3: Total: Before=692286, After=738425, chg +6.66%
That's much more readable! I discussed this with Tom over IRC and the
size difference is certainly a reasonable trade-off for the extra
functionality.
Can you tidy up the series and include DHCP and PXE done through LWIP?
Thanks
/Ilias
>
> BOM 2-3:
>
> add/remove: 287/203 grow/shrink: 3/11 up/down: 43459/-27317 (16142)
> Function old new delta
> tcp_input - 3588 +3588
> tcp_receive - 2884 +2884
> ip4_reass - 1760 +1760
> tcp_output - 1400 +1400
> tcp_write - 1300 +1300
> tcp_slowtmr - 1172 +1172
> httpc_tcp_recv - 1044 +1044
> tftp_recv - 888 +888
> ip4_input - 700 +700
> ip4_frag - 632 +632
> icmp_input - 604 +604
> udp_input - 596 +596
> etharp_input - 512 +512
> tcp_split_unsent_seg - 500 +500
> ip4addr_aton - 492 +492
> tcp_alloc - 484 +484
> ip4_output_if_src - 476 +476
> tcp_close_shutdown - 448 +448
> etharp_query - 436 +436
> httpc_init_connection_common.constprop - 416 +416
> udp_sendto_if_src - 408 +408
> etharp_output - 404 +404
> arp_table - 400 +400
> tcp_connect - 396 +396
> pbuf_alloc - 376 +376
> etharp_find_entry - 372 +372
> tcp_abandon - 368 +368
> tcp_zero_window_probe - 356 +356
> raw_sendto_if_src - 328 +328
> pbuf_copy_partial_pbuf - 328 +328
> ip_reass_free_complete_datagram - 328 +328
> tcp_create_segment - 300 +300
> raw_input - 292 +292
> uboot_lwip_init - 284 +284
> ethernet_input - 284 +284
> etharp_raw - 284 +284
> tcp_output_alloc_header_common.constprop - 280 +280
> cmds - 280 +280
> udp_bind - 276 +276
> tcp_oos_insert_segment - 276 +276
> ip_reass_remove_oldest_datagram - 272 +272
> icmp_send_response - 268 +268
> netif_add - 260 +260
> ping_send - 244 +244
> tcp_rexmit - 232 +232
> tcp_parseopt - 220 +220
> tcp_free_acked_segments.constprop - 220 +220
> send_request - 220 +220
> inet_chksum_pseudo - 216 +216
> ip4addr_ntoa_r - 212 +212
> do_lwip_ping - 212 +212
> tcp_enqueue_flags - 208 +208
> etharp_output_to_arp_index - 208 +208
> netif_set_addr - 204 +204
> tcp_fasttmr - 200 +200
> tcp_rexmit_rto_prepare - 196 +196
> tcp_process_refused_data - 196 +196
> send_data - 196 +196
> lwip_wget - 192 +192
> ethernet_output - 192 +192
> ping_recv - 188 +188
> pbuf_memcmp - 184 +184
> pbuf_copy_partial - 184 +184
> httpc_free_state - 180 +180
> tcp_send_fin - 172 +172
> httpc_recv - 168 +168
> tcp_output_control_segment_netif - 164 +164
> send_error.isra - 164 +164
> do_ops - 164 +164
> raw_sendto - 160 +160
> pbuf_realloc - 160 +160
> pbuf_free - 160 +160
> do_lwip_wget - 160 +160
> do_lwip_tftp - 160 +160
> tftp_init_common - 156 +156
> tcp_rst_netif - 152 +152
> udp_sendto - 144 +144
> tftp_tmr - 144 +144
> tcp_rst - 144 +144
> uboot_lwip_if_init - 140 +140
> tcp_pcb_remove - 140 +140
> tcp_pbuf_prealloc - 140 +140
> sys_timeout_abs - 140 +140
> lwip_tftp - 140 +140
> netif_do_set_ipaddr.isra - 136 +136
> ip4_route - 136 +136
> tcp_netif_ip_addr_changed - 132 +132
> resend_data.isra - 132 +132
> inet_chksum_pbuf - 132 +132
> tcp_output_control_segment - 128 +128
> pbuf_memfind - 128 +128
> lwip_standard_chksum - 128 +128
> tcp_rexmit_fast - 124 +124
> tcp_new_port - 124 +124
> tcp_close_shutdown_fin - 124 +124
> pbuf_add_header_impl - 124 +124
> tcp_send_empty_ack - 120 +120
> httpc_create_request_string.constprop.isra - 120 +120
> tftp_get - 116 +116
> tcp_recved - 116 +116
> tcp_pcb_purge - 116 +116
> tftp_write - 112 +112
> pbuf_free_header - 112 +112
> httpc_tcp_connected - 112 +112
> tftp_error - 108 +108
> send_ack.isra - 108 +108
> low_level_input.constprop - 108 +108
> tcp_input_delayed_close - 104 +104
> close_handle - 100 +100
> sys_untimeout - 96 +96
> memp_pools - 96 +96
> tcp_keepalive - 92 +92
> ip4_addr_isbroadcast_u32 - 92 +92
> init_packet - 92 +92
> tcp_kill_state - 88 +88
> raw_new - 88 +88
> ping_raw_init - 88 +88
> lwip_ping_init - 88 +88
> udp_sendto_if - 84 +84
> tcp_update_rcv_ann_wnd - 84 +84
> tcp_recv_null - 84 +84
> pbuf_remove_header - 84 +84
> pbuf_alloc_reference - 84 +84
> udp_remove - 80 +80
> tcp_get_next_optbyte - 80 +80
> pbuf_alloced_custom - 80 +80
> ip4_input_accept - 80 +80
> httpc_close - 80 +80
> etharp_free_entry - 80 +80
> uboot_lwip_poll - 76 +76
> tcpip_tcp_timer - 76 +76
> udp_netif_ip_addr_changed - 72 +72
> uboot_netif - 72 +72
> tcp_output_alloc_header.constprop - 72 +72
> raw_netif_ip_addr_changed - 72 +72
> tcpip_try_callback - 68 +68
> tcp_timer_needed - 68 +68
> tcp_seg_copy - 68 +68
> tcp_netif_ip_addr_changed_pcblist - 68 +68
> ping_timeout - 68 +68
> ethernetif_input - 68 +68
> udp_new - 64 +64
> pbuf_try_get_at - 64 +64
> sys_timeout - 60 +60
> pbuf_clone - 60 +60
> tcp_seg_free - 56 +56
> pbuf_cat - 56 +56
> netif_get_by_index - 56 +56
> low_level_output - 56 +56
> _u_boot_list_2_cmd_2_lwipinfo - 56 +56
> _u_boot_list_2_cmd_2_lwip - 56 +56
> tftp_state 4 56 +52
> tcp_tmr - 52 +52
> tcp_rexmit_rto - 52 +52
> tcp_segs_free - 48 +48
> tcp_eff_send_mss_netif - 48 +48
> pbuf_skip_const - 48 +48
> ipfrag_free_pbuf_custom - 48 +48
> httpc_tcp_poll - 48 +48
> tcp_free_ooseq - 44 +44
> tcp_close - 44 +44
> pbuf_free_ooseq_callback - 44 +44
> netif_issue_reports - 44 +44
> ip_reass_dequeue_datagram - 44 +44
> httpc_get_internal_addr - 44 +44
> tftp_read - 40 +40
> tftp - 40 +40
> ip_data - 40 +40
> etharp_request - 40 +40
> do_lwip_info - 40 +40
> ulwip_timeout_handler - 36 +36
> raw_bind - 36 +36
> memp_malloc - 36 +36
> ip4_output_if - 36 +36
> tcp_pcb_lists - 32 +32
> pbuf_header_force - 32 +32
> pbuf_clen - 32 +32
> netif_set_up - 32 +32
> netif_set_link_up - 32 +32
> inseg - 32 +32
> inet_chksum - 32 +32
> tcp_next_iss - 28 +28
> pbuf_get_at - 28 +28
> httpc_tcp_err - 28 +28
> do_lwip_init - 28 +28
> tcp_rexmit_rto_commit - 24 +24
> sys_now - 24 +24
> settings - 24 +24
> pbuf_copy - 24 +24
> pbuf_chain - 24 +24
> memp_free - 24 +24
> __func__ 1243 1266 +23
> ulwip_exit - 20 +20
> tcp_trigger_input_pcb_close - 20 +20
> tcp_poll - 20 +20
> ping_send_now - 20 +20
> pbuf_ref - 20 +20
> str - 16 +16
> ip4addr_ntoa - 16 +16
> daddr - 16 +16
> tcp_backoff - 13 +13
> ulwip_loop_set - 12 +12
> ulwip_in_loop - 12 +12
> ulwip_enabled - 12 +12
> ulwip_app_get_err - 12 +12
> udp_recv - 12 +12
> tftp_init_client - 12 +12
> tcp_sent - 12 +12
> tcp_recv - 12 +12
> tcp_free - 12 +12
> tcp_err - 12 +12
> tcp_arg - 12 +12
> net_process_received_packet 800 812 +12
> icmp_time_exceeded - 12 +12
> icmp_dest_unreach - 12 +12
> udp_pcbs - 8 +8
> tftp_open - 8 +8
> tftp_close - 8 +8
> tcphdr_opt2 - 8 +8
> tcphdr - 8 +8
> tcp_tw_pcbs - 8 +8
> tcp_new - 8 +8
> tcp_listen_pcbs - 8 +8
> tcp_input_pcb - 8 +8
> tcp_bound_pcbs - 8 +8
> tcp_active_pcbs - 8 +8
> tcp_abort - 8 +8
> recv_data - 8 +8
> reassdatagrams - 8 +8
> raw_recv - 8 +8
> raw_pcbs - 8 +8
> ping_target - 8 +8
> ping_pcb - 8 +8
> pbuf_add_header - 8 +8
> next_timeout - 8 +8
> netif_null_output_ip4 - 8 +8
> netif_list - 8 +8
> netif_default - 8 +8
> lwip_htons - 8 +8
> lwip_htonl - 8 +8
> httpc_tcp_sent - 8 +8
> tcp_persist_backoff - 7 +7
> ethzero - 6 +6
> ethbroadcast - 6 +6
> ulwip_app_err - 4 +4
> udp_new_ip_type - 4 +4
> uboot_net_use_lwip - 4 +4
> tcpip_tcp_timer_active - 4 +4
> tcp_ticks - 4 +4
> seqno - 4 +4
> mem_trim - 4 +4
> mem_malloc - 4 +4
> mem_free - 4 +4
> loop_lwip - 4 +4
> iss - 4 +4
> ip_target - 4 +4
> ip_chksum_pseudo - 4 +4
> ip_addr_any - 4 +4
> httpc_init_connection - 4 +4
> ackno - 4 +4
> udp_port - 2 +2
> tcplen - 2 +2
> tcphdr_optlen - 2 +2
> tcphdr_opt1len - 2 +2
> tcp_port - 2 +2
> tcp_optidx - 2 +2
> recv_acked - 2 +2
> ping_seq_num - 2 +2
> memp_UDP_PCB - 2 +2
> memp_TCP_SEG - 2 +2
> memp_TCP_PCB_LISTEN - 2 +2
> memp_TCP_PCB - 2 +2
> memp_TCPIP_MSG_INPKT - 2 +2
> memp_TCPIP_MSG_API - 2 +2
> memp_SYS_TIMEOUT - 2 +2
> memp_REASSDATA - 2 +2
> memp_RAW_PCB - 2 +2
> memp_PBUF_POOL - 2 +2
> memp_PBUF - 2 +2
> memp_FRAG_PBUF - 2 +2
> ip_reass_pbufcount - 2 +2
> ip_id - 2 +2
> tcp_timer_ctr - 1 +1
> tcp_timer - 1 +1
> tcp_active_pcbs_changed - 1 +1
> recv_flags - 1 +1
> pbuf_free_ooseq_pending - 1 +1
> netif_num - 1 +1
> flags - 1 +1
> etharp_cached_entry - 1 +1
> supported_nfs_versions 1 - -1
> retry_action 1 - -1
> net_boot_file_name_explicit 1 - -1
> dhcp_option_overload 1 - -1
> tftp_windowsize 2 - -2
> tftp_window_size_option 2 - -2
> tftp_next_ack 2 - -2
> tftp_last_nack 2 - -2
> tftp_block_size_option 2 - -2
> tftp_block_size 2 - -2
> ping_seq_number 2 - -2
> last_op 2 - -2
> env_flags_vartype_rep 7 5 -2
> linefeed 3 - -3
> wget_timeout_count 4 - -4
> wget_loop_state 4 - -4
> web_server_ip 4 - -4
> timeout_count_max 4 - -4
> timeout_count 4 - -4
> tftp_timeout_count_max 4 - -4
> tftp_remote_port 4 - -4
> tftp_remote_ip 4 - -4
> tftp_our_port 4 - -4
> saved_tftp_block_size_option 4 - -4
> retry_tcp_seq_num 4 - -4
> retry_tcp_ack_num 4 - -4
> retry_len 4 - -4
> pkt_q_idx 4 - -4
> packets 4 - -4
> our_port 4 - -4
> nfs_timeout_count 4 - -4
> nfs_state 4 - -4
> nfs_server_port 4 - -4
> nfs_server_mount_port 4 - -4
> nfs_server_ip 4 - -4
> nfs_our_port 4 - -4
> nfs_offset 4 - -4
> nfs_len 4 - -4
> nfs_download_state 4 - -4
> net_ping_ip 4 - -4
> net_dns_server 4 - -4
> net_boot_file_expected_size_in_blocks 4 - -4
> last_reg_lo 4 - -4
> last_reg_hi 4 - -4
> last_mask 4 - -4
> last_data 4 - -4
> last_addr_lo 4 - -4
> last_addr_hi 4 - -4
> initial_data_seq_num 4 - -4
> http_ok 4 - -4
> fs_mounted 4 - -4
> filefh3_length 4 - -4
> eth_common_init 4 - -4
> dummy_handler 8 4 -4
> dhcp_state 4 - -4
> dhcp_server_ip 4 - -4
> dhcp_leasetime 4 - -4
> current_wget_state 4 - -4
> bootp_try 4 - -4
> bootp_num_ids 4 - -4
> http_eom 5 - -5
> bootfile1 5 - -5
> timeout_ms 8 - -8
> time_taken_max 8 - -8
> time_start 16 8 -8
> tftp_prev_block 8 - -8
> tftp_load_size 8 - -8
> tftp_load_addr 8 - -8
> tftp_cur_block 8 - -8
> tftp_block_wrap_offset 8 - -8
> tftp_block_wrap 8 - -8
> rpc_id 8 - -8
> nfs_path 8 - -8
> nfs_filename 8 - -8
> miiphy_is_1000base_x 8 - -8
> init_sequence_r 264 256 -8
> image_url 8 - -8
> distro_pxe_check 8 - -8
> current_mii 8 - -8
> content_length 8 - -8
> bootp_timeout 8 - -8
> bootp_start 8 - -8
> tcp_get_tcp_state 12 - -12
> do_wget 12 - -12
> do_tftpb 12 - -12
> do_nfs 12 - -12
> do_dhcp 12 - -12
> do_bootp 12 - -12
> default_filename 13 - -13
> bootfile3 14 - -14
> content_len 15 - -15
> reg_2_desc_tbl 16 - -16
> pkt_q 16 - -16
> mii_devs 16 - -16
> bootp_ids 16 - -16
> miiphy_get_current_dev 20 - -20
> tcp_set_tcp_handler 24 - -24
> pxe_default_paths 24 - -24
> net_set_udp_handler 24 - -24
> net_check_prereq 256 232 -24
> miiphy_init 28 - -28
> ping_timeout_handler 32 - -32
> net_nis_domain 32 - -32
> net_hostname 32 - -32
> distro_bootmeth_pxe_ids 32 - -32
> dirfh 32 - -32
> initr_net 36 - -36
> distro_bootmeth_pxe_bind 36 - -36
> ip_to_string 40 - -40
> distro_bootmeth_pxe_ops 40 - -40
> net_send_udp_packet 44 - -44
> label_boot 1944 1900 -44
> env_flags_validate 632 588 -44
> reg_3_desc_tbl 48 - -48
> do_get_tftp 56 - -56
> cmd_net 56 - -56
> _u_boot_list_2_cmd_2_wget 56 - -56
> _u_boot_list_2_cmd_2_tftpboot 56 - -56
> _u_boot_list_2_cmd_2_pxe 56 - -56
> _u_boot_list_2_cmd_2_ping 56 - -56
> _u_boot_list_2_cmd_2_nfs 56 - -56
> _u_boot_list_2_cmd_2_net 56 - -56
> _u_boot_list_2_cmd_2_mii 56 - -56
> _u_boot_list_2_cmd_2_dhcp 56 - -56
> _u_boot_list_2_cmd_2_bootp 56 - -56
> net_loop 652 592 -60
> net_eth_hdr_size 60 - -60
> bootp_reset 60 - -60
> net_root_path 64 - -64
> filefh 64 - -64
> do_bootvx 816 748 -68
> miiphy_set_current_dev 72 - -72
> basename 72 - -72
> pxe_get_file_size 76 - -76
> copy_filename 76 - -76
> distro_pxe_getfile 80 - -80
> tftp_init_load_addr 92 - -92
> miiphy_read 92 - -92
> extract_range 92 - -92
> miiphy_write 96 - -96
> miiphy_get_active_dev 96 - -96
> distro_pxe_read_file 96 - -96
> wget_fail 104 - -104
> skip_num 104 - -104
> miiphy_get_dev_by_name 104 - -104
> dump_field 104 - -104
> do_bdinfo 432 328 -104
> bootp_timeout_handler 104 - -104
> nfs_timeout_handler 108 - -108
> cmd_pxe_sub 112 - -112
> nfs_umountall_req 120 - -120
> _u_boot_list_2_driver_2_bootmeth_pxe 120 - -120
> do_ping 124 - -124
> tftp_filename 128 - -128
> reg_9_desc_tbl 128 - -128
> reg_10_desc_tbl 128 - -128
> distro_pxe_boot 128 - -128
> tftp_timeout_handler 132 - -132
> do_pxe 132 - -132
> nfs_umountall_reply 136 - -136
> lmb_get_free_size 136 - -136
> format_mac_pxe 136 - -136
> miiphy_listdev 144 - -144
> efi_net_set_dhcp_ack 144 - -144
> wget_timeout_handler 148 - -148
> nfs_mount_reply 148 - -148
> dhcp_packet_process_options 148 - -148
> eth_validate_ethaddr_str 152 - -152
> do_pxe_get 156 - -156
> reg_0_desc_tbl 160 - -160
> net_parse_bootfile 160 - -160
> miiphy_info 160 - -160
> get_pxelinux_path 160 - -160
> do_net 164 - -164
> net_auto_load 172 - -172
> do_net_list 176 - -176
> rpc_lookup_reply 180 - -180
> nfs_readlink_req 184 - -184
> nfs_mount_req 188 - -188
> reg_5_desc_tbl 192 - -192
> reg_4_desc_tbl 192 - -192
> miiphy_speed 200 - -200
> miiphy_duplex 200 - -200
> nfs_read_req 224 - -224
> do_pxe_boot 248 - -248
> reg_1_desc_tbl 256 - -256
> mii_reg_desc_tbl 256 - -256
> nfs_send 260 - -260
> wget_start 268 - -268
> ping_start 276 - -276
> nfs_lookup_reply 280 - -280
> rpc_req 300 - -300
> eth_initialize 300 - -300
> distro_pxe_read_bootflow 300 - -300
> nfs_readlink_reply 328 - -328
> nfs_lookup_req 328 - -328
> ping_receive 332 - -332
> pxe_get 376 - -376
> nfs_read_reply 396 - -396
> wget_send_stored 444 - -444
> nfs_start 468 - -468
> dhcp_process_options 508 - -508
> tftp_send 560 - -560
> nfs_handler 580 - -580
> bootp_request 612 - -612
> dhcp_extended 616 - -616
> netboot_common 632 - -632
> default_environment 4444 3800 -644
> tftp_start 912 - -912
> dhcp_handler 1000 - -1000
> wget_handler 1092 - -1092
> tftp_handler 1304 - -1304
> nfs_path_buff 2048 - -2048
> do_mii 2124 - -2124
> Total: Before=722283, After=738425, chg +2.23%
>
>
>
> On Thu, 8 Jun 2023 at 02:07, Tom Rini <trini@konsulko.com> wrote:
>>
>> On Wed, May 24, 2023 at 10:18:13PM +0200, Simon Goldschmidt wrote:
>> > Hi Maxim, Tom,
>> >
>> > On 24.05.2023 16:05, Maxim Uvarov wrote:
>> > > On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
>> > >
>> > > > On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
>> > > > > On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
>> > > > >
>> > > > > > On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
>> > > > > > > Hi Maxim
>> > > > > > >
>> > > > > > > On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
>> > > > > > wrote:
>> > > > > > > >
>> > > > > > > > My measurements for binary after LTO looks like:
>> > > > > > > >
>> > > > > > > > U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
>> > > > > > > > 870728 | 915000 | 912560 |
>> > > > > > 41832 | 4.8
>> > > > > > >
>> > > > > > >
>> > > > > > > I think you'll need to analyze that a bit more. First of all I don't
>> > > > > > > think the '+ping' tab is useful. What is is trying to achieve?
>> > > > > >
>> > > > >
>> > > > > To show the difference of extra bytes if we add a ping app on top.
>> > > > >
>> > > > >
>> > > > > > > - How was LWIP compiled?
>> > > > > >
>> > > > >
>> > > > > It has a really huge configuration. I tried to turn off everything off
>> > > > > everything what can impact on size but still make http app work:
>> > > > > #define LWIP_HAVE_LOOPIF 0
>> > > > > #define LWIP_NETCONN 0
>> > > > > #define LWIP_SOCKET 0
>> > > > > #define SO_REUSE 0
>> > > > > #define LWIP_STATS 0
>> > > > > #define PPP_SUPPORT 0
>> > > > >
>> > > > > Disabling loopback:
>> > > > > #define LWIP_NETIF_LOOPBACK 0
>> > > > > can lower to 912288 bytes.
>> > > > >
>> > > > > And it's the same compilation option (optimization for size) as the main
>> > > > > u-boot. I will do more experiments, but I think the goal is not to turn
>> > > > off
>> > > > > everything.
>> > > > >
>> > > > >
>> > > > > > > - Was ipv6 supported?
>> > > > > >
>> > > > >
>> > > > > No. I.e. when I sent results it was enabled on the compilation stage but
>> > > > > not used. I just checked that size remains the same if IPv6 is not even
>> > > > > compiled.
>> > > > >
>> > > > >
>> > > > > > > - Can we strip it down even further?
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > > There is always room for optimization. I think I tried to turn off
>> > > > > everything that is configurable with defines. I can play with disable IP
>> > > > > reassembly and things like that or figure out which functions have more
>> > > > > size and if it's possible to exclude them.
>> > > > >
>> > > > >
>> > > > > > > In general please give as much information as you can with what we
>> > > > > > > gain in functionality from LWIP with those extra bytes of code.
>> > > > > >
>> > > > > >
>> > > > > The main idea is to reuse a maintainable IP stack outside of U-boot.
>> > > > LWIP
>> > > > > can give a nice separation between IP stack code and network application
>> > > > > code. I.e. application should not take care about any TCP details (SYN,
>> > > > > ACK, retransmission, reassembly etc) and should open connection and use
>> > > > > functions similar to recv() and send() to transfer data. Data means
>> > > > > application data, no network packets. And LWIP allows
>> > > > > us to do that.
>> > > > > Because LWIP has an API similar to sockets, it has to be very easy to
>> > > > port
>> > > > > a linux application to LWIP. Then you can test it with a tap device. Then
>> > > > > copy sources to U-boot, add a small integration layer (cmd command to
>> > > > > call), compile and use.
>> > > > >
>> > > > > So my suggestion was:
>> > > > > - do not maintain new network stack code in the current U-boot. Use lwip
>> > > > > sources as an external project. All bugs related to network stack go to
>> > > > > lwip project first, then sync with U-boot.
>> > > > > - maintain network apps code* or
>> > > > > -- inside U-boot. Write our own code for application and maintain it
>> > > > > inside U-boot.
>> > > > > -- inside LWIP. Add examples to LWIP which are suitable for both
>> > > > U-boot
>> > > > > and LWIP.
>> > > > >
>> > > > > * Let's define a U-boot network application as a cmd command. It might be
>> > > > > ping, wget (http or https download), telnet, arp dns etc..
>> > > > >
>> > > > > Let's consider the real use case, like HTTPS download client. We need to
>> > > > > enable TLS connection, validate certificates, then do http download.
>> > > > > Looking at the current code of wget command it's quite difficult to
>> > > > > implement this due to the application having some protol level things. On
>> > > > > the other side we can find embedTLS examples to do https download on
>> > > > > sockets. If LWIP socket API is ported then the only thing you need to do
>> > > > is
>> > > > > change socket() -> lwip_socket(), recv()->lwip_recv(),
>> > > > send()->lwip_send()
>> > > > > and etc, even function names are similar. If LWIP socket API is not
>> > > > > supported, then use callback API for recv() and send(), which are also
>> > > > > easy.
>> > > > >
>> > > > > So yes we add extra bytes, but that will allow us to write more complex
>> > > > > apps, use standard debug tools, use applications with very minimal
>> > > > > integration changes, use help from the LWIP community to fix protocol
>> > > > bugs,
>> > > > > etc..
>> > > > > Bunch of things already implemented there:
>> > > > > - ipv6
>> > > > > - dhcp
>> > > > > - snmp
>> > > > > - igmp
>> > > > > - dns
>> > > > > - tcp and udp and raw.
>> > > > > - loopback
>> > > > > - netconn
>> > > > > - socket
>> > > > > - stats
>> > > > > - ppp
>> > > > > (I just followed configurable defines).
>> > > > >
>> > > > >
>> > > > > And please make sure to disable the previous support, my guess fro that
>> > > > > > much growth is that you didn't.
>> > > > > >
>> > > > >
>> > > > > # CONFIG_PROT_TCP is not set
>> > > > > # CONFIG_PROT_UDP is not set
>> > > > > # CONFIG_UDP_CHECKSUM is not set
>> > > > > # CONFIG_UDP_FUNCTION_FASTBOOT is not set
>> > > > > # CONFIG_CMD_WGET is not set
>> > > >
>> > > > I think you need to step back and figure out a better way to measure the
>> > > > size change and growth.
>> > > >
>> > > > I am not interested in a path that long term means two networking stacks
>> > > > in U-Boot.
>> > > >
>> > > > I am not interested in massively growing the overall binary size for
>> > > > every platform. Given how much larger just TCP support is, that's
>> > > > strongly implying a huge growth for the older use cases too.
>> > > >
>> > > > But I also suspect given the overall reputation that LWIP enjoys,
>> > > > there's something amiss here.
>> > > >
>> > > > --
>> > > > Tom
>> > > >
>> > >
>> > > +cc lwip-devel@ mailing list, maybe they have something to add.
>> >
>> > I do think using lwIP instead of "inventing yet another IP stack" is a
>> > good idea! However, in terms of code size, lwIP will lose against what's
>> > in U-Boot at present. And this is only natural, as lwIP is a "full-size"
>> > stack supporting multiple concurrently running applications while the
>> > current IP stack in U-Boot is rather "crippled" down to just what the
>> > implementor needed at the time of writing.
>> >
>> > One example of this is that (if I remember correctly), U-Boot only has
>> > one single network packet buffer, while lwIP has support for multiple
>> > buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
>> > of that development in U-Boot about 3 years ago), we're comparing "we
>> > have implemented everything we need so that it just kind of works" to
>> > "we can easily add a HTTPS client to download something over the
>> > internet just by enabling more compile options".
>> >
>> > Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
>> > U-Boot TCP (at least that of some years ago) is far from complete when
>> > compared to lwIP!
>> >
>> > lwIP is meant to be highly configurable and we're always open to add yet
>> > more options to leave out more code when it's not needed. However, I
>> > think there are some design decisions that will make lwIP larger than
>> > the current IP stack in U-Boot. To me, that's a natural result of having
>> > a "generic code" approach vs "developed to our needs". However, while
>> > DHCP + BOOTP and even a simple network console was rather easy to
>> > implement, I would not recommend implementing your own HTTPS download
>> > but rather using the existing lwIP + apps for that.
>> >
>> > In the end, I cannot take the decision from you. In my opinion, lwIP
>> > would be the better decision in terms of future work load and
>> > compatibility, but in the short run, it *will* lead to bigger binaries
>> > at least in some setups. And I do know from my past that it sometimes
>> > has been a pain to try and stuff a new U-Boot release into the existing
>> > space of flash or RAM, so that's not an easy decision.
>> >
>> > If you do take the lwIP approach however, let us know if we can help!
>>
>> Given Maxim's more recent experiments, I'm sure we can come up with
>> something that works overall. There's hopefully a place or two U-Boot
>> people can help introduce a tunable or two to lwIP to bring some sizes
>> down. But I think it's overall looking to be the right direction.
>>
>> --
>> Tom
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-06-08 17:52 ` Ilias Apalodimas
@ 2023-06-09 7:37 ` Peter Robinson
0 siblings, 0 replies; 37+ messages in thread
From: Peter Robinson @ 2023-06-09 7:37 UTC (permalink / raw)
To: Ilias Apalodimas
Cc: Maxim Uvarov, Tom Rini, Simon Goldschmidt, u-boot, pbrobinson,
joe.hershberger, rfried.dev, lwip-devel
On Thu, Jun 8, 2023 at 6:56 PM Ilias Apalodimas
<ilias.apalodimas@linaro.org> wrote:
>
> Thanks Maxim,
>
> On Thu, 8 Jun 2023 at 13:14, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> >
> > Ilias asked to make more clear results to compare the original stack and LWIP stack. So the difference between the current U-boot stack and the LWIP stack with 3 network commands is:
> > a) 18Kb - ls -lh size
> > b) 15Kb - bloat-o-meter script total line report.
> >
> > BOM=linux/scripts/bloat-o-meter (script)
> >
> > 1. 893K - U-boot CMD_NET=n
> > 2. 928K - U-boot CMD_NET=y TFTP=y PING=y WGET=y
> > BOM 1-2: Total: Before=692286, After=722283, chg +4.33%
> > 3. 940K - U-boot CMD_NET=n, LWIP_TFTP=y LWIP_PING=y LWIP_PING=y
> > BOM 1-3: Total: Before=692286, After=738425, chg +6.66%
>
> That's much more readable! I discussed this with Tom over IRC and the
> size difference is certainly a reasonable trade-off for the extra
> functionality.
Yes, this looks great! I'm also sure with a closer look there could be
further optimisations here in time as well. I feel having a widely
used IP stack that's also widely audited is a big win here, it will
also provide things like HTTP, DNS and IPv6 without having to reinvent
the wheel.
> Can you tidy up the series and include DHCP and PXE done through LWIP?
I'll look forward to this.
> Thanks
> /Ilias
> >
> > BOM 2-3:
> >
> > add/remove: 287/203 grow/shrink: 3/11 up/down: 43459/-27317 (16142)
> > Function old new delta
> > tcp_input - 3588 +3588
> > tcp_receive - 2884 +2884
> > ip4_reass - 1760 +1760
> > tcp_output - 1400 +1400
> > tcp_write - 1300 +1300
> > tcp_slowtmr - 1172 +1172
> > httpc_tcp_recv - 1044 +1044
> > tftp_recv - 888 +888
> > ip4_input - 700 +700
> > ip4_frag - 632 +632
> > icmp_input - 604 +604
> > udp_input - 596 +596
> > etharp_input - 512 +512
> > tcp_split_unsent_seg - 500 +500
> > ip4addr_aton - 492 +492
> > tcp_alloc - 484 +484
> > ip4_output_if_src - 476 +476
> > tcp_close_shutdown - 448 +448
> > etharp_query - 436 +436
> > httpc_init_connection_common.constprop - 416 +416
> > udp_sendto_if_src - 408 +408
> > etharp_output - 404 +404
> > arp_table - 400 +400
> > tcp_connect - 396 +396
> > pbuf_alloc - 376 +376
> > etharp_find_entry - 372 +372
> > tcp_abandon - 368 +368
> > tcp_zero_window_probe - 356 +356
> > raw_sendto_if_src - 328 +328
> > pbuf_copy_partial_pbuf - 328 +328
> > ip_reass_free_complete_datagram - 328 +328
> > tcp_create_segment - 300 +300
> > raw_input - 292 +292
> > uboot_lwip_init - 284 +284
> > ethernet_input - 284 +284
> > etharp_raw - 284 +284
> > tcp_output_alloc_header_common.constprop - 280 +280
> > cmds - 280 +280
> > udp_bind - 276 +276
> > tcp_oos_insert_segment - 276 +276
> > ip_reass_remove_oldest_datagram - 272 +272
> > icmp_send_response - 268 +268
> > netif_add - 260 +260
> > ping_send - 244 +244
> > tcp_rexmit - 232 +232
> > tcp_parseopt - 220 +220
> > tcp_free_acked_segments.constprop - 220 +220
> > send_request - 220 +220
> > inet_chksum_pseudo - 216 +216
> > ip4addr_ntoa_r - 212 +212
> > do_lwip_ping - 212 +212
> > tcp_enqueue_flags - 208 +208
> > etharp_output_to_arp_index - 208 +208
> > netif_set_addr - 204 +204
> > tcp_fasttmr - 200 +200
> > tcp_rexmit_rto_prepare - 196 +196
> > tcp_process_refused_data - 196 +196
> > send_data - 196 +196
> > lwip_wget - 192 +192
> > ethernet_output - 192 +192
> > ping_recv - 188 +188
> > pbuf_memcmp - 184 +184
> > pbuf_copy_partial - 184 +184
> > httpc_free_state - 180 +180
> > tcp_send_fin - 172 +172
> > httpc_recv - 168 +168
> > tcp_output_control_segment_netif - 164 +164
> > send_error.isra - 164 +164
> > do_ops - 164 +164
> > raw_sendto - 160 +160
> > pbuf_realloc - 160 +160
> > pbuf_free - 160 +160
> > do_lwip_wget - 160 +160
> > do_lwip_tftp - 160 +160
> > tftp_init_common - 156 +156
> > tcp_rst_netif - 152 +152
> > udp_sendto - 144 +144
> > tftp_tmr - 144 +144
> > tcp_rst - 144 +144
> > uboot_lwip_if_init - 140 +140
> > tcp_pcb_remove - 140 +140
> > tcp_pbuf_prealloc - 140 +140
> > sys_timeout_abs - 140 +140
> > lwip_tftp - 140 +140
> > netif_do_set_ipaddr.isra - 136 +136
> > ip4_route - 136 +136
> > tcp_netif_ip_addr_changed - 132 +132
> > resend_data.isra - 132 +132
> > inet_chksum_pbuf - 132 +132
> > tcp_output_control_segment - 128 +128
> > pbuf_memfind - 128 +128
> > lwip_standard_chksum - 128 +128
> > tcp_rexmit_fast - 124 +124
> > tcp_new_port - 124 +124
> > tcp_close_shutdown_fin - 124 +124
> > pbuf_add_header_impl - 124 +124
> > tcp_send_empty_ack - 120 +120
> > httpc_create_request_string.constprop.isra - 120 +120
> > tftp_get - 116 +116
> > tcp_recved - 116 +116
> > tcp_pcb_purge - 116 +116
> > tftp_write - 112 +112
> > pbuf_free_header - 112 +112
> > httpc_tcp_connected - 112 +112
> > tftp_error - 108 +108
> > send_ack.isra - 108 +108
> > low_level_input.constprop - 108 +108
> > tcp_input_delayed_close - 104 +104
> > close_handle - 100 +100
> > sys_untimeout - 96 +96
> > memp_pools - 96 +96
> > tcp_keepalive - 92 +92
> > ip4_addr_isbroadcast_u32 - 92 +92
> > init_packet - 92 +92
> > tcp_kill_state - 88 +88
> > raw_new - 88 +88
> > ping_raw_init - 88 +88
> > lwip_ping_init - 88 +88
> > udp_sendto_if - 84 +84
> > tcp_update_rcv_ann_wnd - 84 +84
> > tcp_recv_null - 84 +84
> > pbuf_remove_header - 84 +84
> > pbuf_alloc_reference - 84 +84
> > udp_remove - 80 +80
> > tcp_get_next_optbyte - 80 +80
> > pbuf_alloced_custom - 80 +80
> > ip4_input_accept - 80 +80
> > httpc_close - 80 +80
> > etharp_free_entry - 80 +80
> > uboot_lwip_poll - 76 +76
> > tcpip_tcp_timer - 76 +76
> > udp_netif_ip_addr_changed - 72 +72
> > uboot_netif - 72 +72
> > tcp_output_alloc_header.constprop - 72 +72
> > raw_netif_ip_addr_changed - 72 +72
> > tcpip_try_callback - 68 +68
> > tcp_timer_needed - 68 +68
> > tcp_seg_copy - 68 +68
> > tcp_netif_ip_addr_changed_pcblist - 68 +68
> > ping_timeout - 68 +68
> > ethernetif_input - 68 +68
> > udp_new - 64 +64
> > pbuf_try_get_at - 64 +64
> > sys_timeout - 60 +60
> > pbuf_clone - 60 +60
> > tcp_seg_free - 56 +56
> > pbuf_cat - 56 +56
> > netif_get_by_index - 56 +56
> > low_level_output - 56 +56
> > _u_boot_list_2_cmd_2_lwipinfo - 56 +56
> > _u_boot_list_2_cmd_2_lwip - 56 +56
> > tftp_state 4 56 +52
> > tcp_tmr - 52 +52
> > tcp_rexmit_rto - 52 +52
> > tcp_segs_free - 48 +48
> > tcp_eff_send_mss_netif - 48 +48
> > pbuf_skip_const - 48 +48
> > ipfrag_free_pbuf_custom - 48 +48
> > httpc_tcp_poll - 48 +48
> > tcp_free_ooseq - 44 +44
> > tcp_close - 44 +44
> > pbuf_free_ooseq_callback - 44 +44
> > netif_issue_reports - 44 +44
> > ip_reass_dequeue_datagram - 44 +44
> > httpc_get_internal_addr - 44 +44
> > tftp_read - 40 +40
> > tftp - 40 +40
> > ip_data - 40 +40
> > etharp_request - 40 +40
> > do_lwip_info - 40 +40
> > ulwip_timeout_handler - 36 +36
> > raw_bind - 36 +36
> > memp_malloc - 36 +36
> > ip4_output_if - 36 +36
> > tcp_pcb_lists - 32 +32
> > pbuf_header_force - 32 +32
> > pbuf_clen - 32 +32
> > netif_set_up - 32 +32
> > netif_set_link_up - 32 +32
> > inseg - 32 +32
> > inet_chksum - 32 +32
> > tcp_next_iss - 28 +28
> > pbuf_get_at - 28 +28
> > httpc_tcp_err - 28 +28
> > do_lwip_init - 28 +28
> > tcp_rexmit_rto_commit - 24 +24
> > sys_now - 24 +24
> > settings - 24 +24
> > pbuf_copy - 24 +24
> > pbuf_chain - 24 +24
> > memp_free - 24 +24
> > __func__ 1243 1266 +23
> > ulwip_exit - 20 +20
> > tcp_trigger_input_pcb_close - 20 +20
> > tcp_poll - 20 +20
> > ping_send_now - 20 +20
> > pbuf_ref - 20 +20
> > str - 16 +16
> > ip4addr_ntoa - 16 +16
> > daddr - 16 +16
> > tcp_backoff - 13 +13
> > ulwip_loop_set - 12 +12
> > ulwip_in_loop - 12 +12
> > ulwip_enabled - 12 +12
> > ulwip_app_get_err - 12 +12
> > udp_recv - 12 +12
> > tftp_init_client - 12 +12
> > tcp_sent - 12 +12
> > tcp_recv - 12 +12
> > tcp_free - 12 +12
> > tcp_err - 12 +12
> > tcp_arg - 12 +12
> > net_process_received_packet 800 812 +12
> > icmp_time_exceeded - 12 +12
> > icmp_dest_unreach - 12 +12
> > udp_pcbs - 8 +8
> > tftp_open - 8 +8
> > tftp_close - 8 +8
> > tcphdr_opt2 - 8 +8
> > tcphdr - 8 +8
> > tcp_tw_pcbs - 8 +8
> > tcp_new - 8 +8
> > tcp_listen_pcbs - 8 +8
> > tcp_input_pcb - 8 +8
> > tcp_bound_pcbs - 8 +8
> > tcp_active_pcbs - 8 +8
> > tcp_abort - 8 +8
> > recv_data - 8 +8
> > reassdatagrams - 8 +8
> > raw_recv - 8 +8
> > raw_pcbs - 8 +8
> > ping_target - 8 +8
> > ping_pcb - 8 +8
> > pbuf_add_header - 8 +8
> > next_timeout - 8 +8
> > netif_null_output_ip4 - 8 +8
> > netif_list - 8 +8
> > netif_default - 8 +8
> > lwip_htons - 8 +8
> > lwip_htonl - 8 +8
> > httpc_tcp_sent - 8 +8
> > tcp_persist_backoff - 7 +7
> > ethzero - 6 +6
> > ethbroadcast - 6 +6
> > ulwip_app_err - 4 +4
> > udp_new_ip_type - 4 +4
> > uboot_net_use_lwip - 4 +4
> > tcpip_tcp_timer_active - 4 +4
> > tcp_ticks - 4 +4
> > seqno - 4 +4
> > mem_trim - 4 +4
> > mem_malloc - 4 +4
> > mem_free - 4 +4
> > loop_lwip - 4 +4
> > iss - 4 +4
> > ip_target - 4 +4
> > ip_chksum_pseudo - 4 +4
> > ip_addr_any - 4 +4
> > httpc_init_connection - 4 +4
> > ackno - 4 +4
> > udp_port - 2 +2
> > tcplen - 2 +2
> > tcphdr_optlen - 2 +2
> > tcphdr_opt1len - 2 +2
> > tcp_port - 2 +2
> > tcp_optidx - 2 +2
> > recv_acked - 2 +2
> > ping_seq_num - 2 +2
> > memp_UDP_PCB - 2 +2
> > memp_TCP_SEG - 2 +2
> > memp_TCP_PCB_LISTEN - 2 +2
> > memp_TCP_PCB - 2 +2
> > memp_TCPIP_MSG_INPKT - 2 +2
> > memp_TCPIP_MSG_API - 2 +2
> > memp_SYS_TIMEOUT - 2 +2
> > memp_REASSDATA - 2 +2
> > memp_RAW_PCB - 2 +2
> > memp_PBUF_POOL - 2 +2
> > memp_PBUF - 2 +2
> > memp_FRAG_PBUF - 2 +2
> > ip_reass_pbufcount - 2 +2
> > ip_id - 2 +2
> > tcp_timer_ctr - 1 +1
> > tcp_timer - 1 +1
> > tcp_active_pcbs_changed - 1 +1
> > recv_flags - 1 +1
> > pbuf_free_ooseq_pending - 1 +1
> > netif_num - 1 +1
> > flags - 1 +1
> > etharp_cached_entry - 1 +1
> > supported_nfs_versions 1 - -1
> > retry_action 1 - -1
> > net_boot_file_name_explicit 1 - -1
> > dhcp_option_overload 1 - -1
> > tftp_windowsize 2 - -2
> > tftp_window_size_option 2 - -2
> > tftp_next_ack 2 - -2
> > tftp_last_nack 2 - -2
> > tftp_block_size_option 2 - -2
> > tftp_block_size 2 - -2
> > ping_seq_number 2 - -2
> > last_op 2 - -2
> > env_flags_vartype_rep 7 5 -2
> > linefeed 3 - -3
> > wget_timeout_count 4 - -4
> > wget_loop_state 4 - -4
> > web_server_ip 4 - -4
> > timeout_count_max 4 - -4
> > timeout_count 4 - -4
> > tftp_timeout_count_max 4 - -4
> > tftp_remote_port 4 - -4
> > tftp_remote_ip 4 - -4
> > tftp_our_port 4 - -4
> > saved_tftp_block_size_option 4 - -4
> > retry_tcp_seq_num 4 - -4
> > retry_tcp_ack_num 4 - -4
> > retry_len 4 - -4
> > pkt_q_idx 4 - -4
> > packets 4 - -4
> > our_port 4 - -4
> > nfs_timeout_count 4 - -4
> > nfs_state 4 - -4
> > nfs_server_port 4 - -4
> > nfs_server_mount_port 4 - -4
> > nfs_server_ip 4 - -4
> > nfs_our_port 4 - -4
> > nfs_offset 4 - -4
> > nfs_len 4 - -4
> > nfs_download_state 4 - -4
> > net_ping_ip 4 - -4
> > net_dns_server 4 - -4
> > net_boot_file_expected_size_in_blocks 4 - -4
> > last_reg_lo 4 - -4
> > last_reg_hi 4 - -4
> > last_mask 4 - -4
> > last_data 4 - -4
> > last_addr_lo 4 - -4
> > last_addr_hi 4 - -4
> > initial_data_seq_num 4 - -4
> > http_ok 4 - -4
> > fs_mounted 4 - -4
> > filefh3_length 4 - -4
> > eth_common_init 4 - -4
> > dummy_handler 8 4 -4
> > dhcp_state 4 - -4
> > dhcp_server_ip 4 - -4
> > dhcp_leasetime 4 - -4
> > current_wget_state 4 - -4
> > bootp_try 4 - -4
> > bootp_num_ids 4 - -4
> > http_eom 5 - -5
> > bootfile1 5 - -5
> > timeout_ms 8 - -8
> > time_taken_max 8 - -8
> > time_start 16 8 -8
> > tftp_prev_block 8 - -8
> > tftp_load_size 8 - -8
> > tftp_load_addr 8 - -8
> > tftp_cur_block 8 - -8
> > tftp_block_wrap_offset 8 - -8
> > tftp_block_wrap 8 - -8
> > rpc_id 8 - -8
> > nfs_path 8 - -8
> > nfs_filename 8 - -8
> > miiphy_is_1000base_x 8 - -8
> > init_sequence_r 264 256 -8
> > image_url 8 - -8
> > distro_pxe_check 8 - -8
> > current_mii 8 - -8
> > content_length 8 - -8
> > bootp_timeout 8 - -8
> > bootp_start 8 - -8
> > tcp_get_tcp_state 12 - -12
> > do_wget 12 - -12
> > do_tftpb 12 - -12
> > do_nfs 12 - -12
> > do_dhcp 12 - -12
> > do_bootp 12 - -12
> > default_filename 13 - -13
> > bootfile3 14 - -14
> > content_len 15 - -15
> > reg_2_desc_tbl 16 - -16
> > pkt_q 16 - -16
> > mii_devs 16 - -16
> > bootp_ids 16 - -16
> > miiphy_get_current_dev 20 - -20
> > tcp_set_tcp_handler 24 - -24
> > pxe_default_paths 24 - -24
> > net_set_udp_handler 24 - -24
> > net_check_prereq 256 232 -24
> > miiphy_init 28 - -28
> > ping_timeout_handler 32 - -32
> > net_nis_domain 32 - -32
> > net_hostname 32 - -32
> > distro_bootmeth_pxe_ids 32 - -32
> > dirfh 32 - -32
> > initr_net 36 - -36
> > distro_bootmeth_pxe_bind 36 - -36
> > ip_to_string 40 - -40
> > distro_bootmeth_pxe_ops 40 - -40
> > net_send_udp_packet 44 - -44
> > label_boot 1944 1900 -44
> > env_flags_validate 632 588 -44
> > reg_3_desc_tbl 48 - -48
> > do_get_tftp 56 - -56
> > cmd_net 56 - -56
> > _u_boot_list_2_cmd_2_wget 56 - -56
> > _u_boot_list_2_cmd_2_tftpboot 56 - -56
> > _u_boot_list_2_cmd_2_pxe 56 - -56
> > _u_boot_list_2_cmd_2_ping 56 - -56
> > _u_boot_list_2_cmd_2_nfs 56 - -56
> > _u_boot_list_2_cmd_2_net 56 - -56
> > _u_boot_list_2_cmd_2_mii 56 - -56
> > _u_boot_list_2_cmd_2_dhcp 56 - -56
> > _u_boot_list_2_cmd_2_bootp 56 - -56
> > net_loop 652 592 -60
> > net_eth_hdr_size 60 - -60
> > bootp_reset 60 - -60
> > net_root_path 64 - -64
> > filefh 64 - -64
> > do_bootvx 816 748 -68
> > miiphy_set_current_dev 72 - -72
> > basename 72 - -72
> > pxe_get_file_size 76 - -76
> > copy_filename 76 - -76
> > distro_pxe_getfile 80 - -80
> > tftp_init_load_addr 92 - -92
> > miiphy_read 92 - -92
> > extract_range 92 - -92
> > miiphy_write 96 - -96
> > miiphy_get_active_dev 96 - -96
> > distro_pxe_read_file 96 - -96
> > wget_fail 104 - -104
> > skip_num 104 - -104
> > miiphy_get_dev_by_name 104 - -104
> > dump_field 104 - -104
> > do_bdinfo 432 328 -104
> > bootp_timeout_handler 104 - -104
> > nfs_timeout_handler 108 - -108
> > cmd_pxe_sub 112 - -112
> > nfs_umountall_req 120 - -120
> > _u_boot_list_2_driver_2_bootmeth_pxe 120 - -120
> > do_ping 124 - -124
> > tftp_filename 128 - -128
> > reg_9_desc_tbl 128 - -128
> > reg_10_desc_tbl 128 - -128
> > distro_pxe_boot 128 - -128
> > tftp_timeout_handler 132 - -132
> > do_pxe 132 - -132
> > nfs_umountall_reply 136 - -136
> > lmb_get_free_size 136 - -136
> > format_mac_pxe 136 - -136
> > miiphy_listdev 144 - -144
> > efi_net_set_dhcp_ack 144 - -144
> > wget_timeout_handler 148 - -148
> > nfs_mount_reply 148 - -148
> > dhcp_packet_process_options 148 - -148
> > eth_validate_ethaddr_str 152 - -152
> > do_pxe_get 156 - -156
> > reg_0_desc_tbl 160 - -160
> > net_parse_bootfile 160 - -160
> > miiphy_info 160 - -160
> > get_pxelinux_path 160 - -160
> > do_net 164 - -164
> > net_auto_load 172 - -172
> > do_net_list 176 - -176
> > rpc_lookup_reply 180 - -180
> > nfs_readlink_req 184 - -184
> > nfs_mount_req 188 - -188
> > reg_5_desc_tbl 192 - -192
> > reg_4_desc_tbl 192 - -192
> > miiphy_speed 200 - -200
> > miiphy_duplex 200 - -200
> > nfs_read_req 224 - -224
> > do_pxe_boot 248 - -248
> > reg_1_desc_tbl 256 - -256
> > mii_reg_desc_tbl 256 - -256
> > nfs_send 260 - -260
> > wget_start 268 - -268
> > ping_start 276 - -276
> > nfs_lookup_reply 280 - -280
> > rpc_req 300 - -300
> > eth_initialize 300 - -300
> > distro_pxe_read_bootflow 300 - -300
> > nfs_readlink_reply 328 - -328
> > nfs_lookup_req 328 - -328
> > ping_receive 332 - -332
> > pxe_get 376 - -376
> > nfs_read_reply 396 - -396
> > wget_send_stored 444 - -444
> > nfs_start 468 - -468
> > dhcp_process_options 508 - -508
> > tftp_send 560 - -560
> > nfs_handler 580 - -580
> > bootp_request 612 - -612
> > dhcp_extended 616 - -616
> > netboot_common 632 - -632
> > default_environment 4444 3800 -644
> > tftp_start 912 - -912
> > dhcp_handler 1000 - -1000
> > wget_handler 1092 - -1092
> > tftp_handler 1304 - -1304
> > nfs_path_buff 2048 - -2048
> > do_mii 2124 - -2124
> > Total: Before=722283, After=738425, chg +2.23%
> >
> >
> >
> > On Thu, 8 Jun 2023 at 02:07, Tom Rini <trini@konsulko.com> wrote:
> >>
> >> On Wed, May 24, 2023 at 10:18:13PM +0200, Simon Goldschmidt wrote:
> >> > Hi Maxim, Tom,
> >> >
> >> > On 24.05.2023 16:05, Maxim Uvarov wrote:
> >> > > On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
> >> > >
> >> > > > On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
> >> > > > > On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
> >> > > > >
> >> > > > > > On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> >> > > > > > > Hi Maxim
> >> > > > > > >
> >> > > > > > > On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
> >> > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > My measurements for binary after LTO looks like:
> >> > > > > > > >
> >> > > > > > > > U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> >> > > > > > > > 870728 | 915000 | 912560 |
> >> > > > > > 41832 | 4.8
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > I think you'll need to analyze that a bit more. First of all I don't
> >> > > > > > > think the '+ping' tab is useful. What is is trying to achieve?
> >> > > > > >
> >> > > > >
> >> > > > > To show the difference of extra bytes if we add a ping app on top.
> >> > > > >
> >> > > > >
> >> > > > > > > - How was LWIP compiled?
> >> > > > > >
> >> > > > >
> >> > > > > It has a really huge configuration. I tried to turn off everything off
> >> > > > > everything what can impact on size but still make http app work:
> >> > > > > #define LWIP_HAVE_LOOPIF 0
> >> > > > > #define LWIP_NETCONN 0
> >> > > > > #define LWIP_SOCKET 0
> >> > > > > #define SO_REUSE 0
> >> > > > > #define LWIP_STATS 0
> >> > > > > #define PPP_SUPPORT 0
> >> > > > >
> >> > > > > Disabling loopback:
> >> > > > > #define LWIP_NETIF_LOOPBACK 0
> >> > > > > can lower to 912288 bytes.
> >> > > > >
> >> > > > > And it's the same compilation option (optimization for size) as the main
> >> > > > > u-boot. I will do more experiments, but I think the goal is not to turn
> >> > > > off
> >> > > > > everything.
> >> > > > >
> >> > > > >
> >> > > > > > > - Was ipv6 supported?
> >> > > > > >
> >> > > > >
> >> > > > > No. I.e. when I sent results it was enabled on the compilation stage but
> >> > > > > not used. I just checked that size remains the same if IPv6 is not even
> >> > > > > compiled.
> >> > > > >
> >> > > > >
> >> > > > > > > - Can we strip it down even further?
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > > > There is always room for optimization. I think I tried to turn off
> >> > > > > everything that is configurable with defines. I can play with disable IP
> >> > > > > reassembly and things like that or figure out which functions have more
> >> > > > > size and if it's possible to exclude them.
> >> > > > >
> >> > > > >
> >> > > > > > > In general please give as much information as you can with what we
> >> > > > > > > gain in functionality from LWIP with those extra bytes of code.
> >> > > > > >
> >> > > > > >
> >> > > > > The main idea is to reuse a maintainable IP stack outside of U-boot.
> >> > > > LWIP
> >> > > > > can give a nice separation between IP stack code and network application
> >> > > > > code. I.e. application should not take care about any TCP details (SYN,
> >> > > > > ACK, retransmission, reassembly etc) and should open connection and use
> >> > > > > functions similar to recv() and send() to transfer data. Data means
> >> > > > > application data, no network packets. And LWIP allows
> >> > > > > us to do that.
> >> > > > > Because LWIP has an API similar to sockets, it has to be very easy to
> >> > > > port
> >> > > > > a linux application to LWIP. Then you can test it with a tap device. Then
> >> > > > > copy sources to U-boot, add a small integration layer (cmd command to
> >> > > > > call), compile and use.
> >> > > > >
> >> > > > > So my suggestion was:
> >> > > > > - do not maintain new network stack code in the current U-boot. Use lwip
> >> > > > > sources as an external project. All bugs related to network stack go to
> >> > > > > lwip project first, then sync with U-boot.
> >> > > > > - maintain network apps code* or
> >> > > > > -- inside U-boot. Write our own code for application and maintain it
> >> > > > > inside U-boot.
> >> > > > > -- inside LWIP. Add examples to LWIP which are suitable for both
> >> > > > U-boot
> >> > > > > and LWIP.
> >> > > > >
> >> > > > > * Let's define a U-boot network application as a cmd command. It might be
> >> > > > > ping, wget (http or https download), telnet, arp dns etc..
> >> > > > >
> >> > > > > Let's consider the real use case, like HTTPS download client. We need to
> >> > > > > enable TLS connection, validate certificates, then do http download.
> >> > > > > Looking at the current code of wget command it's quite difficult to
> >> > > > > implement this due to the application having some protol level things. On
> >> > > > > the other side we can find embedTLS examples to do https download on
> >> > > > > sockets. If LWIP socket API is ported then the only thing you need to do
> >> > > > is
> >> > > > > change socket() -> lwip_socket(), recv()->lwip_recv(),
> >> > > > send()->lwip_send()
> >> > > > > and etc, even function names are similar. If LWIP socket API is not
> >> > > > > supported, then use callback API for recv() and send(), which are also
> >> > > > > easy.
> >> > > > >
> >> > > > > So yes we add extra bytes, but that will allow us to write more complex
> >> > > > > apps, use standard debug tools, use applications with very minimal
> >> > > > > integration changes, use help from the LWIP community to fix protocol
> >> > > > bugs,
> >> > > > > etc..
> >> > > > > Bunch of things already implemented there:
> >> > > > > - ipv6
> >> > > > > - dhcp
> >> > > > > - snmp
> >> > > > > - igmp
> >> > > > > - dns
> >> > > > > - tcp and udp and raw.
> >> > > > > - loopback
> >> > > > > - netconn
> >> > > > > - socket
> >> > > > > - stats
> >> > > > > - ppp
> >> > > > > (I just followed configurable defines).
> >> > > > >
> >> > > > >
> >> > > > > And please make sure to disable the previous support, my guess fro that
> >> > > > > > much growth is that you didn't.
> >> > > > > >
> >> > > > >
> >> > > > > # CONFIG_PROT_TCP is not set
> >> > > > > # CONFIG_PROT_UDP is not set
> >> > > > > # CONFIG_UDP_CHECKSUM is not set
> >> > > > > # CONFIG_UDP_FUNCTION_FASTBOOT is not set
> >> > > > > # CONFIG_CMD_WGET is not set
> >> > > >
> >> > > > I think you need to step back and figure out a better way to measure the
> >> > > > size change and growth.
> >> > > >
> >> > > > I am not interested in a path that long term means two networking stacks
> >> > > > in U-Boot.
> >> > > >
> >> > > > I am not interested in massively growing the overall binary size for
> >> > > > every platform. Given how much larger just TCP support is, that's
> >> > > > strongly implying a huge growth for the older use cases too.
> >> > > >
> >> > > > But I also suspect given the overall reputation that LWIP enjoys,
> >> > > > there's something amiss here.
> >> > > >
> >> > > > --
> >> > > > Tom
> >> > > >
> >> > >
> >> > > +cc lwip-devel@ mailing list, maybe they have something to add.
> >> >
> >> > I do think using lwIP instead of "inventing yet another IP stack" is a
> >> > good idea! However, in terms of code size, lwIP will lose against what's
> >> > in U-Boot at present. And this is only natural, as lwIP is a "full-size"
> >> > stack supporting multiple concurrently running applications while the
> >> > current IP stack in U-Boot is rather "crippled" down to just what the
> >> > implementor needed at the time of writing.
> >> >
> >> > One example of this is that (if I remember correctly), U-Boot only has
> >> > one single network packet buffer, while lwIP has support for multiple
> >> > buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
> >> > of that development in U-Boot about 3 years ago), we're comparing "we
> >> > have implemented everything we need so that it just kind of works" to
> >> > "we can easily add a HTTPS client to download something over the
> >> > internet just by enabling more compile options".
> >> >
> >> > Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
> >> > U-Boot TCP (at least that of some years ago) is far from complete when
> >> > compared to lwIP!
> >> >
> >> > lwIP is meant to be highly configurable and we're always open to add yet
> >> > more options to leave out more code when it's not needed. However, I
> >> > think there are some design decisions that will make lwIP larger than
> >> > the current IP stack in U-Boot. To me, that's a natural result of having
> >> > a "generic code" approach vs "developed to our needs". However, while
> >> > DHCP + BOOTP and even a simple network console was rather easy to
> >> > implement, I would not recommend implementing your own HTTPS download
> >> > but rather using the existing lwIP + apps for that.
> >> >
> >> > In the end, I cannot take the decision from you. In my opinion, lwIP
> >> > would be the better decision in terms of future work load and
> >> > compatibility, but in the short run, it *will* lead to bigger binaries
> >> > at least in some setups. And I do know from my past that it sometimes
> >> > has been a pain to try and stuff a new U-Boot release into the existing
> >> > space of flash or RAM, so that's not an easy decision.
> >> >
> >> > If you do take the lwIP approach however, let us know if we can help!
> >>
> >> Given Maxim's more recent experiments, I'm sure we can come up with
> >> something that works overall. There's hopefully a place or two U-Boot
> >> people can help introduce a tunable or two to lwIP to bring some sizes
> >> down. But I think it's overall looking to be the right direction.
> >>
> >> --
> >> Tom
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-06-07 9:46 ` Ilias Apalodimas
2023-06-07 12:01 ` Maxim Uvarov
@ 2023-06-11 8:24 ` Simon Glass
2023-06-11 18:34 ` Tom Rini
1 sibling, 1 reply; 37+ messages in thread
From: Simon Glass @ 2023-06-11 8:24 UTC (permalink / raw)
To: Ilias Apalodimas
Cc: Maxim Uvarov, Simon Goldschmidt, Tom Rini, u-boot, pbrobinson,
joe.hershberger, rfried.dev, lwip-devel
Hi,
On Wed, 7 Jun 2023 at 10:47, Ilias Apalodimas
<ilias.apalodimas@linaro.org> wrote:
>
> Hi Maxim,
>
> On Tue, 6 Jun 2023 at 17:33, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> >
> > Greetings,
> >
> > I implemented the tftp client (that was quick due to lwip has example app for tftp), and did some more measurements.
> > I uploaded patches here if somebody want to do his own measurements:
> > https://github.com/muvarov/uboot-lwip
> >
> > measure 1:
> > 976K - total (total means lwip with all 3 commands ping, tftp, wget)
> > 971K - total - tftp (total, but disable/minus tftp)
> > 965K - total - tftp - wget (disable tftp and wget)
> > 963K - total - tftp - wget - ping (disable tftp, wget, ping)
> > 931K - no lwip
> >
> > result 1: lwip tftp (+ udp protocol) protocol 976-971k = 5kb
> > result 2: lwip ping command 965- 963 = 2kb
> > result 3: lwip wget command 971- 965 = 6kb
> > result 4: lwip core stack with apps 976 - 931 = 45kb
>
> So tftp = 5kb, wget = 6kb ping =2kb and lwip = 32kb
>
> >
> > measure 2:
> > 890K - no CONFIG_NET_CMD
> > 930K - + lwip tftp only
> > 937K - + full lwip (ping wget tftp)
> >
> > result 1: 937-890=47kb ( lwip + all 3 commands)
> > result 2: 937-930=7kb (ping and lwip command)
>
> I am not sure I understand this measurement. How is this different
> from measurement 1 where the entire binary was 976K?
>
> >
> > measure 3:
> > 904K - no lwip, CMD_NET_TFTP=y
> > 900K - no lwip, CMD_NET_TFTP=n
> > result 1: original u-boot tftp command 904-900=4kb
> > 890K - no lwip, CMD_NET=n
> > result 2: 900-890=10k original u-boot net/IP stack.
> >
> > My findings for all that measurements and lwip configuration:
> > 1. The original u-boot net stack (packet process and up layers) is 10k vs lwip 40k (the very minimal settings were 30k).
> > 2. Network applications size is about the same 4kb for tftp original command 5kb for lwip.
> > 3. It's quite easy to reuse LWIP examples to implement the same functionality for the U-boot.
> > 4. I still think that there are other criterias which might have more priority than size (bug free code, code reuse, development speed, compatible API to posix and etc).
>
> Yes, there are other criteria and certainly having a complete network
> stack might be worth it in many cases, but we need to keep in mind
> 30kb might be a lot for some systems.
>
> I personally think this is decent and we can optimize lwip more in the
> future. Tom, Simon, how about adding lwip as 'experimental' and
> making it depend on !CMD_NET or something similar?
That seems OK to me, but we don't really want two network stacks, so
we'd need to set an expectation that we would move to lwip.
I wonder why it is so large? I saw mention of it supporting multiple
buffers and perhaps having a fuller implementation of the protocols.
But it makes U-Boot's stack seem super-slim in comparison. I wonder if
lwip could support just a single buffer and reduced functionality in
other areas?
Regards,
Simon
>
> Thanks
> /Ilias
> >
> > BR,
> > Maxim.
> >
> > On Thu, 25 May 2023 at 02:18, Simon Goldschmidt <goldsimon@gmx.de> wrote:
> >>
> >> Hi Maxim, Tom,
> >>
> >> On 24.05.2023 16:05, Maxim Uvarov wrote:
> >> > On Tue, 23 May 2023 at 03:23, Tom Rini <trini@konsulko.com> wrote:
> >> >
> >> >> On Mon, May 22, 2023 at 12:40:49PM -0400, Maxim Uvarov wrote:
> >> >>> On Mon, 22 May 2023 at 10:20, Tom Rini <trini@konsulko.com> wrote:
> >> >>>
> >> >>>> On Mon, May 22, 2023 at 04:33:57PM +0300, Ilias Apalodimas wrote:
> >> >>>>> Hi Maxim
> >> >>>>>
> >> >>>>> On Mon, 22 May 2023 at 12:01, Maxim Uvarov <maxim.uvarov@linaro.org>
> >> >>>> wrote:
> >> >>>>>>
> >> >>>>>> My measurements for binary after LTO looks like:
> >> >>>>>>
> >> >>>>>> U-boot WGET | LWIP WGET + ping | LWIP WGET| diff bytes| diff %
> >> >>>>>> 870728 | 915000 | 912560 |
> >> >>>> 41832 | 4.8
> >> >>>>>
> >> >>>>>
> >> >>>>> I think you'll need to analyze that a bit more. First of all I don't
> >> >>>>> think the '+ping' tab is useful. What is is trying to achieve?
> >> >>>>
> >> >>>
> >> >>> To show the difference of extra bytes if we add a ping app on top.
> >> >>>
> >> >>>
> >> >>>>> - How was LWIP compiled?
> >> >>>>
> >> >>>
> >> >>> It has a really huge configuration. I tried to turn off everything off
> >> >>> everything what can impact on size but still make http app work:
> >> >>> #define LWIP_HAVE_LOOPIF 0
> >> >>> #define LWIP_NETCONN 0
> >> >>> #define LWIP_SOCKET 0
> >> >>> #define SO_REUSE 0
> >> >>> #define LWIP_STATS 0
> >> >>> #define PPP_SUPPORT 0
> >> >>>
> >> >>> Disabling loopback:
> >> >>> #define LWIP_NETIF_LOOPBACK 0
> >> >>> can lower to 912288 bytes.
> >> >>>
> >> >>> And it's the same compilation option (optimization for size) as the main
> >> >>> u-boot. I will do more experiments, but I think the goal is not to turn
> >> >> off
> >> >>> everything.
> >> >>>
> >> >>>
> >> >>>>> - Was ipv6 supported?
> >> >>>>
> >> >>>
> >> >>> No. I.e. when I sent results it was enabled on the compilation stage but
> >> >>> not used. I just checked that size remains the same if IPv6 is not even
> >> >>> compiled.
> >> >>>
> >> >>>
> >> >>>>> - Can we strip it down even further?
> >> >>>>>
> >> >>>>
> >> >>>
> >> >>> There is always room for optimization. I think I tried to turn off
> >> >>> everything that is configurable with defines. I can play with disable IP
> >> >>> reassembly and things like that or figure out which functions have more
> >> >>> size and if it's possible to exclude them.
> >> >>>
> >> >>>
> >> >>>>> In general please give as much information as you can with what we
> >> >>>>> gain in functionality from LWIP with those extra bytes of code.
> >> >>>>
> >> >>>>
> >> >>> The main idea is to reuse a maintainable IP stack outside of U-boot.
> >> >> LWIP
> >> >>> can give a nice separation between IP stack code and network application
> >> >>> code. I.e. application should not take care about any TCP details (SYN,
> >> >>> ACK, retransmission, reassembly etc) and should open connection and use
> >> >>> functions similar to recv() and send() to transfer data. Data means
> >> >>> application data, no network packets. And LWIP allows
> >> >>> us to do that.
> >> >>> Because LWIP has an API similar to sockets, it has to be very easy to
> >> >> port
> >> >>> a linux application to LWIP. Then you can test it with a tap device. Then
> >> >>> copy sources to U-boot, add a small integration layer (cmd command to
> >> >>> call), compile and use.
> >> >>>
> >> >>> So my suggestion was:
> >> >>> - do not maintain new network stack code in the current U-boot. Use lwip
> >> >>> sources as an external project. All bugs related to network stack go to
> >> >>> lwip project first, then sync with U-boot.
> >> >>> - maintain network apps code* or
> >> >>> -- inside U-boot. Write our own code for application and maintain it
> >> >>> inside U-boot.
> >> >>> -- inside LWIP. Add examples to LWIP which are suitable for both
> >> >> U-boot
> >> >>> and LWIP.
> >> >>>
> >> >>> * Let's define a U-boot network application as a cmd command. It might be
> >> >>> ping, wget (http or https download), telnet, arp dns etc..
> >> >>>
> >> >>> Let's consider the real use case, like HTTPS download client. We need to
> >> >>> enable TLS connection, validate certificates, then do http download.
> >> >>> Looking at the current code of wget command it's quite difficult to
> >> >>> implement this due to the application having some protol level things. On
> >> >>> the other side we can find embedTLS examples to do https download on
> >> >>> sockets. If LWIP socket API is ported then the only thing you need to do
> >> >> is
> >> >>> change socket() -> lwip_socket(), recv()->lwip_recv(),
> >> >> send()->lwip_send()
> >> >>> and etc, even function names are similar. If LWIP socket API is not
> >> >>> supported, then use callback API for recv() and send(), which are also
> >> >>> easy.
> >> >>>
> >> >>> So yes we add extra bytes, but that will allow us to write more complex
> >> >>> apps, use standard debug tools, use applications with very minimal
> >> >>> integration changes, use help from the LWIP community to fix protocol
> >> >> bugs,
> >> >>> etc..
> >> >>> Bunch of things already implemented there:
> >> >>> - ipv6
> >> >>> - dhcp
> >> >>> - snmp
> >> >>> - igmp
> >> >>> - dns
> >> >>> - tcp and udp and raw.
> >> >>> - loopback
> >> >>> - netconn
> >> >>> - socket
> >> >>> - stats
> >> >>> - ppp
> >> >>> (I just followed configurable defines).
> >> >>>
> >> >>>
> >> >>> And please make sure to disable the previous support, my guess fro that
> >> >>>> much growth is that you didn't.
> >> >>>>
> >> >>>
> >> >>> # CONFIG_PROT_TCP is not set
> >> >>> # CONFIG_PROT_UDP is not set
> >> >>> # CONFIG_UDP_CHECKSUM is not set
> >> >>> # CONFIG_UDP_FUNCTION_FASTBOOT is not set
> >> >>> # CONFIG_CMD_WGET is not set
> >> >>
> >> >> I think you need to step back and figure out a better way to measure the
> >> >> size change and growth.
> >> >>
> >> >> I am not interested in a path that long term means two networking stacks
> >> >> in U-Boot.
> >> >>
> >> >> I am not interested in massively growing the overall binary size for
> >> >> every platform. Given how much larger just TCP support is, that's
> >> >> strongly implying a huge growth for the older use cases too.
> >> >>
> >> >> But I also suspect given the overall reputation that LWIP enjoys,
> >> >> there's something amiss here.
> >> >>
> >> >> --
> >> >> Tom
> >> >>
> >> >
> >> > +cc lwip-devel@ mailing list, maybe they have something to add.
> >>
> >> I do think using lwIP instead of "inventing yet another IP stack" is a
> >> good idea! However, in terms of code size, lwIP will lose against what's
> >> in U-Boot at present. And this is only natural, as lwIP is a "full-size"
> >> stack supporting multiple concurrently running applications while the
> >> current IP stack in U-Boot is rather "crippled" down to just what the
> >> implementor needed at the time of writing.
> >>
> >> One example of this is that (if I remember correctly), U-Boot only has
> >> one single network packet buffer, while lwIP has support for multiple
> >> buffers. When speaking of TCP (forgive me if I'm wrong, I've lost track
> >> of that development in U-Boot about 3 years ago), we're comparing "we
> >> have implemented everything we need so that it just kind of works" to
> >> "we can easily add a HTTPS client to download something over the
> >> internet just by enabling more compile options".
> >>
> >> Also, when comparing lwIP to U-Boot TCP code size, keep in mind that
> >> U-Boot TCP (at least that of some years ago) is far from complete when
> >> compared to lwIP!
> >>
> >> lwIP is meant to be highly configurable and we're always open to add yet
> >> more options to leave out more code when it's not needed. However, I
> >> think there are some design decisions that will make lwIP larger than
> >> the current IP stack in U-Boot. To me, that's a natural result of having
> >> a "generic code" approach vs "developed to our needs". However, while
> >> DHCP + BOOTP and even a simple network console was rather easy to
> >> implement, I would not recommend implementing your own HTTPS download
> >> but rather using the existing lwIP + apps for that.
> >>
> >> In the end, I cannot take the decision from you. In my opinion, lwIP
> >> would be the better decision in terms of future work load and
> >> compatibility, but in the short run, it *will* lead to bigger binaries
> >> at least in some setups. And I do know from my past that it sometimes
> >> has been a pain to try and stuff a new U-Boot release into the existing
> >> space of flash or RAM, so that's not an easy decision.
> >>
> >> If you do take the lwIP approach however, let us know if we can help!
> >>
> >> Regards,
> >> Simon
> >>
> >> >
> >> > My measurements say that the current U-boot IP stack + wget command adds an
> >> > additional 9 Kbytes.
> >> > The minimal configuration of LWIP with wget command is 30 Kbytes.
> >> > (compiled out all asserts, debugs, not used protocols etc.).
> >> >
> >> > And the most bigger functions are tcp in/out itself:
> >> > * These functions are generally called in the order (ip_input() ->)
> >> > * tcp_input() -> * tcp_process() -> tcp_receive() (-> application).
> >> >
> >> > +tcp_input - 4364 +4364
> >> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n118
> >> > +tcp_receive - 3444 +3444
> >> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_in.c#n1154
> >> > +tcp_write - 2192 +2192
> >> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n393
> >> > +ip4_reass - 2096 +2096
> >> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/ipv4/ip4_frag.c#n503
> >> > +tcp_output - 1616 +1616
> >> > https://git.savannah.nongnu.org/cgit/lwip.git/tree/src/core/tcp_out.c#n1241
> >> >
> >> > If we transfer current net commands to lwip then we can decrease the size,
> >> > because of functions reuse.
> >> > And if we turn on all features in lwip it will be about 50 Kbytes.
> >> >
> >> > BR,
> >> > Maxim.
> >> >
> >> >
> >> > _______________________________________________
> >> > lwip-devel mailing list
> >> > lwip-devel@nongnu.org
> >> > https://lists.nongnu.org/mailman/listinfo/lwip-devel
^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [lwip-devel] [RFC PATCH 0/5] LWIP stack integration
2023-06-11 8:24 ` Simon Glass
@ 2023-06-11 18:34 ` Tom Rini
0 siblings, 0 replies; 37+ messages in thread
From: Tom Rini @ 2023-06-11 18:34 UTC (permalink / raw)
To: Simon Glass
Cc: Ilias Apalodimas, Maxim Uvarov, Simon Goldschmidt, u-boot,
pbrobinson, joe.hershberger, rfried.dev, lwip-devel
[-- Attachment #1: Type: text/plain, Size: 3548 bytes --]
On Sun, Jun 11, 2023 at 09:24:14AM +0100, Simon Glass wrote:
> Hi,
>
> On Wed, 7 Jun 2023 at 10:47, Ilias Apalodimas
> <ilias.apalodimas@linaro.org> wrote:
> >
> > Hi Maxim,
> >
> > On Tue, 6 Jun 2023 at 17:33, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
> > >
> > > Greetings,
> > >
> > > I implemented the tftp client (that was quick due to lwip has example app for tftp), and did some more measurements.
> > > I uploaded patches here if somebody want to do his own measurements:
> > > https://github.com/muvarov/uboot-lwip
> > >
> > > measure 1:
> > > 976K - total (total means lwip with all 3 commands ping, tftp, wget)
> > > 971K - total - tftp (total, but disable/minus tftp)
> > > 965K - total - tftp - wget (disable tftp and wget)
> > > 963K - total - tftp - wget - ping (disable tftp, wget, ping)
> > > 931K - no lwip
> > >
> > > result 1: lwip tftp (+ udp protocol) protocol 976-971k = 5kb
> > > result 2: lwip ping command 965- 963 = 2kb
> > > result 3: lwip wget command 971- 965 = 6kb
> > > result 4: lwip core stack with apps 976 - 931 = 45kb
> >
> > So tftp = 5kb, wget = 6kb ping =2kb and lwip = 32kb
> >
> > >
> > > measure 2:
> > > 890K - no CONFIG_NET_CMD
> > > 930K - + lwip tftp only
> > > 937K - + full lwip (ping wget tftp)
> > >
> > > result 1: 937-890=47kb ( lwip + all 3 commands)
> > > result 2: 937-930=7kb (ping and lwip command)
> >
> > I am not sure I understand this measurement. How is this different
> > from measurement 1 where the entire binary was 976K?
> >
> > >
> > > measure 3:
> > > 904K - no lwip, CMD_NET_TFTP=y
> > > 900K - no lwip, CMD_NET_TFTP=n
> > > result 1: original u-boot tftp command 904-900=4kb
> > > 890K - no lwip, CMD_NET=n
> > > result 2: 900-890=10k original u-boot net/IP stack.
> > >
> > > My findings for all that measurements and lwip configuration:
> > > 1. The original u-boot net stack (packet process and up layers) is 10k vs lwip 40k (the very minimal settings were 30k).
> > > 2. Network applications size is about the same 4kb for tftp original command 5kb for lwip.
> > > 3. It's quite easy to reuse LWIP examples to implement the same functionality for the U-boot.
> > > 4. I still think that there are other criterias which might have more priority than size (bug free code, code reuse, development speed, compatible API to posix and etc).
> >
> > Yes, there are other criteria and certainly having a complete network
> > stack might be worth it in many cases, but we need to keep in mind
> > 30kb might be a lot for some systems.
> >
> > I personally think this is decent and we can optimize lwip more in the
> > future. Tom, Simon, how about adding lwip as 'experimental' and
> > making it depend on !CMD_NET or something similar?
>
> That seems OK to me, but we don't really want two network stacks, so
> we'd need to set an expectation that we would move to lwip.
Yes, we'll need to move on to evaluating that once we can show and use
lwip as a replacement for most cases.
> I wonder why it is so large? I saw mention of it supporting multiple
> buffers and perhaps having a fuller implementation of the protocols.
> But it makes U-Boot's stack seem super-slim in comparison. I wonder if
> lwip could support just a single buffer and reduced functionality in
> other areas?
Well, right. Seeing what space related tuneables we can introduce
and/or further tune down will be of interest. But that will be easier
to do once it's easier to try out lwip in U-Boot itself.
--
Tom
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]
^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2023-06-11 18:35 UTC | newest]
Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-05 10:25 [RFC PATCH 0/5] LWIP stack integration Maxim Uvarov
2023-05-05 10:25 ` [RFC PATCH 1/5] add lwip-external submodule Maxim Uvarov
2023-05-08 14:43 ` Simon Glass
2023-05-10 7:40 ` Ilias Apalodimas
2023-05-10 14:46 ` Peter Robinson
2023-05-11 13:51 ` Tom Rini
2023-05-11 13:51 ` Tom Rini
2023-05-05 10:25 ` [RFC PATCH 2/5] lib/lwip: compile-in core files Maxim Uvarov
2023-05-05 10:25 ` [RFC PATCH 3/5] add doc/README.lwip Maxim Uvarov
2023-05-11 13:51 ` Tom Rini
2023-05-05 10:25 ` [RFC PATCH 4/5] add doc/README.lwip.size Maxim Uvarov
2023-05-11 13:51 ` Tom Rini
2023-05-05 10:25 ` [RFC PATCH 5/5] lwip: implement wget command from http_client.c example Maxim Uvarov
2023-05-05 10:27 ` [RFC PATCH 0/5] LWIP stack integration Ilias Apalodimas
2023-05-08 21:23 ` Simon Glass
2023-05-11 13:28 ` Maxim Uvarov
2023-05-11 13:52 ` Tom Rini
2023-05-15 15:25 ` Maxim Uvarov
2023-05-15 15:39 ` Tom Rini
2023-05-19 13:17 ` Ilias Apalodimas
2023-05-19 13:52 ` Tom Rini
2023-05-22 9:01 ` Maxim Uvarov
2023-05-22 13:33 ` Ilias Apalodimas
2023-05-22 14:20 ` Tom Rini
2023-05-22 16:40 ` Maxim Uvarov
2023-05-22 21:23 ` Tom Rini
2023-05-24 14:05 ` Maxim Uvarov
2023-05-24 20:18 ` [lwip-devel] " Simon Goldschmidt
2023-06-06 14:33 ` Maxim Uvarov
2023-06-07 9:46 ` Ilias Apalodimas
2023-06-07 12:01 ` Maxim Uvarov
2023-06-11 8:24 ` Simon Glass
2023-06-11 18:34 ` Tom Rini
2023-06-07 20:07 ` Tom Rini
2023-06-08 10:14 ` Maxim Uvarov
2023-06-08 17:52 ` Ilias Apalodimas
2023-06-09 7:37 ` Peter Robinson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.