[PATCH v3 00/60] Complete 'continuous' Online Check Mode Implementation.

[PATCH v3 00/60] Complete 'continuous' Online Check Mode Implementation.

[Grant Erickson]

This completes the implementation of 'EnableOnlineToReadyTransition'
begun by Emmanuel Vautrin with commit f0bd0e8fe578 ("service: Add
online to ready transition feature").

When asserted along with 'EnableOnlineCheck',
'EnableOnlineToReadyTransition' effects what can be thought of as a
"continuous" online check mode, which is different from the "one-shot"
online check mode when 'EnableOnlineToReadyTransition' is not asserted
but 'EnableOnlineCheck' is. Effectively, these two Booleans encode
three online check modes:

    1. None (!EnableOnlineCheck)
    2. One-shot (EnableOnlineCheck && !EnableOnlineToReadyTransition)
    3. Continuous (EnableOnlineCheck && EnableOnlineToReadyTransition)

With this change, these three modes are formalized and introduced as a
new 'OnlineCheckMode' configuration setting which is intended to
eventually deprecate 'EnableOnlineCheck' (long-term) and
'EnableOnlineToReadyTransition' (short-term, as it never made it
beyond the EXPERIMENTAL designation).

In "none" mode, as has been the case historically, there are no
"online" HTTP-based Internet reachability checks. Any connected
service and the manager state will terminate at the "ready" state and
will not progress to "online".

In "one-shot" mode, as has been the case historically, there is a
single, one-shot "online" HTTP-based Internet reachability check for
the default service. When the check succeeds, the associated service
and the manager state will terminate at the "online" state. When the
check fails, subsequent checks will be rescheduled according to
"OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
"OnlineCheckMaxInterval" and will continue indefinitely until one
succeeds or until the service is disconnected.

In "continuous" mode, which is finalized with this change, there are
ongoing "online" HTTP-based Internet reachability check for the
default service. As with "one-shot" mode, when the first check
succeeds, the associated service and the manager state will terminate
at the "online" state. Thereafter, subsequent checks will be scheduled
according to "OnlineCheckIntervalStyle" and
"OnlineCheckMaxInterval". When the check fails, subsequent checks will
be rescheduled according to "OnlineCheckIntervalStyle",
"OnlineCheckInitialInterval", and "OnlineCheckMaxInterval". This is
largely unchanged. However, what is new with this change is that when
and if "OnlineCheckFailuresThreshold" is met, the service and manager
state will be demoted to "ready" and the service will have its "Error"
property set to "online-check-failed" while subsequent checks will
continue. In the interim, if available, another service may be
promoted to the default service and online checks will be initiated
for it. When and if, for the demoted service,
"OnlineCheckSuccessesThreshold" is met, the service "Error" property
will be cleared and the service state promoted to "online",
potentially causing it to become the default service again.

Grant Erickson (60):
  service: Change return type of '__connman_service_wispr_start'.
  service: Sanity check the 'type' param in
    '__connman_service_wispr_start'.
  service: Rename 'online_check_enabled_check'.
  service: Change return type of 'start_online_check'.
  service: Change return type of 'start_wispr_if_connected'.
  service: Introduce and use online check 'active' state.
  service: Document 'online_check_state' 'active' member.
  service: Document 'online_check_is_active'.
  service: Document 'online_check_active_set_value'.
  service: Document 'online_check_active_{clear,set}'.
  service: Rename 'start_wispr_if_connected'.
  service: Refactor 'start_online_check_if_connected'.
  service: Document 'start_online_check_if_connected_with_type'.
  service: Add an explicit conditional for IPv6 in
    'reschedule_online_check'.
  session: Add service identifier to 'DBG' in four functions.
  service: Add online check failure service error enumeration and
    string.
  doc/service: Document online check failure service error.
  service: Introduce and use online check 'failures/successes' state.
  service: Document 'online_check_state' 'failures/successes' members.
  main: Introduce the 'OnlineCheck{Failures,Successes}Threshold'
    settings.
  doc: Document the 'OnlineCheck{Failures,Successes}Threshold' settings.
  service: Add per-service online check failure threshold state.
  service: Document 'connman_service'
    'online_check_failures_met_threshold' member.
  service: Document 'connman_service' 'online_check_state_ipv{4,6}'
    members.
  service: Move 'set_error' forward declaration earlier in the source.
  service: Capture IP configuration state by value rather than by
    pointer.
  service: Complete 'continuous' online check mode implementation.
  service: Document 'online_check_failures_threshold_was_met'.
  service: Document 'online_check_failures_threshold_was_met_set_value'.
  service: Document
    'online_check_failures_threshold_was_met_{set,clear}'.
  service: Document 'online_check_counter_reset'.
  service: Document 'online_check_{failures,successes}_reset'.
  service: Document 'online_check_state_reset'.
  service: Document 'online_check_counters_log'.
  service: Document 'online_check_counter_threshold_is_met'.
  service: Document 'is_online_check_failure_threshold_met_predicate'.
  service: Document 'online_check_failures_threshold_is_met'.
  service: Document 'is_online_check_success_threshold_met_predicate'.
  service: Document 'online_check_successes_threshold_is_met'.
  service: Document 'online_check_counter_increment_and_log'.
  service: Document 'online_check_log_success'.
  service: Document
    'continuous_online_check_log_{counter,failures,successes}_threshold_met'.
  service: Document 'handle_{continuous,oneshot}_online_check_success'.
  service: Document 'online_check_log_failure'.
  service: Document 'handle_{continuous,oneshot}_online_check_failure'.
  service: Add @sa documentation references.
  service: Document the internals of 'service_compare'.
  service: Document interval assignment in
    '__connman_service_wispr_start'.
  service: Document 'downgrade_connected_services'.
  service: Document 'service_schedule_changed'.
  service: Document 'service_route_changed'.
  service: Add function parameter to 'default_changed' and
    'service_list_sort'.
  service: Add @sa documentation reference to 'current_default'.
  service: Add 'DBG' to 'downgrade_connected_services'.
  main/service: Introduce 'OnlineCheckMode' setting.
  service: Leverage 'OnlineCheckMode' setting.
  doc: Document the 'OnlineCheckMode' setting.
  main: Add comments tags for online check-related settings.
  main: Consolidate logging of online check-related settings.
  main: Add deprecation warnings for 'EnableOnlineCheck' and
    'EnableOnlineToReadyTransition'.

 README                |   91 +-
 doc/connman.conf.5.in |   85 +-
 doc/service-api.txt   |    3 +-
 include/service.h     |   26 +-
 src/connman.h         |   18 +-
 src/main.c            |  177 +++-
 src/main.conf         |  122 ++-
 src/service.c         | 2133 ++++++++++++++++++++++++++++++++++++-----
 8 files changed, 2356 insertions(+), 299 deletions(-)

-- 
2.42.0

[PATCH v3 01/60] service: Change return type of '__connman_service_wispr_start'.

[Grant Erickson]

This changes the return type of '__connman_service_wispr_start' to
allow callers to conditionally handle its return status which, as of
this revision, is fairly simple but will be expanded over time.
---
 src/connman.h |  2 +-
 src/service.c | 10 +++++++++-
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/src/connman.h b/src/connman.h
index 622a778590b1..796cb202570e 100644
--- a/src/connman.h
+++ b/src/connman.h
@@ -743,7 +743,7 @@ struct connman_ipconfig *__connman_service_get_ipconfig(
 				struct connman_service *service, int family);
 void __connman_service_notify_ipv4_configuration(
 				struct connman_service *service);
-void __connman_service_wispr_start(struct connman_service *service,
+int __connman_service_wispr_start(struct connman_service *service,
                                 enum connman_ipconfig_type type);
 bool __connman_service_is_connected_state(const struct connman_service *service,
 					enum connman_ipconfig_type type);
diff --git a/src/service.c b/src/service.c
index 8e057e051d63..7af0d4a5f60c 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2410,13 +2410,16 @@ static void start_wispr_if_connected(struct connman_service *service)
  *                           "online" reachability check is to be
  *                           started.
  *
+ *  @retval  0          If successful.
+ *  @retval  -EINVAL    If @a service is null or @a type is invalid.
+ *
  *  @sa cancel_online_check
  *  @sa start_online_check
  *  @sa complete_online_check
  *  @sa start_wispr_if_connected
  *
  */
-void __connman_service_wispr_start(struct connman_service *service,
+int __connman_service_wispr_start(struct connman_service *service,
 					enum connman_ipconfig_type type)
 {
 	DBG("service %p (%s) type %d (%s)",
@@ -2424,6 +2427,9 @@ void __connman_service_wispr_start(struct connman_service *service,
 		connman_service_get_identifier(service),
 		type, __connman_ipconfig_type2string(type));
 
+	if (!service)
+		return -EINVAL;
+
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV4)
 		service->online_check_state_ipv4.interval =
 					online_check_initial_interval;
@@ -2433,6 +2439,8 @@ void __connman_service_wispr_start(struct connman_service *service,
 
 	__connman_wispr_start(service, type,
 			online_check_connect_timeout_ms, complete_online_check);
+
+	return 0;
 }
 
 /**
-- 
2.42.0

[PATCH v3 02/60] service: Sanity check the 'type' param in '__connman_service_wispr_start'.

[Grant Erickson]

This adds sanity checking for the 'type' parameter in
'__connman_service_wispr_start'. In addition, it adds an explicit
conditional test for the 'CONNMAN_IPCONFIG_TYPE_IPV6'.

Both of these ensure that neither 'CONNMAN_IPCONFIG_TYPE_ALL' or
'CONNMAN_IPCONFIG_TYPE_UNKNOWN' trigger a WISPr start but, instead,
error out.
---
 src/service.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/src/service.c b/src/service.c
index 7af0d4a5f60c..e5cd6fcd306a 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2430,10 +2430,18 @@ int __connman_service_wispr_start(struct connman_service *service,
 	if (!service)
 		return -EINVAL;
 
+	switch (type) {
+	case CONNMAN_IPCONFIG_TYPE_IPV4:
+	case CONNMAN_IPCONFIG_TYPE_IPV6:
+		break;
+	default:
+		return -EINVAL;
+	}
+
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV4)
 		service->online_check_state_ipv4.interval =
 					online_check_initial_interval;
-	else
+	else if (type == CONNMAN_IPCONFIG_TYPE_IPV6)
 		service->online_check_state_ipv6.interval =
 					online_check_initial_interval;
 
-- 
2.42.0

[PATCH v3 03/60] service: Rename 'online_check_enabled_check'.

[Grant Erickson]

Since 'online_check_enabled_check' is a predicate function, follow
prevailing function name style for such predicates by adding '_is_' to
the name.
---
 src/service.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/src/service.c b/src/service.c
index e5cd6fcd306a..87ef3b575b1f 100644
--- a/src/service.c
+++ b/src/service.c
@@ -1881,7 +1881,8 @@ static void cancel_online_check(struct connman_service *service,
  *  @sa start_wispr_if_connected
  *
  */
-static bool online_check_enabled_check(const struct connman_service *service)
+static bool online_check_is_enabled_check(
+		const struct connman_service *service)
 {
 	if (!connman_setting_get_bool("EnableOnlineCheck")) {
 		connman_info("Online check disabled. "
@@ -1927,7 +1928,7 @@ static void start_online_check(struct connman_service *service,
 		type,
 		__connman_ipconfig_type2string(type));
 
-	if (!online_check_enabled_check(service))
+	if (!online_check_is_enabled_check(service))
 		return;
 
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV6 || check_proxy_setup(service)) {
@@ -2380,7 +2381,7 @@ static void start_wispr_if_connected(struct connman_service *service)
 		service,
 		connman_service_get_identifier(service));
 
-	if (!online_check_enabled_check(service))
+	if (!online_check_is_enabled_check(service))
 		return;
 
 	if (__connman_service_is_connected_state(service,
-- 
2.42.0

[PATCH v3 04/60] service: Change return type of 'start_online_check'.

[Grant Erickson]

This changes the return type of 'start_online_check' from 'void' to
'int' to allow callers to conditionally handle its return status.
---
 src/service.c | 28 ++++++++++++++++++++++++----
 1 file changed, 24 insertions(+), 4 deletions(-)

diff --git a/src/service.c b/src/service.c
index 87ef3b575b1f..0c8a4bc0a0c9 100644
--- a/src/service.c
+++ b/src/service.c
@@ -1913,28 +1913,48 @@ static bool online_check_is_enabled_check(
  *                           "online" reachability check is to be
  *                           started.
  *
+ *  @retval  0          If successful.
+ *  @retval  -EINVAL    If @a service is null or @a type is invalid.
+ *  @retval  -EPERM     If online checks are disabled via
+ *                      configuration.
+ *
  *  @sa cancel_online_check
  *  @sa complete_online_check
  *  @sa start_wispr_if_connected
  *  @sa __connman_service_wispr_start
  *
  */
-static void start_online_check(struct connman_service *service,
+static int start_online_check(struct connman_service *service,
 				enum connman_ipconfig_type type)
 {
+	int status = 0;
+
 	DBG("service %p (%s) type %d (%s) maybe start WISPr",
 		service,
 		connman_service_get_identifier(service),
 		type,
 		__connman_ipconfig_type2string(type));
 
-	if (!online_check_is_enabled_check(service))
-		return;
+	if (!service) {
+		status = -EINVAL;
+		goto done;
+	}
+
+	if (!online_check_is_enabled_check(service)) {
+		status = -EPERM;
+		goto done;
+	}
 
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV6 || check_proxy_setup(service)) {
 		cancel_online_check(service, type);
-		__connman_service_wispr_start(service, type);
+
+		status = __connman_service_wispr_start(service, type);
 	}
+
+done:
+	DBG("status %d (%s)", status, strerror(-status));
+
+	return status;
 }
 
 /**
-- 
2.42.0

[PATCH v3 05/60] service: Change return type of 'start_wispr_if_connected'.

[Grant Erickson]

This changes the return type of 'start_wispr_if_connected' from 'void'
to 'int' to allow callers to conditionally handle its return status.
---
 src/service.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/src/service.c b/src/service.c
index 0c8a4bc0a0c9..5077e3fb2d1a 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2394,15 +2394,23 @@ static void complete_online_check(struct connman_service *service,
  *                       reachability probes if the IPv4 or IPv6 state
  *                       is "connected" (that is, "ready" or "online").
  *
+ *  @retval  0          If successful.
+ *  @retval  -EINVAL    If @a service is null or @a type is invalid.
+ *  @retval  -EPERM     If online checks are disabled via
+ *                      configuration.
+ *
  */
-static void start_wispr_if_connected(struct connman_service *service)
+static int start_wispr_if_connected(struct connman_service *service)
 {
 	DBG("service %p (%s) maybe start WISPr",
 		service,
 		connman_service_get_identifier(service));
 
+	if (!service)
+		return -EINVAL;
+
 	if (!online_check_is_enabled_check(service))
-		return;
+		return -EPERM;
 
 	if (__connman_service_is_connected_state(service,
 			CONNMAN_IPCONFIG_TYPE_IPV4))
@@ -2413,6 +2421,8 @@ static void start_wispr_if_connected(struct connman_service *service)
 			CONNMAN_IPCONFIG_TYPE_IPV6))
 		__connman_service_wispr_start(service,
 					CONNMAN_IPCONFIG_TYPE_IPV6);
+
+	return 0;
 }
 
 /**
-- 
2.42.0

[PATCH v3 06/60] service: Introduce and use online check 'active' state.

[Grant Erickson]

This introduces a per-IP configuration online check 'active' state
member that tracks whether an online check is active / in-flight.

Even with commit 864e48f34e8c ("service: Leverage
'__connman_wispr_cancel'.") it remains too easy to inadvertently
schedule more outstanding online checks for a service than desired
(ideally, there should be one each for IPv4 and IPv6, to the extent
they are "connected"). This is all the more so when
'EnableOnlineToReadyTransition' is asserted / online check mode is
continuous.

Rather than scheduling redundant online check, the 'active' state is
now used to enable the return of -EALREADY when it is asserted from
functions capable of initiating and scheduling an online check.
---
 src/service.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 92 insertions(+)

diff --git a/src/service.c b/src/service.c
index 5077e3fb2d1a..669337ab6734 100644
--- a/src/service.c
+++ b/src/service.c
@@ -93,6 +93,8 @@ struct connman_stats_counter {
  *
  */
 struct online_check_state {
+	bool active;
+
 	/**
 	 *  The current GLib main loop timer identifier.
 	 *
@@ -1690,6 +1692,79 @@ static bool check_proxy_setup(struct connman_service *service)
 	return false;
 }
 
+static bool online_check_is_active(const struct connman_service *service,
+		enum connman_ipconfig_type type)
+{
+	bool do_ipv4 = false, do_ipv6 = false;
+	bool active = false;
+
+	DBG("service %p (%s) type %d (%s)",
+		service, connman_service_get_identifier(service),
+		type, __connman_ipconfig_type2string(type));
+
+	if (!service)
+		goto done;
+
+	if (type == CONNMAN_IPCONFIG_TYPE_IPV4)
+		do_ipv4 = true;
+	else if (type == CONNMAN_IPCONFIG_TYPE_IPV6)
+		do_ipv6 = true;
+	else if (type == CONNMAN_IPCONFIG_TYPE_ALL)
+		do_ipv4 = do_ipv6 = true;
+	else
+		goto done;
+
+	active = (do_ipv4 && service->online_check_state_ipv4.active) ||
+			 (do_ipv6 && service->online_check_state_ipv6.active);
+
+	DBG("active? %u", active);
+
+ done:
+	return active;
+}
+
+static void online_check_active_set_value(struct connman_service *service,
+		enum connman_ipconfig_type type,
+		bool active)
+{
+	bool do_ipv4 = false, do_ipv6 = false;
+
+	DBG("service %p (%s) type %d (%s) active? %u",
+		service, connman_service_get_identifier(service),
+		type, __connman_ipconfig_type2string(type),
+		active);
+
+	if (!service)
+		return;
+
+	if (type == CONNMAN_IPCONFIG_TYPE_IPV4)
+		do_ipv4 = true;
+	else if (type == CONNMAN_IPCONFIG_TYPE_IPV6)
+		do_ipv6 = true;
+	else if (type == CONNMAN_IPCONFIG_TYPE_ALL)
+		do_ipv4 = do_ipv6 = true;
+	else
+		return;
+
+	if (do_ipv4)
+		service->online_check_state_ipv4.active = active;
+
+	if (do_ipv6)
+		service->online_check_state_ipv6.active = active;
+}
+
+static void online_check_active_set(struct connman_service *service,
+		enum connman_ipconfig_type type)
+{
+	online_check_active_set_value(service, type, true);
+}
+
+static void online_check_active_clear(struct connman_service *service,
+		enum connman_ipconfig_type type)
+{
+	online_check_active_set_value(service, type, false);
+}
+
 /**
  *  @brief
  *    Compute a Fibonacci online check timeout based on the specified
@@ -1858,6 +1933,10 @@ static void cancel_online_check(struct connman_service *service,
 		 */
 		connman_service_unref(service);
 	}
+
+    /* Mark the online check state as inactive. */
+
+	online_check_active_clear(service, type);
 }
 
 /**
@@ -1917,6 +1996,8 @@ static bool online_check_is_enabled_check(
  *  @retval  -EINVAL    If @a service is null or @a type is invalid.
  *  @retval  -EPERM     If online checks are disabled via
  *                      configuration.
+ *  @retval  -EALREADY  If online checks are already active for @a
+ *                      service.
  *
  *  @sa cancel_online_check
  *  @sa complete_online_check
@@ -2379,6 +2460,8 @@ static void complete_online_check(struct connman_service *service,
 
 	if (reschedule)
 		reschedule_online_check(service, type, online_check_state);
+	else
+		online_check_active_clear(service, type);
 }
 
 /**
@@ -2443,6 +2526,8 @@ static int start_wispr_if_connected(struct connman_service *service)
  *
  *  @retval  0          If successful.
  *  @retval  -EINVAL    If @a service is null or @a type is invalid.
+ *  @retval  -EALREADY  If online checks are already active for @a
+ *                      service.
  *
  *  @sa cancel_online_check
  *  @sa start_online_check
@@ -2469,6 +2554,9 @@ int __connman_service_wispr_start(struct connman_service *service,
 		return -EINVAL;
 	}
 
+	if (online_check_is_active(service, type))
+		return -EALREADY;
+
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV4)
 		service->online_check_state_ipv4.interval =
 					online_check_initial_interval;
@@ -2479,6 +2567,10 @@ int __connman_service_wispr_start(struct connman_service *service,
 	__connman_wispr_start(service, type,
 			online_check_connect_timeout_ms, complete_online_check);
 
+	/* Mark the online check state as active. */
+
+	online_check_active_set(service, type);
+
 	return 0;
 }
 
-- 
2.42.0

[PATCH v3 07/60] service: Document 'online_check_state' 'active' member.

[Grant Erickson]

This documents the 'active' member of the 'online_check_state'
structure.
---
 src/service.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/src/service.c b/src/service.c
index 669337ab6734..27d261861b7c 100644
--- a/src/service.c
+++ b/src/service.c
@@ -93,6 +93,9 @@ struct connman_stats_counter {
  *
  */
 struct online_check_state {
+	/**
+	 *  Indicates whether an online check is active and in-flight.
+	 */
 	bool active;
 
 	/**
-- 
2.42.0

[PATCH v3 08/60] service: Document 'online_check_is_active'.

[Grant Erickson]

This adds documentation to the 'online_check_is_active' function.
---
 src/service.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/src/service.c b/src/service.c
index 27d261861b7c..0c1ed63c0ade 100644
--- a/src/service.c
+++ b/src/service.c
@@ -1695,6 +1695,28 @@ static bool check_proxy_setup(struct connman_service *service)
 	return false;
 }
 
+/**
+ *  @brief
+ *    Determine whether an "online" HTTP-based Internet reachability
+ *    check is active.
+ *
+ *  This determines whether an "online" HTTP-based Internet
+ *  reachability check is active for the specified network service IP
+ *  configuration type.
+ *
+ *  @param[in]  service  A pointer to the immutable network service
+ *                       for which to determine whether an "online"
+ *                       HTTP-based Internet reachability is active.
+ *  @param[in]  type     The IP configuration type for which to
+ *                       determine whether an "online" HTTP-based
+ *                       Internet reachability is active.
+ *
+ *  @returns
+ *    True if an "online" HTTP-based Internet reachability check is
+ *    active for the specified network service IP configuration type;
+ *    otherwise, false.
+ *
+ */
 static bool online_check_is_active(const struct connman_service *service,
 		enum connman_ipconfig_type type)
 {
-- 
2.42.0

[PATCH v3 09/60] service: Document 'online_check_active_set_value'.

[Grant Erickson]

This adds documentation to the 'online_check_active_set_value'
function.
---
 src/service.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/src/service.c b/src/service.c
index 0c1ed63c0ade..4238d81c7c0d 100644
--- a/src/service.c
+++ b/src/service.c
@@ -1748,6 +1748,28 @@ static bool online_check_is_active(const struct connman_service *service,
 	return active;
 }
 
+/**
+ *  @brief
+ *    Assign the "online" HTTP-based Internet reachability check
+ *    active state.
+ *
+ *  This assigns the "online" HTTP-based Internet reachability check
+ *  active state for the specified network service IP configuration
+ *  type.
+ *
+ *  @param[in,out]  service  A pointer to the mutable network service
+ *                           for which to assign the "online" HTTP-
+ *                           based Internet reachability active
+ *                           state.
+ *  @param[in]      type     The IP configuration type for which to
+ *                           assign the "online" HTTP-based Internet
+ *                           reachability active state.
+ *  @param[in]      active   The "online" HTTP-based Internet
+ *                           reachability active state to assign.
+ *
+ *  @sa online_check_is_active
+ *
+ */
 static void online_check_active_set_value(struct connman_service *service,
 		enum connman_ipconfig_type type,
 		bool active)
-- 
2.42.0

[PATCH v3 10/60] service: Document 'online_check_active_{clear,set}'.

[Grant Erickson]

This adds documentation to the 'online_check_active_{clear,set}'
functions.
---
 src/service.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/src/service.c b/src/service.c
index 4238d81c7c0d..cd1e56d0cf7a 100644
--- a/src/service.c
+++ b/src/service.c
@@ -1800,12 +1800,54 @@ static void online_check_active_set_value(struct connman_service *service,
 		service->online_check_state_ipv6.active = active;
 }
 
+/**
+ *  @brief
+ *    Set, or assert, the "online" HTTP-based Internet reachability
+ *    check active state.
+ *
+ *  This sets, or asserts, the "online" HTTP-based Internet
+ *  reachability check active state for the specified network service
+ *  IP configuration type.
+ *
+ *  @param[in,out]  service  A pointer to the mutable network service
+ *                           for which to set the "online" HTTP-
+ *                           based Internet reachability active
+ *                           state.
+ *  @param[in]      type     The IP configuration type for which to
+ *                           set the "online" HTTP-based Internet
+ *                           reachability active state.
+ *
+ *  @sa online_check_active_set_value
+ *  @sa online_check_is_active
+ *
+ */
 static void online_check_active_set(struct connman_service *service,
 		enum connman_ipconfig_type type)
 {
 	online_check_active_set_value(service, type, true);
 }
 
+/**
+ *  @brief
+ *    Clear, or deassert, the "online" HTTP-based Internet
+ *    reachability check active state.
+ *
+ *  This clears, or deasserts, the "online" HTTP-based Internet
+ *  reachability check active state for the specified network service
+ *  IP configuration type.
+ *
+ *  @param[in,out]  service  A pointer to the mutable network service
+ *                           for which to clear the "online" HTTP-
+ *                           based Internet reachability active
+ *                           state.
+ *  @param[in]      type     The IP configuration type for which to
+ *                           clear the "online" HTTP-based Internet
+ *                           reachability active state.
+ *
+ *  @sa online_check_active_set_value
+ *  @sa online_check_is_active
+ *
+ */
 static void online_check_active_clear(struct connman_service *service,
 		enum connman_ipconfig_type type)
 {
-- 
2.42.0

[PATCH v3 11/60] service: Rename 'start_wispr_if_connected'.

[Grant Erickson]

This renames 'start_wispr_if_connected' to
'start_online_check_if_connected' since it aligns more closely with
other '*_online_check' functions and simply uses WISPr to effect the
online check.
---
 src/service.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/src/service.c b/src/service.c
index cd1e56d0cf7a..cd684fbc0b4f 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2046,7 +2046,7 @@ static void cancel_online_check(struct connman_service *service,
  *    enabled for the specified network service; otherwise, false.
  *
  *  @sa start_online_check
- *  @sa start_wispr_if_connected
+ *  @sa start_online_check_if_connected
  *
  */
 static bool online_check_is_enabled_check(
@@ -2090,7 +2090,7 @@ static bool online_check_is_enabled_check(
  *
  *  @sa cancel_online_check
  *  @sa complete_online_check
- *  @sa start_wispr_if_connected
+ *  @sa start_online_check_if_connected
  *  @sa __connman_service_wispr_start
  *
  */
@@ -2503,7 +2503,7 @@ done:
  *
  *  @sa cancel_online_check
  *  @sa start_online_check
- *  @sa start_wispr_if_connected
+ *  @sa start_online_check_if_connected
  *  @sa __connman_service_wispr_start
  *
  */
@@ -2572,7 +2572,7 @@ static void complete_online_check(struct connman_service *service,
  *                      configuration.
  *
  */
-static int start_wispr_if_connected(struct connman_service *service)
+static int start_online_check_if_connected(struct connman_service *service)
 {
 	DBG("service %p (%s) maybe start WISPr",
 		service,
@@ -2621,7 +2621,7 @@ static int start_wispr_if_connected(struct connman_service *service)
  *  @sa cancel_online_check
  *  @sa start_online_check
  *  @sa complete_online_check
- *  @sa start_wispr_if_connected
+ *  @sa start_online_check_if_connected
  *
  */
 int __connman_service_wispr_start(struct connman_service *service,
@@ -2908,7 +2908,7 @@ static void default_changed(void)
 				connman_setting_get_bool("AllowDomainnameUpdates"))
 			__connman_utsname_set_domainname(service->domainname);
 
-		start_wispr_if_connected(service);
+		start_online_check_if_connected(service);
 
 		/*
 		 * Connect VPN automatically when new default service
@@ -4927,7 +4927,7 @@ static DBusMessage *set_property(DBusConnection *conn,
 		nameserver_add_all(service, CONNMAN_IPCONFIG_TYPE_ALL);
 		dns_configuration_changed(service);
 
-		start_wispr_if_connected(service);
+		start_online_check_if_connected(service);
 
 		service_save(service);
 	} else if (g_str_equal(name, "Timeservers.Configuration")) {
-- 
2.42.0

[PATCH v3 12/60] service: Refactor 'start_online_check_if_connected'.

[Grant Erickson]

This refactors 'start_online_check_if_connected' into a support
function, 'start_online_check_if_connected_with_type' to reduce
copy-and-paste and to make common, checking the 'type' parameter,
returning '-ENOTCONN' if the '__connman_service_is_connected_state'
predicate is false, and passing along return status from
'__connman_service_wispr_start'.
---
 src/service.c | 63 +++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 49 insertions(+), 14 deletions(-)

diff --git a/src/service.c b/src/service.c
index cd684fbc0b4f..b1880fbc6457 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2553,6 +2553,28 @@ static void complete_online_check(struct connman_service *service,
 		online_check_active_clear(service, type);
 }
 
+static int start_online_check_if_connected_with_type(
+					struct connman_service *service,
+					enum connman_ipconfig_type type)
+{
+	int status = 0;
+
+	switch (type) {
+	case CONNMAN_IPCONFIG_TYPE_IPV4:
+	case CONNMAN_IPCONFIG_TYPE_IPV6:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	if (!__connman_service_is_connected_state(service, type))
+		status = -ENOTCONN;
+	else
+		status = __connman_service_wispr_start(service, type);
+
+	return status;
+}
+
 /**
  *  @brief
  *    Start HTTP-based Internet reachability probes if the specified
@@ -2562,21 +2584,34 @@ static void complete_online_check(struct connman_service *service,
  *  reachability probes if the IPv4 state or IPv6 state is connected
  *  (that is, "ready" or "online").
  *
- *  @param[in]  service  A pointer to a mutable service on which to start
- *                       reachability probes if the IPv4 or IPv6 state
- *                       is "connected" (that is, "ready" or "online").
+ *  @param[in,out]  service  A pointer to a mutable service on which
+ *                           to start "online" HTTP-based Internet
+ *                           reachability checks if the IPv4 or IPv6
+ *                           state is "connected" (that is, "ready" or
+ *                           "online").
  *
  *  @retval  0          If successful.
  *  @retval  -EINVAL    If @a service is null or @a type is invalid.
  *  @retval  -EPERM     If online checks are disabled via
  *                      configuration.
+ *  @retval  -ENOTCONN  If @a service is not "connected" (that is,
+ *                      "ready" or "online").
+ *  @retval  -EALEADY   If online checks are already active for @a
+ *                      service.
+ *
+ *  @sa start_online_check
+ *  @sa start_online_check_if_connected_with_type
  *
  */
 static int start_online_check_if_connected(struct connman_service *service)
 {
-	DBG("service %p (%s) maybe start WISPr",
+	int status4 = 0, status6 = 0;
+
+	DBG("service %p (%s) state4 %d (%s) state6 %d (%s) maybe start WISPr",
 		service,
-		connman_service_get_identifier(service));
+		connman_service_get_identifier(service),
+		service->state_ipv4, state2string(service->state_ipv4),
+		service->state_ipv6, state2string(service->state_ipv6));
 
 	if (!service)
 		return -EINVAL;
@@ -2584,17 +2619,17 @@ static int start_online_check_if_connected(struct connman_service *service)
 	if (!online_check_is_enabled_check(service))
 		return -EPERM;
 
-	if (__connman_service_is_connected_state(service,
-			CONNMAN_IPCONFIG_TYPE_IPV4))
-		__connman_service_wispr_start(service,
-					CONNMAN_IPCONFIG_TYPE_IPV4);
+	status4 = start_online_check_if_connected_with_type(service,
+			CONNMAN_IPCONFIG_TYPE_IPV4);
 
-	if (__connman_service_is_connected_state(service,
-			CONNMAN_IPCONFIG_TYPE_IPV6))
-		__connman_service_wispr_start(service,
-					CONNMAN_IPCONFIG_TYPE_IPV6);
+	status6 = start_online_check_if_connected_with_type(service,
+			CONNMAN_IPCONFIG_TYPE_IPV6);
 
-	return 0;
+	DBG("status4 %d (%s) status6 %d (%s)",
+		status4, strerror(-status4),
+		status6, strerror(-status6));
+
+	return (status4 < 0 ? status4 : status6);
 }
 
 /**
-- 
2.42.0

[PATCH v3 13/60] service: Document 'start_online_check_if_connected_with_type'.

[Grant Erickson]

This adds documentation to the
'start_online_check_if_connected_with_type' function.
---
 src/service.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/src/service.c b/src/service.c
index b1880fbc6457..b35a4dad1f29 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2553,6 +2553,40 @@ static void complete_online_check(struct connman_service *service,
 		online_check_active_clear(service, type);
 }
 
+/**
+ *  @brief
+ *    Start HTTP-based Internet reachability probes if the specified
+ *    service is connected.
+ *
+ *  This attempts to start IPv4 or IPv6 HTTP-based Internet
+ *  reachability probes if the IPv4 state or IPv6 state is connected
+ *  (that is, "ready" or "online") and if the online check state is
+ *  not already active for the specified network service IP
+ *  configuration type.
+ *
+ *  @param[in,out]  service  A pointer to a mutable service on which
+ *                           to start "online" HTTP-based Internet
+ *                           reachability checks if the IP
+ *                           configuration state associated with @a
+ *                           type is "connected" (that is, "ready" or
+ *                           "online").
+ *  @param[in]      type     The IP configuration type for which to
+ *                           start the "online" HTTP-based Internet
+ *                           reachability checks.
+ *
+ *  @retval  0          If successful.
+ *  @retval  -EINVAL    If @a service is null or @a type is invalid.
+ *  @retval  -EPERM     If online checks are disabled via
+ *                      configuration.
+ *  @retval  -ENOTCONN  If @a service is not "connected" (that is,
+ *                      "ready" or "online").
+ *  @retval  -EALREADY  If online checks are already active for @a
+ *                      service.
+ *
+ *  @sa start_online_check
+ *  @sa start_online_check_if_connected_with_type
+ *
+ */
 static int start_online_check_if_connected_with_type(
 					struct connman_service *service,
 					enum connman_ipconfig_type type)
-- 
2.42.0

[PATCH v3 14/60] service: Add an explicit conditional for IPv6 in 'reschedule_online_check'.

[Grant Erickson]

This adds an explicit conditional check for the 'type' parameter
against 'CONNMAN_IPCONFIG_TYPE_IPV6' to ensure that neither
'CONNMAN_IPCONFIG_TYPE_ALL' or 'CONNMAN_IPCONFIG_TYPE_UNKNOWN'
reschedule an IPv6 online check.
---
 src/service.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/src/service.c b/src/service.c
index b35a4dad1f29..0e25d4879d77 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2293,8 +2293,10 @@ static void reschedule_online_check(struct connman_service *service,
 
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV4)
 		redo_func = redo_wispr_ipv4;
-	else
+	else if (type == CONNMAN_IPCONFIG_TYPE_IPV6)
 		redo_func = redo_wispr_ipv6;
+	else
+		return;
 
 	DBG("updating online checkout timeout period");
 
-- 
2.42.0

[PATCH v3 15/60] session: Add service identifier to 'DBG' in four functions.

[Grant Erickson]

This adds the service identifier to the 'DBG' statements in:

  *  'service_save'
  * '__connman_service_nameserver_append'
  * 'service_schedule_added'
  * 'service_route_changed'

to aid debugging in a multi-technology environment with
"EnableOnlineToReadyTransition" asserted and for consistency with
other 'DBG' statements in this and other modules.
---
 src/service.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/src/service.c b/src/service.c
index 0e25d4879d77..fb901ef16c9b 100644
--- a/src/service.c
+++ b/src/service.c
@@ -705,7 +705,9 @@ static int service_save(struct connman_service *service)
 	const char *cst_str = NULL;
 	int err = 0;
 
-	DBG("service %p new %d", service, service->new_service);
+	DBG("service %p (%s) new %d",
+		service, connman_service_get_identifier(service),
+		service->new_service);
 
 	if (service->new_service)
 		return -ESRCH;
@@ -1231,7 +1233,9 @@ int __connman_service_nameserver_append(struct connman_service *service,
 	char **nameservers;
 	int len, i;
 
-	DBG("service %p nameserver %s auto %d",	service, nameserver, is_auto);
+	DBG("service %p (%s) nameserver %s auto %d",
+		service, connman_service_get_identifier(service),
+		nameserver, is_auto);
 
 	if (!nameserver)
 		return -EINVAL;
@@ -6415,7 +6419,8 @@ static DBusMessage *reset_counters(DBusConnection *conn,
 
 static void service_schedule_added(struct connman_service *service)
 {
-	DBG("service %p", service);
+	DBG("service %p (%s)",
+		service, connman_service_get_identifier(service));
 
 	g_hash_table_remove(services_notify->remove, service->path);
 	g_hash_table_replace(services_notify->add, service->path, service);
@@ -8643,8 +8648,10 @@ static void service_ip_bound(struct connman_ipconfig *ipconfig,
 	type = __connman_ipconfig_get_config_type(ipconfig);
 	method = __connman_ipconfig_get_method(ipconfig);
 
-	DBG("service %p ipconfig %p type %d method %d", service, ipconfig,
-							type, method);
+	DBG("service %p (%s) type %d (%s) ipconfig %p method %d (%s)",
+		service, connman_service_get_identifier(service),
+		type, __connman_ipconfig_type2string(type),
+		ipconfig, method, __connman_ipconfig_method2string(method));
 
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV6 &&
 			method == CONNMAN_IPCONFIG_METHOD_AUTO)
@@ -8691,7 +8698,10 @@ static void service_route_changed(struct connman_ipconfig *ipconfig,
 {
 	struct connman_service *service = __connman_ipconfig_get_data(ipconfig);
 
-	DBG("%s route changed", ifname);
+	DBG("service %p (%s) ipconfig %p ifname %s route changed",
+		service, connman_service_get_identifier(service),
+		ipconfig,
+		ifname);
 
 	settings_changed(service, ipconfig);
 }
-- 
2.42.0

[PATCH v3 16/60] service: Add online check failure service error enumeration and string.

[Grant Erickson]

This adds the 'CONNMAN_SERVICE_ERROR_ONLINE_CHECK_FAILED' service
error enumeration and "online-check-failed" string, used for services
that have met or exceeded the continuous mode online check failure
threshold.
---
 include/service.h | 19 ++++++++++---------
 src/service.c     |  2 ++
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/include/service.h b/include/service.h
index 69f93574bafb..bfa3ab2592e4 100644
--- a/include/service.h
+++ b/include/service.h
@@ -71,15 +71,16 @@ enum connman_service_state {
 };
 
 enum connman_service_error {
-	CONNMAN_SERVICE_ERROR_UNKNOWN        = 0,
-	CONNMAN_SERVICE_ERROR_OUT_OF_RANGE   = 1,
-	CONNMAN_SERVICE_ERROR_PIN_MISSING    = 2,
-	CONNMAN_SERVICE_ERROR_DHCP_FAILED    = 3,
-	CONNMAN_SERVICE_ERROR_CONNECT_FAILED = 4,
-	CONNMAN_SERVICE_ERROR_LOGIN_FAILED  = 5,
-	CONNMAN_SERVICE_ERROR_AUTH_FAILED    = 6,
-	CONNMAN_SERVICE_ERROR_INVALID_KEY    = 7,
-	CONNMAN_SERVICE_ERROR_BLOCKED        = 8,
+	CONNMAN_SERVICE_ERROR_UNKNOWN             = 0,
+	CONNMAN_SERVICE_ERROR_OUT_OF_RANGE        = 1,
+	CONNMAN_SERVICE_ERROR_PIN_MISSING         = 2,
+	CONNMAN_SERVICE_ERROR_DHCP_FAILED         = 3,
+	CONNMAN_SERVICE_ERROR_CONNECT_FAILED      = 4,
+	CONNMAN_SERVICE_ERROR_LOGIN_FAILED        = 5,
+	CONNMAN_SERVICE_ERROR_AUTH_FAILED         = 6,
+	CONNMAN_SERVICE_ERROR_INVALID_KEY         = 7,
+	CONNMAN_SERVICE_ERROR_BLOCKED             = 8,
+	CONNMAN_SERVICE_ERROR_ONLINE_CHECK_FAILED = 9,
 };
 
 enum connman_service_proxy_method {
diff --git a/src/service.c b/src/service.c
index fb901ef16c9b..2599acd832ce 100644
--- a/src/service.c
+++ b/src/service.c
@@ -384,6 +384,8 @@ static const char *error2string(enum connman_service_error error)
 		return "invalid-key";
 	case CONNMAN_SERVICE_ERROR_BLOCKED:
 		return "blocked";
+	case CONNMAN_SERVICE_ERROR_ONLINE_CHECK_FAILED:
+		return "online-check-failed";
 	}
 
 	return NULL;
-- 
2.42.0

[PATCH v3 17/60] doc/service: Document online check failure service error.

[Grant Erickson]

This adds documentation for the online check failure service error
enumeration and string.
---
 doc/service-api.txt | 3 ++-
 include/service.h   | 7 +++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/doc/service-api.txt b/doc/service-api.txt
index c0d5adbb2b57..89208a6dc55f 100644
--- a/doc/service-api.txt
+++ b/doc/service-api.txt
@@ -157,7 +157,8 @@ Properties	string State [readonly]
 
 			Currently defined error codes are: "out-of-range",
 			"pin-missing", "dhcp-failed", "connect-failed",
-			"login-failed", "auth-failed" and "invalid-key".
+			"login-failed", "auth-failed", "invalid-key",
+			"blocked", and "online-check-failed".
 
 		string Name [readonly]
 
diff --git a/include/service.h b/include/service.h
index bfa3ab2592e4..f194422fae48 100644
--- a/include/service.h
+++ b/include/service.h
@@ -80,6 +80,13 @@ enum connman_service_error {
 	CONNMAN_SERVICE_ERROR_AUTH_FAILED         = 6,
 	CONNMAN_SERVICE_ERROR_INVALID_KEY         = 7,
 	CONNMAN_SERVICE_ERROR_BLOCKED             = 8,
+
+	/**
+	 *	In "continuous" online check mode,
+	 *	the back-to-back online check
+	 *	failures threshold was met or
+	 *	exceeded.
+	 */
 	CONNMAN_SERVICE_ERROR_ONLINE_CHECK_FAILED = 9,
 };
 
-- 
2.42.0

[PATCH v3 18/60] service: Introduce and use online check 'failures/successes' state.

[Grant Erickson]

This introduces a per-IP configuration online check 'failures' and
'successes' state member that represent the number of sustained,
back-to-back "online" reachability check failures and successes,
respectively, for "continuous" online check mode.
---
 src/service.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/service.c b/src/service.c
index 2599acd832ce..db1d45f61063 100644
--- a/src/service.c
+++ b/src/service.c
@@ -109,6 +109,8 @@ struct online_check_state {
 	 *
 	 */
 	unsigned int interval;
+	unsigned int successes;
+	unsigned int failures;
 };
 
 struct connman_service {
-- 
2.42.0

[PATCH v3 19/60] service: Document 'online_check_state' 'failures/successes' members.

[Grant Erickson]

This documents the 'failures/successes' members of the
'online_check_state' structure.
---
 src/service.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/src/service.c b/src/service.c
index db1d45f61063..5f9dcab58092 100644
--- a/src/service.c
+++ b/src/service.c
@@ -109,7 +109,17 @@ struct online_check_state {
 	 *
 	 */
 	unsigned int interval;
+
+	/**
+	 *	The number of sustained, back-to-back "online" reachability
+	 *	check successes for "continuous" online check mode.
+	 */
 	unsigned int successes;
+
+	/**
+	 *	The number of sustained, back-to-back "online" reachability
+	 *	check failures for "continuous" online check mode.
+	 */
 	unsigned int failures;
 };
 
-- 
2.42.0

[PATCH v3 20/60] main: Introduce the 'OnlineCheck{Failures,Successes}Threshold' settings.

[Grant Erickson]

When both "EnableOnlineCheck" and "EnableOnlineToReadyTransition" are
asserted, "OnlineCheckFailuresThreshold" is the number of failed
back-to-back "ready" to "online" HTTP-based Internet reachability
checks that will be allowed before marking a service as "failed" from
a reachability perspective, sorting it at a lower priority than other
services not so marked.

When both "EnableOnlineCheck" and "EnableOnlineToReadyTransition" are
asserted, "OnlineCheckSuccessesThreshold" is the number of successful
back-to-back "ready" to "online" HTTP-based Internet reachability
checks that must be met before clearing a service as "failed" from a
reachability perspective and allowing it to transition to the "online"
state again, allowing it to sort back to a higher priority relative to
other network services.
---
 src/main.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/src/main.c b/src/main.c
index d2ec8296f490..8e47426f2e8c 100644
--- a/src/main.c
+++ b/src/main.c
@@ -56,6 +56,9 @@
 #define DEFAULT_ONLINE_CHECK_INITIAL_INTERVAL 1
 #define DEFAULT_ONLINE_CHECK_MAX_INTERVAL 12
 
+#define DEFAULT_ONLINE_CHECK_FAILURES_THRESHOLD 6
+#define DEFAULT_ONLINE_CHECK_SUCCESSES_THRESHOLD 6
+
 #define ONLINE_CHECK_INTERVAL_STYLE_FIBONACCI "fibonacci"
 #define ONLINE_CHECK_INTERVAL_STYLE_GEOMETRIC "geometric"
 
@@ -114,6 +117,8 @@ static struct {
 	unsigned int online_check_connect_timeout_ms;
 	unsigned int online_check_initial_interval;
 	unsigned int online_check_max_interval;
+	unsigned int online_check_failures_threshold;
+	unsigned int online_check_successes_threshold;
 	char *online_check_interval_style;
 	bool auto_connect_roaming_services;
 	bool acd;
@@ -146,6 +151,10 @@ static struct {
 	.online_check_connect_timeout_ms = DEFAULT_ONLINE_CHECK_CONNECT_TIMEOUT,
 	.online_check_initial_interval = DEFAULT_ONLINE_CHECK_INITIAL_INTERVAL,
 	.online_check_max_interval = DEFAULT_ONLINE_CHECK_MAX_INTERVAL,
+	.online_check_failures_threshold =
+		DEFAULT_ONLINE_CHECK_FAILURES_THRESHOLD,
+	.online_check_successes_threshold =
+		DEFAULT_ONLINE_CHECK_SUCCESSES_THRESHOLD,
 	.online_check_interval_style = NULL,
 	.auto_connect_roaming_services = false,
 	.acd = false,
@@ -178,6 +187,8 @@ static struct {
 #define CONF_ONLINE_CHECK_CONNECT_TIMEOUT "OnlineCheckConnectTimeout"
 #define CONF_ONLINE_CHECK_INITIAL_INTERVAL "OnlineCheckInitialInterval"
 #define CONF_ONLINE_CHECK_MAX_INTERVAL     "OnlineCheckMaxInterval"
+#define CONF_ONLINE_CHECK_FAILURES_THRESHOLD "OnlineCheckFailuresThreshold"
+#define CONF_ONLINE_CHECK_SUCCESSES_THRESHOLD "OnlineCheckSuccessesThreshold"
 #define CONF_ONLINE_CHECK_INTERVAL_STYLE "OnlineCheckIntervalStyle"
 #define CONF_AUTO_CONNECT_ROAMING_SERVICES "AutoConnectRoamingServices"
 #define CONF_ACD                        "AddressConflictDetection"
@@ -211,6 +222,8 @@ static const char *supported_options[] = {
 	CONF_ONLINE_CHECK_CONNECT_TIMEOUT,
 	CONF_ONLINE_CHECK_INITIAL_INTERVAL,
 	CONF_ONLINE_CHECK_MAX_INTERVAL,
+	CONF_ONLINE_CHECK_FAILURES_THRESHOLD,
+	CONF_ONLINE_CHECK_SUCCESSES_THRESHOLD,
 	CONF_ONLINE_CHECK_INTERVAL_STYLE,
 	CONF_AUTO_CONNECT_ROAMING_SERVICES,
 	CONF_ACD,
@@ -604,6 +617,38 @@ static void parse_config(GKeyFile *config)
 			DEFAULT_ONLINE_CHECK_MAX_INTERVAL;
 	}
 
+	/* OnlineCheckFailuresThreshold */
+
+	integer = g_key_file_get_integer(config, "General",
+			CONF_ONLINE_CHECK_FAILURES_THRESHOLD, &error);
+	if (!error && integer >= 0)
+		connman_settings.online_check_failures_threshold = integer;
+
+	if (connman_settings.online_check_failures_threshold < 1) {
+		connman_warn("Incorrect online check failures threshold [%d]",
+			connman_settings.online_check_failures_threshold);
+		connman_settings.online_check_failures_threshold =
+			DEFAULT_ONLINE_CHECK_FAILURES_THRESHOLD;
+	}
+
+	g_clear_error(&error);
+
+	/* OnlineCheckSuccessesThreshold */
+
+	integer = g_key_file_get_integer(config, "General",
+			CONF_ONLINE_CHECK_SUCCESSES_THRESHOLD, &error);
+	if (!error && integer >= 0)
+		connman_settings.online_check_successes_threshold = integer;
+
+	if (connman_settings.online_check_successes_threshold < 1) {
+		connman_warn("Incorrect online check successes threshold [%d]",
+			connman_settings.online_check_successes_threshold);
+		connman_settings.online_check_successes_threshold =
+			DEFAULT_ONLINE_CHECK_SUCCESSES_THRESHOLD;
+	}
+
+	g_clear_error(&error);
+
 	string = __connman_config_get_string(config, "General",
 					CONF_ONLINE_CHECK_INTERVAL_STYLE, &error);
 	if (!error) {
@@ -921,6 +966,12 @@ unsigned int connman_setting_get_uint(const char *key)
 	if (g_str_equal(key, CONF_ONLINE_CHECK_MAX_INTERVAL))
 		return connman_settings.online_check_max_interval;
 
+	if (g_str_equal(key, CONF_ONLINE_CHECK_FAILURES_THRESHOLD))
+		return connman_settings.online_check_failures_threshold;
+
+	if (g_str_equal(key, CONF_ONLINE_CHECK_SUCCESSES_THRESHOLD))
+		return connman_settings.online_check_successes_threshold;
+
 	return 0;
 }
 
-- 
2.42.0

[PATCH v3 21/60] doc: Document the 'OnlineCheck{Failures,Successes}Threshold' settings.

[Grant Erickson]

This documents the 'OnlineCheck{Failures,Successes}Threshold' settings.
---
 doc/connman.conf.5.in | 38 ++++++++++++++++++++++++++++++++++++++
 src/main.conf         | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+)

diff --git a/doc/connman.conf.5.in b/doc/connman.conf.5.in
index 9049d7037623..bc5a112f758a 100644
--- a/doc/connman.conf.5.in
+++ b/doc/connman.conf.5.in
@@ -222,6 +222,44 @@ will transition to READY state, enabling another service to become the
 default one, in replacement.
 Default value is false.
 .TP
+.BI OnlineCheckFailuresThreshold= failures
+When both \fBEnableOnlineCheck\fR and
+\fBEnableOnlineToReadyTransition\fR are asserted, this is the number
+of failed back-to-back "ready" to "online" HTTP-based Internet
+reachability checks that will be allowed before marking a service as
+"failed" from a reachability perspective, sorting it at a lower
+priority than other services not so marked.
+
+Lower values may result in higher-frequency network service cycling
+while higher values may result in a longer period of time before
+failing from a non-Internet reachable service to one that might be.
+
+See \fBOnlineCheckIntervalStyle\fR, \fBOnlineCheckInitialInterval\fR,
+and \fBOnlineCheckMaxInterval\fR for other values that influence
+network service failure/recovery transition time.
+
+The default value is 6.
+.TP
+.BI OnlineCheckSuccessesThreshold= successes
+When both \fBEnableOnlineCheck\fR and
+\fBEnableOnlineToReadyTransition\fR are asserted, this is the number
+of successful back-to-back "ready" to "online" HTTP-based Internet
+reachability checks that must be met before clearing a service as
+"failed" from a reachability perspective and allowing it to transition
+to the "online" state again, allowing it to sort back to a higher
+priority relative to other network services.
+
+Lower values may result in higher-frequency network service cycling
+while higher values may result in a longer period of time before
+transitioning back to more a preferred, Internet reachable network
+service.
+
+See \fBOnlineCheckIntervalStyle\fR, \fBOnlineCheckInitialInterval\fR,
+and \fBOnlineCheckMaxInterval\fR for other values that influence
+network service failure/recovery transition time.
+
+The default value is 6.
+.TP
 .BI AutoConnectRoamingServices=true\ \fR|\fB\ false
 Automatically connect roaming services. This is not recommended unless you know
 you won't have any billing problem.
diff --git a/src/main.conf b/src/main.conf
index ddcb3564c246..b0eb93811f85 100644
--- a/src/main.conf
+++ b/src/main.conf
@@ -159,6 +159,44 @@
 # default one, in replacement.
 # EnableOnlineToReadyTransition = false
 
+# When both "EnableOnlineCheck" and "EnableOnlineToReadyTransition"
+# are asserted, this is the number of failed back-to-back "ready" to
+# "online" HTTP-based Internet reachability checks that will be
+# allowed before marking a service as "failed" from a reachability
+# perspective, sorting it at a lower priority than other services not
+# so marked.
+#
+# Lower values may result in higher-frequency network service cycling
+# while higher values may result in a longer period of time before
+# failing from a non-Internet reachable service to one that might be.
+#
+# See "OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
+# "OnlineCheckMaxInterval" for other values that influence network
+# service failure/recovery transition time.
+#
+# Default value is 6.
+# OnlineCheckFailuresThreshold=6
+
+# When both "EnableOnlineCheck" and "EnableOnlineToReadyTransition"
+# are asserted, this is the number of successful back-to-back "ready"
+# to "online" HTTP-based Internet reachability checks that must be met
+# before clearing a service as "failed" from a reachability
+# perspective and allowing it to transition to the "online" state
+# again, allowing it to sort back to a higher priority relative to
+# other network services.
+#
+# Lower values may result in higher-frequency network service cycling
+# while higher values may result in a longer period of time before
+# transitioning back to more a preferred, Internet reachable network
+# service.
+#
+# See "OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
+# "OnlineCheckMaxInterval" for other values that influence network
+# service failure/recovery transition time.
+#
+# Default value is 6.
+# OnlineCheckSuccessesThreshold=6
+
 # The style or mathematical series function used to compute the actual
 # time, in seconds, between two "ready" to "online" HTTP-based Internet
 # reachability checks. The value of which may be either "geometric" or
-- 
2.42.0

[PATCH v3 22/60] service: Add per-service online check failure threshold state.

[Grant Erickson]

This adds per-service state that tracks whether the service has met
the number of sustained, back-to-back "online" reachability check
failures for continuous" online check mode.
---
 src/service.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/service.c b/src/service.c
index 5f9dcab58092..d75399062ed7 100644
--- a/src/service.c
+++ b/src/service.c
@@ -188,6 +188,7 @@ struct connman_service {
 	bool wps_advertizing;
 	struct online_check_state online_check_state_ipv4;
 	struct online_check_state online_check_state_ipv6;
+	bool online_check_failures_met_threshold;
 	bool do_split_routing;
 	bool new_service;
 	bool hidden_service;
-- 
2.42.0

[PATCH v3 23/60] service: Document 'connman_service' 'online_check_failures_met_threshold' member.

[Grant Erickson]

This documents the 'online_check_failures_met_threshold' member of the
'connman_service' structure.
---
 src/service.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/src/service.c b/src/service.c
index d75399062ed7..2ce829ae37a7 100644
--- a/src/service.c
+++ b/src/service.c
@@ -188,6 +188,12 @@ struct connman_service {
 	bool wps_advertizing;
 	struct online_check_state online_check_state_ipv4;
 	struct online_check_state online_check_state_ipv6;
+
+    /**
+     *  Tracks whether the service has met the number of sustained,
+     *  back-to-back "online" reachability check failures for
+     *  "continuous" online check mode.
+     */
 	bool online_check_failures_met_threshold;
 	bool do_split_routing;
 	bool new_service;
-- 
2.42.0

[PATCH v3 24/60] service: Document 'connman_service' 'online_check_state_ipv{4,6}' members.

[Grant Erickson]

This documents the 'online_check_state_ipv{4,6}' members of the
'connman_service' structure.
---
 src/service.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/src/service.c b/src/service.c
index 2ce829ae37a7..6c898476627b 100644
--- a/src/service.c
+++ b/src/service.c
@@ -186,7 +186,15 @@ struct connman_service {
 	char *pac;
 	bool wps;
 	bool wps_advertizing;
+
+    /**
+     *  IPv4-specific "online" reachability check state.
+     */
 	struct online_check_state online_check_state_ipv4;
+
+    /**
+     *  IPv6-specific "online" reachability check state.
+     */
 	struct online_check_state online_check_state_ipv6;
 
     /**
-- 
2.42.0

[PATCH v3 25/60] service: Move 'set_error' forward declaration earlier in the source.

[Grant Erickson]

This moves the 'set_error' forward declaration earlier in the source
such that it may be invoked by functions that precede its position
before this change.
---
 src/service.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/src/service.c b/src/service.c
index 6c898476627b..a51c1685b308 100644
--- a/src/service.c
+++ b/src/service.c
@@ -225,6 +225,8 @@ static void complete_online_check(struct connman_service *service,
 					int err);
 static bool service_downgrade_online_state(struct connman_service *service);
 static bool connman_service_is_default(const struct connman_service *service);
+static void set_error(struct connman_service *service,
+					enum connman_service_error error);
 static void clear_error(struct connman_service *service);
 
 struct find_data {
@@ -4538,9 +4540,6 @@ int __connman_service_check_passphrase(enum connman_service_security security,
 	return 0;
 }
 
-static void set_error(struct connman_service *service,
-					enum connman_service_error error);
-
 int __connman_service_set_passphrase(struct connman_service *service,
 					const char *passphrase)
 {
-- 
2.42.0

[PATCH v3 26/60] service: Capture IP configuration state by value rather than by pointer.

[Grant Erickson]

With recent refactoring, there is no longer any need to capture the IP
configuration state by pointer in 'complete_online_check'. Instead,
capture it by value.
---
 src/service.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/src/service.c b/src/service.c
index a51c1685b308..2017370d7ce8 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2464,11 +2464,12 @@ static bool handle_online_check_failure(struct connman_service *service,
 {
 	bool reschedule = false;
 
-	DBG("service %p (%s) type %d (%s) "
+	DBG("service %p (%s) type %d (%s) state %d (%s) "
 		"one-shot %u err %d (%s)\n",
 		service,
 		connman_service_get_identifier(service),
 		type, __connman_ipconfig_type2string(type),
+		ipconfig_state, state2string(ipconfig_state),
 		oneshot, err, strerror(-err));
 
 	/*
@@ -2550,7 +2551,7 @@ static void complete_online_check(struct connman_service *service,
 					int err)
 {
 	struct online_check_state *online_check_state;
-	enum connman_service_state *ipconfig_state;
+	enum connman_service_state ipconfig_state;
 	bool reschedule = false;
 
 	DBG("service %p (%s) type %d (%s) "
@@ -2562,10 +2563,10 @@ static void complete_online_check(struct connman_service *service,
 
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV4) {
 		online_check_state = &service->online_check_state_ipv4;
-		ipconfig_state = &service->state_ipv4;
+		ipconfig_state = service->state_ipv4;
 	} else if (type == CONNMAN_IPCONFIG_TYPE_IPV6) {
 		online_check_state = &service->online_check_state_ipv6;
-		ipconfig_state = &service->state_ipv6;
+		ipconfig_state = service->state_ipv6;
 	} else
 		return;
 
@@ -2577,7 +2578,7 @@ static void complete_online_check(struct connman_service *service,
 	else
 		reschedule = handle_online_check_failure(service,
 					 type,
-					 *ipconfig_state,
+					 ipconfig_state,
 					 online_check_state,
 					 !enable_online_to_ready_transition,
 					 err);
-- 
2.42.0

[PATCH v3 27/60] service: Complete 'continuous' online check mode implementation.

[Grant Erickson]

This completes the implementation of 'EnableOnlineToReadyTransition'
begun by Emmanuel Vautrin with commit f0bd0e8fe578 ("service: Add
online to ready transition feature").

When asserted along with 'EnableOnlineCheck',
'EnableOnlineToReadyTransition' effects what can be thought of as a
"continuous" online check mode, which is different from the "one-shot"
online check mode when 'EnableOnlineToReadyTransition' is not asserted
but 'EnableOnlineCheck' is. Effectively, these two Booleans encode
three online check modes:

    1. None (!EnableOnlineCheck)
    2. One-shot (EnableOnlineCheck && !EnableOnlineToReadyTransition)
    3. Continuous (EnableOnlineCheck && EnableOnlineToReadyTransition)

With this change, these three modes are all but formalized.

In "none" mode, as has been the case historically, there are no
"online" HTTP-based Internet reachability checks. Any connected
service and the manager state will terminate at the "ready" state and
will not progress to "online".

In "one-shot" mode, as has been the case historically, there is a
single, one-shot "online" HTTP-based Internet reachability check for
the default service. When the check succeeds, the associated service
and the manager state will terminate at the "online" state. When the
check fails, subsequent checks will be rescheduled according to
"OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
"OnlineCheckMaxInterval" and will continue indefinitely until one
succeeds or until the service is disconnected.

In "continuous" mode, which is finalized with this change, there are
ongoing "online" HTTP-based Internet reachability check for the
default service. As with "one-shot" mode, when the first check
succeeds, the associated service and the manager state will terminate
at the "online" state. Thereafter, subsequent checks will be scheduled
according to "OnlineCheckIntervalStyle" and
"OnlineCheckMaxInterval". When the check fails, subsequent checks will
be rescheduled according to "OnlineCheckIntervalStyle",
"OnlineCheckInitialInterval", and "OnlineCheckMaxInterval". This is
largely unchanged. However, what is new with this change is that when
and if "OnlineCheckFailuresThreshold" is met, the service and manager
state will be demoted to "ready" and the service will have its "Error"
property set to "online-check-failed" while subsequent checks will
continue. In the interim, if available, another service may be
promoted to the default service and online checks will be initiated
for it. When and if, for the demoted service,
"OnlineCheckSuccessesThreshold" is met, the service "Error" property
will be cleared and the service state promoted to "online",
potentially causing it to become the default service again.
---
 src/service.c | 661 ++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 643 insertions(+), 18 deletions(-)

diff --git a/src/service.c b/src/service.c
index 2017370d7ce8..15a79c0aa5c8 100644
--- a/src/service.c
+++ b/src/service.c
@@ -46,6 +46,10 @@
 #define VPN_AUTOCONNECT_TIMEOUT_ATTEMPTS_THRESHOLD 270
 
 typedef guint (*online_check_timeout_compute_t)(unsigned int interval);
+typedef bool (*is_counter_threshold_met_predicate_t)(
+	const struct connman_service *service,
+	const char *counter_description,
+	unsigned int counter_threshold);
 
 static DBusConnection *connection = NULL;
 
@@ -219,12 +223,14 @@ static struct connman_ipconfig *create_ip6config(struct connman_service *service
 static void dns_changed(struct connman_service *service);
 static void vpn_auto_connect(void);
 static void trigger_autoconnect(struct connman_service *service);
+static void service_list_sort(void);
 static void complete_online_check(struct connman_service *service,
 					enum connman_ipconfig_type type,
 					bool success,
 					int err);
 static bool service_downgrade_online_state(struct connman_service *service);
 static bool connman_service_is_default(const struct connman_service *service);
+static int start_online_check_if_connected(struct connman_service *service);
 static void set_error(struct connman_service *service,
 					enum connman_service_error error);
 static void clear_error(struct connman_service *service);
@@ -2162,6 +2168,235 @@ done:
 	return status;
 }
 
+static bool online_check_failures_threshold_was_met(
+			const struct connman_service *service)
+{
+	return service->online_check_failures_met_threshold;
+}
+
+static void online_check_failures_threshold_was_met_set_value(
+			struct connman_service *service, bool value)
+{
+	DBG("service %p (%s) failures met threshold %u",
+		service, connman_service_get_identifier(service),
+		value);
+
+	service->online_check_failures_met_threshold = value;
+}
+
+static void online_check_failures_threshold_was_met_set(
+			struct connman_service *service)
+{
+	online_check_failures_threshold_was_met_set_value(service, true);
+}
+
+static void online_check_failures_threshold_was_met_clear(
+			struct connman_service *service)
+{
+	online_check_failures_threshold_was_met_set_value(service, false);
+}
+
+static inline void online_check_counter_reset(
+			unsigned int *counter)
+{
+	if (!counter)
+		return;
+
+	*counter = 0;
+}
+
+static void online_check_failures_reset(struct connman_service *service)
+{
+	DBG("service %p (%s)",
+		service, connman_service_get_identifier(service));
+
+	online_check_counter_reset(&service->online_check_state_ipv4.failures);
+	online_check_counter_reset(&service->online_check_state_ipv6.failures);
+}
+
+static void online_check_successes_reset(struct connman_service *service)
+{
+	DBG("service %p (%s)",
+		service, connman_service_get_identifier(service));
+
+	online_check_counter_reset(&service->online_check_state_ipv4.successes);
+	online_check_counter_reset(&service->online_check_state_ipv6.successes);
+}
+
+static void online_check_state_reset(struct connman_service *service)
+{
+	online_check_failures_reset(service);
+
+	online_check_successes_reset(service);
+
+	online_check_failures_threshold_was_met_clear(service);
+
+	clear_error(service);
+}
+
+static void online_check_counters_log(
+			const struct connman_service *service,
+			const char *counter_description,
+			unsigned int ipv4_counter,
+			unsigned int ipv6_counter)
+{
+	DBG("service %p (%s) "
+		"ipv4 state %d (%s) %s(s/es) %u "
+		"ipv6 state %d (%s) %s(s/es) %u ",
+		service, connman_service_get_identifier(service),
+		service->state_ipv4, state2string(service->state_ipv4),
+		counter_description,
+		ipv4_counter,
+		service->state_ipv6, state2string(service->state_ipv6),
+		counter_description,
+		ipv6_counter);
+}
+
+static bool online_check_counter_threshold_is_met(
+			const struct connman_service *service,
+			const char *counter_threshold_key,
+			const char *counter_description,
+			is_counter_threshold_met_predicate_t predicate)
+{
+	unsigned int counter_threshold;
+	bool threshold_met = false;
+
+	if (!service ||
+		!counter_threshold_key ||
+		!counter_description ||
+		!predicate)
+		goto done;
+
+	counter_threshold = connman_setting_get_uint(counter_threshold_key);
+
+	threshold_met = predicate(service,
+						counter_description,
+						counter_threshold);
+
+	DBG("service %p (%s) %s threshold %u %s(s) met %u",
+		service, connman_service_get_identifier(service),
+		counter_description,
+		counter_threshold,
+		counter_description,
+		threshold_met);
+
+done:
+	return threshold_met;
+}
+
+static bool is_online_check_failure_threshold_met_predicate(
+			const struct connman_service *service,
+			const char *counter_description,
+			unsigned int counter_threshold)
+{
+	bool ipv4_is_connected;
+	bool ipv6_is_connected;
+	bool threshold_met = false;
+
+	online_check_counters_log(service,
+		counter_description,
+		service->online_check_state_ipv4.failures,
+		service->online_check_state_ipv6.failures);
+
+	ipv4_is_connected = is_connected(service->state_ipv4);
+	ipv6_is_connected = is_connected(service->state_ipv6);
+
+	/*
+	 * It is entirely possible that IPv4 reachability is fine and that
+	 * IPv6 reachablity is not due to the premises ISP, premises
+	 * Internet access equipment (that is, CPE), availability of the
+	 * reachability endpoint infrastructure, etc.
+	 *
+	 * Consequently, we want to see bilateral failures of BOTH IPv4
+	 * AND IPv6 in excess of the threshold, to the extent either is
+	 * connected (based on the #is_connected predicate).
+	 */
+	if ((!ipv6_is_connected &&
+		 ipv4_is_connected &&
+		 service->online_check_state_ipv4.failures >=
+		 counter_threshold) ||
+
+		(!ipv4_is_connected &&
+		ipv6_is_connected &&
+		service->online_check_state_ipv6.failures >=
+		counter_threshold) ||
+
+		(ipv4_is_connected &&
+		service->online_check_state_ipv4.failures >=
+		counter_threshold &&
+		ipv6_is_connected &&
+		service->online_check_state_ipv6.failures >=
+		counter_threshold)) {
+		threshold_met = true;
+	}
+
+	return threshold_met;
+}
+
+static bool online_check_failures_threshold_is_met(
+			const struct connman_service *service)
+{
+	const char * const counter_threshold_key =
+		"OnlineCheckFailuresThreshold";
+	const char * const counter_description =
+		"failure";
+
+	return online_check_counter_threshold_is_met(service,
+			counter_threshold_key,
+			counter_description,
+			is_online_check_failure_threshold_met_predicate);
+}
+
+static bool is_online_check_success_threshold_met_predicate(
+			const struct connman_service *service,
+			const char *counter_description,
+			unsigned int counter_threshold)
+{
+	bool threshold_met = false;
+
+	online_check_counters_log(service,
+		counter_description,
+		service->online_check_state_ipv4.successes,
+		service->online_check_state_ipv6.successes);
+
+	/*
+	 * It is entirely possible that IPv4 reachability is fine and that
+	 * IPv6 reachablity is not due to the premises ISP, premises
+	 * Internet access equipment (that is, CPE), availability of the
+	 * reachability endpoint infrastructure, etc.
+	 *
+	 * Consequently, we want to see bilateral successes of EITHER IPv4
+	 * OR IPv6 (as with #combine_state) in excess of the threshold, to
+	 * the extent either is connected (based on the #is_connected
+	 * predicate).
+	 */
+
+	if ((is_connected(service->state_ipv4) &&
+		service->online_check_state_ipv4.successes >=
+		counter_threshold) ||
+		(is_connected(service->state_ipv6) &&
+		service->online_check_state_ipv6.successes >=
+		counter_threshold)) {
+		threshold_met = true;
+	}
+
+	return threshold_met;
+}
+
+static bool online_check_successes_threshold_is_met(
+			const struct connman_service *service)
+{
+	const char * const counter_threshold_key =
+		"OnlineCheckSuccessesThreshold";
+	const char * const counter_description =
+		"success";
+
+	return online_check_counter_threshold_is_met(service,
+			counter_threshold_key,
+			counter_description,
+			is_online_check_success_threshold_met_predicate);
+}
+
 /**
  *  @brief
  *    Retry an "online" HTTP-based Internet reachability check.
@@ -2356,6 +2591,214 @@ static void reschedule_online_check(struct connman_service *service,
 		online_check_state->interval++;
 }
 
+static void online_check_counter_increment_and_log(
+			const struct connman_service *service,
+			enum connman_ipconfig_type type,
+			const char *counter_description,
+			unsigned int *counter)
+{
+	if (!service || !counter_description || !counter)
+		return;
+
+	(*counter)++;
+
+	DBG("service %p (%s) type %d (%s) %s %u",
+		service, connman_service_get_identifier(service),
+		type, __connman_ipconfig_type2string(type),
+		counter_description, *counter);
+}
+
+static void online_check_log_success(const struct connman_service *service,
+			enum connman_ipconfig_type type)
+{
+	g_autofree char *interface = NULL;
+
+	interface = connman_service_get_interface(service);
+
+	connman_info("Interface %s [ %s ] %s online check to %s succeeded",
+		interface,
+		__connman_service_type2string(service->type),
+		__connman_ipconfig_type2string(type),
+		type == CONNMAN_IPCONFIG_TYPE_IPV4 ?
+			connman_setting_get_string("OnlineCheckIPv4URL") :
+			connman_setting_get_string("OnlineCheckIPv6URL"));
+}
+
+static void continuous_online_check_log_counter_threshold_met(
+			const struct connman_service *service,
+			const char *counter_threshold_key,
+			const char *counter_description)
+{
+	g_autofree char *interface = NULL;
+
+	interface = connman_service_get_interface(service);
+
+	connman_warn("Interface %s [ %s ] online check had %u back-to-back "
+		"%s; %s threshold met",
+		interface,
+		__connman_service_type2string(service->type),
+		connman_setting_get_uint(counter_threshold_key),
+				 counter_description,
+				 counter_description);
+}
+
+static void continuous_online_check_log_successes_threshold_met(
+			const struct connman_service *service
+)
+{
+	static const char *const counter_threshold_key =
+		"OnlineCheckSuccessesThreshold";
+	static const char *const counter_description =
+		"success(es)";
+
+	continuous_online_check_log_counter_threshold_met(service,
+		counter_threshold_key,
+		counter_description);
+}
+
+static void continuous_online_check_log_failures_threshold_met(
+			const struct connman_service *service
+)
+{
+	static const char *const counter_threshold_key =
+		"OnlineCheckFailuresThreshold";
+	static const char *const counter_description =
+		"failure(s)";
+
+	continuous_online_check_log_counter_threshold_met(service,
+		counter_threshold_key,
+		counter_description);
+}
+
+static bool handle_oneshot_online_check_success(
+			struct connman_service *service,
+			enum connman_ipconfig_type type,
+			struct online_check_state *online_check_state)
+{
+	const bool reschedule = true;
+
+	/*
+	 * Simply log the success, mark the service IP configuration state
+	 * as ONLINE, and return.
+	 */
+	online_check_log_success(service, type);
+
+	__connman_service_ipconfig_indicate_state(service,
+		CONNMAN_SERVICE_STATE_ONLINE,
+		type);
+
+	return !reschedule;
+}
+
+static bool handle_continuous_online_check_success(
+			struct connman_service *service,
+			enum connman_ipconfig_type type,
+			struct online_check_state *online_check_state)
+{
+	bool failures_threshold_was_met;
+	bool successes_threshold_is_met;
+	const bool reschedule = true;
+
+	/* Unconditionally increment and log the success counter. */
+
+	online_check_counter_increment_and_log(service, type,
+		"successes", &online_check_state->successes);
+
+	/*
+	 * Ultimately, for failures, we are looking for a STRING of
+	 * SUSTAINED, BACK-TO-BACK failures to meet the failures
+	 * threshold. Consequently, any success should reset the
+	 * corresponding failure count back to zero (0).
+	 */
+	online_check_counter_reset(&online_check_state->failures);
+
+	failures_threshold_was_met =
+		online_check_failures_threshold_was_met(service);
+	successes_threshold_is_met =
+		online_check_successes_threshold_is_met(service);
+
+	DBG("failures threshold was met %u, "
+		"successes threshold is met %u, "
+		"default %u",
+		failures_threshold_was_met,
+		successes_threshold_is_met,
+		connman_service_is_default(service));
+
+	/*
+	 * If the service HAD previously-exceeded the failure threshold
+	 * AND if this is the first success, then reset the online check
+	 * interval to the initial, minimum value since we want to recover
+	 * as quickly as possible with a STRING of SUSTAINED, BACK-TO-BACK
+	 * successes, where the length of that string is dictated by the
+	 * "OnlineCheckSuccessesThreshold" settings value.
+	 *
+	 * Otherwise, if the service HAD NOT previously-exceeded the
+	 * failure threshold OR if it HAD previously-exceeded the failure
+	 * threshold AND the successes threshold was met, then reset the
+	 * online check interval to the maximum value.
+	 */
+	if (failures_threshold_was_met &&
+		online_check_state->successes == 1)
+		online_check_state->interval = online_check_initial_interval;
+	else if (!failures_threshold_was_met ||
+		(failures_threshold_was_met && successes_threshold_is_met))
+		online_check_state->interval = online_check_max_interval;
+
+	/*
+	 * If the service HAD NOT previously-exceeded the failure
+	 * threshold, then simply mark the service IP configuration state
+	 * as ONLINE.
+	 *
+	 * Otherwise, if the service HAD previously exceeded the failure
+	 * threshold AND successes meet or exceed the configured success
+	 * threshold, then re-sort the network services and update the
+	 * gateways accordingly.
+	 *
+	 * The succeeding service will be promoted until such time as it
+	 * has a configured number of failures, at which time, we will
+	 * resort again.
+	 *
+	 */
+	if (!failures_threshold_was_met) {
+		if (online_check_state->successes == 1)
+			online_check_log_success(service, type);
+
+		if (connman_service_is_default(service))
+			__connman_service_ipconfig_indicate_state(service,
+				CONNMAN_SERVICE_STATE_ONLINE,
+				type);
+	} else if (failures_threshold_was_met &&
+			   successes_threshold_is_met) {
+		online_check_log_success(service, type);
+
+		continuous_online_check_log_successes_threshold_met(service);
+
+		online_check_state_reset(service);
+
+		/*
+		 * The ordering here is considered and intentional. FIRST, now
+		 * that this service has cleared / reset the online check
+		 * state, re-sort the service list. This may promote this
+		 * service back to the default. SECOND, make the READY to
+		 * ONLINE promotion, since that promotion is qualified with
+		 * this service being the default (that is, has the default
+		 * route) service.
+		 */
+		service_list_sort();
+
+		if (connman_service_is_default(service)) {
+			__connman_service_ipconfig_indicate_state(
+				service,
+				CONNMAN_SERVICE_STATE_ONLINE,
+				type);
+		}
+
+		__connman_gateway_update();
+	}
+
+	return reschedule;
+}
+
 /**
  *  @brief
  *    Handle the successful completion of an "online" HTTP-based
@@ -2395,7 +2838,7 @@ static bool handle_online_check_success(struct connman_service *service,
 				struct online_check_state *online_check_state,
 				bool oneshot)
 {
-	const bool reschedule = !oneshot;
+	bool reschedule;
 
 	DBG("service %p (%s) type %d (%s) "
 		"one-shot %u\n",
@@ -2404,12 +2847,144 @@ static bool handle_online_check_success(struct connman_service *service,
 		type, __connman_ipconfig_type2string(type),
 		oneshot);
 
-	__connman_service_ipconfig_indicate_state(service,
-		CONNMAN_SERVICE_STATE_ONLINE,
-		type);
+	if (oneshot)
+		reschedule = handle_oneshot_online_check_success(service,
+						type,
+						online_check_state);
+	else
+		reschedule = handle_continuous_online_check_success(service,
+						type,
+						online_check_state);
 
-	if (!oneshot)
-		online_check_state->interval = online_check_max_interval;
+	return reschedule;
+}
+
+static void online_check_log_failure(const struct connman_service *service,
+			enum connman_ipconfig_type type,
+			int err)
+{
+	g_autofree char *interface = NULL;
+
+	interface = connman_service_get_interface(service);
+
+	connman_warn("Interface %s [ %s ] %s online check to %s failed: %d: %s",
+		interface,
+		__connman_service_type2string(service->type),
+		__connman_ipconfig_type2string(type),
+		type == CONNMAN_IPCONFIG_TYPE_IPV4 ?
+			connman_setting_get_string("OnlineCheckIPv4URL") :
+			connman_setting_get_string("OnlineCheckIPv6URL"),
+		err,
+		strerror(-err));
+}
+
+static bool handle_oneshot_online_check_failure(
+			struct connman_service *service,
+			enum connman_ipconfig_type type,
+			enum connman_service_state ipconfig_state,
+			struct online_check_state *online_check_state,
+			int err)
+{
+	const bool reschedule = true;
+
+	/* Simply indicate rescheduling another check is desired. */
+
+	DBG("online check mode is one-shot; requesting another check");
+
+	return reschedule;
+}
+
+static bool handle_continuous_online_check_failure(
+			struct connman_service *service,
+			enum connman_ipconfig_type type,
+			enum connman_service_state ipconfig_state,
+			struct online_check_state *online_check_state,
+			int err)
+{
+	bool reschedule = false;
+
+	/* Unconditionally increment and log the failure counter. */
+
+	online_check_counter_increment_and_log(service, type,
+		"failures", &online_check_state->failures);
+
+	/*
+	 * Ultimately, for successes, we are looking for a STRING of
+	 * SUSTAINED, BACK-TO-BACK successes to meet the successes
+	 * threshold. Consequently, any failure should reset the
+	 * corresponding success count back to zero (0).
+	 */
+	online_check_counter_reset(&online_check_state->successes);
+
+	/*
+	 * If this is the first failure, then reset the online check
+	 * interval to the initial, minimum value. Subsequent failures
+	 * will increment the interval on reschedule from here until the
+	 * maximum interval is hit.
+	 */
+	if (online_check_state->failures == 1)
+		online_check_state->interval = online_check_initial_interval;
+
+	DBG("failures threshold was met %u failures threshold is met %u "
+		"default %u",
+		online_check_failures_threshold_was_met(service),
+		online_check_failures_threshold_is_met(service),
+		connman_service_is_default(service));
+
+	/*
+	 * If the service HAD NOT previously-exceeded the failure
+	 * threshold AND failures meet or exceed the configured failure
+	 * threshold, then:
+	 *
+	 *	  1. Assert the failure threshold state.
+	 *	  2. Reset the success counters.
+	 *	  3. Attempt to downgrade the service IP configuration state
+	 *		 from ONLINE to READY.
+	 *	  4. Re-sort the network services.
+	 *	  5. Update the gateways accordingly.
+	 *
+	 * The failing service will be demoted until such time as it has a
+	 * configured number of successes, at which time, we will resort
+	 * again.
+	 *
+	 */
+	if (!online_check_failures_threshold_was_met(service) &&
+		online_check_failures_threshold_is_met(service)) {
+		online_check_failures_threshold_was_met_set(service);
+
+		continuous_online_check_log_failures_threshold_met(service);
+
+		online_check_successes_reset(service);
+
+		/*
+		 * Attempt to downgrade the service state from ONLINE to
+		 * READY.
+		 *
+		 * We attempt BOTH IPv4 and IPv6 IP configuration states since
+		 * the #online_check_failures_threshold_is_met predicate tells
+		 * us that both IP configurations have met the failures
+		 * threshold.
+		 */
+		service_downgrade_online_state(service);
+
+		set_error(service, CONNMAN_SERVICE_ERROR_ONLINE_CHECK_FAILED);
+
+		service_list_sort();
+
+		__connman_gateway_update();
+	}
+
+	DBG("failures threshold was met %u, default %u",
+		online_check_failures_threshold_was_met(service),
+		connman_service_is_default(service));
+
+	/*
+	 * We only want to reschedule future online checks for
+	 * the default service or those that are in failure.
+	 */
+	if (connman_service_is_default(service) ||
+		online_check_failures_threshold_was_met(service))
+		reschedule = true;
 
 	return reschedule;
 }
@@ -2473,10 +3048,10 @@ static bool handle_online_check_failure(struct connman_service *service,
 		oneshot, err, strerror(-err));
 
 	/*
-	 * If this completion closure was a failure with error status
-	 * -ECANCELED, then it was canceled by #__connman_wispr_cancel.
-	 * Simply ignore it and DO NOT reschedule; another check will be
-	 * along to replace the canceled one.
+	 * Regardless of online check mode, if this completion closure
+	 * was a failure with error status -ECANCELED, then it was canceled
+	 * by #__connman_wispr_cancel. Simply ignore it and DO NOT indicate
+	 * rescheduling another check is desired.
 	 */
 	if (err == -ECANCELED) {
 		DBG("online check was canceled; no action taken");
@@ -2484,15 +3059,26 @@ static bool handle_online_check_failure(struct connman_service *service,
 		goto done;
 	}
 
-	if (oneshot)
-		reschedule = true;
-	else {
-		if (service_downgrade_online_state(service))
-			online_check_state->interval =
-				online_check_initial_interval;
+	/* Unconditionally log the failure, regardless of online check mode. */
 
-		reschedule = connman_service_is_default(service);
-	}
+	online_check_log_failure(service, type, err);
+
+	/* Handle the failure according to the online check mode. */
+
+	if (oneshot)
+		reschedule = handle_oneshot_online_check_failure(
+						service,
+						type,
+						ipconfig_state,
+						online_check_state,
+						err);
+	else
+		reschedule = handle_continuous_online_check_failure(
+						service,
+						type,
+						ipconfig_state,
+						online_check_state,
+						err);
 
 done:
 	return reschedule;
@@ -3004,6 +3590,34 @@ static void default_changed(void)
 
 	__connman_service_timeserver_changed(current_default, NULL);
 
+	/*
+	 * If there is a current default service, then it may either have
+	 * been temporarily:
+	 *
+	 *	 1. promoted as a failover from another senior service that
+	 *		was temporarily demoted
+	 *	 2. demoted as a failover to another junior service that is
+	 *		being temporarily promoted
+	 *
+	 * due to a continuous mode online check failure.
+	 *
+	 * Regardless, only services in online check failure or the default
+	 * service should be running online checks and only the default
+	 * service should be online. Consequently, make the appropriate
+	 * calls on the current default to ensure that is the case BEFORE
+	 * assigning the proposed new default as the current default.
+	 */
+	if (current_default) {
+		if (!online_check_failures_threshold_was_met(current_default) &&
+			current_default->error !=
+				CONNMAN_SERVICE_ERROR_ONLINE_CHECK_FAILED) {
+			cancel_online_check(current_default,
+				CONNMAN_IPCONFIG_TYPE_ALL);
+
+			service_downgrade_online_state(current_default);
+		}
+	}
+
 	current_default = service;
 
 	if (service) {
@@ -6867,6 +7481,15 @@ static gint service_compare(gconstpointer a, gconstpointer b)
 
 	if (a_connected && b_connected) {
 		int rval;
+		if (!online_check_failures_threshold_was_met(service_a) &&
+			online_check_failures_threshold_was_met(service_b)) {
+			return -1;
+		}
+
+		if (online_check_failures_threshold_was_met(service_a) &&
+			!online_check_failures_threshold_was_met(service_b)) {
+			return 1;
+		}
 
 		/* Compare the VPN transport and the service */
 		if ((service_a->type == CONNMAN_SERVICE_TYPE_VPN ||
@@ -7814,6 +8437,8 @@ static int service_indicate_state(struct connman_service *service)
 
 		__connman_wispr_stop(service);
 
+		online_check_state_reset(service);
+
 		__connman_wpad_stop(service);
 
 		domain_changed(service);
-- 
2.42.0

[PATCH v3 28/60] service: Document 'online_check_failures_threshold_was_met'.

[Grant Erickson]

This adds documentation to the
'online_check_failures_threshold_was_met' function.
---
 src/service.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/src/service.c b/src/service.c
index 15a79c0aa5c8..e8fc14f1c4ca 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2168,6 +2168,23 @@ done:
 	return status;
 }
 
+/**
+ *  @brief
+ *    Return the online check failures threshold state.
+ *
+ *  @param[in]  service  A pointer to the immutable service for which
+ *                       to return the online check failures threshold
+ *                       state.
+ *
+ *  @returns
+ *    True if the online check failures threshold was met; otherwise,
+ *    false.
+ *
+ *  @sa online_check_failures_threshold_was_met_set_value
+ *  @sa online_check_failures_threshold_was_met_set
+ *  @sa online_check_failures_threshold_was_met_clear
+ *
+ */
 static bool online_check_failures_threshold_was_met(
 			const struct connman_service *service)
 {
-- 
2.42.0

[PATCH v3 29/60] service: Document 'online_check_failures_threshold_was_met_set_value'.

[Grant Erickson]

This adds documentation to the
'online_check_failures_threshold_was_met_set_value' function.
---
 src/service.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/src/service.c b/src/service.c
index e8fc14f1c4ca..b41f29d73d60 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2191,6 +2191,20 @@ static bool online_check_failures_threshold_was_met(
 	return service->online_check_failures_met_threshold;
 }
 
+/**
+ *  @brief
+ *    Set the online check failures threshold state to the specified
+ *    value.
+ *
+ *  @param[in,out]  service  A pointer to the mutable service for which
+ *                           to set the failures threshold state.
+ *  @param[in]      value    The value to set the @a service failures
+ *                           threshold state to.
+ *
+ *  @sa online_check_failures_threshold_was_met_set
+ *  @sa online_check_failures_threshold_was_met_clear
+ *
+ */
 static void online_check_failures_threshold_was_met_set_value(
 			struct connman_service *service, bool value)
 {
-- 
2.42.0

[PATCH v3 30/60] service: Document 'online_check_failures_threshold_was_met_{set,clear}'.

[Grant Erickson]

This adds documentation to the
'online_check_failures_threshold_was_met_{set,clear}' functions.
---
 src/service.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/src/service.c b/src/service.c
index b41f29d73d60..6e34b4c8ef34 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2215,12 +2215,35 @@ static void online_check_failures_threshold_was_met_set_value(
 	service->online_check_failures_met_threshold = value;
 }
 
+/**
+ *  @brief
+ *    Set (that is, assert) the online check failures threshold state.
+ *
+ *  @param[in,out]  service  A pointer to the mutable service for which
+ *                           to set the failures threshold state.
+ *
+ *  @sa online_check_failures_threshold_was_met_set_value
+ *  @sa online_check_failures_threshold_was_met_clear
+ *
+ */
 static void online_check_failures_threshold_was_met_set(
 			struct connman_service *service)
 {
 	online_check_failures_threshold_was_met_set_value(service, true);
 }
 
+/**
+ *  @brief
+ *    Clear (that is, deassert) the online check failures threshold
+ *    state.
+ *
+ *  @param[in,out]  service  A pointer to the mutable service for which
+ *                           to clear the failures threshold state.
+ *
+ *  @sa online_check_failures_threshold_was_met_set_value
+ *  @sa online_check_failures_threshold_was_met_set
+ *
+ */
 static void online_check_failures_threshold_was_met_clear(
 			struct connman_service *service)
 {
-- 
2.42.0

[PATCH v3 31/60] service: Document 'online_check_counter_reset'.

[Grant Erickson]

This adds documentation to the 'online_check_counter_reset' function.
---
 src/service.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/service.c b/src/service.c
index 6e34b4c8ef34..8a1d09f1a4fd 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2250,6 +2250,13 @@ static void online_check_failures_threshold_was_met_clear(
 	online_check_failures_threshold_was_met_set_value(service, false);
 }
 
+/**
+ *  Reset the specified counter to zero (0).
+ *
+ *  @param[in,out]  counter  A pointer to the counter to reset by
+ *                           setting it to zero (0).
+ *
+ */
 static inline void online_check_counter_reset(
 			unsigned int *counter)
 {
-- 
2.42.0

[PATCH v3 32/60] service: Document 'online_check_{failures,successes}_reset'.

[Grant Erickson]

This adds documentation to the
'online_check_{failures,successes}_reset' functions.
---
 src/service.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/src/service.c b/src/service.c
index 8a1d09f1a4fd..810ff68cf4f4 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2266,6 +2266,18 @@ static inline void online_check_counter_reset(
 	*counter = 0;
 }
 
+/**
+ *  @brief
+ *    Reset to zero (0) the IPv4 and IPv6 online check failure
+ *    counters for the specified service.
+ *
+ *  @param[in]   service   A pointer to the mutable service for which
+ *                         to reset the IPv4 and IPv6 online check
+ *                         failure counters.
+ *
+ *  @sa online_check_successes_reset
+ *
+ */
 static void online_check_failures_reset(struct connman_service *service)
 {
 	DBG("service %p (%s)",
@@ -2275,6 +2287,18 @@ static void online_check_failures_reset(struct connman_service *service)
 	online_check_counter_reset(&service->online_check_state_ipv6.failures);
 }
 
+/**
+ *  @brief
+ *    Reset to zero (0) the IPv4 and IPv6 online check success
+ *    counters for the specified service.
+ *
+ *  @param[in]   service   A pointer to the mutable service for which
+ *                         to reset the IPv4 and IPv6 online check
+ *                         success counters.
+ *
+ *  @sa online_check_failures_reset
+ *
+ */
 static void online_check_successes_reset(struct connman_service *service)
 {
 	DBG("service %p (%s)",
-- 
2.42.0

[PATCH v3 33/60] service: Document 'online_check_state_reset'.

[Grant Erickson]

This adds documentation to the 'online_check_state_reset' function.
---
 src/service.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/src/service.c b/src/service.c
index 810ff68cf4f4..5dfada3af3f5 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2308,6 +2308,22 @@ static void online_check_successes_reset(struct connman_service *service)
 	online_check_counter_reset(&service->online_check_state_ipv6.successes);
 }
 
+/**
+ *  @brief
+ *    Reset the online check state for the specified service.
+ *
+ *  This resets the online check state for the specified service,
+ *  including its failure threshold state, failure counters, and
+ *  success counters.
+ *
+ *  @param[in]   service   A pointer to the mutable service for which
+ *                         to reset the online check state.
+ *
+ *  @sa online_check_failures_reset
+ *  @sa online_check_successes_reset
+ *  @sa online_check_failures_threshold_was_met_clear
+ *
+ */
 static void online_check_state_reset(struct connman_service *service)
 {
 	online_check_failures_reset(service);
-- 
2.42.0

[PATCH v3 34/60] service: Document 'online_check_counters_log'.

[Grant Erickson]

This adds documentation to the 'online_check_counters_log' function.
---
 src/service.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/src/service.c b/src/service.c
index 5dfada3af3f5..e6652f53363c 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2335,6 +2335,26 @@ static void online_check_state_reset(struct connman_service *service)
 	clear_error(service);
 }
 
+/**
+ *  @brief
+ *    Log the specified IPv4 and IPv6 online check counters for the
+ *    specified service.
+ *
+ *  This logs the specified IPv4 and IPv6 online check counters
+ *  described by the provided description for the specified network
+ *  service.
+ *
+ *  @param[in]  service              A pointer to the immutable network
+ *                                   service associated with @a
+ *                                   ipv4_counter and @a ipv6_counter.
+ *  @param[in]  counter_description  A pointer to a null-terminated C
+ *                                   string describing @a ipv4_counter
+ *                                   and @a ipv6_counter. For example,
+ *                                   "failure".
+ *  @param[in]  ipv4_counter         The IPv4-specific counter to log.
+ *  @param[in]  ipv6_counter         The IPv6-specific counter to log.
+ *
+ */
 static void online_check_counters_log(
 			const struct connman_service *service,
 			const char *counter_description,
-- 
2.42.0

[PATCH v3 35/60] service: Document 'online_check_counter_threshold_is_met'.

[Grant Erickson]

This adds documentation to the 'online_check_counter_threshold_is_met'
function.
---
 src/service.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/src/service.c b/src/service.c
index e6652f53363c..abf7800b56fa 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2373,6 +2373,38 @@ static void online_check_counters_log(
 		ipv6_counter);
 }
 
+/**
+ *  @brief
+ *    Determine whether an online check counter has met its threshold.
+ *
+ *  This determines whether an online check counter associated with
+ *  the specified network service has met its threshold, where the
+ *  threshold is accessed from the configuration store with the
+ *  specified key.
+ *
+ *  @param[in]  service                A pointer to the immutable
+ *                                     network service associated with
+ *                                     the counter to check.
+ *  @param[in]  counter_threshold_key  A pointer to a null-terminated
+ *                                     C string containing the key to
+ *                                     use with the configuration
+ *                                     store to access the threshold
+ *                                     value to check the counter
+ *                                     against.
+ *  @param[in]  counter_description    A pointer to a null-terminated
+ *                                     C string describing the counter
+ *                                     to check. For example, "failure".
+ *  @param[in]  predicate              A pointer to the predicate
+ *                                     function to invoke to make the
+ *                                     actual determination of whether
+ *                                     the counter has met the
+ *                                     threshold accessed by @a
+ *                                     counter_threshold_key.
+ *
+ *  @returns
+ *    True if the counter has met the threshold; otherwise, false.
+ *
+ */
 static bool online_check_counter_threshold_is_met(
 			const struct connman_service *service,
 			const char *counter_threshold_key,
-- 
2.42.0

[PATCH v3 36/60] service: Document 'is_online_check_failure_threshold_met_predicate'.

[Grant Erickson]

This adds documentation to the
'is_online_check_failure_threshold_met_predicate' function.
---
 src/service.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/src/service.c b/src/service.c
index abf7800b56fa..8c8ff1c7a496 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2437,6 +2437,32 @@ done:
 	return threshold_met;
 }
 
+/**
+ *  @brief
+ *    Determine whether the service has met the online check failure
+ *    threshold.
+ *
+ *  This predicate determines whether the online check failure
+ *  threshold has been met by the specified network service.
+ *
+ *  @param[in]  service                A pointer to the immutable
+ *                                     network service for which to
+ *                                     check whether its has met the
+ *                                     online check failure threshold.
+ *  @param[in]  counter_description    A pointer to a null-terminated
+ *                                     C string describing the failure
+ *                                     counter. For example,
+ *                                     "failure".
+ *  @param[in]  counter_threshold      The threshold value to check the
+ *                                     failure counter against.
+ *
+ *  @returns
+ *    True if the online check failure counter has met the failure
+ *    threshold; otherwise, false.
+ *
+ *  @sa online_check_failures_threshold_is_met
+ *
+ */
 static bool is_online_check_failure_threshold_met_predicate(
 			const struct connman_service *service,
 			const char *counter_description,
-- 
2.42.0

[PATCH v3 37/60] service: Document 'online_check_failures_threshold_is_met'.

[Grant Erickson]

This adds documentation to the
'online_check_failures_threshold_is_met' function.
---
 src/service.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/src/service.c b/src/service.c
index 8c8ff1c7a496..beb339b89256 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2512,6 +2512,26 @@ static bool is_online_check_failure_threshold_met_predicate(
 	return threshold_met;
 }
 
+/**
+ *  @brief
+ *    Determine whether the online check failures threshold is met.
+ *
+ *  This attempts to determine whether the online check failures
+ *  threshold is met, comparing the current IPv4 and IPv6 online check
+ *  failure counts against the "OnlineCheckFailuresThreshold" settings
+ *  value and returning @a true if @b both the IPv4 and IPv6 counts
+ *  meet or exceed the threshold.
+ *
+ *  @param[in]  service  A pointer to the immutable service for which
+ *                       to determine whether the online check failure
+ *                       threshold is met.
+ *
+ *  @returns
+ *    True if the failure threshold is met; otherwise, false.
+ *
+ *  @sa online_check_successes_threshold_is_met
+ *
+ */
 static bool online_check_failures_threshold_is_met(
 			const struct connman_service *service)
 {
-- 
2.42.0

[PATCH v3 38/60] service: Document 'is_online_check_success_threshold_met_predicate'.

[Grant Erickson]

This adds documentation to the
'is_online_check_success_threshold_met_predicate' function.
---
 src/service.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/src/service.c b/src/service.c
index beb339b89256..6a9db3c98e56 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2546,6 +2546,32 @@ static bool online_check_failures_threshold_is_met(
 			is_online_check_failure_threshold_met_predicate);
 }
 
+/**
+ *  @brief
+ *    Determine whether the service has met the online check success
+ *    threshold.
+ *
+ *  This predicate determines whether the online check success
+ *  threshold has been met by the specified network service.
+ *
+ *  @param[in]  service                A pointer to the immutable
+ *                                     network service for which to
+ *                                     check whether its has met the
+ *                                     online check success threshold.
+ *  @param[in]  counter_description    A pointer to a null-terminated
+ *                                     C string describing the success
+ *                                     counter. For example,
+ *                                     "success".
+ *  @param[in]  counter_threshold      The threshold value to check the
+ *                                     success counter against.
+ *
+ *  @returns
+ *    True if the online check success counter has met the success
+ *    threshold; otherwise, false.
+ *
+ *  @sa online_check_successes_threshold_is_met
+ *
+ */
 static bool is_online_check_success_threshold_met_predicate(
 			const struct connman_service *service,
 			const char *counter_description,
-- 
2.42.0

[PATCH v3 39/60] service: Document 'online_check_successes_threshold_is_met'.

[Grant Erickson]

This adds documentation to the
'online_check_successes_threshold_is_met' function.
---
 src/service.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/src/service.c b/src/service.c
index 6a9db3c98e56..333c072739ed 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2608,6 +2608,26 @@ static bool is_online_check_success_threshold_met_predicate(
 	return threshold_met;
 }
 
+/**
+ *  @brief
+ *    Determine whether the online check successes threshold is met.
+ *
+ *  This attempts to determine whether the online check successes
+ *  threshold is met, comparing the current IPv4 and IPv6 online check
+ *  success counts against the "OnlineCheckSuccessesThreshold" settings
+ *  value and returning @a true if @b either the IPv4 @b or IPv6 counts
+ *  meet or exceed the threshold.
+ *
+ *  @param[in]  service  A pointer to the immutable service for which
+ *                       to determine whether the online check success
+ *                       threshold is met.
+ *
+ *  @returns
+ *    True if the success threshold is met; otherwise, false.
+ *
+ *  @sa online_check_failures_threshold_is_met
+ *
+ */
 static bool online_check_successes_threshold_is_met(
 			const struct connman_service *service)
 {
-- 
2.42.0

[PATCH v3 40/60] service: Document 'online_check_counter_increment_and_log'.

[Grant Erickson]

This adds documentation to the
'online_check_counter_increment_and_log' function.
---
 src/service.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/src/service.c b/src/service.c
index 333c072739ed..9d27a5692034 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2836,6 +2836,24 @@ static void reschedule_online_check(struct connman_service *service,
 		online_check_state->interval++;
 }
 
+/**
+ *  @brief
+ *    Increment and log the specified online check counter.
+ *
+ *  This increments by one (1) and logs the post-increment value of
+ *  the specified online check counter associated with the specified
+ *  network service.
+ *
+ *  @param[in]  service              A pointer to the immutable network
+ *                                   service associated with @a
+ *                                   counter.
+ *  @param[in]  type                 The IP configuration type associated
+ *                                   with @a counter.
+ *  @param[in]  counter_description  A pointer to a null-terminated C
+ *                                   string describing @a counter. For
+ *                                   example, "failure".
+ *
+ */
 static void online_check_counter_increment_and_log(
 			const struct connman_service *service,
 			enum connman_ipconfig_type type,
-- 
2.42.0

[PATCH v3 41/60] service: Document 'online_check_log_success'.

[Grant Erickson]

This adds documentation to the 'online_check_log_success' function.
---
 src/service.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/src/service.c b/src/service.c
index 9d27a5692034..029d7b8b6122 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2871,6 +2871,20 @@ static void online_check_counter_increment_and_log(
 		counter_description, *counter);
 }
 
+/**
+ *  @brief
+ *    Log an online check success.
+ *
+ *  This logs an online check success for the specified network
+ *  service IP configuration type.
+ *
+ *  @param[in]  service  A pointer to the immutable network
+ *                       service for which to log an online
+ *                       check success.
+ *  @param[in]  type     The IP configuration type for which
+ *                       the online check was successful.
+ *
+ */
 static void online_check_log_success(const struct connman_service *service,
 			enum connman_ipconfig_type type)
 {
-- 
2.42.0

[PATCH v3 42/60] service: Document 'continuous_online_check_log_{counter,failures,successes}_threshold_met'.

[Grant Erickson]

This adds documentation to the
'continuous_online_check_log_{counter,failures,successes}_threshold_met'
functions.
---
 src/service.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/src/service.c b/src/service.c
index 029d7b8b6122..1e2282572892 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2901,6 +2901,29 @@ static void online_check_log_success(const struct connman_service *service,
 			connman_setting_get_string("OnlineCheckIPv6URL"));
 }
 
+/**
+ *  @brief
+ *    Log that an online check counter has met its threshold.
+ *
+ *  This logs that an online check counter associated with the
+ *  specified network service has met its threshold.
+ *
+ *  @param[in]  service                A pointer to the immutable
+ *                                     network service for which to
+ *                                     log that one of its online
+ *                                     check counters has met its
+ *                                     threshold.
+ *  @param[in]  counter_threshold_key  A pointer to a null-terminated
+ *                                     C string containing the key to
+ *                                     use with the configuration
+ *                                     store to access the threshold
+ *                                     value for the counter.
+ *  @param[in]  counter_description    A pointer to a null-terminated
+ *                                     C string describing the counter
+ *                                     to check. For example,
+ *                                     "failure(s)".
+ *
+ */
 static void continuous_online_check_log_counter_threshold_met(
 			const struct connman_service *service,
 			const char *counter_threshold_key,
@@ -2919,6 +2942,20 @@ static void continuous_online_check_log_counter_threshold_met(
 				 counter_description);
 }
 
+/**
+ *  @brief
+ *    Log that an online check success counter has met its threshold.
+ *
+ *  This logs that an online check success counter associated with the
+ *  specified network service has met its threshold.
+ *
+ *  @param[in]  service                A pointer to the immutable
+ *                                     network service for which to
+ *                                     log that its online check
+ *                                     success counter has met its
+ *                                     threshold.
+ *
+ */
 static void continuous_online_check_log_successes_threshold_met(
 			const struct connman_service *service
 )
@@ -2933,6 +2970,20 @@ static void continuous_online_check_log_successes_threshold_met(
 		counter_description);
 }
 
+/**
+ *  @brief
+ *    Log that an online check failure counter has met its threshold.
+ *
+ *  This logs that an online check failure counter associated with the
+ *  specified network service has met its threshold.
+ *
+ *  @param[in]  service                A pointer to the immutable
+ *                                     network service for which to
+ *                                     log that its online check
+ *                                     failure counter has met its
+ *                                     threshold.
+ *
+ */
 static void continuous_online_check_log_failures_threshold_met(
 			const struct connman_service *service
 )
-- 
2.42.0

[PATCH v3 43/60] service: Document 'handle_{continuous,oneshot}_online_check_success'.

[Grant Erickson]

This adds documentation to the
'handle_{continuous,oneshot}_online_check_success' functions.
---
 src/service.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/src/service.c b/src/service.c
index 1e2282572892..62edabed1b1b 100644
--- a/src/service.c
+++ b/src/service.c
@@ -2998,6 +2998,36 @@ static void continuous_online_check_log_failures_threshold_met(
 		counter_description);
 }
 
+/**
+ *  @brief
+ *    Handle the successful completion of an "online" HTTP-based
+ *    Internet reachability check for the specified network service
+ *    and IP configuration type for the "one-shot" online check mode.
+ *
+ *  This handles the completion of a successful "online" HTTP-based
+ *  Internet reachability check for the specified network service and
+ *  IP configuration type for the "one-shot" online check mode. This
+ *  effectively "bookends" an earlier #__connman_service_wispr_start.
+ *
+ *  @param[in,out]  service             A pointer to the mutable service
+ *                                      for which to handle a
+ *                                      successful previously-requested
+ *                                      online check.
+ *  @param[in]      type                The IP configuration type for
+ *                                      which to handle a successful
+ *                                      previously-requested online
+ *                                      check.
+ *  @param[in,out]  online_check_state  A pointer to the online check
+ *                                      state for @a service
+ *                                      associated with @a type.
+ *
+ *  @returns
+ *    False, unconditionally.
+ *
+ *  @sa handle_oneshot_online_check_failure
+ *  @sa handle_online_check_success
+ *
+ */
 static bool handle_oneshot_online_check_success(
 			struct connman_service *service,
 			enum connman_ipconfig_type type,
@@ -3018,6 +3048,37 @@ static bool handle_oneshot_online_check_success(
 	return !reschedule;
 }
 
+/**
+ *  @brief
+ *    Handle the successful completion of an "online" HTTP-based
+ *    Internet reachability check for the specified network service
+ *    and IP configuration type for the "continuous" online check mode.
+ *
+ *  This handles the completion of a successful "online" HTTP-based
+ *  Internet reachability check for the specified network service and
+ *  IP configuration type for the "continuous" online check mode. This
+ *  effectively "bookends" an earlier #__connman_service_wispr_start.
+ *
+ *  @param[in,out]  service             A pointer to the mutable service
+ *                                      for which to handle a
+ *                                      successful previously-requested
+ *                                      online check.
+ *  @param[in]      type                The IP configuration type for
+ *                                      which to handle a successful
+ *                                      previously-requested online
+ *                                      check.
+ *  @param[in,out]  online_check_state  A pointer to the online check
+ *                                      state for @a service
+ *                                      associated with @a type.
+ *
+ *  @returns
+ *    True if another online check should be scheduled; otherwise,
+ *    false.
+ *
+ *  @sa handle_continuous_online_check_failure
+ *  @sa handle_online_check_success
+ *
+ */
 static bool handle_continuous_online_check_success(
 			struct connman_service *service,
 			enum connman_ipconfig_type type,
-- 
2.42.0

[PATCH v3 44/60] service: Document 'online_check_log_failure'.

[Grant Erickson]

This adds documentation to the 'online_check_log_failure' function.
---
 src/service.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/src/service.c b/src/service.c
index 62edabed1b1b..b2eb00a3e774 100644
--- a/src/service.c
+++ b/src/service.c
@@ -3248,6 +3248,22 @@ static bool handle_online_check_success(struct connman_service *service,
 	return reschedule;
 }
 
+/**
+ *  @brief
+ *    Log an online check failure.
+ *
+ *  This logs an online check failure for the specified network
+ *  service IP configuration type.
+ *
+ *  @param[in]  service  A pointer to the immutable network
+ *                       service for which to log an online
+ *                       check failure.
+ *  @param[in]  type     The IP configuration type for which
+ *                       the online check failed.
+ *  @param[in]  err      The error status, in the POSIX domain,
+ *                       associated with the online check failure.
+ *
+ */
 static void online_check_log_failure(const struct connman_service *service,
 			enum connman_ipconfig_type type,
 			int err)
-- 
2.42.0

[PATCH v3 45/60] service: Document 'handle_{continuous,oneshot}_online_check_failure'.

[Grant Erickson]

This adds documentation to the
'handle_{continuous,oneshot}_online_check_failure' functions.
---
 src/service.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)

diff --git a/src/service.c b/src/service.c
index b2eb00a3e774..20cc5914a016 100644
--- a/src/service.c
+++ b/src/service.c
@@ -3283,6 +3283,47 @@ static void online_check_log_failure(const struct connman_service *service,
 		strerror(-err));
 }
 
+/**
+ *  @brief
+ *    Handle the failed completion of an one-shot mode "online"
+ *    HTTP-based Internet reachability check for the specified network
+ *    service and IP configuration type for the "one-shot" online
+ *    check mode.
+ *
+ *  This handles the completion of a failed one-shot mode "online"
+ *  HTTP-based Internet reachability check for the specified network
+ *  service and IP configuration type for the "one-shot" online check
+ *  mode. This effectively "bookends" an earlier
+ *  #__connman_service_wispr_start.
+ *
+ *  This simply indicates that rescheduling another check is desired.
+ *
+ *  @param[in,out]  service             A pointer to the mutable service
+ *                                      for which to handle a
+ *                                      failed previously-requested
+ *                                      online check.
+ *  @param[in]      type                The IP configuration type for
+ *                                      which to handle a failed
+ *                                      previously-requested online
+ *                                      check.
+ *  @param[in]      ipconfig_state      The current @a type IP
+ *                                      configuration state for @a
+ *                                      service.
+ *  @param[in,out]  online_check_state  A pointer to the online check
+ *                                      state for @a service
+ *                                      associated with @a type.
+ *  @param[in]      err                 The error status associated with
+ *                                      the failed previously-requested
+ *                                      online check. This is expected
+ *                                      to be less than zero ('< 0').
+ *
+ *  @returns
+ *    True, unconditionally.
+ *
+ *  @sa handle_online_check_failure
+ *  @sa handle_oneshot_online_check_failure
+ *
+ */
 static bool handle_oneshot_online_check_failure(
 			struct connman_service *service,
 			enum connman_ipconfig_type type,
@@ -3299,6 +3340,46 @@ static bool handle_oneshot_online_check_failure(
 	return reschedule;
 }
 
+/**
+ *  @brief
+ *    Handle the failed completion of an one-shot mode "online"
+ *    HTTP-based Internet reachability check for the specified network
+ *    service and IP configuration type for the "continuous" online
+ *    check mode.
+ *
+ *  This handles the completion of a failed continuous mode "online"
+ *  HTTP-based Internet reachability check for the specified network
+ *  service and IP configuration type for the "continuous" online check
+ *  mode. This effectively "bookends" an earlier
+ *  #__connman_service_wispr_start.
+ *
+ *  @param[in,out]  service             A pointer to the mutable service
+ *                                      for which to handle a
+ *                                      failed previously-requested
+ *                                      online check.
+ *  @param[in]      type                The IP configuration type for
+ *                                      which to handle a failed
+ *                                      previously-requested online
+ *                                      check.
+ *  @param[in]      ipconfig_state      The current @a type IP
+ *                                      configuration state for @a
+ *                                      service.
+ *  @param[in,out]  online_check_state  A pointer to the online check
+ *                                      state for @a service
+ *                                      associated with @a type.
+ *  @param[in]      err                 The error status associated with
+ *                                      the failed previously-requested
+ *                                      online check. This is expected
+ *                                      to be less than zero ('< 0').
+ *
+ *  @returns
+ *    True if another online check should be scheduled; otherwise,
+ *    false.
+ *
+ *  @sa handle_online_check_failure
+ *  @sa handle_continuous_online_check_failure
+ *
+ */
 static bool handle_continuous_online_check_failure(
 			struct connman_service *service,
 			enum connman_ipconfig_type type,
-- 
2.42.0

[PATCH v3 46/60] service: Add @sa documentation references.

[Grant Erickson]

This adds additional @sa ("see also") documentation references to the
'handle_online_check_{failure,success}' and 'complete_online_check'
functions.
---
 src/service.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/service.c b/src/service.c
index 20cc5914a016..c376b559e915 100644
--- a/src/service.c
+++ b/src/service.c
@@ -3220,6 +3220,8 @@ static bool handle_continuous_online_check_success(
  *    false.
  *
  *  @sa handle_online_check_failure
+ *  @sa handle_oneshot_online_check_success
+ *  @sa handle_continuous_online_check_success
  *
  */
 static bool handle_online_check_success(struct connman_service *service,
@@ -3514,6 +3516,8 @@ static bool handle_continuous_online_check_failure(
  *    false.
  *
  *  @sa handle_online_check_success
+ *  @sa handle_oneshot_online_check_failure
+ *  @sa handle_continuous_online_check_failure
  *
  */
 static bool handle_online_check_failure(struct connman_service *service,
@@ -3615,6 +3619,9 @@ done:
  *  @sa start_online_check
  *  @sa start_online_check_if_connected
  *  @sa __connman_service_wispr_start
+ *  @sa handle_online_check_success
+ *  @sa handle_online_check_failure
+ *  @sa reschedule_online_check
  *
  */
 static void complete_online_check(struct connman_service *service,
-- 
2.42.0

[PATCH v3 47/60] service: Document the internals of 'service_compare'.

[Grant Erickson]

This documents the major internal comparison blocks of the
'service_compare' function.
---
 src/service.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/src/service.c b/src/service.c
index c376b559e915..b1fb6a3e532a 100644
--- a/src/service.c
+++ b/src/service.c
@@ -7972,8 +7972,21 @@ static gint service_compare(gconstpointer a, gconstpointer b)
 	a_connected = is_connected(state_a);
 	b_connected = is_connected(state_b);
 
+	/*
+	 * If both services are connected (that is, "ready" or "online"),
+	 * then further sort by whether the services are VPN type, then
+	 * service order if there is VPN equivalence, and then by their
+	 * preferred technology status.
+	 */
 	if (a_connected && b_connected) {
 		int rval;
+
+		/*
+		 * If at this point the services are still comparing as
+		 * equivalent, then use online check failure status, giving
+		 * priority to the service that has not met the failure
+		 * threshold.
+		 */
 		if (!online_check_failures_threshold_was_met(service_a) &&
 			online_check_failures_threshold_was_met(service_b)) {
 			return -1;
@@ -8004,6 +8017,15 @@ static gint service_compare(gconstpointer a, gconstpointer b)
 			return rval;
 	}
 
+	/*
+	 * If at this point the services are still comparing as
+	 * equilvalent, then check whether their combined states are
+	 * different. If they are, then prefer the service that is
+	 * "online" to that which is only "ready", then prefer @a a being
+	 * connected versus @a b being connected, and, finally, then
+	 * prefer @a a being in the process of connecting to @a b being in
+	 * the process of connecting.
+	 */
 	if (state_a != state_b) {
 		if (a_connected && b_connected) {
 			/* We prefer online over ready state */
@@ -8025,12 +8047,26 @@ static gint service_compare(gconstpointer a, gconstpointer b)
 			return 1;
 	}
 
+	/*
+	 * If at this point the services are still comparing as
+	 * equivalent, then use favorite status, giving priority to @a a
+	 * as a favorite versus @a b as a favorite.
+	 */
 	if (service_a->favorite && !service_b->favorite)
 		return -1;
 
 	if (!service_a->favorite && service_b->favorite)
 		return 1;
 
+	/*
+	 * If at this point the services are still comparing as
+	 * equivalent, then check whether their types are different. If
+	 * they are, then compare their types. First, against the
+	 * PreferredTechnologies priority list and then by an internal
+	 * prioritization favoring Ethernet over Wi-Fi, Wi-Fi over
+	 * Cellular, Cellular over Bluetooth, Bluetooth over VPN, and VPN
+	 * over Gadget (that is, USB Ethernet).
+	 */
 	if (service_a->type != service_b->type) {
 		int rval;
 
@@ -8069,10 +8105,18 @@ static gint service_compare(gconstpointer a, gconstpointer b)
 			return 1;
 	}
 
+	/*
+	 * If at this point the services are still comparing as
+	 * equivalent, then check their strengths.
+	 */
 	strength = (gint) service_b->strength - (gint) service_a->strength;
 	if (strength)
 		return strength;
 
+	/*
+	 * Finally, if at this point the services are still comparing as
+	 * equivalent, then check their names.
+	 */
 	return g_strcmp0(service_a->name, service_b->name);
 }
 
-- 
2.42.0

[PATCH v3 48/60] service: Document interval assignment in '__connman_service_wispr_start'.

[Grant Erickson]

This adds a comment documenting the rationale for setting the check
interval to the initial interval in '__connman_service_wispr_start'.
---
 src/service.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/src/service.c b/src/service.c
index b1fb6a3e532a..cae4258fca7f 100644
--- a/src/service.c
+++ b/src/service.c
@@ -3832,6 +3832,11 @@ int __connman_service_wispr_start(struct connman_service *service,
 	if (online_check_is_active(service, type))
 		return -EALREADY;
 
+	/*
+	 * At this particular entry point, we assume to be starting an
+	 * "online" HTTP-based Internet reachability check
+	 * afresh. Consequently, set the check interval to initial.
+	 */
 	if (type == CONNMAN_IPCONFIG_TYPE_IPV4)
 		service->online_check_state_ipv4.interval =
 					online_check_initial_interval;
-- 
2.42.0

[PATCH v3 49/60] service: Document 'downgrade_connected_services'.

[Grant Erickson]

This adds documentation to the 'downgrade_connected_services'
functions.
---
 src/service.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/src/service.c b/src/service.c
index cae4258fca7f..efed2fc50ac9 100644
--- a/src/service.c
+++ b/src/service.c
@@ -8709,6 +8709,19 @@ static void request_input_cb(struct connman_service *service,
 	}
 }
 
+/**
+ *  @brief
+ *    Downgrade the service IPv4 and IPv6 states from "online" to
+ *    "ready" of all connected services.
+ *
+ *  This attempts to downgrade the IPv4 and IPv6 states of all
+ *  @a is_connected services to "ready" if they are "online".
+ *
+ *  @sa service_ipconfig_downgrade_online_state
+ *  @sa service_downgrade_online_state
+ *  @sa service_downgrade_online_state_if_default
+ *
+ */
 static void downgrade_connected_services(void)
 {
 	struct connman_service *up_service;
-- 
2.42.0

[PATCH v3 50/60] service: Document 'service_schedule_changed'.

[Grant Erickson]

This adds documentation to the 'service_schedule_changed' functions.
---
 src/service.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/src/service.c b/src/service.c
index efed2fc50ac9..0f3ea526967c 100644
--- a/src/service.c
+++ b/src/service.c
@@ -7389,6 +7389,15 @@ static gboolean service_send_changed(gpointer data)
 	return FALSE;
 }
 
+/**
+ *  @brief
+ *    Schedule a D-Bus "ServicesChanged" signal at 100 milliseconds
+ *    from now.
+ *
+ *  @sa service_send_changed
+ *  @sa service_list_sort
+ *
+ */
 static void service_schedule_changed(void)
 {
 	if (services_notify->id != 0)
-- 
2.42.0

[PATCH v3 51/60] service: Document 'service_route_changed'.

[Grant Erickson]

This adds documentation to the 'service_route_changed' functions.
---
 src/service.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/src/service.c b/src/service.c
index 0f3ea526967c..07322b80bdf7 100644
--- a/src/service.c
+++ b/src/service.c
@@ -9911,6 +9911,25 @@ static void service_ip_release(struct connman_ipconfig *ipconfig,
 	settings_changed(service, ipconfig);
 }
 
+/**
+ *  @brief
+ *    Handler for IP configuration routes changes.
+ *
+ *  This is the IP configuration handler for route set (add) and unset
+ *  (delete) operations for the specified IP configuration and its
+ *  associated network interface name.
+ *
+ *  @param[in]  ipconfig  A pointer to the IP configuration associated
+ *                        with the network service route change.
+ *  @param[in]  ifname    A pointer to an immutable null-terminated
+ *                        C string containing the network interface
+ *                        name associated with the route change.
+ *
+ *  @sa __connman_ipconfig_set_data
+ *  @sa __connman_ipconfig_set_ops
+ *  @sa settings_changed
+ *
+ */
 static void service_route_changed(struct connman_ipconfig *ipconfig,
 				const char *ifname)
 {
-- 
2.42.0

[PATCH v3 52/60] service: Add function parameter to 'default_changed' and 'service_list_sort'.

[Grant Erickson]

There are four (4) invocations of 'default_changed' and nine (9)
invocations of 'service_list_sort'.

To aid debugging, a function parameter is added to each, which is a
pointer to an immutable null-terminated C string, ostenisbly the name
of the function that invoked either 'default_changed' or
'service_list_sort'.

In addition, 'DEFAULT_CHANGED' and 'SERVICE_LIST_SORT' macros are
added and leveraged at those 13 call sites. These macros invoke
'default_changed' or 'service_list_sort', passing the C preprocessor
predefined '__func__' macro as the function parameter.
---
 src/service.c | 53 +++++++++++++++++++++++++++++++++++----------------
 1 file changed, 37 insertions(+), 16 deletions(-)

diff --git a/src/service.c b/src/service.c
index 07322b80bdf7..e04e24f5db97 100644
--- a/src/service.c
+++ b/src/service.c
@@ -45,6 +45,18 @@
 #define VPN_AUTOCONNECT_TIMEOUT_STEP 30
 #define VPN_AUTOCONNECT_TIMEOUT_ATTEMPTS_THRESHOLD 270
 
+/*
+ * There are many call sites throughout this module for these
+ * functions. These are macros to help, during debugging, to acertain
+ * where they were called from.
+ */
+
+#define DEFAULT_CHANGED() \
+	default_changed(__func__)
+
+#define SERVICE_LIST_SORT() \
+	service_list_sort(__func__)
+
 typedef guint (*online_check_timeout_compute_t)(unsigned int interval);
 typedef bool (*is_counter_threshold_met_predicate_t)(
 	const struct connman_service *service,
@@ -223,7 +235,7 @@ static struct connman_ipconfig *create_ip6config(struct connman_service *service
 static void dns_changed(struct connman_service *service);
 static void vpn_auto_connect(void);
 static void trigger_autoconnect(struct connman_service *service);
-static void service_list_sort(void);
+static void service_list_sort(const char *function);
 static void complete_online_check(struct connman_service *service,
 					enum connman_ipconfig_type type,
 					bool success,
@@ -3173,7 +3185,7 @@ static bool handle_continuous_online_check_success(
 		 * this service being the default (that is, has the default
 		 * route) service.
 		 */
-		service_list_sort();
+		SERVICE_LIST_SORT();
 
 		if (connman_service_is_default(service)) {
 			__connman_service_ipconfig_indicate_state(
@@ -3457,7 +3469,7 @@ static bool handle_continuous_online_check_failure(
 
 		set_error(service, CONNMAN_SERVICE_ERROR_ONLINE_CHECK_FAILED);
 
-		service_list_sort();
+		SERVICE_LIST_SORT();
 
 		__connman_gateway_update();
 	}
@@ -4075,10 +4087,12 @@ bool __connman_service_index_is_default(int index)
 	return __connman_service_get_index(service) == index;
 }
 
-static void default_changed(void)
+static void default_changed(const char *function)
 {
 	struct connman_service *service = connman_service_get_default();
 
+	DBG("from %s()", function);
+
 	if (service == current_default)
 		return;
 
@@ -8143,13 +8157,20 @@ static gint service_compare(gconstpointer a, gconstpointer b)
  *  the network services list. On completion of the sort, a D-Bus
  *  "ServicesChanged" signal is scheduled.
  *
+ *  @param[in]  function  A pointer to an immutable null-terminated
+ *                        C string containing the function name to
+ *                        which the call to this function should be
+ *                        attributed.
+ *
  *  @sa service_compare
  *  @sa service_compare_preferred
  *  @sa service_schedule_changed
  *
  */
-static void service_list_sort(void)
+static void service_list_sort(const char *function)
 {
+	DBG("from %s()", function);
+
 	if (service_list && service_list->next) {
 		service_list = g_list_sort(service_list, service_compare);
 		service_schedule_changed();
@@ -8434,7 +8455,7 @@ int __connman_service_set_favorite_delayed(struct connman_service *service,
 
 	if (!delay_ordering) {
 
-		service_list_sort();
+		SERVICE_LIST_SORT();
 
 		__connman_gateway_update();
 	}
@@ -8573,7 +8594,7 @@ static void report_error_cb(void *user_context, bool retry,
 		__connman_service_clear_error(service);
 
 		service_complete(service);
-		service_list_sort();
+		SERVICE_LIST_SORT();
 		__connman_gateway_update();
 	}
 }
@@ -8945,7 +8966,7 @@ static int service_indicate_state(struct connman_service *service)
 
 		service_update_preferred_order(def_service, service, new_state);
 
-		default_changed();
+		DEFAULT_CHANGED();
 
 		__connman_service_set_favorite(service, true);
 
@@ -8995,7 +9016,7 @@ static int service_indicate_state(struct connman_service *service)
 
 		reply_pending(service, ECONNABORTED);
 
-		default_changed();
+		DEFAULT_CHANGED();
 
 		cancel_online_check(service, CONNMAN_IPCONFIG_TYPE_ALL);
 
@@ -9032,7 +9053,7 @@ static int service_indicate_state(struct connman_service *service)
 		break;
 	}
 
-	service_list_sort();
+	SERVICE_LIST_SORT();
 
 	__connman_gateway_update();
 
@@ -9046,7 +9067,7 @@ notifier:
 
 	if (is_online(new_state)) {
 		__connman_notifier_enter_online(service->type);
-		default_changed();
+		DEFAULT_CHANGED();
 	}
 
 	return 0;
@@ -9117,7 +9138,7 @@ int __connman_service_indicate_default(struct connman_service *service)
 		return -EINPROGRESS;
 	}
 
-	default_changed();
+	DEFAULT_CHANGED();
 
 	return 0;
 }
@@ -9734,7 +9755,7 @@ int __connman_service_provision_changed(const char *ident)
 	if (services_dirty) {
 		services_dirty = false;
 
-		service_list_sort();
+		SERVICE_LIST_SORT();
 
 		__connman_gateway_update();
 	}
@@ -9807,7 +9828,7 @@ static int service_register(struct connman_service *service)
 	if (__connman_config_provision_service(service) < 0)
 		service_load(service);
 
-	service_list_sort();
+	SERVICE_LIST_SORT();
 
 	__connman_gateway_update();
 
@@ -10232,7 +10253,7 @@ static void update_from_network(struct connman_service *service,
 	if (!service->network)
 		service->network = connman_network_ref(network);
 
-	service_list_sort();
+	SERVICE_LIST_SORT();
 }
 
 static void trigger_autoconnect(struct connman_service *service)
@@ -10439,7 +10460,7 @@ roaming:
 
 sorting:
 	if (need_sort) {
-		service_list_sort();
+		SERVICE_LIST_SORT();
 	}
 }
 
-- 
2.42.0

[PATCH v3 53/60] service: Add @sa documentation reference to 'current_default'.

[Grant Erickson]

This adds an additional @sa ("see also") documentation reference to
'current_default'.
---
 src/service.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/service.c b/src/service.c
index e04e24f5db97..6975b55acaa2 100644
--- a/src/service.c
+++ b/src/service.c
@@ -78,6 +78,7 @@ static unsigned int vpn_autoconnect_id = 0;
  *
  *  @sa connman_service_get_default
  *  @sa connman_service_is_default
+ *  @sa default_changed
  *
  */
 static struct connman_service *current_default = NULL;
-- 
2.42.0

[PATCH v3 54/60] service: Add 'DBG' to 'downgrade_connected_services'.

[Grant Erickson]

This adds an empty 'DBG' to 'downgrade_connected_services' to make it
clear in debug logs that it is the caller of
'service_downgrade_online_state' in that instance.
---
 src/service.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/service.c b/src/service.c
index 6975b55acaa2..e44eea376636 100644
--- a/src/service.c
+++ b/src/service.c
@@ -8758,6 +8758,8 @@ static void downgrade_connected_services(void)
 	struct connman_service *up_service;
 	GList *list;
 
+	DBG("");
+
 	for (list = service_list; list; list = list->next) {
 		up_service = list->data;
 
-- 
2.42.0

[PATCH v3 55/60] main/service: Introduce 'OnlineCheckMode' setting.

[Grant Erickson]

This introduces a new 'OnlineCheckMode' configuration setting.

This setting is intended to eventually deprecate 'EnableOnlineCheck'
(long-term) and 'EnableOnlineToReadyTransition' (short-term).

When asserted along with 'EnableOnlineCheck',
'EnableOnlineToReadyTransition' effects what can be thought of as a
"continuous" online check mode, which is different from the "one-shot"
online check mode when 'EnableOnlineToReadyTransition' is not asserted
but 'EnableOnlineCheck' is. Effectively, these two Booleans encode
three online check modes:

    1. None (!EnableOnlineCheck)
    2. One-shot (EnableOnlineCheck && !EnableOnlineToReadyTransition)
    3. Continuous (EnableOnlineCheck && EnableOnlineToReadyTransition)

With this change, these three modes are all formalized.

In "none" mode, as has been the case historically, there are no
"online" HTTP-based Internet reachability checks. Any connected
service and the manager state will terminate at the "ready" state and
will not progress to "online".

In "one-shot" mode, as has been the case historically, there is a
single, one-shot "online" HTTP-based Internet reachability check for
the default service. When the check succeeds, the associated service
and the manager state will terminate at the "online" state. When the
check fails, subsequent checks will be rescheduled according to
"OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
"OnlineCheckMaxInterval" and will continue indefinitely until one
succeeds or until the service is disconnected.

In "continuous" mode, which is finalized with this change, there are
ongoing "online" HTTP-based Internet reachability check for the
default service. As with "one-shot" mode, when the first check
succeeds, the associated service and the manager state will terminate
at the "online" state. Thereafter, subsequent checks will be scheduled
according to "OnlineCheckIntervalStyle" and
"OnlineCheckMaxInterval". When the check fails, subsequent checks will
be rescheduled according to "OnlineCheckIntervalStyle",
"OnlineCheckInitialInterval", and "OnlineCheckMaxInterval". This is
largely unchanged. However, what is new with this change is that when
and if "OnlineCheckFailuresThreshold" is met, the service and manager
state will be demoted to "ready" and the service will have its "Error"
property set to "online-check-failed" while subsequent checks will
continue. In the interim, if available, another service may be
promoted to the default service and online checks will be initiated
for it. When and if, for the demoted service,
"OnlineCheckSuccessesThreshold" is met, the service "Error" property
will be cleared and the service state promoted to "online",
potentially causing it to become the default service again.
---
 src/connman.h |  9 ++++++++
 src/main.c    | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++
 src/service.c | 16 +++++++++++++++
 3 files changed, 82 insertions(+)

diff --git a/src/connman.h b/src/connman.h
index 796cb202570e..28280a55b0ec 100644
--- a/src/connman.h
+++ b/src/connman.h
@@ -800,6 +800,15 @@ const char *__connman_service_type2string(enum connman_service_type type);
 enum connman_service_type __connman_service_string2type(const char *str);
 enum connman_service_security __connman_service_string2security(const char *str);
 
+enum service_online_check_mode {
+	CONNMAN_SERVICE_ONLINE_CHECK_MODE_UNKNOWN    = 0,
+	CONNMAN_SERVICE_ONLINE_CHECK_MODE_NONE       = 1,
+	CONNMAN_SERVICE_ONLINE_CHECK_MODE_ONE_SHOT   = 2,
+	CONNMAN_SERVICE_ONLINE_CHECK_MODE_CONTINUOUS = 3
+};
+
+enum service_online_check_mode __connman_service_online_check_string2mode(
+		const char *mode);
 int __connman_service_nameserver_append(struct connman_service *service,
 				const char *nameserver, bool is_auto);
 int __connman_service_nameserver_remove(struct connman_service *service,
diff --git a/src/main.c b/src/main.c
index 8e47426f2e8c..97ed3ed744b1 100644
--- a/src/main.c
+++ b/src/main.c
@@ -112,6 +112,7 @@ static struct {
 	char *vendor_class_id;
 	bool enable_online_check;
 	bool enable_online_to_ready_transition;
+	enum service_online_check_mode online_check_mode;
 	char *online_check_ipv4_url;
 	char *online_check_ipv6_url;
 	unsigned int online_check_connect_timeout_ms;
@@ -146,6 +147,7 @@ static struct {
 	.vendor_class_id = NULL,
 	.enable_online_check = true,
 	.enable_online_to_ready_transition = false,
+	.online_check_mode = CONNMAN_SERVICE_ONLINE_CHECK_MODE_UNKNOWN,
 	.online_check_ipv4_url = NULL,
 	.online_check_ipv6_url = NULL,
 	.online_check_connect_timeout_ms = DEFAULT_ONLINE_CHECK_CONNECT_TIMEOUT,
@@ -182,6 +184,7 @@ static struct {
 #define CONF_VENDOR_CLASS_ID            "VendorClassID"
 #define CONF_ENABLE_ONLINE_CHECK        "EnableOnlineCheck"
 #define CONF_ENABLE_ONLINE_TO_READY_TRANSITION "EnableOnlineToReadyTransition"
+#define CONF_ONLINE_CHECK_MODE          "OnlineCheckMode"
 #define CONF_ONLINE_CHECK_IPV4_URL      "OnlineCheckIPv4URL"
 #define CONF_ONLINE_CHECK_IPV6_URL      "OnlineCheckIPv6URL"
 #define CONF_ONLINE_CHECK_CONNECT_TIMEOUT "OnlineCheckConnectTimeout"
@@ -217,6 +220,7 @@ static const char *supported_options[] = {
 	CONF_VENDOR_CLASS_ID,
 	CONF_ENABLE_ONLINE_CHECK,
 	CONF_ENABLE_ONLINE_TO_READY_TRANSITION,
+	CONF_ONLINE_CHECK_MODE,
 	CONF_ONLINE_CHECK_IPV4_URL,
 	CONF_ONLINE_CHECK_IPV6_URL,
 	CONF_ONLINE_CHECK_CONNECT_TIMEOUT,
@@ -345,6 +349,36 @@ static void check_config(GKeyFile *config)
 	g_strfreev(keys);
 }
 
+static void online_check_mode_set_from_deprecated(void)
+{
+	connman_settings.online_check_mode =
+		connman_settings.enable_online_check ?
+		connman_settings.enable_online_to_ready_transition ?
+			CONNMAN_SERVICE_ONLINE_CHECK_MODE_CONTINUOUS :
+			CONNMAN_SERVICE_ONLINE_CHECK_MODE_ONE_SHOT :
+		CONNMAN_SERVICE_ONLINE_CHECK_MODE_NONE;
+}
+
+static void online_check_mode_set_to_deprecated(void)
+{
+	switch (connman_settings.online_check_mode) {
+	case CONNMAN_SERVICE_ONLINE_CHECK_MODE_NONE:
+		connman_settings.enable_online_check = false;
+		connman_settings.enable_online_to_ready_transition = false;
+		break;
+	case CONNMAN_SERVICE_ONLINE_CHECK_MODE_ONE_SHOT:
+		connman_settings.enable_online_check = true;
+		connman_settings.enable_online_to_ready_transition = false;
+		break;
+	case CONNMAN_SERVICE_ONLINE_CHECK_MODE_CONTINUOUS:
+		connman_settings.enable_online_check = true;
+		connman_settings.enable_online_to_ready_transition = true;
+		break;
+	default:
+		break;
+	}
+}
+
 static void parse_config(GKeyFile *config)
 {
 	GError *error = NULL;
@@ -549,6 +583,26 @@ static void parse_config(GKeyFile *config)
 
 	g_clear_error(&error);
 
+	/* OnlineCheckMode */
+
+	string = __connman_config_get_string(config, "General",
+				CONF_ONLINE_CHECK_MODE, &error);
+	if (!error) {
+		connman_settings.online_check_mode =
+			__connman_service_online_check_string2mode(string);
+		if (connman_settings.online_check_mode ==
+			CONNMAN_SERVICE_ONLINE_CHECK_MODE_UNKNOWN) {
+			connman_error("Invalid online check mode \"%s\"",
+				string);
+
+			online_check_mode_set_from_deprecated();
+		} else
+			online_check_mode_set_to_deprecated();
+	} else
+		online_check_mode_set_from_deprecated();
+
+	g_clear_error(&error);
+
 	/* OnlineCheckConnecTimeout */
 
 	real = g_key_file_get_double(config, "General",
@@ -966,6 +1020,9 @@ unsigned int connman_setting_get_uint(const char *key)
 	if (g_str_equal(key, CONF_ONLINE_CHECK_MAX_INTERVAL))
 		return connman_settings.online_check_max_interval;
 
+	if (g_str_equal(key, CONF_ONLINE_CHECK_MODE))
+		return connman_settings.online_check_mode;
+
 	if (g_str_equal(key, CONF_ONLINE_CHECK_FAILURES_THRESHOLD))
 		return connman_settings.online_check_failures_threshold;
 
diff --git a/src/service.c b/src/service.c
index e44eea376636..c427fc441193 100644
--- a/src/service.c
+++ b/src/service.c
@@ -1749,6 +1749,22 @@ static bool check_proxy_setup(struct connman_service *service)
 	return false;
 }
 
+enum service_online_check_mode __connman_service_online_check_string2mode(
+				const char *mode)
+{
+	if (!mode)
+		return CONNMAN_SERVICE_ONLINE_CHECK_MODE_UNKNOWN;
+
+	if (g_strcmp0(mode, "none") == 0)
+		return CONNMAN_SERVICE_ONLINE_CHECK_MODE_NONE;
+	else if (g_strcmp0(mode, "one-shot") == 0)
+		return CONNMAN_SERVICE_ONLINE_CHECK_MODE_ONE_SHOT;
+	else if (g_strcmp0(mode, "continuous") == 0)
+		return CONNMAN_SERVICE_ONLINE_CHECK_MODE_CONTINUOUS;
+
+	return CONNMAN_SERVICE_ONLINE_CHECK_MODE_UNKNOWN;
+}
+
 /**
  *  @brief
  *    Determine whether an "online" HTTP-based Internet reachability
-- 
2.42.0

[PATCH v3 56/60] service: Leverage 'OnlineCheckMode' setting.

[Grant Erickson]

This leverages the newly-introduced 'OnlineCheckMode' setting,
displacing and deprecating both the use of 'EnableOnlineCheck' and
'EnableOnlineToReadyTransition' in this module.
---
 src/connman.h |   7 ++++
 src/service.c | 113 +++++++++++++++++++++++++++++++++++++++++---------
 2 files changed, 100 insertions(+), 20 deletions(-)

diff --git a/src/connman.h b/src/connman.h
index 28280a55b0ec..b33704d31bc4 100644
--- a/src/connman.h
+++ b/src/connman.h
@@ -807,8 +807,15 @@ enum service_online_check_mode {
 	CONNMAN_SERVICE_ONLINE_CHECK_MODE_CONTINUOUS = 3
 };
 
+const char *__connman_service_online_check_mode2string(
+		enum service_online_check_mode mode);
 enum service_online_check_mode __connman_service_online_check_string2mode(
 		const char *mode);
+enum service_online_check_mode __connman_service_get_online_check_mode(void);
+bool __connman_service_is_online_check_enabled(void);
+bool __connman_service_is_online_check_mode(
+		enum service_online_check_mode mode);
+
 int __connman_service_nameserver_append(struct connman_service *service,
 				const char *nameserver, bool is_auto);
 int __connman_service_nameserver_remove(struct connman_service *service,
diff --git a/src/service.c b/src/service.c
index c427fc441193..99b322a91684 100644
--- a/src/service.c
+++ b/src/service.c
@@ -83,7 +83,6 @@ static unsigned int vpn_autoconnect_id = 0;
  */
 static struct connman_service *current_default = NULL;
 static bool services_dirty = false;
-static bool enable_online_to_ready_transition = false;
 static unsigned int online_check_connect_timeout_ms = 0;
 static unsigned int online_check_initial_interval = 0;
 static unsigned int online_check_max_interval = 0;
@@ -1749,6 +1748,25 @@ static bool check_proxy_setup(struct connman_service *service)
 	return false;
 }
 
+const char *__connman_service_online_check_mode2string(
+				enum service_online_check_mode mode)
+{
+	switch (mode) {
+	case CONNMAN_SERVICE_ONLINE_CHECK_MODE_UNKNOWN:
+		break;
+	case CONNMAN_SERVICE_ONLINE_CHECK_MODE_NONE:
+		return "none";
+	case CONNMAN_SERVICE_ONLINE_CHECK_MODE_ONE_SHOT:
+		return "one-shot";
+	case CONNMAN_SERVICE_ONLINE_CHECK_MODE_CONTINUOUS:
+		return "continuous";
+	default:
+		break;
+	}
+
+	return NULL;
+}
+
 enum service_online_check_mode __connman_service_online_check_string2mode(
 				const char *mode)
 {
@@ -1765,6 +1783,61 @@ enum service_online_check_mode __connman_service_online_check_string2mode(
 	return CONNMAN_SERVICE_ONLINE_CHECK_MODE_UNKNOWN;
 }
 
+/**
+ *  @brief
+ *    Return the "online" HTTP-based Internet reachability check mode.
+ *
+ *  @returns
+ *    The "online" HTTP-based Internet reachability check mode.
+ *
+ */
+enum service_online_check_mode __connman_service_get_online_check_mode(void)
+{
+	return connman_setting_get_uint("OnlineCheckMode");
+}
+
+/**
+ *  @brief
+ *    Return whether the "online" HTTP-based Internet reachability
+ *    checks are enabled.
+ *
+ *  @returns
+ *    True if "online" HTTP-based Internet reachability checks are
+ *    enabled; otherwise, false.
+ *
+ *  @sa __connman_service_get_online_check_mode
+ *
+ */
+bool __connman_service_is_online_check_enabled(void)
+{
+	const enum service_online_check_mode mode =
+		__connman_service_get_online_check_mode();
+
+	return mode != CONNMAN_SERVICE_ONLINE_CHECK_MODE_UNKNOWN &&
+		mode != CONNMAN_SERVICE_ONLINE_CHECK_MODE_NONE;
+}
+
+/**
+ *  @brief
+ *    Determines whether the "online" HTTP-based Internet reachability
+ *    check mode is the specified mode.
+ *
+ *  @param[in]  mode  The "online" HTTP-based Internet reachability
+ *                    check mode to confirm.
+ *
+ *  @returns
+ *    True if the current "online" HTTP-based Internet reachability
+ *    check mode is @a mode; otherwise, false.
+ *
+ *  @sa __connman_service_get_online_check_mode
+ *
+ */
+bool __connman_service_is_online_check_mode(
+		enum service_online_check_mode mode)
+{
+	return __connman_service_get_online_check_mode() == mode;
+}
+
 /**
  *  @brief
  *    Determine whether an "online" HTTP-based Internet reachability
@@ -2122,7 +2195,7 @@ static void cancel_online_check(struct connman_service *service,
 static bool online_check_is_enabled_check(
 		const struct connman_service *service)
 {
-	if (!connman_setting_get_bool("EnableOnlineCheck")) {
+	if (!__connman_service_is_online_check_enabled()) {
 		connman_info("Online check disabled. "
 			"Default service remains in READY state.");
 		return false;
@@ -2142,7 +2215,7 @@ static bool online_check_is_enabled_check(
  *
  *  @note
  *    Any check is skipped, with an informational log message, if @a
- *    EnableOnlineCheck is not asserted.
+ *    OnlineCheckMode is "none".
  *
  *  @param[in,out]  service  A pointer to the mutable network service
  *                           for which to start the "online"
@@ -3614,22 +3687,22 @@ done:
  *  configuration type. This effectively "bookends" an earlier
  *  #__connman_service_wispr_start.
  *
- *  If "EnableOnlineToReadyTransition" is deasserted and if @a success
- *  is asserted, then the state for the specified IP configuration
- *  type is transitioned to "online" and a future online check is
+ *  If "OnlineCheckMode" is "one-shot" and if @a success is asserted,
+ *  then the state for the specified IP configuration type is
+ *  transitioned to "online" and a future online check is scheduled
+ *  based on the current interval and the "OnlineCheckIntervalStyle"
+ *  setting.
+ *
+ *  Otherwise, if "OnlineCheckMode" is "continuous", then counters are
+ *  managed for the success or failure and state is managed and
+ *  tracked resulting in the potential demotion of the service,
+ *  placing it into a temporary failure state until such time as a
+ *  series of back-to-back online checks successfully complete. If the
+ *  service is a non-default after demotion and it is in failure state
+ *  or if it is the default service, then a future online check is
  *  scheduled based on the current interval and the
  *  "OnlineCheckIntervalStyle" setting.
  *
- *  Otherwise, if "EnableOnlineToReadyTransition" is asserted, then
- *  counters are managed for the success or failure and state is
- *  managed and tracked resulting in the potential demotion of the
- *  service, placing it into a temporary failure state until such time
- *  as a series of back-to-back online checks successfully
- *  complete. If the service is a non-default after demotion and it is
- *  in failure state or if it is the default service, then a future
- *  online check is scheduled based on the current interval and the
- *  "OnlineCheckIntervalStyle" setting.
- *
  *  @param[in,out]  service  A pointer to the mutable service for which
  *                           to complete a previously-requested online
  *                           check.
@@ -3658,6 +3731,8 @@ static void complete_online_check(struct connman_service *service,
 					bool success,
 					int err)
 {
+	const bool oneshot = __connman_service_is_online_check_mode(
+		CONNMAN_SERVICE_ONLINE_CHECK_MODE_ONE_SHOT);
 	struct online_check_state *online_check_state;
 	enum connman_service_state ipconfig_state;
 	bool reschedule = false;
@@ -3682,13 +3757,13 @@ static void complete_online_check(struct connman_service *service,
 		reschedule = handle_online_check_success(service,
 					 type,
 					 online_check_state,
-					 !enable_online_to_ready_transition);
+					 oneshot);
 	else
 		reschedule = handle_online_check_failure(service,
 					 type,
 					 ipconfig_state,
 					 online_check_state,
-					 !enable_online_to_ready_transition,
+					 oneshot,
 					 err);
 
 	DBG("reschedule online check %u", reschedule);
@@ -10689,8 +10764,6 @@ int __connman_service_init(void)
 
 	remove_unprovisioned_services();
 
-	enable_online_to_ready_transition =
-		connman_setting_get_bool("EnableOnlineToReadyTransition");
 	online_check_timeout_interval_style =
 		connman_setting_get_string("OnlineCheckIntervalStyle");
 	if (g_strcmp0(online_check_timeout_interval_style, "fibonacci") == 0)
-- 
2.42.0

[PATCH v3 57/60] doc: Document the 'OnlineCheckMode' setting.

[Grant Erickson]

This documents the 'OnlineCheckMode' configuration setting.
---
 README                |  91 +++++++++++++++++++++++++++---------
 doc/connman.conf.5.in |  73 +++++++++++++++++++++++------
 src/main.conf         | 106 ++++++++++++++++++++++++++++++------------
 3 files changed, 203 insertions(+), 67 deletions(-)

diff --git a/README b/README
index 75cb549d513e..2dee1e5fe73c 100644
--- a/README
+++ b/README
@@ -408,29 +408,74 @@ from ipv4.connman.net (for IPv4 connectivity) and ipv6.connman.net
 (for IPv6 connectivity). The used URL looks like this
 http://ipv{4|6}.connman.net/online/status.html
 
-When an online check request fails, another one is triggered after a
-longer interval. The intervals follows one of two mathemetical
-sequences, depending on the "OnlineCheckIntervalStyle" setting:
-"fibonacci" or "geometric", with a default of "geometric". The
-geometric setting is the square series of numbers in the range
-specified by "OnlineCheckInitialInterval" and "OnlineCheckMaxInterval".
-The default values for "OnlineCheckInitialInterval" and
-"OnlineCheckMaxInterval" are the range [1, 12], which correspond to the
-following "geometric" intervals, in seconds: 1, 4, 9, 16, 25, 36, 49, 64,
-81, 100, 121 and 144 over that range. By contrast, the correspending
-"fibonacci" sequence over that range is 1, 1, 2, 3, 5, 8, 13, 21, 34, 55,
-89, and 144. The "fibonacci" series and style is more aggressive in check
-rate up to 12 steps (its equivalence point with "geometric" at 144 seconds)
-than "geometric" but backs off far more aggressively past that point
-reaching an hour at interval 19 which "geometric" does not reach until
-interval 60.
-
-See connman.conf(5) for the EnableOnlineCheck option, if you need to
-disable the feature.
-It is also possible to specify other URLs via OnlineCheckIPv4URL and
-OnlineCheckIPv6URL options.
-The range of intervals between two online check requests can be fine-tuned
-via OnlineCheckInitialInterval and OnlineCheckMaxInterval options.
+The online check operates in one of three modes:
+
+  * "none"
+  * "one-shot"
+  * "continuous"
+
+where "one-shot" is the default and is governed by the
+"OnlineCheckMode" setting.
+
+In "none" mode, there are no "online" HTTP-based Internet
+reachability checks. Any connected service and the manager state
+will terminate at the Ready state and will not progress to
+Online.
+
+In "one-shot", the default mode, there is a single, one-shot "online"
+HTTP-based Internet reachability check for the default service (that
+is, the service with the high-priority (metric 0) gateway default
+route). When the check succeeds, the associated service and the
+manager state will terminate at the "online" state. When the check
+fails, subsequent checks will be rescheduled according to
+"OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
+"OnlineCheckMaxInterval" and will continue indefinitely until one
+succeeds or until the service is disconnected.
+
+In "continuous" mode, there are ongoing "online" HTTP-based Internet
+reachability checks for the default service (that is, the service with
+the high-priority (metric 0) gateway default route). As with
+"one-shot" mode, when the first check succeeds, the associated service
+and the manager state will terminate at the Online state. Thereafter,
+subsequent checks will be scheduled according to
+"OnlineCheckIntervalStyle" and "OnlineCheckMaxInterval". When the
+check fails, subsequent checks will be rescheduled according to
+"OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
+"OnlineCheckMaxInterval". When and if "OnlineCheckFailuresThreshold"
+is met, the service and manager state will be demoted to Ready and the
+service will have its "Error" property set to "online-check-failed"
+while subsequent checks will continue. In the interim, if available,
+another service may be promoted to the default service and online
+checks will be initiated for it. When and if, for the demoted service,
+"OnlineCheckSuccessesThreshold" is met, the service "Error" property
+will be cleared and the service state promoted to Online, potentially
+causing it to become the default service again.
+
+See connman.conf(5) for the "OnlineCheckMode" option, if you need to
+disable the feature. It is also possible to specify other URLs via
+"OnlineCheckIPv4URL" and "OnlineCheckIPv6URL" options. The range of
+intervals between two online check requests can be fine-tuned via
+"OnlineCheckInitialInterval" and "OnlineCheckMaxInterval" options as
+well as with the "OnlineCheckIntervalStyle" option.
+
+As intimated above, for the "one-shot" and "continuous" modes, when an
+online check request fails (or, in the case of "continuous" mode,
+succeeds as well), another one is triggered after a longer
+interval. The intervals follows one of two mathemetical sequences,
+depending on the "OnlineCheckIntervalStyle" setting: "fibonacci" or
+"geometric", with a default of "geometric". The geometric setting is
+the square series of numbers in the range specified by
+"OnlineCheckInitialInterval" and "OnlineCheckMaxInterval".  The
+default values for "OnlineCheckInitialInterval" and
+"OnlineCheckMaxInterval" are the range [1, 12], which correspond to
+the following "geometric" intervals, in seconds: 1, 4, 9, 16, 25, 36,
+49, 64, 81, 100, 121 and 144 over that range. By contrast, the
+correspending "fibonacci" sequence over that range is 1, 1, 2, 3, 5,
+8, 13, 21, 34, 55, 89, and 144. The "fibonacci" series and style is
+more aggressive in check rate up to 12 steps (its equivalence point
+with "geometric" at 144 seconds) than "geometric" but backs off far
+more aggressively past that point reaching an hour at interval 19
+which "geometric" does not reach until interval 60.
 
 During the online check procedure, ConnMan will temporarily install
 a host route to both the ipv4.connman.net and ipv6.connman.net so that
diff --git a/doc/connman.conf.5.in b/doc/connman.conf.5.in
index bc5a112f758a..966c20709724 100644
--- a/doc/connman.conf.5.in
+++ b/doc/connman.conf.5.in
@@ -166,7 +166,49 @@ ConnMan will issue an HTTP GET request to verify that end-to-end
 connectivity is successful. Only then the service will be
 transitioned to ONLINE state.
 If this setting is false, the default service will remain in READY state.
-Default value is true.
+Default value is \fBtrue\fR.
+
+\fBNOTE:\fR \fBEnableOnlineCheck\fR is deprecated; please use
+\fBOnlineCheckMode\fR with a mode of either \fBone-shot\fR or
+\fBnone\fR.
+
+.TP
+.BI OnlineCheckMode= mode
+This indicates the "online" HTTP-based Internet reachability check
+mode and supercedes and deprecates \fBEnableOnlineCheck\fR. Possible
+values for \fImode\fR are \fBnone\fR, \fBone-shot\fR, and
+\fBcontinuous\fR. The default value of \fImode\fR is \fBone-shot\fR.
+
+In \fBnone\fR mode, there are no "online" HTTP-based Internet reachability
+checks. Any connected service and the manager state will terminate at
+the "ready" state and will not progress to "online".
+
+In \fBone-shot\fR mode, there is a single, one-shot "online" HTTP-based
+Internet reachability check for the default service. When the check
+succeeds, the associated service and the manager state will terminate
+at the "online" state. When the check fails, subsequent checks will be
+rescheduled according to \fBOnlineCheckIntervalStyle\fR,
+\fBOnlineCheckInitialInterval\fR, and \fBOnlineCheckMaxInterval\fR and will
+continue indefinitely until one succeeds or unti the service is
+disconnected.
+
+In \fBcontinuous\fR mode, there are ongoing "online" HTTP-based Internet
+reachability check for the default service. As with "one-shot" mode,
+when the first check succeeds, the associated service and the manager
+state will terminate at the "online" state. Thereafter, subsequent
+checks will be scheduled according to \fBOnlineCheckIntervalStyle\fR and
+\fBOnlineCheckMaxInterval\fR. When the check fails, subsequent checks will
+be rescheduled according to \fBOnlineCheckIntervalStyle\fR,
+\fBOnlineCheckInitialInterval\fR, and \fBOnlineCheckMaxInterval\fR. When and
+if \fBOnlineCheckFailuresThreshold\fR is met, the service and manager
+state will be demoted to "ready" and the service will have its "Error"
+property set to "online-check-failed" while subsequent checks will
+continue. In the interim, if available, another service may be
+promoted to the default service and online checks will be initiated
+for it. When and if, for the demoted service,
+\fBOnlineCheckSuccessesThreshold\fR is met, the service "Error" property
+will be cleared and the service state promoted to "online",
+potentially causing it to become the default service again.
 .TP
 .BI OnlineCheckIPv4URL= url, OnlineCheckIPv6URL= url
 Urls (IPv4 and IPv6 respectively) used during the online status check.
@@ -220,12 +262,15 @@ If this setting is true, the HTTP GET request keeps beeing called to guarantee
 that end-to-end connectivity is still successful. If not, the default service
 will transition to READY state, enabling another service to become the
 default one, in replacement.
-Default value is false.
+Default value is \fBfalse\fR.
+
+\fBNOTE:\fR \fBEnableOnlineToReadyTransition\fR is deprecated; please use
+\fBOnlineCheckMode\fR with a mode of either \fBcontinuous\fR or
+\fBnone\fR.
 .TP
 .BI OnlineCheckFailuresThreshold= failures
-When both \fBEnableOnlineCheck\fR and
-\fBEnableOnlineToReadyTransition\fR are asserted, this is the number
-of failed back-to-back "ready" to "online" HTTP-based Internet
+When \fBOnlineCheckMode\fR is "continuous", this is the number of
+failed back-to-back "ready" to "online" HTTP-based Internet
 reachability checks that will be allowed before marking a service as
 "failed" from a reachability perspective, sorting it at a lower
 priority than other services not so marked.
@@ -234,16 +279,14 @@ Lower values may result in higher-frequency network service cycling
 while higher values may result in a longer period of time before
 failing from a non-Internet reachable service to one that might be.
 
-See \fBOnlineCheckIntervalStyle\fR, \fBOnlineCheckInitialInterval\fR,
-and \fBOnlineCheckMaxInterval\fR for other values that influence
-network service failure/recovery transition time.
-
+See \fBOnlineCheckIntervalStyle\fR, \fBOnlineCheckInitialInterval\fR, and
+\fBOnlineCheckMaxInterval\fR for other values that influence network
+service failure/recovery transition time.
 The default value is 6.
 .TP
 .BI OnlineCheckSuccessesThreshold= successes
-When both \fBEnableOnlineCheck\fR and
-\fBEnableOnlineToReadyTransition\fR are asserted, this is the number
-of successful back-to-back "ready" to "online" HTTP-based Internet
+When \fBOnlineCheckMode\fR is "continuous", this is the number of
+successful back-to-back "ready" to "online" HTTP-based Internet
 reachability checks that must be met before clearing a service as
 "failed" from a reachability perspective and allowing it to transition
 to the "online" state again, allowing it to sort back to a higher
@@ -254,9 +297,9 @@ while higher values may result in a longer period of time before
 transitioning back to more a preferred, Internet reachable network
 service.
 
-See \fBOnlineCheckIntervalStyle\fR, \fBOnlineCheckInitialInterval\fR,
-and \fBOnlineCheckMaxInterval\fR for other values that influence
-network service failure/recovery transition time.
+See \fBOnlineCheckIntervalStyle\fR, \fBOnlineCheckInitialInterval\fR, and
+\fBOnlineCheckMaxInterval\fR for other values that influence network
+service failure/recovery transition time.
 
 The default value is 6.
 .TP
diff --git a/src/main.conf b/src/main.conf
index b0eb93811f85..5357edb82f90 100644
--- a/src/main.conf
+++ b/src/main.conf
@@ -116,11 +116,51 @@
 # section 4.1).
 # Enable6to4 = false
 
-# Enable use of http get as on online status check.
-# When a service is in a READY state, and is selected as default,
-# ConnMan will issue an HTTP GET request to verify that end-to-end
-# connectivity is successful. Only then the service will be
-# transitioned to ONLINE state.
+# This indicates the "online" HTTP-based Internet reachability check
+# mode. Possible values are "none", "one-shot", "continuous".
+#
+# In "none" mode, there are no "online" HTTP-based Internet
+# reachability checks. Any connected service and the manager state
+# will terminate at the "ready" state and will not progress to
+# "online".
+#
+# In "one-shot" mode, there is a single, one-shot "online" HTTP-based
+# Internet reachability check for the default service. When the check
+# succeeds, the associated service and the manager state will
+# terminate at the "online" state. When the check fails, subsequent
+# checks will be rescheduled according to "OnlineCheckIntervalStyle",
+# "OnlineCheckInitialInterval", and "OnlineCheckMaxInterval" and will
+# continue indefinitely until one succeeds or until the service is
+# disconnected.
+#
+# In "continuous" mode, there are ongoing "online" HTTP-based Internet
+# reachability check for the default service. As with "one-shot" mode,
+# when the first check succeeds, the associated service and the
+# manager state will terminate at the "online" state. Thereafter,
+# subsequent checks will be scheduled according to
+# "OnlineCheckIntervalStyle" and "OnlineCheckMaxInterval". When the
+# check fails, subsequent checks will be rescheduled according to
+# "OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
+# "OnlineCheckMaxInterval". When and if "OnlineCheckFailuresThreshold"
+# is met, the service and manager state will be demoted to "ready" and
+# the service will have its "Error" property set to
+# "online-check-failed" while subsequent checks will continue. In the
+# interim, if available, another service may be promoted to the
+# default service and online checks will be initiated for it. When and
+# if, for the demoted service, "OnlineCheckSuccessesThreshold" is met,
+# the service "Error" property will be cleared and the service state
+# promoted to "online", potentially causing it to become the default
+# service again.
+# Default value is "one-shot".
+# OnlineCheckMode = one-shot
+
+# NOTE: This setting is deprecated; use "OnlineCheckMode" instead with
+# a value of "one-shot" or "none".
+# Enable the use of "online" HTTP-baesd Internet reachability check as
+# an online status check.  When a service is in a READY state, and is
+# selected as default, ConnMan will issue an HTTP GET request to
+# verify that end-to-end connectivity is successful. Only then the
+# service will be transitioned to ONLINE state.
 # If this setting is false, the default service will remain in READY state.
 # Default value is true.
 # EnableOnlineCheck = false
@@ -147,24 +187,28 @@
 # OnlineCheckInitialInterval = 1
 # OnlineCheckMaxInterval = 12
 
-# WARNING: Experimental feature!!!
-# In addition to EnableOnlineCheck setting, enable or disable use of HTTP GET
-# to detect the loss of end-to-end connectivity.
-# If this setting is false, when the default service transitions to ONLINE
-# state, the HTTP GET request is no more called until next cycle, initiated
-# by a transition of the default service to DISCONNECT state.
-# If this setting is true, the HTTP GET request keeps beeing called to guarantee
-# that end-to-end connectivity is still successful. If not, the default service
-# will transition to READY state, enabling another service to become the
-# default one, in replacement.
+# NOTE: This setting is deprecated; use "OnlineCheckMode" instead with
+# a value of "continuous" or "none".
+# In addition to the "EnableOnlineCheck" setting, enable or disable
+# continuous use of "online" HTTP-based Internet reachability checks
+# to detect the loss of end-to-end connectivity.
+# If this setting is false, when the default service transitions to
+# ONLINE state, the "online" HTTP-based Internet reachability check is
+# terminated until a transition of the default service to DISCONNECT
+# state.
+# If this setting is true, the "online" HTTP-based Internet
+# reachability checks continue to be rescheduled to guarantee that
+# end-to-end connectivity is still successful. If not, the default
+# service will transition to READY state, enabling another service to
+# become the default one, in replacement.
 # EnableOnlineToReadyTransition = false
 
 # When both "EnableOnlineCheck" and "EnableOnlineToReadyTransition"
-# are asserted, this is the number of failed back-to-back "ready" to
-# "online" HTTP-based Internet reachability checks that will be
-# allowed before marking a service as "failed" from a reachability
-# perspective, sorting it at a lower priority than other services not
-# so marked.
+# are asserted or "OnlineCheckMode" is "continuous", this is the
+# number of failed back-to-back "ready" to "online" HTTP-based
+# Internet reachability checks that will be allowed before marking a
+# service as "failed" from a reachability perspective, sorting it at a
+# lower priority than other services not so marked.
 #
 # Lower values may result in higher-frequency network service cycling
 # while higher values may result in a longer period of time before
@@ -178,12 +222,12 @@
 # OnlineCheckFailuresThreshold=6
 
 # When both "EnableOnlineCheck" and "EnableOnlineToReadyTransition"
-# are asserted, this is the number of successful back-to-back "ready"
-# to "online" HTTP-based Internet reachability checks that must be met
-# before clearing a service as "failed" from a reachability
-# perspective and allowing it to transition to the "online" state
-# again, allowing it to sort back to a higher priority relative to
-# other network services.
+# are asserted or "OnlineCheckMode" is "continuous", this is the
+# number of successful back-to-back "ready" to "online" HTTP-based
+# Internet reachability checks that must be met before clearing a
+# service as "failed" from a reachability perspective and allowing it
+# to transition to the "online" state again, allowing it to sort back
+# to a higher priority relative to other network services.
 #
 # Lower values may result in higher-frequency network service cycling
 # while higher values may result in a longer period of time before
@@ -211,9 +255,13 @@
 # interval of 6, the time, in seconds, is 8 seconds.
 #
 # "fibonacci" is more aggressive in check rate up to 12 steps (its
-# equivalence point with "geometric" at 144 seconds) but is far less
-# aggressive past that point, reacing an hour at interval 19 compared
-# to "geometric" which does not reach an hour until interval 60.
+# equivalence point with "geometric" at 144 seconds) than "geometric"
+# and yields quicker recovery for transient failures. For example, for
+# an "OnlineCheckSuccessesThreshold" of six (6), "fibonacci" gets the
+# failing service back to "online" in 20 seconds versus 91 seconds for
+# "geometric". By comparison, past 12 steps, "fibonacci" backs off far
+# more aggressively, which is less wasteful, particularly for a
+# metered interface like Cellular, when the failures are more enduring.
 #
 # Default value is "geometric".
 # OnlineCheckIntervalStyle=geometric
-- 
2.42.0

[PATCH v3 58/60] main: Add comments tags for online check-related settings.

[Grant Erickson]

At this point, there are a sufficient number of settings in
'parse_config' that it is easy to lose track of where the block for
one settings handler ends and the next begins.

This adds comment tags for online check-related settings to make it
easy, at a glance, to see where the block for each is.
---
 src/main.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/src/main.c b/src/main.c
index 97ed3ed744b1..c8d97ee80f03 100644
--- a/src/main.c
+++ b/src/main.c
@@ -565,6 +565,8 @@ static void parse_config(GKeyFile *config)
 
 	g_clear_error(&error);
 
+	/* EnableOnlineCheck */
+
 	boolean = __connman_config_get_bool(config, "General",
 					CONF_ENABLE_ONLINE_CHECK, &error);
 	if (!error) {
@@ -575,6 +577,8 @@ static void parse_config(GKeyFile *config)
 
 	g_clear_error(&error);
 
+	/* EnableOnlineToReadyTransition */
+
 	boolean = __connman_config_get_bool(config, "General",
 			CONF_ENABLE_ONLINE_TO_READY_TRANSITION, &error);
 	if (!error) {
@@ -603,7 +607,7 @@ static void parse_config(GKeyFile *config)
 
 	g_clear_error(&error);
 
-	/* OnlineCheckConnecTimeout */
+	/* OnlineCheckConnectTimeout */
 
 	real = g_key_file_get_double(config, "General",
 			CONF_ONLINE_CHECK_CONNECT_TIMEOUT, &error);
@@ -624,6 +628,8 @@ static void parse_config(GKeyFile *config)
 
 	g_clear_error(&error);
 
+	/* OnlineCheckIPv4URL */
+
 	string = __connman_config_get_string(config, "General",
 					CONF_ONLINE_CHECK_IPV4_URL, &error);
 	if (!error)
@@ -634,6 +640,8 @@ static void parse_config(GKeyFile *config)
 
 	g_clear_error(&error);
 
+	/* OnlineCheckIPv6URL */
+
 	string = __connman_config_get_string(config, "General",
 					CONF_ONLINE_CHECK_IPV6_URL, &error);
 	if (!error)
@@ -645,6 +653,8 @@ static void parse_config(GKeyFile *config)
 
 	g_clear_error(&error);
 
+	/* OnlineCheck{Initial,Max}Interval */
+
 	integer = g_key_file_get_integer(config, "General",
 			CONF_ONLINE_CHECK_INITIAL_INTERVAL, &error);
 	if (!error && integer >= 0)
@@ -703,6 +713,8 @@ static void parse_config(GKeyFile *config)
 
 	g_clear_error(&error);
 
+	/* OnlineCheckIntervalStyle */
+
 	string = __connman_config_get_string(config, "General",
 					CONF_ONLINE_CHECK_INTERVAL_STYLE, &error);
 	if (!error) {
-- 
2.42.0

[PATCH v3 59/60] main: Consolidate logging of online check-related settings.

[Grant Erickson]

This consolidates logging, at the info level, of all online
check-related settings to a single place, 'online_check_settings_log'
such that when the 'OnlineCheckMode' is "none", the following is
logged:

    Online check mode "none"

when 'OnlineCheckMode' is "one-shot", the following are logged:

    Online check mode "one-shot"
    Online check IPv4 URL "http://ipv4.connman.net/online/status.html"
    Online check IPv6 URL "http://ipv6.connman.net/online/status.html"
    Online check interval style "geometric"
    Online check interval range [1, 12]
    Online check connect timeout 12100 ms

and when 'OnlineCheckMode' is "continuous the following are logged:

    Online check mode "continuous"
    Online check IPv4 URL "http://ipv4.connman.net/online/status.html"
    Online check IPv6 URL "http://ipv6.connman.net/online/status.html"
    Online check interval style "geometric"
    Online check interval range [1, 12]
    Online check connect timeout 12100 ms
    Online check continuous mode failures threshold 6
    Online check continuous mode successes threshold 6
---
 src/main.c | 47 ++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 40 insertions(+), 7 deletions(-)

diff --git a/src/main.c b/src/main.c
index c8d97ee80f03..f8007cb3be65 100644
--- a/src/main.c
+++ b/src/main.c
@@ -379,6 +379,44 @@ static void online_check_mode_set_to_deprecated(void)
 	}
 }
 
+static void online_check_settings_log(void)
+{
+	connman_info("Online check mode \"%s\"",
+				 __connman_service_online_check_mode2string(
+					connman_settings.online_check_mode));
+
+	if (connman_settings.online_check_mode ==
+			CONNMAN_SERVICE_ONLINE_CHECK_MODE_NONE)
+		return;
+
+	connman_info("Online check IPv4 URL \"%s\"",
+		connman_settings.online_check_ipv4_url);
+
+	connman_info("Online check IPv6 URL \"%s\"",
+		connman_settings.online_check_ipv6_url);
+
+	connman_info("Online check interval style \"%s\"",
+		connman_settings.online_check_interval_style);
+
+	connman_info("Online check interval range [%u, %u]",
+		connman_settings.online_check_initial_interval,
+		connman_settings.online_check_max_interval);
+
+	if (connman_settings.online_check_connect_timeout_ms)
+		connman_info("Online check connect timeout %u ms",
+			connman_settings.online_check_connect_timeout_ms);
+
+	if (connman_settings.online_check_mode !=
+			CONNMAN_SERVICE_ONLINE_CHECK_MODE_CONTINUOUS)
+		return;
+
+	connman_info("Online check continuous mode failures threshold %d",
+		connman_settings.online_check_failures_threshold);
+
+	connman_info("Online check continuous mode successes threshold %d",
+		connman_settings.online_check_successes_threshold);
+}
+
 static void parse_config(GKeyFile *config)
 {
 	GError *error = NULL;
@@ -571,8 +609,6 @@ static void parse_config(GKeyFile *config)
 					CONF_ENABLE_ONLINE_CHECK, &error);
 	if (!error) {
 		connman_settings.enable_online_check = boolean;
-		if (!boolean)
-			connman_info("Online check disabled by main config.");
 	}
 
 	g_clear_error(&error);
@@ -622,10 +658,6 @@ static void parse_config(GKeyFile *config)
 				real * 1000;
 	}
 
-	if (connman_settings.online_check_connect_timeout_ms)
-		connman_info("Online check connect timeout %ums",
-			connman_settings.online_check_connect_timeout_ms);
-
 	g_clear_error(&error);
 
 	/* OnlineCheckIPv4URL */
@@ -650,7 +682,6 @@ static void parse_config(GKeyFile *config)
 		connman_settings.online_check_ipv6_url =
 			g_strdup(DEFAULT_ONLINE_CHECK_IPV6_URL);
 
-
 	g_clear_error(&error);
 
 	/* OnlineCheck{Initial,Max}Interval */
@@ -777,6 +808,8 @@ static void parse_config(GKeyFile *config)
 		g_free(string);
 
 	g_clear_error(&error);
+
+	online_check_settings_log();
 }
 
 static int config_init(const char *file)
-- 
2.42.0

[PATCH v3 60/60] main: Add deprecation warnings for 'EnableOnlineCheck' and 'EnableOnlineToReadyTransition'.

[Grant Erickson]

This adds deprecation warning log messages for 'EnableOnlineCheck' and
'EnableOnlineToReadyTransition' with guidance to transition to
'OnlineCheckMode'.
---
 src/main.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/src/main.c b/src/main.c
index f8007cb3be65..241c713d7980 100644
--- a/src/main.c
+++ b/src/main.c
@@ -608,6 +608,10 @@ static void parse_config(GKeyFile *config)
 	boolean = __connman_config_get_bool(config, "General",
 					CONF_ENABLE_ONLINE_CHECK, &error);
 	if (!error) {
+		connman_warn("\"%s\" is deprecated; use \"%s\" instead.",
+			CONF_ENABLE_ONLINE_CHECK,
+			CONF_ONLINE_CHECK_MODE);
+
 		connman_settings.enable_online_check = boolean;
 	}
 
@@ -618,6 +622,10 @@ static void parse_config(GKeyFile *config)
 	boolean = __connman_config_get_bool(config, "General",
 			CONF_ENABLE_ONLINE_TO_READY_TRANSITION, &error);
 	if (!error) {
+		connman_warn("\"%s\" is deprecated; use \"%s\" instead.",
+			CONF_ENABLE_ONLINE_TO_READY_TRANSITION,
+			CONF_ONLINE_CHECK_MODE);
+
 		connman_settings.enable_online_to_ready_transition = boolean;
 	}
 
-- 
2.42.0

Re: [PATCH v3 00/60] Complete 'continuous' Online Check Mode Implementation.

[Marcel Holtmann]

Hi Grant,

> This completes the implementation of 'EnableOnlineToReadyTransition'
> begun by Emmanuel Vautrin with commit f0bd0e8fe578 ("service: Add
> online to ready transition feature").
> 
> When asserted along with 'EnableOnlineCheck',
> 'EnableOnlineToReadyTransition' effects what can be thought of as a
> "continuous" online check mode, which is different from the "one-shot"
> online check mode when 'EnableOnlineToReadyTransition' is not asserted
> but 'EnableOnlineCheck' is. Effectively, these two Booleans encode
> three online check modes:
> 
>    1. None (!EnableOnlineCheck)
>    2. One-shot (EnableOnlineCheck && !EnableOnlineToReadyTransition)
>    3. Continuous (EnableOnlineCheck && EnableOnlineToReadyTransition)
> 
> With this change, these three modes are formalized and introduced as a
> new 'OnlineCheckMode' configuration setting which is intended to
> eventually deprecate 'EnableOnlineCheck' (long-term) and
> 'EnableOnlineToReadyTransition' (short-term, as it never made it
> beyond the EXPERIMENTAL designation).
> 
> In "none" mode, as has been the case historically, there are no
> "online" HTTP-based Internet reachability checks. Any connected
> service and the manager state will terminate at the "ready" state and
> will not progress to "online".
> 
> In "one-shot" mode, as has been the case historically, there is a
> single, one-shot "online" HTTP-based Internet reachability check for
> the default service. When the check succeeds, the associated service
> and the manager state will terminate at the "online" state. When the
> check fails, subsequent checks will be rescheduled according to
> "OnlineCheckIntervalStyle", "OnlineCheckInitialInterval", and
> "OnlineCheckMaxInterval" and will continue indefinitely until one
> succeeds or until the service is disconnected.
> 
> In "continuous" mode, which is finalized with this change, there are
> ongoing "online" HTTP-based Internet reachability check for the
> default service. As with "one-shot" mode, when the first check
> succeeds, the associated service and the manager state will terminate
> at the "online" state. Thereafter, subsequent checks will be scheduled
> according to "OnlineCheckIntervalStyle" and
> "OnlineCheckMaxInterval". When the check fails, subsequent checks will
> be rescheduled according to "OnlineCheckIntervalStyle",
> "OnlineCheckInitialInterval", and "OnlineCheckMaxInterval". This is
> largely unchanged. However, what is new with this change is that when
> and if "OnlineCheckFailuresThreshold" is met, the service and manager
> state will be demoted to "ready" and the service will have its "Error"
> property set to "online-check-failed" while subsequent checks will
> continue. In the interim, if available, another service may be
> promoted to the default service and online checks will be initiated
> for it. When and if, for the demoted service,
> "OnlineCheckSuccessesThreshold" is met, the service "Error" property
> will be cleared and the service state promoted to "online",
> potentially causing it to become the default service again.
> 
> Grant Erickson (60):
>  service: Change return type of '__connman_service_wispr_start'.
>  service: Sanity check the 'type' param in
>    '__connman_service_wispr_start'.
>  service: Rename 'online_check_enabled_check'.
>  service: Change return type of 'start_online_check'.
>  service: Change return type of 'start_wispr_if_connected'.
>  service: Introduce and use online check 'active' state.
>  service: Document 'online_check_state' 'active' member.
>  service: Document 'online_check_is_active'.
>  service: Document 'online_check_active_set_value'.
>  service: Document 'online_check_active_{clear,set}'.
>  service: Rename 'start_wispr_if_connected'.
>  service: Refactor 'start_online_check_if_connected'.
>  service: Document 'start_online_check_if_connected_with_type'.
>  service: Add an explicit conditional for IPv6 in
>    'reschedule_online_check'.
>  session: Add service identifier to 'DBG' in four functions.
>  service: Add online check failure service error enumeration and
>    string.
>  doc/service: Document online check failure service error.
>  service: Introduce and use online check 'failures/successes' state.
>  service: Document 'online_check_state' 'failures/successes' members.
>  main: Introduce the 'OnlineCheck{Failures,Successes}Threshold'
>    settings.
>  doc: Document the 'OnlineCheck{Failures,Successes}Threshold' settings.
>  service: Add per-service online check failure threshold state.
>  service: Document 'connman_service'
>    'online_check_failures_met_threshold' member.
>  service: Document 'connman_service' 'online_check_state_ipv{4,6}'
>    members.
>  service: Move 'set_error' forward declaration earlier in the source.
>  service: Capture IP configuration state by value rather than by
>    pointer.
>  service: Complete 'continuous' online check mode implementation.
>  service: Document 'online_check_failures_threshold_was_met'.
>  service: Document 'online_check_failures_threshold_was_met_set_value'.
>  service: Document
>    'online_check_failures_threshold_was_met_{set,clear}'.
>  service: Document 'online_check_counter_reset'.
>  service: Document 'online_check_{failures,successes}_reset'.
>  service: Document 'online_check_state_reset'.
>  service: Document 'online_check_counters_log'.
>  service: Document 'online_check_counter_threshold_is_met'.
>  service: Document 'is_online_check_failure_threshold_met_predicate'.
>  service: Document 'online_check_failures_threshold_is_met'.
>  service: Document 'is_online_check_success_threshold_met_predicate'.
>  service: Document 'online_check_successes_threshold_is_met'.
>  service: Document 'online_check_counter_increment_and_log'.
>  service: Document 'online_check_log_success'.
>  service: Document
>    'continuous_online_check_log_{counter,failures,successes}_threshold_met'.
>  service: Document 'handle_{continuous,oneshot}_online_check_success'.
>  service: Document 'online_check_log_failure'.
>  service: Document 'handle_{continuous,oneshot}_online_check_failure'.
>  service: Add @sa documentation references.
>  service: Document the internals of 'service_compare'.
>  service: Document interval assignment in
>    '__connman_service_wispr_start'.
>  service: Document 'downgrade_connected_services'.
>  service: Document 'service_schedule_changed'.
>  service: Document 'service_route_changed'.
>  service: Add function parameter to 'default_changed' and
>    'service_list_sort'.
>  service: Add @sa documentation reference to 'current_default'.
>  service: Add 'DBG' to 'downgrade_connected_services'.
>  main/service: Introduce 'OnlineCheckMode' setting.
>  service: Leverage 'OnlineCheckMode' setting.
>  doc: Document the 'OnlineCheckMode' setting.
>  main: Add comments tags for online check-related settings.
>  main: Consolidate logging of online check-related settings.
>  main: Add deprecation warnings for 'EnableOnlineCheck' and
>    'EnableOnlineToReadyTransition'.
> 
> README                |   91 +-
> doc/connman.conf.5.in |   85 +-
> doc/service-api.txt   |    3 +-
> include/service.h     |   26 +-
> src/connman.h         |   18 +-
> src/main.c            |  177 +++-
> src/main.conf         |  122 ++-
> src/service.c         | 2133 ++++++++++++++++++++++++++++++++++++-----
> 8 files changed, 2356 insertions(+), 299 deletions(-)

all SIXTY (60 ;) patches have been applied.

Regards

Marcel