From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CDB2C41604 for ; Sat, 3 Oct 2020 20:01:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1C012206DD for ; Sat, 3 Oct 2020 20:01:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BlJVAInN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C012206DD Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 411B46B005D; Sat, 3 Oct 2020 16:01:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 39CD18E0001; Sat, 3 Oct 2020 16:01:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23C2D6B0068; Sat, 3 Oct 2020 16:01:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id D7ED56B005D for ; Sat, 3 Oct 2020 16:01:42 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 757CA8249980 for ; Sat, 3 Oct 2020 20:01:42 +0000 (UTC) X-FDA: 77331684444.18.pull43_260c831271af Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 5B080100ED9CD for ; Sat, 3 Oct 2020 20:01:42 +0000 (UTC) X-HE-Tag: pull43_260c831271af X-Filterd-Recvd-Size: 22220 Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Sat, 3 Oct 2020 20:01:41 +0000 (UTC) Received: by mail-pg1-f194.google.com with SMTP id h6so2499018pgk.4 for ; Sat, 03 Oct 2020 13:01:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zdkNiZAzQabxH3e8QC7RyG9Qd3opRmCFR4QjQnA9NuM=; b=BlJVAInN+cu3FjsT+JvvvQcUgueaPYXfBH2FIWyQJsFveOBTkDgSt/Kz2LvPUh72Pf oRqKYKX7Kj7PnYIskFyALzAAD9y+Fsm+fSnc051RHOk4LV8fXJiZ9Gd/nIuox33pgOOM 7a3WLXw8H4x70V29IRr+vJLLlufEz3Zy5mE6kujVTQgX++jl2LZifXXAROJLIGrUfrlO XV1g1bxFLznLRJ9UlmDVZSs/mK2TnIWlkdgpzqxg3hP2aLuXxJLNvWjeTDSEIla5f0+l 58rcZ8kQ8gjKvdKCxyiL5PxjUiI+vlssGJMY5gwKLsCPPef+HrKIokzH89goWNsTwBkY HCxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zdkNiZAzQabxH3e8QC7RyG9Qd3opRmCFR4QjQnA9NuM=; b=EfNlPv8kMui5mWFpdIyU5hpQYqRtNL0vlYY3OLc5FVFx6ppCSRgiL1fsBJAL9MX6vw b+JQHhgfsyb74FaiIg854czsnnU9TN3wiR6GkL+7mdxBEfjlb2mI7l6+v0aW0ng4UnrH G2Rb4M0CtNttB+KBno8OjYmqmcOfpF/8k0a4SNisXyQb8irZ44csiWYA7VgRPnDvrZ3I c0i4ZjbefBRepXt+NHBFOx0Lj7OuTXdLCJudnNvxwIbvoG49W4q8tuAoDqKDYC2U7/Te whCgHiDxVw2ELeomF7JbjrNJLozxy+wW4UT6pRN8dvWYl2keN/zm4EnTVLxJ+sEVd6up vLHg== X-Gm-Message-State: AOAM530H6ldioytH4EGFl4w4gCP1OQYYmD8YU8VetsHLfJx4mDctR8Dl 2djOcaKnNyo4xaE0lPcCxPZiPKtA2XXTauU83LASKw== X-Google-Smtp-Source: ABdhPJxME1FScZvZeBAIdEUVSyUsbP0PfJEgeVaiR0x9jx4bxGH8CSJEbSSpyIMm6lvSm70TXrFtay+n8/vSvl87fs4= X-Received: by 2002:a05:6a00:d2:b029:152:5ebd:426 with SMTP id e18-20020a056a0000d2b02901525ebd0426mr1285968pfj.5.1601755300128; Sat, 03 Oct 2020 13:01:40 -0700 (PDT) MIME-Version: 1.0 References: <20201002162101.665549-1-kaleshsingh@google.com> <20201002162101.665549-2-kaleshsingh@google.com> <39b9988b-e196-926b-8d1f-dd474fb53f89@nvidia.com> In-Reply-To: <39b9988b-e196-926b-8d1f-dd474fb53f89@nvidia.com> From: Kalesh Singh Date: Sat, 3 Oct 2020 16:01:29 -0400 Message-ID: Subject: Re: [PATCH v2 1/6] kselftests: vm: Add mremap tests To: John Hubbard Cc: Suren Baghdasaryan , Minchan Kim , Joel Fernandes , Lokesh Gidra , "Cc: Android Kernel" , Catalin Marinas , Will Deacon , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "the arch/x86 maintainers" , "H. Peter Anvin" , Andrew Morton , Shuah Khan , Kees Cook , "Aneesh Kumar K.V" , Peter Zijlstra , Arnd Bergmann , Masahiro Yamada , Sami Tolvanen , Frederic Weisbecker , Krzysztof Kozlowski , Hassan Naveed , Christian Brauner , Mark Rutland , Mark Brown , Mike Rapoport , Gavin Shan , Chris von Recklinghausen , Jia He , Zhenyu Ye , Thiago Jung Bauermann , Greg Kroah-Hartman , "Kirill A. Shutemov" , Mina Almasry , Ram Pai , Ralph Campbell , Sandipan Das , Dave Hansen , Colin Ian King , Masami Hiramatsu , Kamalesh Babulal , Ira Weiny , SeongJae Park , Brian Geffon , LKML , "moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)" , "open list:MEMORY MANAGEMENT" , "open list:KERNEL SELFTEST FRAMEWORK" Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Oct 3, 2020 at 3:13 AM John Hubbard wrote: > > On 10/2/20 9:20 AM, Kalesh Singh wrote: > > Test mremap on regions of various sizes and alignments and validate > > data after remapping. Also provide total time for remapping > > the region which is useful for performance comparison of the mremap > > optimizations that move pages at the PMD/PUD levels if HAVE_MOVE_PMD > > and/or HAVE_MOVE_PUD are enabled. > > > > Signed-off-by: Kalesh Singh > > --- > > Changes in v2: > > - Reduce test time by only validating a certain threshold of the > > remapped region (4MB by default). The -t flag can be used to > > set a custom threshold in MB or no threshold by passing 0. (-t0). > > mremap time is not provided in stdout for only partially validated > > regions. This time is only applicable for comparison if the entire > > mapped region was faulted in. > > - Use a random pattern for validating the remapped region. The -p > > flag can be used to run the tests with a specified seed for the > > random pattern. > > - Print test configs (threshold_mb and pattern_seed) to stdout. > > - Remove MAKE_SIMPLE_TEST macro. > > - Define named flags instead of 0 / 1. > > - Add comments for destination address' align_mask and offset. > > Thanks for making those changes. This all looks much nicer, both in the > code and on the screen. Now it's easy to see which code generates which > output, and the tests are readable. The new comments are also helpful. > > I ran it locally and it came in at 0.6 sec, so that looks good. > > A remaining nit: there are lots of > 80 col lines here. Only those lines > that contain printf output strings really need to be extra-long. The > others can all be easily made to fit. Thanks for the review John. I can fix the lines that don't need to be >80 col. FWIW checkpatch now allows 100 col lines: https://lore.kernel.org/r/9c360bfa43580ce7726dd3d9d247f1216a690ef0.camel@perches.com > > ... > > + > > +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) > > +#define MIN(X, Y) ((X) < (Y) ? (X) : (Y)) > > > Not really a comment on this patch, but: at least two other test suites > in selftests/ have had to recreate MIN() or min(), and probably a few > other common things as well. It would be nice to gather up some of these > common things into one of the shared header files in selftests/. > > I'm certainly not saying that this patchset has that responsibility, > though. Just pointing it out in case someone has a moment to clean up a > few here and there. Thanks for pointing this out. I'll leave it as a task for a separate patch if time permits or no one gets around to it before :-) > > I'll assume that any changes to make things fit within 80 cols are > trivial, so you can add: > > Reviewed-by: John Hubbard > > thanks, > -- > John Hubbard > NVIDIA > > > + > > +struct config { > > + unsigned long long src_alignment; > > + unsigned long long dest_alignment; > > + unsigned long long region_size; > > + int overlapping; > > +}; > > + > > +struct test { > > + const char *name; > > + struct config config; > > + int expect_failure; > > +}; > > + > > +enum { > > + _1KB = 1ULL << 10, /* 1KB -> not page aligned */ > > + _4KB = 4ULL << 10, > > + _8KB = 8ULL << 10, > > + _1MB = 1ULL << 20, > > + _2MB = 2ULL << 20, > > + _4MB = 4ULL << 20, > > + _1GB = 1ULL << 30, > > + _2GB = 2ULL << 30, > > + PTE = _4KB, > > + PMD = _2MB, > > + PUD = _1GB, > > +}; > > + > > +#define MAKE_TEST(source_align, destination_align, size, \ > > + overlaps, should_fail, test_name) \ > > +{ \ > > + .name = test_name, \ > > + .config = { \ > > + .src_alignment = source_align, \ > > + .dest_alignment = destination_align, \ > > + .region_size = size, \ > > + .overlapping = overlaps, \ > > + }, \ > > + .expect_failure = should_fail \ > > +} > > + > > +/* > > + * Returns the start address of the mapping on success, else returns > > + * NULL on failure. > > + */ > > +static void *get_source_mapping(struct config c) > > +{ > > + unsigned long long addr = 0ULL; > > + void *src_addr = NULL; > > +retry: > > + addr += c.src_alignment; > > + src_addr = mmap((void *) addr, c.region_size, PROT_READ | PROT_WRITE, > > + MAP_FIXED | MAP_ANONYMOUS | MAP_SHARED, -1, 0); > > + if (src_addr == MAP_FAILED) { > > + if (errno == EPERM) > > + goto retry; > > + goto error; > > + } > > + /* > > + * Check that the address is aligned to the specified alignment. Addresses > > + * which have alignments that are multiples of that specified are not considered > > + * valid. For instance, 1GB address is 2MB-aligned, however it will not be > > + * considered valid for a requested alignment of 2MB. This is done to > > + * reduce coincidental alignment in the tests. > > + */ > > + if (((unsigned long long) src_addr & (c.src_alignment - 1)) || > > + !((unsigned long long) src_addr & c.src_alignment)) > > + goto retry; > > + > > + if (!src_addr) > > + goto error; > > + > > + return src_addr; > > +error: > > + ksft_print_msg("Failed to map source region: %s\n", > > + strerror(errno)); > > + return NULL; > > +} > > + > > +/* Returns the time taken for the remap on success else returns -1. */ > > +static long long remap_region(struct config c, unsigned int threshold_mb, > > + char pattern_seed) > > +{ > > + void *addr, *src_addr, *dest_addr; > > + unsigned long long i; > > + struct timespec t_start = {0, 0}, t_end = {0, 0}; > > + long long start_ns, end_ns, align_mask, ret, offset; > > + unsigned long long threshold; > > + > > + if (threshold_mb == VALIDATION_NO_THRESHOLD) > > + threshold = c.region_size; > > + else > > + threshold = MIN(threshold_mb * _1MB, c.region_size); > > + > > + src_addr = get_source_mapping(c); > > + if (!src_addr) { > > + ret = -1; > > + goto out; > > + } > > + > > + /* Set byte pattern */ > > + srand(pattern_seed); > > + for (i = 0; i < threshold; i++) > > + memset((char *) src_addr + i, (char) rand(), 1); > > + > > + /* Mask to zero out lower bits of address for alignment */ > > + align_mask = ~(c.dest_alignment - 1); > > + /* Offset of destination address from the end of the source region */ > > + offset = (c.overlapping) ? -c.dest_alignment : c.dest_alignment; > > + addr = (void *) (((unsigned long long) src_addr + c.region_size + offset) > > + & align_mask); > > + > > + /* See comment in get_source_mapping() */ > > + if (!((unsigned long long) addr & c.dest_alignment)) > > + addr = (void *) ((unsigned long long) addr | c.dest_alignment); > > + > > + clock_gettime(CLOCK_MONOTONIC, &t_start); > > + dest_addr = mremap(src_addr, c.region_size, c.region_size, > > + MREMAP_MAYMOVE|MREMAP_FIXED, (char *) addr); > > + clock_gettime(CLOCK_MONOTONIC, &t_end); > > + > > + if (dest_addr == MAP_FAILED) { > > + ksft_print_msg("mremap failed: %s\n", strerror(errno)); > > + ret = -1; > > + goto clean_up_src; > > + } > > + > > + /* Verify byte pattern after remapping */ > > + srand(pattern_seed); > > + for (i = 0; i < threshold; i++) { > > + char c = (char) rand(); > > + > > + if (((char *) dest_addr)[i] != c) { > > + ksft_print_msg("Data after remap doesn't match at offset %d\n", > > + i); > > + ksft_print_msg("Expected: %#x\t Got: %#x\n", c & 0xff, > > + ((char *) dest_addr)[i] & 0xff); > > + ret = -1; > > + goto clean_up_dest; > > + } > > + } > > + > > + start_ns = t_start.tv_sec * NS_PER_SEC + t_start.tv_nsec; > > + end_ns = t_end.tv_sec * NS_PER_SEC + t_end.tv_nsec; > > + ret = end_ns - start_ns; > > + > > +/* > > + * Since the destination address is specified using MREMAP_FIXED, subsequent mremap will unmap any > > + * previous mapping at the address range specified by dest_addr and region_size. This significantly > > + * affects the remap time of subsequent tests. So we clean up mappings after each test. > > + */ > > +clean_up_dest: > > + munmap(dest_addr, c.region_size); > > +clean_up_src: > > + munmap(src_addr, c.region_size); > > +out: > > + return ret; > > +} > > + > > +static void run_mremap_test_case(struct test test_case, int *failures, unsigned int threshold_mb, > > + unsigned int pattern_seed) > > +{ > > + long long remap_time = remap_region(test_case.config, threshold_mb, pattern_seed); > > + > > + if (remap_time < 0) { > > + if (test_case.expect_failure) > > + ksft_test_result_pass("%s\n\tExpected mremap failure\n", test_case.name); > > + else { > > + ksft_test_result_fail("%s\n", test_case.name); > > + *failures += 1; > > + } > > + } else { > > + /* Comparing mremap time is only applicable if entire region was faulted in */ > > + if (threshold_mb == VALIDATION_NO_THRESHOLD || > > + test_case.config.region_size <= threshold_mb * _1MB) > > + ksft_test_result_pass("%s\n\tmremap time: %12lldns\n", > > + test_case.name, remap_time); > > + else > > + ksft_test_result_pass("%s\n", test_case.name); > > + } > > +} > > + > > +static void usage(const char *cmd) > > +{ > > + fprintf(stderr, > > + "Usage: %s [[-t ] [-p ]]\n" > > + "-t\t only validate threshold_mb of the remapped region\n" > > + " \t if 0 is supplied no threshold is used; all tests\n" > > + " \t are run and remapped regions validated fully.\n" > > + " \t The default threshold used is 4MB.\n" > > + "-p\t provide a seed to generate the random pattern for\n" > > + " \t validating the remapped region.\n", cmd); > > +} > > + > > +static int parse_args(int argc, char **argv, unsigned int *threshold_mb, > > + unsigned int *pattern_seed) > > +{ > > + const char *optstr = "t:p:"; > > + int opt; > > + > > + while ((opt = getopt(argc, argv, optstr)) != -1) { > > + switch (opt) { > > + case 't': > > + *threshold_mb = atoi(optarg); > > + break; > > + case 'p': > > + *pattern_seed = atoi(optarg); > > + break; > > + default: > > + usage(argv[0]); > > + return -1; > > + } > > + } > > + > > + if (optind < argc) { > > + usage(argv[0]); > > + return -1; > > + } > > + > > + return 0; > > +} > > + > > +int main(int argc, char **argv) > > +{ > > + int failures = 0; > > + int i, run_perf_tests; > > + unsigned int threshold_mb = VALIDATION_DEFAULT_THRESHOLD; > > + unsigned int pattern_seed; > > + time_t t; > > + > > + pattern_seed = (unsigned int) time(&t); > > + > > + if (parse_args(argc, argv, &threshold_mb, &pattern_seed) < 0) > > + exit(EXIT_FAILURE); > > + > > + ksft_print_msg("Test configs:\n\tthreshold_mb=%u\n\tpattern_seed=%u\n\n", > > + threshold_mb, pattern_seed); > > + > > + struct test test_cases[] = { > > + /* Expected mremap failures */ > > + MAKE_TEST(_4KB, _4KB, _4KB, OVERLAPPING, EXPECT_FAILURE, > > + "mremap - Source and Destination Regions Overlapping"), > > + MAKE_TEST(_4KB, _1KB, _4KB, NON_OVERLAPPING, EXPECT_FAILURE, > > + "mremap - Destination Address Misaligned (1KB-aligned)"), > > + MAKE_TEST(_1KB, _4KB, _4KB, NON_OVERLAPPING, EXPECT_FAILURE, > > + "mremap - Source Address Misaligned (1KB-aligned)"), > > + > > + /* Src addr PTE aligned */ > > + MAKE_TEST(PTE, PTE, _8KB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "8KB mremap - Source PTE-aligned, Destination PTE-aligned"), > > + > > + /* Src addr 1MB aligned */ > > + MAKE_TEST(_1MB, PTE, _2MB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "2MB mremap - Source 1MB-aligned, Destination PTE-aligned"), > > + MAKE_TEST(_1MB, _1MB, _2MB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "2MB mremap - Source 1MB-aligned, Destination 1MB-aligned"), > > + > > + /* Src addr PMD aligned */ > > + MAKE_TEST(PMD, PTE, _4MB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "4MB mremap - Source PMD-aligned, Destination PTE-aligned"), > > + MAKE_TEST(PMD, _1MB, _4MB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "4MB mremap - Source PMD-aligned, Destination 1MB-aligned"), > > + MAKE_TEST(PMD, PMD, _4MB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "4MB mremap - Source PMD-aligned, Destination PMD-aligned"), > > + > > + /* Src addr PUD aligned */ > > + MAKE_TEST(PUD, PTE, _2GB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "2GB mremap - Source PUD-aligned, Destination PTE-aligned"), > > + MAKE_TEST(PUD, _1MB, _2GB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "2GB mremap - Source PUD-aligned, Destination 1MB-aligned"), > > + MAKE_TEST(PUD, PMD, _2GB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "2GB mremap - Source PUD-aligned, Destination PMD-aligned"), > > + MAKE_TEST(PUD, PUD, _2GB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "2GB mremap - Source PUD-aligned, Destination PUD-aligned"), > > + }; > > + > > + struct test perf_test_cases[] = { > > + /* mremap 1GB region - Page table level aligned time comparison */ > > + MAKE_TEST(PTE, PTE, _1GB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "1GB mremap - Source PTE-aligned, Destination PTE-aligned"), > > + MAKE_TEST(PMD, PMD, _1GB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "1GB mremap - Source PMD-aligned, Destination PMD-aligned"), > > + MAKE_TEST(PUD, PUD, _1GB, NON_OVERLAPPING, EXPECT_SUCCESS, > > + "1GB mremap - Source PUD-aligned, Destination PUD-aligned"), > > + }; > > + > > + run_perf_tests = (threshold_mb == VALIDATION_NO_THRESHOLD) || > > + (threshold_mb * _1MB >= _1GB); > > + > > + ksft_set_plan(ARRAY_SIZE(test_cases) + (run_perf_tests ? > > + ARRAY_SIZE(perf_test_cases) : 0)); > > + > > + for (i = 0; i < ARRAY_SIZE(test_cases); i++) > > + run_mremap_test_case(test_cases[i], &failures, threshold_mb, pattern_seed); > > + > > + if (run_perf_tests) { > > + ksft_print_msg("\n%s\n", > > + "mremap HAVE_MOVE_PMD/PUD optimization time comparison for 1GB region:"); > > + for (i = 0; i < ARRAY_SIZE(perf_test_cases); i++) > > + run_mremap_test_case(perf_test_cases[i], &failures, threshold_mb, > > + pattern_seed); > > + } > > + > > + if (failures > 0) > > + ksft_exit_fail(); > > + else > > + ksft_exit_pass(); > > +} > > diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests > > index a3f4f30f0a2e..d578ad831813 100755 > > --- a/tools/testing/selftests/vm/run_vmtests > > +++ b/tools/testing/selftests/vm/run_vmtests > > @@ -241,6 +241,17 @@ else > > echo "[PASS]" > > fi > > > > +echo "-------------------" > > +echo "running mremap_test" > > +echo "-------------------" > > +./mremap_test > > +if [ $? -ne 0 ]; then > > + echo "[FAIL]" > > + exitcode=1 > > +else > > + echo "[PASS]" > > +fi > > + > > echo "-----------------" > > echo "running thuge-gen" > > echo "-----------------" > > > > -- > To unsubscribe from this group and stop receiving emails from it, send an email to kernel-team+unsubscribe@android.com. >