qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
@ 2020-05-22 18:43 Aleksandar Markovic
  2020-05-22 19:18 ` Alex Bennée
                   ` (7 more replies)
  0 siblings, 8 replies; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-22 18:43 UTC (permalink / raw)
  To: qemu-devel

Public bug reported:

This issue is observer with QEMU ToT, checked out around May 15th (but I
believe it is present in current master too), and wasn't present in QEMU
v5.0.0.

I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

Arm cross-compiler is a standard cross-compiler that comes with Debian-
based distributions, and gcc version is:

$ arm-linux-gnueabi-gcc --version
arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

Compile this program with cross compiler:

$ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o toupper_string-
arm

Emulation with QEMU v5.0.0 is correct, and gives expected output:

$ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
CONTROL RESULT: (toupper_string)
 nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
 NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

While, in case of QEMU master it fails:

$ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
Aborted

There are many other programs that exibit the same behavior. The failure
is arm-sprecific.


-----------------------------------------------------

source code: (let's call this file toupper_string.c) (similar file is
also in attachment)


#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <unistd.h>


#define MAX_STRING_LENGHT              15
#define NUMBER_OF_RANDOM_STRINGS       100
#define DEFAULT_NUMBER_OF_REPETITIONS  30000
#define MAX_NUMBER_OF_REPETITIONS      1000000000
#define NUMBER_OF_CONTROL_PRINT_ITEMS  5

/* Structure for keeping an array of strings */
struct StringStruct {
    char chars[MAX_STRING_LENGHT + 1];
};

/**
 * Sets characters of the given string to random small letters a-z.
 * @param s String to get random characters.
 * @len Length of the input string.
 */
static void gen_random_string(char *chars, const int len)
{
    static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

    for (size_t i = 0; i < len; i++) {
        chars[i] = letters[rand() % (sizeof(letters) - 1)];
    }
    chars[len] = 0;
}

void main (int argc, char* argv[])
{
    struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
    struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
    int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
    int32_t option;

    /* Parse command line options */
    while ((option = getopt(argc, argv, "n:")) != -1) {
        if (option == 'n') {
            int32_t user_number_of_repetitions = atoi(optarg);
            /* Check if the value is a negative number */
            if (user_number_of_repetitions < 1) {
                fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                "negative number.\n");
                exit(EXIT_FAILURE);
            }
            /* Check if the value is a string or zero */
            if (user_number_of_repetitions == 0) {
                fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                exit(EXIT_FAILURE);
            }
            /* Check if the value is too large */
            if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                exit(EXIT_FAILURE);
            }
            number_of_repetitions = user_number_of_repetitions;
        } else {
            exit(EXIT_FAILURE);
        }
    }

    /* Create an array of strings with random content */
    srand(1);
    for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
        gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
    }

    /* Perform uppercasing of a set of random strings multiple times */
    for (size_t j = 0; j < number_of_repetitions; j++) {
        /* Copy initial set of random strings to the set to be uppercased */
        memcpy(strings_to_be_uppercased, random_strings,
               NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
        /* Do actual changing case to uppercase */
        for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
            int k = 0;
  
            while (strings_to_be_uppercased[i].chars[k]) { 
                char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                memcpy((void *)strings_to_be_uppercased[i].chars + k,
                       &ch, 1);
                k++; 
            } 
        }
    }

    /* Control printing */
    printf("CONTROL RESULT: (toupper_string)\n");
    for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
        printf(" %s", random_strings[i].chars);
    }
    printf("\n");
    for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
        printf(" %s", strings_to_be_uppercased[i].chars);
    }
    printf("\n");
}

** Affects: qemu
     Importance: Undecided
         Status: New


** Tags: arm

** Attachment added: "toupper_string.c"
   https://bugs.launchpad.net/bugs/1880225/+attachment/5375706/+files/toupper_string.c

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  New

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
@ 2020-05-22 19:18 ` Alex Bennée
  2020-05-22 19:18   ` Alex Bennée
  2020-05-22 19:27 ` [Bug 1880225] " Alex Bennée
                   ` (6 subsequent siblings)
  7 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-05-22 19:18 UTC (permalink / raw)
  To: Bug 1880225; +Cc: qemu-devel


Aleksandar Markovic <1880225@bugs.launchpad.net> writes:

> Public bug reported:
>
> This issue is observer with QEMU ToT, checked out around May 15th (but I
> believe it is present in current master too), and wasn't present in QEMU
> v5.0.0.
>
> I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.
>
> Arm cross-compiler is a standard cross-compiler that comes with Debian-
> based distributions, and gcc version is:
>
> $ arm-linux-gnueabi-gcc --version
> arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0
>
> Compile this program with cross compiler:
>
> $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o toupper_string-
> arm
>
> Emulation with QEMU v5.0.0 is correct, and gives expected output:
>
> $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
> CONTROL RESULT: (toupper_string)
>  nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
>  NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ
>
> While, in case of QEMU master it fails:
>
> $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
> qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
> Aborted
<snip>

Works for me in our TCG tests on master:

20:15:43 [alex@zen:~/l/q/b/user.static] review/aarch64-vms-v7|… + ./arm-linux-user/qemu-arm ./tests/tcg/arm-linux-user/toupper
CONTROL RESULT: (toupper_string)
 nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
 NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

I have submitted a fix to the list that affected programs that couldn't
see /proc/self/maps but I guess that isn't the case here.

-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 19:18 ` Alex Bennée
@ 2020-05-22 19:18   ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-22 19:18 UTC (permalink / raw)
  To: qemu-devel

Aleksandar Markovic <1880225@bugs.launchpad.net> writes:

> Public bug reported:
>
> This issue is observer with QEMU ToT, checked out around May 15th (but I
> believe it is present in current master too), and wasn't present in QEMU
> v5.0.0.
>
> I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.
>
> Arm cross-compiler is a standard cross-compiler that comes with Debian-
> based distributions, and gcc version is:
>
> $ arm-linux-gnueabi-gcc --version
> arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0
>
> Compile this program with cross compiler:
>
> $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o toupper_string-
> arm
>
> Emulation with QEMU v5.0.0 is correct, and gives expected output:
>
> $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
> CONTROL RESULT: (toupper_string)
>  nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
>  NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ
>
> While, in case of QEMU master it fails:
>
> $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
> qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
> Aborted
<snip>

Works for me in our TCG tests on master:

20:15:43 [alex@zen:~/l/q/b/user.static] review/aarch64-vms-v7|… + ./arm-linux-user/qemu-arm ./tests/tcg/arm-linux-user/toupper
CONTROL RESULT: (toupper_string)
 nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
 NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

I have submitted a fix to the list that affected programs that couldn't
see /proc/self/maps but I guess that isn't the case here.

-- 
Alex Bennée

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  New

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
  2020-05-22 19:18 ` Alex Bennée
@ 2020-05-22 19:27 ` Alex Bennée
  2020-05-23  1:07 ` Aleksandar Markovic
                   ` (5 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-22 19:27 UTC (permalink / raw)
  To: qemu-devel

** Tags added: testcase

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  New

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
  2020-05-22 19:18 ` Alex Bennée
  2020-05-22 19:27 ` [Bug 1880225] " Alex Bennée
@ 2020-05-23  1:07 ` Aleksandar Markovic
  2020-05-23  1:14 ` Aleksandar Markovic
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-23  1:07 UTC (permalink / raw)
  To: qemu-devel

Using bisection, it can be deduced that this behavior appears to be
caused by this commit:


commit ee94743034bfb443cf246eda4971bdc15d8ee066 (HEAD)
Author: Alex Bennée <alex.bennee@linaro.org>
Date:   Wed May 13 18:51:28 2020 +0100

    linux-user: completely re-write init_guest_space
    
    First we ensure all guest space initialisation logic comes through
    probe_guest_base once we understand the nature of the binary we are
    loading. The convoluted init_guest_space routine is removed and
    replaced with a number of pgb_* helpers which are called depending on
    what requirements we have when loading the binary.
    
    We first try to do what is requested by the host. Failing that we try
    and satisfy the guest requested base address. If all those options
    fail we fall back to finding a space in the memory map using our
    recently written read_self_maps() helper.
    
    There are some additional complications we try and take into account
    when looking for holes in the address space. We try not to go directly
    after the system brk() space so there is space for a little growth. We
    also don't want to have to use negative offsets which would result in
    slightly less efficient code on x86 when it's unable to use the
    segment offset register.
    
    Less mind-binding gotos and hopefully clearer logic throughout.
    
    Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
    Acked-by: Laurent Vivier <laurent@vivier.eu>
    
    Message-Id: <20200513175134.19619-5-alex.bennee@linaro.org>

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  New

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
                   ` (2 preceding siblings ...)
  2020-05-23  1:07 ` Aleksandar Markovic
@ 2020-05-23  1:14 ` Aleksandar Markovic
  2020-05-23  7:40   ` Alex Bennée
  2020-05-23  1:31 ` Aleksandar Markovic
                   ` (3 subsequent siblings)
  7 siblings, 1 reply; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-23  1:14 UTC (permalink / raw)
  To: qemu-devel

I just want to stress once again that the test was performed on a 32-bit
Intel host.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  New

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
                   ` (3 preceding siblings ...)
  2020-05-23  1:14 ` Aleksandar Markovic
@ 2020-05-23  1:31 ` Aleksandar Markovic
  2020-05-23  7:50 ` Alex Bennée
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-23  1:31 UTC (permalink / raw)
  To: qemu-devel

It appear that there is no problem on Intel 64-bit hosts.

Perhaps the problem is manifested on all 32-bit hosts. I currently don't
have access to any other 320bit host due to remote work.

The arm is the only target were I noticed this happens. I checked hppa,
mips, mipsel, m68k, ppc, and sh4, they ae all fine,  with the same
program/example, on the same 32-bit Intel host. I did not checked other
target except those I just mentioned.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  New

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-23  1:14 ` Aleksandar Markovic
@ 2020-05-23  7:40   ` Alex Bennée
  2020-05-23  7:40     ` Alex Bennée
  0 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-05-23  7:40 UTC (permalink / raw)
  To: Bug 1880225; +Cc: qemu-devel


Aleksandar Markovic <1880225@bugs.launchpad.net> writes:

> I just want to stress once again that the test was performed on a 32-bit
> Intel host.

Ahh - OK that makes sense. I'll see if I can replicate.


-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-23  7:40   ` Alex Bennée
@ 2020-05-23  7:40     ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-23  7:40 UTC (permalink / raw)
  To: qemu-devel

Aleksandar Markovic <1880225@bugs.launchpad.net> writes:

> I just want to stress once again that the test was performed on a 32-bit
> Intel host.

Ahh - OK that makes sense. I'll see if I can replicate.


-- 
Alex Bennée

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  New

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
                   ` (4 preceding siblings ...)
  2020-05-23  1:31 ` Aleksandar Markovic
@ 2020-05-23  7:50 ` Alex Bennée
  2020-05-23  7:52 ` Aleksandar Markovic
  2020-08-20 15:08 ` Thomas Huth
  7 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-23  7:50 UTC (permalink / raw)
  To: qemu-devel

** Changed in: qemu
       Status: New => Confirmed

** Changed in: qemu
     Assignee: (unassigned) => Alex Bennée (ajbennee)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  Confirmed

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
                   ` (5 preceding siblings ...)
  2020-05-23  7:50 ` Alex Bennée
@ 2020-05-23  7:52 ` Aleksandar Markovic
  2020-05-23 10:14   ` Alex Bennée
  2020-08-20 15:08 ` Thomas Huth
  7 siblings, 1 reply; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-23  7:52 UTC (permalink / raw)
  To: qemu-devel

The problem may be in int_guest_commpage() - it returns false.

>From gdb debugging session:

(gdb) p addr
$1 = (void *) 0xb7ffd000
(gdb) p want
$2 = (void *) 0xffff0000
(gdb) n
398	    if (addr != want) {
(gdb) p qemu_host_page_size
$3 = 4096
(gdb) l
393	
394	    if (addr == MAP_FAILED) {
395	        perror("Allocating guest commpage");
396	        exit(EXIT_FAILURE);
397	    }
398	    if (addr != want) {
399	        return false;
400	    }
401	
402	    /* Set kernel helper versions; rest of page is 0.  */
(gdb)

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  Confirmed

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-23  7:52 ` Aleksandar Markovic
@ 2020-05-23 10:14   ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-23 10:14 UTC (permalink / raw)
  To: qemu-devel

Aleksandar Markovic <1880225@bugs.launchpad.net> writes:

> The problem may be in int_guest_commpage() - it returns false.
>
>>From gdb debugging session:
>
> (gdb) p addr
> $1 = (void *) 0xb7ffd000
> (gdb) p want
> $2 = (void *) 0xffff0000
> (gdb) n
> 398	    if (addr != want) {
> (gdb) p qemu_host_page_size
> $3 = 4096
> (gdb) l
> 393	
> 394	    if (addr == MAP_FAILED) {
> 395	        perror("Allocating guest commpage");
> 396	        exit(EXIT_FAILURE);
> 397	    }
> 398	    if (addr != want) {
> 399	        return false;
> 400	    }
> 401	
> 402	    /* Set kernel helper versions; rest of page is 0.  */
> (gdb)

I'm not totally convinced the calculation that we do to work out the
extended size of the guest space in 32 bit:

  11:10 alex@debian-buster-i386/i686  [arm/bugs/add-mmap-fallback@github] >./arm-linux-user/qemu-arm tests/tcg/arm-linux-user/sha1
  pgb_static: loaddr: 10000
  pgb_static: loaddr: ffff0000
  pgb_find_hole: ffff0000:10809a8 (1000)
  pgb_find_hole: 0:10809a8
  init_guest_commpage: 0xffff0000 -> 0xb7f48000 (4096)
  qemu-arm: /home/alex/lsrc/qemu.git/linux-user/elfload.c:2350: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted (core dumped)

Or in fact why we don't do a MAP_FIXED ass we should have ensured we
have enough space allocated for the guest. Richard any ideas?

-- 
Alex Bennée

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  Confirmed

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [PATCH  v1 0/3] some linux-user guest_base fixes
@ 2020-05-27 10:05 Alex Bennée
  2020-05-27 10:05 ` [PATCH v1 1/3] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
                   ` (3 more replies)
  0 siblings, 4 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-27 10:05 UTC (permalink / raw)
  To: qemu-devel; +Cc: qemu-arm, Alex Bennée

Hi,

Here are a couple of patches to clean-up guest_base handling for
linux-user guests. There are two cases we hadn't accounted for/tested
enough in the original clean-up:

  - chroots without /proc access
  - 32 bit hosts and the 32 bit ARM COMMPAGE

The solution to the first problem is a simple fall-back to mmap
probing but without the ugliness of the first approach.

The second is a bit of a hack but works. Given we only ever expose one
value from the COMMPAGE it does make me wonder if we could rip out the
special casing and come up with a cleaner ARM only approach by hooking
exceptions? Anyway I've included a test case for the COMMPAGE as well.

Please review

Alex Bennée (3):
  linux-user: provide fallback pgd_find_hole for bare chroots
  linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  tests/tcg: add simple commpage test case

 linux-user/elfload.c          | 66 ++++++++++++++++++++++++++++++-----
 tests/tcg/arm/commpage.c      | 61 ++++++++++++++++++++++++++++++++
 tests/tcg/arm/Makefile.target |  2 ++
 3 files changed, 121 insertions(+), 8 deletions(-)
 create mode 100644 tests/tcg/arm/commpage.c

-- 
2.20.1



^ permalink raw reply	[flat|nested] 76+ messages in thread

* [PATCH v1 1/3] linux-user: provide fallback pgd_find_hole for bare chroots
  2020-05-27 10:05 [PATCH v1 0/3] some linux-user guest_base fixes Alex Bennée
@ 2020-05-27 10:05 ` Alex Bennée
  2020-06-02  0:37   ` Richard Henderson
  2020-05-27 10:05 ` [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-05-27 10:05 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Riku Voipio, qemu-arm, Alex Bennée, Laurent Vivier

When running QEMU out of a chroot environment we may not have access
to /proc/self/maps. As there is no other "official" way to introspect
our memory map we need to fall back to the original technique of
repeatedly trying to mmap an address range until we find one that
works.

Fortunately it's not quite as ugly as the original code given we
already re-factored the complications of dealing with the
ARM_COMMPAGE. We do make an attempt to skip over brk() which is about
the only concrete piece of information we have about the address map
at this moment.

Fixes: ee9474303
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 linux-user/elfload.c | 48 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 01a9323a637..d6027867a1a 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2099,6 +2099,50 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
     }
 }
 
+/**
+ * pgd_find_hole_fallback: potential mmap address
+ * @guest_size: size of available space
+ * @brk: location of break
+ * @align: memory alignment
+ *
+ * This is a fallback method for finding a hole in the host address
+ * space if we don't have the benefit of being able to access
+ * /proc/self/map. It can potentially take a very long time as we can
+ * only dumbly iterate up the host address space seeing if the
+ * allocation would work.
+ */
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)
+{
+    uintptr_t base;
+
+    /* Start at the bottom and work our way up */
+    base = mmap_min_addr;
+
+    while (true) {
+        uintptr_t align_start, end;
+        align_start = ROUND_UP(base, align);
+        end = align_start + guest_size;
+
+        /* if brk is anywhere in the range give ourselves some room to grow. */
+        if (align_start <= brk && brk < end) {
+            base += 16 * MiB;
+            continue;
+        } else if (align_start + guest_size < align_start) {
+            /* we have run out of space */
+            return -1;
+        } else {
+            int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE | MAP_FIXED;
+            void * mmap_start = mmap((void *) align_start, guest_size,
+                                     PROT_NONE, flags, -1, 0);
+            if (mmap_start != MAP_FAILED) {
+                munmap((void *) align_start, guest_size);
+                return (uintptr_t) mmap_start;
+            }
+            base += qemu_host_page_size;
+        }
+    }
+}
+
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
                                long align)
@@ -2114,6 +2158,10 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
     /* Read brk after we've read the maps, which will malloc. */
     brk = (uintptr_t)sbrk(0);
 
+    if (!maps) {
+        return pgd_find_hole_fallback(guest_size, brk, align);
+    }
+
     /* The first hole is before the first map entry. */
     this_start = mmap_min_addr;
 
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 10:05 [PATCH v1 0/3] some linux-user guest_base fixes Alex Bennée
  2020-05-27 10:05 ` [PATCH v1 1/3] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
@ 2020-05-27 10:05 ` Alex Bennée
  2020-05-27 10:05   ` [Bug 1880225] " Alex Bennée
                     ` (2 more replies)
  2020-05-27 10:05 ` [PATCH v1 3/3] tests/tcg: add simple commpage test case Alex Bennée
  2020-05-27 14:12 ` [PATCH v1 0/3] some linux-user guest_base fixes no-reply
  3 siblings, 3 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-27 10:05 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Bug 1880225, Riku Voipio, Richard Henderson,
	Laurent Vivier, qemu-arm, Alex Bennée

We rely on the pointer to wrap when accessing the high address of the
COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
cannot afford just to map the entire 4gb address range. The old mmap
trial and error code handled this by just checking we could map both
the guest_base and the computed COMMPAGE address.

We can't just manipulate loadaddr to get what we want so we introduce
an offset which pgb_find_hole can apply when looking for a gap for
guest_base that ensures there is space left to map the COMMPAGE
afterwards.

This is arguably a little inefficient for the one 32 bit
value (kuser_helper_version) we need to keep there given all the
actual code entries are picked up during the translation phase.

Fixes: ee94743034b
Bug: https://bugs.launchpad.net/qemu/+bug/1880225
Cc: Bug 1880225 <1880225@bugs.launchpad.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
---
 linux-user/elfload.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index d6027867a1a..31defce95b5 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
 
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
-                               long align)
+                               long align, uintptr_t offset)
 {
     GSList *maps, *iter;
     uintptr_t this_start, this_end, next_start, brk;
@@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
 
         this_end = ((MapInfo *)iter->data)->start;
         next_start = ((MapInfo *)iter->data)->end;
-        align_start = ROUND_UP(this_start, align);
+        align_start = ROUND_UP(this_start + offset, align);
 
         /* Skip holes that are too small. */
         if (align_start >= this_end) {
@@ -2221,6 +2221,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
 {
     uintptr_t loaddr = orig_loaddr;
     uintptr_t hiaddr = orig_hiaddr;
+    uintptr_t offset = 0;
     uintptr_t addr;
 
     if (hiaddr != orig_hiaddr) {
@@ -2234,18 +2235,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
     if (ARM_COMMPAGE) {
         /*
          * Extend the allocation to include the commpage.
-         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
-         * the address arithmetic will wrap around, but the difference
-         * will produce the correct allocation size.
+         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
+         * need to ensure there is space bellow the guest_base so we
+         * can map the commpage in the place needed when the address
+         * arithmetic wraps around.
          */
         if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
             hiaddr = (uintptr_t)4 << 30;
         } else {
-            loaddr = ARM_COMMPAGE & -align;
+            offset = (128 * KiB);
         }
     }
 
-    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
+    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
     if (addr == -1) {
         /*
          * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
@@ -2280,7 +2282,7 @@ static void pgb_dynamic(const char *image_name, long align)
          * just above that, and maximises the positive guest addresses.
          */
         commpage = ARM_COMMPAGE & -align;
-        addr = pgb_find_hole(commpage, -commpage, align);
+        addr = pgb_find_hole(commpage, -commpage, align, 0);
         assert(addr != -1);
         guest_base = addr;
     }
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [Bug 1880225] [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 10:05 ` [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
@ 2020-05-27 10:05   ` Alex Bennée
  2020-05-27 12:05   ` Aleksandar Markovic
  2020-06-02  0:28   ` Richard Henderson
  2 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-27 10:05 UTC (permalink / raw)
  To: qemu-devel

We rely on the pointer to wrap when accessing the high address of the
COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
cannot afford just to map the entire 4gb address range. The old mmap
trial and error code handled this by just checking we could map both
the guest_base and the computed COMMPAGE address.

We can't just manipulate loadaddr to get what we want so we introduce
an offset which pgb_find_hole can apply when looking for a gap for
guest_base that ensures there is space left to map the COMMPAGE
afterwards.

This is arguably a little inefficient for the one 32 bit
value (kuser_helper_version) we need to keep there given all the
actual code entries are picked up during the translation phase.

Fixes: ee94743034b
Bug: https://bugs.launchpad.net/qemu/+bug/1880225
Cc: Bug 1880225 <1880225@bugs.launchpad.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
---
 linux-user/elfload.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index d6027867a1a..31defce95b5 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
 
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
-                               long align)
+                               long align, uintptr_t offset)
 {
     GSList *maps, *iter;
     uintptr_t this_start, this_end, next_start, brk;
@@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
 
         this_end = ((MapInfo *)iter->data)->start;
         next_start = ((MapInfo *)iter->data)->end;
-        align_start = ROUND_UP(this_start, align);
+        align_start = ROUND_UP(this_start + offset, align);
 
         /* Skip holes that are too small. */
         if (align_start >= this_end) {
@@ -2221,6 +2221,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
 {
     uintptr_t loaddr = orig_loaddr;
     uintptr_t hiaddr = orig_hiaddr;
+    uintptr_t offset = 0;
     uintptr_t addr;
 
     if (hiaddr != orig_hiaddr) {
@@ -2234,18 +2235,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
     if (ARM_COMMPAGE) {
         /*
          * Extend the allocation to include the commpage.
-         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
-         * the address arithmetic will wrap around, but the difference
-         * will produce the correct allocation size.
+         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
+         * need to ensure there is space bellow the guest_base so we
+         * can map the commpage in the place needed when the address
+         * arithmetic wraps around.
          */
         if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
             hiaddr = (uintptr_t)4 << 30;
         } else {
-            loaddr = ARM_COMMPAGE & -align;
+            offset = (128 * KiB);
         }
     }
 
-    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
+    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
     if (addr == -1) {
         /*
          * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
@@ -2280,7 +2282,7 @@ static void pgb_dynamic(const char *image_name, long align)
          * just above that, and maximises the positive guest addresses.
          */
         commpage = ARM_COMMPAGE & -align;
-        addr = pgb_find_hole(commpage, -commpage, align);
+        addr = pgb_find_hole(commpage, -commpage, align, 0);
         assert(addr != -1);
         guest_base = addr;
     }
-- 
2.20.1


** Changed in: qemu
       Status: Confirmed => In Progress

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  In Progress

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH  v1 3/3] tests/tcg: add simple commpage test case
  2020-05-27 10:05 [PATCH v1 0/3] some linux-user guest_base fixes Alex Bennée
  2020-05-27 10:05 ` [PATCH v1 1/3] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
  2020-05-27 10:05 ` [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
@ 2020-05-27 10:05 ` Alex Bennée
  2020-06-02  0:40   ` Richard Henderson
  2020-05-27 14:12 ` [PATCH v1 0/3] some linux-user guest_base fixes no-reply
  3 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-05-27 10:05 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, qemu-arm, Alex Bennée

The COMMPAGE are a number of kernel provided user-space routines for
32 bit ARM systems. Add a basic series of smoke tests to ensure it is
working as it should.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/arm/commpage.c      | 61 +++++++++++++++++++++++++++++++++++
 tests/tcg/arm/Makefile.target |  2 ++
 2 files changed, 63 insertions(+)
 create mode 100644 tests/tcg/arm/commpage.c

diff --git a/tests/tcg/arm/commpage.c b/tests/tcg/arm/commpage.c
new file mode 100644
index 00000000000..c76e70cb8bd
--- /dev/null
+++ b/tests/tcg/arm/commpage.c
@@ -0,0 +1,61 @@
+/*
+ * Verify the COMMPAGE emulation
+ *
+ * The ARM commpage is a set of user space helper functions provided
+ * by the kernel in an effort to ease portability of user space code
+ * between different CPUs with potentially different capabilities. It
+ * is a 32 bit invention and similar to the vdso segment in many ways.
+ *
+ * The ABI is documented in the Linux kernel:
+ *     Documentation/arm/kernel_userspace_helpers.rst
+ *
+ * Copyright (c) 2020 Linaro Ltd
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+
+#define ARM_COMMPAGE      (0xffff0f00u)
+#define ARM_KUSER_VERSION (*(int32_t *)(ARM_COMMPAGE + 0xfc))
+typedef void * (get_tls_fn)(void);
+#define ARM_KUSER_GET_TLS (*(get_tls_fn *)(ARM_COMMPAGE + 0xe0))
+typedef int (cmpxchg_fn)(int oldval, int newval, volatile int *ptr);
+#define ARM_KUSER_CMPXCHG (*(cmpxchg_fn *)(ARM_COMMPAGE + 0xc0))
+typedef void (dmb_fn)(void);
+#define ARM_KUSER_DMB (*(dmb_fn *)(ARM_COMMPAGE + 0xa0))
+typedef int (cmpxchg64_fn)(const int64_t *oldval,
+                           const int64_t *newval,
+                           volatile int64_t *ptr);
+#define ARM_KUSER_CMPXCHG64 (*(cmpxchg64_fn *)(ARM_COMMPAGE + 0x60))
+
+#define fail_unless(x)                                                  \
+    do {                                                                \
+        if (!(x)) {                                                     \
+            fprintf(stderr, "FAILED at %s:%d\n", __FILE__, __LINE__);   \
+            exit(EXIT_FAILURE);                                         \
+        }                                                               \
+    } while (0)
+
+
+int main(int argc, char *argv[argc])
+{
+    void *kuser_tls;
+    int val = 1;
+    const int64_t oldval = 1, newval = 2;
+    int64_t val64 = 1;
+
+    fail_unless(ARM_KUSER_VERSION == 0x5);
+    kuser_tls = ARM_KUSER_GET_TLS();
+    printf("TLS = %p\n", kuser_tls);
+    fail_unless(kuser_tls != 0);
+    fail_unless(ARM_KUSER_CMPXCHG(1, 2, &val) == 0);
+    printf("val = %d\n", val);
+    /* this is a crash test, not checking an actual barrier occurs */
+    ARM_KUSER_DMB();
+    fail_unless(ARM_KUSER_CMPXCHG64(&oldval, &newval, &val64) == 0);
+    printf("val64 = %lld\n", val64);
+    return 0;
+}
diff --git a/tests/tcg/arm/Makefile.target b/tests/tcg/arm/Makefile.target
index 11c39c601ea..3da09a38be7 100644
--- a/tests/tcg/arm/Makefile.target
+++ b/tests/tcg/arm/Makefile.target
@@ -68,6 +68,8 @@ run-semiconsole-arm: semiconsole-arm
 run-plugin-semiconsole-arm-with-%:
 	$(call skip-test, $<, "MANUAL ONLY")
 
+ARM_TESTS += commpage
+
 TESTS += $(ARM_TESTS)
 
 # On ARM Linux only supports 4k pages
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 10:05 ` [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
  2020-05-27 10:05   ` [Bug 1880225] " Alex Bennée
@ 2020-05-27 12:05   ` Aleksandar Markovic
  2020-05-27 12:05     ` [Bug 1880225] " Aleksandar Markovic
  2020-05-27 14:47     ` Aleksandar Markovic
  2020-06-02  0:28   ` Richard Henderson
  2 siblings, 2 replies; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-27 12:05 UTC (permalink / raw)
  To: Alex Bennée
  Cc: Peter Maydell, Bug 1880225, Riku Voipio, Richard Henderson,
	QEMU Developers, Laurent Vivier, qemu-arm

сре, 27. мај 2020. у 12:07 Alex Bennée <alex.bennee@linaro.org> је написао/ла:
>
> We rely on the pointer to wrap when accessing the high address of the
> COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
> cannot afford just to map the entire 4gb address range. The old mmap
> trial and error code handled this by just checking we could map both
> the guest_base and the computed COMMPAGE address.
>
> We can't just manipulate loadaddr to get what we want so we introduce
> an offset which pgb_find_hole can apply when looking for a gap for
> guest_base that ensures there is space left to map the COMMPAGE
> afterwards.
>
> This is arguably a little inefficient for the one 32 bit
> value (kuser_helper_version) we need to keep there given all the
> actual code entries are picked up during the translation phase.
>
> Fixes: ee94743034b
> Bug: https://bugs.launchpad.net/qemu/+bug/1880225

For the scenario in this bug report, for today's master, before and after
applying this patch:

BEFORE:

$ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294:
probe_guest_base: Assertion `have_guest_base' failed.
Aborted

AFTER:

$ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
CONTROL RESULT: (toupper_string)
 nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
 NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

So, it works in my test bed.

Tested-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

> Cc: Bug 1880225 <1880225@bugs.launchpad.net>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Richard Henderson <richard.henderson@linaro.org>
> Cc: Peter Maydell <peter.maydell@linaro.org>
> ---
>  linux-user/elfload.c | 18 ++++++++++--------
>  1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
> index d6027867a1a..31defce95b5 100644
> --- a/linux-user/elfload.c
> +++ b/linux-user/elfload.c
> @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
>
>  /* Return value for guest_base, or -1 if no hole found. */
>  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
> -                               long align)
> +                               long align, uintptr_t offset)
>  {
>      GSList *maps, *iter;
>      uintptr_t this_start, this_end, next_start, brk;
> @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>
>          this_end = ((MapInfo *)iter->data)->start;
>          next_start = ((MapInfo *)iter->data)->end;
> -        align_start = ROUND_UP(this_start, align);
> +        align_start = ROUND_UP(this_start + offset, align);
>
>          /* Skip holes that are too small. */
>          if (align_start >= this_end) {
> @@ -2221,6 +2221,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
>  {
>      uintptr_t loaddr = orig_loaddr;
>      uintptr_t hiaddr = orig_hiaddr;
> +    uintptr_t offset = 0;
>      uintptr_t addr;
>
>      if (hiaddr != orig_hiaddr) {
> @@ -2234,18 +2235,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
>      if (ARM_COMMPAGE) {
>          /*
>           * Extend the allocation to include the commpage.
> -         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
> -         * the address arithmetic will wrap around, but the difference
> -         * will produce the correct allocation size.
> +         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
> +         * need to ensure there is space bellow the guest_base so we
> +         * can map the commpage in the place needed when the address
> +         * arithmetic wraps around.
>           */
>          if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
>              hiaddr = (uintptr_t)4 << 30;
>          } else {
> -            loaddr = ARM_COMMPAGE & -align;
> +            offset = (128 * KiB);
>          }
>      }
>
> -    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
> +    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
>      if (addr == -1) {
>          /*
>           * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
> @@ -2280,7 +2282,7 @@ static void pgb_dynamic(const char *image_name, long align)
>           * just above that, and maximises the positive guest addresses.
>           */
>          commpage = ARM_COMMPAGE & -align;
> -        addr = pgb_find_hole(commpage, -commpage, align);
> +        addr = pgb_find_hole(commpage, -commpage, align, 0);
>          assert(addr != -1);
>          guest_base = addr;
>      }
> --
> 2.20.1
>
>


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 12:05   ` Aleksandar Markovic
@ 2020-05-27 12:05     ` Aleksandar Markovic
  2020-05-27 14:47     ` Aleksandar Markovic
  1 sibling, 0 replies; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-27 12:05 UTC (permalink / raw)
  To: qemu-devel

сре, 27. мај 2020. у 12:07 Alex Bennée <alex.bennee@linaro.org> је написао/ла:
>
> We rely on the pointer to wrap when accessing the high address of the
> COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
> cannot afford just to map the entire 4gb address range. The old mmap
> trial and error code handled this by just checking we could map both
> the guest_base and the computed COMMPAGE address.
>
> We can't just manipulate loadaddr to get what we want so we introduce
> an offset which pgb_find_hole can apply when looking for a gap for
> guest_base that ensures there is space left to map the COMMPAGE
> afterwards.
>
> This is arguably a little inefficient for the one 32 bit
> value (kuser_helper_version) we need to keep there given all the
> actual code entries are picked up during the translation phase.
>
> Fixes: ee94743034b
> Bug: https://bugs.launchpad.net/qemu/+bug/1880225

For the scenario in this bug report, for today's master, before and after
applying this patch:

BEFORE:

$ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294:
probe_guest_base: Assertion `have_guest_base' failed.
Aborted

AFTER:

$ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
CONTROL RESULT: (toupper_string)
 nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
 NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

So, it works in my test bed.

Tested-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>

> Cc: Bug 1880225 <1880225@bugs.launchpad.net>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Richard Henderson <richard.henderson@linaro.org>
> Cc: Peter Maydell <peter.maydell@linaro.org>
> ---
>  linux-user/elfload.c | 18 ++++++++++--------
>  1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
> index d6027867a1a..31defce95b5 100644
> --- a/linux-user/elfload.c
> +++ b/linux-user/elfload.c
> @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
>
>  /* Return value for guest_base, or -1 if no hole found. */
>  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
> -                               long align)
> +                               long align, uintptr_t offset)
>  {
>      GSList *maps, *iter;
>      uintptr_t this_start, this_end, next_start, brk;
> @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>
>          this_end = ((MapInfo *)iter->data)->start;
>          next_start = ((MapInfo *)iter->data)->end;
> -        align_start = ROUND_UP(this_start, align);
> +        align_start = ROUND_UP(this_start + offset, align);
>
>          /* Skip holes that are too small. */
>          if (align_start >= this_end) {
> @@ -2221,6 +2221,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
>  {
>      uintptr_t loaddr = orig_loaddr;
>      uintptr_t hiaddr = orig_hiaddr;
> +    uintptr_t offset = 0;
>      uintptr_t addr;
>
>      if (hiaddr != orig_hiaddr) {
> @@ -2234,18 +2235,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
>      if (ARM_COMMPAGE) {
>          /*
>           * Extend the allocation to include the commpage.
> -         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
> -         * the address arithmetic will wrap around, but the difference
> -         * will produce the correct allocation size.
> +         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
> +         * need to ensure there is space bellow the guest_base so we
> +         * can map the commpage in the place needed when the address
> +         * arithmetic wraps around.
>           */
>          if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
>              hiaddr = (uintptr_t)4 << 30;
>          } else {
> -            loaddr = ARM_COMMPAGE & -align;
> +            offset = (128 * KiB);
>          }
>      }
>
> -    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
> +    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
>      if (addr == -1) {
>          /*
>           * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
> @@ -2280,7 +2282,7 @@ static void pgb_dynamic(const char *image_name, long align)
>           * just above that, and maximises the positive guest addresses.
>           */
>          commpage = ARM_COMMPAGE & -align;
> -        addr = pgb_find_hole(commpage, -commpage, align);
> +        addr = pgb_find_hole(commpage, -commpage, align, 0);
>          assert(addr != -1);
>          guest_base = addr;
>      }
> --
> 2.20.1
>
>

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  In Progress

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH  v1 0/3] some linux-user guest_base fixes
  2020-05-27 10:05 [PATCH v1 0/3] some linux-user guest_base fixes Alex Bennée
                   ` (2 preceding siblings ...)
  2020-05-27 10:05 ` [PATCH v1 3/3] tests/tcg: add simple commpage test case Alex Bennée
@ 2020-05-27 14:12 ` no-reply
  3 siblings, 0 replies; 76+ messages in thread
From: no-reply @ 2020-05-27 14:12 UTC (permalink / raw)
  To: alex.bennee; +Cc: qemu-arm, alex.bennee, qemu-devel

Patchew URL: https://patchew.org/QEMU/20200527100546.29297-1-alex.bennee@linaro.org/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20200527100546.29297-1-alex.bennee@linaro.org
Subject: [PATCH  v1 0/3] some linux-user guest_base fixes
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

From https://github.com/patchew-project/qemu
 - [tag update]      patchew/20200527100546.29297-1-alex.bennee@linaro.org -> patchew/20200527100546.29297-1-alex.bennee@linaro.org
Switched to a new branch 'test'
a537e13 tests/tcg: add simple commpage test case
26c424e linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
8825cb1 linux-user: provide fallback pgd_find_hole for bare chroots

=== OUTPUT BEGIN ===
1/3 Checking commit 8825cb148fa8 (linux-user: provide fallback pgd_find_hole for bare chroots)
WARNING: line over 80 characters
#44: FILE: linux-user/elfload.c:2114:
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)

total: 0 errors, 1 warnings, 60 lines checked

Patch 1/3 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/3 Checking commit 26c424e346df (linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit)
3/3 Checking commit a537e13dbea5 (tests/tcg: add simple commpage test case)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#28: 
new file mode 100644

ERROR: Use of volatile is usually wrong, please add a comment
#57: FILE: tests/tcg/arm/commpage.c:25:
+typedef int (cmpxchg_fn)(int oldval, int newval, volatile int *ptr);

ERROR: Use of volatile is usually wrong, please add a comment
#63: FILE: tests/tcg/arm/commpage.c:31:
+                           volatile int64_t *ptr);

total: 2 errors, 1 warnings, 69 lines checked

Patch 3/3 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200527100546.29297-1-alex.bennee@linaro.org/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 12:05   ` Aleksandar Markovic
  2020-05-27 12:05     ` [Bug 1880225] " Aleksandar Markovic
@ 2020-05-27 14:47     ` Aleksandar Markovic
  2020-05-27 14:47       ` [Bug 1880225] " Aleksandar Markovic
  2020-05-27 16:14       ` Alex Bennée
  1 sibling, 2 replies; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-27 14:47 UTC (permalink / raw)
  To: Alex Bennée
  Cc: Peter Maydell, Bug 1880225, Riku Voipio, Richard Henderson,
	QEMU Developers, Laurent Vivier, qemu-arm

сре, 27. мај 2020. у 14:05 Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> је написао/ла:
>
> сре, 27. мај 2020. у 12:07 Alex Bennée <alex.bennee@linaro.org> је написао/ла:
> >
> > We rely on the pointer to wrap when accessing the high address of the
> > COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
> > cannot afford just to map the entire 4gb address range. The old mmap
> > trial and error code handled this by just checking we could map both
> > the guest_base and the computed COMMPAGE address.
> >
> > We can't just manipulate loadaddr to get what we want so we introduce
> > an offset which pgb_find_hole can apply when looking for a gap for
> > guest_base that ensures there is space left to map the COMMPAGE
> > afterwards.
> >
> > This is arguably a little inefficient for the one 32 bit
> > value (kuser_helper_version) we need to keep there given all the
> > actual code entries are picked up during the translation phase.
> >
> > Fixes: ee94743034b
> > Bug: https://bugs.launchpad.net/qemu/+bug/1880225
>
> For the scenario in this bug report, for today's master, before and after
> applying this patch:
>

I am not sure how significant is this info, but I executed the test
without applying patch 1/3, so only 2/3 was applied in the case
"AFTER".

Aleksandar

> BEFORE:
>
> $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
> qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294:
> probe_guest_base: Assertion `have_guest_base' failed.
> Aborted
>
> AFTER:
>
> $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
> CONTROL RESULT: (toupper_string)
>  nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
>  NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ
>
> So, it works in my test bed.
>
> Tested-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
>
> > Cc: Bug 1880225 <1880225@bugs.launchpad.net>
> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> > Cc: Richard Henderson <richard.henderson@linaro.org>
> > Cc: Peter Maydell <peter.maydell@linaro.org>
> > ---
> >  linux-user/elfload.c | 18 ++++++++++--------
> >  1 file changed, 10 insertions(+), 8 deletions(-)
> >
> > diff --git a/linux-user/elfload.c b/linux-user/elfload.c
> > index d6027867a1a..31defce95b5 100644
> > --- a/linux-user/elfload.c
> > +++ b/linux-user/elfload.c
> > @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
> >
> >  /* Return value for guest_base, or -1 if no hole found. */
> >  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
> > -                               long align)
> > +                               long align, uintptr_t offset)
> >  {
> >      GSList *maps, *iter;
> >      uintptr_t this_start, this_end, next_start, brk;
> > @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
> >
> >          this_end = ((MapInfo *)iter->data)->start;
> >          next_start = ((MapInfo *)iter->data)->end;
> > -        align_start = ROUND_UP(this_start, align);
> > +        align_start = ROUND_UP(this_start + offset, align);
> >
> >          /* Skip holes that are too small. */
> >          if (align_start >= this_end) {
> > @@ -2221,6 +2221,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
> >  {
> >      uintptr_t loaddr = orig_loaddr;
> >      uintptr_t hiaddr = orig_hiaddr;
> > +    uintptr_t offset = 0;
> >      uintptr_t addr;
> >
> >      if (hiaddr != orig_hiaddr) {
> > @@ -2234,18 +2235,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
> >      if (ARM_COMMPAGE) {
> >          /*
> >           * Extend the allocation to include the commpage.
> > -         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
> > -         * the address arithmetic will wrap around, but the difference
> > -         * will produce the correct allocation size.
> > +         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
> > +         * need to ensure there is space bellow the guest_base so we
> > +         * can map the commpage in the place needed when the address
> > +         * arithmetic wraps around.
> >           */
> >          if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
> >              hiaddr = (uintptr_t)4 << 30;
> >          } else {
> > -            loaddr = ARM_COMMPAGE & -align;
> > +            offset = (128 * KiB);
> >          }
> >      }
> >
> > -    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
> > +    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
> >      if (addr == -1) {
> >          /*
> >           * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
> > @@ -2280,7 +2282,7 @@ static void pgb_dynamic(const char *image_name, long align)
> >           * just above that, and maximises the positive guest addresses.
> >           */
> >          commpage = ARM_COMMPAGE & -align;
> > -        addr = pgb_find_hole(commpage, -commpage, align);
> > +        addr = pgb_find_hole(commpage, -commpage, align, 0);
> >          assert(addr != -1);
> >          guest_base = addr;
> >      }
> > --
> > 2.20.1
> >
> >


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 14:47     ` Aleksandar Markovic
@ 2020-05-27 14:47       ` Aleksandar Markovic
  2020-05-27 16:14       ` Alex Bennée
  1 sibling, 0 replies; 76+ messages in thread
From: Aleksandar Markovic @ 2020-05-27 14:47 UTC (permalink / raw)
  To: qemu-devel

сре, 27. мај 2020. у 14:05 Aleksandar Markovic
<aleksandar.qemu.devel@gmail.com> је написао/ла:
>
> сре, 27. мај 2020. у 12:07 Alex Bennée <alex.bennee@linaro.org> је написао/ла:
> >
> > We rely on the pointer to wrap when accessing the high address of the
> > COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
> > cannot afford just to map the entire 4gb address range. The old mmap
> > trial and error code handled this by just checking we could map both
> > the guest_base and the computed COMMPAGE address.
> >
> > We can't just manipulate loadaddr to get what we want so we introduce
> > an offset which pgb_find_hole can apply when looking for a gap for
> > guest_base that ensures there is space left to map the COMMPAGE
> > afterwards.
> >
> > This is arguably a little inefficient for the one 32 bit
> > value (kuser_helper_version) we need to keep there given all the
> > actual code entries are picked up during the translation phase.
> >
> > Fixes: ee94743034b
> > Bug: https://bugs.launchpad.net/qemu/+bug/1880225
>
> For the scenario in this bug report, for today's master, before and after
> applying this patch:
>

I am not sure how significant is this info, but I executed the test
without applying patch 1/3, so only 2/3 was applied in the case
"AFTER".

Aleksandar

> BEFORE:
>
> $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
> qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294:
> probe_guest_base: Assertion `have_guest_base' failed.
> Aborted
>
> AFTER:
>
> $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
> CONTROL RESULT: (toupper_string)
>  nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
>  NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ
>
> So, it works in my test bed.
>
> Tested-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
>
> > Cc: Bug 1880225 <1880225@bugs.launchpad.net>
> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> > Cc: Richard Henderson <richard.henderson@linaro.org>
> > Cc: Peter Maydell <peter.maydell@linaro.org>
> > ---
> >  linux-user/elfload.c | 18 ++++++++++--------
> >  1 file changed, 10 insertions(+), 8 deletions(-)
> >
> > diff --git a/linux-user/elfload.c b/linux-user/elfload.c
> > index d6027867a1a..31defce95b5 100644
> > --- a/linux-user/elfload.c
> > +++ b/linux-user/elfload.c
> > @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
> >
> >  /* Return value for guest_base, or -1 if no hole found. */
> >  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
> > -                               long align)
> > +                               long align, uintptr_t offset)
> >  {
> >      GSList *maps, *iter;
> >      uintptr_t this_start, this_end, next_start, brk;
> > @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
> >
> >          this_end = ((MapInfo *)iter->data)->start;
> >          next_start = ((MapInfo *)iter->data)->end;
> > -        align_start = ROUND_UP(this_start, align);
> > +        align_start = ROUND_UP(this_start + offset, align);
> >
> >          /* Skip holes that are too small. */
> >          if (align_start >= this_end) {
> > @@ -2221,6 +2221,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
> >  {
> >      uintptr_t loaddr = orig_loaddr;
> >      uintptr_t hiaddr = orig_hiaddr;
> > +    uintptr_t offset = 0;
> >      uintptr_t addr;
> >
> >      if (hiaddr != orig_hiaddr) {
> > @@ -2234,18 +2235,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
> >      if (ARM_COMMPAGE) {
> >          /*
> >           * Extend the allocation to include the commpage.
> > -         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
> > -         * the address arithmetic will wrap around, but the difference
> > -         * will produce the correct allocation size.
> > +         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
> > +         * need to ensure there is space bellow the guest_base so we
> > +         * can map the commpage in the place needed when the address
> > +         * arithmetic wraps around.
> >           */
> >          if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
> >              hiaddr = (uintptr_t)4 << 30;
> >          } else {
> > -            loaddr = ARM_COMMPAGE & -align;
> > +            offset = (128 * KiB);
> >          }
> >      }
> >
> > -    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
> > +    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
> >      if (addr == -1) {
> >          /*
> >           * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
> > @@ -2280,7 +2282,7 @@ static void pgb_dynamic(const char *image_name, long align)
> >           * just above that, and maximises the positive guest addresses.
> >           */
> >          commpage = ARM_COMMPAGE & -align;
> > -        addr = pgb_find_hole(commpage, -commpage, align);
> > +        addr = pgb_find_hole(commpage, -commpage, align, 0);
> >          assert(addr != -1);
> >          guest_base = addr;
> >      }
> > --
> > 2.20.1
> >
> >

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  In Progress

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 14:47     ` Aleksandar Markovic
  2020-05-27 14:47       ` [Bug 1880225] " Aleksandar Markovic
@ 2020-05-27 16:14       ` Alex Bennée
  2020-05-27 16:14         ` [Bug 1880225] " Alex Bennée
  1 sibling, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-05-27 16:14 UTC (permalink / raw)
  To: Aleksandar Markovic
  Cc: Peter Maydell, Bug 1880225, Riku Voipio, Richard Henderson,
	QEMU Developers, Laurent Vivier, qemu-arm


Aleksandar Markovic <aleksandar.qemu.devel@gmail.com> writes:

> сре, 27. мај 2020. у 14:05 Aleksandar Markovic
> <aleksandar.qemu.devel@gmail.com> је написао/ла:
>>
>> сре, 27. мај 2020. у 12:07 Alex Bennée <alex.bennee@linaro.org> је написао/ла:
>> >
>> > We rely on the pointer to wrap when accessing the high address of the
>> > COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
>> > cannot afford just to map the entire 4gb address range. The old mmap
>> > trial and error code handled this by just checking we could map both
>> > the guest_base and the computed COMMPAGE address.
>> >
>> > We can't just manipulate loadaddr to get what we want so we introduce
>> > an offset which pgb_find_hole can apply when looking for a gap for
>> > guest_base that ensures there is space left to map the COMMPAGE
>> > afterwards.
>> >
>> > This is arguably a little inefficient for the one 32 bit
>> > value (kuser_helper_version) we need to keep there given all the
>> > actual code entries are picked up during the translation phase.
>> >
>> > Fixes: ee94743034b
>> > Bug: https://bugs.launchpad.net/qemu/+bug/1880225
>>
>> For the scenario in this bug report, for today's master, before and after
>> applying this patch:
>>
>
> I am not sure how significant is this info, but I executed the test
> without applying patch 1/3, so only 2/3 was applied in the case
> "AFTER".

That is expected. The other fix only affects binaries run inside a
/proc-less chroot and the final patch is a test case for COMMPAGE
support.

-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 16:14       ` Alex Bennée
@ 2020-05-27 16:14         ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-05-27 16:14 UTC (permalink / raw)
  To: qemu-devel

Aleksandar Markovic <aleksandar.qemu.devel@gmail.com> writes:

> сре, 27. мај 2020. у 14:05 Aleksandar Markovic
> <aleksandar.qemu.devel@gmail.com> је написао/ла:
>>
>> сре, 27. мај 2020. у 12:07 Alex Bennée <alex.bennee@linaro.org> је написао/ла:
>> >
>> > We rely on the pointer to wrap when accessing the high address of the
>> > COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
>> > cannot afford just to map the entire 4gb address range. The old mmap
>> > trial and error code handled this by just checking we could map both
>> > the guest_base and the computed COMMPAGE address.
>> >
>> > We can't just manipulate loadaddr to get what we want so we introduce
>> > an offset which pgb_find_hole can apply when looking for a gap for
>> > guest_base that ensures there is space left to map the COMMPAGE
>> > afterwards.
>> >
>> > This is arguably a little inefficient for the one 32 bit
>> > value (kuser_helper_version) we need to keep there given all the
>> > actual code entries are picked up during the translation phase.
>> >
>> > Fixes: ee94743034b
>> > Bug: https://bugs.launchpad.net/qemu/+bug/1880225
>>
>> For the scenario in this bug report, for today's master, before and after
>> applying this patch:
>>
>
> I am not sure how significant is this info, but I executed the test
> without applying patch 1/3, so only 2/3 was applied in the case
> "AFTER".

That is expected. The other fix only affects binaries run inside a
/proc-less chroot and the final patch is a test case for COMMPAGE
support.

-- 
Alex Bennée

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  In Progress

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-05-27 10:05 ` [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
  2020-05-27 10:05   ` [Bug 1880225] " Alex Bennée
  2020-05-27 12:05   ` Aleksandar Markovic
@ 2020-06-02  0:28   ` Richard Henderson
  2020-06-05  9:45     ` Alex Bennée
  2 siblings, 1 reply; 76+ messages in thread
From: Richard Henderson @ 2020-06-02  0:28 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel
  Cc: Peter Maydell, Bug 1880225, qemu-arm, Riku Voipio, Laurent Vivier

On 5/27/20 3:05 AM, Alex Bennée wrote:
> @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
>  
>  /* Return value for guest_base, or -1 if no hole found. */
>  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
> -                               long align)
> +                               long align, uintptr_t offset)
>  {
>      GSList *maps, *iter;
>      uintptr_t this_start, this_end, next_start, brk;
> @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>  
>          this_end = ((MapInfo *)iter->data)->start;
>          next_start = ((MapInfo *)iter->data)->end;
> -        align_start = ROUND_UP(this_start, align);
> +        align_start = ROUND_UP(this_start + offset, align);
>  
>          /* Skip holes that are too small. */

I suppose offset is supposed to mean we start from -offset?  You didn't update
pgb_find_hole_fallback.

> -            loaddr = ARM_COMMPAGE & -align;
> +            offset = (128 * KiB);

Why 128K?  Surely this should be an expression against ARM_COMMPAGE.


r~


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 1/3] linux-user: provide fallback pgd_find_hole for bare chroots
  2020-05-27 10:05 ` [PATCH v1 1/3] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
@ 2020-06-02  0:37   ` Richard Henderson
  0 siblings, 0 replies; 76+ messages in thread
From: Richard Henderson @ 2020-06-02  0:37 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel
  Cc: Peter Maydell, Riku Voipio, Laurent Vivier, qemu-arm

On 5/27/20 3:05 AM, Alex Bennée wrote:
> +static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)
> +{
> +    uintptr_t base;
> +
> +    /* Start at the bottom and work our way up */
> +    base = mmap_min_addr;
> +
> +    while (true) {
> +        uintptr_t align_start, end;
> +        align_start = ROUND_UP(base, align);
> +        end = align_start + guest_size;
> +
> +        /* if brk is anywhere in the range give ourselves some room to grow. */
> +        if (align_start <= brk && brk < end) {
> +            base += 16 * MiB;

You should skip the entire brk region with base = brk + 16 * MiB.

> +            base += qemu_host_page_size;

If align < qemu_host_page_size, then we'll try the same page multiple times.
Better as base = align_start + qemu_host_page_size.

Or even base = ROUND_UP(base, align) right at the beginning.


r~


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 3/3] tests/tcg: add simple commpage test case
  2020-05-27 10:05 ` [PATCH v1 3/3] tests/tcg: add simple commpage test case Alex Bennée
@ 2020-06-02  0:40   ` Richard Henderson
  0 siblings, 0 replies; 76+ messages in thread
From: Richard Henderson @ 2020-06-02  0:40 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel; +Cc: Peter Maydell, qemu-arm

On 5/27/20 3:05 AM, Alex Bennée wrote:
> The COMMPAGE are a number of kernel provided user-space routines for
> 32 bit ARM systems. Add a basic series of smoke tests to ensure it is
> working as it should.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/tcg/arm/commpage.c      | 61 +++++++++++++++++++++++++++++++++++
>  tests/tcg/arm/Makefile.target |  2 ++
>  2 files changed, 63 insertions(+)
>  create mode 100644 tests/tcg/arm/commpage.c

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-06-02  0:28   ` Richard Henderson
@ 2020-06-05  9:45     ` Alex Bennée
  2020-06-05  9:45       ` [Bug 1880225] " Alex Bennée
  2020-06-05 10:24       ` Alex Bennée
  0 siblings, 2 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05  9:45 UTC (permalink / raw)
  To: Richard Henderson
  Cc: Peter Maydell, Bug 1880225, Riku Voipio, qemu-devel,
	Laurent Vivier, qemu-arm


Richard Henderson <richard.henderson@linaro.org> writes:

> On 5/27/20 3:05 AM, Alex Bennée wrote:
>> @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
>>  
>>  /* Return value for guest_base, or -1 if no hole found. */
>>  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>> -                               long align)
>> +                               long align, uintptr_t offset)
>>  {
>>      GSList *maps, *iter;
>>      uintptr_t this_start, this_end, next_start, brk;
>> @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>>  
>>          this_end = ((MapInfo *)iter->data)->start;
>>          next_start = ((MapInfo *)iter->data)->end;
>> -        align_start = ROUND_UP(this_start, align);
>> +        align_start = ROUND_UP(this_start + offset, align);
>>  
>>          /* Skip holes that are too small. */
>
> I suppose offset is supposed to mean we start from -offset?

Well guest_base will start higher meaning we have space for the
commpage beneath it.

> You didn't update
> pgb_find_hole_fallback.

Fixed.

>
>> -            loaddr = ARM_COMMPAGE & -align;
>> +            offset = (128 * KiB);
>
> Why 128K?  Surely this should be an expression against ARM_COMMPAGE.

In theory:

            offset = -(ARM_COMMPAGE & -align);

should do the trick but I found it failed every now and again.
Frustratingly putting printfs in made it go away so in frustration I
just upped the offset until it stopped happening.

I do kinda wish rr worked on i386 :-/


>
>
> r~


-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-06-05  9:45     ` Alex Bennée
@ 2020-06-05  9:45       ` Alex Bennée
  2020-06-05 10:24       ` Alex Bennée
  1 sibling, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05  9:45 UTC (permalink / raw)
  To: qemu-devel

Richard Henderson <richard.henderson@linaro.org> writes:

> On 5/27/20 3:05 AM, Alex Bennée wrote:
>> @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
>>  
>>  /* Return value for guest_base, or -1 if no hole found. */
>>  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>> -                               long align)
>> +                               long align, uintptr_t offset)
>>  {
>>      GSList *maps, *iter;
>>      uintptr_t this_start, this_end, next_start, brk;
>> @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>>  
>>          this_end = ((MapInfo *)iter->data)->start;
>>          next_start = ((MapInfo *)iter->data)->end;
>> -        align_start = ROUND_UP(this_start, align);
>> +        align_start = ROUND_UP(this_start + offset, align);
>>  
>>          /* Skip holes that are too small. */
>
> I suppose offset is supposed to mean we start from -offset?

Well guest_base will start higher meaning we have space for the
commpage beneath it.

> You didn't update
> pgb_find_hole_fallback.

Fixed.

>
>> -            loaddr = ARM_COMMPAGE & -align;
>> +            offset = (128 * KiB);
>
> Why 128K?  Surely this should be an expression against ARM_COMMPAGE.

In theory:

            offset = -(ARM_COMMPAGE & -align);

should do the trick but I found it failed every now and again.
Frustratingly putting printfs in made it go away so in frustration I
just upped the offset until it stopped happening.

I do kinda wish rr worked on i386 :-/


>
>
> r~


-- 
Alex Bennée

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  In Progress

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-06-05  9:45     ` Alex Bennée
  2020-06-05  9:45       ` [Bug 1880225] " Alex Bennée
@ 2020-06-05 10:24       ` Alex Bennée
  2020-06-05 10:24         ` [Bug 1880225] " Alex Bennée
  1 sibling, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 10:24 UTC (permalink / raw)
  To: Richard Henderson
  Cc: Peter Maydell, Bug 1880225, Riku Voipio, qemu-devel,
	Laurent Vivier, qemu-arm


Alex Bennée <alex.bennee@linaro.org> writes:

> Richard Henderson <richard.henderson@linaro.org> writes:
>
>> On 5/27/20 3:05 AM, Alex Bennée wrote:
>>> @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
>>>  
>>>  /* Return value for guest_base, or -1 if no hole found. */
>>>  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>>> -                               long align)
>>> +                               long align, uintptr_t offset)
>>>  {
>>>      GSList *maps, *iter;
>>>      uintptr_t this_start, this_end, next_start, brk;
>>> @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>>>  
>>>          this_end = ((MapInfo *)iter->data)->start;
>>>          next_start = ((MapInfo *)iter->data)->end;
>>> -        align_start = ROUND_UP(this_start, align);
>>> +        align_start = ROUND_UP(this_start + offset, align);
>>>  
>>>          /* Skip holes that are too small. */
>>
>> I suppose offset is supposed to mean we start from -offset?
>
> Well guest_base will start higher meaning we have space for the
> commpage beneath it.
>
>> You didn't update
>> pgb_find_hole_fallback.
>
> Fixed.
>
>>
>>> -            loaddr = ARM_COMMPAGE & -align;
>>> +            offset = (128 * KiB);
>>
>> Why 128K?  Surely this should be an expression against ARM_COMMPAGE.
>
> In theory:
>
>             offset = -(ARM_COMMPAGE & -align);
>
> should do the trick but I found it failed every now and again.
> Frustratingly putting printfs in made it go away so in frustration I
> just upped the offset until it stopped happening.
>
> I do kinda wish rr worked on i386 :-/

Ahh all I needed was a MAP_FIXED for init_commpage

-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-06-05 10:24       ` Alex Bennée
@ 2020-06-05 10:24         ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 10:24 UTC (permalink / raw)
  To: qemu-devel

Alex Bennée <alex.bennee@linaro.org> writes:

> Richard Henderson <richard.henderson@linaro.org> writes:
>
>> On 5/27/20 3:05 AM, Alex Bennée wrote:
>>> @@ -2145,7 +2145,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
>>>  
>>>  /* Return value for guest_base, or -1 if no hole found. */
>>>  static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>>> -                               long align)
>>> +                               long align, uintptr_t offset)
>>>  {
>>>      GSList *maps, *iter;
>>>      uintptr_t this_start, this_end, next_start, brk;
>>> @@ -2171,7 +2171,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
>>>  
>>>          this_end = ((MapInfo *)iter->data)->start;
>>>          next_start = ((MapInfo *)iter->data)->end;
>>> -        align_start = ROUND_UP(this_start, align);
>>> +        align_start = ROUND_UP(this_start + offset, align);
>>>  
>>>          /* Skip holes that are too small. */
>>
>> I suppose offset is supposed to mean we start from -offset?
>
> Well guest_base will start higher meaning we have space for the
> commpage beneath it.
>
>> You didn't update
>> pgb_find_hole_fallback.
>
> Fixed.
>
>>
>>> -            loaddr = ARM_COMMPAGE & -align;
>>> +            offset = (128 * KiB);
>>
>> Why 128K?  Surely this should be an expression against ARM_COMMPAGE.
>
> In theory:
>
>             offset = -(ARM_COMMPAGE & -align);
>
> should do the trick but I found it failed every now and again.
> Frustratingly putting printfs in made it go away so in frustration I
> just upped the offset until it stopped happening.
>
> I do kinda wish rr worked on i386 :-/

Ahh all I needed was a MAP_FIXED for init_commpage

-- 
Alex Bennée

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  In Progress

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [PATCH  v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes)
@ 2020-06-05 15:49 Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 01/14] qemu-plugin.h: add missing include <stddef.h> to define size_t Alex Bennée
                   ` (16 more replies)
  0 siblings, 17 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Alex Bennée

Hi,

These are all the patches I've currently got which are ready for a
pull request next week. I've included some patches which are destined
to go in via other trees so I can keep the testing green on the CI.

In summary:

 Some simple plugin cleanups (the reset remain in plugins/next)
 Reliability fixes for travis/shippable
 iotest 194 fix (going in via block tree?)
 docker updates (ubuntu and tricore fix)
 vhost-user and TCG fix
 more linux-user guest_base fixes

I'll certainly include the testing stuff in my PR but if others are
happy for me to include bits touching their areas then shout and I'll
include them in the PR.

The following need review:

 - linux-user: detect overflow of MAP_FIXED mmap
 - linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
 - linux-user: provide fallback pgd_find_hole for bare chroots
 - tests/docker: fix pre-requisite for debian-tricore-cross
 - hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
 - .shippable: temporaily disable some cross builds
 - exec: flush the whole TLB if a watchpoint crosses a page boundary

Alex Bennée (10):
  tests/plugin: correctly honour io_count
  exec: flush the whole TLB if a watchpoint crosses a page boundary
  .travis.yml: allow failure for unreliable hosts
  .shippable: temporaily disable some cross builds
  tests/docker: fix pre-requisite for debian-tricore-cross
  hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
  linux-user: provide fallback pgd_find_hole for bare chroots
  linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  tests/tcg: add simple commpage test case
  linux-user: detect overflow of MAP_FIXED mmap

Emilio G. Cota (1):
  qemu-plugin.h: add missing include <stddef.h> to define size_t

Paolo Bonzini (1):
  docker: update Ubuntu to 20.04

Philippe Mathieu-Daudé (1):
  scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header

Vladimir Sementsov-Ogievskiy (1):
  iotests: 194: wait migration completion on target too

 include/qemu/qemu-plugin.h             |  1 +
 exec.c                                 |  8 ++-
 hw/virtio/vhost.c                      | 57 +++++++++++++++------
 linux-user/elfload.c                   | 71 ++++++++++++++++++++++----
 linux-user/mmap.c                      |  2 +-
 tests/plugin/mem.c                     |  2 +-
 tests/tcg/arm/commpage.c               | 61 ++++++++++++++++++++++
 .shippable.yml                         | 12 ++---
 .travis.yml                            |  5 ++
 hw/virtio/trace-events                 |  3 +-
 scripts/clean-includes                 |  1 +
 tests/docker/Makefile.include          |  2 +-
 tests/docker/dockerfiles/ubuntu.docker |  2 +-
 tests/qemu-iotests/194                 | 10 ++++
 tests/qemu-iotests/194.out             |  5 ++
 tests/tcg/arm/Makefile.target          |  2 +
 16 files changed, 206 insertions(+), 38 deletions(-)
 create mode 100644 tests/tcg/arm/commpage.c

-- 
2.20.1



^ permalink raw reply	[flat|nested] 76+ messages in thread

* [PATCH v1 01/14] qemu-plugin.h: add missing include <stddef.h> to define size_t
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 02/14] scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header Alex Bennée
                   ` (15 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Emilio G. Cota, Alex Bennée, Philippe Mathieu-Daudé

From: "Emilio G. Cota" <cota@braap.org>

Signed-off-by: Emilio G. Cota <cota@braap.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200524202427.951784-1-cota@braap.org>
---
 include/qemu/qemu-plugin.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/qemu/qemu-plugin.h b/include/qemu/qemu-plugin.h
index 89ed579f559..bab8b0d4b3a 100644
--- a/include/qemu/qemu-plugin.h
+++ b/include/qemu/qemu-plugin.h
@@ -12,6 +12,7 @@
 
 #include <inttypes.h>
 #include <stdbool.h>
+#include <stddef.h>
 
 /*
  * For best performance, build the plugin with -fvisibility=hidden so that
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH v1 02/14] scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 01/14] qemu-plugin.h: add missing include <stddef.h> to define size_t Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 03/14] tests/plugin: correctly honour io_count Alex Bennée
                   ` (14 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Emilio G . Cota, Alex Bennée, Philippe Mathieu-Daudé

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

"qemu/qemu-plugin.h" isn't meant to be include by QEMU codebase,
but by 3rd party plugins that QEMU can use. These plugins can be
built out of QEMU and don't include "qemu/osdep.h".
Mark "qemu/qemu-plugin.h" as a special header that doesn't need
to be cleaned for "qemu/osdep.h".

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200524215654.13256-1-f4bug@amsat.org>
---
 scripts/clean-includes | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/clean-includes b/scripts/clean-includes
index dd938daa3ec..795b3bea318 100755
--- a/scripts/clean-includes
+++ b/scripts/clean-includes
@@ -123,6 +123,7 @@ for f in "$@"; do
       ;;
     *include/qemu/osdep.h | \
     *include/qemu/compiler.h | \
+    *include/qemu/qemu-plugin.h | \
     *include/glib-compat.h | \
     *include/sysemu/os-posix.h | \
     *include/sysemu/os-win32.h | \
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH  v1 03/14] tests/plugin: correctly honour io_count
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 01/14] qemu-plugin.h: add missing include <stddef.h> to define size_t Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 02/14] scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 04/14] exec: flush the whole TLB if a watchpoint crosses a page boundary Alex Bennée
                   ` (13 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Alex Bennée, Philippe Mathieu-Daudé

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/plugin/mem.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/plugin/mem.c b/tests/plugin/mem.c
index 878abf09d19..4725bd851d8 100644
--- a/tests/plugin/mem.c
+++ b/tests/plugin/mem.c
@@ -28,7 +28,7 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
 
     g_string_printf(out, "mem accesses: %" PRIu64 "\n", mem_count);
     if (do_haddr) {
-        g_string_append_printf(out, "io accesses: %" PRIu64 "\n", mem_count);
+        g_string_append_printf(out, "io accesses: %" PRIu64 "\n", io_count);
     }
     qemu_plugin_outs(out->str);
 }
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH v1 04/14] exec: flush the whole TLB if a watchpoint crosses a page boundary
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (2 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 03/14] tests/plugin: correctly honour io_count Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 05/14] .travis.yml: allow failure for unreliable hosts Alex Bennée
                   ` (12 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Alexander Bulekov, Paolo Bonzini, Alex Bennée, Richard Henderson

There is no particular reason why you can't have a watchpoint in TCG
that covers a large chunk of the address space. We could be clever
about it but these cases are pretty rare and we can assume the user
will expect a little performance degradation.

NB: In my testing gdb will silently squash a watchpoint like:

  watch (char[0x7fffffffff]) *0x0

to a 4 byte watchpoint. Practically it will limit the maximum size
based on max-value-size. However given enough of a tweak the sky is
the limit.

Reported-by: Alexander Bulekov <alxndr@bu.edu>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - use cleaner in_page = -(addr | TARGET_PAGE_MASK) logic per rth
---
 exec.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/exec.c b/exec.c
index 3d4c94a9dc3..7cd45e94fce 100644
--- a/exec.c
+++ b/exec.c
@@ -1036,6 +1036,7 @@ int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
                           int flags, CPUWatchpoint **watchpoint)
 {
     CPUWatchpoint *wp;
+    vaddr in_page;
 
     /* forbid ranges which are empty or run off the end of the address space */
     if (len == 0 || (addr + len - 1) < addr) {
@@ -1056,7 +1057,12 @@ int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
         QTAILQ_INSERT_TAIL(&cpu->watchpoints, wp, entry);
     }
 
-    tlb_flush_page(cpu, addr);
+    in_page = -(addr | TARGET_PAGE_MASK);
+    if (len <= in_page) {
+        tlb_flush_page(cpu, addr);
+    } else {
+        tlb_flush(cpu);
+    }
 
     if (watchpoint)
         *watchpoint = wp;
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH  v1 05/14] .travis.yml: allow failure for unreliable hosts
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (3 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 04/14] exec: flush the whole TLB if a watchpoint crosses a page boundary Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 06/14] .shippable: temporaily disable some cross builds Alex Bennée
                   ` (11 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Philippe Mathieu-Daudé,
	Alex Bennée, Philippe Mathieu-Daudé

They will still run but they won't get in the way of the result.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .travis.yml | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 564be50a3c1..ec6367af1f0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -429,6 +429,7 @@ jobs:
       env:
         - TEST_CMD="make check check-tcg V=1"
         - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS}"
+        - UNRELIABLE=true
 
     - name: "[ppc64] GCC check-tcg"
       arch: ppc64le
@@ -493,6 +494,7 @@ jobs:
       env:
         - TEST_CMD="make check check-tcg V=1"
         - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS},s390x-linux-user"
+        - UNRELIABLE=true
       script:
         - ( cd ${SRC_DIR} ; git submodule update --init roms/SLOF )
         - BUILD_RC=0 && make -j${JOBS} || BUILD_RC=$?
@@ -535,6 +537,7 @@ jobs:
         - TEST_CMD="make check-unit"
         - CONFIG="--disable-containers --disable-tcg --enable-kvm
                   --disable-tools --host-cc=clang --cxx=clang++"
+        - UNRELIABLE=true
 
     # Release builds
     # The make-release script expect a QEMU version, so our tag must start with a 'v'.
@@ -556,3 +559,5 @@ jobs:
         - mkdir -p release-build && cd release-build
         - ../configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; }
         - make install
+  allow_failures:
+    - env: UNRELIABLE=true
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH  v1 06/14] .shippable: temporaily disable some cross builds
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (4 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 05/14] .travis.yml: allow failure for unreliable hosts Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 16:12   ` Philippe Mathieu-Daudé
  2020-06-05 15:49 ` [PATCH v1 07/14] iotests: 194: wait migration completion on target too Alex Bennée
                   ` (10 subsequent siblings)
  16 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée

These currently fail due to Debian bug #960271 as the
linux-libc-library has a user-space build breaking symbol in it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .shippable.yml | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/.shippable.yml b/.shippable.yml
index 2cce7b56890..10cf219bff4 100644
--- a/.shippable.yml
+++ b/.shippable.yml
@@ -5,8 +5,8 @@ env:
   global:
     - LC_ALL=C
   matrix:
-    - IMAGE=debian-amd64
-      TARGET_LIST=x86_64-softmmu,x86_64-linux-user
+    # - IMAGE=debian-amd64
+    #   TARGET_LIST=x86_64-softmmu,x86_64-linux-user
     - IMAGE=debian-win32-cross
       TARGET_LIST=arm-softmmu,i386-softmmu,lm32-softmmu
     - IMAGE=debian-win64-cross
@@ -19,10 +19,10 @@ env:
       TARGET_LIST=aarch64-softmmu,aarch64-linux-user
     - IMAGE=debian-s390x-cross
       TARGET_LIST=s390x-softmmu,s390x-linux-user
-    - IMAGE=debian-mips-cross
-      TARGET_LIST=mips-softmmu,mipsel-linux-user
-    - IMAGE=debian-mips64el-cross
-      TARGET_LIST=mips64el-softmmu,mips64el-linux-user
+    # - IMAGE=debian-mips-cross
+    #   TARGET_LIST=mips-softmmu,mipsel-linux-user
+    # - IMAGE=debian-mips64el-cross
+    #   TARGET_LIST=mips64el-softmmu,mips64el-linux-user
     - IMAGE=debian-ppc64el-cross
       TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user
 build:
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH v1 07/14] iotests: 194: wait migration completion on target too
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (5 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 06/14] .shippable: temporaily disable some cross builds Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 08/14] tests/docker: fix pre-requisite for debian-tricore-cross Alex Bennée
                   ` (9 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Kevin Wolf, Thomas Huth, Vladimir Sementsov-Ogievskiy,
	open list:Block layer core, Max Reitz, Alex Bennée

From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

It is possible, that shutdown on target occurs earlier than migration
finish. In this case we crash in bdrv_release_dirty_bitmap_locked()
on assertion "assert(!bdrv_dirty_bitmap_busy(bitmap));" as we do have
busy bitmap, as bitmap migration is ongoing.

We'll fix bitmap migration to gracefully cancel on early shutdown soon.
Now let's fix iotest 194 to wait migration completion before shutdown.

Note that in this test dest_vm.shutdown() is called implicitly, as vms
used as context-providers, see __exit__() method of QEMUMachine class.

Actually, not waiting migration finish is a wrong thing, but the test
started to crash after commit ae00aa239847682
"iotests: 194: test also migration of dirty bitmap", which added dirty
bitmaps here. So, Fixes: tag won't hurt.

Fixes: ae00aa2398476824f0eca80461da215e7cdc1c3b
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200604083341.26978-1-vsementsov@virtuozzo.com>
---
 tests/qemu-iotests/194     | 10 ++++++++++
 tests/qemu-iotests/194.out |  5 +++++
 2 files changed, 15 insertions(+)

diff --git a/tests/qemu-iotests/194 b/tests/qemu-iotests/194
index 3fad7c6c1ab..6dc2bc94d7e 100755
--- a/tests/qemu-iotests/194
+++ b/tests/qemu-iotests/194
@@ -87,4 +87,14 @@ with iotests.FilePath('source.img') as source_img_path, \
             iotests.log(dest_vm.qmp('nbd-server-stop'))
             break
 
+    iotests.log('Wait migration completion on target...')
+    migr_events = (('MIGRATION', {'data': {'status': 'completed'}}),
+                   ('MIGRATION', {'data': {'status': 'failed'}}))
+    event = dest_vm.events_wait(migr_events)
+    iotests.log(event, filters=[iotests.filter_qmp_event])
+
+    iotests.log('Check bitmaps on source:')
     iotests.log(source_vm.qmp('query-block')['return'][0]['dirty-bitmaps'])
+
+    iotests.log('Check bitmaps on target:')
+    iotests.log(dest_vm.qmp('query-block')['return'][0]['dirty-bitmaps'])
diff --git a/tests/qemu-iotests/194.out b/tests/qemu-iotests/194.out
index dd60dcc14f1..f70cf7610e0 100644
--- a/tests/qemu-iotests/194.out
+++ b/tests/qemu-iotests/194.out
@@ -21,4 +21,9 @@ Gracefully ending the `drive-mirror` job on source...
 {"data": {"device": "mirror-job0", "len": 1073741824, "offset": 1073741824, "speed": 0, "type": "mirror"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
 Stopping the NBD server on destination...
 {"return": {}}
+Wait migration completion on target...
+{"data": {"status": "completed"}, "event": "MIGRATION", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Check bitmaps on source:
+[{"busy": false, "count": 0, "granularity": 65536, "name": "bitmap0", "persistent": false, "recording": true, "status": "active"}]
+Check bitmaps on target:
 [{"busy": false, "count": 0, "granularity": 65536, "name": "bitmap0", "persistent": false, "recording": true, "status": "active"}]
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH v1 08/14] tests/docker: fix pre-requisite for debian-tricore-cross
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (6 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 07/14] iotests: 194: wait migration completion on target too Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 09/14] docker: update Ubuntu to 20.04 Alex Bennée
                   ` (8 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, Bastian Koppelmann, Alex Bennée,
	Philippe Mathieu-Daudé

Reported-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ed46bd98eb5..981b7fcf2a5 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -130,7 +130,7 @@ docker-image-debian-sparc64-cross: docker-image-debian10
 docker-image-travis: NOUSER=1
 
 # Specialist build images, sometimes very limited tools
-docker-image-tricore-cross: docker-image-debian9
+docker-image-debian-tricore-cross: docker-image-debian9
 docker-image-debian-arm64-test-cross: docker-image-debian11
 
 # These images may be good enough for building tests but not for test builds
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH  v1 09/14] docker: update Ubuntu to 20.04
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (7 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 08/14] tests/docker: fix pre-requisite for debian-tricore-cross Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 10/14] hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE Alex Bennée
                   ` (7 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Alex Bennée, Paolo Bonzini, Fam Zheng, Philippe Mathieu-Daudé

From: Paolo Bonzini <pbonzini@redhat.com>

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200604231716.11354-1-pbonzini@redhat.com>
---
 tests/docker/dockerfiles/ubuntu.docker | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index eeb3b22bf20..43872417dec 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -9,7 +9,7 @@
 # system won't pick up that it has changed.
 #
 
-FROM ubuntu:19.04
+FROM ubuntu:20.04
 ENV PACKAGES flex bison \
     ccache \
     clang \
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH v1 10/14] hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (8 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 09/14] docker: update Ubuntu to 20.04 Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 11/14] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
                   ` (6 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Michael S . Tsirkin, Alex Bennée, Dr . David Alan Gilbert,
	Stefan Hajnoczi, Fabiano Rosas

The purpose of vhost_section is to identify RAM regions that need to
be made available to a vhost client. However when running under TCG
all RAM sections have DIRTY_MEMORY_CODE set which leads to problems
down the line.

Re-factor the code so:

  - steps are clearer to follow
  - reason for rejection is recorded in the trace point
  - we allow DIRTY_MEMORY_CODE

We expand the comment to explain that kernel based vhost has specific
support for migration tracking.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Fabiano Rosas <farosas@linux.ibm.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200604134022.10564-1-alex.bennee@linaro.org>

---
v3
  - more tweaks to comments
  - merge the flag setting into one line
---
 hw/virtio/vhost.c      | 57 ++++++++++++++++++++++++++++++------------
 hw/virtio/trace-events |  3 ++-
 2 files changed, 43 insertions(+), 17 deletions(-)

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index aff98a0ede5..e3e21812905 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -27,6 +27,7 @@
 #include "migration/blocker.h"
 #include "migration/qemu-file-types.h"
 #include "sysemu/dma.h"
+#include "sysemu/tcg.h"
 #include "trace.h"
 
 /* enabled until disconnected backend stabilizes */
@@ -403,26 +404,50 @@ static int vhost_verify_ring_mappings(struct vhost_dev *dev,
     return r;
 }
 
+/*
+ * vhost_section: identify sections needed for vhost access
+ *
+ * We only care about RAM sections here (where virtqueue and guest
+ * internals accessed by virtio might live). If we find one we still
+ * allow the backend to potentially filter it out of our list.
+ */
 static bool vhost_section(struct vhost_dev *dev, MemoryRegionSection *section)
 {
-    bool result;
-    bool log_dirty = memory_region_get_dirty_log_mask(section->mr) &
-                     ~(1 << DIRTY_MEMORY_MIGRATION);
-    result = memory_region_is_ram(section->mr) &&
-        !memory_region_is_rom(section->mr);
-
-    /* Vhost doesn't handle any block which is doing dirty-tracking other
-     * than migration; this typically fires on VGA areas.
-     */
-    result &= !log_dirty;
+    MemoryRegion *mr = section->mr;
+
+    if (memory_region_is_ram(mr) && !memory_region_is_rom(mr)) {
+        uint8_t dirty_mask = memory_region_get_dirty_log_mask(mr);
+        uint8_t handled_dirty;
+
+        /*
+         * Kernel based vhost doesn't handle any block which is doing
+         * dirty-tracking other than migration for which it has
+         * specific logging support. However for TCG the kernel never
+         * gets involved anyway so we can also ignore it's
+         * self-modiying code detection flags. However a vhost-user
+         * client could still confuse a TCG guest if it re-writes
+         * executable memory that has already been translated.
+         */
+        handled_dirty = (1 << DIRTY_MEMORY_MIGRATION) |
+            (1 << DIRTY_MEMORY_CODE);
 
-    if (result && dev->vhost_ops->vhost_backend_mem_section_filter) {
-        result &=
-            dev->vhost_ops->vhost_backend_mem_section_filter(dev, section);
-    }
+        if (dirty_mask & ~handled_dirty) {
+            trace_vhost_reject_section(mr->name, 1);
+            return false;
+        }
+
+        if (dev->vhost_ops->vhost_backend_mem_section_filter &&
+            !dev->vhost_ops->vhost_backend_mem_section_filter(dev, section)) {
+            trace_vhost_reject_section(mr->name, 2);
+            return false;
+        }
 
-    trace_vhost_section(section->mr->name, result);
-    return result;
+        trace_vhost_section(mr->name);
+        return true;
+    } else {
+        trace_vhost_reject_section(mr->name, 3);
+        return false;
+    }
 }
 
 static void vhost_begin(MemoryListener *listener)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index e83500bee92..6427a0047df 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -5,7 +5,8 @@ vhost_commit(bool started, bool changed) "Started: %d Changed: %d"
 vhost_region_add_section(const char *name, uint64_t gpa, uint64_t size, uint64_t host) "%s: 0x%"PRIx64"+0x%"PRIx64" @ 0x%"PRIx64
 vhost_region_add_section_merge(const char *name, uint64_t new_size, uint64_t gpa, uint64_t owr) "%s: size: 0x%"PRIx64 " gpa: 0x%"PRIx64 " owr: 0x%"PRIx64
 vhost_region_add_section_aligned(const char *name, uint64_t gpa, uint64_t size, uint64_t host) "%s: 0x%"PRIx64"+0x%"PRIx64" @ 0x%"PRIx64
-vhost_section(const char *name, int r) "%s:%d"
+vhost_section(const char *name) "%s"
+vhost_reject_section(const char *name, int d) "%s:%d"
 vhost_iotlb_miss(void *dev, int step) "%p step %d"
 
 # vhost-user.c
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH v1 11/14] linux-user: provide fallback pgd_find_hole for bare chroots
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (9 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 10/14] hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 12/14] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
                   ` (5 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Peter Maydell, Riku Voipio, Alex Bennée, Laurent Vivier

When running QEMU out of a chroot environment we may not have access
to /proc/self/maps. As there is no other "official" way to introspect
our memory map we need to fall back to the original technique of
repeatedly trying to mmap an address range until we find one that
works.

Fortunately it's not quite as ugly as the original code given we
already re-factored the complications of dealing with the
ARM_COMMPAGE. We do make an attempt to skip over brk() which is about
the only concrete piece of information we have about the address map
at this moment.

Fixes: ee9474303
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200527100546.29297-2-alex.bennee@linaro.org>

---
v3
  - skip to brk if we get there
  - round up early
---
 linux-user/elfload.c | 48 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index ebc663ea0b3..475d243f3bd 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2101,6 +2101,50 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
     }
 }
 
+/**
+ * pgd_find_hole_fallback: potential mmap address
+ * @guest_size: size of available space
+ * @brk: location of break
+ * @align: memory alignment
+ *
+ * This is a fallback method for finding a hole in the host address
+ * space if we don't have the benefit of being able to access
+ * /proc/self/map. It can potentially take a very long time as we can
+ * only dumbly iterate up the host address space seeing if the
+ * allocation would work.
+ */
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)
+{
+    uintptr_t base;
+
+    /* Start (aligned) at the bottom and work our way up */
+    base = ROUND_UP(mmap_min_addr, align);
+
+    while (true) {
+        uintptr_t align_start, end;
+        align_start = ROUND_UP(base, align);
+        end = align_start + guest_size;
+
+        /* if brk is anywhere in the range give ourselves some room to grow. */
+        if (align_start <= brk && brk < end) {
+            base = brk + (16 * MiB);
+            continue;
+        } else if (align_start + guest_size < align_start) {
+            /* we have run out of space */
+            return -1;
+        } else {
+            int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE | MAP_FIXED;
+            void * mmap_start = mmap((void *) align_start, guest_size,
+                                     PROT_NONE, flags, -1, 0);
+            if (mmap_start != MAP_FAILED) {
+                munmap((void *) align_start, guest_size);
+                return (uintptr_t) mmap_start;
+            }
+            base += qemu_host_page_size;
+        }
+    }
+}
+
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
                                long align)
@@ -2116,6 +2160,10 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
     /* Read brk after we've read the maps, which will malloc. */
     brk = (uintptr_t)sbrk(0);
 
+    if (!maps) {
+        return pgd_find_hole_fallback(guest_size, brk, align);
+    }
+
     /* The first hole is before the first map entry. */
     this_start = mmap_min_addr;
 
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH v1 12/14] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (10 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 11/14] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49   ` [Bug 1880225] " Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 13/14] tests/tcg: add simple commpage test case Alex Bennée
                   ` (4 subsequent siblings)
  16 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Peter Maydell, Bug 1880225, Riku Voipio, Richard Henderson,
	Laurent Vivier, Aleksandar Markovic, Alex Bennée

We rely on the pointer to wrap when accessing the high address of the
COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
cannot afford just to map the entire 4gb address range. The old mmap
trial and error code handled this by just checking we could map both
the guest_base and the computed COMMPAGE address.

We can't just manipulate loadaddr to get what we want so we introduce
an offset which pgb_find_hole can apply when looking for a gap for
guest_base that ensures there is space left to map the COMMPAGE
afterwards.

This is arguably a little inefficient for the one 32 bit
value (kuser_helper_version) we need to keep there given all the
actual code entries are picked up during the translation phase.

Fixes: ee94743034b
Bug: https://bugs.launchpad.net/qemu/+bug/1880225
Cc: Bug 1880225 <1880225@bugs.launchpad.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200527100546.29297-3-alex.bennee@linaro.org>

---
v3
  - base offset on ARM_COMMPAGE
  - ensure we MAP_FIXED the commpage
  - support offset in pgd_find_hole_fallback
---
 linux-user/elfload.c | 31 +++++++++++++++++--------------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 475d243f3bd..b5cb21384a1 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -389,7 +389,7 @@ static bool init_guest_commpage(void)
 {
     void *want = g2h(ARM_COMMPAGE & -qemu_host_page_size);
     void *addr = mmap(want, qemu_host_page_size, PROT_READ | PROT_WRITE,
-                      MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+                      MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0);
 
     if (addr == MAP_FAILED) {
         perror("Allocating guest commpage");
@@ -2113,7 +2113,8 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
  * only dumbly iterate up the host address space seeing if the
  * allocation would work.
  */
-static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk,
+                                        long align, uintptr_t offset)
 {
     uintptr_t base;
 
@@ -2123,7 +2124,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
     while (true) {
         uintptr_t align_start, end;
         align_start = ROUND_UP(base, align);
-        end = align_start + guest_size;
+        end = align_start + guest_size + offset;
 
         /* if brk is anywhere in the range give ourselves some room to grow. */
         if (align_start <= brk && brk < end) {
@@ -2138,7 +2139,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
                                      PROT_NONE, flags, -1, 0);
             if (mmap_start != MAP_FAILED) {
                 munmap((void *) align_start, guest_size);
-                return (uintptr_t) mmap_start;
+                return (uintptr_t) mmap_start + offset;
             }
             base += qemu_host_page_size;
         }
@@ -2147,7 +2148,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
 
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
-                               long align)
+                               long align, uintptr_t offset)
 {
     GSList *maps, *iter;
     uintptr_t this_start, this_end, next_start, brk;
@@ -2161,7 +2162,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
     brk = (uintptr_t)sbrk(0);
 
     if (!maps) {
-        return pgd_find_hole_fallback(guest_size, brk, align);
+        return pgd_find_hole_fallback(guest_size, brk, align, offset);
     }
 
     /* The first hole is before the first map entry. */
@@ -2173,7 +2174,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
 
         this_end = ((MapInfo *)iter->data)->start;
         next_start = ((MapInfo *)iter->data)->end;
-        align_start = ROUND_UP(this_start, align);
+        align_start = ROUND_UP(this_start + offset, align);
 
         /* Skip holes that are too small. */
         if (align_start >= this_end) {
@@ -2223,6 +2224,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
 {
     uintptr_t loaddr = orig_loaddr;
     uintptr_t hiaddr = orig_hiaddr;
+    uintptr_t offset = 0;
     uintptr_t addr;
 
     if (hiaddr != orig_hiaddr) {
@@ -2236,18 +2238,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
     if (ARM_COMMPAGE) {
         /*
          * Extend the allocation to include the commpage.
-         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
-         * the address arithmetic will wrap around, but the difference
-         * will produce the correct allocation size.
+         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
+         * need to ensure there is space bellow the guest_base so we
+         * can map the commpage in the place needed when the address
+         * arithmetic wraps around.
          */
         if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
-            hiaddr = (uintptr_t)4 << 30;
+            hiaddr = (uintptr_t) 4 << 30;
         } else {
-            loaddr = ARM_COMMPAGE & -align;
+            offset = -(ARM_COMMPAGE & -align);
         }
     }
 
-    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
+    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
     if (addr == -1) {
         /*
          * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
@@ -2282,7 +2285,7 @@ static void pgb_dynamic(const char *image_name, long align)
          * just above that, and maximises the positive guest addresses.
          */
         commpage = ARM_COMMPAGE & -align;
-        addr = pgb_find_hole(commpage, -commpage, align);
+        addr = pgb_find_hole(commpage, -commpage, align, 0);
         assert(addr != -1);
         guest_base = addr;
     }
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [Bug 1880225] [PATCH v1 12/14] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-06-05 15:49 ` [PATCH v1 12/14] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
@ 2020-06-05 15:49   ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel

We rely on the pointer to wrap when accessing the high address of the
COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
cannot afford just to map the entire 4gb address range. The old mmap
trial and error code handled this by just checking we could map both
the guest_base and the computed COMMPAGE address.

We can't just manipulate loadaddr to get what we want so we introduce
an offset which pgb_find_hole can apply when looking for a gap for
guest_base that ensures there is space left to map the COMMPAGE
afterwards.

This is arguably a little inefficient for the one 32 bit
value (kuser_helper_version) we need to keep there given all the
actual code entries are picked up during the translation phase.

Fixes: ee94743034b
Bug: https://bugs.launchpad.net/qemu/+bug/1880225
Cc: Bug 1880225 <1880225@bugs.launchpad.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200527100546.29297-3-alex.bennee@linaro.org>

---
v3
  - base offset on ARM_COMMPAGE
  - ensure we MAP_FIXED the commpage
  - support offset in pgd_find_hole_fallback
---
 linux-user/elfload.c | 31 +++++++++++++++++--------------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 475d243f3bd..b5cb21384a1 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -389,7 +389,7 @@ static bool init_guest_commpage(void)
 {
     void *want = g2h(ARM_COMMPAGE & -qemu_host_page_size);
     void *addr = mmap(want, qemu_host_page_size, PROT_READ | PROT_WRITE,
-                      MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+                      MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0);
 
     if (addr == MAP_FAILED) {
         perror("Allocating guest commpage");
@@ -2113,7 +2113,8 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
  * only dumbly iterate up the host address space seeing if the
  * allocation would work.
  */
-static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk,
+                                        long align, uintptr_t offset)
 {
     uintptr_t base;
 
@@ -2123,7 +2124,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
     while (true) {
         uintptr_t align_start, end;
         align_start = ROUND_UP(base, align);
-        end = align_start + guest_size;
+        end = align_start + guest_size + offset;
 
         /* if brk is anywhere in the range give ourselves some room to grow. */
         if (align_start <= brk && brk < end) {
@@ -2138,7 +2139,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
                                      PROT_NONE, flags, -1, 0);
             if (mmap_start != MAP_FAILED) {
                 munmap((void *) align_start, guest_size);
-                return (uintptr_t) mmap_start;
+                return (uintptr_t) mmap_start + offset;
             }
             base += qemu_host_page_size;
         }
@@ -2147,7 +2148,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
 
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
-                               long align)
+                               long align, uintptr_t offset)
 {
     GSList *maps, *iter;
     uintptr_t this_start, this_end, next_start, brk;
@@ -2161,7 +2162,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
     brk = (uintptr_t)sbrk(0);
 
     if (!maps) {
-        return pgd_find_hole_fallback(guest_size, brk, align);
+        return pgd_find_hole_fallback(guest_size, brk, align, offset);
     }
 
     /* The first hole is before the first map entry. */
@@ -2173,7 +2174,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
 
         this_end = ((MapInfo *)iter->data)->start;
         next_start = ((MapInfo *)iter->data)->end;
-        align_start = ROUND_UP(this_start, align);
+        align_start = ROUND_UP(this_start + offset, align);
 
         /* Skip holes that are too small. */
         if (align_start >= this_end) {
@@ -2223,6 +2224,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
 {
     uintptr_t loaddr = orig_loaddr;
     uintptr_t hiaddr = orig_hiaddr;
+    uintptr_t offset = 0;
     uintptr_t addr;
 
     if (hiaddr != orig_hiaddr) {
@@ -2236,18 +2238,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
     if (ARM_COMMPAGE) {
         /*
          * Extend the allocation to include the commpage.
-         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
-         * the address arithmetic will wrap around, but the difference
-         * will produce the correct allocation size.
+         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
+         * need to ensure there is space bellow the guest_base so we
+         * can map the commpage in the place needed when the address
+         * arithmetic wraps around.
          */
         if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
-            hiaddr = (uintptr_t)4 << 30;
+            hiaddr = (uintptr_t) 4 << 30;
         } else {
-            loaddr = ARM_COMMPAGE & -align;
+            offset = -(ARM_COMMPAGE & -align);
         }
     }
 
-    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
+    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
     if (addr == -1) {
         /*
          * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
@@ -2282,7 +2285,7 @@ static void pgb_dynamic(const char *image_name, long align)
          * just above that, and maximises the positive guest addresses.
          */
         commpage = ARM_COMMPAGE & -align;
-        addr = pgb_find_hole(commpage, -commpage, align);
+        addr = pgb_find_hole(commpage, -commpage, align, 0);
         assert(addr != -1);
         guest_base = addr;
     }
-- 
2.20.1

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  In Progress

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH  v1 13/14] tests/tcg: add simple commpage test case
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (11 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 12/14] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 15:49 ` [PATCH v1 14/14] linux-user: detect overflow of MAP_FIXED mmap Alex Bennée
                   ` (3 subsequent siblings)
  16 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel
  Cc: Richard Henderson, open list:ARM TCG CPUs, Alex Bennée,
	Peter Maydell

The COMMPAGE are a number of kernel provided user-space routines for
32 bit ARM systems. Add a basic series of smoke tests to ensure it is
working as it should.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200527100546.29297-4-alex.bennee@linaro.org>
---
 tests/tcg/arm/commpage.c      | 61 +++++++++++++++++++++++++++++++++++
 tests/tcg/arm/Makefile.target |  2 ++
 2 files changed, 63 insertions(+)
 create mode 100644 tests/tcg/arm/commpage.c

diff --git a/tests/tcg/arm/commpage.c b/tests/tcg/arm/commpage.c
new file mode 100644
index 00000000000..c76e70cb8bd
--- /dev/null
+++ b/tests/tcg/arm/commpage.c
@@ -0,0 +1,61 @@
+/*
+ * Verify the COMMPAGE emulation
+ *
+ * The ARM commpage is a set of user space helper functions provided
+ * by the kernel in an effort to ease portability of user space code
+ * between different CPUs with potentially different capabilities. It
+ * is a 32 bit invention and similar to the vdso segment in many ways.
+ *
+ * The ABI is documented in the Linux kernel:
+ *     Documentation/arm/kernel_userspace_helpers.rst
+ *
+ * Copyright (c) 2020 Linaro Ltd
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+
+#define ARM_COMMPAGE      (0xffff0f00u)
+#define ARM_KUSER_VERSION (*(int32_t *)(ARM_COMMPAGE + 0xfc))
+typedef void * (get_tls_fn)(void);
+#define ARM_KUSER_GET_TLS (*(get_tls_fn *)(ARM_COMMPAGE + 0xe0))
+typedef int (cmpxchg_fn)(int oldval, int newval, volatile int *ptr);
+#define ARM_KUSER_CMPXCHG (*(cmpxchg_fn *)(ARM_COMMPAGE + 0xc0))
+typedef void (dmb_fn)(void);
+#define ARM_KUSER_DMB (*(dmb_fn *)(ARM_COMMPAGE + 0xa0))
+typedef int (cmpxchg64_fn)(const int64_t *oldval,
+                           const int64_t *newval,
+                           volatile int64_t *ptr);
+#define ARM_KUSER_CMPXCHG64 (*(cmpxchg64_fn *)(ARM_COMMPAGE + 0x60))
+
+#define fail_unless(x)                                                  \
+    do {                                                                \
+        if (!(x)) {                                                     \
+            fprintf(stderr, "FAILED at %s:%d\n", __FILE__, __LINE__);   \
+            exit(EXIT_FAILURE);                                         \
+        }                                                               \
+    } while (0)
+
+
+int main(int argc, char *argv[argc])
+{
+    void *kuser_tls;
+    int val = 1;
+    const int64_t oldval = 1, newval = 2;
+    int64_t val64 = 1;
+
+    fail_unless(ARM_KUSER_VERSION == 0x5);
+    kuser_tls = ARM_KUSER_GET_TLS();
+    printf("TLS = %p\n", kuser_tls);
+    fail_unless(kuser_tls != 0);
+    fail_unless(ARM_KUSER_CMPXCHG(1, 2, &val) == 0);
+    printf("val = %d\n", val);
+    /* this is a crash test, not checking an actual barrier occurs */
+    ARM_KUSER_DMB();
+    fail_unless(ARM_KUSER_CMPXCHG64(&oldval, &newval, &val64) == 0);
+    printf("val64 = %lld\n", val64);
+    return 0;
+}
diff --git a/tests/tcg/arm/Makefile.target b/tests/tcg/arm/Makefile.target
index 11c39c601ea..3da09a38be7 100644
--- a/tests/tcg/arm/Makefile.target
+++ b/tests/tcg/arm/Makefile.target
@@ -68,6 +68,8 @@ run-semiconsole-arm: semiconsole-arm
 run-plugin-semiconsole-arm-with-%:
 	$(call skip-test, $<, "MANUAL ONLY")
 
+ARM_TESTS += commpage
+
 TESTS += $(ARM_TESTS)
 
 # On ARM Linux only supports 4k pages
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PATCH  v1 14/14] linux-user: detect overflow of MAP_FIXED mmap
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (12 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 13/14] tests/tcg: add simple commpage test case Alex Bennée
@ 2020-06-05 15:49 ` Alex Bennée
  2020-06-05 16:16   ` Philippe Mathieu-Daudé
  2020-06-05 15:54 ` [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Eric Blake
                   ` (2 subsequent siblings)
  16 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-06-05 15:49 UTC (permalink / raw)
  To: qemu-devel; +Cc: Riku Voipio, Alex Bennée, Laurent Vivier

Relaxing the restrictions on 64 bit guests leads to the user being
able to attempt to map right at the edge of addressable memory. This
in turn lead to address overflow tripping the assert in page_set_flags
when the end address wrapped around.

Detect the wrap earlier and correctly -ENOMEM the guest (in the
reported case LTP mmap15).

Fixes: 7d8cbbabcb
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 linux-user/mmap.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index e3780337974..2e05bd499e6 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -467,7 +467,7 @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int prot,
          * It can fail only on 64-bit host with 32-bit target.
          * On any other target/host host mmap() handles this error correctly.
          */
-        if (!guest_range_valid(start, len)) {
+        if (end < start || !guest_range_valid(start, len)) {
             errno = ENOMEM;
             goto fail;
         }
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes)
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (13 preceding siblings ...)
  2020-06-05 15:49 ` [PATCH v1 14/14] linux-user: detect overflow of MAP_FIXED mmap Alex Bennée
@ 2020-06-05 15:54 ` Eric Blake
  2020-06-05 17:46 ` no-reply
  2020-06-07  6:55 ` Thomas Huth
  16 siblings, 0 replies; 76+ messages in thread
From: Eric Blake @ 2020-06-05 15:54 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel

On 6/5/20 10:49 AM, Alex Bennée wrote:
> Hi,
> 
> These are all the patches I've currently got which are ready for a
> pull request next week. I've included some patches which are destined
> to go in via other trees so I can keep the testing green on the CI.
> 
> In summary:
> 
>   Some simple plugin cleanups (the reset remain in plugins/next)
>   Reliability fixes for travis/shippable
>   iotest 194 fix (going in via block tree?)

I've got that one queued for a PR Monday with some typo fixes included.

>   docker updates (ubuntu and tricore fix)
>   vhost-user and TCG fix
>   more linux-user guest_base fixes
> 
> I'll certainly include the testing stuff in my PR but if others are
> happy for me to include bits touching their areas then shout and I'll
> include them in the PR.
> 
> The following need review:
> 
>   - linux-user: detect overflow of MAP_FIXED mmap
>   - linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
>   - linux-user: provide fallback pgd_find_hole for bare chroots
>   - tests/docker: fix pre-requisite for debian-tricore-cross
>   - hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
>   - .shippable: temporaily disable some cross builds
>   - exec: flush the whole TLB if a watchpoint crosses a page boundary
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 06/14] .shippable: temporaily disable some cross builds
  2020-06-05 15:49 ` [PATCH v1 06/14] .shippable: temporaily disable some cross builds Alex Bennée
@ 2020-06-05 16:12   ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 76+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-06-05 16:12 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel; +Cc: Fam Zheng

On 6/5/20 5:49 PM, Alex Bennée wrote:
> These currently fail due to Debian bug #960271 as the
> linux-libc-library has a user-space build breaking symbol in it.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .shippable.yml | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/.shippable.yml b/.shippable.yml
> index 2cce7b56890..10cf219bff4 100644
> --- a/.shippable.yml
> +++ b/.shippable.yml
> @@ -5,8 +5,8 @@ env:
>    global:
>      - LC_ALL=C
>    matrix:
> -    - IMAGE=debian-amd64
> -      TARGET_LIST=x86_64-softmmu,x86_64-linux-user
> +    # - IMAGE=debian-amd64
> +    #   TARGET_LIST=x86_64-softmmu,x86_64-linux-user
>      - IMAGE=debian-win32-cross
>        TARGET_LIST=arm-softmmu,i386-softmmu,lm32-softmmu
>      - IMAGE=debian-win64-cross
> @@ -19,10 +19,10 @@ env:
>        TARGET_LIST=aarch64-softmmu,aarch64-linux-user
>      - IMAGE=debian-s390x-cross
>        TARGET_LIST=s390x-softmmu,s390x-linux-user
> -    - IMAGE=debian-mips-cross
> -      TARGET_LIST=mips-softmmu,mipsel-linux-user
> -    - IMAGE=debian-mips64el-cross
> -      TARGET_LIST=mips64el-softmmu,mips64el-linux-user
> +    # - IMAGE=debian-mips-cross
> +    #   TARGET_LIST=mips-softmmu,mipsel-linux-user
> +    # - IMAGE=debian-mips64el-cross
> +    #   TARGET_LIST=mips64el-softmmu,mips64el-linux-user
>      - IMAGE=debian-ppc64el-cross
>        TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user
>  build:
> 

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 14/14] linux-user: detect overflow of MAP_FIXED mmap
  2020-06-05 15:49 ` [PATCH v1 14/14] linux-user: detect overflow of MAP_FIXED mmap Alex Bennée
@ 2020-06-05 16:16   ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 76+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-06-05 16:16 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel; +Cc: Riku Voipio, Laurent Vivier

On 6/5/20 5:49 PM, Alex Bennée wrote:
> Relaxing the restrictions on 64 bit guests leads to the user being
> able to attempt to map right at the edge of addressable memory. This
> in turn lead to address overflow tripping the assert in page_set_flags
> when the end address wrapped around.
> 
> Detect the wrap earlier and correctly -ENOMEM the guest (in the
> reported case LTP mmap15).
> 
> Fixes: 7d8cbbabcb

Reported-by: Laurent Vivier <laurent@vivier.eu>

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  linux-user/mmap.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/linux-user/mmap.c b/linux-user/mmap.c
> index e3780337974..2e05bd499e6 100644
> --- a/linux-user/mmap.c
> +++ b/linux-user/mmap.c
> @@ -467,7 +467,7 @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int prot,
>           * It can fail only on 64-bit host with 32-bit target.
>           * On any other target/host host mmap() handles this error correctly.
>           */
> -        if (!guest_range_valid(start, len)) {
> +        if (end < start || !guest_range_valid(start, len)) {
>              errno = ENOMEM;
>              goto fail;
>          }
> 



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH  v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes)
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (14 preceding siblings ...)
  2020-06-05 15:54 ` [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Eric Blake
@ 2020-06-05 17:46 ` no-reply
  2020-06-07  6:55 ` Thomas Huth
  16 siblings, 0 replies; 76+ messages in thread
From: no-reply @ 2020-06-05 17:46 UTC (permalink / raw)
  To: alex.bennee; +Cc: alex.bennee, qemu-devel

Patchew URL: https://patchew.org/QEMU/20200605154929.26910-1-alex.bennee@linaro.org/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20200605154929.26910-1-alex.bennee@linaro.org
Subject: [PATCH  v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes)
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20200605165007.12095-1-peter.maydell@linaro.org -> patchew/20200605165007.12095-1-peter.maydell@linaro.org
 * [new tag]         patchew/20200605165656.17578-1-philmd@redhat.com -> patchew/20200605165656.17578-1-philmd@redhat.com
Auto packing the repository for optimum performance. You may also
run "git gc" manually. See "git help gc" for more information.
Switched to a new branch 'test'
8c192c8 linux-user: detect overflow of MAP_FIXED mmap
343b114 tests/tcg: add simple commpage test case
3ea9b6b linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
bd10df6 linux-user: provide fallback pgd_find_hole for bare chroots
140a4b0 hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
c3fdeb9 docker: update Ubuntu to 20.04
3827a17 tests/docker: fix pre-requisite for debian-tricore-cross
00ba0df iotests: 194: wait migration completion on target too
8252e14 .shippable: temporaily disable some cross builds
1027a95 .travis.yml: allow failure for unreliable hosts
d545513 exec: flush the whole TLB if a watchpoint crosses a page boundary
c286730 tests/plugin: correctly honour io_count
573745f scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header
592de76 qemu-plugin.h: add missing include <stddef.h> to define size_t

=== OUTPUT BEGIN ===
1/14 Checking commit 592de76606a7 (qemu-plugin.h: add missing include <stddef.h> to define size_t)
2/14 Checking commit 573745fa77cd (scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header)
3/14 Checking commit c286730e2e6a (tests/plugin: correctly honour io_count)
4/14 Checking commit d545513f07b5 (exec: flush the whole TLB if a watchpoint crosses a page boundary)
5/14 Checking commit 1027a95a2300 (.travis.yml: allow failure for unreliable hosts)
6/14 Checking commit 8252e1457d91 (.shippable: temporaily disable some cross builds)
7/14 Checking commit 00ba0dfa3911 (iotests: 194: wait migration completion on target too)
8/14 Checking commit 3827a17199bf (tests/docker: fix pre-requisite for debian-tricore-cross)
9/14 Checking commit c3fdeb9b0a44 (docker: update Ubuntu to 20.04)
10/14 Checking commit 140a4b0c679f (hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE)
11/14 Checking commit bd10df6e7aff (linux-user: provide fallback pgd_find_hole for bare chroots)
WARNING: line over 80 characters
#46: FILE: linux-user/elfload.c:2116:
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)

total: 0 errors, 1 warnings, 60 lines checked

Patch 11/14 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
12/14 Checking commit 3ea9b6bac8e8 (linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit)
13/14 Checking commit 343b11418044 (tests/tcg: add simple commpage test case)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#30: 
new file mode 100644

ERROR: Use of volatile is usually wrong, please add a comment
#59: FILE: tests/tcg/arm/commpage.c:25:
+typedef int (cmpxchg_fn)(int oldval, int newval, volatile int *ptr);

ERROR: Use of volatile is usually wrong, please add a comment
#65: FILE: tests/tcg/arm/commpage.c:31:
+                           volatile int64_t *ptr);

total: 2 errors, 1 warnings, 69 lines checked

Patch 13/14 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

14/14 Checking commit 8c192c8751c3 (linux-user: detect overflow of MAP_FIXED mmap)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200605154929.26910-1-alex.bennee@linaro.org/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes)
  2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
                   ` (15 preceding siblings ...)
  2020-06-05 17:46 ` no-reply
@ 2020-06-07  6:55 ` Thomas Huth
  2020-06-08 15:58   ` Alex Bennée
  16 siblings, 1 reply; 76+ messages in thread
From: Thomas Huth @ 2020-06-07  6:55 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel

On 05/06/2020 17.49, Alex Bennée wrote:
> Hi,
> 
> These are all the patches I've currently got which are ready for a
> pull request next week. I've included some patches which are destined
> to go in via other trees so I can keep the testing green on the CI.
> 
> In summary:
> 
>  Some simple plugin cleanups (the reset remain in plugins/next)
>  Reliability fixes for travis/shippable
>  iotest 194 fix (going in via block tree?)
>  docker updates (ubuntu and tricore fix)
>  vhost-user and TCG fix
>  more linux-user guest_base fixes
> 
> I'll certainly include the testing stuff in my PR but if others are
> happy for me to include bits touching their areas then shout and I'll
> include them in the PR.
> 
> The following need review:
> 
>  - linux-user: detect overflow of MAP_FIXED mmap
>  - linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
>  - linux-user: provide fallback pgd_find_hole for bare chroots
>  - tests/docker: fix pre-requisite for debian-tricore-cross
>  - hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
>  - .shippable: temporaily disable some cross builds
>  - exec: flush the whole TLB if a watchpoint crosses a page boundary
> 
> Alex Bennée (10):
>   tests/plugin: correctly honour io_count
>   exec: flush the whole TLB if a watchpoint crosses a page boundary
>   .travis.yml: allow failure for unreliable hosts
>   .shippable: temporaily disable some cross builds
>   tests/docker: fix pre-requisite for debian-tricore-cross
>   hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
>   linux-user: provide fallback pgd_find_hole for bare chroots
>   linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
>   tests/tcg: add simple commpage test case
>   linux-user: detect overflow of MAP_FIXED mmap
> 
> Emilio G. Cota (1):
>   qemu-plugin.h: add missing include <stddef.h> to define size_t
> 
> Paolo Bonzini (1):
>   docker: update Ubuntu to 20.04
> 
> Philippe Mathieu-Daudé (1):
>   scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header

What about Philippe's "tests: Remove unused bison/flex packages" v2
patch series from May 15th? I think you could include it here, too.

 Thomas



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes)
  2020-06-07  6:55 ` Thomas Huth
@ 2020-06-08 15:58   ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-08 15:58 UTC (permalink / raw)
  To: Thomas Huth; +Cc: qemu-devel


Thomas Huth <thuth@redhat.com> writes:

> On 05/06/2020 17.49, Alex Bennée wrote:
>> Hi,
>> 
>> These are all the patches I've currently got which are ready for a
>> pull request next week. I've included some patches which are destined
>> to go in via other trees so I can keep the testing green on the CI.
>> 
>> In summary:
>> 
>>  Some simple plugin cleanups (the reset remain in plugins/next)
>>  Reliability fixes for travis/shippable
>>  iotest 194 fix (going in via block tree?)
>>  docker updates (ubuntu and tricore fix)
>>  vhost-user and TCG fix
>>  more linux-user guest_base fixes
>> 
>> I'll certainly include the testing stuff in my PR but if others are
>> happy for me to include bits touching their areas then shout and I'll
>> include them in the PR.
>> 
>> The following need review:
>> 
>>  - linux-user: detect overflow of MAP_FIXED mmap
>>  - linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
>>  - linux-user: provide fallback pgd_find_hole for bare chroots
>>  - tests/docker: fix pre-requisite for debian-tricore-cross
>>  - hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
>>  - .shippable: temporaily disable some cross builds
>>  - exec: flush the whole TLB if a watchpoint crosses a page boundary
>> 
>> Alex Bennée (10):
>>   tests/plugin: correctly honour io_count
>>   exec: flush the whole TLB if a watchpoint crosses a page boundary
>>   .travis.yml: allow failure for unreliable hosts
>>   .shippable: temporaily disable some cross builds
>>   tests/docker: fix pre-requisite for debian-tricore-cross
>>   hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
>>   linux-user: provide fallback pgd_find_hole for bare chroots
>>   linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
>>   tests/tcg: add simple commpage test case
>>   linux-user: detect overflow of MAP_FIXED mmap
>> 
>> Emilio G. Cota (1):
>>   qemu-plugin.h: add missing include <stddef.h> to define size_t
>> 
>> Paolo Bonzini (1):
>>   docker: update Ubuntu to 20.04
>> 
>> Philippe Mathieu-Daudé (1):
>>   scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header
>
> What about Philippe's "tests: Remove unused bison/flex packages" v2
> patch series from May 15th? I think you could include it here, too.

Sure - grabbed.

-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [PULL 00/17] testing and misc fixes
@ 2020-06-09 10:37 Alex Bennée
  2020-06-09 10:37 ` [PULL 01/17] qemu-plugin.h: add missing include <stddef.h> to define size_t Alex Bennée
                   ` (17 more replies)
  0 siblings, 18 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:37 UTC (permalink / raw)
  To: peter.maydell; +Cc: Alex Bennée, qemu-devel

The following changes since commit 49ee11555262a256afec592dfed7c5902d5eefd2:

  Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-5.1-pull-request' into staging (2020-06-08 11:04:57 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-testing-and-misc-080620-1

for you to fetch changes up to a5b04ccd742f6c58a1d305530d9e07ad9731b8e6:

  scripts/coverity-scan: Remove flex/bison packages (2020-06-08 17:04:19 +0100)

----------------------------------------------------------------
Various testing and misc fixes:

  - header cleanups for plugins
  - support wider watchpoints
  - tweaks for unreliable and broken CI
  - docker image fixes and verion bumps
  - linux-user guest_base fixes
  - remove flex/bison from various test images

----------------------------------------------------------------
Alex Bennée (10):
      tests/plugin: correctly honour io_count
      exec: flush the whole TLB if a watchpoint crosses a page boundary
      .travis.yml: allow failure for unreliable hosts
      .shippable: temporaily disable some cross builds
      tests/docker: fix pre-requisite for debian-tricore-cross
      hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
      linux-user: provide fallback pgd_find_hole for bare chroots
      linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
      tests/tcg: add simple commpage test case
      linux-user: detect overflow of MAP_FIXED mmap

Emilio G. Cota (1):
      qemu-plugin.h: add missing include <stddef.h> to define size_t

Paolo Bonzini (1):
      docker: update Ubuntu to 20.04

Philippe Mathieu-Daudé (5):
      scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header
      tests/docker: Remove flex/bison packages
      tests/vm: Remove flex/bison packages
      cirrus-ci: Remove flex/bison packages
      scripts/coverity-scan: Remove flex/bison packages

 include/qemu/qemu-plugin.h                         |  1 +
 exec.c                                             |  8 ++-
 hw/virtio/vhost.c                                  | 57 ++++++++++++-----
 linux-user/elfload.c                               | 71 +++++++++++++++++++---
 linux-user/mmap.c                                  |  2 +-
 tests/plugin/mem.c                                 |  2 +-
 tests/tcg/arm/commpage.c                           | 61 +++++++++++++++++++
 .cirrus.yml                                        |  2 +-
 .shippable.yml                                     | 12 ++--
 .travis.yml                                        |  5 ++
 hw/virtio/trace-events                             |  3 +-
 scripts/clean-includes                             |  1 +
 scripts/coverity-scan/coverity-scan.docker         |  2 -
 tests/docker/Makefile.include                      |  2 +-
 tests/docker/dockerfiles/centos7.docker            |  2 -
 tests/docker/dockerfiles/centos8.docker            |  2 -
 .../docker/dockerfiles/debian-xtensa-cross.docker  |  2 -
 tests/docker/dockerfiles/debian10.docker           |  2 -
 tests/docker/dockerfiles/debian9.docker            |  2 -
 tests/docker/dockerfiles/fedora.docker             |  2 -
 tests/docker/dockerfiles/ubuntu.docker             |  4 +-
 tests/docker/dockerfiles/ubuntu1804.docker         |  2 +-
 tests/tcg/arm/Makefile.target                      |  2 +
 tests/vm/fedora                                    |  1 -
 tests/vm/freebsd                                   |  1 -
 tests/vm/netbsd                                    |  1 -
 tests/vm/openbsd                                   |  1 -
 tests/vm/ubuntu.i386                               |  2 +-
 28 files changed, 195 insertions(+), 60 deletions(-)
 create mode 100644 tests/tcg/arm/commpage.c

-- 
2.20.1



^ permalink raw reply	[flat|nested] 76+ messages in thread

* [PULL 01/17] qemu-plugin.h: add missing include <stddef.h> to define size_t
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
@ 2020-06-09 10:37 ` Alex Bennée
  2020-06-09 10:37 ` [PULL 02/17] scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header Alex Bennée
                   ` (16 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:37 UTC (permalink / raw)
  To: peter.maydell
  Cc: Emilio G. Cota, Alex Bennée, qemu-devel,
	Philippe Mathieu-Daudé

From: "Emilio G. Cota" <cota@braap.org>

Signed-off-by: Emilio G. Cota <cota@braap.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200524202427.951784-1-cota@braap.org>
Message-Id: <20200605154929.26910-2-alex.bennee@linaro.org>

diff --git a/include/qemu/qemu-plugin.h b/include/qemu/qemu-plugin.h
index 89ed579f559..bab8b0d4b3a 100644
--- a/include/qemu/qemu-plugin.h
+++ b/include/qemu/qemu-plugin.h
@@ -12,6 +12,7 @@
 
 #include <inttypes.h>
 #include <stdbool.h>
+#include <stddef.h>
 
 /*
  * For best performance, build the plugin with -fvisibility=hidden so that
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 02/17] scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
  2020-06-09 10:37 ` [PULL 01/17] qemu-plugin.h: add missing include <stddef.h> to define size_t Alex Bennée
@ 2020-06-09 10:37 ` Alex Bennée
  2020-06-09 10:37 ` [PULL 03/17] tests/plugin: correctly honour io_count Alex Bennée
                   ` (15 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:37 UTC (permalink / raw)
  To: peter.maydell
  Cc: Emilio G . Cota, Alex Bennée, qemu-devel,
	Philippe Mathieu-Daudé

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

"qemu/qemu-plugin.h" isn't meant to be include by QEMU codebase,
but by 3rd party plugins that QEMU can use. These plugins can be
built out of QEMU and don't include "qemu/osdep.h".
Mark "qemu/qemu-plugin.h" as a special header that doesn't need
to be cleaned for "qemu/osdep.h".

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200524215654.13256-1-f4bug@amsat.org>
Message-Id: <20200605154929.26910-3-alex.bennee@linaro.org>

diff --git a/scripts/clean-includes b/scripts/clean-includes
index dd938daa3ec..795b3bea318 100755
--- a/scripts/clean-includes
+++ b/scripts/clean-includes
@@ -123,6 +123,7 @@ for f in "$@"; do
       ;;
     *include/qemu/osdep.h | \
     *include/qemu/compiler.h | \
+    *include/qemu/qemu-plugin.h | \
     *include/glib-compat.h | \
     *include/sysemu/os-posix.h | \
     *include/sysemu/os-win32.h | \
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 03/17] tests/plugin: correctly honour io_count
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
  2020-06-09 10:37 ` [PULL 01/17] qemu-plugin.h: add missing include <stddef.h> to define size_t Alex Bennée
  2020-06-09 10:37 ` [PULL 02/17] scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header Alex Bennée
@ 2020-06-09 10:37 ` Alex Bennée
  2020-06-09 10:37 ` [PULL 04/17] exec: flush the whole TLB if a watchpoint crosses a page boundary Alex Bennée
                   ` (14 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:37 UTC (permalink / raw)
  To: peter.maydell; +Cc: Alex Bennée, qemu-devel, Philippe Mathieu-Daudé

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200605154929.26910-4-alex.bennee@linaro.org>

diff --git a/tests/plugin/mem.c b/tests/plugin/mem.c
index 878abf09d19..4725bd851d8 100644
--- a/tests/plugin/mem.c
+++ b/tests/plugin/mem.c
@@ -28,7 +28,7 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
 
     g_string_printf(out, "mem accesses: %" PRIu64 "\n", mem_count);
     if (do_haddr) {
-        g_string_append_printf(out, "io accesses: %" PRIu64 "\n", mem_count);
+        g_string_append_printf(out, "io accesses: %" PRIu64 "\n", io_count);
     }
     qemu_plugin_outs(out->str);
 }
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 04/17] exec: flush the whole TLB if a watchpoint crosses a page boundary
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (2 preceding siblings ...)
  2020-06-09 10:37 ` [PULL 03/17] tests/plugin: correctly honour io_count Alex Bennée
@ 2020-06-09 10:37 ` Alex Bennée
  2020-06-09 10:37 ` [PULL 05/17] .travis.yml: allow failure for unreliable hosts Alex Bennée
                   ` (13 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:37 UTC (permalink / raw)
  To: peter.maydell
  Cc: Richard Henderson, qemu-devel, Alexander Bulekov, Paolo Bonzini,
	Alex Bennée, Richard Henderson

There is no particular reason why you can't have a watchpoint in TCG
that covers a large chunk of the address space. We could be clever
about it but these cases are pretty rare and we can assume the user
will expect a little performance degradation.

NB: In my testing gdb will silently squash a watchpoint like:

  watch (char[0x7fffffffff]) *0x0

to a 4 byte watchpoint. Practically it will limit the maximum size
based on max-value-size. However given enough of a tweak the sky is
the limit.

Reported-by: Alexander Bulekov <alxndr@bu.edu>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200605154929.26910-5-alex.bennee@linaro.org>

diff --git a/exec.c b/exec.c
index be4be2df3a1..a0bf9d61c87 100644
--- a/exec.c
+++ b/exec.c
@@ -1038,6 +1038,7 @@ int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
                           int flags, CPUWatchpoint **watchpoint)
 {
     CPUWatchpoint *wp;
+    vaddr in_page;
 
     /* forbid ranges which are empty or run off the end of the address space */
     if (len == 0 || (addr + len - 1) < addr) {
@@ -1058,7 +1059,12 @@ int cpu_watchpoint_insert(CPUState *cpu, vaddr addr, vaddr len,
         QTAILQ_INSERT_TAIL(&cpu->watchpoints, wp, entry);
     }
 
-    tlb_flush_page(cpu, addr);
+    in_page = -(addr | TARGET_PAGE_MASK);
+    if (len <= in_page) {
+        tlb_flush_page(cpu, addr);
+    } else {
+        tlb_flush(cpu);
+    }
 
     if (watchpoint)
         *watchpoint = wp;
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 05/17] .travis.yml: allow failure for unreliable hosts
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (3 preceding siblings ...)
  2020-06-09 10:37 ` [PULL 04/17] exec: flush the whole TLB if a watchpoint crosses a page boundary Alex Bennée
@ 2020-06-09 10:37 ` Alex Bennée
  2020-06-09 10:37 ` [PULL 06/17] .shippable: temporaily disable some cross builds Alex Bennée
                   ` (12 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:37 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Philippe Mathieu-Daudé,
	Alex Bennée, qemu-devel, Philippe Mathieu-Daudé

They will still run but they won't get in the way of the result.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200605154929.26910-6-alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 564be50a3c1..ec6367af1f0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -429,6 +429,7 @@ jobs:
       env:
         - TEST_CMD="make check check-tcg V=1"
         - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS}"
+        - UNRELIABLE=true
 
     - name: "[ppc64] GCC check-tcg"
       arch: ppc64le
@@ -493,6 +494,7 @@ jobs:
       env:
         - TEST_CMD="make check check-tcg V=1"
         - CONFIG="--disable-containers --target-list=${MAIN_SOFTMMU_TARGETS},s390x-linux-user"
+        - UNRELIABLE=true
       script:
         - ( cd ${SRC_DIR} ; git submodule update --init roms/SLOF )
         - BUILD_RC=0 && make -j${JOBS} || BUILD_RC=$?
@@ -535,6 +537,7 @@ jobs:
         - TEST_CMD="make check-unit"
         - CONFIG="--disable-containers --disable-tcg --enable-kvm
                   --disable-tools --host-cc=clang --cxx=clang++"
+        - UNRELIABLE=true
 
     # Release builds
     # The make-release script expect a QEMU version, so our tag must start with a 'v'.
@@ -556,3 +559,5 @@ jobs:
         - mkdir -p release-build && cd release-build
         - ../configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; }
         - make install
+  allow_failures:
+    - env: UNRELIABLE=true
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 06/17] .shippable: temporaily disable some cross builds
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (4 preceding siblings ...)
  2020-06-09 10:37 ` [PULL 05/17] .travis.yml: allow failure for unreliable hosts Alex Bennée
@ 2020-06-09 10:37 ` Alex Bennée
  2020-06-09 10:37 ` [PULL 07/17] tests/docker: fix pre-requisite for debian-tricore-cross Alex Bennée
                   ` (11 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:37 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Philippe Mathieu-Daudé,
	Alex Bennée, qemu-devel, Philippe Mathieu-Daudé

These currently fail due to Debian bug #960271 as the
linux-libc-library has a user-space build breaking symbol in it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200605154929.26910-7-alex.bennee@linaro.org>

diff --git a/.shippable.yml b/.shippable.yml
index 2cce7b56890..10cf219bff4 100644
--- a/.shippable.yml
+++ b/.shippable.yml
@@ -5,8 +5,8 @@ env:
   global:
     - LC_ALL=C
   matrix:
-    - IMAGE=debian-amd64
-      TARGET_LIST=x86_64-softmmu,x86_64-linux-user
+    # - IMAGE=debian-amd64
+    #   TARGET_LIST=x86_64-softmmu,x86_64-linux-user
     - IMAGE=debian-win32-cross
       TARGET_LIST=arm-softmmu,i386-softmmu,lm32-softmmu
     - IMAGE=debian-win64-cross
@@ -19,10 +19,10 @@ env:
       TARGET_LIST=aarch64-softmmu,aarch64-linux-user
     - IMAGE=debian-s390x-cross
       TARGET_LIST=s390x-softmmu,s390x-linux-user
-    - IMAGE=debian-mips-cross
-      TARGET_LIST=mips-softmmu,mipsel-linux-user
-    - IMAGE=debian-mips64el-cross
-      TARGET_LIST=mips64el-softmmu,mips64el-linux-user
+    # - IMAGE=debian-mips-cross
+    #   TARGET_LIST=mips-softmmu,mipsel-linux-user
+    # - IMAGE=debian-mips64el-cross
+    #   TARGET_LIST=mips64el-softmmu,mips64el-linux-user
     - IMAGE=debian-ppc64el-cross
       TARGET_LIST=ppc64-softmmu,ppc64-linux-user,ppc64abi32-linux-user
 build:
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 07/17] tests/docker: fix pre-requisite for debian-tricore-cross
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (5 preceding siblings ...)
  2020-06-09 10:37 ` [PULL 06/17] .shippable: temporaily disable some cross builds Alex Bennée
@ 2020-06-09 10:37 ` Alex Bennée
  2020-06-09 10:38 ` [PULL 08/17] docker: update Ubuntu to 20.04 Alex Bennée
                   ` (10 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:37 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Bastian Koppelmann, Alex Bennée, qemu-devel,
	Philippe Mathieu-Daudé

Reported-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200605154929.26910-9-alex.bennee@linaro.org>

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ed46bd98eb5..981b7fcf2a5 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -130,7 +130,7 @@ docker-image-debian-sparc64-cross: docker-image-debian10
 docker-image-travis: NOUSER=1
 
 # Specialist build images, sometimes very limited tools
-docker-image-tricore-cross: docker-image-debian9
+docker-image-debian-tricore-cross: docker-image-debian9
 docker-image-debian-arm64-test-cross: docker-image-debian11
 
 # These images may be good enough for building tests but not for test builds
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 08/17] docker: update Ubuntu to 20.04
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (6 preceding siblings ...)
  2020-06-09 10:37 ` [PULL 07/17] tests/docker: fix pre-requisite for debian-tricore-cross Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:38 ` [PULL 09/17] hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE Alex Bennée
                   ` (9 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell
  Cc: Alex Bennée, Paolo Bonzini, Fam Zheng,
	Philippe Mathieu-Daudé,
	qemu-devel

From: Paolo Bonzini <pbonzini@redhat.com>

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200604231716.11354-1-pbonzini@redhat.com>
Message-Id: <20200605154929.26910-10-alex.bennee@linaro.org>

diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index eeb3b22bf20..43872417dec 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -9,7 +9,7 @@
 # system won't pick up that it has changed.
 #
 
-FROM ubuntu:19.04
+FROM ubuntu:20.04
 ENV PACKAGES flex bison \
     ccache \
     clang \
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 09/17] hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (7 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 08/17] docker: update Ubuntu to 20.04 Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:38 ` [PULL 10/17] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
                   ` (8 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell
  Cc: Michael S . Tsirkin, Fabiano Rosas, qemu-devel,
	Dr . David Alan Gilbert, Stefan Hajnoczi, Alex Bennée

The purpose of vhost_section is to identify RAM regions that need to
be made available to a vhost client. However when running under TCG
all RAM sections have DIRTY_MEMORY_CODE set which leads to problems
down the line.

Re-factor the code so:

  - steps are clearer to follow
  - reason for rejection is recorded in the trace point
  - we allow DIRTY_MEMORY_CODE

We expand the comment to explain that kernel based vhost has specific
support for migration tracking.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Fabiano Rosas <farosas@linux.ibm.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20200605154929.26910-11-alex.bennee@linaro.org>

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index aff98a0ede5..e3e21812905 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -27,6 +27,7 @@
 #include "migration/blocker.h"
 #include "migration/qemu-file-types.h"
 #include "sysemu/dma.h"
+#include "sysemu/tcg.h"
 #include "trace.h"
 
 /* enabled until disconnected backend stabilizes */
@@ -403,26 +404,50 @@ static int vhost_verify_ring_mappings(struct vhost_dev *dev,
     return r;
 }
 
+/*
+ * vhost_section: identify sections needed for vhost access
+ *
+ * We only care about RAM sections here (where virtqueue and guest
+ * internals accessed by virtio might live). If we find one we still
+ * allow the backend to potentially filter it out of our list.
+ */
 static bool vhost_section(struct vhost_dev *dev, MemoryRegionSection *section)
 {
-    bool result;
-    bool log_dirty = memory_region_get_dirty_log_mask(section->mr) &
-                     ~(1 << DIRTY_MEMORY_MIGRATION);
-    result = memory_region_is_ram(section->mr) &&
-        !memory_region_is_rom(section->mr);
-
-    /* Vhost doesn't handle any block which is doing dirty-tracking other
-     * than migration; this typically fires on VGA areas.
-     */
-    result &= !log_dirty;
+    MemoryRegion *mr = section->mr;
+
+    if (memory_region_is_ram(mr) && !memory_region_is_rom(mr)) {
+        uint8_t dirty_mask = memory_region_get_dirty_log_mask(mr);
+        uint8_t handled_dirty;
+
+        /*
+         * Kernel based vhost doesn't handle any block which is doing
+         * dirty-tracking other than migration for which it has
+         * specific logging support. However for TCG the kernel never
+         * gets involved anyway so we can also ignore it's
+         * self-modiying code detection flags. However a vhost-user
+         * client could still confuse a TCG guest if it re-writes
+         * executable memory that has already been translated.
+         */
+        handled_dirty = (1 << DIRTY_MEMORY_MIGRATION) |
+            (1 << DIRTY_MEMORY_CODE);
 
-    if (result && dev->vhost_ops->vhost_backend_mem_section_filter) {
-        result &=
-            dev->vhost_ops->vhost_backend_mem_section_filter(dev, section);
-    }
+        if (dirty_mask & ~handled_dirty) {
+            trace_vhost_reject_section(mr->name, 1);
+            return false;
+        }
+
+        if (dev->vhost_ops->vhost_backend_mem_section_filter &&
+            !dev->vhost_ops->vhost_backend_mem_section_filter(dev, section)) {
+            trace_vhost_reject_section(mr->name, 2);
+            return false;
+        }
 
-    trace_vhost_section(section->mr->name, result);
-    return result;
+        trace_vhost_section(mr->name);
+        return true;
+    } else {
+        trace_vhost_reject_section(mr->name, 3);
+        return false;
+    }
 }
 
 static void vhost_begin(MemoryListener *listener)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index e83500bee92..6427a0047df 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -5,7 +5,8 @@ vhost_commit(bool started, bool changed) "Started: %d Changed: %d"
 vhost_region_add_section(const char *name, uint64_t gpa, uint64_t size, uint64_t host) "%s: 0x%"PRIx64"+0x%"PRIx64" @ 0x%"PRIx64
 vhost_region_add_section_merge(const char *name, uint64_t new_size, uint64_t gpa, uint64_t owr) "%s: size: 0x%"PRIx64 " gpa: 0x%"PRIx64 " owr: 0x%"PRIx64
 vhost_region_add_section_aligned(const char *name, uint64_t gpa, uint64_t size, uint64_t host) "%s: 0x%"PRIx64"+0x%"PRIx64" @ 0x%"PRIx64
-vhost_section(const char *name, int r) "%s:%d"
+vhost_section(const char *name) "%s"
+vhost_reject_section(const char *name, int d) "%s:%d"
 vhost_iotlb_miss(void *dev, int step) "%p step %d"
 
 # vhost-user.c
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 10/17] linux-user: provide fallback pgd_find_hole for bare chroots
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (8 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 09/17] hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:38 ` [PULL 11/17] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
                   ` (7 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell; +Cc: Riku Voipio, Alex Bennée, qemu-devel, Laurent Vivier

When running QEMU out of a chroot environment we may not have access
to /proc/self/maps. As there is no other "official" way to introspect
our memory map we need to fall back to the original technique of
repeatedly trying to mmap an address range until we find one that
works.

Fortunately it's not quite as ugly as the original code given we
already re-factored the complications of dealing with the
ARM_COMMPAGE. We do make an attempt to skip over brk() which is about
the only concrete piece of information we have about the address map
at this moment.

Fixes: ee9474303
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200605154929.26910-12-alex.bennee@linaro.org>

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index ebc663ea0b3..475d243f3bd 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2101,6 +2101,50 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
     }
 }
 
+/**
+ * pgd_find_hole_fallback: potential mmap address
+ * @guest_size: size of available space
+ * @brk: location of break
+ * @align: memory alignment
+ *
+ * This is a fallback method for finding a hole in the host address
+ * space if we don't have the benefit of being able to access
+ * /proc/self/map. It can potentially take a very long time as we can
+ * only dumbly iterate up the host address space seeing if the
+ * allocation would work.
+ */
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)
+{
+    uintptr_t base;
+
+    /* Start (aligned) at the bottom and work our way up */
+    base = ROUND_UP(mmap_min_addr, align);
+
+    while (true) {
+        uintptr_t align_start, end;
+        align_start = ROUND_UP(base, align);
+        end = align_start + guest_size;
+
+        /* if brk is anywhere in the range give ourselves some room to grow. */
+        if (align_start <= brk && brk < end) {
+            base = brk + (16 * MiB);
+            continue;
+        } else if (align_start + guest_size < align_start) {
+            /* we have run out of space */
+            return -1;
+        } else {
+            int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE | MAP_FIXED;
+            void * mmap_start = mmap((void *) align_start, guest_size,
+                                     PROT_NONE, flags, -1, 0);
+            if (mmap_start != MAP_FAILED) {
+                munmap((void *) align_start, guest_size);
+                return (uintptr_t) mmap_start;
+            }
+            base += qemu_host_page_size;
+        }
+    }
+}
+
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
                                long align)
@@ -2116,6 +2160,10 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
     /* Read brk after we've read the maps, which will malloc. */
     brk = (uintptr_t)sbrk(0);
 
+    if (!maps) {
+        return pgd_find_hole_fallback(guest_size, brk, align);
+    }
+
     /* The first hole is before the first map entry. */
     this_start = mmap_min_addr;
 
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 11/17] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (9 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 10/17] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:38   ` [Bug 1880225] " Alex Bennée
  2020-06-09 10:38 ` [PULL 12/17] tests/tcg: add simple commpage test case Alex Bennée
                   ` (6 subsequent siblings)
  17 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell
  Cc: Bug 1880225, Riku Voipio, Richard Henderson, qemu-devel,
	Laurent Vivier, Aleksandar Markovic, Alex Bennée

We rely on the pointer to wrap when accessing the high address of the
COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
cannot afford just to map the entire 4gb address range. The old mmap
trial and error code handled this by just checking we could map both
the guest_base and the computed COMMPAGE address.

We can't just manipulate loadaddr to get what we want so we introduce
an offset which pgb_find_hole can apply when looking for a gap for
guest_base that ensures there is space left to map the COMMPAGE
afterwards.

This is arguably a little inefficient for the one 32 bit
value (kuser_helper_version) we need to keep there given all the
actual code entries are picked up during the translation phase.

Fixes: ee94743034b
Bug: https://bugs.launchpad.net/qemu/+bug/1880225
Cc: Bug 1880225 <1880225@bugs.launchpad.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200605154929.26910-13-alex.bennee@linaro.org>

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 475d243f3bd..b5cb21384a1 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -389,7 +389,7 @@ static bool init_guest_commpage(void)
 {
     void *want = g2h(ARM_COMMPAGE & -qemu_host_page_size);
     void *addr = mmap(want, qemu_host_page_size, PROT_READ | PROT_WRITE,
-                      MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+                      MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0);
 
     if (addr == MAP_FAILED) {
         perror("Allocating guest commpage");
@@ -2113,7 +2113,8 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
  * only dumbly iterate up the host address space seeing if the
  * allocation would work.
  */
-static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk,
+                                        long align, uintptr_t offset)
 {
     uintptr_t base;
 
@@ -2123,7 +2124,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
     while (true) {
         uintptr_t align_start, end;
         align_start = ROUND_UP(base, align);
-        end = align_start + guest_size;
+        end = align_start + guest_size + offset;
 
         /* if brk is anywhere in the range give ourselves some room to grow. */
         if (align_start <= brk && brk < end) {
@@ -2138,7 +2139,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
                                      PROT_NONE, flags, -1, 0);
             if (mmap_start != MAP_FAILED) {
                 munmap((void *) align_start, guest_size);
-                return (uintptr_t) mmap_start;
+                return (uintptr_t) mmap_start + offset;
             }
             base += qemu_host_page_size;
         }
@@ -2147,7 +2148,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
 
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
-                               long align)
+                               long align, uintptr_t offset)
 {
     GSList *maps, *iter;
     uintptr_t this_start, this_end, next_start, brk;
@@ -2161,7 +2162,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
     brk = (uintptr_t)sbrk(0);
 
     if (!maps) {
-        return pgd_find_hole_fallback(guest_size, brk, align);
+        return pgd_find_hole_fallback(guest_size, brk, align, offset);
     }
 
     /* The first hole is before the first map entry. */
@@ -2173,7 +2174,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
 
         this_end = ((MapInfo *)iter->data)->start;
         next_start = ((MapInfo *)iter->data)->end;
-        align_start = ROUND_UP(this_start, align);
+        align_start = ROUND_UP(this_start + offset, align);
 
         /* Skip holes that are too small. */
         if (align_start >= this_end) {
@@ -2223,6 +2224,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
 {
     uintptr_t loaddr = orig_loaddr;
     uintptr_t hiaddr = orig_hiaddr;
+    uintptr_t offset = 0;
     uintptr_t addr;
 
     if (hiaddr != orig_hiaddr) {
@@ -2236,18 +2238,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
     if (ARM_COMMPAGE) {
         /*
          * Extend the allocation to include the commpage.
-         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
-         * the address arithmetic will wrap around, but the difference
-         * will produce the correct allocation size.
+         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
+         * need to ensure there is space bellow the guest_base so we
+         * can map the commpage in the place needed when the address
+         * arithmetic wraps around.
          */
         if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
-            hiaddr = (uintptr_t)4 << 30;
+            hiaddr = (uintptr_t) 4 << 30;
         } else {
-            loaddr = ARM_COMMPAGE & -align;
+            offset = -(ARM_COMMPAGE & -align);
         }
     }
 
-    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
+    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
     if (addr == -1) {
         /*
          * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
@@ -2282,7 +2285,7 @@ static void pgb_dynamic(const char *image_name, long align)
          * just above that, and maximises the positive guest addresses.
          */
         commpage = ARM_COMMPAGE & -align;
-        addr = pgb_find_hole(commpage, -commpage, align);
+        addr = pgb_find_hole(commpage, -commpage, align, 0);
         assert(addr != -1);
         guest_base = addr;
     }
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [Bug 1880225] [PULL 11/17] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit
  2020-06-09 10:38 ` [PULL 11/17] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
@ 2020-06-09 10:38   ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: qemu-devel

We rely on the pointer to wrap when accessing the high address of the
COMMPAGE so it lands somewhere reasonable. However on 32 bit hosts we
cannot afford just to map the entire 4gb address range. The old mmap
trial and error code handled this by just checking we could map both
the guest_base and the computed COMMPAGE address.

We can't just manipulate loadaddr to get what we want so we introduce
an offset which pgb_find_hole can apply when looking for a gap for
guest_base that ensures there is space left to map the COMMPAGE
afterwards.

This is arguably a little inefficient for the one 32 bit
value (kuser_helper_version) we need to keep there given all the
actual code entries are picked up during the translation phase.

Fixes: ee94743034b
Bug: https://bugs.launchpad.net/qemu/+bug/1880225
Cc: Bug 1880225 <1880225@bugs.launchpad.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200605154929.26910-13-alex.bennee@linaro.org>

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 475d243f3bd..b5cb21384a1 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -389,7 +389,7 @@ static bool init_guest_commpage(void)
 {
     void *want = g2h(ARM_COMMPAGE & -qemu_host_page_size);
     void *addr = mmap(want, qemu_host_page_size, PROT_READ | PROT_WRITE,
-                      MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+                      MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED, -1, 0);
 
     if (addr == MAP_FAILED) {
         perror("Allocating guest commpage");
@@ -2113,7 +2113,8 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
  * only dumbly iterate up the host address space seeing if the
  * allocation would work.
  */
-static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, long align)
+static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk,
+                                        long align, uintptr_t offset)
 {
     uintptr_t base;
 
@@ -2123,7 +2124,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
     while (true) {
         uintptr_t align_start, end;
         align_start = ROUND_UP(base, align);
-        end = align_start + guest_size;
+        end = align_start + guest_size + offset;
 
         /* if brk is anywhere in the range give ourselves some room to grow. */
         if (align_start <= brk && brk < end) {
@@ -2138,7 +2139,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
                                      PROT_NONE, flags, -1, 0);
             if (mmap_start != MAP_FAILED) {
                 munmap((void *) align_start, guest_size);
-                return (uintptr_t) mmap_start;
+                return (uintptr_t) mmap_start + offset;
             }
             base += qemu_host_page_size;
         }
@@ -2147,7 +2148,7 @@ static uintptr_t pgd_find_hole_fallback(uintptr_t guest_size, uintptr_t brk, lon
 
 /* Return value for guest_base, or -1 if no hole found. */
 static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
-                               long align)
+                               long align, uintptr_t offset)
 {
     GSList *maps, *iter;
     uintptr_t this_start, this_end, next_start, brk;
@@ -2161,7 +2162,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
     brk = (uintptr_t)sbrk(0);
 
     if (!maps) {
-        return pgd_find_hole_fallback(guest_size, brk, align);
+        return pgd_find_hole_fallback(guest_size, brk, align, offset);
     }
 
     /* The first hole is before the first map entry. */
@@ -2173,7 +2174,7 @@ static uintptr_t pgb_find_hole(uintptr_t guest_loaddr, uintptr_t guest_size,
 
         this_end = ((MapInfo *)iter->data)->start;
         next_start = ((MapInfo *)iter->data)->end;
-        align_start = ROUND_UP(this_start, align);
+        align_start = ROUND_UP(this_start + offset, align);
 
         /* Skip holes that are too small. */
         if (align_start >= this_end) {
@@ -2223,6 +2224,7 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
 {
     uintptr_t loaddr = orig_loaddr;
     uintptr_t hiaddr = orig_hiaddr;
+    uintptr_t offset = 0;
     uintptr_t addr;
 
     if (hiaddr != orig_hiaddr) {
@@ -2236,18 +2238,19 @@ static void pgb_static(const char *image_name, abi_ulong orig_loaddr,
     if (ARM_COMMPAGE) {
         /*
          * Extend the allocation to include the commpage.
-         * For a 64-bit host, this is just 4GiB; for a 32-bit host,
-         * the address arithmetic will wrap around, but the difference
-         * will produce the correct allocation size.
+         * For a 64-bit host, this is just 4GiB; for a 32-bit host we
+         * need to ensure there is space bellow the guest_base so we
+         * can map the commpage in the place needed when the address
+         * arithmetic wraps around.
          */
         if (sizeof(uintptr_t) == 8 || loaddr >= 0x80000000u) {
-            hiaddr = (uintptr_t)4 << 30;
+            hiaddr = (uintptr_t) 4 << 30;
         } else {
-            loaddr = ARM_COMMPAGE & -align;
+            offset = -(ARM_COMMPAGE & -align);
         }
     }
 
-    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align);
+    addr = pgb_find_hole(loaddr, hiaddr - loaddr, align, offset);
     if (addr == -1) {
         /*
          * If ARM_COMMPAGE, there *might* be a non-consecutive allocation
@@ -2282,7 +2285,7 @@ static void pgb_dynamic(const char *image_name, long align)
          * just above that, and maximises the positive guest addresses.
          */
         commpage = ARM_COMMPAGE & -align;
-        addr = pgb_find_hole(commpage, -commpage, align);
+        addr = pgb_find_hole(commpage, -commpage, align, 0);
         assert(addr != -1);
         guest_base = addr;
     }
-- 
2.20.1

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  In Progress

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 12/17] tests/tcg: add simple commpage test case
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (10 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 11/17] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:38 ` [PULL 13/17] linux-user: detect overflow of MAP_FIXED mmap Alex Bennée
                   ` (5 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell
  Cc: Richard Henderson, open list:ARM TCG CPUs, Alex Bennée, qemu-devel

The COMMPAGE are a number of kernel provided user-space routines for
32 bit ARM systems. Add a basic series of smoke tests to ensure it is
working as it should.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200605154929.26910-14-alex.bennee@linaro.org>

diff --git a/tests/tcg/arm/commpage.c b/tests/tcg/arm/commpage.c
new file mode 100644
index 00000000000..c76e70cb8bd
--- /dev/null
+++ b/tests/tcg/arm/commpage.c
@@ -0,0 +1,61 @@
+/*
+ * Verify the COMMPAGE emulation
+ *
+ * The ARM commpage is a set of user space helper functions provided
+ * by the kernel in an effort to ease portability of user space code
+ * between different CPUs with potentially different capabilities. It
+ * is a 32 bit invention and similar to the vdso segment in many ways.
+ *
+ * The ABI is documented in the Linux kernel:
+ *     Documentation/arm/kernel_userspace_helpers.rst
+ *
+ * Copyright (c) 2020 Linaro Ltd
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdint.h>
+
+#define ARM_COMMPAGE      (0xffff0f00u)
+#define ARM_KUSER_VERSION (*(int32_t *)(ARM_COMMPAGE + 0xfc))
+typedef void * (get_tls_fn)(void);
+#define ARM_KUSER_GET_TLS (*(get_tls_fn *)(ARM_COMMPAGE + 0xe0))
+typedef int (cmpxchg_fn)(int oldval, int newval, volatile int *ptr);
+#define ARM_KUSER_CMPXCHG (*(cmpxchg_fn *)(ARM_COMMPAGE + 0xc0))
+typedef void (dmb_fn)(void);
+#define ARM_KUSER_DMB (*(dmb_fn *)(ARM_COMMPAGE + 0xa0))
+typedef int (cmpxchg64_fn)(const int64_t *oldval,
+                           const int64_t *newval,
+                           volatile int64_t *ptr);
+#define ARM_KUSER_CMPXCHG64 (*(cmpxchg64_fn *)(ARM_COMMPAGE + 0x60))
+
+#define fail_unless(x)                                                  \
+    do {                                                                \
+        if (!(x)) {                                                     \
+            fprintf(stderr, "FAILED at %s:%d\n", __FILE__, __LINE__);   \
+            exit(EXIT_FAILURE);                                         \
+        }                                                               \
+    } while (0)
+
+
+int main(int argc, char *argv[argc])
+{
+    void *kuser_tls;
+    int val = 1;
+    const int64_t oldval = 1, newval = 2;
+    int64_t val64 = 1;
+
+    fail_unless(ARM_KUSER_VERSION == 0x5);
+    kuser_tls = ARM_KUSER_GET_TLS();
+    printf("TLS = %p\n", kuser_tls);
+    fail_unless(kuser_tls != 0);
+    fail_unless(ARM_KUSER_CMPXCHG(1, 2, &val) == 0);
+    printf("val = %d\n", val);
+    /* this is a crash test, not checking an actual barrier occurs */
+    ARM_KUSER_DMB();
+    fail_unless(ARM_KUSER_CMPXCHG64(&oldval, &newval, &val64) == 0);
+    printf("val64 = %lld\n", val64);
+    return 0;
+}
diff --git a/tests/tcg/arm/Makefile.target b/tests/tcg/arm/Makefile.target
index 11c39c601ea..3da09a38be7 100644
--- a/tests/tcg/arm/Makefile.target
+++ b/tests/tcg/arm/Makefile.target
@@ -68,6 +68,8 @@ run-semiconsole-arm: semiconsole-arm
 run-plugin-semiconsole-arm-with-%:
 	$(call skip-test, $<, "MANUAL ONLY")
 
+ARM_TESTS += commpage
+
 TESTS += $(ARM_TESTS)
 
 # On ARM Linux only supports 4k pages
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 13/17] linux-user: detect overflow of MAP_FIXED mmap
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (11 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 12/17] tests/tcg: add simple commpage test case Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:38 ` [PULL 14/17] tests/docker: Remove flex/bison packages Alex Bennée
                   ` (4 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell; +Cc: Riku Voipio, Alex Bennée, qemu-devel, Laurent Vivier

Relaxing the restrictions on 64 bit guests leads to the user being
able to attempt to map right at the edge of addressable memory. This
in turn lead to address overflow tripping the assert in page_set_flags
when the end address wrapped around.

Detect the wrap earlier and correctly -ENOMEM the guest (in the
reported case LTP mmap15).

Fixes: 7d8cbbabcb
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20200605154929.26910-15-alex.bennee@linaro.org>

diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index caab62909eb..0019447892e 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -467,7 +467,7 @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int prot,
          * It can fail only on 64-bit host with 32-bit target.
          * On any other target/host host mmap() handles this error correctly.
          */
-        if (!guest_range_valid(start, len)) {
+        if (end < start || !guest_range_valid(start, len)) {
             errno = ENOMEM;
             goto fail;
         }
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 14/17] tests/docker: Remove flex/bison packages
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (12 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 13/17] linux-user: detect overflow of MAP_FIXED mmap Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:38 ` [PULL 15/17] tests/vm: " Alex Bennée
                   ` (3 subsequent siblings)
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Claudio Fontana

From: Philippe Mathieu-Daudé <philmd@redhat.com>

QEMU does not use flex/bison packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200515163029.12917-2-philmd@redhat.com>

diff --git a/tests/docker/dockerfiles/centos7.docker b/tests/docker/dockerfiles/centos7.docker
index 9a2a2e515d7..e197acdc3ce 100644
--- a/tests/docker/dockerfiles/centos7.docker
+++ b/tests/docker/dockerfiles/centos7.docker
@@ -5,13 +5,11 @@ RUN yum -y update
 
 # Please keep this list sorted alphabetically
 ENV PACKAGES \
-    bison \
     bzip2 \
     bzip2-devel \
     ccache \
     csnappy-devel \
     dbus-daemon \
-    flex \
     gcc-c++ \
     gcc \
     gettext \
diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker
index bfa0d33c9c8..9852c5b9eea 100644
--- a/tests/docker/dockerfiles/centos8.docker
+++ b/tests/docker/dockerfiles/centos8.docker
@@ -3,11 +3,9 @@ FROM centos:8.1.1911
 RUN dnf -y update
 ENV PACKAGES \
     SDL-devel \
-    bison \
     bzip2 \
     bzip2-devel \
     dbus-daemon \
-    flex \
     gcc \
     gcc-c++ \
     gettext \
diff --git a/tests/docker/dockerfiles/debian-xtensa-cross.docker b/tests/docker/dockerfiles/debian-xtensa-cross.docker
index e6f93f65ee2..beb73f46baa 100644
--- a/tests/docker/dockerfiles/debian-xtensa-cross.docker
+++ b/tests/docker/dockerfiles/debian-xtensa-cross.docker
@@ -11,11 +11,9 @@ RUN apt-get update && \
     DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
     apt-get install -y --no-install-recommends \
-        bison \
         build-essential \
         ca-certificates \
         curl \
-        flex \
         gettext \
         git \
         python3-minimal
diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
index 0769700a416..bcdff04ddfe 100644
--- a/tests/docker/dockerfiles/debian10.docker
+++ b/tests/docker/dockerfiles/debian10.docker
@@ -18,12 +18,10 @@ RUN apt update && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
     apt install -y --no-install-recommends \
         bc \
-        bison \
         build-essential \
         ca-certificates \
         clang \
         dbus \
-        flex \
         gdb-multiarch \
         gettext \
         git \
diff --git a/tests/docker/dockerfiles/debian9.docker b/tests/docker/dockerfiles/debian9.docker
index 08cc970feb1..0f0ebe530af 100644
--- a/tests/docker/dockerfiles/debian9.docker
+++ b/tests/docker/dockerfiles/debian9.docker
@@ -18,11 +18,9 @@ RUN apt update && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
     apt install -y --no-install-recommends \
         bc \
-        bison \
         build-essential \
         ca-certificates \
         clang \
-        flex \
         gdb-multiarch \
         gettext \
         git \
diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 179575ecaaa..92b6e11c8a8 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -3,7 +3,6 @@ FROM fedora:30
 # Please keep this list sorted alphabetically
 ENV PACKAGES \
     bc \
-    bison \
     brlapi-devel \
     bzip2 \
     bzip2-devel \
@@ -13,7 +12,6 @@ ENV PACKAGES \
     dbus-daemon \
     device-mapper-multipath-devel \
     findutils \
-    flex \
     gcc \
     gcc-c++ \
     gettext \
diff --git a/tests/docker/dockerfiles/ubuntu.docker b/tests/docker/dockerfiles/ubuntu.docker
index 43872417dec..161806e6b8c 100644
--- a/tests/docker/dockerfiles/ubuntu.docker
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -10,7 +10,7 @@
 #
 
 FROM ubuntu:20.04
-ENV PACKAGES flex bison \
+ENV PACKAGES \
     ccache \
     clang \
     dbus \
diff --git a/tests/docker/dockerfiles/ubuntu1804.docker b/tests/docker/dockerfiles/ubuntu1804.docker
index f66b06f4cff..a10ea2850b6 100644
--- a/tests/docker/dockerfiles/ubuntu1804.docker
+++ b/tests/docker/dockerfiles/ubuntu1804.docker
@@ -1,5 +1,5 @@
 FROM ubuntu:18.04
-ENV PACKAGES flex bison \
+ENV PACKAGES \
     ccache \
     clang \
     gcc \
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 15/17] tests/vm: Remove flex/bison packages
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (13 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 14/17] tests/docker: Remove flex/bison packages Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:53   ` Claudio Fontana
  2020-06-09 10:38 ` [PULL 16/17] cirrus-ci: " Alex Bennée
                   ` (2 subsequent siblings)
  17 siblings, 1 reply; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell
  Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Claudio Fontana

From: Philippe Mathieu-Daudé <philmd@redhat.com>

QEMU does not use flex/bison packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200515163029.12917-3-philmd@redhat.com>

diff --git a/tests/vm/fedora b/tests/vm/fedora
index bd9c6cf295c..a9195670f4b 100755
--- a/tests/vm/fedora
+++ b/tests/vm/fedora
@@ -32,7 +32,6 @@ class FedoraVM(basevm.BaseVM):
     pkgs = [
         # tools
         'git-core',
-        'flex', 'bison',
         'gcc', 'binutils', 'make',
 
         # perl
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 298967fe9cf..f87db2b126e 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -38,7 +38,6 @@ class FreeBSDVM(basevm.BaseVM):
         "bash",
         "gmake",
         "gsed",
-        "flex", "bison",
 
         # libs: crypto
         "gnutls",
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index b10c9d429de..cdac502dad8 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -36,7 +36,6 @@ class NetBSDVM(basevm.BaseVM):
         "bash",
         "gmake",
         "gsed",
-        "flex", "bison",
 
         # libs: crypto
         "gnutls",
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 0b705f49452..13e7f9a6d56 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -35,7 +35,6 @@ class OpenBSDVM(basevm.BaseVM):
         "bash",
         "gmake",
         "gsed",
-        "bison",
 
         # libs: usb
         "libusb1",
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 15707753353..24527cc78c7 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -52,7 +52,7 @@ class UbuntuX86VM(basevm.BaseVM):
         self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list")
         self.ssh_root_check("apt-get update")
         self.ssh_root_check("apt-get build-dep -y qemu")
-        self.ssh_root_check("apt-get install -y libfdt-dev flex bison language-pack-en")
+        self.ssh_root_check("apt-get install -y libfdt-dev language-pack-en")
         self.ssh_root("poweroff")
         self.wait()
         os.rename(img_tmp, img)
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 16/17] cirrus-ci: Remove flex/bison packages
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (14 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 15/17] tests/vm: " Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-09 10:38 ` [PULL 17/17] scripts/coverity-scan: " Alex Bennée
  2020-06-11 10:20 ` [PULL 00/17] testing and misc fixes Peter Maydell
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell
  Cc: Alex Bennée, Ed Maste, Philippe Mathieu-Daudé,
	qemu-devel, Li-Wen Hsu

From: Philippe Mathieu-Daudé <philmd@redhat.com>

QEMU does not use flex/bison packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Li-Wen Hsu <lwhsu@freebsd.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200515163029.12917-5-philmd@redhat.com>

diff --git a/.cirrus.yml b/.cirrus.yml
index de0727cb097..ce7850a320e 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -7,7 +7,7 @@ freebsd_12_task:
     cpu: 8
     memory: 8G
   install_script: ASSUME_ALWAYS_YES=yes pkg bootstrap -f ; pkg install -y
-    bash bison curl cyrus-sasl git glib gmake gnutls gsed
+    bash curl cyrus-sasl git glib gmake gnutls gsed
     nettle perl5 pixman pkgconf png usbredir
   script:
     - mkdir build
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* [PULL 17/17] scripts/coverity-scan: Remove flex/bison packages
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (15 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 16/17] cirrus-ci: " Alex Bennée
@ 2020-06-09 10:38 ` Alex Bennée
  2020-06-11 10:20 ` [PULL 00/17] testing and misc fixes Peter Maydell
  17 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 10:38 UTC (permalink / raw)
  To: peter.maydell
  Cc: Alex Bennée, Philippe Mathieu-Daudé,
	qemu-devel, Claudio Fontana

From: Philippe Mathieu-Daudé <philmd@redhat.com>

QEMU does not use flex/bison packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200515163029.12917-6-philmd@redhat.com>

diff --git a/scripts/coverity-scan/coverity-scan.docker b/scripts/coverity-scan/coverity-scan.docker
index a4f64d12834..ad4d64c0f81 100644
--- a/scripts/coverity-scan/coverity-scan.docker
+++ b/scripts/coverity-scan/coverity-scan.docker
@@ -19,7 +19,6 @@ FROM fedora:30
 ENV PACKAGES \
     alsa-lib-devel \
     bc \
-    bison \
     brlapi-devel \
     bzip2 \
     bzip2-devel \
@@ -30,7 +29,6 @@ ENV PACKAGES \
     dbus-daemon \
     device-mapper-multipath-devel \
     findutils \
-    flex \
     gcc \
     gcc-c++ \
     gettext \
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 76+ messages in thread

* Re: [PULL 15/17] tests/vm: Remove flex/bison packages
  2020-06-09 10:38 ` [PULL 15/17] tests/vm: " Alex Bennée
@ 2020-06-09 10:53   ` Claudio Fontana
  2020-06-09 11:08     ` Alex Bennée
  0 siblings, 1 reply; 76+ messages in thread
From: Claudio Fontana @ 2020-06-09 10:53 UTC (permalink / raw)
  To: Alex Bennée, peter.maydell
  Cc: Fam Zheng, Philippe Mathieu-Daudé, qemu-devel, David Gibson

This series incidentally removes the build warnings when not having flex/bison,

could someone queue it?

Thanks,

Claudio

https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg01851.html


On 6/9/20 12:38 PM, Alex Bennée wrote:
> From: Philippe Mathieu-Daudé <philmd@redhat.com>
> 
> QEMU does not use flex/bison packages.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Reviewed-by: Claudio Fontana <cfontana@suse.de>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200515163029.12917-3-philmd@redhat.com>
> 
> diff --git a/tests/vm/fedora b/tests/vm/fedora
> index bd9c6cf295c..a9195670f4b 100755
> --- a/tests/vm/fedora
> +++ b/tests/vm/fedora
> @@ -32,7 +32,6 @@ class FedoraVM(basevm.BaseVM):
>      pkgs = [
>          # tools
>          'git-core',
> -        'flex', 'bison',
>          'gcc', 'binutils', 'make',
>  
>          # perl
> diff --git a/tests/vm/freebsd b/tests/vm/freebsd
> index 298967fe9cf..f87db2b126e 100755
> --- a/tests/vm/freebsd
> +++ b/tests/vm/freebsd
> @@ -38,7 +38,6 @@ class FreeBSDVM(basevm.BaseVM):
>          "bash",
>          "gmake",
>          "gsed",
> -        "flex", "bison",
>  
>          # libs: crypto
>          "gnutls",
> diff --git a/tests/vm/netbsd b/tests/vm/netbsd
> index b10c9d429de..cdac502dad8 100755
> --- a/tests/vm/netbsd
> +++ b/tests/vm/netbsd
> @@ -36,7 +36,6 @@ class NetBSDVM(basevm.BaseVM):
>          "bash",
>          "gmake",
>          "gsed",
> -        "flex", "bison",
>  
>          # libs: crypto
>          "gnutls",
> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> index 0b705f49452..13e7f9a6d56 100755
> --- a/tests/vm/openbsd
> +++ b/tests/vm/openbsd
> @@ -35,7 +35,6 @@ class OpenBSDVM(basevm.BaseVM):
>          "bash",
>          "gmake",
>          "gsed",
> -        "bison",
>  
>          # libs: usb
>          "libusb1",
> diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
> index 15707753353..24527cc78c7 100755
> --- a/tests/vm/ubuntu.i386
> +++ b/tests/vm/ubuntu.i386
> @@ -52,7 +52,7 @@ class UbuntuX86VM(basevm.BaseVM):
>          self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list")
>          self.ssh_root_check("apt-get update")
>          self.ssh_root_check("apt-get build-dep -y qemu")
> -        self.ssh_root_check("apt-get install -y libfdt-dev flex bison language-pack-en")
> +        self.ssh_root_check("apt-get install -y libfdt-dev language-pack-en")
>          self.ssh_root("poweroff")
>          self.wait()
>          os.rename(img_tmp, img)
> 



^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PULL 15/17] tests/vm: Remove flex/bison packages
  2020-06-09 10:53   ` Claudio Fontana
@ 2020-06-09 11:08     ` Alex Bennée
  0 siblings, 0 replies; 76+ messages in thread
From: Alex Bennée @ 2020-06-09 11:08 UTC (permalink / raw)
  To: Claudio Fontana
  Cc: Fam Zheng, peter.maydell, Philippe Mathieu-Daudé,
	qemu-devel, David Gibson


Claudio Fontana <cfontana@suse.de> writes:

> This series incidentally removes the build warnings when not having flex/bison,
>
> could someone queue it?
>
> Thanks,
>
> Claudio
>
> https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg01851.html

If it is all reviewed I can pick it up in the next testing/next bits if
no one else cares.

>
>
> On 6/9/20 12:38 PM, Alex Bennée wrote:
>> From: Philippe Mathieu-Daudé <philmd@redhat.com>
>> 
>> QEMU does not use flex/bison packages.
>> 
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> Reviewed-by: Claudio Fontana <cfontana@suse.de>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Message-Id: <20200515163029.12917-3-philmd@redhat.com>
>> 
>> diff --git a/tests/vm/fedora b/tests/vm/fedora
>> index bd9c6cf295c..a9195670f4b 100755
>> --- a/tests/vm/fedora
>> +++ b/tests/vm/fedora
>> @@ -32,7 +32,6 @@ class FedoraVM(basevm.BaseVM):
>>      pkgs = [
>>          # tools
>>          'git-core',
>> -        'flex', 'bison',
>>          'gcc', 'binutils', 'make',
>>  
>>          # perl
>> diff --git a/tests/vm/freebsd b/tests/vm/freebsd
>> index 298967fe9cf..f87db2b126e 100755
>> --- a/tests/vm/freebsd
>> +++ b/tests/vm/freebsd
>> @@ -38,7 +38,6 @@ class FreeBSDVM(basevm.BaseVM):
>>          "bash",
>>          "gmake",
>>          "gsed",
>> -        "flex", "bison",
>>  
>>          # libs: crypto
>>          "gnutls",
>> diff --git a/tests/vm/netbsd b/tests/vm/netbsd
>> index b10c9d429de..cdac502dad8 100755
>> --- a/tests/vm/netbsd
>> +++ b/tests/vm/netbsd
>> @@ -36,7 +36,6 @@ class NetBSDVM(basevm.BaseVM):
>>          "bash",
>>          "gmake",
>>          "gsed",
>> -        "flex", "bison",
>>  
>>          # libs: crypto
>>          "gnutls",
>> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
>> index 0b705f49452..13e7f9a6d56 100755
>> --- a/tests/vm/openbsd
>> +++ b/tests/vm/openbsd
>> @@ -35,7 +35,6 @@ class OpenBSDVM(basevm.BaseVM):
>>          "bash",
>>          "gmake",
>>          "gsed",
>> -        "bison",
>>  
>>          # libs: usb
>>          "libusb1",
>> diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
>> index 15707753353..24527cc78c7 100755
>> --- a/tests/vm/ubuntu.i386
>> +++ b/tests/vm/ubuntu.i386
>> @@ -52,7 +52,7 @@ class UbuntuX86VM(basevm.BaseVM):
>>          self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list")
>>          self.ssh_root_check("apt-get update")
>>          self.ssh_root_check("apt-get build-dep -y qemu")
>> -        self.ssh_root_check("apt-get install -y libfdt-dev flex bison language-pack-en")
>> +        self.ssh_root_check("apt-get install -y libfdt-dev language-pack-en")
>>          self.ssh_root("poweroff")
>>          self.wait()
>>          os.rename(img_tmp, img)
>> 


-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 76+ messages in thread

* Re: [PULL 00/17] testing and misc fixes
  2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
                   ` (16 preceding siblings ...)
  2020-06-09 10:38 ` [PULL 17/17] scripts/coverity-scan: " Alex Bennée
@ 2020-06-11 10:20 ` Peter Maydell
  17 siblings, 0 replies; 76+ messages in thread
From: Peter Maydell @ 2020-06-11 10:20 UTC (permalink / raw)
  To: Alex Bennée; +Cc: QEMU Developers

On Tue, 9 Jun 2020 at 11:38, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 49ee11555262a256afec592dfed7c5902d5eefd2:
>
>   Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-5.1-pull-request' into staging (2020-06-08 11:04:57 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-testing-and-misc-080620-1
>
> for you to fetch changes up to a5b04ccd742f6c58a1d305530d9e07ad9731b8e6:
>
>   scripts/coverity-scan: Remove flex/bison packages (2020-06-08 17:04:19 +0100)
>
> ----------------------------------------------------------------
> Various testing and misc fixes:
>
>   - header cleanups for plugins
>   - support wider watchpoints
>   - tweaks for unreliable and broken CI
>   - docker image fixes and verion bumps
>   - linux-user guest_base fixes
>   - remove flex/bison from various test images


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM


^ permalink raw reply	[flat|nested] 76+ messages in thread

* [Bug 1880225] Re: Emulation of some arm programs fail with "Assertion `have_guest_base' failed."
  2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
                   ` (6 preceding siblings ...)
  2020-05-23  7:52 ` Aleksandar Markovic
@ 2020-08-20 15:08 ` Thomas Huth
  7 siblings, 0 replies; 76+ messages in thread
From: Thomas Huth @ 2020-08-20 15:08 UTC (permalink / raw)
  To: qemu-devel

Fixed here:
https://git.qemu.org/?p=qemu.git;a=commitdiff;h=5c3e87f345ac93de9260f

** Changed in: qemu
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1880225

Title:
  Emulation of some arm programs fail with "Assertion `have_guest_base'
  failed."

Status in QEMU:
  Fix Released

Bug description:
  This issue is observer with QEMU ToT, checked out around May 15th (but
  I believe it is present in current master too), and wasn't present in
  QEMU v5.0.0.

  I am using 32-bit Intel(R) Pentium(R) M processor 1.73GHz host.

  Arm cross-compiler is a standard cross-compiler that comes with
  Debian-based distributions, and gcc version is:

  $ arm-linux-gnueabi-gcc --version
  arm-linux-gnueabi-gcc (Debian 8.3.0-2) 8.3.0

  Compile this program with cross compiler:

  $ arm-linux-gnueabi-gcc -O2 -static toupper_string.c -o
  toupper_string-arm

  Emulation with QEMU v5.0.0 is correct, and gives expected output:

  $ ~/Build/qemu-5.0.0/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  CONTROL RESULT: (toupper_string)
   nwlrbbmqbhcdarz owkkyhiddqscdxr jmowfrxsjybldbe fsarcbynecdyggx xpklorellnmpapq
   NWLRBBMQBHCDARZ OWKKYHIDDQSCDXR JMOWFRXSJYBLDBE FSARCBYNECDYGGX XPKLORELLNMPAPQ

  While, in case of QEMU master it fails:

  $ ~/Build/qemu-master/build-gcc/arm-linux-user/qemu-arm ./toupper_string-arm
  qemu-arm: /home/rtrk/Build/qemu-master/linux-user/elfload.c:2294: probe_guest_base: Assertion `have_guest_base' failed.
  Aborted

  There are many other programs that exibit the same behavior. The
  failure is arm-sprecific.

  
  -----------------------------------------------------

  source code: (let's call this file toupper_string.c) (similar file is
  also in attachment)

  
  #include <stdlib.h>
  #include <string.h>
  #include <stdio.h>
  #include <unistd.h>

  
  #define MAX_STRING_LENGHT              15
  #define NUMBER_OF_RANDOM_STRINGS       100
  #define DEFAULT_NUMBER_OF_REPETITIONS  30000
  #define MAX_NUMBER_OF_REPETITIONS      1000000000
  #define NUMBER_OF_CONTROL_PRINT_ITEMS  5

  /* Structure for keeping an array of strings */
  struct StringStruct {
      char chars[MAX_STRING_LENGHT + 1];
  };

  /**
   * Sets characters of the given string to random small letters a-z.
   * @param s String to get random characters.
   * @len Length of the input string.
   */
  static void gen_random_string(char *chars, const int len)
  {
      static const char letters[] = "abcdefghijklmnopqrstuvwxyz";

      for (size_t i = 0; i < len; i++) {
          chars[i] = letters[rand() % (sizeof(letters) - 1)];
      }
      chars[len] = 0;
  }

  void main (int argc, char* argv[])
  {
      struct StringStruct random_strings[NUMBER_OF_RANDOM_STRINGS];
      struct StringStruct strings_to_be_uppercased[NUMBER_OF_RANDOM_STRINGS];
      int32_t number_of_repetitions = DEFAULT_NUMBER_OF_REPETITIONS;
      int32_t option;

      /* Parse command line options */
      while ((option = getopt(argc, argv, "n:")) != -1) {
          if (option == 'n') {
              int32_t user_number_of_repetitions = atoi(optarg);
              /* Check if the value is a negative number */
              if (user_number_of_repetitions < 1) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be a "
                                  "negative number.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is a string or zero */
              if (user_number_of_repetitions == 0) {
                  fprintf(stderr, "Error ... Invalid value for option '-n'.\n");
                  exit(EXIT_FAILURE);
              }
              /* Check if the value is too large */
              if (user_number_of_repetitions > MAX_NUMBER_OF_REPETITIONS) {
                  fprintf(stderr, "Error ... Value for option '-n' cannot be "
                                  "more than %d.\n", MAX_NUMBER_OF_REPETITIONS);
                  exit(EXIT_FAILURE);
              }
              number_of_repetitions = user_number_of_repetitions;
          } else {
              exit(EXIT_FAILURE);
          }
      }

      /* Create an array of strings with random content */
      srand(1);
      for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
          gen_random_string(random_strings[i].chars, MAX_STRING_LENGHT);
      }

      /* Perform uppercasing of a set of random strings multiple times */
      for (size_t j = 0; j < number_of_repetitions; j++) {
          /* Copy initial set of random strings to the set to be uppercased */
          memcpy(strings_to_be_uppercased, random_strings,
                 NUMBER_OF_RANDOM_STRINGS * (MAX_STRING_LENGHT + 1));
          /* Do actual changing case to uppercase */
          for (size_t i = 0; i < NUMBER_OF_RANDOM_STRINGS; i++) {
              int k = 0;
    
              while (strings_to_be_uppercased[i].chars[k]) { 
                  char ch = strings_to_be_uppercased[i].chars[k] - 32; 
                  memcpy((void *)strings_to_be_uppercased[i].chars + k,
                         &ch, 1);
                  k++; 
              } 
          }
      }

      /* Control printing */
      printf("CONTROL RESULT: (toupper_string)\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", random_strings[i].chars);
      }
      printf("\n");
      for (size_t i = 0; i < NUMBER_OF_CONTROL_PRINT_ITEMS; i++) {
          printf(" %s", strings_to_be_uppercased[i].chars);
      }
      printf("\n");
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1880225/+subscriptions


^ permalink raw reply	[flat|nested] 76+ messages in thread

end of thread, other threads:[~2020-08-20 15:26 UTC | newest]

Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-22 18:43 [Bug 1880225] [NEW] Emulation of some arm programs fail with "Assertion `have_guest_base' failed." Aleksandar Markovic
2020-05-22 19:18 ` Alex Bennée
2020-05-22 19:18   ` Alex Bennée
2020-05-22 19:27 ` [Bug 1880225] " Alex Bennée
2020-05-23  1:07 ` Aleksandar Markovic
2020-05-23  1:14 ` Aleksandar Markovic
2020-05-23  7:40   ` Alex Bennée
2020-05-23  7:40     ` Alex Bennée
2020-05-23  1:31 ` Aleksandar Markovic
2020-05-23  7:50 ` Alex Bennée
2020-05-23  7:52 ` Aleksandar Markovic
2020-05-23 10:14   ` Alex Bennée
2020-08-20 15:08 ` Thomas Huth
2020-05-27 10:05 [PATCH v1 0/3] some linux-user guest_base fixes Alex Bennée
2020-05-27 10:05 ` [PATCH v1 1/3] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
2020-06-02  0:37   ` Richard Henderson
2020-05-27 10:05 ` [PATCH v1 2/3] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
2020-05-27 10:05   ` [Bug 1880225] " Alex Bennée
2020-05-27 12:05   ` Aleksandar Markovic
2020-05-27 12:05     ` [Bug 1880225] " Aleksandar Markovic
2020-05-27 14:47     ` Aleksandar Markovic
2020-05-27 14:47       ` [Bug 1880225] " Aleksandar Markovic
2020-05-27 16:14       ` Alex Bennée
2020-05-27 16:14         ` [Bug 1880225] " Alex Bennée
2020-06-02  0:28   ` Richard Henderson
2020-06-05  9:45     ` Alex Bennée
2020-06-05  9:45       ` [Bug 1880225] " Alex Bennée
2020-06-05 10:24       ` Alex Bennée
2020-06-05 10:24         ` [Bug 1880225] " Alex Bennée
2020-05-27 10:05 ` [PATCH v1 3/3] tests/tcg: add simple commpage test case Alex Bennée
2020-06-02  0:40   ` Richard Henderson
2020-05-27 14:12 ` [PATCH v1 0/3] some linux-user guest_base fixes no-reply
2020-06-05 15:49 [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Alex Bennée
2020-06-05 15:49 ` [PATCH v1 01/14] qemu-plugin.h: add missing include <stddef.h> to define size_t Alex Bennée
2020-06-05 15:49 ` [PATCH v1 02/14] scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header Alex Bennée
2020-06-05 15:49 ` [PATCH v1 03/14] tests/plugin: correctly honour io_count Alex Bennée
2020-06-05 15:49 ` [PATCH v1 04/14] exec: flush the whole TLB if a watchpoint crosses a page boundary Alex Bennée
2020-06-05 15:49 ` [PATCH v1 05/14] .travis.yml: allow failure for unreliable hosts Alex Bennée
2020-06-05 15:49 ` [PATCH v1 06/14] .shippable: temporaily disable some cross builds Alex Bennée
2020-06-05 16:12   ` Philippe Mathieu-Daudé
2020-06-05 15:49 ` [PATCH v1 07/14] iotests: 194: wait migration completion on target too Alex Bennée
2020-06-05 15:49 ` [PATCH v1 08/14] tests/docker: fix pre-requisite for debian-tricore-cross Alex Bennée
2020-06-05 15:49 ` [PATCH v1 09/14] docker: update Ubuntu to 20.04 Alex Bennée
2020-06-05 15:49 ` [PATCH v1 10/14] hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE Alex Bennée
2020-06-05 15:49 ` [PATCH v1 11/14] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
2020-06-05 15:49 ` [PATCH v1 12/14] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
2020-06-05 15:49   ` [Bug 1880225] " Alex Bennée
2020-06-05 15:49 ` [PATCH v1 13/14] tests/tcg: add simple commpage test case Alex Bennée
2020-06-05 15:49 ` [PATCH v1 14/14] linux-user: detect overflow of MAP_FIXED mmap Alex Bennée
2020-06-05 16:16   ` Philippe Mathieu-Daudé
2020-06-05 15:54 ` [PATCH v1 00/14] various fixes for next PR (testing, vhost, guest_base fixes) Eric Blake
2020-06-05 17:46 ` no-reply
2020-06-07  6:55 ` Thomas Huth
2020-06-08 15:58   ` Alex Bennée
2020-06-09 10:37 [PULL 00/17] testing and misc fixes Alex Bennée
2020-06-09 10:37 ` [PULL 01/17] qemu-plugin.h: add missing include <stddef.h> to define size_t Alex Bennée
2020-06-09 10:37 ` [PULL 02/17] scripts/clean-includes: Mark 'qemu/qemu-plugin.h' as special header Alex Bennée
2020-06-09 10:37 ` [PULL 03/17] tests/plugin: correctly honour io_count Alex Bennée
2020-06-09 10:37 ` [PULL 04/17] exec: flush the whole TLB if a watchpoint crosses a page boundary Alex Bennée
2020-06-09 10:37 ` [PULL 05/17] .travis.yml: allow failure for unreliable hosts Alex Bennée
2020-06-09 10:37 ` [PULL 06/17] .shippable: temporaily disable some cross builds Alex Bennée
2020-06-09 10:37 ` [PULL 07/17] tests/docker: fix pre-requisite for debian-tricore-cross Alex Bennée
2020-06-09 10:38 ` [PULL 08/17] docker: update Ubuntu to 20.04 Alex Bennée
2020-06-09 10:38 ` [PULL 09/17] hw/virtio/vhost: re-factor vhost-section and allow DIRTY_MEMORY_CODE Alex Bennée
2020-06-09 10:38 ` [PULL 10/17] linux-user: provide fallback pgd_find_hole for bare chroots Alex Bennée
2020-06-09 10:38 ` [PULL 11/17] linux-user: deal with address wrap for ARM_COMMPAGE on 32 bit Alex Bennée
2020-06-09 10:38   ` [Bug 1880225] " Alex Bennée
2020-06-09 10:38 ` [PULL 12/17] tests/tcg: add simple commpage test case Alex Bennée
2020-06-09 10:38 ` [PULL 13/17] linux-user: detect overflow of MAP_FIXED mmap Alex Bennée
2020-06-09 10:38 ` [PULL 14/17] tests/docker: Remove flex/bison packages Alex Bennée
2020-06-09 10:38 ` [PULL 15/17] tests/vm: " Alex Bennée
2020-06-09 10:53   ` Claudio Fontana
2020-06-09 11:08     ` Alex Bennée
2020-06-09 10:38 ` [PULL 16/17] cirrus-ci: " Alex Bennée
2020-06-09 10:38 ` [PULL 17/17] scripts/coverity-scan: " Alex Bennée
2020-06-11 10:20 ` [PULL 00/17] testing and misc fixes Peter Maydell

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).