[Bug 1850378] [NEW] RISC-V unreliable IPIs

[Bug 1850378] [NEW] RISC-V unreliable IPIs

[tm]

Public bug reported:

I am working on a project with custom inter processor interrupts (IPIs) on the RISC-V virt machine.
After upgrading from version 3.1.0 to 4.1.0 which fixes a related issue (https://github.com/riscv/riscv-qemu/issues/132) I am able to use the CPU hotplug feature.

However, if I try to use IPIs for communication between two cores, the
wfi instruction behaves strangely. Either it does not return, or it
returns on timer interrupts, even though they are disabled. The code, I
use on one core to wait for an interrupt is the following.

	csr_clear(sie, SIE_SEIE | SIE_STIE);
	do {
		wait_for_interrupt();
		sipval = csr_read(sip);
		sieval = csr_read(sie);
		scauseval = csr_read(scause) & 0xFF;
	/* only break if wfi returns for an software interrupt */
	} while ((sipval & sieval) == 0 && scauseval != 1);
	csr_set(sie, SIE_SEIE | SIE_STIE);

Since the resulting sequence does not seem to be deterministic, my guess
is, that it has something to do with the communication of qemu's threads
for the different cores.

** Affects: qemu
     Importance: Undecided
         Status: New


** Tags: riscv

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1850378

Title:
  RISC-V unreliable IPIs

Status in QEMU:
  New

Bug description:
  I am working on a project with custom inter processor interrupts (IPIs) on the RISC-V virt machine.
  After upgrading from version 3.1.0 to 4.1.0 which fixes a related issue (https://github.com/riscv/riscv-qemu/issues/132) I am able to use the CPU hotplug feature.

  However, if I try to use IPIs for communication between two cores, the
  wfi instruction behaves strangely. Either it does not return, or it
  returns on timer interrupts, even though they are disabled. The code,
  I use on one core to wait for an interrupt is the following.

  	csr_clear(sie, SIE_SEIE | SIE_STIE);
  	do {
  		wait_for_interrupt();
  		sipval = csr_read(sip);
  		sieval = csr_read(sie);
  		scauseval = csr_read(scause) & 0xFF;
  	/* only break if wfi returns for an software interrupt */
  	} while ((sipval & sieval) == 0 && scauseval != 1);
  	csr_set(sie, SIE_SEIE | SIE_STIE);

  Since the resulting sequence does not seem to be deterministic, my
  guess is, that it has something to do with the communication of qemu's
  threads for the different cores.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1850378/+subscriptions

[Bug 1850378] Re: RISC-V unreliable IPIs

[Palmer Dabbelt]

Can you post a whole program that reproduces this?  freedom-e-sdk
<https://github.com/sifive/freedom-e-sdk> will run bare-metal code on
QEMU if you don't want to post the rest of the surrounding
infrastructure.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1850378

Title:
  RISC-V unreliable IPIs

Status in QEMU:
  New

Bug description:
  I am working on a project with custom inter processor interrupts (IPIs) on the RISC-V virt machine.
  After upgrading from version 3.1.0 to 4.1.0 which fixes a related issue (https://github.com/riscv/riscv-qemu/issues/132) I am able to use the CPU hotplug feature.

  However, if I try to use IPIs for communication between two cores, the
  wfi instruction behaves strangely. Either it does not return, or it
  returns on timer interrupts, even though they are disabled. The code,
  I use on one core to wait for an interrupt is the following.

  	csr_clear(sie, SIE_SEIE | SIE_STIE);
  	do {
  		wait_for_interrupt();
  		sipval = csr_read(sip);
  		sieval = csr_read(sie);
  		scauseval = csr_read(scause) & 0xFF;
  	/* only break if wfi returns for an software interrupt */
  	} while ((sipval & sieval) == 0 && scauseval != 1);
  	csr_set(sie, SIE_SEIE | SIE_STIE);

  Since the resulting sequence does not seem to be deterministic, my
  guess is, that it has something to do with the communication of qemu's
  threads for the different cores.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1850378/+subscriptions

[Bug 1850378] Re: RISC-V unreliable IPIs

[tm]

I created a minimal example from my setup. I'm running a kernel 4.19.57 with a custom firmware based on bbl (https://github.com/riscv/riscv-pk). 
An ioctl device from a kernel module is used to execute the code above in kernel space.
In the example, the userspace application proceeds after a couple of seconds without receiving the custom IPI.

** Attachment added: "example.tar.gz"
   https://bugs.launchpad.net/qemu/+bug/1850378/+attachment/5301219/+files/example.tar.gz

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1850378

Title:
  RISC-V unreliable IPIs

Status in QEMU:
  New

Bug description:
  I am working on a project with custom inter processor interrupts (IPIs) on the RISC-V virt machine.
  After upgrading from version 3.1.0 to 4.1.0 which fixes a related issue (https://github.com/riscv/riscv-qemu/issues/132) I am able to use the CPU hotplug feature.

  However, if I try to use IPIs for communication between two cores, the
  wfi instruction behaves strangely. Either it does not return, or it
  returns on timer interrupts, even though they are disabled. The code,
  I use on one core to wait for an interrupt is the following.

  	csr_clear(sie, SIE_SEIE | SIE_STIE);
  	do {
  		wait_for_interrupt();
  		sipval = csr_read(sip);
  		sieval = csr_read(sie);
  		scauseval = csr_read(scause) & 0xFF;
  	/* only break if wfi returns for an software interrupt */
  	} while ((sipval & sieval) == 0 && scauseval != 1);
  	csr_set(sie, SIE_SEIE | SIE_STIE);

  Since the resulting sequence does not seem to be deterministic, my
  guess is, that it has something to do with the communication of qemu's
  threads for the different cores.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1850378/+subscriptions

[Bug 1850378] Re: RISC-V unreliable IPIs

[tm]

** Description changed:

  I am working on a project with custom inter processor interrupts (IPIs) on the RISC-V virt machine.
  After upgrading from version 3.1.0 to 4.1.0 which fixes a related issue (https://github.com/riscv/riscv-qemu/issues/132) I am able to use the CPU hotplug feature.
  
  However, if I try to use IPIs for communication between two cores, the
  wfi instruction behaves strangely. Either it does not return, or it
  returns on timer interrupts, even though they are disabled. The code, I
  use on one core to wait for an interrupt is the following.
  
- 	csr_clear(sie, SIE_SEIE | SIE_STIE);
- 	do {
- 		wait_for_interrupt();
- 		sipval = csr_read(sip);
- 		sieval = csr_read(sie);
- 		scauseval = csr_read(scause) & 0xFF;
- 	/* only break if wfi returns for an software interrupt */
- 	} while ((sipval & sieval) == 0 && scauseval != 1);
- 	csr_set(sie, SIE_SEIE | SIE_STIE);
+  csr_clear(sie, SIE_SEIE | SIE_STIE);
+  do {
+   wait_for_interrupt();
+   sipval = csr_read(sip);
+   sieval = csr_read(sie);
+   scauseval = csr_read(scause) & 0xFF;
+  /* only break if wfi returns for an software interrupt */
+  } while ((sipval & sieval) == 0 && scauseval != 1);
+  csr_set(sie, SIE_SEIE | SIE_STIE);
  
  Since the resulting sequence does not seem to be deterministic, my guess
  is, that it has something to do with the communication of qemu's threads
  for the different cores.
+ 
+ Update:
+ The exact same setup works fine in spike (the actual sim, not the qemu board), which might give a hint, that it is related to the interrupt controller implementation.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1850378

Title:
  RISC-V unreliable IPIs

Status in QEMU:
  New

Bug description:
  I am working on a project with custom inter processor interrupts (IPIs) on the RISC-V virt machine.
  After upgrading from version 3.1.0 to 4.1.0 which fixes a related issue (https://github.com/riscv/riscv-qemu/issues/132) I am able to use the CPU hotplug feature.

  However, if I try to use IPIs for communication between two cores, the
  wfi instruction behaves strangely. Either it does not return, or it
  returns on timer interrupts, even though they are disabled. The code,
  I use on one core to wait for an interrupt is the following.

   csr_clear(sie, SIE_SEIE | SIE_STIE);
   do {
    wait_for_interrupt();
    sipval = csr_read(sip);
    sieval = csr_read(sie);
    scauseval = csr_read(scause) & 0xFF;
   /* only break if wfi returns for an software interrupt */
   } while ((sipval & sieval) == 0 && scauseval != 1);
   csr_set(sie, SIE_SEIE | SIE_STIE);

  Since the resulting sequence does not seem to be deterministic, my
  guess is, that it has something to do with the communication of qemu's
  threads for the different cores.

  Update:
  The exact same setup works fine in spike (the actual sim, not the qemu board), which might give a hint, that it is related to the interrupt controller implementation.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1850378/+subscriptions

[Bug 1850378] Re: RISC-V unreliable IPIs

[Thomas Huth]

The QEMU project is currently considering to move its bug tracking to
another system. For this we need to know which bugs are still valid
and which could be closed already. Thus we are setting older bugs to
"Incomplete" now.

If you still think this bug report here is valid, then please switch
the state back to "New" within the next 60 days, otherwise this report
will be marked as "Expired". Or please mark it as "Fix Released" if
the problem has been solved with a newer version of QEMU already.

Thank you and sorry for the inconvenience.


** Changed in: qemu
       Status: New => Incomplete

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1850378

Title:
  RISC-V unreliable IPIs

Status in QEMU:
  Incomplete

Bug description:
  I am working on a project with custom inter processor interrupts (IPIs) on the RISC-V virt machine.
  After upgrading from version 3.1.0 to 4.1.0 which fixes a related issue (https://github.com/riscv/riscv-qemu/issues/132) I am able to use the CPU hotplug feature.

  However, if I try to use IPIs for communication between two cores, the
  wfi instruction behaves strangely. Either it does not return, or it
  returns on timer interrupts, even though they are disabled. The code,
  I use on one core to wait for an interrupt is the following.

   csr_clear(sie, SIE_SEIE | SIE_STIE);
   do {
    wait_for_interrupt();
    sipval = csr_read(sip);
    sieval = csr_read(sie);
    scauseval = csr_read(scause) & 0xFF;
   /* only break if wfi returns for an software interrupt */
   } while ((sipval & sieval) == 0 && scauseval != 1);
   csr_set(sie, SIE_SEIE | SIE_STIE);

  Since the resulting sequence does not seem to be deterministic, my
  guess is, that it has something to do with the communication of qemu's
  threads for the different cores.

  Update:
  The exact same setup works fine in spike (the actual sim, not the qemu board), which might give a hint, that it is related to the interrupt controller implementation.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1850378/+subscriptions

[Bug 1850378] Re: RISC-V unreliable IPIs

[Launchpad Bug Tracker]

[Expired for QEMU because there has been no activity for 60 days.]

** Changed in: qemu
       Status: Incomplete => Expired

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1850378

Title:
  RISC-V unreliable IPIs

Status in QEMU:
  Expired

Bug description:
  I am working on a project with custom inter processor interrupts (IPIs) on the RISC-V virt machine.
  After upgrading from version 3.1.0 to 4.1.0 which fixes a related issue (https://github.com/riscv/riscv-qemu/issues/132) I am able to use the CPU hotplug feature.

  However, if I try to use IPIs for communication between two cores, the
  wfi instruction behaves strangely. Either it does not return, or it
  returns on timer interrupts, even though they are disabled. The code,
  I use on one core to wait for an interrupt is the following.

   csr_clear(sie, SIE_SEIE | SIE_STIE);
   do {
    wait_for_interrupt();
    sipval = csr_read(sip);
    sieval = csr_read(sie);
    scauseval = csr_read(scause) & 0xFF;
   /* only break if wfi returns for an software interrupt */
   } while ((sipval & sieval) == 0 && scauseval != 1);
   csr_set(sie, SIE_SEIE | SIE_STIE);

  Since the resulting sequence does not seem to be deterministic, my
  guess is, that it has something to do with the communication of qemu's
  threads for the different cores.

  Update:
  The exact same setup works fine in spike (the actual sim, not the qemu board), which might give a hint, that it is related to the interrupt controller implementation.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1850378/+subscriptions