xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
* [Xen-devel] Having a DOM-U guest with 1:1 mapping in the second stage MMU.
@ 2020-01-14 21:39 Jorge Pereira
  2020-01-15 11:07 ` Julien Grall
  0 siblings, 1 reply; 14+ messages in thread
From: Jorge Pereira @ 2020-01-14 21:39 UTC (permalink / raw)
  To: xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 656 bytes --]

Hi Guys,

I'm currently using XEN in order to run side-by-side a DOM-0 with a DOM-U guest. My use-case scenario requires in the DOM-U direct access to some dma-capable devices such ethernet and some GPUs.

Since our target platform (i.MX8MM) does not support IOMMU, we can't assign dma-capable devices to the DOM-U guest because XEN does not create 1:1 mapping for that guest in the 2nd stage MMU. So, guest-virtual addresses are different than the physical ones.

Is it possible to have 1:1 mapping for DOM-U guests? If not, I'm interested to know what would be the estimated effort to support this feature?

Thanks in advance,

Cheers,
Jorge

[-- Attachment #1.2: Type: text/html, Size: 2909 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread
[parent not found: <VI1PR04MB5807A7F83F1B2763BD7EEB20F91B0@VI1PR04MB5807.eurprd04.prod.outlook.com>]
* Re: [Xen-devel] Having a DOM-U guest with 1:1 mapping in the second stage MMU.
@ 2020-02-13 17:27 Andrei Cherechesu
  2020-02-13 21:23 ` Julien Grall
  2020-02-13 21:28 ` Stefano Stabellini
  0 siblings, 2 replies; 14+ messages in thread
From: Andrei Cherechesu @ 2020-02-13 17:27 UTC (permalink / raw)
  To: Julien Grall; +Cc: xen-devel, Stefano Stabellini

-----Original Message-----
From: Julien Grall <julien@xen.org> 
Sent: Thursday, February 13, 2020 00:03
To: Stefano Stabellini <sstabellini@kernel.org>; Andrei Cherechesu <andrei.cherechesu@nxp.com>
Cc: Jorge Pereira <jorge.pereira@nxp.com>; xen-devel@lists.xenproject.org
Subject: Re: [Xen-devel] Having a DOM-U guest with 1:1 mapping in the second stage MMU.

Hello,

I used the Xen from Stefano's tree and made the updates to the partial
dtb that he specified.

> This is mostly likely because Linux is trying to access a region
> that is not mapped in stage-2. You can rebuild Xen with debug enabled
> and you should see a message "traps.c:..." telling the exact physical
> address accessed.
> 
> In general I would recommend to build Xen with debug enabled during development as the hypervisor will give you more information of what's going on.
> 
> Cheers,
> 
> --
> Julien Grall

I enabled debug config and gave it another try. But I'm still
getting the same unhandled fault error, that seems to match what
Julien described above.

It is indeed a stage-2 abort caused by the guest.

I attached the DomU1 crash log at [0].

[0] https://pastebin.com/BSHVFQiK

How should I proceed in this case?

Thank you again for your support,
Andrei
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [Xen-devel] Having a DOM-U guest with 1:1 mapping in the second stage MMU.
@ 2020-04-01 10:14 Andrei Cherechesu
  2020-04-01 16:26 ` Stefano Stabellini
  0 siblings, 1 reply; 14+ messages in thread
From: Andrei Cherechesu @ 2020-04-01 10:14 UTC (permalink / raw)
  To: Stefano Stabellini; +Cc: xen-devel, Julien Grall

Hi,

And thanks for your help.

> Great to hear!
> 
> 
> > However, I am encountering another problem now: in Dom0 and in 
> > dom0less-booted DomUs, I cannot use /dev/hvc0.
> 
> For dom0less-booted DomUs it is normal because they don't get a PV console,
> they get an emulated PL011 UART instead.  Make sure to have a "vpl011" tag in
> device tree to enable it (ImageBuilder generates it by default.) The device
> name is usually ttyAMA0.

Ok, understood. I had my vpl011 tag in the dom0less DomUs nodes in the DT,
so that's not the problem. I am able to see DomUs boot prompt when booting
dom0less. The problem is after DomU boots, that I am not able to switch to
its console from Dom0, in order to be able to log in.

> > Even though I'm specifying "console=hvc0" in dom0-bootargs, when dom0 
> > finishes booting, it looks like I cannot use the getty spawned on /dev/hvc0.
> >
> > This is the end of the boot log:
> > [    2.947845] random: rngd: uninitialized urandom read (4 bytes read)
> > [    2.958415] random: rngd: uninitialized urandom read (4 bytes read)
> > [    2.965452] random: rngd: uninitialized urandom read (2500 bytes read)
> > .
> > [    2.972410] random: crng init done
> > Starting OpenBSD Secure Shell server: sshd done.
> > Starting /usr/sbin/xenstored...
> > Setting domain 0 name, domid and JSON config...
> > Done setting up Dom0
> > Starting xenconsoled...
> > Starting QEMU as disk backend for dom0 Starting domain watchdog 
> > daemon: xenwatchdogd startup
> >
> > [done]
> >
> > Auto Linux BSP 1.0 s32g274aevb /dev/hvc0
> >
> > s32g274aevb login:
> > Auto Linux BSP 1.0 s32g274aevb /dev/ttyLF0
> >
> > s32g274aevb login:
> >
> > ----- END -----
> >
> > It seems that the getty spawned on /dev/ttyLF0 overwrites the one 
> > spawned on /dev/hvc0. Which I do not understand, since Dom0 should not 
> > have access (?) directly to ttyLF0 (the serial console device on our 
> > boards). If I remove the line which spawns the getty on ttyLF0 from 
> > /etc/inittab, the system hangs when waiting for the username, and it does not let me type in any characters. For the record, hvc0 is added to /etc/securetty.
> >
> > In a system where I boot DomU via xl from Dom0, I can switch to its 
> > console with xl console, and hvc0 works there.
> >
> > The problem that comes with this is that I can not use the CTRL-AAA 
> > command to switch between Dom0 console and DomU console in a dom0less 
> > case, and I cannot therefore test that the passthrough works. But at least Dom0 does not have an entry for it under /dev, anymore, and DomU boot prompt tells that the driver has been registered.
> 
> It looks like there is some kind of interference between the dom0 ttyLF0 driver and the Xen serial driver.
> 
> Is your Xen UART driver marking the device as "used by Xen"? See for instance the pl011 driver, at the end of
> xen/drivers/char/pl011.c:pl011_dt_uart_init:
> 
>     dt_device_set_used_by(dev, DOMID_XEN);
> 
> Devices that are marked as "used by Xen" are not exposed to dom0, so you shouldn't see the ttyLF0 device come up in Linux at all.

I've checked my Xen UART Driver and that call is there. So ttyLF0 should be
marked for Xen to use.

I don't have any ideas why this happens.

I've attached the driver [0], if you want to take a look.

[0] https://pastebin.com/PuXi50H0

Thanks,
Andrei Cherechesu,
NXP Semiconductors



^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [Xen-devel] Having a DOM-U guest with 1:1 mapping in the second stage MMU.
@ 2020-04-01 19:58 Andrei Cherechesu
  2020-04-01 21:07 ` Julien Grall
  2020-04-01 21:35 ` Stefano Stabellini
  0 siblings, 2 replies; 14+ messages in thread
From: Andrei Cherechesu @ 2020-04-01 19:58 UTC (permalink / raw)
  To: Stefano Stabellini; +Cc: xen-devel, Julien Grall

> I cannot see any issues with the driver. Can you paste the device tree as dom0
> sees it? You can access it from /proc/device-tree (you can convert it to dts
> with dtc -I fs -O dts). And also the host device tree?
> 
> We should see something like the following (this example is taken from a Xilinx
> ZynqMP):
> 
> 1) host device tree
> 
>         serial@ff000000 {
>                 u-boot,dm-pre-reloc;
>                 compatible = "cdns,uart-r1p12", "xlnx,xuartps";
>                 status = "okay";
>                 interrupt-parent = <0x4>;
>                 interrupts = <0x0 0x15 0x4>;
>                 reg = <0x0 0xff000000 0x0 0x1000>;
>                 clock-names = "uart_clk", "pclk";
>                 power-domains = <0x26 0x21>;
>                 clocks = <0x3 0x38 0x3 0x1f>;
>                 pinctrl-names = "default";
>                 pinctrl-0 = <0x37>;
>                 cts-override;
>                 device_type = "serial";
>                 port-number = <0x0>;
>         };
> 
> 
> 2) dom0 device tree
> 
>     The node is missing
> 
> 
> The key is that dom0 should not see the UART node in device tree at all.
> 
> If Dom0 is seeing the node, then it is a problem with Xen that somehow is not
> hiding it (see "Skip nodes used by Xen" under
> xen/arch/arm/domain_build.c:handle_node)
> 
> If Dom0 is *not* seeing the node, that what is the underlying device tree node
> that the dom0 driver is using to bring up /dev/ttyLF0?
> 

I looked under /proc/device-tree and Dom0 actually sees the node corresponding
to the serial device. That means Xen is not hiding it. 

I've pasted the host device tree at [0] and the Dom0 device tree at [1]
[0] https://pastebin.com/ir7VkfES
[1] https://pastebin.com/UqRDycHF

So this is the device in the host device tree:

    uart0:serial@401C8000 {
        compatible = "fsl,s32-linflexuart";
        reg = <0x0 0x401C8000 0x0 0x3000>;
        interrupts = <0 82 1>;
        clocks = <&clks S32GEN1_CLK_LIN_BAUD>,
                <&clks S32GEN1_CLK_LIN>;
        clock-names = "lin", "ipg";
        dmas = <&edma0 0 4>,
                <&edma0 0 3>;
        dma-names = "rx", "tx";
    };


And this is the device in the Dom0 device tree:

    serial@401C8000 {
        clock-names = "lin\0ipg";
        interrupts = <0x00 0x52 0x01>;
        clocks = <0x04 0x2a 0x04 0x2b>;
        dma-names = "rx\0tx";
        compatible = "fsl,s32-linflexuart";
        reg = <0x00 0x401c8000 0x00 0x3000>;
        dmas = <0x0e 0x00 0x04 0x0e 0x00 0x03>;
    };

From this story I figured out something else: that I was not passing to Xen's
bootargs "console=dtuart dtuart=serial0". Maybe that's why Xen is not hiding
the node from Dom0 (serial0 is an alias to uart0).

So I went on and I added that to bootargs. And now my Xen boot stops here
(DEBUG print on):
...
...
(XEN) fixed up name for siul2_reg@0x4401170C -> siul2_reg
(XEN) fixed up name for siul2_reg@0x4401174C -> siul2_reg
(XEN) fixed up name for usbmisc@44064200 -> usbmisc
(XEN) fixed up name for usb@44064000 -> usb
(XEN) fixed up name for memory_DDR0@80000000 -> memory_DDR0
(XEN) fixed up name for memory_DDR1@c0000000 -> memory_DDR1
(XEN) fixed up name for memory_DDR2@880000000 -> memory_DDR2
(XEN)  <- unflatten_device_tree()
(XEN) adding DT alias:can0: stem=can id=0 node=/flexcan@401B4000
(XEN) adding DT alias:can1: stem=can id=1 node=/flexcan@401BE000
(XEN) adding DT alias:can2: stem=can id=2 node=/flexcan@402A8000
(XEN) adding DT alias:can3: stem=can id=3 node=/flexcan@402B2000
(XEN) adding DT alias:serial0: stem=serial id=0 node=/serial@401C8000
(XEN) adding DT alias:serial1: stem=serial id=1 node=/serial@401CC000
(XEN) adding DT alias:serial2: stem=serial id=2 node=/serial@402BC000
(XEN) Platform: Generic System
(XEN) Looking for dtuart at "serial0", options ""
(XEN) DT: ** translation for device /serial@401C8000 **
(XEN) DT: bus is default (na=2, ns=2) on /
(XEN) DT: translating address:<3> 00000000<3> 401c8000<3>
(XEN) DT: reached root node
(XEN) dt_device_get_raw_irq: dev=/serial@401C8000, index=0
(XEN)  intspec=0 intlen=3
(XEN)  intsize=3 intlen=3
(XEN) dt_irq_map_raw: par=/interrupt-controller@50800000,intspec=[0x00000000 0x00000052...],ointsize=3
(XEN) dt_irq_map_raw: ipar=/interrupt-controller@50800000, size=3
(XEN)  -> addrsize=2
(XEN)  -> got it� 

I started debugging and I found out that it hangs in:
console_init_preirq() -> __putstr(xen_banner()) -> sercon_puts() -> serial_puts() -> __serial_putc(),
where it spins at line 178:         
        /* Synchronous finite-capacity transmitter. */
        while ( !(n = port->driver->tx_ready(port)) )
            cpu_relax();
 
Which is a bit strange, considering that my serial device is asynchronous,
I think it should not get there. But it gets on that "else" branch because
port->txbuf is actually NULL at line 120 when it performs the check, and
it does not enter the branch for asynchronous transmitters.

Thanks,
Andrei Cherechesu,
NXP Semiconductors


^ permalink raw reply	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2020-04-01 21:36 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-14 21:39 [Xen-devel] Having a DOM-U guest with 1:1 mapping in the second stage MMU Jorge Pereira
2020-01-15 11:07 ` Julien Grall
2020-01-16 23:55   ` Stefano Stabellini
     [not found] <VI1PR04MB5807A7F83F1B2763BD7EEB20F91B0@VI1PR04MB5807.eurprd04.prod.outlook.com>
2020-02-12 13:45 ` Andrei Cherechesu
2020-02-12 21:36   ` Stefano Stabellini
2020-02-12 22:03     ` Julien Grall
2020-02-13 17:27 Andrei Cherechesu
2020-02-13 21:23 ` Julien Grall
2020-02-13 21:28 ` Stefano Stabellini
2020-04-01 10:14 Andrei Cherechesu
2020-04-01 16:26 ` Stefano Stabellini
2020-04-01 19:58 Andrei Cherechesu
2020-04-01 21:07 ` Julien Grall
2020-04-01 21:35 ` Stefano Stabellini

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).