Re: [PATCH v3 00/32] Introduce HVM without dm and new boot ABI

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

> > > I started briefly looking today but don't really have time to look into
> > > it properly.  I do think we want to use as much of the native/HVM code
> > > as possible and avoid the PV path (the current Linux PVH code uses too
> > > much of the PV entry point, IMO).
> > 
> > I'm afraid I'm not that familiar with the native Linux entry point in
> > order to know how much of that can be reused. Does the amd64 kernel
> > contain a 32bit entry point or the kernel is expected to be launched in
> > long mode with some page tables already setup?
> 
> It does : arch/x86/kernel/head_64[32].S:startup_32
> 
> The problem is that it depends to have the boot_params (a Linux structure)
> filled out.
> 
> Which then calls 'default_entry' [32-bit] or 'startup_64', and
> then it goes off somewhere (there is a lot of EFI code in there).
> 
> Anyhow it eventually ends up:
> 
>  x86_64_start_kernel -> x86_64_start_reservations
>  
> But the problem is that during this execution none of the pvops get
> set to the Xen ones (which we need to tell the kernel that there is
> almost no hardware).
> 
> One could change initial_code to some xen_x86_64_start_kernel that
> would be similar to xen_startup - in that it would set up the
> basic pvops structures and then call x86_64_start_kernel.
> 
> In essence something like:
>  startup_32
>    \-> startup_64 [in compressed/head_64.S]
>       \-> startup_64 [ head_64.S]
>          \-> secondary_startup_64
>             \-> jmpq *initial_code [which is by default x86_64_start_kernel]
                   \-> x86_64_start_reservations

> 
>          
> [if I got the assmebler call chain correctly figured out] 

Just to give an comparison, the PV entry point is:

 xen_start_kernel [copies appropiate things to boot_parms - such as command line, makes
     tons of hypercalls to set appropiate tweaks in boot_params]
   \-> x86_64_start_reservations

If we were to hook up to the above one, we would need:

xen_no_paging_start_kernel [copy minimum amount of things we need on the bootparams
    to keep on going, sets initial_code to our xen_secondary_start]
   \- startup_32
	\-> startup_64
            \-> secondary_startup_64
		 -> jmpq *initial_code
			\-> xen_secondary_start [set pvops, copies rest of bootparams,
                                                 etc]
				\-> x86_64_start_kernel
					\-> x86_64_start_reservation

If we were to borrow from what Roger has [http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=blob;f=sys/amd64/amd64/xen-locore32.S;h=fd73d150009ab21059d4df219e6a62cc40d5994e;hb=refs/heads/new_entry_point_v2]
we would do:

xen_no_paging_start_kernel [xen_start32 copy of FreeBSD]
  \-> xen_secondary_start [sets pvops, copies rest of params, etc]
    \-> x86_64_start_reservation

But that to me looks quite similar to the PV path, except more code, and
also x86 folks asking us why we aren't using the baremetal boot paths.

We could omit some of this if the toolstack/hyprecall knew the Linux
boot_params similar to how GRUB2 knows it. And just populate that.
But then if we are doing that, why not just go the simple route
and use the Xen PV entry point.