On Thu, 3 Mar 2022, Wei Chen wrote: > > On Thu, 3 Mar 2022, Wei Chen wrote: > > > > On Wed, 2 Mar 2022, Wei Chen wrote: > > > > > > > > > > If not, and considering that we have to generate > > > > > > > > > > ARM_MPU_*_MEMORY_START/END anyway at build time, would it > > make > > > > > > sense > > > > > > > > to > > > > > > > > > > also generate mpu,guest-memory-section, xen,static-mem, > > etc. > > > > at > > > > > > build > > > > > > > > > > time rather than passing it via device tree to Xen at > > runtime? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Did you mean we still add these information in device tree, > > but > > > > for > > > > > > > > build > > > > > > > > > time only. In runtime we don't parse them? > > > > > > > > > > > > > > > > Yes, something like that, but see below. > > > > > > > > > > > > > > > > > > > > > > > > > > What's the value of doing ARM_MPU_*_MEMORY_START/END at > > build > > > > time > > > > > > and > > > > > > > > > > everything else at runtime? > > > > > > > > > > > > > > > > > > ARM_MPU_*_MEMORY_START/END is defined by platform. But other > > > > things > > > > > > are > > > > > > > > > users customized. They can change their usage without > > rebuild > > > > the > > > > > > image. > > > > > > > > > > > > > > > > Good point. > > > > > > > > > > > > > > > > We don't want to have to rebuild Xen if the user updated a > > guest > > > > > > kernel, > > > > > > > > resulting in a larger boot-module-section. > > > > > > > > > > > > > > > > So I think it makes sense that "mpu,boot-module-section" is > > > > generated > > > > > > by > > > > > > > > the scripts (e.g. ImageBuilder) at build time, and Xen reads > > the > > > > > > > > property at boot from the runtime device tree. > > > > > > > > > > > > > > > > I think we need to divide the information into two groups: > > > > > > > > > > > > > > > > > > > > > > > > # Group1: board info > > > > > > > > > > > > > > > > This information is platform specific and it is not meant to > > > > change > > > > > > > > depending on the VM configuration. Ideally, we build Xen for a > > > > > > platform > > > > > > > > once, then we can use the same Xen binary together with any > > > > > > combination > > > > > > > > of dom0/domU kernels and ramdisks. > > > > > > > > > > > > > > > > This kind of information doesn't need to be exposed to the > > runtime > > > > > > > > device tree. But we can still use a build-time device tree to > > > > generate > > > > > > > > the addresses if it is convenient. > > > > > > > > > > > > > > > > XEN_START_ADDRESS, ARM_MPU_DEVICE_MEMORY_*, and > > > > > > ARM_MPU_NORMAL_MEMORY_* > > > > > > > > seem to be part of this group. > > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > > > > > > > > > > # Group2: boot configuration > > > > > > > > > > > > > > > > This information is about the specific set of binaries and VMs > > > > that we > > > > > > > > need to boot. It is conceptually similar to the dom0less > > device > > > > tree > > > > > > > > nodes that we already have. If we change one of the VM > > binaries, > > > > we > > > > > > > > likely have to refresh the information here. > > > > > > > > > > > > > > > > "mpu,boot-module-section" probably belongs to this group > > (unless > > > > we > > > > > > find > > > > > > > > a way to define "mpu,boot-module-section" generically so that > > we > > > > don't > > > > > > > > need to change it any time the set of boot modules change.) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I agree. > > > > > > > > > > > > > > > > > It looks like we are forced to have the sections > > definitions > > > > at > > > > > > build > > > > > > > > > > time because we need them before we can parse device tree. > > In > > > > that > > > > > > > > case, > > > > > > > > > > we might as well define all the sections at build time. > > > > > > > > > > > > > > > > > > > > But I think it would be even better if Xen could > > automatically > > > > > > choose > > > > > > > > > > xen,static-mem, mpu,guest-memory-section, etc. on its own > > > > based on > > > > > > the > > > > > > > > > > regular device tree information (/memory, /amba, etc.), > > > > without > > > > > > any > > > > > > > > need > > > > > > > > > > for explicitly describing each range with these new > > properties. > > > > > > > > > > > > > > > > > > > > > > > > > > > > for mpu,guest-memory-section, with the limitations: no other > > > > usage > > > > > > > > between > > > > > > > > > different guest' memory nodes, this is OK. But for > > xen,static- > > > > mem > > > > > > (heap), > > > > > > > > > we just want everything on a MPU system is dertermistic. But, > > of > > > > > > course > > > > > > > > Xen > > > > > > > > > can select left memory for heap without static-mem. > > > > > > > > > > > > > > > > It is good that you think they can be chosen by Xen. > > > > > > > > > > > > > > > > Differently from "boot-module-section", which has to do with > > the > > > > boot > > > > > > > > modules selected by the user for a specific execution, > > > > > > > > guest-memory-section and static-mem are Xen specific memory > > > > > > > > policies/allocations. > > > > > > > > > > > > > > > > A user wouldn't know how to fill them in. And I worry that > > even a > > > > > > script > > > > > > > > > > > > > > But users should know it, because static-mem for guest must be > > > > allocated > > > > > > > in this range. And users take the responsibility to set the > > DomU's > > > > > > > static allocate memory ranges. > > > > > > > > > > > > Let me premise that my goal is to avoid having many users > > reporting > > > > > > errors to xen-devel and xen-users when actually it is just a wrong > > > > > > choice of addresses. > > > > > > > > > > > > I think we need to make a distinction between addresses for the > > boot > > > > > > modules, e.g. addresses where to load xen, the dom0/U kernel, > > dom0/U > > > > > > ramdisk in memory at boot time, and VM static memory addresses. > > > > > > > > > > > > The boot modules addresses are particularly difficult to fill in > > > > because > > > > > > they are many and a small update in one of the modules could > > > > invalidate > > > > > > all the other addresses. This is why I ended up writing > > ImageBuilder. > > > > > > Since them, I received several emails from users thanking me for > > > > > > ImageBuilder :-) > > > > > > > > > > > > > > > > Thanks +999 😊 > > > > > > > > > > > > > > > > The static VM memory addresses (xen,static-mem) should be a bit > > easier > > > > > > to fill in correctly. They are meant to be chosen once, and it > > > > shouldn't > > > > > > happen that an update on a kernel forces the user to change all > > the VM > > > > > > static memory addresses. Also, I know that some users actually > > want to > > > > > > be able to choose the domU addresses by hand because they have > > > > specific > > > > > > needs. So it is good that we can let the user choose the addresses > > if > > > > > > they want to. > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > With all of that said, I do think that many users won't have an > > > > opinion > > > > > > on the VM static memory addresses and won't know how to choose > > them. > > > > > > It would be error prone to let them try to fill them in by hand. > > So I > > > > > > was already planning on adding support to ImageBuilder to > > > > automatically > > > > > > generate xen,static-mem for dom0less domains. > > > > > > > > > > > > > > > > Let me make sure that's what you said: Users give an VM memory size > > to > > > > > ImageBuilder, and ImageBuilder will generate xen,static-mem = > > > size>. > > > > > For specific VM, ImageBuilder also can accept start and size as > > inputs? > > > > > > > > > > Do I understand this correctly? > > > > > > > > Yes, exactly > > > > > > > > > > > > > > Going back to this specific discussion about boot-module-section: > > I > > > > can > > > > > > see now that, given xen,static-mem is chosen by ImageBuilder (or > > > > > > > > > > By hand : ) > > > > > > > > > > > similar) and not Xen, then it makes sense to have ImageBuilder (or > > > > > > similar) also generate boot-module-section. > > > > > > > > > > > > > > > > If my above understanding is right, then yes. > > > > > > > > Yes, I think we are on the same page > > > > > > > > > > > > > > > > like ImageBuilder wouldn't be the best place to pick these > > values > > > > -- > > > > > > > > they seem too "important" to leave to a script. > > > > > > > > > > > > > > > > But it seems possible to choose the values in Xen: > > > > > > > > - Xen knows ARM_MPU_NORMAL_MEMORY_* because it was defined at > > > > build > > > > > > time > > > > > > > > - Xen reads boot-module-section from device tree > > > > > > > > > > > > > > > > It should be possible at this point for Xen to pick the best > > > > values > > > > > > for > > > > > > > > guest-memory-section and static-mem based on the memory > > available. > > > > > > > > > > > > > > > > > > > > > > How Xen to pick? Does it mean in static allocation DomU DT node, > > we > > > > just > > > > > > > need a size, but don't require a start address for static-mem? > > > > > > > > > > > > Yes the idea was that the user would only provide the size (e.g. > > > > > > DOMU_STATIC_MEM[1]=1024) and the addresses would be automatically > > > > > > calculated. But I didn't mean to change the existing xen,static- > > mem > > > > > > device tree bindings. So it is best if the xen,static-mem > > addresses > > > > > > generation is done by ImageBuilder (or similar tool) instead of > > Xen. > > > > > > > > > > > > > > > > If we still keep the option for user to specify the start and size > > > > > parameters for VM memory, because it maybe very important for a > > > > > deterministic system (fully static system), I agree with you. > > > > > > > > > > And in current static-allocation, I think Xen doesn't generate > > > > > xen,static-mem addresses, all by hands... > > > > > > > > Yeah > > > > > > > > > > I will update my proposal to cover our above discussion, but I forgot > > one > > > thing. As the platform header files will be generated from DTS, does it > > > mean we have to maintain platform dts files in Xen like what Zephyr has > > > done? > > > > I would prefer not to have to maintain platform dts files in Xen like > > Zephyr is doing. Ideally, the user should be able to take any > > spec-compliant device tree file and use it. I would say: let's start > > without adding the dts files to Xen (we might have one under docs/ but > > just as an example.) We can add them later if the need arise. > > > > But without any default dts, that means we can't start to build Xen for > v8R? It seems in this case, we need Makefile to print some message to > tell users to specific his/her dts path. Yes, exactly. > > > And do you have some idea to integrate the "ImageBuilder"? Make it > > > as a submodule of Xen or integrate to xen-tools? > > > > I think it would be best if ImageBuilder was kept as a separate > > repository because there should be no strong ties between ImageBuilder > > versions and Xen versions. It is more convenient to handle it in a > > separate repository, especially as Yocto and other build systems might > > clone ImageBuilder during the build to generate boot.scr (it is already > > the case). > > > > That said, it might be good to make it more "official" but moving it to > > Xen Project. I can talk to George about creating > > http://xenbits.xen.org/git-http/imagebuilder.git or > > https://gitlab.com/xen-project/imagebuilder. > > That's good : )