* [RFC] Standard way of generating assembler offsets @ 2001-10-04 11:47 Keith Owens 2001-10-04 15:36 ` george anzinger ` (3 more replies) 0 siblings, 4 replies; 14+ messages in thread From: Keith Owens @ 2001-10-04 11:47 UTC (permalink / raw) To: linux-kernel Almost every architecture generates Assembler values to map the offsets of fields in C structures, about the only exception is i386 and that is because its offsets are hard coded into entry.S. Every arch has done it differently, none of them have got it exactly right. As part of kbuild 2.5 I am standardizing on one method for generating Assembler offsets. This change is required for kbuild 2.5 but it can be added to 2.4 without disturbing the current kbuild, I want to do this gradually now instead of a single massive change in kernel 2.5. I will be issuing per architecture changes for generating Assembler offsets against 2.4. The kbuild 2.5 method for generating Assembler offsets satisfies these requirements: * No manual intervention required. Many architectures rely on users running make dep after changing config options that affect the Assembler offsets. If the user forgets to run make dep then the C and Assembler code is out of sync - totally unacceptable. This is completely fixed in kbuild 2.5; I cannot do a complete fix in kbuild 2.4 but it is still better than the existing manual system. * Standard name for the related files. There are 6+ different names for the files used to generate Assembler offsets, kbuild 2.5 uses asm-offsets.[csh] on all architectures. * Allows for multiple parallel compiles from the same source tree. Writing the generated asm-offsets.h to include/asm is not an option, it prevents concurrent compiles. * The method must work in native and cross compile mode and give exactly the same results. Some 2.4 code only works in native mode, some architectures have different methods for native and cross compile with different output formats. Yeuch! * Standard scripts for generating the output. Every arch does it differently in 2.4, standards are good! * Correct dependency trees. Because 2.4 make dep does not scan .S files, there is little or no dependency information. Even if the offsets are regenerated, the affected Assembler code does not always get rebuilt. kbuild 2.5 handles dependencies for Assembler as well as C; I cannot get kbuild 2.4 perfect but I can improve on the existing (or non-existent) 2.4 dependencies. All architectures will define arch/$(ARCH)/asm-offsets.c. This has a standard prologue for the macros that convert offsets to Assembler, followed by arch specific field references. arch/$(ARCH)/asm-offsets.s is generated from arch/$(ARCH)/asm-offsets.c using standard rules, although kbuild 2.4 needs some tweaking. arch/$(ARCH)/asm-offsets.h is generated from arch/$(ARCH)/asm-offsets.s by a semi-standard script. Most of the script is common to all architectures but the precise format of the Assembler output is arch specific. The final result is included in *only* the Assembler programs that need it, as #include "asm-offsets.h" with -I arch/$(ARCH) in the relevant Makefiles. Hard coding relative paths in source files is a pet hate, use #include "localname.h" and -I instead. Including the generated file in C code is not allowed, it severly pollutes the dependency chain, to the extent that any config change can force a complete recompile, unacceptable. Example from i386: arch/i386/asm-offsets.c /* * Generate definitions needed by assembly language modules. * This code generates raw asm output which is post-processed to extract * and format the required data. */ #include <linux/types.h> #include <linux/stddef.h> #include <linux/sched.h> /* Use marker if you need to separate the values later */ #define DEFINE(sym, val, marker) \ asm volatile("\n-> " #sym " %0 " #val " " #marker : : "i" (val)) #define BLANK() asm volatile("\n->" : : ) int main(void) { DEFINE(state, offsetof(struct task_struct, state),); DEFINE(flags, offsetof(struct task_struct, flags),); DEFINE(sigpending, offsetof(struct task_struct, sigpending),); DEFINE(addr_limit, offsetof(struct task_struct, addr_limit),); DEFINE(exec_domain, offsetof(struct task_struct, exec_domain),); DEFINE(need_resched, offsetof(struct task_struct, need_resched),); DEFINE(tsk_ptrace, offsetof(struct task_struct, ptrace),); DEFINE(processor, offsetof(struct task_struct, processor),); BLANK(); DEFINE(ENOSYS, ENOSYS,); return 0; } asm-offsets.s to asm-offsets.h. # Convert raw asm offsets into something that can be included as # assembler definitions. It converts # -> symbol $value source # into # symbol = value /* 0xvalue source */ echo '#ifndef __ASM_OFFSETS_H__' echo '#define __ASM_OFFSETS_H__' echo '/*' echo ' * DO NOT MODIFY' echo ' *' echo " * This file was generated by arch/$(ARCH)/Makefile.in." echo ' *' echo ' */' echo '' awk ' /^->$/{printf("\n")} /^-> /{ sym = $2; val = $3; sub(/^\$/, "", val); $1 = ""; $2 = ""; $3 = ""; printf("%-20s = %3d\t/* 0x%x\t%s */\n", sym, val, val, $0) } ' echo '#endif' Generated arch/i386/asm-offsets.h #ifndef __ASM_OFFSETS_H__ #define __ASM_OFFSETS_H__ /* * DO NOT MODIFY * * This file was generated by arch/i386/Makefile.in. * */ state = 0 /* 0x0 offsetof(struct task_struct, state) */ flags = 4 /* 0x4 offsetof(struct task_struct, flags) */ sigpending = 8 /* 0x8 offsetof(struct task_struct, sigpending) */ addr_limit = 12 /* 0xc offsetof(struct task_struct, addr_limit) */ exec_domain = 16 /* 0x10 offsetof(struct task_struct, exec_domain) */ need_resched = 20 /* 0x14 offsetof(struct task_struct, need_resched) */ tsk_ptrace = 24 /* 0x18 offsetof(struct task_struct, ptrace) */ processor = 52 /* 0x34 offsetof(struct task_struct, processor) */ ENOSYS = 38 /* 0x26 ENOSYS */ #endif ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-04 11:47 [RFC] Standard way of generating assembler offsets Keith Owens @ 2001-10-04 15:36 ` george anzinger 2001-10-05 5:48 ` Keith Owens 2001-10-06 5:21 ` Keith Owens ` (2 subsequent siblings) 3 siblings, 1 reply; 14+ messages in thread From: george anzinger @ 2001-10-04 15:36 UTC (permalink / raw) To: Keith Owens; +Cc: linux-kernel This sound great. It has been a struggle for me for some time. A couple of comments: The symbol name IMHO should contain both the member name and the structure name. Otherwise there may be a problem if two structures use the same member name (flags comes to mind). This is way down on the list, but is it possible to generate a separate file for each *.S AND put the "required symbols" in the *.S. When I last did this we put a comment in the *.S that had a special format and changed the make rule for *.S->*.o to first build the required file. The *.S then used #include "sym*.h" to get the symbols. I think the standard make rule even removed the sys*.h file after the *.o was built. The actual section in the *.S that defined the required symbols was written as cpp macros which were stripped with a sed script. The result (with a standard header and tail) was compiled and run to generate the sym*.h file. Even the includes needed to generate the symbols were in the *.S so a dep script can generate the needed stuff for the *.S. The nice thing about this method is that it is easy to keep track of the required symbols and the make dependencies are simple (and easily derived). George Keith Owens wrote: > > Almost every architecture generates Assembler values to map the offsets > of fields in C structures, about the only exception is i386 and that is > because its offsets are hard coded into entry.S. Every arch has done > it differently, none of them have got it exactly right. > > As part of kbuild 2.5 I am standardizing on one method for generating > Assembler offsets. This change is required for kbuild 2.5 but it can > be added to 2.4 without disturbing the current kbuild, I want to do > this gradually now instead of a single massive change in kernel 2.5. I > will be issuing per architecture changes for generating Assembler > offsets against 2.4. > > The kbuild 2.5 method for generating Assembler offsets satisfies these > requirements: > > * No manual intervention required. Many architectures rely on users > running make dep after changing config options that affect the > Assembler offsets. If the user forgets to run make dep then the C > and Assembler code is out of sync - totally unacceptable. This is > completely fixed in kbuild 2.5; I cannot do a complete fix in kbuild > 2.4 but it is still better than the existing manual system. > > * Standard name for the related files. There are 6+ different names > for the files used to generate Assembler offsets, kbuild 2.5 uses > asm-offsets.[csh] on all architectures. > > * Allows for multiple parallel compiles from the same source tree. > Writing the generated asm-offsets.h to include/asm is not an option, > it prevents concurrent compiles. > > * The method must work in native and cross compile mode and give > exactly the same results. Some 2.4 code only works in native mode, > some architectures have different methods for native and cross > compile with different output formats. Yeuch! > > * Standard scripts for generating the output. Every arch does it > differently in 2.4, standards are good! > > * Correct dependency trees. Because 2.4 make dep does not scan .S > files, there is little or no dependency information. Even if the > offsets are regenerated, the affected Assembler code does not always > get rebuilt. kbuild 2.5 handles dependencies for Assembler as well > as C; I cannot get kbuild 2.4 perfect but I can improve on the > existing (or non-existent) 2.4 dependencies. > > All architectures will define arch/$(ARCH)/asm-offsets.c. This has a > standard prologue for the macros that convert offsets to Assembler, > followed by arch specific field references. > > arch/$(ARCH)/asm-offsets.s is generated from arch/$(ARCH)/asm-offsets.c > using standard rules, although kbuild 2.4 needs some tweaking. > > arch/$(ARCH)/asm-offsets.h is generated from arch/$(ARCH)/asm-offsets.s > by a semi-standard script. Most of the script is common to all > architectures but the precise format of the Assembler output is arch > specific. > > The final result is included in *only* the Assembler programs that need > it, as #include "asm-offsets.h" with -I arch/$(ARCH) in the relevant > Makefiles. Hard coding relative paths in source files is a pet hate, > use #include "localname.h" and -I instead. Including the generated > file in C code is not allowed, it severly pollutes the dependency > chain, to the extent that any config change can force a complete > recompile, unacceptable. > > Example from i386: > > arch/i386/asm-offsets.c > > /* > * Generate definitions needed by assembly language modules. > * This code generates raw asm output which is post-processed to extract > * and format the required data. > */ > > #include <linux/types.h> > #include <linux/stddef.h> > #include <linux/sched.h> > > /* Use marker if you need to separate the values later */ > > #define DEFINE(sym, val, marker) \ > asm volatile("\n-> " #sym " %0 " #val " " #marker : : "i" (val)) > > #define BLANK() asm volatile("\n->" : : ) > > int > main(void) > { > DEFINE(state, offsetof(struct task_struct, state),); > DEFINE(flags, offsetof(struct task_struct, flags),); > DEFINE(sigpending, offsetof(struct task_struct, sigpending),); > DEFINE(addr_limit, offsetof(struct task_struct, addr_limit),); > DEFINE(exec_domain, offsetof(struct task_struct, exec_domain),); > DEFINE(need_resched, offsetof(struct task_struct, need_resched),); > DEFINE(tsk_ptrace, offsetof(struct task_struct, ptrace),); > DEFINE(processor, offsetof(struct task_struct, processor),); > BLANK(); > DEFINE(ENOSYS, ENOSYS,); > return 0; > } > > asm-offsets.s to asm-offsets.h. > > # Convert raw asm offsets into something that can be included as > # assembler definitions. It converts > # -> symbol $value source > # into > # symbol = value /* 0xvalue source */ > > echo '#ifndef __ASM_OFFSETS_H__' > echo '#define __ASM_OFFSETS_H__' > echo '/*' > echo ' * DO NOT MODIFY' > echo ' *' > echo " * This file was generated by arch/$(ARCH)/Makefile.in." > echo ' *' > echo ' */' > echo '' > awk ' > /^->$/{printf("\n")} > /^-> /{ > sym = $2; > val = $3; > sub(/^\$/, "", val); > $1 = ""; > $2 = ""; > $3 = ""; > printf("%-20s = %3d\t/* 0x%x\t%s */\n", sym, val, val, $0) > } > ' > echo '#endif' > > Generated arch/i386/asm-offsets.h > > #ifndef __ASM_OFFSETS_H__ > #define __ASM_OFFSETS_H__ > /* > * DO NOT MODIFY > * > * This file was generated by arch/i386/Makefile.in. > * > */ > > state = 0 /* 0x0 offsetof(struct task_struct, state) */ > flags = 4 /* 0x4 offsetof(struct task_struct, flags) */ > sigpending = 8 /* 0x8 offsetof(struct task_struct, sigpending) */ > addr_limit = 12 /* 0xc offsetof(struct task_struct, addr_limit) */ > exec_domain = 16 /* 0x10 offsetof(struct task_struct, exec_domain) */ > need_resched = 20 /* 0x14 offsetof(struct task_struct, need_resched) */ > tsk_ptrace = 24 /* 0x18 offsetof(struct task_struct, ptrace) */ > processor = 52 /* 0x34 offsetof(struct task_struct, processor) */ > > ENOSYS = 38 /* 0x26 ENOSYS */ > #endif > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-04 15:36 ` george anzinger @ 2001-10-05 5:48 ` Keith Owens 0 siblings, 0 replies; 14+ messages in thread From: Keith Owens @ 2001-10-05 5:48 UTC (permalink / raw) To: george anzinger; +Cc: linux-kernel On Thu, 04 Oct 2001 08:36:54 -0700, george anzinger <george@mvista.com> wrote: >The symbol name IMHO should contain both the member name and the >structure name. Otherwise there may be a problem if two structures use >the same member name (flags comes to mind). The asm symbol name is arch defined, I am defining the standard method, not the asm names. It is up to the arch maintainers to pick suitable names, e.g. ia64 does DEFINE(IA64_SWITCH_STACK_AR_BSPSTORE_OFFSET, offsetof(struct switch_stack, ar_bspstore),); >This is way down on the list, but is it possible to generate a separate >file for each *.S AND put the "required symbols" in the *.S. That works for a small number of mappings but not when there are a large number that are required in several places. Take a look at arch/ia64/tools/print_offsets.c, 130+ mappings used by 5 or 6 different asm sources. There are also technical reasons (to do with the kernel CONFIG system) why a single asm-offsets file is easier to maintain. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-04 11:47 [RFC] Standard way of generating assembler offsets Keith Owens 2001-10-04 15:36 ` george anzinger @ 2001-10-06 5:21 ` Keith Owens 2001-10-08 9:35 ` Pantelis Antoniou 2001-10-08 9:49 ` David S. Miller 3 siblings, 0 replies; 14+ messages in thread From: Keith Owens @ 2001-10-06 5:21 UTC (permalink / raw) To: linux-kernel First patch to clean up Assembler offsets, as described in http://marc.theaimsgroup.com/?l=linux-kernel&m=100219616028442&w=2 This patch is larger than normal because my testing highlighted a long standing race in the top level kernel Makefile. i386 patch. Yes, I know that doing asm-offsets.c for i386 is overkill, it only needs a few values that "never" change so they can be hard coded. However I need a reference implementation, plus the race needs to be mitigated. BTW, this is *much* cleaner in kbuild 2.5. Bits of the patch are workarounds for recursive make problems and to handle mkdep not processing Assembler dependencies :(. Index: 11-pre4.1/Makefile --- 11-pre4.1/Makefile Fri, 05 Oct 2001 15:05:09 +1000 kaos (linux-2.4/T/c/50_Makefile 1.1.2.15.1.2.2.25.2.2.1.17.1.4.1.29 644) +++ 11-pre4.1(w)/Makefile Sat, 06 Oct 2001 15:07:25 +1000 kaos (linux-2.4/T/c/50_Makefile 1.1.2.15.1.2.2.25.2.2.1.17.1.4.1.29 644) @@ -205,7 +205,8 @@ CLEAN_FILES = \ drivers/scsi/53c700-mem.c \ net/khttpd/make_times_h \ net/khttpd/times.h \ - submenu* + submenu* \ + arch/$(ARCH)/asm-offsets.[sh] # directories removed with 'make clean' CLEAN_DIRS = \ modules @@ -245,9 +246,9 @@ export CPPFLAGS CFLAGS AFLAGS export NETWORKS DRIVERS LIBS HEAD LDFLAGS LINKFLAGS MAKEBOOT ASFLAGS .S.s: - $(CPP) $(AFLAGS) -traditional -o $*.s $< + $(CPP) $(AFLAGS) $(AFLAGS_$@) -traditional -o $@ $< .S.o: - $(CC) $(AFLAGS) -traditional -c -o $*.o $< + $(CC) $(AFLAGS) $(AFLAGS_$@) -traditional -c -o $@ $< Version: dummy @rm -f include/linux/compile.h @@ -273,29 +274,63 @@ symlinks: mkdir include/linux/modules; \ fi -oldconfig: symlinks +# There is a race here against anything built by the top level Makefile or by +# arch/$(ARCH)/Makefile (directly included by the top level Makefile). + +# split-include changes timestamps on config settings without telling make. +# Because make is not aware that config files have been changed, the timestamp +# checks in .hdepend (see Rules.make) use the old times for config files. +# Therefore anything built at the same time that split-include is running gets +# wrong timestamps for config settings and for the include files that depend on +# config settings. + +# I cannot completely fix this in kbuild 2.4 without moving most of the top +# level Makefile into a second file and doing make -C on that file after +# split-include has run. Explicitly running split-include after *config instead +# of delaying until make dep mitigates the race, although it does not completely +# remove it. + +# The race is not a problem for SUBDIRS processed from the top level Makefile +# because split-include has completed before they are entered. However it is a +# problem for top level builds and for directories invoked from +# arch/$(ARCH)/Makefile without using SUBDIRS. + +# kbuild 2.5 has a clean separation between configuration (top level and +# scripts/Makefile) and the rest of the kernel build, so it does not suffer from +# this race. Also kbuild 2.5 splits arch/$(ARCH)/Makefile into definitions +# (Makefile.defs.{no}config) and kernel building commands (Makefile.in). +# KAO + +split_include := scripts/split-include include/linux/autoconf.h include/config && touch include/config/MARKER + +oldconfig: symlinks scripts/split-include $(CONFIG_SHELL) scripts/Configure -d arch/$(ARCH)/config.in + @$(split_include) -xconfig: symlinks +xconfig: symlinks scripts/split-include $(MAKE) -C scripts kconfig.tk wish -f scripts/kconfig.tk + @$(split_include) -menuconfig: include/linux/version.h symlinks +menuconfig: include/linux/version.h symlinks scripts/split-include $(MAKE) -C scripts/lxdialog all $(CONFIG_SHELL) scripts/Menuconfig arch/$(ARCH)/config.in + @$(split_include) -config: symlinks +config: symlinks scripts/split-include $(CONFIG_SHELL) scripts/Configure arch/$(ARCH)/config.in + @$(split_include) include/config/MARKER: scripts/split-include include/linux/autoconf.h - scripts/split-include include/linux/autoconf.h include/config - @ touch include/config/MARKER + @$(split_include) linuxsubdirs: $(patsubst %, _dir_%, $(SUBDIRS)) -$(patsubst %, _dir_%, $(SUBDIRS)) : dummy include/linux/version.h include/config/MARKER +$(patsubst %, _dir_%, $(SUBDIRS)) : dummy include/linux/version.h include/config/MARKER before_subdirs $(MAKE) CFLAGS="$(CFLAGS) $(CFLAGS_KERNEL)" -C $(patsubst _dir_%, %, $@) +.PHONY: before_subdirs + $(TOPDIR)/include/linux/version.h: include/linux/version.h $(TOPDIR)/include/linux/compile.h: include/linux/compile.h @@ -356,7 +391,7 @@ endif modules: $(patsubst %, _mod_%, $(SUBDIRS)) .PHONY: $(patsubst %, _mod_%, $(SUBDIRS)) -$(patsubst %, _mod_%, $(SUBDIRS)) : include/linux/version.h include/config/MARKER +$(patsubst %, _mod_%, $(SUBDIRS)) : include/linux/version.h include/config/MARKER before_subdirs $(MAKE) -C $(patsubst _mod_%, %, $@) CFLAGS="$(CFLAGS) $(MODFLAGS)" MAKING_MODULES=1 modules .PHONY: modules_install Index: 11-pre4.1/arch/i386/asm-offsets.c --- 11-pre4.1/arch/i386/asm-offsets.c Sat, 06 Oct 2001 15:07:38 +1000 kaos () +++ 11-pre4.1(w)/arch/i386/asm-offsets.c Fri, 05 Oct 2001 21:53:18 +1000 kaos (linux-2.4/q/f/50_asm-offset 644) @@ -0,0 +1,32 @@ +/* + * Generate definitions needed by assembly language modules. + * This code generates raw asm output which is post-processed to extract + * and format the required data. + */ + +#include <linux/types.h> +#include <linux/stddef.h> +#include <linux/sched.h> + +/* Use marker if you need to separate the values later */ + +#define DEFINE(sym, val, marker) \ + asm volatile("\n-> " #sym " %0 " #val " " #marker : : "i" (val)) + +#define BLANK() asm volatile("\n->" : : ) + +int +main(void) +{ + DEFINE(state, offsetof(struct task_struct, state),); + DEFINE(flags, offsetof(struct task_struct, flags),); + DEFINE(sigpending, offsetof(struct task_struct, sigpending),); + DEFINE(addr_limit, offsetof(struct task_struct, addr_limit),); + DEFINE(exec_domain, offsetof(struct task_struct, exec_domain),); + DEFINE(need_resched, offsetof(struct task_struct, need_resched),); + DEFINE(tsk_ptrace, offsetof(struct task_struct, ptrace),); + DEFINE(processor, offsetof(struct task_struct, processor),); + BLANK(); + DEFINE(ENOSYS, ENOSYS,); + return 0; +} Index: 11-pre4.1/arch/i386/kernel/entry.S --- 11-pre4.1/arch/i386/kernel/entry.S Mon, 01 Oct 2001 12:23:40 +1000 kaos (linux-2.4/S/c/24_entry.S 1.1.5.5 644) +++ 11-pre4.1(w)/arch/i386/kernel/entry.S Sat, 06 Oct 2001 13:04:03 +1000 kaos (linux-2.4/S/c/24_entry.S 1.1.5.5 644) @@ -68,20 +68,7 @@ IF_MASK = 0x00000200 NT_MASK = 0x00004000 VM_MASK = 0x00020000 -/* - * these are offsets into the task-struct. - */ -state = 0 -flags = 4 -sigpending = 8 -addr_limit = 12 -exec_domain = 16 -need_resched = 20 -tsk_ptrace = 24 -processor = 52 - -ENOSYS = 38 - +#include "asm-offsets.h" #define SAVE_ALL \ cld; \ Index: 11-pre4.1/arch/i386/kernel/Makefile --- 11-pre4.1/arch/i386/kernel/Makefile Wed, 19 Sep 2001 14:59:20 +1000 kaos (linux-2.4/S/c/21_Makefile 1.1.2.1 644) +++ 11-pre4.1(w)/arch/i386/kernel/Makefile Fri, 05 Oct 2001 22:37:10 +1000 kaos (linux-2.4/S/c/21_Makefile 1.1.2.1 644) @@ -8,7 +8,7 @@ # Note 2! The CFLAGS definitions are now in the main makefile... .S.o: - $(CC) $(AFLAGS) -traditional -c $< -o $*.o + $(CC) $(AFLAGS) $(EXTRA_AFLAGS) $(AFLAGS_$@) -traditional -c $< -o $@ all: kernel.o head.o init_task.o @@ -40,5 +40,8 @@ obj-$(CONFIG_SMP) += smp.o smpboot.o tr obj-$(CONFIG_X86_LOCAL_APIC) += mpparse.o apic.o nmi.o obj-$(CONFIG_X86_IO_APIC) += io_apic.o obj-$(CONFIG_X86_VISWS_APIC) += visws_apic.o + +AFLAGS_entry.o += -I $(TOPDIR)/arch/$(ARCH) +entry.o: $(TOPDIR)/arch/$(ARCH)/asm-offsets.h include $(TOPDIR)/Rules.make Index: 11-pre4.1/arch/i386/Makefile --- 11-pre4.1/arch/i386/Makefile Wed, 18 Apr 2001 11:00:10 +1000 kaos (linux-2.4/T/c/37_Makefile 1.1.2.1.2.1 644) +++ 11-pre4.1(w)/arch/i386/Makefile Sat, 06 Oct 2001 15:07:31 +1000 kaos (linux-2.4/T/c/37_Makefile 1.1.2.1.2.1 644) @@ -145,3 +145,45 @@ archmrproper: archdep: @$(MAKEBOOT) dep + +# Convert raw asm offsets into something that can be included as +# assembler definitions. It converts +# -> symbol $value source +# into +# symbol = value /* 0xvalue source */ + +arch/$(ARCH)/asm-offsets.h: arch/$(ARCH)/asm-offsets.s + @echo Creating $@ + @set -e; \ + (echo "#ifndef __ASM_OFFSETS_H__"; \ + echo "#define __ASM_OFFSETS_H__"; \ + echo "/*"; \ + echo " * DO NOT MODIFY"; \ + echo " *"; \ + echo " * This file was generated by arch/$(ARCH)/Makefile."; \ + echo " *"; \ + echo " */"; \ + echo ""; \ + awk "/^->\$$/{printf(\"\\n\");} \ + /^-> /{ \ + sym = \$$2; \ + val = \$$3; \ + sub(/^\\\$$/, \"\", val); \ + \$$1 = \"\"; \ + \$$2 = \"\"; \ + \$$3 = \"\"; \ + printf(\"%-20s = %3d\\t\\t\\t/* 0x%x\\t%s */\\n\", \ + sym, val, val, \$$0) \ + }"; \ + echo ""; \ + echo "#endif"; \ + ) < $< > $@; + +arch/$(ARCH)/asm-offsets.s: include/config/MARKER \ + include/linux/sched.h \ + include/linux/types.h \ + include/linux/stddef.h + +# Kludge to prevent subdirs from being processed before asm-offsets.h is created. +# I hate recursive make! KAO. +before_subdirs: arch/$(ARCH)/asm-offsets.h ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-04 11:47 [RFC] Standard way of generating assembler offsets Keith Owens 2001-10-04 15:36 ` george anzinger 2001-10-06 5:21 ` Keith Owens @ 2001-10-08 9:35 ` Pantelis Antoniou 2001-10-08 17:29 ` george anzinger 2001-10-08 17:56 ` Georg Nikodym 2001-10-08 9:49 ` David S. Miller 3 siblings, 2 replies; 14+ messages in thread From: Pantelis Antoniou @ 2001-10-08 9:35 UTC (permalink / raw) To: linux-kernel Hi there. If anyone is interested I have already made a perl script that produces assembler offsets from structure members. It doesn't need to run native since it reads the header files, extract the structures and by using objdump calculates the offsets automatically. Maybe it needs some more work for what you describe, but it's exactly what you describe. If you're interested please email me directly for more information. Regards ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-08 9:35 ` Pantelis Antoniou @ 2001-10-08 17:29 ` george anzinger 2001-10-08 17:56 ` Georg Nikodym 1 sibling, 0 replies; 14+ messages in thread From: george anzinger @ 2001-10-08 17:29 UTC (permalink / raw) To: Pantelis Antoniou; +Cc: linux-kernel Pantelis Antoniou wrote: > > Hi there. > > If anyone is interested I have already made a perl > script that produces assembler offsets from structure > members. > > It doesn't need to run native since it reads the > header files, extract the structures and by using > objdump calculates the offsets automatically. > > Maybe it needs some more work for what you describe, > but it's exactly what you describe. > > If you're interested please email me directly for > more information. > One of the problems with this sort of thing is that it has a hard time getting the CPP macros right. The best way to do this sort of thing is to actually compile the header file with all the CONFIG defines and a set of tools (read macros) that produce the required offsets. This way you get what you want and don't have to reinvent the CPP stuff. It also allows production of #define constants and other constructs that folks push into CPP, in a very simple and straight forward manner. Been there, done that. George ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-08 9:35 ` Pantelis Antoniou 2001-10-08 17:29 ` george anzinger @ 2001-10-08 17:56 ` Georg Nikodym 2001-10-08 19:00 ` george anzinger 1 sibling, 1 reply; 14+ messages in thread From: Georg Nikodym @ 2001-10-08 17:56 UTC (permalink / raw) To: george anzinger; +Cc: Pantelis Antoniou, Linux Kernel List At the risk of sticking my foot in it, is there something wrong with the ANSI C offsetof() macro, defined in <stddef.h>? --Georg ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-08 17:56 ` Georg Nikodym @ 2001-10-08 19:00 ` george anzinger 2001-10-09 7:38 ` Pantelis Antoniou 2001-10-09 9:28 ` David S. Miller 0 siblings, 2 replies; 14+ messages in thread From: george anzinger @ 2001-10-08 19:00 UTC (permalink / raw) To: Georg Nikodym; +Cc: Pantelis Antoniou, Linux Kernel List Georg Nikodym wrote: > > At the risk of sticking my foot in it, is there something wrong with the > ANSI C offsetof() macro, defined in <stddef.h>? > > --Georg No, and it could have been (and was) written prio to ANSI C defining it. Something like: #define offsetof(x, instruct) &((struct instruct *)0)->x The issues that CPP resolves have to deal with the following sort of structure: struct instruct { struct foo * bar; #ifdef CONFIG_OPTION_DIDDLE int diddle_flag; int diddle_array[CONFIG_DIDDLE_SIZE]; #endif int x; } Or for the simple need for a constant: #define Const (CONFIG_DIDDLE_SIZE * sizeof(int)) Of course you could have any number of constant operators in the expression. Note also, that the array in the structure above is defined by a CONFIG symbol. This could also involve math, i.e.: #define CONFIG_DIDDLE_SIZE CLOCK_RATE / HZ and so on. All in all, it best to let CPP do what it does best and scarf up the result: #define GENERATE_CONSTANT(name,c) printf(#name " equ %d\n",c) then: GENERATE_CONSTANT(diddle_size,CONFIG_DIDDLE_SIZE); In the code we did, we put all the GENERATE macros in a .h file. The the code looked like: #include.... all the headers needed.... #include <generate.h> GENERATE.... all the generate macro calls... } // all done (assumes that the "main(){" is in the generate.h file) This whole mess was included as comments in the asm file. The make rule then used a sed script to extract it, compile and run it to produce a new header file which the asm source included outside of the above stuff. George ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-08 19:00 ` george anzinger @ 2001-10-09 7:38 ` Pantelis Antoniou 2001-10-09 9:28 ` David S. Miller 1 sibling, 0 replies; 14+ messages in thread From: Pantelis Antoniou @ 2001-10-09 7:38 UTC (permalink / raw) To: linux-kernel [-- Attachment #1: Type: text/plain, Size: 2041 bytes --] george anzinger wrote: > > Georg Nikodym wrote: > > > > At the risk of sticking my foot in it, is there something wrong with the > > ANSI C offsetof() macro, defined in <stddef.h>? > > > > --Georg > No, and it could have been (and was) written prio to ANSI C defining > it. Something like: > > #define offsetof(x, instruct) &((struct instruct *)0)->x > > The issues that CPP resolves have to deal with the following sort of > structure: > > struct instruct { > struct foo * bar; > #ifdef CONFIG_OPTION_DIDDLE > int diddle_flag; > int diddle_array[CONFIG_DIDDLE_SIZE]; > #endif > int x; > } > > Or for the simple need for a constant: > > #define Const (CONFIG_DIDDLE_SIZE * sizeof(int)) > > Of course you could have any number of constant operators in the > expression. Note also, that the array in the structure above is defined > by a CONFIG symbol. This could also involve math, i.e.: > > #define CONFIG_DIDDLE_SIZE CLOCK_RATE / HZ > > and so on. All in all, it best to let CPP do what it does best and > scarf up the result: > > #define GENERATE_CONSTANT(name,c) printf(#name " equ %d\n",c) > > then: > > GENERATE_CONSTANT(diddle_size,CONFIG_DIDDLE_SIZE); > > In the code we did, we put all the GENERATE macros in a .h file. The > the code looked like: > > #include.... all the headers needed.... > > #include <generate.h> > > GENERATE.... all the generate macro calls... > > } // all done (assumes that the "main(){" is in the generate.h file) > > This whole mess was included as comments in the asm file. The make rule > then used a sed script to extract it, compile and run it to produce a > new header file which the asm source included outside of the above > stuff. > > George My script already handles that, everything is first passed through CPP and the member offset are varied correctly according to any compilation options. I included the script and the results of two runs. 1. ./h2inc tst.h >tst.inc 2. ./h2inc --cflags="-DSHOW_HIDDEN -I./" >tst-hidden.inc Regards [-- Attachment #2: h2inc --] [-- Type: text/plain, Size: 18360 bytes --] #!/usr/bin/perl -w use integer; use Getopt::Long; use File::Basename; use File::stat; my $CC = "cc"; my $CFLAGS = "-I./"; my $CPPFLAGS = "-E -dD"; my $OBJCOPY = "objcopy"; my $OBJFLAGS = "-O binary"; sub find_32bit_type; sub target_endianess; sub alligned_type_size; sub base_type_size; sub members_offset; sub tmpfile; sub inputfile; sub decode_type; sub decode_member; sub find_complex_member_name; sub find_matching_brace; $Getopt::Long::ignorecase = 0; # Don't ignore case my @filelist = (); GetOptions ( "cc=s" => \$CC, "cflags=s" => \$CFLAGS, "cppflags=s" => \$CPPFLAGS, "objcopy=s" => \$OBJCOPY, "objflags=s" => \$OBJFLAGS ); `$CC 2>/dev/null -v`; die "Compiler is not present", if $? != 0; my $u32 = &find_32bit_type(); my $endianess = &target_endianess($u32); for ($i = 0; $i <= $#ARGV; $i++) { $_ = $ARGV[$i]; # print STDERR "file: $_\n"; push @filelist, $_; } $#filelist >= 0 || die "Filename missing\n"; my %members = (); my %typedefs = (); my %structs = (); my %unions = (); my %ilist = (); # hash of included files my $ilist; my $defines; foreach $f (@filelist) { $f = basename($f); %ilist = (); undef $ilist; $defines = ""; &inputfile($f); my @defines = split(/\n/, $defines); my $incfile = $f; $incfile =~ s/\.h$/.inc/g; $incfile =~ s/\S*\/(\S+\.inc)/$1/g; my $incfiledef = "_" . uc($incfile); $incfiledef =~ s/\./_/g; print "#ifndef $incfiledef\n"; print "#define $incfiledef\n\n"; if (defined ($ilist)) { my $if; foreach $if (split /\s/, $ilist) { $if =~ s/\.h$/.inc/g; $if =~ s/\S*\/(\S+\.inc)/$1/g; $if = basename($if); print "#include \"$if\"\n"; } print "\n"; } my @offsets; my $b; my $i; my $j; my $k; foreach $b (sort keys %members) { my @m = split /\s/, $members{$b}; my $m; print "/****************************************************************\n\n"; print "\tOffsets for $b\n\n"; print "****************************************************************/\n\n"; my $size_define; my $complete_type; if ($typedefs{$b}) { $complete_type = $b; $size_define = $b . "_SIZE"; } elsif ($structs{$b}) { $complete_type = "struct $b"; $size_define = "struct_" . $b . "_SIZE"; } elsif ($unions{$b}) { $complete_type = "union $b"; $size_define = "union_" . $b . "_SIZE"; } else { die; } @offsets = &members_offset($endianess, $u32, "#include \"$f\"\n", $complete_type, \@m); $j = 0; foreach (@offsets) { my $cm = $m[$j++]; my @cm = split(/\./, $cm); for ($k = 0; $k <= $#defines; $k++) { my @tt = split(/\s+/, $defines[$k]); if (defined($tt[2]) && $tt[2] eq $cm) { # print STDERR "define alias found for $cm\n"; $defines[$k] = "/* $tt[1] removed as an alias for $cm */"; @cm = ($tt[1]); last; } } $cm = join('_', @cm); printf("#define %-30s %3d\n", $cm, $_); } printf("#define %-30s %3d\n", $size_define, &base_type_size($endianess, $u32, "#include \"$f\"\n", $complete_type)); print "\n"; } $defines = join("\n", @defines); if ($defines ne "") { print "/****************************************************************\n\n"; print "\tSimple defines list \n\n"; print "****************************************************************/\n\n"; print "$defines\n\n"; } print "#endif\n"; } sub inputfile { my $file = shift(@_); $file = basename($file); my @wf = (); # whole file my $wf = ''; my $size; my $align; # print STDERR "Processing : $file\n"; my $tmp = &tmpfile(); my $tmp_c = $tmp . ".c"; my $tmp_o = $tmp . ".o"; local $SIG{'INT'} = sub { unlink $tmp_c; unlink $tmp_o; die $_[0]; }; open(TMPFILE, ">$tmp_c") || die; print TMPFILE "#include \"$file\"\n"; close(TMPFILE); # first verify that the header file is correct `$CC $CFLAGS -c $tmp_c -o $tmp_o`; if ($? != 0) { # try system wide include open(TMPFILE, ">$tmp_c") || die; print TMPFILE "#include \<$file\>\n"; close(TMPFILE); # first verify that the header file is correct `$CC $CFLAGS -c $tmp_c -o $tmp_o`; die, if ($? != 0); } unlink $tmp_o; # remove object file # open(LOGFILE, ">log.i") || die; # print LOGFILE "$CC $CFLAGS $CPPFLAGS $tmp_c |\n"; open(CPPPIPE, "$CC $CFLAGS $CPPFLAGS $tmp_c |") || die; $wf = ""; my $cf = ""; # current file - only output for current file my $last; my $lll; while (<CPPPIPE>) { # print LOGFILE $_; $lll = $_; chop; if (s/\\$//) { $_ .= <CPPPIPE>; redo; } # check for escape at the end of line $last = $_; my $ll = $_; if (! /\s*\#\s*/) { if (basename($cf) eq basename($file)) { $wf .= "$_\n"; } next; } # print STDERR "\$\_='$_'\n"; # print STDERR "o: \$\_='$_', \$\`='$`', \$\&='$&', \$\'='$'\n"; if (/\s+[0-9]+\s*\"([^\"].*)\"/) { # print STDERR "$cf - $1\n"; my $c = basename($1); if (!defined($ilist{$c})) { next, if ($c eq basename($tmp_c)); # do not add the dummy if ($c ne basename($file) && basename($cf) eq basename($file)) { # do not add self, and only directly included # print STDERR "$ll: $1\n"; $ilist{$c} = 1; if (defined($ilist)) { $ilist .= " $c"; } else { $ilist = $c; } } } $cf = basename($1); } elsif (/\s*define\s*([a-zA-Z_][a-zA-Z0-9_]*)/) { # $_ = $'; next, if ($cf ne $file); # print STDERR "\$lll='$lll'\n\$last='$last'\n\$\_='$_'\n\$\`='$`'\n\$\&='$&'\n\$\'='$'\n"; my $defname = $1; next, if (/\(.*\)/); # ignore arguments $_ = $'; if (/\S+.*$/) { my $what = $&; # next, if ($what =~ /[()]/); # only simple defines pass # $defines .= "#define $defname $1\n"; # print STDERR "1-> '$defname' '$what'\n"; $defines .= sprintf("#define %-30s %s\n", $defname, $what); } else { # print STDERR "2-> $defname\n"; $defines .= sprintf("#define %-30s\n", $defname); } } else { # print STDERR "$_"; } } # close(LOGFILE); # print STDERR "FILE:\n" . $wf ."FEND:\n"; $wf =~ s/([*;,.!~{}()+\-\\\/\[\]])/ $1 /gsx; # add spaces $wf =~ s/\s+/ /gsx; # print STDERR $wf; @wf = split /\s/, $wf; $i = 0; OUTTER: while ($i < ($#wf + 1)) { # print STDERR "$i: '$wf[$i]'\n"; if ($wf[$i] eq "typedef" && $wf[$i+1] eq "struct") { &decode_type(\@wf, \$i); } elsif ($wf[$i] eq "typedef" && $wf[$i+1] eq "union") { &decode_type(\@wf, \$i); } elsif ($wf[$i] eq "struct") { &decode_type(\@wf, \$i); } elsif ($wf[$i] eq "union") { &decode_type(\@wf, \$i); } else { $i++; } } close CPPPIPE; unlink $tmp_c; local $SIG{'INT'} = 'DEFAULT'; return 1; } sub decode_member { my $wf = shift(@_); # reference to the whole body of the file my $ms = shift(@_); # start of member definition my $me = shift(@_); # end of member definition my $ii; my $i; my $n; # decode right to left $i = $me; $i--, if ($$wf[$i] eq ";"); while ($$wf[$i] eq "]") { # array definition, find match # print STDERR "'$$wf[$i]'\n"; $n = 1; do { $i--; # print STDERR "'$$wf[$i]'\n"; $n++, if ($$wf[$i] eq "]"); $n--, if ($$wf[$i] eq "["); } while ($n > 0 || $$wf[$i] ne "["); $i--; } while ($$wf[$i] eq ")") { # function pointer definition, find match $n = 1; $ii = $i; # print STDERR "'$$wf[$ii]'\n"; do { $ii--; # print STDERR "'$$wf[$ii]'\n"; $n++, if ($$wf[$ii] eq ")"); $n--, if ($$wf[$ii] eq "("); } while ($n > 0 || $$wf[$ii] ne "("); # now check if next token is a pointer then we've found it # print STDERR "i=$i, ii=$ii, '" . $$wf[$ii] . "' '" . $$wf[$ii+1] . "' '" . $$wf[$ii+2] . "'\n"; if ($i - $ii == 3 && $$wf[$ii+1] eq "*") { # found it! $i = $ii + 2; } else { $i = $ii - 1; } } # print STDERR "found; $$wf[$i]\n"; return $$wf[$i]; } sub find_complex_member_name { my $wf = shift(@_); # reference to the whole body of the file my $k = shift(@_); # start of member definition my $last = shift(@_); # end of member definition my $ct = $$wf[$k]; my $ccs = $k + 2; my $n = 0; do { $k++; $n++, if ($$wf[$k] eq "{"); $n--, if ($$wf[$k] eq "}"); } while ($n > 0); my $cce = $k; $k++; my $cs = $k; $k++, while ($$wf[$k] ne ";"); my $ce = $k; $k++; my $cn = &decode_member($wf, $cs, $ce); return $cn; } sub find_matching_brace { my $wf = shift(@_); # reference to the whole body of the file my $k = shift(@_); # start of member definition my $last = shift(@_); # end of member definition my $n = 0; do { $k++; $n++, if ($$wf[$k] eq "{"); $n--, if ($$wf[$k] eq "}"); } while ($n > 0); $k++; $k++, while ($$wf[$k] ne ";"); $k++; return $k; } sub get_next_type_name { } sub decode_type { my $wf = shift(@_); # reference to the whole body of the file my $i = shift(@_); # reference to the token index at the file my $n = 0; # nesting level my $k; my $j; my $anchor; my $first_brace; my $last_brace; my $trailer; my @result = (); $anchor = $$i; # keep this for later # now find the brace $j = $anchor; # first find opening brace or terminating semicolon $j++, while ($$wf[$j] ne "{" && $$wf[$j] ne ";"); if ($$wf[$j] eq ";") { # not a structure definition $$i = $j + 1; return ""; } $first_brace = $j; $n = 1; do { $j++; $n++, if ($$wf[$j] eq "{"); $n--, if ($$wf[$j] eq "}"); } while ($n > 0 || $$wf[$j] ne "}"); $last_brace = $j; # find terminating semicolon $j++, while ($$wf[$j] ne ";"); my $is_typedef = $$wf[$anchor] eq "typedef"; my $is_struct = $$wf[$anchor] eq "struct"; my $is_union = $$wf[$anchor] eq "union"; my $is_declaration = 0; my $base_type; if ($is_typedef) { my $ii = $j - 1; # remove semicolon # print STDERR "is_typedef\n"; # print STDERR "'$$wf[$ii]' "; while ($$wf[$ii] eq "]") { # array definition, find match $n = 1; do { $ii--; # print STDERR "'$$wf[$ii]' "; $n++, if ($$wf[$ii] eq "]"); $n--, if ($$wf[$ii] eq "["); } while ($n > 0 || $$wf[$ii] ne "["); $ii--; } $base_type = $$wf[$ii]; if ($$wf[$j-1] eq "]") { $$i = $j; print STDERR "typedef arrays not supported; skipping $base_type\n"; return ""; } $typedefs{$base_type} = $is_typedef; $structs{$base_type} = $is_struct; $unions{$base_type} = $is_union; } else { # print STDERR "!is_typedef\n"; $base_type = $$wf[$anchor] . " " . $$wf[$anchor + 1]; $base_type = $$wf[$anchor + 1]; if ($base_type ne "{") { $typedefs{$base_type} = $is_typedef; $structs{$base_type} = $is_struct; $unions{$base_type} = $is_union; } else { $is_declaration = 1; } } # print STDERR "sizeof($base_type)\n"; my $ms; my $me; my $member; my @nt = (); my $cs; my $ce; my $ct; my $cn; my $ccs; my $cce; $ct = ""; $n = 0; for ($ms = $first_brace + 1, $me = $first_brace+1; $me < $last_brace; ) { # print STDERR "$$wf[$me-1] \>$$wf[$me]\< $$wf[$me+1]\n"; if (($$wf[$me] eq "union" || $$wf[$me] eq "struct") && $$wf[$me+1] eq "{") { $cn = &find_complex_member_name($wf, $me, $last_brace); # print STDERR "found $ct $cn\n"; if (! $is_declaration) { if ($ct eq "struct" || length($cn) > 2) { if (defined($members{$base_type})) { $members{$base_type} .= " $cn"; } else { $members{$base_type} = $cn; } } } $ms = $me + 2; $me = $ms; push @nt, $cn; # print STDERR "pushed $cn, \@nt = @nt\n"; next; } if ($$wf[$me] eq "}") { $me++; $cn = pop @nt; # print STDERR "poped $cn, \@nt = @nt\n"; $me++, while ($me < $last_brace && $$wf[$me] ne ";"); $ms = $me + 1; $me = $ms; next; } $me++, while ($me < $last_brace && $$wf[$me] ne ";"); last, if ($me >= $last_brace); $me--; # remove semicolon last, if ($ms >= $me); # protection for empty types if (! $is_declaration) { $member = &decode_member($wf, $ms, $me); # print STDERR "offsetof($base_type, $member)\n"; $member = join('.', @nt, $member); if (defined($members{$base_type})) { $members{$base_type} .= " $member"; } else { $members{$base_type} = $member; } } $ms = $me + 2; $me = $ms; } $$i = $j; return ""; } sub tmpfile { my $tmp_dir = -d '/tmp' ? '/tmp' : $ENV{TMP} || $ENV{TEMP}; my $tmp_name = sprintf("%s/%s-%d-%d", $tmp_dir, basename($0), $$, time()); return $tmp_name; } sub target_endianess { my $thirty_two_bits_type = shift(@_); my $extra_code = shift(@_); if (!defined($extra_code)) { $extra_code = ""; } my $tmp_dir = -d '/tmp' ? '/tmp' : $ENV{TMP} || $ENV{TEMP}; my $tmp_name = sprintf("%s/%s-%d-%d", $tmp_dir, basename($0), $$, time()); my $tmp_c = "$tmp_name.c"; my $tmp_o = "$tmp_name.o"; my $tmp_bin = "$tmp_name.bin"; my $t_size; my $sb; my $fsize; my $i; unlink $tmp_c; local $SIG{'INT'} = sub { unlink $tmp_c; unlink $tmp_o; unlink $tmp_bin; die $_[0]; }; open(TMPFILE, ">$tmp_c") || die "Could not create temp file"; print TMPFILE "$extra_code\n"; print TMPFILE "const $thirty_two_bits_type tmp_val = 0x01234567LU;\n"; close(TMPFILE); `$CC $CFLAGS -c $tmp_c -o $tmp_o`; die, if $? != 0; `$OBJCOPY $OBJFLAGS $tmp_o $tmp_bin`; die, if $? != 0; $sb = stat($tmp_bin); $fsize = $sb->size; my $read_bytes; my $buf; open(TMPFILE, "<$tmp_bin") || die "Could not open binary file"; ($read_bytes = read TMPFILE, $buf, 4) == 4 || die "File has illegal size"; my @v; ($v[0], $v[1], $v[2], $v[3]) = unpack("C4", $buf); close(TMPFILE); unlink $tmp_c; unlink $tmp_o; unlink $tmp_bin; local $SIG{'INT'} = 'DEFAULT'; if ($v[0] == 0x01 && $v[1] == 0x23 && $v[2] == 0x45 && $v[3] == 0x67) { return("big"); } if ($v[3] == 0x01 && $v[2] == 0x23 && $v[1] == 0x45 && $v[0] == 0x67) { return("little"); } return("unknown"); } sub alligned_type_size { my $type = shift(@_); my $type_val = shift(@_); if (!defined($type_val)) { $type_val = "0"; } my $array_size = shift(@_); if (!defined($array_size)) { $array_size = 16; } my $extra_code = shift(@_); if (!defined($extra_code)) { $extra_code = ""; } my $tmp_dir = -d '/tmp' ? '/tmp' : $ENV{TMP} || $ENV{TEMP}; my $tmp_name = sprintf("%s/%s-%d-%d", $tmp_dir, basename($0), $$, time()); my $tmp_c = "$tmp_name.c"; my $tmp_o = "$tmp_name.o"; my $tmp_bin = "$tmp_name.bin"; my $t_size; my $sb; my $fsize; my $i; # first find out the size of the size_t type unlink $tmp_c; local $SIG{'INT'} = sub { unlink $tmp_c; unlink $tmp_o; unlink $tmp_bin; die $_[0]; }; open(TMPFILE, ">$tmp_c") || die "Could not create temp file"; print TMPFILE "$extra_code\n"; print TMPFILE "const $type tmp_array[$array_size] = {\n"; for ($i = 0; $i < $array_size; $i++) { print TMPFILE "$type_val,\n"; } print TMPFILE "};\n"; close(TMPFILE); `$CC $CFLAGS -c $tmp_c -o $tmp_o`; die, if $? != 0; `$OBJCOPY $OBJFLAGS $tmp_o $tmp_bin`; die, if $? != 0; $sb = stat($tmp_bin); $fsize = $sb->size; $t_size = $fsize / $array_size; unlink $tmp_c; unlink $tmp_o; unlink $tmp_bin; local $SIG{'INT'} = 'DEFAULT'; return $t_size; } sub members_offset { my $endianess = shift(@_); # big or little my $thirty_two_bits_type = shift(@_); my $extra_code = shift(@_); # my $base = shift(@_); # base type my $members = shift(@_); # member to find the offset my $offset = 0; my @offsets = (); my $i; my $j; my $tmp_dir = -d '/tmp' ? '/tmp' : $ENV{TMP} || $ENV{TEMP}; my $tmp_name = sprintf("%s/%s-%d-%d", $tmp_dir, basename($0), $$, time()); my $tmp_c = "$tmp_name.c"; my $tmp_o = "$tmp_name.o"; my $tmp_bin = "$tmp_name.bin"; unlink $tmp_c; local $SIG{'INT'} = sub { unlink $tmp_c; unlink $tmp_o; unlink $tmp_bin; die $_[0]; }; open(TMPFILE, ">$tmp_c") || die "Could not create temp file"; print TMPFILE "$extra_code\n"; print TMPFILE "#include <stddef.h>\n"; $j = $#$members + 1; print TMPFILE "const $thirty_two_bits_type offset_table[$j] = {\n"; foreach (@$members) { print TMPFILE "\toffsetof($base, $_),\n"; } print TMPFILE "};\n"; close(TMPFILE); `$CC $CFLAGS -c $tmp_c -o $tmp_o`; die, if $? != 0; `$OBJCOPY $OBJFLAGS $tmp_o $tmp_bin`; die, if $? != 0; my $read_bytes; my $buf; open(TMPFILE, "<$tmp_bin") || die "Could not open binary file"; for ($i = 0; $i < $j; $i++) { ($read_bytes = read TMPFILE, $buf, 4) == 4 || die "File has illegal size"; if ($endianess eq "big") { ($offset) = unpack("N", $buf); # big endian 32 bits } else { ($offset) = unpack("V", $buf); # little endian 32 bits } push @offsets, $offset; } close(TMPFILE); unlink $tmp_c; unlink $tmp_o; unlink $tmp_bin; local $SIG{'INT'} = 'DEFAULT'; return(@offsets); } sub base_type_size { my $endianess = shift(@_); # big or little my $thirty_two_bits_type = shift(@_); my $extra_code = shift(@_); # my $base = shift(@_); # base type my $size = 0; my $tmp_dir = -d '/tmp' ? '/tmp' : $ENV{TMP} || $ENV{TEMP}; my $tmp_name = sprintf("%s/%s-%d-%d", $tmp_dir, basename($0), $$, time()); my $tmp_c = "$tmp_name.c"; my $tmp_o = "$tmp_name.o"; my $tmp_bin = "$tmp_name.bin"; unlink $tmp_c; local $SIG{'INT'} = sub { unlink $tmp_c; unlink $tmp_o; unlink $tmp_bin; die $_[0]; }; open(TMPFILE, ">$tmp_c") || die "Could not create temp file"; print TMPFILE "$extra_code\n"; print TMPFILE "#include <stddef.h>\n"; print TMPFILE "const $thirty_two_bits_type size = sizeof($base);\n"; close(TMPFILE); `$CC $CFLAGS -c $tmp_c -o $tmp_o`; die, if $? != 0; `$OBJCOPY $OBJFLAGS $tmp_o $tmp_bin`; die, if $? != 0; my $read_bytes; my $buf; open(TMPFILE, "<$tmp_bin") || die "Could not open binary file"; ($read_bytes = read TMPFILE, $buf, 4) == 4 || die "File has illegal size"; if ($endianess eq "big") { ($size) = unpack("N", $buf); # big endian 32 bits } else { ($size) = unpack("V", $buf); # little endian 32 bits } close(TMPFILE); unlink $tmp_c; unlink $tmp_o; unlink $tmp_bin; local $SIG{'INT'} = 'DEFAULT'; return($size); } sub find_32bit_type { my $sizeof_int = &alligned_type_size("int", "0", 16, ""); my $sizeof_short = &alligned_type_size("short", "0", 16, ""); my $sizeof_long = &alligned_type_size("long", "0", 16, ""); my $u32; if ($sizeof_int == 4) { $u32 = "unsigned int"; $s32 = "signed int"; } elsif ($sizeof_long == 4) { $u32 = "unsigned long"; $s32 = "signed long"; } elsif ($sizeof_short == 4) { $u32 = "unsigned short"; $s32 = "signed short"; } else { die "Not one 32 bit type found!\n"; } return $u32; } [-- Attachment #3: tst.h --] [-- Type: text/plain, Size: 201 bytes --] #ifndef TST_H #define TST_H #include "tst2.h" #define TST_DEF 0x10 struct tst { int tst_member_a; union { char *tst_u_str; } tst_member_b; #ifdef SHOW_HIDDEN int tst_hidden; #endif }; #endif [-- Attachment #4: tst2.h --] [-- Type: text/plain, Size: 38 bytes --] #ifndef TST2_H #define TST2_H #endif [-- Attachment #5: tst.inc --] [-- Type: text/plain, Size: 629 bytes --] #ifndef _TST_INC #define _TST_INC #include "tst2.inc" /**************************************************************** Offsets for tst ****************************************************************/ #define tst_member_a 0 #define tst_member_b 4 #define tst_member_b_tst_u_str 4 #define struct_tst_SIZE 8 /**************************************************************** Simple defines list ****************************************************************/ #define TST_H #define TST_DEF 0x10 #endif [-- Attachment #6: tst-hidden.inc --] [-- Type: text/plain, Size: 672 bytes --] #ifndef _TST_INC #define _TST_INC #include "tst2.inc" /**************************************************************** Offsets for tst ****************************************************************/ #define tst_member_a 0 #define tst_member_b 4 #define tst_member_b_tst_u_str 4 #define tst_hidden 8 #define struct_tst_SIZE 12 /**************************************************************** Simple defines list ****************************************************************/ #define TST_H #define TST_DEF 0x10 #endif [-- Attachment #7: tst2.inc --] [-- Type: text/plain, Size: 241 bytes --] #ifndef _TST2_INC #define _TST2_INC /**************************************************************** Simple defines list ****************************************************************/ #define TST2_H #endif ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-08 19:00 ` george anzinger 2001-10-09 7:38 ` Pantelis Antoniou @ 2001-10-09 9:28 ` David S. Miller 1 sibling, 0 replies; 14+ messages in thread From: David S. Miller @ 2001-10-09 9:28 UTC (permalink / raw) To: panto; +Cc: linux-kernel I think your work is way over-engineered and that you really need to look at the very simple method by which we do this on sparc64. Franks a lot, David S. Miller davem@redhat.com ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-04 11:47 [RFC] Standard way of generating assembler offsets Keith Owens ` (2 preceding siblings ...) 2001-10-08 9:35 ` Pantelis Antoniou @ 2001-10-08 9:49 ` David S. Miller 2001-10-09 7:25 ` Pantelis Antoniou ` (2 more replies) 3 siblings, 3 replies; 14+ messages in thread From: David S. Miller @ 2001-10-08 9:49 UTC (permalink / raw) To: panto; +Cc: linux-kernel From: Pantelis Antoniou <panto@intracom.gr> Date: Mon, 08 Oct 2001 12:35:27 +0300 If anyone is interested I have already made a perl script that produces assembler offsets from structure members. It doesn't need to run native since it reads the header files, extract the structures and by using objdump calculates the offsets automatically. BTW, I assume you have already taken a look at how we do this on Sparc64. See arch/sparc64/kernel/check_asm.sh and the "check_asm" target in arch/sparc64/kernel/Makefile It also works in all cross-compilation etc. environments. And I bet it would work on every platform with very minimal changes, if any. Franks a lot, David S. Miller davem@redhat.com ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-08 9:49 ` David S. Miller @ 2001-10-09 7:25 ` Pantelis Antoniou 2001-10-09 9:24 ` David S. Miller 2001-10-14 11:27 ` Keith Owens 2 siblings, 0 replies; 14+ messages in thread From: Pantelis Antoniou @ 2001-10-09 7:25 UTC (permalink / raw) To: linux-kernel "David S. Miller" wrote: > > From: Pantelis Antoniou <panto@intracom.gr> > Date: Mon, 08 Oct 2001 12:35:27 +0300 > > If anyone is interested I have already made a perl > script that produces assembler offsets from structure > members. > > It doesn't need to run native since it reads the > header files, extract the structures and by using > objdump calculates the offsets automatically. > > BTW, I assume you have already taken a look at how we > do this on Sparc64. See arch/sparc64/kernel/check_asm.sh > and the "check_asm" target in arch/sparc64/kernel/Makefile > > It also works in all cross-compilation etc. environments. > And I bet it would work on every platform with very minimal > changes, if any. > > Franks a lot, > David S. Miller > davem@redhat.com I've look at your script and it kinda flew over my head. Would you mind explain this a bit? Thanks ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-08 9:49 ` David S. Miller 2001-10-09 7:25 ` Pantelis Antoniou @ 2001-10-09 9:24 ` David S. Miller 2001-10-14 11:27 ` Keith Owens 2 siblings, 0 replies; 14+ messages in thread From: David S. Miller @ 2001-10-09 9:24 UTC (permalink / raw) To: panto; +Cc: linux-kernel From: Pantelis Antoniou <panto@intracom.gr> Date: Tue, 09 Oct 2001 10:25:19 +0300 I've look at your script and it kinda flew over my head. Would you mind explain this a bit? We generate the offsets as data items in an assembler file, then we parse out those data section entries and spit them into the header. It allows cross compilation setups to work, ie. even in cases when you cannot generate and run a binary. Franks a lot, David S. Miller davem@redhat.com ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [RFC] Standard way of generating assembler offsets 2001-10-08 9:49 ` David S. Miller 2001-10-09 7:25 ` Pantelis Antoniou 2001-10-09 9:24 ` David S. Miller @ 2001-10-14 11:27 ` Keith Owens 2 siblings, 0 replies; 14+ messages in thread From: Keith Owens @ 2001-10-14 11:27 UTC (permalink / raw) To: David S. Miller; +Cc: linux-kernel On Mon, 08 Oct 2001 02:49:46 -0700 (PDT), "David S. Miller" <davem@redhat.com> wrote: >BTW, I assume you have already taken a look at how we >do this on Sparc64. See arch/sparc64/kernel/check_asm.sh >and the "check_asm" target in arch/sparc64/kernel/Makefile > >It also works in all cross-compilation etc. environments. >And I bet it would work on every platform with very minimal >changes, if any. I finally had time to look at sparc64 check_asm. The code suffers from several problems: * It is only built at make dep time so any patches after make dep that affect the asm offsets are not picked up. * The code has to know about all the config options that affect the structures, it manually tests for SMP and SPIN_LOCK_DEBUG. That is bad enough but other architectures are much worse, several config settings can change the asm offsets, each config setting would have to be manually defined. * The asm output lists all fields in the input structures. It is arguable if this is a good or bad thing, I prefer to explicitly define just the symbols required by asm so I view this as a bad feature. * check_asm can only generate offsetof and sizeof fields. So calculated fields like AOFF_task_fpregs have to be done elsewhere. * The code is complex. I thought my asm-offsets.c->.s->.h conversion was complicated but asm_check takes the cake. Here is a first cut at doing sparc64 asm offsets the same way that i386 and ia64 are done in kbuild 2.5. * It uses .c->.s->h so it plugs directly into the kbuild 2.5 dependency handling, any patch or config changes that affect asm-offsets.h will automatically rebuild asm-offsets.h. * Only the fields actually used by asm are listed. I may have missed a field or two but gcc will tell us that. * asm-offsets.c DEFINE() can do any calculation. So AOFF_task_fpregs is now calculated here. * Less complex :). * The addition of comments against each define makes the result more readable. ==== arch/sparc64/asm-offsets.c ==== /* * Generate definitions needed by assembly language modules. * This code generates raw asm output which is post-processed to extract * and format the required data. */ #include <linux/types.h> #include <linux/stddef.h> #include <linux/sched.h> /* Use marker if you need to separate the values later */ #define DEFINE(sym, val, marker) \ asm volatile("\n-> " #sym " %0 " #val " " #marker : : "i" (val)) #define BLANK() asm volatile("\n->" : : ) int main(void) { DEFINE(AOFF_mm_context, offsetof(struct mm_struct, context),); BLANK(); DEFINE(AOFF_task_blocked, offsetof(struct task_struct, blocked),); DEFINE(AOFF_task_egid, offsetof(struct task_struct, egid),); DEFINE(ASIZ_task_egid, sizeof(((struct task_struct *)NULL)->egid),); DEFINE(AOFF_task_euid, offsetof(struct task_struct, euid),); DEFINE(ASIZ_task_euid, sizeof(((struct task_struct *)NULL)->euid),); DEFINE(AOFF_task_flags, offsetof(struct task_struct, flags),); DEFINE(AOFF_task_fpregs, (sizeof(struct task_struct) + (64 - 1)) & ~(64 - 1),); DEFINE(AOFF_task_gid, offsetof(struct task_struct, gid),); DEFINE(ASIZ_task_gid, sizeof(((struct task_struct *)NULL)->gid),); DEFINE(AOFF_task_need_resched, offsetof(struct task_struct, need_resched),); DEFINE(AOFF_task_personality, offsetof(struct task_struct, personality),); DEFINE(ASIZ_task_personality, sizeof(((struct task_struct *)NULL)->personality),); DEFINE(AOFF_task_pid, offsetof(struct task_struct, pid),); DEFINE(AOFF_task_p_opptr, offsetof(struct task_struct, p_opptr),); DEFINE(AOFF_task_processor, offsetof(struct task_struct, processor),); DEFINE(AOFF_task_ptrace, offsetof(struct task_struct, ptrace),); DEFINE(AOFF_task_sigpending, offsetof(struct task_struct, sigpending),); DEFINE(AOFF_task_thread, offsetof(struct task_struct, thread),); DEFINE(AOFF_task_uid, offsetof(struct task_struct, uid),); DEFINE(ASIZ_task_uid, sizeof(((struct task_struct *)NULL)->uid),); BLANK(); DEFINE(AOFF_thread_current_ds, offsetof(struct thread_struct, current_ds),); DEFINE(AOFF_thread_fault_address, offsetof(struct thread_struct, fault_address),); DEFINE(AOFF_thread_fault_code, offsetof(struct thread_struct, fault_code),); DEFINE(AOFF_thread_flags, offsetof(struct thread_struct, flags),); DEFINE(AOFF_thread_fork_kpsr, offsetof(struct thread_struct, fork_kpsr),); DEFINE(AOFF_thread_fpdepth, offsetof(struct thread_struct, fpdepth),); DEFINE(AOFF_thread_fpsaved, offsetof(struct thread_struct, fpsaved),); DEFINE(AOFF_thread_gsr, offsetof(struct thread_struct, gsr),); DEFINE(AOFF_thread_kernel_cntd0, offsetof(struct thread_struct, kernel_cntd0),); DEFINE(AOFF_thread_pcr_reg, offsetof(struct thread_struct, pcr_reg),); DEFINE(AOFF_thread_reg_window, offsetof(struct thread_struct, reg_window),); DEFINE(AOFF_thread_rwbuf_stkptrs, offsetof(struct thread_struct, rwbuf_stkptrs),); DEFINE(AOFF_thread_use_blkcommit, offsetof(struct thread_struct, use_blkcommit),); DEFINE(AOFF_thread_utraps, offsetof(struct thread_struct, utraps),); DEFINE(AOFF_thread_uwinmask, offsetof(struct thread_struct, uwinmask),); DEFINE(AOFF_thread_w_saved, offsetof(struct thread_struct, w_saved),); DEFINE(AOFF_thread_xfsr, offsetof(struct thread_struct, xfsr),); return 0; } ==== arch/sparc64/Makefile.in ==== # Convert raw asm offsets into something that can be included as # assembler definitions. It converts # -> symbol $value source # into # #define symbol value /* 0xvalue source */ user_command(asm-offsets.h ($(objfile asm-offsets.s)) (set -e; (echo "#ifndef __ASM_OFFSETS_H__"; echo "#define __ASM_OFFSETS_H__"; echo "/*"; echo " * DO NOT MODIFY"; echo " *"; echo " * This file was generated by arch/$(ARCH)/Makefile.in."; echo " *"; echo " */"; echo ""; awk "/^->\$$/{printf(\"\\n\");} /^-> /{ sym = \$$2; val = \$$3; sub(/^\\\$$/, \"\", val); \$$1 = \"\"; \$$2 = \"\"; \$$3 = \"\"; printf(\"#define %-24s %4d\\t\\t\\t/* 0x%x\\t%s */\\n\", sym, val, val, \$$0) }"; echo ""; echo "#endif"; ) < $< > $@; ) () ) ==== arch/sparc64/asm-offsets.h output ==== This is partial output and was actually generated on i386 so the numbers are wrong but it shows what the output will look like. I do not have access to a machine for compiling 2.4 sparc64 kernels so none of the thread values are in this sample. #ifndef __ASM_OFFSETS_H__ #define __ASM_OFFSETS_H__ /* * DO NOT MODIFY * * This file was generated by arch/sparc64/Makefile.in. * */ #define AOFF_mm_context 128 /* 0x80 offsetof(struct mm_struct, context) */ #define AOFF_task_blocked 1632 /* 0x660 offsetof(struct task_struct, blocked) */ #define AOFF_task_egid 572 /* 0x23c offsetof(struct task_struct, egid) */ #define ASIZ_task_egid 4 /* 0x4 sizeof(((struct task_struct *)NULL)->egid) */ #define AOFF_task_euid 556 /* 0x22c offsetof(struct task_struct, euid) */ #define ASIZ_task_euid 4 /* 0x4 sizeof(((struct task_struct *)NULL)->euid) */ #define AOFF_task_flags 4 /* 0x4 offsetof(struct task_struct, flags) */ #define AOFF_task_fpregs 1728 /* 0x6c0 (sizeof(struct task_struct) + (64 - 1)) & ~(64 - 1) */ #define AOFF_task_gid 568 /* 0x238 offsetof(struct task_struct, gid) */ #define ASIZ_task_gid 4 /* 0x4 sizeof(((struct task_struct *)NULL)->gid) */ #define AOFF_task_need_resched 20 /* 0x14 offsetof(struct task_struct, need_resched) */ #define AOFF_task_personality 116 /* 0x74 offsetof(struct task_struct, personality) */ #define ASIZ_task_personality 4 /* 0x4 sizeof(((struct task_struct *)NULL)->personality) */ #define AOFF_task_pid 124 /* 0x7c offsetof(struct task_struct, pid) */ #define AOFF_task_p_opptr 148 /* 0x94 offsetof(struct task_struct, p_opptr) */ #define AOFF_task_processor 52 /* 0x34 offsetof(struct task_struct, processor) */ #define AOFF_task_ptrace 24 /* 0x18 offsetof(struct task_struct, ptrace) */ #define AOFF_task_sigpending 8 /* 0x8 offsetof(struct task_struct, sigpending) */ #define AOFF_task_thread 880 /* 0x370 offsetof(struct task_struct, thread) */ #define AOFF_task_uid 552 /* 0x228 offsetof(struct task_struct, uid) */ #define ASIZ_task_uid 4 /* 0x4 sizeof(((struct task_struct *)NULL)->uid) */ /* Thread values would be here if I could compile for sparc64. */ #endif ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2001-10-14 11:27 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2001-10-04 11:47 [RFC] Standard way of generating assembler offsets Keith Owens 2001-10-04 15:36 ` george anzinger 2001-10-05 5:48 ` Keith Owens 2001-10-06 5:21 ` Keith Owens 2001-10-08 9:35 ` Pantelis Antoniou 2001-10-08 17:29 ` george anzinger 2001-10-08 17:56 ` Georg Nikodym 2001-10-08 19:00 ` george anzinger 2001-10-09 7:38 ` Pantelis Antoniou 2001-10-09 9:28 ` David S. Miller 2001-10-08 9:49 ` David S. Miller 2001-10-09 7:25 ` Pantelis Antoniou 2001-10-09 9:24 ` David S. Miller 2001-10-14 11:27 ` Keith Owens
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).