* [PATCH v3 00/21] backports: add kernel integration support
@ 2014-11-11 8:14 Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper Luis R. Rodriguez
` (21 more replies)
0 siblings, 22 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
This third series should hopefully address all pending feedback.
I've split out patches even more, but more importantly have
made the parser simpler by both integrating the logic required
to understand a backport by identifying the backport by type
and strategy, sharing that, and then providing ignore lists on
the parsers.
The more complex change was dealing with the recursive _walk() in such
a way that we can have a modified src_line regexp depending on the type
of backport we are using. The parser will use a regexp that requires a
full path for the sources, it will also allow for using a variable for
the backport project directory if such is defined, and also provide a
verification helper which ensures backported will be used rather than
the older kernel code.
Luis R. Rodriguez (21):
backports: move legacy and SmPL patch application into helper
backports: ifdef around module_init() module_exit() for modules
backports: move packaging kconfig hacks to its own file
backports: grant parsers access to kconfig config and menuconfig
backports: keep track of original backported source symbols early on
backports: allow for different backport prefix and simple integration
support
backports: add prefix for integration on dependencies
backports: replace BACKPORT_PWD with BACKPORT_DIR
backports: fold outdir as a backport identity target
backports: avoid git tree reinitialization
backports: modify kconfig parser to use bpid and target_dir
backports: make kconfig src_line backport / ConfigTree specific
backports: use BACKPORT_DIR prefix on kconfig sources
backports: update dependencies map file
backports: split Kconfig into Kconfig.package and Kconfig.sources
backports: move version file generation to run earlier
backports: define C code backport version info using CPTCFG_
backports: add backport version parsing for kernel integration
backports: prefix c-file / h-file auto backport with BPAUTO
backports: remove extra BACKPORT_ prefix from kernel versioning
backports: add full kernel integration support
backport/Kconfig | 47 --
backport/Kconfig.integrate | 36 ++
backport/Kconfig.package | 24 +
backport/Kconfig.package.hacks | 8 +
backport/Kconfig.sources | 23 +
backport/Makefile | 8 +-
backport/Makefile.build | 4 +-
backport/Makefile.kernel | 19 +-
backport/Makefile.real | 2 +-
backport/backport-include/asm/dma-mapping.h | 4 +-
backport/backport-include/backport/backport.h | 5 +
backport/backport-include/backport/leds-disabled.h | 2 +-
backport/backport-include/linux/module.h | 8 +-
backport/compat/Kconfig | 84 +--
backport/compat/Makefile | 42 +-
backport/compat/backports.h | 4 +-
backport/compat/main.c | 40 +-
backport/scripts/uninstall.sh | 4 +-
dependencies | 31 +-
devel/doc/kconfig-operation | 13 +-
gentree.py | 599 ++++++++++++++-------
.../0001-enable-backports-built-in.patch | 40 ++
lib/bpversion.py | 48 ++
lib/kconfig.py | 196 ++++++-
patches/backport-adjustments/devcoredump.patch | 4 +-
.../media/0002-no_dmabuf/v4l2.patch | 6 +-
26 files changed, 910 insertions(+), 391 deletions(-)
delete mode 100644 backport/Kconfig
create mode 100644 backport/Kconfig.integrate
create mode 100644 backport/Kconfig.package
create mode 100644 backport/Kconfig.package.hacks
create mode 100644 backport/Kconfig.sources
create mode 100644 integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch
create mode 100644 lib/bpversion.py
--
2.1.1
^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
@ 2014-11-11 8:14 ` Luis R. Rodriguez
2016-07-04 9:33 ` Johannes Berg
2014-11-11 8:14 ` [PATCH v3 02/21] backports: ifdef around module_init() module_exit() for modules Luis R. Rodriguez
` (20 subsequent siblings)
21 siblings, 1 reply; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
This allows us to extend how backports uses patches for
different types of applications. This will later be used
for kernel integration support, for example.
This should have no functional change.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
gentree.py | 279 +++++++++++++++++++++++++++++++------------------------------
1 file changed, 144 insertions(+), 135 deletions(-)
diff --git a/gentree.py b/gentree.py
index 9b4db98..59ae19d 100755
--- a/gentree.py
+++ b/gentree.py
@@ -438,6 +438,149 @@ def upload_release(args, rel_prep, logwrite=lambda x:None):
kup_cmd = "kup put /\n\t\t%s /\n\t\t%s /\n\t\t%s" % (gzip_name, tar_name + '.asc', korg_path)
logwrite("kup-test: skipping cmd: %s" % kup_cmd)
+def apply_patches(args, desc, source_dir, patch_src, target_dir, logwrite=lambda x:None):
+ """
+ Given a path of a directories of patches and SmPL patches apply
+ them on the target directory. If requested refresh patches, or test
+ a specific SmPL patch.
+ """
+ logwrite('Applying patches from %s to %s ...' % (patch_src, target_dir))
+ test_cocci = args.test_cocci or args.profile_cocci
+ test_cocci_found = False
+ patches = []
+ sempatches = []
+ for root, dirs, files in os.walk(os.path.join(source_dir, patch_src)):
+ for f in files:
+ if not test_cocci and f.endswith('.patch'):
+ patches.append(os.path.join(root, f))
+ if f.endswith('.cocci'):
+ if test_cocci:
+ if f not in test_cocci:
+ continue
+ test_cocci_found = True
+ if args.test_cocci:
+ logwrite("Testing Coccinelle SmPL patch: %s" % test_cocci)
+ elif args.profile_cocci:
+ logwrite("Profiling Coccinelle SmPL patch: %s" % test_cocci)
+ sempatches.append(os.path.join(root, f))
+ patches.sort()
+ prefix_len = len(os.path.join(source_dir, patch_src)) + 1
+ for pfile in patches:
+ print_name = pfile[prefix_len:]
+ # read the patch file
+ p = patch.fromfile(pfile)
+ # complain if it's not a patch
+ if not p:
+ raise Exception('No patch content found in %s' % print_name)
+ # leading / seems to be stripped?
+ if 'dev/null' in p.items[0].source:
+ raise Exception('Patches creating files are not supported (in %s)' % print_name)
+ # check if the first file the patch touches exists, if so
+ # assume the patch needs to be applied -- otherwise continue
+ patched_file = '/'.join(p.items[0].source.split('/')[1:])
+ fullfn = os.path.join(target_dir, patched_file)
+ if not os.path.exists(fullfn):
+ if args.verbose:
+ logwrite("Not applying %s, not needed" % print_name)
+ continue
+ if args.verbose:
+ logwrite("Applying patch %s" % print_name)
+
+ if args.refresh:
+ # but for refresh, of course look at all files the patch touches
+ for patchitem in p.items:
+ patched_file = '/'.join(patchitem.source.split('/')[1:])
+ fullfn = os.path.join(target_dir, patched_file)
+ shutil.copyfile(fullfn, fullfn + '.orig_file')
+
+ process = subprocess.Popen(['patch', '-p1'], stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT, stdin=subprocess.PIPE,
+ close_fds=True, universal_newlines=True,
+ cwd=target_dir)
+ output = process.communicate(input=open(pfile, 'r').read())[0]
+ output = output.split('\n')
+ if output[-1] == '':
+ output = output[:-1]
+ if args.verbose:
+ for line in output:
+ logwrite('> %s' % line)
+ if process.returncode != 0:
+ if not args.verbose:
+ logwrite("Failed to apply changes from %s" % print_name)
+ for line in output:
+ logwrite('> %s' % line)
+ raise Exception('Patch failed')
+
+ if args.refresh:
+ pfilef = open(pfile + '.tmp', 'a')
+ pfilef.write(p.top_header)
+ pfilef.flush()
+ for patchitem in p.items:
+ patched_file = '/'.join(patchitem.source.split('/')[1:])
+ fullfn = os.path.join(target_dir, patched_file)
+ process = subprocess.Popen(['diff', '-p', '-u', patched_file + '.orig_file', patched_file,
+ '--label', 'a/' + patched_file,
+ '--label', 'b/' + patched_file],
+ stdout=pfilef, close_fds=True,
+ universal_newlines=True, cwd=target_dir)
+ process.wait()
+ os.unlink(fullfn + '.orig_file')
+ if not process.returncode in (0, 1):
+ logwrite("Failed to diff to refresh %s" % print_name)
+ pfilef.close()
+ os.unlink(pfile + '.tmp')
+ raise Exception('Refresh failed')
+ pfilef.close()
+ os.rename(pfile + '.tmp', pfile)
+
+ # remove orig/rej files that patch sometimes creates
+ for root, dirs, files in os.walk(target_dir):
+ for f in files:
+ if f[-5:] == '.orig' or f[-4:] == '.rej':
+ os.unlink(os.path.join(root, f))
+ git_debug_snapshot(args, "apply %s patch %s" % (desc, print_name))
+
+ sempatches.sort()
+ prefix_len = len(os.path.join(source_dir, patch_src)) + 1
+
+ for cocci_file in sempatches:
+ # Until Coccinelle picks this up
+ pycocci = os.path.join(source_dir, 'devel/pycocci')
+ cmd = [pycocci, cocci_file]
+ extra_spatch_args = []
+ if args.profile_cocci:
+ cmd.append('--profile-cocci')
+ cmd.append(os.path.abspath(target_dir))
+ print_name = cocci_file[prefix_len:]
+ if args.verbose:
+ logwrite("Applying SmPL patch %s" % print_name)
+ sprocess = subprocess.Popen(cmd,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+ close_fds=True, universal_newlines=True,
+ cwd=target_dir)
+ output = sprocess.communicate()[0]
+ sprocess.wait()
+ if sprocess.returncode != 0:
+ logwrite("Failed to process SmPL patch %s" % print_name)
+ raise Exception('SmPL patch failed')
+ output = output.split('\n')
+ if output[-1] == '':
+ output = output[:-1]
+ if args.verbose:
+ for line in output:
+ logwrite('> %s' % line)
+
+ # remove cocci_backup files
+ for root, dirs, files in os.walk(target_dir):
+ for f in files:
+ if f.endswith('.cocci_backup'):
+ os.unlink(os.path.join(root, f))
+ git_debug_snapshot(args, "apply %s SmPL patch %s" % (desc, print_name))
+
+ if test_cocci and test_cocci_found:
+ logwrite('Done!')
+ sys.exit(0)
+
def _main():
# Our binary requirements go here
req = reqs.Req()
@@ -620,141 +763,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
bpcfg.disable_symbols(disable_list)
git_debug_snapshot(args, 'Add automatic backports')
- test_cocci = args.test_cocci or args.profile_cocci
-
- logwrite('Apply patches ...')
- patches = []
- sempatches = []
- for root, dirs, files in os.walk(os.path.join(source_dir, 'patches')):
- for f in files:
- if not test_cocci and f.endswith('.patch'):
- patches.append(os.path.join(root, f))
- if f.endswith('.cocci'):
- if test_cocci:
- if f not in test_cocci:
- continue
- if args.test_cocci:
- logwrite("Testing Coccinelle SmPL patch: %s" % test_cocci)
- elif args.profile_cocci:
- logwrite("Profiling Coccinelle SmPL patch: %s" % test_cocci)
- sempatches.append(os.path.join(root, f))
- patches.sort()
- prefix_len = len(os.path.join(source_dir, 'patches')) + 1
- for pfile in patches:
- print_name = pfile[prefix_len:]
- # read the patch file
- p = patch.fromfile(pfile)
- # complain if it's not a patch
- if not p:
- raise Exception('No patch content found in %s' % print_name)
- # leading / seems to be stripped?
- if 'dev/null' in p.items[0].source:
- raise Exception('Patches creating files are not supported (in %s)' % print_name)
- # check if the first file the patch touches exists, if so
- # assume the patch needs to be applied -- otherwise continue
- patched_file = '/'.join(p.items[0].source.split('/')[1:])
- fullfn = os.path.join(args.outdir, patched_file)
- if not os.path.exists(fullfn):
- if args.verbose:
- logwrite("Not applying %s, not needed" % print_name)
- continue
- if args.verbose:
- logwrite("Applying patch %s" % print_name)
-
- if args.refresh:
- # but for refresh, of course look at all files the patch touches
- for patchitem in p.items:
- patched_file = '/'.join(patchitem.source.split('/')[1:])
- fullfn = os.path.join(args.outdir, patched_file)
- shutil.copyfile(fullfn, fullfn + '.orig_file')
-
- process = subprocess.Popen(['patch', '-p1'], stdout=subprocess.PIPE,
- stderr=subprocess.STDOUT, stdin=subprocess.PIPE,
- close_fds=True, universal_newlines=True,
- cwd=args.outdir)
- output = process.communicate(input=open(pfile, 'r').read())[0]
- output = output.split('\n')
- if output[-1] == '':
- output = output[:-1]
- if args.verbose:
- for line in output:
- logwrite('> %s' % line)
- if process.returncode != 0:
- if not args.verbose:
- logwrite("Failed to apply changes from %s" % print_name)
- for line in output:
- logwrite('> %s' % line)
- return 2
-
- if args.refresh:
- pfilef = open(pfile + '.tmp', 'a')
- pfilef.write(p.top_header)
- pfilef.flush()
- for patchitem in p.items:
- patched_file = '/'.join(patchitem.source.split('/')[1:])
- fullfn = os.path.join(args.outdir, patched_file)
- process = subprocess.Popen(['diff', '-p', '-u', patched_file + '.orig_file', patched_file,
- '--label', 'a/' + patched_file,
- '--label', 'b/' + patched_file],
- stdout=pfilef, close_fds=True,
- universal_newlines=True, cwd=args.outdir)
- process.wait()
- os.unlink(fullfn + '.orig_file')
- if not process.returncode in (0, 1):
- logwrite("Failed to diff to refresh %s" % print_name)
- pfilef.close()
- os.unlink(pfile + '.tmp')
- return 3
- pfilef.close()
- os.rename(pfile + '.tmp', pfile)
-
- # remove orig/rej files that patch sometimes creates
- for root, dirs, files in os.walk(args.outdir):
- for f in files:
- if f[-5:] == '.orig' or f[-4:] == '.rej':
- os.unlink(os.path.join(root, f))
- git_debug_snapshot(args, "apply backport patch %s" % print_name)
-
- sempatches.sort()
- prefix_len = len(os.path.join(source_dir, 'patches')) + 1
-
- for cocci_file in sempatches:
- # Until Coccinelle picks this up
- pycocci = os.path.join(source_dir, 'devel/pycocci')
- cmd = [pycocci, cocci_file]
- extra_spatch_args = []
- if args.profile_cocci:
- cmd.append('--profile-cocci')
- cmd.append(os.path.abspath(args.outdir))
- print_name = cocci_file[prefix_len:]
- if args.verbose:
- logwrite("Applying SmPL patch %s" % print_name)
- sprocess = subprocess.Popen(cmd,
- stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
- close_fds=True, universal_newlines=True,
- cwd=args.outdir)
- output = sprocess.communicate()[0]
- sprocess.wait()
- if sprocess.returncode != 0:
- logwrite("Failed to process SmPL patch %s" % print_name)
- return 3
- output = output.split('\n')
- if output[-1] == '':
- output = output[:-1]
- if args.verbose:
- for line in output:
- logwrite('> %s' % line)
-
- # remove cocci_backup files
- for root, dirs, files in os.walk(args.outdir):
- for f in files:
- if f.endswith('.cocci_backup'):
- os.unlink(os.path.join(root, f))
- git_debug_snapshot(args, "apply backport SmPL patch %s" % print_name)
-
- if test_cocci:
- logwrite('Done!')
- return 0
+ apply_patches(args, "backport", source_dir, 'patches', args.outdir, logwrite)
# some post-processing is required
configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 02/21] backports: ifdef around module_init() module_exit() for modules
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper Luis R. Rodriguez
@ 2014-11-11 8:14 ` Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 03/21] backports: move packaging kconfig hacks to its own file Luis R. Rodriguez
` (19 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
We only need this for modules.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/backport-include/linux/module.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/backport/backport-include/linux/module.h b/backport/backport-include/linux/module.h
index 82c96bd..e8f28b1 100644
--- a/backport/backport-include/linux/module.h
+++ b/backport/backport-include/linux/module.h
@@ -25,6 +25,7 @@ extern void backport_dependency_symbol(void);
" using backports " BACKPORTS_VERSION);
#endif
+#ifdef MODULE
#undef module_init
#define module_init(initfn) \
static int __init __init_backport(void) \
@@ -58,6 +59,7 @@ extern void backport_dependency_symbol(void);
rcu_barrier(); \
} \
void cleanup_module(void) __attribute__((alias("__exit_compat")));
+#endif
#if LINUX_VERSION_CODE < KERNEL_VERSION(3,3,0)
#undef param_check_bool
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 03/21] backports: move packaging kconfig hacks to its own file
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 02/21] backports: ifdef around module_init() module_exit() for modules Luis R. Rodriguez
@ 2014-11-11 8:14 ` Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 04/21] backports: grant parsers access to kconfig config and menuconfig Luis R. Rodriguez
` (18 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
Move packaging kconfig hacks to its own file, we'll be ignoring
this file later for postprocessing. Keeping this in a separate
file will make parsing much simpler.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/Kconfig | 9 ++-------
backport/Kconfig.package.hacks | 8 ++++++++
gentree.py | 3 ++-
3 files changed, 12 insertions(+), 8 deletions(-)
create mode 100644 backport/Kconfig.package.hacks
diff --git a/backport/Kconfig b/backport/Kconfig
index b14a268..a8f2867 100644
--- a/backport/Kconfig
+++ b/backport/Kconfig
@@ -14,13 +14,8 @@ config BACKPORTED_KERNEL_NAME
source Kconfig.kernel
source Kconfig.versions
-# some hacks ...
-config WIRELESS
- def_bool y
-config NET_CORE
- def_bool y
-config EXPERT
- def_bool y
+# Packaging hacks
+source Kconfig.package.hacks
# this has the configuration for the backport code
source compat/Kconfig
diff --git a/backport/Kconfig.package.hacks b/backport/Kconfig.package.hacks
new file mode 100644
index 0000000..6a429dd
--- /dev/null
+++ b/backport/Kconfig.package.hacks
@@ -0,0 +1,8 @@
+# some hacks for when we use backports to generate a package
+# to build modules out of tree.
+config WIRELESS
+ def_bool y
+config NET_CORE
+ def_bool y
+config EXPERT
+ def_bool y
diff --git a/gentree.py b/gentree.py
index 59ae19d..7a6deef 100755
--- a/gentree.py
+++ b/gentree.py
@@ -731,7 +731,8 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# do the copy
backport_files = [(x, x) for x in [
- 'Kconfig', 'Makefile', 'Makefile.build', 'Makefile.kernel', '.gitignore',
+ 'Kconfig', 'Kconfig.package.hacks',
+ 'Makefile', 'Makefile.build', 'Makefile.kernel', '.gitignore',
'Makefile.real', 'compat/', 'backport-include/', 'kconf/',
'scripts/', '.blacklist.map',
]]
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 04/21] backports: grant parsers access to kconfig config and menuconfig
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (2 preceding siblings ...)
2014-11-11 8:14 ` [PATCH v3 03/21] backports: move packaging kconfig hacks to its own file Luis R. Rodriguez
@ 2014-11-11 8:14 ` Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 05/21] backports: keep track of original backported source symbols early on Luis R. Rodriguez
` (17 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
Allow parsers to query the type of option used, either config
or menuconfig. This will be used later.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
lib/kconfig.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/kconfig.py b/lib/kconfig.py
index 179121a..3562630 100644
--- a/lib/kconfig.py
+++ b/lib/kconfig.py
@@ -7,7 +7,7 @@ import os, re
src_line = re.compile(r'^\s*source\s+"?(?P<src>[^\s"]*)"?\s*$')
tri_line = re.compile(r'^(?P<spc>\s+)tristate')
bool_line = re.compile(r'^(?P<spc>\s+)bool')
-cfg_line = re.compile(r'^(config|menuconfig)\s+(?P<sym>[^\s]*)')
+cfg_line = re.compile(r'^(?P<opt>config|menuconfig)\s+(?P<sym>[^\s]*)')
sel_line = re.compile(r'^(?P<spc>\s+)select\s+(?P<sym>[^\s]*)\s*$')
backport_line = re.compile(r'^\s+#(?P<key>[ch]-file|module-name)\s*(?P<name>.*)')
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 05/21] backports: keep track of original backported source symbols early on
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (3 preceding siblings ...)
2014-11-11 8:14 ` [PATCH v3 04/21] backports: grant parsers access to kconfig config and menuconfig Luis R. Rodriguez
@ 2014-11-11 8:14 ` Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 06/21] backports: allow for different backport prefix and simple integration support Luis R. Rodriguez
` (16 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
We'll later want to do some modifications of symbols we backport
when integrating, but before we do any modification on kconfig
files we should keep track of the original source symbols.
This commit introduces no functional changes right now, it will
become more useful after we start adjusting kconfig entries
for integration.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
gentree.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/gentree.py b/gentree.py
index 7a6deef..cb39a3f 100755
--- a/gentree.py
+++ b/gentree.py
@@ -768,6 +768,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# some post-processing is required
configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
+ orig_symbols = configtree.symbols()
logwrite('Modify Kconfig tree ...')
configtree.prune_sources(ignore=['Kconfig.kernel', 'Kconfig.versions'])
git_debug_snapshot(args, "prune Kconfig tree")
@@ -829,7 +830,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# rewrite Makefile and source symbols
regexes = []
- for some_symbols in [symbols[i:i + 50] for i in range(0, len(symbols), 50)]:
+ for some_symbols in [orig_symbols[i:i + 50] for i in range(0, len(orig_symbols), 50)]:
r = 'CONFIG_((' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
regexes.append(re.compile(r, re.MULTILINE))
for root, dirs, files in os.walk(args.outdir):
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 06/21] backports: allow for different backport prefix and simple integration support
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (4 preceding siblings ...)
2014-11-11 8:14 ` [PATCH v3 05/21] backports: keep track of original backported source symbols early on Luis R. Rodriguez
@ 2014-11-11 8:14 ` Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 07/21] backports: add prefix for integration on dependencies Luis R. Rodriguez
` (15 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
The way we backport when packaging is to minimize the amount of
changes required by taking advantage of the fact that Kconfig
can treat CONFIG_ as an environment variable with getenv()
when parsing menu entries. When doing integration we don't want
to do this so instead we'll rely on the CONFIG_BACKPORT prefix.
This requires a bit of work on our current parsers, so to do that
identify the backport and provide that to helpers.
This also provides initial basic support for integration where no
additional code was required other than prefixes or ensuring certain
things are not done for integration. Before we expose integration
support we'll need to add quite a bit more things so this just assumes
integration is never desired.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
devel/doc/kconfig-operation | 4 +--
gentree.py | 83 +++++++++++++++++++++++++++++++++++----------
2 files changed, 67 insertions(+), 20 deletions(-)
diff --git a/devel/doc/kconfig-operation b/devel/doc/kconfig-operation
index 0245889..f1ecf60 100644
--- a/devel/doc/kconfig-operation
+++ b/devel/doc/kconfig-operation
@@ -63,7 +63,7 @@ being compiled against is older than X.
All together, this allows the correct options to be selected by the user.
-There's one more caveat: At backport configuration time, the CONFIG_
+When using Linux backports as a packaged solution the the CONFIG_
environment variable is set to CPTCFG_ ("compat config", but also chosen
because it has the same length as CONFIG_). This shows up in the .config
file and other places, and in fact in all makefiles and in the code. The
@@ -73,8 +73,6 @@ This allows code to, for example, have "#ifdef CONFIG_PM" which can only
be set or cleared in the kernel, not in the backport configuration. Since
this is needed, a transformation step is done at backport creation time.
-
-
Backport creation for Kconfig
-------------------------------
diff --git a/gentree.py b/gentree.py
index cb39a3f..40b0a55 100755
--- a/gentree.py
+++ b/gentree.py
@@ -17,6 +17,28 @@ from lib import bpkup as kup
from lib.tempdir import tempdir
from lib import bpreqs as reqs
+class Bp_Identity(object):
+ """
+ folks considering multiple integrations may want to
+ consider stuffing versioning info here as well but
+ that will need thought/design on sharing compat and
+ module namespaces.
+
+ Use the *_resafe when combining on regexps, although we currently
+ don't support regexps there perhaps later we will and this will
+ just make things safer for the output regardless. Once and if those
+ are added, how we actually use the others for regular printing will
+ need to be considered.
+ """
+ def __init__(self, integrate=False, kconfig_prefix='CPTCFG_', project_prefix=''):
+ self.integrate = integrate
+ self.kconfig_prefix = kconfig_prefix
+ self.kconfig_prefix_resafe = re.escape(kconfig_prefix)
+ self.project_prefix = project_prefix
+ self.project_prefix_resafe = re.escape(project_prefix)
+ self.full_prefix = kconfig_prefix + project_prefix
+ self.full_prefix_resafe = re.escape(self.full_prefix)
+
def read_copy_list(copyfile):
"""
Read a copy-list file and return a list of (source, target)
@@ -195,7 +217,7 @@ def automatic_backport_mangle_c_file(name):
return name.replace('/', '-')
-def add_automatic_backports(args):
+def add_automatic_backports(args, bpid):
disable_list = []
export = re.compile(r'^EXPORT_SYMBOL(_GPL)?\((?P<sym>[^\)]*)\)')
bpi = kconfig.get_backport_info(os.path.join(args.outdir, 'compat', 'Kconfig'))
@@ -228,9 +250,9 @@ def add_automatic_backports(args):
raise Exception('backporting a module requires a #module-name')
for of in o_files:
mf.write('%s-objs += %s\n' % (module_name, of))
- mf.write('obj-$(CPTCFG_%s) += %s.o\n' % (sym, module_name))
+ mf.write('obj-$(%s%s) += %s.o\n' % (bpid.full_prefix, sym, module_name))
elif symtype == 'bool':
- mf.write('compat-$(CPTCFG_%s) += %s\n' % (sym, ' '.join(o_files)))
+ mf.write('compat-$(%s%s) += %s\n' % (bpid.full_prefix, sym, ' '.join(o_files)))
# finally create the include file
syms = []
@@ -243,14 +265,14 @@ def add_automatic_backports(args):
for f in h_files:
outf = open(os.path.join(args.outdir, 'include', f), 'w')
outf.write('/* Automatically created during backport process */\n')
- outf.write('#ifndef CPTCFG_%s\n' % sym)
+ outf.write('#ifndef %s%s\n' % (bpid.full_prefix, sym))
outf.write('#include_next <%s>\n' % f)
outf.write('#else\n');
for s in syms:
outf.write('#undef %s\n' % s)
outf.write('#define %s LINUX_BACKPORT(%s)\n' % (s, s))
outf.write('#include <%s>\n' % (os.path.dirname(f) + '/backport-' + os.path.basename(f), ))
- outf.write('#endif /* CPTCFG_%s */\n' % sym)
+ outf.write('#endif /* %s%s */\n' % (bpid.full_prefix, sym))
return disable_list
def git_debug_init(args):
@@ -695,6 +717,25 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
gitdebug, verbose, extra_driver, kup, kup_test,
test_cocci, profile_cocci)
rel_prep = None
+ integrate = False
+
+ # When building a package we use CPTCFG as we can rely on the
+ # fact that kconfig treats CONFIG_ as an environment variable
+ # requring less changes on code. For kernel integration we use
+ # the longer CONFIG_BACKPORT given that we'll be sticking to
+ # the kernel symbol namespace, to address that we do a final
+ # search / replace. Technically its possible to rely on the
+ # same prefix for packaging as with kernel integration but
+ # there are already some users of the CPTCFG prefix.
+ bpid = None
+ if integrate:
+ bpid = Bp_Identity(integrate = integrate,
+ kconfig_prefix = 'CONFIG_',
+ project_prefix = 'BACKPORT_')
+ else:
+ bpid = Bp_Identity(integrate = integrate,
+ kconfig_prefix = 'CPTCFG_',
+ project_prefix = '')
# start processing ...
if (args.kup or args.kup_test):
@@ -758,7 +799,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
git_debug_snapshot(args, 'Add driver sources')
- disable_list = add_automatic_backports(args)
+ disable_list = add_automatic_backports(args, bpid)
if disable_list:
bpcfg = kconfig.ConfigTree(os.path.join(args.outdir, 'compat', 'Kconfig'))
bpcfg.disable_symbols(disable_list)
@@ -769,11 +810,15 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# some post-processing is required
configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
orig_symbols = configtree.symbols()
+
logwrite('Modify Kconfig tree ...')
configtree.prune_sources(ignore=['Kconfig.kernel', 'Kconfig.versions'])
git_debug_snapshot(args, "prune Kconfig tree")
- configtree.force_tristate_modular()
- git_debug_snapshot(args, "force tristate options modular")
+
+ if not bpid.integrate:
+ configtree.force_tristate_modular()
+ git_debug_snapshot(args, "force tristate options modular")
+
configtree.modify_selects()
git_debug_snapshot(args, "convert select to depends on")
@@ -793,16 +838,17 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
if git_tracked_version:
f.write('BACKPORTS_GIT_TRACKED="backport tracker ID: $(shell git rev-parse HEAD 2>/dev/null || echo \'not built in git tree\')"\n')
f.close()
+ git_debug_snapshot(args, "add versions files")
symbols = configtree.symbols()
- # write local symbol list -- needed during build
- f = open(os.path.join(args.outdir, '.local-symbols'), 'w')
- for sym in symbols:
- f.write('%s=\n' % sym)
- f.close()
-
- git_debug_snapshot(args, "add versions/symbols files")
+ # write local symbol list -- needed during packaging build
+ if not bpid.integrate:
+ f = open(os.path.join(args.outdir, '.local-symbols'), 'w')
+ for sym in symbols:
+ f.write('%s=\n' % sym)
+ f.close()
+ git_debug_snapshot(args, "add symbols files")
# add defconfigs that we want
defconfigs_dir = os.path.join(source_dir, 'backport', 'defconfigs')
@@ -829,6 +875,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
logwrite('Rewrite Makefiles and Kconfig files ...')
# rewrite Makefile and source symbols
+
regexes = []
for some_symbols in [orig_symbols[i:i + 50] for i in range(0, len(orig_symbols), 50)]:
r = 'CONFIG_((' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
@@ -840,9 +887,11 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
for f in files:
data = open(os.path.join(root, f), 'r').read()
for r in regexes:
- data = r.sub(r'CPTCFG_\1', data)
+ data = r.sub(r'' + bpid.full_prefix + '\\1', data)
data = re.sub(r'\$\(srctree\)', '$(backport_srctree)', data)
data = re.sub(r'-Idrivers', '-I$(backport_srctree)/drivers', data)
+ if bpid.integrate:
+ data = re.sub(r'CPTCFG_', bpid.full_prefix, data)
fo = open(os.path.join(root, f), 'w')
fo.write(data)
fo.close()
@@ -886,7 +935,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# groups -- 50 seemed safer and is still fast)
regexes = []
for some_symbols in [disable_makefile[i:i + 50] for i in range(0, len(disable_makefile), 50)]:
- r = '^([^#].*((CPTCFG|CONFIG)_(' + '|'.join([s for s in some_symbols]) + ')))'
+ r = '^([^#].*((' + bpid.full_prefix_resafe + '|CONFIG_)(' + '|'.join([s for s in some_symbols]) + ')))'
regexes.append(re.compile(r, re.MULTILINE))
for f in maketree.get_makefiles():
data = open(f, 'r').read()
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 07/21] backports: add prefix for integration on dependencies
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (5 preceding siblings ...)
2014-11-11 8:14 ` [PATCH v3 06/21] backports: allow for different backport prefix and simple integration support Luis R. Rodriguez
@ 2014-11-11 8:14 ` Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 08/21] backports: replace BACKPORT_PWD with BACKPORT_DIR Luis R. Rodriguez
` (14 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
When integrating we need the BACKPORT prefix added to
dependency symbols.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
gentree.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/gentree.py b/gentree.py
index 40b0a55..dd7af84 100755
--- a/gentree.py
+++ b/gentree.py
@@ -921,7 +921,10 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
new.append('BACKPORT_DISABLED_KCONFIG_OPTION')
else:
new.append('!BACKPORT_KERNEL_%s' % dep.replace('.', '_'))
- deplist[sym] = new
+ if bpid.integrate:
+ deplist[sym] = ["BACKPORT_" + x for x in new]
+ else:
+ deplist[sym] = new
configtree.add_dependencies(deplist)
git_debug_snapshot(args, "add kernel version dependencies")
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 08/21] backports: replace BACKPORT_PWD with BACKPORT_DIR
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (6 preceding siblings ...)
2014-11-11 8:14 ` [PATCH v3 07/21] backports: add prefix for integration on dependencies Luis R. Rodriguez
@ 2014-11-11 8:14 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 09/21] backports: fold outdir as a backport identity target Luis R. Rodriguez
` (13 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:14 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
This will also be used for built-in kernel integration, by
using this we can share more code with the built-in kernel
approach.
Reviewed-by: Johannes Berg <johannes@sipsolutions.net>
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/Makefile | 6 +++---
backport/Makefile.build | 4 ++--
backport/Makefile.real | 2 +-
backport/scripts/uninstall.sh | 4 ++--
4 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/backport/Makefile b/backport/Makefile
index ea3982b..fcf2f01 100644
--- a/backport/Makefile
+++ b/backport/Makefile
@@ -6,7 +6,7 @@ ifeq ($(KERNELRELEASE),)
MAKEFLAGS += --no-print-directory
SHELL := /bin/bash
-BACKPORT_PWD := $(shell pwd)
+BACKPORT_DIR := $(shell pwd)
KMODDIR ?= updates
ifneq ($(origin KLIB), undefined)
@@ -20,7 +20,7 @@ KERNEL_CONFIG := $(KLIB_BUILD)/.config
KERNEL_MAKEFILE := $(KLIB_BUILD)/Makefile
CONFIG_MD5 := $(shell md5sum $(KERNEL_CONFIG) 2>/dev/null | sed 's/\s.*//')
-export KLIB KLIB_BUILD BACKPORT_PWD KMODDIR KMODPATH_ARG
+export KLIB KLIB_BUILD BACKPORT_DIR KMODDIR KMODPATH_ARG
# disable built-in rules for this file
.SUFFIXES:
@@ -159,5 +159,5 @@ help: defconfig-help
@echo ""
@echo "Execute "make" or "make all" to build all targets marked with [*]"
else
-include $(BACKPORT_PWD)/Makefile.kernel
+include $(BACKPORT_DIR)/Makefile.kernel
endif
diff --git a/backport/Makefile.build b/backport/Makefile.build
index d209041..a848b37 100644
--- a/backport/Makefile.build
+++ b/backport/Makefile.build
@@ -3,8 +3,8 @@ export
.PHONY: modules
modules:
- @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_PWD) modules
+ @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_DIR) modules
.PHONY: clean
clean:
- @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_PWD) clean
+ @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_DIR) clean
diff --git a/backport/Makefile.real b/backport/Makefile.real
index f60d5ca..a0f4916 100644
--- a/backport/Makefile.real
+++ b/backport/Makefile.real
@@ -89,7 +89,7 @@ modules: backport-include/backport/autoconf.h
.PHONY: install
install: modules
- @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_PWD) \
+ @$(MAKE) -C $(KLIB_BUILD) M=$(BACKPORT_DIR) \
INSTALL_MOD_DIR=$(KMODDIR) $(KMODPATH_ARG) \
modules_install
@./scripts/blacklist.sh $(KLIB)/ $(KLIB)/$(KMODDIR)
diff --git a/backport/scripts/uninstall.sh b/backport/scripts/uninstall.sh
index 1eb42b0..b46d3fe 100755
--- a/backport/scripts/uninstall.sh
+++ b/backport/scripts/uninstall.sh
@@ -10,8 +10,8 @@ else
compr=""
fi
-for driver in $(find ${BACKPORT_PWD} -type f -name *.ko); do
- mod_name=${driver/${BACKPORT_PWD}/${KLIB}${KMODDIR}}${compr}
+for driver in $(find ${BACKPORT_DIR} -type f -name *.ko); do
+ mod_name=${driver/${BACKPORT_DIR}/${KLIB}${KMODDIR}}${compr}
echo " uninstall" $mod_name
rm -f $mod_name
done
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 09/21] backports: fold outdir as a backport identity target
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (7 preceding siblings ...)
2014-11-11 8:14 ` [PATCH v3 08/21] backports: replace BACKPORT_PWD with BACKPORT_DIR Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 10/21] backports: avoid git tree reinitialization Luis R. Rodriguez
` (12 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
The output directory is always the same as the project directory
for packaging backports but for integration these will differ,
to make introduction of integration easier to follow use the
outdir as both project directory and target directory and fold
the existing args.outdir under the backport identity object as
both "project_dir" and "target_dir".
This commit has no functional changes.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
gentree.py | 140 ++++++++++++++++++++++++++++++++++---------------------------
1 file changed, 78 insertions(+), 62 deletions(-)
diff --git a/gentree.py b/gentree.py
index dd7af84..04e8f5e 100755
--- a/gentree.py
+++ b/gentree.py
@@ -30,7 +30,9 @@ class Bp_Identity(object):
are added, how we actually use the others for regular printing will
need to be considered.
"""
- def __init__(self, integrate=False, kconfig_prefix='CPTCFG_', project_prefix=''):
+ def __init__(self, integrate=False, kconfig_prefix='CPTCFG_',
+ project_prefix='', project_dir='',
+ target_dir='', target_dir_name=''):
self.integrate = integrate
self.kconfig_prefix = kconfig_prefix
self.kconfig_prefix_resafe = re.escape(kconfig_prefix)
@@ -38,6 +40,9 @@ class Bp_Identity(object):
self.project_prefix_resafe = re.escape(project_prefix)
self.full_prefix = kconfig_prefix + project_prefix
self.full_prefix_resafe = re.escape(self.full_prefix)
+ self.project_dir = project_dir
+ self.target_dir = target_dir
+ self.target_dir_name = target_dir_name
def read_copy_list(copyfile):
"""
@@ -217,11 +222,11 @@ def automatic_backport_mangle_c_file(name):
return name.replace('/', '-')
-def add_automatic_backports(args, bpid):
+def add_automatic_backports(args):
disable_list = []
export = re.compile(r'^EXPORT_SYMBOL(_GPL)?\((?P<sym>[^\)]*)\)')
- bpi = kconfig.get_backport_info(os.path.join(args.outdir, 'compat', 'Kconfig'))
- configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
+ bpi = kconfig.get_backport_info(os.path.join(args.bpid.target_dir, 'compat', 'Kconfig'))
+ configtree = kconfig.ConfigTree(os.path.join(args.bpid.target_dir, 'Kconfig'))
all_selects = configtree.all_selects()
for sym, vals in bpi.items():
if sym.startswith('BACKPORT_BUILD_'):
@@ -238,41 +243,41 @@ def add_automatic_backports(args, bpid):
files.append((os.path.join('include', f),
os.path.join('include', os.path.dirname(f), 'backport-' + os.path.basename(f))))
if args.git_revision:
- copy_git_files(args.kerneldir, files, args.git_revision, args.outdir)
+ copy_git_files(args.kerneldir, files, args.git_revision, args.bpid.target_dir)
else:
- copy_files(args.kerneldir, files, args.outdir)
+ copy_files(args.kerneldir, files, args.bpid.target_dir)
# now add the Makefile line
- mf = open(os.path.join(args.outdir, 'compat', 'Makefile'), 'a+')
+ mf = open(os.path.join(args.bpid.target_dir, 'compat', 'Makefile'), 'a+')
o_files = [automatic_backport_mangle_c_file(f)[:-1] + 'o' for f in c_files]
if symtype == 'tristate':
if not module_name:
raise Exception('backporting a module requires a #module-name')
for of in o_files:
mf.write('%s-objs += %s\n' % (module_name, of))
- mf.write('obj-$(%s%s) += %s.o\n' % (bpid.full_prefix, sym, module_name))
+ mf.write('obj-$(%s%s) += %s.o\n' % (args.bpid.full_prefix, sym, module_name))
elif symtype == 'bool':
- mf.write('compat-$(%s%s) += %s\n' % (bpid.full_prefix, sym, ' '.join(o_files)))
+ mf.write('compat-$(%s%s) += %s\n' % (args.bpid.full_prefix, sym, ' '.join(o_files)))
# finally create the include file
syms = []
for f in c_files:
- for l in open(os.path.join(args.outdir, 'compat',
+ for l in open(os.path.join(args.bpid.target_dir, 'compat',
automatic_backport_mangle_c_file(f)), 'r'):
m = export.match(l)
if m:
syms.append(m.group('sym'))
for f in h_files:
- outf = open(os.path.join(args.outdir, 'include', f), 'w')
+ outf = open(os.path.join(args.bpid.target_dir, 'include', f), 'w')
outf.write('/* Automatically created during backport process */\n')
- outf.write('#ifndef %s%s\n' % (bpid.full_prefix, sym))
+ outf.write('#ifndef %s%s\n' % (args.bpid.full_prefix, sym))
outf.write('#include_next <%s>\n' % f)
outf.write('#else\n');
for s in syms:
outf.write('#undef %s\n' % s)
outf.write('#define %s LINUX_BACKPORT(%s)\n' % (s, s))
outf.write('#include <%s>\n' % (os.path.dirname(f) + '/backport-' + os.path.basename(f), ))
- outf.write('#endif /* %s%s */\n' % (bpid.full_prefix, sym))
+ outf.write('#endif /* %s%s */\n' % (args.bpid.full_prefix, sym))
return disable_list
def git_debug_init(args):
@@ -283,8 +288,8 @@ def git_debug_init(args):
"""
if not args.gitdebug:
return
- git.init(tree=args.outdir)
- git.commit_all("Copied backport", tree=args.outdir)
+ git.init(tree=args.bpid.project_dir)
+ git.commit_all("Copied backport", tree=args.bpid.project_dir)
def git_debug_snapshot(args, name):
@@ -293,7 +298,7 @@ def git_debug_snapshot(args, name):
"""
if not args.gitdebug:
return
- git.commit_all(name, tree=args.outdir)
+ git.commit_all(name, tree=args.bpid.project_dir)
def get_rel_spec_stable(rel):
"""
@@ -426,12 +431,12 @@ def upload_release(args, rel_prep, logwrite=lambda x:None):
if (rel_prep['stable']):
korg_path += "/stable"
- parent = os.path.dirname(args.outdir)
- release = os.path.basename(args.outdir)
+ parent = os.path.dirname(args.bpid.project_dir)
+ release = os.path.basename(args.bpid.project_dir)
tar_name = parent + '/' + release + ".tar"
gzip_name = tar_name + ".gz"
- create_tar_and_gz(tar_name, args.outdir)
+ create_tar_and_gz(tar_name, args.bpid.project_dir)
logwrite(gpg.sign(tar_name, extra_args=['--armor', '--detach-sign']))
@@ -656,13 +661,42 @@ def _main():
'of changes done by Coccinelle.')
args = parser.parse_args()
+ # When building a package we use CPTCFG as we can rely on the
+ # fact that kconfig treats CONFIG_ as an environment variable
+ # requring less changes on code. For kernel integration we use
+ # the longer CONFIG_BACKPORT given that we'll be sticking to
+ # the kernel symbol namespace, to address that we do a final
+ # search / replace. Technically its possible to rely on the
+ # same prefix for packaging as with kernel integration but
+ # there are already some users of the CPTCFG prefix.
+ bpid = None
+ integrate = False
+ if integrate:
+ bpid = Bp_Identity(integrate = integrate,
+ kconfig_prefix = 'CONFIG_',
+ project_prefix = 'BACKPORT_',
+ project_dir = args.outdir,
+ target_dir = os.path.join(args.outdir, 'backports/'),
+ target_dir_name = 'backports/',
+ )
+ else:
+ bpid = Bp_Identity(integrate = integrate,
+ kconfig_prefix = 'CPTCFG_',
+ project_prefix = '',
+ project_dir = args.outdir,
+ target_dir = args.outdir,
+ target_dir_name = '',
+ )
+
def logwrite(msg):
sys.stdout.write(msg)
sys.stdout.write('\n')
sys.stdout.flush()
- return process(args.kerneldir, args.outdir, args.copy_list,
- git_revision=args.git_revision, clean=args.clean,
+ return process(args.kerneldir, args.copy_list,
+ git_revision=args.git_revision,
+ bpid=bpid,
+ clean=args.clean,
refresh=args.refresh, base_name=args.base_name,
gitdebug=args.gitdebug, verbose=args.verbose,
extra_driver=args.extra_driver,
@@ -672,7 +706,8 @@ def _main():
profile_cocci=args.profile_cocci,
logwrite=logwrite)
-def process(kerneldir, outdir, copy_list_file, git_revision=None,
+def process(kerneldir, copy_list_file, git_revision=None,
+ bpid=None,
clean=False, refresh=False, base_name="Linux", gitdebug=False,
verbose=False, extra_driver=[], kup=False,
kup_test=False,
@@ -681,16 +716,16 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
logwrite=lambda x:None,
git_tracked_version=False):
class Args(object):
- def __init__(self, kerneldir, outdir, copy_list_file,
- git_revision, clean, refresh, base_name,
+ def __init__(self, kerneldir, copy_list_file,
+ git_revision, bpid, clean, refresh, base_name,
gitdebug, verbose, extra_driver, kup,
kup_test,
test_cocci,
profile_cocci):
self.kerneldir = kerneldir
- self.outdir = outdir
self.copy_list = copy_list_file
self.git_revision = git_revision
+ self.bpid = bpid
self.clean = clean
self.refresh = refresh
self.base_name = base_name
@@ -712,30 +747,11 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
else:
logwrite('Validated tree: %s' % tree)
- args = Args(kerneldir, outdir, copy_list_file,
- git_revision, clean, refresh, base_name,
+ args = Args(kerneldir, copy_list_file,
+ git_revision, bpid, clean, refresh, base_name,
gitdebug, verbose, extra_driver, kup, kup_test,
test_cocci, profile_cocci)
rel_prep = None
- integrate = False
-
- # When building a package we use CPTCFG as we can rely on the
- # fact that kconfig treats CONFIG_ as an environment variable
- # requring less changes on code. For kernel integration we use
- # the longer CONFIG_BACKPORT given that we'll be sticking to
- # the kernel symbol namespace, to address that we do a final
- # search / replace. Technically its possible to rely on the
- # same prefix for packaging as with kernel integration but
- # there are already some users of the CPTCFG prefix.
- bpid = None
- if integrate:
- bpid = Bp_Identity(integrate = integrate,
- kconfig_prefix = 'CONFIG_',
- project_prefix = 'BACKPORT_')
- else:
- bpid = Bp_Identity(integrate = integrate,
- kconfig_prefix = 'CPTCFG_',
- project_prefix = '')
# start processing ...
if (args.kup or args.kup_test):
@@ -743,7 +759,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
git_paranoia(kerneldir, logwrite)
rel_describe = git.describe(rev=None, tree=source_dir, extra_args=['--dirty'])
- release = os.path.basename(args.outdir)
+ release = os.path.basename(bpid.target_dir)
version = release.replace("backports-", "")
rel_prep = get_rel_prep(version)
@@ -768,7 +784,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
deplist = read_dependencies(os.path.join(source_dir, 'dependencies'))
# validate output directory
- check_output_dir(args.outdir, args.clean)
+ check_output_dir(bpid.target_dir, args.clean)
# do the copy
backport_files = [(x, x) for x in [
@@ -782,33 +798,33 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
else:
logwrite('Get original source files from git ...')
- copy_files(os.path.join(source_dir, 'backport'), backport_files, args.outdir)
+ copy_files(os.path.join(source_dir, 'backport'), backport_files, bpid.target_dir)
git_debug_init(args)
if not args.git_revision:
- copy_files(args.kerneldir, copy_list, args.outdir)
+ copy_files(args.kerneldir, copy_list, bpid.target_dir)
else:
- copy_git_files(args.kerneldir, copy_list, args.git_revision, args.outdir)
+ copy_git_files(args.kerneldir, copy_list, args.git_revision, bpid.target_dir)
# FIXME: should we add a git version of this (e.g. --git-extra-driver)?
for src, copy_list in args.extra_driver:
if (args.kup or args.kup_test):
git_paranoia(src)
- copy_files(src, read_copy_list(open(copy_list, 'r')), args.outdir)
+ copy_files(src, read_copy_list(open(copy_list, 'r')), bpid.target_dir)
git_debug_snapshot(args, 'Add driver sources')
- disable_list = add_automatic_backports(args, bpid)
+ disable_list = add_automatic_backports(args)
if disable_list:
- bpcfg = kconfig.ConfigTree(os.path.join(args.outdir, 'compat', 'Kconfig'))
+ bpcfg = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'compat', 'Kconfig'))
bpcfg.disable_symbols(disable_list)
git_debug_snapshot(args, 'Add automatic backports')
- apply_patches(args, "backport", source_dir, 'patches', args.outdir, logwrite)
+ apply_patches(args, "backport", source_dir, 'patches', bpid.target_dir, logwrite)
# some post-processing is required
- configtree = kconfig.ConfigTree(os.path.join(args.outdir, 'Kconfig'))
+ configtree = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'Kconfig'))
orig_symbols = configtree.symbols()
logwrite('Modify Kconfig tree ...')
@@ -831,7 +847,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
kernel_version = git.describe(rev=args.git_revision or 'HEAD',
tree=args.kerneldir,
extra_args=['--long'])
- f = open(os.path.join(args.outdir, 'versions'), 'w')
+ f = open(os.path.join(bpid.target_dir, 'versions'), 'w')
f.write('BACKPORTS_VERSION="%s"\n' % backports_version)
f.write('BACKPORTED_KERNEL_VERSION="%s"\n' % kernel_version)
f.write('BACKPORTED_KERNEL_NAME="%s"\n' % args.base_name)
@@ -844,7 +860,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# write local symbol list -- needed during packaging build
if not bpid.integrate:
- f = open(os.path.join(args.outdir, '.local-symbols'), 'w')
+ f = open(os.path.join(bpid.project_dir, '.local-symbols'), 'w')
for sym in symbols:
f.write('%s=\n' % sym)
f.close()
@@ -852,7 +868,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
# add defconfigs that we want
defconfigs_dir = os.path.join(source_dir, 'backport', 'defconfigs')
- os.mkdir(os.path.join(args.outdir, 'defconfigs'))
+ os.mkdir(os.path.join(bpid.target_dir, 'defconfigs'))
for dfbase in os.listdir(defconfigs_dir):
copy_defconfig = True
dfsrc = os.path.join(defconfigs_dir, dfbase)
@@ -868,7 +884,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
copy_defconfig = False
break
if copy_defconfig:
- shutil.copy(dfsrc, os.path.join(args.outdir, 'defconfigs', dfbase))
+ shutil.copy(dfsrc, os.path.join(bpid.target_dir, 'defconfigs', dfbase))
git_debug_snapshot(args, "add (useful) defconfig files")
@@ -880,7 +896,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
for some_symbols in [orig_symbols[i:i + 50] for i in range(0, len(orig_symbols), 50)]:
r = 'CONFIG_((' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
regexes.append(re.compile(r, re.MULTILINE))
- for root, dirs, files in os.walk(args.outdir):
+ for root, dirs, files in os.walk(bpid.target_dir):
# don't go into .git dir (possible debug thing)
if '.git' in dirs:
dirs.remove('.git')
@@ -899,7 +915,7 @@ def process(kerneldir, outdir, copy_list_file, git_revision=None,
git_debug_snapshot(args, "rename config symbol / srctree usage")
# disable unbuildable Kconfig symbols and stuff Makefiles that doesn't exist
- maketree = make.MakeTree(os.path.join(args.outdir, 'Makefile.kernel'))
+ maketree = make.MakeTree(os.path.join(bpid.target_dir, 'Makefile.kernel'))
disable_kconfig = []
disable_makefile = []
for sym in maketree.get_impossible_symbols():
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 10/21] backports: avoid git tree reinitialization
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (8 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 09/21] backports: fold outdir as a backport identity target Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-14 20:47 ` Johannes Berg
2014-11-11 8:15 ` [PATCH v3 11/21] backports: modify kconfig parser to use bpid and target_dir Luis R. Rodriguez
` (11 subsequent siblings)
21 siblings, 1 reply; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
When using backports against for packaging with --gitdebug you either
have to use --clean or assume the directory is empty already. In either
case you start fresh. With integration this will be a bit different, you
could end up with a project directory where the git tree was present but
only the target directory was empty.
Calling git init on an already existing git directory will reinitialize
your git tree, that can reset some config stuff, let's avoid that.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
gentree.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/gentree.py b/gentree.py
index 04e8f5e..68efce9 100755
--- a/gentree.py
+++ b/gentree.py
@@ -288,7 +288,10 @@ def git_debug_init(args):
"""
if not args.gitdebug:
return
- git.init(tree=args.bpid.project_dir)
+ # Git supports re-initialization, although not well documented it can
+ # reset config stuff, lets avoid that if the tree already exists.
+ if not os.path.exists(os.path.join(args.bpid.project_dir, '.git')):
+ git.init(tree=args.bpid.project_dir)
git.commit_all("Copied backport", tree=args.bpid.project_dir)
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 11/21] backports: modify kconfig parser to use bpid and target_dir
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (9 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 10/21] backports: avoid git tree reinitialization Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 12/21] backports: make kconfig src_line backport / ConfigTree specific Luis R. Rodriguez
` (10 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
The kconfig parser sets a base directory using the directory
in which the Kconfig file passed lives in. This strategy assumes the
kernel uses relative paths, while this is truly supported in
practice though upstream only deals with full paths based on
the actual root directory of the kernel. Its safer, cleaner and
much simpler to support full paths, and since the kernel uses
full paths anyway lets stick to that and complain when relative
paths are found. This also modifies our own provided Kconfig to
use full paths as well, we'll need this later to adjust our
Kconfigs for kernel integration.
$ time /home/mcgrof/backports/devel/ckmake --allyesconfig
1 3.0.101 [ OK ]
2 3.1.10 [ OK ]
3 3.2.62 [ OK ]
4 3.3.8 [ OK ]
5 3.4.104 [ OK ]
6 3.5.7 [ OK ]
7 3.6.11 [ OK ]
8 3.7.10 [ OK ]
9 3.8.13 [ OK ]
10 3.9.11 [ OK ]
11 3.10.58 [ OK ]
12 3.11.10 [ OK ]
13 3.12.31 [ OK ]
14 3.13.11 [ OK ]
15 3.14.22 [ OK ]
16 3.15.10 [ OK ]
17 3.16.6 [ OK ]
18 3.17.1 [ OK ]
19 3.18-rc1 [ OK ]
real 44m3.002s
user 1188m20.708s
sys 140m25.540s
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/Kconfig | 36 ++++++++++++++++++------------------
gentree.py | 6 +++---
lib/kconfig.py | 51 +++++++++++++++++++++++++++++++++------------------
3 files changed, 54 insertions(+), 39 deletions(-)
diff --git a/backport/Kconfig b/backport/Kconfig
index a8f2867..415db89 100644
--- a/backport/Kconfig
+++ b/backport/Kconfig
@@ -11,32 +11,32 @@ config BACKPORTED_KERNEL_NAME
option env="BACKPORTED_KERNEL_NAME"
# these will be generated
-source Kconfig.kernel
-source Kconfig.versions
+source "Kconfig.kernel"
+source "Kconfig.versions"
# Packaging hacks
-source Kconfig.package.hacks
+source "Kconfig.package.hacks"
# this has the configuration for the backport code
-source compat/Kconfig
+source "compat/Kconfig"
# these are copied from the kernel
-source net/wireless/Kconfig
-source net/mac80211/Kconfig
-source net/bluetooth/Kconfig
-source drivers/net/wireless/Kconfig
-source drivers/net/ethernet/Kconfig
-source drivers/net/usb/Kconfig
+source "net/wireless/Kconfig"
+source "net/mac80211/Kconfig"
+source "net/bluetooth/Kconfig"
+source "drivers/net/wireless/Kconfig"
+source "drivers/net/ethernet/Kconfig"
+source "drivers/net/usb/Kconfig"
-source drivers/ssb/Kconfig
-source drivers/bcma/Kconfig
+source "drivers/ssb/Kconfig"
+source "drivers/bcma/Kconfig"
-source net/nfc/Kconfig
+source "net/nfc/Kconfig"
-source drivers/media/Kconfig
+source "drivers/media/Kconfig"
-source net/ieee802154/Kconfig
-source net/mac802154/Kconfig
-source drivers/net/ieee802154/Kconfig
+source "net/ieee802154/Kconfig"
+source "net/mac802154/Kconfig"
+source "drivers/net/ieee802154/Kconfig"
-source drivers/usb/class/Kconfig
+source "drivers/usb/class/Kconfig"
diff --git a/gentree.py b/gentree.py
index 68efce9..2c08bba 100755
--- a/gentree.py
+++ b/gentree.py
@@ -226,7 +226,7 @@ def add_automatic_backports(args):
disable_list = []
export = re.compile(r'^EXPORT_SYMBOL(_GPL)?\((?P<sym>[^\)]*)\)')
bpi = kconfig.get_backport_info(os.path.join(args.bpid.target_dir, 'compat', 'Kconfig'))
- configtree = kconfig.ConfigTree(os.path.join(args.bpid.target_dir, 'Kconfig'))
+ configtree = kconfig.ConfigTree(os.path.join(args.bpid.target_dir, 'Kconfig'), args.bpid)
all_selects = configtree.all_selects()
for sym, vals in bpi.items():
if sym.startswith('BACKPORT_BUILD_'):
@@ -820,14 +820,14 @@ def process(kerneldir, copy_list_file, git_revision=None,
disable_list = add_automatic_backports(args)
if disable_list:
- bpcfg = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'compat', 'Kconfig'))
+ bpcfg = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'compat', 'Kconfig'), bpid)
bpcfg.disable_symbols(disable_list)
git_debug_snapshot(args, 'Add automatic backports')
apply_patches(args, "backport", source_dir, 'patches', bpid.target_dir, logwrite)
# some post-processing is required
- configtree = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'Kconfig'))
+ configtree = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'Kconfig'), bpid)
orig_symbols = configtree.symbols()
logwrite('Modify Kconfig tree ...')
diff --git a/lib/kconfig.py b/lib/kconfig.py
index 3562630..f348d4a 100644
--- a/lib/kconfig.py
+++ b/lib/kconfig.py
@@ -4,7 +4,8 @@
import os, re
-src_line = re.compile(r'^\s*source\s+"?(?P<src>[^\s"]*)"?\s*$')
+src_line = re.compile(r'^\s*source\s+"(?P<src>[^\s"]*)"?\s*$')
+src_line_rel = re.compile(r'^\s*source\s+(?P<src>[^\s"]*)"?\s*$')
tri_line = re.compile(r'^(?P<spc>\s+)tristate')
bool_line = re.compile(r'^(?P<spc>\s+)bool')
cfg_line = re.compile(r'^(?P<opt>config|menuconfig)\s+(?P<sym>[^\s]*)')
@@ -12,32 +13,46 @@ sel_line = re.compile(r'^(?P<spc>\s+)select\s+(?P<sym>[^\s]*)\s*$')
backport_line = re.compile(r'^\s+#(?P<key>[ch]-file|module-name)\s*(?P<name>.*)')
class ConfigTree(object):
- def __init__(self, rootfile):
- self.basedir = os.path.dirname(rootfile)
+ def __init__(self, rootfile, bpid):
+ self.bpid = bpid
self.rootfile = os.path.basename(rootfile)
+ def _check_relative_source(self, f, l):
+ #
+ # Although we can support relative kconfig source lines its a lot safer,
+ # clearer to use full paths; it also makes it easier to support / parse and
+ # modify kconfig entries. The kernel also uses full paths anyway but if
+ # a relative path is found we should consider changing that upstream to
+ # streamline usage of full path.
+ m = src_line_rel.match(l)
+ if m:
+ raise Exception('File: %s uses relative kconfig source entries (line: \'%s\'), use full path with quotes' %
+ (os.path.join(self.bpid.target_dir, f), l))
def _walk(self, f):
yield f
- for l in open(os.path.join(self.basedir, f), 'r'):
+ for l in open(os.path.join(self.bpid.target_dir, f), 'r'):
m = src_line.match(l)
- if m and os.path.exists(os.path.join(self.basedir, m.group('src'))):
+ if m and os.path.exists(os.path.join(self.bpid.target_dir, m.group('src'))):
for i in self._walk(m.group('src')):
yield i
+ else:
+ self._check_relative_source(f, l)
def _prune_sources(self, f, ignore):
for nf in self._walk(f):
out = ''
- for l in open(os.path.join(self.basedir, nf), 'r'):
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
m = src_line.match(l)
if not m:
+ self._check_relative_source(nf, l)
out += l
continue
src = m.group('src')
- if src in ignore or os.path.exists(os.path.join(self.basedir, src)):
+ if src in ignore or os.path.exists(os.path.join(self.bpid.target_dir, src)):
out += l
else:
out += '#' + l
- outf = open(os.path.join(self.basedir, nf), 'w')
+ outf = open(os.path.join(self.bpid.target_dir, nf), 'w')
outf.write(out)
outf.close()
@@ -47,19 +62,19 @@ class ConfigTree(object):
def force_tristate_modular(self):
for nf in self._walk(self.rootfile):
out = ''
- for l in open(os.path.join(self.basedir, nf), 'r'):
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
m = tri_line.match(l)
out += l
if m:
out += m.group('spc') + "depends on m\n"
- outf = open(os.path.join(self.basedir, nf), 'w')
+ outf = open(os.path.join(self.bpid.target_dir, nf), 'w')
outf.write(out)
outf.close()
def symbols(self):
syms = []
for nf in self._walk(self.rootfile):
- for l in open(os.path.join(self.basedir, nf), 'r'):
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
m = cfg_line.match(l)
if m:
syms.append(m.group('sym'))
@@ -68,7 +83,7 @@ class ConfigTree(object):
def all_selects(self):
result = []
for nf in self._walk(self.rootfile):
- for l in open(os.path.join(self.basedir, nf), 'r'):
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
m = sel_line.match(l)
if m:
result.append(m.group('sym'))
@@ -78,7 +93,7 @@ class ConfigTree(object):
syms = self.symbols()
for nf in self._walk(self.rootfile):
out = ''
- for l in open(os.path.join(self.basedir, nf), 'r'):
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
m = sel_line.match(l)
if m and not m.group('sym') in syms:
if 'BACKPORT_' + m.group('sym') in syms:
@@ -87,32 +102,32 @@ class ConfigTree(object):
out += m.group('spc') + "depends on " + m.group('sym') + '\n'
else:
out += l
- outf = open(os.path.join(self.basedir, nf), 'w')
+ outf = open(os.path.join(self.bpid.target_dir, nf), 'w')
outf.write(out)
outf.close()
def disable_symbols(self, syms):
for nf in self._walk(self.rootfile):
out = ''
- for l in open(os.path.join(self.basedir, nf), 'r'):
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
m = cfg_line.match(l)
out += l
if m and m.group('sym') in syms:
out += "\tdepends on n\n"
- outf = open(os.path.join(self.basedir, nf), 'w')
+ outf = open(os.path.join(self.bpid.target_dir, nf), 'w')
outf.write(out)
outf.close()
def add_dependencies(self, deps):
for nf in self._walk(self.rootfile):
out = ''
- for l in open(os.path.join(self.basedir, nf), 'r'):
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
m = cfg_line.match(l)
out += l
if m:
for dep in deps.get(m.group('sym'), []):
out += "\tdepends on %s\n" % dep
- outf = open(os.path.join(self.basedir, nf), 'w')
+ outf = open(os.path.join(self.bpid.target_dir, nf), 'w')
outf.write(out)
outf.close()
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 12/21] backports: make kconfig src_line backport / ConfigTree specific
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (10 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 11/21] backports: modify kconfig parser to use bpid and target_dir Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 13/21] backports: use BACKPORT_DIR prefix on kconfig sources Luis R. Rodriguez
` (9 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
The kconfig source line used on a backport project is very
project specific. For backport packaging the source line simply
should follow the full path of the project but for integration
this will vary depending on the backport prefix directory used.
We'll add support for that next.
$ time /home/mcgrof/backports/devel/ckmake --allyesconfig
1 3.0.101 [ OK ]
2 3.1.10 [ OK ]
3 3.2.62 [ OK ]
4 3.3.8 [ OK ]
5 3.4.104 [ OK ]
6 3.5.7 [ OK ]
7 3.6.11 [ OK ]
8 3.7.10 [ OK ]
9 3.8.13 [ OK ]
10 3.9.11 [ OK ]
11 3.10.58 [ OK ]
12 3.11.10 [ OK ]
13 3.12.31 [ OK ]
14 3.13.11 [ OK ]
15 3.14.22 [ OK ]
16 3.15.10 [ OK ]
17 3.16.6 [ OK ]
18 3.17.1 [ OK ]
19 3.18-rc1 [ OK ]
real 43m56.439s
user 1187m4.516s
sys 138m53.328s
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
lib/kconfig.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/kconfig.py b/lib/kconfig.py
index f348d4a..ac96613 100644
--- a/lib/kconfig.py
+++ b/lib/kconfig.py
@@ -4,7 +4,6 @@
import os, re
-src_line = re.compile(r'^\s*source\s+"(?P<src>[^\s"]*)"?\s*$')
src_line_rel = re.compile(r'^\s*source\s+(?P<src>[^\s"]*)"?\s*$')
tri_line = re.compile(r'^(?P<spc>\s+)tristate')
bool_line = re.compile(r'^(?P<spc>\s+)bool')
@@ -16,6 +15,7 @@ class ConfigTree(object):
def __init__(self, rootfile, bpid):
self.bpid = bpid
self.rootfile = os.path.basename(rootfile)
+ self.src_line = re.compile(r'^\s*source\s+"(?P<src>[^\s"]*)"?\s*$')
def _check_relative_source(self, f, l):
#
@@ -31,7 +31,7 @@ class ConfigTree(object):
def _walk(self, f):
yield f
for l in open(os.path.join(self.bpid.target_dir, f), 'r'):
- m = src_line.match(l)
+ m = self.src_line.match(l)
if m and os.path.exists(os.path.join(self.bpid.target_dir, m.group('src'))):
for i in self._walk(m.group('src')):
yield i
@@ -42,7 +42,7 @@ class ConfigTree(object):
for nf in self._walk(f):
out = ''
for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
- m = src_line.match(l)
+ m = self.src_line.match(l)
if not m:
self._check_relative_source(nf, l)
out += l
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 13/21] backports: use BACKPORT_DIR prefix on kconfig sources
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (11 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 12/21] backports: make kconfig src_line backport / ConfigTree specific Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 14/21] backports: update dependencies map file Luis R. Rodriguez
` (8 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
This will allow us to do less work for built-in integration support.
When packaging we only use BACKPORT_DIR for Kconfig sources we control,
to avoid a large change we avoid doing this for other Kconfig files.
For integration we'll always have to use BACKPORT_DIR on kconfig sources.
Suporting requires a bit of self evaluation of the variable within our
kconfig library, ideally we'd have support for groking all variables
defined within the Kconfig setup but that requires quite a lot more
work as it means also parsing Makefiles and inheriting these definitions
within config symbols when used. Since we only define one right now and
its used for built-in support we deal with it ourselves for now.
Please consider the complexity of adding others, it doesn't seem like
there would be much need for others though. If others wanted to use a
different BACKPORT_DIR path other than 'backports' that would require
tweaking here, but we'll start by assuming no one will want to do that.
$ time /home/mcgrof/backports/devel/ckmake --allyesconfig
1 3.0.101 [ OK ]
2 3.1.10 [ OK ]
3 3.2.62 [ OK ]
4 3.3.8 [ OK ]
5 3.4.104 [ OK ]
6 3.5.7 [ OK ]
7 3.6.11 [ OK ]
8 3.7.10 [ OK ]
9 3.8.13 [ OK ]
10 3.9.11 [ OK ]
11 3.10.58 [ OK ]
12 3.11.10 [ OK ]
13 3.12.31 [ OK ]
14 3.13.11 [ OK ]
15 3.14.22 [ OK ]
16 3.15.10 [ OK ]
17 3.16.6 [ OK ]
18 3.17.1 [ OK ]
19 3.18-rc1 [ OK ]
real 42m41.825s
user 1187m38.880s
sys 141m11.760s
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/Kconfig | 40 +++++++++++++++---------------
gentree.py | 18 ++++++++++++--
lib/kconfig.py | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++---
3 files changed, 107 insertions(+), 25 deletions(-)
diff --git a/backport/Kconfig b/backport/Kconfig
index 415db89..b729d47 100644
--- a/backport/Kconfig
+++ b/backport/Kconfig
@@ -1,5 +1,8 @@
mainmenu "Linux Backports from $BACKPORTED_KERNEL_NAME $BACKPORTED_KERNEL_VERSION (with backports $BACKPORTS_VERSION)"
+config BACKPORT_DIR
+ string
+ option env="BACKPORT_DIR"
config BACKPORTS_VERSION
string
option env="BACKPORTS_VERSION"
@@ -10,33 +13,32 @@ config BACKPORTED_KERNEL_NAME
string
option env="BACKPORTED_KERNEL_NAME"
-# these will be generated
-source "Kconfig.kernel"
-source "Kconfig.versions"
+source "$BACKPORT_DIR/Kconfig.kernel"
+source "$BACKPORT_DIR/Kconfig.versions"
# Packaging hacks
-source "Kconfig.package.hacks"
+source "$BACKPORT_DIR/Kconfig.package.hacks"
# this has the configuration for the backport code
-source "compat/Kconfig"
+source "$BACKPORT_DIR/compat/Kconfig"
# these are copied from the kernel
-source "net/wireless/Kconfig"
-source "net/mac80211/Kconfig"
-source "net/bluetooth/Kconfig"
-source "drivers/net/wireless/Kconfig"
-source "drivers/net/ethernet/Kconfig"
-source "drivers/net/usb/Kconfig"
+source "$BACKPORT_DIR/net/wireless/Kconfig"
+source "$BACKPORT_DIR/net/mac80211/Kconfig"
+source "$BACKPORT_DIR/net/bluetooth/Kconfig"
+source "$BACKPORT_DIR/drivers/net/wireless/Kconfig"
+source "$BACKPORT_DIR/drivers/net/ethernet/Kconfig"
+source "$BACKPORT_DIR/drivers/net/usb/Kconfig"
-source "drivers/ssb/Kconfig"
-source "drivers/bcma/Kconfig"
+source "$BACKPORT_DIR/drivers/ssb/Kconfig"
+source "$BACKPORT_DIR/drivers/bcma/Kconfig"
-source "net/nfc/Kconfig"
+source "$BACKPORT_DIR/net/nfc/Kconfig"
-source "drivers/media/Kconfig"
+source "$BACKPORT_DIR/drivers/media/Kconfig"
-source "net/ieee802154/Kconfig"
-source "net/mac802154/Kconfig"
-source "drivers/net/ieee802154/Kconfig"
+source "$BACKPORT_DIR/net/ieee802154/Kconfig"
+source "$BACKPORT_DIR/net/mac802154/Kconfig"
+source "$BACKPORT_DIR/drivers/net/ieee802154/Kconfig"
-source "drivers/usb/class/Kconfig"
+source "$BACKPORT_DIR/drivers/usb/class/Kconfig"
diff --git a/gentree.py b/gentree.py
index 2c08bba..7632b90 100755
--- a/gentree.py
+++ b/gentree.py
@@ -32,7 +32,8 @@ class Bp_Identity(object):
"""
def __init__(self, integrate=False, kconfig_prefix='CPTCFG_',
project_prefix='', project_dir='',
- target_dir='', target_dir_name=''):
+ target_dir='', target_dir_name='',
+ kconfig_source_var=None):
self.integrate = integrate
self.kconfig_prefix = kconfig_prefix
self.kconfig_prefix_resafe = re.escape(kconfig_prefix)
@@ -43,6 +44,11 @@ class Bp_Identity(object):
self.project_dir = project_dir
self.target_dir = target_dir
self.target_dir_name = target_dir_name
+ self.kconfig_source_var = kconfig_source_var
+ if self.kconfig_source_var:
+ self.kconfig_source_var_resafe = re.escape(self.kconfig_source_var)
+ else:
+ self.kconfig_source_var_resafe = None
def read_copy_list(copyfile):
"""
@@ -681,6 +687,7 @@ def _main():
project_dir = args.outdir,
target_dir = os.path.join(args.outdir, 'backports/'),
target_dir_name = 'backports/',
+ kconfig_source_var = '$BACKPORT_DIR',
)
else:
bpid = Bp_Identity(integrate = integrate,
@@ -689,6 +696,7 @@ def _main():
project_dir = args.outdir,
target_dir = args.outdir,
target_dir_name = '',
+ kconfig_source_var = '$BACKPORT_DIR',
)
def logwrite(msg):
@@ -820,6 +828,7 @@ def process(kerneldir, copy_list_file, git_revision=None,
disable_list = add_automatic_backports(args)
if disable_list:
+ # No need to verify_sources() as compat's Kconfig has no 'source' call
bpcfg = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'compat', 'Kconfig'), bpid)
bpcfg.disable_symbols(disable_list)
git_debug_snapshot(args, 'Add automatic backports')
@@ -828,10 +837,15 @@ def process(kerneldir, copy_list_file, git_revision=None,
# some post-processing is required
configtree = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'Kconfig'), bpid)
+ ignore=['Kconfig.kernel', 'Kconfig.versions']
+
+ configtree.verify_sources(ignore=ignore)
+ git_debug_snapshot(args, "verify sources on top level backports Kconfig")
+
orig_symbols = configtree.symbols()
logwrite('Modify Kconfig tree ...')
- configtree.prune_sources(ignore=['Kconfig.kernel', 'Kconfig.versions'])
+ configtree.prune_sources(ignore=ignore)
git_debug_snapshot(args, "prune Kconfig tree")
if not bpid.integrate:
diff --git a/lib/kconfig.py b/lib/kconfig.py
index ac96613..ca3af63 100644
--- a/lib/kconfig.py
+++ b/lib/kconfig.py
@@ -12,10 +12,41 @@ sel_line = re.compile(r'^(?P<spc>\s+)select\s+(?P<sym>[^\s]*)\s*$')
backport_line = re.compile(r'^\s+#(?P<key>[ch]-file|module-name)\s*(?P<name>.*)')
class ConfigTree(object):
+ """
+ If you suppport kernel integration you should use bpid.kconfig_source_var
+ and set that to the the variable name you have used on your Kconfig file
+ for use as a directory prefix. When packaging you only need to set this on
+ the Kconfig sources part of backports. For integration this variable will
+ be used to prefix all Kconfig sources taken from the kernel. This variable
+ is used by Kconfig to expand to bpid.target_dir_name. All kernel Kconfig
+ source entries use a full path based on bpid.project_dir, when using
+ integration a prefix is needed on backported Kconfig entries to
+ help ensure that the base directory used for backported code is
+ bpid.target_dir and not bpid.project_dir. To help with we provide a
+ verification / modifier, verify_sources(), of the Kconfig entries, you
+ run this before working on the ConfigTree with any other helper.
+
+ Verification is only needed when integrating and if the top level Kconfig
+ entries that have a source entry.
+ """
def __init__(self, rootfile, bpid):
self.bpid = bpid
self.rootfile = os.path.basename(rootfile)
- self.src_line = re.compile(r'^\s*source\s+"(?P<src>[^\s"]*)"?\s*$')
+ if self.bpid.kconfig_source_var:
+ self.src_line = re.compile(r'^\s*source\s+"(?P<kconfig_var>' + self.bpid.kconfig_source_var_resafe + ')?/?(?P<src>[^\s"]*)"?\s*$')
+ self.kconfig_var_line = re.compile(r'.*(?P<kconfig_var>' + self.bpid.kconfig_source_var_resafe + ')+/+')
+ else:
+ self.src_line = re.compile(r'^\s*source\s+"(?P<src>[^\s"]*)"?\s*$')
+ self.kconfig_var_line = None
+ self.verified = False
+ self.need_verification = False
+ if self.bpid.integrate:
+ for l in open(os.path.join(self.bpid.target_dir, rootfile), 'r'):
+ m = self.src_line.match(l)
+ mrel = src_line_rel.match(l)
+ if m or mrel:
+ self.need_verification = True
+ break
def _check_relative_source(self, f, l):
#
@@ -28,12 +59,17 @@ class ConfigTree(object):
if m:
raise Exception('File: %s uses relative kconfig source entries (line: \'%s\'), use full path with quotes' %
(os.path.join(self.bpid.target_dir, f), l))
- def _walk(self, f):
+ def _walk(self, f, first_pass=False):
+ if self.bpid.integrate:
+ if not self.bpid.kconfig_source_var and self.need_verification:
+ raise Exception("You enabled integration but haven't set bpid.kconfig_source_var, this seems incorrect")
+ if not first_pass and not self.verified and self.need_verification:
+ raise Exception("You must run verify_sources() first, we don't do that for you as you may want to ignore some source files")
yield f
for l in open(os.path.join(self.bpid.target_dir, f), 'r'):
m = self.src_line.match(l)
if m and os.path.exists(os.path.join(self.bpid.target_dir, m.group('src'))):
- for i in self._walk(m.group('src')):
+ for i in self._walk(m.group('src'), first_pass=first_pass):
yield i
else:
self._check_relative_source(f, l)
@@ -49,12 +85,42 @@ class ConfigTree(object):
continue
src = m.group('src')
if src in ignore or os.path.exists(os.path.join(self.bpid.target_dir, src)):
+ out += l
+ else:
+ out += '#' + l
+ outf = open(os.path.join(self.bpid.target_dir, nf), 'w')
+ outf.write(out)
+ outf.close()
+
+ def verify_sources(self, ignore=[]):
+ if not self.need_verification:
+ return
+ for nf in self._walk(self.rootfile, first_pass=True):
+ out = ''
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
+ m = self.src_line.match(l)
+ if not m:
+ self._check_relative_source(nf, l)
out += l
+ continue
+ src = m.group('src')
+ k = self.kconfig_var_line.match(l)
+ if src in ignore or os.path.exists(os.path.join(self.bpid.target_dir, src)):
+ if k:
+ out += l
+ else:
+ out += 'source "%s/%s"\n' % (self.bpid.kconfig_source_var, m.group('src'))
else:
- out += '#' + l
+ if k:
+ out += '# source "' + self.bpid.kconfig_source_var + '/' + src + '"\n'
+ else:
+ out += '#' + l
outf = open(os.path.join(self.bpid.target_dir, nf), 'w')
outf.write(out)
outf.close()
+ # Now the kconfig_var is always required from now on
+ self.src_line = re.compile(r'^\s*source\s+"(?P<kconfig_var>' + self.bpid.kconfig_source_var_resafe + ')+/+(?P<src>[^\s"]*)"?\s*$')
+ self.verified = True
def prune_sources(self, ignore=[]):
self._prune_sources(self.rootfile, ignore)
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 14/21] backports: update dependencies map file
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (12 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 13/21] backports: use BACKPORT_DIR prefix on kconfig sources Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 15/21] backports: split Kconfig into Kconfig.package and Kconfig.sources Luis R. Rodriguez
` (7 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
During development of kernel integration support using CONFIG_BACKPORT
was evaluated as a prefix over CPTCFG even for packaging backports,
for some reason this change lifted some restrictions one some device
drivers which was present before and as such requires some changes to
the dependencies map file to ensure correct compilation for respective
kernel versions.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
dependencies | 31 +++++++++++++++++++++++--------
1 file changed, 23 insertions(+), 8 deletions(-)
diff --git a/dependencies b/dependencies
index 4964aa6..64903dc 100644
--- a/dependencies
+++ b/dependencies
@@ -16,6 +16,8 @@ USB_GSPCA_OV534 3.5
# we only support on >= 3.4 give that reguatory itself depends
# on -EPROBE_DEFER.
SOC_CAMERA 3.4
+VIDEO_S5K5BAF 3.4
+VIDEO_S5K4ECGX 3.4
# 49920bc6 and 1003cab8 and while it seems there is a
# one to one map DMA_FROM_DEVICE to DMA_DEV_TO_MEM
# I can't verify this fully yet.
@@ -29,16 +31,29 @@ DVB_USB_RTL28XXU 3.4
# missing linux/gpio/consumer.h
VIDEO_ADV7604 3.13
# missing function clk_prepare_enable(), clk_disable_unprepare() and devm_regulator_bulk_get()
-VIDEO_MT9P031 3.3
-VIDEO_MT9T001 3.3
VIDEO_MT9V032 3.3
-VIDEO_NOON010PC30 3.3
-VIDEO_S5K6A3 3.3
VIDEO_S5K6AA 3.3
-VIDEO_S5K5BAF 3.3
-VIDEO_S5K4ECGX 3.3
-VIDEO_M5MOLS 3.3
-VIDEO_S5C73M3 3.3
+
+# missing devm_regulator_bulk_get
+VIDEO_MT9T001 3.5
+VIDEO_NOON010PC30 3.5
+VIDEO_MT9P031 3.5
+
+# missing of_property_read_bool() added via fa4d34cc on v3.4-rc1~71, backport
+# that attomically and you can enable this for older kernels.
+VIDEO_ADV7343 3.4
+
+# for whoever wants to backport this: GPIOF_EXPORT was redefined via commit
+# 79a9becd on v3.13-rc1~120^2~15, before this it was redefined via commit
+# f567fde2 on v3.5-rc7~32^2~4 and finally added via commit fc3a1f04 on
+# v3.5-rc1~93^2~25. We don't address any of these changes yet so we require
+# the latest interpretation.
+VIDEO_S5K6AA 3.13
+VIDEO_S5K6A3 3.13
+
+# missing linux/sizes.h
+VIDEO_M5MOLS 3.6
+VIDEO_S5C73M3 3.6
# missing function devm_regmap_init_i2c()
VIDEO_LM3646 3.3
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 15/21] backports: split Kconfig into Kconfig.package and Kconfig.sources
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (13 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 14/21] backports: update dependencies map file Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 16/21] backports: move version file generation to run earlier Luis R. Rodriguez
` (6 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
This splits up the Kconfig from things which are needed due to
our copy of code from the kernel and things which we need
specifically only for packaging backports. We will later then
share the Kconfig.souce for instance for kernel integration.
While at it, split up the list of files that we need to copy
into what we know could be shared for integration. The the
case of backport packaging Kconfig.package will be copied to
the package's Kconfig when building the package with gentree.py.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/Kconfig | 44 --------------------------------------------
backport/Kconfig.package | 24 ++++++++++++++++++++++++
backport/Kconfig.sources | 23 +++++++++++++++++++++++
gentree.py | 27 ++++++++++++++++++++++-----
4 files changed, 69 insertions(+), 49 deletions(-)
delete mode 100644 backport/Kconfig
create mode 100644 backport/Kconfig.package
create mode 100644 backport/Kconfig.sources
diff --git a/backport/Kconfig b/backport/Kconfig
deleted file mode 100644
index b729d47..0000000
--- a/backport/Kconfig
+++ /dev/null
@@ -1,44 +0,0 @@
-mainmenu "Linux Backports from $BACKPORTED_KERNEL_NAME $BACKPORTED_KERNEL_VERSION (with backports $BACKPORTS_VERSION)"
-
-config BACKPORT_DIR
- string
- option env="BACKPORT_DIR"
-config BACKPORTS_VERSION
- string
- option env="BACKPORTS_VERSION"
-config BACKPORTED_KERNEL_VERSION
- string
- option env="BACKPORTED_KERNEL_VERSION"
-config BACKPORTED_KERNEL_NAME
- string
- option env="BACKPORTED_KERNEL_NAME"
-
-source "$BACKPORT_DIR/Kconfig.kernel"
-source "$BACKPORT_DIR/Kconfig.versions"
-
-# Packaging hacks
-source "$BACKPORT_DIR/Kconfig.package.hacks"
-
-# this has the configuration for the backport code
-source "$BACKPORT_DIR/compat/Kconfig"
-
-# these are copied from the kernel
-source "$BACKPORT_DIR/net/wireless/Kconfig"
-source "$BACKPORT_DIR/net/mac80211/Kconfig"
-source "$BACKPORT_DIR/net/bluetooth/Kconfig"
-source "$BACKPORT_DIR/drivers/net/wireless/Kconfig"
-source "$BACKPORT_DIR/drivers/net/ethernet/Kconfig"
-source "$BACKPORT_DIR/drivers/net/usb/Kconfig"
-
-source "$BACKPORT_DIR/drivers/ssb/Kconfig"
-source "$BACKPORT_DIR/drivers/bcma/Kconfig"
-
-source "$BACKPORT_DIR/net/nfc/Kconfig"
-
-source "$BACKPORT_DIR/drivers/media/Kconfig"
-
-source "$BACKPORT_DIR/net/ieee802154/Kconfig"
-source "$BACKPORT_DIR/net/mac802154/Kconfig"
-source "$BACKPORT_DIR/drivers/net/ieee802154/Kconfig"
-
-source "$BACKPORT_DIR/drivers/usb/class/Kconfig"
diff --git a/backport/Kconfig.package b/backport/Kconfig.package
new file mode 100644
index 0000000..9684574
--- /dev/null
+++ b/backport/Kconfig.package
@@ -0,0 +1,24 @@
+mainmenu "Backports from $BACKPORTED_KERNEL_NAME $BACKPORTED_KERNEL_VERSION (backports $BACKPORTS_VERSION)"
+
+config BACKPORT_DIR
+ string
+ option env="BACKPORT_DIR"
+config BACKPORTS_VERSION
+ string
+ option env="BACKPORTS_VERSION"
+config BACKPORTED_KERNEL_VERSION
+ string
+ option env="BACKPORTED_KERNEL_VERSION"
+config BACKPORTED_KERNEL_NAME
+ string
+ option env="BACKPORTED_KERNEL_NAME"
+
+# Packaging hacks
+source "$BACKPORT_DIR/Kconfig.package.hacks"
+
+# Code we backport
+source "$BACKPORT_DIR/Kconfig.sources"
+
+# these will be generated
+source "$BACKPORT_DIR/Kconfig.kernel"
+source "$BACKPORT_DIR/Kconfig.versions"
diff --git a/backport/Kconfig.sources b/backport/Kconfig.sources
new file mode 100644
index 0000000..cdf993c
--- /dev/null
+++ b/backport/Kconfig.sources
@@ -0,0 +1,23 @@
+# this has the configuration for the backport code
+source "$BACKPORT_DIR/compat/Kconfig"
+
+# these are copied from the kernel
+source "$BACKPORT_DIR/net/wireless/Kconfig"
+source "$BACKPORT_DIR/net/mac80211/Kconfig"
+source "$BACKPORT_DIR/net/bluetooth/Kconfig"
+source "$BACKPORT_DIR/drivers/net/wireless/Kconfig"
+source "$BACKPORT_DIR/drivers/net/ethernet/Kconfig"
+source "$BACKPORT_DIR/drivers/net/usb/Kconfig"
+
+source "$BACKPORT_DIR/drivers/ssb/Kconfig"
+source "$BACKPORT_DIR/drivers/bcma/Kconfig"
+
+source "$BACKPORT_DIR/net/nfc/Kconfig"
+
+source "$BACKPORT_DIR/drivers/media/Kconfig"
+
+source "$BACKPORT_DIR/net/ieee802154/Kconfig"
+source "$BACKPORT_DIR/net/mac802154/Kconfig"
+source "$BACKPORT_DIR/drivers/net/ieee802154/Kconfig"
+
+source "$BACKPORT_DIR/drivers/usb/class/Kconfig"
diff --git a/gentree.py b/gentree.py
index 7632b90..71e52e4 100755
--- a/gentree.py
+++ b/gentree.py
@@ -798,12 +798,28 @@ def process(kerneldir, copy_list_file, git_revision=None,
check_output_dir(bpid.target_dir, args.clean)
# do the copy
+ backport_package_files = [(x, x) for x in [
+ 'Makefile',
+ 'kconf/',
+ 'Makefile.real',
+ 'Makefile.kernel',
+ 'Kconfig.package.hacks',
+ 'scripts/',
+ '.blacklist.map',
+ '.gitignore',
+ 'Makefile.build'] ]
+ backport_package_files += [
+ ('Kconfig.package', 'Kconfig'),
+ ]
backport_files = [(x, x) for x in [
- 'Kconfig', 'Kconfig.package.hacks',
- 'Makefile', 'Makefile.build', 'Makefile.kernel', '.gitignore',
- 'Makefile.real', 'compat/', 'backport-include/', 'kconf/',
- 'scripts/', '.blacklist.map',
+ 'Kconfig.sources',
+ 'compat/',
+ 'backport-include/',
]]
+
+ if not bpid.integrate:
+ backport_files += backport_package_files
+
if not args.git_revision:
logwrite('Copy original source files ...')
else:
@@ -827,6 +843,7 @@ def process(kerneldir, copy_list_file, git_revision=None,
git_debug_snapshot(args, 'Add driver sources')
disable_list = add_automatic_backports(args)
+
if disable_list:
# No need to verify_sources() as compat's Kconfig has no 'source' call
bpcfg = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'compat', 'Kconfig'), bpid)
@@ -877,7 +894,7 @@ def process(kerneldir, copy_list_file, git_revision=None,
# write local symbol list -- needed during packaging build
if not bpid.integrate:
- f = open(os.path.join(bpid.project_dir, '.local-symbols'), 'w')
+ f = open(os.path.join(bpid.target_dir, '.local-symbols'), 'w')
for sym in symbols:
f.write('%s=\n' % sym)
f.close()
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 16/21] backports: move version file generation to run earlier
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (14 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 15/21] backports: split Kconfig into Kconfig.package and Kconfig.sources Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 17/21] backports: define C code backport version info using CPTCFG_ Luis R. Rodriguez
` (5 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
The version file contains:
* BACKPORTS_VERSION
* BACKPORTED_KERNEL_VERSION
* BACKPORTED_KERNEL_NAME
This file is used by the Makefile to export variable
definitions used to identify the backport. For kernel
integration we'll be using a generated Kconfig file
instead and since these are needed when processing
kconfig logic this will need to be defined earlier.
Just move the generation of the version file for packaging
up early to match where we will generate the Kconfig with
the backports versioning identification.
This change has no real functional change other than the
move of some code running earlier.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
gentree.py | 36 ++++++++++++++++++------------------
1 file changed, 18 insertions(+), 18 deletions(-)
diff --git a/gentree.py b/gentree.py
index 71e52e4..afa4967 100755
--- a/gentree.py
+++ b/gentree.py
@@ -843,6 +843,24 @@ def process(kerneldir, copy_list_file, git_revision=None,
git_debug_snapshot(args, 'Add driver sources')
disable_list = add_automatic_backports(args)
+ if git_tracked_version:
+ backports_version = "(see git)"
+ kernel_version = "(see git)"
+ else:
+ backports_version = git.describe(tree=source_dir, extra_args=['--long'])
+ kernel_version = git.describe(rev=args.git_revision or 'HEAD',
+ tree=args.kerneldir,
+ extra_args=['--long'])
+
+ if not bpid.integrate:
+ f = open(os.path.join(bpid.target_dir, 'versions'), 'w')
+ f.write('BACKPORTS_VERSION="%s"\n' % backports_version)
+ f.write('BACKPORTED_KERNEL_VERSION="%s"\n' % kernel_version)
+ f.write('BACKPORTED_KERNEL_NAME="%s"\n' % args.base_name)
+ if git_tracked_version:
+ f.write('BACKPORTS_GIT_TRACKED="backport tracker ID: $(shell git rev-parse HEAD 2>/dev/null || echo \'not built in git tree\')"\n')
+ f.close()
+ git_debug_snapshot(args, "add versions files")
if disable_list:
# No need to verify_sources() as compat's Kconfig has no 'source' call
@@ -872,24 +890,6 @@ def process(kerneldir, copy_list_file, git_revision=None,
configtree.modify_selects()
git_debug_snapshot(args, "convert select to depends on")
- # write the versioning file
- if git_tracked_version:
- backports_version = "(see git)"
- kernel_version = "(see git)"
- else:
- backports_version = git.describe(tree=source_dir, extra_args=['--long'])
- kernel_version = git.describe(rev=args.git_revision or 'HEAD',
- tree=args.kerneldir,
- extra_args=['--long'])
- f = open(os.path.join(bpid.target_dir, 'versions'), 'w')
- f.write('BACKPORTS_VERSION="%s"\n' % backports_version)
- f.write('BACKPORTED_KERNEL_VERSION="%s"\n' % kernel_version)
- f.write('BACKPORTED_KERNEL_NAME="%s"\n' % args.base_name)
- if git_tracked_version:
- f.write('BACKPORTS_GIT_TRACKED="backport tracker ID: $(shell git rev-parse HEAD 2>/dev/null || echo \'not built in git tree\')"\n')
- f.close()
- git_debug_snapshot(args, "add versions files")
-
symbols = configtree.symbols()
# write local symbol list -- needed during packaging build
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 17/21] backports: define C code backport version info using CPTCFG_
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (15 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 16/21] backports: move version file generation to run earlier Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 18/21] backports: add backport version parsing for kernel integration Luis R. Rodriguez
` (4 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
In order to help unify the naming scheme for shared
backports versioning information rely on the CPTCFG_
prefix, when integration support gets added that will
translate to the respective CONFIG_BACKPORT_ prefix.
Kconfig opt env entries don't get propagated out, so
we need to define these ourselves. This leaves all
other names in place for packaging and just focuses
on sharing on the C / header code.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/Makefile.kernel | 6 +++---
backport/backport-include/linux/module.h | 6 +++---
backport/compat/main.c | 32 ++++++++++++++++----------------
3 files changed, 22 insertions(+), 22 deletions(-)
diff --git a/backport/Makefile.kernel b/backport/Makefile.kernel
index dcb2ba7..a63184b 100644
--- a/backport/Makefile.kernel
+++ b/backport/Makefile.kernel
@@ -10,9 +10,9 @@ NOSTDINC_FLAGS := \
-I$(M)/include/uapi \
-include $(M)/backport-include/backport/backport.h \
$(call backport-cc-disable-warning, unused-but-set-variable) \
- -DBACKPORTS_VERSION=\"$(BACKPORTS_VERSION)\" \
- -DBACKPORTED_KERNEL_VERSION=\"$(BACKPORTED_KERNEL_VERSION)\" \
- -DBACKPORTED_KERNEL_NAME=\"$(BACKPORTED_KERNEL_NAME)\" \
+ -DCPTCFG_VERSION=\"$(BACKPORTS_VERSION)\" \
+ -DCPTCFG_KERNEL_VERSION=\"$(BACKPORTED_KERNEL_VERSION)\" \
+ -DCPTCFG_KERNEL_NAME=\"$(BACKPORTED_KERNEL_NAME)\" \
$(BACKPORTS_GIT_TRACKER_DEF) \
$(CFLAGS)
diff --git a/backport/backport-include/linux/module.h b/backport/backport-include/linux/module.h
index e8f28b1..8870abb 100644
--- a/backport/backport-include/linux/module.h
+++ b/backport/backport-include/linux/module.h
@@ -20,9 +20,9 @@ extern void backport_dependency_symbol(void);
#define BACKPORT_MOD_VERSIONS MODULE_VERSION(BACKPORTS_GIT_TRACKED);
#else
#define BACKPORT_MOD_VERSIONS \
- MODULE_VERSION("backported from " BACKPORTED_KERNEL_NAME \
- " (" BACKPORTED_KERNEL_VERSION ")" \
- " using backports " BACKPORTS_VERSION);
+ MODULE_VERSION("backported from " CPTCFG_KERNEL_NAME \
+ " (" CPTCFG_KERNEL_VERSION ")" \
+ " using backports " CPTCFG_VERSION);
#endif
#ifdef MODULE
diff --git a/backport/compat/main.c b/backport/compat/main.c
index d3f8944..04ebbfd 100644
--- a/backport/compat/main.c
+++ b/backport/compat/main.c
@@ -8,40 +8,40 @@ MODULE_AUTHOR("Luis R. Rodriguez");
MODULE_DESCRIPTION("Kernel backport module");
MODULE_LICENSE("GPL");
-#ifndef BACKPORTED_KERNEL_NAME
-#error "You need a BACKPORTED_KERNEL_NAME"
+#ifndef CPTCFG_KERNEL_NAME
+#error "You need a CPTCFG_KERNEL_NAME"
#endif
-#ifndef BACKPORTED_KERNEL_VERSION
-#error "You need a BACKPORTED_KERNEL_VERSION"
+#ifndef CPTCFG_KERNEL_VERSION
+#error "You need a CPTCFG_KERNEL_VERSION"
#endif
-#ifndef BACKPORTS_VERSION
-#error "You need a BACKPORTS_VERSION"
+#ifndef CPTCFG_VERSION
+#error "You need a CPTCFG_VERSION"
#endif
-static char *backported_kernel_name = BACKPORTED_KERNEL_NAME;
+static char *backported_kernel_name = CPTCFG_KERNEL_NAME;
module_param(backported_kernel_name, charp, 0400);
MODULE_PARM_DESC(backported_kernel_name,
- "The kernel tree name that was used for this backport (" BACKPORTED_KERNEL_NAME ")");
+ "The kernel tree name that was used for this backport (" CPTCFG_KERNEL_NAME ")");
-#ifdef BACKPORTS_GIT_TRACKED
+#ifdef BACKPORTS_GIT_TRACKED
static char *backports_tracker_id = BACKPORTS_GIT_TRACKED;
module_param(backports_tracker_id, charp, 0400);
MODULE_PARM_DESC(backports_tracker_id,
"The version of the tree containing this backport (" BACKPORTS_GIT_TRACKED ")");
#else
-static char *backported_kernel_version = BACKPORTED_KERNEL_VERSION;
-static char *backports_version = BACKPORTS_VERSION;
+static char *backported_kernel_version = CPTCFG_KERNEL_VERSION;
+static char *backports_version = CPTCFG_VERSION;
module_param(backported_kernel_version, charp, 0400);
MODULE_PARM_DESC(backported_kernel_version,
- "The kernel version that was used for this backport (" BACKPORTED_KERNEL_VERSION ")");
+ "The kernel version that was used for this backport (" CPTCFG_KERNEL_VERSION ")");
module_param(backports_version, charp, 0400);
MODULE_PARM_DESC(backports_version,
- "The git version of the backports tree used to generate this backport (" BACKPORTS_VERSION ")");
+ "The git version of the backports tree used to generate this backport (" CPTCFG_VERSION ")");
#endif
@@ -63,15 +63,15 @@ static int __init backport_init(void)
return ret;
}
- printk(KERN_INFO "Loading modules backported from " BACKPORTED_KERNEL_NAME
+ printk(KERN_INFO "Loading modules backported from " CPTCFG_KERNEL_NAME
#ifndef BACKPORTS_GIT_TRACKED
- " version " BACKPORTED_KERNEL_VERSION
+ " version " CPTCFG_KERNEL_VERSION
#endif
"\n");
#ifdef BACKPORTS_GIT_TRACKED
printk(KERN_INFO BACKPORTS_GIT_TRACKED "\n");
#else
- printk(KERN_INFO "Backport generated by backports.git " BACKPORTS_VERSION "\n");
+ printk(KERN_INFO "Backport generated by backports.git " CPTCFG_VERSION "\n");
#endif
return 0;
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 18/21] backports: add backport version parsing for kernel integration
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (16 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 17/21] backports: define C code backport version info using CPTCFG_ Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 19/21] backports: prefix c-file / h-file auto backport with BPAUTO Luis R. Rodriguez
` (3 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
The way we'll define backports versioning information for
kernel integration is slightly different, we'll rely completely
on Kconfig for the job, but in the end share the same C code
defines.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
gentree.py | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/gentree.py b/gentree.py
index afa4967..3e130f1 100755
--- a/gentree.py
+++ b/gentree.py
@@ -861,6 +861,24 @@ def process(kerneldir, copy_list_file, git_revision=None,
f.write('BACKPORTS_GIT_TRACKED="backport tracker ID: $(shell git rev-parse HEAD 2>/dev/null || echo \'not built in git tree\')"\n')
f.close()
git_debug_snapshot(args, "add versions files")
+ else:
+ kconf_regexes = [
+ (re.compile(r'.*(?P<key>%%BACKPORT_DIR%%)'), '%%BACKPORT_DIR%%', 'backports/'),
+ (re.compile(r'.*(?P<key>%%BACKPORTS_VERSION%%).*'), '%%BACKPORTS_VERSION%%', backports_version),
+ (re.compile(r'.*(?P<key>%%BACKPORTED_KERNEL_VERSION%%).*'), '%%BACKPORTED_KERNEL_VERSION%%', kernel_version),
+ (re.compile(r'.*(?P<key>%%BACKPORTED_KERNEL_NAME%%).*'), '%%BACKPORTED_KERNEL_NAME%%', args.base_name),
+ ]
+ out = ''
+ for l in open(os.path.join(bpid.target_dir, 'Kconfig'), 'r'):
+ for r in kconf_regexes:
+ m = r[0].match(l)
+ if m:
+ l = re.sub(r'(' + r[1] + ')', r'' + r[2] + '', l)
+ out += l
+ outf = open(os.path.join(bpid.target_dir, 'Kconfig'), 'w')
+ outf.write(out)
+ outf.close()
+ git_debug_snapshot(args, "modify top level backports/Kconfig with backports identity")
if disable_list:
# No need to verify_sources() as compat's Kconfig has no 'source' call
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 19/21] backports: prefix c-file / h-file auto backport with BPAUTO
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (17 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 18/21] backports: add backport version parsing for kernel integration Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 20/21] backports: remove extra BACKPORT_ prefix from kernel versioning Luis R. Rodriguez
` (2 subsequent siblings)
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
Things we backport are now prefixed with BACKPORT_, when we copy
over libraries with the c-files / h-files trick on the compat/Kconfig
trick we prefix these with BACKPORT_ already so in order to help
distinguish them use BACKPORT_BPAUTO_ for them and prevent a double
BACKPORT_ prefix.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/backport-include/asm/dma-mapping.h | 4 +-
backport/backport-include/backport/leds-disabled.h | 2 +-
backport/compat/Kconfig | 74 +++++++++++-----------
backport/compat/Makefile | 4 +-
backport/compat/backports.h | 4 +-
gentree.py | 2 +-
patches/backport-adjustments/devcoredump.patch | 4 +-
7 files changed, 47 insertions(+), 47 deletions(-)
diff --git a/backport/backport-include/asm/dma-mapping.h b/backport/backport-include/asm/dma-mapping.h
index 844fe3b..b73b7da 100644
--- a/backport/backport-include/asm/dma-mapping.h
+++ b/backport/backport-include/asm/dma-mapping.h
@@ -3,12 +3,12 @@
#include_next <asm/dma-mapping.h>
#include <linux/version.h>
-#if defined(CPTCFG_BACKPORT_BUILD_DMA_SHARED_HELPERS)
+#if defined(CPTCFG_BPAUTO_BUILD_DMA_SHARED_HELPERS)
#define dma_common_get_sgtable LINUX_BACKPORT(dma_common_get_sgtable)
int
dma_common_get_sgtable(struct device *dev, struct sg_table *sgt,
void *cpu_addr, dma_addr_t dma_addr, size_t size);
-#endif /* defined(CPTCFG_BACKPORT_BUILD_DMA_SHARED_HELPERS) */
+#endif /* defined(CPTCFG_BPAUTO_BUILD_DMA_SHARED_HELPERS) */
#if LINUX_VERSION_CODE < KERNEL_VERSION(3,6,0)
diff --git a/backport/backport-include/backport/leds-disabled.h b/backport/backport-include/backport/leds-disabled.h
index 501f2a0..156d7fa 100644
--- a/backport/backport-include/backport/leds-disabled.h
+++ b/backport/backport-include/backport/leds-disabled.h
@@ -8,7 +8,7 @@
* allows compilation.
*/
-#ifdef CPTCFG_BACKPORT_BUILD_LEDS
+#ifdef CPTCFG_BPAUTO_BUILD_LEDS
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/rwsem.h>
diff --git a/backport/compat/Kconfig b/backport/compat/Kconfig
index 49699bd..06ef8d5 100644
--- a/backport/compat/Kconfig
+++ b/backport/compat/Kconfig
@@ -1,16 +1,16 @@
#
# backport Kconfig
#
-# Some options are user-selectable ("BACKPORT_USERSEL_*")
+# Some options are user-selectable ("BPAUTO_USERSEL_*")
#
# Most options, however, follow a few different schemes:
#
# A) An option that is selected by drivers ("select FOO") will be
-# changed to "select BACKPORT_FOO" (if the option BACKPORT_FOO
-# exists). The option BACKPORT_FOO then controls setting of the
-# BACKPORT_BUILD_FOO option, which is a module, like this:
+# changed to "select BPAUTO_FOO" (if the option BPAUTO_FOO
+# exists). The option BPAUTO_FOO then controls setting of the
+# BPAUTO_BUILD_FOO option, which is a module, like this:
#
-# config BACKPORT_BUILD_FOO
+# config BPAUTO_BUILD_FOO
# tristate
# # or bool
#
@@ -25,13 +25,13 @@
# # kernel version X.Z
# depends on !FOO || BACKPORT_KERNEL_X_Z
#
-# # build if driver needs it (it selects BACKPORT_FOO)
-# default m if BACKPORT_FOO
+# # build if driver needs it (it selects BPAUTO_FOO)
+# default m if BPAUTO_FOO
#
-# # or for build-testing (BACKPORT_USERSEL_BUILD_ALL is enabled)
-# default m if BACKPORT_USERSEL_BUILD_ALL
+# # or for build-testing (BPAUTO_USERSEL_BUILD_ALL is enabled)
+# default m if BPAUTO_USERSEL_BUILD_ALL
#
-# config BACKPORT_FOO
+# config BPAUTO_FOO
# bool
#
# This only works as-is if the kernel code is usable on any version,
@@ -41,44 +41,44 @@
# B) An option for code always present on some kernels (e.g. KFIFO).
# This simply depends on/sets the default based on the version:
#
-# config BACKPORT_BUILD_KFIFO
+# config BPAUTO_BUILD_KFIFO
# def_bool y
# depends on BACKPORT_KERNEL_2_6_36
#
#
# C) similarly, a kconfig symbol for an option, e.g.
-# BACKPORT_OPTION_SOME_FIX (no examples provided) check git log
+# BPAUTO_OPTION_SOME_FIX (no examples provided) check git log
#
#
# Variations are obviously possible.
#
-config BACKPORT_BUILD_CORDIC
+config BPAUTO_BUILD_CORDIC
tristate
depends on !CORDIC
- default m if BACKPORT_CORDIC
- default m if BACKPORT_USERSEL_BUILD_ALL
+ default m if BPAUTO_CORDIC
+ default m if BPAUTO_USERSEL_BUILD_ALL
#module-name cordic
#c-file lib/cordic.c
-config BACKPORT_CORDIC
+config BPAUTO_CORDIC
bool
-config BACKPORT_BUILD_AVERAGE
+config BPAUTO_BUILD_AVERAGE
bool
depends on !AVERAGE
- default y if BACKPORT_USERSEL_BUILD_ALL
- default y if BACKPORT_AVERAGE
+ default y if BPAUTO_USERSEL_BUILD_ALL
+ default y if BPAUTO_AVERAGE
#h-file linux/average.h
#c-file lib/average.c
-config BACKPORT_AVERAGE
+config BPAUTO_AVERAGE
bool
-config BACKPORT_MII
+config BPAUTO_MII
bool
-config BACKPORT_BUILD_DMA_SHARED_HELPERS
+config BPAUTO_BUILD_DMA_SHARED_HELPERS
bool
depends on HAS_DMA
# Build on other kernels kernels < 3.9 if HAVE_GENERIC_DMA_COHERENT is
@@ -87,25 +87,25 @@ config BACKPORT_BUILD_DMA_SHARED_HELPERS
# Always build if on 3.3 - 3.5
default y if (BACKPORT_KERNEL_3_4 || BACKPORT_KERNEL_3_5 || BACKPORT_KERNEL_3_6)
# build for testing
- default y if BACKPORT_USERSEL_BUILD_ALL
+ default y if BPAUTO_USERSEL_BUILD_ALL
-config BACKPORT_BUILD_LEDS
+config BPAUTO_BUILD_LEDS
bool
depends on !NEW_LEDS || LEDS_CLASS=n || !LEDS_TRIGGERS
- default y if BACKPORT_NEW_LEDS
- default y if BACKPORT_LEDS_CLASS
- default y if BACKPORT_LEDS_TRIGGERS
+ default y if BPAUTO_NEW_LEDS
+ default y if BPAUTO_LEDS_CLASS
+ default y if BPAUTO_LEDS_TRIGGERS
-config BACKPORT_NEW_LEDS
+config BPAUTO_NEW_LEDS
bool
-config BACKPORT_LEDS_CLASS
+config BPAUTO_LEDS_CLASS
bool
-config BACKPORT_LEDS_TRIGGERS
+config BPAUTO_LEDS_TRIGGERS
bool
-config BACKPORT_USERSEL_BUILD_ALL
+config BPAUTO_USERSEL_BUILD_ALL
bool "Build all compat code"
help
This option selects all the compat code options
@@ -114,24 +114,24 @@ config BACKPORT_USERSEL_BUILD_ALL
It's only really useful for compat testing, so
you probably shouldn't enable it.
-config BACKPORT_CRYPTO_CCM
+config BPAUTO_CRYPTO_CCM
depends on CRYPTO_AEAD
depends on CRYPTO_CTR
bool
-config BACKPORT_BUILD_CRYPTO_CCM
+config BPAUTO_BUILD_CRYPTO_CCM
bool
default n if CRYPTO_CCM
- default y if BACKPORT_CRYPTO_CCM
+ default y if BPAUTO_CRYPTO_CCM
#c-file crypto/ccm.c
-config BACKPORT_WANT_DEV_COREDUMP
+config BPAUTO_WANT_DEV_COREDUMP
bool
-config BACKPORT_BUILD_WANT_DEV_COREDUMP
+config BPAUTO_BUILD_WANT_DEV_COREDUMP
bool
default n if DEV_COREDUMP
default n if DISABLE_DEV_COREDUMP
- default y if BACKPORT_WANT_DEV_COREDUMP
+ default y if BPAUTO_WANT_DEV_COREDUMP
#h-file linux/devcoredump.h
#c-file drivers/base/devcoredump.c
diff --git a/backport/compat/Makefile b/backport/compat/Makefile
index 6d210b0..0dd69fe 100644
--- a/backport/compat/Makefile
+++ b/backport/compat/Makefile
@@ -21,5 +21,5 @@ compat-$(CPTCFG_BACKPORT_KERNEL_3_15) += backport-3.15.o
compat-$(CPTCFG_BACKPORT_KERNEL_3_17) += backport-3.17.o
compat-$(CPTCFG_BACKPORT_KERNEL_3_18) += backport-3.18.o
-compat-$(CPTCFG_BACKPORT_BUILD_CRYPTO_CCM) += crypto-ccm.o
-compat-$(CPTCFG_BACKPORT_BUILD_DMA_SHARED_HELPERS) += dma-shared-helpers.o
+compat-$(CPTCFG_BPAUTO_BUILD_CRYPTO_CCM) += crypto-ccm.o
+compat-$(CPTCFG_BPAUTO_BUILD_DMA_SHARED_HELPERS) += dma-shared-helpers.o
diff --git a/backport/compat/backports.h b/backport/compat/backports.h
index c9094ac..323c908 100644
--- a/backport/compat/backports.h
+++ b/backport/compat/backports.h
@@ -3,7 +3,7 @@
#include <linux/version.h>
-#ifdef CPTCFG_BACKPORT_BUILD_CRYPTO_CCM
+#ifdef CPTCFG_BPAUTO_BUILD_CRYPTO_CCM
int crypto_ccm_module_init(void);
void crypto_ccm_module_exit(void);
#else
@@ -13,7 +13,7 @@ static inline void crypto_ccm_module_exit(void)
{}
#endif
-#ifdef CPTCFG_BACKPORT_BUILD_WANT_DEV_COREDUMP
+#ifdef CPTCFG_BPAUTO_WANT_DEV_COREDUMP
int devcoredump_init(void);
void devcoredump_exit(void);
#else
diff --git a/gentree.py b/gentree.py
index 3e130f1..bf2112b 100755
--- a/gentree.py
+++ b/gentree.py
@@ -235,7 +235,7 @@ def add_automatic_backports(args):
configtree = kconfig.ConfigTree(os.path.join(args.bpid.target_dir, 'Kconfig'), args.bpid)
all_selects = configtree.all_selects()
for sym, vals in bpi.items():
- if sym.startswith('BACKPORT_BUILD_'):
+ if sym.startswith('BPAUTO_BUILD_'):
if not sym[15:] in all_selects:
disable_list.append(sym)
continue
diff --git a/patches/backport-adjustments/devcoredump.patch b/patches/backport-adjustments/devcoredump.patch
index 3378884..3bf35f9 100644
--- a/patches/backport-adjustments/devcoredump.patch
+++ b/patches/backport-adjustments/devcoredump.patch
@@ -107,7 +107,7 @@ index c0a360e99f64..da20e61f6c06 100644
#include <linux/vmalloc.h>
-#ifdef CONFIG_DEV_COREDUMP
-+#ifdef CPTCFG_BACKPORT_BUILD_WANT_DEV_COREDUMP
++#ifdef CPTCFG_BPAUTO_WANT_DEV_COREDUMP
void dev_coredumpv(struct device *dev, const void *data, size_t datalen,
gfp_t gfp);
@@ -116,6 +116,6 @@ index c0a360e99f64..da20e61f6c06 100644
free(data);
}
-#endif /* CONFIG_DEV_COREDUMP */
-+#endif /* CPTCFG_BACKPORT_BUILD_WANT_DEV_COREDUMP */
++#endif /* CPTCFG_BPAUTO_WANT_DEV_COREDUMP */
#endif /* __DEVCOREDUMP_H */
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 20/21] backports: remove extra BACKPORT_ prefix from kernel versioning
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (18 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 19/21] backports: prefix c-file / h-file auto backport with BPAUTO Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 21/21] backports: add full kernel integration support Luis R. Rodriguez
2014-11-14 20:50 ` [PATCH v3 00/21] backports: add " Johannes Berg
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
The CPTCFG_ prefix already implies backport, when integration
is used we'd end up with a double BACKPORT_ prefix, so just
remove the existing one as its not needed.
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/Makefile | 2 +-
backport/compat/Kconfig | 10 +++----
backport/compat/Makefile | 34 +++++++++++-----------
devel/doc/kconfig-operation | 6 ++--
gentree.py | 2 +-
.../media/0002-no_dmabuf/v4l2.patch | 6 ++--
6 files changed, 30 insertions(+), 30 deletions(-)
diff --git a/backport/Makefile b/backport/Makefile
index fcf2f01..c21b813 100644
--- a/backport/Makefile
+++ b/backport/Makefile
@@ -93,7 +93,7 @@ mrproper:
print=0 ;\
for v in $$kvers ; do \
if [ "$$print" = "1" ] ; then \
- echo config BACKPORT_KERNEL_$$(echo $$v | tr . _) ;\
+ echo config KERNEL_$$(echo $$v | tr . _) ;\
echo " def_bool y" ;\
fi ;\
if [ "$$v" = "$$kver" ] ; then print=1 ; fi ;\
diff --git a/backport/compat/Kconfig b/backport/compat/Kconfig
index 06ef8d5..aadf246 100644
--- a/backport/compat/Kconfig
+++ b/backport/compat/Kconfig
@@ -16,14 +16,14 @@
#
# # not possible on kernel < X.Y, build will fail if any
# # drivers are allowed to build on kernels < X.Y
-# depends on BACKPORT_KERNEL_X_Y
+# depends on KERNEL_X_Y
#
# # don't build the backport code if FOO is in the kernel
# # already, but only if the kernel version is also >= X.Z;
# # this is an example of backporting where the version of
# # the FOO subsystem that we need is only available from
# # kernel version X.Z
-# depends on !FOO || BACKPORT_KERNEL_X_Z
+# depends on !FOO || KERNEL_X_Z
#
# # build if driver needs it (it selects BPAUTO_FOO)
# default m if BPAUTO_FOO
@@ -43,7 +43,7 @@
#
# config BPAUTO_BUILD_KFIFO
# def_bool y
-# depends on BACKPORT_KERNEL_2_6_36
+# depends on KERNEL_2_6_36
#
#
# C) similarly, a kconfig symbol for an option, e.g.
@@ -83,9 +83,9 @@ config BPAUTO_BUILD_DMA_SHARED_HELPERS
depends on HAS_DMA
# Build on other kernels kernels < 3.9 if HAVE_GENERIC_DMA_COHERENT is
# not set. Kernels >= 3.8 have this if HAS_DMA is set.
- depends on (!HAVE_GENERIC_DMA_COHERENT || BACKPORT_KERNEL_3_9)
+ depends on (!HAVE_GENERIC_DMA_COHERENT || KERNEL_3_9)
# Always build if on 3.3 - 3.5
- default y if (BACKPORT_KERNEL_3_4 || BACKPORT_KERNEL_3_5 || BACKPORT_KERNEL_3_6)
+ default y if (KERNEL_3_4 || KERNEL_3_5 || KERNEL_3_6)
# build for testing
default y if BPAUTO_USERSEL_BUILD_ALL
diff --git a/backport/compat/Makefile b/backport/compat/Makefile
index 0dd69fe..e787763 100644
--- a/backport/compat/Makefile
+++ b/backport/compat/Makefile
@@ -3,23 +3,23 @@ obj-m += compat.o
compat-y += main.o
# Kernel backport compatibility code
-compat-$(CPTCFG_BACKPORT_KERNEL_3_0) += compat-3.0.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_1) += compat-3.1.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_2) += backport-3.2.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_3) += compat-3.3.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_4) += compat-3.4.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_5) += compat-3.5.o user_namespace.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_6) += compat-3.6.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_7) += compat-3.7.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_8) += compat-3.8.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_9) += compat-3.9.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_10) += backport-3.10.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_12) += backport-3.12.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_13) += backport-3.13.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_14) += backport-3.14.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_15) += backport-3.15.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_17) += backport-3.17.o
-compat-$(CPTCFG_BACKPORT_KERNEL_3_18) += backport-3.18.o
+compat-$(CPTCFG_KERNEL_3_0) += compat-3.0.o
+compat-$(CPTCFG_KERNEL_3_1) += compat-3.1.o
+compat-$(CPTCFG_KERNEL_3_2) += backport-3.2.o
+compat-$(CPTCFG_KERNEL_3_3) += compat-3.3.o
+compat-$(CPTCFG_KERNEL_3_4) += compat-3.4.o
+compat-$(CPTCFG_KERNEL_3_5) += compat-3.5.o user_namespace.o
+compat-$(CPTCFG_KERNEL_3_6) += compat-3.6.o
+compat-$(CPTCFG_KERNEL_3_7) += compat-3.7.o
+compat-$(CPTCFG_KERNEL_3_8) += compat-3.8.o
+compat-$(CPTCFG_KERNEL_3_9) += compat-3.9.o
+compat-$(CPTCFG_KERNEL_3_10) += backport-3.10.o
+compat-$(CPTCFG_KERNEL_3_12) += backport-3.12.o
+compat-$(CPTCFG_KERNEL_3_13) += backport-3.13.o
+compat-$(CPTCFG_KERNEL_3_14) += backport-3.14.o
+compat-$(CPTCFG_KERNEL_3_15) += backport-3.15.o
+compat-$(CPTCFG_KERNEL_3_17) += backport-3.17.o
+compat-$(CPTCFG_KERNEL_3_18) += backport-3.18.o
compat-$(CPTCFG_BPAUTO_BUILD_CRYPTO_CCM) += crypto-ccm.o
compat-$(CPTCFG_BPAUTO_BUILD_DMA_SHARED_HELPERS) += dma-shared-helpers.o
diff --git a/devel/doc/kconfig-operation b/devel/doc/kconfig-operation
index f1ecf60..ddb4de7 100644
--- a/devel/doc/kconfig-operation
+++ b/devel/doc/kconfig-operation
@@ -53,12 +53,12 @@ The second file (Kconfig.versions) is generated from just the version of
the kernel and also contains invisible and unselectable boolean options
like
-config BACKPORT_KERNEL_X
+config KERNEL_X
def_bool y
to indicate that APIs introduced in kernel version X (e.g. 3_3 for 3.3
or 2_6_24 for 2.6.24) must be backported. Essentially, the presence of
-a symbol BACKPORT_KERNEL_X indicates that the kernel that the code is
+a symbol KERNEL_X indicates that the kernel that the code is
being compiled against is older than X.
All together, this allows the correct options to be selected by the user.
@@ -121,7 +121,7 @@ These are the problems and their solutions:
requires at least 2.6.29 due to the shash crypto code. This cannot be
determined easily automatically, so the 'dependencies' file is read
and options that are listed there are rewritten to include
- depends on !BACKPORT_KERNEL_X_Y
+ depends on !KERNEL_X_Y
(where X_Y is the listed kernel version), which makes them available
only on kernels >= X.Y.
diff --git a/gentree.py b/gentree.py
index bf2112b..90334ef 100755
--- a/gentree.py
+++ b/gentree.py
@@ -988,7 +988,7 @@ def process(kerneldir, copy_list_file, git_revision=None,
elif (dep == "DISABLE"):
new.append('BACKPORT_DISABLED_KCONFIG_OPTION')
else:
- new.append('!BACKPORT_KERNEL_%s' % dep.replace('.', '_'))
+ new.append('!KERNEL_%s' % dep.replace('.', '_'))
if bpid.integrate:
deplist[sym] = ["BACKPORT_" + x for x in new]
else:
diff --git a/patches/collateral-evolutions/media/0002-no_dmabuf/v4l2.patch b/patches/collateral-evolutions/media/0002-no_dmabuf/v4l2.patch
index 02d042a..7f88c3e 100644
--- a/patches/collateral-evolutions/media/0002-no_dmabuf/v4l2.patch
+++ b/patches/collateral-evolutions/media/0002-no_dmabuf/v4l2.patch
@@ -5,7 +5,7 @@
# Used by drivers that need Videobuf2 modules
config VIDEOBUF2_CORE
- select DMA_SHARED_BUFFER
-+ select DMA_SHARED_BUFFER if !BACKPORT_KERNEL_3_5
++ select DMA_SHARED_BUFFER if !KERNEL_3_5
tristate
config VIDEOBUF2_MEMOPS
@@ -14,14 +14,14 @@
select VIDEOBUF2_CORE
select VIDEOBUF2_MEMOPS
- select DMA_SHARED_BUFFER
-+ select DMA_SHARED_BUFFER if !BACKPORT_KERNEL_3_5
++ select DMA_SHARED_BUFFER if !KERNEL_3_5
config VIDEOBUF2_VMALLOC
tristate
select VIDEOBUF2_CORE
select VIDEOBUF2_MEMOPS
- select DMA_SHARED_BUFFER
-+ select DMA_SHARED_BUFFER if !BACKPORT_KERNEL_3_5
++ select DMA_SHARED_BUFFER if !KERNEL_3_5
config VIDEOBUF2_DMA_SG
tristate
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v3 21/21] backports: add full kernel integration support
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (19 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 20/21] backports: remove extra BACKPORT_ prefix from kernel versioning Luis R. Rodriguez
@ 2014-11-11 8:15 ` Luis R. Rodriguez
2014-11-14 20:50 ` [PATCH v3 00/21] backports: add " Johannes Berg
21 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-11 8:15 UTC (permalink / raw)
To: backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
From: "Luis R. Rodriguez" <mcgrof@suse.com>
This enables support for using the backports project
to integrate device drivers from a future version of Linux
into an older version of Linux. What you end up seeing is
a backports submenu when configuring your kernel and the
ability to select specific device drivers from subsystems
supported through the Linux backports project.
At this time enabling one device driver from a future version
of Linux will require using only the latest version of the
subsystem modules and other subsystem drivers. For example
enabling cfg80211 and mac80211 from a future version of Linux
will require you to only use future version of the respective
device drivers. In order to enable the backported version of
802.11 drivers for example, you will have to enable first:
Networking support -->
Wireless -->
But under that menu disable all options, then jump to the backports
submenu to now enable:
Backports -->
cfg80211
mac80211
Wireless LAN --->
etc
You build these device drivers modular or built-in to the kernel.
Integration support requires only slight modifications to the original
kernel sources, one to the top level Kconfig to add our entry, and also
the top level Makefile to enable backports code to be part of the
built-in vmlinux.
Support for integration takes advantage over the existing infrastructure
added by Johannes to keep track of each indvidual change done by the
backports infrastructure if --gitdebug is used.
mcgrof@drvbp1 ~/backports (git::master)$ time ./gentree.py --clean \
/home/mcgrof/linux-next /home/mcgrof/build/backports-20141023
Copy original source files ...
Applying patches from patches to /home/mcgrof/build/backports-20141023
...
Modify Kconfig tree ...
Rewrite Makefiles and Kconfig files ...
Done!
real 1m27.942s
user 13m23.752s
sys 0m47.608s
1 3.0.101 [ OK ]
2 3.1.10 [ OK ]
3 3.2.62 [ OK ]
4 3.3.8 [ OK ]
5 3.4.104 [ OK ]
6 3.5.7 [ OK ]
7 3.6.11 [ OK ]
8 3.7.10 [ OK ]
9 3.8.13 [ OK ]
10 3.9.11 [ OK ]
11 3.10.58 [ OK ]
12 3.11.10 [ OK ]
13 3.12.31 [ OK ]
14 3.13.11 [ OK ]
15 3.14.22 [ OK ]
16 3.15.10 [ OK ]
17 3.16.6 [ OK ]
18 3.17.1 [ OK ]
19 3.18-rc1 [ OK ]
real 42m44.838s
user 1190m5.092s
sys 140m37.208s
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
---
backport/Kconfig.integrate | 36 +++++++++++
backport/Makefile.kernel | 13 ++++
backport/backport-include/backport/backport.h | 5 ++
backport/compat/Makefile | 4 ++
backport/compat/main.c | 8 ++-
devel/doc/kconfig-operation | 5 ++
gentree.py | 72 ++++++++++++++++++++--
.../0001-enable-backports-built-in.patch | 40 ++++++++++++
lib/bpversion.py | 48 +++++++++++++++
lib/kconfig.py | 67 ++++++++++++++++++++
10 files changed, 291 insertions(+), 7 deletions(-)
create mode 100644 backport/Kconfig.integrate
create mode 100644 integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch
create mode 100644 lib/bpversion.py
diff --git a/backport/Kconfig.integrate b/backport/Kconfig.integrate
new file mode 100644
index 0000000..f64a3f6
--- /dev/null
+++ b/backport/Kconfig.integrate
@@ -0,0 +1,36 @@
+config BACKPORT_INTEGRATE
+ bool
+ def_bool y
+
+config BACKPORT_DIR
+ string
+ default "%%BACKPORT_DIR%%"
+
+config BACKPORT_VERSION
+ string
+ default "%%BACKPORTS_VERSION%%"
+
+config BACKPORT_KERNEL_VERSION
+ string
+ default "%%BACKPORTED_KERNEL_VERSION%%"
+
+config BACKPORT_KERNEL_NAME
+ string
+ default "%%BACKPORTED_KERNEL_NAME%%"
+
+menuconfig BACKPORT_LINUX
+ bool "Backport %%BACKPORTED_KERNEL_NAME%% %%BACKPORTED_KERNEL_VERSION%% (backports %%BACKPORTS_VERSION%%)"
+ default n
+ ---help---
+ Enabling this will let give you the opportunity to use features and
+ drivers backported from %%BACKPORTED_KERNEL_NAME%% %%BACKPORTED_KERNEL_VERSION%%
+ on the kernel your are using. This is experimental and you should
+ say no unless you'd like to help test things or want to help debug
+ this should we run into any issues.
+
+if BACKPORT_LINUX
+
+source "$BACKPORT_DIR/Kconfig.versions"
+source "$BACKPORT_DIR/Kconfig.sources"
+
+endif # BACKPORT_LINUX
diff --git a/backport/Makefile.kernel b/backport/Makefile.kernel
index a63184b..c57b2d3 100644
--- a/backport/Makefile.kernel
+++ b/backport/Makefile.kernel
@@ -1,3 +1,4 @@
+ifeq ($(CONFIG_BACKPORT_INTEGRATE),)
# Since 2.6.21, try-run is available, but cc-disable-warning
# was only added later, so we add it here ourselves:
backport-cc-disable-warning = $(call try-run,\
@@ -17,6 +18,18 @@ NOSTDINC_FLAGS := \
$(CFLAGS)
export backport_srctree = $(M)
+else
+export BACKPORT_DIR = backports/
+export backport_srctree = $(BACKPORT_DIR)
+NOSTDINC_FLAGS := \
+ -I$(BACKPORT_DIR)/backport-include/ \
+ -I$(BACKPORT_DIR)/backport-include/uapi \
+ -I$(BACKPORT_DIR)/include/ \
+ -I$(BACKPORT_DIR)/include/uapi \
+ -include $(BACKPORT_DIR)/backport-include/backport/backport.h \
+ $(CFLAGS)
+endif
+
obj-y += compat/
diff --git a/backport/backport-include/backport/backport.h b/backport/backport-include/backport/backport.h
index 7cf21aa..d1d3b10 100644
--- a/backport/backport-include/backport/backport.h
+++ b/backport/backport-include/backport/backport.h
@@ -1,11 +1,16 @@
#ifndef __BACKPORT_H
#define __BACKPORT_H
+#include <generated/autoconf.h>
+#ifndef CONFIG_BACKPORT_INTEGRATE
#include <backport/autoconf.h>
+#endif
#include <linux/kconfig.h>
#ifndef __ASSEMBLY__
#define LINUX_BACKPORT(__sym) backport_ ##__sym
+#ifndef CONFIG_BACKPORT_INTEGRATE
#include <backport/checks.h>
#endif
+#endif
#endif /* __BACKPORT_H */
diff --git a/backport/compat/Makefile b/backport/compat/Makefile
index e787763..f14b516 100644
--- a/backport/compat/Makefile
+++ b/backport/compat/Makefile
@@ -1,5 +1,9 @@
ccflags-y += -I$(src)
+ifeq ($(CONFIG_BACKPORT_INTEGRATE),)
obj-m += compat.o
+else
+obj-y += compat.o
+endif
compat-y += main.o
# Kernel backport compatibility code
diff --git a/backport/compat/main.c b/backport/compat/main.c
index 04ebbfd..5d45e3d 100644
--- a/backport/compat/main.c
+++ b/backport/compat/main.c
@@ -71,8 +71,14 @@ static int __init backport_init(void)
#ifdef BACKPORTS_GIT_TRACKED
printk(KERN_INFO BACKPORTS_GIT_TRACKED "\n");
#else
+
+#ifdef CONFIG_BACKPORT_INTEGRATE
+ printk(KERN_INFO "Backport integrated by backports.git " CPTCFG_VERSION "\n");
+#else
printk(KERN_INFO "Backport generated by backports.git " CPTCFG_VERSION "\n");
-#endif
+#endif /* CONFIG_BACKPORT_INTEGRATE */
+
+#endif /* BACKPORTS_GIT_TRACKED */
return 0;
}
diff --git a/devel/doc/kconfig-operation b/devel/doc/kconfig-operation
index ddb4de7..35e198c 100644
--- a/devel/doc/kconfig-operation
+++ b/devel/doc/kconfig-operation
@@ -73,6 +73,11 @@ This allows code to, for example, have "#ifdef CONFIG_PM" which can only
be set or cleared in the kernel, not in the backport configuration. Since
this is needed, a transformation step is done at backport creation time.
+When using Linux backports to integrate into an existing Linux tree
+the CONFIG_BACKPORT_ prefix is used, this allows a CONFIG_BACKPORT_
+symbol to depend on the non-backported respective symbol to be selected
+allowing these to be mutually exclusive.
+
Backport creation for Kconfig
-------------------------------
diff --git a/gentree.py b/gentree.py
index 90334ef..00e2d46 100755
--- a/gentree.py
+++ b/gentree.py
@@ -16,6 +16,7 @@ from lib import bpgpg as gpg
from lib import bpkup as kup
from lib.tempdir import tempdir
from lib import bpreqs as reqs
+from lib import bpversion as gen_version
class Bp_Identity(object):
"""
@@ -233,6 +234,9 @@ def add_automatic_backports(args):
export = re.compile(r'^EXPORT_SYMBOL(_GPL)?\((?P<sym>[^\)]*)\)')
bpi = kconfig.get_backport_info(os.path.join(args.bpid.target_dir, 'compat', 'Kconfig'))
configtree = kconfig.ConfigTree(os.path.join(args.bpid.target_dir, 'Kconfig'), args.bpid)
+ ignore=['Kconfig.kernel', 'Kconfig.versions']
+ configtree.verify_sources(ignore=ignore)
+ git_debug_snapshot(args, "verify sources for automatic backports")
all_selects = configtree.all_selects()
for sym, vals in bpi.items():
if sym.startswith('BPAUTO_BUILD_'):
@@ -640,6 +644,9 @@ def _main():
'and we use git ls-tree to get the files.')
parser.add_argument('--clean', const=True, default=False, action="store_const",
help='Clean output directory instead of erroring if it isn\'t empty')
+ parser.add_argument('--integrate', const=True, default=False, action="store_const",
+ help='Integrate a future backported kernel solution into ' +
+ 'an older kernel tree source directory.')
parser.add_argument('--refresh', const=True, default=False, action="store_const",
help='Refresh patches as they are applied, the source dir will be modified!')
parser.add_argument('--base-name', metavar='<name>', type=str, default='Linux',
@@ -679,9 +686,8 @@ def _main():
# same prefix for packaging as with kernel integration but
# there are already some users of the CPTCFG prefix.
bpid = None
- integrate = False
- if integrate:
- bpid = Bp_Identity(integrate = integrate,
+ if args.integrate:
+ bpid = Bp_Identity(integrate = args.integrate,
kconfig_prefix = 'CONFIG_',
project_prefix = 'BACKPORT_',
project_dir = args.outdir,
@@ -690,7 +696,7 @@ def _main():
kconfig_source_var = '$BACKPORT_DIR',
)
else:
- bpid = Bp_Identity(integrate = integrate,
+ bpid = Bp_Identity(integrate = args.integrate,
kconfig_prefix = 'CPTCFG_',
project_prefix = '',
project_dir = args.outdir,
@@ -764,6 +770,11 @@ def process(kerneldir, copy_list_file, git_revision=None,
test_cocci, profile_cocci)
rel_prep = None
+ if bpid.integrate:
+ if args.kup_test or args.test_cocci or args.profile_cocci or args.refresh:
+ logwrite('Cannot use integration with:\n\tkup_test\n\ttest_cocci\n\tprofile_cocci\n\trefresh\n');
+ sys.exit(1)
+
# start processing ...
if (args.kup or args.kup_test):
git_paranoia(source_dir, logwrite)
@@ -798,6 +809,10 @@ def process(kerneldir, copy_list_file, git_revision=None,
check_output_dir(bpid.target_dir, args.clean)
# do the copy
+ backport_integrate_files = [
+ ('Makefile.kernel', 'Makefile'),
+ ('Kconfig.integrate', 'Kconfig'),
+ ]
backport_package_files = [(x, x) for x in [
'Makefile',
'kconf/',
@@ -819,6 +834,8 @@ def process(kerneldir, copy_list_file, git_revision=None,
if not bpid.integrate:
backport_files += backport_package_files
+ else:
+ backport_files += backport_integrate_files
if not args.git_revision:
logwrite('Copy original source files ...')
@@ -888,6 +905,21 @@ def process(kerneldir, copy_list_file, git_revision=None,
apply_patches(args, "backport", source_dir, 'patches', bpid.target_dir, logwrite)
+ # Kernel integration requires Kconfig.versions already generated for you,
+ # we cannot do this for a package as we have no idea what kernel folks
+ # will be using.
+ if bpid.integrate:
+ kver = gen_version.kernelversion(bpid.project_dir)
+ rel_specs = gen_version.get_rel_spec_stable(kver)
+ if not rel_specs:
+ logwrite('Cannot parse source kernel version, update parser')
+ sys.exit(1)
+ data = gen_version.genkconfig_versions(rel_specs)
+ fo = open(os.path.join(bpid.target_dir, 'Kconfig.versions'), 'w')
+ fo.write(data)
+ fo.close()
+ git_debug_snapshot(args, "generate kernel version requirement Kconfig file")
+
# some post-processing is required
configtree = kconfig.ConfigTree(os.path.join(bpid.target_dir, 'Kconfig'), bpid)
ignore=['Kconfig.kernel', 'Kconfig.versions']
@@ -905,6 +937,15 @@ def process(kerneldir, copy_list_file, git_revision=None,
configtree.force_tristate_modular()
git_debug_snapshot(args, "force tristate options modular")
+ ignore = [os.path.join(bpid.target_dir, x) for x in [
+ 'Kconfig.package.hacks',
+ 'Kconfig.versions',
+ 'Kconfig',
+ ]
+ ]
+ configtree.adjust_backported_configs(ignore=ignore, orig_symbols=orig_symbols)
+ git_debug_snapshot(args, "adjust backports config symbols we port")
+
configtree.modify_selects()
git_debug_snapshot(args, "convert select to depends on")
@@ -944,8 +985,15 @@ def process(kerneldir, copy_list_file, git_revision=None,
# rewrite Makefile and source symbols
+ # symbols we know only we can provide under the backport project prefix
+ # for which we need an exception.
+ skip_orig_syms = [ bpid.project_prefix + x for x in [
+ 'INTEGRATE',
+ ]
+ ]
+ parse_orig_syms = [x for x in orig_symbols if x not in skip_orig_syms ]
regexes = []
- for some_symbols in [orig_symbols[i:i + 50] for i in range(0, len(orig_symbols), 50)]:
+ for some_symbols in [parse_orig_syms[i:i + 50] for i in range(0, len(parse_orig_syms), 50)]:
r = 'CONFIG_((' + '|'.join([s + '(_MODULE)?' for s in some_symbols]) + ')([^A-Za-z0-9_]|$))'
regexes.append(re.compile(r, re.MULTILINE))
for root, dirs, files in os.walk(bpid.target_dir):
@@ -967,7 +1015,10 @@ def process(kerneldir, copy_list_file, git_revision=None,
git_debug_snapshot(args, "rename config symbol / srctree usage")
# disable unbuildable Kconfig symbols and stuff Makefiles that doesn't exist
- maketree = make.MakeTree(os.path.join(bpid.target_dir, 'Makefile.kernel'))
+ if bpid.integrate:
+ maketree = make.MakeTree(os.path.join(bpid.target_dir, 'Makefile'))
+ else:
+ maketree = make.MakeTree(os.path.join(bpid.target_dir, 'Makefile.kernel'))
disable_kconfig = []
disable_makefile = []
for sym in maketree.get_impossible_symbols():
@@ -1017,6 +1068,15 @@ def process(kerneldir, copy_list_file, git_revision=None,
fo.close()
git_debug_snapshot(args, "disable unsatisfied Makefile parts")
+ if bpid.integrate:
+ f = open(os.path.join(bpid.project_dir, 'Kconfig'), 'a')
+ f.write('source "backports/Kconfig"\n')
+ f.close()
+ git_debug_snapshot(args, "hooked backport to top level Kconfig")
+
+ apply_patches(args, "integration", source_dir, 'integration-patches/',
+ bpid.project_dir, logwrite)
+
if (args.kup or args.kup_test):
req = reqs.Req()
req.kup()
diff --git a/integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch b/integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch
new file mode 100644
index 0000000..d66b203
--- /dev/null
+++ b/integration-patches/0001-enable-backports/0001-enable-backports-built-in.patch
@@ -0,0 +1,40 @@
+Allow backports to be integrated into vmlinux.
+
+diff --git a/Makefile b/Makefile
+index 6d1e304..de26b18 100644
+--- a/Makefile
++++ b/Makefile
+@@ -542,6 +542,7 @@ scripts: scripts_basic include/config/auto.conf include/config/tristate.conf \
+ $(Q)$(MAKE) $(build)=$(@)
+
+ # Objects we will link into vmlinux / subdirs we need to visit
++backports-y := backports/
+ init-y := init/
+ drivers-y := drivers/ sound/ firmware/
+ net-y := net/
+@@ -820,13 +821,16 @@ core-y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/
+
+ vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m) \
+ $(core-y) $(core-m) $(drivers-y) $(drivers-m) \
++ $(backports-y) $(backports-m) \
+ $(net-y) $(net-m) $(libs-y) $(libs-m)))
+
+ vmlinux-alldirs := $(sort $(vmlinux-dirs) $(patsubst %/,%,$(filter %/, \
+ $(init-n) $(init-) \
+ $(core-n) $(core-) $(drivers-n) $(drivers-) \
++ $(backports-n) $(backports-) \
+ $(net-n) $(net-) $(libs-n) $(libs-))))
+
++backports-y := $(patsubst %/, %/built-in.o, $(backports-y))
+ init-y := $(patsubst %/, %/built-in.o, $(init-y))
+ core-y := $(patsubst %/, %/built-in.o, $(core-y))
+ drivers-y := $(patsubst %/, %/built-in.o, $(drivers-y))
+@@ -837,7 +841,7 @@ libs-y := $(libs-y1) $(libs-y2)
+
+ # Externally visible symbols (used by link-vmlinux.sh)
+ export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
+-export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y)
++export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y) $(backports-y)
+ export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds
+ export LDFLAGS_vmlinux
+ # used by scripts/pacmage/Makefile
diff --git a/lib/bpversion.py b/lib/bpversion.py
new file mode 100644
index 0000000..aefdcf0
--- /dev/null
+++ b/lib/bpversion.py
@@ -0,0 +1,48 @@
+import subprocess, os, re
+
+class VersionError(Exception):
+ pass
+class ExecutionError(VersionError):
+ def __init__(self, errcode):
+ self.error_code = errcode
+
+def _check(process):
+ if process.returncode != 0:
+ raise ExecutionError(process.returncode)
+
+def get_rel_spec_stable(rel):
+ """
+ Returns release specs for a linux-stable backports based release.
+ """
+ m = None
+ if ("rc" in rel):
+ m = re.match(r"v*(?P<VERSION>\d+)\.+" \
+ "(?P<PATCHLEVEL>\d+)[.]*" \
+ "(?P<SUBLEVEL>\d*)" \
+ "[-rc]+(?P<RC_VERSION>\d+)",
+ rel)
+ else:
+ m = re.match(r"(?P<VERSION>\d+)\.+" \
+ "(?P<PATCHLEVEL>\d+)[.]*" \
+ "(?P<SUBLEVEL>\d*)",
+ rel)
+ if (not m):
+ return m
+ return m.groupdict()
+
+def kernelversion(tree):
+ cmd = ['make', '--no-print-directory', '-C', tree, 'kernelversion' ]
+ process = subprocess.Popen(cmd,
+ stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+ close_fds=True, universal_newlines=True)
+ stdout = process.communicate()[0]
+ process.wait()
+ _check(process)
+ return stdout.strip()
+
+def genkconfig_versions(rel_specs):
+ data = ''
+ for i in range(int(rel_specs['PATCHLEVEL']) + 1, 99):
+ data += "config BACKPORT_KERNEL_%s_%s\n" % (rel_specs['VERSION'], i)
+ data += " def_bool y\n"
+ return data
diff --git a/lib/kconfig.py b/lib/kconfig.py
index ca3af63..ea9c121 100644
--- a/lib/kconfig.py
+++ b/lib/kconfig.py
@@ -137,6 +137,73 @@ class ConfigTree(object):
outf.write(out)
outf.close()
+ def _mod_kconfig_line(self, l, orig_symbols):
+ if self.bpid.project_prefix != '':
+ for sym in orig_symbols:
+ if sym in l:
+ return re.sub(r' (' + sym + ')', r' ' + self.bpid.project_prefix + '\\1', l)
+ return l
+
+ def adjust_backported_configs(self, ignore=[], orig_symbols=[]):
+ m = None
+ old_l = None
+ for nf in self._walk(self.rootfile):
+ if os.path.join(self.bpid.target_dir, nf) in ignore:
+ continue
+ out = ''
+ for l in open(os.path.join(self.bpid.target_dir, nf), 'r'):
+ n = cfg_line.match(l)
+ if n:
+ m = n
+ old_l = l
+ continue
+ # We're now on the second line for the config symbol
+ if m:
+ built_in_sym = m.group('sym')
+ if self.bpid.project_prefix != '':
+ built_in_sym = re.sub(r'' + self.bpid.project_prefix + '(.*)', r'\1', m.group('sym'))
+ # These are things that we carry as part of our backports
+ # module or things we automatically copy over into our
+ # backports module. What we want to do is ensure that we
+ # always negate a backported symbol we provide with the one
+ # provided by the old kernel.
+ #
+ # Note that this means that since project_prefix is empty
+ # it likely means the kconfig getenv() trick was used to
+ # backport and when that's used the same symbol is used,
+ # that means you can't easily negate an internal symbol
+ # as kconfig will always apply the kconfig_prefix.
+ #
+ # XXX: only do this for symbols that have C files
+ # depending on it
+ if self.bpid.project_prefix != '' and self.bpid.project_prefix in m.group('sym'):
+ out += old_l
+ out += l
+ out += "\tdepends on !%s\n" % (built_in_sym)
+ else:
+ # First rewrite the upstream symbol with our prefix if
+ # needed
+ if self.bpid.project_prefix != '':
+ out += m.group('opt') + ' ' + self.bpid.project_prefix + m.group('sym') + '\n'
+ out += l
+ # Packaging uses the checks.h but that's a reactive
+ # measure, this is proactive.
+ if not self.bpid.integrate:
+ # This doesn't happen right now as packaging
+ # always uses an empty project prefix.
+ out += "\tdepends on %s!=y\n" % (built_in_sym)
+ else:
+ out += "\tdepends on !%s\n" % (built_in_sym)
+ else:
+ out += m.group('opt') + ' ' + m.group('sym') + '\n'
+ out += l
+ m = None
+ else:
+ out += self._mod_kconfig_line(l, orig_symbols)
+ outf = open(os.path.join(self.bpid.target_dir, nf), 'w')
+ outf.write(out)
+ outf.close()
+
def symbols(self):
syms = []
for nf in self._walk(self.rootfile):
--
2.1.1
^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH v3 10/21] backports: avoid git tree reinitialization
2014-11-11 8:15 ` [PATCH v3 10/21] backports: avoid git tree reinitialization Luis R. Rodriguez
@ 2014-11-14 20:47 ` Johannes Berg
2014-11-14 20:51 ` Luis R. Rodriguez
0 siblings, 1 reply; 32+ messages in thread
From: Johannes Berg @ 2014-11-14 20:47 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: backports, linux-kernel, yann.morin.1998, mmarek, sassmann,
Luis R. Rodriguez
On Tue, 2014-11-11 at 00:15 -0800, Luis R. Rodriguez wrote:
> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>
> When using backports against for packaging with --gitdebug you either
> have to use --clean or assume the directory is empty already. In either
> case you start fresh. With integration this will be a bit different, you
> could end up with a project directory where the git tree was present but
> only the target directory was empty.
>
> Calling git init on an already existing git directory will reinitialize
> your git tree, that can reset some config stuff, let's avoid that.
I guess I don't care much, but it seems strange to me that you'd use
--gitdebug in this case, you'd end up committing crappy patches to the
top-level kernel tree?
johannes
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 00/21] backports: add kernel integration support
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
` (20 preceding siblings ...)
2014-11-11 8:15 ` [PATCH v3 21/21] backports: add full kernel integration support Luis R. Rodriguez
@ 2014-11-14 20:50 ` Johannes Berg
2014-11-14 20:52 ` Luis R. Rodriguez
21 siblings, 1 reply; 32+ messages in thread
From: Johannes Berg @ 2014-11-14 20:50 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: backports, linux-kernel, yann.morin.1998, mmarek, sassmann,
Luis R. Rodriguez
On Tue, 2014-11-11 at 00:14 -0800, Luis R. Rodriguez wrote:
> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>
> This third series should hopefully address all pending feedback.
> I've split out patches even more, but more importantly have
> made the parser simpler by both integrating the logic required
> to understand a backport by identifying the backport by type
> and strategy, sharing that, and then providing ignore lists on
> the parsers.
I think from a code change POV I should be OK with this, but I'll
definitely want to run it and see the differences. I'll have to do that
next week, it's getting late here.
johannes
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 10/21] backports: avoid git tree reinitialization
2014-11-14 20:47 ` Johannes Berg
@ 2014-11-14 20:51 ` Luis R. Rodriguez
0 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-14 20:51 UTC (permalink / raw)
To: Johannes Berg
Cc: backports, linux-kernel, Yann E. MORIN, Michal Marek, Stefan Assmann
On Fri, Nov 14, 2014 at 12:47 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2014-11-11 at 00:15 -0800, Luis R. Rodriguez wrote:
>> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>>
>> When using backports against for packaging with --gitdebug you either
>> have to use --clean or assume the directory is empty already. In either
>> case you start fresh. With integration this will be a bit different, you
>> could end up with a project directory where the git tree was present but
>> only the target directory was empty.
>>
>> Calling git init on an already existing git directory will reinitialize
>> your git tree, that can reset some config stuff, let's avoid that.
>
> I guess I don't care much, but it seems strange to me that you'd use
> --gitdebug in this case, you'd end up committing crappy patches to the
> top-level kernel tree?
It was very useful to me when debugging, hence --gitdebug. Also, each
step is actually what some folks may want to see done / committed as
that may have been work done manually before for each kernel release
bump, now its just automated.
Luis
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 00/21] backports: add kernel integration support
2014-11-14 20:50 ` [PATCH v3 00/21] backports: add " Johannes Berg
@ 2014-11-14 20:52 ` Luis R. Rodriguez
2014-11-14 20:54 ` Johannes Berg
0 siblings, 1 reply; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-14 20:52 UTC (permalink / raw)
To: Johannes Berg
Cc: backports, linux-kernel, Yann E. MORIN, Michal Marek, Stefan Assmann
On Fri, Nov 14, 2014 at 12:50 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2014-11-11 at 00:14 -0800, Luis R. Rodriguez wrote:
>> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>>
>> This third series should hopefully address all pending feedback.
>> I've split out patches even more, but more importantly have
>> made the parser simpler by both integrating the logic required
>> to understand a backport by identifying the backport by type
>> and strategy, sharing that, and then providing ignore lists on
>> the parsers.
>
> I think from a code change POV I should be OK with this, but I'll
> definitely want to run it and see the differences. I'll have to do that
> next week, it's getting late here.
OK. So I shouldn't merge yet?
Luis
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 00/21] backports: add kernel integration support
2014-11-14 20:52 ` Luis R. Rodriguez
@ 2014-11-14 20:54 ` Johannes Berg
2014-11-14 21:10 ` Luis R. Rodriguez
0 siblings, 1 reply; 32+ messages in thread
From: Johannes Berg @ 2014-11-14 20:54 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: backports, linux-kernel, Yann E. MORIN, Michal Marek, Stefan Assmann
On Fri, 2014-11-14 at 12:52 -0800, Luis R. Rodriguez wrote:
> On Fri, Nov 14, 2014 at 12:50 PM, Johannes Berg
> <johannes@sipsolutions.net> wrote:
> > On Tue, 2014-11-11 at 00:14 -0800, Luis R. Rodriguez wrote:
> >> From: "Luis R. Rodriguez" <mcgrof@suse.com>
> >>
> >> This third series should hopefully address all pending feedback.
> >> I've split out patches even more, but more importantly have
> >> made the parser simpler by both integrating the logic required
> >> to understand a backport by identifying the backport by type
> >> and strategy, sharing that, and then providing ignore lists on
> >> the parsers.
> >
> > I think from a code change POV I should be OK with this, but I'll
> > definitely want to run it and see the differences. I'll have to do that
> > next week, it's getting late here.
>
> OK. So I shouldn't merge yet?
I would prefer you didn't, but OTOH we can always fix up issues later as
well.
johannes
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 00/21] backports: add kernel integration support
2014-11-14 20:54 ` Johannes Berg
@ 2014-11-14 21:10 ` Luis R. Rodriguez
2014-11-15 1:20 ` Luis R. Rodriguez
0 siblings, 1 reply; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-14 21:10 UTC (permalink / raw)
To: Johannes Berg
Cc: backports, linux-kernel, Yann E. MORIN, Michal Marek, Stefan Assmann
On Fri, Nov 14, 2014 at 12:54 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Fri, 2014-11-14 at 12:52 -0800, Luis R. Rodriguez wrote:
>> On Fri, Nov 14, 2014 at 12:50 PM, Johannes Berg
>> <johannes@sipsolutions.net> wrote:
>> > On Tue, 2014-11-11 at 00:14 -0800, Luis R. Rodriguez wrote:
>> >> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>> >>
>> >> This third series should hopefully address all pending feedback.
>> >> I've split out patches even more, but more importantly have
>> >> made the parser simpler by both integrating the logic required
>> >> to understand a backport by identifying the backport by type
>> >> and strategy, sharing that, and then providing ignore lists on
>> >> the parsers.
>> >
>> > I think from a code change POV I should be OK with this, but I'll
>> > definitely want to run it and see the differences. I'll have to do that
>> > next week, it's getting late here.
>>
>> OK. So I shouldn't merge yet?
>
> I would prefer you didn't, but OTOH we can always fix up issues later as
> well.
I'll wait.
Luis
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 00/21] backports: add kernel integration support
2014-11-14 21:10 ` Luis R. Rodriguez
@ 2014-11-15 1:20 ` Luis R. Rodriguez
0 siblings, 0 replies; 32+ messages in thread
From: Luis R. Rodriguez @ 2014-11-15 1:20 UTC (permalink / raw)
To: Johannes Berg
Cc: backports, linux-kernel, Yann E. MORIN, Michal Marek, Stefan Assmann
On Fri, Nov 14, 2014 at 1:10 PM, Luis R. Rodriguez
<mcgrof@do-not-panic.com> wrote:
> On Fri, Nov 14, 2014 at 12:54 PM, Johannes Berg
> <johannes@sipsolutions.net> wrote:
>> On Fri, 2014-11-14 at 12:52 -0800, Luis R. Rodriguez wrote:
>>> On Fri, Nov 14, 2014 at 12:50 PM, Johannes Berg
>>> <johannes@sipsolutions.net> wrote:
>>> > On Tue, 2014-11-11 at 00:14 -0800, Luis R. Rodriguez wrote:
>>> >> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>>> >>
>>> >> This third series should hopefully address all pending feedback.
>>> >> I've split out patches even more, but more importantly have
>>> >> made the parser simpler by both integrating the logic required
>>> >> to understand a backport by identifying the backport by type
>>> >> and strategy, sharing that, and then providing ignore lists on
>>> >> the parsers.
>>> >
>>> > I think from a code change POV I should be OK with this, but I'll
>>> > definitely want to run it and see the differences. I'll have to do that
>>> > next week, it's getting late here.
>>>
>>> OK. So I shouldn't merge yet?
>>
>> I would prefer you didn't, but OTOH we can always fix up issues later as
>> well.
>
> I'll wait.
Pushed.
Luis
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper
2014-11-11 8:14 ` [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper Luis R. Rodriguez
@ 2016-07-04 9:33 ` Johannes Berg
2016-07-07 0:10 ` Luis R. Rodriguez
0 siblings, 1 reply; 32+ messages in thread
From: Johannes Berg @ 2016-07-04 9:33 UTC (permalink / raw)
To: Luis R. Rodriguez, backports
Cc: linux-kernel, yann.morin.1998, mmarek, sassmann, Luis R. Rodriguez
On Tue, 2014-11-11 at 00:14 -0800, Luis R. Rodriguez wrote:
> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>
> This allows us to extend how backports uses patches for
> different types of applications. This will later be used
> for kernel integration support, for example.
>
> This should have no functional change.
Obviously this patch was applied a long time ago, but you lied - it has
a functional change:
> + if process.returncode != 0:
> + if not args.verbose:
> + logwrite("Failed to apply changes from %s" %
> print_name)
> + for line in output:
> + logwrite('> %s' % line)
> + raise Exception('Patch failed')
vs.
> - if process.returncode != 0:
> - if not args.verbose:
> - logwrite("Failed to apply changes from %s" %
> print_name)
> - for line in output:
> - logwrite('> %s' % line)
> - return 2
This had a major impact on the devel/git-tracker.py tool.
johannes
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper
2016-07-04 9:33 ` Johannes Berg
@ 2016-07-07 0:10 ` Luis R. Rodriguez
2016-07-07 4:14 ` Johannes Berg
0 siblings, 1 reply; 32+ messages in thread
From: Luis R. Rodriguez @ 2016-07-07 0:10 UTC (permalink / raw)
To: Johannes Berg
Cc: Luis R. Rodriguez, backports, linux-kernel, yann.morin.1998,
mmarek, sassmann
On Mon, Jul 04, 2016 at 11:33:03AM +0200, Johannes Berg wrote:
> On Tue, 2014-11-11 at 00:14 -0800, Luis R. Rodriguez wrote:
> > From: "Luis R. Rodriguez" <mcgrof@suse.com>
> >
> > This allows us to extend how backports uses patches for
> > different types of applications. This will later be used
> > for kernel integration support, for example.
> >
> > This should have no functional change.
>
> Obviously this patch was applied a long time ago,
Geesh yes over 2 year ago.
> but you lied - it has a functional change:
>
> > + if process.returncode != 0:
> > + if not args.verbose:
> > + logwrite("Failed to apply changes from %s" %
> > print_name)
> > + for line in output:
> > + logwrite('> %s' % line)
> > + raise Exception('Patch failed')
>
> vs.
>
> > - if process.returncode != 0:
> > - if not args.verbose:
> > - logwrite("Failed to apply changes from %s" %
> > print_name)
> > - for line in output:
> > - logwrite('> %s' % line)
> > - return 2
>
> This had a major impact on the devel/git-tracker.py tool.
Sorry about that, is there an easy fix for it? Is there a test
we can do to avoid further regressions against the tracker ?
Luis
^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper
2016-07-07 0:10 ` Luis R. Rodriguez
@ 2016-07-07 4:14 ` Johannes Berg
0 siblings, 0 replies; 32+ messages in thread
From: Johannes Berg @ 2016-07-07 4:14 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Luis R. Rodriguez, backports, linux-kernel, yann.morin.1998,
mmarek, sassmann
On Thu, 2016-07-07 at 02:10 +0200, Luis R. Rodriguez wrote:
> On Mon, Jul 04, 2016 at 11:33:03AM +0200, Johannes Berg wrote:
> > On Tue, 2014-11-11 at 00:14 -0800, Luis R. Rodriguez wrote:
> > > From: "Luis R. Rodriguez" <mcgrof@suse.com>
> > >
> > > This allows us to extend how backports uses patches for
> > > different types of applications. This will later be used
> > > for kernel integration support, for example.
> > >
> > > This should have no functional change.
> >
> > Obviously this patch was applied a long time ago,
>
> Geesh yes over 2 year ago.
:)
> > but you lied - it has a functional change:
> >
> > > + if process.returncode != 0:
> > > + if not args.verbose:
> > > + logwrite("Failed to apply changes from %s" %
> > > print_name)
> > > + for line in output:
> > > + logwrite('> %s' % line)
> > > + raise Exception('Patch failed')
> >
> > vs.
> >
> > > - if process.returncode != 0:
> > > - if not args.verbose:
> > > - logwrite("Failed to apply changes from %s" %
> > > print_name)
> > > - for line in output:
> > > - logwrite('> %s' % line)
> > > - return 2
> >
> > This had a major impact on the devel/git-tracker.py tool.
>
> Sorry about that, is there an easy fix for it? Is there a test
> we can do to avoid further regressions against the tracker ?
>
Luca has a fix.
johannes
^ permalink raw reply [flat|nested] 32+ messages in thread
end of thread, other threads:[~2016-07-07 4:14 UTC | newest]
Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-11-11 8:14 [PATCH v3 00/21] backports: add kernel integration support Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 01/21] backports: move legacy and SmPL patch application into helper Luis R. Rodriguez
2016-07-04 9:33 ` Johannes Berg
2016-07-07 0:10 ` Luis R. Rodriguez
2016-07-07 4:14 ` Johannes Berg
2014-11-11 8:14 ` [PATCH v3 02/21] backports: ifdef around module_init() module_exit() for modules Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 03/21] backports: move packaging kconfig hacks to its own file Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 04/21] backports: grant parsers access to kconfig config and menuconfig Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 05/21] backports: keep track of original backported source symbols early on Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 06/21] backports: allow for different backport prefix and simple integration support Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 07/21] backports: add prefix for integration on dependencies Luis R. Rodriguez
2014-11-11 8:14 ` [PATCH v3 08/21] backports: replace BACKPORT_PWD with BACKPORT_DIR Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 09/21] backports: fold outdir as a backport identity target Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 10/21] backports: avoid git tree reinitialization Luis R. Rodriguez
2014-11-14 20:47 ` Johannes Berg
2014-11-14 20:51 ` Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 11/21] backports: modify kconfig parser to use bpid and target_dir Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 12/21] backports: make kconfig src_line backport / ConfigTree specific Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 13/21] backports: use BACKPORT_DIR prefix on kconfig sources Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 14/21] backports: update dependencies map file Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 15/21] backports: split Kconfig into Kconfig.package and Kconfig.sources Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 16/21] backports: move version file generation to run earlier Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 17/21] backports: define C code backport version info using CPTCFG_ Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 18/21] backports: add backport version parsing for kernel integration Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 19/21] backports: prefix c-file / h-file auto backport with BPAUTO Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 20/21] backports: remove extra BACKPORT_ prefix from kernel versioning Luis R. Rodriguez
2014-11-11 8:15 ` [PATCH v3 21/21] backports: add full kernel integration support Luis R. Rodriguez
2014-11-14 20:50 ` [PATCH v3 00/21] backports: add " Johannes Berg
2014-11-14 20:52 ` Luis R. Rodriguez
2014-11-14 20:54 ` Johannes Berg
2014-11-14 21:10 ` Luis R. Rodriguez
2014-11-15 1:20 ` Luis R. Rodriguez
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).