* [PATCH 0/3] sanity: improve mirrors check
@ 2014-08-25 22:46 Christopher Larson
2014-08-25 22:46 ` [PATCH 1/3] sanity: handle both \n and \n in mirror vars Christopher Larson
` (3 more replies)
0 siblings, 4 replies; 12+ messages in thread
From: Christopher Larson @ 2014-08-25 22:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Christopher Larson
From: Christopher Larson <chris_larson@mentor.com>
The following changes since commit 217aa4d9802609d2c3628c8751a27a5d25900898:
autogen-native: inherit pkgconfig to fix a build failure (2014-08-25 10:25:54 +0100)
are available in the git repository at:
git@github.com:kergoth/openembedded-core improve-sanity-mirrors-check
for you to fetch changes up to 47b5636f4c87979aecfbcb743235a9e010bedd23:
sanity: refactor mirrors checks to be more pythonic (2014-08-25 15:45:19 -0700)
----------------------------------------------------------------
Christopher Larson (3):
sanity: handle both \n and \n in mirror vars
sanity: fix support for regex schemes in mirrors check
sanity: refactor mirrors checks to be more pythonic
meta/classes/sanity.bbclass | 67 ++++++++++++++++++++++++---------------------
1 file changed, 36 insertions(+), 31 deletions(-)
--
1.8.3.4
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH 1/3] sanity: handle both \n and \n in mirror vars 2014-08-25 22:46 [PATCH 0/3] sanity: improve mirrors check Christopher Larson @ 2014-08-25 22:46 ` Christopher Larson 2014-08-25 22:50 ` Christopher Larson 2014-08-25 22:46 ` [PATCH 2/3] sanity: fix support for regex schemes in mirrors check Christopher Larson ` (2 subsequent siblings) 3 siblings, 1 reply; 12+ messages in thread From: Christopher Larson @ 2014-08-25 22:46 UTC (permalink / raw) To: openembedded-core; +Cc: Christopher Larson From: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Christopher Larson <chris_larson@mentor.com> --- meta/classes/sanity.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass index dbcc26b..7cfc4be 100644 --- a/meta/classes/sanity.bbclass +++ b/meta/classes/sanity.bbclass @@ -759,7 +759,7 @@ def check_sanity_everybuild(status, d): 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ 'bzr://', 'cvs://'] for mir_type in mir_types: - mirros = (d.getVar(mir_type, True) or '').split('\\n') + mirros = (d.getVar(mir_type, True) or '').replace('\\n', '\n').split('\n') for mir in mirros: mir_list = mir.split() # Should be two members. -- 1.8.3.4 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH 1/3] sanity: handle both \n and \n in mirror vars 2014-08-25 22:46 ` [PATCH 1/3] sanity: handle both \n and \n in mirror vars Christopher Larson @ 2014-08-25 22:50 ` Christopher Larson 0 siblings, 0 replies; 12+ messages in thread From: Christopher Larson @ 2014-08-25 22:50 UTC (permalink / raw) To: Patches and discussions about the oe-core layer; +Cc: Christopher Larson [-- Attachment #1: Type: text/plain, Size: 258 bytes --] On Mon, Aug 25, 2014 at 3:46 PM, Christopher Larson <kergoth@gmail.com> wrote: > From: Christopher Larson <chris_larson@mentor.com> > > Signed-off-by: Christopher Larson <chris_larson@mentor.com> > Bah, typo in the subject, fixing. :) -Chris [-- Attachment #2: Type: text/html, Size: 786 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 2/3] sanity: fix support for regex schemes in mirrors check 2014-08-25 22:46 [PATCH 0/3] sanity: improve mirrors check Christopher Larson 2014-08-25 22:46 ` [PATCH 1/3] sanity: handle both \n and \n in mirror vars Christopher Larson @ 2014-08-25 22:46 ` Christopher Larson 2014-08-25 22:46 ` [PATCH 3/3] sanity: refactor mirrors checks to be more pythonic Christopher Larson 2014-08-25 22:57 ` [PATCHv2 0/3] sanity: improve mirrors check Christopher Larson 3 siblings, 0 replies; 12+ messages in thread From: Christopher Larson @ 2014-08-25 22:46 UTC (permalink / raw) To: openembedded-core; +Cc: Christopher Larson From: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Christopher Larson <chris_larson@mentor.com> --- meta/classes/sanity.bbclass | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass index 7cfc4be..f655d8b 100644 --- a/meta/classes/sanity.bbclass +++ b/meta/classes/sanity.bbclass @@ -754,8 +754,9 @@ def check_sanity_everybuild(status, d): status.addresult("Error, you have a space in your COREBASE directory path. Please move the installation to a directory which doesn't include a space since autotools doesn't support this.") # Check the format of MIRRORS, PREMIRRORS and SSTATE_MIRRORS + import re mir_types = ['MIRRORS', 'PREMIRRORS', 'SSTATE_MIRRORS'] - protocols = ['http://', 'ftp://', 'file://', 'https://', 'https?$://', \ + protocols = ['http://', 'ftp://', 'file://', 'https://', \ 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ 'bzr://', 'cvs://'] for mir_type in mir_types: @@ -767,12 +768,19 @@ def check_sanity_everybuild(status, d): bb.warn('Invalid %s: %s, should be 2 members, but found %s.' \ % (mir_type, mir.strip(), len(mir_list))) elif len(mir_list) == 2: + decoded = bb.fetch2.decodeurl(mir_list[0]) + try: + pattern_scheme = re.compile(decoded[0]) + except re.error as exc: + bb.warn('Invalid scheme regex (%s) in %s: %s' % (decoded[0], mir_type, mir.strip())) + continue + # Each member should start with protocols valid_protocol_0 = False valid_protocol_1 = False file_absolute = True for protocol in protocols: - if not valid_protocol_0 and mir_list[0].startswith(protocol): + if not valid_protocol_0 and pattern_scheme.match(protocol[:-3]): valid_protocol_0 = True if not valid_protocol_1 and mir_list[1].startswith(protocol): valid_protocol_1 = True -- 1.8.3.4 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 3/3] sanity: refactor mirrors checks to be more pythonic 2014-08-25 22:46 [PATCH 0/3] sanity: improve mirrors check Christopher Larson 2014-08-25 22:46 ` [PATCH 1/3] sanity: handle both \n and \n in mirror vars Christopher Larson 2014-08-25 22:46 ` [PATCH 2/3] sanity: fix support for regex schemes in mirrors check Christopher Larson @ 2014-08-25 22:46 ` Christopher Larson 2014-08-25 22:57 ` [PATCHv2 0/3] sanity: improve mirrors check Christopher Larson 3 siblings, 0 replies; 12+ messages in thread From: Christopher Larson @ 2014-08-25 22:46 UTC (permalink / raw) To: openembedded-core; +Cc: Christopher Larson From: Christopher Larson <chris_larson@mentor.com> - Use clearer variable names - Use variable unpacking to reference elements by name rather than index - Sacrifice a small amount of time (iterate over protocols twice per entry rather than once) for clarity: use readable generator expressions with any() rather than maintaining state. Signed-off-by: Christopher Larson <chris_larson@mentor.com> --- meta/classes/sanity.bbclass | 73 ++++++++++++++++++++++----------------------- 1 file changed, 35 insertions(+), 38 deletions(-) diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass index f655d8b..5be5efb 100644 --- a/meta/classes/sanity.bbclass +++ b/meta/classes/sanity.bbclass @@ -755,44 +755,41 @@ def check_sanity_everybuild(status, d): # Check the format of MIRRORS, PREMIRRORS and SSTATE_MIRRORS import re - mir_types = ['MIRRORS', 'PREMIRRORS', 'SSTATE_MIRRORS'] - protocols = ['http://', 'ftp://', 'file://', 'https://', \ - 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ - 'bzr://', 'cvs://'] - for mir_type in mir_types: - mirros = (d.getVar(mir_type, True) or '').replace('\\n', '\n').split('\n') - for mir in mirros: - mir_list = mir.split() - # Should be two members. - if len(mir_list) not in [0, 2]: - bb.warn('Invalid %s: %s, should be 2 members, but found %s.' \ - % (mir_type, mir.strip(), len(mir_list))) - elif len(mir_list) == 2: - decoded = bb.fetch2.decodeurl(mir_list[0]) - try: - pattern_scheme = re.compile(decoded[0]) - except re.error as exc: - bb.warn('Invalid scheme regex (%s) in %s: %s' % (decoded[0], mir_type, mir.strip())) - continue - - # Each member should start with protocols - valid_protocol_0 = False - valid_protocol_1 = False - file_absolute = True - for protocol in protocols: - if not valid_protocol_0 and pattern_scheme.match(protocol[:-3]): - valid_protocol_0 = True - if not valid_protocol_1 and mir_list[1].startswith(protocol): - valid_protocol_1 = True - # The file:// must be an absolute path. - if protocol == 'file://' and not mir_list[1].startswith('file:///'): - file_absolute = False - if valid_protocol_0 and valid_protocol_1: - break - if not (valid_protocol_0 and valid_protocol_1): - bb.warn('Invalid protocol in %s: %s' % (mir_type, mir.strip())) - if not file_absolute: - bb.warn('Invalid file url in %s: %s, must be absolute path (file:///)' % (mir_type, mir.strip())) + mirror_vars = ['MIRRORS', 'PREMIRRORS', 'SSTATE_MIRRORS'] + protocols = ['http', 'ftp', 'file', 'https', \ + 'git', 'gitsm', 'hg', 'osc', 'p4', 'svk', 'svn', \ + 'bzr', 'cvs'] + for mirror_var in mirror_vars: + mirrors = (d.getVar(mirror_var, True) or '').replace('\\n', '\n').split('\n') + for mirror_entry in mirrors: + mirror_entry = mirror_entry.strip() + if not mirror_entry: + # ignore blank lines + continue + + try: + pattern, mirror = mirror_entry.split() + except ValueError: + bb.warn('Invalid %s: %s, should be 2 members.' % (mirror_var, mirror_entry.strip())) + continue + + decoded = bb.fetch2.decodeurl(pattern) + try: + pattern_scheme = re.compile(decoded[0]) + except re.error as exc: + bb.warn('Invalid scheme regex (%s) in %s; %s' % (pattern, mirror_var, mirror_entry)) + continue + + if not any(pattern_scheme.match(protocol) for protocol in protocols): + bb.warn('Invalid protocol (%s) in %s: %s' % (decoded[0], mirror_var, mirror_entry)) + continue + + if not any(mirror.startswith(protocol + '://') for protocol in protocols): + bb.warn('Invalid protocol in %s: %s' % (mirror_var, mirror_entry)) + continue + + if mirror.startswith('file://') and not mirror.startswith('file:///'): + bb.warn('Invalid file url in %s: %s, must be absolute path (file:///)' % (mirror_var, mirror_entry)) # Check that TMPDIR hasn't changed location since the last time we were run tmpdir = d.getVar('TMPDIR', True) -- 1.8.3.4 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCHv2 0/3] sanity: improve mirrors check 2014-08-25 22:46 [PATCH 0/3] sanity: improve mirrors check Christopher Larson ` (2 preceding siblings ...) 2014-08-25 22:46 ` [PATCH 3/3] sanity: refactor mirrors checks to be more pythonic Christopher Larson @ 2014-08-25 22:57 ` Christopher Larson 2014-08-25 22:57 ` [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars Christopher Larson ` (2 more replies) 3 siblings, 3 replies; 12+ messages in thread From: Christopher Larson @ 2014-08-25 22:57 UTC (permalink / raw) To: openembedded-core; +Cc: Christopher Larson From: Christopher Larson <chris_larson@mentor.com> v2: fixed subject line of first commit to mention \\n The following changes since commit 217aa4d9802609d2c3628c8751a27a5d25900898: autogen-native: inherit pkgconfig to fix a build failure (2014-08-25 10:25:54 +0100) are available in the git repository at: git@github.com:kergoth/openembedded-core improve-sanity-mirrors-check for you to fetch changes up to 8fa6adaaa363a88c888f2afab2f0117b199a54c0: sanity: refactor mirrors checks to be more pythonic (2014-08-25 15:50:53 -0700) ---------------------------------------------------------------- Christopher Larson (3): sanity: handle both \n and \\n in mirror vars sanity: fix support for regex schemes in mirrors check sanity: refactor mirrors checks to be more pythonic meta/classes/sanity.bbclass | 67 ++++++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 31 deletions(-) -- 1.8.3.4 ^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars 2014-08-25 22:57 ` [PATCHv2 0/3] sanity: improve mirrors check Christopher Larson @ 2014-08-25 22:57 ` Christopher Larson 2014-08-26 10:51 ` Martin Jansa 2014-08-27 11:50 ` Koen Kooi 2014-08-25 22:57 ` [PATCHv2 2/3] sanity: fix support for regex schemes in mirrors check Christopher Larson 2014-08-25 22:57 ` [PATCHv2 3/3] sanity: refactor mirrors checks to be more pythonic Christopher Larson 2 siblings, 2 replies; 12+ messages in thread From: Christopher Larson @ 2014-08-25 22:57 UTC (permalink / raw) To: openembedded-core; +Cc: Christopher Larson From: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Christopher Larson <chris_larson@mentor.com> --- meta/classes/sanity.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass index dbcc26b..7cfc4be 100644 --- a/meta/classes/sanity.bbclass +++ b/meta/classes/sanity.bbclass @@ -759,7 +759,7 @@ def check_sanity_everybuild(status, d): 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ 'bzr://', 'cvs://'] for mir_type in mir_types: - mirros = (d.getVar(mir_type, True) or '').split('\\n') + mirros = (d.getVar(mir_type, True) or '').replace('\\n', '\n').split('\n') for mir in mirros: mir_list = mir.split() # Should be two members. -- 1.8.3.4 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars 2014-08-25 22:57 ` [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars Christopher Larson @ 2014-08-26 10:51 ` Martin Jansa 2014-08-26 10:59 ` Martin Jansa 2014-08-27 11:50 ` Koen Kooi 1 sibling, 1 reply; 12+ messages in thread From: Martin Jansa @ 2014-08-26 10:51 UTC (permalink / raw) To: Christopher Larson; +Cc: Christopher Larson, openembedded-core [-- Attachment #1: Type: text/plain, Size: 1301 bytes --] On Mon, Aug 25, 2014 at 03:57:42PM -0700, Christopher Larson wrote: > From: Christopher Larson <chris_larson@mentor.com> > > Signed-off-by: Christopher Larson <chris_larson@mentor.com> > --- > meta/classes/sanity.bbclass | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass > index dbcc26b..7cfc4be 100644 > --- a/meta/classes/sanity.bbclass > +++ b/meta/classes/sanity.bbclass > @@ -759,7 +759,7 @@ def check_sanity_everybuild(status, d): > 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ > 'bzr://', 'cvs://'] > for mir_type in mir_types: > - mirros = (d.getVar(mir_type, True) or '').split('\\n') > + mirros = (d.getVar(mir_type, True) or '').replace('\\n', '\n').split('\n') Would you mind fixing the typo in variable name as well? > for mir in mirros: > mir_list = mir.split() > # Should be two members. > -- > 1.8.3.4 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 188 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars 2014-08-26 10:51 ` Martin Jansa @ 2014-08-26 10:59 ` Martin Jansa 0 siblings, 0 replies; 12+ messages in thread From: Martin Jansa @ 2014-08-26 10:59 UTC (permalink / raw) To: Christopher Larson; +Cc: Christopher Larson, openembedded-core [-- Attachment #1: Type: text/plain, Size: 1554 bytes --] On Tue, Aug 26, 2014 at 12:51:49PM +0200, Martin Jansa wrote: > On Mon, Aug 25, 2014 at 03:57:42PM -0700, Christopher Larson wrote: > > From: Christopher Larson <chris_larson@mentor.com> > > > > Signed-off-by: Christopher Larson <chris_larson@mentor.com> > > --- > > meta/classes/sanity.bbclass | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass > > index dbcc26b..7cfc4be 100644 > > --- a/meta/classes/sanity.bbclass > > +++ b/meta/classes/sanity.bbclass > > @@ -759,7 +759,7 @@ def check_sanity_everybuild(status, d): > > 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ > > 'bzr://', 'cvs://'] > > for mir_type in mir_types: > > - mirros = (d.getVar(mir_type, True) or '').split('\\n') > > + mirros = (d.getVar(mir_type, True) or '').replace('\\n', '\n').split('\n') > > Would you mind fixing the typo in variable name as well? Nevermind, it's refactored in 3/3, sorry for noise. > > for mir in mirros: > > mir_list = mir.split() > > # Should be two members. > > -- > > 1.8.3.4 > > > > -- > > _______________________________________________ > > Openembedded-core mailing list > > Openembedded-core@lists.openembedded.org > > http://lists.openembedded.org/mailman/listinfo/openembedded-core > > -- > Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com -- Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 188 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars 2014-08-25 22:57 ` [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars Christopher Larson 2014-08-26 10:51 ` Martin Jansa @ 2014-08-27 11:50 ` Koen Kooi 1 sibling, 0 replies; 12+ messages in thread From: Koen Kooi @ 2014-08-27 11:50 UTC (permalink / raw) To: Christopher Larson; +Cc: Christopher Larson, openembedded-core Op 26 aug. 2014, om 00:57 heeft Christopher Larson <kergoth@gmail.com> het volgende geschreven: > From: Christopher Larson <chris_larson@mentor.com> > > Signed-off-by: Christopher Larson <chris_larson@mentor.com> Tested-by: Koen Kooi <koen@dominion.thruhere.net> > --- > meta/classes/sanity.bbclass | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass > index dbcc26b..7cfc4be 100644 > --- a/meta/classes/sanity.bbclass > +++ b/meta/classes/sanity.bbclass > @@ -759,7 +759,7 @@ def check_sanity_everybuild(status, d): > 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ > 'bzr://', 'cvs://'] > for mir_type in mir_types: > - mirros = (d.getVar(mir_type, True) or '').split('\\n') > + mirros = (d.getVar(mir_type, True) or '').replace('\\n', '\n').split('\n') > for mir in mirros: > mir_list = mir.split() > # Should be two members. > -- > 1.8.3.4 > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core > ^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCHv2 2/3] sanity: fix support for regex schemes in mirrors check 2014-08-25 22:57 ` [PATCHv2 0/3] sanity: improve mirrors check Christopher Larson 2014-08-25 22:57 ` [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars Christopher Larson @ 2014-08-25 22:57 ` Christopher Larson 2014-08-25 22:57 ` [PATCHv2 3/3] sanity: refactor mirrors checks to be more pythonic Christopher Larson 2 siblings, 0 replies; 12+ messages in thread From: Christopher Larson @ 2014-08-25 22:57 UTC (permalink / raw) To: openembedded-core; +Cc: Christopher Larson From: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Christopher Larson <chris_larson@mentor.com> --- meta/classes/sanity.bbclass | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass index 7cfc4be..f655d8b 100644 --- a/meta/classes/sanity.bbclass +++ b/meta/classes/sanity.bbclass @@ -754,8 +754,9 @@ def check_sanity_everybuild(status, d): status.addresult("Error, you have a space in your COREBASE directory path. Please move the installation to a directory which doesn't include a space since autotools doesn't support this.") # Check the format of MIRRORS, PREMIRRORS and SSTATE_MIRRORS + import re mir_types = ['MIRRORS', 'PREMIRRORS', 'SSTATE_MIRRORS'] - protocols = ['http://', 'ftp://', 'file://', 'https://', 'https?$://', \ + protocols = ['http://', 'ftp://', 'file://', 'https://', \ 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ 'bzr://', 'cvs://'] for mir_type in mir_types: @@ -767,12 +768,19 @@ def check_sanity_everybuild(status, d): bb.warn('Invalid %s: %s, should be 2 members, but found %s.' \ % (mir_type, mir.strip(), len(mir_list))) elif len(mir_list) == 2: + decoded = bb.fetch2.decodeurl(mir_list[0]) + try: + pattern_scheme = re.compile(decoded[0]) + except re.error as exc: + bb.warn('Invalid scheme regex (%s) in %s: %s' % (decoded[0], mir_type, mir.strip())) + continue + # Each member should start with protocols valid_protocol_0 = False valid_protocol_1 = False file_absolute = True for protocol in protocols: - if not valid_protocol_0 and mir_list[0].startswith(protocol): + if not valid_protocol_0 and pattern_scheme.match(protocol[:-3]): valid_protocol_0 = True if not valid_protocol_1 and mir_list[1].startswith(protocol): valid_protocol_1 = True -- 1.8.3.4 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCHv2 3/3] sanity: refactor mirrors checks to be more pythonic 2014-08-25 22:57 ` [PATCHv2 0/3] sanity: improve mirrors check Christopher Larson 2014-08-25 22:57 ` [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars Christopher Larson 2014-08-25 22:57 ` [PATCHv2 2/3] sanity: fix support for regex schemes in mirrors check Christopher Larson @ 2014-08-25 22:57 ` Christopher Larson 2 siblings, 0 replies; 12+ messages in thread From: Christopher Larson @ 2014-08-25 22:57 UTC (permalink / raw) To: openembedded-core; +Cc: Christopher Larson From: Christopher Larson <chris_larson@mentor.com> - Use clearer variable names - Use variable unpacking to reference elements by name rather than index - Sacrifice a small amount of time (iterate over protocols twice per entry rather than once) for clarity: use readable generator expressions with any() rather than maintaining state. Signed-off-by: Christopher Larson <chris_larson@mentor.com> --- meta/classes/sanity.bbclass | 73 ++++++++++++++++++++++----------------------- 1 file changed, 35 insertions(+), 38 deletions(-) diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass index f655d8b..5be5efb 100644 --- a/meta/classes/sanity.bbclass +++ b/meta/classes/sanity.bbclass @@ -755,44 +755,41 @@ def check_sanity_everybuild(status, d): # Check the format of MIRRORS, PREMIRRORS and SSTATE_MIRRORS import re - mir_types = ['MIRRORS', 'PREMIRRORS', 'SSTATE_MIRRORS'] - protocols = ['http://', 'ftp://', 'file://', 'https://', \ - 'git://', 'gitsm://', 'hg://', 'osc://', 'p4://', 'svk://', 'svn://', \ - 'bzr://', 'cvs://'] - for mir_type in mir_types: - mirros = (d.getVar(mir_type, True) or '').replace('\\n', '\n').split('\n') - for mir in mirros: - mir_list = mir.split() - # Should be two members. - if len(mir_list) not in [0, 2]: - bb.warn('Invalid %s: %s, should be 2 members, but found %s.' \ - % (mir_type, mir.strip(), len(mir_list))) - elif len(mir_list) == 2: - decoded = bb.fetch2.decodeurl(mir_list[0]) - try: - pattern_scheme = re.compile(decoded[0]) - except re.error as exc: - bb.warn('Invalid scheme regex (%s) in %s: %s' % (decoded[0], mir_type, mir.strip())) - continue - - # Each member should start with protocols - valid_protocol_0 = False - valid_protocol_1 = False - file_absolute = True - for protocol in protocols: - if not valid_protocol_0 and pattern_scheme.match(protocol[:-3]): - valid_protocol_0 = True - if not valid_protocol_1 and mir_list[1].startswith(protocol): - valid_protocol_1 = True - # The file:// must be an absolute path. - if protocol == 'file://' and not mir_list[1].startswith('file:///'): - file_absolute = False - if valid_protocol_0 and valid_protocol_1: - break - if not (valid_protocol_0 and valid_protocol_1): - bb.warn('Invalid protocol in %s: %s' % (mir_type, mir.strip())) - if not file_absolute: - bb.warn('Invalid file url in %s: %s, must be absolute path (file:///)' % (mir_type, mir.strip())) + mirror_vars = ['MIRRORS', 'PREMIRRORS', 'SSTATE_MIRRORS'] + protocols = ['http', 'ftp', 'file', 'https', \ + 'git', 'gitsm', 'hg', 'osc', 'p4', 'svk', 'svn', \ + 'bzr', 'cvs'] + for mirror_var in mirror_vars: + mirrors = (d.getVar(mirror_var, True) or '').replace('\\n', '\n').split('\n') + for mirror_entry in mirrors: + mirror_entry = mirror_entry.strip() + if not mirror_entry: + # ignore blank lines + continue + + try: + pattern, mirror = mirror_entry.split() + except ValueError: + bb.warn('Invalid %s: %s, should be 2 members.' % (mirror_var, mirror_entry.strip())) + continue + + decoded = bb.fetch2.decodeurl(pattern) + try: + pattern_scheme = re.compile(decoded[0]) + except re.error as exc: + bb.warn('Invalid scheme regex (%s) in %s; %s' % (pattern, mirror_var, mirror_entry)) + continue + + if not any(pattern_scheme.match(protocol) for protocol in protocols): + bb.warn('Invalid protocol (%s) in %s: %s' % (decoded[0], mirror_var, mirror_entry)) + continue + + if not any(mirror.startswith(protocol + '://') for protocol in protocols): + bb.warn('Invalid protocol in %s: %s' % (mirror_var, mirror_entry)) + continue + + if mirror.startswith('file://') and not mirror.startswith('file:///'): + bb.warn('Invalid file url in %s: %s, must be absolute path (file:///)' % (mirror_var, mirror_entry)) # Check that TMPDIR hasn't changed location since the last time we were run tmpdir = d.getVar('TMPDIR', True) -- 1.8.3.4 ^ permalink raw reply related [flat|nested] 12+ messages in thread
end of thread, other threads:[~2014-08-27 11:50 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-08-25 22:46 [PATCH 0/3] sanity: improve mirrors check Christopher Larson 2014-08-25 22:46 ` [PATCH 1/3] sanity: handle both \n and \n in mirror vars Christopher Larson 2014-08-25 22:50 ` Christopher Larson 2014-08-25 22:46 ` [PATCH 2/3] sanity: fix support for regex schemes in mirrors check Christopher Larson 2014-08-25 22:46 ` [PATCH 3/3] sanity: refactor mirrors checks to be more pythonic Christopher Larson 2014-08-25 22:57 ` [PATCHv2 0/3] sanity: improve mirrors check Christopher Larson 2014-08-25 22:57 ` [PATCHv2 1/3] sanity: handle both \n and \\n in mirror vars Christopher Larson 2014-08-26 10:51 ` Martin Jansa 2014-08-26 10:59 ` Martin Jansa 2014-08-27 11:50 ` Koen Kooi 2014-08-25 22:57 ` [PATCHv2 2/3] sanity: fix support for regex schemes in mirrors check Christopher Larson 2014-08-25 22:57 ` [PATCHv2 3/3] sanity: refactor mirrors checks to be more pythonic Christopher Larson
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.