[OE-core][PATCH] oe-pkgdata-util: ignore SIGPIPE

[OE-core][PATCH] oe-pkgdata-util: ignore SIGPIPE

[Chen Qi]

oe-pkgdata-util sometimes outputs a large amount of data. When used
with pipe, it's likely to get the following error.

  BrokenPipeError: [Errno 32] Broken pipe

The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'.
Type 'q' after running the above command, and we get the error.

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 scripts/oe-pkgdata-util | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
index 75dd23efa3..4aeb28879d 100755
--- a/scripts/oe-pkgdata-util
+++ b/scripts/oe-pkgdata-util
@@ -17,6 +17,7 @@ import re
 import argparse
 import logging
 from collections import defaultdict, OrderedDict
+from signal import signal, SIGPIPE, SIG_DFL
 
 scripts_path = os.path.dirname(os.path.realpath(__file__))
 lib_path = scripts_path + '/lib'
@@ -615,6 +616,10 @@ def main():
         logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
         sys.exit(1)
 
+    # It's possible that this program will output large contents, and when used with a pipe in command line,
+    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
+    signal(SIGPIPE, SIG_DFL)
+
     ret = args.func(args)
 
     return ret
-- 
2.17.1

Re: [OE-core][PATCH] oe-pkgdata-util: ignore SIGPIPE

[Richard Purdie]

On Tue, 2021-04-20 at 16:37 +0800, Chen Qi wrote:
> oe-pkgdata-util sometimes outputs a large amount of data. When used
> with pipe, it's likely to get the following error.
> 
>   BrokenPipeError: [Errno 32] Broken pipe
> 
> The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'.
> Type 'q' after running the above command, and we get the error.
> 
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>  scripts/oe-pkgdata-util | 5 +++++
>  1 file changed, 5 insertions(+)

This is far from clearcut. 

https://mail.python.org/pipermail/python-dev/2018-June/154191.html

Not quite sure what to do about this as there are pros/cons...

Cheers,

Richard

> diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
> index 75dd23efa3..4aeb28879d 100755
> --- a/scripts/oe-pkgdata-util
> +++ b/scripts/oe-pkgdata-util
> @@ -17,6 +17,7 @@ import re
>  import argparse
>  import logging
>  from collections import defaultdict, OrderedDict
> +from signal import signal, SIGPIPE, SIG_DFL
>  
> 
> 
> 
>  scripts_path = os.path.dirname(os.path.realpath(__file__))
>  lib_path = scripts_path + '/lib'
> @@ -615,6 +616,10 @@ def main():
>          logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
>          sys.exit(1)
>  
> 
> 
> 
> +    # It's possible that this program will output large contents, and when used with a pipe in command line,
> +    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
> +    signal(SIGPIPE, SIG_DFL)
> +
>      ret = args.func(args)
>  
> 
> 
> 
>      return ret
> 
> 

Re: [OE-core][PATCH] oe-pkgdata-util: ignore SIGPIPE

[Chen Qi]

On 04/30/2021 09:34 PM, Richard Purdie wrote:
> On Tue, 2021-04-20 at 16:37 +0800, Chen Qi wrote:
>> oe-pkgdata-util sometimes outputs a large amount of data. When used
>> with pipe, it's likely to get the following error.
>>
>>    BrokenPipeError: [Errno 32] Broken pipe
>>
>> The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'.
>> Type 'q' after running the above command, and we get the error.
>>
>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>> ---
>>   scripts/oe-pkgdata-util | 5 +++++
>>   1 file changed, 5 insertions(+)
> This is far from clearcut.
>
> https://mail.python.org/pipermail/python-dev/2018-June/154191.html
>
> Not quite sure what to do about this as there are pros/cons...
>
> Cheers,
>
> Richard

Thanks Richard. I've read the material above and grabbed more knowledge 
on this topic.

At first, I also wanted to use the 'try ... except' clause to handle the 
output of oe-pkgdata-utils. But there are multiple places for the 
output, so I figured it's ugly and not necessary.

People mentioned situations where setting SIG_DFL for SIGPIPE is 
harmful. That's true. But in respect to this oe-pkgutils-data program, 
when setting SIG_DFL for SIGPIPE, what would be the actual problem in 
practice?

So I think https://github.com/python/cpython/pull/6773 is meaningful, 
but it does not apply to all programs.

Anyway, if you think using the 'try...except' clause is better, I could 
do it and send out V2. But I'm still wondering what extra benefit it 
could bring us.

Best Regards,
Chen Qi

>> diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
>> index 75dd23efa3..4aeb28879d 100755
>> --- a/scripts/oe-pkgdata-util
>> +++ b/scripts/oe-pkgdata-util
>> @@ -17,6 +17,7 @@ import re
>>   import argparse
>>   import logging
>>   from collections import defaultdict, OrderedDict
>> +from signal import signal, SIGPIPE, SIG_DFL
>>   
>>
>>
>>
>>   scripts_path = os.path.dirname(os.path.realpath(__file__))
>>   lib_path = scripts_path + '/lib'
>> @@ -615,6 +616,10 @@ def main():
>>           logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
>>           sys.exit(1)
>>   
>>
>>
>>
>> +    # It's possible that this program will output large contents, and when used with a pipe in command line,
>> +    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
>> +    signal(SIGPIPE, SIG_DFL)
>> +
>>       ret = args.func(args)
>>   
>>
>>
>>
>>       return ret
>> 
>>
>
>

Re: [OE-core][PATCH] oe-pkgdata-util: ignore SIGPIPE

[Chen Qi]

On 04/30/2021 09:34 PM, Richard Purdie wrote:
> On Tue, 2021-04-20 at 16:37 +0800, Chen Qi wrote:
>> oe-pkgdata-util sometimes outputs a large amount of data. When used
>> with pipe, it's likely to get the following error.
>>
>>    BrokenPipeError: [Errno 32] Broken pipe
>>
>> The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'.
>> Type 'q' after running the above command, and we get the error.
>>
>> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>> ---
>>   scripts/oe-pkgdata-util | 5 +++++
>>   1 file changed, 5 insertions(+)
> This is far from clearcut.
>
> https://mail.python.org/pipermail/python-dev/2018-June/154191.html
>
> Not quite sure what to do about this as there are pros/cons...
>
> Cheers,
>
> Richard

Thanks Richard. I've read the material above and grabbed more knowledge 
on this topic.

At first, I also wanted to use the 'try ... except' clause to handle the 
output of oe-pkgdata-utils. But there are multiple places for the 
output, so I figured it's ugly and not necessary.

People mentioned situations where setting SIG_DFL for SIGPIPE is 
harmful. That's true. But in respect to this oe-pkgutils-data program, 
when setting SIG_DFL for SIGPIPE, what would be the actual problem in 
practice?

So I think https://github.com/python/cpython/pull/6773 is meaningful, 
but it does not apply to all programs.

Anyway, if you think using the 'try...except' clause is better, I could 
do it and send out V2. But I'm still wondering what extra benefit it 
could bring us.

Best Regards,
Chen Qi

>> diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
>> index 75dd23efa3..4aeb28879d 100755
>> --- a/scripts/oe-pkgdata-util
>> +++ b/scripts/oe-pkgdata-util
>> @@ -17,6 +17,7 @@ import re
>>   import argparse
>>   import logging
>>   from collections import defaultdict, OrderedDict
>> +from signal import signal, SIGPIPE, SIG_DFL
>>   
>>
>>
>>
>>   scripts_path = os.path.dirname(os.path.realpath(__file__))
>>   lib_path = scripts_path + '/lib'
>> @@ -615,6 +616,10 @@ def main():
>>           logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
>>           sys.exit(1)
>>   
>>
>>
>>
>> +    # It's possible that this program will output large contents, and when used with a pipe in command line,
>> +    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
>> +    signal(SIGPIPE, SIG_DFL)
>> +
>>       ret = args.func(args)
>>   
>>
>>
>>
>>       return ret
>> 
>>
>
>

Re: [OE-core][PATCH] oe-pkgdata-util: ignore SIGPIPE

[Chen Qi]

[-- Attachment #1: Type: text/plain, Size: 1446 bytes --]

ping

On 04/20/2021 04:37 PM, Chen Qi wrote:
> oe-pkgdata-util sometimes outputs a large amount of data. When used
> with pipe, it's likely to get the following error.
>
>    BrokenPipeError: [Errno 32] Broken pipe
>
> The problem could be reproduced by running `oe-pkgdata-util list-pkg | less'.
> Type 'q' after running the above command, and we get the error.
>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> ---
>   scripts/oe-pkgdata-util | 5 +++++
>   1 file changed, 5 insertions(+)
>
> diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
> index 75dd23efa3..4aeb28879d 100755
> --- a/scripts/oe-pkgdata-util
> +++ b/scripts/oe-pkgdata-util
> @@ -17,6 +17,7 @@ import re
>   import argparse
>   import logging
>   from collections import defaultdict, OrderedDict
> +from signal import signal, SIGPIPE, SIG_DFL
>   
>   scripts_path = os.path.dirname(os.path.realpath(__file__))
>   lib_path = scripts_path + '/lib'
> @@ -615,6 +616,10 @@ def main():
>           logger.error('Unable to find pkgdata directory %s' % args.pkgdata_dir)
>           sys.exit(1)
>   
> +    # It's possible that this program will output large contents, and when used with a pipe in command line,
> +    # we will get a 'BrokenPipeError: [Errno 32] Broken pipe'. Ignore the SIGPIPE to avoid such error.
> +    signal(SIGPIPE, SIG_DFL)
> +
>       ret = args.func(args)
>   
>       return ret
>
>
> 
>


[-- Attachment #2: Type: text/html, Size: 1924 bytes --]