oddity: file_5.37 printing unexpected "octet-stream" mime type for ISO

oddity: file_5.37 printing unexpected "octet-stream" mime type for ISO

[Robert P. J. Day]

[-- Attachment #1: Type: text/plain, Size: 2587 bytes --]


  (i asked about this on the OE list a couple days ago, but i have
more info and wanted to get a bit wider coverage as i really, really
want to de-mystify this issue.)

  on current project, upgrading from wind river WRL9 to LTS19
(effectively morty to zeus) involves going from file_5.28 to file_5.37
recipe, which is used by various install and upgrade scripts to
validate something is an actual ISO image by checking its mime type
(not the way i would have done it, but it's done).

  in WRL9, the mime type of an ISO image as printed by "file -i"
included the string "application/x-iso9660-image", and that's what
these scripts use to "verify" ISOness. so far, so good.

  suddenly, in LTS19, the very same "file -i" invocation instead
prints "application/octet-stream" -- this very same weirdness was
noticed elsewhere, like here in ubuntu launchpad:

  https://bugs.launchpad.net/ubuntu/+source/file/+bug/1763570

that bug report describes *precisely* the behaviour we're seeing.

  it's hard to believe the "file" command could have been that broken
so i looked around for another explanation, and noticed that any
single file could match multiple mime types, and that the file command
had a "--keep-going" option, so my colleague tried that and, sure
enough:

#   file -i --keep-going /var/tmp/ptx.iso
  /var/tmp/ptx.iso:
  application/x-iso9660-imageapplication/x-dosexec\012-
  application/octet-stream; charset=binary

so now we see both mime types being displayed, but it's not feasible
to change all the scripts to add the "--keep-going" option so i'm
baffled as to why the newer file command suddenly decides to print
"octet-stream" as the mime type rather than the more precise
"x-iso9669-image".

  more puzzlingly, the man page for the command states:

     -k, --keep-going
             Don't stop at the first match, keep going.  Subsequent
             matches will be have the string ‘\012- ’ prepended.  (If
             you want a newline, see the -r option.)  The magic pat‐
             tern with the highest strength (see the -l option) comes
             first.

hang on ... if the pattern with the highest strength is allegedly
printed first, in the above, that is "x-iso9660-iamge", so why would
that not be the single mime type displayed by a normal invocation?

  has anyone else seen this? can it be reproduced? i'll try the latest
version of file later this weekend, but i'm open to assistance since
this weirdness is currently breaking all sorts of install and upgrade
procedures.

rday

Re: [yocto] oddity: file_5.37 printing unexpected "octet-stream" mime type for ISO

[Ross Burton]

You might have better luck talking to the file maintainers.

Ross

On Sat, 30 Jan 2021 at 11:27, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
>
>
>   (i asked about this on the OE list a couple days ago, but i have
> more info and wanted to get a bit wider coverage as i really, really
> want to de-mystify this issue.)
>
>   on current project, upgrading from wind river WRL9 to LTS19
> (effectively morty to zeus) involves going from file_5.28 to file_5.37
> recipe, which is used by various install and upgrade scripts to
> validate something is an actual ISO image by checking its mime type
> (not the way i would have done it, but it's done).
>
>   in WRL9, the mime type of an ISO image as printed by "file -i"
> included the string "application/x-iso9660-image", and that's what
> these scripts use to "verify" ISOness. so far, so good.
>
>   suddenly, in LTS19, the very same "file -i" invocation instead
> prints "application/octet-stream" -- this very same weirdness was
> noticed elsewhere, like here in ubuntu launchpad:
>
>   https://bugs.launchpad.net/ubuntu/+source/file/+bug/1763570
>
> that bug report describes *precisely* the behaviour we're seeing.
>
>   it's hard to believe the "file" command could have been that broken
> so i looked around for another explanation, and noticed that any
> single file could match multiple mime types, and that the file command
> had a "--keep-going" option, so my colleague tried that and, sure
> enough:
>
> #   file -i --keep-going /var/tmp/ptx.iso
>   /var/tmp/ptx.iso:
>   application/x-iso9660-imageapplication/x-dosexec\012-
>   application/octet-stream; charset=binary
>
> so now we see both mime types being displayed, but it's not feasible
> to change all the scripts to add the "--keep-going" option so i'm
> baffled as to why the newer file command suddenly decides to print
> "octet-stream" as the mime type rather than the more precise
> "x-iso9669-image".
>
>   more puzzlingly, the man page for the command states:
>
>      -k, --keep-going
>              Don't stop at the first match, keep going.  Subsequent
>              matches will be have the string ‘\012- ’ prepended.  (If
>              you want a newline, see the -r option.)  The magic pat‐
>              tern with the highest strength (see the -l option) comes
>              first.
>
> hang on ... if the pattern with the highest strength is allegedly
> printed first, in the above, that is "x-iso9660-iamge", so why would
> that not be the single mime type displayed by a normal invocation?
>
>   has anyone else seen this? can it be reproduced? i'll try the latest
> version of file later this weekend, but i'm open to assistance since
> this weirdness is currently breaking all sorts of install and upgrade
> procedures.
>
> rday
>
> 
>