win create failing at mcopy #poky #raspberrypi

win create failing at mcopy #poky #raspberrypi

[proclivis]

[-- Attachment #1: Type: text/plain, Size: 5310 bytes --]

I am getting a Codepage 850 error from mcopy when using meta-raspberrypi with the openbmc layers.

What is strange is I had no problems for months, set aside my project, came back and had the failure. The only thing that can be different is my command line actions, or the code pulled from repositories has changed, which in my understanding can't happen.

Before I give details, note that if I use /usr/bin/mtools on the vfat, it succeeds. It is only the mtools from the build that are a problem. Therefore, I concluded that the vfat itself is ok, and the problem is with the tools. Also note I tried older/newer versions of mtools, going back to .18, which is what the base linux OS has and works. But the results are the same. From this I conclude it is not a bug in mtools per se.

There is another thread on this with the context of two docker images, and the result was some missing mtools files. However, I don't have the knowledge of poky and mtools and could not use that hint to actually solve the problem. So if there is something missing in a layer, conf, etc, I need somewhat complete pointers, examples, or learning resources to succeed. Just saying a file is missing is not going to get me far, because I am not a yocto expert yet.

I will break down the info into some categories, and hopefully someone can point me in the right direction:

* SOURCE
* ENVIRONMENT
* CONF
* ERROR
* MANUAL WIC CREATE RESULTS

-- DETAILS --

SOURCE

openbmc tag 2.8 or 2.9
Adding some layers for device tree and driver patches, but nothing that effects the image

ENVIORNMENT

Ubuntu 14, 16, 18, 20, all have the same results
I use these commands to set the environment and build:

* scripts/install-buildtools
* . poky/buildtools/environment-setup-x86_64-pokysdk-linux
* . oe-init-build-env
* bitbake obmc-phosphor-image

Note that if I build core-image-base I don't have problems. It is in the context of openbmc layers that I have problems. The question may have to be asked on the obmc list, but if so, perhaps someone can help me frame the post to best advantage.

CONF

local.conf has these pertinent settings:

require conf/machine/include/obmc-bsp-common.inc

IMAGE_FSTYPES = "tar.bz2 ext3 wic.bz2 wic.bmap"

The second line prevented obmc image layers from making images incorrectly, as images are normally for ASPEED, not Raspberry Pi. And this worked for months and has not been changed.

ERROR

output: Error converting to codepage 850 Invalid argument

| Cannot initialize '::'

| Bad target ::/

MANUAL WIC CREATE RESULTS

I ran wic create manually and get the same result:

wic create sdimage-raspberrypi -e obmc-phosphor-image

INFO: Building wic-tools...

Loading cache: 100% |###################################################################################################################| Time: 0:00:01

Loaded 3376 entries from dependency cache.

Parsing recipes: 100% |#################################################################################################################| Time: 0:00:04

Parsing of 2286 .bb files complete (2285 cached, 1 parsed). 3377 targets, 352 skipped, 0 masked, 0 errors.

NOTE: Resolving any missing task queue dependencies

Build Configuration:

BB_VERSION           = "1.46.0"

BUILD_SYS            = "x86_64-linux"

NATIVELSBSTRING      = "ubuntu-18.04"

TARGET_SYS           = "arm-openbmc-linux-gnueabi"

MACHINE              = "raspberrypi4"

DISTRO               = "openbmc-phosphor"

DISTRO_VERSION       = "0.1.0"

TUNE_FEATURES        = "arm vfp cortexa7 neon vfpv4 thumb callconvention-hard"

TARGET_FPU           = "hard"

meta

meta-poky

meta-oe

meta-networking

meta-python

meta-webserver

meta-raspberrypi     = "HEAD:35a774200999ac2fca48693c1c169bf99d2f63ea"

meta-pmbus-raspberrypi = "master:ab92ff0828a0278494e5deeb5ea1af2d4dcec2d4"

meta-phosphor        = "HEAD:35a774200999ac2fca48693c1c169bf99d2f63ea"

meta-pmbus-phosphor  = "master:6f2f8a4f8dcdad057f8ff9d81583f277a3aea0f2"

Initialising tasks: 100% |##############################################################################################################| Time: 0:00:02

Sstate summary: Wanted 4 Found 0 Missed 4 Current 52 (0% match, 92% complete)

NOTE: Executing Tasks

NOTE: Tasks Summary: Attempted 437 tasks of which 416 didn't need to be rerun and all succeeded.

INFO: Creating image(s)...

WARNING: bootloader config not specified, using defaults

ERROR: _exec_cmd: export PATH=/home/openbmc/share/design/code/openbmc/build/tmp/work/cortexa7t2hf-neon-vfpv4-openbmc-linux-gnueabi/wic-tools/1.0-r0/recipe-sysroot-native/sbin:/home/openbmc/share/design/code/openbmc/build/tmp/work/cortexa7t2hf-neon-vfpv4-openbmc-linux-gnueabi/wic-tools/1.0-r0/recipe-sysroot-native/usr/sbin:/home/openbmc/share/design/code/openbmc/build/tmp/work/cortexa7t2hf-neon-vfpv4-openbmc-linux-gnueabi/wic-tools/1.0-r0/recipe-sysroot-native/usr/bin:$PATH;mcopy -i ./tmp.wic.yrj_5bbq/rootfs_boot.1.vfat -s ./tmp.wic.yrj_5bbq/boot.1/* ::/ returned '1' instead of 0

output: Error converting to codepage 850 Invalid argument

Cannot initialize '::'

Bad target ::/

[-- Attachment #2: Type: text/html, Size: 14395 bytes --]

Re: [poky] win create failing at mcopy #poky #raspberrypi

[proclivis]

[-- Attachment #1: Type: text/plain, Size: 6699 bytes --]

I have a little more info on this and a more specific question.

I copied the mtools code and hooked it into the layer so I could put in debug statements.

The problem occurs in charsetConv.c in the function cp_open, when it tries to do an iconv_open using “CP850”, which fails.

I ran “iconv —list” to see what it can convert, and CP850 is not listed.

To prove this is the problem, I hardcoded the iconv_open with “ASCII” and bit bake can go all the way through and “wic create” works properly.

So I ran iconv —version and saw it was “GNU libc 2.31” so I poked around those layers.

Best I can guess is this line from the mtools layer should tell the libc to provide CP850 via this line:

RDEPENDS_${PN}_libc-glibc = "glibc-gconv-ibm850”

Except that that is for runtime, and this is build time.

So I am not sure how glibc decides what converters are used. Is it just a set of defaults? Is there a layer somewhere defining them I could not find?

It seems that somewhere along the way, mtools changed its default to use 850, and it is not compatible anymore.

The solution is probably to tell glibc to provide CP850.

Does anyone know how to do that?

Mike

> On May 3, 2021, at 11:37 AM, proclivis via lists.yoctoproject.org <proclivis=gmail.com@lists.yoctoproject.org> wrote:
> 
> I am getting a Codepage 850 error from mcopy when using meta-raspberrypi with the openbmc layers.
> 
> What is strange is I had no problems for months, set aside my project, came back and had the failure. The only thing that can be different is my command line actions, or the code pulled from repositories has changed, which in my understanding can't happen.
> 
> Before I give details, note that if I use /usr/bin/mtools on the vfat, it succeeds. It is only the mtools from the build that are a problem. Therefore, I concluded that the vfat itself is ok, and the problem is with the tools. Also note I tried older/newer versions of mtools, going back to .18, which is what the base linux OS has and works. But the results are the same. From this I conclude it is not a bug in mtools per se.
> 
> There is another thread on this with the context of two docker images, and the result was some missing mtools files. However, I don't have the knowledge of poky and mtools and could not use that hint to actually solve the problem. So if there is something missing in a layer, conf, etc, I need somewhat complete pointers, examples, or learning resources to succeed. Just saying a file is missing is not going to get me far, because I am not a yocto expert yet.
> 
> I will break down the info into some categories, and hopefully someone can point me in the right direction:
> 
> SOURCE
> ENVIRONMENT
> CONF
> ERROR
> MANUAL WIC CREATE RESULTS
> 
> -- DETAILS --
> 
> SOURCE
> 
> openbmc tag 2.8 or 2.9
> Adding some layers for device tree and driver patches, but nothing that effects the image
> 
> ENVIORNMENT
> 
> Ubuntu 14, 16, 18, 20, all have the same results
> I use these commands to set the environment and build:
> 
> scripts/install-buildtools
> . poky/buildtools/environment-setup-x86_64-pokysdk-linux
> . oe-init-build-env
> bitbake obmc-phosphor-image
> 
> Note that if I build core-image-base I don't have problems. It is in the context of openbmc layers that I have problems. The question may have to be asked on the obmc list, but if so, perhaps someone can help me frame the post to best advantage.
> 
> CONF
> 
> local.conf has these pertinent settings:
> 
> require conf/machine/include/obmc-bsp-common.inc
> IMAGE_FSTYPES = "tar.bz2 ext3 wic.bz2 wic.bmap"
> 
> The second line prevented obmc image layers from making images incorrectly, as images are normally for ASPEED, not Raspberry Pi. And this worked for months and has not been changed.
> 
> ERROR
> 
>  output: Error converting to codepage 850 Invalid argument
> | Cannot initialize '::'
> | Bad target ::/
> 
> MANUAL WIC CREATE RESULTS
> 
> I ran wic create manually and get the same result:
> 
> wic create sdimage-raspberrypi -e obmc-phosphor-image
> INFO: Building wic-tools...
>
> Loading cache: 100% |###################################################################################################################| Time: 0:00:01
> Loaded 3376 entries from dependency cache.
> Parsing recipes: 100% |#################################################################################################################| Time: 0:00:04
> Parsing of 2286 .bb files complete (2285 cached, 1 parsed). 3377 targets, 352 skipped, 0 masked, 0 errors.
> NOTE: Resolving any missing task queue dependencies
>
> Build Configuration:
> BB_VERSION           = "1.46.0"
> BUILD_SYS            = "x86_64-linux"
> NATIVELSBSTRING      = "ubuntu-18.04"
> TARGET_SYS           = "arm-openbmc-linux-gnueabi"
> MACHINE              = "raspberrypi4"
> DISTRO               = "openbmc-phosphor"
> DISTRO_VERSION       = "0.1.0"
> TUNE_FEATURES        = "arm vfp cortexa7 neon vfpv4 thumb callconvention-hard"
> TARGET_FPU           = "hard"
> meta                 
> meta-poky
> meta-oe
> meta-networking
> meta-python
> meta-webserver       
> meta-raspberrypi     = "HEAD:35a774200999ac2fca48693c1c169bf99d2f63ea"
> meta-pmbus-raspberrypi = "master:ab92ff0828a0278494e5deeb5ea1af2d4dcec2d4"
> meta-phosphor        = "HEAD:35a774200999ac2fca48693c1c169bf99d2f63ea"
> meta-pmbus-phosphor  = "master:6f2f8a4f8dcdad057f8ff9d81583f277a3aea0f2"
>
> Initialising tasks: 100% |##############################################################################################################| Time: 0:00:02
> Sstate summary: Wanted 4 Found 0 Missed 4 Current 52 (0% match, 92% complete)
> NOTE: Executing Tasks
> NOTE: Tasks Summary: Attempted 437 tasks of which 416 didn't need to be rerun and all succeeded.
> INFO: Creating image(s)...
>
> WARNING: bootloader config not specified, using defaults
>
>
> ERROR: _exec_cmd: export PATH=/home/openbmc/share/design/code/openbmc/build/tmp/work/cortexa7t2hf-neon-vfpv4-openbmc-linux-gnueabi/wic-tools/1.0-r0/recipe-sysroot-native/sbin:/home/openbmc/share/design/code/openbmc/build/tmp/work/cortexa7t2hf-neon-vfpv4-openbmc-linux-gnueabi/wic-tools/1.0-r0/recipe-sysroot-native/usr/sbin:/home/openbmc/share/design/code/openbmc/build/tmp/work/cortexa7t2hf-neon-vfpv4-openbmc-linux-gnueabi/wic-tools/1.0-r0/recipe-sysroot-native/usr/bin:$PATH;mcopy -i ./tmp.wic.yrj_5bbq/rootfs_boot.1.vfat -s ./tmp.wic.yrj_5bbq/boot.1/* ::/ returned '1' instead of 0
> output: Error converting to codepage 850 Invalid argument
> Cannot initialize '::'
> Bad target ::/
> 
> 
> 
> 


[-- Attachment #2: Type: text/html, Size: 15876 bytes --]

Re: [poky] win create failing at mcopy #poky #raspberrypi

[Richard Purdie]

On Mon, 2021-05-10 at 18:54 -0600, Mike Jones wrote:
> I have a little more info on this and a more specific question.
> 
> I copied the mtools code and hooked it into the layer so I could put in debug statements.
> 
> The problem occurs in charsetConv.c in the function cp_open, when it tries to do an iconv_open using
> “CP850”, which fails.
> 
> I ran “iconv —list” to see what it can convert, and CP850 is not listed.
> 
> To prove this is the problem, I hardcoded the iconv_open with “ASCII” and bit bake can go all the way
> through and “wic create” works properly.
> 
> So I ran iconv —version and saw it was “GNU libc 2.31” so I poked around those layers.
> 
> Best I can guess is this line from the mtools layer should tell the libc to provide CP850 via this line:
> 
> RDEPENDS_${PN}_libc-glibc = "glibc-gconv-ibm850”
> 
> Except that that is for runtime, and this is build time.
> 
> So I am not sure how glibc decides what converters are used. Is it just a set of defaults? Is there a layer
> somewhere defining them I could not find?
> 
> It seems that somewhere along the way, mtools changed its default to use 850, and it is not compatible
> anymore.
> 
> The solution is probably to tell glibc to provide CP850.
> 
> Does anyone know how to do that?

Which version of Yocto Project are you using? Are you using buildtools-tarball 
or uninative in your build?

I suspect you need a newer buildtools-tarball version which includes CP850.

Cheers,

Richard

Re: [poky] win create failing at mcopy #poky #raspberrypi

[Mike Jones]

> On May 11, 2021, at 4:20 AM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> 
> On Mon, 2021-05-10 at 18:54 -0600, Mike Jones wrote:
>> I have a little more info on this and a more specific question.
>> 
>> I copied the mtools code and hooked it into the layer so I could put in debug statements.
>> 
>> The problem occurs in charsetConv.c in the function cp_open, when it tries to do an iconv_open using
>> “CP850”, which fails.
>> 
>> I ran “iconv —list” to see what it can convert, and CP850 is not listed.
>> 
>> To prove this is the problem, I hardcoded the iconv_open with “ASCII” and bit bake can go all the way
>> through and “wic create” works properly.
>> 
>> So I ran iconv —version and saw it was “GNU libc 2.31” so I poked around those layers.
>> 
>> Best I can guess is this line from the mtools layer should tell the libc to provide CP850 via this line:
>> 
>> RDEPENDS_${PN}_libc-glibc = "glibc-gconv-ibm850”
>> 
>> Except that that is for runtime, and this is build time.
>> 
>> So I am not sure how glibc decides what converters are used. Is it just a set of defaults? Is there a layer
>> somewhere defining them I could not find?
>> 
>> It seems that somewhere along the way, mtools changed its default to use 850, and it is not compatible
>> anymore.
>> 
>> The solution is probably to tell glibc to provide CP850.
>> 
>> Does anyone know how to do that?
> 
> Which version of Yocto Project are you using? Are you using buildtools-tarball 
> or uninative in your build?

I run poky/scripts/install-buildtools with no command line options.

It contains these defaults:

DEFAULT_INSTALL_DIR = os.path.join(os.path.split(scripts_path)[0],'buildtools')
DEFAULT_BASE_URL = 'http://downloads.yoctoproject.org/releases/yocto'
DEFAULT_RELEASE = 'yocto-3.2_M3'
DEFAULT_INSTALLER_VERSION = '3.1+snapshot'
DEFAULT_BUILDDATE = '20200923'

I see I can use command line options to use a different release if you have guidance on what to use.

Mike

> 
> I suspect you need a newer buildtools-tarball version which includes CP850.
> 
> Cheers,
> 
> Richard
> 

Re: [poky] win create failing at mcopy #poky #raspberrypi

[Richard Purdie]

On Tue, 2021-05-11 at 11:20 +0100, Richard Purdie via lists.yoctoproject.org wrote:
> On Mon, 2021-05-10 at 18:54 -0600, Mike Jones wrote:
> > I have a little more info on this and a more specific question.
> > 
> > I copied the mtools code and hooked it into the layer so I could put in debug statements.
> > 
> > The problem occurs in charsetConv.c in the function cp_open, when it tries to do an iconv_open using
> > “CP850”, which fails.
> > 
> > I ran “iconv —list” to see what it can convert, and CP850 is not listed.
> > 
> > To prove this is the problem, I hardcoded the iconv_open with “ASCII” and bit bake can go all the way
> > through and “wic create” works properly.
> > 
> > So I ran iconv —version and saw it was “GNU libc 2.31” so I poked around those layers.
> > 
> > Best I can guess is this line from the mtools layer should tell the libc to provide CP850 via this line:
> > 
> > RDEPENDS_${PN}_libc-glibc = "glibc-gconv-ibm850”
> > 
> > Except that that is for runtime, and this is build time.
> > 
> > So I am not sure how glibc decides what converters are used. Is it just a set of defaults? Is there a layer
> > somewhere defining them I could not find?
> > 
> > It seems that somewhere along the way, mtools changed its default to use 850, and it is not compatible
> > anymore.
> > 
> > The solution is probably to tell glibc to provide CP850.
> > 
> > Does anyone know how to do that?
> 
> Which version of Yocto Project are you using? Are you using buildtools-tarball 
> or uninative in your build?
> 
> I suspect you need a newer buildtools-tarball version which includes CP850.

In particular, the CP850 requirement was removed recently:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=aaef86ad03d0d7904e42056c5a3e93a960e04ede

I remembered a dependency being added:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=268ea5544d204ca35d7709104382bad16705434b

but that was a while ago in 2012 so it seems unlikely you'd be missing the dependency.

The dependency was added to uninative in 2015:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=d8d4ce720e5c5823d00c2000d182e0f409240138

Cheers,

Richard

Re: [poky] win create failing at mcopy #poky #raspberrypi

[Mike Jones]

> On May 11, 2021, at 6:59 AM, Mike Jones via lists.yoctoproject.org <proclivis=gmail.com@lists.yoctoproject.org> wrote:
> 
> 
>>> On May 11, 2021, at 4:20 AM, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
>>> 
>>> On Mon, 2021-05-10 at 18:54 -0600, Mike Jones wrote:
>>> I have a little more info on this and a more specific question.
>>> 
>>> I copied the mtools code and hooked it into the layer so I could put in debug statements.
>>> 
>>> The problem occurs in charsetConv.c in the function cp_open, when it tries to do an iconv_open using
>>> “CP850”, which fails.
>>> 
>>> I ran “iconv —list” to see what it can convert, and CP850 is not listed.
>>> 
>>> To prove this is the problem, I hardcoded the iconv_open with “ASCII” and bit bake can go all the way
>>> through and “wic create” works properly.
>>> 
>>> So I ran iconv —version and saw it was “GNU libc 2.31” so I poked around those layers.
>>> 
>>> Best I can guess is this line from the mtools layer should tell the libc to provide CP850 via this line:
>>> 
>>> RDEPENDS_${PN}_libc-glibc = "glibc-gconv-ibm850”
>>> 
>>> Except that that is for runtime, and this is build time.
>>> 
>>> So I am not sure how glibc decides what converters are used. Is it just a set of defaults? Is there a layer
>>> somewhere defining them I could not find?
>>> 
>>> It seems that somewhere along the way, mtools changed its default to use 850, and it is not compatible
>>> anymore.
>>> 
>>> The solution is probably to tell glibc to provide CP850.
>>> 
>>> Does anyone know how to do that?
>> 
>> Which version of Yocto Project are you using? Are you using buildtools-tarball 
>> or uninative in your build?
> 
> I run poky/scripts/install-buildtools with no command line options.
> 
> It contains these defaults:
> 
> DEFAULT_INSTALL_DIR = os.path.join(os.path.split(scripts_path)[0],'buildtools')
> DEFAULT_BASE_URL = 'http://downloads.yoctoproject.org/releases/yocto'
> DEFAULT_RELEASE = 'yocto-3.2_M3'
> DEFAULT_INSTALLER_VERSION = '3.1+snapshot'
> DEFAULT_BUILDDATE = '20200923'
> 
> I see I can use command line options to use a different release if you have guidance on what to use.

After tweaking the script to change sha256 to sha256sum and removing one \s from the regular expression, I installed 3.2.3 and I will bitbake and see it it works.

I am guessing the script changes are due to an older version of poky, but poky is part of the openbmc project clone, not something we manually clone from a yocto repository.

If this works, I’ll have to address that issue on the openbmc listserv.

Mike

> 
> Mike
> 
>> 
>> I suspect you need a newer buildtools-tarball version which includes CP850.
>> 
>> Cheers,
>> 
>> Richard
>> 
> 
> 
>