How to use mirror to download linux-yocto?

How to use mirror to download linux-yocto?

[Bo YU]

Hi,

I hope the mail list is the right place to ask the simple question about yocto.

I am using meta-riscv[0] to build riscv32 qemu.
And after `$(MACHINE=qemuriscv32 bitbake core-image-full-cmdline)`
Everything is downloaded well but linux-yocto:
```
0: linux-yocto-6.1.14+gitAUTOINC+e8d08fc4c0_b05ca3429c-r0 do_fetch -
12h26m11s (pid 4110422)  28% |#######                    | 2.00 KiB/s
openembedded-core/meta/recipes-kernel/linux/linux-yocto_6.1.bb
```
I tried many time but nothing change.

There is one mirror[1] I can use under my network condition.
So I want to know what is right way to alter URI in
openembedded-core/meta/recipes-kernel/linux/linux-yocto_6.1.bb?

I googled one article to suggest use Insteadof in .gitconfig[2], is it ok?

Many thanks for your help.

BR,
Bo

[0]: https://github.com/riscv/meta-riscv
[1]: https://isrc.iscas.ac.cn/gitlab/mirrors/git.yoctoproject.org/linux-yocto
[3]: https://www.wpgdadatong.com/cn/blog/detail/44614

Re: [yocto] How to use mirror to download linux-yocto?

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1685 bytes --]

When the slow download is happening, can you please check (e.g. from a
process listing), which server specifically is being accessed and with
which command? Just to be sure where the problem is.

Alex

On Fri 17. Mar 2023 at 15.08, Bo YU <tsu.yubo@gmail.com> wrote:

> Hi,
>
> I hope the mail list is the right place to ask the simple question about
> yocto.
>
> I am using meta-riscv[0] to build riscv32 qemu.
> And after `$(MACHINE=qemuriscv32 bitbake core-image-full-cmdline)`
> Everything is downloaded well but linux-yocto:
> ```
> 0: linux-yocto-6.1.14+gitAUTOINC+e8d08fc4c0_b05ca3429c-r0 do_fetch -
> 12h26m11s (pid 4110422)  28% |#######                    | 2.00 KiB/s
> openembedded-core/meta/recipes-kernel/linux/linux-yocto_6.1.bb
> ```
> I tried many time but nothing change.
>
> There is one mirror[1] I can use under my network condition.
> So I want to know what is right way to alter URI in
> openembedded-core/meta/recipes-kernel/linux/linux-yocto_6.1.bb?
>
> I googled one article to suggest use Insteadof in .gitconfig[2], is it ok?
>
> Many thanks for your help.
>
> BR,
> Bo
>
> [0]: https://github.com/riscv/meta-riscv
> [1]:
> https://isrc.iscas.ac.cn/gitlab/mirrors/git.yoctoproject.org/linux-yocto
> [3]: https://www.wpgdadatong.com/cn/blog/detail/44614
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#59449):
> https://lists.yoctoproject.org/g/yocto/message/59449
> Mute This Topic: https://lists.yoctoproject.org/mt/97675021/1686489
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 3052 bytes --]

Re: [yocto] How to use mirror to download linux-yocto?

[Bo YU]

[-- Attachment #1: Type: text/plain, Size: 944 bytes --]

Hi,
On Sat, Mar 18, 2023 at 09:29:34AM +0000, Alexander Kanavin wrote:
>   When the slow download is happening, can you please check (e.g. from a
>   process listing), which server specifically is being accessed and with
>   which command? Just to be sure where the problem is.

Thanks for tips. I downloaded linux-yocto about 3 days but failed and
one night to download it after just sent the mail.

I suspected the accessing to git://git.yoctoproject.org/linux-yocto.git 
is very slow under my network. So I think there is one way to replace 

```
SRC_URI =
"git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRANCH};
\
git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-6.1;destsuffix=${KMETA}"
```

Or use another bb file to download linux-yocto kernel with mirror I
mentioned. But I am fail to try this.

Anway, Thank you very much!

>

-- 
Regards,
--
   Bo YU


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [yocto] How to use mirror to download linux-yocto?

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2137 bytes --]

On Sun, Mar 19, 2023 at 3:50 PM Bo YU <tsu.yubo@gmail.com> wrote:

> Hi,
> On Sat, Mar 18, 2023 at 09:29:34AM +0000, Alexander Kanavin wrote:
> >   When the slow download is happening, can you please check (e.g. from a
> >   process listing), which server specifically is being accessed and with
> >   which command? Just to be sure where the problem is.
>
> Thanks for tips. I downloaded linux-yocto about 3 days but failed and
> one night to download it after just sent the mail.
>
> I suspected the accessing to git://git.yoctoproject.org/linux-yocto.git
> is very slow under my network. So I think there is one way to replace
>
> ```
> SRC_URI =
> "git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRANCH}
> <http://git.yoctoproject.org/linux-yocto.git;name=machine;branch=$%7BKBRANCH%7D>
> ;
> \
> git://
> git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-6.1;destsuffix=${KMETA}
> <http://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-6.1;destsuffix=$%7BKMETA%7D>
> "
> ```
>
> Or use another bb file to download linux-yocto kernel with mirror I
> mentioned. But I am fail to try this.
>

You can and should use PREMIRROR like with any other source fetched by
bitbake fetcher.

Use BB_GENERATE_MIRROR_TARBALLS, then rsync your downloads directory
(exclude .done, bad-checksum files and git2, svn, ... dirs) to some shared
location on your network and you have a good starting PREMIRROR to use in
all the builds, so you don't ever need to re-download everything again.
Update your PREMIRROR with rsync after succesful build (every time or from
time to time).

You can also share whole DL_DIR directly, but that has small risk, that if
fetcher fails to update e.g. linux-yocto due to temporary network glitch,
it will remove whole git clone from the DL_DIR and start again, while with
PREMIRROR it will start from the tarball stored there. So it's useful to
combine both (PREMIRROR updated after successful builds and shared DL_DIR
if you have many builders with fast access to the same NFS/CIFS share).

Cheers,

[-- Attachment #2: Type: text/html, Size: 2709 bytes --]