All of lore.kernel.org
 help / color / mirror / Atom feed
* Fetch private gitlab repo using ssh with Yocto recipe #bitbake
@ 2022-01-26  7:16 hrsourabh011
       [not found] ` <664d7ef6f28584b0d7310774ad48660437562589.camel@delisys.ch>
  0 siblings, 1 reply; 6+ messages in thread
From: hrsourabh011 @ 2022-01-26  7:16 UTC (permalink / raw)
  To: yocto

[-- Attachment #1: Type: text/plain, Size: 2048 bytes --]

I am trying to fetch a private gitlab repo within Yocto image recipe using SSH protocol. In my image recipe I have passed SRC_URI as:

SRC_URI = " \
       gitsm://git@git.example.com:2224/blah/blah/blah/blah;protocol=ssh;branch=master \
" But this results in the error:

ERROR: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; export PATH="/root /build-swu-v2/tmp/sysroots-uninative/x86_64-linux/usr/bin:/root/sources/poky/scripts: /root/build-swu-v2/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/awsclient/1.4- r0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi:/root/build-swu-v2/tmp /work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/awsclient/1.4-r0/recipe-sysroot/usr/bin /crossscripts:/root/build-swu-v2/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi /awsclient/1.4-r0/recipe-sysroot-native/usr/sbin:/root/build-swu-v2/tmp/work/cortexa7t2hf- neon-vfpv4-poky-linux-gnueabi/awsclient/1.4-r0/recipe-sysroot-native/usr/bin:/root/build- swu-v2/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/awsclient/1.4-r0/recipe-sysroot- native/sbin:/root/build-swu-v2/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi /awsclient/1.4-r0/recipe-sysroot-native/bin:/root/sources/poky/bitbake/bin:/root/build-swu- v2/tmp/hosttools"; export HOME="/root"; LANG=C git -c core.fsyncobjectfiles=0 clone --bare  --mirror "ssh://git@git.osb-connagtive.com:2224/iot/products/iot-device-suite_client/aws- iot-client" /root/downloads//git2/git@git.example.com:2224/blah/blah/blah/blah --progress failed with exit code 128, no output
ERROR: Bitbake Fetcher Error: FetchError('Unable to fetch URL from any source.',  'gitsm://git@git.example.com:2224/blah/blah/blah/blah;protocol=ssh;branch=master')
DEBUG: Python function base_do_fetch finished

But I am able to clone the repo using git clone.

SSH key is already added to the Gitlab. There is no config file in my ~/.ssh. Do I need to create a config file? What should be the content of the config file?

Can anyone please let me know how to resolve this?

Thanks in advance.

[-- Attachment #2: Type: text/html, Size: 2185 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [yocto] Fetch private gitlab repo using ssh with Yocto recipe #bitbake
       [not found] ` <664d7ef6f28584b0d7310774ad48660437562589.camel@delisys.ch>
@ 2022-01-28  9:46   ` Sourabh Hegde
  2022-01-28 10:27     ` AW: " embedded (VIVAVIS AG)
  0 siblings, 1 reply; 6+ messages in thread
From: Sourabh Hegde @ 2022-01-28  9:46 UTC (permalink / raw)
  To: Nicolas Jeker; +Cc: yocto

[-- Attachment #1: Type: text/plain, Size: 1687 bytes --]

Hi Nicolas,

Thanks for your answer.

That's great. Even I am building inside a docker container. I tried with
creating a "config" file in .ssh directory. But I still have same issue.

Can you please let me know how to "forward SSH_AGENT into it to be able
to fetch from internal projects without the need to mount the key into the
container."? I never did that before.

Thanks in advance.

On Fri, Jan 28, 2022, 10:42 Nicolas Jeker <n.jeker@delisys.ch> wrote:

> On Tue, 2022-01-25 at 23:16 -0800, hrsourabh011@gmail.com wrote:
> > I am trying to fetch a private gitlab repo within Yocto image recipe
> > using SSH protocol. In my image recipe I have passed SRC_URI as:
> >
> > SRC_URI = " \
> >         gitsm://git@git.example.com:2224/blah/blah/blah/blah;protocol
> > =ssh;branch=master \
> > "
>
> I use almost the same, just without submodules.
>
> SRC_URI =
> "git://git@git.example.com:1234/group/project.git;protocol=ssh"
>
> It should "just work" if ssh is able to find your key. I often build in
> a docker container, so I have to forward SSH_AGENT into it to be able
> to fetch from internal projects without the need to mount the key into
> the container. I don't really have any insight for builds outside
> docker, if git clone works, the bitbake fetcher should too.
>
> > But this results in the error:
> >
> <snip>
> >
> > But I am able to clone the repo using git clone.
> > SSH key is already added to the Gitlab. There is no config file in my
> > ~/.ssh. Do I need to create a config file? What should be the content
> > of the config file?
>
> You should not need a ssh config file.
>
> > Can anyone please let me know how to resolve this?
> > Thanks in advance.
>
>

[-- Attachment #2: Type: text/html, Size: 2699 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* AW: [yocto] Fetch private gitlab repo using ssh with Yocto recipe #bitbake
  2022-01-28  9:46   ` [yocto] " Sourabh Hegde
@ 2022-01-28 10:27     ` embedded (VIVAVIS AG)
       [not found]       ` <90997c2cb92ea9700451b0e5d9afaeef64f4662c.camel@delisys.ch>
  2022-01-28 17:37       ` Khem Raj
  0 siblings, 2 replies; 6+ messages in thread
From: embedded (VIVAVIS AG) @ 2022-01-28 10:27 UTC (permalink / raw)
  To: Sourabh Hegde, Nicolas Jeker; +Cc: yocto

Hi,
 
> Von: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> Im Auftrag von Sourabh Hegde
> Gesendet: Freitag, 28. Januar 2022 10:47
>
> Can you please let me know how to "forward SSH_AGENT into it to be able
> to fetch from internal projects without the need to mount the key into the container."? I never did that before.

I use the following options within the Docker run command:
  -v $SSH_AUTH_SOCK:/ssh.socket \
  -e SSH_AUTH_SOCK=/ssh.socket \

Furthermore, I had to mount the .ssh folder into the container to make it working (be aware of security risk).
Additionally, you should check that uid, gid of the user in the container is the same on the host.

Regards,

Carsten

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [yocto] Fetch private gitlab repo using ssh with Yocto recipe #bitbake
       [not found]       ` <90997c2cb92ea9700451b0e5d9afaeef64f4662c.camel@delisys.ch>
@ 2022-01-28 12:33         ` Erik Botö
  0 siblings, 0 replies; 6+ messages in thread
From: Erik Botö @ 2022-01-28 12:33 UTC (permalink / raw)
  To: Nicolas Jeker; +Cc: VIVAVIS AG, Sourabh Hegde, yocto

On Fri, Jan 28, 2022 at 11:50 AM Nicolas Jeker <n.jeker@delisys.ch> wrote:
>
> On Fri, 2022-01-28 at 10:27 +0000, VIVAVIS AG wrote:
> > Hi,
> >
> > > Von: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> Im
> > > Auftrag von Sourabh Hegde
> > > Gesendet: Freitag, 28. Januar 2022 10:47
> > >
> > > Can you please let me know how to "forward SSH_AGENT into it to be
> > > able
> > > to fetch from internal projects without the need to mount the key
> > > into the container."? I never did that before.
> >
> > I use the following options within the Docker run command:
> >   -v $SSH_AUTH_SOCK:/ssh.socket \
> >   -e SSH_AUTH_SOCK=/ssh.socket \
> >
>
> That's pretty much what I use.
>
> > Furthermore, I had to mount the .ssh folder into the container to
> > make it working (be aware of security risk).
> > Additionally, you should check that uid, gid of the user in the
> > container is the same on the host.
>
> I do something similar, my "problem" was that ssh needs the
> .ssh/known_hosts file with a matching entry in addition to your
> key/agent, but mounting the .ssh folder was not possible for me because
> of permissions. Currently, I just created a little script that wraps
> "oe-init-build-env" and populates the known_hosts file accordingly.
>
> mkdir -p ~/.ssh
>
> cat <<EOF >> ~/.ssh/known_hosts
> git.example.com ssh-ed25519 <base64key>
> EOF
>

I use my own Dockerfile based on crops/poky where I do the following,
which might be helpful if you also use this. It sets up the config
changes in /etc/skel/ since it creates users "on the fly" with
matching uid.

# Remove strict host key checking for ssh
# This is needed since the build will pull source over git-ssh
RUN mkdir -p /etc/skel/.ssh/
COPY ci-scripts/docker-stuff/config /etc/skel/.ssh/
RUN echo 'export GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/dev/null
-o StrictHostKeyChecking=no"' >> /etc/skel/.bashrc


The ci-scripts/docker-stuff/config file contains:
Host *
   StrictHostKeyChecking no
   UserKnownHostsFile=/dev/null

Now it was ages ago I set this up, and right now I can't really
understand why I basically do the same thing twice. So you'd have to
check which of the two things that actually solves the issue :-)

Cheers,
Erik


> > Regards,
> >
> > Carsten
> >
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#56009): https://lists.yoctoproject.org/g/yocto/message/56009
> Mute This Topic: https://lists.yoctoproject.org/mt/88691891/3618217
> Mute #bitbake:https://lists.yoctoproject.org/g/yocto/mutehashtag/bitbake
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [erik.boto@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [yocto] Fetch private gitlab repo using ssh with Yocto recipe #bitbake
  2022-01-28 10:27     ` AW: " embedded (VIVAVIS AG)
       [not found]       ` <90997c2cb92ea9700451b0e5d9afaeef64f4662c.camel@delisys.ch>
@ 2022-01-28 17:37       ` Khem Raj
  2022-01-31 10:54         ` Sourabh Hegde
  1 sibling, 1 reply; 6+ messages in thread
From: Khem Raj @ 2022-01-28 17:37 UTC (permalink / raw)
  To: VIVAVIS AG; +Cc: Sourabh Hegde, Nicolas Jeker, yocto

On Fri, Jan 28, 2022 at 2:27 AM VIVAVIS AG <embedded@vivavis.com> wrote:
>
> Hi,
>
> > Von: yocto@lists.yoctoproject.org <yocto@lists.yoctoproject.org> Im Auftrag von Sourabh Hegde
> > Gesendet: Freitag, 28. Januar 2022 10:47
> >
> > Can you please let me know how to "forward SSH_AGENT into it to be able
> > to fetch from internal projects without the need to mount the key into the container."? I never did that before.
>
> I use the following options within the Docker run command:
>   -v $SSH_AUTH_SOCK:/ssh.socket \
>   -e SSH_AUTH_SOCK=/ssh.socket \
>
> Furthermore, I had to mount the .ssh folder into the container to make it working (be aware of security risk).
> Additionally, you should check that uid, gid of the user in the container is the same on the host.
>

yeah something like that works, we use it for yoe which always uses
container to build
see

https://github.com/YoeDistro/yoe-distro/blob/master/envsetup.sh#L528-L541

> Regards,
>
> Carsten
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> You automatically follow any topics you start or reply to.
> View/Reply Online (#56008): https://lists.yoctoproject.org/g/yocto/message/56008
> Mute This Topic: https://lists.yoctoproject.org/mt/88691891/1997914
> Mute #bitbake:https://lists.yoctoproject.org/g/yocto/mutehashtag/bitbake
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Fetch private gitlab repo using ssh with Yocto recipe #bitbake
  2022-01-28 17:37       ` Khem Raj
@ 2022-01-31 10:54         ` Sourabh Hegde
  0 siblings, 0 replies; 6+ messages in thread
From: Sourabh Hegde @ 2022-01-31 10:54 UTC (permalink / raw)
  To: yocto

[-- Attachment #1: Type: text/plain, Size: 2399 bytes --]

Hello @Nicolas @Erik @Khem,

Update from my side:

After following some discussion from other posts, I added "config" file.

~/.ssh/config:

Host git.example.com
HostName git.example.com
User git
PreferredAuthentications publickey
IdentityFile ~/.ssh/id_ed25519.pub
# LogLevel DEBUG3

Then I did "eval `ssh-agent -s`"

Then doing "ssh-add ~/.ssh/id_ed25519.pub" results in:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/root/.ssh/id_ed25519.pub' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.

Whereas the permissions are set as:

ls -l -a ~/.ssh

-rw-r--r-- 1 root root  157 Jan 31 10:48 config
-rw------- 1 root root  464 Jan 20 15:26 id_ed25519
-rw-r--r-- 1 root root  109 Jan 20 15:26 id_ed25519.pub
-rw-r--r-- 1 root root  888 Jan 26 08:43 known_hosts

"ssh-agent" is running

ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-lcft54A4nriC/agent.2833; export SSH_AUTH_SOCK;
SSH_AGENT_PID=2834; export SSH_AGENT_PID;
echo Agent pid 2834;

After doing these changes, when I try to "ssh -v git.example.com" to test the connection before running bitbake, I get

OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f  31 Mar 2020
debug1: Reading configuration data /root/.ssh/config
debug1: /root/.ssh/config line 1: Applying options for git.example.com
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to git.example.com [116.203.241.xxx] port 22.
debug1: connect to address 116.203.241.xxx port 22: Connection refused
ssh: connect to host git.example.com port 22: Connection refused

I don't understand what is the issue here.

@Nicolas Can you please let me know where and how to run below commands? Do I need to run them every time before fetching from gitlab?
-v $SSH_AUTH_SOCK:/ssh.socket \
-e SSH_AUTH_SOCK=/ssh.socket \

And also I already have "known_hosts" file with matching entries for key/agent pair.

Can you please let me know how to make this working?

Your help will be much appreciated.

Thanks in advance.

[-- Attachment #2: Type: text/html, Size: 2785 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2022-01-31 10:54 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-26  7:16 Fetch private gitlab repo using ssh with Yocto recipe #bitbake hrsourabh011
     [not found] ` <664d7ef6f28584b0d7310774ad48660437562589.camel@delisys.ch>
2022-01-28  9:46   ` [yocto] " Sourabh Hegde
2022-01-28 10:27     ` AW: " embedded (VIVAVIS AG)
     [not found]       ` <90997c2cb92ea9700451b0e5d9afaeef64f4662c.camel@delisys.ch>
2022-01-28 12:33         ` Erik Botö
2022-01-28 17:37       ` Khem Raj
2022-01-31 10:54         ` Sourabh Hegde

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.