bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

[Samuel Jiang (江騏先)]

[-- Attachment #1.1: Type: text/plain, Size: 10100 bytes --]

Dear yocto developer,
The below disscussed about azure devops clone issue including test result with Microsoft support team member.
We wonder know how Bitbake access repository through SSH. Does it different between git and Azue DevOps?

Thanks,

Samuel Jiang
---------- Forwarded message ----------
From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
Date: 2019年10月31日 PM5:39 [+0800]
To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
Cc: support <support@mail.support.microsoft.com>, Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

Hi Samuel,

Thank you for reply.

I test and read the BitBake doc, let me share with you my finding and analysis.

  1.  I have verified that our Azure DevOps Organization Repo could be clone through SSH authentication successfully.

  1.  I create a new SSH key and add it to Azure DevOps SSH public keys.
[cid:image008.png@01D5900C.6DB2CF60] [cid:image009.png@01D5900C.6DB2CF60]

  1.  Then I use git clone command and clone this repo successfully. The url is “v-chucho-microsoft@vs-ssh.visualstudio.com:v3/v-chucho-microsoft/testCodeCoverage/testCodeCoverage”
[cid:image011.png@01D5900C.C1971910]
My testing verify that SSH Authentication is available, and the url given by Azure DevOps is correct.


  1.  Then I also test it in GitHub, and succeed. The only difference is the url of GitHub is “git@github.com:xxxxxxxxxxxxxxxxxx.git”


  1.  So let’s come back to BitBake. I read more of the BitBake doc.( https://www.yoctoproject.org/docs/1.8/bitbake-user-manual/bitbake-user-manual.html#bb-fetchers) . Your command SRC_URI = "git://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobranch=1". The BitBake example is below.
[cid:image012.png@01D59011.A1A7F340]
We can see that the BitBake syntax is like git SSH but not Azure DevOps SSH. Do you try to use this command clone a repo from GitHub?


  1.  Then I read the doc you shared. The customer also met your issue when trying to clone Azure DevOps repo with SSH url. In my understanding, seems BitBake Git fetcher is good for Git SSH but not ready to access the Azure DevOps SSH.
[cid:image010.png@01D5900C.6DB2CF60]

Action Plan
Could you involve BitBake support team? We can deliver this testing result and consult them how BitBake access Azure DevOps through SSH.

If you have any conern or query, please feel free to let me know.

Best Regards,

Alex Chong

[image001]
Support Engineer
Microsoft APAC Developer Support Team
Customer Service & Support (CSS)
Email: v-chucho@microsoft.com<mailto:v-chucho@microsoft.com>
Office: +86 (21) 52638610
Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri

From: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
Sent: Thursday, October 31, 2019 12:16 PM
To: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
Cc: support <support@mail.support.microsoft.com>
Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

Hi Alex,

I catch same problem on the yocto mail list
link: https://lists.yoctoproject.org/pipermail/yocto/2018-October/042736.html<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.yoctoproject.org%2Fpipermail%2Fyocto%2F2018-October%2F042736.html&data=02%7C01%7Cv-chucho%40microsoft.com%7C5b51185a798a4fdf6cd608d75db90d70%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637080921717407210&sdata=Uf0%2Fq1FwlmsXRaoFfMbDfpiWKNzN3MgBlgZXphM3jwo%3D&reserved=0>

when I use SRC_URI = "git://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobranch=1"

the bitbake response below error message:
git -c core.fsyncobjectfiles=0 ls-remote ssh://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump  failed with exit code 128, output:
ssh: Could not resolve hostname vs-ssh.visualstudio.com:v3: Name or service not known
fatal: Could not read from remote repository.

I think the bitbake call “git ls-remote” command however it could not parser below host vs-ssh.visualstudio.com:v3
Could you help confirm the  “git ls-remote”command could work on ssh method clone with ssh:// prefix?

BTW, the protocol will define the uri prefix, I didn’t find any way removing this prefix.

Thanks,
Samuel Jiang

From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
Sent: Thursday, October 31, 2019 10:32 AM
To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
Cc: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>; support <support@mail.support.microsoft.com>
Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

Hi Samuel,

How are you?

Do you try our Azure DevOps SSH authentication? Do you contact BitBake support for help? Is there any suggestion or new issue? Please feel free to contact me if there is something I can do for you.

Best Regards,

Alex Chong

[image001]
Support Engineer
Microsoft APAC Developer Support Team
Customer Service & Support (CSS)
Email: v-chucho@microsoft.com<mailto:v-chucho@microsoft.com>
Office: +86 (21) 52638610
Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri

From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com<mailto:v-chucho@microsoft.com>>
Sent: Tuesday, October 29, 2019 3:50 PM
To: Samuel.Jiang@quantatw.com<mailto:Samuel.Jiang@quantatw.com>
Cc: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com<mailto:v-chucho@microsoft.com>>
Subject: 119102923000220 use yocto bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic




[Image removed by sender. 郵件模板標頭]



Hi Samuel,



Thank you for contacting Microsoft.


My name is Alex and I am the Microsoft Support Engineer from APGC who will be working with you on this support request . You can reach me using the contact information in my signature.

I just called you at +886 3 327 2345 and had a quick discussion.
Issue Description
1.Customer is using a 3rd-party application BitBake to clone Azure DevOps repo.
2.Customer could fetch file from https url successfully.
3.Customer wants to know whether Azure DevOps support SSH authentication method.


Analysis
1.Yes, Azure DevOps has SSH authentication method. (Reference: https://docs.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdevops%2Frepos%2Fgit%2Fuse-ssh-keys-to-authenticate%3Fview%3Dazure-devops&data=02%7C01%7Cv-chucho%40microsoft.com%7C5b51185a798a4fdf6cd608d75db90d70%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637080921717417205&sdata=RT5w4vibwi5VapG3Sr8cDthwGeu%2BudWf98s9FdsbxP4%3D&reserved=0>)

  *   Create your SSH keys
    Create your SSH keys with the ssh-keygen command from the bash prompt.

  *   Add the public key to your Azure DevOps Organization
Open your security settings by browsing to the web portal and selecting your avatar in the upper right of the user interface. Select Security in the menu that appears.
[cid:image002.png@01D58FF1.31F5A490]
Select SSH public keys , then select +New Key.
[cid:image003.png@01D58FF1.31F5A490]
Copy the contents of the public key (for example, id_rsa.pub) that you generated into the Public Key Data field.

[cid:image004.png@01D58FF1.31F5A490]



  *   Clone the Git repository with SSH
Copy the SSH clone URL from the web portal. In this example the SSL clone URL is for a repo in an organization named fabrikam-fiber, as indicated by the first part of the URL after dev.azure.com.
[cid:image005.png@01D58FF1.31F5A490]
Run git clone from the command prompt.
My example is git clone v-chucho-microsoft@vs-ssh.visualstudio.com:v3/v-chucho-microsoft/testCodeCoverage/testCodeCoverage<mailto:v-chucho-microsoft@vs-ssh.visualstudio.com:v3/v-chucho-microsoft/testCodeCoverage/testCodeCoverage>

2.Regarding to BitBake. (Reference: https://www.yoctoproject.org/docs/1.8/bitbake-user-manual/bitbake-user-manual.html#bb-fetchers<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.yoctoproject.org%2Fdocs%2F1.8%2Fbitbake-user-manual%2Fbitbake-user-manual.html%23bb-fetchers&data=02%7C01%7Cv-chucho%40microsoft.com%7C5b51185a798a4fdf6cd608d75db90d70%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637080921717417205&sdata=pAVBdFRKoXlV2dcCBZi4aHVvIWN%2BfLYhnXg2Yf1fbYY%3D&reserved=0>)

This is a 3rd-party application which is out of our Microsoft support. Our Azure DevOps has SSH authentication method. But I have no experience whether there is such option in BitBake.

I did some research on this application and find Git fetcher and ssh:// fetcher functionalities. In my understanding, it should support git with SSH or SSH fecher directly. It's suggested to consult BitBake support for more details of these 2 fetchers. If there is something our Microsoft could help, please feel free to involve me directly.
[cid:image006.png@01D58FF1.31F5A490]
[cid:image007.png@01D58FF1.31F5A490]


If you have any concern or query, please feel free to contact me.


Best Regards,

Alex Chong

Support Engineer
Microsoft APAC Developer Support Team
Customer Service & Support (CSS)
Email: v-chucho@microsoft.com<mailto:v-chucho@microsoft.com>
Office: +86 (21) 52638610
Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri



這則來自 Microsoft 的訊息是您或貴公司所購買或參與之計畫、服務或產品的重要部分。請檢閱我們的線上隱私權聲明<https://go.microsoft.com/fwlink/?LinkId=521839>

Microsoft Corporation
One Microsoft Way
Redmond, WA, USA 98052

[Image removed by sender. Microsoft]

[-- Attachment #1.2: Type: text/html, Size: 38896 bytes --]

[-- Attachment #2: image005.png --]
[-- Type: image/png, Size: 29915 bytes --]

[-- Attachment #3: image001.png --]
[-- Type: image/png, Size: 1999 bytes --]

[-- Attachment #4: image003.png --]
[-- Type: image/png, Size: 141341 bytes --]

[-- Attachment #5: image010.png --]
[-- Type: image/png, Size: 5383 bytes --]

[-- Attachment #6: image012.png --]
[-- Type: image/png, Size: 4298 bytes --]

[-- Attachment #7: image006.png --]
[-- Type: image/png, Size: 44254 bytes --]

[-- Attachment #8: image008.png --]
[-- Type: image/png, Size: 15632 bytes --]

[-- Attachment #9: image004.png --]
[-- Type: image/png, Size: 37907 bytes --]

[-- Attachment #10: image009.png --]
[-- Type: image/png, Size: 10853 bytes --]

[-- Attachment #11: image011.png --]
[-- Type: image/png, Size: 10066 bytes --]

[-- Attachment #12: ~WRD331.jpg --]
[-- Type: image/jpeg, Size: 203 bytes --]

[-- Attachment #13: image007.png --]
[-- Type: image/png, Size: 19994 bytes --]

[-- Attachment #14: image002.png --]
[-- Type: image/png, Size: 28693 bytes --]

Re: bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

[Mark Hatle]

When cloning a repository using the builtin fetcher that is git based, the
default protocol is 'git'.

If you want to use an alternative protocol, such as http or ssh, you must
specify the protocol to use.

In your example below, you have specified http:

SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http"

Change the http to 'ssh' to switch to that protocol.  Note, for the fetcher the
system MUST be able to automatically login without user interaction.  This means
that you must have authorized keys enabled.

Note, ssh is not suggested as a protocol -- except for internal private
development -- because you can't easily share you recipes with others.  Many
corporations block ssh, and will only allow http via a proxy.

--Mark

On 11/3/19 8:52 PM, Samuel Jiang (江騏先) wrote:
> Dear yocto developer,
> The below disscussed about azure devops clone issue including test result with
> Microsoft support team member.
> We wonder know how Bitbake access repository through SSH. Does it different
> between git and Azue DevOps?
> 
> Thanks,
> 
> Samuel Jiang
> ---------- Forwarded message ----------
> *From:* Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
> *Date:* 2019年10月31日 PM5:39 [+0800]
> *To:* Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
> *Cc:* support <support@mail.support.microsoft.com>, Alex Chong (Shanghai
> Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
> *Subject:* RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure DevOps
> repository Azure DevOps Services Basic
> 
>> Hi Samuel,
>>
>>  
>>
>> Thank you for reply.
>>
>>  
>>
>> I test and read the BitBake doc, let me share with you my finding and analysis.
>>
>>  1. I have verified that our Azure DevOps Organization Repo could be clone
>>     through SSH authentication successfully. 
>>
>>  1. I create a new SSH key and add it to Azure DevOps SSH public keys. 
>>
>>  2. Then I use git clone command and clone this repo successfully. The url is
>>     “v-chucho-microsoft@vs-ssh.visualstudio.com:v3/v-chucho-microsoft/testCodeCoverage/testCodeCoverage”
>>
>>
>> My testing verify that SSH Authentication is available, and the url given by
>> Azure DevOps is correct.
>>
>>  
>>
>>  2. Then I also test it in GitHub, and succeed. The only difference is the url
>>     of GitHub is “git@github.com:xxxxxxxxxxxxxxxxxx.git” 
>>
>>  
>>
>>  3. So let’s come back to BitBake. I read more of the BitBake doc.(
>>     https://www.yoctoproject.org/docs/1.8/bitbake-user-manual/bitbake-user-manual.html#bb-fetchers)
>>     . Your command SRC_URI =
>>     "git://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobranch=1".
>>     The BitBake example is below. 
>>
>> We can see that the BitBake syntax is like git SSH but not Azure DevOps SSH.
>> Do you try to use this command clone a repo from GitHub?
>>
>>  
>>
>>  4. Then I read the doc you shared. The customer also met your issue when
>>     trying to clone Azure DevOps repo with SSH url. In my understanding, seems
>>     BitBake Git fetcher is good for Git SSH but not ready to access the Azure
>>     DevOps SSH. 
>>
>>  
>>
>> *_Action Plan_*
>>
>> Could you involve BitBake support team? We can deliver this testing result and
>> consult them how BitBake access Azure DevOps through SSH.
>>
>>  
>>
>> If you have any conern or query, please feel free to let me know.
>>
>>  
>>
>> Best Regards,
>>
>>  
>>
>> Alex Chong
>>
>>  
>>
>> image001
>>
>> Support Engineer
>>
>> Microsoft APAC Developer Support Team
>>
>> Customer Service & Support (CSS)
>>
>> Email: v-chucho@microsoft.com <mailto:v-chucho@microsoft.com>
>>
>> Office: +86 (21) 52638610
>>
>> Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
>>
>> Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
>>
>>  
>>
>> *From:* Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
>> *Sent:* Thursday, October 31, 2019 12:16 PM
>> *To:* Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
>> *Cc:* support <support@mail.support.microsoft.com>
>> *Subject:* RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure DevOps
>> repository Azure DevOps Services Basic
>>
>>  
>>
>> Hi Alex,
>>
>> I catch same problem on the yocto mail list
>> link: https://lists.yoctoproject.org/pipermail/yocto/2018-October/042736.html
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.yoctoproject.org%2Fpipermail%2Fyocto%2F2018-October%2F042736.html&data=02%7C01%7Cv-chucho%40microsoft.com%7C5b51185a798a4fdf6cd608d75db90d70%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637080921717407210&sdata=Uf0%2Fq1FwlmsXRaoFfMbDfpiWKNzN3MgBlgZXphM3jwo%3D&reserved=0>
>>
>> when I use SRC_URI =
>> "git://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobranch=1"
>>
>> the bitbake response below error message:
>> git -c core.fsyncobjectfiles=0 ls-remote
>> ssh://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump  failed
>> with exit code 128, output:
>>
>> ssh: Could not resolve hostname vs-ssh.visualstudio.com:v3: Name or service
>> not known
>>
>> fatal: Could not read from remote repository.
>>
>> I think the bitbake call “git ls-remote” command however it could not parser
>> below host vs-ssh.visualstudio.com:v3
>>
>> Could you help confirm the  “git ls-remote”command could work on ssh method
>> clone with ssh:// prefix?
>>
>> BTW, the protocol will define the uri prefix, I didn’t find any way removing
>> this prefix.
>>
>> Thanks,
>> Samuel Jiang
>>
>>  
>>
>> *From:* Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
>> *Sent:* Thursday, October 31, 2019 10:32 AM
>> *To:* Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
>> *Cc:* Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>;
>> support <support@mail.support.microsoft.com>
>> *Subject:* RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure DevOps
>> repository Azure DevOps Services Basic
>>
>>  
>>
>> Hi Samuel,
>>
>>  
>>
>> How are you?
>>
>>  
>>
>> Do you try our Azure DevOps SSH authentication? Do you contact BitBake support
>> for help? Is there any suggestion or new issue? Please feel free to contact me
>> if there is something I can do for you.
>>
>>  
>>
>> Best Regards,
>>
>>  
>>
>> Alex Chong
>>
>>  
>>
>> image001
>>
>> Support Engineer
>>
>> Microsoft APAC Developer Support Team
>>
>> Customer Service & Support (CSS)
>>
>> Email: v-chucho@microsoft.com <mailto:v-chucho@microsoft.com>
>>
>> Office: +86 (21) 52638610
>>
>> Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
>>
>> Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
>>
>>  
>>
>> *From:* Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com
>> <mailto:v-chucho@microsoft.com>>
>> *Sent:* Tuesday, October 29, 2019 3:50 PM
>> *To:* Samuel.Jiang@quantatw.com <mailto:Samuel.Jiang@quantatw.com>
>> *Cc:* Alex Chong (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com
>> <mailto:v-chucho@microsoft.com>>
>> *Subject:* 119102923000220 use yocto bitbake SRC_URI fetch Azure DevOps
>> repository Azure DevOps Services Basic
>>
>>  
>>
>>  
>>
>>  
>>
>> Image removed by sender. 郵件模板標頭
>>
>>  
>>
>> Hi Samuel,
>>
>>  
>>
>> Thank you for contacting Microsoft.
>>
>>  
>>
>> My name is Alex and I am the Microsoft Support Engineer from APGC who will be
>> working with you on this support request . You can reach me using the contact
>> information in my signature.
>>
>>  
>>
>> I just called you at +886 3 327 2345 and had a quick discussion.
>>
>> *_Issue Description_*
>>
>> 1.Customer is using a 3rd-party application BitBake to clone Azure DevOps repo. 
>>
>> 2.Customer could fetch file from https url successfully.
>>
>> 3.Customer wants to know whether Azure DevOps support SSH authentication method.
>>
>>  
>>
>> *_Analysis_*
>>
>> 1.Yes, Azure DevOps has SSH authentication method. (Reference:
>> https://docs.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdevops%2Frepos%2Fgit%2Fuse-ssh-keys-to-authenticate%3Fview%3Dazure-devops&data=02%7C01%7Cv-chucho%40microsoft.com%7C5b51185a798a4fdf6cd608d75db90d70%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637080921717417205&sdata=RT5w4vibwi5VapG3Sr8cDthwGeu%2BudWf98s9FdsbxP4%3D&reserved=0>)
>>
>>   * Create your SSH keys
>>
>>     Create your SSH keys with the |ssh-keygen| command from the |bash| prompt.
>>
>>   * Add the public key to your Azure DevOps Organization
>>
>> Open your security settings by browsing to the web portal and selecting your
>> avatar in the upper right of the user interface. Select Security in the menu
>> that appears.
>>
>> Select SSH public keys , then select +New Key.
>>
>> Copy the contents of the public key (for example, id_rsa.pub) that you
>> generated into the Public Key Data field.
>>
>>  
>>
>>  
>>
>>   * Clone the Git repository with SSH
>>
>> Copy the SSH clone URL from the web portal. In this example the SSL clone URL
>> is for a repo in an organization named fabrikam-fiber, as indicated by the
>> first part of the URL after dev.azure.com.
>>
>> Run git clone from the command prompt.
>>
>> My example is /git clone/
>> /v-chucho-microsoft@vs-ssh.visualstudio.com:v3/v-chucho-microsoft/testCodeCoverage/testCodeCoverage/
>> <mailto:v-chucho-microsoft@vs-ssh.visualstudio.com:v3/v-chucho-microsoft/testCodeCoverage/testCodeCoverage>
>>
>>  
>>
>> 2.Regarding to BitBake. (Reference:
>> https://www.yoctoproject.org/docs/1.8/bitbake-user-manual/bitbake-user-manual.html#bb-fetchers
>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.yoctoproject.org%2Fdocs%2F1.8%2Fbitbake-user-manual%2Fbitbake-user-manual.html%23bb-fetchers&data=02%7C01%7Cv-chucho%40microsoft.com%7C5b51185a798a4fdf6cd608d75db90d70%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637080921717417205&sdata=pAVBdFRKoXlV2dcCBZi4aHVvIWN%2BfLYhnXg2Yf1fbYY%3D&reserved=0>)
>>
>> This is a 3rd-party application which is out of our Microsoft support. Our
>> Azure DevOps has SSH authentication method. But I have no experience
>> whether there is such option in BitBake. 
>>
>> I did some research on this application and find Git fetcher and ssh://
>> fetcher functionalities. In my understanding, it should support git with SSH
>> or SSH fecher directly. It's suggested to consult BitBake support for more
>> details of these 2 fetchers. If there is something our Microsoft could help,
>> please feel free to involve me directly.
>>
>>  
>>
>> If you have any concern or query, please feel free to contact me.
>>
>>  
>>
>> Best Regards,
>>
>>  
>>
>> Alex Chong
>>
>>  
>>
>> Support Engineer
>>
>> Microsoft APAC Developer Support Team
>>
>> Customer Service & Support (CSS)
>>
>> Email: v-chucho@microsoft.com <mailto:v-chucho@microsoft.com>
>>
>> Office: +86 (21) 52638610
>>
>> Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
>>
>> Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
>>
>>  
>>
>> 這則來自 Microsoft 的訊息是您或貴公司所購買或參與之計畫、服務或產品的重要部
>> 分。請檢閱我們的線上隱私權聲明 <https://go.microsoft.com/fwlink/?LinkId=521839>
>>
>> Microsoft Corporation
>> One Microsoft Way
>> Redmond, WA, USA 98052
>>
>> Image removed by sender. Microsoft
>>
> 

Re: bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

[Richard Purdie]

On Mon, 2019-11-04 at 02:52 +0000, Samuel Jiang (江騏先) wrote:
> Dear yocto developer,
> The below disscussed about azure devops clone issue including test
> result with Microsoft support team member.
> We wonder know how Bitbake access repository through SSH. Does it
> different between git and Azue DevOps?


Try:

SRC_URI = "git://quanta01@vs-ssh.visualstudio.com/v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobranch=1"

as I think you didn't convert the url to our syntax correctly.

Cheers,

Richard


> Thanks,
> 
> Samuel Jiang
> ---------- Forwarded message ----------
> From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <
> v-chucho@microsoft.com>
> Date: 2019年10月31日 PM5:39 [+0800]
> To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
> Cc: support <support@mail.support.microsoft.com>, Alex Chong
> (Shanghai Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
> Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure
> DevOps repository Azure DevOps Services Basic
> 
> > Hi Samuel,
> > 
> >  
> > 
> > Thank you for reply.
> > 
> >  
> > 
> > I test and read the BitBake doc, let me share with you my finding
> > and analysis.
> > 
> > I have verified that our Azure DevOps Organization Repo could be
> > clone through SSH authentication successfully.
> > I create a new SSH key and add it to Azure DevOps SSH public keys.
> >  
> > 
> > Then I use git clone command and clone this repo successfully. The
> > url is “v-chucho-microsoft@vs-ssh.visualstudio.com:v3/v-chucho-
> > microsoft/testCodeCoverage/testCodeCoverage”
> > 
> > 
> > My testing verify that SSH Authentication is available, and the url
> > given by Azure DevOps is correct.
> > 
> >  
> > 
> > Then I also test it in GitHub, and succeed. The only difference is
> > the url of GitHub is “git@github.com:xxxxxxxxxxxxxxxxxx.git”
> >  
> > 
> > So let’s come back to BitBake. I read more of the BitBake doc.( 
> > https://www.yoctoproject.org/docs/1.8/bitbake-user-manual/bitbake-user-manual.html#bb-fetchers
> > ) . Your command SRC_URI = "git://quanta01@vs-
> > ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump;protocol=ssh;nob
> > ranch=1". The BitBake example is below.
> > 
> > 
> > We can see that the BitBake syntax is like git SSH but not Azure
> > DevOps SSH. Do you try to use this command clone a repo from
> > GitHub?
> > 
> >  
> > 
> > Then I read the doc you shared. The customer also met your issue
> > when trying to clone Azure DevOps repo with SSH url. In my
> > understanding, seems BitBake Git fetcher is good for Git SSH but
> > not ready to access the Azure DevOps SSH.
> > 
> > 
> >  
> > 
> > Action Plan
> > 
> > Could you involve BitBake support team? We can deliver this testing
> > result and consult them how BitBake access Azure DevOps through
> > SSH.
> > 
> >  
> > 
> > If you have any conern or query, please feel free to let me know.
> > 
> >  
> > 
> > Best Regards,
> > 
> >  
> > 
> > Alex Chong
> > 
> >  
> > 
> > 
> > 
> > Support Engineer
> > 
> > Microsoft APAC Developer Support Team
> > 
> > Customer Service & Support (CSS)
> > 
> > Email: v-chucho@microsoft.com
> > 
> > Office: +86 (21) 52638610
> > 
> > Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
> > 
> > Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
> > 
> >  
> > 
> > From: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
> > Sent: Thursday, October 31, 2019 12:16 PM
> > To: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <
> > v-chucho@microsoft.com>
> > Cc: support <support@mail.support.microsoft.com>
> > Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure
> > DevOps repository Azure DevOps Services Basic
> > 
> >  
> > 
> > Hi Alex,
> > 
> > I catch same problem on the yocto mail list
> > link: 
> > https://lists.yoctoproject.org/pipermail/yocto/2018-October/042736.html
> > 
> > when I use SRC_URI = "git://quanta01@vs-
> > ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump;protocol=ssh;nob
> > ranch=1"
> > 
> > the bitbake response below error message:
> > git -c core.fsyncobjectfiles=0 ls-remote 
> > ssh://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump
> >   failed with exit code 128, output:
> > 
> > ssh: Could not resolve hostname vs-ssh.visualstudio.com:v3: Name or
> > service not known
> > 
> > fatal: Could not read from remote repository.
> > 
> > I think the bitbake call “git ls-remote” command however it could
> > not parser below host vs-ssh.visualstudio.com:v3
> > 
> > Could you help confirm the  “git ls-remote”command could work on
> > ssh method clone with ssh:// prefix?
> > 
> > BTW, the protocol will define the uri prefix, I didn’t find any way
> > removing this prefix.
> > 
> > Thanks,
> > Samuel Jiang
> > 
> >  
> > 
> > From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <
> > v-chucho@microsoft.com>
> > Sent: Thursday, October 31, 2019 10:32 AM
> > To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
> > Cc: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <
> > v-chucho@microsoft.com>; support <
> > support@mail.support.microsoft.com>
> > Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure
> > DevOps repository Azure DevOps Services Basic
> > 
> >  
> > 
> > Hi Samuel,
> > 
> >  
> > 
> > How are you?
> > 
> >  
> > 
> > Do you try our Azure DevOps SSH authentication? Do you contact
> > BitBake support for help? Is there any suggestion or new issue?
> > Please feel free to contact me if there is something I can do for
> > you.
> > 
> >  
> > 
> > Best Regards,
> > 
> >  
> > 
> > Alex Chong
> > 
> >  
> > 
> > 
> > 
> > Support Engineer
> > 
> > Microsoft APAC Developer Support Team
> > 
> > Customer Service & Support (CSS)
> > 
> > Email: v-chucho@microsoft.com
> > 
> > Office: +86 (21) 52638610
> > 
> > Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
> > 
> > Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
> > 
> >  
> > 
> > From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <
> > v-chucho@microsoft.com>
> > Sent: Tuesday, October 29, 2019 3:50 PM
> > To: Samuel.Jiang@quantatw.com
> > Cc: Alex Chong (Shanghai Wicresoft Co,.Ltd.) <
> > v-chucho@microsoft.com>
> > Subject: 119102923000220 use yocto bitbake SRC_URI fetch Azure
> > DevOps repository Azure DevOps Services Basic
> > 
> >  
> > 
> >  
> > 
> >  
> > 
> > 
> > 
> >  
> > 
> > Hi Samuel,
> > 
> >  
> > 
> > Thank you for contacting Microsoft.
> > 
> >  
> > 
> > My name is Alex and I am the Microsoft Support Engineer from APGC
> > who will be working with you on this support request . You can
> > reach me using the contact information in my signature.
> > 
> >  
> > 
> > I just called you at +886 3 327 2345 and had a quick discussion.
> > 
> > Issue Description
> > 
> > 1.Customer is using a 3rd-party application BitBake to clone Azure
> > DevOps repo. 
> > 
> > 2.Customer could fetch file from https url successfully.
> > 
> > 3.Customer wants to know whether Azure DevOps support SSH
> > authentication method.
> > 
> >  
> > 
> > Analysis
> > 
> > 1.Yes, Azure DevOps has SSH authentication method. (Reference: 
> > https://docs.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops
> > )
> > 
> >  Create your SSH keys
> >     Create your SSH keys with the ssh-keygen command from
> > the bash prompt.
> > 
> >  Add the public key to your Azure DevOps Organization
> > Open your security settings by browsing to the web portal and
> > selecting your avatar in the upper right of the user interface.
> > Select Security in the menu that appears.
> > 
> > 
> > 
> > Select SSH public keys , then select +New Key.
> > 
> > 
> > 
> > Copy the contents of the public key (for example, id_rsa.pub) that
> > you generated into the Public Key Data field.
> > 
> >  
> > 
> > 
> > 
> >  
> > 
> >  Clone the Git repository with SSH
> > Copy the SSH clone URL from the web portal. In this example the SSL
> > clone URL is for a repo in an organization named fabrikam-fiber, as
> > indicated by the first part of the URL after dev.azure.com.
> > 
> > 
> > 
> > Run git clone from the command prompt.
> > 
> > My example is git clone v-chucho-microsoft@vs-ssh.visualstudio.com:
> > v3/v-chucho-microsoft/testCodeCoverage/testCodeCoverage
> > 
> >  
> > 
> > 2.Regarding to BitBake. (Reference: 
> > https://www.yoctoproject.org/docs/1.8/bitbake-user-manual/bitbake-user-manual.html#bb-fetchers
> > )
> > 
> > This is a 3rd-party application which is out of our Microsoft
> > support. Our Azure DevOps has SSH authentication method. But I have
> > no experience whether there is such option in BitBake. 
> > 
> > I did some research on this application and find Git fetcher and
> > ssh:// fetcher functionalities. In my understanding, it should
> > support git with SSH or SSH fecher directly. It's suggested to
> > consult BitBake support for more details of these 2 fetchers. If
> > there is something our Microsoft could help, please feel free to
> > involve me directly.
> > 
> > 
> > 
> > 
> > 
> >  
> > 
> > If you have any concern or query, please feel free to contact me.
> > 
> >  
> > 
> > Best Regards,
> > 
> >  
> > 
> > Alex Chong
> > 
> >  
> > 
> > Support Engineer
> > 
> > Microsoft APAC Developer Support Team
> > 
> > Customer Service & Support (CSS)
> > 
> > Email: v-chucho@microsoft.com
> > 
> > Office: +86 (21) 52638610
> > 
> > Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
> > 
> > Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
> > 
> >  
> > 
> > 這則來自 Microsoft 的訊息是您或貴公司所購買或參與之計畫、服務或產品的重要部分。請檢閱我們的線上隱私權聲明
> > 
> > Microsoft Corporation
> > One Microsoft Way
> > Redmond, WA, USA 98052
> > 
> > 
> > 

Re: bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

[Samuel Jiang (江騏先)]

Hi Richard,

I try use 
SRC_URI = "git://quanta01@vs-ssh.visualstudio.com/v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobranch=1"

the bitbake response below error message:
 git -c core.fsyncobjectfiles=0 ls-remote 
ssh://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump
  failed with exit code 128, output:

ssh: Could not resolve hostname vs-ssh.visualstudio.com:v3: Name or 
 service not known
fatal: Could not read from remote repository.

I discussed with Microsoft support team,
the Azure DevOps support SSH url clone without [ ssh:// ] prefix, like below:
git -c core.fsyncobjectfiles=0 ls-remote 
quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump

Does bitbake support git SRC_URI without any prefix or need extra with Azure SSH protocol support it?

Thanks,
Samuel Jiang


-----Original Message-----
From: Richard Purdie <richard.purdie@linuxfoundation.org> 
Sent: Tuesday, November 5, 2019 11:30 PM
To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>; yocto@yoctoproject.org
Cc: Alex Chong <v-chucho@microsoft.com>
Subject: Re: [yocto] bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

On Mon, 2019-11-04 at 02:52 +0000, Samuel Jiang (江騏先) wrote:
> Dear yocto developer,
> The below disscussed about azure devops clone issue including test 
> result with Microsoft support team member.
> We wonder know how Bitbake access repository through SSH. Does it 
> different between git and Azue DevOps?


Try:

SRC_URI = "git://quanta01@vs-ssh.visualstudio.com/v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobranch=1"

as I think you didn't convert the url to our syntax correctly.

Cheers,

Richard


> Thanks,
> 
> Samuel Jiang
> ---------- Forwarded message ----------
> From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) < 
> v-chucho@microsoft.com>
> Date: 2019年10月31日 PM5:39 [+0800]
> To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
> Cc: support <support@mail.support.microsoft.com>, Alex Chong (Shanghai 
> Wicresoft Co,.Ltd.) <v-chucho@microsoft.com>
> Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure 
> DevOps repository Azure DevOps Services Basic
> 
> > Hi Samuel,
> > 
> >  
> > 
> > Thank you for reply.
> > 
> >  
> > 
> > I test and read the BitBake doc, let me share with you my finding 
> > and analysis.
> > 
> > I have verified that our Azure DevOps Organization Repo could be 
> > clone through SSH authentication successfully.
> > I create a new SSH key and add it to Azure DevOps SSH public keys.
> >  
> > 
> > Then I use git clone command and clone this repo successfully. The 
> > url is “v-chucho-microsoft@vs-ssh.visualstudio.com:v3/v-chucho-
> > microsoft/testCodeCoverage/testCodeCoverage”
> > 
> > 
> > My testing verify that SSH Authentication is available, and the url 
> > given by Azure DevOps is correct.
> > 
> >  
> > 
> > Then I also test it in GitHub, and succeed. The only difference is 
> > the url of GitHub is “git@github.com:xxxxxxxxxxxxxxxxxx.git”
> >  
> > 
> > So let’s come back to BitBake. I read more of the BitBake doc.( 
> > https://www.yoctoproject.org/docs/1.8/bitbake-user-manual/bitbake-us
> > er-manual.html#bb-fetchers
> > ) . Your command SRC_URI = "git://quanta01@vs- 
> > ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump;protocol=ssh;nob
> > ranch=1". The BitBake example is below.
> > 
> > 
> > We can see that the BitBake syntax is like git SSH but not Azure 
> > DevOps SSH. Do you try to use this command clone a repo from GitHub?
> > 
> >  
> > 
> > Then I read the doc you shared. The customer also met your issue 
> > when trying to clone Azure DevOps repo with SSH url. In my 
> > understanding, seems BitBake Git fetcher is good for Git SSH but not 
> > ready to access the Azure DevOps SSH.
> > 
> > 
> >  
> > 
> > Action Plan
> > 
> > Could you involve BitBake support team? We can deliver this testing 
> > result and consult them how BitBake access Azure DevOps through SSH.
> > 
> >  
> > 
> > If you have any conern or query, please feel free to let me know.
> > 
> >  
> > 
> > Best Regards,
> > 
> >  
> > 
> > Alex Chong
> > 
> >  
> > 
> > 
> > 
> > Support Engineer
> > 
> > Microsoft APAC Developer Support Team
> > 
> > Customer Service & Support (CSS)
> > 
> > Email: v-chucho@microsoft.com
> > 
> > Office: +86 (21) 52638610
> > 
> > Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
> > 
> > Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
> > 
> >  
> > 
> > From: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
> > Sent: Thursday, October 31, 2019 12:16 PM
> > To: Alex Chong (Shanghai Wicresoft Co,.Ltd.) < 
> > v-chucho@microsoft.com>
> > Cc: support <support@mail.support.microsoft.com>
> > Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure 
> > DevOps repository Azure DevOps Services Basic
> > 
> >  
> > 
> > Hi Alex,
> > 
> > I catch same problem on the yocto mail list
> > link: 
> > https://lists.yoctoproject.org/pipermail/yocto/2018-October/042736.h
> > tml
> > 
> > when I use SRC_URI = "git://quanta01@vs- 
> > ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump;protocol=ssh;nob
> > ranch=1"
> > 
> > the bitbake response below error message:
> > git -c core.fsyncobjectfiles=0 ls-remote 
> > ssh://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump
> >   failed with exit code 128, output:
> > 
> > ssh: Could not resolve hostname vs-ssh.visualstudio.com:v3: Name or 
> > service not known
> > 
> > fatal: Could not read from remote repository.
> > 
> > I think the bitbake call “git ls-remote” command however it could 
> > not parser below host vs-ssh.visualstudio.com:v3
> > 
> > Could you help confirm the  “git ls-remote”command could work on ssh 
> > method clone with ssh:// prefix?
> > 
> > BTW, the protocol will define the uri prefix, I didn’t find any way 
> > removing this prefix.
> > 
> > Thanks,
> > Samuel Jiang
> > 
> >  
> > 
> > From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) < 
> > v-chucho@microsoft.com>
> > Sent: Thursday, October 31, 2019 10:32 AM
> > To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>
> > Cc: Alex Chong (Shanghai Wicresoft Co,.Ltd.) < 
> > v-chucho@microsoft.com>; support < 
> > support@mail.support.microsoft.com>
> > Subject: RE: 119102923000220 use yocto bitbake SRC_URI fetch Azure 
> > DevOps repository Azure DevOps Services Basic
> > 
> >  
> > 
> > Hi Samuel,
> > 
> >  
> > 
> > How are you?
> > 
> >  
> > 
> > Do you try our Azure DevOps SSH authentication? Do you contact 
> > BitBake support for help? Is there any suggestion or new issue?
> > Please feel free to contact me if there is something I can do for 
> > you.
> > 
> >  
> > 
> > Best Regards,
> > 
> >  
> > 
> > Alex Chong
> > 
> >  
> > 
> > 
> > 
> > Support Engineer
> > 
> > Microsoft APAC Developer Support Team
> > 
> > Customer Service & Support (CSS)
> > 
> > Email: v-chucho@microsoft.com
> > 
> > Office: +86 (21) 52638610
> > 
> > Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
> > 
> > Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
> > 
> >  
> > 
> > From: Alex Chong (Shanghai Wicresoft Co,.Ltd.) < 
> > v-chucho@microsoft.com>
> > Sent: Tuesday, October 29, 2019 3:50 PM
> > To: Samuel.Jiang@quantatw.com
> > Cc: Alex Chong (Shanghai Wicresoft Co,.Ltd.) < 
> > v-chucho@microsoft.com>
> > Subject: 119102923000220 use yocto bitbake SRC_URI fetch Azure 
> > DevOps repository Azure DevOps Services Basic
> > 
> >  
> > 
> >  
> > 
> >  
> > 
> > 
> > 
> >  
> > 
> > Hi Samuel,
> > 
> >  
> > 
> > Thank you for contacting Microsoft.
> > 
> >  
> > 
> > My name is Alex and I am the Microsoft Support Engineer from APGC 
> > who will be working with you on this support request . You can reach 
> > me using the contact information in my signature.
> > 
> >  
> > 
> > I just called you at +886 3 327 2345 and had a quick discussion.
> > 
> > Issue Description
> > 
> > 1.Customer is using a 3rd-party application BitBake to clone Azure 
> > DevOps repo.
> > 
> > 2.Customer could fetch file from https url successfully.
> > 
> > 3.Customer wants to know whether Azure DevOps support SSH 
> > authentication method.
> > 
> >  
> > 
> > Analysis
> > 
> > 1.Yes, Azure DevOps has SSH authentication method. (Reference: 
> > https://docs.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys
> > -to-authenticate?view=azure-devops
> > )
> > 
> >  Create your SSH keys
> >     Create your SSH keys with the ssh-keygen command from the bash 
> > prompt.
> > 
> >  Add the public key to your Azure DevOps Organization Open your 
> > security settings by browsing to the web portal and selecting your 
> > avatar in the upper right of the user interface.
> > Select Security in the menu that appears.
> > 
> > 
> > 
> > Select SSH public keys , then select +New Key.
> > 
> > 
> > 
> > Copy the contents of the public key (for example, id_rsa.pub) that 
> > you generated into the Public Key Data field.
> > 
> >  
> > 
> > 
> > 
> >  
> > 
> >  Clone the Git repository with SSH
> > Copy the SSH clone URL from the web portal. In this example the SSL 
> > clone URL is for a repo in an organization named fabrikam-fiber, as 
> > indicated by the first part of the URL after dev.azure.com.
> > 
> > 
> > 
> > Run git clone from the command prompt.
> > 
> > My example is git clone v-chucho-microsoft@vs-ssh.visualstudio.com:
> > v3/v-chucho-microsoft/testCodeCoverage/testCodeCoverage
> > 
> >  
> > 
> > 2.Regarding to BitBake. (Reference: 
> > https://www.yoctoproject.org/docs/1.8/bitbake-user-manual/bitbake-us
> > er-manual.html#bb-fetchers
> > )
> > 
> > This is a 3rd-party application which is out of our Microsoft 
> > support. Our Azure DevOps has SSH authentication method. But I have 
> > no experience whether there is such option in BitBake.
> > 
> > I did some research on this application and find Git fetcher and 
> > ssh:// fetcher functionalities. In my understanding, it should 
> > support git with SSH or SSH fecher directly. It's suggested to 
> > consult BitBake support for more details of these 2 fetchers. If 
> > there is something our Microsoft could help, please feel free to 
> > involve me directly.
> > 
> > 
> > 
> > 
> > 
> >  
> > 
> > If you have any concern or query, please feel free to contact me.
> > 
> >  
> > 
> > Best Regards,
> > 
> >  
> > 
> > Alex Chong
> > 
> >  
> > 
> > Support Engineer
> > 
> > Microsoft APAC Developer Support Team
> > 
> > Customer Service & Support (CSS)
> > 
> > Email: v-chucho@microsoft.com
> > 
> > Office: +86 (21) 52638610
> > 
> > Time zone: (UTC+08:00) Beijing, Chongqing, Hong Kong, Urumqi
> > 
> > Working time: 9:00am-6:00pm, Mon-Tue-Wed-Thu-Fri
> > 
> >  
> > 
> > 這則來自 Microsoft 的訊息是您或貴公司所購買或參與之計畫、服務或產品的重要部分。請檢閱我們的線上隱私權聲明
> > 
> > Microsoft Corporation
> > One Microsoft Way
> > Redmond, WA, USA 98052
> > 
> > 
> > 

Re: bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

[Richard Purdie]

On Wed, 2019-11-13 at 03:06 +0000, Samuel Jiang (江騏先) wrote:
> I try use 
> SRC_URI = "git://quanta01@vs-
> ssh.visualstudio.com/v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobra
> nch=1"
> 
> the bitbake response below error message:
>  git -c core.fsyncobjectfiles=0 ls-remote 
> ssh://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump
>   failed with exit code 128, output:
> 
> ssh: Could not resolve hostname vs-ssh.visualstudio.com:v3: Name or 
>  service not known
> fatal: Could not read from remote repository.
> 
> I discussed with Microsoft support team,
> the Azure DevOps support SSH url clone without [ ssh:// ] prefix,
> like below:
> git -c core.fsyncobjectfiles=0 ls-remote 
> quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump
> 
> Does bitbake support git SRC_URI without any prefix or need extra
> with Azure SSH protocol support it?

Have you tried:

SRC_URI = "git://quanta01@vs-ssh.visualstudio.com/v3/quanta01/OpenBMC/crashdump;protocol=git;nobranch=1"

(changing protocol to git)
?

Cheers,

Richard

Re: bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

[Samuel Jiang (江騏先)]

I got below error message:
crashdump-git-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export SSH_AGENT_PID="2255"; export SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export PATH="[private_data]"; export HOME="/home/samueljiang";
git -c core.fsyncobjectfiles=0 ls-remote git://quanta01.visualstudio.com/OpenBMC/_git/crashdump  failed with exit code 128, output:
fatal: unable to connect to quanta01.visualstudio.com:
quanta01.visualstudio.com[0: 2620:1ec:21::18]: errno=Network is unreachable
quanta01.visualstudio.com[1: 13.107.42.18]: errno=Connection timed out

Seems this url with git:// prefix could not connect in Azure DevOps 

Document[https://docs.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops].

Thanks,
Samuel Jiang


-----Original Message-----
From: Richard Purdie <richard.purdie@linuxfoundation.org> 
Sent: Wednesday, November 13, 2019 4:10 PM
To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>; yocto@yoctoproject.org
Cc: Alex Chong <v-chucho@microsoft.com>
Subject: Re: [yocto] bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

On Wed, 2019-11-13 at 03:06 +0000, Samuel Jiang (江騏先) wrote:
> I try use
> SRC_URI = "git://quanta01@vs-
> ssh.visualstudio.com/v3/quanta01/OpenBMC/crashdump;protocol=ssh;nobra
> nch=1"
> 
> the bitbake response below error message:
>  git -c core.fsyncobjectfiles=0 ls-remote 
> ssh://quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump
>   failed with exit code 128, output:
> 
> ssh: Could not resolve hostname vs-ssh.visualstudio.com:v3: Name or  
> service not known
> fatal: Could not read from remote repository.
> 
> I discussed with Microsoft support team, the Azure DevOps support SSH 
> url clone without [ ssh:// ] prefix, like below:
> git -c core.fsyncobjectfiles=0 ls-remote 
> quanta01@vs-ssh.visualstudio.com:v3/quanta01/OpenBMC/crashdump
> 
> Does bitbake support git SRC_URI without any prefix or need extra with 
> Azure SSH protocol support it?

Have you tried:

SRC_URI = "git://quanta01@vs-ssh.visualstudio.com/v3/quanta01/OpenBMC/crashdump;protocol=git;nobranch=1"

(changing protocol to git)
?

Cheers,

Richard

Re: bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

[Richard Purdie]

On Wed, 2019-11-13 at 09:02 +0000, Samuel Jiang (江騏先) wrote:
> I got below error message:
> crashdump-git-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export SSH_AGENT_PID="2255"; export SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export PATH="[private_data]"; export HOME="/home/samueljiang";
> git -c core.fsyncobjectfiles=0 ls-remote git://quanta01.visualstudio.com/OpenBMC/_git/crashdump  failed with exit code 128, output:
> fatal: unable to connect to quanta01.visualstudio.com:
> quanta01.visualstudio.com[0: 2620:1ec:21::18]: errno=Network is unreachable
> quanta01.visualstudio.com[1: 13.107.42.18]: errno=Connection timed out
> 
> Seems this url with git:// prefix could not connect in Azure DevOps 
> 
> Document[https://docs.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops].

I now realise what the problem is. Whatever this git server you're
talking to is, it does not work the same as is documented for the "git
clone" manpage.

There is no option to force bitbake to use the syntax you're asking for
since the manpage says the synax its using is valid. You could patch
bitbake to force it to use that syntax with a patch like:

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index fa41b078f12..f20c1f36b82 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -588,6 +588,8 @@ class Git(FetchMethod):
             username = ud.user + '@'
         else:
             username = ""
+        if ud.proto == "ssh":
+            return "%s%s:%s" % (username, ud.host, ud.path[1:])
         return "%s://%s%s%s" % (ud.proto, username, ud.host, ud.path)
 
     def _revision_key(self, ud, d, name):


which in my local tests appeared to give the result you're asking for
but that shouldn't be required with a server that adheres to what the
git manual says is supported.

Cheers,

Richard

Re: bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

[Samuel Jiang (江騏先)]

Hi Richard,
Thanks for your patch which could work for us.
I study git-clone document and found the git urls support alternative scp-like syntax like below:

[user@]host.xz:path/to/repo.git/

It seems like Azure DevOps provides SSH url.
Will yocto support this alternative url?

Thanks, 
Samuel Jiang

-----Original Message-----
From: Richard Purdie <richard.purdie@linuxfoundation.org> 
Sent: Thursday, November 14, 2019 2:52 AM
To: Samuel Jiang (江騏先) <Samuel.Jiang@quantatw.com>; yocto@yoctoproject.org
Cc: Alex Chong <v-chucho@microsoft.com>
Subject: Re: [yocto] bitbake SRC_URI fetch Azure DevOps repository Azure DevOps Services Basic

On Wed, 2019-11-13 at 09:02 +0000, Samuel Jiang (江騏先) wrote:
> I got below error message:
> crashdump-git-r0 do_fetch: Fetcher failure: Fetch command export 
> PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1000/bus"; export SSH_AGENT_PID="2255"; export SSH_AUTH_SOCK="/run/user/1000/keyring/ssh"; export PATH="[private_data]"; export HOME="/home/samueljiang"; git -c core.fsyncobjectfiles=0 ls-remote git://quanta01.visualstudio.com/OpenBMC/_git/crashdump  failed with exit code 128, output:
> fatal: unable to connect to quanta01.visualstudio.com:
> quanta01.visualstudio.com[0: 2620:1ec:21::18]: errno=Network is 
> unreachable
> quanta01.visualstudio.com[1: 13.107.42.18]: errno=Connection timed out
> 
> Seems this url with git:// prefix could not connect in Azure DevOps
> 
> Document[https://docs.microsoft.com/en-us/azure/devops/repos/git/use-ssh-keys-to-authenticate?view=azure-devops].

I now realise what the problem is. Whatever this git server you're talking to is, it does not work the same as is documented for the "git clone" manpage.

There is no option to force bitbake to use the syntax you're asking for since the manpage says the synax its using is valid. You could patch bitbake to force it to use that syntax with a patch like:

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py index fa41b078f12..f20c1f36b82 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -588,6 +588,8 @@ class Git(FetchMethod):
             username = ud.user + '@'
         else:
             username = ""
+        if ud.proto == "ssh":
+            return "%s%s:%s" % (username, ud.host, ud.path[1:])
         return "%s://%s%s%s" % (ud.proto, username, ud.host, ud.path)
 
     def _revision_key(self, ud, d, name):


which in my local tests appeared to give the result you're asking for but that shouldn't be required with a server that adheres to what the git manual says is supported.

Cheers,

Richard