Source Code provision without full git repository history

Source Code provision without full git repository history

[Waldvogel, Sebastian]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

Hello,

I am searching for a way in Yocto/BitBake to extract the source codes of the software applications developed by my company for delivery to our customers.
The source codes shall be delivered as archive containing only the source files without the whole git repository history.

Setup is that we have own Yocto layers/recipes referencing the company internal git repository (SRC_URI="git://git.company.url") and the concrete SRCREV to be used. Basic idea was to use the option BB_GENERATE_MIRROR_TARBALLS = "1" which generates an archive file with the application source code. Unfortunately the generated tarball contains the full bare cloned git repository with full history and not only the source files of the concrete SRCREV.

As alternative I tried to use the archiver class<http://www.yoctoproject.org/docs/2.0.1/dev-manual/dev-manual.html#providing-the-source-code>. The generated tarballs contains the concrete source files but in addition the full git repository with history is contained too.

Does anyone has an idea how to extract the concrete source files without git repository and history?

Thank you very much for your help
Sebastian Waldvogel



[-- Attachment #2: Type: text/html, Size: 3360 bytes --]

Re: Source Code provision without full git repository history

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]

On Thu, Mar 9, 2017 at 5:02 AM Waldvogel, Sebastian <
Sebastian.Waldvogel@vector.com> wrote:

> Hello,
>
>
>
> I am searching for a way in Yocto/BitBake to extract the source codes of
> the software applications developed by my company for delivery to our
> customers.
>
> The source codes shall be delivered as archive containing only the source
> files without the whole git repository history.
>
>
>
> Setup is that we have own Yocto layers/recipes referencing the company
> internal git repository (SRC_URI=”git://git.company.url”) and the concrete
> SRCREV to be used. Basic idea was to use the option
> BB_GENERATE_MIRROR_TARBALLS = "1" which generates an archive file with the
> application source code. Unfortunately the generated tarball contains the
> full bare cloned git repository with full history and not only the source
> files of the concrete SRCREV.
>
>
>
> As alternative I tried to use the archiver class
> <http://www.yoctoproject.org/docs/2.0.1/dev-manual/dev-manual.html#providing-the-source-code>.
> The generated tarballs contains the concrete source files but in addition
> the full git repository with history is contained too.
>

I am not sure if it is intended to keep SCM history though you might want
to file a bug

>
>
> Does anyone has an idea how to extract the concrete source files without
> git repository and history?
>
>
>
> Thank you very much for your help
>
> Sebastian Waldvogel
>
>
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

[-- Attachment #2: Type: text/html, Size: 4025 bytes --]

Re: Source Code provision without full git repository history

[Daniel.]

I'm not trying to be simplistic here, but, why not package the source
releases to a tarball (with git archive or somthing) and provide it
through a FTP server?

Regards,

2017-03-09 11:29 GMT-03:00 Khem Raj <raj.khem@gmail.com>:
>
> On Thu, Mar 9, 2017 at 5:02 AM Waldvogel, Sebastian
> <Sebastian.Waldvogel@vector.com> wrote:
>>
>> Hello,
>>
>>
>>
>> I am searching for a way in Yocto/BitBake to extract the source codes of
>> the software applications developed by my company for delivery to our
>> customers.
>>
>> The source codes shall be delivered as archive containing only the source
>> files without the whole git repository history.
>>
>>
>>
>> Setup is that we have own Yocto layers/recipes referencing the company
>> internal git repository (SRC_URI=”git://git.company.url”) and the concrete
>> SRCREV to be used. Basic idea was to use the option
>> BB_GENERATE_MIRROR_TARBALLS = "1" which generates an archive file with the
>> application source code. Unfortunately the generated tarball contains the
>> full bare cloned git repository with full history and not only the source
>> files of the concrete SRCREV.
>>
>>
>>
>> As alternative I tried to use the archiver class. The generated tarballs
>> contains the concrete source files but in addition the full git repository
>> with history is contained too.
>
>
> I am not sure if it is intended to keep SCM history though you might want to
> file a bug
>>
>>
>>
>> Does anyone has an idea how to extract the concrete source files without
>> git repository and history?
>>
>>
>>
>> Thank you very much for your help
>>
>> Sebastian Waldvogel
>>
>>
>>
>>
>>
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>



-- 
“If you're going to try, go all the way. Otherwise, don't even start. ..."
  Charles Bukowski

Re: Source Code provision without full git repository history

[Daniel.]

There are some tools to automate the testing and delivery that may
make the deployment less painfull. Jenkins and Buildbot are examples

Regards,

2017-03-09 13:28 GMT-03:00 Daniel. <danielhilst@gmail.com>:
> I'm not trying to be simplistic here, but, why not package the source
> releases to a tarball (with git archive or somthing) and provide it
> through a FTP server?
>
> Regards,
>
> 2017-03-09 11:29 GMT-03:00 Khem Raj <raj.khem@gmail.com>:
>>
>> On Thu, Mar 9, 2017 at 5:02 AM Waldvogel, Sebastian
>> <Sebastian.Waldvogel@vector.com> wrote:
>>>
>>> Hello,
>>>
>>>
>>>
>>> I am searching for a way in Yocto/BitBake to extract the source codes of
>>> the software applications developed by my company for delivery to our
>>> customers.
>>>
>>> The source codes shall be delivered as archive containing only the source
>>> files without the whole git repository history.
>>>
>>>
>>>
>>> Setup is that we have own Yocto layers/recipes referencing the company
>>> internal git repository (SRC_URI=”git://git.company.url”) and the concrete
>>> SRCREV to be used. Basic idea was to use the option
>>> BB_GENERATE_MIRROR_TARBALLS = "1" which generates an archive file with the
>>> application source code. Unfortunately the generated tarball contains the
>>> full bare cloned git repository with full history and not only the source
>>> files of the concrete SRCREV.
>>>
>>>
>>>
>>> As alternative I tried to use the archiver class. The generated tarballs
>>> contains the concrete source files but in addition the full git repository
>>> with history is contained too.
>>
>>
>> I am not sure if it is intended to keep SCM history though you might want to
>> file a bug
>>>
>>>
>>>
>>> Does anyone has an idea how to extract the concrete source files without
>>> git repository and history?
>>>
>>>
>>>
>>> Thank you very much for your help
>>>
>>> Sebastian Waldvogel
>>>
>>>
>>>
>>>
>>>
>>> --
>>> _______________________________________________
>>> yocto mailing list
>>> yocto@yoctoproject.org
>>> https://lists.yoctoproject.org/listinfo/yocto
>>
>>
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>>
>
>
>
> --
> “If you're going to try, go all the way. Otherwise, don't even start. ..."
>   Charles Bukowski



-- 
“If you're going to try, go all the way. Otherwise, don't even start. ..."
  Charles Bukowski

Re: Source Code provision without full git repository history

[Waldvogel, Sebastian]

The point is that we want to deliver the Yocto build environment including the layers/recipes and the download directory (DL_DIR) to our customers as well.
The download directory shall contain an archive of the software modules / applications developed by my company. The customer has no access to the company's git repository.

We try to use as much as possible default functionality of Yocto and avoid developing additional delivery tools around Yocto.
A workaround would be manual extraction of the generated tarballs (including full git repo) and creating a new archive with git archive containing only the source files of the desired git revision.
In addition the SRC_URI of the related recipe need to be overwritten  by an .bbappend pointing to the tarball instead of the company git repo.

My favorite solution would be an additional attribute in the SRC_URI to control the behavior of the mirror tarball generation. E.g. SRC_URI = " git://git.company.url;nogithistory=1"

Best regards
Sebastian

>-----Original Message-----
>From: Daniel. [mailto:danielhilst@gmail.com]
>Sent: Thursday, March 09, 2017 5:28 PM
>To: Khem Raj
>Cc: Waldvogel, Sebastian; yocto@yoctoproject.org
>Subject: Re: [yocto] Source Code provision without full git repository history
>
>I'm not trying to be simplistic here, but, why not package the source
>releases to a tarball (with git archive or somthing) and provide it
>through a FTP server?
>
>Regards,
>
>2017-03-09 11:29 GMT-03:00 Khem Raj <raj.khem@gmail.com>:
>>
>> On Thu, Mar 9, 2017 at 5:02 AM Waldvogel, Sebastian
>> <Sebastian.Waldvogel@vector.com> wrote:
>>>
>>> Hello,
>>>
>>>
>>>
>>> I am searching for a way in Yocto/BitBake to extract the source codes of
>>> the software applications developed by my company for delivery to our
>>> customers.
>>>
>>> The source codes shall be delivered as archive containing only the source
>>> files without the whole git repository history.
>>>
>>>
>>>
>>> Setup is that we have own Yocto layers/recipes referencing the company
>>> internal git repository (SRC_URI=”git://git.company.url”) and the concrete
>>> SRCREV to be used. Basic idea was to use the option
>>> BB_GENERATE_MIRROR_TARBALLS = "1" which generates an archive file
>with the
>>> application source code. Unfortunately the generated tarball contains the
>>> full bare cloned git repository with full history and not only the source
>>> files of the concrete SRCREV.
>>>
>>>
>>>
>>> As alternative I tried to use the archiver class. The generated tarballs
>>> contains the concrete source files but in addition the full git repository
>>> with history is contained too.
>>
>>
>> I am not sure if it is intended to keep SCM history though you might want to
>> file a bug
>>>
>>>
>>>
>>> Does anyone has an idea how to extract the concrete source files without
>>> git repository and history?
>>>
>>>
>>>
>>> Thank you very much for your help
>>>
>>> Sebastian Waldvogel
>>>
>>>
>>>
>>>
>>>
>>> --
>>> _______________________________________________
>>> yocto mailing list
>>> yocto@yoctoproject.org
>>> https://lists.yoctoproject.org/listinfo/yocto
>>
>>
>> --
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>>
>
>
>
>--
>“If you're going to try, go all the way. Otherwise, don't even start. ..."
>  Charles Bukowski