Size limit in the cip-kernel-sec repository?

Size limit in the cip-kernel-sec repository?

[Masami Ichikawa]

Hi !

When I push commits to the cip-kernel-sec repository
(https://gitlab.com/cip-project/cip-kernel/cip-kernel-sec), I get the
following message.

$ git push cip master
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 4 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 457 bytes | 457.00 KiB/s, done.
Total 4 (delta 3), reused 0 (delta 0)
remote:
remote: ========================================================================
remote:
remote:    Your push has been rejected, because this repository has exceeded
remote:     its size limit of 0 Bytes by 400 MB. Please contact your GitLab
remote:                   administrator for more information.
remote:
remote: ========================================================================
remote:
To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
   78850bfd..92d679f5  master -> master

Even Though, I get the message, commits are pushed successfully.
I can see commits in the gitlab's UI.

Do we need something in the repository setting?

Regards,
-- 
Masami Ichikawa
Cybertrust Japan Co., Ltd.

Email :masami.ichikawa@cybertrust.co.jp
          :masami.ichikawa@miraclelinux.com

Re: Size limit in the cip-kernel-sec repository?

[Jan Kiszka]

On 09.05.23 01:34, Masami Ichikawa wrote:
> Hi !
> 
> When I push commits to the cip-kernel-sec repository
> (https://gitlab.com/cip-project/cip-kernel/cip-kernel-sec), I get the
> following message.
> 
> $ git push cip master
> Enumerating objects: 7, done.
> Counting objects: 100% (7/7), done.
> Delta compression using up to 4 threads
> Compressing objects: 100% (4/4), done.
> Writing objects: 100% (4/4), 457 bytes | 457.00 KiB/s, done.
> Total 4 (delta 3), reused 0 (delta 0)
> remote:
> remote: ========================================================================
> remote:
> remote:    Your push has been rejected, because this repository has exceeded
> remote:     its size limit of 0 Bytes by 400 MB. Please contact your GitLab
> remote:                   administrator for more information.
> remote:
> remote: ========================================================================
> remote:
> To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
>    78850bfd..92d679f5  master -> master
> 
> Even Though, I get the message, commits are pushed successfully.
> I can see commits in the gitlab's UI.
> 
> Do we need something in the repository setting?
> 

This issue affects the complete cip-project. The reason is likely

https://gitlab.com/cip-project/cip-testing/linux-stable-rc

which hit the 10G limit, see

https://gitlab.com/groups/cip-project/-/usage_quotas#storage-quota-tab

Folks, can we avoid this?

Chris, didn't we apply for an OSS project rank on gitlab.com?

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: Size limit in the cip-kernel-sec repository?

[Jan Kiszka]

On 09.05.23 08:39, Jan Kiszka wrote:
> On 09.05.23 01:34, Masami Ichikawa wrote:
>> Hi !
>>
>> When I push commits to the cip-kernel-sec repository
>> (https://gitlab.com/cip-project/cip-kernel/cip-kernel-sec), I get the
>> following message.
>>
>> $ git push cip master
>> Enumerating objects: 7, done.
>> Counting objects: 100% (7/7), done.
>> Delta compression using up to 4 threads
>> Compressing objects: 100% (4/4), done.
>> Writing objects: 100% (4/4), 457 bytes | 457.00 KiB/s, done.
>> Total 4 (delta 3), reused 0 (delta 0)
>> remote:
>> remote: ========================================================================
>> remote:
>> remote:    Your push has been rejected, because this repository has exceeded
>> remote:     its size limit of 0 Bytes by 400 MB. Please contact your GitLab
>> remote:                   administrator for more information.
>> remote:
>> remote: ========================================================================
>> remote:
>> To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
>>    78850bfd..92d679f5  master -> master
>>
>> Even Though, I get the message, commits are pushed successfully.
>> I can see commits in the gitlab's UI.
>>
>> Do we need something in the repository setting?
>>
> 
> This issue affects the complete cip-project. The reason is likely
> 
> https://gitlab.com/cip-project/cip-testing/linux-stable-rc
> 
> which hit the 10G limit, see
> 
> https://gitlab.com/groups/cip-project/-/usage_quotas#storage-quota-tab
> 

Wait - I only looked a the repo column, but there is way more,
specifically under "Jobs". Are these artifacts?

Jan

> Folks, can we avoid this?
> 
> Chris, didn't we apply for an OSS project rank on gitlab.com?
> 
> Jan
> 

-- 
Siemens AG, Technology
Competence Center Embedded Linux

RE: Size limit in the cip-kernel-sec repository?

[Chris Paterson]

Hello Jan,

> From: Jan Kiszka <jan.kiszka@siemens.com>
> Sent: Tuesday, May 9, 2023 7:39 AM
> 
> On 09.05.23 01:34, Masami Ichikawa wrote:
> > Hi !
> >
> > When I push commits to the cip-kernel-sec repository
> >
> (https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitla
> b.com%2Fcip-project%2Fcip-kernel%2Fcip-kernel-
> sec&data=05%7C01%7CChris.Paterson2%40renesas.com%7Ccf0d089f5f4f424
> 1aa6f08db5058242b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C
> 638192111707052389%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
> DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=Cjxru6HLSW5Kq%2BwYCwcuzgmvfKwr4G4x4V8jbdakiT8%3D&reser
> ved=0), I get the
> > following message.
> >
> > $ git push cip master
> > Enumerating objects: 7, done.
> > Counting objects: 100% (7/7), done.
> > Delta compression using up to 4 threads
> > Compressing objects: 100% (4/4), done.
> > Writing objects: 100% (4/4), 457 bytes | 457.00 KiB/s, done.
> > Total 4 (delta 3), reused 0 (delta 0)
> > remote:
> > remote:
> ==========================================================
> ==============
> > remote:
> > remote:    Your push has been rejected, because this repository has
> exceeded
> > remote:     its size limit of 0 Bytes by 400 MB. Please contact your GitLab
> > remote:                   administrator for more information.
> > remote:
> > remote:
> ==========================================================
> ==============
> > remote:
> > To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
> >    78850bfd..92d679f5  master -> master
> >
> > Even Though, I get the message, commits are pushed successfully.
> > I can see commits in the gitlab's UI.
> >
> > Do we need something in the repository setting?
> >
> 
> This issue affects the complete cip-project. The reason is likely
> 
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab
> .com%2Fcip-project%2Fcip-testing%2Flinux-stable-
> rc&data=05%7C01%7CChris.Paterson2%40renesas.com%7Ccf0d089f5f4f4241
> aa6f08db5058242b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6
> 38192111707052389%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD
> AiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C
> &sdata=8YzAuSt5tXmv6m61DlGl8KZCTfvNXGyD%2FrAfTs8bS1c%3D&reserve
> d=0
> 
> which hit the 10G limit, see
> 
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab
> .com%2Fgroups%2Fcip-project%2F-%2Fusage_quotas%23storage-quota-
> tab&data=05%7C01%7CChris.Paterson2%40renesas.com%7Ccf0d089f5f4f424
> 1aa6f08db5058242b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C
> 638192111707052389%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
> DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=6TajRY6KeI9VMENausm3UpzfY5EggnVtjKcHYvVaPXg%3D&reserved
> =0
> 
> Folks, can we avoid this?
> 
> Chris, didn't we apply for an OSS project rank on gitlab.com?

We did!
And according to the billings page [0] this isn't due to expire until 22/06/23.
[0] https://gitlab.com/groups/cip-project/-/billings

I don't know if there is a storage limit even for GitLab ultimate users though.

Actually, according to [1] there is a 10GB limit for a git repository, but it hints that this setup may be changing to "namespace" storage limits
[1] https://about.gitlab.com/pricing/faq-paid-storage-transfer/#q-what-is-changing-with-storage-and-transfer-limits

I'll create a support ticket with GitLab and see if they can provide some clarification.

linux-stable-rc is a mirror of the kernel.org repo. There isn't much we can do to reduce the size.
If we can't get the quota increased we'll have to go back to using kernel.org directly again and remove our mirror.

Regards, Chris

> 
> Jan
> 
> --
> Siemens AG, Technology
> Competence Center Embedded Linux

Re: Size limit in the cip-kernel-sec repository?

[Masami Ichikawa]

Hi,

On Tue, May 9, 2023 at 7:39 PM Chris Paterson
<Chris.Paterson2@renesas.com> wrote:
>
> Hello Jan,
>
> > From: Jan Kiszka <jan.kiszka@siemens.com>
> > Sent: Tuesday, May 9, 2023 7:39 AM
> >
> > On 09.05.23 01:34, Masami Ichikawa wrote:
> > > Hi !
> > >
> > > When I push commits to the cip-kernel-sec repository
> > >
> > (https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitla
> > b.com%2Fcip-project%2Fcip-kernel%2Fcip-kernel-
> > sec&data=05%7C01%7CChris.Paterson2%40renesas.com%7Ccf0d089f5f4f424
> > 1aa6f08db5058242b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C
> > 638192111707052389%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
> > DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> > C&sdata=Cjxru6HLSW5Kq%2BwYCwcuzgmvfKwr4G4x4V8jbdakiT8%3D&reser
> > ved=0), I get the
> > > following message.
> > >
> > > $ git push cip master
> > > Enumerating objects: 7, done.
> > > Counting objects: 100% (7/7), done.
> > > Delta compression using up to 4 threads
> > > Compressing objects: 100% (4/4), done.
> > > Writing objects: 100% (4/4), 457 bytes | 457.00 KiB/s, done.
> > > Total 4 (delta 3), reused 0 (delta 0)
> > > remote:
> > > remote:
> > ==========================================================
> > ==============
> > > remote:
> > > remote:    Your push has been rejected, because this repository has
> > exceeded
> > > remote:     its size limit of 0 Bytes by 400 MB. Please contact your GitLab
> > > remote:                   administrator for more information.
> > > remote:
> > > remote:
> > ==========================================================
> > ==============
> > > remote:
> > > To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
> > >    78850bfd..92d679f5  master -> master
> > >
> > > Even Though, I get the message, commits are pushed successfully.
> > > I can see commits in the gitlab's UI.
> > >
> > > Do we need something in the repository setting?
> > >
> >
> > This issue affects the complete cip-project. The reason is likely
> >
> > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab
> > .com%2Fcip-project%2Fcip-testing%2Flinux-stable-
> > rc&data=05%7C01%7CChris.Paterson2%40renesas.com%7Ccf0d089f5f4f4241
> > aa6f08db5058242b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6
> > 38192111707052389%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMD
> > AiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C
> > &sdata=8YzAuSt5tXmv6m61DlGl8KZCTfvNXGyD%2FrAfTs8bS1c%3D&reserve
> > d=0
> >
> > which hit the 10G limit, see
> >
> > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab
> > .com%2Fgroups%2Fcip-project%2F-%2Fusage_quotas%23storage-quota-
> > tab&data=05%7C01%7CChris.Paterson2%40renesas.com%7Ccf0d089f5f4f424
> > 1aa6f08db5058242b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C
> > 638192111707052389%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
> > DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> > C&sdata=6TajRY6KeI9VMENausm3UpzfY5EggnVtjKcHYvVaPXg%3D&reserved
> > =0
> >
> > Folks, can we avoid this?
> >
> > Chris, didn't we apply for an OSS project rank on gitlab.com?
>
> We did!
> And according to the billings page [0] this isn't due to expire until 22/06/23.
> [0] https://gitlab.com/groups/cip-project/-/billings
>
> I don't know if there is a storage limit even for GitLab ultimate users though.
>
> Actually, according to [1] there is a 10GB limit for a git repository, but it hints that this setup may be changing to "namespace" storage limits
> [1] https://about.gitlab.com/pricing/faq-paid-storage-transfer/#q-what-is-changing-with-storage-and-transfer-limits
>
> I'll create a support ticket with GitLab and see if they can provide some clarification.
>
> linux-stable-rc is a mirror of the kernel.org repo. There isn't much we can do to reduce the size.
> If we can't get the quota increased we'll have to go back to using kernel.org directly again and remove our mirror.
>
> Regards, Chris
>
> >
> > Jan
> >
> > --
> > Siemens AG, Technology
> > Competence Center Embedded Linux
>

The warning message disappeared today (Tue 09 May 2023 11:09:07 PM UTC).

Enumerating objects: 12, done.
Counting objects: 100% (12/12), done.
Delta compression using up to 4 threads
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 1.02 KiB | 1.02 MiB/s, done.
Total 8 (delta 6), reused 0 (delta 0)
To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
   92d679f5..ef1cf1be  master -> master

Thanks alot.

Regards,
-- 
Masami Ichikawa
Cybertrust Japan Co., Ltd.

Email :masami.ichikawa@cybertrust.co.jp
          :masami.ichikawa@miraclelinux.com

RE: Size limit in the cip-kernel-sec repository?

[Chris Paterson]

Hello,

> From: Masami Ichikawa <masami.ichikawa@miraclelinux.com>
> Sent: Wednesday, May 10, 2023 12:12 AM

[...]

> >
> 
> The warning message disappeared today (Tue 09 May 2023 11:09:07 PM
> UTC).
> 
> Enumerating objects: 12, done.
> Counting objects: 100% (12/12), done.
> Delta compression using up to 4 threads
> Compressing objects: 100% (8/8), done.
> Writing objects: 100% (8/8), 1.02 KiB | 1.02 MiB/s, done.
> Total 8 (delta 6), reused 0 (delta 0)
> To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
>    92d679f5..ef1cf1be  master -> master

Huzzah.
FYI I went through and removed all the tags and branches we don't care about in our linux-stable-rc mirror, and set the mirror to only pull branches we care about.
So the repo is now only 4.1GB :)

In the meantime GitLab got back to me and basically said there is no way to increase the storage quotas, unless CIP wants to pay.

Kind regards, Chris

Re: Size limit in the cip-kernel-sec repository?

[Jan Kiszka]

On 10.05.23 08:58, Chris Paterson wrote:
> Hello,
> 
>> From: Masami Ichikawa <masami.ichikawa@miraclelinux.com>
>> Sent: Wednesday, May 10, 2023 12:12 AM
> 
> [...]
> 
>>>
>>
>> The warning message disappeared today (Tue 09 May 2023 11:09:07 PM
>> UTC).
>>
>> Enumerating objects: 12, done.
>> Counting objects: 100% (12/12), done.
>> Delta compression using up to 4 threads
>> Compressing objects: 100% (8/8), done.
>> Writing objects: 100% (8/8), 1.02 KiB | 1.02 MiB/s, done.
>> Total 8 (delta 6), reused 0 (delta 0)
>> To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
>>    92d679f5..ef1cf1be  master -> master
> 
> Huzzah.
> FYI I went through and removed all the tags and branches we don't care about in our linux-stable-rc mirror, and set the mirror to only pull branches we care about.
> So the repo is now only 4.1GB :)
> 
> In the meantime GitLab got back to me and basically said there is no way to increase the storage quotas, unless CIP wants to pay.
> 

So the issue was really only this single over-sized repo, not all the
other artifacts we have and that accumulate to hundreds of GB?

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

RE: Size limit in the cip-kernel-sec repository?

[Chris Paterson]

> From: Jan Kiszka <jan.kiszka@siemens.com>
> Sent: Wednesday, May 10, 2023 8:02 AM
> 
> On 10.05.23 08:58, Chris Paterson wrote:
> > Hello,
> >
> >> From: Masami Ichikawa <masami.ichikawa@miraclelinux.com>
> >> Sent: Wednesday, May 10, 2023 12:12 AM
> >
> > [...]
> >
> >>>
> >>
> >> The warning message disappeared today (Tue 09 May 2023 11:09:07 PM
> >> UTC).
> >>
> >> Enumerating objects: 12, done.
> >> Counting objects: 100% (12/12), done.
> >> Delta compression using up to 4 threads
> >> Compressing objects: 100% (8/8), done.
> >> Writing objects: 100% (8/8), 1.02 KiB | 1.02 MiB/s, done.
> >> Total 8 (delta 6), reused 0 (delta 0)
> >> To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
> >>    92d679f5..ef1cf1be  master -> master
> >
> > Huzzah.
> > FYI I went through and removed all the tags and branches we don't care
> about in our linux-stable-rc mirror, and set the mirror to only pull branches
> we care about.
> > So the repo is now only 4.1GB :)
> >
> > In the meantime GitLab got back to me and basically said there is no way to
> increase the storage quotas, unless CIP wants to pay.
> >
> 
> So the issue was really only this single over-sized repo, not all the
> other artifacts we have and that accumulate to hundreds of GB?

Yep!

Re: Size limit in the cip-kernel-sec repository?

[Masami Ichikawa]

Hi.

On Wed, May 10, 2023 at 3:58 PM Chris Paterson
<Chris.Paterson2@renesas.com> wrote:
>
> Hello,
>
> > From: Masami Ichikawa <masami.ichikawa@miraclelinux.com>
> > Sent: Wednesday, May 10, 2023 12:12 AM
>
> [...]
>
> > >
> >
> > The warning message disappeared today (Tue 09 May 2023 11:09:07 PM
> > UTC).
> >
> > Enumerating objects: 12, done.
> > Counting objects: 100% (12/12), done.
> > Delta compression using up to 4 threads
> > Compressing objects: 100% (8/8), done.
> > Writing objects: 100% (8/8), 1.02 KiB | 1.02 MiB/s, done.
> > Total 8 (delta 6), reused 0 (delta 0)
> > To gitlab.com:cip-project/cip-kernel/cip-kernel-sec.git
> >    92d679f5..ef1cf1be  master -> master
>
> Huzzah.
> FYI I went through and removed all the tags and branches we don't care about in our linux-stable-rc mirror, and set the mirror to only pull branches we care about.
> So the repo is now only 4.1GB :)
>

Thank you for reducing the repository size.

> In the meantime GitLab got back to me and basically said there is no way to increase the storage quotas, unless CIP wants to pay.
>
> Kind regards, Chris

Regards,
-- 
Masami Ichikawa
Cybertrust Japan Co., Ltd.

Email :masami.ichikawa@cybertrust.co.jp
          :masami.ichikawa@miraclelinux.com