All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: max_loop limit
@ 2007-03-22 23:23 devzero
  2007-03-23  8:59 ` Tomas M
  0 siblings, 1 reply; 21+ messages in thread
From: devzero @ 2007-03-22 23:23 UTC (permalink / raw)
  To: linux-kernel

wondering that here are 13 postings about loopdevice limitation, but nobody giving any comment about dm-loop ( http://sources.redhat.com/lvm2/wiki/DMLoop ), which is a solution for this problem ......

tomas, you should spend that money to bryn! ;)

regards
roland


> -----Ursprüngliche Nachricht-----
> Von: devzero@web.de
> Gesendet: 22.03.07 14:53:19
> An: linux-kernel@vger.kernel.org
> Betreff: Re: max_loop limit

> oh - i forgot sending this to the list, since this was copy&paste via webmailer.....
> 
> 
> > -----Ursprüngliche Nachricht-----
> > Von: devzero@web.de
> > Gesendet: 22.03.07 14:42:45
> > An: tomas@slax.org
> > CC: breeves@redhat.com
> > Betreff: Re: max_loop limit
> 
> > Hi Tomas, 
> > 
> > you`re completely right.
> > 
> > I have had this problem of loopdev number limitation for years, but i think there is a better solution besides your patch.
> > 
> > Some new module has been created for this and being announced on dm-devel mailinglist : 
> > 
> > dm-loop - the device mapper loopback target.
> > 
> > See http://sources.redhat.com/lvm2/wiki/DMLoop  for further information.
> > 
> > It can be used as a 1:1 replacement for classic loop and should (?) probably be ready for mainline in the not too far future. (i cannot tell, but it works good for me!)
> > 
> > Typically, you need to use dm-setup to setup device-mapper targets, but dm-setup has got support for dm-loop,  so it`s as easy as 1-2-3 to replace "losetup ...." with "dmlosetup" alias for dm-setup.
> > 
> > Feel free to test it and give feedback !
> > 
> > regards
> > Roland
> > 
> > ps:
> > dm-loop-config.patch is being linked wrong in the wiki - this is the right one:  http://sources.redhat.com/lvm2/wiki/DMLoop?action=AttachFile&do=get&target=dm-loop-config.patch
> > 
> > 
> > 
> > > 255 loop devices are insufficient? What kind of scenario do you have
> > > in mind?
> > > 
> > > 
> > 
> > Thank you very much for replying.
> > 
> > In 1981, Bill Gates said that 64KB of memory is enough for everybody.
> > And you know how much RAM do you have right now. :)
> > 
> > Every limit is bad. The limit of 255 loop devices has been introduced 
> > years ago, in the times when minor device number has been limited by 
> > 255. Nowadays, there is no such limitation.
> > 
> > There are many possible/reasonable uses for more than 255 loop devices. 
> > For example CD/ISO server. My project, Slax Linux live, is based on 
> > modular approach where many parts of the root filesystem are stored 
> > separately in compressed read-only loop files, and are mounted and 
> > unioned to a single root by using union fs (aufs).
> > 
> > The question is not "Why do we need more than 255 loops?".
> > The question should be "Why do we need the hardcoded 255-limit in kernel 
> > while there is no reason for it at all?"
> > 
> > My patch simply removes the hardcoded limitation.
> > 
> > 
> > Tomas M
> > slax.org
> 
> 


_______________________________________________________________
SMS schreiben mit WEB.DE FreeMail - einfach, schnell und
kostenguenstig. Jetzt gleich testen! http://f.web.de/?mc=021192


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 23:23 max_loop limit devzero
@ 2007-03-23  8:59 ` Tomas M
  0 siblings, 0 replies; 21+ messages in thread
From: Tomas M @ 2007-03-23  8:59 UTC (permalink / raw)
  To: linux-kernel

> wondering that here are 13 postings about loopdevice limitation, but
> nobody giving any comment about dm-loop (
> http://sources.redhat.com/lvm2/wiki/DMLoop ), which is a solution for
> this problem ......

If I understand it correctly, I would need 'dm' in kernel (or module) 
and moreover I would need some userspace utilities/binaries.

 > It can be used as a 1:1 replacement for classic loop and
 > should (?) probably be ready for mainline in the not too far future.

So it is not in Kernel?

With current loop, people would need kernel patch and *no* userspace 
utilities.
With dm-loop, people would need kernel patch *and* userspace utilities.
So, from my point of view, it would be better to fix/enhance Linux's loop.


Tomas M
slax.org

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 23:37 roland
@ 2007-03-29 14:20 ` Bill Davidsen
  0 siblings, 0 replies; 21+ messages in thread
From: Bill Davidsen @ 2007-03-29 14:20 UTC (permalink / raw)
  To: roland; +Cc: linux-kernel

roland wrote:
> partitions on loop or device-mapper devices ?
> 
> you can use kpartx tool for this.
> 
> bryn m. reeves told me, that it's probably poissible to create udev 
> rules that will automatically create partition maps when a new loop 
> device is setup, which is better than adding partitioning logig into 
> dm-loop for example.

It is certainly possible to create a partitionable RAID device from a 
loop device. Should be possible to use nbd as well, but I can't seem to 
get nbd to work on 2.6.21-rc (my working system runs 2.6.17).
> 
> example:
>> kpartx -a /dev/mapper/loop0
>>
>> # ls /dev/mapper/loop0*
>> /dev/mapper/loop0  /dev/mapper/loop0p1  /dev/mapper/loop0p2
>> /dev/mapper/loop0p3
> 
> i have seen a patch for loop.c doing this, though. search the archives 
> for this
> 
> regards
> roland
> 
> 
> 
> 
> 
> On Thu, Mar 22, 2007 at 02:33:14PM +0000, Al Viro wrote:
>> Correction: current ABI is crap.  To set the thing up you need to open
>> it and issue an ioctl.  Which is a bloody bad idea, for obvious 
>> reasons...
> 
> Agreed.  What would be a right way?  Global device ala ptmx/tun/tap?
> New syscall?  Something else?
> 
>  OG.
> -
> ]


-- 
Bill Davidsen <davidsen@tmr.com>
   "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 11:37   ` Tomas M
  2007-03-22 13:42     ` Eric Dumazet
@ 2007-03-29 14:16     ` Bill Davidsen
  1 sibling, 0 replies; 21+ messages in thread
From: Bill Davidsen @ 2007-03-29 14:16 UTC (permalink / raw)
  To: Tomas M; +Cc: linux-kernel

Tomas M wrote:
>> 255 loop devices are insufficient? What kind of scenario do you have
>> in mind?
>>
>>
> 
> Thank you very much for replying.
> 
> In 1981, Bill Gates said that 64KB of memory is enough for everybody.
> And you know how much RAM do you have right now. :)
> 
Actually, I believe the number was 640k, the quote included the phrase 
"should be," the available memory on the IBM PC. And this was after IBM 
decided to put the video adapter in memory at 640k, Intel decided to 
provide only 1MB of address space on the 8086, and was in the context of 
mainframes of the day, some of which could only address 1MB.

And having run clients with three users on an XT with just that 640kB 
and UNIX, I don't think he was wrong about the memory for that time, 
just the O/S.

BTW: anyone got a copy of PC/IX (SysIII for XT) around? I'd love to run 
that in a VM just for the comparison.

-- 
Bill Davidsen <davidsen@tmr.com>
   "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 16:09               ` Pádraig Brady
@ 2007-03-28 23:34                 ` Karel Zak
  0 siblings, 0 replies; 21+ messages in thread
From: Karel Zak @ 2007-03-28 23:34 UTC (permalink / raw)
  To: Pádraig Brady
  Cc: William Lee Irwin III, Jens Axboe, Eric Dumazet, Tomas M, linux-kernel

On Thu, Mar 22, 2007 at 04:09:13PM +0000, Pádraig Brady wrote:
> William Lee Irwin III wrote:
> > Any chance we can get some kind of devices set up for partitions of
> > loop devices if we're going to redo loopdev setup? That's been a thorn
> > in my side for some time.
> 
> This script might be of use:
> http://www.pixelbeat.org/scripts/lomount.sh

 Ah, lomount... very popular name ;-) Xen guys have lomount too.
 
 Unfortunately, these solution are useless with LVM volumes. The
 kpartx is more usable:

 http://fedoraproject.org/wiki/FedoraXenQuickstartFC6?highlight=%28Xen%29#head-9c5408e750e8184aece3efe822be0ef6dd1871cd

    Karel

-- 
 Karel Zak  <kzak@redhat.com>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 13:42     ` Eric Dumazet
  2007-03-22 13:42       ` Jens Axboe
  2007-03-22 14:25       ` Tomas M
@ 2007-03-23  1:34       ` Jan Engelhardt
  2 siblings, 0 replies; 21+ messages in thread
From: Jan Engelhardt @ 2007-03-23  1:34 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: Tomas M, linux-kernel


On Mar 22 2007 14:42, Eric Dumazet wrote:
>Instead of using :
>
>static struct loop_device *loop_dev;
>loop_dev = kmalloc(max_loop * sizeof(struct loop_device));
>
>Switch to :
>
>static struct loop_device **loop_dev;
>loop_dev = kmalloc(max_loop * sizeof(void *));
>if (!loop_dev) rollback...
>for (i = 0 ; i < max_loop ; i++) {
>	loop_dev[i] = kmalloc(sizeof(struct loop_device));
>	if (!loop_dev[i]) rollback...
>}
>
>This time, you would be limited to 16384 loop devices on x86_64, 32768 on i386
>:)

Oh noes. Please use a linked list (kmalloc cope = perfect) if you really need
loads of loopdevs. Sorta

 struct loopdev {
    struct list_head lh;
    int lo_number;
 };

to keep the /dev/loop%d number consistent across loopdev removal.
Maybe it's better to even use an rbtree (linked list does not scale to it).


Jan
-- 

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
@ 2007-03-22 23:37 roland
  2007-03-29 14:20 ` Bill Davidsen
  0 siblings, 1 reply; 21+ messages in thread
From: roland @ 2007-03-22 23:37 UTC (permalink / raw)
  To: linux-kernel

partitions on loop or device-mapper devices ?

you can use kpartx tool for this.

bryn m. reeves told me, that it's probably poissible to create udev rules 
that will automatically create partition maps when a new loop device is 
setup, which is better than adding partitioning logig into dm-loop for 
example.

example:
>kpartx -a /dev/mapper/loop0
>
># ls /dev/mapper/loop0*
>/dev/mapper/loop0  /dev/mapper/loop0p1  /dev/mapper/loop0p2
>/dev/mapper/loop0p3

i have seen a patch for loop.c doing this, though. search the archives for 
this

regards
roland





On Thu, Mar 22, 2007 at 02:33:14PM +0000, Al Viro wrote:
> Correction: current ABI is crap.  To set the thing up you need to open
> it and issue an ioctl.  Which is a bloody bad idea, for obvious reasons...

Agreed.  What would be a right way?  Global device ala ptmx/tun/tap?
New syscall?  Something else?

  OG.
-
] 


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 14:33         ` Al Viro
@ 2007-03-22 19:51           ` Olivier Galibert
  0 siblings, 0 replies; 21+ messages in thread
From: Olivier Galibert @ 2007-03-22 19:51 UTC (permalink / raw)
  To: Al Viro; +Cc: linux-kernel

On Thu, Mar 22, 2007 at 02:33:14PM +0000, Al Viro wrote:
> Correction: current ABI is crap.  To set the thing up you need to open
> it and issue an ioctl.  Which is a bloody bad idea, for obvious reasons...

Agreed.  What would be a right way?  Global device ala ptmx/tun/tap?
New syscall?  Something else?

  OG.

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 14:11             ` William Lee Irwin III
  2007-03-22 15:22               ` Arjan van de Ven
@ 2007-03-22 16:09               ` Pádraig Brady
  2007-03-28 23:34                 ` Karel Zak
  1 sibling, 1 reply; 21+ messages in thread
From: Pádraig Brady @ 2007-03-22 16:09 UTC (permalink / raw)
  To: William Lee Irwin III; +Cc: Jens Axboe, Eric Dumazet, Tomas M, linux-kernel

William Lee Irwin III wrote:
> Any chance we can get some kind of devices set up for partitions of
> loop devices if we're going to redo loopdev setup? That's been a thorn
> in my side for some time.

This script might be of use:
http://www.pixelbeat.org/scripts/lomount.sh

cheers,
Pádraig.

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 14:11             ` William Lee Irwin III
@ 2007-03-22 15:22               ` Arjan van de Ven
  2007-03-22 16:09               ` Pádraig Brady
  1 sibling, 0 replies; 21+ messages in thread
From: Arjan van de Ven @ 2007-03-22 15:22 UTC (permalink / raw)
  To: William Lee Irwin III; +Cc: Jens Axboe, Eric Dumazet, Tomas M, linux-kernel

On Thu, 2007-03-22 at 07:11 -0700, William Lee Irwin III wrote:
> On Thu, Mar 22 2007, Eric Dumazet wrote:
> >> Sure, but it's the first Tomas patch :)
> 
> On Thu, Mar 22, 2007 at 02:54:57PM +0100, Jens Axboe wrote:
> > The more the reason to guide him in the direction of a right solution,
> > instead of extending the current bad one!
> 
> On Thu, Mar 22 2007, Eric Dumazet wrote:
> >> Apparently the 'current crap' didnt caugth someone else attention.
> 
> On Thu, Mar 22, 2007 at 02:54:57PM +0100, Jens Axboe wrote:
> > I guess most people don't care, 8 is enough for them and the wasted
> > memory isn't too much to care about - 8 devices and only one used, is
> > wasting at least ~14kb on my machine here.
> 
> Any chance we can get some kind of devices set up for partitions of
> loop devices if we're going to redo loopdev setup? That's been a thorn
> in my side for some time.

you can already do that with devmapper and partx combo...



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 13:42       ` Jens Axboe
  2007-03-22 13:52         ` Eric Dumazet
@ 2007-03-22 14:33         ` Al Viro
  2007-03-22 19:51           ` Olivier Galibert
  1 sibling, 1 reply; 21+ messages in thread
From: Al Viro @ 2007-03-22 14:33 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Eric Dumazet, Tomas M, linux-kernel

On Thu, Mar 22, 2007 at 02:42:31PM +0100, Jens Axboe wrote:
> But this still wastes memory, why not just allocate each loop device
> dynamically when it is set up? The current approach is crap, it is just
> wasting memory for loop devices, queues, etc.

Correction: current ABI is crap.  To set the thing up you need to open
it and issue an ioctl.  Which is a bloody bad idea, for obvious reasons...

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 13:42     ` Eric Dumazet
  2007-03-22 13:42       ` Jens Axboe
@ 2007-03-22 14:25       ` Tomas M
  2007-03-23  1:34       ` Jan Engelhardt
  2 siblings, 0 replies; 21+ messages in thread
From: Tomas M @ 2007-03-22 14:25 UTC (permalink / raw)
  To: linux-kernel

> You might want a more radical patch : 

I agree that my patch is not the perfect solution for max_loop problem.
But it nearly doubles max_loop for me (using 386 arch) and moreover it 
is a FIX for incorrect implementation in kernel IMHO. So I can see 
REASON to include it in Kernel. Do I cry at the correct tomb? :)

> 
> Instead of using :
> ::
> Switch to :
> ::

I'm not any professional kernel hacker, so I don't understand the 
mysteries regarding ** (pointers to pointers?). Is there anyone who 
could provide CLEAN patch for loop.c, which would raise the max_loop 
limit to (at least) 1024 and which would be ACCEPTED to mainline kernel 
any soon?

I'm offering MONEY for this task.
Let's say $256 ;-)
I hope I didn't offend anyone by this offer.



Tomas M
slax.org

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 13:54           ` Jens Axboe
@ 2007-03-22 14:11             ` William Lee Irwin III
  2007-03-22 15:22               ` Arjan van de Ven
  2007-03-22 16:09               ` Pádraig Brady
  0 siblings, 2 replies; 21+ messages in thread
From: William Lee Irwin III @ 2007-03-22 14:11 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Eric Dumazet, Tomas M, linux-kernel

On Thu, Mar 22 2007, Eric Dumazet wrote:
>> Sure, but it's the first Tomas patch :)

On Thu, Mar 22, 2007 at 02:54:57PM +0100, Jens Axboe wrote:
> The more the reason to guide him in the direction of a right solution,
> instead of extending the current bad one!

On Thu, Mar 22 2007, Eric Dumazet wrote:
>> Apparently the 'current crap' didnt caugth someone else attention.

On Thu, Mar 22, 2007 at 02:54:57PM +0100, Jens Axboe wrote:
> I guess most people don't care, 8 is enough for them and the wasted
> memory isn't too much to care about - 8 devices and only one used, is
> wasting at least ~14kb on my machine here.

Any chance we can get some kind of devices set up for partitions of
loop devices if we're going to redo loopdev setup? That's been a thorn
in my side for some time.


-- wli

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 13:52         ` Eric Dumazet
@ 2007-03-22 13:54           ` Jens Axboe
  2007-03-22 14:11             ` William Lee Irwin III
  0 siblings, 1 reply; 21+ messages in thread
From: Jens Axboe @ 2007-03-22 13:54 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: Tomas M, linux-kernel

On Thu, Mar 22 2007, Eric Dumazet wrote:
> On Thu, 22 Mar 2007 14:42:31 +0100
> Jens Axboe <jens.axboe@oracle.com> wrote:
> 
> 
> > > This time, you would be limited to 16384 loop devices on x86_64, 32768 on i386 :)
> > 
> > But this still wastes memory, why not just allocate each loop device
> > dynamically when it is set up? The current approach is crap, it is just
> > wasting memory for loop devices, queues, etc.
> > 
> 
> Sure, but it's the first Tomas patch :)

The more the reason to guide him in the direction of a right solution,
instead of extending the current bad one!

> Apparently the 'current crap' didnt caugth someone else attention.

I guess most people don't care, 8 is enough for them and the wasted
memory isn't too much to care about - 8 devices and only one used, is
wasting at least ~14kb on my machine here.

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
@ 2007-03-22 13:53 devzero
  0 siblings, 0 replies; 21+ messages in thread
From: devzero @ 2007-03-22 13:53 UTC (permalink / raw)
  To: linux-kernel

oh - i forgot sending this to the list, since this was copy&paste via webmailer.....


> -----Ursprüngliche Nachricht-----
> Von: devzero@web.de
> Gesendet: 22.03.07 14:42:45
> An: tomas@slax.org
> CC: breeves@redhat.com
> Betreff: Re: max_loop limit

> Hi Tomas, 
> 
> you`re completely right.
> 
> I have had this problem of loopdev number limitation for years, but i think there is a better solution besides your patch.
> 
> Some new module has been created for this and being announced on dm-devel mailinglist : 
> 
> dm-loop - the device mapper loopback target.
> 
> See http://sources.redhat.com/lvm2/wiki/DMLoop  for further information.
> 
> It can be used as a 1:1 replacement for classic loop and should (?) probably be ready for mainline in the not too far future. (i cannot tell, but it works good for me!)
> 
> Typically, you need to use dm-setup to setup device-mapper targets, but dm-setup has got support for dm-loop,  so it`s as easy as 1-2-3 to replace "losetup ...." with "dmlosetup" alias for dm-setup.
> 
> Feel free to test it and give feedback !
> 
> regards
> Roland
> 
> ps:
> dm-loop-config.patch is being linked wrong in the wiki - this is the right one:  http://sources.redhat.com/lvm2/wiki/DMLoop?action=AttachFile&do=get&target=dm-loop-config.patch
> 
> 
> 
> > 255 loop devices are insufficient? What kind of scenario do you have
> > in mind?
> > 
> > 
> 
> Thank you very much for replying.
> 
> In 1981, Bill Gates said that 64KB of memory is enough for everybody.
> And you know how much RAM do you have right now. :)
> 
> Every limit is bad. The limit of 255 loop devices has been introduced 
> years ago, in the times when minor device number has been limited by 
> 255. Nowadays, there is no such limitation.
> 
> There are many possible/reasonable uses for more than 255 loop devices. 
> For example CD/ISO server. My project, Slax Linux live, is based on 
> modular approach where many parts of the root filesystem are stored 
> separately in compressed read-only loop files, and are mounted and 
> unioned to a single root by using union fs (aufs).
> 
> The question is not "Why do we need more than 255 loops?".
> The question should be "Why do we need the hardcoded 255-limit in kernel 
> while there is no reason for it at all?"
> 
> My patch simply removes the hardcoded limitation.
> 
> 
> Tomas M
> slax.org


_______________________________________________________________
SMS schreiben mit WEB.DE FreeMail - einfach, schnell und
kostenguenstig. Jetzt gleich testen! http://f.web.de/?mc=021192


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 13:42       ` Jens Axboe
@ 2007-03-22 13:52         ` Eric Dumazet
  2007-03-22 13:54           ` Jens Axboe
  2007-03-22 14:33         ` Al Viro
  1 sibling, 1 reply; 21+ messages in thread
From: Eric Dumazet @ 2007-03-22 13:52 UTC (permalink / raw)
  To: Jens Axboe; +Cc: Tomas M, linux-kernel

On Thu, 22 Mar 2007 14:42:31 +0100
Jens Axboe <jens.axboe@oracle.com> wrote:


> > This time, you would be limited to 16384 loop devices on x86_64, 32768 on i386 :)
> 
> But this still wastes memory, why not just allocate each loop device
> dynamically when it is set up? The current approach is crap, it is just
> wasting memory for loop devices, queues, etc.
> 

Sure, but it's the first Tomas patch :)

Apparently the 'current crap' didnt caugth someone else attention.

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 13:42     ` Eric Dumazet
@ 2007-03-22 13:42       ` Jens Axboe
  2007-03-22 13:52         ` Eric Dumazet
  2007-03-22 14:33         ` Al Viro
  2007-03-22 14:25       ` Tomas M
  2007-03-23  1:34       ` Jan Engelhardt
  2 siblings, 2 replies; 21+ messages in thread
From: Jens Axboe @ 2007-03-22 13:42 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: Tomas M, linux-kernel

On Thu, Mar 22 2007, Eric Dumazet wrote:
> On Thu, 22 Mar 2007 12:37:54 +0100
> Tomas M <tomas@slax.org> wrote:
> 
> > The question is not "Why do we need more than 255 loops?".
> > The question should be "Why do we need the hardcoded 255-limit in kernel 
> > while there is no reason for it at all?"
> > 
> > My patch simply removes the hardcoded limitation.
> 
> Hello Tomas, welcome !
> 
> Well, its an attempt to remove a hardcoded limit, but as you said in the Changelog, it really depends on kmalloc() being able to allocate a large continous memory zone. Alas it might fail.
> The golden rule is to avoid all allocations larger than PAGE_SIZE :)
> 
> On x86_64, sizeof(struct loop_device) is 368, so the 'new limit' would be 356 instead of 256...
> 
> You might want a more radical patch : 
> 
> Instead of using :
> 
> static struct loop_device *loop_dev;
> loop_dev = kmalloc(max_loop * sizeof(struct loop_device));
> 
> Switch to :
> 
> static struct loop_device **loop_dev;
> loop_dev = kmalloc(max_loop * sizeof(void *));
> if (!loop_dev) rollback...
> for (i = 0 ; i < max_loop ; i++) {
> 	loop_dev[i] = kmalloc(sizeof(struct loop_device));
> 	if (!loop_dev[i]) rollback...
> }
> 
> This time, you would be limited to 16384 loop devices on x86_64, 32768 on i386 :)

But this still wastes memory, why not just allocate each loop device
dynamically when it is set up? The current approach is crap, it is just
wasting memory for loop devices, queues, etc.

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 11:37   ` Tomas M
@ 2007-03-22 13:42     ` Eric Dumazet
  2007-03-22 13:42       ` Jens Axboe
                         ` (2 more replies)
  2007-03-29 14:16     ` Bill Davidsen
  1 sibling, 3 replies; 21+ messages in thread
From: Eric Dumazet @ 2007-03-22 13:42 UTC (permalink / raw)
  To: Tomas M; +Cc: linux-kernel

On Thu, 22 Mar 2007 12:37:54 +0100
Tomas M <tomas@slax.org> wrote:

> The question is not "Why do we need more than 255 loops?".
> The question should be "Why do we need the hardcoded 255-limit in kernel 
> while there is no reason for it at all?"
> 
> My patch simply removes the hardcoded limitation.

Hello Tomas, welcome !

Well, its an attempt to remove a hardcoded limit, but as you said in the Changelog, it really depends on kmalloc() being able to allocate a large continous memory zone. Alas it might fail.
The golden rule is to avoid all allocations larger than PAGE_SIZE :)

On x86_64, sizeof(struct loop_device) is 368, so the 'new limit' would be 356 instead of 256...

You might want a more radical patch : 

Instead of using :

static struct loop_device *loop_dev;
loop_dev = kmalloc(max_loop * sizeof(struct loop_device));

Switch to :

static struct loop_device **loop_dev;
loop_dev = kmalloc(max_loop * sizeof(void *));
if (!loop_dev) rollback...
for (i = 0 ; i < max_loop ; i++) {
	loop_dev[i] = kmalloc(sizeof(struct loop_device));
	if (!loop_dev[i]) rollback...
}

This time, you would be limited to 16384 loop devices on x86_64, 32768 on i386 :)

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22 11:00 ` markus reichelt
@ 2007-03-22 11:37   ` Tomas M
  2007-03-22 13:42     ` Eric Dumazet
  2007-03-29 14:16     ` Bill Davidsen
  0 siblings, 2 replies; 21+ messages in thread
From: Tomas M @ 2007-03-22 11:37 UTC (permalink / raw)
  To: linux-kernel

> 255 loop devices are insufficient? What kind of scenario do you have
> in mind?
> 
> 

Thank you very much for replying.

In 1981, Bill Gates said that 64KB of memory is enough for everybody.
And you know how much RAM do you have right now. :)

Every limit is bad. The limit of 255 loop devices has been introduced 
years ago, in the times when minor device number has been limited by 
255. Nowadays, there is no such limitation.

There are many possible/reasonable uses for more than 255 loop devices. 
For example CD/ISO server. My project, Slax Linux live, is based on 
modular approach where many parts of the root filesystem are stored 
separately in compressed read-only loop files, and are mounted and 
unioned to a single root by using union fs (aufs).

The question is not "Why do we need more than 255 loops?".
The question should be "Why do we need the hardcoded 255-limit in kernel 
while there is no reason for it at all?"

My patch simply removes the hardcoded limitation.


Tomas M
slax.org


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: max_loop limit
  2007-03-22  7:57 Tomas M
@ 2007-03-22 11:00 ` markus reichelt
  2007-03-22 11:37   ` Tomas M
  0 siblings, 1 reply; 21+ messages in thread
From: markus reichelt @ 2007-03-22 11:00 UTC (permalink / raw)
  To: linux-kernel

[-- Attachment #1: Type: text/plain, Size: 334 bytes --]

* Tomas M <tomas@slax.org> wrote:

> I hope you will like it and you will include it in kernel.
> Or, if not, maybe this patch will start some debate regarding
> the current insufficient limit of 255 loop devices.

255 loop devices are insufficient? What kind of scenario do you have
in mind?


-- 
left blank, right bald

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* max_loop limit
@ 2007-03-22  7:57 Tomas M
  2007-03-22 11:00 ` markus reichelt
  0 siblings, 1 reply; 21+ messages in thread
From: Tomas M @ 2007-03-22  7:57 UTC (permalink / raw)
  To: linux-kernel

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

Hello,

this is my first code submitted to kernel, I hope you won't hate it.

This 4-lines-change patch adds support for nearly two-times more loop
devices. Explanation follows:

The maximum amount of loop devices has been 255 for many years, while
there is a lot of space for more. The maximum depends on max memory
available from kmalloc(), which is usually 128KB, but can be increased
in some cases.

It would be better to support thousands of loop devices of course, but
the change could be more complicated (perhaps replace kmalloc by
vmalloc). This four lines change is just simple and sufficient, without
any need for kmalloc replacement.

I only removed the test if (max_loop > 255), so now we support much more
loop devices then before, without ANY OTHER CHANGE to the code.

Information: The maximum max_loop is 455 if kmalloc can allocate 128KB,
so the amount is nearly doubled without any significant change to kernel
code. The maximum could be even bigger I guess, it probably depends on:
NR_CPUS, MAX_NUMNODES, CONFIG_MMU and CONFIG_LARGE_ALLOCS.

If kmalloc can't allocate enough RAM, loop is simply unloaded.


Thank you for your consideration.

I hope you will like it and you will include it in kernel.
Or, if not, maybe this patch will start some debate regarding
the current insufficient limit of 255 loop devices.


Tomas M
slax.org



[-- Attachment #2: loop.c.diff --]
[-- Type: text/x-patch, Size: 1861 bytes --]

--- linux/drivers/block/loop_old.c	2007-02-04 18:44:54.000000000 +0000
+++ linux/drivers/block/loop.c	2007-03-22 08:31:55.000000000 +0000
@@ -44,6 +44,16 @@
  * backing filesystem.
  * Anton Altaparmakov, 16 Feb 2005
  *
+ * The maximum amount of loop devices has been 255 for many years, while there
+ * is a lot of space for more. The maximum depends on max memory available 
+ * from kmalloc, which is usually 128KB, but can be even more.
+ * I removed the test if (max_loop > 255), so now we support much more loop 
+ * devices then before; it probably depends on:
+ *    NR_CPUS, MAX_NUMNODES, CONFIG_MMU and CONFIG_LARGE_ALLOCS.
+ * Information: The maximum max_loop is 455 if kmalloc handles only 128KB.
+ * If kmalloc can't allocate enough RAM, loop is simply unloaded.
+ * Author: Tomas Matejicek, www.slax.org, 21 Mar 2007
+ *
  * Still To Fix:
  * - Advisory locking is ignored here.
  * - Should use an own CAP_* category instead of CAP_SYS_ADMIN
@@ -1358,7 +1368,7 @@
  * And now the modules code and kernel interface.
  */
 module_param(max_loop, int, 0);
-MODULE_PARM_DESC(max_loop, "Maximum number of loop devices (1-256)");
+MODULE_PARM_DESC(max_loop, "Maximum number of loop devices (1-455 on i386)");
 MODULE_LICENSE("GPL");
 MODULE_ALIAS_BLOCKDEV_MAJOR(LOOP_MAJOR);
 
@@ -1402,9 +1412,9 @@
 {
 	int	i;
 
-	if (max_loop < 1 || max_loop > 256) {
-		printk(KERN_WARNING "loop: invalid max_loop (must be between"
-				    " 1 and 256), using default (8)\n");
+	if (max_loop < 1) {
+		printk(KERN_WARNING "loop: invalid max_loop (must be at least 1"
+				    ", using default (8)\n");
 		max_loop = 8;
 	}
 
@@ -1465,7 +1475,7 @@
 	kfree(loop_dev);
 out_mem1:
 	unregister_blkdev(LOOP_MAJOR, "loop");
-	printk(KERN_ERR "loop: ran out of memory\n");
+	printk(KERN_ERR "loop: ran out of memory for max_loop=%d\n", max_loop);
 	return -ENOMEM;
 }
 

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2007-03-29 14:19 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-03-22 23:23 max_loop limit devzero
2007-03-23  8:59 ` Tomas M
  -- strict thread matches above, loose matches on Subject: below --
2007-03-22 23:37 roland
2007-03-29 14:20 ` Bill Davidsen
2007-03-22 13:53 devzero
2007-03-22  7:57 Tomas M
2007-03-22 11:00 ` markus reichelt
2007-03-22 11:37   ` Tomas M
2007-03-22 13:42     ` Eric Dumazet
2007-03-22 13:42       ` Jens Axboe
2007-03-22 13:52         ` Eric Dumazet
2007-03-22 13:54           ` Jens Axboe
2007-03-22 14:11             ` William Lee Irwin III
2007-03-22 15:22               ` Arjan van de Ven
2007-03-22 16:09               ` Pádraig Brady
2007-03-28 23:34                 ` Karel Zak
2007-03-22 14:33         ` Al Viro
2007-03-22 19:51           ` Olivier Galibert
2007-03-22 14:25       ` Tomas M
2007-03-23  1:34       ` Jan Engelhardt
2007-03-29 14:16     ` Bill Davidsen

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.