Limit on return string from program map

Limit on return string from program map

[Keith Owens]

Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3).  I added
"/net /etc/auto.net" to /etc/auto.master and it works fine, up to a
point.  ps shows

/usr/sbin/automount --timeout=60 /net program /etc/auto.net

If the target server has a lot of mounted filesystems then automount
will only mount the first few entries on the list.  Then it tries to
run mount with invalid input.  It looks like there is a hard coded
limit on the string that automount expects from a program map.  My
virtual CD server has lots of CD images mounted over loopback, each
with its own export entry, so the return string from auto.net is quite
long.

"/etc/auto.net virtcd" returns a long string, over 20K.  The start of
this string is below, some of the names have been sanitized without
changing their length.  irix-6.5.21-overlays-4 mounts correctly,
irix-6.5.22-applications-1 does not.  Instead of issuing

  /bin/mount -t nfs -s -o hard,intr,nodev,nosuid virtcd:/virtcd/mnt/irix-6.5.22-applications-1 /net/virtcd/virtcd/mnt/irix-6.5.22-applications-1

it issues

  /bin/mount -t nfs -s -o hard,intr,nodev,nosuid virtcd:/virtcd/mnt /net/virtcd/virtcd/mnt/irix-6.5.22-applications-1

The NFS directory string has been truncated.  Interesting that the
failure point is around the 4K mark on the input string.

-fstype=nfs,hard,intr,nodev,nosuid \
	/ virtcd:/ \
	/virtcd virtcd:/virtcd \
	/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-009.iso virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-009.iso \
	/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-1162-xxx.iso virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-1162-xxx.iso \
	/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-004.iso virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxxxxxxxxxx-004.iso \
	/virtcd/mnt/xxxxxxxxxxxxxxxx-MR-1 virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxx-MR-1 \
	/virtcd/mnt/xxxxxxxxxxxxxxxx-RC1-1 virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxx-RC1-1 \
	/virtcd/mnt/xxxxxxxxxxxxxxxx-RC2-1 virtcd:/virtcd/mnt/xxxxxxxxxxxxxxxx-RC2-1 \
	/virtcd/mnt/xxxxxxxxxxxxxx_28_05_02-1 virtcd:/virtcd/mnt/xxxxxxxxxxxxxx_28_05_02-1 \
	/virtcd/mnt/xxxxx-xxxxx-1 virtcd:/virtcd/mnt/xxxxx-xxxxx-1 \
	/virtcd/mnt/FC2-i386-disc1.iso virtcd:/virtcd/mnt/FC2-i386-disc1.iso \
	/virtcd/mnt/FC2-i386-disc2.iso virtcd:/virtcd/mnt/FC2-i386-disc2.iso \
	/virtcd/mnt/FC2-i386-disc3.iso virtcd:/virtcd/mnt/FC2-i386-disc3.iso \
	/virtcd/mnt/FC2-i386-disc4.iso virtcd:/virtcd/mnt/FC2-i386-disc4.iso \
	/virtcd/mnt/FC2-i386-rescuecd.iso virtcd:/virtcd/mnt/FC2-i386-rescuecd.iso \
	/virtcd/mnt/FC2-i386-SRPMS-disc1.iso virtcd:/virtcd/mnt/FC2-i386-SRPMS-disc1.iso \
	/virtcd/mnt/FC2-i386-SRPMS-disc2.iso virtcd:/virtcd/mnt/FC2-i386-SRPMS-disc2.iso \
	/virtcd/mnt/FC2-i386-SRPMS-disc3.iso virtcd:/virtcd/mnt/FC2-i386-SRPMS-disc3.iso \
	/virtcd/mnt/FC2-i386-SRPMS-disc4.iso virtcd:/virtcd/mnt/FC2-i386-SRPMS-disc4.iso \
	/virtcd/mnt/FC3-i386-disc1.iso virtcd:/virtcd/mnt/FC3-i386-disc1.iso \
	/virtcd/mnt/FC3-i386-disc2.iso virtcd:/virtcd/mnt/FC3-i386-disc2.iso \
	/virtcd/mnt/FC3-i386-disc3.iso virtcd:/virtcd/mnt/FC3-i386-disc3.iso \
	/virtcd/mnt/FC3-i386-disc4.iso virtcd:/virtcd/mnt/FC3-i386-disc4.iso \
	/virtcd/mnt/FC3-i386-rescuecd.iso virtcd:/virtcd/mnt/FC3-i386-rescuecd.iso \
	/virtcd/mnt/FC3-i386-SRPMS-disc1.iso virtcd:/virtcd/mnt/FC3-i386-SRPMS-disc1.iso \
	/virtcd/mnt/FC3-i386-SRPMS-disc2.iso virtcd:/virtcd/mnt/FC3-i386-SRPMS-disc2.iso \
	/virtcd/mnt/FC3-i386-SRPMS-disc3.iso virtcd:/virtcd/mnt/FC3-i386-SRPMS-disc3.iso \
	/virtcd/mnt/FC3-i386-SRPMS-disc4.iso virtcd:/virtcd/mnt/FC3-i386-SRPMS-disc4.iso \
	/virtcd/mnt/xxxxxxxx-xxxxxx-1 virtcd:/virtcd/mnt/xxxxxxxx-xxxxxx-1 \
	/virtcd/mnt/xxxxxxxx-xxxxxx-2 virtcd:/virtcd/mnt/xxxxxxxx-xxxxxx-2 \
	/virtcd/mnt/xxxxxxxx-xxxxxx-3 virtcd:/virtcd/mnt/xxxxxxxx-xxxxxx-3 \
	/virtcd/mnt/xxxxxxxx-xxxxxx-4 virtcd:/virtcd/mnt/xxxxxxxx-xxxxxx-4 \
	/virtcd/mnt/irix-6.5.18-applications-1 virtcd:/virtcd/mnt/irix-6.5.18-applications-1 \
	/virtcd/mnt/irix-6.5.18-overlays-1 virtcd:/virtcd/mnt/irix-6.5.18-overlays-1 \
	/virtcd/mnt/irix-6.5.18-overlays-2 virtcd:/virtcd/mnt/irix-6.5.18-overlays-2 \
	/virtcd/mnt/irix-6.5.18-overlays-3 virtcd:/virtcd/mnt/irix-6.5.18-overlays-3 \
	/virtcd/mnt/irix-6.5.18-overlays-4 virtcd:/virtcd/mnt/irix-6.5.18-overlays-4 \
	/virtcd/mnt/irix-6.5.19-applications-1 virtcd:/virtcd/mnt/irix-6.5.19-applications-1 \
	/virtcd/mnt/irix-6.5.19-overlays-1 virtcd:/virtcd/mnt/irix-6.5.19-overlays-1 \
	/virtcd/mnt/irix-6.5.19-overlays-2 virtcd:/virtcd/mnt/irix-6.5.19-overlays-2 \
	/virtcd/mnt/irix-6.5.19-overlays-3 virtcd:/virtcd/mnt/irix-6.5.19-overlays-3 \
	/virtcd/mnt/irix-6.5.19-overlays-4 virtcd:/virtcd/mnt/irix-6.5.19-overlays-4 \
	/virtcd/mnt/irix-6.5.20-applications-1 virtcd:/virtcd/mnt/irix-6.5.20-applications-1 \
	/virtcd/mnt/irix-6.5.20-overlays-1 virtcd:/virtcd/mnt/irix-6.5.20-overlays-1 \
	/virtcd/mnt/irix-6.5.20-overlays-2 virtcd:/virtcd/mnt/irix-6.5.20-overlays-2 \
	/virtcd/mnt/irix-6.5.20-overlays-3 virtcd:/virtcd/mnt/irix-6.5.20-overlays-3 \
	/virtcd/mnt/irix-6.5.20-overlays-4 virtcd:/virtcd/mnt/irix-6.5.20-overlays-4 \
	/virtcd/mnt/irix-6.5.21-applications-1 virtcd:/virtcd/mnt/irix-6.5.21-applications-1 \
	/virtcd/mnt/irix-6.5.21-overlays-1 virtcd:/virtcd/mnt/irix-6.5.21-overlays-1 \
	/virtcd/mnt/irix-6.5.21-overlays-2 virtcd:/virtcd/mnt/irix-6.5.21-overlays-2 \
	/virtcd/mnt/irix-6.5.21-overlays-3 virtcd:/virtcd/mnt/irix-6.5.21-overlays-3 \
	/virtcd/mnt/irix-6.5.21-overlays-4 virtcd:/virtcd/mnt/irix-6.5.21-overlays-4 \
	/virtcd/mnt/irix-6.5.22-applications-1 virtcd:/virtcd/mnt/irix-6.5.22-applications-1 \
	/virtcd/mnt/irix-6.5.22-applications-2 virtcd:/virtcd/mnt/irix-6.5.22-applications-2 \
	/virtcd/mnt/irix-6.5.22-overlays-1 virtcd:/virtcd/mnt/irix-6.5.22-overlays-1 \
	/virtcd/mnt/irix-6.5.22-overlays-2 virtcd:/virtcd/mnt/irix-6.5.22-overlays-2 \
	/virtcd/mnt/irix-6.5.22-overlays-3 virtcd:/virtcd/mnt/irix-6.5.22-overlays-3 \

	Another 16K of entries suppressed.

Re: Limit on return string from program map

[Ian Kent]

On Thu, 16 Dec 2004, Keith Owens wrote:

> Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3).  I added
> "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a
> point.  ps shows
> 
> /usr/sbin/automount --timeout=60 /net program /etc/auto.net
> 
> If the target server has a lot of mounted filesystems then automount
> will only mount the first few entries on the list.  Then it tries to
> run mount with invalid input.  It looks like there is a hard coded
> limit on the string that automount expects from a program map.  My
> virtual CD server has lots of CD images mounted over loopback, each
> with its own export entry, so the return string from auto.net is quite
> long.

The buffer size for map entries is 4096.

Ian

Re: Limit on return string from program map

[Jeff Moyer]

==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <raven@themaw.net> adds:

raven> On Thu, 16 Dec 2004, Keith Owens wrote:
>> Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3).  I added
>> "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a
>> point.  ps shows
>> 
>> /usr/sbin/automount --timeout=60 /net program /etc/auto.net
>> 
>> If the target server has a lot of mounted filesystems then automount
>> will only mount the first few entries on the list.  Then it tries to run
>> mount with invalid input.  It looks like there is a hard coded limit on
>> the string that automount expects from a program map.  My virtual CD
>> server has lots of CD images mounted over loopback, each with its own
>> export entry, so the return string from auto.net is quite long.

raven> The buffer size for map entries is 4096.

I think I posted a patch here that would extend that?  I can't seem to find
it in the archives, so I'll repost.  This is the version of the patch that
is in our tree, so no guarantees about it applying to Ian's tree.  *sigh*
(patch originally done by from Neil Horman)

Ian, when is 4.1.4 coming out.  ;-)

-Jeff

--- autofs-4.1.3/modules/lookup_program.c~	2004-01-29 11:01:22.000000000 -0500
+++ autofs-4.1.3/modules/lookup_program.c	2004-11-15 19:03:06.860316496 -0500
@@ -84,7 +84,7 @@ int lookup_ghost(const char *root, int g
 int lookup_mount(const char *root, const char *name, int name_len, void *context)
 {
 	struct lookup_context *ctxt = (struct lookup_context *) context;
-	char mapent[MAPENT_MAX_LEN + 1], *mapp;
+	char *mapent, *mapp, *tmp;
 	char errbuf[1024], *errp;
 	char ch;
 	int pipefd[2], epipefd[2];
@@ -94,11 +94,19 @@ int lookup_mount(const char *root, const
 	fd_set readfds, ourfds;
 	enum state { st_space, st_map, st_done } state;
 	int quoted = 0;
-	int ret;
+	int ret = 1;
 	int max_fd;
+	int distance;
+	int alloci = 1;
 
 	debug(MODPREFIX "looking up %s", name);
 
+	mapent = (char *)malloc(MAPENT_MAX_LEN + 1);
+	if (!mapent) {
+		error(MODPREFIX "malloc: %s\n", strerror(errno));
+		return 1;
+	}
+
 	/*
 	 * We don't use popen because we don't want to run /bin/sh plus we
 	 * want to send stderr to the syslog, and we don't use spawnl()
@@ -107,12 +115,12 @@ int lookup_mount(const char *root, const
 
 	if (pipe(pipefd)) {
 		error(MODPREFIX "pipe: %m");
-		return 1;
+		goto out_free;
 	}
 	if (pipe(epipefd)) {
 		close(pipefd[0]);
 		close(pipefd[1]);
-		return 1;
+		goto out_free;
 	}
 
 	f = fork();
@@ -122,7 +130,7 @@ int lookup_mount(const char *root, const
 		close(epipefd[0]);
 		close(epipefd[1]);
 		error(MODPREFIX "fork: %m");
-		return 1;
+		goto out_free;
 	} else if (f == 0) {
 		reset_signals();
 		close(pipefd[0]);
@@ -177,21 +185,40 @@ int lookup_mount(const char *root, const
 				if (!quoted && ch == '\n') {
 					*mapp = '\0';
 					state = st_done;
-				} else if (mapp - mapent < MAPENT_MAX_LEN - 1) {
-					/* 
-					 * Eat \ quoting \n, otherwise pass it
-					 * through for the parser
+				/* We overwrite up to 3 characters, so we
+				 * need to make sure we have enough room
+				 * in the buffer for this. */
+				} else if (mapp - mapent > 
+					   ((MAPENT_MAX_LEN+1) * alloci) - 3) {
+					/*
+					 * Alloc another page for map entries.
 					 */
-					if (quoted) {
-						if (ch == '\n')
-							*mapp++ = ' ';
-						else {
-							*mapp++ = '\\';
-							*mapp++ = ch;
-						}
-					} else
-						*mapp++ = ch;
+					distance = mapp - mapent;
+					tmp = realloc(mapent,
+						      ((MAPENT_MAX_LEN + 1) * 
+						       ++alloci));
+					if (!tmp) {
+						alloci--;
+						error(MODPREFIX "realloc: %s\n",
+						      strerror(errno));
+						break;
+					}
+					mapent = tmp;
+					mapp = tmp + distance;
 				}
+				/* 
+				 * Eat \ quoting \n, otherwise pass it
+				 * through for the parser
+				 */
+				if (quoted) {
+					if (ch == '\n')
+						*mapp++ = ' ';
+					else {
+						*mapp++ = '\\';
+						*mapp++ = ch;
+					}
+				} else
+					*mapp++ = ch;
 				break;
 			case st_done:
 				/* Eat characters till there's no more output */
@@ -233,18 +260,20 @@ int lookup_mount(const char *root, const
 
 	if (waitpid(f, &status, 0) != f) {
 		error(MODPREFIX "waitpid: %m");
-		return 1;
+		goto out_free;
 	}
 
 	if (mapp == mapent || !WIFEXITED(status) || WEXITSTATUS(status) != 0) {
 		error(MODPREFIX "lookup for %s failed", name);
-		return 1;
+		goto out_free;
 	}
 
 	debug(MODPREFIX "%s -> %s", name, mapent);
 
 	ret = ctxt->parse->parse_mount(root, name, name_len,
 				       mapent, ctxt->parse->context);
+out_free:
+	free(mapent);
 	return ret;
 }

Re: Limit on return string from program map

[Thomas Jahns]

On 12/17/04 16:19:30, Jeff Moyer wrote:
>>> the string that automount expects from a program map.  My virtual
>>> CD server has lots of CD images mounted over loopback, each with  
>>> its own export entry, so the return string from auto.net is quite  
>>> long.
> 
> raven> The buffer size for map entries is 4096.

not sure it's related: wasn't there also a very small limit on the  
number of loop-devices?

Thomas Jahns
-- 
"Computers are good at following instructions,
 but not at reading your mind."
D. E. Knuth, The TeXbook, Addison-Wesley 1984, 1986, 1996, p. 9

Re: Limit on return string from program map

[Jeff Moyer]

==> Regarding Re: [autofs] Limit on return string from program map; Thomas Jahns <Thomas.Jahns@gmx.net> adds:

Thomas.Jahns> On 12/17/04 16:19:30, Jeff Moyer wrote:
>>>> the string that automount expects from a program map.  My virtual CD
>>>> server has lots of CD images mounted over loopback, each with its own
>>>> export entry, so the return string from auto.net is quite long.
>>
raven> The buffer size for map entries is 4096.

Thomas.Jahns> not sure it's related: wasn't there also a very small limit
Thomas.Jahns> on the number of loop-devices?

MODULE_PARM(max_loop, "i");
MODULE_PARM_DESC(max_loop, "Maximum number of loop devices (1-256)");

-Jeff

Re: Limit on return string from program map

[Jeff Moyer]

==> Regarding Re: [autofs] Limit on return string from program map; Jeff Moyer <jmoyer@redhat.com> adds:

==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <raven@themaw.net> adds:
raven> On Thu, 16 Dec 2004, Keith Owens wrote:
>>> Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3).  I added
>>> "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a
>>> point.  ps shows
>>> 
>>> /usr/sbin/automount --timeout=60 /net program /etc/auto.net
>>> 
>>> If the target server has a lot of mounted filesystems then automount
>>> will only mount the first few entries on the list.  Then it tries to
>>> run mount with invalid input.  It looks like there is a hard coded
>>> limit on the string that automount expects from a program map.  My
>>> virtual CD server has lots of CD images mounted over loopback, each
>>> with its own export entry, so the return string from auto.net is quite
>>> long.

raven> The buffer size for map entries is 4096.

jmoyer> I think I posted a patch here that would extend that?  I can't seem
jmoyer> to find it in the archives, so I'll repost.  This is the version of
jmoyer> the patch that is in our tree, so no guarantees about it applying
jmoyer> to Ian's tree.  *sigh* (patch originally done by from Neil Horman)

jmoyer> Ian, when is 4.1.4 coming out.  ;-)

Bah, and it was just pointed out to me that this patch adds a bug.  I'll
repost the patch when the bug is fixed.  (see red hat bz #138994).

No ETA just yet.

-Jeff

4.1.4?

[Greg Bradner]

I'm curious too.

>Ian, when is 4.1.4 coming out.   ;-) 
>
>-Jeff



-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rhythm & Hues
5404 Jandy Place
Los Angeles, CA 90066
Voice: 310 448-7763
Fax:   310 448-7600
gregb@rhythm.com

Re: 4.1.4?

[Ian Kent]

On Fri, 17 Dec 2004, Greg Bradner wrote:

> I'm curious too.
> 
> >Ian, when is 4.1.4 coming out.   ;-) 
> >
> >-Jeff

I'm working kernel patch updates now.

I have a couple of difficult daemon patches to merge.
In particular init script patches.

A couple of patches that have been submitted have not been applied, 
largely because of the large number of changes already added. In 
particular the debug info to standard out. I must point out that this is 
not because of problems with the patch but because of the number of 
changes. I don't want to add additional function until this lot has 
settled.

So when?

I should be able to finish over Xmass and get something out in the first 
or second week of Jan.

Ian

Re: Limit on return string from program map

[Ian Kent]

On Fri, 17 Dec 2004, Jeff Moyer wrote:

> ==> Regarding Re: [autofs] Limit on return string from program map; Jeff Moyer <jmoyer@redhat.com> adds:
> 
> ==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <raven@themaw.net> adds:
> raven> On Thu, 16 Dec 2004, Keith Owens wrote:
> >>> Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3).  I added
> >>> "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a
> >>> point.  ps shows
> >>> 
> >>> /usr/sbin/automount --timeout=60 /net program /etc/auto.net
> >>> 
> >>> If the target server has a lot of mounted filesystems then automount
> >>> will only mount the first few entries on the list.  Then it tries to
> >>> run mount with invalid input.  It looks like there is a hard coded
> >>> limit on the string that automount expects from a program map.  My
> >>> virtual CD server has lots of CD images mounted over loopback, each
> >>> with its own export entry, so the return string from auto.net is quite
> >>> long.
> 
> raven> The buffer size for map entries is 4096.
> 
> jmoyer> I think I posted a patch here that would extend that?  I can't seem
> jmoyer> to find it in the archives, so I'll repost.  This is the version of
> jmoyer> the patch that is in our tree, so no guarantees about it applying
> jmoyer> to Ian's tree.  *sigh* (patch originally done by from Neil Horman)
> 
> jmoyer> Ian, when is 4.1.4 coming out.  ;-)
> 
> Bah, and it was just pointed out to me that this patch adds a bug.  I'll
> repost the patch when the bug is fixed.  (see red hat bz #138994).
> 
> No ETA just yet.

OK.

I think I've already merge the above patch.
I'll look forward to the fix.

Ian

Re: Limit on return string from program map

[raven]

On Mon, 20 Dec 2004, Jeff Moyer wrote:

> ==> Regarding Re: [autofs] Limit on return string from program map; Ian Kent <raven@themaw.net> adds:
>
> raven> On Fri, 17 Dec 2004, Jeff Moyer wrote:
>>> ==> Regarding Re: [autofs] Limit on return string from program map; Jeff
>>> Moyer <jmoyer@redhat.com> adds:
>>>
>>> ==> Regarding Re: [autofs] Limit on return string from program map; Ian
>>> Kent <raven@themaw.net> adds:
> raven> On Thu, 16 Dec 2004, Keith Owens wrote:
>>>>>> Kernel 2.6.9, autofs v4 as a module, autofs-4.1.3-28 (FC3).  I added
>>>>>> "/net /etc/auto.net" to /etc/auto.master and it works fine, up to a
>>>>>> point.  ps shows
>>>>>>
>>>>>> /usr/sbin/automount --timeout=60 /net program /etc/auto.net
>>>>>>
>>>>>> If the target server has a lot of mounted filesystems then automount
>>>>>> will only mount the first few entries on the list.  Then it tries to
>>>>>> run mount with invalid input.  It looks like there is a hard coded
>>>>>> limit on the string that automount expects from a program map.  My
>>>>>> virtual CD server has lots of CD images mounted over loopback, each
>>>>>> with its own export entry, so the return string from auto.net is
>>> quite >>> long.
>>>
> raven> The buffer size for map entries is 4096.
>>>
> jmoyer> I think I posted a patch here that would extend that?  I can't seem
> jmoyer> to find it in the archives, so I'll repost.  This is the version of
> jmoyer> the patch that is in our tree, so no guarantees about it applying
> jmoyer> to Ian's tree.  *sigh* (patch originally done by from Neil Horman)
>>>
> jmoyer> Ian, when is 4.1.4 coming out.  ;-)
>>> Bah, and it was just pointed out to me that this patch adds a bug.  I'll
>>> repost the patch when the bug is fixed.  (see red hat bz #138994).
>>>
>>> No ETA just yet.
>
> raven> OK.
>
> raven> I think I've already merge the above patch.  I'll look forward to
> raven> the fix.
>
> New patch attached.  Let me know if you need an incremental.  This is as of
> yet untested.

Incremental would probably be better if it's not to much trouble.

There will have to be quite a bit of testing when I'm finished patching. I 
expect it won't even compile first off.

Ian