From mboxrd@z Thu Jan  1 00:00:00 1970
From: Daniel Stodden <daniel.stodden@citrix.com>
Subject: Re: blktap2 device creation failing after 162 devices
	w/Xen4.0 + linux-2.6.31.13
Date: Wed, 14 Apr 2010 13:27:24 -0700
Message-ID: <1271276844.25413.3823.camel@agari.van.xensource.com>
References: <4BC50A5A.1020501@cs.ucsd.edu>
	<1271216280.25413.1782.camel@agari.van.xensource.com>
	<4BC55531.7080005@cs.ucsd.edu>
	<1271233446.2449.59.camel@ramone.somacoma.net>
	<4BC5ED97.9090608@cs.ucsd.edu>
	<1271275946.25413.3785.camel@agari.van.xensource.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xensource.com>
In-Reply-To: <1271275946.25413.3785.camel@agari.van.xensource.com>
List-Unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: John McCullough <jmccullo@cs.ucsd.edu>
Cc: Ian Pratt <Ian.Pratt@eu.citrix.com>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>
List-Id: xen-devel@lists.xenproject.org

On Wed, 2010-04-14 at 16:12 -0400, Daniel Stodden wrote:
> On Wed, 2010-04-14 at 12:30 -0400, John McCullough wrote:
> > On 04/14/2010 01:24 AM, Daniel Stodden wrote:
> > > On Wed, 2010-04-14 at 01:40 -0400, John McCullough wrote:
> > >    
> > >> Daniel,
> > >>
> > >> That did the trick and got us up to 256, Thanks!
> > >>
> > >> Out of curiosity, what's standing in the way of more devices?
> > >>      
> > > I must admit I never tried. Lack of maybe a couple sparse tables here
> > > and there?
> > >
> > >    
> > >> We tried raising the MAX_*_DEVICES constants in these files to 512, but
> > >> didn't have any luck:
> > >> linux-2.6-pvops.git/drivers/xen/blktap/blktap.h
> > >> tools/blktap2/include/blktaplib.h
> > >> tools/blktap/lib/blktaplib.h
> > >>
> > >> (The error is now "vbd open failed: -6")
> > >>      
> > > That would be an ENXIO, probably while trying to open the ring (can you
> > > verify that with an strace -f?)
> > >    
> > 
> > Looks like it is ENXIO:
> > 
> > ...
> > [pid  4154] open("/dev/xen/blktap-2/blktap256", O_RDWR <unfinished ...>
> > 
> > [pid  4153] close(4)                    = 0
> > [pid  4153] fcntl(3, F_GETFL)           = 0 (flags O_RDONLY)
> > [pid  4153] brk(0)                      = 0xa1e000
> > [pid  4153] brk(0xa3f000)               = 0xa3f000
> > [pid  4153] fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
> > [pid  4153] mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
> > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6844b40000
> > [pid  4153] lseek(3, 0, SEEK_CUR)       = -1 ESPIPE (Illegal seek)
> > [pid  4153] read(3, <unfinished ...>
> > 
> > [pid  4154] <... open resumed> )        = -1 ENXIO (No such device or 
> > address)
> > [pid  4154] gettimeofday({1271262219, 398530}, NULL) = 0
> > [pid  4154] sendto(0, "<27>Apr 14 09:23:39 tapdisk2[415"..., 115, 
> > MSG_NOSIGNAL, NULL, 0) = 115
> > [pid  4154] close(3)                    = 0
> > [pid  4154] gettimeofday({1271262219, 398836}, NULL) = 0
> > [pid  4154] sendto(0, "<30>Apr 14 09:23:39 tapdisk2[415"..., 109, 
> > MSG_NOSIGNAL, NULL, 0) = 109
> > [pid  4154] gettimeofday({1271262219, 399161}, NULL) = 0
> > [pid  4154] sendto(0, "<27>Apr 14 09:23:39 tapdisk2[415"..., 86, 
> > MSG_NOSIGNAL, NULL, 0) = 86
> > [pid  4154] write(1, "-6: vbd open failed: -6\n", 24) = 24
> > ... (err, ioctl to control, close and exit)
> 
> Yep. So that's the ring device, which was apparently created ok but
> remains somehow inaccessible.
> 
> Like I said, I think this fails earlier in the call stack than juyst
> blktap_ring_file_operations. Just from looking at some code I don't
> immediately see the issue.
> 
> I guess this presently returns only halfway through chrdev_open. 
> Broken device registration somewhere? Does it take more magic to
> register minors beyond 8 bit?

Oh.

--snip
 * This function registers a range of 256 minor numbers. The first minor
number
 * is 0.
 */
int register_chrdev(unsigned int major, const char *name,
		    const struct file_operations *fops)
--snip--

Judging from the rest of the kernel, we'll need to reimplement the
registration somewhat different that that.

Daniel