From mboxrd@z Thu Jan  1 00:00:00 1970
From: Daniel Stodden <daniel.stodden@citrix.com>
Subject: Re: blktap2 device creation failing after 162 devices
	w/Xen4.0 + linux-2.6.31.13
Date: Wed, 14 Apr 2010 13:12:26 -0700
Message-ID: <1271275946.25413.3785.camel@agari.van.xensource.com>
References: <4BC50A5A.1020501@cs.ucsd.edu>
	<1271216280.25413.1782.camel@agari.van.xensource.com>
	<4BC55531.7080005@cs.ucsd.edu>
	<1271233446.2449.59.camel@ramone.somacoma.net>
	<4BC5ED97.9090608@cs.ucsd.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xensource.com>
In-Reply-To: <4BC5ED97.9090608@cs.ucsd.edu>
List-Unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: John McCullough <jmccullo@cs.ucsd.edu>
Cc: Ian Pratt <Ian.Pratt@eu.citrix.com>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>
List-Id: xen-devel@lists.xenproject.org

On Wed, 2010-04-14 at 12:30 -0400, John McCullough wrote:
> On 04/14/2010 01:24 AM, Daniel Stodden wrote:
> > On Wed, 2010-04-14 at 01:40 -0400, John McCullough wrote:
> >    
> >> Daniel,
> >>
> >> That did the trick and got us up to 256, Thanks!
> >>
> >> Out of curiosity, what's standing in the way of more devices?
> >>      
> > I must admit I never tried. Lack of maybe a couple sparse tables here
> > and there?
> >
> >    
> >> We tried raising the MAX_*_DEVICES constants in these files to 512, but
> >> didn't have any luck:
> >> linux-2.6-pvops.git/drivers/xen/blktap/blktap.h
> >> tools/blktap2/include/blktaplib.h
> >> tools/blktap/lib/blktaplib.h
> >>
> >> (The error is now "vbd open failed: -6")
> >>      
> > That would be an ENXIO, probably while trying to open the ring (can you
> > verify that with an strace -f?)
> >    
> 
> Looks like it is ENXIO:
> 
> ...
> [pid  4154] open("/dev/xen/blktap-2/blktap256", O_RDWR <unfinished ...>
> 
> [pid  4153] close(4)                    = 0
> [pid  4153] fcntl(3, F_GETFL)           = 0 (flags O_RDONLY)
> [pid  4153] brk(0)                      = 0xa1e000
> [pid  4153] brk(0xa3f000)               = 0xa3f000
> [pid  4153] fstat(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
> [pid  4153] mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f6844b40000
> [pid  4153] lseek(3, 0, SEEK_CUR)       = -1 ESPIPE (Illegal seek)
> [pid  4153] read(3, <unfinished ...>
> 
> [pid  4154] <... open resumed> )        = -1 ENXIO (No such device or 
> address)
> [pid  4154] gettimeofday({1271262219, 398530}, NULL) = 0
> [pid  4154] sendto(0, "<27>Apr 14 09:23:39 tapdisk2[415"..., 115, 
> MSG_NOSIGNAL, NULL, 0) = 115
> [pid  4154] close(3)                    = 0
> [pid  4154] gettimeofday({1271262219, 398836}, NULL) = 0
> [pid  4154] sendto(0, "<30>Apr 14 09:23:39 tapdisk2[415"..., 109, 
> MSG_NOSIGNAL, NULL, 0) = 109
> [pid  4154] gettimeofday({1271262219, 399161}, NULL) = 0
> [pid  4154] sendto(0, "<27>Apr 14 09:23:39 tapdisk2[415"..., 86, 
> MSG_NOSIGNAL, NULL, 0) = 86
> [pid  4154] write(1, "-6: vbd open failed: -6\n", 24) = 24
> ... (err, ioctl to control, close and exit)

Yep. So that's the ring device, which was apparently created ok but
remains somehow inaccessible.

Like I said, I think this fails earlier in the call stack than juyst
blktap_ring_file_operations. Just from looking at some code I don't
immediately see the issue.

I guess this presently returns only halfway through chrdev_open. 
Broken device registration somewhere? Does it take more magic to
register minors beyond 8 bit?

Thanks for the trace. Can't spent much time on it right now. Want to
take it on?

Daniel