EBUSY when connecting in parallel to btle devices

EBUSY when connecting in parallel to btle devices

[Tobias Preclik]

[-- Attachment #1: Type: text/plain, Size: 2447 bytes --]

Dear devs,

when performing two subsequent non-blocking connects to two different 
BTLE devices (using the same HCI device) results in the first connection 
attempt completing successfully (POLLOUT event when polling the socket) 
and the second connection hanging up (POLLOUT, POLLHUP and POLLERR 
events when polling the socket). The error is usually EBUSY (Device or 
resource busy), sometimes also ELOOP (Too many levels of symbolic 
links). This behavior is somewhat reasonable even though I would expect 
the connection attempt to be queued. However, the HCI device still seems 
to establish _both_ connections (judging from the status LEDs on the 
BTLE devices I connect to). And even if I close the hung-up socket and 
exit the process the connection remains established until the 
corresponding BTLE device is powered down. This can be verified using 
netstat built with HAVE_AFBLUETOOTH=1:

Active Bluetooth connections (w/o servers)
Proto  Destination       Source            State         PSM DCID   SCID 
      IMTU    OMTU Security
l2cap  00:BA:DC:AB:1E:00 00:de:ca:fb:ad:00 CONNECTED       0 0x0004 
0x0004     672     672 LOW

I suspect this is erroneous behavior. I attached a small program 
demonstrating the behavior. Currently, I am using kernel 4.4.0 (Ubuntu) 
and bluez 5.40. The problem can be reproduced using various HCI devices. 
The sample output of the behavior described above is:

$ g++ -Wall -std=c++11 -o bttest bttest.cpp -lbluetooth
$ ./bttest
Connecting socket (00:DE:AD:BE:EF:00)...
Connection in progress (00:DE:AD:BE:EF:00)...
Connecting socket (00:DE:CA:FB:AD:00)...
Connection in progress (00:DE:CA:FB:AD:00)...
Events from socket connection (00:DE:CA:FB:AD:00): POLLOUT POLLHUP POLLERR
Socket error occurred: Device or resource busy
Events from socket connection (00:DE:CA:FB:AD:00): POLLOUT POLLHUP
Closing socket (00:DE:CA:FB:AD:00) due to hang-up...
Events from socket connection (00:DE:AD:BE:EF:00): POLLOUT
Connected socket (00:DE:AD:BE:EF:00).
# Powering down device 00:DE:AD:BE:EF:00...
Events from socket connection (00:DE:AD:BE:EF:00): POLLHUP POLLERR
Socket error occurred: Connection timed out
Events from socket connection (00:DE:AD:BE:EF:00): POLLHUP
Closing socket (00:DE:AD:BE:EF:00) due to hang-up...

The EBUSY/ELOOP error is also reported if neither of the hard-coded BTLE 
devices (00:DE:AD:BE:EF:00 and 00:DE:CA:FB:AD:00) exists.

Can anyone confirm that this is a bug?

Regards,
Tobias

[-- Attachment #2: bttest.cpp --]
[-- Type: text/plain, Size: 4999 bytes --]

#include <sys/types.h>
#include <sys/socket.h>
#include <unistd.h>
#include <fcntl.h>
#include <poll.h>
#include <errno.h>
#include <iostream>

#include <bluetooth/bluetooth.h>
#include <bluetooth/hci.h>
#include <bluetooth/hci_lib.h>
#include <bluetooth/l2cap.h>

#define ATT_CID 4

struct Device {
	Device(const std::string &address) : address_(address) {}
	Device() {}
	~Device() {
		close();
	}

	bool close() {
		if (sock_ < 0)
			return true;

		if (::close(sock_) < 0) {
			std::cout << "Failed to close socket: " << strerror(errno) << std::endl;
			return false;
		}

		sock_ = -1;
		return true;
	}

	bool openSocket(const char *hciMac) {
		struct sockaddr_l2 srcaddr, dstaddr;

		int dev_id;
		if (hciMac == nullptr)
			dev_id = hci_get_route(nullptr);
		else
			dev_id = hci_devid(hciMac);

		struct hci_dev_info dev_info;
		if (hci_devinfo(dev_id, &dev_info) < 0) {
			std::cout << "Failed to obtain device info: " << strerror(errno) << std::endl;
			return false;
		}

		sock_ = socket(PF_BLUETOOTH, SOCK_SEQPACKET, BTPROTO_L2CAP);
		if (sock_ < 0) {
			std::cout << "Failed to create L2CAP socket: " << strerror(errno) << std::endl;
			return false;
		}

		memset(&srcaddr, 0, sizeof(srcaddr));
		srcaddr.l2_family = AF_BLUETOOTH;
		srcaddr.l2_cid = htobs(ATT_CID);
		srcaddr.l2_bdaddr = dev_info.bdaddr;

		if (bind(sock_, (struct sockaddr *)&srcaddr, sizeof(srcaddr)) < 0) {
			std::cout << "Failed to bind: " << strerror(errno) << std::endl;
			close();
			return false;
		}

		// Set up destination address.
		memset(&dstaddr, 0, sizeof(dstaddr));
		dstaddr.l2_family = AF_BLUETOOTH;
		dstaddr.l2_cid = htobs(ATT_CID);
		dstaddr.l2_bdaddr_type = BDADDR_LE_RANDOM;
		str2ba(address_.c_str(), &dstaddr.l2_bdaddr);

		// Set socket to non-blocking mode.
		int flags = fcntl(sock_, F_GETFL, NULL);
		if (flags < 0) {
			std::cout << "Failed to get socket status flags: " << strerror(errno) << std::endl;
			close();
			return false;
		}

		flags |= O_NONBLOCK;

		if (fcntl(sock_, F_SETFL, flags) < 0) {
			std::cout << "Failed to set socket status flags: " << strerror(errno) << std::endl;
			close();
			return false;
		}

		std::cout << "Connecting socket (" << address_ << ")..." << std::endl;

		int err = ::connect(sock_, (struct sockaddr *)&dstaddr, sizeof(dstaddr));
		if (err < 0 && errno != EINPROGRESS) {
			std::cout << "Failed to connect: " << strerror(errno) << std::endl;
			close();
			return false;
		}

		if (errno == EINPROGRESS)
			std::cout << "Connection in progress (" << address_ << ")..." << std::endl;
		else
			std::cout << "Connected socket (" << address_ << ")." << std::endl;

		return true;
	}

	int sock_{-1};
	std::string address_;
};

int main()
{
	const std::size_t numDevices = 2;

	Device devices[numDevices];
	devices[0] = Device("00:DE:AD:BE:EF:00");
	devices[1] = Device("00:DE:CA:FB:AD:00");
	
	pollfd fds[numDevices];

	for (std::size_t i = 0; i < numDevices; ++i) {
		bool ret = devices[i].openSocket(nullptr);
		if (!ret)
			std::cout << "Failed to open socket (" << devices[i].address_ << ")." << std::endl;

		fds[i].fd = devices[i].sock_;
		fds[i].events = POLLIN | POLLPRI | POLLOUT | POLLRDHUP;
	}

	// Wait for events...
	for (;;) {
		std::size_t numActive = 0;
		for (std::size_t i = 0; i < numDevices; ++i)
			if (fds[i].fd != -1)
				++numActive;

		if (numActive == 0)
			break;

		int ret = poll(fds, numDevices, -1);
		if (ret < 0) {
			std::cout << "Failed to poll sockets: " << strerror(errno) << std::endl;
			std::exit(EXIT_FAILURE);
		}

		for (std::size_t i = 0; i < numDevices; ++i) {
			short revents = fds[i].revents;

			if (revents == 0)
				continue;

			std::cout << "Events from socket connection (" << devices[i].address_ << "):";
			if (revents & POLLIN)
				std::cout << " POLLIN";
			if (revents & POLLPRI)
				std::cout << " POLLPRI";
			if (revents & POLLOUT)
				std::cout << " POLLOUT";
			if (revents & POLLRDHUP)
				std::cout << " POLLRDHUP";
			if (revents & POLLHUP)
				std::cout << " POLLHUP";
			if (revents & POLLERR)
				std::cout << " POLLERR";
			if (revents & POLLNVAL)
				std::cout << " POLLNVAL";
			std::cout << std::endl;

			if (revents & POLLNVAL) {
				std::cout << "Invalid file descriptor detected: " << fds[i].fd << std::endl;
				fds[i].fd = -1;
			}
			else if (revents & POLLERR) {
				int optionValue = 0;
				socklen_t optionLength = sizeof(optionValue);

				if (getsockopt(devices[i].sock_, SOL_SOCKET, SO_ERROR, &optionValue, &optionLength) < 0)
					std::cout << "Failed to determine reason of socket error: " << strerror(errno) << std::endl;
				else
					std::cout << "Socket error occurred: " << strerror(optionValue) << std::endl;
			}
			else if (revents & POLLHUP) {
				std::cout << "Closing socket (" << devices[i].address_ << ") due to hang-up..." << std::endl;

				devices[i].close();
				fds[i].fd = -1;
			}
			else if (revents & POLLOUT) {
				std::cout << "Connected socket (" << devices[i].address_ << ")." << std::endl;
				fds[i].events = 0;
			}
		}
	}
}