Unable to register memory, ibv_reg_mr returning EACCESS

[Vinit Agnihotri <vinita@ryussi.com>]

Hi,

we have a userspace server which would allow user to access files using 
his/her own credentials via UID mapping.

while we are running server as root user, but to in order to honor posix 
mappings, we use  setfsgid()/setfssid() before

accessing any file and restore them back to user access. However when we 
try to ibv_reg_mr() EACCESS is returned.

We are ensuring to be root before we call to ibv_reg_mr(), yet it keeps 
failing.

While going through libibverbs sources we realize EACCESS might be 
returned by this call:

if (write(pd->context->cmd_fd, cmd, cmd_size) != cmd_size)
         return errno;


As we are ensuring that setfsgid()/setfssid() are set to root before we 
call to ibv_reg_mr(), why EACCESS is still returned?

Since these calls are thread specific only why it should affect entire 
process?

Can anyone provide any insight into this behavior?


Thanks & Regards,

Vinit.