[Xenomai] Regarding Xenomai and RTNET

[Xenomai] Regarding Xenomai and RTNET

[Leopold Palomo-Avellaneda]

Hi,


I'm fighting with some code to run with Xenomai 3. One of the projects is soem 
[1]. 

The original library uses POSIX to open a raw socket to send Ethercat 
messages. Our fork, uses rtnet and some Xenomai functionalities to accomplish 
it. It has been working with 2.6.x without any problems. 

OTOH, also we have done another fork fril  [2] that, using the POSIX 
compatibility and rtnet use a rt_ethX device to communicate with a robot. We 
have used it without any problem too with 2.6.x

Now, we try to migrate all of this code to Xenomai-3 but we are having crashes 
all the time.

With fril, a pure POSIX code, using the compatibility mode, we have a crash, 
showing:

 BUG: unable to handle kernel paging request at 00007f47ea0ef878
 IP: [<ffffffffa0231580>] rt_udp_ioctl+0x50/0x74 [rtudp]
 PGD 458887067 PUD 4590a1067 PMD 45921f067 PTE 8000000438863867
 Oops: 0001 [#1] PREEMPT SMP 
 Modules linked in: rt_igb rt_loopback rtcfg rtudp rtipv4 rtmac rtpacket rtnet 
e100 mii ctr ccm binfmt_misc nfsd
 CPU: 4 PID: 6773 Comm: LWRJointPositio Not tainted 4.1.18-xenomai-3.0.3 #1
 Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./Q170M-D3H-
CF, BIOS F1 10/13/2015
 task: ffff880459a26010 ti: ffff880459a38000 task.ti: ffff880459a38000
 RIP: 0010:[<ffffffffa0231580>]  [<ffffffffa0231580>] rt_udp_ioctl+0x50/0x74 [rtudp]
 RSP: 0018:ffff880459a3be08  EFLAGS: 00010246
 RAX: 00007f47ea0ef870 RBX: ffff880458d59400 RCX: ffff880458d59440
 RDX: 0000000000000000 RSI: 0000000040100022 RDI: ffff880458d59400
 RBP: 0000000000000003 R08: ffff880460297420 R09: 000000000000004e
 R10: 00000000000000dc R11: ffff880459a3bdc0 R12: ffff880459a26010
 R13: ffffc90001f05008 R14: 0000000040100022 R15: ffffffff81b85ec0
 FS:  00007f47ea0f0700(0000) GS:ffff880460200000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
 CR2: 00007f47ea0ef878 CR3: 000000045890c000 CR4: 00000000003406e0
 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
 I-pipe domain Linux
 Stack:
  ffffffffa0231535 ffffffff8116fb70 ffff880459a265c0 00007f47ea0ef870
  ffff8804599975d0 0000000000000010 ffff880459a3beb8 ffff880459a3be48
  0000000000000002 ffff880459a26010 00007f47ea0ef870 ffff880459a26010
 Call Trace:
  [<ffffffffa0231535>] ? rt_udp_ioctl+0x5/0x74 [rtudp]
  [<ffffffff8116fb70>] ? rtdm_fd_ioctl+0x100/0x270
  [<ffffffff81174b40>] ? CoBaLt_fcntl+0x20/0x20
  [<ffffffff81174b40>] ? CoBaLt_fcntl+0x20/0x20
  [<ffffffff81174b50>] ? CoBaLt_ioctl+0x10/0x20
  [<ffffffff81174b45>] ? CoBaLt_ioctl+0x5/0x20
  [<ffffffff8118450a>] ? ipipe_syscall_hook+0x11a/0x360      
  [<ffffffff81108da7>] ? __ipipe_notify_syscall+0xe7/0x1d0   
  [<ffffffff81107185>] ? __ipipe_restore_root_nosync+0x5/0x30
  [<ffffffff8158fb34>] ? pipeline_syscall+0x9/0x16
 Code: 23 00 10 40 75 15 8b 50 08 48 8b 30 48 89 cf 48 83 c4 08 e9 a3 fd ff ff 
0f 1f 00 48 89 c2 48 83 c4 08 e9 5
 RIP  [<ffffffffa0231580>] rt_udp_ioctl+0x50/0x74 [rtudp]
  RSP <ffff880459a3be08>
 CR2: 00007f47ea0ef878
 ---[ end trace 085d23e71de3ae4b ]---


with soem, not using compatibility, just native (alchemy) code:

 BUG: unable to handle kernel paging request at 00007ffd028e0190
 IP: [<ffffffff81319ca8>] strncmp+0x8/0x50
 PGD 458468067 PUD 459b58067 PMD 4592ac067 PTE 8000000451078867
 Oops: 0001 [#2] PREEMPT SMP 
 Modules linked in: rt_igb rt_loopback rtcfg rtudp rtipv4 rtmac rtpacket rtnet 
e100 mii ctr ccm binfmt_misc nfsd
 CPU: 0 PID: 7752 Comm: slaveinfo_rt Tainted: G      D         4.1.18-
xenomai-3.0.3 #1
 Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./Q170M-D3H-
CF, BIOS F1 10/13/2015
 task: ffff8804581bcc90 ti: ffff88045b434000 task.ti: ffff88045b434000
 RIP: 0010:[<ffffffff81319ca8>]  [<ffffffff81319ca8>] strncmp+0x8/0x50
 RSP: 0018:ffff88045b437da0  EFLAGS: 00010202
 RAX: ffffc90001f02008 RBX: ffffffffa01a4740 RCX: 0000000000000072
 RDX: 0000000000000010 RSI: 00007ffd028e0190 RDI: ffff88045b457404
 RBP: ffff88045b457400 R08: ffff88045fe97420 R09: 000000000000004d
 R10: 00000000000000dc R11: ffff88045b437dc0 R12: 00007ffd028e0190
 R13: 00007ffd028e0190 R14: 0000000040180021 R15: ffffffff81b85ec0
 FS:  00007f6dbcbf0740(0000) GS:ffff88045fe00000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
 CR2: 00007ffd028e0190 CR3: 0000000458df7000 CR4: 00000000003406f0
 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
 I-pipe domain Linux
 Stack:
  ffffffffa019d151 0000000000052f08 0000000000000000 00007ffd028e0190
  ffffffffa019d621 ffff880459184a00 0000000040180021 ffff8804581bcc90
  ffffffffa01a07de ffff880459184a00 0000000000000003 ffff8804581bcc90
 Call Trace:
  [<ffffffffa019d151>] ? __rtdev_get_by_name+0x31/0x60 [rtnet]
  [<ffffffffa019d621>] ? rtdev_get_by_name+0x51/0xd0 [rtnet]  
  [<ffffffffa01a07de>] ? rt_socket_if_ioctl+0x2e/0x2f0 [rtnet]
  [<ffffffff8116fb70>] ? rtdm_fd_ioctl+0x100/0x270
  [<ffffffff81174b40>] ? CoBaLt_fcntl+0x20/0x20
  [<ffffffff81174b40>] ? CoBaLt_fcntl+0x20/0x20
  [<ffffffff81174b50>] ? CoBaLt_ioctl+0x10/0x20
  [<ffffffff81174b45>] ? CoBaLt_ioctl+0x5/0x20
  [<ffffffff8118450a>] ? ipipe_syscall_hook+0x11a/0x360   
  [<ffffffff81108da7>] ? __ipipe_notify_syscall+0xe7/0x1d0
  [<ffffffff811feb25>] ? fput+0x5/0x90
  [<ffffffff81107185>] ? __ipipe_restore_root_nosync+0x5/0x30
  [<ffffffff8158fb34>] ? pipeline_syscall+0x9/0x16
 Code: 46 ff 75 0f 84 c0 75 eb 31 c0 c3 0f 1f 84 00 00 00 00 00 19 c0 83 c8 01 
c3 66 2e 0f 1f 84 00 00 00 00 00 4
 RIP  [<ffffffff81319ca8>] strncmp+0x8/0x50
  RSP <ffff88045b437da0>
 CR2: 00007ffd028e0190
 ---[ end trace 085d23e71de3ae4c ]---


If both the program crashes "unable to handle kernel paging request". 

Also, I have found that using the --compat parameter in xeno-config, the values 
obtained (--posix --compat --cflags) makes that the applications doesn't 
compile:

 /home/leopold.palomo/fril/fril.git/src/FastResearchInterfaceLibrary/Console.cpp:50:
/usr/xenomai/include/trank/posix/pthread.h: In function ‘int 
pthread_set_mode_np(int, int, int*)’:
/usr/xenomai/include/trank/posix/pthread.h:55:52: error: ‘pthread_setmode_np’ 
was not declared in this scope
  return pthread_setmode_np(clrmask, setmask, mask_r);

I guess that there's a missing includes hidden here.

So, any idea to find a solution?
Do you have some example that works with xenomai-3 and rtnet?

Best regards,


Leopold



[1] https://github.com/iocroblab/soem
[2] https://github.com/iocroblab/fril

-- 
--
Linux User 152692     GPG: 05F4A7A949A2D9AA
Catalonia
-------------------------------------
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part.
URL: <http://xenomai.org/pipermail/xenomai/attachments/20160930/0b67fb4e/attachment.sig>

Re: [Xenomai] Regarding Xenomai and RTNET

[Philippe Gerum]

On 09/30/2016 02:15 PM, Leopold Palomo-Avellaneda wrote:
> Hi,
> 
> 
> I'm fighting with some code to run with Xenomai 3. One of the projects is soem 
> [1]. 
> 
> The original library uses POSIX to open a raw socket to send Ethercat 
> messages. Our fork, uses rtnet and some Xenomai functionalities to accomplish 
> it. It has been working with 2.6.x without any problems. 
> 
> OTOH, also we have done another fork fril  [2] that, using the POSIX 
> compatibility and rtnet use a rt_ethX device to communicate with a robot. We 
> have used it without any problem too with 2.6.x
> 
> Now, we try to migrate all of this code to Xenomai-3 but we are having crashes 
> all the time.
> 
> With fril, a pure POSIX code, using the compatibility mode, we have a crash, 
> showing:
> 
>  BUG: unable to handle kernel paging request at 00007f47ea0ef878
>  IP: [<ffffffffa0231580>] rt_udp_ioctl+0x50/0x74 [rtudp]
>  PGD 458887067 PUD 4590a1067 PMD 45921f067 PTE 8000000438863867
>

[snip]

> If both the program crashes "unable to handle kernel paging request". 
>

The compat mode is not related to this issue, it is merely a source code
wrapping trick at user level, nothing more. If a driver breaks the
kernel, then it's the driver's fault no matter what, regardless of the
input data userland might have sent.

You should check the RTnet stack for this. Btw, assuming your 2.6 setup
relies on the original out-of-tree RTnet, and that your 3.0
configuration relies on the RTnet code integrated in the Xenomai tree,
then you end up comparing two different RTnet implementations.

> Also, I have found that using the --compat parameter in xeno-config, the values 
> obtained (--posix --compat --cflags) makes that the applications doesn't 
> compile:
> 
>  /home/leopold.palomo/fril/fril.git/src/FastResearchInterfaceLibrary/Console.cpp:50:
> /usr/xenomai/include/trank/posix/pthread.h: In function ‘int 
> pthread_set_mode_np(int, int, int*)’:
> /usr/xenomai/include/trank/posix/pthread.h:55:52: error: ‘pthread_setmode_np’ 
> was not declared in this scope
>   return pthread_setmode_np(clrmask, setmask, mask_r);
> 
> I guess that there's a missing includes hidden here.
> 

No, it's rather an issue in the include directory chain when --compat is
enabled. Fixed in xeno-config, in the stable maintenance tree.

> So, any idea to find a solution?
> Do you have some example that works with xenomai-3 and rtnet?
> 

No, but I'm no RTnet expert.

-- 
Philippe.

Re: [Xenomai] Regarding Xenomai and RTNET

[Leopold Palomo-Avellaneda]

El Divendres, 30 de setembre de 2016, a les 16:06:15, Philippe Gerum va 
escriure:
> On 09/30/2016 02:15 PM, Leopold Palomo-Avellaneda wrote:
> > Hi,
> > 
> > 
> > I'm fighting with some code to run with Xenomai 3. One of the projects is
> > soem [1].
> > 
> > The original library uses POSIX to open a raw socket to send Ethercat
> > messages. Our fork, uses rtnet and some Xenomai functionalities to
> > accomplish it. It has been working with 2.6.x without any problems.
> > 
> > OTOH, also we have done another fork fril  [2] that, using the POSIX
> > compatibility and rtnet use a rt_ethX device to communicate with a robot.
> > We have used it without any problem too with 2.6.x
> > 
> > Now, we try to migrate all of this code to Xenomai-3 but we are having
> > crashes all the time.
> > 
> > With fril, a pure POSIX code, using the compatibility mode, we have a
> > crash,> 
> > showing:
> >  BUG: unable to handle kernel paging request at 00007f47ea0ef878
> >  IP: [<ffffffffa0231580>] rt_udp_ioctl+0x50/0x74 [rtudp]
> >  PGD 458887067 PUD 4590a1067 PMD 45921f067 PTE 8000000438863867
> 
> [snip]
> 
> > If both the program crashes "unable to handle kernel paging request".
> 
> The compat mode is not related to this issue, it is merely a source code
> wrapping trick at user level, nothing more. If a driver breaks the
> kernel, then it's the driver's fault no matter what, regardless of the
> input data userland might have sent.

Ok.

> You should check the RTnet stack for this. Btw, assuming your 2.6 setup
> relies on the original out-of-tree RTnet, and that your 3.0
> configuration relies on the RTnet code integrated in the Xenomai tree,
> then you end up comparing two different RTnet implementations.

I'm investigating this issue. I have found an interesting detail. I'm using 
rt_igb module. The first time I upload the module, it's loaded (showed with 
lsmod) but no rtethX interface is found. So, if I rmmod it and I load it 
again, then the rtethX are found.

Any idea why this?

In any case, we are talking to debug rtnet modules. Right?


> > Also, I have found that using the --compat parameter in xeno-config, the
> > values obtained (--posix --compat --cflags) makes that the applications
> > doesn't> 
> > compile:
> >  
/home/leopold.palomo/fril/fril.git/src/FastResearchInterfaceLibrary/Console.cpp:50:
> > /usr/xenomai/include/trank/posix/pthread.h: In function ‘int
> > pthread_set_mode_np(int, int, int*)’:
> > /usr/xenomai/include/trank/posix/pthread.h:55:52: error:
> > ‘pthread_setmode_np’ was not declared in this scope
> > 
> >   return pthread_setmode_np(clrmask, setmask, mask_r);
> > 
> > I guess that there's a missing includes hidden here.
> 
> No, it's rather an issue in the include directory chain when --compat is
> enabled. Fixed in xeno-config, in the stable maintenance tree.
> 
> > So, any idea to find a solution?
> > Do you have some example that works with xenomai-3 and rtnet?
> 
> No, but I'm no RTnet expert.

Jan?

Leopold

-- 
--
Linux User 152692     GPG: 05F4A7A949A2D9AA
Catalonia
-------------------------------------
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?