Re: [PATCH 1/3] [POC] test implementaion of rt-signals

[Jan Kiszka <jan.kiszka@siemens.com>]

On 16.05.23 08:46, Johannes Kirchmair wrote:
> Hello, 
> 
>> -----Original Message-----
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>> Sent: Montag, 15. Mai 2023 12:38
>> To: Johannes Kirchmair <johannes.kirchmair@sigmatek.at>;
>> xenomai@lists.linux.dev; Schaffner, Tobias <tobias.schaffner@siemens.com>
>> Subject: Re: [PATCH 1/3] [POC] test implementaion of rt-signals
>>
>> CAUTION: External E-Mail !
>>
>> On 15.05.23 08:50, Johannes Kirchmair wrote:
>>>> -----Original Message-----
>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>> Sent: Freitag, 12. Mai 2023 19:39
>>>> To: Johannes Kirchmair <johannes.kirchmair@sigmatek.at>;
>>>> xenomai@lists.linux.dev; Schaffner, Tobias <tobias.schaffner@siemens.com>
>>>> Subject: Re: [PATCH 1/3] [POC] test implementaion of rt-signals
>>>>
>>>> CAUTION: External E-Mail !
>>>>
>>>> On 09.05.23 15:17, Johannes Kirchmair wrote:
>>>>> Hello Jan and Tobias,
>>>>>
>>>>> I just dropped the patches on the mailing list.
>>>>> For the dovetail functions I took a rather naive approach, using most of the
>>>> frame setup code provided by Linux.
>>>>> I just tested it for 32bit and 64bit x86 Applications and it seems to work.
>>>>>
>>>>> I also added a tool rt_signal_hist that runs in a cycle and measures the time an
>>>> exception handling takes.
>>>>> It is afterwards put into an txt file that reassembles an histogram of the
>> timings.
>>>>>
>>>>> Unfortunately as a 32bit application the rt_signal_hist tool does not run to
>>>> completion, printing the following:
>>>>> " Xenomai/cobalt: sleeping while holding mutex
>>>>> CPU time limit exceeded"
>>>>>
>>>>> Maybe the naive approach is not so reliable ;-/
>>>>> Hope I find the time the look into it.
>>>>>
>>>>
>>>> I looked into the x86 signalframe helpers before and found some traces
>>>> of locking that make them incompatible - if I rember correctly. After
>>>> that, I dropped the idea of just removing the "static".
>>> Do you remember where you saw the locking in the code, was it rather obvious
>> or hidden?
>>> When I had a look I did not see locking in the functions I used, so I thought
>> dropping the "static" would be fine.
>>>
>>> Would you still try to use some of the functions Linux provides?
>>>
>>
>> I don't recall too many details, specficially not which kernel version I
>> looked at. Maybe part of my concerns was that I started from
>> setup_rt_frame (rseq_signal_deliver), rather than x64_setup_rt_frame & Co.
>>
>> I was now digging into 6.3, and one area that is sensitive is the FPU
>> handling. We end up in fault_in_readable, and I'm not sure yet if that
>> is ok. The whole FPU save/restore needs a careful check, or more, if we
>> are not violating the assumptions of OOB.
>>
>> But did you run your code with full lock debugging enabled already? Did
>> you stress faulting sigframe setups/restores? That would be needed as well.
> 
> Yes, thought about activating debugging in kernel, yesterday. Resulting in the following kernel output:
> [   83.338288] ------------[ cut here ]------------
> [   83.338292] WARNING: CPU: 3 PID: 268 at mm/memory.c:5640 __might_fault+0x28/0x30
> [   83.338299] Modules linked in:
> [   83.338301] CPU: 3 PID: 268 Comm: signal_test_tas Tainted: G        W          6.3.0+ #7
> [   83.338304] Hardware name: Default string Default string/Default string, BIOS 5.19 04/28/2022
> [   83.338305] IRQ stage: Xenomai
> [   83.338306] RIP: 0010:__might_fault+0x28/0x30
> [   83.338308] Code: 90 90 f3 0f 1e fa 0f 1f 44 00 00 55 65 8b 05 57 11 24 6a 48 89 e5 a9 00 00 00 02 75 06 5d c3 cc cc cc cc 9c 58 f6 c4 02 75 f3 <0f> 0b 5d c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90
> [   83.338311] RSP: 0000:ffffafbc417d3cb8 EFLAGS: 00010046
> [   83.338313] RAX: 0000000000000006 RBX: 00000000df53adbc RCX: ffffafbc417d3d68
> [   83.338314] RDX: 00000000000002b4 RSI: 0000000000000060 RDI: ffffffff9782ead0
> [   83.338315] RBP: ffffafbc417d3cb8 R08: ffffafbc417d3e48 R09: 0000000000000001
> [   83.338316] R10: 0000000000000001 R11: 0000000000000000 R12: 00000000df53af40
> [   83.338317] R13: ffffafbc417d3db8 R14: 0000000002000000 R15: 0000000000000000
> [   83.338318] FS:  0000000000000000(0000) GS:ffff966e2e000000(0063) knlGS:00000000df53bb40
> [   83.338320] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
> [   83.338321] CR2: 00000000f7ee7080 CR3: 0000000006550000 CR4: 0000000000350ee0
> [   83.338323] Call Trace:
> [   83.338325]  <TASK>
> [   83.338328]  copy_fpstate_to_sigframe+0xaa/0x410
> [   83.338335]  get_sigframe+0xdc/0x2e0
> [   83.338339]  ia32_setup_rt_frame+0x67/0x2f0
> [   83.338343]  dovetail_setup_rt_signal_frame+0x43/0x50
> [   83.338346]  xnthread_handle_rt_signals+0xf2/0x130
> [   83.338349]  ? handle_head_syscall+0x179/0x550
> [   83.338352]  ? __pfx_CoBaLt_sigaction+0x10/0x10
> [   83.338355]  ? handle_head_syscall+0x179/0x550
> [   83.338357]  ? __pfx_CoBaLt_sigaction+0x10/0x10
> [   83.338359]  handle_oob_trap_entry+0x1b6/0x3c0
> [   83.338362]  ? syscall_enter_from_user_mode_work+0x31/0xb0
> [   83.338366]  __oob_trap_notify+0x2f/0x40
> [   83.338368]  exc_invalid_op+0xb7/0x110
> [   83.338371]  asm_exc_invalid_op+0x1f/0x30
> [   83.338374] RIP: 0023:0x5659868a
> [   83.338376] Code: Unable to access opcode bytes at 0x56598660.
> [   83.338377] RSP: 002b:00000000df53b1c0 EFLAGS: 00010246
> [   83.338378] RAX: 00000000f453ff4d RBX: 000000005659b000 RCX: 0000000000000000
> [   83.338379] RDX: 0000000000000012 RSI: 00000000f453ff4d RDI: 00000000f7ed3ac8
> [   83.338380] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
> [   83.338381] R10: 0000000000000000 R11: 0000000000000296 R12: 0000000000000000
> [   83.338382] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [   83.338386]  </TASK>
> [   83.338387] irq event stamp: 0
> [   83.338388] hardirqs last  enabled at (0): [<0000000000000000>] 0x0
> [   83.338389] hardirqs last disabled at (0): [<ffffffff95cb68eb>] copy_process+0x7fb/0x21b0
> [   83.338393] softirqs last  enabled at (0): [<ffffffff95cb68ef>] copy_process+0x7ff/0x21b0
> [   83.338395] softirqs last disabled at (0): [<0000000000000000>] 0x0
> [   83.338396] ---[ end trace 0000000000000000 ]---
> 
> Do you think, this is related to the message I get on execution.

I don't think so unless we actually ran into a fault in your 32-bit case.

This one here is unhappy that a fault might be taken with hard IRQs off,
see __might_fault:

 #if defined(CONFIG_PROVE_LOCKING) || defined(CONFIG_DEBUG_ATOMIC_SLEEP)
 void __might_fault(const char *file, int line)
 {
+       /*
+        * When running over the oob stage (e.g. some co-kernel's own
+        * thread), we should only make sure to run with hw IRQs
+        * enabled before accessing the memory.
+        */
+       if (running_oob()) {
+               WARN_ON_ONCE(hard_irqs_disabled());
+               return;
+       }

> 
> Other question: By "stress faulting" you mean running tests while generating a lot of cpu load?

No, I meant constructing fault-causing sigframes or stacks so that we
actually take faults. Normally, the involved memory should all be
present and locked into the process.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux