Got a WARN_ON for supposedly unreachable code.

Got a WARN_ON for supposedly unreachable code.

[Ilya Lesokhin]

Hi, 
Just in case anyone is interested, I've hit a WARN_ON that shouldn't happen:
http://elixir.free-electrons.com/linux/v4.13.10/source/arch/x86/kernel/traps.c#L788

I was single stepping in GDB connected to a QEMU target
and got the trace below inside the VM.

I'm not sure If it's a kernel bug or a KVM bug and I did try to reproduce or debug it.

Hypervisor was running 3.10.0-514.21.1.el7.x86_64.
VM was running a modified 4.13.0.

Thanks,
Ilya


[  750.284330] ------------[ cut here ]------------
[  750.286461] WARNING: CPU: 3 PID: 29 at arch/x86/kernel/traps.c:776 do_debug+0x102/0x1f0
[  750.286468] Modules linked in: xt_nat iptable_nat mlx5_ib mlx5_core tls ptp pps_core nfsv3 nfs_acl rpcsec_gss_krb5 auth_rpcgss oid_registry nfsv4 nfs lockd grace sunrpc fscache vport_vxlan vxlan ip6_udp_tunnel udp_tunnel openvswitch nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_defrag_ipv6 nf_nat nf_conntrack rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm configfs ib_cm iw_cm dm_mirror dm_region_hash dm_log dm_mod dax snd_hda_codec_generic ib_core ppdev snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm pcspkr virtio_balloon snd_timer snd soundcore parport_pc parport i2c_piix4 i2c_core uinput ip_tables xfs libcrc32c virtio_blk virtio_net ata_generic pata_acpi ahci libahci virtio_pci virtio_ring virtio ata_piix floppy serio_raw autofs4 [last unloaded: tls]
[  750.286893] CPU: 3 PID: 29 Comm: ksoftirqd/3 Not tainted 4.13.0-tls+ #108
[  750.286899] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
[  750.286905] task: ffff88013a30c800 task.stack: ffffc90000728000
[  750.286914] RIP: 0010:do_debug+0x102/0x1f0
[  750.286920] RSP: 0018:ffff88013fd87f20 EFLAGS: 00010246
[  750.286931] RAX: ffff88013a30c800 RBX: ffff88013fd87f58 RCX: 0000000000000002
[  750.286937] RDX: 0000000000001801 RSI: 0000000000000000 RDI: ffff88013a30c800
[  750.286943] RBP: ffff88013fd87f48 R08: 0000000000000000 R09: 0000000000000000
[  750.286949] R10: 0000000000000001 R11: 0000000000000000 R12: ffff88013a30c800
[  750.286955] R13: 0000000000000000 R14: 0000000000000000 R15: ffff88013441a0a0
[  750.286962] FS:  0000000000000000(0000) GS:ffff88013fd80000(0000) knlGS:0000000000000000
[  750.286968] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  750.286974] CR2: 00007fa3972905d0 CR3: 000000012b1de000 CR4: 00000000000006e0
[  750.286988] Call Trace:
[  750.287043]  <#DB>
[  750.287056]  debug+0x3e/0x70
[  750.287067] RIP: 0010:skb_checksum_help+0x1d/0x1c0
[  750.287073] RSP: 0018:ffffc9000072bce0 EFLAGS: 00000306
[  750.287084] RAX: 00000000000000c0 RBX: ffff880132aa0600 RCX: 0000000000000000
[  750.287089] RDX: 00000000000000c0 RSI: 0000000000000000 RDI: ffff880132aa0600
[  750.287095] RBP: ffffc9000072bd00 R08: 0000000000000001 R09: 0000000000000001
[  750.287101] R10: 0000000000000001 R11: 0000000000000000 R12: 0000002000004a21
[  750.287107] R13: ffff880133cd1000 R14: 0000000000000000 R15: ffff88013441a0a0
[  750.287123]  </#DB>
[  750.287134]  skb_csum_hwoffload_help+0x25/0x40
[  750.287142]  validate_xmit_skb+0x20f/0x2e0
[  750.287154]  validate_xmit_skb_list+0x42/0x70
[  750.287165]  sch_direct_xmit+0xb4/0x190
[  750.287175]  __qdisc_run+0xb2/0x420
[  750.287188]  net_tx_action+0x18f/0x3c0
[  750.287195]  ? __do_softirq+0xd1/0x48a
[  750.287205]  __do_softirq+0xd1/0x48a
[  750.287219]  run_ksoftirqd+0x25/0x70
[  750.287228]  smpboot_thread_fn+0x11a/0x1e0
[  750.287239]  kthread+0x152/0x190
[  750.287246]  ? sort_range+0x30/0x30
[  750.287254]  ? kthread_create_on_node+0x40/0x40
[  750.287264]  ret_from_fork+0x2a/0x40
[  750.287279] Code: 00 3d 01 80 00 00 74 aa 65 ff 05 ce 22 ff 7e f6 83 91 00 00 00 02 0f 85 8d 00 00 00 f6 45 d9 40 74 28 f6 83 88 00 00 00 03 75 1f <0f> ff 49 81 a4 24 e8 1d 00 00 ff bf ff ff f0 41 80 0c 24 10 48 
[  750.287688] ---[ end trace ba659993f997ef67 ]---

Thanks,
Ilya

Re: Got a WARN_ON for supposedly unreachable code.

[Paolo Bonzini]

On 02/11/2017 12:27, Ilya Lesokhin wrote:
> Hi, 
> Just in case anyone is interested, I've hit a WARN_ON that shouldn't happen:
> http://elixir.free-electrons.com/linux/v4.13.10/source/arch/x86/kernel/traps.c#L788
> 
> I was single stepping in GDB connected to a QEMU target
> and got the trace below inside the VM.
> 
> I'm not sure If it's a kernel bug or a KVM bug and I did try to reproduce or debug it.
> 
> Hypervisor was running 3.10.0-514.21.1.el7.x86_64.
> VM was running a modified 4.13.0.

It's a KVM bug, though I'm not sure if it's easily fixable.

Paolo

Re: Got a WARN_ON for supposedly unreachable code.

[Nadav Amit]

Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 02/11/2017 12:27, Ilya Lesokhin wrote:
>> Hi, 
>> Just in case anyone is interested, I've hit a WARN_ON that shouldn't happen:
>> http://elixir.free-electrons.com/linux/v4.13.10/source/arch/x86/kernel/traps.c#L788
>> 
>> I was single stepping in GDB connected to a QEMU target
>> and got the trace below inside the VM.
>> 
>> I'm not sure If it's a kernel bug or a KVM bug and I did try to reproduce or debug it.
>> 
>> Hypervisor was running 3.10.0-514.21.1.el7.x86_64.
>> VM was running a modified 4.13.0.
> 
> It's a KVM bug, though I'm not sure if it's easily fixable.

What’s wrong with MTF? That’s what I used for debugging the Intel tests.

Re: Got a WARN_ON for supposedly unreachable code.

[Paolo Bonzini]

On 02/11/2017 18:56, Nadav Amit wrote:
> Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
>> On 02/11/2017 12:27, Ilya Lesokhin wrote:
>>> Hi, 
>>> Just in case anyone is interested, I've hit a WARN_ON that shouldn't happen:
>>> http://elixir.free-electrons.com/linux/v4.13.10/source/arch/x86/kernel/traps.c#L788
>>>
>>> I was single stepping in GDB connected to a QEMU target
>>> and got the trace below inside the VM.
>>>
>>> I'm not sure If it's a kernel bug or a KVM bug and I did try to reproduce or debug it.
>>>
>>> Hypervisor was running 3.10.0-514.21.1.el7.x86_64.
>>> VM was running a modified 4.13.0.
>>
>> It's a KVM bug, though I'm not sure if it's easily fixable.
> 
> What’s wrong with MTF? That’s what I used for debugging the Intel tests.

Nothing, but I haven't checked if you might get the same failure on
AMD---which doesn't have it.

Thanks,

Paolo

Re: Got a WARN_ON for supposedly unreachable code.

[Andy Lutomirski]

> On Nov 2, 2017, at 6:57 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
>> On 02/11/2017 18:56, Nadav Amit wrote:
>> Paolo Bonzini <pbonzini@redhat.com> wrote:
>> 
>>>> On 02/11/2017 12:27, Ilya Lesokhin wrote:
>>>> Hi, 
>>>> Just in case anyone is interested, I've hit a WARN_ON that shouldn't happen:
>>>> http://elixir.free-electrons.com/linux/v4.13.10/source/arch/x86/kernel/traps.c#L788
>>>> 
>>>> I was single stepping in GDB connected to a QEMU target
>>>> and got the trace below inside the VM.
>>>> 
>>>> I'm not sure If it's a kernel bug or a KVM bug and I did try to reproduce or debug it.
>>>> 
>>>> Hypervisor was running 3.10.0-514.21.1.el7.x86_64.
>>>> VM was running a modified 4.13.0.
>>> 
>>> It's a KVM bug, though I'm not sure if it's easily fixable.
>> 
>> What’s wrong with MTF? That’s what I used for debugging the Intel tests.
> 
> Nothing, but I haven't checked if you might get the same failure on
> AMD---which doesn't have it.

Is this that old SYSCALL CVE?

> 
> Thanks,
> 
> Paolo

Re: Got a WARN_ON for supposedly unreachable code.

[Paolo Bonzini]

On 02/11/2017 19:17, Andy Lutomirski wrote:
> 
> 
>> On Nov 2, 2017, at 6:57 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>>> On 02/11/2017 18:56, Nadav Amit wrote:
>>> Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>
>>>>> On 02/11/2017 12:27, Ilya Lesokhin wrote:
>>>>> Hi, 
>>>>> Just in case anyone is interested, I've hit a WARN_ON that shouldn't happen:
>>>>> http://elixir.free-electrons.com/linux/v4.13.10/source/arch/x86/kernel/traps.c#L788
>>>>>
>>>>> I was single stepping in GDB connected to a QEMU target
>>>>> and got the trace below inside the VM.
>>>>>
>>>>> I'm not sure If it's a kernel bug or a KVM bug and I did try to reproduce or debug it.
>>>>>
>>>>> Hypervisor was running 3.10.0-514.21.1.el7.x86_64.
>>>>> VM was running a modified 4.13.0.
>>>>
>>>> It's a KVM bug, though I'm not sure if it's easily fixable.
>>>
>>> What’s wrong with MTF? That’s what I used for debugging the Intel tests.
>>
>> Nothing, but I haven't checked if you might get the same failure on
>> AMD---which doesn't have it.
> 
> Is this that old SYSCALL CVE?

No, he's just using QEMU's gdb server and bits of DR6 sometimes sneak
into a guest that is e.g. using watchpoints.

Paolo

Re: Got a WARN_ON for supposedly unreachable code.

[Andy Lutomirski]

On Thu, Nov 2, 2017 at 11:25 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> On 02/11/2017 19:17, Andy Lutomirski wrote:
>>
>>
>>> On Nov 2, 2017, at 6:57 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>
>>>> On 02/11/2017 18:56, Nadav Amit wrote:
>>>> Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>>
>>>>>> On 02/11/2017 12:27, Ilya Lesokhin wrote:
>>>>>> Hi,
>>>>>> Just in case anyone is interested, I've hit a WARN_ON that shouldn't happen:
>>>>>> http://elixir.free-electrons.com/linux/v4.13.10/source/arch/x86/kernel/traps.c#L788
>>>>>>
>>>>>> I was single stepping in GDB connected to a QEMU target
>>>>>> and got the trace below inside the VM.
>>>>>>
>>>>>> I'm not sure If it's a kernel bug or a KVM bug and I did try to reproduce or debug it.
>>>>>>
>>>>>> Hypervisor was running 3.10.0-514.21.1.el7.x86_64.
>>>>>> VM was running a modified 4.13.0.
>>>>>
>>>>> It's a KVM bug, though I'm not sure if it's easily fixable.
>>>>
>>>> What’s wrong with MTF? That’s what I used for debugging the Intel tests.
>>>
>>> Nothing, but I haven't checked if you might get the same failure on
>>> AMD---which doesn't have it.
>>
>> Is this that old SYSCALL CVE?
>
> No, he's just using QEMU's gdb server and bits of DR6 sometimes sneak
> into a guest that is e.g. using watchpoints.
>

If you have a good description of the symptoms from the guest's
perspective, I can try to fix up the entry code to handle it more
gracefully and to at least say something like "congrats!  you just hit
a QEMU bug.".