Re: EFI GetNextVariableName crashes when running under Xen, but not under Linux. efi-rs=0 works. No memmap issues

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

[-- Attachment #1: Type: text/plain, Size: 9717 bytes --]

On Tue, Jan 27, 2015 at 04:17:05PM +0000, Jan Beulich wrote:
> >>> On 27.01.15 at 15:26, <konrad.wilk@oracle.com> wrote:
> > On Tue, Jan 27, 2015 at 07:54:30AM +0000, Jan Beulich wrote:
> >> (re-adding xen-devel)
> >> 
> >> >>> On 27.01.15 at 01:32, <andrew.cooper3@citrix.com> wrote:
> >> > On 27/01/2015 00:02, Daniel Kiper wrote:
> >> >> On Mon, Jan 26, 2015 at 05:00:41PM +0000, Jan Beulich wrote:
> >> >>>>>> On 26.01.15 at 17:27, <konrad.wilk@oracle.com> wrote:
> >> >>>> Anyhow I am bit stuck:
> >> >>>>  1) It works with Linux, so what is it that Linux does that
> >> >>>>     Xen does not?
> >> >>> They map more than just what is marked for runtime use.
> >> >> IIRC, Linux maps boot services unconditionally (and states in comment
> >> >> that this is not in line with spec). We do not have such mechanism.
> >> >> Could we ease life of our users and add a boot option (e.g. map-efi-bs)
> >> >> which will enforce mapping of BS regions on platforms with buggy EFI/UEFI
> >> >> implementations? We should not penalize owners of such hardware because
> >> >> they are not guilty of these crazy bugs. We should educate firmware devs...
> >> >> Ehh... Please, do not curse at me. I remember discussion about EFI reset
> >> >> stuff which happened here a few days ago.
> >> > 
> >> > While, in principle, I would like to take a tough stand against buggy
> >> > firmware, the truth is that firmware is always going to be buggy, and
> >> > many users are going to be in a position where their buggy firmware is
> >> > not going to be fixed by their vendors.  Much as I would prefer not to,
> >> > I feel that the only rational course of action to take is to behave like
> >> > Linux in cases like this.
> >> > 
> >> > Therefore, I am a begrudgingly +1 "work around EFI firmware bugs",
> >> > despite it being the wrong pragmatic thing to do.
> >> 
> >> And I agree that we will need to accept in such workarounds. But
> >> two remarks to whoever is going to implement it: We already have
> >> the efi-rs workaround option - we should deprecate that one, and
> >> have a consolidated efi= one instead, covering the case here too.
> >> Plus the issue here is not just a matter of mapping BS memory, but
> >> also not making it available to the allocator. That in turn may yield
> >> problems with the conversion of the EFI memory map to E820 form,
> >> both because of the number of entries needed, and because that
> >> conversion happens _before_ the normal command line parsing.
> > 
> > Twisty maze. 
> > 
> > However even with my 'debug' patch and mapping the boot services
> > it still fails on this laptop. So I fear there is something more
> > to my woes with Lenovo's EFI firmware implementation.
> 
> Again - apart from mapping the range, did you also make sure it
> didn't get passed to the allocator (and hence couldn't have got
> overwritten)?

Yes, see patch:

Also see attached of the code with what Linux sees and what Xen sees
(Linux first). I am thinking that the firmware is under the assumption
that if SetVirtualAddressMap is not called then you MUST be still
before ExitBootServices has been called. Going to verify that by
implementing an GetNextVariableName before calling ExitBootServices)

diff --git a/xen/Rules.mk b/xen/Rules.mk
index b4315a5..6692242 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -7,10 +7,10 @@ verbose       ?= y
 perfc         ?= n
 perfc_arrays  ?= n
 lock_profile  ?= n
-crash_debug   ?= y
-frame_pointer ?= y
+crash_debug   ?= n
+frame_pointer ?= n
 lto           ?= n
-debug := y
+debug := n
 
 include $(XEN_ROOT)/Config.mk
 
diff --git a/xen/arch/x86/efi/efi-boot.h b/xen/arch/x86/efi/efi-boot.h
index 3a3b4fe..c3bdb8d 100644
--- a/xen/arch/x86/efi/efi-boot.h
+++ b/xen/arch/x86/efi/efi-boot.h
@@ -152,8 +152,6 @@ static void __init efi_arch_process_memory_map(EFI_SYSTEM_TABLE *SystemTable,
             type = E820_RESERVED;
             break;
         case EfiConventionalMemory:
-        case EfiBootServicesCode:
-        case EfiBootServicesData:
             if ( !trampoline_phys && desc->PhysicalStart + len <= 0x100000 &&
                  len >= cfg.size && desc->PhysicalStart + len > cfg.addr )
                 cfg.addr = (desc->PhysicalStart + len - cfg.size) & PAGE_MASK;
diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
index c11b572..e7c939e 100644
--- a/xen/common/efi/boot.c
+++ b/xen/common/efi/boot.c
@@ -1159,17 +1221,27 @@ void __init efi_init_memory(void)
         u64 len = desc->NumberOfPages << EFI_PAGE_SHIFT;
         unsigned long smfn, emfn;
         unsigned int prot = PAGE_HYPERVISOR;
+        unsigned int skip = 1;
 
         printk(XENLOG_INFO " %013" PRIx64 "-%013" PRIx64
                            " type=%u attr=%016" PRIx64 "\n",
                desc->PhysicalStart, desc->PhysicalStart + len - 1,
                desc->Type, desc->Attribute);
 
-        if ( !efi_rs_enable || !(desc->Attribute & EFI_MEMORY_RUNTIME) )
-	{
-	    printk(XENLOG_INFO " .. skipped!\n");
+        if ( desc->Attribute & EFI_MEMORY_RUNTIME )
+            skip = 0;
+
+        if ( desc->Type == 4 && desc->Attribute != 0 )
+            skip = 0;
+
+        if ( desc->Type == 3 && desc->Attribute != 0 )
+            skip = 0;
+
+        if ( !efi_rs_enable || skip )
+        {
+            printk(XENLOG_INFO " .. skipped!\n");
             continue;
-	}
+        }
         desc->VirtualStart = INVALID_VIRTUAL_ADDRESS;
 
         smfn = PFN_DOWN(desc->PhysicalStart);
@@ -1246,18 +1318,28 @@ void __init efi_init_memory(void)
 
     copy_mapping(0, max_page, ram_range_valid);
 
+    printk(XENLOG_INFO "Copying..\n");
     /* Insert non-RAM runtime mappings inside the direct map. */
     for ( i = 0; i < efi_memmap_size; i += efi_mdesc_size )
     {
         const EFI_MEMORY_DESCRIPTOR *desc = efi_memmap + i;
 
-        if ( (desc->Attribute & EFI_MEMORY_RUNTIME) &&
+        if ( ((desc->Attribute & EFI_MEMORY_RUNTIME) ||
+             (desc->Type == 3 && desc->Attribute != 0 ) ||
+             (desc->Type == 4 && desc->Attribute != 0 )) &&
              desc->VirtualStart != INVALID_VIRTUAL_ADDRESS &&
-             desc->VirtualStart != desc->PhysicalStart )
+             desc->VirtualStart != desc->PhysicalStart ) {
+        
+            printk(XENLOG_INFO " %013" PRIx64 "-%013" PRIx64
+                           " type=%u attr=%016" PRIx64 "\n",
+               PFN_DOWN(desc->PhysicalStart), PFN_UP(desc->PhysicalStart + (desc->NumberOfPages << EFI_PAGE_SHIFT)),
+               desc->Type, desc->Attribute);
+
             copy_mapping(PFN_DOWN(desc->PhysicalStart),
                          PFN_UP(desc->PhysicalStart +
                                 (desc->NumberOfPages << EFI_PAGE_SHIFT)),
                          rt_range_valid);
+            }
     }
 
     /* Insert non-RAM runtime mappings outside of the direct map. */
diff --git a/xen/common/efi/runtime.c b/xen/common/efi/runtime.c
index 0750436..15401a4 100644
--- a/xen/common/efi/runtime.c
+++ b/xen/common/efi/runtime.c
@@ -146,6 +146,44 @@ static void _delay(void)
     }
     printk("\n");
 }
+static void _dumpcode(char *code, unsigned long s, unsigned long e)
+{
+    unsigned long idx, e_idx;
+    unsigned long cr3;
+    unsigned int i;
+
+    if ( s > e )
+        return;
+
+    idx = s;
+
+    printk("%lx -> %lx\nCode: ", s, e);
+    do {
+        e_idx = idx + 4095;
+        if ( e_idx > e )
+            e_idx = e;
+
+        process_pending_softirqs();
+
+        memset(code, 0, 4096);
+
+        cr3 = efi_rs_enter();
+        memcpy(code, (void *)idx, e_idx - idx);
+        efi_rs_leave(cr3);
+
+        for ( i = 0; i < e_idx - idx ;i++)
+        {
+            if ( i & 0xFF )
+                process_pending_softirqs();
+            printk(" %02x", (unsigned short)code[i] & 0xFF);
+        }
+        printk("\n");
+        idx = e_idx + 1;
+    } while ( idx < e );
+
+    printk("\n");
+    _delay();
+}
 
 long efi_debug(void)
 {
@@ -162,12 +200,13 @@ long efi_debug(void)
     unsigned int rev;
     unsigned long getnext, get;
     char *code;
+    unsigned long val[13];
 
     if ( !cr3 )
         return -EOPNOTSUPP;
     efi_rs_leave(cr3);
 
-    code = xzalloc_bytes(size);
+    code = xzalloc_bytes(4096);
     if ( !code )
         return -ENOMEM;
 
@@ -193,18 +232,41 @@ long efi_debug(void)
 
     cr3 = efi_rs_enter();
     getnext = (unsigned long)efi_rs->GetNextVariableName;
-    memcpy(code, efi_rs->GetNextVariableName, 1024);
-    get = (unsigned long)efi_rs->GetVariable;
+    get = (unsigned long)efi_rs;
     efi_rs_leave(cr3);
 
-    printk(", GetNextVariableName: %lx, GetVariable: %lx\n", getnext, get);
-    printk(" Code: ");
-    for ( i = 0; i < 1024;i++)
-        printk(" %02x", (unsigned short)code[i] & 0xFF);
-    printk("\n");
+    printk(", GetNextVariableName: %lx, efi_rs: %lx\n", getnext, get);
+
+    val[0] = 0xcfdba230;	/* Saw it somewhere Boot Services?? */
+    val[1] = 0xcfdba270;
+    val[2] = val[0] + 0x18;
+    val[3] = val[1] + 0x18;
+    val[4] = 0xcfdc9cc0;
+	val[5] = getnext + + 0x11bc; /* 3f: */
+    val[6] = getnext + 0x11fc;
+    val[7] = getnext + 0x11e4;
+    val[8] = getnext + 0x1154;
+    val[9] = getnext + 0x116c;
+    val[10] = getnext + 0x11d4;
+    val[11] = getnext + 0x1154;
+    val[12] = getnext + 0x116c;
+
+    for ( i = 0; i < 13; i++)
+	{
+		printk("val[%d]:\n", i);
+        _dumpcode(code, val[i], val[i] + 8);
+	}
+#if 0
+    _dumpcode(code, get, get+4096);
+    _delay();
 
+    _dumpcode(code, 0x00000d6929000, 0x00000d6a4ffff);
     _delay();
 
+    _dumpcode(code, 0x00000cfdba000,0x00000cfdcffff);
+    _delay();
+#endif
+    _dumpcode(code, 0, 512);
     idx = 1;
     do {
         printk("%4d:", idx++);
> 
> Jan
> 

[-- Attachment #2: print.txt --]
[-- Type: text/plain, Size: 7204 bytes --]

   0:	48 89 5c 24 08       	mov    %rbx,0x8(%rsp)
   5:	48 89 6c 24 10       	mov    %rbp,0x10(%rsp)
   a:	48 89 74 24 18       	mov    %rsi,0x18(%rsp)
   f:	57                   	push   %rdi
  10:	41 54                	push   %r12
  12:	41 55                	push   %r13
  14:	48 83 ec 20          	sub    $0x20,%rsp
  18:	45 33 ed             	xor    %r13d,%r13d
  1b:	48 85 c9             	test   %rcx,%rcx
  1e:	4d 8b e0             	mov    %r8,%r12
  21:	48 8b fa             	mov    %rdx,%rdi
  24:	48 8b e9             	mov    %rcx,%rbp
  27:	0f 84 09 01 00 00    	je     0x136
  2d:	48 85 d2             	test   %rdx,%rdx
  30:	0f 84 00 01 00 00    	je     0x136
  36:	4d 85 c0             	test   %r8,%r8
  39:	0f 84 f7 00 00 00    	je     0x136
  3f:	48 8b 05 76 11 00 00 	mov    0x1176(%rip),%rax        # 0x11bc [20 53 c3 fa fe ff ff ff][20 53 a3 d6 00 00 00 00
  46:	48 8d 15 af 11 00 00 	lea    0x11af(%rip),%rdx        # 0x11fc [00 54 f3 41 60 06 1c 8][00 6d 15 d8 d6 db 40 8d
  4d:	48 8b c8             	mov    %rax,%rcx
  50:	ff 50 20             	callq  *0x20(%rax)
  53:	80 3d a2 11 00 00 01 	cmpb   $0x1,0x11a2(%rip)        # 0x11fc [00 54 f3 41 60 06 1c 8][00 6d 15 d8 d6 db 40 8d]
  5a:	75 1b                	jne    0x77
  5c:	48 8b 05 81 11 00 00 	mov    0x1181(%rip),%rax        # 0x11e4 [80 62 2b db 00 00 00 00][80 62 2b db 00 00 00 0]
  63:	4d 8b c4             	mov    %r12,%r8
  66:	48 8b d7             	mov    %rdi,%rdx
  69:	48 8b cd             	mov    %rbp,%rcx
  6c:	ff 50 08             	callq  *0x8(%rax)
  6f:	48 8b d8             	mov    %rax,%rbx
  72:	e9 ba 00 00 00       	jmpq   0x131
  77:	48 8b cf             	mov    %rdi,%rcx
  7a:	e8 bd 0f 00 00       	callq  0x103c
  7f:	48 3d 00 01 00 00    	cmp    $0x100,%rax
  85:	0f 87 ab 00 00 00    	ja     0x136
  8b:	44 38 2d c2 10 00 00 	cmp    %r13b,0x10c2(%rip)        # 0x115 [01 01 00 00 00 00 00 00][00 01 00 00 00 00 00 00
  92:	75 12                	jne    0xa6
  94:	48 8b 05 d1 10 00 00 	mov    0x10d1(%rip),%rax        # 0x116c [70 a2 db cf 00 00 00 00][70 a2 db cf 00 00 00 00
  9b:	b9 1f 00 00 00       	mov    $0x1f,%ecx
  a0:	ff 50 18             	callq  *0x18(%rax)
  a3:	4c 8b e8             	mov    %rax,%r13
  a6:	48 8b 35 27 11 00 00 	mov    0x1127(%rip),%rsi        # 0x11d4 [10 d0 87 fa fe ff ff ff][10 d0 47 da 00 00 00 00
  ad:	48 8b d7             	mov    %rdi,%rdx
  b0:	c6 06 5a             	movb   $0x5a,(%rsi)
  b3:	c6 46 01 6b          	movb   $0x6b,0x1(%rsi)
  b7:	48 8b 4d 00          	mov    0x0(%rbp),%rcx
  bb:	48 89 4e 18          	mov    %rcx,0x18(%rsi)
  bf:	48 8d 4e 20          	lea    0x20(%rsi),%rcx
  c3:	e8 58 0f 00 00       	callq  0x1020
  c8:	48 8d 8e 20 02 00 00 	lea    0x220(%rsi),%rcx
  cf:	41 b8 10 00 00 00    	mov    $0x10,%r8d
  d5:	49 8b d4             	mov    %r12,%rdx
  d8:	e8 ff 0e 00 00       	callq  0xfdc
  dd:	e8 3a fb ff ff       	callq  0xfffffffffffffc1c
  e2:	44 8a 1e             	mov    (%rsi),%r11b
  e5:	41 80 fb 5a          	cmp    $0x5a,%r11b
  e9:	74 bb                	je     0xa6
  eb:	48 8b 5e 08          	mov    0x8(%rsi),%rbx
  ef:	48 8b 46 18          	mov    0x18(%rsi),%rax
  f3:	48 85 db             	test   %rbx,%rbx
  f6:	48 89 45 00          	mov    %rax,0x0(%rbp)
  fa:	75 1f                	jne    0x11b
  fc:	48 8d 56 20          	lea    0x20(%rsi),%rdx
 100:	48 8b cf             	mov    %rdi,%rcx
 103:	e8 18 0f 00 00       	callq  0x1020
 108:	48 8d 96 20 02 00 00 	lea    0x220(%rsi),%rdx
 10f:	44 8d 43 10          	lea    0x10(%rbx),%r8d
 113:	49 8b cc             	mov    %r12,%rcx
 116:	e8 c1 0e 00 00       	callq  0xfdc
 11b:	80 3d 32 10 00 00 00 	cmpb   $0x0,0x1032(%rip)        # 0x1154 [70 a2 db cf 00 00 00 00][00 01 00 00 00 00 00 00]
 122:	75 0d                	jne    0x131
 124:	48 8b 05 41 10 00 00 	mov    0x1041(%rip),%rax        # 0x116c [70 a2 db cf 00 00 00 00][70 a2 db cf 00 00 00 00]
 12b:	49 8b cd             	mov    %r13,%rcx
 12e:	ff 50 20             	callq  *0x20(%rax)
 131:	48 8b c3             	mov    %rbx,%rax
 134:	eb 0a                	jmp    0x140
 136:	48 b8 02 00 00 00 00 	movabs $0x8000000000000002,%rax
 13d:	00 00 80 
 140:	48 8b 5c 24 40       	mov    0x40(%rsp),%rbx
 145:	48 8b 6c 24 48       	mov    0x48(%rsp),%rbp
 14a:	48 8b 74 24 50       	mov    0x50(%rsp),%rsi
 14f:	48 83 c4 20          	add    $0x20,%rsp
 153:	41 5d                	pop    %r13
 155:	41 5c                	pop    %r12
 157:	5f                   	pop    %rdi
 158:	c3                   	retq   
 159:	cc                   	int3   
 15a:	cc                   	int3   
 15b:	cc                   	int3   
(XEN)    1:----[ Xen-4.6-unstable  x86_64  debug=n  Not tainted ]---- 
(XEN) CPU:    0
(XEN) RIP:    e008:[<0000000000000007>] 0000000000000007
(XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor
(XEN) rax: 00000000cfdba270   rbx: ffff830214cfea00   rcx: 000000000000001f
(XEN) rdx: 00000000d6995ed0   rsi: 0000000000150670   rdi: ffff830214cfe580
(XEN) rbp: ffff82d080457d80   rsp: ffff82d080457cf0   r8:  ffff82d080457d88
(XEN) r9:  0000000000008000   r10: ffff82d080457bfc   r11: 00000000db002700
(XEN) r12: ffff82d080457d88   r13: 0000000000000000   r14: 0000000000000001
(XEN) r15: 00000000d1079000   cr0: 0000000080050033   cr4: 00000000001506f0
(XEN) cr3: 0000000216b3b000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen stack trace from rsp=ffff82d080457cf0:
(XEN)    0000000068f00002 00000000d6994d77 ffff82d080498b30 0000000000000206
(XEN)    00000000d1079000 ffff830214cfe580 00000000d6995e40 ffff82d080457d90
(XEN)    0000000216b3b000 ffff82d080229e16 ffff830214cfea00 ffff830214cfe580
(XEN)    0000000000150670 ffff82d080229de6 000000000000000a ffff82d080457d88
(XEN)    ffff82d080457d80 ffff830214cf3000 0000000000000400 0000000000000246
(XEN)    ffff82d080457da0 00000000cfdba230 00000000cfdba270 00000000cfdba248
(XEN)    00000000cfdba288 00000000cfdc9cc0 00000000d6995e90 00000000d6995ed0
(XEN)    00000000d6995eb8 00000000d6995e28 00000000d6995e40 00000000d6995ea8
(XEN)    00000000d6995e28 00000000d6995e40 0000000000000003 ffff830216b314f0
(XEN)    0000000000000002 0000000000000008 0000000000000003 ffff8300d124b040
(XEN)    ffff82d080269d80 ffff82d0804259b6 ffff8300d124b040 ffff8300d124afe0
(XEN)    00007d2f00000002 ffff8300d123ac25 00000000012b0000 000000021ab35000
(XEN)    0000000000000000 00000000ffffffff 000000000021e600 0000000000000000
(XEN)    00000000d124afe0 ffffffd080499780 0000000000499780 00000000012b0fff
(XEN)    0000000000100000 0058bf9000000000 0000000800000000 000000010000006e
(XEN)    0000000000000003 00000000000002f8 0000000000000000 00000000d123a280
(XEN)    00000000d0793408 00000000d0eff388 0000000000057000 00000000fed20000
(XEN)    0000000000002960 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<0000000000000007>] 0000000000000007
(XEN)    [<ffff82d080229e16>] efi_debug+0x246/0x3b0
(XEN)    [<ffff82d080229de6>] efi_debug+0x216/0x3b0
(XEN)    [<ffff82d0804259b6>] __start_xen+0x25b6/0x3bc0

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel