Re: [PATCH 00/10] ima: measure events between kexec load and execute

[Tushar Sugandhi <tusharsu@linux.microsoft.com>]

On 7/11/23 10:51, Tushar Sugandhi wrote:
> Thanks for reviewing this series Mimi. Appreciate it.
> 
> Adding Eric to cc.
> 
> On 7/7/23 08:55, Mimi Zohar wrote:
>> On Mon, 2023-07-03 at 14:56 -0700, Tushar Sugandhi wrote:
>>> The current Kernel behavior is IMA measurements snapshot is taken at
>>> kexec 'load' and not at kexec 'execute'.  IMA log is then carried
>>> over to the new Kernel after kexec 'execute'.
>>>
>>> Some devices can be configured to call kexec 'load' first, and followed
>>> by kexec 'execute' after some time. (as opposed to calling 'load' and
>>> 'execute' in one single kexec command).  In such scenario, if new IMA
>>> measurements are added between kexec 'load' and kexec 'execute', the
>>> TPM PCRs are extended with the IMA events between 'load' and 'execute';
>>> but those IMA events are not carried over to the new kernel after kexec
>>> soft reboot.  This results in mismatch between TPM PCR quotes and the
>>> actual IMA measurements list after the device boots into the new kexec
>>> image.  This mismatch results in the remote attestation failing for that
>>> device.
>>>
>>> This patch series proposes a solution to solve this problem by 
>>> allocating
>>> the necessary buffer at kexec 'load' time, and populating the buffer
>>> with the IMA measurements at kexec 'execute' time.
>> Thanks, Tushar.   Depending on the IMA policy, the above problem
>> statement is correct, but not all policies are affected by it.  It's
>> also unclear why so much needs to change.  Instead of copying the
>> measurement list at kexec 'load',  using the existing method and simply
>> copying them at kexec 'exec' would suffice.
> My understanding is the buffer must be allocated at kexec ‘load’ time.
> The segment size cannot change between kexec ‘load’ and kexec ‘execute’.
> Not sure if this is a technical limitation of IMA, or KEXEC.
> 
> Could you/someone from kexec side let me know?
> 
> If my current understanding is not correct, then I agree, simply copying
> the measurements at kexec ‘execute’ using ima_dump_measurement_list should
> suffice.
> 
>>
>> Also as mentioned in comment on 3/10, the ordering of this patch set is
>> not bisect safe.  If the same method of copying the measurement list
>> was used, changing from copying at  kexec 'load'  to kexec 'exec' could
>> be done in the same patch.
>>
>> Mimi
> Ok. Based on my above response, if there is no such technical limitation
> to allocate and copy at kexec ‘execute’ – I will simply move the call
> to ima_dump_measurement_list from kexec 'load' to 'exec'.
> And that can be done inthe same patch as you mentioned.
> 
> ~Tushar
Hello all,
For the last few days, I did several experiments to allocate the memory
during kexec 'execute' rather than at the existing kexec 'load'.

My experiments either resulted in Kernel panics, or didn't copy the IMA
log entries added between 'load' and 'execute'.

As per my understanding, and also documented in several comments in
kexec code (e.g. in kexec_file.c [1]), adding buffer after allocating
control pages is not allowed. All segments need to be placed first
before any control pages are allocated.

And control page allocation happens at kexec 'load' [3][4][5].

There are several checks that ensures new buffer segments are not
allocated after control pages are allocated at kexec 'load'.
e.g. kexec_file.c[1][3], kexec_core.c[2].

If we want to allocate buffer at kexec 'execute' to address this issue
of "missing entries in IMA log during kexec load-execute window", we
will have to bypass those checks. That would be a very significant
change in kexec logic. And I do not believe its the right approach.

I believe we have to stick with the current approach of allocating
memory segments at kexec 'load', and copy whatever we can, during
kexe 'execute'.

Thanks,
Tushar

[1] https://elixir.bootlin.com/linux/latest/source/kernel/kexec_file.c#L652
int kexec_add_buffer(struct kexec_buf *kbuf)
{
...

  /*
  * Make sure we are not trying to add buffer after allocating
  * control pages. All segments need to be placed first before
  * any control pages are allocated. As control page allocation
  * logic goes through list of segments to make sure there are
  * no destination overlaps.
  */
  if (!list_empty(&kbuf->image->control_pages)) {
      WARN_ON(1);
      return -EINVAL;
  }
...
}

[2] https://elixir.bootlin.com/linux/latest/source/kernel/kexec_core.c#L352
static struct page *kimage_alloc_normal_control_pages(struct kimage *image,
                                          unsigned int order)
{
      /* Control pages are special, they are the intermediaries
       * that are needed while we copy the rest of the pages
       * to their final resting place.  As such they must
       * not conflict with either the destination addresses
       * or memory the kernel is already using.
       *
       * The only case where we really need more than one of
       * these are for architectures where we cannot disable
       * the MMU and must instead generate an identity mapped
       * page table for all of the memory.
       *
       * At worst this runs in O(N) of the image size.
       */
...
}

[3] https://elixir.bootlin.com/linux/latest/source/kernel/kexec_file.c#L299
static int
kimage_file_alloc_init(struct kimage **rimage, int kernel_fd,
                   int initrd_fd, const char __user *cmdline_ptr,
                   unsigned long cmdline_len, unsigned long flags)
{
      image->control_code_page = kimage_alloc_control_pages(image,
                                 get_order(KEXEC_CONTROL_PAGE_SIZE));
      if (!image->control_code_page) {
            pr_err("Could not allocate control_code_buffer\n");
            goto out_free_post_load_bufs;
      }
}

[4] https://elixir.bootlin.com/linux/latest/source/kernel/kexec_file.c#L366
SYSCALL_DEFINE5(kexec_file_load, int, kernel_fd, int, initrd_fd,
            unsigned long, cmdline_len, const char __user *, cmdline_ptr,
            unsigned long, flags)
{
...
ret = kimage_file_alloc_init(&image, kernel_fd, initrd_fd, cmdline_ptr,
...                          cmdline_len, flags);
}
                        
[5] https://elixir.bootlin.com/linux/latest/source/kernel/kexec.c#L63
static int kimage_alloc_init(struct kimage **rimage, unsigned long entry,
                       unsigned long nr_segments,
                       struct kexec_segment *segments,
                       unsigned long flags)
{
     ...
     image->control_code_page = kimage_alloc_control_pages(image,
                           get_order(KEXEC_CONTROL_PAGE_SIZE));
     if (!image->control_code_page) {
       pr_err("Could not allocate control_code_buffer\n");
      goto out_free_image;
     }
     ...
}

>>
>>> The solution includes:
>>>   - addition of new functionality to allocate a buffer to hold IMA
>>>     measurements at kexec 'load',
>>>
>>>   - ima functionality to suspend and resume measurements as needed 
>>> during
>>>     buffer copy at kexec 'execute',
>>>
>>>   - ima functionality for mapping the measurement list from the current
>>>     Kernel to the subsequent one,
>>>
>>>   - necessary changes to the kexec_file_load syscall, enabling it to 
>>> call
>>>     the ima functions
>>>
>>>   - registering a reboot notifier which gets called during kexec 
>>> 'execute',
>>>
>>>   - and removal of deprecated functions.
>>>
>>> The modifications proposed in this series ensure the integrity of the 
>>> ima
>>> measurements is preserved across kexec soft reboots, thus significantly
>>> improving the security of the Kernel post kexec soft reboots.
>>>
>>> There were previous attempts to fix this issue [1], [2], [3].  But they
>>> were not merged into the mainline Kernel.
>>>
>>> We took inspiration from the past work [1] and [2] while working on this
>>> patch series.
>>>
>>> References:
>>> -----------
>>>
>>> [1] [PATHC v2 5/9] ima: on soft reboot, save the measurement list
>>> https://lore.kernel.org/lkml/1472596811-9596-6-git-send-email-zohar@linux.vnet.ibm.com/
>>>
>>> [2] PATCH v2 4/6] kexec_file: Add mechanism to update kexec segments.
>>> https://lkml.org/lkml/2016/8/16/577
>>>
>>> [3] [PATCH 1/6] kexec_file: Add buffer hand-over support
>>> https://lore.kernel.org/linuxppc-dev/1466473476-10104-6-git-send-email-bauerman@linux.vnet.ibm.com/T/
>>>
>>> Tushar Sugandhi (10):
>>>    ima: implement function to allocate buffer at kexec load
>>>    ima: implement function to populate buffer at kexec execute
>>>    ima: allocate buffer at kexec load to hold ima measurements
>>>    ima: implement functions to suspend and resume measurements
>>>    kexec: implement functions to map and unmap segment to kimage
>>>    ima: update buffer at kexec execute with ima measurements
>>>    ima: remove function ima_dump_measurement_list
>>>    ima: implement and register a reboot notifier function to update 
>>> kexec
>>>      buffer
>>>    ima: suspend measurements while the kexec buffer is being copied
>>>    kexec: update kexec_file_load syscall to call ima_kexec_post_load
>>>
>>>   include/linux/ima.h                |   3 +
>>>   include/linux/kexec.h              |  13 ++
>>>   kernel/kexec_core.c                |  72 +++++++++-
>>>   kernel/kexec_file.c                |   7 +
>>>   kernel/kexec_internal.h            |   1 +
>>>   security/integrity/ima/ima.h       |   4 +
>>>   security/integrity/ima/ima_kexec.c | 211 +++++++++++++++++++++++------
>>>   security/integrity/ima/ima_queue.c |  32 +++++
>>>   8 files changed, 295 insertions(+), 48 deletions(-)
>>>