Re: Memory management facing a 400Gpbs network link

[Jerome Glisse <jglisse@redhat.com>]

On Thu, Feb 21, 2019 at 06:15:14PM +0000, Christopher Lameter wrote:
> On Wed, 20 Feb 2019, Michal Hocko wrote:
> 
> > > I dont like the existing approaches but I can present them?
> >
> > Please give us at least some rough outline so that we can evaluate a
> > general interest and see how/whether to schedule such a topic.
> 
> Ok. I am fuzzy on this one too. Lets give this another shot:
> 
> In the HPC world we often have to bypass operating system mechanisms for
> full speed. Usually this has been through accellerators in the network
> card, in sharing memory between multiple systems (with NUMA being a
> special case of this) or with devices that provide some specialized memory
> access. There is a whole issue here with pinned memory access (I think
> that is handled in another session at the MM summit)
> 
> The intend was typically to bring the data into the system so that an
> application can act on it. However, with the increasing speeds of the
> interconnect that may even be faster than the internal busses on
> contemporary platforms that may have to change since the processor and the
> system as a whole is no longer able to handle the inbound data stream.
> This is partially due to the I/O bus speeds no longer increasing.
> 
> The solutions to this issue coming from some vendors are falling
> mostly into the following categories:
> 
> A) Provide preprocessing in the NIC.
> 
>    This can compress data, modify it and direct it to certain cores of
>    the system. Preprocessing may allow multiple hosts to use one NIC
>    (Makes sense since a single host may no longer be able to handle the
>    data).
> 
> B) Provide fast memory in the NIC
> 
>    Since the NIC is at capacity limits when it comes to pushing data
>    from the NIC into memory the obvious solution is to not go to main
>    memory but provide faster on NIC memory that can then be accessed
>    from the host as needed. Now the applications creates I/O bottlenecks
>    when accessing their data or they need to implement complicated
>    transfer mechanisms to retrieve and store data onto the NIC memory.
> 
> C) Direct passthrough to other devices
> 
>    The host I/O bus is used or another enhanced bus is provided to reach
>    other system components without the constraints imposed by the OS or
>    hardware. This means for example that a NIC can directly write to an
>    NVME storage device (f.e. NVMEoF). A NIC can directly exchange data with
>    another NIC. In an extreme case a hardware addressable global data fabric
>    exists that is shared between multiple systems and the devices can
>    share memory areas with one another. In the ultra extreme case there
>    is a bypass  even using the memory channels since non volatile memory
>    (a storage device essentially) is now  supported that way.
> 
> All of this leads to the development of numerous specialized accellerators
> and special mechamisms to access memory on such devices. We already see a
> proliferation of various remote memory schemes (HMM, PCI device memory
> etc)
> 
> So how does memory work in the systems of the future? It seems that we may
> need some new way of tracking memory that is remote on some device in
> additional to the classic NUMA nodes? Or can we change the existing NUMA
> schemes to cover these use cases?
> 
> We need some consistent and hopefully vendor neutral way to work with
> memory I think.

Note that i proposed a topic about that [1] NUMA is really hard to work
with for device memory and adding memory that might not be cache coherent
or not support atomic operation, is not a good idea to report as regular
NUMA as existing application might start using such memory unaware of all
its peculiarities.

Anyway it is definitly a topic i beliew we need to discuss and i intend
to present the problem from GPU/accelerator point of view (as today this
are the hardware with sizeable fast local memory).

Cheers,
Jérôme

[1] https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1904033.html


> 
> 
> 
> 
> 
> ----- Old proposal
> 
> 
> 00G Infiniband will become available this year. This means that the data
> ingest speeds can be higher than the bandwidth of the processor
> interacting with its own memory.
> 
> For example a single hardware thread is limited to 20Gbyte/sec whereas the
> network interface provides 50Gbytes/sec. These rates can only be obtained
> currently with pinned memory.
> 
> How can we evolve the memory management subsystem to operate at higher
> speeds with more the comforts of paging and system calls that we are used
> to?
> 
> It is likely that these speeds with increase further and since the lead
> processor vendor seems to be caught in a management induced corporate
> suicide attempt we will not likely see any process on the processors from
> there. The straightforward solution would be to use the high speed tech
> for fabrics for the internal busses (doh!). Alternate processors are
> likely to show up in 2019 and 2020 but those will take a long time to
> mature.
> 
> So what does the future hold and how do we scale up our HPC systems given
> these problems?
>