Re: File system for scratch space (in HPC cluster)

[Paul Menzel <pmenzel@molgen.mpg.de>]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

Dear Andreas,


On 2019-10-24 19:51, Andreas Dilger wrote:
> On Oct 24, 2019, at 4:43 AM, Paul Menzel <pmenzel@molgen.mpg.de> 
> wrote:

>> In our cluster, we offer scratch space for temporary files. As 
>> these files are temporary, we do not need any safety requirements
>> – especially not those when the system crashes or shuts down. So
>> no `sync` is for example needed.
>> 
>> Are there file systems catering to this need? I couldn’t find any? 
>> Maybe I missed some options for existing file systems.
> 
> How big do you need the scratch filesystem to be?  Is it local to
> the node or does it need to be shared between nodes?

In this case local.

> If it needs to be large and shared between nodes then Lustre is 
> typically used for this.  If it is local and relatively small you 
> could consider using tmpfs backed by swab on an NVMe flash device 
> (M.2 or U.2, Optane if you can afford it) inside the node.
> 
> That way you get RAM-like performance for many files, with a larger 
> capacity than RAM when needed (tmpfs can use swap).
> 
> You might consider to mount a new tmpfs filesystem per job (no 
> formatting is needed for tmpfs), and then unmount it when the job is 
> done, so that the old files are automatically cleaned up.
That is a good idea, but probably not practical for 10 TB. Out of
curiosity, what is the limit for “relatively small” in your
experience?


Kind regards,

Paul


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5174 bytes --]