On Mon, Mar 09, 2015 at 09:52:18AM -0700, Linus Torvalds wrote:
> On Mon, Mar 9, 2015 at 4:29 AM, Dave Chinner <david@fromorbit.com> wrote:
> >
> >> Also, is there some sane way for me to actually see this behavior on a
> >> regular machine with just a single socket? Dave is apparently running
> >> in some fake-numa setup, I'm wondering if this is easy enough to
> >> reproduce that I could see it myself.
> >
> > Should be - I don't actually use 500TB of storage to generate this -
> > 50GB on an SSD is all you need from the storage side. I just use a
> > sparse backing file to make it look like a 500TB device. :P
> 
> What's your virtual environment setup? Kernel config, and
> virtualization environment to actually get that odd fake NUMA thing
> happening?

I don't have the exact .config with me (test machines at home
are shut down because I'm half a world away), but it's pretty much
this (copied and munged from a similar test vm on my laptop):

$ cat run-vm-4.sh
sudo qemu-system-x86_64 \
        -machine accel=kvm \
        -no-fd-bootchk \
        -localtime \
        -boot c \
        -serial pty \
        -nographic \
        -alt-grab \
        -smp 16 -m 16384 \
        -hda /data/vm-2/root.img \
        -drive file=/vm/vm-4/vm-4-test.img,if=virtio,cache=none \
        -drive file=/vm/vm-4/vm-4-scratch.img,if=virtio,cache=none \
        -drive file=/vm/vm-4/vm-4-500TB.img,if=virtio,cache=none \
        -kernel /vm/vm-4/vmlinuz \
        -append "console=ttyS0,115200 root=/dev/sda1,numa=fake=4"
$

And on the host I have /vm on a ssd that is an XFS filesystem, and
I've created /vm/vm-4/vm-4-500TB.img by doing:

$ xfs_io -f -c "truncate 500t" -c "extsize 1m" /vm/vm-4/vm-4-500TB.img

and in the guest the filesystem is created with:

# mkfs.xfs -f -mcrc=1,finobt=1 /dev/vdc

And that will create a 500TB filesystem that you can then mount and
run fsmark on it, then unmount and run xfs_repair on it.

the .config I have on my laptop is from 3.18-rc something, but it
should work just with a make oldconfig update. I'ts attached below.

Hopefully this will be sufficient for you, otherwise it'll have to
wait until I get home to get the exact configs for you.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com