inter-vm shared memory (ivshmem) documentation

inter-vm shared memory (ivshmem) documentation

[Valdir Stumm Junior]

Hi all,

I need a mechanism to share memory between my kvm guests to use at an
intrusion tolerance project. I found the Inter-VM Shared Memory
(ivshmem) device driver posted in this mailing list. I am trying to
install and use it, but I have some questions.

Is there a documentation (like a roadmap) about how to put ivshmem
device driver to work and how to test it?

I've compiled the patched kernel with ivshmem enabled as a module.
Now, I'm running kvm over this kernel. So, what is the next step?

I've tried this on my kvm guests:
num =`cat /proc/devices | grep kvm_ivshmem | awk '{print $1}`
mknod --mode=666 /dev/ivshmem $num 0
But, the /proc/devices file has no reference to kvm_ivshmem.


Thanks,

Valdir.

Re: inter-vm shared memory (ivshmem) documentation

[Cameron Macdonell]

On 22-Apr-09, at 5:23 PM, Valdir Stumm Junior wrote:

> Hi all,
>
> I need a mechanism to share memory between my kvm guests to use at an
> intrusion tolerance project. I found the Inter-VM Shared Memory
> (ivshmem) device driver posted in this mailing list. I am trying to
> install and use it, but I have some questions.
>
> Is there a documentation (like a roadmap) about how to put ivshmem
> device driver to work and how to test it?

Hi Valdir,

Documentation is under development.  I have some simple test programs  
that I can supply when you get it up and running.  As well as boot  
scripts to create the /dev file.

> I've compiled the patched kernel with ivshmem enabled as a module.
> Now, I'm running kvm over this kernel. So, what is the next step?
>
> I've tried this on my kvm guests:
> num =`cat /proc/devices | grep kvm_ivshmem | awk '{print $1}`
> mknod --mode=666 /dev/ivshmem $num 0
> But, the /proc/devices file has no reference to kvm_ivshmem.

Are you running the patched kernel on the host or in the guest.  It's  
meant to run in the guest.

Cam

-----------------------------------------------
A. Cameron Macdonell
Ph.D. Student
Department of Computing Science
University of Alberta
cam@cs.ualberta.ca

Re: inter-vm shared memory (ivshmem) documentation

[Valdir Stumm Junior]

On Thu, Apr 23, 2009 at 12:52 AM, Cameron Macdonell <cam@cs.ualberta.ca> wrote:
> Hi Valdir,
>
> Documentation is under development.  I have some simple test programs that I
> can supply when you get it up and running.  As well as boot scripts to
> create the /dev file.
> Are you running the patched kernel on the host or in the guest.  It's meant
> to run in the guest.

Cameron,

now I am running the patched kernel in the guest and it's working.
I modprobed "kvm_jvshmem" and then I made the following steps:

    num =`cat /proc/devices | grep kvm_ivshmem | awk '{print $1}`
    mknod --mode=666 /dev/ivshmem c $num 0

I've added to the mknod command the "c" option, to create a character
device (in the original post it was "mknod --mode=666 /dev/ivshmem
$num 0", without that option). The file creation was succesful. So,
what's next? Could you send your own test programs?

Nevertheless, I will create some simple tests, writing and reading
from the device.


Thanks in advance,

Valdir.



--
Valdir Stumm Junior
http://www.inf.ufsc.br/~stummjr

Re: inter-vm shared memory (ivshmem) documentation

[Cam Macdonell]

[-- Attachment #1: Type: text/plain, Size: 1758 bytes --]

Valdir Stumm Junior wrote:
> On Thu, Apr 23, 2009 at 12:52 AM, Cameron Macdonell <cam@cs.ualberta.ca> wrote:
>> Hi Valdir,
>>
>> Documentation is under development.  I have some simple test programs that I
>> can supply when you get it up and running.  As well as boot scripts to
>> create the /dev file.
>> Are you running the patched kernel on the host or in the guest.  It's meant
>> to run in the guest.
> 
> Cameron,
> 
> now I am running the patched kernel in the guest and it's working.
> I modprobed "kvm_jvshmem" and then I made the following steps:
> 
>     num =`cat /proc/devices | grep kvm_ivshmem | awk '{print $1}`
>     mknod --mode=666 /dev/ivshmem c $num 0
> 
> I've added to the mknod command the "c" option, to create a character
> device (in the original post it was "mknod --mode=666 /dev/ivshmem
> $num 0", without that option). The file creation was succesful. So,
> what's next? Could you send your own test programs?

Great.  I've attached some simple programs "dump" and "sum".  They are 
meant to run in the guest.  dump will fill the shared memory region with 
random numbers via the character device, so you run it

dump /dev/ivshmem 256 (if your device is 256 MB)

And on the other guest

sum /dev/ivshmem 256

will do a SHA1 sum on the region.  Dump also prints out the SHA1 sum, so 
make sure they match.

There is a third source file for the host - sum_host.c.

sum_host <shm object> 256

256 will do the sum on the host shm object file.  All the sums should 
match.  On the host, only specify the name of the shm object, not the 
full path since shm_open just takes the name "foo" and then creates 
/dev/shm/foo.

Eventually, I will move these tests to the wiki if/when the patch is 
accepted in some form.

Good luck,
Cam

[-- Attachment #2: dump.c --]
[-- Type: text/x-csrc, Size: 1341 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <string.h>
#include <openssl/sha.h>
#include <unistd.h>
#include <errno.h>

int main(int argc, char ** argv){

    int fd, length=4*1024;
    void * memptr;
    long * long_array;
    int i;
    SHA_CTX context;
    char md[20];

    if (argc != 3){
        printf("USAGE: dump <filename> <size>\n");
        exit(-1);
    }

    printf("[DUMP] opening file %s\n", argv[1]);
    length=atoi(argv[2])*1024*1024;
//    length=atoi(argv[2]);
    printf("[DUMP] size is %d\n", length);

    fd=open(argv[1], O_RDWR);

    if ((memptr = mmap(NULL, length, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0)) == (caddr_t)-1){
        printf("mmap failed (0x%x)\n", memptr);
        close (fd);
        exit (-1);
    }

    srand(time());
    long_array=(long *)memptr;
    for (i=0; i < length/sizeof(long); i++){
        long_array[i]=rand();
    }

    memset(md,0,20);

    SHA1_Init(&context);
    SHA1_Update(&context,memptr,length);
    SHA1_Final(md,&context);

    printf("[DUMP] ");
    for(i = 0; i < SHA_DIGEST_LENGTH; ++i )
    {
        unsigned char c = md[i];
        printf("%2.2x",c);
    }
    printf("\n");

    printf("munmap is unmapping %x\n", memptr);
    munmap(memptr, length);

    close(fd);

}

[-- Attachment #3: sum.c --]
[-- Type: text/x-csrc, Size: 1357 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <string.h>
#include <openssl/sha.h>
#include <unistd.h>
#include <sys/stat.h>
#include <errno.h>

int main(int argc, char ** argv){

    long size;
    char * file;
    void * map_region;
    long * long_array;
    int i,fd;
    SHA_CTX context;
    unsigned char md[20];
    struct test * myptr;

    if (argc!=3){
        fprintf(stderr, "USAGE: sum <file> <size in MB>\n");
        exit(-1);
    }

    size=atol(argv[2])*1024*1024;
    file=strdup(argv[1]);

    printf("[SUM] reading %d bytes from %s\n", size, file);

    if ((fd=open(file, O_RDWR)) < 0){
        fprintf(stderr, "ERROR: cannot open file\n");
        exit(-1);
    }

    if ((map_region=mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0))<0){
        fprintf(stderr, "ERROR: cannot mmap file\n");
    } else {
        printf("[SUM] mapped to %p\n", map_region);
    }

    memset(md,0,20);

    SHA1_Init(&context);
    SHA1_Update(&context,map_region,size);
    SHA1_Final(md,&context);

    printf("[SUM] ");
    for(i = 0; i < SHA_DIGEST_LENGTH; ++i )
    {
        unsigned char c = md[i];
        printf("%2.2x",c);
    }
    printf("\n");

//    printf("md is *%20s*\n", md);

    munmap(map_region,size);
    close(fd);

    printf("[SUM] Exiting...\n");
}

[-- Attachment #4: Makefile --]
[-- Type: text/plain, Size: 293 bytes --]

CC=gcc
CFLAGS= -g -lcrypto -lrt

all: sum dump writedump unlink_mem

sum:	sum.c
	$(CC) $^ -o $@ $(CFLAGS)

writedump:	writedump.c
	$(CC) $^ -o $@ $(CFLAGS)

dump:	dump.c
	$(CC) $^ -o $@ $(CFLAGS)

unlink_mem:	unlink_mem.c
	$(CC) $^ -o $@ $(CFLAGS)

clean:
	rm -f sum dump unlink_mem writedump

[-- Attachment #5: sum_host.c --]
[-- Type: text/x-csrc, Size: 1574 bytes --]

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <string.h>
#include <openssl/sha.h>
#include <unistd.h>
#include <sys/stat.h>
#include <errno.h>

int main(int argc, char ** argv){

    long size;
    char file[1024];
    void * map_region;
    long * long_array;
    int i,fd;
    SHA_CTX context;
    unsigned char md[20];
    struct test * myptr;

    if (argc!=3){
        fprintf(stderr, "USAGE: sum <shm object> <size in MB>\n");
        fprintf(stderr, "this is meant to run on the host\n");
        exit(-1);
    }

    size=atol(argv[2])*1024*1024;
    memset(file,0,1024);
    snprintf(file, 1024, "/%s",argv[1]);

    printf("[SUM] reading %ld bytes from %s\n", size, file);

    if ((fd=shm_open(file, O_RDWR|O_CREAT, S_IREAD | S_IWRITE)) > 0){
        printf("[SUM] second\n");
    } else {
        fprintf(stderr, "ERROR: cannot open file\n");
        exit(-1);
    }

    ftruncate(fd,size);
    if ((map_region=mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0))<0){
        fprintf(stderr, "ERROR: cannot mmap file\n");
    } else {
        printf("[SUM] mapped to %p\n", map_region);
    }

    memset(md,0,20);

    SHA1_Init(&context);
    SHA1_Update(&context,map_region,size);
    SHA1_Final(md,&context);

    printf("[SUM] ");
    for(i = 0; i < SHA_DIGEST_LENGTH; ++i )
    {
        unsigned char c = md[i];
        printf("%2.2x",c);
    }
    printf("\n");

//    printf("md is *%20s*\n", md);

    munmap(map_region,size);
    close(fd);

    printf("[SUM] Exiting...\n");
}