All of lore.kernel.org
 help / color / mirror / Atom feed
* A question about selinux userspace tools
@ 2009-08-04  3:11 TaurusHarry
       [not found] ` <A0FAB46BC43C4D8ABE2581D6EF3E2713@CIMSLAB.NL>
                   ` (2 more replies)
  0 siblings, 3 replies; 20+ messages in thread
From: TaurusHarry @ 2009-08-04  3:11 UTC (permalink / raw)
  To: selinux-mailing-list

[-- Attachment #1: Type: text/plain, Size: 1722 bytes --]


Hi all,

I am a newcomer to SELinux and have a question about the latest SELinux userspace tools.

Before I try to install the latest refpolicy-20090730 from tresys website on Ubuntu 8.10 on Dell 6101 laptop(x86_32), I have installed the below latest SELinux userspace tools on the laptop:

libsepol-2.0.37
libselinux-2.0.85
libsemanage-2.0.33
policycoreutils-2.0.69
checkpolicy-2.0.19
sepolgen-1.0.17

If I specify MONOLITHIC=n in build.conf and compile refpolicy-20090730 by the latest SELinux userspace tools and then do "make load", I would run into following error message in the end:

libsepol.sepol_module_package_read: out of memory
libsemanage.semanage_load_module: Error while reading from module file /etc/selinux/refpolicy-20090730/modules/tmp/base.pp.
/usr/sbin/semodule:  Failed!
make: *** [load] Error 1

The same error would crop up when I try out following second-to-the-latest SELinux userspace tools(also from tresys website):

libsepol-2.0.36
libselinux-2.0.79
libsemanage-2.0.31
policycoreutils-2.0.62
checkpolicy-2.0.19
sepolgen-1.0.16

Only if I degrade libsemanage from 2.0.31 to 2.0.27, and policycoreutils from 2.0.62 to 2.0.55, would the "make load" finish uneventfully, then I could do "semanage login/user -l" and "semodule -l" successfully.

So far I have very shallow experience on SELinux userspace tools implementation and couldn't find out the reason  why the latest stable version would have above error message on my laptop, has anyone run into the same problem before? Any suggestions are greatly appreciated!

Thanks!

Harry
_________________________________________________________________
张三挖到了元宝,小美又掉进陷阱了,快来MClub与好友齐乐乐!立刻访问!
http://club.msn.cn/?from=3

[-- Attachment #2: Type: text/html, Size: 1977 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: A question about selinux userspace tools
       [not found] ` <A0FAB46BC43C4D8ABE2581D6EF3E2713@CIMSLAB.NL>
@ 2009-08-04  8:57   ` TaurusHarry
  0 siblings, 0 replies; 20+ messages in thread
From: TaurusHarry @ 2009-08-04  8:57 UTC (permalink / raw)
  To: remmolt; +Cc: selinux-mailing-list

[-- Attachment #1: Type: text/plain, Size: 3933 bytes --]


Hi Remmolt,

Many thanks for your reply!

I guess you may think I am running Ubuntu 8.10 by vmware on top of Windows OS, but this is not the case, I have booted into Ubuntu directly without any virtualization tool, there is no PAGEFILE.SYS nor INDEXF.SYS on linux, they seem to be very Windows-like.

Yeah, I know I could install a bootable selinux kit simply by "sudo apt-get install selinux" and everything will be installed and set up correctly, however, I would like to follow all the necessary steps by hand to get a better understanding of how to deploy selinux on Ubuntu, so I started from scratch by installing the latest selinux userspace tools and loading refpolicy package.

I just wondering why the latest stable libsemanage may cause this "libsepol.sepol_module_package_read: out of memory" problem but the version of 2.0.27 won't.

Best regards,

Harry


From: remmolt@zwartsenberg.eu
To: harrytaurus2002@hotmail.com
CC: selinux@tycho.nsa.gov
Subject: RE: A question about selinux userspace tools
Date: Tue, 4 Aug 2009 10:38:46 +0200



















Hello Harry,

 

You are getting a traceback error because
your build is trying to reference RAM memory on your laptop that either does
not exist or is addressed thru a virtualization tool ( like say Vmware,
Softtricity or CITRIX)

Further, your PAGEFILE.SYS may be too
small. Mind you the referencing of the INDEXF.SYS (HD memory) from ‘userspace’ is
handled thru a windows i/o handler.

 

What you could try (you seem to be young
and have plenty of time) is create a bootable Selinux kit and before you reboot
your laptop change the boot sequence in the BIOS (Basic Input Output System) to
the device on your laptop you stored the kit on.

We did this successfully on Knoppix (cr
Klaus Knoppers). 

 

Viel spass,

 

~remmolt     

 

www.bundesnachrichtendienst.de


 









From:
owner-selinux@tycho.nsa.gov [mailto:owner-selinux@tycho.nsa.gov] On Behalf Of TaurusHarry

Sent: dinsdag 4 augustus 2009 5:11

To: selinux-mailing-list

Subject: A question about selinux
userspace tools



 

Hi all,



I am a newcomer to SELinux and have a question about the latest SELinux
userspace tools.



Before I try to install the latest refpolicy-20090730 from tresys website on
Ubuntu 8.10 on Dell 6101 laptop(x86_32), I have installed the below latest
SELinux userspace tools on the laptop:



libsepol-2.0.37

libselinux-2.0.85

libsemanage-2.0.33

policycoreutils-2.0.69

checkpolicy-2.0.19

sepolgen-1.0.17



If I specify MONOLITHIC=n in build.conf and compile refpolicy-20090730 by the
latest SELinux userspace tools and then do "make load", I would run
into following error message in the end:



libsepol.sepol_module_package_read: out of memory

libsemanage.semanage_load_module: Error while reading from module file
/etc/selinux/refpolicy-20090730/modules/tmp/base.pp.

/usr/sbin/semodule:  Failed!

make: *** [load] Error 1



The same error would crop up when I try out following second-to-the-latest
SELinux userspace tools(also ! from tresys website):



libsepol-2.0.36

libselinux-2.0.79

libsemanage-2.0.31

policycoreutils-2.0.62

checkpolicy-2.0.19

sepolgen-1.0.16



Only if I degrade libsemanage from 2.0.31 to 2.0.27, and policycoreutils from
2.0.62 to 2.0.55, would the "make load" finish uneventfully, then I
could do "semanage login/user -l" and "semodule -l"
successfully.



So far I have very shallow experience on SELinux userspace tools implementation
and couldn't find out the reason  why the latest stable version would have
above error message on my laptop, has anyone run into the same problem before?
Any suggestions are greatly appreciated!



Thanks!



Harry







聊天+搜索+邮箱 想要轻松出游,手机MSN帮你搞定! 立刻下载!


_________________________________________________________________
您可以借助 Windows Live 整理、编辑和共享您的照片。
http://www.microsoft.com/china/windows/windowslive/products/photo-gallery-edit.aspx

[-- Attachment #2: Type: text/html, Size: 9556 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: A question about selinux userspace tools
  2009-08-04  3:11 A question about selinux userspace tools TaurusHarry
       [not found] ` <A0FAB46BC43C4D8ABE2581D6EF3E2713@CIMSLAB.NL>
@ 2009-08-04 12:13 ` Stephen Smalley
  2009-08-04 12:40   ` Stephen Smalley
       [not found] ` <1249397076.9193.101.camel@moss-pluto.epoch.ncsc.mil>
  2 siblings, 1 reply; 20+ messages in thread
From: Stephen Smalley @ 2009-08-04 12:13 UTC (permalink / raw)
  To: TaurusHarry; +Cc: selinux-mailing-list, Daniel J Walsh, Joshua Brindle

On Tue, 2009-08-04 at 03:11 +0000, TaurusHarry wrote:
> Hi all,
> 
> I am a newcomer to SELinux and have a question about the latest
> SELinux userspace tools.
> 
> Before I try to install the latest refpolicy-20090730 from tresys
> website on Ubuntu 8.10 on Dell 6101 laptop(x86_32), I have installed
> the below latest SELinux userspace tools on the laptop:
> 
> libsepol-2.0.37
> libselinux-2.0.85
> libsemanage-2.0.33
> policycoreutils-2.0.69
> checkpolicy-2.0.19
> sepolgen-1.0.17
> 
> If I specify MONOLITHIC=n in build.conf and compile refpolicy-20090730
> by the latest SELinux userspace tools and then do "make load", I would
> run into following error message in the end:
> 
> libsepol.sepol_module_package_read: out of memory
> libsemanage.semanage_load_module: Error while reading from module
> file /etc/selinux/refpolicy-20090730/modules/tmp/base.pp.
> /usr/sbin/semodule:  Failed!
> make: *** [load] Error 1
> 
> The same error would crop up when I try out following
> second-to-the-latest SELinux userspace tools(also ! from tresys
> website):
> 
> libsepol-2.0.36
> libselinux-2.0.79
> libsemanage-2.0.31
> policycoreutils-2.0.62
> checkpolicy-2.0.19
> sepolgen-1.0.16
> 
> Only if I degrade libsemanage from 2.0.31 to 2.0.27, and
> policycoreutils from 2.0.62 to 2.0.55, would the "make load" finish
> uneventfully, then I could do "semanage login/user -l" and "semodule
> -l" successfully.
> 
> So far I have very shallow experience on SELinux userspace tools
> implementation and couldn't find out the reason  why the latest stable
> version would have above error message on my laptop, has anyone run
> into the same problem before? Any suggestions are greatly appreciated!

Things that would help debug:
1) A copy of the base.pp file on which it failed,
2) The amount of memory and swap configured on your laptop,

Possibly this could be due to the introduction of policy module
compression (bzip) support in libsemanage 2.0.31.

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: A question about selinux userspace tools
  2009-08-04 12:13 ` Stephen Smalley
@ 2009-08-04 12:40   ` Stephen Smalley
  2009-08-04 13:25     ` Stephen Smalley
  0 siblings, 1 reply; 20+ messages in thread
From: Stephen Smalley @ 2009-08-04 12:40 UTC (permalink / raw)
  To: TaurusHarry; +Cc: selinux-mailing-list, Daniel J Walsh, Joshua Brindle

On Tue, 2009-08-04 at 08:13 -0400, Stephen Smalley wrote:
> On Tue, 2009-08-04 at 03:11 +0000, TaurusHarry wrote:
> > Hi all,
> > 
> > I am a newcomer to SELinux and have a question about the latest
> > SELinux userspace tools.
> > 
> > Before I try to install the latest refpolicy-20090730 from tresys
> > website on Ubuntu 8.10 on Dell 6101 laptop(x86_32), I have installed
> > the below latest SELinux userspace tools on the laptop:
> > 
> > libsepol-2.0.37
> > libselinux-2.0.85
> > libsemanage-2.0.33
> > policycoreutils-2.0.69
> > checkpolicy-2.0.19
> > sepolgen-1.0.17
> > 
> > If I specify MONOLITHIC=n in build.conf and compile refpolicy-20090730
> > by the latest SELinux userspace tools and then do "make load", I would
> > run into following error message in the end:
> > 
> > libsepol.sepol_module_package_read: out of memory
> > libsemanage.semanage_load_module: Error while reading from module
> > file /etc/selinux/refpolicy-20090730/modules/tmp/base.pp.
> > /usr/sbin/semodule:  Failed!
> > make: *** [load] Error 1
> > 
> > The same error would crop up when I try out following
> > second-to-the-latest SELinux userspace tools(also ! from tresys
> > website):
> > 
> > libsepol-2.0.36
> > libselinux-2.0.79
> > libsemanage-2.0.31
> > policycoreutils-2.0.62
> > checkpolicy-2.0.19
> > sepolgen-1.0.16
> > 
> > Only if I degrade libsemanage from 2.0.31 to 2.0.27, and
> > policycoreutils from 2.0.62 to 2.0.55, would the "make load" finish
> > uneventfully, then I could do "semanage login/user -l" and "semodule
> > -l" successfully.
> > 
> > So far I have very shallow experience on SELinux userspace tools
> > implementation and couldn't find out the reason  why the latest stable
> > version would have above error message on my laptop, has anyone run
> > into the same problem before? Any suggestions are greatly appreciated!
> 
> Things that would help debug:
> 1) A copy of the base.pp file on which it failed,
> 2) The amount of memory and swap configured on your laptop,
> 
> Possibly this could be due to the introduction of policy module
> compression (bzip) support in libsemanage 2.0.31.

Some quick observations on the code:
- direct_api.c:bzip() always passes 9 as the blocksize to
BZ2_bzWriteOpen().  I'd suggest making that tunable via semanage.conf or
something so that we can balance memory use against compression level.

- direct_api.c:bunzip() always passes 0 as the small flag to
BZ2_bzReadOpen().  I'd likewise make that tunable via semanage.conf so
that we can balance memory use against runtime performance.

- What is the rationale for the buffer size (1<<18) in bunzip()?

- Doubling the buffer size within the loop in bunzip() may be too
greedy; possibly it should just increment by a fixed chunk size.

- We're effectively allocating the memory twice - inside bunzip() we
allocate a buffer and fill it, and then in map_file() we mmap memory and
copy into it.  I assume that was to avoid changing the exit paths of the
callers so that they can always just munmap it, but we might want to
reconsider it as it temporarily puts us in a position of holding twice
the required memory.

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: A question about selinux userspace tools
  2009-08-04 12:40   ` Stephen Smalley
@ 2009-08-04 13:25     ` Stephen Smalley
  2009-08-04 14:08       ` [PATCH v2] libsemanage: Enable configuration of bzip behavior Stephen Smalley
  0 siblings, 1 reply; 20+ messages in thread
From: Stephen Smalley @ 2009-08-04 13:25 UTC (permalink / raw)
  To: TaurusHarry; +Cc: selinux-mailing-list, Daniel J Walsh, Joshua Brindle

On Tue, 2009-08-04 at 08:40 -0400, Stephen Smalley wrote:
> On Tue, 2009-08-04 at 08:13 -0400, Stephen Smalley wrote:
> > On Tue, 2009-08-04 at 03:11 +0000, TaurusHarry wrote:
> > > Hi all,
> > > 
> > > I am a newcomer to SELinux and have a question about the latest
> > > SELinux userspace tools.
> > > 
> > > Before I try to install the latest refpolicy-20090730 from tresys
> > > website on Ubuntu 8.10 on Dell 6101 laptop(x86_32), I have installed
> > > the below latest SELinux userspace tools on the laptop:
> > > 
> > > libsepol-2.0.37
> > > libselinux-2.0.85
> > > libsemanage-2.0.33
> > > policycoreutils-2.0.69
> > > checkpolicy-2.0.19
> > > sepolgen-1.0.17
> > > 
> > > If I specify MONOLITHIC=n in build.conf and compile refpolicy-20090730
> > > by the latest SELinux userspace tools and then do "make load", I would
> > > run into following error message in the end:
> > > 
> > > libsepol.sepol_module_package_read: out of memory
> > > libsemanage.semanage_load_module: Error while reading from module
> > > file /etc/selinux/refpolicy-20090730/modules/tmp/base.pp.
> > > /usr/sbin/semodule:  Failed!
> > > make: *** [load] Error 1
> > > 
> > > The same error would crop up when I try out following
> > > second-to-the-latest SELinux userspace tools(also ! from tresys
> > > website):
> > > 
> > > libsepol-2.0.36
> > > libselinux-2.0.79
> > > libsemanage-2.0.31
> > > policycoreutils-2.0.62
> > > checkpolicy-2.0.19
> > > sepolgen-1.0.16
> > > 
> > > Only if I degrade libsemanage from 2.0.31 to 2.0.27, and
> > > policycoreutils from 2.0.62 to 2.0.55, would the "make load" finish
> > > uneventfully, then I could do "semanage login/user -l" and "semodule
> > > -l" successfully.
> > > 
> > > So far I have very shallow experience on SELinux userspace tools
> > > implementation and couldn't find out the reason  why the latest stable
> > > version would have above error message on my laptop, has anyone run
> > > into the same problem before? Any suggestions are greatly appreciated!
> > 
> > Things that would help debug:
> > 1) A copy of the base.pp file on which it failed,
> > 2) The amount of memory and swap configured on your laptop,
> > 
> > Possibly this could be due to the introduction of policy module
> > compression (bzip) support in libsemanage 2.0.31.
> 
> Some quick observations on the code:
> - direct_api.c:bzip() always passes 9 as the blocksize to
> BZ2_bzWriteOpen().  I'd suggest making that tunable via semanage.conf or
> something so that we can balance memory use against compression level.
> 
> - direct_api.c:bunzip() always passes 0 as the small flag to
> BZ2_bzReadOpen().  I'd likewise make that tunable via semanage.conf so
> that we can balance memory use against runtime performance.

Allow the administrator to customize the bzip block size and "small"
flag via semanage.conf.  After applying you can add entries like these
to your /etc/selinux/semanage.conf to trade off memory vs disk space
(block size) and to trade off memory vs runtime (small):
bzip-blocksize=4
bzip-small=1

Signed-off-by:  Stephen Smalley <sds@tycho.nsa.gov>

diff --git a/libsemanage/src/conf-parse.y b/libsemanage/src/conf-parse.y
index 2001afb..5b2c859 100644
--- a/libsemanage/src/conf-parse.y
+++ b/libsemanage/src/conf-parse.y
@@ -58,6 +58,7 @@ static int parse_errors;
 
 %token MODULE_STORE VERSION EXPAND_CHECK FILE_MODE SAVE_PREVIOUS SAVE_LINKED
 %token LOAD_POLICY_START SETFILES_START DISABLE_GENHOMEDIRCON HANDLE_UNKNOWN
+%token BZIP_BLOCKSIZE BZIP_SMALL
 %token VERIFY_MOD_START VERIFY_LINKED_START VERIFY_KERNEL_START BLOCK_END
 %token PROG_PATH PROG_ARGS
 %token <s> ARG
@@ -82,6 +83,8 @@ single_opt:     module_store
         |       save_linked
         |       disable_genhomedircon
         |       handle_unknown
+	|	bzip_blocksize
+	|	bzip_small
         ;
 
 module_store:   MODULE_STORE '=' ARG {
@@ -163,6 +166,16 @@ handle_unknown: HANDLE_UNKNOWN '=' ARG {
 	free($3);
  }
 
+bzip_blocksize:  BZIP_BLOCKSIZE '=' ARG {
+	current_conf->bzip_blocksize = atoi($3);
+	free($3);
+}
+	
+bzip_small:  BZIP_SMALL '=' ARG {
+	current_conf->bzip_small = atoi($3);
+	free($3);
+}
+
 command_block: 
                 command_start external_opts BLOCK_END  {
                         if (new_external->path == NULL) {
@@ -230,6 +243,8 @@ static int semanage_conf_init(semanage_conf_t * conf)
 	conf->expand_check = 1;
 	conf->handle_unknown = -1;
 	conf->file_mode = 0644;
+	conf->bzip_blocksize = 9;
+	conf->bzip_small = 0;
 
 	conf->save_previous = 0;
 	conf->save_linked = 0;
diff --git a/libsemanage/src/conf-scan.l b/libsemanage/src/conf-scan.l
index faa0aeb..8af5137 100644
--- a/libsemanage/src/conf-scan.l
+++ b/libsemanage/src/conf-scan.l
@@ -47,6 +47,8 @@ save-previous     return SAVE_PREVIOUS;
 save-linked       return SAVE_LINKED;
 disable-genhomedircon return DISABLE_GENHOMEDIRCON;
 handle-unknown    return HANDLE_UNKNOWN;
+bzip-blocksize	return BZIP_BLOCKSIZE;
+bzip-small	return BZIP_SMALL;
 "[load_policy]"   return LOAD_POLICY_START;
 "[setfiles]"      return SETFILES_START;
 "[verify module]" return VERIFY_MOD_START;
diff --git a/libsemanage/src/direct_api.c b/libsemanage/src/direct_api.c
index bd13387..780ba14 100644
--- a/libsemanage/src/direct_api.c
+++ b/libsemanage/src/direct_api.c
@@ -401,7 +401,9 @@ static int parse_base_headers(semanage_handle_t * sh,
 
 /* bzip() a data to a file, returning the total number of compressed bytes
  * in the file.  Returns -1 if file could not be compressed. */
-static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
+static ssize_t bzip(semanage_handle_t *sh, const char *filename, char *data,
+			size_t num_bytes) 
+{
 	BZFILE* b;
 	size_t  size = 1<<16;
 	int     bzerror;
@@ -413,7 +415,7 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
 		return -1;
 	}
 
-	b = BZ2_bzWriteOpen( &bzerror, f, 9, 0, 0);
+	b = BZ2_bzWriteOpen( &bzerror, f, sh->conf->bzip_blocksize, 0, 0);
 	if (bzerror != BZ_OK) {
 		BZ2_bzWriteClose ( &bzerror, b, 1, 0, 0 );
 		return -1;
@@ -443,7 +445,8 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
 
 /* bunzip() a file to '*data', returning the total number of uncompressed bytes
  * in the file.  Returns -1 if file could not be decompressed. */
-ssize_t bunzip(FILE *f, char **data) {
+ssize_t bunzip(semanage_handle_t *sh, FILE *f, char **data)
+{
 	BZFILE* b;
 	size_t  nBuf;
 	char    buf[1<<18];
@@ -451,7 +454,7 @@ ssize_t bunzip(FILE *f, char **data) {
 	int     bzerror;
 	size_t  total=0;
 	
-	b = BZ2_bzReadOpen ( &bzerror, f, 0, 0, NULL, 0 );
+	b = BZ2_bzReadOpen ( &bzerror, f, 0, sh->conf->bzip_small, NULL, 0 );
 	if ( bzerror != BZ_OK ) {
 		BZ2_bzReadClose ( &bzerror, b );
 		return -1;
@@ -486,11 +489,12 @@ ssize_t bunzip(FILE *f, char **data) {
  * the file into '*data'.
  * Returns the total number of bytes in memory .
  * Returns -1 if file could not be opened or mapped. */
-static ssize_t map_file(int fd, char **data, int *compressed)
+static ssize_t map_file(semanage_handle_t *sh, int fd, char **data,
+			int *compressed)
 {
 	ssize_t size = -1;
 	char *uncompress;
-	if ((size = bunzip(fdopen(fd, "r"), &uncompress)) > 0) {
+	if ((size = bunzip(sh, fdopen(fd, "r"), &uncompress)) > 0) {
 		*data = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
 		if (*data == MAP_FAILED) {
 			free(uncompress);
@@ -997,7 +1001,7 @@ static int semanage_direct_install(semanage_handle_t * sh,
 					   &filename)) != 0) {
 		goto cleanup;
 	}
-	if (bzip(filename, data, data_len) <= 0) {
+	if (bzip(sh, filename, data, data_len) <= 0) {
 		ERR(sh, "Error while writing to %s.", filename);
 		retval = -3;
 		goto cleanup;
@@ -1029,7 +1033,7 @@ static int semanage_direct_install_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 		
@@ -1127,7 +1131,7 @@ static int semanage_direct_upgrade(semanage_handle_t * sh,
 						 data, data_len, 
 						 &filename);
 	if (retval == 0) {
-		if (bzip(filename, data, data_len) <= 0) {
+		if (bzip(sh, filename, data, data_len) <= 0) {
 			ERR(sh, "Error while writing to %s.", filename);
 			retval = -3;
 		}
@@ -1155,7 +1159,7 @@ static int semanage_direct_upgrade_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 
@@ -1197,7 +1201,7 @@ static int semanage_direct_install_base(semanage_handle_t * sh,
 	if ((filename = semanage_path(SEMANAGE_TMP, SEMANAGE_BASE)) == NULL) {
 		goto cleanup;
 	}
-	if (bzip(filename, base_data, data_len) <= 0) {
+	if (bzip(sh, filename, base_data, data_len) <= 0) {
 		ERR(sh, "Error while writing to %s.", filename);
 		retval = -3;
 		goto cleanup;
@@ -1225,7 +1229,7 @@ static int semanage_direct_install_base_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 		
@@ -1347,7 +1351,7 @@ static int semanage_direct_list(semanage_handle_t * sh,
 		ssize_t size;
 		char *data = NULL;
 
-		if ((size = bunzip(fp, &data)) > 0) {
+		if ((size = bunzip(sh, fp, &data)) > 0) {
 			fclose(fp);
 			fp = fmemopen(data, size, "rb");
 			if (!fp) {
diff --git a/libsemanage/src/direct_api.h b/libsemanage/src/direct_api.h
index 1ad7d1d..e56107b 100644
--- a/libsemanage/src/direct_api.h
+++ b/libsemanage/src/direct_api.h
@@ -41,6 +41,6 @@ int semanage_direct_mls_enabled(struct semanage_handle *sh);
 
 #include <stdio.h>
 #include <unistd.h>
-ssize_t bunzip(FILE *f, char **data);
+ssize_t bunzip(struct semanage_handle *sh, FILE *f, char **data);
 
 #endif
diff --git a/libsemanage/src/semanage_conf.h b/libsemanage/src/semanage_conf.h
index 7ee139f..4118910 100644
--- a/libsemanage/src/semanage_conf.h
+++ b/libsemanage/src/semanage_conf.h
@@ -40,6 +40,8 @@ typedef struct semanage_conf {
 	int disable_genhomedircon;
 	int handle_unknown;
 	mode_t file_mode;
+	int bzip_blocksize;
+	int bzip_small;
 	struct external_prog *load_policy;
 	struct external_prog *setfiles;
 	struct external_prog *mod_prog, *linked_prog, *kernel_prog;
diff --git a/libsemanage/src/semanage_store.c b/libsemanage/src/semanage_store.c
index 6d4c3ce..a3b0819 100644
--- a/libsemanage/src/semanage_store.c
+++ b/libsemanage/src/semanage_store.c
@@ -1529,7 +1529,7 @@ static int semanage_load_module(semanage_handle_t * sh, const char *filename,
 	ssize_t size;
 	char *data = NULL;
 
-	if ((size = bunzip(fp, &data)) > 0) {
+	if ((size = bunzip(sh, fp, &data)) > 0) {
 		fclose(fp);
 		fp = fmemopen(data, size, "rb");
 		if (!fp) {

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v2] libsemanage:  Enable configuration of bzip behavior
  2009-08-04 13:25     ` Stephen Smalley
@ 2009-08-04 14:08       ` Stephen Smalley
  2009-08-04 14:11         ` Joshua Brindle
  2009-08-04 14:38         ` [PATCH v3] " Stephen Smalley
  0 siblings, 2 replies; 20+ messages in thread
From: Stephen Smalley @ 2009-08-04 14:08 UTC (permalink / raw)
  To: TaurusHarry; +Cc: selinux-mailing-list, Daniel J Walsh, Joshua Brindle

Allow the administrator to customize the bzip block size and "small"
flag via semanage.conf.  After applying you can add entries like these
to your /etc/selinux/semanage.conf to trade off memory vs disk space
(block size) and to trade off memory vs runtime (small):

bzip-blocksize=4
bzip-small=1

You can also disable bzip compression altogether for your module store
via:
bzip-blocksize=0

Signed-off-by:  Stephen Smalley <sds@tycho.nsa.gov>

diff --git a/libsemanage/src/conf-parse.y b/libsemanage/src/conf-parse.y
index 2001afb..5b2c859 100644
--- a/libsemanage/src/conf-parse.y
+++ b/libsemanage/src/conf-parse.y
@@ -58,6 +58,7 @@ static int parse_errors;
 
 %token MODULE_STORE VERSION EXPAND_CHECK FILE_MODE SAVE_PREVIOUS SAVE_LINKED
 %token LOAD_POLICY_START SETFILES_START DISABLE_GENHOMEDIRCON HANDLE_UNKNOWN
+%token BZIP_BLOCKSIZE BZIP_SMALL
 %token VERIFY_MOD_START VERIFY_LINKED_START VERIFY_KERNEL_START BLOCK_END
 %token PROG_PATH PROG_ARGS
 %token <s> ARG
@@ -82,6 +83,8 @@ single_opt:     module_store
         |       save_linked
         |       disable_genhomedircon
         |       handle_unknown
+	|	bzip_blocksize
+	|	bzip_small
         ;
 
 module_store:   MODULE_STORE '=' ARG {
@@ -163,6 +166,16 @@ handle_unknown: HANDLE_UNKNOWN '=' ARG {
 	free($3);
  }
 
+bzip_blocksize:  BZIP_BLOCKSIZE '=' ARG {
+	current_conf->bzip_blocksize = atoi($3);
+	free($3);
+}
+	
+bzip_small:  BZIP_SMALL '=' ARG {
+	current_conf->bzip_small = atoi($3);
+	free($3);
+}
+
 command_block: 
                 command_start external_opts BLOCK_END  {
                         if (new_external->path == NULL) {
@@ -230,6 +243,8 @@ static int semanage_conf_init(semanage_conf_t * conf)
 	conf->expand_check = 1;
 	conf->handle_unknown = -1;
 	conf->file_mode = 0644;
+	conf->bzip_blocksize = 9;
+	conf->bzip_small = 0;
 
 	conf->save_previous = 0;
 	conf->save_linked = 0;
diff --git a/libsemanage/src/conf-scan.l b/libsemanage/src/conf-scan.l
index faa0aeb..8af5137 100644
--- a/libsemanage/src/conf-scan.l
+++ b/libsemanage/src/conf-scan.l
@@ -47,6 +47,8 @@ save-previous     return SAVE_PREVIOUS;
 save-linked       return SAVE_LINKED;
 disable-genhomedircon return DISABLE_GENHOMEDIRCON;
 handle-unknown    return HANDLE_UNKNOWN;
+bzip-blocksize	return BZIP_BLOCKSIZE;
+bzip-small	return BZIP_SMALL;
 "[load_policy]"   return LOAD_POLICY_START;
 "[setfiles]"      return SETFILES_START;
 "[verify module]" return VERIFY_MOD_START;
diff --git a/libsemanage/src/direct_api.c b/libsemanage/src/direct_api.c
index bd13387..92799ad 100644
--- a/libsemanage/src/direct_api.c
+++ b/libsemanage/src/direct_api.c
@@ -401,7 +401,9 @@ static int parse_base_headers(semanage_handle_t * sh,
 
 /* bzip() a data to a file, returning the total number of compressed bytes
  * in the file.  Returns -1 if file could not be compressed. */
-static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
+static ssize_t bzip(semanage_handle_t *sh, const char *filename, char *data,
+			size_t num_bytes) 
+{
 	BZFILE* b;
 	size_t  size = 1<<16;
 	int     bzerror;
@@ -413,7 +415,16 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
 		return -1;
 	}
 
-	b = BZ2_bzWriteOpen( &bzerror, f, 9, 0, 0);
+	if (!sh->conf->bzip_blocksize) {
+		if (fwrite(data, 1, num_bytes, f) < num_bytes) {
+			fclose(f);
+			return -1;
+		}
+		fclose(f);
+		return num_bytes;
+	}
+
+	b = BZ2_bzWriteOpen( &bzerror, f, sh->conf->bzip_blocksize, 0, 0);
 	if (bzerror != BZ_OK) {
 		BZ2_bzWriteClose ( &bzerror, b, 1, 0, 0 );
 		return -1;
@@ -443,15 +454,19 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
 
 /* bunzip() a file to '*data', returning the total number of uncompressed bytes
  * in the file.  Returns -1 if file could not be decompressed. */
-ssize_t bunzip(FILE *f, char **data) {
+ssize_t bunzip(semanage_handle_t *sh, FILE *f, char **data)
+{
 	BZFILE* b;
 	size_t  nBuf;
 	char    buf[1<<18];
 	size_t  size = sizeof(buf);
 	int     bzerror;
 	size_t  total=0;
+
+	if (!sh->conf->bzip_blocksize)
+		return -1;
 	
-	b = BZ2_bzReadOpen ( &bzerror, f, 0, 0, NULL, 0 );
+	b = BZ2_bzReadOpen ( &bzerror, f, 0, sh->conf->bzip_small, NULL, 0 );
 	if ( bzerror != BZ_OK ) {
 		BZ2_bzReadClose ( &bzerror, b );
 		return -1;
@@ -486,11 +501,12 @@ ssize_t bunzip(FILE *f, char **data) {
  * the file into '*data'.
  * Returns the total number of bytes in memory .
  * Returns -1 if file could not be opened or mapped. */
-static ssize_t map_file(int fd, char **data, int *compressed)
+static ssize_t map_file(semanage_handle_t *sh, int fd, char **data,
+			int *compressed)
 {
 	ssize_t size = -1;
 	char *uncompress;
-	if ((size = bunzip(fdopen(fd, "r"), &uncompress)) > 0) {
+	if ((size = bunzip(sh, fdopen(fd, "r"), &uncompress)) > 0) {
 		*data = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
 		if (*data == MAP_FAILED) {
 			free(uncompress);
@@ -997,7 +1013,7 @@ static int semanage_direct_install(semanage_handle_t * sh,
 					   &filename)) != 0) {
 		goto cleanup;
 	}
-	if (bzip(filename, data, data_len) <= 0) {
+	if (bzip(sh, filename, data, data_len) <= 0) {
 		ERR(sh, "Error while writing to %s.", filename);
 		retval = -3;
 		goto cleanup;
@@ -1029,7 +1045,7 @@ static int semanage_direct_install_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 		
@@ -1127,7 +1143,7 @@ static int semanage_direct_upgrade(semanage_handle_t * sh,
 						 data, data_len, 
 						 &filename);
 	if (retval == 0) {
-		if (bzip(filename, data, data_len) <= 0) {
+		if (bzip(sh, filename, data, data_len) <= 0) {
 			ERR(sh, "Error while writing to %s.", filename);
 			retval = -3;
 		}
@@ -1155,7 +1171,7 @@ static int semanage_direct_upgrade_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 
@@ -1197,7 +1213,7 @@ static int semanage_direct_install_base(semanage_handle_t * sh,
 	if ((filename = semanage_path(SEMANAGE_TMP, SEMANAGE_BASE)) == NULL) {
 		goto cleanup;
 	}
-	if (bzip(filename, base_data, data_len) <= 0) {
+	if (bzip(sh, filename, base_data, data_len) <= 0) {
 		ERR(sh, "Error while writing to %s.", filename);
 		retval = -3;
 		goto cleanup;
@@ -1225,7 +1241,7 @@ static int semanage_direct_install_base_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 		
@@ -1347,7 +1363,7 @@ static int semanage_direct_list(semanage_handle_t * sh,
 		ssize_t size;
 		char *data = NULL;
 
-		if ((size = bunzip(fp, &data)) > 0) {
+		if ((size = bunzip(sh, fp, &data)) > 0) {
 			fclose(fp);
 			fp = fmemopen(data, size, "rb");
 			if (!fp) {
diff --git a/libsemanage/src/direct_api.h b/libsemanage/src/direct_api.h
index 1ad7d1d..e56107b 100644
--- a/libsemanage/src/direct_api.h
+++ b/libsemanage/src/direct_api.h
@@ -41,6 +41,6 @@ int semanage_direct_mls_enabled(struct semanage_handle *sh);
 
 #include <stdio.h>
 #include <unistd.h>
-ssize_t bunzip(FILE *f, char **data);
+ssize_t bunzip(struct semanage_handle *sh, FILE *f, char **data);
 
 #endif
diff --git a/libsemanage/src/semanage_conf.h b/libsemanage/src/semanage_conf.h
index 7ee139f..4118910 100644
--- a/libsemanage/src/semanage_conf.h
+++ b/libsemanage/src/semanage_conf.h
@@ -40,6 +40,8 @@ typedef struct semanage_conf {
 	int disable_genhomedircon;
 	int handle_unknown;
 	mode_t file_mode;
+	int bzip_blocksize;
+	int bzip_small;
 	struct external_prog *load_policy;
 	struct external_prog *setfiles;
 	struct external_prog *mod_prog, *linked_prog, *kernel_prog;
diff --git a/libsemanage/src/semanage_store.c b/libsemanage/src/semanage_store.c
index 6d4c3ce..a3b0819 100644
--- a/libsemanage/src/semanage_store.c
+++ b/libsemanage/src/semanage_store.c
@@ -1529,7 +1529,7 @@ static int semanage_load_module(semanage_handle_t * sh, const char *filename,
 	ssize_t size;
 	char *data = NULL;
 
-	if ((size = bunzip(fp, &data)) > 0) {
+	if ((size = bunzip(sh, fp, &data)) > 0) {
 		fclose(fp);
 		fp = fmemopen(data, size, "rb");
 		if (!fp) {


-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* RE: [PATCH v2] libsemanage:  Enable configuration of bzip behavior
  2009-08-04 14:08       ` [PATCH v2] libsemanage: Enable configuration of bzip behavior Stephen Smalley
@ 2009-08-04 14:11         ` Joshua Brindle
  2009-08-04 14:22           ` Stephen Smalley
  2009-08-04 14:38         ` [PATCH v3] " Stephen Smalley
  1 sibling, 1 reply; 20+ messages in thread
From: Joshua Brindle @ 2009-08-04 14:11 UTC (permalink / raw)
  To: Stephen Smalley, TaurusHarry; +Cc: selinux-mailing-list, Daniel J Walsh

> From: Stephen Smalley [mailto:sds@tycho.nsa.gov] 
> 
> Allow the administrator to customize the bzip block size and "small"
> flag via semanage.conf.  After applying you can add entries 
> like these to your /etc/selinux/semanage.conf to trade off 
> memory vs disk space (block size) and to trade off memory vs 
> runtime (small):
> 
> bzip-blocksize=4
> bzip-small=1
> 
> You can also disable bzip compression altogether for your module store
> via:
> bzip-blocksize=0
> 

Are these the right config entries for everyone? Why not just have a max
compression, less compression, no compression kind of scheme? How badly
can you mess up the compression by putting semi-random values in these
fields?

> Signed-off-by:  Stephen Smalley <sds@tycho.nsa.gov>
> 
> diff --git a/libsemanage/src/conf-parse.y 
> b/libsemanage/src/conf-parse.y index 2001afb..5b2c859 100644
> --- a/libsemanage/src/conf-parse.y
> +++ b/libsemanage/src/conf-parse.y
> @@ -58,6 +58,7 @@ static int parse_errors;
>  
>  %token MODULE_STORE VERSION EXPAND_CHECK FILE_MODE 
> SAVE_PREVIOUS SAVE_LINKED  %token LOAD_POLICY_START 
> SETFILES_START DISABLE_GENHOMEDIRCON HANDLE_UNKNOWN
> +%token BZIP_BLOCKSIZE BZIP_SMALL
>  %token VERIFY_MOD_START VERIFY_LINKED_START 
> VERIFY_KERNEL_START BLOCK_END  %token PROG_PATH PROG_ARGS  
> %token <s> ARG
> @@ -82,6 +83,8 @@ single_opt:     module_store
>          |       save_linked
>          |       disable_genhomedircon
>          |       handle_unknown
> +	|	bzip_blocksize
> +	|	bzip_small
>          ;
>  
>  module_store:   MODULE_STORE '=' ARG {
> @@ -163,6 +166,16 @@ handle_unknown: HANDLE_UNKNOWN '=' ARG {
>  	free($3);
>   }
>  
> +bzip_blocksize:  BZIP_BLOCKSIZE '=' ARG {
> +	current_conf->bzip_blocksize = atoi($3);
> +	free($3);
> +}
> +	
> +bzip_small:  BZIP_SMALL '=' ARG {
> +	current_conf->bzip_small = atoi($3);
> +	free($3);
> +}
> +
>  command_block: 
>                  command_start external_opts BLOCK_END  {
>                          if (new_external->path == NULL) { @@ 
> -230,6 +243,8 @@ static int semanage_conf_init(semanage_conf_t * conf)
>  	conf->expand_check = 1;
>  	conf->handle_unknown = -1;
>  	conf->file_mode = 0644;
> +	conf->bzip_blocksize = 9;
> +	conf->bzip_small = 0;
>  
>  	conf->save_previous = 0;
>  	conf->save_linked = 0;
> diff --git a/libsemanage/src/conf-scan.l 
> b/libsemanage/src/conf-scan.l index faa0aeb..8af5137 100644
> --- a/libsemanage/src/conf-scan.l
> +++ b/libsemanage/src/conf-scan.l
> @@ -47,6 +47,8 @@ save-previous     return SAVE_PREVIOUS;
>  save-linked       return SAVE_LINKED;
>  disable-genhomedircon return DISABLE_GENHOMEDIRCON;
>  handle-unknown    return HANDLE_UNKNOWN;
> +bzip-blocksize	return BZIP_BLOCKSIZE;
> +bzip-small	return BZIP_SMALL;
>  "[load_policy]"   return LOAD_POLICY_START;
>  "[setfiles]"      return SETFILES_START;
>  "[verify module]" return VERIFY_MOD_START; diff --git 
> a/libsemanage/src/direct_api.c b/libsemanage/src/direct_api.c 
> index bd13387..92799ad 100644
> --- a/libsemanage/src/direct_api.c
> +++ b/libsemanage/src/direct_api.c
> @@ -401,7 +401,9 @@ static int 
> parse_base_headers(semanage_handle_t * sh,
>  
>  /* bzip() a data to a file, returning the total number of 
> compressed bytes
>   * in the file.  Returns -1 if file could not be compressed. 
> */ -static ssize_t bzip(const char *filename, char *data, 
> size_t num_bytes) {
> +static ssize_t bzip(semanage_handle_t *sh, const char 
> *filename, char *data,
> +			size_t num_bytes)
> +{
>  	BZFILE* b;
>  	size_t  size = 1<<16;
>  	int     bzerror;
> @@ -413,7 +415,16 @@ static ssize_t bzip(const char 
> *filename, char *data, size_t num_bytes) {
>  		return -1;
>  	}
>  
> -	b = BZ2_bzWriteOpen( &bzerror, f, 9, 0, 0);
> +	if (!sh->conf->bzip_blocksize) {
> +		if (fwrite(data, 1, num_bytes, f) < num_bytes) {
> +			fclose(f);
> +			return -1;
> +		}
> +		fclose(f);
> +		return num_bytes;
> +	}
> +
> +	b = BZ2_bzWriteOpen( &bzerror, f, 
> sh->conf->bzip_blocksize, 0, 0);
>  	if (bzerror != BZ_OK) {
>  		BZ2_bzWriteClose ( &bzerror, b, 1, 0, 0 );
>  		return -1;
> @@ -443,15 +454,19 @@ static ssize_t bzip(const char 
> *filename, char *data, size_t num_bytes) {
>  
>  /* bunzip() a file to '*data', returning the total number of 
> uncompressed bytes
>   * in the file.  Returns -1 if file could not be 
> decompressed. */ -ssize_t bunzip(FILE *f, char **data) {
> +ssize_t bunzip(semanage_handle_t *sh, FILE *f, char **data) {
>  	BZFILE* b;
>  	size_t  nBuf;
>  	char    buf[1<<18];
>  	size_t  size = sizeof(buf);
>  	int     bzerror;
>  	size_t  total=0;
> +
> +	if (!sh->conf->bzip_blocksize)
> +		return -1;
>  	
> -	b = BZ2_bzReadOpen ( &bzerror, f, 0, 0, NULL, 0 );
> +	b = BZ2_bzReadOpen ( &bzerror, f, 0, 
> sh->conf->bzip_small, NULL, 0 );
>  	if ( bzerror != BZ_OK ) {
>  		BZ2_bzReadClose ( &bzerror, b );
>  		return -1;
> @@ -486,11 +501,12 @@ ssize_t bunzip(FILE *f, char **data) {
>   * the file into '*data'.
>   * Returns the total number of bytes in memory .
>   * Returns -1 if file could not be opened or mapped. */ 
> -static ssize_t map_file(int fd, char **data, int *compressed)
> +static ssize_t map_file(semanage_handle_t *sh, int fd, char **data,
> +			int *compressed)
>  {
>  	ssize_t size = -1;
>  	char *uncompress;
> -	if ((size = bunzip(fdopen(fd, "r"), &uncompress)) > 0) {
> +	if ((size = bunzip(sh, fdopen(fd, "r"), &uncompress)) > 0) {
>  		*data = mmap(0, size, PROT_READ|PROT_WRITE, 
> MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
>  		if (*data == MAP_FAILED) {
>  			free(uncompress);
> @@ -997,7 +1013,7 @@ static int 
> semanage_direct_install(semanage_handle_t * sh,
>  					   &filename)) != 0) {
>  		goto cleanup;
>  	}
> -	if (bzip(filename, data, data_len) <= 0) {
> +	if (bzip(sh, filename, data, data_len) <= 0) {
>  		ERR(sh, "Error while writing to %s.", filename);
>  		retval = -3;
>  		goto cleanup;
> @@ -1029,7 +1045,7 @@ static int 
> semanage_direct_install_file(semanage_handle_t * sh,
>  		return -1;
>  	}
>  
> -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
>  		goto cleanup;
>  	}
>  		
> @@ -1127,7 +1143,7 @@ static int 
> semanage_direct_upgrade(semanage_handle_t * sh,
>  						 data, data_len, 
>  						 &filename);
>  	if (retval == 0) {
> -		if (bzip(filename, data, data_len) <= 0) {
> +		if (bzip(sh, filename, data, data_len) <= 0) {
>  			ERR(sh, "Error while writing to %s.", filename);
>  			retval = -3;
>  		}
> @@ -1155,7 +1171,7 @@ static int 
> semanage_direct_upgrade_file(semanage_handle_t * sh,
>  		return -1;
>  	}
>  
> -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
>  		goto cleanup;
>  	}
>  
> @@ -1197,7 +1213,7 @@ static int 
> semanage_direct_install_base(semanage_handle_t * sh,
>  	if ((filename = semanage_path(SEMANAGE_TMP, 
> SEMANAGE_BASE)) == NULL) {
>  		goto cleanup;
>  	}
> -	if (bzip(filename, base_data, data_len) <= 0) {
> +	if (bzip(sh, filename, base_data, data_len) <= 0) {
>  		ERR(sh, "Error while writing to %s.", filename);
>  		retval = -3;
>  		goto cleanup;
> @@ -1225,7 +1241,7 @@ static int 
> semanage_direct_install_base_file(semanage_handle_t * sh,
>  		return -1;
>  	}
>  
> -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
>  		goto cleanup;
>  	}
>  		
> @@ -1347,7 +1363,7 @@ static int 
> semanage_direct_list(semanage_handle_t * sh,
>  		ssize_t size;
>  		char *data = NULL;
>  
> -		if ((size = bunzip(fp, &data)) > 0) {
> +		if ((size = bunzip(sh, fp, &data)) > 0) {
>  			fclose(fp);
>  			fp = fmemopen(data, size, "rb");
>  			if (!fp) {
> diff --git a/libsemanage/src/direct_api.h 
> b/libsemanage/src/direct_api.h index 1ad7d1d..e56107b 100644
> --- a/libsemanage/src/direct_api.h
> +++ b/libsemanage/src/direct_api.h
> @@ -41,6 +41,6 @@ int semanage_direct_mls_enabled(struct 
> semanage_handle *sh);
>  
>  #include <stdio.h>
>  #include <unistd.h>
> -ssize_t bunzip(FILE *f, char **data);
> +ssize_t bunzip(struct semanage_handle *sh, FILE *f, char **data);
>  
>  #endif
> diff --git a/libsemanage/src/semanage_conf.h 
> b/libsemanage/src/semanage_conf.h index 7ee139f..4118910 100644
> --- a/libsemanage/src/semanage_conf.h
> +++ b/libsemanage/src/semanage_conf.h
> @@ -40,6 +40,8 @@ typedef struct semanage_conf {
>  	int disable_genhomedircon;
>  	int handle_unknown;
>  	mode_t file_mode;
> +	int bzip_blocksize;
> +	int bzip_small;
>  	struct external_prog *load_policy;
>  	struct external_prog *setfiles;
>  	struct external_prog *mod_prog, *linked_prog, 
> *kernel_prog; diff --git a/libsemanage/src/semanage_store.c 
> b/libsemanage/src/semanage_store.c
> index 6d4c3ce..a3b0819 100644
> --- a/libsemanage/src/semanage_store.c
> +++ b/libsemanage/src/semanage_store.c
> @@ -1529,7 +1529,7 @@ static int 
> semanage_load_module(semanage_handle_t * sh, const char *filename,
>  	ssize_t size;
>  	char *data = NULL;
>  
> -	if ((size = bunzip(fp, &data)) > 0) {
> +	if ((size = bunzip(sh, fp, &data)) > 0) {
>  		fclose(fp);
>  		fp = fmemopen(data, size, "rb");
>  		if (!fp) {
> 
> 
> --
> Stephen Smalley
> National Security Agency
> 
> 


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: [PATCH v2] libsemanage:  Enable configuration of bzip behavior
  2009-08-04 14:11         ` Joshua Brindle
@ 2009-08-04 14:22           ` Stephen Smalley
  2009-08-04 14:40             ` Stephen Smalley
  0 siblings, 1 reply; 20+ messages in thread
From: Stephen Smalley @ 2009-08-04 14:22 UTC (permalink / raw)
  To: Joshua Brindle; +Cc: TaurusHarry, selinux-mailing-list, Daniel J Walsh

On Tue, 2009-08-04 at 10:11 -0400, Joshua Brindle wrote:
> > From: Stephen Smalley [mailto:sds@tycho.nsa.gov] 
> > 
> > Allow the administrator to customize the bzip block size and "small"
> > flag via semanage.conf.  After applying you can add entries 
> > like these to your /etc/selinux/semanage.conf to trade off 
> > memory vs disk space (block size) and to trade off memory vs 
> > runtime (small):
> > 
> > bzip-blocksize=4
> > bzip-small=1
> > 
> > You can also disable bzip compression altogether for your module store
> > via:
> > bzip-blocksize=0
> > 
> 
> Are these the right config entries for everyone? Why not just have a max
> compression, less compression, no compression kind of scheme? How badly
> can you mess up the compression by putting semi-random values in these
> fields?

The default values in the absence of any semanage.conf entries will
remain blocksize 9, small 0 (i.e. maximize compression and minimize
decompression runtime at a cost in memory use).  If the admin chooses to
configure entries in semanage.conf, then he can use any value from 0 (no
compression) through 9 for blocksize, and if he uses anything outside of
that range he'll get an error upon attempting to insert a module (it
will fail in bzip when trying to write the module file).  bzip-small
gets treated as a boolean so any non-zero value will cause it to
optimize for memory use over runtime during decompression.

I can't see any situation where the admin can hurt himself without
knowing about it, as it requires manual configuration to change the
defaults, setting bzip-blocksize > 9 will be immediately evident upon
the next attempt to run semodule, setting bzip-small to any non-zero
value is treated identically to setting it to 1, and the two values are
completely independent of one another.  And admins will already be
familiar with the -1 through -9 options of bzip2 and gzip.



--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [PATCH v3] libsemanage:  Enable configuration of bzip behavior
  2009-08-04 14:08       ` [PATCH v2] libsemanage: Enable configuration of bzip behavior Stephen Smalley
  2009-08-04 14:11         ` Joshua Brindle
@ 2009-08-04 14:38         ` Stephen Smalley
  2009-08-04 15:23           ` Daniel J Walsh
  1 sibling, 1 reply; 20+ messages in thread
From: Stephen Smalley @ 2009-08-04 14:38 UTC (permalink / raw)
  To: TaurusHarry; +Cc: selinux-mailing-list, Daniel J Walsh, Joshua Brindle

Allow the administrator to customize the bzip block size and "small"
flag via semanage.conf.  After applying you can add entries like these
to your /etc/selinux/semanage.conf to trade off memory vs disk space
(block size) and to trade off memory vs runtime (small):

bzip-blocksize=4
bzip-small=true

You can also disable bzip compression altogether for your module store
via:
bzip-blocksize=0

The semanage.conf entries are now validated against legal value ranges
at handle creation time.

Signed-off-by:  Stephen Smalley <sds@tycho.nsa.gov>

diff --git a/libsemanage/src/conf-parse.y b/libsemanage/src/conf-parse.y
index 2001afb..f2473aa 100644
--- a/libsemanage/src/conf-parse.y
+++ b/libsemanage/src/conf-parse.y
@@ -58,6 +58,7 @@ static int parse_errors;
 
 %token MODULE_STORE VERSION EXPAND_CHECK FILE_MODE SAVE_PREVIOUS SAVE_LINKED
 %token LOAD_POLICY_START SETFILES_START DISABLE_GENHOMEDIRCON HANDLE_UNKNOWN
+%token BZIP_BLOCKSIZE BZIP_SMALL
 %token VERIFY_MOD_START VERIFY_LINKED_START VERIFY_KERNEL_START BLOCK_END
 %token PROG_PATH PROG_ARGS
 %token <s> ARG
@@ -82,6 +83,8 @@ single_opt:     module_store
         |       save_linked
         |       disable_genhomedircon
         |       handle_unknown
+	|	bzip_blocksize
+	|	bzip_small
         ;
 
 module_store:   MODULE_STORE '=' ARG {
@@ -163,6 +166,26 @@ handle_unknown: HANDLE_UNKNOWN '=' ARG {
 	free($3);
  }
 
+bzip_blocksize:  BZIP_BLOCKSIZE '=' ARG {
+	int blocksize = atoi($3);
+	free($3);
+	if (blocksize > 9)
+		yyerror("bzip-blocksize can only be in the range 0-9");
+	else
+		current_conf->bzip_blocksize = blocksize;
+}
+	
+bzip_small:  BZIP_SMALL '=' ARG {
+	if (strcasecmp($3, "false") == 0) {
+		current_conf->bzip_small = 0;
+	} else if (strcasecmp($3, "true") == 0) {
+		current_conf->bzip_small = 1;
+	} else {
+		yyerror("bzip-small can only be 'true' or 'false'");
+	}
+	free($3);
+}
+
 command_block: 
                 command_start external_opts BLOCK_END  {
                         if (new_external->path == NULL) {
@@ -230,6 +253,8 @@ static int semanage_conf_init(semanage_conf_t * conf)
 	conf->expand_check = 1;
 	conf->handle_unknown = -1;
 	conf->file_mode = 0644;
+	conf->bzip_blocksize = 9;
+	conf->bzip_small = 0;
 
 	conf->save_previous = 0;
 	conf->save_linked = 0;
diff --git a/libsemanage/src/conf-scan.l b/libsemanage/src/conf-scan.l
index faa0aeb..8af5137 100644
--- a/libsemanage/src/conf-scan.l
+++ b/libsemanage/src/conf-scan.l
@@ -47,6 +47,8 @@ save-previous     return SAVE_PREVIOUS;
 save-linked       return SAVE_LINKED;
 disable-genhomedircon return DISABLE_GENHOMEDIRCON;
 handle-unknown    return HANDLE_UNKNOWN;
+bzip-blocksize	return BZIP_BLOCKSIZE;
+bzip-small	return BZIP_SMALL;
 "[load_policy]"   return LOAD_POLICY_START;
 "[setfiles]"      return SETFILES_START;
 "[verify module]" return VERIFY_MOD_START;
diff --git a/libsemanage/src/direct_api.c b/libsemanage/src/direct_api.c
index bd13387..92799ad 100644
--- a/libsemanage/src/direct_api.c
+++ b/libsemanage/src/direct_api.c
@@ -401,7 +401,9 @@ static int parse_base_headers(semanage_handle_t * sh,
 
 /* bzip() a data to a file, returning the total number of compressed bytes
  * in the file.  Returns -1 if file could not be compressed. */
-static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
+static ssize_t bzip(semanage_handle_t *sh, const char *filename, char *data,
+			size_t num_bytes) 
+{
 	BZFILE* b;
 	size_t  size = 1<<16;
 	int     bzerror;
@@ -413,7 +415,16 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
 		return -1;
 	}
 
-	b = BZ2_bzWriteOpen( &bzerror, f, 9, 0, 0);
+	if (!sh->conf->bzip_blocksize) {
+		if (fwrite(data, 1, num_bytes, f) < num_bytes) {
+			fclose(f);
+			return -1;
+		}
+		fclose(f);
+		return num_bytes;
+	}
+
+	b = BZ2_bzWriteOpen( &bzerror, f, sh->conf->bzip_blocksize, 0, 0);
 	if (bzerror != BZ_OK) {
 		BZ2_bzWriteClose ( &bzerror, b, 1, 0, 0 );
 		return -1;
@@ -443,15 +454,19 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
 
 /* bunzip() a file to '*data', returning the total number of uncompressed bytes
  * in the file.  Returns -1 if file could not be decompressed. */
-ssize_t bunzip(FILE *f, char **data) {
+ssize_t bunzip(semanage_handle_t *sh, FILE *f, char **data)
+{
 	BZFILE* b;
 	size_t  nBuf;
 	char    buf[1<<18];
 	size_t  size = sizeof(buf);
 	int     bzerror;
 	size_t  total=0;
+
+	if (!sh->conf->bzip_blocksize)
+		return -1;
 	
-	b = BZ2_bzReadOpen ( &bzerror, f, 0, 0, NULL, 0 );
+	b = BZ2_bzReadOpen ( &bzerror, f, 0, sh->conf->bzip_small, NULL, 0 );
 	if ( bzerror != BZ_OK ) {
 		BZ2_bzReadClose ( &bzerror, b );
 		return -1;
@@ -486,11 +501,12 @@ ssize_t bunzip(FILE *f, char **data) {
  * the file into '*data'.
  * Returns the total number of bytes in memory .
  * Returns -1 if file could not be opened or mapped. */
-static ssize_t map_file(int fd, char **data, int *compressed)
+static ssize_t map_file(semanage_handle_t *sh, int fd, char **data,
+			int *compressed)
 {
 	ssize_t size = -1;
 	char *uncompress;
-	if ((size = bunzip(fdopen(fd, "r"), &uncompress)) > 0) {
+	if ((size = bunzip(sh, fdopen(fd, "r"), &uncompress)) > 0) {
 		*data = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
 		if (*data == MAP_FAILED) {
 			free(uncompress);
@@ -997,7 +1013,7 @@ static int semanage_direct_install(semanage_handle_t * sh,
 					   &filename)) != 0) {
 		goto cleanup;
 	}
-	if (bzip(filename, data, data_len) <= 0) {
+	if (bzip(sh, filename, data, data_len) <= 0) {
 		ERR(sh, "Error while writing to %s.", filename);
 		retval = -3;
 		goto cleanup;
@@ -1029,7 +1045,7 @@ static int semanage_direct_install_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 		
@@ -1127,7 +1143,7 @@ static int semanage_direct_upgrade(semanage_handle_t * sh,
 						 data, data_len, 
 						 &filename);
 	if (retval == 0) {
-		if (bzip(filename, data, data_len) <= 0) {
+		if (bzip(sh, filename, data, data_len) <= 0) {
 			ERR(sh, "Error while writing to %s.", filename);
 			retval = -3;
 		}
@@ -1155,7 +1171,7 @@ static int semanage_direct_upgrade_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 
@@ -1197,7 +1213,7 @@ static int semanage_direct_install_base(semanage_handle_t * sh,
 	if ((filename = semanage_path(SEMANAGE_TMP, SEMANAGE_BASE)) == NULL) {
 		goto cleanup;
 	}
-	if (bzip(filename, base_data, data_len) <= 0) {
+	if (bzip(sh, filename, base_data, data_len) <= 0) {
 		ERR(sh, "Error while writing to %s.", filename);
 		retval = -3;
 		goto cleanup;
@@ -1225,7 +1241,7 @@ static int semanage_direct_install_base_file(semanage_handle_t * sh,
 		return -1;
 	}
 
-	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
+	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
 		goto cleanup;
 	}
 		
@@ -1347,7 +1363,7 @@ static int semanage_direct_list(semanage_handle_t * sh,
 		ssize_t size;
 		char *data = NULL;
 
-		if ((size = bunzip(fp, &data)) > 0) {
+		if ((size = bunzip(sh, fp, &data)) > 0) {
 			fclose(fp);
 			fp = fmemopen(data, size, "rb");
 			if (!fp) {
diff --git a/libsemanage/src/direct_api.h b/libsemanage/src/direct_api.h
index 1ad7d1d..e56107b 100644
--- a/libsemanage/src/direct_api.h
+++ b/libsemanage/src/direct_api.h
@@ -41,6 +41,6 @@ int semanage_direct_mls_enabled(struct semanage_handle *sh);
 
 #include <stdio.h>
 #include <unistd.h>
-ssize_t bunzip(FILE *f, char **data);
+ssize_t bunzip(struct semanage_handle *sh, FILE *f, char **data);
 
 #endif
diff --git a/libsemanage/src/semanage_conf.h b/libsemanage/src/semanage_conf.h
index 7ee139f..4118910 100644
--- a/libsemanage/src/semanage_conf.h
+++ b/libsemanage/src/semanage_conf.h
@@ -40,6 +40,8 @@ typedef struct semanage_conf {
 	int disable_genhomedircon;
 	int handle_unknown;
 	mode_t file_mode;
+	int bzip_blocksize;
+	int bzip_small;
 	struct external_prog *load_policy;
 	struct external_prog *setfiles;
 	struct external_prog *mod_prog, *linked_prog, *kernel_prog;
diff --git a/libsemanage/src/semanage_store.c b/libsemanage/src/semanage_store.c
index 6d4c3ce..a3b0819 100644
--- a/libsemanage/src/semanage_store.c
+++ b/libsemanage/src/semanage_store.c
@@ -1529,7 +1529,7 @@ static int semanage_load_module(semanage_handle_t * sh, const char *filename,
 	ssize_t size;
 	char *data = NULL;
 
-	if ((size = bunzip(fp, &data)) > 0) {
+	if ((size = bunzip(sh, fp, &data)) > 0) {
 		fclose(fp);
 		fp = fmemopen(data, size, "rb");
 		if (!fp) {

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* RE: [PATCH v2] libsemanage:  Enable configuration of bzip behavior
  2009-08-04 14:22           ` Stephen Smalley
@ 2009-08-04 14:40             ` Stephen Smalley
  0 siblings, 0 replies; 20+ messages in thread
From: Stephen Smalley @ 2009-08-04 14:40 UTC (permalink / raw)
  To: Joshua Brindle; +Cc: TaurusHarry, selinux-mailing-list, Daniel J Walsh

On Tue, 2009-08-04 at 10:22 -0400, Stephen Smalley wrote:
> On Tue, 2009-08-04 at 10:11 -0400, Joshua Brindle wrote:
> > > From: Stephen Smalley [mailto:sds@tycho.nsa.gov] 
> > > 
> > > Allow the administrator to customize the bzip block size and "small"
> > > flag via semanage.conf.  After applying you can add entries 
> > > like these to your /etc/selinux/semanage.conf to trade off 
> > > memory vs disk space (block size) and to trade off memory vs 
> > > runtime (small):
> > > 
> > > bzip-blocksize=4
> > > bzip-small=1
> > > 
> > > You can also disable bzip compression altogether for your module store
> > > via:
> > > bzip-blocksize=0
> > > 
> > 
> > Are these the right config entries for everyone? Why not just have a max
> > compression, less compression, no compression kind of scheme? How badly
> > can you mess up the compression by putting semi-random values in these
> > fields?
> 
> The default values in the absence of any semanage.conf entries will
> remain blocksize 9, small 0 (i.e. maximize compression and minimize
> decompression runtime at a cost in memory use).  If the admin chooses to
> configure entries in semanage.conf, then he can use any value from 0 (no
> compression) through 9 for blocksize, and if he uses anything outside of
> that range he'll get an error upon attempting to insert a module (it
> will fail in bzip when trying to write the module file).  bzip-small
> gets treated as a boolean so any non-zero value will cause it to
> optimize for memory use over runtime during decompression.
> 
> I can't see any situation where the admin can hurt himself without
> knowing about it, as it requires manual configuration to change the
> defaults, setting bzip-blocksize > 9 will be immediately evident upon
> the next attempt to run semodule, setting bzip-small to any non-zero
> value is treated identically to setting it to 1, and the two values are
> completely independent of one another.  And admins will already be
> familiar with the -1 through -9 options of bzip2 and gzip.

Added range checking for bzip-blocksize, and changed bzip-small to a
boolean in the config in the latest patch version.

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH v3] libsemanage:  Enable configuration of bzip behavior
  2009-08-04 14:38         ` [PATCH v3] " Stephen Smalley
@ 2009-08-04 15:23           ` Daniel J Walsh
  2009-08-04 15:37             ` Mike Edenfield
  2009-08-04 16:07             ` Stephen Smalley
  0 siblings, 2 replies; 20+ messages in thread
From: Daniel J Walsh @ 2009-08-04 15:23 UTC (permalink / raw)
  To: Stephen Smalley; +Cc: TaurusHarry, selinux-mailing-list, Joshua Brindle

On 08/04/2009 10:38 AM, Stephen Smalley wrote:
> Allow the administrator to customize the bzip block size and "small"
> flag via semanage.conf.  After applying you can add entries like these
> to your /etc/selinux/semanage.conf to trade off memory vs disk space
> (block size) and to trade off memory vs runtime (small):
> 
> bzip-blocksize=4
> bzip-small=true
> 
> You can also disable bzip compression altogether for your module store
> via:
> bzip-blocksize=0
> 
> The semanage.conf entries are now validated against legal value ranges
> at handle creation time.
> 
> Signed-off-by:  Stephen Smalley <sds@tycho.nsa.gov>
> 
> diff --git a/libsemanage/src/conf-parse.y b/libsemanage/src/conf-parse.y
> index 2001afb..f2473aa 100644
> --- a/libsemanage/src/conf-parse.y
> +++ b/libsemanage/src/conf-parse.y
> @@ -58,6 +58,7 @@ static int parse_errors;
>  
>  %token MODULE_STORE VERSION EXPAND_CHECK FILE_MODE SAVE_PREVIOUS SAVE_LINKED
>  %token LOAD_POLICY_START SETFILES_START DISABLE_GENHOMEDIRCON HANDLE_UNKNOWN
> +%token BZIP_BLOCKSIZE BZIP_SMALL
>  %token VERIFY_MOD_START VERIFY_LINKED_START VERIFY_KERNEL_START BLOCK_END
>  %token PROG_PATH PROG_ARGS
>  %token <s> ARG
> @@ -82,6 +83,8 @@ single_opt:     module_store
>          |       save_linked
>          |       disable_genhomedircon
>          |       handle_unknown
> +	|	bzip_blocksize
> +	|	bzip_small
>          ;
>  
>  module_store:   MODULE_STORE '=' ARG {
> @@ -163,6 +166,26 @@ handle_unknown: HANDLE_UNKNOWN '=' ARG {
>  	free($3);
>   }
>  
> +bzip_blocksize:  BZIP_BLOCKSIZE '=' ARG {
> +	int blocksize = atoi($3);
> +	free($3);
> +	if (blocksize > 9)
> +		yyerror("bzip-blocksize can only be in the range 0-9");
> +	else
> +		current_conf->bzip_blocksize = blocksize;
> +}
> +	
> +bzip_small:  BZIP_SMALL '=' ARG {
> +	if (strcasecmp($3, "false") == 0) {
> +		current_conf->bzip_small = 0;
> +	} else if (strcasecmp($3, "true") == 0) {
> +		current_conf->bzip_small = 1;
> +	} else {
> +		yyerror("bzip-small can only be 'true' or 'false'");
> +	}
> +	free($3);
> +}
> +
>  command_block: 
>                  command_start external_opts BLOCK_END  {
>                          if (new_external->path == NULL) {
> @@ -230,6 +253,8 @@ static int semanage_conf_init(semanage_conf_t * conf)
>  	conf->expand_check = 1;
>  	conf->handle_unknown = -1;
>  	conf->file_mode = 0644;
> +	conf->bzip_blocksize = 9;
> +	conf->bzip_small = 0;
>  
>  	conf->save_previous = 0;
>  	conf->save_linked = 0;
> diff --git a/libsemanage/src/conf-scan.l b/libsemanage/src/conf-scan.l
> index faa0aeb..8af5137 100644
> --- a/libsemanage/src/conf-scan.l
> +++ b/libsemanage/src/conf-scan.l
> @@ -47,6 +47,8 @@ save-previous     return SAVE_PREVIOUS;
>  save-linked       return SAVE_LINKED;
>  disable-genhomedircon return DISABLE_GENHOMEDIRCON;
>  handle-unknown    return HANDLE_UNKNOWN;
> +bzip-blocksize	return BZIP_BLOCKSIZE;
> +bzip-small	return BZIP_SMALL;
>  "[load_policy]"   return LOAD_POLICY_START;
>  "[setfiles]"      return SETFILES_START;
>  "[verify module]" return VERIFY_MOD_START;
> diff --git a/libsemanage/src/direct_api.c b/libsemanage/src/direct_api.c
> index bd13387..92799ad 100644
> --- a/libsemanage/src/direct_api.c
> +++ b/libsemanage/src/direct_api.c
> @@ -401,7 +401,9 @@ static int parse_base_headers(semanage_handle_t * sh,
>  
>  /* bzip() a data to a file, returning the total number of compressed bytes
>   * in the file.  Returns -1 if file could not be compressed. */
> -static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
> +static ssize_t bzip(semanage_handle_t *sh, const char *filename, char *data,
> +			size_t num_bytes) 
> +{
>  	BZFILE* b;
>  	size_t  size = 1<<16;
>  	int     bzerror;
> @@ -413,7 +415,16 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
>  		return -1;
>  	}
>  
> -	b = BZ2_bzWriteOpen( &bzerror, f, 9, 0, 0);
> +	if (!sh->conf->bzip_blocksize) {
> +		if (fwrite(data, 1, num_bytes, f) < num_bytes) {
> +			fclose(f);
> +			return -1;
> +		}
> +		fclose(f);
> +		return num_bytes;
> +	}
> +
> +	b = BZ2_bzWriteOpen( &bzerror, f, sh->conf->bzip_blocksize, 0, 0);
>  	if (bzerror != BZ_OK) {
>  		BZ2_bzWriteClose ( &bzerror, b, 1, 0, 0 );
>  		return -1;
> @@ -443,15 +454,19 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
>  
>  /* bunzip() a file to '*data', returning the total number of uncompressed bytes
>   * in the file.  Returns -1 if file could not be decompressed. */
> -ssize_t bunzip(FILE *f, char **data) {
> +ssize_t bunzip(semanage_handle_t *sh, FILE *f, char **data)
> +{
>  	BZFILE* b;
>  	size_t  nBuf;
>  	char    buf[1<<18];
>  	size_t  size = sizeof(buf);
>  	int     bzerror;
>  	size_t  total=0;
> +
> +	if (!sh->conf->bzip_blocksize)
> +		return -1;
>  	
> -	b = BZ2_bzReadOpen ( &bzerror, f, 0, 0, NULL, 0 );
> +	b = BZ2_bzReadOpen ( &bzerror, f, 0, sh->conf->bzip_small, NULL, 0 );
>  	if ( bzerror != BZ_OK ) {
>  		BZ2_bzReadClose ( &bzerror, b );
>  		return -1;
> @@ -486,11 +501,12 @@ ssize_t bunzip(FILE *f, char **data) {
>   * the file into '*data'.
>   * Returns the total number of bytes in memory .
>   * Returns -1 if file could not be opened or mapped. */
> -static ssize_t map_file(int fd, char **data, int *compressed)
> +static ssize_t map_file(semanage_handle_t *sh, int fd, char **data,
> +			int *compressed)
>  {
>  	ssize_t size = -1;
>  	char *uncompress;
> -	if ((size = bunzip(fdopen(fd, "r"), &uncompress)) > 0) {
> +	if ((size = bunzip(sh, fdopen(fd, "r"), &uncompress)) > 0) {
>  		*data = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
>  		if (*data == MAP_FAILED) {
>  			free(uncompress);
> @@ -997,7 +1013,7 @@ static int semanage_direct_install(semanage_handle_t * sh,
>  					   &filename)) != 0) {
>  		goto cleanup;
>  	}
> -	if (bzip(filename, data, data_len) <= 0) {
> +	if (bzip(sh, filename, data, data_len) <= 0) {
>  		ERR(sh, "Error while writing to %s.", filename);
>  		retval = -3;
>  		goto cleanup;
> @@ -1029,7 +1045,7 @@ static int semanage_direct_install_file(semanage_handle_t * sh,
>  		return -1;
>  	}
>  
> -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
>  		goto cleanup;
>  	}
>  		
> @@ -1127,7 +1143,7 @@ static int semanage_direct_upgrade(semanage_handle_t * sh,
>  						 data, data_len, 
>  						 &filename);
>  	if (retval == 0) {
> -		if (bzip(filename, data, data_len) <= 0) {
> +		if (bzip(sh, filename, data, data_len) <= 0) {
>  			ERR(sh, "Error while writing to %s.", filename);
>  			retval = -3;
>  		}
> @@ -1155,7 +1171,7 @@ static int semanage_direct_upgrade_file(semanage_handle_t * sh,
>  		return -1;
>  	}
>  
> -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
>  		goto cleanup;
>  	}
>  
> @@ -1197,7 +1213,7 @@ static int semanage_direct_install_base(semanage_handle_t * sh,
>  	if ((filename = semanage_path(SEMANAGE_TMP, SEMANAGE_BASE)) == NULL) {
>  		goto cleanup;
>  	}
> -	if (bzip(filename, base_data, data_len) <= 0) {
> +	if (bzip(sh, filename, base_data, data_len) <= 0) {
>  		ERR(sh, "Error while writing to %s.", filename);
>  		retval = -3;
>  		goto cleanup;
> @@ -1225,7 +1241,7 @@ static int semanage_direct_install_base_file(semanage_handle_t * sh,
>  		return -1;
>  	}
>  
> -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
>  		goto cleanup;
>  	}
>  		
> @@ -1347,7 +1363,7 @@ static int semanage_direct_list(semanage_handle_t * sh,
>  		ssize_t size;
>  		char *data = NULL;
>  
> -		if ((size = bunzip(fp, &data)) > 0) {
> +		if ((size = bunzip(sh, fp, &data)) > 0) {
>  			fclose(fp);
>  			fp = fmemopen(data, size, "rb");
>  			if (!fp) {
> diff --git a/libsemanage/src/direct_api.h b/libsemanage/src/direct_api.h
> index 1ad7d1d..e56107b 100644
> --- a/libsemanage/src/direct_api.h
> +++ b/libsemanage/src/direct_api.h
> @@ -41,6 +41,6 @@ int semanage_direct_mls_enabled(struct semanage_handle *sh);
>  
>  #include <stdio.h>
>  #include <unistd.h>
> -ssize_t bunzip(FILE *f, char **data);
> +ssize_t bunzip(struct semanage_handle *sh, FILE *f, char **data);
>  
>  #endif
> diff --git a/libsemanage/src/semanage_conf.h b/libsemanage/src/semanage_conf.h
> index 7ee139f..4118910 100644
> --- a/libsemanage/src/semanage_conf.h
> +++ b/libsemanage/src/semanage_conf.h
> @@ -40,6 +40,8 @@ typedef struct semanage_conf {
>  	int disable_genhomedircon;
>  	int handle_unknown;
>  	mode_t file_mode;
> +	int bzip_blocksize;
> +	int bzip_small;
>  	struct external_prog *load_policy;
>  	struct external_prog *setfiles;
>  	struct external_prog *mod_prog, *linked_prog, *kernel_prog;
> diff --git a/libsemanage/src/semanage_store.c b/libsemanage/src/semanage_store.c
> index 6d4c3ce..a3b0819 100644
> --- a/libsemanage/src/semanage_store.c
> +++ b/libsemanage/src/semanage_store.c
> @@ -1529,7 +1529,7 @@ static int semanage_load_module(semanage_handle_t * sh, const char *filename,
>  	ssize_t size;
>  	char *data = NULL;
>  
> -	if ((size = bunzip(fp, &data)) > 0) {
> +	if ((size = bunzip(sh, fp, &data)) > 0) {
>  		fclose(fp);
>  		fp = fmemopen(data, size, "rb");
>  		if (!fp) {
> 
I just want to make sure that if I change the size in semanage.conf, it will still handle all sizes of bzipped files.
So if we ship xguest.pp.bz2 format it will work.

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH v3] libsemanage:  Enable configuration of bzip behavior
  2009-08-04 15:23           ` Daniel J Walsh
@ 2009-08-04 15:37             ` Mike Edenfield
  2009-08-04 16:07             ` Stephen Smalley
  1 sibling, 0 replies; 20+ messages in thread
From: Mike Edenfield @ 2009-08-04 15:37 UTC (permalink / raw)
  To: Daniel J Walsh
  Cc: Stephen Smalley, TaurusHarry, selinux-mailing-list, Joshua Brindle

On 8/4/2009 11:23 AM, Daniel J Walsh wrote:

> I just want to make sure that if I change the size in semanage.conf, it will still handle all sizes of bzipped files.
> So if we ship xguest.pp.bz2 format it will work.

At compression time, bzip stores the selected block size in the header, 
so the decompression routine knows what to use.  So, yes, it will work fine.

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH v3] libsemanage:  Enable configuration of bzip behavior
  2009-08-04 15:23           ` Daniel J Walsh
  2009-08-04 15:37             ` Mike Edenfield
@ 2009-08-04 16:07             ` Stephen Smalley
  1 sibling, 0 replies; 20+ messages in thread
From: Stephen Smalley @ 2009-08-04 16:07 UTC (permalink / raw)
  To: Daniel J Walsh; +Cc: TaurusHarry, selinux-mailing-list, Joshua Brindle

On Tue, 2009-08-04 at 11:23 -0400, Daniel J Walsh wrote:
> On 08/04/2009 10:38 AM, Stephen Smalley wrote:
> > Allow the administrator to customize the bzip block size and "small"
> > flag via semanage.conf.  After applying you can add entries like these
> > to your /etc/selinux/semanage.conf to trade off memory vs disk space
> > (block size) and to trade off memory vs runtime (small):
> > 
> > bzip-blocksize=4
> > bzip-small=true
> > 
> > You can also disable bzip compression altogether for your module store
> > via:
> > bzip-blocksize=0
> > 
> > The semanage.conf entries are now validated against legal value ranges
> > at handle creation time.
> > 
> > Signed-off-by:  Stephen Smalley <sds@tycho.nsa.gov>
> > 
> > diff --git a/libsemanage/src/conf-parse.y b/libsemanage/src/conf-parse.y
> > index 2001afb..f2473aa 100644
> > --- a/libsemanage/src/conf-parse.y
> > +++ b/libsemanage/src/conf-parse.y
> > @@ -58,6 +58,7 @@ static int parse_errors;
> >  
> >  %token MODULE_STORE VERSION EXPAND_CHECK FILE_MODE SAVE_PREVIOUS SAVE_LINKED
> >  %token LOAD_POLICY_START SETFILES_START DISABLE_GENHOMEDIRCON HANDLE_UNKNOWN
> > +%token BZIP_BLOCKSIZE BZIP_SMALL
> >  %token VERIFY_MOD_START VERIFY_LINKED_START VERIFY_KERNEL_START BLOCK_END
> >  %token PROG_PATH PROG_ARGS
> >  %token <s> ARG
> > @@ -82,6 +83,8 @@ single_opt:     module_store
> >          |       save_linked
> >          |       disable_genhomedircon
> >          |       handle_unknown
> > +	|	bzip_blocksize
> > +	|	bzip_small
> >          ;
> >  
> >  module_store:   MODULE_STORE '=' ARG {
> > @@ -163,6 +166,26 @@ handle_unknown: HANDLE_UNKNOWN '=' ARG {
> >  	free($3);
> >   }
> >  
> > +bzip_blocksize:  BZIP_BLOCKSIZE '=' ARG {
> > +	int blocksize = atoi($3);
> > +	free($3);
> > +	if (blocksize > 9)
> > +		yyerror("bzip-blocksize can only be in the range 0-9");
> > +	else
> > +		current_conf->bzip_blocksize = blocksize;
> > +}
> > +	
> > +bzip_small:  BZIP_SMALL '=' ARG {
> > +	if (strcasecmp($3, "false") == 0) {
> > +		current_conf->bzip_small = 0;
> > +	} else if (strcasecmp($3, "true") == 0) {
> > +		current_conf->bzip_small = 1;
> > +	} else {
> > +		yyerror("bzip-small can only be 'true' or 'false'");
> > +	}
> > +	free($3);
> > +}
> > +
> >  command_block: 
> >                  command_start external_opts BLOCK_END  {
> >                          if (new_external->path == NULL) {
> > @@ -230,6 +253,8 @@ static int semanage_conf_init(semanage_conf_t * conf)
> >  	conf->expand_check = 1;
> >  	conf->handle_unknown = -1;
> >  	conf->file_mode = 0644;
> > +	conf->bzip_blocksize = 9;
> > +	conf->bzip_small = 0;
> >  
> >  	conf->save_previous = 0;
> >  	conf->save_linked = 0;
> > diff --git a/libsemanage/src/conf-scan.l b/libsemanage/src/conf-scan.l
> > index faa0aeb..8af5137 100644
> > --- a/libsemanage/src/conf-scan.l
> > +++ b/libsemanage/src/conf-scan.l
> > @@ -47,6 +47,8 @@ save-previous     return SAVE_PREVIOUS;
> >  save-linked       return SAVE_LINKED;
> >  disable-genhomedircon return DISABLE_GENHOMEDIRCON;
> >  handle-unknown    return HANDLE_UNKNOWN;
> > +bzip-blocksize	return BZIP_BLOCKSIZE;
> > +bzip-small	return BZIP_SMALL;
> >  "[load_policy]"   return LOAD_POLICY_START;
> >  "[setfiles]"      return SETFILES_START;
> >  "[verify module]" return VERIFY_MOD_START;
> > diff --git a/libsemanage/src/direct_api.c b/libsemanage/src/direct_api.c
> > index bd13387..92799ad 100644
> > --- a/libsemanage/src/direct_api.c
> > +++ b/libsemanage/src/direct_api.c
> > @@ -401,7 +401,9 @@ static int parse_base_headers(semanage_handle_t * sh,
> >  
> >  /* bzip() a data to a file, returning the total number of compressed bytes
> >   * in the file.  Returns -1 if file could not be compressed. */
> > -static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
> > +static ssize_t bzip(semanage_handle_t *sh, const char *filename, char *data,
> > +			size_t num_bytes) 
> > +{
> >  	BZFILE* b;
> >  	size_t  size = 1<<16;
> >  	int     bzerror;
> > @@ -413,7 +415,16 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
> >  		return -1;
> >  	}
> >  
> > -	b = BZ2_bzWriteOpen( &bzerror, f, 9, 0, 0);
> > +	if (!sh->conf->bzip_blocksize) {
> > +		if (fwrite(data, 1, num_bytes, f) < num_bytes) {
> > +			fclose(f);
> > +			return -1;
> > +		}
> > +		fclose(f);
> > +		return num_bytes;
> > +	}
> > +
> > +	b = BZ2_bzWriteOpen( &bzerror, f, sh->conf->bzip_blocksize, 0, 0);
> >  	if (bzerror != BZ_OK) {
> >  		BZ2_bzWriteClose ( &bzerror, b, 1, 0, 0 );
> >  		return -1;
> > @@ -443,15 +454,19 @@ static ssize_t bzip(const char *filename, char *data, size_t num_bytes) {
> >  
> >  /* bunzip() a file to '*data', returning the total number of uncompressed bytes
> >   * in the file.  Returns -1 if file could not be decompressed. */
> > -ssize_t bunzip(FILE *f, char **data) {
> > +ssize_t bunzip(semanage_handle_t *sh, FILE *f, char **data)
> > +{
> >  	BZFILE* b;
> >  	size_t  nBuf;
> >  	char    buf[1<<18];
> >  	size_t  size = sizeof(buf);
> >  	int     bzerror;
> >  	size_t  total=0;
> > +
> > +	if (!sh->conf->bzip_blocksize)
> > +		return -1;
> >  	
> > -	b = BZ2_bzReadOpen ( &bzerror, f, 0, 0, NULL, 0 );
> > +	b = BZ2_bzReadOpen ( &bzerror, f, 0, sh->conf->bzip_small, NULL, 0 );
> >  	if ( bzerror != BZ_OK ) {
> >  		BZ2_bzReadClose ( &bzerror, b );
> >  		return -1;
> > @@ -486,11 +501,12 @@ ssize_t bunzip(FILE *f, char **data) {
> >   * the file into '*data'.
> >   * Returns the total number of bytes in memory .
> >   * Returns -1 if file could not be opened or mapped. */
> > -static ssize_t map_file(int fd, char **data, int *compressed)
> > +static ssize_t map_file(semanage_handle_t *sh, int fd, char **data,
> > +			int *compressed)
> >  {
> >  	ssize_t size = -1;
> >  	char *uncompress;
> > -	if ((size = bunzip(fdopen(fd, "r"), &uncompress)) > 0) {
> > +	if ((size = bunzip(sh, fdopen(fd, "r"), &uncompress)) > 0) {
> >  		*data = mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
> >  		if (*data == MAP_FAILED) {
> >  			free(uncompress);
> > @@ -997,7 +1013,7 @@ static int semanage_direct_install(semanage_handle_t * sh,
> >  					   &filename)) != 0) {
> >  		goto cleanup;
> >  	}
> > -	if (bzip(filename, data, data_len) <= 0) {
> > +	if (bzip(sh, filename, data, data_len) <= 0) {
> >  		ERR(sh, "Error while writing to %s.", filename);
> >  		retval = -3;
> >  		goto cleanup;
> > @@ -1029,7 +1045,7 @@ static int semanage_direct_install_file(semanage_handle_t * sh,
> >  		return -1;
> >  	}
> >  
> > -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> > +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
> >  		goto cleanup;
> >  	}
> >  		
> > @@ -1127,7 +1143,7 @@ static int semanage_direct_upgrade(semanage_handle_t * sh,
> >  						 data, data_len, 
> >  						 &filename);
> >  	if (retval == 0) {
> > -		if (bzip(filename, data, data_len) <= 0) {
> > +		if (bzip(sh, filename, data, data_len) <= 0) {
> >  			ERR(sh, "Error while writing to %s.", filename);
> >  			retval = -3;
> >  		}
> > @@ -1155,7 +1171,7 @@ static int semanage_direct_upgrade_file(semanage_handle_t * sh,
> >  		return -1;
> >  	}
> >  
> > -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> > +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
> >  		goto cleanup;
> >  	}
> >  
> > @@ -1197,7 +1213,7 @@ static int semanage_direct_install_base(semanage_handle_t * sh,
> >  	if ((filename = semanage_path(SEMANAGE_TMP, SEMANAGE_BASE)) == NULL) {
> >  		goto cleanup;
> >  	}
> > -	if (bzip(filename, base_data, data_len) <= 0) {
> > +	if (bzip(sh, filename, base_data, data_len) <= 0) {
> >  		ERR(sh, "Error while writing to %s.", filename);
> >  		retval = -3;
> >  		goto cleanup;
> > @@ -1225,7 +1241,7 @@ static int semanage_direct_install_base_file(semanage_handle_t * sh,
> >  		return -1;
> >  	}
> >  
> > -	if ((data_len = map_file(in_fd, &data, &compressed)) <= 0) {
> > +	if ((data_len = map_file(sh, in_fd, &data, &compressed)) <= 0) {
> >  		goto cleanup;
> >  	}
> >  		
> > @@ -1347,7 +1363,7 @@ static int semanage_direct_list(semanage_handle_t * sh,
> >  		ssize_t size;
> >  		char *data = NULL;
> >  
> > -		if ((size = bunzip(fp, &data)) > 0) {
> > +		if ((size = bunzip(sh, fp, &data)) > 0) {
> >  			fclose(fp);
> >  			fp = fmemopen(data, size, "rb");
> >  			if (!fp) {
> > diff --git a/libsemanage/src/direct_api.h b/libsemanage/src/direct_api.h
> > index 1ad7d1d..e56107b 100644
> > --- a/libsemanage/src/direct_api.h
> > +++ b/libsemanage/src/direct_api.h
> > @@ -41,6 +41,6 @@ int semanage_direct_mls_enabled(struct semanage_handle *sh);
> >  
> >  #include <stdio.h>
> >  #include <unistd.h>
> > -ssize_t bunzip(FILE *f, char **data);
> > +ssize_t bunzip(struct semanage_handle *sh, FILE *f, char **data);
> >  
> >  #endif
> > diff --git a/libsemanage/src/semanage_conf.h b/libsemanage/src/semanage_conf.h
> > index 7ee139f..4118910 100644
> > --- a/libsemanage/src/semanage_conf.h
> > +++ b/libsemanage/src/semanage_conf.h
> > @@ -40,6 +40,8 @@ typedef struct semanage_conf {
> >  	int disable_genhomedircon;
> >  	int handle_unknown;
> >  	mode_t file_mode;
> > +	int bzip_blocksize;
> > +	int bzip_small;
> >  	struct external_prog *load_policy;
> >  	struct external_prog *setfiles;
> >  	struct external_prog *mod_prog, *linked_prog, *kernel_prog;
> > diff --git a/libsemanage/src/semanage_store.c b/libsemanage/src/semanage_store.c
> > index 6d4c3ce..a3b0819 100644
> > --- a/libsemanage/src/semanage_store.c
> > +++ b/libsemanage/src/semanage_store.c
> > @@ -1529,7 +1529,7 @@ static int semanage_load_module(semanage_handle_t * sh, const char *filename,
> >  	ssize_t size;
> >  	char *data = NULL;
> >  
> > -	if ((size = bunzip(fp, &data)) > 0) {
> > +	if ((size = bunzip(sh, fp, &data)) > 0) {
> >  		fclose(fp);
> >  		fp = fmemopen(data, size, "rb");
> >  		if (!fp) {
> > 
> I just want to make sure that if I change the size in semanage.conf, it will still handle all sizes of bzipped files.
> So if we ship xguest.pp.bz2 format it will work.

Yes, it will only affect the block size used when libsemanage writes
modules to the store.  It doesn't affect the ability to read any block
size.  And the compile-time defaults will remain blocksize=9 (900k) and
small=0 so you'd only need to set the values in semanage.conf if you
wanted a different setting.

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: A question about selinux userspace tools
       [not found] ` <1249397076.9193.101.camel@moss-pluto.epoch.ncsc.mil>
@ 2009-08-05  3:03   ` TaurusHarry
  2009-08-05 12:09     ` Stephen Smalley
  0 siblings, 1 reply; 20+ messages in thread
From: TaurusHarry @ 2009-08-05  3:03 UTC (permalink / raw)
  To: sds; +Cc: selinux-mailing-list

[-- Attachment #1: Type: text/plain, Size: 1637 bytes --]


Hey Smalley,

So many thanks for giving me your help so promptly, you are such a kind person !:-D

Yep, the quick test reveal that only with disabling bzip feature by "bzip-blocksize=0" would the libsemanage-2.0.33 successfully build a policy store, the option of bzip-blocksize=4 and bzip-small=true would result in the same libsepol out of memeory error.

If there is any thing I could ever help test for you, just let me know.

Best regards,

Harry


> Subject: Re: A question about selinux userspace tools
> From: sds@tycho.nsa.gov
> To: harrytaurus2002@hotmail.com
> Date: Tue, 4 Aug 2009 10:44:36 -0400
> 
> On Tue, 2009-08-04 at 03:11 +0000, TaurusHarry wrote:
> > Hi all,
> > 
> > I am a newcomer to SELinux and have a question about the latest
> > SELinux userspace tools.
> > 
> > Before I try to install the latest refpolicy-20090730 from tresys
> > website on Ubuntu 8.10 on Dell 6101 laptop(x86_32), I have installed
> > the below latest SELinux userspace tools on the laptop:
> 
> Hi Harry,
> 
> If you could try the last version of the patch I posted for libsemanage
> configure your /etc/selinux/semanage.conf with different settings, try
> again, and see if that resolves your problem on your laptop, that would
> be helpful.
> 
> I'd suggest first trying something like:
> bzip-blocksize=4
> bzip-small=true
> 
> If that still doesn't help, then I'd suggest disabling bzip compression
> altogether via:
> bzip-blocksize=0
> 
> -- 
> Stephen Smalley
> National Security Agency
> 

_________________________________________________________________
上Windows Live 中国首页,下载最新版Messenger!
http://www.windowslive.cn

[-- Attachment #2: Type: text/html, Size: 2133 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: A question about selinux userspace tools
  2009-08-05  3:03   ` A question about selinux userspace tools TaurusHarry
@ 2009-08-05 12:09     ` Stephen Smalley
  2009-08-06  1:33       ` TaurusHarry
  0 siblings, 1 reply; 20+ messages in thread
From: Stephen Smalley @ 2009-08-05 12:09 UTC (permalink / raw)
  To: TaurusHarry; +Cc: selinux-mailing-list

On Wed, 2009-08-05 at 03:03 +0000, TaurusHarry wrote:
> Hey Smalley,
> 
> So many thanks for giving me your help so promptly, you are such a
> kind person !:-D
> 
> Yep, the quick test reveal that only with disabling bzip feature by
> "bzip-blocksize=0" would the libsemanage-2.0.33 successfully build a
> policy store, the option of bzip-blocksize=4 and bzip-small=true would
> result in the same libsepol out of memeory error.
> 
> If there is any thing I could ever help test for you, just let me
> know.

Thanks for testing.  Could you let us know the amount of memory and swap
you had configured on your laptop to help with reproducing the behavior?
And also, what your /proc/sys/vm/overcommit_memory
and /proc/sys/vm/overcommit_ratio settings were?

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: A question about selinux userspace tools
  2009-08-05 12:09     ` Stephen Smalley
@ 2009-08-06  1:33       ` TaurusHarry
  2009-08-06 12:10         ` Stephen Smalley
  0 siblings, 1 reply; 20+ messages in thread
From: TaurusHarry @ 2009-08-06  1:33 UTC (permalink / raw)
  To: Stephen Smalley; +Cc: selinux-mailing-list

[-- Attachment #1: Type: text/plain, Size: 1469 bytes --]


Hey Smalley,

On my laptop. both RAM and swap are about 4G bytes, and the overcomit_memory is 0 while the overcommit_ration is 50.

Is it possible to tune these vm settings in some way so that bzip-blocksize=4 or 9 would succeed in building a policy store?

Thanks!

Harry


> Subject: RE: A question about selinux userspace tools
> From: sds@tycho.nsa.gov
> To: harrytaurus2002@hotmail.com
> CC: selinux@tycho.nsa.gov
> Date: Wed, 5 Aug 2009 08:09:11 -0400
> 
> On Wed, 2009-08-05 at 03:03 +0000, TaurusHarry wrote:
> > Hey Smalley,
> > 
> > So many thanks for giving me your help so promptly, you are such a
> > kind person !:-D
> > 
> > Yep, the quick test reveal that only with disabling bzip feature by
> > "bzip-blocksize=0" would the libsemanage-2.0.33 successfully build a
> > policy store, the option of bzip-blocksize=4 and bzip-small=true would
> > result in the same libsepol out of memeory error.
> > 
> > If there is any thing I could ever help test for you, just let me
> > know.
> 
> Thanks for testing.  Could you let us know the amount of memory and swap
> you had configured on your laptop to help with reproducing the behavior?
> And also, what your /proc/sys/vm/overcommit_memory
> and /proc/sys/vm/overcommit_ratio settings were?
> 
> -- 
> Stephen Smalley
> National Security Agency
> 

_________________________________________________________________
张三挖到了元宝,小美又掉进陷阱了,快来MClub与好友齐乐乐!立刻访问!
http://club.msn.cn/?from=3

[-- Attachment #2: Type: text/html, Size: 1852 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: A question about selinux userspace tools
  2009-08-06  1:33       ` TaurusHarry
@ 2009-08-06 12:10         ` Stephen Smalley
  2009-08-07  2:04           ` TaurusHarry
  0 siblings, 1 reply; 20+ messages in thread
From: Stephen Smalley @ 2009-08-06 12:10 UTC (permalink / raw)
  To: TaurusHarry; +Cc: selinux-mailing-list

On Thu, 2009-08-06 at 01:33 +0000, TaurusHarry wrote:
> Hey Smalley,
> 
> On my laptop. both RAM and swap are about 4G bytes, and the
> overcomit_memory is 0 while the overcommit_ration is 50.
> 
> Is it possible to tune these vm settings in some way so that
> bzip-blocksize=4 or 9 would succeed in building a policy store?

That's curious, as I've tested on a much smaller memory configuration (a
VM with 512M RAM, 1G swap) without any problems using the same
overcommit settings.

Is your arch i686 or x86_64?

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: A question about selinux userspace tools
  2009-08-06 12:10         ` Stephen Smalley
@ 2009-08-07  2:04           ` TaurusHarry
  2009-08-07 12:54             ` Stephen Smalley
  0 siblings, 1 reply; 20+ messages in thread
From: TaurusHarry @ 2009-08-07  2:04 UTC (permalink / raw)
  To: Stephen Smalley; +Cc: selinux-mailing-list

[-- Attachment #1: Type: text/plain, Size: 1079 bytes --]


Hi Smalley,

My laptop is i686.

Is there anything I could do to facilitate reproducing the same issue on your side?

Cheers,

Harry


> Subject: RE: A question about selinux userspace tools
> From: sds@tycho.nsa.gov
> To: harrytaurus2002@hotmail.com
> CC: selinux@tycho.nsa.gov
> Date: Thu, 6 Aug 2009 08:10:16 -0400
> 
> On Thu, 2009-08-06 at 01:33 +0000, TaurusHarry wrote:
> > Hey Smalley,
> > 
> > On my laptop. both RAM and swap are about 4G bytes, and the
> > overcomit_memory is 0 while the overcommit_ration is 50.
> > 
> > Is it possible to tune these vm settings in some way so that
> > bzip-blocksize=4 or 9 would succeed in building a policy store?
> 
> That's curious, as I've tested on a much smaller memory configuration (a
> VM with 512M RAM, 1G swap) without any problems using the same
> overcommit settings.
> 
> Is your arch i686 or x86_64?
> 
> -- 
> Stephen Smalley
> National Security Agency
> 

_________________________________________________________________
张三挖到了元宝,小美又掉进陷阱了,快来MClub与好友齐乐乐!立刻访问!
http://club.msn.cn/?from=3

[-- Attachment #2: Type: text/html, Size: 1423 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: A question about selinux userspace tools
  2009-08-07  2:04           ` TaurusHarry
@ 2009-08-07 12:54             ` Stephen Smalley
  2009-08-10  7:00               ` TaurusHarry
  0 siblings, 1 reply; 20+ messages in thread
From: Stephen Smalley @ 2009-08-07 12:54 UTC (permalink / raw)
  To: TaurusHarry; +Cc: selinux-mailing-list

On Fri, 2009-08-07 at 02:04 +0000, TaurusHarry wrote:
> Hi Smalley,
> 
> My laptop is i686.
> 
> Is there anything I could do to facilitate reproducing the same issue
> on your side?

grep flags /proc/cpuinfo
cat /proc/meminfo

-- 
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* RE: A question about selinux userspace tools
  2009-08-07 12:54             ` Stephen Smalley
@ 2009-08-10  7:00               ` TaurusHarry
  0 siblings, 0 replies; 20+ messages in thread
From: TaurusHarry @ 2009-08-10  7:00 UTC (permalink / raw)
  To: Stephen Smalley; +Cc: selinux-mailing-list

[-- Attachment #1: Type: text/plain, Size: 2717 bytes --]


Hi Smalley,

Below are the information of my laptop you may want to know.

Anything I may help you investigate this issue just let me know.

Best regards,

Harry

---

cao@cao-laptop:~$ uname -a
Linux cao-laptop 2.6.27-7-generic #1 SMP Fri Oct 24 06:42:44 UTC 2008 i686 GNU/Linux
cao@cao-laptop:~$ grep flags /proc/cpuinfo 
flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm ida
flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm ida
cao@cao-laptop:~$ cat /proc/meminfo 
MemTotal:      3620032 kB
MemFree:       2152164 kB
Buffers:        157004 kB
Cached:         924304 kB
SwapCached:          0 kB
Active:         598820 kB
Inactive:       718548 kB
HighTotal:     2740532 kB
HighFree:      1562548 kB
LowTotal:       879500 kB
LowFree:        589616 kB
SwapTotal:     4000176 kB
SwapFree:      4000176 kB
Dirty:             144 kB
Writeback:           0 kB
AnonPages:      236060 kB
Mapped:          82024 kB
Slab:            92920 kB
SReclaimable:    67280 kB
SUnreclaim:      25640 kB
PageTables:       2700 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
WritebackTmp:        0 kB
CommitLimit:   5810192 kB
Committed_AS:   601344 kB
VmallocTotal:   110584 kB
VmallocUsed:     47668 kB
VmallocChunk:    59892 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
HugePages_Surp:      0
Hugepagesize:     4096 kB
DirectMap4k:     81920 kB
DirectMap4M:    835584 kB
cao@cao-laptop:~$ 

> Subject: RE: A question about selinux userspace tools
> From: sds@tycho.nsa.gov
> To: harrytaurus2002@hotmail.com
> CC: selinux@tycho.nsa.gov
> Date: Fri, 7 Aug 2009 08:54:17 -0400
> 
> On Fri, 2009-08-07 at 02:04 +0000, TaurusHarry wrote:
> > Hi Smalley,
> > 
> > My laptop is i686.
> > 
> > Is there anything I could do to facilitate reproducing the same issue
> > on your side?
> 
> grep flags /proc/cpuinfo
> cat /proc/meminfo
> 
> -- 
> Stephen Smalley
> National Security Agency
> 
> 
> --
> This message was distributed to subscribers of the selinux mailing list.
> If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
> the words "unsubscribe selinux" without quotes as the message.

_________________________________________________________________
您可以借助 Windows Live 整理、编辑和共享您的照片。
http://www.microsoft.com/china/windows/windowslive/products/photo-gallery-edit.aspx

[-- Attachment #2: Type: text/html, Size: 4337 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2009-08-10  7:00 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-08-04  3:11 A question about selinux userspace tools TaurusHarry
     [not found] ` <A0FAB46BC43C4D8ABE2581D6EF3E2713@CIMSLAB.NL>
2009-08-04  8:57   ` TaurusHarry
2009-08-04 12:13 ` Stephen Smalley
2009-08-04 12:40   ` Stephen Smalley
2009-08-04 13:25     ` Stephen Smalley
2009-08-04 14:08       ` [PATCH v2] libsemanage: Enable configuration of bzip behavior Stephen Smalley
2009-08-04 14:11         ` Joshua Brindle
2009-08-04 14:22           ` Stephen Smalley
2009-08-04 14:40             ` Stephen Smalley
2009-08-04 14:38         ` [PATCH v3] " Stephen Smalley
2009-08-04 15:23           ` Daniel J Walsh
2009-08-04 15:37             ` Mike Edenfield
2009-08-04 16:07             ` Stephen Smalley
     [not found] ` <1249397076.9193.101.camel@moss-pluto.epoch.ncsc.mil>
2009-08-05  3:03   ` A question about selinux userspace tools TaurusHarry
2009-08-05 12:09     ` Stephen Smalley
2009-08-06  1:33       ` TaurusHarry
2009-08-06 12:10         ` Stephen Smalley
2009-08-07  2:04           ` TaurusHarry
2009-08-07 12:54             ` Stephen Smalley
2009-08-10  7:00               ` TaurusHarry

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.