Future of SETools and CIL

Future of SETools and CIL

[Steve Lawrence]

It has become clear that SETools has fallen behind userspace in terms of
features and general maintenance. We would like to get it to the point
where this is not the case, and to find a way to make sure it does not
happen again. We think the solution to the maintenance issue is to make
it more visible by merging the more useful parts of SETools into the
userspace repo, while deprecating/removing the remaining pieces.

However, we are well aware of the complexity of SETools, primarily
libapol, and that upstreaming it without any changes would not solve the
problems. So, we have done a little work behind the scenes to find a way
to reduce the complexity of libapol. As a first stab at it, we started
with an older version of libapol that is quite a bit less complex and
began porting it forward for use with modern userspace, and seeing if it
would make sense to eventually merge. But before we get too deep into
this port, we wanted to start a discussion with the SELinux community to
make sure we are headed in the right direction. So to start, does this
seem like a good idea (both merging with userspace and porting older
libapol)? Or should we take a completely different direction (e.g. the
use of graphing databases as a replacement of apol has been mentioned in
the past)?

Another discussion we would like to have, which may affect the future of
SETools/apol, is CIL. Is there still interest in CIL? And if so, have
there been any thoughts on using and migrating to CIL? Is more work
needed before this can happen? Has anyone put thought into higher level
languages that could sit on top of CIL? If there is interest, this may
affect the SETools changes, for example, syntactic policy analysis for
CIL is likely very different than current policy.

Thanks,
- Steve


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[Daniel J Walsh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/16/2013 09:33 AM, Steve Lawrence wrote:
> It has become clear that SETools has fallen behind userspace in terms of 
> features and general maintenance. We would like to get it to the point 
> where this is not the case, and to find a way to make sure it does not 
> happen again. We think the solution to the maintenance issue is to make it
> more visible by merging the more useful parts of SETools into the userspace
> repo, while deprecating/removing the remaining pieces.
> 
I think this would be a good idea.   I have been adding sepolicy which uses
libapol and libqpol, to gather data from the installed policy.  We have
several patches for setools that never made it upstream.  We also are heavy
users of sesearch and seinfo, although would could replace these with python
tools using the seinfo and search python bindings.

Over the summer we beginning to build a new gui based on the sesearch and
seinfo python bindings.  along with a lot of the work we have done in sepolicy.

Our first goal is to reveal all of the infomation that we currently have in
the SELinux Policy Man pages in an active presentation.   The idea is to allow
an administrator to "browse" all of the policy that effects a particular
executable.  For example the admin selects httpd and sees tabs for all of the
booleans, network ports, entry point paths, file types, places apache can
write, applications that apache can transition too.  Not just the types but
also the actual values.

# sepolicy network -d httpd_t
httpd_t: tcp name_connect
	dns_port_t: 53
	http_port_t: 80,81,443,488,8008,8009,8443,9000
	ocsp_port_t: 9080
	kerberos_port_t: 88,750,4444
	pop_port_t: 106,109,110,143,220,993,995,1109
	smtp_port_t: 25,465,587
httpd_t: tcp name_bind
	ntop_port_t: 3000-3001
	http_cache_port_t: 8080,8118,8123,10001-10010
	http_port_t: 80,81,443,488,8008,8009,8443,9000
	puppet_port_t: 8140
	jboss_messaging_port_t: 5445,5455
	jboss_management_port_t: 4712,4447,7600,9123,9990,9999,18001
httpd_t: udp name_bind

# sepolicy transition -s httpd_t | head
httpd_t @ httpd_suexec_exec_t --> httpd_suexec_t
httpd_t @ mailman_cgi_exec_t --> mailman_cgi_t
httpd_t @ abrt_retrace_worker_exec_t --> abrt_retrace_worker_t
httpd_t @ dirsrvadmin_unconfined_script_exec_t --> dirsrvadmin_unconfined_script_t
httpd_t @ nagios_services_plugin_exec_t --> nagios_services_plugin_t
httpd_t @ httpd_rotatelogs_exec_t --> httpd_rotatelogs_t
httpd_t @ pwauth_exec_t --> pwauth_t
httpd_t @ abrt_helper_exec_t --> abrt_helper_t
httpd_t @ nagios_system_plugin_exec_t --> nagios_system_plugin_t
httpd_t @ sepgsql_trusted_proc_exec_t --> sepgsql_trusted_proc_t

Then the next step would be to allow users, to customize the policy by turning
on booleans or changing network ports or adding file context.

libapol and libqpol become critical to getting to this point.


In fedora and RHEL7 we are dropping support for a few of the executables that
we do not want to support.  Also apps that have more traditional ways of
discovering the data.

rpm -qla setools\* | grep bin
/usr/bin/apol
/usr/bin/seaudit
/usr/sbin/seaudit
/usr/bin/sediff
/usr/bin/seinfo
/usr/bin/sesearch

> However, we are well aware of the complexity of SETools, primarily libapol,
> and that upstreaming it without any changes would not solve the problems.
> So, we have done a little work behind the scenes to find a way to reduce
> the complexity of libapol. As a first stab at it, we started with an older
> version of libapol that is quite a bit less complex and began porting it
> forward for use with modern userspace, and seeing if it would make sense to
> eventually merge. But before we get too deep into this port, we wanted to
> start a discussion with the SELinux community to make sure we are headed in
> the right direction. So to start, does this seem like a good idea (both
> merging with userspace and porting older libapol)? Or should we take a
> completely different direction (e.g. the use of graphing databases as a
> replacement of apol has been mentioned in the past)?
> 
> Another discussion we would like to have, which may affect the future of 
> SETools/apol, is CIL. Is there still interest in CIL? And if so, have there
> been any thoughts on using and migrating to CIL? Is more work needed before
> this can happen? Has anyone put thought into higher level languages that
> could sit on top of CIL? If there is interest, this may affect the SETools
> changes, for example, syntactic policy analysis for CIL is likely very
> different than current policy.
> 
As far as CIL is concerned, we love the idea, and would love to use it, but we
need to get it as a replacement for current policy with limited work.

> Thanks, - Steve
> 
> 
> -- This message was distributed to subscribers of the selinux mailing
> list. If you no longer wish to subscribe, send mail to
> majordomo@tycho.nsa.gov with the words "unsubscribe selinux" without quotes
> as the message.
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlGU6M0ACgkQrlYvE4MpobM2xwCfXG/YGDbDuR+mOpfc3qyByGVv
+hIAoMCLJu27AhJ5NG7WsaT0c+Rk+FH2
=Unbw
-----END PGP SIGNATURE-----

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[Steve Lawrence]

On 05/16/2013 10:10 AM, Daniel J Walsh wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 05/16/2013 09:33 AM, Steve Lawrence wrote:
>> It has become clear that SETools has fallen behind userspace in terms of
>> features and general maintenance. We would like to get it to the point
>> where this is not the case, and to find a way to make sure it does not
>> happen again. We think the solution to the maintenance issue is to make it
>> more visible by merging the more useful parts of SETools into the userspace
>> repo, while deprecating/removing the remaining pieces.
>>
> I think this would be a good idea.   I have been adding sepolicy which uses
> libapol and libqpol, to gather data from the installed policy.  We have
> several patches for setools that never made it upstream.  We also are heavy
> users of sesearch and seinfo, although would could replace these with python
> tools using the seinfo and search python bindings.
>
> Over the summer we beginning to build a new gui based on the sesearch and
> seinfo python bindings.  along with a lot of the work we have done in sepolicy.
>

Is this code anywhere. We'd love to take a look at it.

Also, it sounds like reverting to an older verstion of libapol might 
break more things than we originally anticipated, so that might not be 
the best idea. Perhaps merging the current libapol into userspace and 
gradually working to reduce the complexity is the better route.

> Our first goal is to reveal all of the infomation that we currently have in
> the SELinux Policy Man pages in an active presentation.   The idea is to allow
> an administrator to "browse" all of the policy that effects a particular
> executable.  For example the admin selects httpd and sees tabs for all of the
> booleans, network ports, entry point paths, file types, places apache can
> write, applications that apache can transition too.  Not just the types but
> also the actual values.
>
> # sepolicy network -d httpd_t
> httpd_t: tcp name_connect
> 	dns_port_t: 53
> 	http_port_t: 80,81,443,488,8008,8009,8443,9000
> 	ocsp_port_t: 9080
> 	kerberos_port_t: 88,750,4444
> 	pop_port_t: 106,109,110,143,220,993,995,1109
> 	smtp_port_t: 25,465,587
> httpd_t: tcp name_bind
> 	ntop_port_t: 3000-3001
> 	http_cache_port_t: 8080,8118,8123,10001-10010
> 	http_port_t: 80,81,443,488,8008,8009,8443,9000
> 	puppet_port_t: 8140
> 	jboss_messaging_port_t: 5445,5455
> 	jboss_management_port_t: 4712,4447,7600,9123,9990,9999,18001
> httpd_t: udp name_bind
>
> # sepolicy transition -s httpd_t | head
> httpd_t @ httpd_suexec_exec_t --> httpd_suexec_t
> httpd_t @ mailman_cgi_exec_t --> mailman_cgi_t
> httpd_t @ abrt_retrace_worker_exec_t --> abrt_retrace_worker_t
> httpd_t @ dirsrvadmin_unconfined_script_exec_t --> dirsrvadmin_unconfined_script_t
> httpd_t @ nagios_services_plugin_exec_t --> nagios_services_plugin_t
> httpd_t @ httpd_rotatelogs_exec_t --> httpd_rotatelogs_t
> httpd_t @ pwauth_exec_t --> pwauth_t
> httpd_t @ abrt_helper_exec_t --> abrt_helper_t
> httpd_t @ nagios_system_plugin_exec_t --> nagios_system_plugin_t
> httpd_t @ sepgsql_trusted_proc_exec_t --> sepgsql_trusted_proc_t
>
> Then the next step would be to allow users, to customize the policy by turning
> on booleans or changing network ports or adding file context.
>
> libapol and libqpol become critical to getting to this point.
>
>
> In fedora and RHEL7 we are dropping support for a few of the executables that
> we do not want to support.  Also apps that have more traditional ways of
> discovering the data.
>
> rpm -qla setools\* | grep bin
> /usr/bin/apol
> /usr/bin/seaudit
> /usr/sbin/seaudit
> /usr/bin/sediff
> /usr/bin/seinfo
> /usr/bin/sesearch
>
>> However, we are well aware of the complexity of SETools, primarily libapol,
>> and that upstreaming it without any changes would not solve the problems.
>> So, we have done a little work behind the scenes to find a way to reduce
>> the complexity of libapol. As a first stab at it, we started with an older
>> version of libapol that is quite a bit less complex and began porting it
>> forward for use with modern userspace, and seeing if it would make sense to
>> eventually merge. But before we get too deep into this port, we wanted to
>> start a discussion with the SELinux community to make sure we are headed in
>> the right direction. So to start, does this seem like a good idea (both
>> merging with userspace and porting older libapol)? Or should we take a
>> completely different direction (e.g. the use of graphing databases as a
>> replacement of apol has been mentioned in the past)?
>>
>> Another discussion we would like to have, which may affect the future of
>> SETools/apol, is CIL. Is there still interest in CIL? And if so, have there
>> been any thoughts on using and migrating to CIL? Is more work needed before
>> this can happen? Has anyone put thought into higher level languages that
>> could sit on top of CIL? If there is interest, this may affect the SETools
>> changes, for example, syntactic policy analysis for CIL is likely very
>> different than current policy.
>>
> As far as CIL is concerned, we love the idea, and would love to use it, but we
> need to get it as a replacement for current policy with limited work.
>

Good to hear. We'll keep that in mind.

>> Thanks, - Steve
>>
>>
>> -- This message was distributed to subscribers of the selinux mailing
>> list. If you no longer wish to subscribe, send mail to
>> majordomo@tycho.nsa.gov with the words "unsubscribe selinux" without quotes
>> as the message.
>>
>>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.13 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlGU6M0ACgkQrlYvE4MpobM2xwCfXG/YGDbDuR+mOpfc3qyByGVv
> +hIAoMCLJu27AhJ5NG7WsaT0c+Rk+FH2
> =Unbw
> -----END PGP SIGNATURE-----
>


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[Daniel J Walsh]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/23/2013 02:32 PM, Steve Lawrence wrote:
> On 05/16/2013 10:10 AM, Daniel J Walsh wrote: On 05/16/2013 09:33 AM, Steve
> Lawrence wrote:
>>>> It has become clear that SETools has fallen behind userspace in terms
>>>> of features and general maintenance. We would like to get it to the
>>>> point where this is not the case, and to find a way to make sure it
>>>> does not happen again. We think the solution to the maintenance issue
>>>> is to make it more visible by merging the more useful parts of
>>>> SETools into the userspace repo, while deprecating/removing the
>>>> remaining pieces.
>>>> 
> I think this would be a good idea.   I have been adding sepolicy which
> uses libapol and libqpol, to gather data from the installed policy.  We
> have several patches for setools that never made it upstream.  We also are
> heavy users of sesearch and seinfo, although would could replace these with
> python tools using the seinfo and search python bindings.
> 
> Over the summer we beginning to build a new gui based on the sesearch and 
> seinfo python bindings.  along with a lot of the work we have done in
> sepolicy.
> 
> 
>> Is this code anywhere. We'd love to take a look at it.
> 
>> Also, it sounds like reverting to an older verstion of libapol might
>> break more things than we originally anticipated, so that might not be
>> the best idea. Perhaps merging the current libapol into userspace and
>> gradually working to reduce the complexity is the better route.
> 
> Our first goal is to reveal all of the infomation that we currently have
> in the SELinux Policy Man pages in an active presentation.   The idea is to
> allow an administrator to "browse" all of the policy that effects a
> particular executable.  For example the admin selects httpd and sees tabs
> for all of the booleans, network ports, entry point paths, file types,
> places apache can write, applications that apache can transition too.  Not
> just the types but also the actual values.
> 
> # sepolicy network -d httpd_t httpd_t: tcp name_connect dns_port_t: 53 
> http_port_t: 80,81,443,488,8008,8009,8443,9000 ocsp_port_t: 9080 
> kerberos_port_t: 88,750,4444 pop_port_t: 106,109,110,143,220,993,995,1109 
> smtp_port_t: 25,465,587 httpd_t: tcp name_bind ntop_port_t: 3000-3001 
> http_cache_port_t: 8080,8118,8123,10001-10010 http_port_t:
> 80,81,443,488,8008,8009,8443,9000 puppet_port_t: 8140 
> jboss_messaging_port_t: 5445,5455 jboss_management_port_t:
> 4712,4447,7600,9123,9990,9999,18001 httpd_t: udp name_bind
> 
> # sepolicy transition -s httpd_t | head httpd_t @ httpd_suexec_exec_t -->
> httpd_suexec_t httpd_t @ mailman_cgi_exec_t --> mailman_cgi_t httpd_t @
> abrt_retrace_worker_exec_t --> abrt_retrace_worker_t httpd_t @
> dirsrvadmin_unconfined_script_exec_t --> dirsrvadmin_unconfined_script_t 
> httpd_t @ nagios_services_plugin_exec_t --> nagios_services_plugin_t 
> httpd_t @ httpd_rotatelogs_exec_t --> httpd_rotatelogs_t httpd_t @
> pwauth_exec_t --> pwauth_t httpd_t @ abrt_helper_exec_t --> abrt_helper_t 
> httpd_t @ nagios_system_plugin_exec_t --> nagios_system_plugin_t httpd_t @
> sepgsql_trusted_proc_exec_t --> sepgsql_trusted_proc_t
> 
> Then the next step would be to allow users, to customize the policy by
> turning on booleans or changing network ports or adding file context.
> 
> libapol and libqpol become critical to getting to this point.
> 
> 
> In fedora and RHEL7 we are dropping support for a few of the executables
> that we do not want to support.  Also apps that have more traditional ways
> of discovering the data.
> 
> rpm -qla setools\* | grep bin /usr/bin/apol /usr/bin/seaudit 
> /usr/sbin/seaudit /usr/bin/sediff /usr/bin/seinfo /usr/bin/sesearch
> 
>>>> However, we are well aware of the complexity of SETools, primarily
>>>> libapol, and that upstreaming it without any changes would not solve
>>>> the problems. So, we have done a little work behind the scenes to
>>>> find a way to reduce the complexity of libapol. As a first stab at
>>>> it, we started with an older version of libapol that is quite a bit
>>>> less complex and began porting it forward for use with modern
>>>> userspace, and seeing if it would make sense to eventually merge. But
>>>> before we get too deep into this port, we wanted to start a
>>>> discussion with the SELinux community to make sure we are headed in 
>>>> the right direction. So to start, does this seem like a good idea
>>>> (both merging with userspace and porting older libapol)? Or should we
>>>> take a completely different direction (e.g. the use of graphing
>>>> databases as a replacement of apol has been mentioned in the past)?
>>>> 
>>>> Another discussion we would like to have, which may affect the future
>>>> of SETools/apol, is CIL. Is there still interest in CIL? And if so,
>>>> have there been any thoughts on using and migrating to CIL? Is more
>>>> work needed before this can happen? Has anyone put thought into
>>>> higher level languages that could sit on top of CIL? If there is
>>>> interest, this may affect the SETools changes, for example, syntactic
>>>> policy analysis for CIL is likely very different than current
>>>> policy.
>>>> 
> As far as CIL is concerned, we love the idea, and would love to use it, but
> we need to get it as a replacement for current policy with limited work.
> 
> 
>> Good to hear. We'll keep that in mind.
> 
>>>> Thanks, - Steve
>>>> 
>>>> 
>>>> -- This message was distributed to subscribers of the selinux
>>>> mailing list. If you no longer wish to subscribe, send mail to 
>>>> majordomo@tycho.nsa.gov with the words "unsubscribe selinux" without
>>>> quotes as the message.
>>>> 
>>>> 
> 
>> 
> 

Most of it is upstream now

policycoreutils/sepolicy/*.c in the upstream have the bindings.  Basically is
just taking sesearch and seinfo and making the data returned by those modules
into python dictionaries.

Then you can do stuff like

python
>>> import sepolicy sepolicy.get_all_domains() print
>>> sepolicy.get_all_domains()
['sosreport_t', 'git_session_t', 'sshd_sandbox_t', 'cfengine_execd_t',
'bootloader_t', 'netutils_t', 'qmail_tcp_env_t', 'devicekit_power_t',
'zoneminder_t', 'httpd_collectd_script_t', 'sandbox_x_client_t', 'nova_api_t',
'sblim_reposd_t', 'dkim_milter_t', 'admin_crontab_t', 'consolekit_t',
'nova_compute_t', 'nova_console_t', 'pam_console_t', 'zarafa_gateway_t',
'policykit_grant_t', 'logrotate_t', 'openvswitch_t', 'update_modules_t',
'ssh_keysign_t', 'nova_network_t', 'qmail_rspawn_t', 'uml_switch_t',
...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlGfaZ4ACgkQrlYvE4MpobNWRQCeO8NLwMRBz+XCptQNf4ruUXff
CqgAoK36YGZ5UGDNQ4dSPTgbBcKdLPWT
=sfuv
-----END PGP SIGNATURE-----

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[James Carter]

On 05/16/2013 09:33 AM, Steve Lawrence wrote:
> Another discussion we would like to have, which may affect the future of
> SETools/apol, is CIL. Is there still interest in CIL? And if so, have
> there been any thoughts on using and migrating to CIL? Is more work
> needed before this can happen? Has anyone put thought into higher level
> languages that could sit on top of CIL? If there is interest, this may
> affect the SETools changes, for example, syntactic policy analysis for
> CIL is likely very different than current policy.

I am still interested in CIL. In fact, I just got CIL to work on a 
translation of Refpolicy from early 2012. (And by work I mean produce a 
binary policy equivalent, according to sediff, with the binary produced 
by the Refpolicy build.) I just started this week on trying it against a 
recent version of Refpolicy. There are some issues that I need to work 
through; the biggest being how to handle the optional parameters to 
filetrans_pattern() and filetrans_add_pattern(). I hope to make both the 
CIL translation of Refpolicy and my many modifications to CIL available 
shortly.

I am also interested in resurrecting the earlier policy toolchain work 
to convert to the use of source modules and allow the use of CIL for 
policy builds.

-- 
James Carter <jwcart2@tycho.nsa.gov>
National Security Agency

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[Steve Lawrence]

On 05/16/2013 10:46 AM, James Carter wrote:
> On 05/16/2013 09:33 AM, Steve Lawrence wrote:
>> Another discussion we would like to have, which may affect the future of
>> SETools/apol, is CIL. Is there still interest in CIL? And if so, have
>> there been any thoughts on using and migrating to CIL? Is more work
>> needed before this can happen? Has anyone put thought into higher level
>> languages that could sit on top of CIL? If there is interest, this may
>> affect the SETools changes, for example, syntactic policy analysis for
>> CIL is likely very different than current policy.
>
> I am still interested in CIL. In fact, I just got CIL to work on a
> translation of Refpolicy from early 2012. (And by work I mean produce a
> binary policy equivalent, according to sediff, with the binary produced
> by the Refpolicy build.) I just started this week on trying it against a
> recent version of Refpolicy. There are some issues that I need to work
> through; the biggest being how to handle the optional parameters to
> filetrans_pattern() and filetrans_add_pattern(). I hope to make both the
> CIL translation of Refpolicy and my many modifications to CIL available
> shortly.
>
> I am also interested in resurrecting the earlier policy toolchain work
> to convert to the use of source modules and allow the use of CIL for
> policy builds.
>

That's great to hear! Did this require any patches to CIL at all? I'd be 
happy to review any changes.

Regarding the policy toolchain, I just tried to rebase to previous 
policy toolchain work/CIL integration and, not surprisingly, it ran into 
conflict issues on the very first patch. So it's probably not trivial, 
but I imagine it's not too difficult either.

- Steve

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[James Carter]

On 05/23/2013 02:43 PM, Steve Lawrence wrote:
> On 05/16/2013 10:46 AM, James Carter wrote:
>> On 05/16/2013 09:33 AM, Steve Lawrence wrote:
>>> Another discussion we would like to have, which may affect the future of
>>> SETools/apol, is CIL. Is there still interest in CIL? And if so, have
>>> there been any thoughts on using and migrating to CIL? Is more work
>>> needed before this can happen? Has anyone put thought into higher level
>>> languages that could sit on top of CIL? If there is interest, this may
>>> affect the SETools changes, for example, syntactic policy analysis for
>>> CIL is likely very different than current policy.
>>
>> I am still interested in CIL. In fact, I just got CIL to work on a
>> translation of Refpolicy from early 2012. (And by work I mean produce a
>> binary policy equivalent, according to sediff, with the binary produced
>> by the Refpolicy build.) I just started this week on trying it against a
>> recent version of Refpolicy. There are some issues that I need to work
>> through; the biggest being how to handle the optional parameters to
>> filetrans_pattern() and filetrans_add_pattern(). I hope to make both the
>> CIL translation of Refpolicy and my many modifications to CIL available
>> shortly.
>>
>> I am also interested in resurrecting the earlier policy toolchain work
>> to convert to the use of source modules and allow the use of CIL for
>> policy builds.
>>
>
> That's great to hear! Did this require any patches to CIL at all? I'd be happy
> to review any changes.
>

A few changes were made. ;)

  b/Makefile              |   38
  b/include/cil/cil.h     |    1
  b/secilc.c              |   51 -
  b/src/cil.c             |  237 ++---
  b/src/cil_binary.c      | 1144 ++++++++++++++--------------
  b/src/cil_binary.h      |    4
  b/src/cil_build_ast.c   | 1717 +++++++++++++++++-------------------------
  b/src/cil_build_ast.h   |   26
  b/src/cil_copy_ast.c    | 1265 +++++++-----------------------
  b/src/cil_copy_ast.h    |   18
  b/src/cil_fqn.c         |    9
  b/src/cil_internal.h    |  136 +--
  b/src/cil_list.c        |  157 +--
  b/src/cil_list.h        |   25
  b/src/cil_mem.c         |   33
  b/src/cil_mem.h         |    4
  b/src/cil_parser.c      |    6
  b/src/cil_policy.c      |  610 ++++++++------
  b/src/cil_post.c        |  639 +++++++++++----
  b/src/cil_resolve_ast.c |  889 ++++++++-------------
  b/src/cil_resolve_ast.h |    6
  b/src/cil_symtab.c      |  140 +--
  b/src/cil_symtab.h      |    9
  b/src/cil_tree.c        | 1959 +++++++++++++++++++++++++-----------------------
  b/src/cil_verify.c      |  847 ++++++++------------
  b/src/cil_verify.h      |    5
  b/test/policy.cil       |    2
  src/Makefile            |   57 -
  28 files changed, 4591 insertions(+), 5443 deletions(-)

I will gladly send you patches if you want them, but I have 45 commits in my tree.

Many of these commits are small. And many of the ones that are large are just 
applying a similar change in many places. One of the big ones is just to improve 
debug messages. Another big one is just a refactoring of cil_list handling. 
Another very large one is the removal of cil_conditional.

A surprisingly large number of the commits went towards adding support for 
expression operations in classpermsets and classmappings. I haven't been able to 
test CIL with actual permission expressions yet, because I had been expanding 
them all in my parser when converting Refpolicy to CIL policy. I don't expect 
any major problems because permission expressions are handled with much of the 
same code that attributes and other expressions are.

I am currently updating my parser to work with the latest Refpolicy and testing 
should begin shortly. Hopefully, I will soon be able to report success.

> Regarding the policy toolchain, I just tried to rebase to previous policy
> toolchain work/CIL integration and, not surprisingly, it ran into conflict
> issues on the very first patch. So it's probably not trivial, but I imagine it's
> not too difficult either.

That is my thought as well.


-- 
James Carter <jwcart2@tycho.nsa.gov>
National Security Agency

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[Steve Lawrence]

On 05/23/2013 03:43 PM, James Carter wrote:
> On 05/23/2013 02:43 PM, Steve Lawrence wrote:
>> On 05/16/2013 10:46 AM, James Carter wrote:
>>> On 05/16/2013 09:33 AM, Steve Lawrence wrote:
>>>> Another discussion we would like to have, which may affect the
>>>> future of
>>>> SETools/apol, is CIL. Is there still interest in CIL? And if so, have
>>>> there been any thoughts on using and migrating to CIL? Is more work
>>>> needed before this can happen? Has anyone put thought into higher level
>>>> languages that could sit on top of CIL? If there is interest, this may
>>>> affect the SETools changes, for example, syntactic policy analysis for
>>>> CIL is likely very different than current policy.
>>>
>>> I am still interested in CIL. In fact, I just got CIL to work on a
>>> translation of Refpolicy from early 2012. (And by work I mean produce a
>>> binary policy equivalent, according to sediff, with the binary produced
>>> by the Refpolicy build.) I just started this week on trying it against a
>>> recent version of Refpolicy. There are some issues that I need to work
>>> through; the biggest being how to handle the optional parameters to
>>> filetrans_pattern() and filetrans_add_pattern(). I hope to make both the
>>> CIL translation of Refpolicy and my many modifications to CIL available
>>> shortly.
>>>
>>> I am also interested in resurrecting the earlier policy toolchain work
>>> to convert to the use of source modules and allow the use of CIL for
>>> policy builds.
>>>
>>
>> That's great to hear! Did this require any patches to CIL at all? I'd
>> be happy
>> to review any changes.
>>
>
> A few changes were made. ;)
>
>   b/Makefile              |   38
>   b/include/cil/cil.h     |    1
>   b/secilc.c              |   51 -
>   b/src/cil.c             |  237 ++---
>   b/src/cil_binary.c      | 1144 ++++++++++++++--------------
>   b/src/cil_binary.h      |    4
>   b/src/cil_build_ast.c   | 1717 +++++++++++++++++-------------------------
>   b/src/cil_build_ast.h   |   26
>   b/src/cil_copy_ast.c    | 1265 +++++++-----------------------
>   b/src/cil_copy_ast.h    |   18
>   b/src/cil_fqn.c         |    9
>   b/src/cil_internal.h    |  136 +--
>   b/src/cil_list.c        |  157 +--
>   b/src/cil_list.h        |   25
>   b/src/cil_mem.c         |   33
>   b/src/cil_mem.h         |    4
>   b/src/cil_parser.c      |    6
>   b/src/cil_policy.c      |  610 ++++++++------
>   b/src/cil_post.c        |  639 +++++++++++----
>   b/src/cil_resolve_ast.c |  889 ++++++++-------------
>   b/src/cil_resolve_ast.h |    6
>   b/src/cil_symtab.c      |  140 +--
>   b/src/cil_symtab.h      |    9
>   b/src/cil_tree.c        | 1959
> +++++++++++++++++++++++++-----------------------
>   b/src/cil_verify.c      |  847 ++++++++------------
>   b/src/cil_verify.h      |    5
>   b/test/policy.cil       |    2
>   src/Makefile            |   57 -
>   28 files changed, 4591 insertions(+), 5443 deletions(-)
>
> I will gladly send you patches if you want them, but I have 45 commits
> in my tree.
>

Nice! That would probably be a little difficult to deal with over email. 
I've given you commit access the CIL repo on oss.tresys.com. If you'd 
like, feel free to push your changes to a branch and I can give them a 
quick review before they go to master.

> Many of these commits are small. And many of the ones that are large are
> just applying a similar change in many places. One of the big ones is
> just to improve debug messages. Another big one is just a refactoring of
> cil_list handling. Another very large one is the removal of
> cil_conditional.
>
> A surprisingly large number of the commits went towards adding support
> for expression operations in classpermsets and classmappings. I haven't
> been able to test CIL with actual permission expressions yet, because I
> had been expanding them all in my parser when converting Refpolicy to
> CIL policy. I don't expect any major problems because permission
> expressions are handled with much of the same code that attributes and
> other expressions are.
>
> I am currently updating my parser to work with the latest Refpolicy and
> testing should begin shortly. Hopefully, I will soon be able to report
> success.
>
>> Regarding the policy toolchain, I just tried to rebase to previous policy
>> toolchain work/CIL integration and, not surprisingly, it ran into
>> conflict
>> issues on the very first patch. So it's probably not trivial, but I
>> imagine it's
>> not too difficult either.
>
> That is my thought as well.
>
>


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[Stephen Smalley]

On 05/16/2013 09:33 AM, Steve Lawrence wrote:
> It has become clear that SETools has fallen behind userspace in terms of
> features and general maintenance. We would like to get it to the point
> where this is not the case, and to find a way to make sure it does not
> happen again. We think the solution to the maintenance issue is to make
> it more visible by merging the more useful parts of SETools into the
> userspace repo, while deprecating/removing the remaining pieces.
>
> However, we are well aware of the complexity of SETools, primarily
> libapol, and that upstreaming it without any changes would not solve the
> problems. So, we have done a little work behind the scenes to find a way
> to reduce the complexity of libapol. As a first stab at it, we started
> with an older version of libapol that is quite a bit less complex and
> began porting it forward for use with modern userspace, and seeing if it
> would make sense to eventually merge. But before we get too deep into
> this port, we wanted to start a discussion with the SELinux community to
> make sure we are headed in the right direction. So to start, does this
> seem like a good idea (both merging with userspace and porting older
> libapol)? Or should we take a completely different direction (e.g. the
> use of graphing databases as a replacement of apol has been mentioned in
> the past)?

What is it that makes the modern version of libapol more complex, or 
what are you giving up by going back to the older version?



--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[Steve Lawrence]

On 05/16/2013 12:37 PM, Stephen Smalley wrote:
> On 05/16/2013 09:33 AM, Steve Lawrence wrote:
>> It has become clear that SETools has fallen behind userspace in terms of
>> features and general maintenance. We would like to get it to the point
>> where this is not the case, and to find a way to make sure it does not
>> happen again. We think the solution to the maintenance issue is to make
>> it more visible by merging the more useful parts of SETools into the
>> userspace repo, while deprecating/removing the remaining pieces.
>>
>> However, we are well aware of the complexity of SETools, primarily
>> libapol, and that upstreaming it without any changes would not solve the
>> problems. So, we have done a little work behind the scenes to find a way
>> to reduce the complexity of libapol. As a first stab at it, we started
>> with an older version of libapol that is quite a bit less complex and
>> began porting it forward for use with modern userspace, and seeing if it
>> would make sense to eventually merge. But before we get too deep into
>> this port, we wanted to start a discussion with the SELinux community to
>> make sure we are headed in the right direction. So to start, does this
>> seem like a good idea (both merging with userspace and porting older
>> libapol)? Or should we take a completely different direction (e.g. the
>> use of graphing databases as a replacement of apol has been mentioned in
>> the past)?
>
> What is it that makes the modern version of libapol more complex, or
> what are you giving up by going back to the older version?
>

Sorry it took so long to get back to you. The current version of libapol 
heavily relies on object oriented techniques, but is entirely written in 
C. For example, it uses iterators and a lot of function pointers to use 
something akin to inheritance. This makes it very awkward to program in 
and adds an unnecessary level of complexity.

Going back to an older version would lose features and support for 
policy versions later than 20. The idea was that we would go back to a 
simpler point and rebuild or pull in the necessary features and newer 
policy version support. A because of the much simpler code base of an 
older version, adding back the features would hopefully not be too 
difficult.

However, perhaps reducing the complexity isn't worth the effort. And 
maybe the dependence of tools in userspace would add more work. It 
sounds like maybe the best route might be to merge in libapol in it's 
current state. And as updates need to be made for Dan's and other tools 
that use libapol, we can gradually reduce the complexity without 
breaking everything?

- Steve

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.

Re: Future of SETools and CIL

[Sven Vermeulen]

On Thu, May 16, 2013 at 09:33:19AM -0400, Steve Lawrence wrote:
> It has become clear that SETools has fallen behind userspace in terms of
> features and general maintenance. We would like to get it to the point
> where this is not the case, and to find a way to make sure it does not
> happen again. We think the solution to the maintenance issue is to make
> it more visible by merging the more useful parts of SETools into the
> userspace repo, while deprecating/removing the remaining pieces.
> 
> However, we are well aware of the complexity of SETools, primarily
> libapol, and that upstreaming it without any changes would not solve the
> problems.

Only SETools complex? I find the combination of all libraries and tools
(libselinux, libsemanage, libsepol, sepolgen, policycoreutils, setools,
checkpolicy) not always that obvious to manage. And many of them then try to
have their interfaces supported in other languages (like python or ruby) and use
swig to generate these bindings, which makes it - imo - even less obvious
for troubleshooting. And then you have the complexity of python-2 versus
python-3 that still needs to be sorted out :-(

I'm in favor of all efforts to get the userspace and libraries things well
documented and maintained and using a simple codebase...

> So, we have done a little work behind the scenes to find a way
> to reduce the complexity of libapol. As a first stab at it, we started
> with an older version of libapol that is quite a bit less complex and
> began porting it forward for use with modern userspace, and seeing if it
> would make sense to eventually merge. 

Please take care not to depend on unknown patches only available in fedora.

> Another discussion we would like to have, which may affect the future of
> SETools/apol, is CIL. Is there still interest in CIL? And if so, have
> there been any thoughts on using and migrating to CIL? Is more work
> needed before this can happen? Has anyone put thought into higher level
> languages that could sit on top of CIL? If there is interest, this may
> affect the SETools changes, for example, syntactic policy analysis for
> CIL is likely very different than current policy.

I have been hearing of CIL for a few years, never tried it out. For me, as
one of the SELinux guys for Gentoo Hardened, CIL only comes to play when the
reference policy upon which we base ourselves is available in CIL and/or the
updates of the reference policy are easily migrated to CIL as well.

Wkr,
	Sven Vermeulen

--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to majordomo@tycho.nsa.gov with
the words "unsubscribe selinux" without quotes as the message.