Making a universal list of syscalls?

Making a universal list of syscalls?

[Andy Lutomirski]

Currently, dealing with Linux syscalls in an architecture-independent
way is a mess.  Here are some issues:

 1. There's no clean way to map between syscall names and numbers on
different architectures.  The kernel contains a number of tables (that
work differently for different architectures).  strace has some arcane
mechanism.  libseccomp has another.

 2. There's no clean way to map between syscall argument registers and
logical syscall arguments.  Each architecture knows how to do it, as
do strace and glibc, but I suspect that *everyone* else gets it wrong.
 Especially on ARM.

 3. Determining which architectures have which syscalls is a mess.
Recent kernel builds love to warn me that finit_module is missing on
x86_64.  This is simply not true.  I have no idea why.

 4. Actually issuing a nontrivial syscall is annoying.  syscall(2) can
do it for the native architecture (only).

 5. Decoding ucontext from SIGSYS is a mess.  I have prototype code
for libseccomp that can do it, but it gets the arguments wrong due to
ABI issues.  See (2).

I'd like to see a master list in the kernel that lists, for every
syscall, the name, the number for each architecture that implements it
(using the AUDIT_ARCH semantics, probably), and the signature.  The
build process could parse this table to replace the current per-arch
mess.

Issues here: some syscalls have different signatures on different
architectures.  Maybe we could require that a canonical syscall name
would have the same signature everywhere, but architectures could
specify alternate names.  So, for things like clone (?), there could
actually be a few syscalls that all have alternate names of "clone".

More importantly, we could add a library in tools that exposes this
information to userspace.  Useful operations:

 - For a given (arch, nr), indicate, for each logical argument, which
physical argument slot is used or, if the argument is split into a
high and low part, which pair of slots is used.

 - For a given (nr, logical args), issue the syscall for the
architecture that build the library.

 - For a given (arch, nr, logical args), issue the syscall if
possible.  An x86_32 build could issue x86_64 syscalls with some
effort, and an x86_64 build could easily issue 32-bit syscalls.

 - For a given arch, map between name and nr, and give access to the signature.

If this happened, presumably all architectures that supported it would
have to have valid AUDIT_ARCH support.  That means that someone would
have to fix ARM OABI (sigh).

Thoughts?


--Andy

Re: [libseccomp-discuss] Making a universal list of syscalls?

[Eric Paris]

On Thu, 2014-02-27 at 12:40 -0800, Andy Lutomirski wrote:
> Currently, dealing with Linux syscalls in an architecture-independent
> way is a mess.  Here are some issues:
> 
>  1. There's no clean way to map between syscall names and numbers on
> different architectures.  The kernel contains a number of tables (that
> work differently for different architectures).  strace has some arcane
> mechanism.  libseccomp has another.

userspace audit a 3rd.

> I'd like to see a master list in the kernel that lists, for every
> syscall, the name, the number for each architecture that implements it
> (using the AUDIT_ARCH semantics, probably), and the signature.  The
> build process could parse this table to replace the current per-arch
> mess.

I know for audit it would be huge if userspace didn't try to organically
grow this knowledge on their own!  So +1 from me!

> 
> Issues here: some syscalls have different signatures on different
> architectures.  Maybe we could require that a canonical syscall name
> would have the same signature everywhere, but architectures could
> specify alternate names.  So, for things like clone (?), there could
> actually be a few syscalls that all have alternate names of "clone".
> 
> More importantly, we could add a library in tools that exposes this
> information to userspace.  Useful operations:
> 
>  - For a given (arch, nr), indicate, for each logical argument, which
> physical argument slot is used or, if the argument is split into a
> high and low part, which pair of slots is used.
> 
>  - For a given (nr, logical args), issue the syscall for the
> architecture that build the library.
> 
>  - For a given (arch, nr, logical args), issue the syscall if
> possible.  An x86_32 build could issue x86_64 syscalls with some
> effort, and an x86_64 build could easily issue 32-bit syscalls.
> 
>  - For a given arch, map between name and nr, and give access to the signature.
> 
> If this happened, presumably all architectures that supported it would
> have to have valid AUDIT_ARCH support.  That means that someone would
> have to fix ARM OABI (sigh).
> 
> Thoughts?

Re: Making a universal list of syscalls?

[Eric Paris]

On Thu, 2014-02-27 at 12:40 -0800, Andy Lutomirski wrote:
> Currently, dealing with Linux syscalls in an architecture-independent
> way is a mess.  Here are some issues:
> 
>  1. There's no clean way to map between syscall names and numbers on
> different architectures.  The kernel contains a number of tables (that
> work differently for different architectures).  strace has some arcane
> mechanism.  libseccomp has another.

userspace audit a 3rd.

> I'd like to see a master list in the kernel that lists, for every
> syscall, the name, the number for each architecture that implements it
> (using the AUDIT_ARCH semantics, probably), and the signature.  The
> build process could parse this table to replace the current per-arch
> mess.

I know for audit it would be huge if userspace didn't try to organically
grow this knowledge on their own!  So +1 from me!

> 
> Issues here: some syscalls have different signatures on different
> architectures.  Maybe we could require that a canonical syscall name
> would have the same signature everywhere, but architectures could
> specify alternate names.  So, for things like clone (?), there could
> actually be a few syscalls that all have alternate names of "clone".
> 
> More importantly, we could add a library in tools that exposes this
> information to userspace.  Useful operations:
> 
>  - For a given (arch, nr), indicate, for each logical argument, which
> physical argument slot is used or, if the argument is split into a
> high and low part, which pair of slots is used.
> 
>  - For a given (nr, logical args), issue the syscall for the
> architecture that build the library.
> 
>  - For a given (arch, nr, logical args), issue the syscall if
> possible.  An x86_32 build could issue x86_64 syscalls with some
> effort, and an x86_64 build could easily issue 32-bit syscalls.
> 
>  - For a given arch, map between name and nr, and give access to the signature.
> 
> If this happened, presumably all architectures that supported it would
> have to have valid AUDIT_ARCH support.  That means that someone would
> have to fix ARM OABI (sigh).
> 
> Thoughts?


------------------------------------------------------------------------------
Flow-based real-time traffic analytics software. Cisco certified tool.
Monitor traffic, SLAs, QoS, Medianet, WAAS etc. with NetFlow Analyzer
Customize your own dashboards, set traffic alerts and generate reports.
Network behavioral analysis & security monitoring. All-in-one tool.
http://pubads.g.doubleclick.net/gampad/clk?id=126839071&iu=/4140/ostg.clktrk

Re: [libseccomp-discuss] Making a universal list of syscalls?

[Paul Moore]

On Thursday, February 27, 2014 12:40:32 PM Andy Lutomirski wrote:
> Currently, dealing with Linux syscalls in an architecture-independent
> way is a mess.  Here are some issues:
> 
>  1. There's no clean way to map between syscall names and numbers on
> different architectures.  The kernel contains a number of tables (that
> work differently for different architectures).  strace has some arcane
> mechanism.  libseccomp has another.

This is a major pain point for libseccomp, what we have now is passable, and 
it works, but I cringe each time I look at it because I worry about 
maintaining it.  I would be very happy if the kernel had some 
header/file/whatever that could be used by userspace applications to map 
syscall names/numbers for each architecture.

>  2. There's no clean way to map between syscall argument registers and
> logical syscall arguments.  Each architecture knows how to do it, as
> do strace and glibc, but I suspect that *everyone* else gets it wrong.
>  Especially on ARM.

I remember looking into this with libseccomp, around the ARM time frame with 
Andy, and I believe I managed to reassure myself - not well, mind you - that 
we were *ok* with seccomp/libseccomp.  However, having a argument mapping 
document/header/etc. would go a long way here.

>  3. Determining which architectures have which syscalls is a mess.
> Recent kernel builds love to warn me that finit_module is missing on
> x86_64.  This is simply not true.  I have no idea why.

Closely related to item #1.  Also a major pain for libseccomp for the same 
reasons.

>  5. Decoding ucontext from SIGSYS is a mess.  I have prototype code
> for libseccomp that can do it, but it gets the arguments wrong due to
> ABI issues.  See (2).

I've actually been sitting on some of Andy's libseccomp code for this for a 
while now because the solution is very fiddly.  Improvements here could make 
life much easier for us and remove a lot of my hesitation in merging Andy's 
code.

-- 
paul moore
security and virtualization @ redhat

Re: Making a universal list of syscalls?

[H. Peter Anvin]

On 02/27/2014 12:40 PM, Andy Lutomirski wrote:
> Currently, dealing with Linux syscalls in an architecture-independent
> way is a mess.  Here are some issues:
> 
>  1. There's no clean way to map between syscall names and numbers on
> different architectures.  The kernel contains a number of tables (that
> work differently for different architectures).  strace has some arcane
> mechanism.  libseccomp has another.
> 
>  2. There's no clean way to map between syscall argument registers and
> logical syscall arguments.  Each architecture knows how to do it, as
> do strace and glibc, but I suspect that *everyone* else gets it wrong.
>  Especially on ARM.
> 
>  3. Determining which architectures have which syscalls is a mess.
> Recent kernel builds love to warn me that finit_module is missing on
> x86_64.  This is simply not true.  I have no idea why.
> 
>  4. Actually issuing a nontrivial syscall is annoying.  syscall(2) can
> do it for the native architecture (only).
> 
>  5. Decoding ucontext from SIGSYS is a mess.  I have prototype code
> for libseccomp that can do it, but it gets the arguments wrong due to
> ABI issues.  See (2).
> 
> I'd like to see a master list in the kernel that lists, for every
> syscall, the name, the number for each architecture that implements it
> (using the AUDIT_ARCH semantics, probably), and the signature.  The
> build process could parse this table to replace the current per-arch
> mess.
> 

Hi Andy,

I have brought that up a lot of times, originally dating back from my
work on klibc.  I have tried to keep the klibc syscall list in a sane
format with architecture annotations, but it doesn't contain all the
syscalls in the system.

Extending that work and making it encompass everything the kernel
exports would be highly useful, but it would take a lot of work.

	-hpa

Re: Making a universal list of syscalls?

[David Howells]

It would be really nice if we could do this in such a manner that we could
build strace from it.

David

Re: Making a universal list of syscalls?

[Andy Lutomirski]

On Tue, Mar 4, 2014 at 11:27 AM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 02/27/2014 12:40 PM, Andy Lutomirski wrote:
>> Currently, dealing with Linux syscalls in an architecture-independent
>> way is a mess.  Here are some issues:
>>
>>  1. There's no clean way to map between syscall names and numbers on
>> different architectures.  The kernel contains a number of tables (that
>> work differently for different architectures).  strace has some arcane
>> mechanism.  libseccomp has another.
>>
>>  2. There's no clean way to map between syscall argument registers and
>> logical syscall arguments.  Each architecture knows how to do it, as
>> do strace and glibc, but I suspect that *everyone* else gets it wrong.
>>  Especially on ARM.
>>
>>  3. Determining which architectures have which syscalls is a mess.
>> Recent kernel builds love to warn me that finit_module is missing on
>> x86_64.  This is simply not true.  I have no idea why.
>>
>>  4. Actually issuing a nontrivial syscall is annoying.  syscall(2) can
>> do it for the native architecture (only).
>>
>>  5. Decoding ucontext from SIGSYS is a mess.  I have prototype code
>> for libseccomp that can do it, but it gets the arguments wrong due to
>> ABI issues.  See (2).
>>
>> I'd like to see a master list in the kernel that lists, for every
>> syscall, the name, the number for each architecture that implements it
>> (using the AUDIT_ARCH semantics, probably), and the signature.  The
>> build process could parse this table to replace the current per-arch
>> mess.
>>
>
> Hi Andy,
>
> I have brought that up a lot of times, originally dating back from my
> work on klibc.  I have tried to keep the klibc syscall list in a sane
> format with architecture annotations, but it doesn't contain all the
> syscalls in the system.
>
> Extending that work and making it encompass everything the kernel
> exports would be highly useful, but it would take a lot of work.

I think that SYSCALLS.def won't work as is -- SYSCALLS.def
references unistd, which ought to be autogenerated from the syscalls
list.  But a somewhat less magical variant should work.

--Andy

Re: Making a universal list of syscalls?

[Andy Lutomirski]

On Tue, Mar 4, 2014 at 11:27 AM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 02/27/2014 12:40 PM, Andy Lutomirski wrote:
>> Currently, dealing with Linux syscalls in an architecture-independent
>> way is a mess.  Here are some issues:
>>
>>  1. There's no clean way to map between syscall names and numbers on
>> different architectures.  The kernel contains a number of tables (that
>> work differently for different architectures).  strace has some arcane
>> mechanism.  libseccomp has another.
>>
>>  2. There's no clean way to map between syscall argument registers and
>> logical syscall arguments.  Each architecture knows how to do it, as
>> do strace and glibc, but I suspect that *everyone* else gets it wrong.
>>  Especially on ARM.
>>
>>  3. Determining which architectures have which syscalls is a mess.
>> Recent kernel builds love to warn me that finit_module is missing on
>> x86_64.  This is simply not true.  I have no idea why.
>>
>>  4. Actually issuing a nontrivial syscall is annoying.  syscall(2) can
>> do it for the native architecture (only).
>>
>>  5. Decoding ucontext from SIGSYS is a mess.  I have prototype code
>> for libseccomp that can do it, but it gets the arguments wrong due to
>> ABI issues.  See (2).
>>
>> I'd like to see a master list in the kernel that lists, for every
>> syscall, the name, the number for each architecture that implements it
>> (using the AUDIT_ARCH semantics, probably), and the signature.  The
>> build process could parse this table to replace the current per-arch
>> mess.
>>
>
> Hi Andy,
>
> I have brought that up a lot of times, originally dating back from my
> work on klibc.  I have tried to keep the klibc syscall list in a sane
> format with architecture annotations, but it doesn't contain all the
> syscalls in the system.
>
> Extending that work and making it encompass everything the kernel
> exports would be highly useful, but it would take a lot of work.

I think that SYSCALLS.def won't work as is -- SYSCALLS.def
references unistd, which ought to be autogenerated from the syscalls
list.  But a somewhat less magical variant should work.

--Andy

------------------------------------------------------------------------------
Subversion Kills Productivity. Get off Subversion & Make the Move to Perforce.
With Perforce, you get hassle-free workflows. Merge that actually works. 
Faster operations. Version large binaries.  Built-in WAN optimization and the
freedom to use Git, Perforce or both. Make the move to Perforce.
http://pubads.g.doubleclick.net/gampad/clk?id=122218951&iu=/4140/ostg.clktrk

Re: Making a universal list of syscalls?

[H. Peter Anvin]

On 03/05/2014 04:13 PM, Andy Lutomirski wrote:
> 
> I think that SYSCALLS.def won't work as is -- SYSCALLS.def
> references unistd, which ought to be autogenerated from the syscalls
> list.  But a somewhat less magical variant should work.
> 

Well, that is because unistd.h is one of the very few things the kernel
actually *does* export (containing system call numbers, and *usually* is
enough to detect presence/absence of a certain system call, which is
actually all you need in the vast majority of cases.)

	-hpa

Re: Making a universal list of syscalls?

[H. Peter Anvin]

On 03/05/2014 03:08 AM, David Howells wrote:
> It would be really nice if we could do this in such a manner that we could
> build strace from it.

strace, seccomp, klibc, ... we all need more or less the same stuff.

	-hpa

Re: Making a universal list of syscalls?

[Andy Lutomirski]

On Thu, Mar 6, 2014 at 3:37 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 03/05/2014 03:08 AM, David Howells wrote:
>> It would be really nice if we could do this in such a manner that we could
>> build strace from it.
>
> strace, seccomp, klibc, ... we all need more or less the same stuff.

strace also needs to know what the pointed-to type is for pointer
arguments, whether pointers are input, output, or both, and what kind
of flags / constants fit in which integer slot.

--Andy

>
>         -hpa
>
>



-- 
Andy Lutomirski
AMA Capital Management, LLC

Re: Making a universal list of syscalls?

[H. Peter Anvin]

On 03/06/2014 03:40 PM, Andy Lutomirski wrote:
> On Thu, Mar 6, 2014 at 3:37 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>> On 03/05/2014 03:08 AM, David Howells wrote:
>>> It would be really nice if we could do this in such a manner that we could
>>> build strace from it.
>>
>> strace, seccomp, klibc, ... we all need more or less the same stuff.
> 
> strace also needs to know what the pointed-to type is for pointer
> arguments, whether pointers are input, output, or both, and what kind
> of flags / constants fit in which integer slot.
> 

Yes.  That kind of information also needs to be annotated.

It might take a while to get there, but it should be part of the design.

	-hpa

Re: Making a universal list of syscalls?

[Michael Kerrisk]

On Fri, Mar 7, 2014 at 12:40 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Thu, Mar 6, 2014 at 3:37 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>> On 03/05/2014 03:08 AM, David Howells wrote:
>>> It would be really nice if we could do this in such a manner that we could
>>> build strace from it.
>>
>> strace, seccomp, klibc, ... we all need more or less the same stuff.
>
> strace also needs to know what the pointed-to type is for pointer
> arguments, whether pointers are input, output, or both, and what kind
> of flags / constants fit in which integer slot.
>
> --Andy


Late to the party, but... Obviously man-pages would be interested in
any results here, so if you CCed me on any future work/results that
would be great.

Thanks,

Michael