fd type from number

fd type from number

[Loris Degioanni]

(resending making sure this is not part of another thread)

I'm looking for an efficient way to determine the type of an fd (file, 
socket...) given its number, from a kernel module.
The closest thing I found by looking at the kernel sources is 
sockfd_lookup(), which works but is limited to telling me if the fd is a 
socket or not.

Is there something else I can look at?

Thank you in advance,
Loris

fd type from number

[Daniel Baluta]

On Tue, Aug 19, 2014 at 6:38 PM, Loris Degioanni <loris@draios.com> wrote:
> (resending making sure this is not part of another thread)
>
> I'm looking for an efficient way to determine the type of an fd (file,
> socket...) given its number, from a kernel module.
> The closest thing I found by looking at the kernel sources is
> sockfd_lookup(), which works but is limited to telling me if the fd is a
> socket or not.
>
> Is there something else I can look at?

https://www.kernel.org/doc/Documentation/filesystems/files.txt

See point 4.

Daniel.

fd type from number

[Valdis.Kletnieks at vt.edu]

On Tue, 19 Aug 2014 08:38:24 -0700, Loris Degioanni said:

> I'm looking for an efficient way to determine the type of an fd (file,
> socket...) given its number, from a kernel module.

What problem are you trying to solve here?  There may be a better API for
your problem.  So step back - what are you trying to accomplish?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 848 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20140819/a71bb92f/attachment.bin 

fd type from number

[Greg KH]

On Tue, Aug 19, 2014 at 08:38:24AM -0700, Loris Degioanni wrote:
> (resending making sure this is not part of another thread)
> 
> I'm looking for an efficient way to determine the type of an fd (file, 
> socket...) given its number, from a kernel module.

You don't have a "number" from within the kernel, you have a pointer to
the full file descriptor structure, right?  If so, what is missing from
that structure that you can not find?

> The closest thing I found by looking at the kernel sources is 
> sockfd_lookup(), which works but is limited to telling me if the fd is a 
> socket or not.
> 
> Is there something else I can look at?

What exactly are you trying to do with this?  And what does it matter
what "type" of thing a file descriptor is?  What can you do with that
information?

thanks,

greg k-h

fd type from number

[Loris Degioanni]

Sure, here's some more context.

I'm one of the developers of sysdig (www.sysdig.org), a tool that 
captures system calls and uses them to offer advanced system monitoring. 
One of the features that our diver offers is the tcpdump-derived concept 
of "snaplen": when a system call with a buffer is captured, it's 
possible to choose how many bytes of that buffer are copied to the 
driver capture buffer. This makes it possible to tune buffer utilization 
and CPU usage vs completeness of data.

Since this feature is important and heavily used, I'd like to extend it 
so that the user has per-fd-type snaplen control. A typical use case is: 
"I want 1000 bytes of each socket buffer, because I'm interested in 
looking at protocol activity, but I don't care about files and so I'm ok 
with just 20 bytes from them". In order for this feature to be useful, 
it needs to be very fast: we use tracepoints to capture system calls, so 
we slow down the original process if we take too long.

And since I'm here, let me expand my question. Another useful thing to 
do would be per-filename snaplen. Use case: "I want the whole content of 
reads and writes to files that are in /etc, but I want only 20 bytes 
from any other system call". This would I guess involve unpacking the 
file structure and retrieving the full file name. Is there any way to do 
it safely and efficiently?

Thanks,
Loris


On 8/19/2014 9:02 AM, Valdis.Kletnieks at vt.edu wrote:
> On Tue, 19 Aug 2014 08:38:24 -0700, Loris Degioanni said:
>
>> I'm looking for an efficient way to determine the type of an fd (file,
>> socket...) given its number, from a kernel module.
> What problem are you trying to solve here?  There may be a better API for
> your problem.  So step back - what are you trying to accomplish?

fd type from number

[Loris Degioanni]

On 8/19/2014 9:10 AM, Greg KH wrote:
> On Tue, Aug 19, 2014 at 08:38:24AM -0700, Loris Degioanni wrote:
>> (resending making sure this is not part of another thread)
>>
>> I'm looking for an efficient way to determine the type of an fd (file,
>> socket...) given its number, from a kernel module.
> You don't have a "number" from within the kernel, you have a pointer to
> the full file descriptor structure, right?  If so, what is missing from
> that structure that you can not find?

I do have the FD number, because it's coming from intercepting a system 
call.

>
>> The closest thing I found by looking at the kernel sources is
>> sockfd_lookup(), which works but is limited to telling me if the fd is a
>> socket or not.
>>
>> Is there something else I can look at?
> What exactly are you trying to do with this?  And what does it matter
> what "type" of thing a file descriptor is?  What can you do with that
> information?

I just sent another email with some explanation.

Loris

>
> thanks,
>
> greg k-h

fd type from number

[Rohan Puri]

On Tue, Aug 19, 2014 at 10:04 PM, Loris Degioanni <loris@draios.com> wrote:
> Sure, here's some more context.
>
> I'm one of the developers of sysdig (www.sysdig.org), a tool that
> captures system calls and uses them to offer advanced system monitoring.
> One of the features that our diver offers is the tcpdump-derived concept
> of "snaplen": when a system call with a buffer is captured, it's
> possible to choose how many bytes of that buffer are copied to the
> driver capture buffer. This makes it possible to tune buffer utilization
> and CPU usage vs completeness of data.
>
> Since this feature is important and heavily used, I'd like to extend it
> so that the user has per-fd-type snaplen control. A typical use case is:
> "I want 1000 bytes of each socket buffer, because I'm interested in
> looking at protocol activity, but I don't care about files and so I'm ok
> with just 20 bytes from them". In order for this feature to be useful,
> it needs to be very fast: we use tracepoints to capture system calls, so
> we slow down the original process if we take too long.
>
> And since I'm here, let me expand my question. Another useful thing to
> do would be per-filename snaplen. Use case: "I want the whole content of
> reads and writes to files that are in /etc, but I want only 20 bytes
> from any other system call". This would I guess involve unpacking the
> file structure and retrieving the full file name. Is there any way to do
> it safely and efficiently?
>
> Thanks,
> Loris
>
>
> On 8/19/2014 9:02 AM, Valdis.Kletnieks at vt.edu wrote:
>> On Tue, 19 Aug 2014 08:38:24 -0700, Loris Degioanni said:
>>
>>> I'm looking for an efficient way to determine the type of an fd (file,
>>> socket...) given its number, from a kernel module.
>> What problem are you trying to solve here?  There may be a better API for
>> your problem.  So step back - what are you trying to accomplish?
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Hi Loris,

You can get the file type from the fd by doing something like this : -

struct file *file = fget(fd);
if(!file)
        return error;
assert(file->f_inode != NULL);
file_type = (file->f_inode->i_mode & S_IFMT) >> 12;

Also, you can make use of S_IS*(mode) macros, to check for file types.

NOTE: fget() makes use of current process's file_struct.

Regards,
- Rohan

fd type from number

[Loris Degioanni]

On 8/20/2014 2:33 AM, Rohan Puri wrote:
> On Tue, Aug 19, 2014 at 10:04 PM, Loris Degioanni <loris@draios.com> wrote:
>> Sure, here's some more context.
>>
>> I'm one of the developers of sysdig (www.sysdig.org), a tool that
>> captures system calls and uses them to offer advanced system monitoring.
>> One of the features that our diver offers is the tcpdump-derived concept
>> of "snaplen": when a system call with a buffer is captured, it's
>> possible to choose how many bytes of that buffer are copied to the
>> driver capture buffer. This makes it possible to tune buffer utilization
>> and CPU usage vs completeness of data.
>>
>> Since this feature is important and heavily used, I'd like to extend it
>> so that the user has per-fd-type snaplen control. A typical use case is:
>> "I want 1000 bytes of each socket buffer, because I'm interested in
>> looking at protocol activity, but I don't care about files and so I'm ok
>> with just 20 bytes from them". In order for this feature to be useful,
>> it needs to be very fast: we use tracepoints to capture system calls, so
>> we slow down the original process if we take too long.
>>
>> And since I'm here, let me expand my question. Another useful thing to
>> do would be per-filename snaplen. Use case: "I want the whole content of
>> reads and writes to files that are in /etc, but I want only 20 bytes
>> from any other system call". This would I guess involve unpacking the
>> file structure and retrieving the full file name. Is there any way to do
>> it safely and efficiently?
>>
>> Thanks,
>> Loris
>>
>>
>> On 8/19/2014 9:02 AM, Valdis.Kletnieks at vt.edu wrote:
>>> On Tue, 19 Aug 2014 08:38:24 -0700, Loris Degioanni said:
>>>
>>>> I'm looking for an efficient way to determine the type of an fd (file,
>>>> socket...) given its number, from a kernel module.
>>> What problem are you trying to solve here?  There may be a better API for
>>> your problem.  So step back - what are you trying to accomplish?
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> Hi Loris,
>
> You can get the file type from the fd by doing something like this : -
>
> struct file *file = fget(fd);
> if(!file)
>          return error;
> assert(file->f_inode != NULL);
> file_type = (file->f_inode->i_mode & S_IFMT) >> 12;
>
> Also, you can make use of S_IS*(mode) macros, to check for file types.
>
> NOTE: fget() makes use of current process's file_struct.
>
> Regards,
> - Rohan

Thanks Rohan,
and for kernels more recent than 3.14 I assume I need to use fdget 
instead of fget, right?

Loris

fd type from number

[Rohan Puri]

On Sat, Aug 23, 2014 at 12:33 AM, Loris Degioanni <loris@draios.com> wrote:
> On 8/20/2014 2:33 AM, Rohan Puri wrote:
>>
>> On Tue, Aug 19, 2014 at 10:04 PM, Loris Degioanni <loris@draios.com>
>> wrote:
>>>
>>> Sure, here's some more context.
>>>
>>> I'm one of the developers of sysdig (www.sysdig.org), a tool that
>>> captures system calls and uses them to offer advanced system monitoring.
>>> One of the features that our diver offers is the tcpdump-derived concept
>>> of "snaplen": when a system call with a buffer is captured, it's
>>> possible to choose how many bytes of that buffer are copied to the
>>> driver capture buffer. This makes it possible to tune buffer utilization
>>> and CPU usage vs completeness of data.
>>>
>>> Since this feature is important and heavily used, I'd like to extend it
>>> so that the user has per-fd-type snaplen control. A typical use case is:
>>> "I want 1000 bytes of each socket buffer, because I'm interested in
>>> looking at protocol activity, but I don't care about files and so I'm ok
>>> with just 20 bytes from them". In order for this feature to be useful,
>>> it needs to be very fast: we use tracepoints to capture system calls, so
>>> we slow down the original process if we take too long.
>>>
>>> And since I'm here, let me expand my question. Another useful thing to
>>> do would be per-filename snaplen. Use case: "I want the whole content of
>>> reads and writes to files that are in /etc, but I want only 20 bytes
>>> from any other system call". This would I guess involve unpacking the
>>> file structure and retrieving the full file name. Is there any way to do
>>> it safely and efficiently?
>>>
>>> Thanks,
>>> Loris
>>>
>>>
>>> On 8/19/2014 9:02 AM, Valdis.Kletnieks at vt.edu wrote:
>>>>
>>>> On Tue, 19 Aug 2014 08:38:24 -0700, Loris Degioanni said:
>>>>
>>>>> I'm looking for an efficient way to determine the type of an fd (file,
>>>>> socket...) given its number, from a kernel module.
>>>>
>>>> What problem are you trying to solve here?  There may be a better API
>>>> for
>>>> your problem.  So step back - what are you trying to accomplish?
>>>
>>>
>>> _______________________________________________
>>> Kernelnewbies mailing list
>>> Kernelnewbies at kernelnewbies.org
>>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>> Hi Loris,
>>
>> You can get the file type from the fd by doing something like this : -
>>
>> struct file *file = fget(fd);
>> if(!file)
>>          return error;
>> assert(file->f_inode != NULL);
>> file_type = (file->f_inode->i_mode & S_IFMT) >> 12;
>>
>> Also, you can make use of S_IS*(mode) macros, to check for file types.
>>
>> NOTE: fget() makes use of current process's file_struct.
>>
>> Regards,
>> - Rohan
>
>
> Thanks Rohan,
> and for kernels more recent than 3.14 I assume I need to use fdget instead
> of fget, right?
fdget() calls __fget_light() internally & if you check out the definition
of __fget_light(), there is a comment there. Pasting it over here : -
/*
 * Lightweight file lookup - no refcnt increment if fd table isn't shared.
 *
 * You can use this instead of fget if you satisfy all of the following
 * conditions:
 * 1) You must call fput_light before exiting the syscall and returning
control
 *    to userspace (i.e. you cannot remember the returned struct file *
after
 *    returning to userspace).
 * 2) You must not call filp_close on the returned struct file * in between
 *    calls to fget_light and fput_light.
 * 3) You must not clone the current task in between the calls to fget_light
 *    and fput_light.
 *
 * The fput_needed flag returned by fget_light should be passed to the
 * corresponding fput_light.
 */

fdget() is different from fget() in 2 ways, if fd table is not shared,
meaning files_struct->count is 1 : -
1. Doesnt take rcu_read_lock()
2. Doesnt increment struct file->f_count.

else it behaves the same as fget(), *so yes i think you can use fdget().*
>
> Loris
>

Regards,
Rohan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20140823/52694a11/attachment-0001.html 

fd type from number

[Greg Freemyer]

On August 18, 2014 8:46:16 PM EDT, Loris Degioanni <loris@draios.com> wrote:
>I'm looking for an efficient way to determine the type of an fd (file, 
>socket...) given its number, from a kernel module.
>The closest thing I found by looking at the kernel sources is 
>sockfd_lookup(), which works but is limited to telling me if the fd is
>a 
>socket or not.
>
>Is there something else I can look at?
>
>Thank you in advance.

Loris,

Apparently you replied to a different email then changed the subject.  Email clients that thread conversations based on "in-reply-to" will lump your supposedly new thread in with the one you replied to.  That guarantees a reduced audience of readers, especially considering the email you replied to.

You should repost by composing a brand new email.

Greg

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

fd type from number

[Loris Degioanni]

I'm looking for an efficient way to determine the type of an fd (file, 
socket...) given its number, from a kernel module.
The closest thing I found by looking at the kernel sources is 
sockfd_lookup(), which works but is limited to telling me if the fd is a 
socket or not.

Is there something else I can look at?

Thank you in advance,
Loris