* Thread scheduling on an SMP
@ 2004-12-23 13:14 Darío Mariani
2004-12-23 17:08 ` Richard Nairn
2004-12-24 13:14 ` file deletion Andy
0 siblings, 2 replies; 13+ messages in thread
From: Darío Mariani @ 2004-12-23 13:14 UTC (permalink / raw)
To: linux-c-programming
Hello:
I have a machine with 4 processors, and I wold like to make the
following: Set 1 processor to do general stuff using the default
scheduler and the other three working on my program exclusively, with
the realtime or my own thread scheduler.
Is this possible?, thanks for any comment or links to documentation
you can give me.
Darío
-
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: file deletion
2004-12-24 13:14 ` file deletion Andy
@ 2004-12-23 13:49 ` Jan-Benedict Glaw
2004-12-27 3:50 ` J.
2004-12-30 11:26 ` Glynn Clements
2 siblings, 0 replies; 13+ messages in thread
From: Jan-Benedict Glaw @ 2004-12-23 13:49 UTC (permalink / raw)
To: linux-c-programming
[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]
On Fri, 2004-12-24 13:14:11 -0000, Andy <andy_webb@onetel.com>
wrote in message <003f01c4e9ba$7679d020$316c4ed5@j0s6l8>:
> Does anybody know of any useful code or c commands that
> you can use to search for duplicate files within a linux/unix directory and
> its subdirectories and remove them?
As long as you believe in cryptographical hashes, something like this
should do the trick:
SOME_DIR=/some/directory
find "${SOME_DIR}" -type f -exec sha1sum {} \; | \
sort | \
uniq -c | \
sed -e 's/^[[:space:]]*//g' | \
egrep -v '\<1\>' | \
cut -f 4- -d ' ' | \
while read DOUBLE_FILE_NAME; do
rm -f "${SOME_DIR}/${DOUBLE_FILE_NAME}"
done
That's untested, but should work (it's essentially a one-liner).
However, it removes *all* instances of files which are believed to be
identical...
MfG, JBG
--
Jan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Thread scheduling on an SMP
2004-12-23 13:14 Thread scheduling on an SMP Darío Mariani
@ 2004-12-23 17:08 ` Richard Nairn
2004-12-24 13:14 ` file deletion Andy
1 sibling, 0 replies; 13+ messages in thread
From: Richard Nairn @ 2004-12-23 17:08 UTC (permalink / raw)
To: Darío Mariani, linux-c-programming
If I am reading your question correctly, you are talking about schedular
affinity. You can set which processors you would like to assign pid's to.
You need a kernel after 2.5.8 for thiese functions...
Excerpt:
#include <sched.h>
int sched_setaffinity(pid_t pid, unsigned int len, unsigned long *mask);
int sched_getaffinity(pid_t pid, unsigned int len, unsigned long *mask);
DESCRIPTION
sched_setaffinity sets the CPU affinity mask of the process denoted by
pid. If pid is zero, then the current process is used.
The affinity mask is represented by the bitmask stored in mask. The least
significant bit corresponds to the first logical processor number on the
system, while the most significant bit corresponds to the last logical
processor number on the system. A set bit corresponds to a legally
schedulable CPU while an unset bit corresponds to an illegally schedulable
CPU. In other words, a process is bound to and will only run on processors
whose corresponding bit is set. Usually, all bits in the mask are set.
The argument len is the length of the data pointed to by mask. Normally
this is the size of a word on the system. For compatibility with future
versions of the Linux kernel, since this size can change, the bitmask
supplied must be at least as large as the affinity mask stored in the
kernel.
The function sched_getaffinity writes into the pointer supplied by mask
that is size len the affinity mask of process pid. If pid is zero, then
the mask of the current process is returned.
On Thu, 23 Dec 2004 10:14:17 -0300, Darío Mariani
<mariani.dario@gmail.com> wrote:
> Hello:
> I have a machine with 4 processors, and I wold like to make the
> following: Set 1 processor to do general stuff using the default
> scheduler and the other three working on my program exclusively, with
> the realtime or my own thread scheduler.
> Is this possible?, thanks for any comment or links to documentation
> you can give me.
>
> Darío
> -
> To unsubscribe from this list: send the line "unsubscribe
> linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
| Richard Nairn Specializing in Linux
| Nairn Consulting Web / Database Solutions
| Calgary, AB
| Richard@NairnConsulting.ca
-
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* file deletion
2004-12-23 13:14 Thread scheduling on an SMP Darío Mariani
2004-12-23 17:08 ` Richard Nairn
@ 2004-12-24 13:14 ` Andy
2004-12-23 13:49 ` Jan-Benedict Glaw
` (2 more replies)
1 sibling, 3 replies; 13+ messages in thread
From: Andy @ 2004-12-24 13:14 UTC (permalink / raw)
To: linux-c-programming
Does anybody know of any useful code or c commands that
you can use to search for duplicate files within a linux/unix directory and
its subdirectories and remove them?
Thanks
Andrew
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: file deletion
2004-12-24 13:14 ` file deletion Andy
2004-12-23 13:49 ` Jan-Benedict Glaw
@ 2004-12-27 3:50 ` J.
2004-12-30 11:26 ` Glynn Clements
2 siblings, 0 replies; 13+ messages in thread
From: J. @ 2004-12-27 3:50 UTC (permalink / raw)
To: linux-c-programming
On Fri, 24 Dec 2004, Andy wrote:
> Does anybody know of any useful code or c commands that
> you can use to search for duplicate files within a linux/unix directory and
> its subdirectories and remove them?
> Thanks
> Andrew
Ehm.. Personally no.. However there is ftw, opendir, readdir, stat,
fstat... etc.. Ones you are able to access all the directory entries you
have to make a decision how you want to compare e.g. md5, crc32, only size
or name etc... Then there is the issue of choosing a optimal ADT and
access/retrieval algo.
If you dont have to c code the program but just looking for a solution I
would most certainly go for a:
`find -type f -exec md5sum '{}' \; >> md5.log`
and parse the md5.log with a simple shell or awk script. That would save
many headache's.. Plus you don't have to reinvent `find`...
J.
--
http://www.rdrs.net
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: file deletion
2004-12-24 13:14 ` file deletion Andy
2004-12-23 13:49 ` Jan-Benedict Glaw
2004-12-27 3:50 ` J.
@ 2004-12-30 11:26 ` Glynn Clements
2004-12-30 12:04 ` wwp
2 siblings, 1 reply; 13+ messages in thread
From: Glynn Clements @ 2004-12-30 11:26 UTC (permalink / raw)
To: Andy; +Cc: linux-c-programming
Andy wrote:
> Does anybody know of any useful code or c commands that
> you can use to search for duplicate files within a linux/unix directory and
> its subdirectories and remove them?
First, you have to define "duplicate". Also, once you've decided that
two or more files are duplicates, you have to decide which one you
wish to keep and which ones are to be deleted.
If you consider any files with identical MD5 hashes as duplicates, and
don't care about which one is kept, you could use something like:
find . -type f -exec md5sum {} \; | sort | uniq -w32 -d | cut -b 35- | \
while read file ; do rm -- "$file" ; done
However, if you have 3 or more copies of a given file, the above will
only delete one of them ("uniq -d ..." only prints one instance of
each duplicate line; "uniq -D ..." prints all instances, which would
result in all copies being removed; there isn't an all-but-one option).
--
Glynn Clements <glynn@gclements.plus.com>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: file deletion
2004-12-30 11:26 ` Glynn Clements
@ 2004-12-30 12:04 ` wwp
2005-01-01 11:09 ` Andy
0 siblings, 1 reply; 13+ messages in thread
From: wwp @ 2004-12-30 12:04 UTC (permalink / raw)
To: linux-c-programming
Hello Glynn et al,
On Thu, 30 Dec 2004 11:26:21 +0000 Glynn Clements <glynn@gclements.plus.com> wrote:
>
> Andy wrote:
>
> > Does anybody know of any useful code or c commands that
> > you can use to search for duplicate files within a linux/unix directory and
> > its subdirectories and remove them?
>
> First, you have to define "duplicate". Also, once you've decided that
> two or more files are duplicates, you have to decide which one you
> wish to keep and which ones are to be deleted.
>
> If you consider any files with identical MD5 hashes as duplicates, and
> don't care about which one is kept, you could use something like:
>
> find . -type f -exec md5sum {} \; | sort | uniq -w32 -d | cut -b 35- | \
> while read file ; do rm -- "$file" ; done
>
> However, if you have 3 or more copies of a given file, the above will
> only delete one of them ("uniq -d ..." only prints one instance of
> each duplicate line; "uniq -D ..." prints all instances, which would
> result in all copies being removed; there isn't an all-but-one option).
An idea would also be to remove all-but-one and create hardlinks in place of
the removed ones, maybe? Of course it would depend Andy's answers to the
questions raised by Glynn: what means 'duplicate' and what to do w/ those
dups (even why are you checkinf for dups and what are you expecting to be
able to do?).
Regards,
--
wwp
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: file deletion
2005-01-01 11:09 ` Andy
@ 2004-12-31 17:55 ` Jan-Benedict Glaw
2005-01-06 7:07 ` Linux source code: malloc.c Venkatesh Joshi
1 sibling, 0 replies; 13+ messages in thread
From: Jan-Benedict Glaw @ 2004-12-31 17:55 UTC (permalink / raw)
To: linux-c-programming
[-- Attachment #1: Type: text/plain, Size: 847 bytes --]
On Sat, 2005-01-01 11:09:37 -0000, Andy <andy_webb@onetel.com>
wrote in message <000b01c4eff2$623ff7e0$8a7e4ed5@j0s6l8>:
> Gents,
> thanks for this, this script seems to do the trick.
> Actually what I meant by "duplicate" but didn't clarify
> was identical name and file size, dates do not matter.
> What does md5sum {} do?
The "find" command will substitute "{}" inside an "-exec ... ;"
expression with the filename it currently checks. So this is only a
md5sum call with a given filename.
MfG, JBG
--
Jan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: file deletion
2004-12-30 12:04 ` wwp
@ 2005-01-01 11:09 ` Andy
2004-12-31 17:55 ` Jan-Benedict Glaw
2005-01-06 7:07 ` Linux source code: malloc.c Venkatesh Joshi
0 siblings, 2 replies; 13+ messages in thread
From: Andy @ 2005-01-01 11:09 UTC (permalink / raw)
To: wwp, linux-c-programming
Gents,
thanks for this, this script seems to do the trick.
Actually what I meant by "duplicate" but didn't clarify
was identical name and file size, dates do not matter.
What does md5sum {} do?
Thanks
Andy
----- Original Message -----
From: "wwp" <subscript@free.fr>
To: <linux-c-programming@vger.kernel.org>
Sent: Thursday, December 30, 2004 12:04 PM
Subject: Re: file deletion
> Hello Glynn et al,
>
>
> On Thu, 30 Dec 2004 11:26:21 +0000 Glynn Clements
<glynn@gclements.plus.com> wrote:
>
> >
> > Andy wrote:
> >
> > > Does anybody know of any useful code or c commands that
> > > you can use to search for duplicate files within a linux/unix
directory and
> > > its subdirectories and remove them?
> >
> > First, you have to define "duplicate". Also, once you've decided that
> > two or more files are duplicates, you have to decide which one you
> > wish to keep and which ones are to be deleted.
> >
> > If you consider any files with identical MD5 hashes as duplicates, and
> > don't care about which one is kept, you could use something like:
> >
> > find . -type f -exec md5sum {} \; | sort | uniq -w32 -d | cut -b 35- | \
> > while read file ; do rm -- "$file" ; done
> >
> > However, if you have 3 or more copies of a given file, the above will
> > only delete one of them ("uniq -d ..." only prints one instance of
> > each duplicate line; "uniq -D ..." prints all instances, which would
> > result in all copies being removed; there isn't an all-but-one option).
>
> An idea would also be to remove all-but-one and create hardlinks in place
of
> the removed ones, maybe? Of course it would depend Andy's answers to the
> questions raised by Glynn: what means 'duplicate' and what to do w/ those
> dups (even why are you checkinf for dups and what are you expecting to be
> able to do?).
>
>
> Regards,
>
> --
> wwp
> -
> To unsubscribe from this list: send the line "unsubscribe
linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Linux source code: malloc.c
2005-01-01 11:09 ` Andy
2004-12-31 17:55 ` Jan-Benedict Glaw
@ 2005-01-06 7:07 ` Venkatesh Joshi
2005-01-06 9:11 ` Steven Smith
1 sibling, 1 reply; 13+ messages in thread
From: Venkatesh Joshi @ 2005-01-06 7:07 UTC (permalink / raw)
To: linux-c-programming
Hi,
Where can I see the implementation of free() and malloc() in Linux
source code? I am not able to locate the file malloc.c.
thanks,
venkatesh
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linux source code: malloc.c
2005-01-06 7:07 ` Linux source code: malloc.c Venkatesh Joshi
@ 2005-01-06 9:11 ` Steven Smith
2005-01-06 15:27 ` davidgn
0 siblings, 1 reply; 13+ messages in thread
From: Steven Smith @ 2005-01-06 9:11 UTC (permalink / raw)
To: Venkatesh Joshi; +Cc: linux-c-programming, sos22
[-- Attachment #1: Type: text/plain, Size: 404 bytes --]
> Where can I see the implementation of free() and malloc() in Linux
> source code? I am not able to locate the file malloc.c.
That's because malloc() isn't in Linux-the-kernel: it's in glibc.
The file you're looking for is malloc/malloc.c in the glibc tarball.
Steven Smith.
--
One day, I'm going to get an Alice-bot to answer all my email for me,
and see how long it takes people to notice.
[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linux source code: malloc.c
2005-01-06 9:11 ` Steven Smith
@ 2005-01-06 15:27 ` davidgn
2005-01-06 16:10 ` Steven Smith
0 siblings, 1 reply; 13+ messages in thread
From: davidgn @ 2005-01-06 15:27 UTC (permalink / raw)
To: linux-c-programming
Mensaje citado por: Steven Smith <sos22@cam.ac.uk>:
> > Where can I see the implementation of free() and malloc() in Linux
> > source code? I am not able to locate the file malloc.c.
> That's because malloc() isn't in Linux-the-kernel: it's in glibc.
> The file you're looking for is malloc/malloc.c in the glibc tarball.
>
Except the kernel implements its own "kmalloc", which if I'm not wrong is called
upon by libc's malloc
-------------------------------------------------
www.correo.unam.mx
UNAMonos Comunicándonos
-
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Linux source code: malloc.c
2005-01-06 15:27 ` davidgn
@ 2005-01-06 16:10 ` Steven Smith
0 siblings, 0 replies; 13+ messages in thread
From: Steven Smith @ 2005-01-06 16:10 UTC (permalink / raw)
To: davidgn; +Cc: linux-c-programming, sos22
[-- Attachment #1: Type: text/plain, Size: 933 bytes --]
> > > Where can I see the implementation of free() and malloc() in Linux
> > > source code? I am not able to locate the file malloc.c.
> > That's because malloc() isn't in Linux-the-kernel: it's in glibc.
> > The file you're looking for is malloc/malloc.c in the glibc tarball.
> Except the kernel implements its own "kmalloc", which if I'm not wrong is called
> upon by libc's malloc
Um, sort of... the kernel does implement a function kmalloc, but it's
not the same as the user-exposed one. For a start, memory allocated by
kmalloc is only accessible from kernel space, rather than user programs.
The kernel does provide two main primitives for userspace programs to
allocated memory: mmap and brk. These are both fairly limited,
though. mmap can only cope with multiples of the page size (4096
bytes on x86), while brk enforces a stack-like discipline.
sys_mmap and sys_brk do call upon kmalloc themselves, though.
Steven.
[-- Attachment #2: Type: application/pgp-signature, Size: 187 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2005-01-06 16:10 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-12-23 13:14 Thread scheduling on an SMP Darío Mariani
2004-12-23 17:08 ` Richard Nairn
2004-12-24 13:14 ` file deletion Andy
2004-12-23 13:49 ` Jan-Benedict Glaw
2004-12-27 3:50 ` J.
2004-12-30 11:26 ` Glynn Clements
2004-12-30 12:04 ` wwp
2005-01-01 11:09 ` Andy
2004-12-31 17:55 ` Jan-Benedict Glaw
2005-01-06 7:07 ` Linux source code: malloc.c Venkatesh Joshi
2005-01-06 9:11 ` Steven Smith
2005-01-06 15:27 ` davidgn
2005-01-06 16:10 ` Steven Smith
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.