* Djprobes questions @ 2007-03-12 22:37 Mathieu Desnoyers 2007-03-13 6:07 ` Masami Hiramatsu 0 siblings, 1 reply; 3+ messages in thread From: Mathieu Desnoyers @ 2007-03-12 22:37 UTC (permalink / raw) To: Masami Hiramatsu, linux-kernel, systemtap, mbligh Hi Masami, I recently had to add support for inline code patching on i386 to my marker infrastructure. Clearly, it looks like what is done in djprobes, with the main difference that I only patch the immediate value of a 2 bytes "load immediate" instruction. I think I found a solution to one of the main issues with djprobes : it currently has to wait for each CPU to hit the probe before being sure that it's safe to patch the code with something else than an int3. This is due to PIII errata 49, which says that a CPU much execute a serializing instruction before executing cross-modified code. Here is what I do : While I use a breakpoint to fall in a trap for the CPUs that hit the site currently being modified, I also send an IPI to all CPUs so they execute cpuid. Once it returns, I am sure that every CPU has executed a serializing instruction, which enables me to go on with the complete code modification, therefore removing the initial breakpoint. Here is my code : http://ltt.polymtl.ca/cgi-bin/gitweb.cgi?p=linux-2.6-lttng.git;a=blob;f=arch/i386/kernel/marker.c;h=89b06f02f0966685be260d6364a0dd94c3d14456;hb=v2.6.20-lttng (Comments are welcome) On a second note, looking at the djprobes code triggered some question in my mind about the safety of using a worker thread to "make sure" every interrupt context has returned (so there is no IP pointing into the modified code). The following scenario might be possible : an interrupt handler (or trap handler) reenables interrupts, does irq_exit() or nmi_exit() (which reenables preemption) but does not do iret yet. My understanding is that it could be scheduled and have a return IP pointing to the code that is being modified. Am I right ? Regards, Mathieu -- Mathieu Desnoyers Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68 ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Djprobes questions 2007-03-12 22:37 Djprobes questions Mathieu Desnoyers @ 2007-03-13 6:07 ` Masami Hiramatsu 2007-03-13 17:24 ` Mathieu Desnoyers 0 siblings, 1 reply; 3+ messages in thread From: Masami Hiramatsu @ 2007-03-13 6:07 UTC (permalink / raw) To: Mathieu Desnoyers Cc: Masami Hiramatsu, linux-kernel, systemtap, mbligh, Satoshi Oshima, Yumiko Sugita, Hideo Aoki Hi Mathieu, Mathieu Desnoyers wrote: > Hi Masami, > > I recently had to add support for inline code patching on i386 to my > marker infrastructure. Clearly, it looks like what is done in djprobes, > with the main difference that I only patch the immediate value of a 2 > bytes "load immediate" instruction. That's interesting. > I think I found a solution to one of the main issues with djprobes : it > currently has to wait for each CPU to hit the probe before being sure > that it's safe to patch the code with something else than an int3. This > is due to PIII errata 49, which says that a CPU much execute a > serializing instruction before executing cross-modified code. Hmm, djprobe already might not wait for each CPU to hit the probe point. It just wait scheduler synchronization instead of that. And after that, it issues cpuid for cache serialization before executing cross-modified code. The most difficult point of the djprobe is that it has to replace "live" instructions. So we must check other processors not to run those instructions carefully. > Here is what I do : While I use a breakpoint to fall in a trap for the > CPUs that hit the site currently being modified, I also send an IPI to > all CPUs so they execute cpuid. Once it returns, I am sure that every > CPU has executed a serializing instruction, which enables me to go on > with the complete code modification, therefore removing the initial > breakpoint. I think its OK. That is the same way which I've done in djprobe. > Here is my code : > > http://ltt.polymtl.ca/cgi-bin/gitweb.cgi?p=linux-2.6-lttng.git;a=blob;f=arch/i386/kernel/marker.c;h=89b06f02f0966685be260d6364a0dd94c3d14456;hb=v2.6.20-lttng > > (Comments are welcome) > > On a second note, looking at the djprobes code triggered some question > in my mind about the safety of using a worker thread to "make sure" > every interrupt context has returned (so there is no IP pointing into > the modified code). The following scenario might be possible : an > interrupt handler (or trap handler) reenables interrupts, does irq_exit() > or nmi_exit() (which reenables preemption) but does not do iret yet. My > understanding is that it could be scheduled and have a return IP > pointing to the code that is being modified. Am I right ? Same idea was already discussed. It might work on normal kernel, but, unfortunately, it doesn't work on preemptive kernel. And actually, that idea is same as synchronize_sched(). So, I've used it on normal kernel. In the case of preemptive kernel, currently, I'm using freeze_processes() suggested by Ingo. Anyway, I and Satoshi are developing a static analysis tool to check whether target instructions can be replaced by long jump. I'd like to release djprobe patch against latest kernel after developed it. Best regards, -- Masami HIRAMATSU Linux Technology Center Hitachi, Ltd., Systems Development Laboratory E-mail: masami.hiramatsu.pt@hitachi.com ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Djprobes questions 2007-03-13 6:07 ` Masami Hiramatsu @ 2007-03-13 17:24 ` Mathieu Desnoyers 0 siblings, 0 replies; 3+ messages in thread From: Mathieu Desnoyers @ 2007-03-13 17:24 UTC (permalink / raw) To: Masami Hiramatsu Cc: Masami Hiramatsu, linux-kernel, systemtap, mbligh, Satoshi Oshima, Yumiko Sugita, Hideo Aoki * Masami Hiramatsu (masami.hiramatsu.pt@hitachi.com) wrote: > Hi Mathieu, > > Mathieu Desnoyers wrote: > > Hi Masami, > > > > I recently had to add support for inline code patching on i386 to my > > marker infrastructure. Clearly, it looks like what is done in djprobes, > > with the main difference that I only patch the immediate value of a 2 > > bytes "load immediate" instruction. > > That's interesting. > > > I think I found a solution to one of the main issues with djprobes : it > > currently has to wait for each CPU to hit the probe before being sure > > that it's safe to patch the code with something else than an int3. This > > is due to PIII errata 49, which says that a CPU much execute a > > serializing instruction before executing cross-modified code. > > Hmm, djprobe already might not wait for each CPU to hit the probe > point. It just wait scheduler synchronization instead of that. Oh, I see. You can do it because you don't support fully preemptible kernels. It makes sense it that scenario. I could not use this scheme for my markers though, since I support fully preemptible kernels. > And after that, it issues cpuid for cache serialization before > executing cross-modified code. > Ok, so you already used the IPI serialization I am talking about. Great :) > The most difficult point of the djprobe is that it has to replace > "live" instructions. So we must check other processors not to run > those instructions carefully. > > > Here is what I do : While I use a breakpoint to fall in a trap for the > > CPUs that hit the site currently being modified, I also send an IPI to > > all CPUs so they execute cpuid. Once it returns, I am sure that every > > CPU has executed a serializing instruction, which enables me to go on > > with the complete code modification, therefore removing the initial > > breakpoint. > > I think its OK. That is the same way which I've done in djprobe. > > > Here is my code : > > > > http://ltt.polymtl.ca/cgi-bin/gitweb.cgi?p=linux-2.6-lttng.git;a=blob;f=arch/i386/kernel/marker.c;h=89b06f02f0966685be260d6364a0dd94c3d14456;hb=v2.6.20-lttng > > > > (Comments are welcome) > > > > On a second note, looking at the djprobes code triggered some question > > in my mind about the safety of using a worker thread to "make sure" > > every interrupt context has returned (so there is no IP pointing into > > the modified code). The following scenario might be possible : an > > interrupt handler (or trap handler) reenables interrupts, does irq_exit() > > or nmi_exit() (which reenables preemption) but does not do iret yet. My > > understanding is that it could be scheduled and have a return IP > > pointing to the code that is being modified. Am I right ? > > Same idea was already discussed. It might work on normal kernel, > but, unfortunately, it doesn't work on preemptive kernel. And actually, > that idea is same as synchronize_sched(). So, I've used it on normal > kernel. In the case of preemptive kernel, currently, I'm using > freeze_processes() suggested by Ingo. > It looks like a good way to know where the processes are stopped. > Anyway, I and Satoshi are developing a static analysis tool to > check whether target instructions can be replaced by long jump. > I'd like to release djprobe patch against latest kernel after > developed it. > Great, continue your good work! Regards, Mathieu > -- > Masami HIRAMATSU > Linux Technology Center > Hitachi, Ltd., Systems Development Laboratory > E-mail: masami.hiramatsu.pt@hitachi.com > > -- Mathieu Desnoyers Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68 ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2007-03-13 17:24 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-03-12 22:37 Djprobes questions Mathieu Desnoyers 2007-03-13 6:07 ` Masami Hiramatsu 2007-03-13 17:24 ` Mathieu Desnoyers
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).