* [PATCH RFC 1/3] RISC-V: KVM: Change the method of calculating cycles to nanoseconds
2020-12-03 12:18 [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM Yifei Jiang
@ 2020-12-03 12:18 ` Yifei Jiang
2020-12-03 12:18 ` [PATCH RFC 2/3] RISC-V: KVM: Support dynamic time frequency from userspace Yifei Jiang
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Yifei Jiang @ 2020-12-03 12:18 UTC (permalink / raw)
To: anup.patel, atish.patra, paul.walmsley, palmer, aou, pbonzini
Cc: kvm-riscv, kvm, linux-riscv, linux-kernel, victor.zhangxiaofeng,
wu.wubin, zhang.zhanghailiang, dengkai1, yinyipeng1, Yifei Jiang
Because we will introduce the dynamic guest frequency later, we
can't use the fixed mult and shift to calculate nanoseconds.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com>
Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com>
---
arch/riscv/include/asm/kvm_vcpu_timer.h | 3 ---
arch/riscv/kvm/vcpu_timer.c | 3 +--
2 files changed, 1 insertion(+), 5 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_vcpu_timer.h b/arch/riscv/include/asm/kvm_vcpu_timer.h
index 375281eb49e0..87e00d878999 100644
--- a/arch/riscv/include/asm/kvm_vcpu_timer.h
+++ b/arch/riscv/include/asm/kvm_vcpu_timer.h
@@ -12,9 +12,6 @@
#include <linux/hrtimer.h>
struct kvm_guest_timer {
- /* Mult & Shift values to get nanoseconds from cycles */
- u32 nsec_mult;
- u32 nsec_shift;
/* Time delta value */
u64 time_delta;
};
diff --git a/arch/riscv/kvm/vcpu_timer.c b/arch/riscv/kvm/vcpu_timer.c
index ddd0ce727b83..f6b35180199a 100644
--- a/arch/riscv/kvm/vcpu_timer.c
+++ b/arch/riscv/kvm/vcpu_timer.c
@@ -33,7 +33,7 @@ static u64 kvm_riscv_delta_cycles2ns(u64 cycles,
cycles_delta = cycles - cycles_now;
else
cycles_delta = 0;
- delta_ns = (cycles_delta * gt->nsec_mult) >> gt->nsec_shift;
+ delta_ns = mul_u64_u64_div_u64(cycles_delta, NSEC_PER_SEC, riscv_timebase);
local_irq_restore(flags);
return delta_ns;
@@ -218,7 +218,6 @@ int kvm_riscv_guest_timer_init(struct kvm *kvm)
{
struct kvm_guest_timer *gt = &kvm->arch.timer;
- riscv_cs_get_mult_shift(>->nsec_mult, >->nsec_shift);
gt->time_delta = -get_cycles64();
return 0;
--
2.19.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH RFC 2/3] RISC-V: KVM: Support dynamic time frequency from userspace
2020-12-03 12:18 [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM Yifei Jiang
2020-12-03 12:18 ` [PATCH RFC 1/3] RISC-V: KVM: Change the method of calculating cycles to nanoseconds Yifei Jiang
@ 2020-12-03 12:18 ` Yifei Jiang
2020-12-03 12:18 ` [PATCH RFC 3/3] RISC-V: KVM: Implement guest time scaling Yifei Jiang
2020-12-16 6:39 ` [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM Anup Patel
3 siblings, 0 replies; 6+ messages in thread
From: Yifei Jiang @ 2020-12-03 12:18 UTC (permalink / raw)
To: anup.patel, atish.patra, paul.walmsley, palmer, aou, pbonzini
Cc: kvm-riscv, kvm, linux-riscv, linux-kernel, victor.zhangxiaofeng,
wu.wubin, zhang.zhanghailiang, dengkai1, yinyipeng1, Yifei Jiang
This patch implements KVM_S/GET_ONE_REG of time frequency to support
setting dynamic time frequency from userspace. When the time frequency
specified by userspace is inconsistent with host 'riscv_timebase',
it will use scale_mult and scale_shift to calculate guest scaling time.
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com>
Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com>
---
arch/riscv/include/asm/kvm_vcpu_timer.h | 9 ++++++
arch/riscv/kvm/vcpu_timer.c | 40 +++++++++++++++++++++----
2 files changed, 44 insertions(+), 5 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_vcpu_timer.h b/arch/riscv/include/asm/kvm_vcpu_timer.h
index 87e00d878999..41b5503de9e4 100644
--- a/arch/riscv/include/asm/kvm_vcpu_timer.h
+++ b/arch/riscv/include/asm/kvm_vcpu_timer.h
@@ -12,6 +12,10 @@
#include <linux/hrtimer.h>
struct kvm_guest_timer {
+ u64 frequency;
+ bool need_scale;
+ u64 scale_mult;
+ u64 scale_shift;
/* Time delta value */
u64 time_delta;
};
@@ -38,4 +42,9 @@ int kvm_riscv_vcpu_timer_reset(struct kvm_vcpu *vcpu);
void kvm_riscv_vcpu_timer_restore(struct kvm_vcpu *vcpu);
int kvm_riscv_guest_timer_init(struct kvm *kvm);
+static inline bool kvm_riscv_need_scale(struct kvm_guest_timer *gt)
+{
+ return gt->need_scale;
+}
+
#endif
diff --git a/arch/riscv/kvm/vcpu_timer.c b/arch/riscv/kvm/vcpu_timer.c
index f6b35180199a..2d203660a7e9 100644
--- a/arch/riscv/kvm/vcpu_timer.c
+++ b/arch/riscv/kvm/vcpu_timer.c
@@ -15,9 +15,38 @@
#include <asm/delay.h>
#include <asm/kvm_vcpu_timer.h>
+#define SCALE_SHIFT_VALUE 48
+#define SCALE_TOLERANCE_HZ 1000
+
+static void kvm_riscv_set_time_freq(struct kvm_guest_timer *gt, u64 freq)
+{
+ /*
+ * Guest time frequency and Host time frequency are identical
+ * if the error between them is limited within SCALE_TOLERANCE_HZ.
+ */
+ u64 diff = riscv_timebase > freq ?
+ riscv_timebase - freq : freq - riscv_timebase;
+ gt->need_scale = (diff >= SCALE_TOLERANCE_HZ);
+ if (gt->need_scale) {
+ gt->scale_shift = SCALE_SHIFT_VALUE;
+ gt->scale_mult = mul_u64_u32_div(1ULL << gt->scale_shift,
+ freq, riscv_timebase);
+ }
+ gt->frequency = freq;
+}
+
+static u64 kvm_riscv_scale_time(struct kvm_guest_timer *gt, u64 time)
+{
+ if (kvm_riscv_need_scale(gt))
+ return mul_u64_u64_shr(time, gt->scale_mult, gt->scale_shift);
+
+ return time;
+}
+
static u64 kvm_riscv_current_cycles(struct kvm_guest_timer *gt)
{
- return get_cycles64() + gt->time_delta;
+ u64 host_time = get_cycles64();
+ return kvm_riscv_scale_time(gt, host_time) + gt->time_delta;
}
static u64 kvm_riscv_delta_cycles2ns(u64 cycles,
@@ -33,7 +62,7 @@ static u64 kvm_riscv_delta_cycles2ns(u64 cycles,
cycles_delta = cycles - cycles_now;
else
cycles_delta = 0;
- delta_ns = mul_u64_u64_div_u64(cycles_delta, NSEC_PER_SEC, riscv_timebase);
+ delta_ns = mul_u64_u64_div_u64(cycles_delta, NSEC_PER_SEC, gt->frequency);
local_irq_restore(flags);
return delta_ns;
@@ -106,7 +135,7 @@ int kvm_riscv_vcpu_get_reg_timer(struct kvm_vcpu *vcpu,
switch (reg_num) {
case KVM_REG_RISCV_TIMER_REG(frequency):
- reg_val = riscv_timebase;
+ reg_val = gt->frequency;
break;
case KVM_REG_RISCV_TIMER_REG(time):
reg_val = kvm_riscv_current_cycles(gt);
@@ -150,10 +179,10 @@ int kvm_riscv_vcpu_set_reg_timer(struct kvm_vcpu *vcpu,
switch (reg_num) {
case KVM_REG_RISCV_TIMER_REG(frequency):
- ret = -EOPNOTSUPP;
+ kvm_riscv_set_time_freq(gt, reg_val);
break;
case KVM_REG_RISCV_TIMER_REG(time):
- gt->time_delta = reg_val - get_cycles64();
+ gt->time_delta = reg_val - kvm_riscv_scale_time(gt, get_cycles64());
break;
case KVM_REG_RISCV_TIMER_REG(compare):
t->next_cycles = reg_val;
@@ -219,6 +248,7 @@ int kvm_riscv_guest_timer_init(struct kvm *kvm)
struct kvm_guest_timer *gt = &kvm->arch.timer;
gt->time_delta = -get_cycles64();
+ gt->frequency = riscv_timebase;
return 0;
}
--
2.19.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH RFC 3/3] RISC-V: KVM: Implement guest time scaling
2020-12-03 12:18 [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM Yifei Jiang
2020-12-03 12:18 ` [PATCH RFC 1/3] RISC-V: KVM: Change the method of calculating cycles to nanoseconds Yifei Jiang
2020-12-03 12:18 ` [PATCH RFC 2/3] RISC-V: KVM: Support dynamic time frequency from userspace Yifei Jiang
@ 2020-12-03 12:18 ` Yifei Jiang
2020-12-16 6:39 ` [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM Anup Patel
3 siblings, 0 replies; 6+ messages in thread
From: Yifei Jiang @ 2020-12-03 12:18 UTC (permalink / raw)
To: anup.patel, atish.patra, paul.walmsley, palmer, aou, pbonzini
Cc: kvm-riscv, kvm, linux-riscv, linux-kernel, victor.zhangxiaofeng,
wu.wubin, zhang.zhanghailiang, dengkai1, yinyipeng1, Yifei Jiang
When time frequency needs to scale, RDTIME/RDTIMEH instruction in guest
doesn't work correctly. Because it still uses the host's time frequency.
To read correct time, the RDTIME/RDTIMEH instruction executed by guest
should trap to HS-mode. The TM bit of HCOUNTEREN CSR could control whether
these instructions are trapped to HS-mode. Therefore, we can implement guest
time scaling by setting TM bit in kvm_riscv_vcpu_timer_restore() and emulating
RDTIME/RDTIMEH instruction in system_opcode_insn().
Signed-off-by: Yifei Jiang <jiangyifei@huawei.com>
Signed-off-by: Yipeng Yin <yinyipeng1@huawei.com>
---
arch/riscv/include/asm/csr.h | 3 +++
arch/riscv/include/asm/kvm_vcpu_timer.h | 1 +
arch/riscv/kvm/vcpu_exit.c | 35 +++++++++++++++++++++++++
arch/riscv/kvm/vcpu_timer.c | 10 +++++++
4 files changed, 49 insertions(+)
diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h
index bc825693e0e3..a4d8ca76cf1d 100644
--- a/arch/riscv/include/asm/csr.h
+++ b/arch/riscv/include/asm/csr.h
@@ -241,6 +241,9 @@
#define IE_TIE (_AC(0x1, UL) << RV_IRQ_TIMER)
#define IE_EIE (_AC(0x1, UL) << RV_IRQ_EXT)
+/* The counteren flag */
+#define CE_TM 1
+
#ifndef __ASSEMBLY__
#define csr_swap(csr, val) \
diff --git a/arch/riscv/include/asm/kvm_vcpu_timer.h b/arch/riscv/include/asm/kvm_vcpu_timer.h
index 41b5503de9e4..61384eb57334 100644
--- a/arch/riscv/include/asm/kvm_vcpu_timer.h
+++ b/arch/riscv/include/asm/kvm_vcpu_timer.h
@@ -41,6 +41,7 @@ int kvm_riscv_vcpu_timer_deinit(struct kvm_vcpu *vcpu);
int kvm_riscv_vcpu_timer_reset(struct kvm_vcpu *vcpu);
void kvm_riscv_vcpu_timer_restore(struct kvm_vcpu *vcpu);
int kvm_riscv_guest_timer_init(struct kvm *kvm);
+u64 kvm_riscv_read_guest_time(struct kvm_vcpu *vcpu);
static inline bool kvm_riscv_need_scale(struct kvm_guest_timer *gt)
{
diff --git a/arch/riscv/kvm/vcpu_exit.c b/arch/riscv/kvm/vcpu_exit.c
index f054406792a6..4beb9d25049a 100644
--- a/arch/riscv/kvm/vcpu_exit.c
+++ b/arch/riscv/kvm/vcpu_exit.c
@@ -18,6 +18,10 @@
#define INSN_MASK_WFI 0xffffff00
#define INSN_MATCH_WFI 0x10500000
+#define INSN_MASK_RDTIME 0xfff03000
+#define INSN_MATCH_RDTIME 0xc0102000
+#define INSN_MASK_RDTIMEH 0xfff03000
+#define INSN_MATCH_RDTIMEH 0xc8102000
#define INSN_MATCH_LB 0x3
#define INSN_MASK_LB 0x707f
@@ -138,6 +142,34 @@ static int truly_illegal_insn(struct kvm_vcpu *vcpu,
return 1;
}
+static int system_opcode_insn_rdtime(struct kvm_vcpu *vcpu,
+ struct kvm_run *run,
+ ulong insn)
+{
+#ifdef CONFIG_64BIT
+ if ((insn & INSN_MASK_RDTIME) == INSN_MATCH_RDTIME) {
+ u64 guest_time = kvm_riscv_read_guest_time(vcpu);
+ SET_RD(insn, &vcpu->arch.guest_context, guest_time);
+ vcpu->arch.guest_context.sepc += INSN_LEN(insn);
+ return 1;
+ }
+#else
+ if ((insn & INSN_MASK_RDTIME) == INSN_MATCH_RDTIME) {
+ u64 guest_time = kvm_riscv_read_guest_time(vcpu);
+ SET_RD(insn, &vcpu->arch.guest_context, (u32)guest_time);
+ vcpu->arch.guest_context.sepc += INSN_LEN(insn);
+ return 1;
+ }
+ if ((insn & INSN_MASK_RDTIMEH) == INSN_MATCH_RDTIMEH) {
+ u64 guest_time = kvm_riscv_read_guest_time(vcpu);
+ SET_RD(insn, &vcpu->arch.guest_context, (u32)(guest_time >> 32));
+ vcpu->arch.guest_context.sepc += INSN_LEN(insn);
+ return 1;
+ }
+#endif
+ return 0;
+}
+
static int system_opcode_insn(struct kvm_vcpu *vcpu,
struct kvm_run *run,
ulong insn)
@@ -154,6 +186,9 @@ static int system_opcode_insn(struct kvm_vcpu *vcpu,
return 1;
}
+ if (system_opcode_insn_rdtime(vcpu, run, insn))
+ return 1;
+
return truly_illegal_insn(vcpu, run, insn);
}
diff --git a/arch/riscv/kvm/vcpu_timer.c b/arch/riscv/kvm/vcpu_timer.c
index 2d203660a7e9..2040dbe57ee6 100644
--- a/arch/riscv/kvm/vcpu_timer.c
+++ b/arch/riscv/kvm/vcpu_timer.c
@@ -49,6 +49,11 @@ static u64 kvm_riscv_current_cycles(struct kvm_guest_timer *gt)
return kvm_riscv_scale_time(gt, host_time) + gt->time_delta;
}
+u64 kvm_riscv_read_guest_time(struct kvm_vcpu *vcpu)
+{
+ return kvm_riscv_current_cycles(&vcpu->kvm->arch.timer);
+}
+
static u64 kvm_riscv_delta_cycles2ns(u64 cycles,
struct kvm_guest_timer *gt,
struct kvm_vcpu_timer *t)
@@ -241,6 +246,11 @@ void kvm_riscv_vcpu_timer_restore(struct kvm_vcpu *vcpu)
csr_write(CSR_HTIMEDELTA, (u32)(gt->time_delta));
csr_write(CSR_HTIMEDELTAH, (u32)(gt->time_delta >> 32));
#endif
+
+ if (kvm_riscv_need_scale(gt))
+ csr_clear(CSR_HCOUNTEREN, 1UL << CE_TM);
+ else
+ csr_set(CSR_HCOUNTEREN, 1UL << CE_TM);
}
int kvm_riscv_guest_timer_init(struct kvm *kvm)
--
2.19.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM
2020-12-03 12:18 [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM Yifei Jiang
` (2 preceding siblings ...)
2020-12-03 12:18 ` [PATCH RFC 3/3] RISC-V: KVM: Implement guest time scaling Yifei Jiang
@ 2020-12-16 6:39 ` Anup Patel
2020-12-16 8:01 ` Jiangyifei
3 siblings, 1 reply; 6+ messages in thread
From: Anup Patel @ 2020-12-16 6:39 UTC (permalink / raw)
To: Yifei Jiang
Cc: Anup Patel, Atish Patra, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Paolo Bonzini, Zhanghailiang, KVM General, yinyipeng,
Zhangxiaofeng (F),
linux-kernel@vger.kernel.org List, kvm-riscv, linux-riscv,
Wubin (H), dengkai (A)
On Thu, Dec 3, 2020 at 5:51 PM Yifei Jiang <jiangyifei@huawei.com> wrote:
>
> This series implements guest time scaling based on RDTIME instruction
> emulation so that we can allow migrating Guest/VM across Hosts with
> different time frequency.
>
> Why not through para-virt. From arm's experience[1], para-virt implementation
> doesn't really solve the problem for the following two main reasons:
> - RDTIME not only be used in linux, but also in firmware and userspace.
> - It is difficult to be compatible with nested virtualization.
I think this approach is rather incomplete. Also, I don't see how para-virt
time scaling will be difficult for nested virtualization.
If trap-n-emulate TIME CSR for Guest Linux then it will have significant
performance impact of systems where TIME CSR is implemented in HW.
Best approach will be to have VDSO-style para-virt time-scale SBI calls
(similar to what KVM x86 does). If the Guest software (Linux/Bootloader)
does not enable para-virt time-scaling then we trap-n-emulate TIME CSR
(this series).
Please propose VDSO-style para-virt time-scale SBI call and expand this
this series to provide both:
1. VDSO-style para-virt time-scaling
2. Trap-n-emulation of TIME CSR when #1 is disabled
Regards,
Anup
>
> [1] https://lore.kernel.org/patchwork/cover/1288153/
>
> Yifei Jiang (3):
> RISC-V: KVM: Change the method of calculating cycles to nanoseconds
> RISC-V: KVM: Support dynamic time frequency from userspace
> RISC-V: KVM: Implement guest time scaling
>
> arch/riscv/include/asm/csr.h | 3 ++
> arch/riscv/include/asm/kvm_vcpu_timer.h | 13 +++++--
> arch/riscv/kvm/vcpu_exit.c | 35 +++++++++++++++++
> arch/riscv/kvm/vcpu_timer.c | 51 ++++++++++++++++++++++---
> 4 files changed, 93 insertions(+), 9 deletions(-)
>
> --
> 2.19.1
>
>
> --
> kvm-riscv mailing list
> kvm-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 6+ messages in thread* RE: [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM
2020-12-16 6:39 ` [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM Anup Patel
@ 2020-12-16 8:01 ` Jiangyifei
0 siblings, 0 replies; 6+ messages in thread
From: Jiangyifei @ 2020-12-16 8:01 UTC (permalink / raw)
To: Anup Patel
Cc: Anup Patel, Atish Patra, Paul Walmsley, Palmer Dabbelt,
Albert Ou, Paolo Bonzini, Zhanghailiang, KVM General, yinyipeng,
Zhangxiaofeng (F),
linux-kernel@vger.kernel.org List, kvm-riscv, linux-riscv,
Wubin (H), dengkai (A)
> -----Original Message-----
> From: Anup Patel [mailto:anup@brainfault.org]
> Sent: Wednesday, December 16, 2020 2:40 PM
> To: Jiangyifei <jiangyifei@huawei.com>
> Cc: Anup Patel <anup.patel@wdc.com>; Atish Patra <atish.patra@wdc.com>;
> Paul Walmsley <paul.walmsley@sifive.com>; Palmer Dabbelt
> <palmer@dabbelt.com>; Albert Ou <aou@eecs.berkeley.edu>; Paolo Bonzini
> <pbonzini@redhat.com>; Zhanghailiang <zhang.zhanghailiang@huawei.com>;
> KVM General <kvm@vger.kernel.org>; yinyipeng <yinyipeng1@huawei.com>;
> Zhangxiaofeng (F) <victor.zhangxiaofeng@huawei.com>;
> linux-kernel@vger.kernel.org List <linux-kernel@vger.kernel.org>;
> kvm-riscv@lists.infradead.org; linux-riscv <linux-riscv@lists.infradead.org>;
> Wubin (H) <wu.wubin@huawei.com>; dengkai (A) <dengkai1@huawei.com>
> Subject: Re: [PATCH RFC 0/3] Implement guest time scaling in RISC-V KVM
>
> On Thu, Dec 3, 2020 at 5:51 PM Yifei Jiang <jiangyifei@huawei.com> wrote:
> >
> > This series implements guest time scaling based on RDTIME instruction
> > emulation so that we can allow migrating Guest/VM across Hosts with
> > different time frequency.
> >
> > Why not through para-virt. From arm's experience[1], para-virt
> > implementation doesn't really solve the problem for the following two main
> reasons:
> > - RDTIME not only be used in linux, but also in firmware and userspace.
> > - It is difficult to be compatible with nested virtualization.
>
> I think this approach is rather incomplete. Also, I don't see how para-virt time
> scaling will be difficult for nested virtualization.
>
> If trap-n-emulate TIME CSR for Guest Linux then it will have significant
> performance impact of systems where TIME CSR is implemented in HW.
>
> Best approach will be to have VDSO-style para-virt time-scale SBI calls (similar
> to what KVM x86 does). If the Guest software (Linux/Bootloader) does not
> enable para-virt time-scaling then we trap-n-emulate TIME CSR (this series).
>
> Please propose VDSO-style para-virt time-scale SBI call and expand this this
> series to provide both:
> 1. VDSO-style para-virt time-scaling
> 2. Trap-n-emulation of TIME CSR when #1 is disabled
>
> Regards,
> Anup
>
OK, it sounds good. We will look into the para-virt time-scaling for more details.
Yifei
> >
> > [1] https://lore.kernel.org/patchwork/cover/1288153/
> >
> > Yifei Jiang (3):
> > RISC-V: KVM: Change the method of calculating cycles to nanoseconds
> > RISC-V: KVM: Support dynamic time frequency from userspace
> > RISC-V: KVM: Implement guest time scaling
> >
> > arch/riscv/include/asm/csr.h | 3 ++
> > arch/riscv/include/asm/kvm_vcpu_timer.h | 13 +++++--
> > arch/riscv/kvm/vcpu_exit.c | 35 +++++++++++++++++
> > arch/riscv/kvm/vcpu_timer.c | 51
> ++++++++++++++++++++++---
> > 4 files changed, 93 insertions(+), 9 deletions(-)
> >
> > --
> > 2.19.1
> >
> >
> > --
> > kvm-riscv mailing list
> > kvm-riscv@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kvm-riscv
^ permalink raw reply [flat|nested] 6+ messages in thread