* [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd
@ 2011-09-30 12:48 Philipp Reisner
2011-09-30 12:48 ` [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now Philipp Reisner
` (10 more replies)
0 siblings, 11 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
This the first request for review of drbd-8.4. The complete set has
471 patches. This is the twelfth installment containing 11 patches.
The whole set is available here:
git://git.drbd.org/linux-2.6-drbd.git for-jens
and is jens_for-3.2_drivers...for-jens
and this part is f23bb45^...ef9bce0
This patch series contains:
* Start to use RCU for the objects containing the configuration.
(Changed seldom, read accessed frequently)
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 02/11] drbd: Inlined drbd_free_mdev(); it got called only from one place Philipp Reisner
` (9 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 2 +-
drivers/block/drbd/drbd_main.c | 12 ++++--------
drivers/block/drbd/drbd_nl.c | 2 +-
3 files changed, 6 insertions(+), 10 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 9aca1f4..ce56aa0 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -1367,7 +1367,7 @@ extern rwlock_t global_state_lock;
extern int conn_lowest_minor(struct drbd_tconn *tconn);
enum drbd_ret_code conn_new_minor(struct drbd_tconn *tconn, unsigned int minor, int vnr);
extern void drbd_free_mdev(struct drbd_conf *mdev);
-extern void drbd_delete_device(unsigned int minor);
+extern void drbd_delete_device(struct drbd_conf *mdev);
struct drbd_tconn *drbd_new_tconn(const char *name);
extern void drbd_free_tconn(struct drbd_tconn *tconn);
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 8097d7a..8466ee7 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2195,15 +2195,10 @@ static void drbd_release_all_peer_reqs(struct drbd_conf *mdev)
}
/* caution. no locking. */
-void drbd_delete_device(unsigned int minor)
+void drbd_delete_device(struct drbd_conf *mdev)
{
- struct drbd_conf *mdev = minor_to_mdev(minor);
-
- if (!mdev)
- return;
-
idr_remove(&mdev->tconn->volumes, mdev->vnr);
- idr_remove(&minors, minor);
+ idr_remove(&minors, mdev_to_minor(mdev));
synchronize_rcu();
/* paranoia asserts */
@@ -2256,7 +2251,8 @@ static void drbd_cleanup(void)
drbd_genl_unregister();
idr_for_each_entry(&minors, mdev, i)
- drbd_delete_device(i);
+ drbd_delete_device(mdev);
+
drbd_destroy_mempools();
unregister_blkdev(DRBD_MAJOR, "drbd");
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 4b60673..be2ca81 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -2916,7 +2916,7 @@ static enum drbd_ret_code adm_delete_minor(struct drbd_conf *mdev)
* we may want to delete a minor from a live replication group.
*/
mdev->state.role == R_SECONDARY) {
- drbd_delete_device(mdev_to_minor(mdev));
+ drbd_delete_device(mdev);
return NO_ERROR;
} else
return ERR_MINOR_CONFIGURED;
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 02/11] drbd: Inlined drbd_free_mdev(); it got called only from one place
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
2011-09-30 12:48 ` [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 03/11] drbd: rcu_read_[un]lock() for all idr accesses that do not sleep Philipp Reisner
` (8 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 1 -
drivers/block/drbd/drbd_main.c | 25 +++++++------------------
2 files changed, 7 insertions(+), 19 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index ce56aa0..c49dc08 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -1366,7 +1366,6 @@ extern rwlock_t global_state_lock;
extern int conn_lowest_minor(struct drbd_tconn *tconn);
enum drbd_ret_code conn_new_minor(struct drbd_tconn *tconn, unsigned int minor, int vnr);
-extern void drbd_free_mdev(struct drbd_conf *mdev);
extern void drbd_delete_device(struct drbd_conf *mdev);
struct drbd_tconn *drbd_new_tconn(const char *name);
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 8466ee7..03b3ffe 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2224,10 +2224,13 @@ void drbd_delete_device(struct drbd_conf *mdev)
kfree(mdev->p_uuid);
/* mdev->p_uuid = NULL; */
- /* cleanup the rest that has been
- * allocated from drbd_new_device
- * and actually free the mdev itself */
- drbd_free_mdev(mdev);
+ kfree(mdev->current_epoch);
+ if (mdev->bitmap) /* should no longer be there. */
+ drbd_bm_cleanup(mdev);
+ __free_page(mdev->md_io_page);
+ put_disk(mdev->vdisk);
+ blk_cleanup_queue(mdev->rq_queue);
+ kfree(mdev);
}
static void drbd_cleanup(void)
@@ -2542,20 +2545,6 @@ out_no_q:
return err;
}
-/* counterpart of drbd_new_device.
- * last part of drbd_delete_device. */
-void drbd_free_mdev(struct drbd_conf *mdev)
-{
- kfree(mdev->current_epoch);
- if (mdev->bitmap) /* should no longer be there. */
- drbd_bm_cleanup(mdev);
- __free_page(mdev->md_io_page);
- put_disk(mdev->vdisk);
- blk_cleanup_queue(mdev->rq_queue);
- kfree(mdev);
-}
-
-
int __init drbd_init(void)
{
int err;
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 03/11] drbd: rcu_read_[un]lock() for all idr accesses that do not sleep
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
2011-09-30 12:48 ` [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now Philipp Reisner
2011-09-30 12:48 ` [PATCH 02/11] drbd: Inlined drbd_free_mdev(); it got called only from one place Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 04/11] drbd: Converted drbd_cfg_mutex into drbd_cfg_rwsem Philipp Reisner
` (7 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_main.c | 12 +++++++---
drivers/block/drbd/drbd_nl.c | 41 +++++++++++++++++++++++++----------
drivers/block/drbd/drbd_receiver.c | 2 +
drivers/block/drbd/drbd_state.c | 34 +++++++++++++++++++++++++++--
drivers/block/drbd/drbd_worker.c | 4 +++
5 files changed, 74 insertions(+), 19 deletions(-)
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 03b3ffe..c4c0bc5 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -453,8 +453,10 @@ void tl_clear(struct drbd_tconn *tconn)
}
/* ensure bit indicating barrier is required is clear */
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
clear_bit(CREATE_BARRIER, &mdev->flags);
+ rcu_read_unlock();
spin_unlock_irq(&tconn->req_lock);
}
@@ -634,13 +636,15 @@ char *drbd_task_to_thread_name(struct drbd_tconn *tconn, struct task_struct *tas
int conn_lowest_minor(struct drbd_tconn *tconn)
{
- int vnr = 0;
struct drbd_conf *mdev;
+ int vnr = 0, m;
+ rcu_read_lock();
mdev = idr_get_next(&tconn->volumes, &vnr);
- if (!mdev)
- return -1;
- return mdev_to_minor(mdev);
+ m = mdev ? mdev_to_minor(mdev) : -1;
+ rcu_read_unlock();
+
+ return m;
}
#ifdef CONFIG_SMP
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index be2ca81..fc918d0 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -407,12 +407,14 @@ static enum drbd_fencing_p highest_fencing_policy(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (get_ldev_if_state(mdev, D_CONSISTENT)) {
fp = max_t(enum drbd_fencing_p, fp, mdev->ldev->dc.fencing);
put_ldev(mdev);
}
}
+ rcu_read_unlock();
return fp;
}
@@ -1664,29 +1666,41 @@ out:
static bool conn_resync_running(struct drbd_tconn *tconn)
{
struct drbd_conf *mdev;
+ bool rv = false;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (mdev->state.conn == C_SYNC_SOURCE ||
mdev->state.conn == C_SYNC_TARGET ||
mdev->state.conn == C_PAUSED_SYNC_S ||
- mdev->state.conn == C_PAUSED_SYNC_T)
- return true;
+ mdev->state.conn == C_PAUSED_SYNC_T) {
+ rv = true;
+ break;
+ }
}
- return false;
+ rcu_read_unlock();
+
+ return rv;
}
static bool conn_ov_running(struct drbd_tconn *tconn)
{
struct drbd_conf *mdev;
+ bool rv = false;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (mdev->state.conn == C_VERIFY_S ||
- mdev->state.conn == C_VERIFY_T)
- return true;
+ mdev->state.conn == C_VERIFY_T) {
+ rv = true;
+ break;
+ }
}
- return false;
+ rcu_read_unlock();
+
+ return rv;
}
int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
@@ -1912,26 +1926,28 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
goto fail;
}
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
if (get_ldev(mdev)) {
enum drbd_fencing_p fp = mdev->ldev->dc.fencing;
put_ldev(mdev);
if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH) {
retcode = ERR_STONITH_AND_PROT_A;
- goto fail;
+ goto fail_rcu_unlock;
}
}
if (mdev->state.role == R_PRIMARY && new_conf->want_lose) {
retcode = ERR_DISCARD;
- goto fail;
+ goto fail_rcu_unlock;
}
if (!mdev->bitmap) {
if(drbd_bm_init(mdev)) {
retcode = ERR_NOMEM;
- goto fail;
+ goto fail_rcu_unlock;
}
}
}
+ rcu_read_unlock();
if (new_conf->on_congestion != OC_BLOCK && new_conf->wire_protocol != DRBD_PROT_A) {
retcode = ERR_CONG_NOT_PROTO_A;
@@ -2045,15 +2061,19 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
retcode = _conn_request_state(tconn, NS(conn, C_UNCONNECTED), CS_VERBOSE);
spin_unlock_irq(&tconn->req_lock);
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
mdev->send_cnt = 0;
mdev->recv_cnt = 0;
kobject_uevent(&disk_to_dev(mdev->vdisk)->kobj, KOBJ_CHANGE);
}
+ rcu_read_unlock();
conn_reconfig_done(tconn);
drbd_adm_finish(info, retcode);
return 0;
+fail_rcu_unlock:
+ rcu_read_unlock();
fail:
kfree(int_dig_in);
kfree(int_dig_vv);
@@ -2616,8 +2636,6 @@ int drbd_adm_get_status_all(struct sk_buff *skb, struct netlink_callback *cb)
/* synchronize with drbd_new_tconn/drbd_free_tconn */
mutex_lock(&drbd_cfg_mutex);
- /* synchronize with drbd_delete_device */
- rcu_read_lock();
next_tconn:
/* revalidate iterator position */
list_for_each_entry(tmp, &drbd_tconns, all_tconn) {
@@ -2680,7 +2698,6 @@ next_tconn:
}
out:
- rcu_read_unlock();
mutex_unlock(&drbd_cfg_mutex);
/* where to start the next iteration */
cb->args[0] = (long)pos;
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 0c45eba..92ecb22 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -4828,11 +4828,13 @@ static int tconn_finish_peer_reqs(struct drbd_tconn *tconn)
set_bit(SIGNAL_ASENDER, &tconn->flags);
spin_lock_irq(&tconn->req_lock);
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
not_empty = !list_empty(&mdev->done_ee);
if (not_empty)
break;
}
+ rcu_read_unlock();
spin_unlock_irq(&tconn->req_lock);
} while (not_empty);
diff --git a/drivers/block/drbd/drbd_state.c b/drivers/block/drbd/drbd_state.c
index 694bd3f..0aed33b 100644
--- a/drivers/block/drbd/drbd_state.c
+++ b/drivers/block/drbd/drbd_state.c
@@ -56,15 +56,21 @@ static inline bool is_susp(union drbd_state s)
bool conn_all_vols_unconf(struct drbd_tconn *tconn)
{
struct drbd_conf *mdev;
+ bool rv = true;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (mdev->state.disk != D_DISKLESS ||
mdev->state.conn != C_STANDALONE ||
- mdev->state.role != R_SECONDARY)
- return false;
+ mdev->state.role != R_SECONDARY) {
+ rv = false;
+ break;
+ }
}
- return true;
+ rcu_read_unlock();
+
+ return rv;
}
enum drbd_role conn_highest_role(struct drbd_tconn *tconn)
@@ -73,8 +79,10 @@ enum drbd_role conn_highest_role(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
role = max_t(enum drbd_role, role, mdev->state.role);
+ rcu_read_unlock();
return role;
}
@@ -85,8 +93,10 @@ enum drbd_role conn_highest_peer(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
peer = max_t(enum drbd_role, peer, mdev->state.peer);
+ rcu_read_unlock();
return peer;
}
@@ -97,8 +107,10 @@ enum drbd_disk_state conn_highest_disk(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
ds = max_t(enum drbd_disk_state, ds, mdev->state.disk);
+ rcu_read_unlock();
return ds;
}
@@ -109,8 +121,10 @@ enum drbd_disk_state conn_lowest_disk(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
ds = min_t(enum drbd_disk_state, ds, mdev->state.disk);
+ rcu_read_unlock();
return ds;
}
@@ -121,8 +135,10 @@ enum drbd_disk_state conn_highest_pdsk(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
ds = max_t(enum drbd_disk_state, ds, mdev->state.pdsk);
+ rcu_read_unlock();
return ds;
}
@@ -133,8 +149,10 @@ enum drbd_disk_state conn_lowest_conn(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
conn = min_t(enum drbd_conns, conn, mdev->state.conn);
+ rcu_read_unlock();
return conn;
}
@@ -1388,12 +1406,14 @@ static int w_after_conn_state_ch(struct drbd_work *w, int unused)
/* case1: The outdate peer handler is successful: */
if (ns_max.pdsk <= D_OUTDATED) {
tl_clear(tconn);
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
if (test_bit(NEW_CUR_UUID, &mdev->flags)) {
drbd_uuid_new_current(mdev);
clear_bit(NEW_CUR_UUID, &mdev->flags);
}
}
+ rcu_read_unlock();
conn_request_state(tconn,
(union drbd_state) { { .susp_fen = 1 } },
(union drbd_state) { { .susp_fen = 0 } },
@@ -1401,8 +1421,10 @@ static int w_after_conn_state_ch(struct drbd_work *w, int unused)
}
/* case2: The connection was established again: */
if (ns_min.conn >= C_CONNECTED) {
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr)
clear_bit(NEW_CUR_UUID, &mdev->flags);
+ rcu_read_unlock();
spin_lock_irq(&tconn->req_lock);
_tl_restart(tconn, RESEND);
_conn_request_state(tconn,
@@ -1427,6 +1449,7 @@ void conn_old_common_state(struct drbd_tconn *tconn, union drbd_state *pcs, enum
struct drbd_conf *mdev;
int vnr, first_vol = 1;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
os = mdev->state;
@@ -1451,6 +1474,7 @@ void conn_old_common_state(struct drbd_tconn *tconn, union drbd_state *pcs, enum
if (cs.pdsk != os.pdsk)
flags &= ~CS_DC_PDSK;
}
+ rcu_read_unlock();
*pf |= CS_DC_MASK;
*pf &= flags;
@@ -1466,6 +1490,7 @@ conn_is_valid_transition(struct drbd_tconn *tconn, union drbd_state mask, union
struct drbd_conf *mdev;
int vnr;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
os = drbd_read_state(mdev);
ns = sanitize_state(mdev, apply_mask_val(os, mask, val), NULL);
@@ -1491,6 +1516,7 @@ conn_is_valid_transition(struct drbd_tconn *tconn, union drbd_state mask, union
if (rv < SS_SUCCESS)
break;
}
+ rcu_read_unlock();
if (rv < SS_SUCCESS && flags & CS_VERBOSE)
print_st_err(mdev, os, ns, rv);
@@ -1516,6 +1542,7 @@ conn_set_state(struct drbd_tconn *tconn, union drbd_state mask, union drbd_state
if (mask.conn == C_MASK)
tconn->cstate = val.conn;
+ rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
os = drbd_read_state(mdev);
ns = apply_mask_val(os, mask, val);
@@ -1541,6 +1568,7 @@ conn_set_state(struct drbd_tconn *tconn, union drbd_state mask, union drbd_state
ns_min.disk = min_t(enum drbd_disk_state, ns.disk, ns_min.disk);
ns_min.pdsk = min_t(enum drbd_disk_state, ns.pdsk, ns_min.pdsk);
}
+ rcu_read_unlock();
ns_min.susp = ns_max.susp = tconn->susp;
ns_min.susp_nod = ns_max.susp_nod = tconn->susp_nod;
diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c
index 62536bc..536eb05 100644
--- a/drivers/block/drbd/drbd_worker.c
+++ b/drivers/block/drbd/drbd_worker.c
@@ -1350,6 +1350,7 @@ static int _drbd_pause_after(struct drbd_conf *mdev)
struct drbd_conf *odev;
int i, rv = 0;
+ rcu_read_lock();
idr_for_each_entry(&minors, odev, i) {
if (odev->state.conn == C_STANDALONE && odev->state.disk == D_DISKLESS)
continue;
@@ -1357,6 +1358,7 @@ static int _drbd_pause_after(struct drbd_conf *mdev)
rv |= (__drbd_set_state(_NS(odev, aftr_isp, 1), CS_HARD, NULL)
!= SS_NOTHING_TO_DO);
}
+ rcu_read_unlock();
return rv;
}
@@ -1372,6 +1374,7 @@ static int _drbd_resume_next(struct drbd_conf *mdev)
struct drbd_conf *odev;
int i, rv = 0;
+ rcu_read_lock();
idr_for_each_entry(&minors, odev, i) {
if (odev->state.conn == C_STANDALONE && odev->state.disk == D_DISKLESS)
continue;
@@ -1382,6 +1385,7 @@ static int _drbd_resume_next(struct drbd_conf *mdev)
!= SS_NOTHING_TO_DO) ;
}
}
+ rcu_read_unlock();
return rv;
}
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 04/11] drbd: Converted drbd_cfg_mutex into drbd_cfg_rwsem
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (2 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 03/11] drbd: rcu_read_[un]lock() for all idr accesses that do not sleep Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 05/11] drbd: protect all idr accesses that might sleep with drbd_cfg_rwsem Philipp Reisner
` (6 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 4 +++-
drivers/block/drbd/drbd_main.c | 10 +++++-----
drivers/block/drbd/drbd_nl.c | 27 ++++++++++++++++-----------
3 files changed, 24 insertions(+), 17 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index c49dc08..7896a64 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -171,7 +171,9 @@ drbd_insert_fault(struct drbd_conf *mdev, unsigned int type) {
extern struct ratelimit_state drbd_ratelimit_state;
extern struct idr minors;
extern struct list_head drbd_tconns;
-extern struct mutex drbd_cfg_mutex;
+extern struct rw_semaphore drbd_cfg_rwsem;
+/* drbd_cfg_rwsem protects: drbd_tconns list,
+ note: non sleeping iterations over the idrs are protoected by RCU */
/* on the wire */
enum drbd_packet {
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index c4c0bc5..d48efb4 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -120,7 +120,7 @@ module_param_string(usermode_helper, usermode_helper, sizeof(usermode_helper), 0
*/
struct idr minors;
struct list_head drbd_tconns; /* list of struct drbd_tconn */
-DEFINE_MUTEX(drbd_cfg_mutex);
+DECLARE_RWSEM(drbd_cfg_rwsem);
struct kmem_cache *drbd_request_cache;
struct kmem_cache *drbd_ee_cache; /* peer requests */
@@ -2321,14 +2321,14 @@ struct drbd_tconn *conn_by_name(const char *name)
if (!name || !name[0])
return NULL;
- mutex_lock(&drbd_cfg_mutex);
+ down_read(&drbd_cfg_rwsem);
list_for_each_entry(tconn, &drbd_tconns, all_tconn) {
if (!strcmp(tconn->name, name))
goto found;
}
tconn = NULL;
found:
- mutex_unlock(&drbd_cfg_mutex);
+ up_read(&drbd_cfg_rwsem);
return tconn;
}
@@ -2395,9 +2395,9 @@ struct drbd_tconn *drbd_new_tconn(const char *name)
DRBD_ON_NO_DATA_DEF, /* on_no_data */
};
- mutex_lock(&drbd_cfg_mutex);
+ down_write(&drbd_cfg_rwsem);
list_add_tail(&tconn->all_tconn, &drbd_tconns);
- mutex_unlock(&drbd_cfg_mutex);
+ up_write(&drbd_cfg_rwsem);
return tconn;
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index fc918d0..5667536 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -1959,7 +1959,7 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
new_my_addr = (struct sockaddr *)&new_conf->my_addr;
new_peer_addr = (struct sockaddr *)&new_conf->peer_addr;
- /* No need to take drbd_cfg_mutex here. All reconfiguration is
+ /* No need to take drbd_cfg_rwsem here. All reconfiguration is
* strictly serialized on genl_lock(). We are protected against
* concurrent reconfiguration/addition/deletion */
list_for_each_entry(oconn, &drbd_tconns, all_tconn) {
@@ -2635,7 +2635,7 @@ int drbd_adm_get_status_all(struct sk_buff *skb, struct netlink_callback *cb)
*/
/* synchronize with drbd_new_tconn/drbd_free_tconn */
- mutex_lock(&drbd_cfg_mutex);
+ down_read(&drbd_cfg_rwsem);
next_tconn:
/* revalidate iterator position */
list_for_each_entry(tmp, &drbd_tconns, all_tconn) {
@@ -2698,7 +2698,7 @@ next_tconn:
}
out:
- mutex_unlock(&drbd_cfg_mutex);
+ up_read(&drbd_cfg_rwsem);
/* where to start the next iteration */
cb->args[0] = (long)pos;
cb->args[1] = (pos == tconn) ? volume + 1 : 0;
@@ -2949,9 +2949,9 @@ int drbd_adm_delete_minor(struct sk_buff *skb, struct genl_info *info)
if (retcode != NO_ERROR)
goto out;
- mutex_lock(&drbd_cfg_mutex);
+ down_write(&drbd_cfg_rwsem);
retcode = adm_delete_minor(adm_ctx.mdev);
- mutex_unlock(&drbd_cfg_mutex);
+ up_write(&drbd_cfg_rwsem);
/* if this was the last volume of this connection,
* this will terminate all threads */
if (retcode == NO_ERROR)
@@ -2979,7 +2979,7 @@ int drbd_adm_down(struct sk_buff *skb, struct genl_info *info)
goto out;
}
- mutex_lock(&drbd_cfg_mutex);
+ down_read(&drbd_cfg_rwsem);
/* demote */
idr_for_each_entry(&adm_ctx.tconn->volumes, mdev, i) {
retcode = drbd_set_role(mdev, R_SECONDARY, 0);
@@ -3006,14 +3006,17 @@ int drbd_adm_down(struct sk_buff *skb, struct genl_info *info)
goto out_unlock;
}
}
+ up_read(&drbd_cfg_rwsem);
/* delete volumes */
+ down_write(&drbd_cfg_rwsem);
idr_for_each_entry(&adm_ctx.tconn->volumes, mdev, i) {
retcode = adm_delete_minor(mdev);
if (retcode != NO_ERROR) {
/* "can not happen" */
drbd_msg_put_info("failed to delete volume");
- goto out_unlock;
+ up_write(&drbd_cfg_rwsem);
+ goto out;
}
}
@@ -3028,10 +3031,12 @@ int drbd_adm_down(struct sk_buff *skb, struct genl_info *info)
/* "can not happen" */
retcode = ERR_CONN_IN_USE;
drbd_msg_put_info("failed to delete connection");
- goto out_unlock;
}
+
+ up_write(&drbd_cfg_rwsem);
+ goto out;
out_unlock:
- mutex_unlock(&drbd_cfg_mutex);
+ up_read(&drbd_cfg_rwsem);
out:
drbd_adm_finish(info, retcode);
return 0;
@@ -3047,14 +3052,14 @@ int drbd_adm_delete_connection(struct sk_buff *skb, struct genl_info *info)
if (retcode != NO_ERROR)
goto out;
- mutex_lock(&drbd_cfg_mutex);
+ down_write(&drbd_cfg_rwsem);
if (conn_lowest_minor(adm_ctx.tconn) < 0) {
drbd_free_tconn(adm_ctx.tconn);
retcode = NO_ERROR;
} else {
retcode = ERR_CONN_IN_USE;
}
- mutex_unlock(&drbd_cfg_mutex);
+ up_write(&drbd_cfg_rwsem);
out:
drbd_adm_finish(info, retcode);
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 05/11] drbd: protect all idr accesses that might sleep with drbd_cfg_rwsem
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (3 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 04/11] drbd: Converted drbd_cfg_mutex into drbd_cfg_rwsem Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 06/11] drbd: Runtime changeable wire protocol Philipp Reisner
` (5 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
With this commit the locking for all accesses to IDRs is complete:
* Non sleeping read accesses are protected by RCU
* sleeping read accesses are protocted by a read lock on drbd_cfg_rwsem
* accesses that add anything are protected by a write lock
* accesses that remove an object are protoected by a write lock
and a call to synchronize_rcu() after it is removed from the IDR
and before the object is actually free()ed.
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 2 +-
drivers/block/drbd/drbd_main.c | 2 ++
drivers/block/drbd/drbd_nl.c | 4 ++++
drivers/block/drbd/drbd_proc.c | 2 ++
drivers/block/drbd/drbd_receiver.c | 13 +++++++++++--
drivers/block/drbd/drbd_worker.c | 2 ++
6 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 7896a64..2119d9b 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -172,7 +172,7 @@ extern struct ratelimit_state drbd_ratelimit_state;
extern struct idr minors;
extern struct list_head drbd_tconns;
extern struct rw_semaphore drbd_cfg_rwsem;
-/* drbd_cfg_rwsem protects: drbd_tconns list,
+/* drbd_cfg_rwsem protects: drbd_tconns list, minors idr, tconn->volumes idr
note: non sleeping iterations over the idrs are protoected by RCU */
/* on the wire */
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index d48efb4..a2cac15 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2257,8 +2257,10 @@ static void drbd_cleanup(void)
drbd_genl_unregister();
+ down_write(&drbd_cfg_rwsem);
idr_for_each_entry(&minors, mdev, i)
drbd_delete_device(mdev);
+ up_write(&drbd_cfg_rwsem);
drbd_destroy_mempools();
unregister_blkdev(DRBD_MAJOR, "drbd");
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 5667536..331da4f 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -363,8 +363,10 @@ static void conn_md_sync(struct drbd_tconn *tconn)
struct drbd_conf *mdev;
int vnr;
+ down_read(&drbd_cfg_rwsem);
idr_for_each_entry(&tconn->volumes, mdev, vnr)
drbd_md_sync(mdev);
+ up_read(&drbd_cfg_rwsem);
}
int conn_khelper(struct drbd_tconn *tconn, char *cmd)
@@ -2920,7 +2922,9 @@ int drbd_adm_add_minor(struct sk_buff *skb, struct genl_info *info)
goto out;
}
+ down_write(&drbd_cfg_rwsem);
retcode = conn_new_minor(adm_ctx.tconn, dh->minor, adm_ctx.volume);
+ up_write(&drbd_cfg_rwsem);
out:
drbd_adm_finish(info, retcode);
return 0;
diff --git a/drivers/block/drbd/drbd_proc.c b/drivers/block/drbd/drbd_proc.c
index a4dbdbc..4025d08 100644
--- a/drivers/block/drbd/drbd_proc.c
+++ b/drivers/block/drbd/drbd_proc.c
@@ -227,6 +227,7 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
oos .. known out-of-sync kB
*/
+ down_read(&drbd_cfg_rwsem);
idr_for_each_entry(&minors, mdev, i) {
if (prev_i != i - 1)
seq_printf(seq, "\n");
@@ -293,6 +294,7 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
}
}
}
+ up_read(&drbd_cfg_rwsem);
return 0;
}
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 92ecb22..4d9d71a 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -964,7 +964,10 @@ retry:
if (drbd_send_protocol(tconn) == -EOPNOTSUPP)
return -1;
- return !idr_for_each(&tconn->volumes, drbd_connected, tconn);
+ down_read(&drbd_cfg_rwsem);
+ h = !idr_for_each(&tconn->volumes, drbd_connected, tconn);
+ up_read(&drbd_cfg_rwsem);
+ return h;
out_release_sockets:
if (tconn->data.socket) {
@@ -4084,7 +4087,9 @@ static void drbd_disconnect(struct drbd_tconn *tconn)
drbd_thread_stop(&tconn->asender);
drbd_free_sock(tconn);
+ down_read(&drbd_cfg_rwsem);
idr_for_each(&tconn->volumes, drbd_disconnected, tconn);
+ up_read(&drbd_cfg_rwsem);
conn_info(tconn, "Connection closed\n");
if (conn_highest_role(tconn) == R_PRIMARY && conn_highest_pdsk(tconn) >= D_UNKNOWN)
@@ -4821,10 +4826,14 @@ static int tconn_finish_peer_reqs(struct drbd_tconn *tconn)
do {
clear_bit(SIGNAL_ASENDER, &tconn->flags);
flush_signals(current);
+ down_read(&drbd_cfg_rwsem);
idr_for_each_entry(&tconn->volumes, mdev, i) {
- if (drbd_finish_peer_reqs(mdev))
+ if (drbd_finish_peer_reqs(mdev)) {
+ up_read(&drbd_cfg_rwsem);
return 1; /* error */
+ }
}
+ up_read(&drbd_cfg_rwsem);
set_bit(SIGNAL_ASENDER, &tconn->flags);
spin_lock_irq(&tconn->req_lock);
diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c
index 536eb05..b63c08e 100644
--- a/drivers/block/drbd/drbd_worker.c
+++ b/drivers/block/drbd/drbd_worker.c
@@ -1733,12 +1733,14 @@ int drbd_worker(struct drbd_thread *thi)
spin_unlock_irq(&tconn->data.work.q_lock);
drbd_thread_stop(&tconn->receiver);
+ down_read(&drbd_cfg_rwsem);
idr_for_each_entry(&tconn->volumes, mdev, vnr) {
D_ASSERT(mdev->state.disk == D_DISKLESS && mdev->state.conn == C_STANDALONE);
/* _drbd_set_state only uses stop_nowait.
* wait here for the exiting receiver. */
drbd_mdev_cleanup(mdev);
}
+ up_read(&drbd_cfg_rwsem);
clear_bit(OBJECT_DYING, &tconn->flags);
clear_bit(CONFIG_PENDING, &tconn->flags);
wake_up(&tconn->ping_wait);
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 06/11] drbd: Runtime changeable wire protocol
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (4 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 05/11] drbd: protect all idr accesses that might sleep with drbd_cfg_rwsem Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 07/11] drbd: Check consistency of net options when the get changed online Philipp Reisner
` (4 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
The wire protocol is no longer a property that is negotiated
between the two peers. It is now expressed with two bits
(DP_SEND_WRITE_ACK and DP_SEND_RECEIVE_ACK) in each data
packet. Therefore the primary node is free to change the
wire protocol at any time without disconnect/reconnect.
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 6 +++++
drivers/block/drbd/drbd_main.c | 8 ++++++-
drivers/block/drbd/drbd_receiver.c | 39 +++++++++++++++++------------------
drivers/block/drbd/drbd_req.c | 18 ++++++++++++----
drivers/block/drbd/drbd_req.h | 8 +++++++
5 files changed, 53 insertions(+), 26 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 2119d9b..c57cedb 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -327,6 +327,8 @@ extern unsigned int drbd_header_size(struct drbd_tconn *tconn);
#define DP_FUA 16 /* equals REQ_FUA */
#define DP_FLUSH 32 /* equals REQ_FLUSH */
#define DP_DISCARD 64 /* equals REQ_DISCARD */
+#define DP_SEND_RECEIVE_ACK 128 /* This is a proto B write request */
+#define DP_SEND_WRITE_ACK 256 /* This is a proto C write request */
struct p_data {
u64 sector; /* 64 bits sector number */
@@ -656,6 +658,9 @@ enum {
/* Conflicting local requests need to be restarted after this request */
__EE_RESTART_REQUESTS,
+
+ /* The peer wants a write ACK for this (wire proto C) */
+ __EE_SEND_WRITE_ACK,
};
#define EE_CALL_AL_COMPLETE_IO (1<<__EE_CALL_AL_COMPLETE_IO)
#define EE_MAY_SET_IN_SYNC (1<<__EE_MAY_SET_IN_SYNC)
@@ -663,6 +668,7 @@ enum {
#define EE_WAS_ERROR (1<<__EE_WAS_ERROR)
#define EE_HAS_DIGEST (1<<__EE_HAS_DIGEST)
#define EE_RESTART_REQUESTS (1<<__EE_RESTART_REQUESTS)
+#define EE_SEND_WRITE_ACK (1<<__EE_SEND_WRITE_ACK)
/* flag bits per mdev */
enum {
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index a2cac15..6a11e9d 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -1672,6 +1672,12 @@ int drbd_send_dblock(struct drbd_conf *mdev, struct drbd_request *req)
if (mdev->state.conn >= C_SYNC_SOURCE &&
mdev->state.conn <= C_PAUSED_SYNC_T)
dp_flags |= DP_MAY_SET_IN_SYNC;
+ if (mdev->tconn->agreed_pro_version >= 100) {
+ if (req->rq_state & RQ_EXP_RECEIVE_ACK)
+ dp_flags |= DP_SEND_RECEIVE_ACK;
+ if (req->rq_state & RQ_EXP_WRITE_ACK)
+ dp_flags |= DP_SEND_WRITE_ACK;
+ }
p->dp_flags = cpu_to_be32(dp_flags);
if (dgs)
drbd_csum_bio(mdev, mdev->tconn->integrity_w_tfm, req->master_bio, p + 1);
@@ -1688,7 +1694,7 @@ int drbd_send_dblock(struct drbd_conf *mdev, struct drbd_request *req)
* out ok after sending on this side, but does not fit on the
* receiving side, we sure have detected corruption elsewhere.
*/
- if (mdev->tconn->net_conf->wire_protocol == DRBD_PROT_A || dgs)
+ if (!(req->rq_state & (RQ_EXP_RECEIVE_ACK | RQ_EXP_WRITE_ACK)) || dgs)
err = _drbd_send_bio(mdev, req->master_bio);
else
err = _drbd_send_zc_bio(mdev, req->master_bio);
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 4d9d71a..0b648dc 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -1697,7 +1697,7 @@ static int e_end_block(struct drbd_work *w, int cancel)
sector_t sector = peer_req->i.sector;
int err = 0, pcmd;
- if (mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C) {
+ if (peer_req->flags & EE_SEND_WRITE_ACK) {
if (likely((peer_req->flags & EE_WAS_ERROR) == 0)) {
pcmd = (mdev->state.conn >= C_SYNC_SOURCE &&
mdev->state.conn <= C_PAUSED_SYNC_T &&
@@ -2074,20 +2074,28 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
list_add(&peer_req->w.list, &mdev->active_ee);
spin_unlock_irq(&mdev->tconn->req_lock);
- switch (mdev->tconn->net_conf->wire_protocol) {
- case DRBD_PROT_C:
+ if (mdev->tconn->agreed_pro_version < 100) {
+ switch (mdev->tconn->net_conf->wire_protocol) {
+ case DRBD_PROT_C:
+ dp_flags |= DP_SEND_WRITE_ACK;
+ break;
+ case DRBD_PROT_B:
+ dp_flags |= DP_SEND_RECEIVE_ACK;
+ break;
+ }
+ }
+
+ if (dp_flags & DP_SEND_WRITE_ACK) {
+ peer_req->flags |= EE_SEND_WRITE_ACK;
inc_unacked(mdev);
/* corresponding dec_unacked() in e_end_block()
* respective _drbd_clear_done_ee */
- break;
- case DRBD_PROT_B:
+ }
+
+ if (dp_flags & DP_SEND_RECEIVE_ACK) {
/* I really don't like it that the receiver thread
* sends on the msock, but anyways */
drbd_send_ack(mdev, P_RECV_ACK, peer_req);
- break;
- case DRBD_PROT_A:
- /* nothing to do */
- break;
}
if (mdev->state.pdsk < D_INCONSISTENT) {
@@ -2932,7 +2940,7 @@ static int receive_protocol(struct drbd_tconn *tconn, struct packet_info *pi)
if (cf & CF_DRY_RUN)
set_bit(CONN_DRY_RUN, &tconn->flags);
- if (p_proto != tconn->net_conf->wire_protocol) {
+ if (p_proto != tconn->net_conf->wire_protocol && tconn->agreed_pro_version < 100) {
conn_err(tconn, "incompatible communication protocols\n");
goto disconnect;
}
@@ -4622,23 +4630,18 @@ static int got_BlockAck(struct drbd_tconn *tconn, struct packet_info *pi)
}
switch (pi->cmd) {
case P_RS_WRITE_ACK:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C);
what = WRITE_ACKED_BY_PEER_AND_SIS;
break;
case P_WRITE_ACK:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C);
what = WRITE_ACKED_BY_PEER;
break;
case P_RECV_ACK:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_B);
what = RECV_ACKED_BY_PEER;
break;
case P_DISCARD_WRITE:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C);
what = DISCARD_WRITE;
break;
case P_RETRY_WRITE:
- D_ASSERT(mdev->tconn->net_conf->wire_protocol == DRBD_PROT_C);
what = POSTPONE_WRITE;
break;
default:
@@ -4656,8 +4659,6 @@ static int got_NegAck(struct drbd_tconn *tconn, struct packet_info *pi)
struct p_block_ack *p = pi->data;
sector_t sector = be64_to_cpu(p->sector);
int size = be32_to_cpu(p->blksize);
- bool missing_ok = tconn->net_conf->wire_protocol == DRBD_PROT_A ||
- tconn->net_conf->wire_protocol == DRBD_PROT_B;
int err;
mdev = vnr_to_mdev(tconn, pi->vnr);
@@ -4674,15 +4675,13 @@ static int got_NegAck(struct drbd_tconn *tconn, struct packet_info *pi)
err = validate_req_change_req_state(mdev, p->block_id, sector,
&mdev->write_requests, __func__,
- NEG_ACKED, missing_ok);
+ NEG_ACKED, true);
if (err) {
/* Protocol A has no P_WRITE_ACKs, but has P_NEG_ACKs.
The master bio might already be completed, therefore the
request is no longer in the collision hash. */
/* In Protocol B we might already have got a P_RECV_ACK
but then get a P_NEG_ACK afterwards. */
- if (!missing_ok)
- return err;
drbd_set_out_of_sync(mdev, sector, size);
}
return 0;
diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c
index fd0b452..0f1a29f 100644
--- a/drivers/block/drbd/drbd_req.c
+++ b/drivers/block/drbd/drbd_req.c
@@ -323,7 +323,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
struct bio_and_error *m)
{
struct drbd_conf *mdev = req->w.mdev;
- int rv = 0;
+ int p, rv = 0;
if (m)
m->bio = NULL;
@@ -344,6 +344,10 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
* and from w_read_retry_remote */
D_ASSERT(!(req->rq_state & RQ_NET_MASK));
req->rq_state |= RQ_NET_PENDING;
+ p = mdev->tconn->net_conf->wire_protocol;
+ req->rq_state |=
+ p == DRBD_PROT_C ? RQ_EXP_WRITE_ACK :
+ p == DRBD_PROT_B ? RQ_EXP_RECEIVE_ACK : 0;
inc_ap_pending(mdev);
break;
@@ -500,7 +504,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
atomic_add(req->i.size >> 9, &mdev->ap_in_flight);
if (bio_data_dir(req->master_bio) == WRITE &&
- mdev->tconn->net_conf->wire_protocol == DRBD_PROT_A) {
+ !(req->rq_state & (RQ_EXP_RECEIVE_ACK | RQ_EXP_WRITE_ACK))) {
/* this is what is dangerous about protocol A:
* pretend it was successfully written on the peer. */
if (req->rq_state & RQ_NET_PENDING) {
@@ -550,6 +554,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
req->rq_state |= RQ_NET_DONE;
/* fall through */
case WRITE_ACKED_BY_PEER:
+ D_ASSERT(req->rq_state & RQ_EXP_WRITE_ACK);
/* protocol C; successfully written on peer.
* Nothing to do here.
* We want to keep the tl in place for all protocols, to cater
@@ -560,11 +565,14 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
* request could set NET_DONE right here, and not wait for the
* P_BARRIER_ACK, but that is an unnecessary optimization. */
+ goto ack_common;
/* this makes it effectively the same as for: */
case RECV_ACKED_BY_PEER:
+ D_ASSERT(req->rq_state & RQ_EXP_RECEIVE_ACK);
/* protocol B; pretends to be successfully written on peer.
* see also notes above in HANDED_OVER_TO_NETWORK about
* protocol != C */
+ ack_common:
req->rq_state |= RQ_NET_OK;
D_ASSERT(req->rq_state & RQ_NET_PENDING);
dec_ap_pending(mdev);
@@ -574,8 +582,8 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
break;
case POSTPONE_WRITE:
- /*
- * If this node has already detected the write conflict, the
+ D_ASSERT(req->rq_state & RQ_EXP_WRITE_ACK);
+ /* If this node has already detected the write conflict, the
* worker will be waiting on misc_wait. Wake it up once this
* request has completed locally.
*/
@@ -646,7 +654,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
}
if ((req->rq_state & RQ_NET_MASK) != 0) {
req->rq_state |= RQ_NET_DONE;
- if (mdev->tconn->net_conf->wire_protocol == DRBD_PROT_A)
+ if (!(req->rq_state & (RQ_EXP_RECEIVE_ACK | RQ_EXP_WRITE_ACK)))
atomic_sub(req->i.size>>9, &mdev->ap_in_flight);
}
_req_may_be_done(req, m); /* Allowed while state.susp */
diff --git a/drivers/block/drbd/drbd_req.h b/drivers/block/drbd/drbd_req.h
index b9f474a..dfd2cef 100644
--- a/drivers/block/drbd/drbd_req.h
+++ b/drivers/block/drbd/drbd_req.h
@@ -198,6 +198,12 @@ enum drbd_req_state_bits {
/* The peer has sent a retry ACK */
__RQ_POSTPONED,
+
+ /* We expect a receive ACK (wire proto B) */
+ __RQ_EXP_RECEIVE_ACK,
+
+ /* We expect a write ACK (wite proto C) */
+ __RQ_EXP_WRITE_ACK,
};
#define RQ_LOCAL_PENDING (1UL << __RQ_LOCAL_PENDING)
@@ -219,6 +225,8 @@ enum drbd_req_state_bits {
#define RQ_WRITE (1UL << __RQ_WRITE)
#define RQ_IN_ACT_LOG (1UL << __RQ_IN_ACT_LOG)
#define RQ_POSTPONED (1UL << __RQ_POSTPONED)
+#define RQ_EXP_RECEIVE_ACK (1UL << __RQ_EXP_RECEIVE_ACK)
+#define RQ_EXP_WRITE_ACK (1UL << __RQ_EXP_WRITE_ACK)
/* For waking up the frozen transfer log mod_req() has to return if the request
should be counted in the epoch object*/
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 07/11] drbd: Check consistency of net options when the get changed online
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (5 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 06/11] drbd: Runtime changeable wire protocol Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 08/11] drbd: Allow online change of replication protocol only with agreed_pv >= 100 Philipp Reisner
` (3 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_nl.c | 79 ++++++++++++++++++++++++------------------
1 files changed, 45 insertions(+), 34 deletions(-)
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 331da4f..0bf1c2e 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -1705,6 +1705,45 @@ static bool conn_ov_running(struct drbd_tconn *tconn)
return rv;
}
+static enum drbd_ret_code
+check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
+{
+ struct drbd_conf *mdev;
+ int i;
+
+ if (new_conf->two_primaries &&
+ (new_conf->wire_protocol != DRBD_PROT_C))
+ return ERR_NOT_PROTO_C;
+
+ rcu_read_lock();
+ idr_for_each_entry(&tconn->volumes, mdev, i) {
+ if (get_ldev(mdev)) {
+ enum drbd_fencing_p fp = mdev->ldev->dc.fencing;
+ put_ldev(mdev);
+ if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH) {
+ rcu_read_unlock();
+ return ERR_STONITH_AND_PROT_A;
+ }
+ }
+ if (mdev->state.role == R_PRIMARY && new_conf->want_lose) {
+ rcu_read_unlock();
+ return ERR_DISCARD;
+ }
+ if (!mdev->bitmap) {
+ if(drbd_bm_init(mdev)) {
+ rcu_read_unlock();
+ return ERR_NOMEM;
+ }
+ }
+ }
+ rcu_read_unlock();
+
+ if (new_conf->on_congestion != OC_BLOCK && new_conf->wire_protocol != DRBD_PROT_A)
+ return ERR_CONG_NOT_PROTO_A;
+
+ return NO_ERROR;
+}
+
int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
{
enum drbd_ret_code retcode;
@@ -1763,6 +1802,10 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
goto fail;
}
+ retcode = check_net_options(tconn, new_conf);
+ if (retcode != NO_ERROR)
+ goto fail;
+
/* re-sync running */
rsr = conn_resync_running(tconn);
if (rsr && strcmp(new_conf->csums_alg, tconn->net_conf->csums_alg)) {
@@ -1922,39 +1965,9 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
goto fail;
}
- if (new_conf->two_primaries
- && (new_conf->wire_protocol != DRBD_PROT_C)) {
- retcode = ERR_NOT_PROTO_C;
- goto fail;
- }
-
- rcu_read_lock();
- idr_for_each_entry(&tconn->volumes, mdev, i) {
- if (get_ldev(mdev)) {
- enum drbd_fencing_p fp = mdev->ldev->dc.fencing;
- put_ldev(mdev);
- if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH) {
- retcode = ERR_STONITH_AND_PROT_A;
- goto fail_rcu_unlock;
- }
- }
- if (mdev->state.role == R_PRIMARY && new_conf->want_lose) {
- retcode = ERR_DISCARD;
- goto fail_rcu_unlock;
- }
- if (!mdev->bitmap) {
- if(drbd_bm_init(mdev)) {
- retcode = ERR_NOMEM;
- goto fail_rcu_unlock;
- }
- }
- }
- rcu_read_unlock();
-
- if (new_conf->on_congestion != OC_BLOCK && new_conf->wire_protocol != DRBD_PROT_A) {
- retcode = ERR_CONG_NOT_PROTO_A;
+ retcode = check_net_options(tconn, new_conf);
+ if (retcode != NO_ERROR)
goto fail;
- }
retcode = NO_ERROR;
@@ -2074,8 +2087,6 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
drbd_adm_finish(info, retcode);
return 0;
-fail_rcu_unlock:
- rcu_read_unlock();
fail:
kfree(int_dig_in);
kfree(int_dig_vv);
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 08/11] drbd: Allow online change of replication protocol only with agreed_pv >= 100
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (6 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 07/11] drbd: Check consistency of net options when the get changed online Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 09/11] drbd: rcu_read_lock() and rcu_dereference() for tconn->net_conf Philipp Reisner
` (2 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_nl.c | 5 +++++
include/linux/drbd.h | 1 +
2 files changed, 6 insertions(+), 0 deletions(-)
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 0bf1c2e..bb41af1 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -1711,6 +1711,11 @@ check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
struct drbd_conf *mdev;
int i;
+ if (tconn->net_conf && tconn->agreed_pro_version < 100 &&
+ tconn->cstate == C_WF_REPORT_PARAMS &&
+ new_conf->wire_protocol != tconn->net_conf->wire_protocol)
+ return ERR_NEED_APV_100;
+
if (new_conf->two_primaries &&
(new_conf->wire_protocol != DRBD_PROT_C))
return ERR_NOT_PROTO_C;
diff --git a/include/linux/drbd.h b/include/linux/drbd.h
index fe8d6ba..6c7c85d 100644
--- a/include/linux/drbd.h
+++ b/include/linux/drbd.h
@@ -160,6 +160,7 @@ enum drbd_ret_code {
ERR_MINOR_CONFIGURED = 160,
ERR_MINOR_EXISTS = 161,
ERR_INVALID_REQUEST = 162,
+ ERR_NEED_APV_100 = 163,
/* insert new ones above this line */
AFTER_LAST_ERR_CODE
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 09/11] drbd: rcu_read_lock() and rcu_dereference() for tconn->net_conf
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (7 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 08/11] drbd: Allow online change of replication protocol only with agreed_pv >= 100 Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 10/11] drbd: Proper locking for updates to net_conf under RCU Philipp Reisner
2011-09-30 12:48 ` [PATCH 11/11] drbd: Considering that the two_primaries config flag can change Philipp Reisner
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Removing the get_net_conf()/put_net_conf() calls
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 15 ++-
drivers/block/drbd/drbd_main.c | 78 ++++++++----
drivers/block/drbd/drbd_nl.c | 171 +++++++++++++++-----------
drivers/block/drbd/drbd_proc.c | 9 +-
drivers/block/drbd/drbd_receiver.c | 245 ++++++++++++++++++++++++------------
drivers/block/drbd/drbd_req.c | 40 ++++--
drivers/block/drbd/drbd_state.c | 14 ++-
drivers/block/drbd/drbd_worker.c | 26 +++-
8 files changed, 389 insertions(+), 209 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index c57cedb..99da54c 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -832,7 +832,7 @@ struct drbd_tconn { /* is a resource from the config file */
struct mutex cstate_mutex; /* Protects graceful disconnects */
unsigned long flags;
- struct net_conf *net_conf; /* protected by get_net_conf() and put_net_conf() */
+ struct net_conf *net_conf; /* content protected by rcu */
atomic_t net_cnt; /* Users of net_conf */
wait_queue_head_t net_cnt_wait;
wait_queue_head_t ping_wait; /* Woken upon reception of a ping, and a state change */
@@ -2059,11 +2059,14 @@ static inline void drbd_get_syncer_progress(struct drbd_conf *mdev,
* maybe re-implement using semaphores? */
static inline int drbd_get_max_buffers(struct drbd_conf *mdev)
{
- int mxb = 1000000; /* arbitrary limit on open requests */
- if (get_net_conf(mdev->tconn)) {
- mxb = mdev->tconn->net_conf->max_buffers;
- put_net_conf(mdev->tconn);
- }
+ struct net_conf *nc;
+ int mxb;
+
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ mxb = nc ? nc->max_buffers : 1000000; /* arbitrary limit on open requests */
+ rcu_read_unlock();
+
return mxb;
}
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 6a11e9d..d335ecc 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -843,15 +843,19 @@ int drbd_send_sync_param(struct drbd_conf *mdev)
int size;
const int apv = mdev->tconn->agreed_pro_version;
enum drbd_packet cmd;
+ struct net_conf *nc;
sock = &mdev->tconn->data;
p = drbd_prepare_command(mdev, sock);
if (!p)
return -EIO;
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+
size = apv <= 87 ? sizeof(struct p_rs_param)
: apv == 88 ? sizeof(struct p_rs_param)
- + strlen(mdev->tconn->net_conf->verify_alg) + 1
+ + strlen(nc->verify_alg) + 1
: apv <= 94 ? sizeof(struct p_rs_param_89)
: /* apv >= 95 */ sizeof(struct p_rs_param_95);
@@ -876,9 +880,10 @@ int drbd_send_sync_param(struct drbd_conf *mdev)
}
if (apv >= 88)
- strcpy(p->verify_alg, mdev->tconn->net_conf->verify_alg);
+ strcpy(p->verify_alg, nc->verify_alg);
if (apv >= 89)
- strcpy(p->csums_alg, mdev->tconn->net_conf->csums_alg);
+ strcpy(p->csums_alg, nc->csums_alg);
+ rcu_read_unlock();
return drbd_send_command(mdev, sock, cmd, size, NULL, 0);
}
@@ -887,36 +892,44 @@ int drbd_send_protocol(struct drbd_tconn *tconn)
{
struct drbd_socket *sock;
struct p_protocol *p;
+ struct net_conf *nc;
int size, cf;
- if (tconn->net_conf->dry_run && tconn->agreed_pro_version < 92) {
- conn_err(tconn, "--dry-run is not supported by peer");
- return -EOPNOTSUPP;
- }
-
sock = &tconn->data;
p = conn_prepare_command(tconn, sock);
if (!p)
return -EIO;
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+
+ if (nc->dry_run && tconn->agreed_pro_version < 92) {
+ rcu_read_unlock();
+ mutex_unlock(&sock->mutex);
+ conn_err(tconn, "--dry-run is not supported by peer");
+ return -EOPNOTSUPP;
+ }
+
size = sizeof(*p);
if (tconn->agreed_pro_version >= 87)
- size += strlen(tconn->net_conf->integrity_alg) + 1;
+ size += strlen(nc->integrity_alg) + 1;
- p->protocol = cpu_to_be32(tconn->net_conf->wire_protocol);
- p->after_sb_0p = cpu_to_be32(tconn->net_conf->after_sb_0p);
- p->after_sb_1p = cpu_to_be32(tconn->net_conf->after_sb_1p);
- p->after_sb_2p = cpu_to_be32(tconn->net_conf->after_sb_2p);
- p->two_primaries = cpu_to_be32(tconn->net_conf->two_primaries);
+ p->protocol = cpu_to_be32(nc->wire_protocol);
+ p->after_sb_0p = cpu_to_be32(nc->after_sb_0p);
+ p->after_sb_1p = cpu_to_be32(nc->after_sb_1p);
+ p->after_sb_2p = cpu_to_be32(nc->after_sb_2p);
+ p->two_primaries = cpu_to_be32(nc->two_primaries);
cf = 0;
- if (tconn->net_conf->want_lose)
+ if (nc->want_lose)
cf |= CF_WANT_LOSE;
- if (tconn->net_conf->dry_run)
+ if (nc->dry_run)
cf |= CF_DRY_RUN;
p->conn_flags = cpu_to_be32(cf);
if (tconn->agreed_pro_version >= 87)
- strcpy(p->integrity_alg, tconn->net_conf->integrity_alg);
+ strcpy(p->integrity_alg, nc->integrity_alg);
+ rcu_read_unlock();
+
return conn_send_command(tconn, sock, P_PROTOCOL, size, NULL, 0);
}
@@ -938,7 +951,9 @@ int _drbd_send_uuids(struct drbd_conf *mdev, u64 uuid_flags)
mdev->comm_bm_set = drbd_bm_total_weight(mdev);
p->uuid[UI_SIZE] = cpu_to_be64(mdev->comm_bm_set);
- uuid_flags |= mdev->tconn->net_conf->want_lose ? 1 : 0;
+ rcu_read_lock();
+ uuid_flags |= rcu_dereference(mdev->tconn->net_conf)->want_lose ? 1 : 0;
+ rcu_read_unlock();
uuid_flags |= test_bit(CRASHED_PRIMARY, &mdev->flags) ? 2 : 0;
uuid_flags |= mdev->new_state_tmp.disk == D_INCONSISTENT ? 4 : 0;
p->uuid[UI_FLAGS] = cpu_to_be64(uuid_flags);
@@ -1134,12 +1149,14 @@ int fill_bitmap_rle_bits(struct drbd_conf *mdev,
unsigned long rl;
unsigned len;
unsigned toggle;
- int bits;
+ int bits, use_rle;
/* may we use this feature? */
- if ((mdev->tconn->net_conf->use_rle == 0) ||
- (mdev->tconn->agreed_pro_version < 90))
- return 0;
+ rcu_read_lock();
+ use_rle = rcu_dereference(mdev->tconn->net_conf)->use_rle;
+ rcu_read_unlock();
+ if (!use_rle || mdev->tconn->agreed_pro_version < 90)
+ return 0;
if (c->bit_offset >= c->bm_bits)
return 0; /* nothing to do. */
@@ -1803,7 +1820,9 @@ int drbd_send(struct drbd_tconn *tconn, struct socket *sock,
msg.msg_flags = msg_flags | MSG_NOSIGNAL;
if (sock == tconn->data.socket) {
- tconn->ko_count = tconn->net_conf->ko_count;
+ rcu_read_lock();
+ tconn->ko_count = rcu_dereference(tconn->net_conf)->ko_count;
+ rcu_read_unlock();
drbd_update_congested(tconn);
}
do {
@@ -3226,15 +3245,18 @@ const char *cmdname(enum drbd_packet cmd)
*/
int drbd_wait_misc(struct drbd_conf *mdev, struct drbd_interval *i)
{
- struct net_conf *net_conf = mdev->tconn->net_conf;
+ struct net_conf *nc;
DEFINE_WAIT(wait);
long timeout;
- if (!net_conf)
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (!nc) {
+ rcu_read_unlock();
return -ETIMEDOUT;
- timeout = MAX_SCHEDULE_TIMEOUT;
- if (net_conf->ko_count)
- timeout = net_conf->timeout * HZ / 10 * net_conf->ko_count;
+ }
+ timeout = nc->ko_count ? nc->timeout * HZ / 10 * nc->ko_count : MAX_SCHEDULE_TIMEOUT;
+ rcu_read_unlock();
/* Indicate to wake up mdev->misc_wait on progress. */
i->waiting = true;
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index bb41af1..6d28e7c 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -290,29 +290,32 @@ static int drbd_adm_finish(struct genl_info *info, int retcode)
static void setup_khelper_env(struct drbd_tconn *tconn, char **envp)
{
char af[20], ad[60], *afs;
+ struct net_conf *nc;
- if (get_net_conf(tconn)) {
- switch (((struct sockaddr *)tconn->net_conf->peer_addr)->sa_family) {
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ if (nc) {
+ switch (((struct sockaddr *)nc->peer_addr)->sa_family) {
case AF_INET6:
afs = "ipv6";
snprintf(ad, 60, "DRBD_PEER_ADDRESS=%pI6",
- &((struct sockaddr_in6 *)tconn->net_conf->peer_addr)->sin6_addr);
+ &((struct sockaddr_in6 *)nc->peer_addr)->sin6_addr);
break;
case AF_INET:
afs = "ipv4";
snprintf(ad, 60, "DRBD_PEER_ADDRESS=%pI4",
- &((struct sockaddr_in *)tconn->net_conf->peer_addr)->sin_addr);
+ &((struct sockaddr_in *)nc->peer_addr)->sin_addr);
break;
default:
afs = "ssocks";
snprintf(ad, 60, "DRBD_PEER_ADDRESS=%pI4",
- &((struct sockaddr_in *)tconn->net_conf->peer_addr)->sin_addr);
+ &((struct sockaddr_in *)nc->peer_addr)->sin_addr);
}
snprintf(af, 20, "DRBD_PEER_AF=%s", afs);
envp[3]=af;
envp[4]=ad;
- put_net_conf(tconn);
}
+ rcu_read_unlock();
}
int drbd_khelper(struct drbd_conf *mdev, char *cmd)
@@ -519,6 +522,7 @@ drbd_set_role(struct drbd_conf *mdev, enum drbd_role new_role, int force)
{
const int max_tries = 4;
enum drbd_state_rv rv = SS_UNKNOWN_ERROR;
+ struct net_conf *nc;
int try = 0;
int forced = 0;
union drbd_state mask, val;
@@ -576,7 +580,12 @@ drbd_set_role(struct drbd_conf *mdev, enum drbd_role new_role, int force)
if (rv == SS_TWO_PRIMARIES) {
/* Maybe the peer is detected as dead very soon...
retry at most once more in this case. */
- schedule_timeout_interruptible((mdev->tconn->net_conf->ping_timeo+1)*HZ/10);
+ int timeo;
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ timeo = nc ? (nc->ping_timeo + 1) * HZ / 10 : 1;
+ rcu_read_unlock();
+ schedule_timeout_interruptible(timeo);
if (try < max_tries)
try = max_tries - 1;
continue;
@@ -606,10 +615,11 @@ drbd_set_role(struct drbd_conf *mdev, enum drbd_role new_role, int force)
put_ldev(mdev);
}
} else {
- if (get_net_conf(mdev->tconn)) {
- mdev->tconn->net_conf->want_lose = 0;
- put_net_conf(mdev->tconn);
- }
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (nc)
+ nc->want_lose = 0;
+ rcu_read_unlock();
set_disk_ro(mdev->vdisk, false);
if (get_ldev(mdev)) {
if (((mdev->state.conn < C_CONNECTED ||
@@ -1233,6 +1243,7 @@ int drbd_adm_attach(struct sk_buff *skb, struct genl_info *info)
struct lru_cache *resync_lru = NULL;
union drbd_state ns, os;
enum drbd_state_rv rv;
+ struct net_conf *nc;
int cp_discovered = 0;
retcode = drbd_adm_prepare(skb, info, DRBD_ADM_NEED_MINOR);
@@ -1296,14 +1307,16 @@ int drbd_adm_attach(struct sk_buff *skb, struct genl_info *info)
goto fail;
}
- if (get_net_conf(mdev->tconn)) {
- int prot = mdev->tconn->net_conf->wire_protocol;
- put_net_conf(mdev->tconn);
- if (nbc->dc.fencing == FP_STONITH && prot == DRBD_PROT_A) {
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (nc) {
+ if (nbc->dc.fencing == FP_STONITH && nc->wire_protocol == DRBD_PROT_A) {
+ rcu_read_unlock();
retcode = ERR_STONITH_AND_PROT_A;
goto fail;
}
}
+ rcu_read_unlock();
bdev = blkdev_get_by_path(nbc->dc.backing_dev,
FMODE_READ | FMODE_WRITE | FMODE_EXCL, mdev);
@@ -1706,42 +1719,35 @@ static bool conn_ov_running(struct drbd_tconn *tconn)
}
static enum drbd_ret_code
-check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
+_check_net_options(struct drbd_tconn *tconn, struct net_conf *old_conf, struct net_conf *new_conf)
{
struct drbd_conf *mdev;
int i;
- if (tconn->net_conf && tconn->agreed_pro_version < 100 &&
+ if (old_conf && tconn->agreed_pro_version < 100 &&
tconn->cstate == C_WF_REPORT_PARAMS &&
- new_conf->wire_protocol != tconn->net_conf->wire_protocol)
+ new_conf->wire_protocol != old_conf->wire_protocol)
return ERR_NEED_APV_100;
if (new_conf->two_primaries &&
(new_conf->wire_protocol != DRBD_PROT_C))
return ERR_NOT_PROTO_C;
- rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
if (get_ldev(mdev)) {
enum drbd_fencing_p fp = mdev->ldev->dc.fencing;
put_ldev(mdev);
- if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH) {
- rcu_read_unlock();
+ if (new_conf->wire_protocol == DRBD_PROT_A && fp == FP_STONITH)
return ERR_STONITH_AND_PROT_A;
- }
}
- if (mdev->state.role == R_PRIMARY && new_conf->want_lose) {
- rcu_read_unlock();
+ if (mdev->state.role == R_PRIMARY && new_conf->want_lose)
return ERR_DISCARD;
- }
+
if (!mdev->bitmap) {
- if(drbd_bm_init(mdev)) {
- rcu_read_unlock();
+ if(drbd_bm_init(mdev))
return ERR_NOMEM;
- }
}
}
- rcu_read_unlock();
if (new_conf->on_congestion != OC_BLOCK && new_conf->wire_protocol != DRBD_PROT_A)
return ERR_CONG_NOT_PROTO_A;
@@ -1749,11 +1755,23 @@ check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
return NO_ERROR;
}
+static enum drbd_ret_code
+check_net_options(struct drbd_tconn *tconn, struct net_conf *new_conf)
+{
+ static enum drbd_ret_code rv;
+
+ rcu_read_lock();
+ rv = _check_net_options(tconn, rcu_dereference(tconn->net_conf), new_conf);
+ rcu_read_unlock();
+
+ return rv;
+}
+
int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
{
enum drbd_ret_code retcode;
struct drbd_tconn *tconn;
- struct net_conf *new_conf = NULL;
+ struct net_conf *old_conf, *new_conf = NULL;
int err;
int ovr; /* online verify running */
int rsr; /* re-sync running */
@@ -1789,17 +1807,20 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
goto out;
}
- /* we also need a net config
- * to change the options on */
- if (!get_net_conf(tconn)) {
+ conn_reconfig_start(tconn);
+
+ rcu_read_lock();
+ old_conf = rcu_dereference(tconn->net_conf);
+
+ if (!old_conf) {
drbd_msg_put_info("net conf missing, try connect");
retcode = ERR_INVALID_REQUEST;
- goto out;
+ goto fail_rcu_unlock;
}
- conn_reconfig_start(tconn);
+ *new_conf = *old_conf;
+ rcu_read_unlock();
- memcpy(new_conf, tconn->net_conf, sizeof(*new_conf));
err = net_conf_from_attrs(new_conf, info);
if (err) {
retcode = ERR_MANDATORY_TAG;
@@ -1813,10 +1834,13 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
/* re-sync running */
rsr = conn_resync_running(tconn);
- if (rsr && strcmp(new_conf->csums_alg, tconn->net_conf->csums_alg)) {
+ rcu_read_lock();
+ old_conf = rcu_dereference(tconn->net_conf);
+ if (rsr && old_conf && strcmp(new_conf->csums_alg, old_conf->csums_alg)) {
retcode = ERR_CSUMS_RESYNC_RUNNING;
- goto fail;
+ goto fail_rcu_unlock;
}
+ rcu_read_unlock();
if (!rsr && new_conf->csums_alg[0]) {
csums_tfm = crypto_alloc_hash(new_conf->csums_alg, 0, CRYPTO_ALG_ASYNC);
@@ -1834,12 +1858,15 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
/* online verify running */
ovr = conn_ov_running(tconn);
- if (ovr) {
- if (strcmp(new_conf->verify_alg, tconn->net_conf->verify_alg)) {
+ rcu_read_lock();
+ old_conf = rcu_dereference(tconn->net_conf);
+ if (ovr && old_conf) {
+ if (strcmp(new_conf->verify_alg, old_conf->verify_alg)) {
retcode = ERR_VERIFY_RUNNING;
- goto fail;
+ goto fail_rcu_unlock;
}
}
+ rcu_read_unlock();
if (!ovr && new_conf->verify_alg[0]) {
verify_tfm = crypto_alloc_hash(new_conf->verify_alg, 0, CRYPTO_ALG_ASYNC);
@@ -1855,16 +1882,9 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
}
}
-
- /* For now, use struct assignment, not pointer assignment.
- * We don't have any means to determine who might still
- * keep a local alias into the struct,
- * so we cannot just free it and hope for the best :(
- * FIXME
- * To avoid someone looking at a half-updated struct, we probably
- * should have a rw-semaphor on net_conf and disk_conf.
- */
- *tconn->net_conf = *new_conf;
+ rcu_assign_pointer(tconn->net_conf, new_conf);
+ synchronize_rcu();
+ kfree(old_conf);
if (!rsr) {
crypto_free_hash(tconn->csums_tfm);
@@ -1880,11 +1900,12 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
if (tconn->cstate >= C_WF_REPORT_PARAMS)
drbd_send_sync_param(minor_to_mdev(conn_lowest_minor(tconn)));
+ fail_rcu_unlock:
+ rcu_read_unlock();
fail:
crypto_free_hash(csums_tfm);
crypto_free_hash(verify_tfm);
kfree(new_conf);
- put_net_conf(tconn);
conn_reconfig_done(tconn);
out:
drbd_adm_finish(info, retcode);
@@ -1895,7 +1916,7 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
{
char hmac_name[CRYPTO_MAX_ALG_NAME];
struct drbd_conf *mdev;
- struct net_conf *new_conf = NULL;
+ struct net_conf *old_conf, *new_conf = NULL;
struct crypto_hash *tfm = NULL;
struct crypto_hash *integrity_w_tfm = NULL;
struct crypto_hash *integrity_r_tfm = NULL;
@@ -1983,23 +2004,26 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
* strictly serialized on genl_lock(). We are protected against
* concurrent reconfiguration/addition/deletion */
list_for_each_entry(oconn, &drbd_tconns, all_tconn) {
+ struct net_conf *nc;
if (oconn == tconn)
continue;
- if (get_net_conf(oconn)) {
- taken_addr = (struct sockaddr *)&oconn->net_conf->my_addr;
- if (new_conf->my_addr_len == oconn->net_conf->my_addr_len &&
+
+ rcu_read_lock();
+ nc = rcu_dereference(oconn->net_conf);
+ if (nc) {
+ taken_addr = (struct sockaddr *)&nc->my_addr;
+ if (new_conf->my_addr_len == nc->my_addr_len &&
!memcmp(new_my_addr, taken_addr, new_conf->my_addr_len))
retcode = ERR_LOCAL_ADDR;
- taken_addr = (struct sockaddr *)&oconn->net_conf->peer_addr;
- if (new_conf->peer_addr_len == oconn->net_conf->peer_addr_len &&
+ taken_addr = (struct sockaddr *)&nc->peer_addr;
+ if (new_conf->peer_addr_len == nc->peer_addr_len &&
!memcmp(new_peer_addr, taken_addr, new_conf->peer_addr_len))
retcode = ERR_PEER_ADDR;
-
- put_net_conf(oconn);
- if (retcode != NO_ERROR)
- goto fail;
}
+ rcu_read_unlock();
+ if (retcode != NO_ERROR)
+ goto fail;
}
if (new_conf->cram_hmac_alg[0] != 0) {
@@ -2058,12 +2082,15 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
conn_flush_workqueue(tconn);
spin_lock_irq(&tconn->req_lock);
- if (tconn->net_conf != NULL) {
+ rcu_read_lock();
+ old_conf = rcu_dereference(tconn->net_conf);
+ if (old_conf != NULL) {
retcode = ERR_NET_CONFIGURED;
+ rcu_read_unlock();
spin_unlock_irq(&tconn->req_lock);
goto fail;
}
- tconn->net_conf = new_conf;
+ rcu_assign_pointer(tconn->net_conf, new_conf);
crypto_free_hash(tconn->cram_hmac_tfm);
tconn->cram_hmac_tfm = tfm;
@@ -2518,9 +2545,9 @@ int nla_put_status_info(struct sk_buff *skb, struct drbd_conf *mdev,
const struct sib_info *sib)
{
struct state_info *si = NULL; /* for sizeof(si->member); */
+ struct net_conf *nc;
struct nlattr *nla;
int got_ldev;
- int got_net;
int err = 0;
int exclude_sensitive;
@@ -2538,7 +2565,6 @@ int nla_put_status_info(struct sk_buff *skb, struct drbd_conf *mdev,
exclude_sensitive = sib || !capable(CAP_SYS_ADMIN);
got_ldev = get_ldev(mdev);
- got_net = get_net_conf(mdev->tconn);
/* We need to add connection name and volume number information still.
* Minor number is in drbd_genlmsghdr. */
@@ -2551,9 +2577,14 @@ int nla_put_status_info(struct sk_buff *skb, struct drbd_conf *mdev,
if (got_ldev)
if (disk_conf_to_skb(skb, &mdev->ldev->dc, exclude_sensitive))
goto nla_put_failure;
- if (got_net)
- if (net_conf_to_skb(skb, mdev->tconn->net_conf, exclude_sensitive))
- goto nla_put_failure;
+
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (nc)
+ err = net_conf_to_skb(skb, nc, exclude_sensitive);
+ rcu_read_unlock();
+ if (err)
+ goto nla_put_failure;
nla = nla_nest_start(skb, DRBD_NLA_STATE_INFO);
if (!nla)
@@ -2600,8 +2631,6 @@ nla_put_failure:
err = -EMSGSIZE;
if (got_ldev)
put_ldev(mdev);
- if (got_net)
- put_net_conf(mdev->tconn);
return err;
}
diff --git a/drivers/block/drbd/drbd_proc.c b/drivers/block/drbd/drbd_proc.c
index 4025d08..792a71e 100644
--- a/drivers/block/drbd/drbd_proc.c
+++ b/drivers/block/drbd/drbd_proc.c
@@ -197,6 +197,8 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
int i, prev_i = -1;
const char *sn;
struct drbd_conf *mdev;
+ struct net_conf *nc;
+ char wp;
static char write_ordering_chars[] = {
[WO_none] = 'n',
@@ -240,6 +242,10 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
mdev->state.role == R_SECONDARY) {
seq_printf(seq, "%2d: cs:Unconfigured\n", i);
} else {
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ wp = nc ? nc->wire_protocol - DRBD_PROT_A + 'A' : ' ';
+ rcu_read_unlock();
seq_printf(seq,
"%2d: cs:%s ro:%s/%s ds:%s/%s %c %c%c%c%c%c%c\n"
" ns:%u nr:%u dw:%u dr:%u al:%u bm:%u "
@@ -249,8 +255,7 @@ static int drbd_seq_show(struct seq_file *seq, void *v)
drbd_role_str(mdev->state.peer),
drbd_disk_str(mdev->state.disk),
drbd_disk_str(mdev->state.pdsk),
- (mdev->tconn->net_conf == NULL ? ' ' :
- (mdev->tconn->net_conf->wire_protocol - DRBD_PROT_A+'A')),
+ wp,
drbd_suspended(mdev) ? 's' : 'r',
mdev->state.aftr_isp ? 'a' : '-',
mdev->state.peer_isp ? 'p' : '-',
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 0b648dc..a37f2ee 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -244,11 +244,18 @@ struct page *drbd_alloc_pages(struct drbd_conf *mdev, unsigned int number,
bool retry)
{
struct page *page = NULL;
+ struct net_conf *nc;
DEFINE_WAIT(wait);
+ int mxb;
/* Yes, we may run up to @number over max_buffers. If we
* follow it strictly, the admin will get it wrong anyways. */
- if (atomic_read(&mdev->pp_in_use) < mdev->tconn->net_conf->max_buffers)
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ mxb = nc ? nc->max_buffers : 1000000;
+ rcu_read_unlock();
+
+ if (atomic_read(&mdev->pp_in_use) < mxb)
page = __drbd_alloc_pages(mdev, number);
while (page == NULL) {
@@ -256,7 +263,7 @@ struct page *drbd_alloc_pages(struct drbd_conf *mdev, unsigned int number,
drbd_kick_lo_and_reclaim_net(mdev);
- if (atomic_read(&mdev->pp_in_use) < mdev->tconn->net_conf->max_buffers) {
+ if (atomic_read(&mdev->pp_in_use) < mxb) {
page = __drbd_alloc_pages(mdev, number);
if (page)
break;
@@ -607,24 +614,47 @@ static struct socket *drbd_try_connect(struct drbd_tconn *tconn)
const char *what;
struct socket *sock;
struct sockaddr_in6 src_in6;
- int err;
+ struct sockaddr_in6 peer_in6;
+ struct net_conf *nc;
+ int err, peer_addr_len, my_addr_len;
+ int sndbuf_size, rcvbuf_size, try_connect_int;
int disconnect_on_error = 1;
- if (!get_net_conf(tconn))
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ if (!nc) {
+ rcu_read_unlock();
return NULL;
+ }
+
+ sndbuf_size = nc->sndbuf_size;
+ rcvbuf_size = nc->rcvbuf_size;
+ try_connect_int = nc->try_connect_int;
+
+ my_addr_len = min_t(int, nc->my_addr_len, sizeof(src_in6));
+ memcpy(&src_in6, nc->my_addr, my_addr_len);
+
+ if (((struct sockaddr *)nc->my_addr)->sa_family == AF_INET6)
+ src_in6.sin6_port = 0;
+ else
+ ((struct sockaddr_in *)&src_in6)->sin_port = 0; /* AF_INET & AF_SCI */
+
+ peer_addr_len = min_t(int, nc->peer_addr_len, sizeof(src_in6));
+ memcpy(&peer_in6, nc->peer_addr, peer_addr_len);
+
+ rcu_read_unlock();
what = "sock_create_kern";
- err = sock_create_kern(((struct sockaddr *)tconn->net_conf->my_addr)->sa_family,
- SOCK_STREAM, IPPROTO_TCP, &sock);
+ err = sock_create_kern(((struct sockaddr *)&src_in6)->sa_family,
+ SOCK_STREAM, IPPROTO_TCP, &sock);
if (err < 0) {
sock = NULL;
goto out;
}
sock->sk->sk_rcvtimeo =
- sock->sk->sk_sndtimeo = tconn->net_conf->try_connect_int*HZ;
- drbd_setbufsize(sock, tconn->net_conf->sndbuf_size,
- tconn->net_conf->rcvbuf_size);
+ sock->sk->sk_sndtimeo = try_connect_int * HZ;
+ drbd_setbufsize(sock, sndbuf_size, rcvbuf_size);
/* explicitly bind to the configured IP as source IP
* for the outgoing connections.
@@ -633,17 +663,8 @@ static struct socket *drbd_try_connect(struct drbd_tconn *tconn)
* Make sure to use 0 as port number, so linux selects
* a free one dynamically.
*/
- memcpy(&src_in6, tconn->net_conf->my_addr,
- min_t(int, tconn->net_conf->my_addr_len, sizeof(src_in6)));
- if (((struct sockaddr *)tconn->net_conf->my_addr)->sa_family == AF_INET6)
- src_in6.sin6_port = 0;
- else
- ((struct sockaddr_in *)&src_in6)->sin_port = 0; /* AF_INET & AF_SCI */
-
what = "bind before connect";
- err = sock->ops->bind(sock,
- (struct sockaddr *) &src_in6,
- tconn->net_conf->my_addr_len);
+ err = sock->ops->bind(sock, (struct sockaddr *) &src_in6, my_addr_len);
if (err < 0)
goto out;
@@ -651,9 +672,7 @@ static struct socket *drbd_try_connect(struct drbd_tconn *tconn)
* stay C_WF_CONNECTION, don't go Disconnecting! */
disconnect_on_error = 0;
what = "connect";
- err = sock->ops->connect(sock,
- (struct sockaddr *)tconn->net_conf->peer_addr,
- tconn->net_conf->peer_addr_len, 0);
+ err = sock->ops->connect(sock, (struct sockaddr *) &peer_in6, peer_addr_len, 0);
out:
if (err < 0) {
@@ -676,40 +695,52 @@ out:
if (disconnect_on_error)
conn_request_state(tconn, NS(conn, C_DISCONNECTING), CS_HARD);
}
- put_net_conf(tconn);
+
return sock;
}
static struct socket *drbd_wait_for_connect(struct drbd_tconn *tconn)
{
- int timeo, err;
+ int timeo, err, my_addr_len;
+ int sndbuf_size, rcvbuf_size, try_connect_int;
struct socket *s_estab = NULL, *s_listen;
+ struct sockaddr_in6 my_addr;
+ struct net_conf *nc;
const char *what;
- if (!get_net_conf(tconn))
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ if (!nc) {
+ rcu_read_unlock();
return NULL;
+ }
+
+ sndbuf_size = nc->sndbuf_size;
+ rcvbuf_size = nc->rcvbuf_size;
+ try_connect_int = nc->try_connect_int;
+
+ my_addr_len = min_t(int, nc->my_addr_len, sizeof(struct sockaddr_in6));
+ memcpy(&my_addr, nc->my_addr, my_addr_len);
+ rcu_read_unlock();
what = "sock_create_kern";
- err = sock_create_kern(((struct sockaddr *)tconn->net_conf->my_addr)->sa_family,
+ err = sock_create_kern(((struct sockaddr *)&my_addr)->sa_family,
SOCK_STREAM, IPPROTO_TCP, &s_listen);
if (err) {
s_listen = NULL;
goto out;
}
- timeo = tconn->net_conf->try_connect_int * HZ;
+ timeo = try_connect_int * HZ;
timeo += (random32() & 1) ? timeo / 7 : -timeo / 7; /* 28.5% random jitter */
s_listen->sk->sk_reuse = 1; /* SO_REUSEADDR */
s_listen->sk->sk_rcvtimeo = timeo;
s_listen->sk->sk_sndtimeo = timeo;
- drbd_setbufsize(s_listen, tconn->net_conf->sndbuf_size,
- tconn->net_conf->rcvbuf_size);
+ drbd_setbufsize(s_listen, sndbuf_size, rcvbuf_size);
what = "bind before listen";
- err = s_listen->ops->bind(s_listen,
- (struct sockaddr *) tconn->net_conf->my_addr,
- tconn->net_conf->my_addr_len);
+ err = s_listen->ops->bind(s_listen, (struct sockaddr *)&my_addr, my_addr_len);
if (err < 0)
goto out;
@@ -724,7 +755,6 @@ out:
conn_request_state(tconn, NS(conn, C_DISCONNECTING), CS_HARD);
}
}
- put_net_conf(tconn);
return s_estab;
}
@@ -817,7 +847,8 @@ int drbd_connected(int vnr, void *p, void *data)
static int drbd_connect(struct drbd_tconn *tconn)
{
struct socket *sock, *msock;
- int try, h, ok;
+ struct net_conf *nc;
+ int timeout, try, h, ok;
if (conn_request_state(tconn, NS(conn, C_WF_CONNECTION), CS_VERBOSE) < SS_SUCCESS)
return -2;
@@ -924,11 +955,17 @@ retry:
* sock->sk->sk_rcvtimeo = MAX_SCHEDULE_TIMEOUT;
* first set it to the P_CONNECTION_FEATURES timeout,
* which we set to 4x the configured ping_timeout. */
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+
sock->sk->sk_sndtimeo =
- sock->sk->sk_rcvtimeo = tconn->net_conf->ping_timeo*4*HZ/10;
+ sock->sk->sk_rcvtimeo = nc->ping_timeo*4*HZ/10;
+
+ msock->sk->sk_rcvtimeo = nc->ping_int*HZ;
+ timeout = nc->timeout * HZ / 10;
+ rcu_read_unlock();
- msock->sk->sk_sndtimeo = tconn->net_conf->timeout*HZ/10;
- msock->sk->sk_rcvtimeo = tconn->net_conf->ping_int*HZ;
+ msock->sk->sk_sndtimeo = timeout;
/* we don't want delays.
* we use TCP_CORK where appropriate, though */
@@ -956,7 +993,7 @@ retry:
if (conn_request_state(tconn, NS(conn, C_WF_REPORT_PARAMS), CS_VERBOSE) < SS_SUCCESS)
return 0;
- sock->sk->sk_sndtimeo = tconn->net_conf->timeout*HZ/10;
+ sock->sk->sk_sndtimeo = timeout;
sock->sk->sk_rcvtimeo = MAX_SCHEDULE_TIMEOUT;
drbd_thread_start(&tconn->asender);
@@ -1842,7 +1879,9 @@ static int wait_for_and_update_peer_seq(struct drbd_conf *mdev, const u32 peer_s
}
prepare_to_wait(&mdev->seq_wait, &wait, TASK_INTERRUPTIBLE);
spin_unlock(&mdev->peer_seq_lock);
- timeout = mdev->tconn->net_conf->ping_timeo*HZ/10;
+ rcu_read_lock();
+ timeout = rcu_dereference(mdev->tconn->net_conf)->ping_timeo*HZ/10;
+ rcu_read_unlock();
timeout = schedule_timeout(timeout);
spin_lock(&mdev->peer_seq_lock);
if (!timeout) {
@@ -2075,7 +2114,8 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
spin_unlock_irq(&mdev->tconn->req_lock);
if (mdev->tconn->agreed_pro_version < 100) {
- switch (mdev->tconn->net_conf->wire_protocol) {
+ rcu_read_lock();
+ switch (rcu_dereference(mdev->tconn->net_conf)->wire_protocol) {
case DRBD_PROT_C:
dp_flags |= DP_SEND_WRITE_ACK;
break;
@@ -2083,6 +2123,7 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
dp_flags |= DP_SEND_RECEIVE_ACK;
break;
}
+ rcu_read_unlock();
}
if (dp_flags & DP_SEND_WRITE_ACK) {
@@ -2385,6 +2426,7 @@ static int drbd_asb_recover_0p(struct drbd_conf *mdev) __must_hold(local)
{
int self, peer, rv = -100;
unsigned long ch_self, ch_peer;
+ enum drbd_after_sb_p after_sb_0p;
self = mdev->ldev->md.uuid[UI_BITMAP] & 1;
peer = mdev->p_uuid[UI_BITMAP] & 1;
@@ -2392,10 +2434,14 @@ static int drbd_asb_recover_0p(struct drbd_conf *mdev) __must_hold(local)
ch_peer = mdev->p_uuid[UI_SIZE];
ch_self = mdev->comm_bm_set;
- switch (mdev->tconn->net_conf->after_sb_0p) {
+ rcu_read_lock();
+ after_sb_0p = rcu_dereference(mdev->tconn->net_conf)->after_sb_0p;
+ rcu_read_unlock();
+ switch (after_sb_0p) {
case ASB_CONSENSUS:
case ASB_DISCARD_SECONDARY:
case ASB_CALL_HELPER:
+ case ASB_VIOLENTLY:
dev_err(DEV, "Configuration error.\n");
break;
case ASB_DISCONNECT:
@@ -2431,7 +2477,7 @@ static int drbd_asb_recover_0p(struct drbd_conf *mdev) __must_hold(local)
if (ch_peer == 0) { rv = 1; break; }
if (ch_self == 0) { rv = -1; break; }
}
- if (mdev->tconn->net_conf->after_sb_0p == ASB_DISCARD_ZERO_CHG)
+ if (after_sb_0p == ASB_DISCARD_ZERO_CHG)
break;
case ASB_DISCARD_LEAST_CHG:
if (ch_self < ch_peer)
@@ -2456,13 +2502,18 @@ static int drbd_asb_recover_0p(struct drbd_conf *mdev) __must_hold(local)
static int drbd_asb_recover_1p(struct drbd_conf *mdev) __must_hold(local)
{
int hg, rv = -100;
+ enum drbd_after_sb_p after_sb_1p;
- switch (mdev->tconn->net_conf->after_sb_1p) {
+ rcu_read_lock();
+ after_sb_1p = rcu_dereference(mdev->tconn->net_conf)->after_sb_1p;
+ rcu_read_unlock();
+ switch (after_sb_1p) {
case ASB_DISCARD_YOUNGER_PRI:
case ASB_DISCARD_OLDER_PRI:
case ASB_DISCARD_LEAST_CHG:
case ASB_DISCARD_LOCAL:
case ASB_DISCARD_REMOTE:
+ case ASB_DISCARD_ZERO_CHG:
dev_err(DEV, "Configuration error.\n");
break;
case ASB_DISCONNECT:
@@ -2505,8 +2556,12 @@ static int drbd_asb_recover_1p(struct drbd_conf *mdev) __must_hold(local)
static int drbd_asb_recover_2p(struct drbd_conf *mdev) __must_hold(local)
{
int hg, rv = -100;
+ enum drbd_after_sb_p after_sb_2p;
- switch (mdev->tconn->net_conf->after_sb_2p) {
+ rcu_read_lock();
+ after_sb_2p = rcu_dereference(mdev->tconn->net_conf)->after_sb_2p;
+ rcu_read_unlock();
+ switch (after_sb_2p) {
case ASB_DISCARD_YOUNGER_PRI:
case ASB_DISCARD_OLDER_PRI:
case ASB_DISCARD_LEAST_CHG:
@@ -2514,6 +2569,7 @@ static int drbd_asb_recover_2p(struct drbd_conf *mdev) __must_hold(local)
case ASB_DISCARD_REMOTE:
case ASB_CONSENSUS:
case ASB_DISCARD_SECONDARY:
+ case ASB_DISCARD_ZERO_CHG:
dev_err(DEV, "Configuration error.\n");
break;
case ASB_VIOLENTLY:
@@ -2758,9 +2814,10 @@ static int drbd_uuid_compare(struct drbd_conf *mdev, int *rule_nr) __must_hold(l
static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_role peer_role,
enum drbd_disk_state peer_disk) __must_hold(local)
{
- int hg, rule_nr;
enum drbd_conns rv = C_MASK;
enum drbd_disk_state mydisk;
+ struct net_conf *nc;
+ int hg, rule_nr, rr_conflict, dry_run;
mydisk = mdev->state.disk;
if (mydisk == D_NEGOTIATING)
@@ -2797,7 +2854,10 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
if (abs(hg) == 100)
drbd_khelper(mdev, "initial-split-brain");
- if (hg == 100 || (hg == -100 && mdev->tconn->net_conf->always_asbp)) {
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+
+ if (hg == 100 || (hg == -100 && nc->always_asbp)) {
int pcount = (mdev->state.role == R_PRIMARY)
+ (peer_role == R_PRIMARY);
int forced = (hg == -100);
@@ -2826,9 +2886,9 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
}
if (hg == -100) {
- if (mdev->tconn->net_conf->want_lose && !(mdev->p_uuid[UI_FLAGS]&1))
+ if (nc->want_lose && !(mdev->p_uuid[UI_FLAGS]&1))
hg = -1;
- if (!mdev->tconn->net_conf->want_lose && (mdev->p_uuid[UI_FLAGS]&1))
+ if (!nc->want_lose && (mdev->p_uuid[UI_FLAGS]&1))
hg = 1;
if (abs(hg) < 100)
@@ -2836,6 +2896,9 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
"Sync from %s node\n",
(hg < 0) ? "peer" : "this");
}
+ rr_conflict = nc->rr_conflict;
+ dry_run = nc->dry_run;
+ rcu_read_unlock();
if (hg == -100) {
/* FIXME this log message is not correct if we end up here
@@ -2854,7 +2917,7 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
if (hg < 0 && /* by intention we do not use mydisk here. */
mdev->state.role == R_PRIMARY && mdev->state.disk >= D_CONSISTENT) {
- switch (mdev->tconn->net_conf->rr_conflict) {
+ switch (rr_conflict) {
case ASB_CALL_HELPER:
drbd_khelper(mdev, "pri-lost");
/* fall through */
@@ -2867,7 +2930,7 @@ static enum drbd_conns drbd_sync_handshake(struct drbd_conf *mdev, enum drbd_rol
}
}
- if (mdev->tconn->net_conf->dry_run || test_bit(CONN_DRY_RUN, &mdev->tconn->flags)) {
+ if (dry_run || test_bit(CONN_DRY_RUN, &mdev->tconn->flags)) {
if (hg == 0)
dev_info(DEV, "dry-run connect: No resync, would become Connected immediately.\n");
else
@@ -2926,6 +2989,8 @@ static int receive_protocol(struct drbd_tconn *tconn, struct packet_info *pi)
int p_proto, p_after_sb_0p, p_after_sb_1p, p_after_sb_2p;
int p_want_lose, p_two_primaries, cf;
char p_integrity_alg[SHARED_SECRET_MAX] = "";
+ unsigned char *my_alg;
+ struct net_conf *nc;
p_proto = be32_to_cpu(p->protocol);
p_after_sb_0p = be32_to_cpu(p->after_sb_0p);
@@ -2940,38 +3005,43 @@ static int receive_protocol(struct drbd_tconn *tconn, struct packet_info *pi)
if (cf & CF_DRY_RUN)
set_bit(CONN_DRY_RUN, &tconn->flags);
- if (p_proto != tconn->net_conf->wire_protocol && tconn->agreed_pro_version < 100) {
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+
+ if (p_proto != nc->wire_protocol && tconn->agreed_pro_version < 100) {
conn_err(tconn, "incompatible communication protocols\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (cmp_after_sb(p_after_sb_0p, tconn->net_conf->after_sb_0p)) {
+ if (cmp_after_sb(p_after_sb_0p, nc->after_sb_0p)) {
conn_err(tconn, "incompatible after-sb-0pri settings\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (cmp_after_sb(p_after_sb_1p, tconn->net_conf->after_sb_1p)) {
+ if (cmp_after_sb(p_after_sb_1p, nc->after_sb_1p)) {
conn_err(tconn, "incompatible after-sb-1pri settings\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (cmp_after_sb(p_after_sb_2p, tconn->net_conf->after_sb_2p)) {
+ if (cmp_after_sb(p_after_sb_2p, nc->after_sb_2p)) {
conn_err(tconn, "incompatible after-sb-2pri settings\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (p_want_lose && tconn->net_conf->want_lose) {
+ if (p_want_lose && nc->want_lose) {
conn_err(tconn, "both sides have the 'want_lose' flag set\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
- if (p_two_primaries != tconn->net_conf->two_primaries) {
+ if (p_two_primaries != nc->two_primaries) {
conn_err(tconn, "incompatible setting of the two-primaries options\n");
- goto disconnect;
+ goto disconnect_rcu_unlock;
}
+ my_alg = nc->integrity_alg;
+ rcu_read_unlock();
+
if (tconn->agreed_pro_version >= 87) {
- unsigned char *my_alg = tconn->net_conf->integrity_alg;
int err;
err = drbd_recv_all(tconn, p_integrity_alg, pi->size);
@@ -2989,6 +3059,8 @@ static int receive_protocol(struct drbd_tconn *tconn, struct packet_info *pi)
return 0;
+disconnect_rcu_unlock:
+ rcu_read_unlock();
disconnect:
conn_request_state(tconn, NS(conn, C_DISCONNECTING), CS_HARD);
return -EIO;
@@ -4322,19 +4394,26 @@ static int drbd_do_auth(struct drbd_tconn *tconn)
char *response = NULL;
char *right_response = NULL;
char *peers_ch = NULL;
- unsigned int key_len = strlen(tconn->net_conf->shared_secret);
+ unsigned int key_len;
+ char secret[SHARED_SECRET_MAX]; /* 64 byte */
unsigned int resp_size;
struct hash_desc desc;
struct packet_info pi;
+ struct net_conf *nc;
int err, rv;
/* FIXME: Put the challenge/response into the preallocated socket buffer. */
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ key_len = strlen(nc->shared_secret);
+ memcpy(secret, nc->shared_secret, key_len);
+ rcu_read_unlock();
+
desc.tfm = tconn->cram_hmac_tfm;
desc.flags = 0;
- rv = crypto_hash_setkey(tconn->cram_hmac_tfm,
- (u8 *)tconn->net_conf->shared_secret, key_len);
+ rv = crypto_hash_setkey(tconn->cram_hmac_tfm, (u8 *)secret, key_len);
if (rv) {
conn_err(tconn, "crypto_hash_setkey() failed with %d\n", rv);
rv = -1;
@@ -4456,8 +4535,8 @@ static int drbd_do_auth(struct drbd_tconn *tconn)
rv = !memcmp(response, right_response, resp_size);
if (rv)
- conn_info(tconn, "Peer authenticated using %d bytes of '%s' HMAC\n",
- resp_size, tconn->net_conf->cram_hmac_alg);
+ conn_info(tconn, "Peer authenticated using %d bytes HMAC\n",
+ resp_size);
else
rv = -1;
@@ -4884,33 +4963,42 @@ int drbd_asender(struct drbd_thread *thi)
int received = 0;
unsigned int header_size = drbd_header_size(tconn);
int expect = header_size;
- int ping_timeout_active = 0;
+ bool ping_timeout_active = false;
+ struct net_conf *nc;
+ int ping_timeo, no_cork, ping_int;
current->policy = SCHED_RR; /* Make this a realtime task! */
current->rt_priority = 2; /* more important than all other tasks */
while (get_t_state(thi) == RUNNING) {
drbd_thread_current_set_cpu(thi);
+
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ ping_timeo = nc->ping_timeo;
+ no_cork = nc->no_cork;
+ ping_int = nc->ping_int;
+ rcu_read_unlock();
+
if (test_and_clear_bit(SEND_PING, &tconn->flags)) {
if (drbd_send_ping(tconn)) {
conn_err(tconn, "drbd_send_ping has failed\n");
goto reconnect;
}
- tconn->meta.socket->sk->sk_rcvtimeo =
- tconn->net_conf->ping_timeo*HZ/10;
- ping_timeout_active = 1;
+ tconn->meta.socket->sk->sk_rcvtimeo = ping_timeo * HZ / 10;
+ ping_timeout_active = true;
}
/* TODO: conditionally cork; it may hurt latency if we cork without
much to send */
- if (!tconn->net_conf->no_cork)
+ if (!no_cork)
drbd_tcp_cork(tconn->meta.socket);
if (tconn_finish_peer_reqs(tconn)) {
conn_err(tconn, "tconn_finish_peer_reqs() failed\n");
goto reconnect;
}
/* but unconditionally uncork unless disabled */
- if (!tconn->net_conf->no_cork)
+ if (!no_cork)
drbd_tcp_uncork(tconn->meta.socket);
/* short circuit, recv_msg would return EINTR anyways. */
@@ -4984,10 +5072,11 @@ int drbd_asender(struct drbd_thread *thi)
tconn->last_received = jiffies;
- /* the idle_timeout (ping-int)
- * has been restored in got_PingAck() */
- if (cmd == &asender_tbl[P_PING_ACK])
- ping_timeout_active = 0;
+ if (cmd == &asender_tbl[P_PING_ACK]) {
+ /* restore idle timeout */
+ tconn->meta.socket->sk->sk_rcvtimeo = ping_int * HZ;
+ ping_timeout_active = false;
+ }
buf = tconn->meta.rbuf;
received = 0;
diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c
index 0f1a29f..c4e4553 100644
--- a/drivers/block/drbd/drbd_req.c
+++ b/drivers/block/drbd/drbd_req.c
@@ -323,6 +323,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
struct bio_and_error *m)
{
struct drbd_conf *mdev = req->w.mdev;
+ struct net_conf *nc;
int p, rv = 0;
if (m)
@@ -344,7 +345,10 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
* and from w_read_retry_remote */
D_ASSERT(!(req->rq_state & RQ_NET_MASK));
req->rq_state |= RQ_NET_PENDING;
- p = mdev->tconn->net_conf->wire_protocol;
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ p = nc->wire_protocol;
+ rcu_read_unlock();
req->rq_state |=
p == DRBD_PROT_C ? RQ_EXP_WRITE_ACK :
p == DRBD_PROT_B ? RQ_EXP_RECEIVE_ACK : 0;
@@ -474,7 +478,11 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
drbd_queue_work(&mdev->tconn->data.work, &req->w);
/* close the epoch, in case it outgrew the limit */
- if (mdev->tconn->newest_tle->n_writes >= mdev->tconn->net_conf->max_epoch_size)
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ p = nc->max_epoch_size;
+ rcu_read_unlock();
+ if (mdev->tconn->newest_tle->n_writes >= p)
queue_barrier(mdev);
break;
@@ -729,6 +737,7 @@ int __drbd_make_request(struct drbd_conf *mdev, struct bio *bio, unsigned long s
const sector_t sector = bio->bi_sector;
struct drbd_tl_epoch *b = NULL;
struct drbd_request *req;
+ struct net_conf *nc;
int local, remote, send_oos = 0;
int err;
int ret = 0;
@@ -935,17 +944,19 @@ allocate_barrier:
if (send_oos && drbd_set_out_of_sync(mdev, sector, size))
_req_mod(req, QUEUE_FOR_SEND_OOS);
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
if (remote &&
- mdev->tconn->net_conf->on_congestion != OC_BLOCK && mdev->tconn->agreed_pro_version >= 96) {
+ nc->on_congestion != OC_BLOCK && mdev->tconn->agreed_pro_version >= 96) {
int congested = 0;
- if (mdev->tconn->net_conf->cong_fill &&
- atomic_read(&mdev->ap_in_flight) >= mdev->tconn->net_conf->cong_fill) {
+ if (nc->cong_fill &&
+ atomic_read(&mdev->ap_in_flight) >= nc->cong_fill) {
dev_info(DEV, "Congestion-fill threshold reached\n");
congested = 1;
}
- if (mdev->act_log->used >= mdev->tconn->net_conf->cong_extents) {
+ if (mdev->act_log->used >= nc->cong_extents) {
dev_info(DEV, "Congestion-extents threshold reached\n");
congested = 1;
}
@@ -953,12 +964,13 @@ allocate_barrier:
if (congested) {
queue_barrier(mdev); /* last barrier, after mirrored writes */
- if (mdev->tconn->net_conf->on_congestion == OC_PULL_AHEAD)
+ if (nc->on_congestion == OC_PULL_AHEAD)
_drbd_set_state(_NS(mdev, conn, C_AHEAD), 0, NULL);
- else /*mdev->tconn->net_conf->on_congestion == OC_DISCONNECT */
+ else /*nc->on_congestion == OC_DISCONNECT */
_drbd_set_state(_NS(mdev, conn, C_DISCONNECTING), 0, NULL);
}
}
+ rcu_read_unlock();
spin_unlock_irq(&mdev->tconn->req_lock);
kfree(b); /* if someone else has beaten us to it... */
@@ -1058,12 +1070,14 @@ void request_timer_fn(unsigned long data)
struct drbd_tconn *tconn = mdev->tconn;
struct drbd_request *req; /* oldest request */
struct list_head *le;
- unsigned long et = 0; /* effective timeout = ko_count * timeout */
+ struct net_conf *nc;
+ unsigned long et; /* effective timeout = ko_count * timeout */
+
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ et = nc ? nc->timeout * HZ/10 * nc->ko_count : 0;
+ rcu_read_unlock();
- if (get_net_conf(tconn)) {
- et = tconn->net_conf->timeout*HZ/10 * tconn->net_conf->ko_count;
- put_net_conf(tconn);
- }
if (!et || mdev->state.conn < C_WF_REPORT_PARAMS)
return; /* Recurring timer stopped */
diff --git a/drivers/block/drbd/drbd_state.c b/drivers/block/drbd/drbd_state.c
index 0aed33b..2423283 100644
--- a/drivers/block/drbd/drbd_state.c
+++ b/drivers/block/drbd/drbd_state.c
@@ -464,6 +464,7 @@ is_valid_state(struct drbd_conf *mdev, union drbd_state ns)
enum drbd_fencing_p fp;
enum drbd_state_rv rv = SS_SUCCESS;
+ struct net_conf *nc;
fp = FP_DONT_CARE;
if (get_ldev(mdev)) {
@@ -471,14 +472,15 @@ is_valid_state(struct drbd_conf *mdev, union drbd_state ns)
put_ldev(mdev);
}
- if (get_net_conf(mdev->tconn)) {
- if (!mdev->tconn->net_conf->two_primaries && ns.role == R_PRIMARY) {
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ if (nc) {
+ if (!nc->two_primaries && ns.role == R_PRIMARY) {
if (ns.peer == R_PRIMARY)
rv = SS_TWO_PRIMARIES;
else if (conn_highest_peer(mdev->tconn) == R_PRIMARY)
rv = SS_O_VOL_PEER_PRI;
- }
- put_net_conf(mdev->tconn);
+ }
}
if (rv <= 0)
@@ -513,7 +515,7 @@ is_valid_state(struct drbd_conf *mdev, union drbd_state ns)
rv = SS_CONNECTED_OUTDATES;
else if ((ns.conn == C_VERIFY_S || ns.conn == C_VERIFY_T) &&
- (mdev->tconn->net_conf->verify_alg[0] == 0))
+ (nc->verify_alg[0] == 0))
rv = SS_NO_VERIFY_ALG;
else if ((ns.conn == C_VERIFY_S || ns.conn == C_VERIFY_T) &&
@@ -523,6 +525,8 @@ is_valid_state(struct drbd_conf *mdev, union drbd_state ns)
else if (ns.conn >= C_CONNECTED && ns.pdsk == D_UNKNOWN)
rv = SS_CONNECTED_OUTDATES;
+ rcu_read_unlock();
+
return rv;
}
diff --git a/drivers/block/drbd/drbd_worker.c b/drivers/block/drbd/drbd_worker.c
index b63c08e..c54c191 100644
--- a/drivers/block/drbd/drbd_worker.c
+++ b/drivers/block/drbd/drbd_worker.c
@@ -1621,10 +1621,16 @@ void drbd_start_resync(struct drbd_conf *mdev, enum drbd_conns side)
* detect connection loss, then waiting for a ping
* response (implicit in drbd_resync_finished) reduces
* the race considerably, but does not solve it. */
- if (side == C_SYNC_SOURCE)
- schedule_timeout_interruptible(
- mdev->tconn->net_conf->ping_int * HZ +
- mdev->tconn->net_conf->ping_timeo*HZ/9);
+ if (side == C_SYNC_SOURCE) {
+ struct net_conf *nc;
+ int timeo;
+
+ rcu_read_lock();
+ nc = rcu_dereference(mdev->tconn->net_conf);
+ timeo = nc->ping_int * HZ + nc->ping_timeo * HZ / 9;
+ rcu_read_unlock();
+ schedule_timeout_interruptible(timeo);
+ }
drbd_resync_finished(mdev);
}
@@ -1647,22 +1653,30 @@ int drbd_worker(struct drbd_thread *thi)
struct drbd_tconn *tconn = thi->tconn;
struct drbd_work *w = NULL;
struct drbd_conf *mdev;
+ struct net_conf *nc;
LIST_HEAD(work_list);
int vnr, intr = 0;
+ int cork;
while (get_t_state(thi) == RUNNING) {
drbd_thread_current_set_cpu(thi);
if (down_trylock(&tconn->data.work.s)) {
mutex_lock(&tconn->data.mutex);
- if (tconn->data.socket && !tconn->net_conf->no_cork)
+
+ rcu_read_lock();
+ nc = rcu_dereference(tconn->net_conf);
+ cork = nc ? !nc->no_cork : 0;
+ rcu_read_unlock();
+
+ if (tconn->data.socket && cork)
drbd_tcp_uncork(tconn->data.socket);
mutex_unlock(&tconn->data.mutex);
intr = down_interruptible(&tconn->data.work.s);
mutex_lock(&tconn->data.mutex);
- if (tconn->data.socket && !tconn->net_conf->no_cork)
+ if (tconn->data.socket && cork)
drbd_tcp_cork(tconn->data.socket);
mutex_unlock(&tconn->data.mutex);
}
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 10/11] drbd: Proper locking for updates to net_conf under RCU
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (8 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 09/11] drbd: rcu_read_lock() and rcu_dereference() for tconn->net_conf Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
2011-09-30 12:48 ` [PATCH 11/11] drbd: Considering that the two_primaries config flag can change Philipp Reisner
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Removing the get_net_conf()/put_net_conf() functions
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 29 +-----------
drivers/block/drbd/drbd_main.c | 17 ++++++-
drivers/block/drbd/drbd_nl.c | 67 ++++++++++++----------------
drivers/block/drbd/drbd_receiver.c | 83 +++++++++++++++++++++++-------------
4 files changed, 100 insertions(+), 96 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 99da54c..83e6cad 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -833,9 +833,8 @@ struct drbd_tconn { /* is a resource from the config file */
unsigned long flags;
struct net_conf *net_conf; /* content protected by rcu */
- atomic_t net_cnt; /* Users of net_conf */
- wait_queue_head_t net_cnt_wait;
- wait_queue_head_t ping_wait; /* Woken upon reception of a ping, and a state change */
+ struct mutex net_conf_update; /* mutex for ready-copy-update of net_conf */
+ wait_queue_head_t ping_wait; /* Woken upon reception of a ping, and a state change */
struct res_opts res_opts;
struct drbd_socket data; /* data/barrier/cstate/parameter packets */
@@ -1379,6 +1378,7 @@ extern void drbd_delete_device(struct drbd_conf *mdev);
struct drbd_tconn *drbd_new_tconn(const char *name);
extern void drbd_free_tconn(struct drbd_tconn *tconn);
struct drbd_tconn *conn_by_name(const char *name);
+extern void conn_free_crypto(struct drbd_tconn *tconn);
extern int proc_details;
@@ -1935,29 +1935,6 @@ static inline void _sub_unacked(struct drbd_conf *mdev, int n, const char *func,
ERR_IF_CNT_IS_NEGATIVE(unacked_cnt, func, line);
}
-static inline void put_net_conf(struct drbd_tconn *tconn)
-{
- if (atomic_dec_and_test(&tconn->net_cnt))
- wake_up(&tconn->net_cnt_wait);
-}
-
-/**
- * get_net_conf() - Increase ref count on mdev->tconn->net_conf; Returns 0 if nothing there
- * @mdev: DRBD device.
- *
- * You have to call put_net_conf() when finished working with mdev->tconn->net_conf.
- */
-static inline int get_net_conf(struct drbd_tconn *tconn)
-{
- int have_net_conf;
-
- atomic_inc(&tconn->net_cnt);
- have_net_conf = tconn->cstate >= C_UNCONNECTED;
- if (!have_net_conf)
- put_net_conf(tconn);
- return have_net_conf;
-}
-
/**
* get_ldev() - Increase the ref count on mdev->ldev. Returns 0 if there is no ldev
* @M: DRBD device.
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index d335ecc..5eb60e8 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2376,6 +2376,20 @@ static void drbd_free_socket(struct drbd_socket *socket)
free_page((unsigned long) socket->rbuf);
}
+void conn_free_crypto(struct drbd_tconn *tconn)
+{
+ crypto_free_hash(tconn->cram_hmac_tfm);
+ crypto_free_hash(tconn->integrity_w_tfm);
+ crypto_free_hash(tconn->integrity_r_tfm);
+ kfree(tconn->int_dig_in);
+ kfree(tconn->int_dig_vv);
+ tconn->cram_hmac_tfm = NULL;
+ tconn->integrity_w_tfm = NULL;
+ tconn->integrity_r_tfm = NULL;
+ tconn->int_dig_in = NULL;
+ tconn->int_dig_vv = NULL;
+}
+
struct drbd_tconn *drbd_new_tconn(const char *name)
{
struct drbd_tconn *tconn;
@@ -2402,8 +2416,7 @@ struct drbd_tconn *drbd_new_tconn(const char *name)
tconn->cstate = C_STANDALONE;
mutex_init(&tconn->cstate_mutex);
spin_lock_init(&tconn->req_lock);
- atomic_set(&tconn->net_cnt, 0);
- init_waitqueue_head(&tconn->net_cnt_wait);
+ mutex_init(&tconn->net_conf_update);
init_waitqueue_head(&tconn->ping_wait);
idr_init(&tconn->volumes);
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index 6d28e7c..bafa457 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -615,11 +615,12 @@ drbd_set_role(struct drbd_conf *mdev, enum drbd_role new_role, int force)
put_ldev(mdev);
}
} else {
- rcu_read_lock();
- nc = rcu_dereference(mdev->tconn->net_conf);
+ mutex_lock(&mdev->tconn->net_conf_update);
+ nc = mdev->tconn->net_conf;
if (nc)
- nc->want_lose = 0;
- rcu_read_unlock();
+ nc->want_lose = 0; /* without copy; single bit op is atomic */
+ mutex_unlock(&mdev->tconn->net_conf_update);
+
set_disk_ro(mdev->vdisk, false);
if (get_ldev(mdev)) {
if (((mdev->state.conn < C_CONNECTED ||
@@ -1809,17 +1810,16 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
conn_reconfig_start(tconn);
- rcu_read_lock();
- old_conf = rcu_dereference(tconn->net_conf);
+ mutex_lock(&tconn->net_conf_update);
+ old_conf = tconn->net_conf;
if (!old_conf) {
drbd_msg_put_info("net conf missing, try connect");
retcode = ERR_INVALID_REQUEST;
- goto fail_rcu_unlock;
+ goto fail;
}
*new_conf = *old_conf;
- rcu_read_unlock();
err = net_conf_from_attrs(new_conf, info);
if (err) {
@@ -1834,13 +1834,10 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
/* re-sync running */
rsr = conn_resync_running(tconn);
- rcu_read_lock();
- old_conf = rcu_dereference(tconn->net_conf);
if (rsr && old_conf && strcmp(new_conf->csums_alg, old_conf->csums_alg)) {
retcode = ERR_CSUMS_RESYNC_RUNNING;
- goto fail_rcu_unlock;
+ goto fail;
}
- rcu_read_unlock();
if (!rsr && new_conf->csums_alg[0]) {
csums_tfm = crypto_alloc_hash(new_conf->csums_alg, 0, CRYPTO_ALG_ASYNC);
@@ -1858,15 +1855,12 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
/* online verify running */
ovr = conn_ov_running(tconn);
- rcu_read_lock();
- old_conf = rcu_dereference(tconn->net_conf);
- if (ovr && old_conf) {
+ if (ovr) {
if (strcmp(new_conf->verify_alg, old_conf->verify_alg)) {
retcode = ERR_VERIFY_RUNNING;
- goto fail_rcu_unlock;
+ goto fail;
}
}
- rcu_read_unlock();
if (!ovr && new_conf->verify_alg[0]) {
verify_tfm = crypto_alloc_hash(new_conf->verify_alg, 0, CRYPTO_ALG_ASYNC);
@@ -1883,8 +1877,6 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
}
rcu_assign_pointer(tconn->net_conf, new_conf);
- synchronize_rcu();
- kfree(old_conf);
if (!rsr) {
crypto_free_hash(tconn->csums_tfm);
@@ -1897,12 +1889,17 @@ int drbd_adm_net_opts(struct sk_buff *skb, struct genl_info *info)
verify_tfm = NULL;
}
+ mutex_unlock(&tconn->net_conf_update);
+ synchronize_rcu();
+ kfree(old_conf);
+
if (tconn->cstate >= C_WF_REPORT_PARAMS)
drbd_send_sync_param(minor_to_mdev(conn_lowest_minor(tconn)));
- fail_rcu_unlock:
- rcu_read_unlock();
+ goto out;
+
fail:
+ mutex_unlock(&tconn->net_conf_update);
crypto_free_hash(csums_tfm);
crypto_free_hash(verify_tfm);
kfree(new_conf);
@@ -2081,32 +2078,26 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
}
conn_flush_workqueue(tconn);
- spin_lock_irq(&tconn->req_lock);
- rcu_read_lock();
- old_conf = rcu_dereference(tconn->net_conf);
- if (old_conf != NULL) {
+
+ mutex_lock(&tconn->net_conf_update);
+ old_conf = tconn->net_conf;
+ if (old_conf) {
retcode = ERR_NET_CONFIGURED;
- rcu_read_unlock();
- spin_unlock_irq(&tconn->req_lock);
+ mutex_unlock(&tconn->net_conf_update);
goto fail;
}
rcu_assign_pointer(tconn->net_conf, new_conf);
- crypto_free_hash(tconn->cram_hmac_tfm);
+ conn_free_crypto(tconn);
tconn->cram_hmac_tfm = tfm;
-
- crypto_free_hash(tconn->integrity_w_tfm);
tconn->integrity_w_tfm = integrity_w_tfm;
-
- crypto_free_hash(tconn->integrity_r_tfm);
tconn->integrity_r_tfm = integrity_r_tfm;
+ tconn->int_dig_in = int_dig_in;
+ tconn->int_dig_vv = int_dig_vv;
- kfree(tconn->int_dig_in);
- kfree(tconn->int_dig_vv);
- tconn->int_dig_in=int_dig_in;
- tconn->int_dig_vv=int_dig_vv;
- retcode = _conn_request_state(tconn, NS(conn, C_UNCONNECTED), CS_VERBOSE);
- spin_unlock_irq(&tconn->req_lock);
+ mutex_unlock(&tconn->net_conf_update);
+
+ retcode = conn_request_state(tconn, NS(conn, C_UNCONNECTED), CS_VERBOSE);
rcu_read_lock();
idr_for_each_entry(&tconn->volumes, mdev, i) {
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index a37f2ee..05b214c 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -3138,6 +3138,7 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
unsigned int header_size, data_size, exp_max_sz;
struct crypto_hash *verify_tfm = NULL;
struct crypto_hash *csums_tfm = NULL;
+ struct net_conf *old_conf, *new_conf = NULL;
const int apv = tconn->agreed_pro_version;
int *rs_plan_s = NULL;
int fifo_size = 0;
@@ -3212,10 +3213,13 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
p->csums_alg[SHARED_SECRET_MAX-1] = 0;
}
- if (strcmp(mdev->tconn->net_conf->verify_alg, p->verify_alg)) {
+ mutex_lock(&mdev->tconn->net_conf_update);
+ old_conf = mdev->tconn->net_conf;
+
+ if (strcmp(old_conf->verify_alg, p->verify_alg)) {
if (mdev->state.conn == C_WF_REPORT_PARAMS) {
dev_err(DEV, "Different verify-alg settings. me=\"%s\" peer=\"%s\"\n",
- mdev->tconn->net_conf->verify_alg, p->verify_alg);
+ old_conf->verify_alg, p->verify_alg);
goto disconnect;
}
verify_tfm = drbd_crypto_alloc_digest_safe(mdev,
@@ -3226,10 +3230,10 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
}
}
- if (apv >= 89 && strcmp(mdev->tconn->net_conf->csums_alg, p->csums_alg)) {
+ if (apv >= 89 && strcmp(old_conf->csums_alg, p->csums_alg)) {
if (mdev->state.conn == C_WF_REPORT_PARAMS) {
dev_err(DEV, "Different csums-alg settings. me=\"%s\" peer=\"%s\"\n",
- mdev->tconn->net_conf->csums_alg, p->csums_alg);
+ old_conf->csums_alg, p->csums_alg);
goto disconnect;
}
csums_tfm = drbd_crypto_alloc_digest_safe(mdev,
@@ -3259,22 +3263,38 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
put_ldev(mdev);
}
- spin_lock(&mdev->peer_seq_lock);
- /* lock against drbd_nl_syncer_conf() */
- if (verify_tfm) {
- strcpy(mdev->tconn->net_conf->verify_alg, p->verify_alg);
- mdev->tconn->net_conf->verify_alg_len = strlen(p->verify_alg) + 1;
- crypto_free_hash(mdev->tconn->verify_tfm);
- mdev->tconn->verify_tfm = verify_tfm;
- dev_info(DEV, "using verify-alg: \"%s\"\n", p->verify_alg);
+ if (verify_tfm || csums_tfm) {
+ new_conf = kzalloc(sizeof(struct net_conf), GFP_KERNEL);
+ if (!new_conf) {
+ dev_err(DEV, "Allocation of new net_conf failed\n");
+ goto disconnect;
+ }
+
+ *new_conf = *old_conf;
+
+ if (verify_tfm) {
+ strcpy(new_conf->verify_alg, p->verify_alg);
+ new_conf->verify_alg_len = strlen(p->verify_alg) + 1;
+ crypto_free_hash(mdev->tconn->verify_tfm);
+ mdev->tconn->verify_tfm = verify_tfm;
+ dev_info(DEV, "using verify-alg: \"%s\"\n", p->verify_alg);
+ }
+ if (csums_tfm) {
+ strcpy(new_conf->csums_alg, p->csums_alg);
+ new_conf->csums_alg_len = strlen(p->csums_alg) + 1;
+ crypto_free_hash(mdev->tconn->csums_tfm);
+ mdev->tconn->csums_tfm = csums_tfm;
+ dev_info(DEV, "using csums-alg: \"%s\"\n", p->csums_alg);
+ }
+ rcu_assign_pointer(tconn->net_conf, new_conf);
}
- if (csums_tfm) {
- strcpy(mdev->tconn->net_conf->csums_alg, p->csums_alg);
- mdev->tconn->net_conf->csums_alg_len = strlen(p->csums_alg) + 1;
- crypto_free_hash(mdev->tconn->csums_tfm);
- mdev->tconn->csums_tfm = csums_tfm;
- dev_info(DEV, "using csums-alg: \"%s\"\n", p->csums_alg);
+ mutex_unlock(&mdev->tconn->net_conf_update);
+ if (new_conf) {
+ synchronize_rcu();
+ kfree(old_conf);
}
+
+ spin_lock(&mdev->peer_seq_lock);
if (fifo_size != mdev->rs_plan_s.size) {
kfree(mdev->rs_plan_s.values);
mdev->rs_plan_s.values = rs_plan_s;
@@ -3286,6 +3306,7 @@ static int receive_SyncParam(struct drbd_tconn *tconn, struct packet_info *pi)
return 0;
disconnect:
+ mutex_lock(&mdev->tconn->net_conf_update);
/* just for completeness: actually not needed,
* as this is not reached if csums_tfm was ok. */
crypto_free_hash(csums_tfm);
@@ -3715,7 +3736,9 @@ static int receive_state(struct drbd_tconn *tconn, struct packet_info *pi)
}
}
- mdev->tconn->net_conf->want_lose = 0;
+ mutex_lock(&mdev->tconn->net_conf_update);
+ mdev->tconn->net_conf->want_lose = 0; /* without copy; single bit op is atomic */
+ mutex_unlock(&mdev->tconn->net_conf_update);
drbd_md_sync(mdev); /* update connected indicator, la_size, ... */
@@ -4183,13 +4206,17 @@ static void drbd_disconnect(struct drbd_tconn *tconn)
spin_unlock_irq(&tconn->req_lock);
if (oc == C_DISCONNECTING) {
- wait_event(tconn->net_cnt_wait, atomic_read(&tconn->net_cnt) == 0);
+ struct net_conf *old_conf;
- crypto_free_hash(tconn->cram_hmac_tfm);
- tconn->cram_hmac_tfm = NULL;
+ mutex_lock(&tconn->net_conf_update);
+ old_conf = tconn->net_conf;
+ rcu_assign_pointer(tconn->net_conf, NULL);
+ conn_free_crypto(tconn);
+ mutex_unlock(&tconn->net_conf_update);
+
+ synchronize_rcu();
+ kfree(old_conf);
- kfree(tconn->net_conf);
- tconn->net_conf = NULL;
conn_request_state(tconn, NS(conn, C_STANDALONE), CS_VERBOSE);
}
}
@@ -4568,12 +4595,8 @@ int drbdd_init(struct drbd_thread *thi)
}
} while (h == 0);
- if (h > 0) {
- if (get_net_conf(tconn)) {
- drbdd(tconn);
- put_net_conf(tconn);
- }
- }
+ if (h > 0)
+ drbdd(tconn);
drbd_disconnect(tconn);
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 11/11] drbd: Considering that the two_primaries config flag can change
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
` (9 preceding siblings ...)
2011-09-30 12:48 ` [PATCH 10/11] drbd: Proper locking for updates to net_conf under RCU Philipp Reisner
@ 2011-09-30 12:48 ` Philipp Reisner
10 siblings, 0 replies; 12+ messages in thread
From: Philipp Reisner @ 2011-09-30 12:48 UTC (permalink / raw)
To: linux-kernel, Jens Axboe; +Cc: drbd-dev
Now since it is possible to change the two_primaries config
flag while the connection is up, make sure we treat a peer_req
in a consistent way if the config flag changes while the peer_req
is under IO.
Signed-off-by: Philipp Reisner <philipp.reisner@linbit.com>
Signed-off-by: Lars Ellenberg <lars.ellenberg@linbit.com>
---
drivers/block/drbd/drbd_int.h | 4 ++++
drivers/block/drbd/drbd_receiver.c | 19 ++++++++++++++-----
2 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 83e6cad..3833d56 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -661,6 +661,9 @@ enum {
/* The peer wants a write ACK for this (wire proto C) */
__EE_SEND_WRITE_ACK,
+
+ /* Is set when net_conf had two_primaries set while creating this peer_req */
+ __EE_IN_INTERVAL_TREE,
};
#define EE_CALL_AL_COMPLETE_IO (1<<__EE_CALL_AL_COMPLETE_IO)
#define EE_MAY_SET_IN_SYNC (1<<__EE_MAY_SET_IN_SYNC)
@@ -669,6 +672,7 @@ enum {
#define EE_HAS_DIGEST (1<<__EE_HAS_DIGEST)
#define EE_RESTART_REQUESTS (1<<__EE_RESTART_REQUESTS)
#define EE_SEND_WRITE_ACK (1<<__EE_SEND_WRITE_ACK)
+#define EE_IN_INTERVAL_TREE (1<<__EE_IN_INTERVAL_TREE)
/* flag bits per mdev */
enum {
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index 05b214c..ec5b923 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -1752,7 +1752,7 @@ static int e_end_block(struct drbd_work *w, int cancel)
}
/* we delete from the conflict detection hash _after_ we sent out the
* P_WRITE_ACK / P_NEG_ACK, to get the sequence number right. */
- if (mdev->tconn->net_conf->two_primaries) {
+ if (peer_req->flags & EE_IN_INTERVAL_TREE) {
spin_lock_irq(&mdev->tconn->req_lock);
D_ASSERT(!drbd_interval_empty(&peer_req->i));
drbd_remove_epoch_entry_interval(mdev, peer_req);
@@ -1811,14 +1811,19 @@ static u32 seq_max(u32 a, u32 b)
static bool need_peer_seq(struct drbd_conf *mdev)
{
struct drbd_tconn *tconn = mdev->tconn;
+ int tp;
/*
* We only need to keep track of the last packet_seq number of our peer
* if we are in dual-primary mode and we have the discard flag set; see
* handle_write_conflicts().
*/
- return tconn->net_conf->two_primaries &&
- test_bit(DISCARD_CONCURRENT, &tconn->flags);
+
+ rcu_read_lock();
+ tp = rcu_dereference(mdev->tconn->net_conf)->two_primaries;
+ rcu_read_unlock();
+
+ return tp && test_bit(DISCARD_CONCURRENT, &tconn->flags);
}
static void update_peer_seq(struct drbd_conf *mdev, unsigned int peer_seq)
@@ -2049,7 +2054,7 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
u32 peer_seq = be32_to_cpu(p->seq_num);
int rw = WRITE;
u32 dp_flags;
- int err;
+ int err, tp;
mdev = vnr_to_mdev(tconn, pi->vnr);
if (!mdev)
@@ -2094,7 +2099,11 @@ static int receive_Data(struct drbd_tconn *tconn, struct packet_info *pi)
atomic_inc(&peer_req->epoch->active);
spin_unlock(&mdev->epoch_lock);
- if (mdev->tconn->net_conf->two_primaries) {
+ rcu_read_lock();
+ tp = rcu_dereference(mdev->tconn->net_conf)->two_primaries;
+ rcu_read_unlock();
+ if (tp) {
+ peer_req->flags |= EE_IN_INTERVAL_TREE;
err = wait_for_and_update_peer_seq(mdev, peer_seq);
if (err)
goto out_interrupted;
--
1.7.4.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
end of thread, other threads:[~2011-09-30 12:50 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-09-30 12:48 [RFC 00/11] drbd: part 12 of adding multiple volume support to drbd Philipp Reisner
2011-09-30 12:48 ` [PATCH 01/11] drbd: drbd_delete_device() takes a struct drbd_conf * now Philipp Reisner
2011-09-30 12:48 ` [PATCH 02/11] drbd: Inlined drbd_free_mdev(); it got called only from one place Philipp Reisner
2011-09-30 12:48 ` [PATCH 03/11] drbd: rcu_read_[un]lock() for all idr accesses that do not sleep Philipp Reisner
2011-09-30 12:48 ` [PATCH 04/11] drbd: Converted drbd_cfg_mutex into drbd_cfg_rwsem Philipp Reisner
2011-09-30 12:48 ` [PATCH 05/11] drbd: protect all idr accesses that might sleep with drbd_cfg_rwsem Philipp Reisner
2011-09-30 12:48 ` [PATCH 06/11] drbd: Runtime changeable wire protocol Philipp Reisner
2011-09-30 12:48 ` [PATCH 07/11] drbd: Check consistency of net options when the get changed online Philipp Reisner
2011-09-30 12:48 ` [PATCH 08/11] drbd: Allow online change of replication protocol only with agreed_pv >= 100 Philipp Reisner
2011-09-30 12:48 ` [PATCH 09/11] drbd: rcu_read_lock() and rcu_dereference() for tconn->net_conf Philipp Reisner
2011-09-30 12:48 ` [PATCH 10/11] drbd: Proper locking for updates to net_conf under RCU Philipp Reisner
2011-09-30 12:48 ` [PATCH 11/11] drbd: Considering that the two_primaries config flag can change Philipp Reisner
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.