From mboxrd@z Thu Jan 1 00:00:00 1970 From: Somnath Roy Subject: RE: Severe performance degradation with jewel rbd image Date: Wed, 25 May 2016 23:08:15 +0000 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from mail-bn1on0091.outbound.protection.outlook.com ([157.56.110.91]:45312 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752225AbcEYXWv (ORCPT ); Wed, 25 May 2016 19:22:51 -0400 In-Reply-To: Content-Language: en-US Sender: ceph-devel-owner@vger.kernel.org List-ID: To: "dillaman@redhat.com" Cc: "ceph-devel@vger.kernel.org" SGkgSmFzb24sDQpZZXMsIEkgYW0gcnVubmluZyBvbiBzaW5nbGUgaW1hZ2UgYnV0IHdpdGggbXVs dGlwbGUgZmlvIGpvYnMgKHdpdGggbnVtam9icyA9IDEwIGluIHRoZSBmaW8gcGFyYW1ldGVyKSAu IEkgZXhwZWN0ZWQgdGhhdCB0aGlzIGV4Y2x1c2l2ZSBsb2NrIGlzIGxvY2tpbmcgYmV0d2VlbiB0 aGUgam9icyBhbmQgdGhhdCdzIHdoeSBJIGhhdmUgcG9zdGVkIHRoZSBzaW5nbGUgam9iIHJlc3Vs dC4gU2luZ2xlIGpvYiByZXN1bHQgaXMgKm5vdCBkZWdyYWRpbmcqLg0KQnV0LCB3ZSBuZWVkIG51 bWpvYiBhbmQgUUQgY29tYmluYXRpb24gdG8gZXh0cmFjdCBtb3N0IHBlcmZvcm1hbmNlIGZyb20g YW4gaW1hZ2Ugd2l0aCBmaW8tcmJkLiBBIHNpbmdsZSBsaWJyYmQgaW5zdGFuY2UgcGVyZm9ybWFu Y2Ugc2VlbXMgdG8gYmUgc3R1Y2sgYXQgNDBLIGFuZCB3ZSBjYW4ndCBzY2FsZSB0aGlzIGltYWdl IHVwIHdpdGhvdXQgcnVubmluZyBtdWx0aXBsZSBsaWJyYmQgaW5zdGFuY2VzIG9uIHRoaXMgaW4g cGFyYWxsZWwuDQoxNlggcGVyZm9ybWFuY2UgZGVncmFkYXRpb24gYmVjYXVzZSBvZiB0aGlzIGxv Y2sgc2VlbXMgdmVyeSBkZXN0cnVjdGl2ZS4NCg0KVGhhbmtzICYgUmVnYXJkcw0KU29tbmF0aA0K DQotLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KRnJvbTogSmFzb24gRGlsbGFtYW4gW21haWx0 bzpqZGlsbGFtYUByZWRoYXQuY29tXSANClNlbnQ6IFdlZG5lc2RheSwgTWF5IDI1LCAyMDE2IDM6 NDcgUE0NClRvOiBTb21uYXRoIFJveQ0KQ2M6IGNlcGgtZGV2ZWxAdmdlci5rZXJuZWwub3JnDQpT dWJqZWN0OiBSZTogU2V2ZXJlIHBlcmZvcm1hbmNlIGRlZ3JhZGF0aW9uIHdpdGggamV3ZWwgcmJk IGltYWdlDQoNCkp1c3QgdG8gZWxpbWluYXRlIHRoZSBtb3N0IHN0cmFpZ2h0Zm9yd2FyZCBleHBs YW5hdGlvbiwgYXJlIHlvdSBydW5uaW5nIG11bHRpcGxlIGZpbyBqb2JzIGFnYWluc3QgdGhlIHNh bWUgaW1hZ2UgY29uY3VycmVudGx5PyAgSWYgdGhlIGV4Y2x1c2l2ZSBsb2NrIGhhZCB0byBwaW5n LXBvbmcgYmFjay1hbmQtZm9ydGggYmV0d2VlbiBjbGllbnRzLCB0aGF0IHdvdWxkIGNlcnRhaW5s eSBleHBsYWluIHRoZSBzZXZlcmUgcGVyZm9ybWFuY2UgcGVuYWx0eS4NCg0KT3RoZXJ3aXNlLCB0 aGUgZXhjbHVzaXZlIGxvY2sgaXMgbm90IGluIHRoZSBJTyBwYXRoIG9uY2UgdGhlIGNsaWVudCBo YXMgYWNxdWlyZWQgdGhlIGV4Y2x1c2l2ZSBsb2NrLiAgSWYgeW91IGFyZSBzZWVpbmcgYSBwZXJm b3JtYW5jZSBwZW5hbHR5IGZvciBhIHNpbmdsZS1jbGllbnQgc2NlbmFyaW8gd2l0aCBleGNsdXNp dmUgbG9jayBlbmFibGVkLCB0aGlzIGlzIHNvbWV0aGluZyB3ZSBoYXZlbid0IHNlZW4gYW5kIHdp bGwgaGF2ZSB0byBpbnZlc3RpZ2F0ZSBBU0FQLg0KDQpUaGFua3MsDQoNCk9uIFdlZCwgTWF5IDI1 LCAyMDE2IGF0IDQ6NDggUE0sIFNvbW5hdGggUm95IDxTb21uYXRoLlJveUBzYW5kaXNrLmNvbT4g d3JvdGU6DQo+IEhpIE1hcmsvSm9zaCwNCj4gQXMgSSBtZW50aW9uZWQgaW4gdGhlIHBlcmZvcm1h bmNlIG1lZXRpbmcgdG9kYXkgLCBpZiB3ZSBjcmVhdGUgcmJkIGltYWdlIHdpdGggZGVmYXVsdCAn cmJkIGNyZWF0ZScgY29tbWFuZCBpbiBqZXdlbCAsIHRoZSBpbmRpdmlkdWFsIGltYWdlIHBlcmZv cm1hbmNlIGZvciA0ayBSVyBpcyBub3Qgc2NhbGluZyB1cCB3ZWxsLiBCdXQsIG11bHRpcGxlIG9m IHJiZCBpbWFnZXMgcnVubmluZyBpbiBwYXJhbGxlbCBhZ2dyZWdhdGVkIHRocm91Z2hwdXQgaXMg c2NhbGluZy4NCj4gRm9yIHRoZSBzYW1lIFFEIGFuZCBudW1qb2IgY29tYmluYXRpb24sIGltYWdl IGNyZWF0ZWQgd2l0aCBpbWFnZSBmb3JtYXQgMSAoYW5kIHdpdGggaGFtbWVyIGxpa2UgcmJkX2Rl ZmF1bHRfZmVhdHVyZXMgPSAzKSBpcyBwcm9kdWNpbmcgKjE2WCogbW9yZSBwZXJmb3JtYW5jZS4g SSBkaWQgc29tZSBkaWdnaW5nIGFuZCBoZXJlIGlzIG15IGZpbmRpbmdzLg0KPg0KPiBTZXR1cDoN Cj4gLS0tLS0tLS0NCj4NCj4gMzIgb3NkcyAoYWxsIFNTRCkgb3ZlciA0IG5vZGVzLiBQb29sIHNp emUgPSAyICwgbWluX3NpemUgPSAxLg0KPg0KPiByb290QHN0b3JtZWFwLTE6fiMgY2VwaCAtcw0K PiAgICAgY2x1c3RlciBkYjBmZWJmMS1kMmIwLTRmOGQtOGYyMC00MzczMWMxMzQ3NjMNCj4gICAg ICBoZWFsdGggSEVBTFRIX1dBUk4NCj4gICAgICAgICAgICAgbm9zY3J1Yixub2RlZXAtc2NydWIs c29ydGJpdHdpc2UgZmxhZyhzKSBzZXQNCj4gICAgICBtb25tYXAgZTE6IDEgbW9ucyBhdCB7YT0x MC42MC4xOTQuMTA6Njc4OS8wfQ0KPiAgICAgICAgICAgICBlbGVjdGlvbiBlcG9jaCA1LCBxdW9y dW0gMCBhDQo+ICAgICAgb3NkbWFwIGUxMzk6IDMyIG9zZHM6IDMyIHVwLCAzMiBpbg0KPiAgICAg ICAgICAgICBmbGFncyBub3NjcnViLG5vZGVlcC1zY3J1Yixzb3J0Yml0d2lzZQ0KPiAgICAgICBw Z21hcCB2MjA1MzI6IDI1MDAgcGdzLCAxIHBvb2xzLCA3NDIxIEdCIGRhdGEsIDE4NTUga29iamVj dHMNCj4gICAgICAgICAgICAgMTQ4NTAgR0IgdXNlZCwgMjA4IFRCIC8gMjIzIFRCIGF2YWlsDQo+ ICAgICAgICAgICAgICAgICAyNTAwIGFjdGl2ZStjbGVhbg0KPg0KPiBJTyBwcm9maWxlIDogRmlv IHJiZCB3aXRoIFFEIDEyOCBhbmQgbnVtam9iID0gMTAgcmJkIGNhY2hlIGlzIA0KPiBkaXNhYmxl ZC4NCj4NCj4gUmVzdWx0Og0KPiAtLS0tLS0tLQ0KPiByb290QHN0b3JtZWFwLTE6fiMgcmJkIGlu Zm8gcmVjb3ZlcnlfdGVzdC9yYmRfZGVncmFkYXRpb24gcmJkIGltYWdlIA0KPiAncmJkX2RlZ3Jh ZGF0aW9uJzoNCj4gICAgICAgICBzaXplIDE5NTMgR0IgaW4gNTAwMDAwIG9iamVjdHMNCj4gICAg ICAgICBvcmRlciAyMiAoNDA5NiBrQiBvYmplY3RzKQ0KPiAgICAgICAgIGJsb2NrX25hbWVfcHJl Zml4OiByYi4wLjVmNWYuNmI4YjQ1NjcNCj4gICAgICAgICBmb3JtYXQ6IDENCj4NCj4gT24gdGhl IGFib3ZlIGltYWdlIHdpdGggZm9ybWF0IDEgaXQgaXMgZ2l2aW5nICp+MTAySyBpb3BzKg0KPg0K PiByb290QHN0b3JtZWFwLTE6fiMgcmJkIGluZm8gDQo+IHJlY292ZXJ5X3Rlc3QvcmJkX2RlZ3Jh ZGF0aW9uX3dpdGhfaGFtbWVyX2ZlYXR1cmVzDQo+IHJiZCBpbWFnZSAncmJkX2RlZ3JhZGF0aW9u X3dpdGhfaGFtbWVyX2ZlYXR1cmVzJzoNCj4gICAgICAgICBzaXplIDE5NSBHQiBpbiA1MDAwMCBv YmplY3RzDQo+ICAgICAgICAgb3JkZXIgMjIgKDQwOTYga0Igb2JqZWN0cykNCj4gICAgICAgICBi bG9ja19uYW1lX3ByZWZpeDogcmJkX2RhdGEuNWY4ZDZiOGI0NTY3DQo+ICAgICAgICAgZm9ybWF0 OiAyDQo+ICAgICAgICAgZmVhdHVyZXM6IGxheWVyaW5nDQo+ICAgICAgICAgZmxhZ3M6DQo+DQo+ IE9uIHRoZSBhYm92ZSBpbWFnZSB3aXRoIGhhbW1lciByYmQgZmVhdHVyZXMgb24gLCBpdCBpcyBn aXZpbmcgKn4xMDVLIA0KPiBpb3BzKg0KPg0KPiByb290QHN0b3JtZWFwLTE6fiMgcmJkIGluZm8g cmVjb3ZlcnlfdGVzdC9yYmRfZGVncmFkYXRpb25fd2l0aF83DQo+IHJiZCBpbWFnZSAncmJkX2Rl Z3JhZGF0aW9uX3dpdGhfNyc6DQo+ICAgICAgICAgc2l6ZSAxOTUgR0IgaW4gNTAwMDAgb2JqZWN0 cw0KPiAgICAgICAgIG9yZGVyIDIyICg0MDk2IGtCIG9iamVjdHMpDQo+ICAgICAgICAgYmxvY2tf bmFtZV9wcmVmaXg6IHJiZF9kYXRhLjVmZDg2YjhiNDU2Nw0KPiAgICAgICAgIGZvcm1hdDogMg0K PiAgICAgICAgIGZlYXR1cmVzOiBsYXllcmluZywgZXhjbHVzaXZlLWxvY2sNCj4gICAgICAgICBm bGFnczoNCj4NCj4gT24gdGhlIGFib3ZlIGltYWdlIHdpdGggZmVhdHVyZSA3IChleGNsdXNpdmUg bG9jayBmZWF0dXJlIG9uKSAsIGl0IGlzIA0KPiBnaXZpbmcgKn44SyBpb3BzKi4uLlNvLCA+MTJY IGRlZ3JhZGF0aW9uDQo+DQo+IFRyaWVkIHdpdGggc2luZ2xlIG51bWpvYiBhbmQgUUQgPSAxMjgg LCBwZXJmb3JtYW5jZSBidW1wZWQgdXAgdGlsbCB+NDBLLi5GdXJ0aGVyIGluY3JlYXNpbmcgUUQg LCBwZXJmb3JtYW5jZSBpcyBub3QgZ29pbmcgdXAuDQo+DQo+DQo+IHJvb3RAc3Rvcm1lYXAtMTp+ IyByYmQgaW5mbyByZWNvdmVyeV90ZXN0L3JiZF9kZWdyYWRhdGlvbl93aXRoXzE1DQo+IHJiZCBp bWFnZSAncmJkX2RlZ3JhZGF0aW9uX3dpdGhfMTUnOg0KPiAgICAgICAgIHNpemUgMTk1IEdCIGlu IDUwMDAwIG9iamVjdHMNCj4gICAgICAgICBvcmRlciAyMiAoNDA5NiBrQiBvYmplY3RzKQ0KPiAg ICAgICAgIGJsb2NrX25hbWVfcHJlZml4OiByYmRfZGF0YS41ZmFiNmI4YjQ1NjcNCj4gICAgICAg ICBmb3JtYXQ6IDINCj4gICAgICAgICBmZWF0dXJlczogbGF5ZXJpbmcsIGV4Y2x1c2l2ZS1sb2Nr LCBvYmplY3QtbWFwDQo+ICAgICAgICAgZmxhZ3M6DQo+DQo+IE9uIHRoZSBhYm92ZSBpbWFnZSB3 aXRoIGZlYXR1cmUgMTUgKGV4Y2x1c2l2ZSBsb2NrLCBvYmplY3QgbWFwIGZlYXR1cmUgDQo+IG9u KSAsIGl0IGlzIGdpdmluZyAqfjhLIGlvcHMqLi4uU28sID4xMlggZGVncmFkYXRpb24NCj4NCj4g VHJpZWQgd2l0aCBzaW5nbGUgbnVtam9iIGFuZCBRRCA9IDEyOCAsIHBlcmZvcm1hbmNlIGJ1bXBl ZCB1cCB0aWxsIH40MEsuLkZ1cnRoZXIgaW5jcmVhc2luZyBRRCAsIHBlcmZvcm1hbmNlIGlzIG5v dCBnb2luZyB1cC4NCj4NCj4NCj4gcm9vdEBzdG9ybWVhcC0xOn4jIHJiZCBpbmZvIHJlY292ZXJ5 X3Rlc3QvY2VwaF9yZWNvdmVyeV9pbWdfMSByYmQgDQo+IGltYWdlICdjZXBoX3JlY292ZXJ5X2lt Z18xJzoNCj4gICAgICAgICBzaXplIDQ4ODIgR0IgaW4gMTI1MDAwMCBvYmplY3RzDQo+ICAgICAg ICAgb3JkZXIgMjIgKDQwOTYga0Igb2JqZWN0cykNCj4gICAgICAgICBibG9ja19uYW1lX3ByZWZp eDogcmJkX2RhdGEuMzcxYjZiOGI0NTY3DQo+ICAgICAgICAgZm9ybWF0OiAyDQo+ICAgICAgICAg ZmVhdHVyZXM6IGxheWVyaW5nLCBleGNsdXNpdmUtbG9jaywgb2JqZWN0LW1hcCwgZmFzdC1kaWZm LCBkZWVwLWZsYXR0ZW4NCj4gICAgICAgICBmbGFnczoNCj4NCj4gT24gdGhlIGFib3ZlIGltYWdl IHdpdGggZmVhdHVyZSA2MSAoSmV3ZWwgZGVmYXVsdCkgLCBpdCBpcyBnaXZpbmcgKn42SyANCj4g aW9wcyouLi5TbywgKj4xNlgqIGRlZ3JhZGF0aW9uDQo+DQo+IFRyaWVkIHdpdGggc2luZ2xlIG51 bWpvYiBhbmQgUUQgPSAxMjggLCBwZXJmb3JtYW5jZSBidW1wZWQgdXAgdGlsbCB+MzVLLi5GdXJ0 aGVyIGluY3JlYXNpbmcgUUQgLCBwZXJmb3JtYW5jZSBpcyBub3QgZ29pbmcgdXAuDQo+DQo+IFN1 bW1hcnkgOg0KPiAtLS0tLS0tLS0tLS0NCj4NCj4gMS4gSXQgc2VlbXMgZXhjbHVzaXZlIGxvY2sg ZmVhdHVyZSBpcyBkZWdyYWRpbmcgcGVyZm9ybWFuY2UuDQo+DQo+IDIuIEl0IGlzIGRlZ3JhZGlu ZyBhIGJpdCBmdXJ0aGVyIG9uIGVuYWJsaW5nIGZhc3QtZGlmZiwgZGVlcC1mbGF0dGVuDQo+DQo+ DQo+IExldCBtZSBrbm93IGlmIHlvdSBuZWVkIG1vcmUgaW5mb3JtYXRpb24gb24gdGhpcy4NCj4N Cj4gVGhhbmtzICYgUmVnYXJkcw0KPiBTb21uYXRoDQo+DQo+DQo+IFBMRUFTRSBOT1RFOiBUaGUg aW5mb3JtYXRpb24gY29udGFpbmVkIGluIHRoaXMgZWxlY3Ryb25pYyBtYWlsIG1lc3NhZ2UgaXMg aW50ZW5kZWQgb25seSBmb3IgdGhlIHVzZSBvZiB0aGUgZGVzaWduYXRlZCByZWNpcGllbnQocykg bmFtZWQgYWJvdmUuIElmIHRoZSByZWFkZXIgb2YgdGhpcyBtZXNzYWdlIGlzIG5vdCB0aGUgaW50 ZW5kZWQgcmVjaXBpZW50LCB5b3UgYXJlIGhlcmVieSBub3RpZmllZCB0aGF0IHlvdSBoYXZlIHJl Y2VpdmVkIHRoaXMgbWVzc2FnZSBpbiBlcnJvciBhbmQgdGhhdCBhbnkgcmV2aWV3LCBkaXNzZW1p bmF0aW9uLCBkaXN0cmlidXRpb24sIG9yIGNvcHlpbmcgb2YgdGhpcyBtZXNzYWdlIGlzIHN0cmlj dGx5IHByb2hpYml0ZWQuIElmIHlvdSBoYXZlIHJlY2VpdmVkIHRoaXMgY29tbXVuaWNhdGlvbiBp biBlcnJvciwgcGxlYXNlIG5vdGlmeSB0aGUgc2VuZGVyIGJ5IHRlbGVwaG9uZSBvciBlLW1haWwg KGFzIHNob3duIGFib3ZlKSBpbW1lZGlhdGVseSBhbmQgZGVzdHJveSBhbnkgYW5kIGFsbCBjb3Bp ZXMgb2YgdGhpcyBtZXNzYWdlIGluIHlvdXIgcG9zc2Vzc2lvbiAod2hldGhlciBoYXJkIGNvcGll cyBvciBlbGVjdHJvbmljYWxseSBzdG9yZWQgY29waWVzKS4NCj4gLS0NCj4gVG8gdW5zdWJzY3Jp YmUgZnJvbSB0aGlzIGxpc3Q6IHNlbmQgdGhlIGxpbmUgInVuc3Vic2NyaWJlIGNlcGgtZGV2ZWwi IA0KPiBpbiB0aGUgYm9keSBvZiBhIG1lc3NhZ2UgdG8gbWFqb3Jkb21vQHZnZXIua2VybmVsLm9y ZyBNb3JlIG1ham9yZG9tbyANCj4gaW5mbyBhdCAgaHR0cDovL3ZnZXIua2VybmVsLm9yZy9tYWpv cmRvbW8taW5mby5odG1sDQoNCg0KDQotLQ0KSmFzb24NCg==