Question about OSDSuperblock

Question about OSDSuperblock

[xxhdx1985126]

Hi, everyone.


I'm trying to read the source code that boots an OSD instance, and I find something really overwhelms me.
In the OSD::init() method, it read the OSDSuperblock by calling OSD::read_superblock(), and the it tried to get the "current" map : "osdmap = get_map(superblock.current_epoch)". Then OSD uses this osdmap to calculate the acting and up set of each pg. 
I really don't understand this! Since the OSDSuperblock is read from the disk, the content of the superblock.current_epoch must be an old epoch which is recorded by the last OSD instance that run on the ceph osd's directory. Why use an old "current_epoch" to calculate the acting and up set of each pg?


Please help me, thank you:-)

Re: Question about OSDSuperblock

[Sage Weil]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1444 bytes --]

On Mon, 24 Oct 2016, xxhdx1985126 wrote:
> Hi, everyone.
> 
> 
> I'm trying to read the source code that boots an OSD instance, and I find something really overwhelms me.
> In the OSD::init() method, it read the OSDSuperblock by calling OSD::read_superblock(), and the it tried to get the "current" map : "osdmap = get_map(superblock.current_epoch)". Then OSD uses this osdmap to calculate the acting and up set of each pg. 
> I really don't understand this! Since the OSDSuperblock is read from the disk, the content of the superblock.current_epoch must be an old epoch which is recorded by the last OSD instance that run on the ceph osd's directory. Why use an old "current_epoch" to calculate the acting and up set of each pg?

Yes.  load_pgs() instantiates all of the PGs, populates the current (== 
current_epoch) up and acting vectors, and triggers a loaded event to help 
get everything in-memory initialized.

Then, during the 'booting' sequence, we request the latest map from the 
monitor, and respond to any changes in the OSDMap--everything that 
happened while the OSD was down, and eventually the OSDmap that indicates 
the OSD is now up.  At that point it can start doing useful work. Look for 
the AdvMap and ActMap state events.

This process of responding to changes to the OSDMap and synchronizing with 
other OSDs in collectively called "peering" and is one of the more 
difficult areas of the code to understand.  :/

sage

Re:Re: Question about OSDSuperblock

[Sage Weil]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2307 bytes --]

On Mon, 24 Oct 2016, xxhdx1985126 wrote:
> Thanks, sir:-) I got it. I'll try to understand the "peering" part of
> code.:-)
> 
> By the way, please allow me to ask just one more question:-)
> Under what circumstance will the OSDMap mark a newly added osd "UP"? Or, in
> other words, how does the OSDMap judge whether or not an booting osd has
> finished the 'booting' sequence and should be mark "UP"?
> 
> Thank you very much, sir:-)

Once the starting OSD has fetched all of the OSDMaps from the monitor (to 
within a few epochs at least), it will send an MOSDBoot message asking the 
monitor to mark it UP.

See OSD::start_boot() to follow the the full sequence!

sage


 > 
> 
> 
> 
> At 2016-10-24 18:06:45, "Sage Weil" <sage@newdream.net> wrote:
> >On Mon, 24 Oct 2016, xxhdx1985126 wrote:
> >> Hi, everyone.
> >> 
> >> 
> >> I'm trying to read the source code that boots an OSD instance, and I find
>  something really overwhelms me.
> >> In the OSD::init() method, it read the OSDSuperblock by calling OSD::read
> _superblock(), and the it tried to get the "current" map : "osdmap = get_map
> (superblock.current_epoch)". Then OSD uses this osdmap to calculate the acti
> ng and up set of each pg. 
> >> I really don't understand this! Since the OSDSuperblock is read from the 
> disk, the content of the superblock.current_epoch must be an old epoch which
>  is recorded by the last OSD instance that run on the ceph osd's directory. 
> Why use an old "current_epoch" to calculate the acting and up set of each pg
> ?
> >
> >Yes.  load_pgs() instantiates all of the PGs, populates the current (== 
> >current_epoch) up and acting vectors, and triggers a loaded event to help 
> >get everything in-memory initialized.
> >
> >Then, during the 'booting' sequence, we request the latest map from the 
> >monitor, and respond to any changes in the OSDMap--everything that 
> >happened while the OSD was down, and eventually the OSDmap that indicates 
> >the OSD is now up.  At that point it can start doing useful work. Look for 
> >the AdvMap and ActMap state events.
> >
> >This process of responding to changes to the OSDMap and synchronizing with 
> >other OSDs in collectively called "peering" and is one of the more 
> >difficult areas of the code to understand.  :/
> >
> >sage
> 
> 
>  
> 
> 
> 

Re:Re:Re: Question about OSDSuperblock

[xxhdx1985126]

Thanks, sir:-)








At 2016-10-24 18:42:53, "Sage Weil" <sage@newdream.net> wrote:
>On Mon, 24 Oct 2016, xxhdx1985126 wrote:
>> Thanks, sir:-) I got it. I'll try to understand the "peering" part of
>> code.:-)
>> 
>> By the way, please allow me to ask just one more question:-)
>> Under what circumstance will the OSDMap mark a newly added osd "UP"? Or, in
>> other words, how does the OSDMap judge whether or not an booting osd has
>> finished the 'booting' sequence and should be mark "UP"?
>> 
>> Thank you very much, sir:-)
>
>Once the starting OSD has fetched all of the OSDMaps from the monitor (to 
>within a few epochs at least), it will send an MOSDBoot message asking the 
>monitor to mark it UP.
>
>See OSD::start_boot() to follow the the full sequence!
>
>sage
>
>
> > 
>> 
>> 
>> 
>> At 2016-10-24 18:06:45, "Sage Weil" <sage@newdream.net> wrote:
>> >On Mon, 24 Oct 2016, xxhdx1985126 wrote:
>> >> Hi, everyone.
>> >> 
>> >> 
>> >> I'm trying to read the source code that boots an OSD instance, and I find
>>  something really overwhelms me.
>> >> In the OSD::init() method, it read the OSDSuperblock by calling OSD::read
>> _superblock(), and the it tried to get the "current" map : "osdmap = get_map
>> (superblock.current_epoch)". Then OSD uses this osdmap to calculate the acti
>> ng and up set of each pg. 
>> >> I really don't understand this! Since the OSDSuperblock is read from the 
>> disk, the content of the superblock.current_epoch must be an old epoch which
>>  is recorded by the last OSD instance that run on the ceph osd's directory. 
>> Why use an old "current_epoch" to calculate the acting and up set of each pg
>> ?
>> >
>> >Yes.  load_pgs() instantiates all of the PGs, populates the current (== 
>> >current_epoch) up and acting vectors, and triggers a loaded event to help 
>> >get everything in-memory initialized.
>> >
>> >Then, during the 'booting' sequence, we request the latest map from the 
>> >monitor, and respond to any changes in the OSDMap--everything that 
>> >happened while the OSD was down, and eventually the OSDmap that indicates 
>> >the OSD is now up.  At that point it can start doing useful work. Look for 
>> >the AdvMap and ActMap state events.
>> >
>> >This process of responding to changes to the OSDMap and synchronizing with 
>> >other OSDs in collectively called "peering" and is one of the more 
>> >difficult areas of the code to understand.  :/
>> >
>> >sage
>> 
>> 
>>  
>> 
>> 
>>