On 19.05.21 19:10, Julien Grall wrote:
> Hi Juergen,
> 
> On 19/05/2021 13:50, Juergen Gross wrote:
>> On 19.05.21 14:33, Julien Grall wrote:
>>>
>>>
>>> On 19/05/2021 13:32, Julien Grall wrote:
>>>> Hi Juergen,
>>>>
>>>> On 19/05/2021 10:09, Juergen Gross wrote:
>>>>> On 18.05.21 20:11, Julien Grall wrote:
>>>>>>
>>>>>> I have started to look at preserving transaction accross Live-update 
>> in
>>>>>
>>>>>> C Xenstored. So far, I managed to transfer transaction that 
>>>>>> read/write existing nodes.
>>>>>>
>>>>>> Now, I am running into trouble to transfer new/deleted node within 
a 
>>
>>>>>> transaction with the existing migration format.
>>>>>>
>>>>>> C Xenstored will keep track of nodes accessed during the transaction 
>>
>>>>>> but not the children (AFAICT for performance reason).
>>>>>
>>>>> Not performance reasons, but because there isn't any need for that:
>>>>>
>>>>> The children are either unchanged (so the non-transaction node records
>>>>> apply), or they will be among the tracked nodes (transaction node
>>>>> records apply). So in both cases all children should be known.
>>>> In theory, opening a new transaction means you will not see any 
>>>> modification in the global database until the transaction has been 
>>>> committed. What you describe would break that because a client would 

>>>> be able to see new nodes added outside of the transaction.
>>>>
>>>> However, C Xenstored implements neither of the two. Currently, when 
>>>> a node is accessed within the transaction, we will also store the 
>>>> names of the current children.
>>>>
>>>> To give an example with access to the global DB (prefixed with TID0) 

>>>> and within a transaction (TID1)
>>>>
>>>>      1) TID0: MKDIR "data/bar"
>>>>          2) Start transaction TID1
>>>>      3) TID1: DIRECTORY "data"
>>>>          -> This will cache thenode data
>>>>      4) TID0: MKDIR "data/foo"
>>>>          -> This will create 
"foo" in the global database
>>>>      5) TID1: MKDIR "data/fish"
>>>>          -> This will create 
"fish" inthe transaction
>>>>      5) TID1: DIRECTORY "data"
>>>>          -> This will only return "bar" and "fish"
>>>>
>>>> If we Live-Update between 4) and 5). Then we should make sure that 
>>>> "bar" cannot be seen in the listing by TID1.
>>>
>>> I meant "foo" here. Sorry for the confusion.
>>>
>>>>
>>>> Therefore, I don't think we can restore the children using the 
>>>> global node here. Instead we need to find a way to transfer the list 

>>>> of known 
>>
>>>> children within the transaction.
>>>>
>>>> As a fun fact, C Xenstored implements weirdly the transaction, so TID1 
>>
>>>> will be able to access "bar" if it knows the name but not list it.
>>
>> And this is the basic problem, I think.
>>
>> C Xenstored should be repaired by adding all (remaining) children of a
>> node into the TID's database when the list of children is modified
>> either globally or in a transaction. A child having been added globally
>> needs to be added as "deleted" into the TID's database.
> 
> IIUC, for every modifications in the global database, we would need to 
> walk every single transactions and check whether a parent was accessed. 

> Am I correct?

Not really. When a node is being read during a transaction and it is
found in the global data base only, its gen-count can be tested for
being older or newer than the transaction start. If it is newer we can
traverse the path up to "/" and treat each parent the same way (so if
a parent is found in the transaction data base, the presence of the
child can be verified, and if it is global only, the gen-count can be
tested against the transaction again).

> If so, I don't think this is a workable solution because of the cost to 

> execute a single command.

My variant will affect the transaction internal reads only, and the
additional cost will be limited by the distance of the read node from
the root node.

> Is it something you plan to address differently with your rework of the 
DB?

Yes. I want to have the transaction specific variants of nodes linked to
the global ones, which solves this problem in an easy way.


Juergen