Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

[Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>]

[-- Attachment #1: Type: text/plain, Size: 3831 bytes --]

20.09.2019 16:10, Max Reitz wrote:
> On 20.09.19 14:53, Vladimir Sementsov-Ogievskiy wrote:
>> 20.09.2019 15:40, Max Reitz wrote:
>>> On 20.09.19 13:53, Vladimir Sementsov-Ogievskiy wrote:
>>>> 20.09.2019 14:10, Max Reitz wrote:
>>>>> On 16.09.19 19:53, Vladimir Sementsov-Ogievskiy wrote:
>>>>>> Hi all!
>>>>>>
>>>>>> Here is an asynchronous scheme for handling fragmented qcow2
>>>>>> reads and writes. Both qcow2 read and write functions loops through
>>>>>> sequential portions of data. The series aim it to parallelize these
>>>>>> loops iterations.
>>>>>> It improves performance for fragmented qcow2 images, I've tested it
>>>>>> as described below.
>>>>>
>>>>> Thanks again, applied to my block branch:
>>>>>
>>>>> https://git.xanclic.moe/XanClic/qemu/commits/branch/block
>>>>
>>>> Thanks a lot!
>>>>
>>>>>
>>>>>> v5: fix 026 and rebase on Max's block branch [perf results not updated]:
>>>>>>
>>>>>> 01: new, prepare 026 to not fail
>>>>>> 03: - drop read_encrypted blkdbg event [Kevin]
>>>>>>        - assert((x & (BDRV_SECTOR_SIZE - 1)) == 0) -> assert(QEMU_IS_ALIGNED(x, BDRV_SECTOR_SIZE)) [rebase]
>>>>>>        - full host offset in argument of qcow2_co_decrypt [rebase]
>>>>>> 04: - substitute remaining qcow2_co_do_pwritev by qcow2_co_pwritev_task in comment [Max]
>>>>>>        - full host offset in argument of qcow2_co_encrypt [rebase]
>>>>>> 05: - Now patch don't affect 026 iotest, so its output is not changed
>>>>>>
>>>>>> Rebase changes seems trivial, so, I've kept r-b marks.
>>>>>
>>>>> (For the record, I didn’t consider them trivial, or I’d’ve applied
>>>>> Maxim’s series on top of yours.  I consider a conflict to be trivially
>>>>> resolvable only if there is only one way of doing it; but when I
>>>>> resolved the conflicts myself, I resolved the one in patch 3 differently
>>>>> from you – I added an offset_in_cluster variable to
>>>>> qcow2_co_preadv_encrypted().  Sure, it’s still simple and the difference
>>>>> is minor, but that was exactly where I thought that I can’t consider
>>>>> this trivial.)
>>>>>
>>>>
>>>> Hmm. May be it's trivial enough to keep r-b (as my change is trivial itself), but not
>>>> trivial enough to change alien patch on queuing? If you disagree, I'll be more
>>>> careful on keeping r-b in changed patches, sorry.
>>>
>>> It doesn’t matter much to me, I diff all patches anyway. :-)
>>>
>>
>> then a bit offtopic:
>>
>> Which tools are you use?
>>
>> I've some scripts to compare different versions of one serie (or to check, what
>> was changed in patches during some porting process..).. The core thing is to filter
>> some not interesting numbers and hashes, which makes diffs dirty, and then call vimdiff.
>> But maybe I've reinvented the wheel.
> 
> Just kompare as a graphical diff tool; I just scroll past the hash diffs.
> 
> But now that you gave me the idea, maybe I should write a script to
> filter them...  (So, no, I don’t know of a tool that would do that
> already :-/)
> 


Then you may find my scripts somehow useful, at least as a hint (I'm afraid code is not beautiful at all)

---

Usage:

eat-diff-numbers is just sed script, used by both following scripts, to eat git numbers and hashes.

---

check-rebase - searches patches by name in original branch and compares

git check-rebase <original branch> <commits to check>

for example (check several backported commits):

git check-rebase master HEAD^^^^..HEAD

-----------

check-rebase2 - don't search by name, but just compare two sequences of patches of same length.

git check-rebase2 <original top commit>  <new top commit>  <how many commits>

for example (check one series version vs another)

git check-rebase2 salvage-v1 salvage-v2 7


-- 
Best regards,
Vladimir

[-- Attachment #2: eat-diff-numbers --]
[-- Type: text/plain, Size: 170 bytes --]

#!/usr/bin/sh

sed -e 's/^index .*/index <some index>/' -e 's/^@@ .* @@/@@ <some lines> @@/' -e 's/^commit .*/commit <some commit>/' -e 's/^Date:.*00/Date: <some date>/'

[-- Attachment #3: git-check-rebase2 --]
[-- Type: text/plain, Size: 921 bytes --]

#!/usr/bin/bash

function dshow {
    git show $1 | eat-diff-numbers
}

function comp {
    ap="/tmp/[a $3] $(git log --format=%f -n 1 $1).patch"
    bp="/tmp/[b $3] $(git log --format=%f -n 1 $2).patch"
    dshow $1 > "$ap"
    dshow $2 > "$bp"
    if ! diff "$ap" "$bp" > /dev/null ; then
        co=$(git log --format=%s -n 1 $2)
        echo -n $co - differs | tee -a loglog
        echo "$ap - $1"
        echo "$bp - $2"
        echo "type exit to continue"
        #git diff --no-index --color-words --word-diff-regex=. /tmp/{a,b}.patch
        if vimdiff  < /dev/tty -c 0 "$ap" "$bp" ; then
            echo " - ok" | tee -a loglog
        else
            echo " - bad" | tee -a loglog
        fi
        return 1
    fi
}

a=$1
b=$2
n=$3

for ((i=1; i<=n; i++)); do
    rr=$((n - i))
    co=$(git log --format=%s -n 1 $b~$rr)
    if comp $a~$rr $b~$rr $i; then
        echo $co - ok | tee -a loglog
    fi
done

[-- Attachment #4: git-check-rebase --]
[-- Type: text/plain, Size: 1002 bytes --]

#!/usr/bin/bash

function dshow {
    git show $1 | eat-diff-numbers
}

function comp {
    local b="/tmp/[b-$3]-$(git log --format=%f -n 1 $2).patch"
    dshow $2 > $b
    dshow $(git-find-subj $1 $2 || echo "NOT FOUND") > /tmp/a.patch
    if ! diff /tmp/a.patch $b > /dev/null ; then
        co=$(git log --format=%s -n 1 $c)
        echo -n $co - differs | tee -a loglog
        echo "/tmp/a.patch - found in $1: "
        echo "$b - $c from current branch"
        echo "type exit to continue"
        #git diff --no-index --color-words --word-diff-regex=. /tmp/{a,b}.patch
        if vimdiff  < /dev/tty -c 0 /tmp/a.patch $b ; then
            echo " - ok" | tee -a loglog
        else
            echo " - bad" | tee -a loglog
        fi
        return 1
    fi
}

rm loglog logcur
a=0
git log $2 --format="%h" --reverse | while read c; do
    ((a++))
    co=$(git log --format=%s -n 1 $c)
    git log -n 1 $c > logcur
    if comp $1 $c $a; then
        echo $co - ok | tee -a loglog
    fi
done