Towards second edition

Towards second edition

[Paul E. McKenney]

Hello!

I finally found and fixed the rcu_barrier() bug [1], so I should again
be able to devote some big-system test time to redoing performance
results in perfbook.  Once that is done, I expect that it is time for
the second edition.

I might also convert the blank page hiding the solution to the Dining
Philosophers Problem to a quick quiz, but I consider this optional.

Are there any other changes that are needed? [2]

							Thanx, Paul


[1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
    offline no-CBs CPUs") in -rcu, in case anyone is curious.

[2] Here is a list of some things that I believe can follow the second
    edition:

	Expand lock-free algorithm discussion to include LIFO push,
	illustrating the infamous pointer-zap issue.  (See ISO SC22
	WG21 P1726R3, which should appear in a couple of weeks, for
	more details.)

	Add text describing the Issaquah Challenge.

	Add text describing skiplists, one of the more concurrency
	friendly data structures.

	Add text describing data-race detectors such as KCSAN.  (This needs
	to wait for more Linux-kernel experience.)

	Additional material from todo.txt.  ;-)

Re: Towards second edition

[Дмитрий Дьяченко]

[-- Attachment #1: Type: text/plain, Size: 2931 bytes --]

Hello!

I build perfbook sometimes and try build today after read 'Toward second
edition' letter.
I use up-to-day livetex and up-to-day perfbook.

07-feb-2020 build PASS.
Today build FAIL [1].

Thank you,
Dmitry

[1] make
[...]
echo > origpub.tex
latexpand --empty-comments perfbook.tex 1> perfbook_flat.tex 2> /dev/null
sh utilities/extractqqz.sh < perfbook_flat.tex | perl utilities/
qqzreorder.pl > qqz.tex
cat perfbook_flat.tex qqz.tex | sh utilities/extractcontrib.sh > contrib.tex
sh utilities/extractorigpub.sh < perfbook_flat.tex > origpub.tex
sh utilities/runfirstlatex.sh perfbook
pdflatex 1 for perfbook.pdf

LaTeX Warning: Reference `ln:toolsoftrade:Inviting a Store-to-Load
Conversion:l
oad:p' on page 42 undefined on input line 1988.


! Package examplep Error: \PVerb inner delimiter must be brace,
(examplep)                not backslash or control sequence or active char.

See the examplep package documentation for explanation.
Type  H <return>  for immediate help.
 ...

l.1988 ...load:p} fetches \co{p}, but the \qco{if}
                                                   statement on
?
! Emergency stop.
 ...

l.1988 ...load:p} fetches \co{p}, but the \qco{if}
                                                   statement on
Try typing  <return>  to proceed.
----- Fatal latex error, see perfbook.log for details. -----
make: *** [Makefile:183: perfbook.aux] Ошибка 1
Command exited with non-zero status 2

вс, 23 февр. 2020 г. в 18:33, Paul E. McKenney <paulmck@kernel.org>:

> Hello!
>
> I finally found and fixed the rcu_barrier() bug [1], so I should again
> be able to devote some big-system test time to redoing performance
> results in perfbook.  Once that is done, I expect that it is time for
> the second edition.
>
> I might also convert the blank page hiding the solution to the Dining
> Philosophers Problem to a quick quiz, but I consider this optional.
>
> Are there any other changes that are needed? [2]
>
>                                                         Thanx, Paul
>
>
> [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
>     offline no-CBs CPUs") in -rcu, in case anyone is curious.
>
> [2] Here is a list of some things that I believe can follow the second
>     edition:
>
>         Expand lock-free algorithm discussion to include LIFO push,
>         illustrating the infamous pointer-zap issue.  (See ISO SC22
>         WG21 P1726R3, which should appear in a couple of weeks, for
>         more details.)
>
>         Add text describing the Issaquah Challenge.
>
>         Add text describing skiplists, one of the more concurrency
>         friendly data structures.
>
>         Add text describing data-race detectors such as KCSAN.  (This needs
>         to wait for more Linux-kernel experience.)
>
>         Additional material from todo.txt.  ;-)
>

[-- Attachment #2: Type: text/html, Size: 4076 bytes --]

Re: Towards second edition

[Paul E. McKenney]

On Sun, Feb 23, 2020 at 07:49:32PM +0300, Дмитрий Дьяченко wrote:
> Hello!
> 
> I build perfbook sometimes and try build today after read 'Toward second
> edition' letter.
> I use up-to-day livetex and up-to-day perfbook.
> 
> 07-feb-2020 build PASS.
> Today build FAIL [1].

Hello, Dmitry,

There was a change in the build system to fix a rendering problem.
Could you please check recent changes to FAQ-BUILD.txt and see if they
apply to you?  It might be necessary for you download epigraph.zip,
depending on what distro version you happen to be using.  (I had to
download it, for example.)

Either way, does "make clean" followed by "make" help?

							Thanx, Paul

> Thank you,
> Dmitry
> 
> [1] make
> [...]
> echo > origpub.tex
> latexpand --empty-comments perfbook.tex 1> perfbook_flat.tex 2> /dev/null
> sh utilities/extractqqz.sh < perfbook_flat.tex | perl utilities/
> qqzreorder.pl > qqz.tex
> cat perfbook_flat.tex qqz.tex | sh utilities/extractcontrib.sh > contrib.tex
> sh utilities/extractorigpub.sh < perfbook_flat.tex > origpub.tex
> sh utilities/runfirstlatex.sh perfbook
> pdflatex 1 for perfbook.pdf
> 
> LaTeX Warning: Reference `ln:toolsoftrade:Inviting a Store-to-Load
> Conversion:l
> oad:p' on page 42 undefined on input line 1988.
> 
> 
> ! Package examplep Error: \PVerb inner delimiter must be brace,
> (examplep)                not backslash or control sequence or active char.
> 
> See the examplep package documentation for explanation.
> Type  H <return>  for immediate help.
>  ...
> 
> l.1988 ...load:p} fetches \co{p}, but the \qco{if}
>                                                    statement on
> ?
> ! Emergency stop.
>  ...
> 
> l.1988 ...load:p} fetches \co{p}, but the \qco{if}
>                                                    statement on
> Try typing  <return>  to proceed.
> ----- Fatal latex error, see perfbook.log for details. -----
> make: *** [Makefile:183: perfbook.aux] Ошибка 1
> Command exited with non-zero status 2
> 
> вс, 23 февр. 2020 г. в 18:33, Paul E. McKenney <paulmck@kernel.org>:
> 
> > Hello!
> >
> > I finally found and fixed the rcu_barrier() bug [1], so I should again
> > be able to devote some big-system test time to redoing performance
> > results in perfbook.  Once that is done, I expect that it is time for
> > the second edition.
> >
> > I might also convert the blank page hiding the solution to the Dining
> > Philosophers Problem to a quick quiz, but I consider this optional.
> >
> > Are there any other changes that are needed? [2]
> >
> >                                                         Thanx, Paul
> >
> >
> > [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
> >     offline no-CBs CPUs") in -rcu, in case anyone is curious.
> >
> > [2] Here is a list of some things that I believe can follow the second
> >     edition:
> >
> >         Expand lock-free algorithm discussion to include LIFO push,
> >         illustrating the infamous pointer-zap issue.  (See ISO SC22
> >         WG21 P1726R3, which should appear in a couple of weeks, for
> >         more details.)
> >
> >         Add text describing the Issaquah Challenge.
> >
> >         Add text describing skiplists, one of the more concurrency
> >         friendly data structures.
> >
> >         Add text describing data-race detectors such as KCSAN.  (This needs
> >         to wait for more Linux-kernel experience.)
> >
> >         Additional material from todo.txt.  ;-)
> >

Re: Towards second edition

[Дмитрий Дьяченко]

Hello, Paul,

I check FAQ-BUILD.txt and find Q. 10.A
" ... and TeX Live releases prior to 2020 (epigraph)".

I update livetex today and probably I have the latest epigraph' version
./texlive/texmf-dist/tex/latex/epigraph:
total 8
-rw-r--r--. 1 dima dima 4602 Jan  3 01:06 epigraph.sty

To avoid make clean/make problems I
-- rm build-perfbook
-- update perfbook
-- copy perfbook into build-perfbook
-- run make in clean dir.

I use Fedora 31 up-to-day, x86_64.

I 'll try tomorrow to find the first commit after 07 feb with errors.

Thank you,
Dmitry

вс, 23 февр. 2020 г. в 21:47, Paul E. McKenney <paulmck@kernel.org>:
>
> On Sun, Feb 23, 2020 at 07:49:32PM +0300, Дмитрий Дьяченко wrote:
> > Hello!
> >
> > I build perfbook sometimes and try build today after read 'Toward second
> > edition' letter.
> > I use up-to-day livetex and up-to-day perfbook.
> >
> > 07-feb-2020 build PASS.
> > Today build FAIL [1].
>
> Hello, Dmitry,
>
> There was a change in the build system to fix a rendering problem.
> Could you please check recent changes to FAQ-BUILD.txt and see if they
> apply to you?  It might be necessary for you download epigraph.zip,
> depending on what distro version you happen to be using.  (I had to
> download it, for example.)
>
> Either way, does "make clean" followed by "make" help?
>
>                                                         Thanx, Paul
>
> > Thank you,
> > Dmitry
> >
> > [1] make
> > [...]
> > echo > origpub.tex
> > latexpand --empty-comments perfbook.tex 1> perfbook_flat.tex 2> /dev/null
> > sh utilities/extractqqz.sh < perfbook_flat.tex | perl utilities/
> > qqzreorder.pl > qqz.tex
> > cat perfbook_flat.tex qqz.tex | sh utilities/extractcontrib.sh > contrib.tex
> > sh utilities/extractorigpub.sh < perfbook_flat.tex > origpub.tex
> > sh utilities/runfirstlatex.sh perfbook
> > pdflatex 1 for perfbook.pdf
> >
> > LaTeX Warning: Reference `ln:toolsoftrade:Inviting a Store-to-Load
> > Conversion:l
> > oad:p' on page 42 undefined on input line 1988.
> >
> >
> > ! Package examplep Error: \PVerb inner delimiter must be brace,
> > (examplep)                not backslash or control sequence or active char.
> >
> > See the examplep package documentation for explanation.
> > Type  H <return>  for immediate help.
> >  ...
> >
> > l.1988 ...load:p} fetches \co{p}, but the \qco{if}
> >                                                    statement on
> > ?
> > ! Emergency stop.
> >  ...
> >
> > l.1988 ...load:p} fetches \co{p}, but the \qco{if}
> >                                                    statement on
> > Try typing  <return>  to proceed.
> > ----- Fatal latex error, see perfbook.log for details. -----
> > make: *** [Makefile:183: perfbook.aux] Ошибка 1
> > Command exited with non-zero status 2
> >
> > вс, 23 февр. 2020 г. в 18:33, Paul E. McKenney <paulmck@kernel.org>:
> >
> > > Hello!
> > >
> > > I finally found and fixed the rcu_barrier() bug [1], so I should again
> > > be able to devote some big-system test time to redoing performance
> > > results in perfbook.  Once that is done, I expect that it is time for
> > > the second edition.
> > >
> > > I might also convert the blank page hiding the solution to the Dining
> > > Philosophers Problem to a quick quiz, but I consider this optional.
> > >
> > > Are there any other changes that are needed? [2]
> > >
> > >                                                         Thanx, Paul
> > >
> > >
> > > [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
> > >     offline no-CBs CPUs") in -rcu, in case anyone is curious.
> > >
> > > [2] Here is a list of some things that I believe can follow the second
> > >     edition:
> > >
> > >         Expand lock-free algorithm discussion to include LIFO push,
> > >         illustrating the infamous pointer-zap issue.  (See ISO SC22
> > >         WG21 P1726R3, which should appear in a couple of weeks, for
> > >         more details.)
> > >
> > >         Add text describing the Issaquah Challenge.
> > >
> > >         Add text describing skiplists, one of the more concurrency
> > >         friendly data structures.
> > >
> > >         Add text describing data-race detectors such as KCSAN.  (This needs
> > >         to wait for more Linux-kernel experience.)
> > >
> > >         Additional material from todo.txt.  ;-)
> > >

Re: Towards second edition

[Akira Yokosawa]

Hi Dmitry,

On Sun, 23 Feb 2020 23:52:25 +0300, Дмитрий Дьяченко wrote:
> Hello, Paul,
> 
> I check FAQ-BUILD.txt and find Q. 10.A
> " ... and TeX Live releases prior to 2020 (epigraph)".
> 
> I update livetex today and probably I have the latest epigraph' version
> ./texlive/texmf-dist/tex/latex/epigraph:
> total 8
> -rw-r--r--. 1 dima dima 4602 Jan  3 01:06 epigraph.sty

So, you are using upstream TeX Live 2019, not the one distributed by
Fedora.

Due to the fairly massive updates in LaTeX base package to be released
in TeX Live 2020, current upstream TeX Live lost compatibility with
the "examplep" package.

I contacted the author of the package about this issue, but have not
heard from him yet.

As a matter of fact, I have a patch set to resolve the issue in case
the breakage persists. My thought was to submit it if Ubuntu/Fedora
picks up the broken state of TeX Live in the next release.

Now that you have encountered the issue, I'm submitting it for you
to test.

        Thanks, Akira

> 
> To avoid make clean/make problems I
> -- rm build-perfbook
> -- update perfbook
> -- copy perfbook into build-perfbook
> -- run make in clean dir.
> 
> I use Fedora 31 up-to-day, x86_64.
> 
> I 'll try tomorrow to find the first commit after 07 feb with errors.
> 
> Thank you,
> Dmitry
> 
> вс, 23 февр. 2020 г. в 21:47, Paul E. McKenney <paulmck@kernel.org>:
>>
>> On Sun, Feb 23, 2020 at 07:49:32PM +0300, Дмитрий Дьяченко wrote:
>>> Hello!
>>>
>>> I build perfbook sometimes and try build today after read 'Toward second
>>> edition' letter.
>>> I use up-to-day livetex and up-to-day perfbook.
>>>
>>> 07-feb-2020 build PASS.
>>> Today build FAIL [1].
>>
>> Hello, Dmitry,
>>
>> There was a change in the build system to fix a rendering problem.
>> Could you please check recent changes to FAQ-BUILD.txt and see if they
>> apply to you?  It might be necessary for you download epigraph.zip,
>> depending on what distro version you happen to be using.  (I had to
>> download it, for example.)
>>
>> Either way, does "make clean" followed by "make" help?
>>
>>                                                         Thanx, Paul
>>
>>> Thank you,
>>> Dmitry
>>>
>>> [1] make
>>> [...]
>>> echo > origpub.tex
>>> latexpand --empty-comments perfbook.tex 1> perfbook_flat.tex 2> /dev/null
>>> sh utilities/extractqqz.sh < perfbook_flat.tex | perl utilities/
>>> qqzreorder.pl > qqz.tex
>>> cat perfbook_flat.tex qqz.tex | sh utilities/extractcontrib.sh > contrib.tex
>>> sh utilities/extractorigpub.sh < perfbook_flat.tex > origpub.tex
>>> sh utilities/runfirstlatex.sh perfbook
>>> pdflatex 1 for perfbook.pdf
>>>
>>> LaTeX Warning: Reference `ln:toolsoftrade:Inviting a Store-to-Load
>>> Conversion:l
>>> oad:p' on page 42 undefined on input line 1988.
>>>
>>>
>>> ! Package examplep Error: \PVerb inner delimiter must be brace,
>>> (examplep)                not backslash or control sequence or active char.
>>>
>>> See the examplep package documentation for explanation.
>>> Type  H <return>  for immediate help.
>>>  ...
>>>
>>> l.1988 ...load:p} fetches \co{p}, but the \qco{if}
>>>                                                    statement on
>>> ?
>>> ! Emergency stop.
>>>  ...
>>>
>>> l.1988 ...load:p} fetches \co{p}, but the \qco{if}
>>>                                                    statement on
>>> Try typing  <return>  to proceed.
>>> ----- Fatal latex error, see perfbook.log for details. -----
>>> make: *** [Makefile:183: perfbook.aux] Ошибка 1
>>> Command exited with non-zero status 2
>>>
>>> вс, 23 февр. 2020 г. в 18:33, Paul E. McKenney <paulmck@kernel.org>:
>>>
>>>> Hello!
>>>>
>>>> I finally found and fixed the rcu_barrier() bug [1], so I should again
>>>> be able to devote some big-system test time to redoing performance
>>>> results in perfbook.  Once that is done, I expect that it is time for
>>>> the second edition.
>>>>
>>>> I might also convert the blank page hiding the solution to the Dining
>>>> Philosophers Problem to a quick quiz, but I consider this optional.
>>>>
>>>> Are there any other changes that are needed? [2]
>>>>
>>>>                                                         Thanx, Paul
>>>>
>>>>
>>>> [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
>>>>     offline no-CBs CPUs") in -rcu, in case anyone is curious.
>>>>
>>>> [2] Here is a list of some things that I believe can follow the second
>>>>     edition:
>>>>
>>>>         Expand lock-free algorithm discussion to include LIFO push,
>>>>         illustrating the infamous pointer-zap issue.  (See ISO SC22
>>>>         WG21 P1726R3, which should appear in a couple of weeks, for
>>>>         more details.)
>>>>
>>>>         Add text describing the Issaquah Challenge.
>>>>
>>>>         Add text describing skiplists, one of the more concurrency
>>>>         friendly data structures.
>>>>
>>>>         Add text describing data-race detectors such as KCSAN.  (This needs
>>>>         to wait for more Linux-kernel experience.)
>>>>
>>>>         Additional material from todo.txt.  ;-)
>>>>

Re: Towards second edition

[Akira Yokosawa]

On 2020/02/24 0:33, Paul E. McKenney wrote:
> Hello!
> 
> I finally found and fixed the rcu_barrier() bug [1], so I should again
> be able to devote some big-system test time to redoing performance
> results in perfbook.  Once that is done, I expect that it is time for
> the second edition.
> 
> I might also convert the blank page hiding the solution to the Dining
> Philosophers Problem to a quick quiz, but I consider this optional.
> 
> Are there any other changes that are needed? [2]

In response to Junchang's (off the list) proposal, I noticed that
Figure 10.27 needs update to reflect the change in the code done
in early 2019.

Can you update it?

The change simplified the lookup side, but doubled the cost of
updates during resizing. So it is likely the discussion in the text
also needs update.

        Thanks, Akira

> 
> 							Thanx, Paul
> 
> 
> [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
>     offline no-CBs CPUs") in -rcu, in case anyone is curious.
> 
> [2] Here is a list of some things that I believe can follow the second
>     edition:
> 
> 	Expand lock-free algorithm discussion to include LIFO push,
> 	illustrating the infamous pointer-zap issue.  (See ISO SC22
> 	WG21 P1726R3, which should appear in a couple of weeks, for
> 	more details.)
> 
> 	Add text describing the Issaquah Challenge.
> 
> 	Add text describing skiplists, one of the more concurrency
> 	friendly data structures.
> 
> 	Add text describing data-race detectors such as KCSAN.  (This needs
> 	to wait for more Linux-kernel experience.)
> 
> 	Additional material from todo.txt.  ;-)
> 

Re: Towards second edition

[Paul E. McKenney]

On Mon, Feb 24, 2020 at 08:42:09AM +0900, Akira Yokosawa wrote:
> On 2020/02/24 0:33, Paul E. McKenney wrote:
> > Hello!
> > 
> > I finally found and fixed the rcu_barrier() bug [1], so I should again
> > be able to devote some big-system test time to redoing performance
> > results in perfbook.  Once that is done, I expect that it is time for
> > the second edition.
> > 
> > I might also convert the blank page hiding the solution to the Dining
> > Philosophers Problem to a quick quiz, but I consider this optional.
> > 
> > Are there any other changes that are needed? [2]
> 
> In response to Junchang's (off the list) proposal, I noticed that
> Figure 10.27 needs update to reflect the change in the code done
> in early 2019.
> 
> Can you update it?
> 
> The change simplified the lookup side, but doubled the cost of
> updates during resizing. So it is likely the discussion in the text
> also needs update.

I will update this when rerunning on the large system in any case,
but thank you for calling my attention to this.  Yes, it does need
to be fixed.  (My problem in early 2019 was not having access to
a large system, now solved!)

							Thanx, Paul

>         Thanks, Akira
> 
> > 
> > 							Thanx, Paul
> > 
> > 
> > [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
> >     offline no-CBs CPUs") in -rcu, in case anyone is curious.
> > 
> > [2] Here is a list of some things that I believe can follow the second
> >     edition:
> > 
> > 	Expand lock-free algorithm discussion to include LIFO push,
> > 	illustrating the infamous pointer-zap issue.  (See ISO SC22
> > 	WG21 P1726R3, which should appear in a couple of weeks, for
> > 	more details.)
> > 
> > 	Add text describing the Issaquah Challenge.
> > 
> > 	Add text describing skiplists, one of the more concurrency
> > 	friendly data structures.
> > 
> > 	Add text describing data-race detectors such as KCSAN.  (This needs
> > 	to wait for more Linux-kernel experience.)
> > 
> > 	Additional material from todo.txt.  ;-)
> > 

Re: Towards second edition

[Дмитрий Дьяченко]

Hi Akira,

279eca4cc1debdfe5661ad98ed387dc85740c43e build PASS.
thank you.

Dmitry

пн, 24 февр. 2020 г. в 01:59, Akira Yokosawa <akiyks@gmail.com>:
>
> Hi Dmitry,
>
> On Sun, 23 Feb 2020 23:52:25 +0300, Дмитрий Дьяченко wrote:
> > Hello, Paul,
> >
> > I check FAQ-BUILD.txt and find Q. 10.A
> > " ... and TeX Live releases prior to 2020 (epigraph)".
> >
> > I update livetex today and probably I have the latest epigraph' version
> > ./texlive/texmf-dist/tex/latex/epigraph:
> > total 8
> > -rw-r--r--. 1 dima dima 4602 Jan  3 01:06 epigraph.sty
>
> So, you are using upstream TeX Live 2019, not the one distributed by
> Fedora.
>
> Due to the fairly massive updates in LaTeX base package to be released
> in TeX Live 2020, current upstream TeX Live lost compatibility with
> the "examplep" package.
>
> I contacted the author of the package about this issue, but have not
> heard from him yet.
>
> As a matter of fact, I have a patch set to resolve the issue in case
> the breakage persists. My thought was to submit it if Ubuntu/Fedora
> picks up the broken state of TeX Live in the next release.
>
> Now that you have encountered the issue, I'm submitting it for you
> to test.
>
>         Thanks, Akira
>
> >
> > To avoid make clean/make problems I
> > -- rm build-perfbook
> > -- update perfbook
> > -- copy perfbook into build-perfbook
> > -- run make in clean dir.
> >
> > I use Fedora 31 up-to-day, x86_64.
> >
> > I 'll try tomorrow to find the first commit after 07 feb with errors.
> >
> > Thank you,
> > Dmitry
> >
> > вс, 23 февр. 2020 г. в 21:47, Paul E. McKenney <paulmck@kernel.org>:
> >>
> >> On Sun, Feb 23, 2020 at 07:49:32PM +0300, Дмитрий Дьяченко wrote:
> >>> Hello!
> >>>
> >>> I build perfbook sometimes and try build today after read 'Toward second
> >>> edition' letter.
> >>> I use up-to-day livetex and up-to-day perfbook.
> >>>
> >>> 07-feb-2020 build PASS.
> >>> Today build FAIL [1].
> >>
> >> Hello, Dmitry,
> >>
> >> There was a change in the build system to fix a rendering problem.
> >> Could you please check recent changes to FAQ-BUILD.txt and see if they
> >> apply to you?  It might be necessary for you download epigraph.zip,
> >> depending on what distro version you happen to be using.  (I had to
> >> download it, for example.)
> >>
> >> Either way, does "make clean" followed by "make" help?
> >>
> >>                                                         Thanx, Paul
> >>
> >>> Thank you,
> >>> Dmitry
> >>>
> >>> [1] make
> >>> [...]
> >>> echo > origpub.tex
> >>> latexpand --empty-comments perfbook.tex 1> perfbook_flat.tex 2> /dev/null
> >>> sh utilities/extractqqz.sh < perfbook_flat.tex | perl utilities/
> >>> qqzreorder.pl > qqz.tex
> >>> cat perfbook_flat.tex qqz.tex | sh utilities/extractcontrib.sh > contrib.tex
> >>> sh utilities/extractorigpub.sh < perfbook_flat.tex > origpub.tex
> >>> sh utilities/runfirstlatex.sh perfbook
> >>> pdflatex 1 for perfbook.pdf
> >>>
> >>> LaTeX Warning: Reference `ln:toolsoftrade:Inviting a Store-to-Load
> >>> Conversion:l
> >>> oad:p' on page 42 undefined on input line 1988.
> >>>
> >>>
> >>> ! Package examplep Error: \PVerb inner delimiter must be brace,
> >>> (examplep)                not backslash or control sequence or active char.
> >>>
> >>> See the examplep package documentation for explanation.
> >>> Type  H <return>  for immediate help.
> >>>  ...
> >>>
> >>> l.1988 ...load:p} fetches \co{p}, but the \qco{if}
> >>>                                                    statement on
> >>> ?
> >>> ! Emergency stop.
> >>>  ...
> >>>
> >>> l.1988 ...load:p} fetches \co{p}, but the \qco{if}
> >>>                                                    statement on
> >>> Try typing  <return>  to proceed.
> >>> ----- Fatal latex error, see perfbook.log for details. -----
> >>> make: *** [Makefile:183: perfbook.aux] Ошибка 1
> >>> Command exited with non-zero status 2
> >>>
> >>> вс, 23 февр. 2020 г. в 18:33, Paul E. McKenney <paulmck@kernel.org>:
> >>>
> >>>> Hello!
> >>>>
> >>>> I finally found and fixed the rcu_barrier() bug [1], so I should again
> >>>> be able to devote some big-system test time to redoing performance
> >>>> results in perfbook.  Once that is done, I expect that it is time for
> >>>> the second edition.
> >>>>
> >>>> I might also convert the blank page hiding the solution to the Dining
> >>>> Philosophers Problem to a quick quiz, but I consider this optional.
> >>>>
> >>>> Are there any other changes that are needed? [2]
> >>>>
> >>>>                                                         Thanx, Paul
> >>>>
> >>>>
> >>>> [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
> >>>>     offline no-CBs CPUs") in -rcu, in case anyone is curious.
> >>>>
> >>>> [2] Here is a list of some things that I believe can follow the second
> >>>>     edition:
> >>>>
> >>>>         Expand lock-free algorithm discussion to include LIFO push,
> >>>>         illustrating the infamous pointer-zap issue.  (See ISO SC22
> >>>>         WG21 P1726R3, which should appear in a couple of weeks, for
> >>>>         more details.)
> >>>>
> >>>>         Add text describing the Issaquah Challenge.
> >>>>
> >>>>         Add text describing skiplists, one of the more concurrency
> >>>>         friendly data structures.
> >>>>
> >>>>         Add text describing data-race detectors such as KCSAN.  (This needs
> >>>>         to wait for more Linux-kernel experience.)
> >>>>
> >>>>         Additional material from todo.txt.  ;-)
> >>>>
>

Re: Towards second edition

[Junchang Wang]

On Sun, Feb 23, 2020 at 06:45:49PM -0800, Paul E. McKenney wrote:
>On Mon, Feb 24, 2020 at 08:42:09AM +0900, Akira Yokosawa wrote:
>> On 2020/02/24 0:33, Paul E. McKenney wrote:
>> > Hello!
>> > 
>> > I finally found and fixed the rcu_barrier() bug [1], so I should again
>> > be able to devote some big-system test time to redoing performance
>> > results in perfbook.  Once that is done, I expect that it is time for
>> > the second edition.
>> > 
>> > I might also convert the blank page hiding the solution to the Dining
>> > Philosophers Problem to a quick quiz, but I consider this optional.
>> > 
>> > Are there any other changes that are needed? [2]
>> 
>> In response to Junchang's (off the list) proposal, I noticed that
>> Figure 10.27 needs update to reflect the change in the code done
>> in early 2019.
>> 
>> Can you update it?
>> 
>> The change simplified the lookup side, but doubled the cost of
>> updates during resizing. So it is likely the discussion in the text
>> also needs update.
>
>I will update this when rerunning on the large system in any case,
>but thank you for calling my attention to this.  Yes, it does need
>to be fixed.  (My problem in early 2019 was not having access to
>a large system, now solved!)
>

Hi Paul and Akira,

I think you are discussing Figure 10.17 (Overhead of Resizing Hash
Tables Between 1024 ...). Is that right?

Here is a comment on this performance evaluation. Test framework
hashtorture.h indirectly controls the number of elements in a hash table
(and the average load factor) by inserting nupdaters*elperupdater/2
elements before running a test, such that U = nupdaters*elperupdater,
where U is the size of the pool of all of the test elements.  This
approach worked well for old machines. One thing worth noting is that U
could be too small for currently available servers which typically have
larger Last Level Caches of dozens of megabytes. For example, the LLC of
the E5 CPU I used to evaluate rehash can perfectly hold 131,072*2
elements, and hence lookup operations barely reach the memory (The Linux
perf reports close to zero cache misses.) Update/resize operations
update some elements and invalidate some cachelines, but it seems the
hardware prefetcher works well in this case, given that the test set U
is relatively small.

If we want to test the hash algorithm in an environment where lookup
operations may need to fetch data from memory, U should be set to a much
bigger number (e.g., 10M). But a bigger U means a bigger number of
pre-inserted elements if the above mentioned approach is used. So in my
experiments, I extended hashtorture.h and broke the relationship between
these two numbers. Please let me know if there are any other solutions.

But like you said, the code and the evaluation figure should be
simplified for a textbook. So I think it's reasonable that you still use
the existing approach, but the comment may worth being mentioned in the
perfbook to better explain your experimental result.


Thanks,
--Junchang


>							Thanx, Paul
>
>>         Thanks, Akira
>> 
>> > 
>> > 							Thanx, Paul
>> > 
>> > 
>> > [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
>> >     offline no-CBs CPUs") in -rcu, in case anyone is curious.
>> > 
>> > [2] Here is a list of some things that I believe can follow the second
>> >     edition:
>> > 
>> > 	Expand lock-free algorithm discussion to include LIFO push,
>> > 	illustrating the infamous pointer-zap issue.  (See ISO SC22
>> > 	WG21 P1726R3, which should appear in a couple of weeks, for
>> > 	more details.)
>> > 
>> > 	Add text describing the Issaquah Challenge.
>> > 
>> > 	Add text describing skiplists, one of the more concurrency
>> > 	friendly data structures.
>> > 
>> > 	Add text describing data-race detectors such as KCSAN.  (This needs
>> > 	to wait for more Linux-kernel experience.)
>> > 
>> > 	Additional material from todo.txt.  ;-)
>> > 

Re: Towards second edition

[Paul E. McKenney]

On Tue, Feb 25, 2020 at 11:30:57AM +0800, Junchang Wang wrote:
> On Sun, Feb 23, 2020 at 06:45:49PM -0800, Paul E. McKenney wrote:
> >On Mon, Feb 24, 2020 at 08:42:09AM +0900, Akira Yokosawa wrote:
> >> On 2020/02/24 0:33, Paul E. McKenney wrote:
> >> > Hello!
> >> > 
> >> > I finally found and fixed the rcu_barrier() bug [1], so I should again
> >> > be able to devote some big-system test time to redoing performance
> >> > results in perfbook.  Once that is done, I expect that it is time for
> >> > the second edition.
> >> > 
> >> > I might also convert the blank page hiding the solution to the Dining
> >> > Philosophers Problem to a quick quiz, but I consider this optional.
> >> > 
> >> > Are there any other changes that are needed? [2]
> >> 
> >> In response to Junchang's (off the list) proposal, I noticed that
> >> Figure 10.27 needs update to reflect the change in the code done
> >> in early 2019.
> >> 
> >> Can you update it?
> >> 
> >> The change simplified the lookup side, but doubled the cost of
> >> updates during resizing. So it is likely the discussion in the text
> >> also needs update.
> >
> >I will update this when rerunning on the large system in any case,
> >but thank you for calling my attention to this.  Yes, it does need
> >to be fixed.  (My problem in early 2019 was not having access to
> >a large system, now solved!)
> >
> 
> Hi Paul and Akira,
> 
> I think you are discussing Figure 10.17 (Overhead of Resizing Hash
> Tables Between 1024 ...). Is that right?
> 
> Here is a comment on this performance evaluation. Test framework
> hashtorture.h indirectly controls the number of elements in a hash table
> (and the average load factor) by inserting nupdaters*elperupdater/2
> elements before running a test, such that U = nupdaters*elperupdater,
> where U is the size of the pool of all of the test elements.  This
> approach worked well for old machines. One thing worth noting is that U
> could be too small for currently available servers which typically have
> larger Last Level Caches of dozens of megabytes. For example, the LLC of
> the E5 CPU I used to evaluate rehash can perfectly hold 131,072*2
> elements, and hence lookup operations barely reach the memory (The Linux
> perf reports close to zero cache misses.) Update/resize operations
> update some elements and invalidate some cachelines, but it seems the
> hardware prefetcher works well in this case, given that the test set U
> is relatively small.
> 
> If we want to test the hash algorithm in an environment where lookup
> operations may need to fetch data from memory, U should be set to a much
> bigger number (e.g., 10M). But a bigger U means a bigger number of
> pre-inserted elements if the above mentioned approach is used. So in my
> experiments, I extended hashtorture.h and broke the relationship between
> these two numbers. Please let me know if there are any other solutions.
> 
> But like you said, the code and the evaluation figure should be
> simplified for a textbook. So I think it's reasonable that you still use
> the existing approach, but the comment may worth being mentioned in the
> perfbook to better explain your experimental result.

Here is hoping that you are not increasing the hash-table per-bucket
load factor as you increase the number of CPUs, as one unfortunate recent
paper did.  Who is reviewing these things, anyway???  ;-)

							Thanx, Paul

> Thanks,
> --Junchang
> 
> 
> >							Thanx, Paul
> >
> >>         Thanks, Akira
> >> 
> >> > 
> >> > 							Thanx, Paul
> >> > 
> >> > 
> >> > [1] The fix is at 77abca1c358a ("rcu: Make rcu_barrier() account for
> >> >     offline no-CBs CPUs") in -rcu, in case anyone is curious.
> >> > 
> >> > [2] Here is a list of some things that I believe can follow the second
> >> >     edition:
> >> > 
> >> > 	Expand lock-free algorithm discussion to include LIFO push,
> >> > 	illustrating the infamous pointer-zap issue.  (See ISO SC22
> >> > 	WG21 P1726R3, which should appear in a couple of weeks, for
> >> > 	more details.)
> >> > 
> >> > 	Add text describing the Issaquah Challenge.
> >> > 
> >> > 	Add text describing skiplists, one of the more concurrency
> >> > 	friendly data structures.
> >> > 
> >> > 	Add text describing data-race detectors such as KCSAN.  (This needs
> >> > 	to wait for more Linux-kernel experience.)
> >> > 
> >> > 	Additional material from todo.txt.  ;-)
> >> >