From: Terence Kelly <tpkelly@eecs.umich.edu>
To: mtk.manpages@gmail.com
Cc: linux-man@vger.kernel.org, Christoph Hellwig <hch@lst.de>,
"Michael Kerrisk (man7.org)" <mtk@man7.org>
Subject: [patch]: mmap.2: New sample code: persistent data structure
Date: Tue, 26 Nov 2019 00:46:36 -0500 (EST) [thread overview]
Message-ID: <alpine.LRH.2.21.1911260040540.10521@email.eecs.umich.edu> (raw)
[-- Attachment #1: Type: text/plain, Size: 7942 bytes --]
[Verbatim re-transmission of my very recent e-mail (below), this time with
the correct TO: and CC: and SUBJECT: fields. Sorry about the confusion;
I'm new at the manpage patch game.]
---------- Forwarded message ----------
Date: Mon, 25 Nov 2019 23:51:53 -0500 (EST)
From: Terence Kelly <tpkelly@email.eecs.umich.edu>
To: "Michael Kerrisk (man7.org)" <mtk@man7.org>
Cc: Christoph Hellwig <hch@lst.de>
Subject: Re: suggesting an enhancement for man mmap(2)
Hi Michael,
Per our earlier conversation, attached is a patch for the mmap(2) manpage
(against man-pages-5.04/man2/mmap.2, which I downloaded earlier today and which
I believe is the latest version). Per the online instructions, the patch is
also inline below (apologies if my e-mail software mangles it).
My intent is to illustrate succinctly mmap()'s versatility and its most
important virtues:
Laying out application data structures in memory-mapped files obviates the need
for serializing/parsing for persistence by enabling applications to manipulate
persistent data with CPU instructions (LOAD and STORE). Moreover mmap() offers
high efficiency: Only accessed data are faulted in, and only modified data are
pushed back down to durability; in my example program, only the first and last
pages move between storage and memory.
I've tried to strike a tasteful division of labor between shell commands and C
code. My code compiles cleanly with all warnings enabled and it checks syscall
return values carefully.
Regarding my qualifications on this topic: I've been working on it for years;
search the ACM Digital Library for "persistent memory programming" to see a
recent example of my work.
Please let me know what you think. I'm willing to iterate with you on this. I
firmly believe that the full power of mmap() should be documented for developers
and I'll do whatever I can toward that end.
Thanks.
-- Terence
--- mmap.2_latest_from_man-pages-5.04 2019-11-25 19:00:44.908460718 -0800
+++ mmap.2_modified_by_Terence_Kelly 2019-11-25 20:17:13.843893947 -0800
@@ -36,6 +36,7 @@
.\" Modified 2006-12-04, mtk, various parts rewritten
.\" 2007-07-10, mtk, Added an example program.
.\" 2008-11-18, mtk, document MAP_STACK
+.\" 2019-11-25, Terence Kelly <tpkelly@eecs.umich.edu>, Added new example
program.
.\"
.TH MMAP 2 2019-10-10 "Linux" "Linux Programmer's Manual"
.SH NAME
@@ -900,7 +901,7 @@
.BR tmpfs (5)
(for example, when using the POSIX shared memory interface documented in
.BR shm_overview (7)).
-.SH EXAMPLE
+.SH EXAMPLES
.\" FIXME . Add an example here that uses an anonymous shared region for
.\" IPC between parent and child.
.PP
@@ -985,6 +986,100 @@
exit(EXIT_SUCCESS);
}
.EE
+.PP
+The following program maintains within a memory-mapped file a
+.I persistent data structure
+that outlives invocations of the program. Compile the program to
+.B a.out
+and use the
+.B truncate
+shell utility to create a sparse file named "pstack" that the program
+will populate with a persistent stack-of-integers data structure.
+Invoke the program with two kinds of command-line arguments:
+integers, which are pushed onto the persistent stack, and the string
+"pop," which causes the top integer on the stack to be printed and
+removed. In the sample shell session below, note that values pushed
+onto the stack by the first invocation of the program persist beyond
+program exit; the second invocation of the program pops these values.
+.PP
+.in +4n
+.EX
+.B $ truncate -s `getconf PAGESIZE` pstack
+.B $ ./a.out 1 2 3
+.B $ ./a.out pop pop pop pop
+3
+2
+1
+<stack empty>
+.EE
+.in
+.SS Program source
+.EX
+#include <fcntl.h>
+#include <inttypes.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+#define handle_error(msg) \\
+ do { perror(msg); exit(EXIT_FAILURE); } while (0)
+
+int
+main(int argc, char *argv[])
+{
+ int fd, a;
+ struct stat sb;
+ size_t len;
+ struct pstack { /* persistent stack header */
+ int32_t n; /* number of items in stack */
+ int32_t s[]; /* array containing stack */
+ } *p; /* ptr to mmap'd stack file */
+
+ fd = open("pstack", O_RDWR);
+ if (fd == \-1)
+ handle_error("open");
+
+ if (fstat(fd, &sb) != 0) /* to obtain file size */
+ handle_error("fstat");
+
+ len = (size_t) sb.st_size;
+
+ p = (struct pstack *) mmap(NULL, len, PROT_READ | PROT_WRITE,
+ MAP_SHARED, fd, 0);
+ if (p == MAP_FAILED)
+ handle_error("mmap");
+
+ for (a = 1; a < argc; a++) {
+ if (strcmp(argv[a], "pop") == 0) {
+ if (p->n <= 0) {
+ printf("<stack empty>\\n");
+ exit(EXIT_FAILURE);
+ } else {
+ printf("%" PRId32 "\\n", p->s[--p->n]);
+ }
+ } else {
+ if (sizeof *p + (size_t) p->n * sizeof p->n >= len) {
+ printf("<stack full>\\n");
+ exit(EXIT_FAILURE);
+ } else {
+ p->s[p->n++] = atoi(argv[a]);
+ }
+ }
+ }
+
+ if (close(fd) != 0) /* implicit on exit() */
+ handle_error("close");
+
+ if (munmap(p, len) != 0) /* implicit on exit() */
+ handle_error("munmap");
+
+ exit(EXIT_SUCCESS);
+}
+.EE
.SH SEE ALSO
.BR ftruncate (2),
.BR getpagesize (2),
@@ -1010,6 +1105,13 @@
.IR /proc/[pid]/smaps .
.PP
B.O. Gallmeister, POSIX.4, O'Reilly, pp. 128\(en129 and 389\(en391.
+.PP
+T. Kelly, "Persistent Memory Programming on Conventional Hardware,"
+ACM
+.I
+Queue
+magazine, Vol. 17, No. 4, July/August 2019
+\%https://queue.acm.org/detail.cfm?id=3358957
.\"
.\" Repeat after me: private read-only mappings are 100% equivalent to
.\" shared read-only mappings. No ifs, buts, or maybes. -- Linus
On Thu, 21 Nov 2019, Michael Kerrisk (man7.org) wrote:
> Hello Terence,
>
> My apologies for the slow reply. Manual page topics should really be
> directed as per https://www.kernel.org/doc/man-pages/contributing.html
>
> I'm agnostic about your proposal. It could be interesting and useful,
> but I wonder if the example itself might be a large piece of code?
>
> Thanks,
>
> Michael
>
> On Wed, 18 Sep 2019 at 05:59, Terence Kelly <tpkelly@eecs.umich.edu> wrote:
> >
> >
> >
> > Mr. Kerrisk,
> >
> > First, many thanks for your efforts to improve the Linux man pages, and
> > for your excellent book. The community owes you a debt of gratitude.
> >
> > I'm writing to suggest an enhancement for man mmap(2). My goal is to
> > teach readers that good old fashioned mmap(2) can support what we might
> > call "the persistent memory style of programming" on conventional hardware
> > and OSes. I've written an article on this subject containing simple
> > example programs:
> >
> > https://queue.acm.org/detail.cfm?id=3358957
> >
> > Nowadays often lost in the hype surrounding non-volatile memory hardware
> > (Intel Optane) is the simple fact that mmap() alone, together with a few
> > very simple idioms and tricks, can support a useful software abstraction
> > of persistent memory on conventional hardware.
> >
> > With your permission I'd like to write a new example program for the
> > mmap(2) man page. You might also consider including a pointer to the
> > article above if you deem it appropriate.
> >
> > If you're open to my basic suggestion, please advise how I may help.
> >
> > Thank you.
> >
> > -- Terence Kelly
> >
> > P.S.: I notice from your Web site that you're a Kiwi. When the
> > ionosphere is in a cooperative mood I often get my news from Radio New
> > Zealand via shortwave.
> >
>
>
> --
> Michael Kerrisk, man7.org Training and Consulting
> mtk@man7.org, http://man7.org/training/
> "The Linux Programming Interface" -- http://man7.org/tlpi/
[-- Attachment #2: Type: application/x-gzip, Size: 13956 bytes --]
next reply other threads:[~2019-11-26 6:11 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-26 5:46 Terence Kelly [this message]
2019-12-12 4:55 ` [patch]: mmap.2: New sample code: persistent data structure Terence Kelly
2020-02-22 1:04 ` Terence Kelly
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.LRH.2.21.1911260040540.10521@email.eecs.umich.edu \
--to=tpkelly@eecs.umich.edu \
--cc=hch@lst.de \
--cc=linux-man@vger.kernel.org \
--cc=mtk.manpages@gmail.com \
--cc=mtk@man7.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).