[RFC] Porting kernel MTD tests to user space

[RFC] Porting kernel MTD tests to user space

[Richard Weinberger]

David was so kind and did an initial port of Linux's MTD tests to user space.
He took most tests as-is and massaged them to work in user space using libmtd.
In the long run the goal is giving up the kernel tests as much as possible
and improve the tests present in mtd-utils.

Feedback is very welcome!

Thanks,
//richard

[PATCH 1/8] mtd-utils: Fix return status in mtd_torture test function
[PATCH 2/8] mtd-utils: Add multi-block erase function
[PATCH 3/8] mtd-utils: Add flash torture test utility
[PATCH 4/8] mtd-utils: Add flash stress test Utility
[PATCH 5/8] mtd-utils: Add flash speed test Utility
[PATCH 6/8] mtd-utils: Add nand flash bit errors test
[PATCH 7/8] mtd-utils: Add flash read test utility
[PATCH 8/8] mtd-utils: Add nand page test utility

 .gitignore                  |   6 +
 Makefile                    |   7 +-
 include/libmtd.h            |  14 ++
 lib/libmtd.c                |  28 ++-
 misc-utils/flash_readtest.c | 263 ++++++++++++++++++++
 misc-utils/flash_speed.c    | 463 +++++++++++++++++++++++++++++++++++
 misc-utils/flash_stress.c   | 276 +++++++++++++++++++++
 misc-utils/flash_torture.c  | 240 ++++++++++++++++++
 nand-utils/nandbiterrs.c    | 497 +++++++++++++++++++++++++++++++++++++
 nand-utils/nandpagetest.c   | 579 ++++++++++++++++++++++++++++++++++++++++++++
 10 files changed, 2362 insertions(+), 11 deletions(-)

[PATCH 1/8] mtd-utils: Fix return status in mtd_torture test function

[Richard Weinberger]

From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

This patch fixes the return status of the mtd_torture function
in libmtd.

The torture test function is currently only used by the ubiformat
utilitiy to check if a block is bad after a write fails (blocks are
marked bad if the function returns an error status). However, the
way the function was written, it ALWAYS returns an error value
irregardless of whether it failed or not.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
---
 lib/libmtd.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/libmtd.c b/lib/libmtd.c
index bf6d71f..1717468 100644
--- a/lib/libmtd.c
+++ b/lib/libmtd.c
@@ -939,7 +939,7 @@ static uint8_t patterns[] = {0xa5, 0x5a, 0x0};
  * @patt: the pattern to check
  * @size: buffer size in bytes
  *
- * This function returns %1 in there are only @patt bytes in @buf, and %0 if
+ * This function returns %0 if there are only @patt bytes in @buf, and %-1 if
  * something else was also found.
  */
 static int check_pattern(const void *buf, uint8_t patt, int size)
@@ -948,8 +948,8 @@ static int check_pattern(const void *buf, uint8_t patt, int size)
 
 	for (i = 0; i < size; i++)
 		if (((const uint8_t *)buf)[i] != patt)
-			return 0;
-	return 1;
+			return -1;
+	return 0;
 }
 
 int mtd_torture(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
@@ -973,7 +973,7 @@ int mtd_torture(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
 			goto out;
 
 		err = check_pattern(buf, 0xFF, mtd->eb_size);
-		if (err == 0) {
+		if (err) {
 			errmsg("erased PEB %d, but a non-0xFF byte found", eb);
 			errno = EIO;
 			goto out;
@@ -992,7 +992,7 @@ int mtd_torture(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
 			goto out;
 
 		err = check_pattern(buf, patterns[i], mtd->eb_size);
-		if (err == 0) {
+		if (err) {
 			errmsg("pattern %x checking failed for PEB %d",
 				patterns[i], eb);
 			errno = EIO;
@@ -1005,7 +1005,7 @@ int mtd_torture(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
 
 out:
 	free(buf);
-	return -1;
+	return err;
 }
 
 int mtd_is_bad(const struct mtd_dev_info *mtd, int fd, int eb)
-- 
2.7.3

[PATCH 2/8] mtd-utils: Add multi-block erase function

[Richard Weinberger]

From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
---
 include/libmtd.h | 14 ++++++++++++++
 lib/libmtd.c     | 16 +++++++++++++---
 2 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/include/libmtd.h b/include/libmtd.h
index a78c8cb..f3089d4 100644
--- a/include/libmtd.h
+++ b/include/libmtd.h
@@ -174,6 +174,20 @@ int mtd_lock(const struct mtd_dev_info *mtd, int fd, int eb);
 int mtd_unlock(const struct mtd_dev_info *mtd, int fd, int eb);
 
 /**
+ * mtd_erase - erase multiple eraseblocks.
+ * @desc: MTD library descriptor
+ * @mtd: MTD device description object
+ * @fd: MTD device node file descriptor
+ * @eb: index of first eraseblock to erase
+ * @blocks: the number of eraseblocks to erase
+ *
+ * This function erases eraseblock @eb of MTD device described by @fd. Returns
+ * %0 in case of success and %-1 in case of failure.
+ */
+int mtd_erase_multi(libmtd_t desc, const struct mtd_dev_info *mtd,
+			int fd, int eb, int blocks);
+
+/**
  * mtd_erase - erase an eraseblock.
  * @desc: MTD library descriptor
  * @mtd: MTD device description object
diff --git a/lib/libmtd.c b/lib/libmtd.c
index 1717468..6101238 100644
--- a/lib/libmtd.c
+++ b/lib/libmtd.c
@@ -845,7 +845,8 @@ int mtd_unlock(const struct mtd_dev_info *mtd, int fd, int eb)
 	return mtd_xlock(mtd, fd, eb, MEMUNLOCK);
 }
 
-int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
+int mtd_erase_multi(libmtd_t desc, const struct mtd_dev_info *mtd,
+			int fd, int eb, int blocks)
 {
 	int ret;
 	struct libmtd *lib = (struct libmtd *)desc;
@@ -856,8 +857,12 @@ int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
 	if (ret)
 		return ret;
 
-	ei64.start = (__u64)eb * mtd->eb_size;
-	ei64.length = mtd->eb_size;
+	ret = mtd_valid_erase_block(mtd, eb + blocks - 1);
+	if (ret)
+		return ret;
+
+	ei64.start = (__u64)eb * (__u64)mtd->eb_size;
+	ei64.length = (__u64)mtd->eb_size * (__u64)blocks;
 
 	if (lib->offs64_ioctls == OFFS64_IOCTLS_SUPPORTED ||
 	    lib->offs64_ioctls == OFFS64_IOCTLS_UNKNOWN) {
@@ -892,6 +897,11 @@ int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
 	return 0;
 }
 
+int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
+{
+	return mtd_erase_multi(desc, mtd, fd, eb, 1);
+}
+
 int mtd_regioninfo(int fd, int regidx, struct region_info_user *reginfo)
 {
 	int ret;
-- 
2.7.3

[PATCH 3/8] mtd-utils: Add flash torture test utility

[Richard Weinberger]

From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

Basically a user space port of the mtd torture test kernel module. In
addition to the block offset and count module parameters, the utility
supports a block stride and can restore the block contents after test.

In contrast to the kernel module, the torture test is implemented by
the libmtd mtd_toruture function and thus doesn't allow for similarly
fine grained options on diagnostics.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
---
 .gitignore                 |   1 +
 Makefile                   |   2 +-
 misc-utils/flash_torture.c | 240 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 242 insertions(+), 1 deletion(-)
 create mode 100644 misc-utils/flash_torture.c

diff --git a/.gitignore b/.gitignore
index 2aac52c..5b529d1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -35,6 +35,7 @@
 /jffsX-utils/mkfs.jffs2
 /misc-utils/mtd_debug
 /misc-utils/mtdpart
+/misc-utils/flash_torture
 /nand-utils/nanddump
 /nand-utils/nandtest
 /nand-utils/nandwrite
diff --git a/Makefile b/Makefile
index 977c9c5..af3d1fd 100644
--- a/Makefile
+++ b/Makefile
@@ -20,7 +20,7 @@ MISC_BINS = \
 	ftl_format doc_loadbios ftl_check mtd_debug docfdisk \
 	serve_image recv_image mtdpart flash_erase flash_lock \
 	flash_unlock flash_otp_info flash_otp_dump flash_otp_lock \
-	flash_otp_write flashcp
+	flash_otp_write flashcp flash_torture
 UBI_BINS = \
 	ubiupdatevol ubimkvol ubirmvol ubicrc32 ubinfo ubiattach \
 	ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock
diff --git a/misc-utils/flash_torture.c b/misc-utils/flash_torture.c
new file mode 100644
index 0000000..b5625c8
--- /dev/null
+++ b/misc-utils/flash_torture.c
@@ -0,0 +1,240 @@
+/*
+ * Copyright (C) 2006-2008 Artem Bityutskiy
+ * Copyright (C) 2006-2008 Jarkko Lavinen
+ * Copyright (C) 2006-2008 Adrian Hunter
+ * Copyright (C) 2015 sigma star gmbh
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ *
+ * WARNING: this test program may kill your flash and your device. Do not
+ * use it unless you know what you do. Authors are not responsible for any
+ * damage caused by this program.
+ *
+ * Author: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
+ *
+ * Based on linux torturetest.c
+ * Authors: Artem Bityutskiy, Jarkko Lavinen, Adria Hunter
+ */
+
+#define PROGRAM_NAME "flash_torture"
+
+#define KEEP_CONTENTS 0x01
+#define RUN_FOREVER 0x02
+
+#include <mtd/mtd-user.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <libmtd.h>
+#include <signal.h>
+#include <stdio.h>
+#include <fcntl.h>
+
+#include "common.h"
+
+static int peb=-1, blocks=-1, skip=-1;
+static struct mtd_dev_info mtd;
+static sig_atomic_t flags=0;
+static const char *mtddev;
+static libmtd_t mtd_desc;
+static int mtdfd;
+
+static void sighandler(int sig)
+{
+	if (sig == SIGINT || sig == SIGTERM || sig == SIGHUP)
+		flags &= ~RUN_FOREVER;
+}
+
+static void usage(int status)
+{
+	fputs(
+	"Usage: "PROGRAM_NAME" [OPTIONS] <device>\n\n"
+	"Options:\n"
+	"  -h, --help         Display this help output\n"
+	"  -b, --peb <num>    Start from this physical erase block\n"
+	"  -c, --blocks <num> Number of erase blocks to torture\n"
+	"  -s, --skip <num>   Number of erase blocks to skip\n"
+	"  -k, --keep         Try to restore existing contents after test\n"
+	"  -r, --repeate      Repeate the torture test indefinitely\n",
+	status==EXIT_SUCCESS ? stdout : stderr);
+	exit(status);
+}
+
+static long read_num(int idx, int argidx, int argc, char **argv)
+{
+	char *end;
+	long num;
+
+	if (argidx >= argc) {
+		fprintf(stderr, "%s: missing argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+
+	num = strtol(argv[argidx], &end, 0);
+
+	if (!end || *end!='\0') {
+		fprintf(stderr, "%s: expected integer argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+	return num;
+}
+
+static void process_options(int argc, char **argv)
+{
+	int i;
+
+	for (i=1; i<argc; ++i) {
+		if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
+			usage(EXIT_SUCCESS);
+		} else if (!strcmp(argv[i], "--peb") || !strcmp(argv[i], "-b")) {
+			if (peb >= 0)
+				goto failmulti;
+			peb = read_num(i, i+1, argc, argv);
+			if (peb < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--blocks") || !strcmp(argv[i], "-c")) {
+			if (blocks > 0)
+				goto failmulti;
+			blocks = read_num(i, i+1, argc, argv);
+			if (blocks <= 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--skip") || !strcmp(argv[i], "-s")) {
+			if (skip >= 0)
+				goto failmulti;
+			skip = read_num(i, i+1, argc, argv);
+			if (skip < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--keep") || !strcmp(argv[i], "-k")) {
+			if (flags & KEEP_CONTENTS)
+				goto failmulti;
+			flags |= KEEP_CONTENTS;
+		} else if (!strcmp(argv[i], "--repeate") || !strcmp(argv[i], "-r")) {
+			if (flags & RUN_FOREVER)
+				goto failmulti;
+			flags |= RUN_FOREVER;
+		} else {
+			if (mtddev)
+				usage(EXIT_FAILURE);
+			mtddev = argv[i];
+		}
+	}
+
+	if (!mtddev)
+		errmsg_die("No device specified!\n");
+	if (peb < 0)
+		peb = 0;
+	if (skip < 0)
+		skip = 0;
+	if (blocks < 0)
+		blocks = 1;
+	return;
+failmulti:
+	fprintf(stderr, "'%s' specified more than once!\n", argv[i]);
+	exit(EXIT_FAILURE);
+failarg:
+	fprintf(stderr, "Invalid argument for '%s'!\n", argv[i]);
+	exit(EXIT_FAILURE);
+}
+
+int main(int argc, char **argv)
+{
+	int i, eb, err, count = 0;
+	char* is_bad = NULL;
+	void *old=NULL;
+
+	process_options(argc, argv);
+
+	mtd_desc = libmtd_open();
+	if (!mtd_desc)
+		return errmsg("can't initialize libmtd");
+
+	if (mtd_get_dev_info(mtd_desc, mtddev, &mtd) < 0)
+		return errmsg("mtd_get_dev_info failed");
+
+	if (peb >= mtd.eb_cnt)
+		return errmsg("Physical erase block %d is out of range!\n", peb);
+
+	if ((peb + (blocks - 1)*(skip + 1)) >= mtd.eb_cnt) {
+		return errmsg("Given block range exceeds block count of %d!\n",
+				mtd.eb_cnt);
+	}
+
+	signal(SIGINT, sighandler);
+	signal(SIGTERM, sighandler);
+	signal(SIGHUP, sighandler);
+
+	if (flags & KEEP_CONTENTS) {
+		old = xmalloc(mtd.eb_size);
+	}
+
+	is_bad = xmalloc(blocks);
+
+	if ((mtdfd = open(mtddev, O_RDWR)) == -1) {
+		perror(mtddev);
+		free(is_bad);
+		free(old);
+		return EXIT_FAILURE;
+	}
+
+	for (i = 0; i < blocks; ++i) {
+		eb = peb + i * (skip + 1);
+		is_bad[i] = mtd_is_bad(&mtd, mtdfd, eb);
+		if (is_bad[i]) {
+			fprintf(stderr, "PEB %d marked bad, will be skipped\n", eb);
+		}
+	}
+
+	do {
+		for (i = 0; i < blocks; ++i) {
+			if (is_bad[i])
+				continue;
+
+			eb = peb + i * (skip + 1);
+
+			if (flags & KEEP_CONTENTS) {
+				err = mtd_read(&mtd, mtdfd, eb, 0, old, mtd.eb_size);
+				if (err) {
+					fprintf(stderr, "Failed to create backup copy "
+							"of PEB %d, skipping!\n", eb);
+					continue;
+				}
+			}
+
+			if (mtd_torture(mtd_desc, &mtd, mtdfd, eb))
+				fprintf(stderr, "Block %d failed torture test!\n", eb);
+
+			if (flags & KEEP_CONTENTS) {
+				err = mtd_erase(mtd_desc, &mtd, mtdfd, eb);
+				if (err) {
+					fprintf(stderr, "mtd_erase failed for block %d!\n", eb);
+					continue;
+				}
+				err = mtd_write(mtd_desc, &mtd, mtdfd, eb, 0,
+						old, mtd.eb_size, NULL, 0, 0);
+				if (err)
+					fprintf(stderr, "Failed to restore block %d!\n", eb);
+			}
+		}
+
+		printf("Torture test iterations done: %d\n", ++count);
+	} while (flags & RUN_FOREVER);
+
+	free(old);
+	free(is_bad);
+	close(mtdfd);
+	return EXIT_SUCCESS;
+}
-- 
2.7.3

[PATCH 4/8] mtd-utils: Add flash stress test Utility

[Richard Weinberger]

From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

Basically a user space port of the mtd stress test kernel module.
In addition to the block offset and count module parameters, the
utility supports a block stride and can restore the block contents
after test.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
---
 .gitignore                |   1 +
 Makefile                  |   2 +-
 misc-utils/flash_stress.c | 276 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 278 insertions(+), 1 deletion(-)
 create mode 100644 misc-utils/flash_stress.c

diff --git a/.gitignore b/.gitignore
index 5b529d1..3d03708 100644
--- a/.gitignore
+++ b/.gitignore
@@ -36,6 +36,7 @@
 /misc-utils/mtd_debug
 /misc-utils/mtdpart
 /misc-utils/flash_torture
+/misc-utils/flash_stress
 /nand-utils/nanddump
 /nand-utils/nandtest
 /nand-utils/nandwrite
diff --git a/Makefile b/Makefile
index af3d1fd..1bc41e0 100644
--- a/Makefile
+++ b/Makefile
@@ -20,7 +20,7 @@ MISC_BINS = \
 	ftl_format doc_loadbios ftl_check mtd_debug docfdisk \
 	serve_image recv_image mtdpart flash_erase flash_lock \
 	flash_unlock flash_otp_info flash_otp_dump flash_otp_lock \
-	flash_otp_write flashcp flash_torture
+	flash_otp_write flashcp flash_torture flash_stress
 UBI_BINS = \
 	ubiupdatevol ubimkvol ubirmvol ubicrc32 ubinfo ubiattach \
 	ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock
diff --git a/misc-utils/flash_stress.c b/misc-utils/flash_stress.c
new file mode 100644
index 0000000..2dd2da1
--- /dev/null
+++ b/misc-utils/flash_stress.c
@@ -0,0 +1,276 @@
+/*
+ * Copyright (C) 2006-2008 Nokia Corporation
+ * Copyright (C) 2015 sigma star gmbh
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Test random reads, writes and erases on MTD device.
+ *
+ * Author: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
+ *
+ * Based on linux stresstest.c
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ */
+#define PROGRAM_NAME "flash_stress"
+
+#define KEEP_CONTENTS 0x01
+#define COUNT_CHANGED 0x02
+#define SEED_SET 0x04
+
+#include <mtd/mtd-user.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <libmtd.h>
+#include <stdio.h>
+#include <fcntl.h>
+#include <time.h>
+
+#include "common.h"
+
+static struct mtd_dev_info mtd;
+static const char *mtddev;
+static libmtd_t mtd_desc;
+static int fd;
+
+static unsigned char *writebuf;
+static unsigned char *readbuf;
+static unsigned char *old;
+static unsigned char *bbt;
+
+static int pgsize;
+static int pgcnt;
+
+static int count = 10000;
+static int flags = 0;
+
+static void usage(int status)
+{
+	fputs(
+	"Usage: "PROGRAM_NAME" [OPTIONS] <device>\n\n"
+	"Options:\n"
+	"  -h, --help         Display this help output\n"
+	"  -c, --count <num>  Number of operations to do (default is 10000)\n"
+	"  -s, --seed <num>   Seed for pseudor random number generator\n"
+	"  -k, --keep         Restore existing contents after test\n",
+	status==EXIT_SUCCESS ? stdout : stderr);
+	exit(status);
+}
+
+static long read_num(int idx, int argidx, int argc, char **argv)
+{
+	char *end;
+	long num;
+
+	if (argidx >= argc) {
+		fprintf(stderr, "%s: missing argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+
+	num = strtol(argv[argidx], &end, 0);
+
+	if (!end || *end!='\0') {
+		fprintf(stderr, "%s: expected integer argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+	return num;
+}
+
+static void process_options(int argc, char **argv)
+{
+	int i;
+
+	for (i=1; i<argc; ++i) {
+		if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
+			usage(EXIT_SUCCESS);
+		} else if (!strcmp(argv[i], "--keep") || !strcmp(argv[i], "-k")) {
+			if (flags & KEEP_CONTENTS)
+				goto failmulti;
+			flags |= KEEP_CONTENTS;
+		} else if (!strcmp(argv[i], "--seed") || !strcmp(argv[i], "-s")) {
+			if (flags & SEED_SET)
+				goto failmulti;
+			srand(read_num(i, i+1, argc, argv));
+			flags |= SEED_SET;
+		} else if (!strcmp(argv[i], "--count") || !strcmp(argv[i], "-c")) {
+			if (flags & COUNT_CHANGED)
+				goto failmulti;
+			count = read_num(i, i+1, argc, argv);
+			if (count <= 0)
+				goto failarg;
+			++i;
+			flags |= COUNT_CHANGED;
+		} else {
+			if (mtddev)
+				usage(EXIT_FAILURE);
+			mtddev = argv[i];
+		}
+	}
+
+	if (!mtddev)
+		errmsg_die("No device specified!\n");
+
+	if (!(flags & SEED_SET))
+		srand(time(NULL));
+	return;
+failmulti:
+	fprintf(stderr, "'%s' specified more than once!\n", argv[i]);
+	exit(EXIT_FAILURE);
+failarg:
+	fprintf(stderr, "Invalid argument for '%s'!\n", argv[i]);
+	exit(EXIT_FAILURE);
+}
+
+static int rand_eb(void)
+{
+	unsigned int eb;
+
+	/* Read or write up 2 eraseblocks at a time - hence 'mtd.eb_cnt - 1' */
+	do {
+		eb = rand() % (mtd.eb_cnt - 1);
+	} while (bbt[eb]);
+
+	return eb;
+}
+
+static int do_read(void)
+{
+	int eb = rand_eb();
+	int offs = rand() % pgcnt;
+	int len = rand() % (pgcnt - offs);
+
+	offs *= pgsize;
+	len *= pgsize;
+	return mtd_read(&mtd, fd, eb, offs, readbuf, len);
+}
+
+static int do_write(void)
+{
+	int eb = rand_eb(), err, err1;
+	int offs = rand() % pgcnt;
+	int len = rand() % (pgcnt - offs);
+
+	offs *= pgsize;
+	len *= pgsize;
+
+	if (flags & KEEP_CONTENTS) {
+		err = mtd_read(&mtd, fd, eb, 0, old, mtd.eb_size);
+		if (err) {
+			fputs("Error backing up old erase block contents\n", stderr);
+			return -1;
+		}
+	}
+
+	err = mtd_erase(mtd_desc, &mtd, fd, eb);
+	if (err)
+		goto out;
+
+	err = mtd_write(mtd_desc, &mtd, fd, eb, offs,
+			writebuf, len, NULL, 0, 0);
+	if (err)
+		goto out;
+
+	err = 0;
+out:
+	if (flags & KEEP_CONTENTS) {
+		if (mtd_erase(mtd_desc, &mtd, fd, eb)) {
+			fprintf(stderr, "mtd_erase: PEB %d", eb);
+			return -1;
+		}
+
+		err1 = mtd_write(mtd_desc, &mtd, fd, eb, 0,
+					old, mtd.eb_size, NULL, 0, 0);
+
+		if (err1) {
+			fprintf(stderr, "Failed to restore old contents\n");
+			return -1;
+		}
+	}
+	return err;
+}
+
+static void scan_for_bad_eraseblocks(unsigned int eb, int ebcnt)
+{
+	int i, bad = 0;
+
+	puts("scanning for bad eraseblocks");
+
+	for (i = 0; i < ebcnt; ++i) {
+		bbt[i] = mtd_is_bad(&mtd, fd, eb + i) ? 1 : 0;
+		if (bbt[i])
+			bad += 1;
+	}
+
+	printf("scanned %d eraseblocks, %d are bad\n", ebcnt, bad);
+}
+
+int main(int argc, char **argv)
+{
+	int status = EXIT_FAILURE, i, op, err;
+
+	process_options(argc, argv);
+
+	mtd_desc = libmtd_open();
+	if (!mtd_desc)
+		return errmsg("can't initialize libmtd");
+
+	if (mtd_get_dev_info(mtd_desc, mtddev, &mtd) < 0)
+		return errmsg("mtd_get_dev_info failed");
+
+	if (mtd.subpage_size == 1) {
+		puts("not NAND flash, assume page size is 512 bytes.");
+		pgsize = 512;
+	} else {
+		pgsize = mtd.subpage_size;
+	}
+
+	pgcnt = mtd.eb_size / pgsize;
+
+	readbuf = xmalloc(mtd.eb_size);
+	writebuf = xmalloc(mtd.eb_size);
+	bbt = xzalloc(mtd.eb_cnt);
+
+	if (flags & KEEP_CONTENTS)
+		old = xmalloc(mtd.eb_size);
+
+	for (i = 0; i < mtd.eb_size; ++i)
+		writebuf[i] = rand();
+
+	/* Open device file */
+	if ((fd = open(mtddev, O_RDWR)) == -1) {
+		perror(mtddev);
+		goto out;
+	}
+
+	/* Do operations */
+	scan_for_bad_eraseblocks(0, mtd.eb_cnt);
+
+	puts("doing operations");
+	for (op = 0; op < count; op++) {
+		if ((op & 1023) == 0)
+			printf("%d operations done\n", op);
+		err = (rand() & 1) ? do_read() : do_write();
+		if (err)
+			goto out;
+	}
+	printf("finished, %d operations done\n", op);
+
+	status = EXIT_SUCCESS;
+out:
+	close(fd);
+	free(bbt);
+	free(writebuf);
+	free(readbuf);
+	free(old);
+	return status;
+}
-- 
2.7.3

[PATCH 5/8] mtd-utils: Add flash speed test Utility

[Richard Weinberger]

From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

Basically a user space port of the mtd speed test kernel module.
In addition to the block offset and count module parameters, the
utility supports a block stride and can restore the block contents
after test. Furthermore, a flag can be used to disable destructive
tests (i.e. only perform read speed tests).

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
---
 .gitignore               |   1 +
 Makefile                 |   4 +-
 misc-utils/flash_speed.c | 463 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 467 insertions(+), 1 deletion(-)
 create mode 100644 misc-utils/flash_speed.c

diff --git a/.gitignore b/.gitignore
index 3d03708..6d98b3d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -37,6 +37,7 @@
 /misc-utils/mtdpart
 /misc-utils/flash_torture
 /misc-utils/flash_stress
+/misc-utils/flash_speed
 /nand-utils/nanddump
 /nand-utils/nandtest
 /nand-utils/nandwrite
diff --git a/Makefile b/Makefile
index 1bc41e0..2a7715d 100644
--- a/Makefile
+++ b/Makefile
@@ -20,7 +20,7 @@ MISC_BINS = \
 	ftl_format doc_loadbios ftl_check mtd_debug docfdisk \
 	serve_image recv_image mtdpart flash_erase flash_lock \
 	flash_unlock flash_otp_info flash_otp_dump flash_otp_lock \
-	flash_otp_write flashcp flash_torture flash_stress
+	flash_otp_write flashcp flash_torture flash_stress flash_speed
 UBI_BINS = \
 	ubiupdatevol ubimkvol ubirmvol ubicrc32 ubinfo ubiattach \
 	ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock
@@ -97,6 +97,8 @@ LDLIBS_mkfs.jffs2  = -lz $(LZOLDLIBS)
 LDFLAGS_jffs2reader = $(ZLIBLDFLAGS) $(LZOLDFLAGS)
 LDLIBS_jffs2reader  = -lz $(LZOLDLIBS)
 
+LDLIBS_flash_speed = -lrt
+
 $(foreach v,$(MISC_BINS),$(eval $(call mkdep,misc-utils/,$(v))))
 $(foreach v,$(JFFSX_BINS),$(eval $(call mkdep,jffsX-utils/,$(v))))
 $(foreach v,$(NAND_BINS),$(eval $(call mkdep,nand-utils/,$(v))))
diff --git a/misc-utils/flash_speed.c b/misc-utils/flash_speed.c
new file mode 100644
index 0000000..9b5355b
--- /dev/null
+++ b/misc-utils/flash_speed.c
@@ -0,0 +1,463 @@
+/*
+ * Copyright (C) 2007 Nokia Corporation
+ * Copyright (C) 2015 sigma star gmbh
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Test read and write speed of a MTD device.
+ *
+ * Author: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
+ *
+ * Based on linux flash_speed.c
+ * Author: Adrian Hunter <adrian.hunter@nokia.com>
+ */
+#define DESTRUCTIVE 0x01
+
+#define PROGRAM_NAME "flash_speed"
+
+#include <mtd/mtd-user.h>
+#include <sys/ioctl.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libmtd.h>
+#include <stdio.h>
+#include <fcntl.h>
+#include <time.h>
+
+#include "common.h"
+
+static struct mtd_dev_info mtd;
+static unsigned char *iobuf;
+static unsigned char *bbt;
+static const char *mtddev;
+static libmtd_t mtd_desc;
+static int fd;
+
+static int peb=-1, count=-1, skip=-1, flags=0;
+static struct timespec start, finish;
+static int pgsize, pgcnt;
+static int goodebcnt;
+
+static void usage(int status)
+{
+	fputs(
+	"Usage: "PROGRAM_NAME" [OPTIONS] <device>\n\n"
+	"Common options:\n"
+	"  -h, --help          Display this help output\n"
+	"  -b, --peb <num>     Start from this physical erase block\n"
+	"  -c, --count <num>   Number of erase blocks to use (default: all)\n"
+	"  -s, --skip <num>    Number of blocks to skip\n"
+	"  -d, --destructive   Run destructive (erase and write speed) tests\n",
+	status==EXIT_SUCCESS ? stdout : stderr);
+	exit(status);
+}
+
+static long read_num(int idx, int argidx, int argc, char **argv)
+{
+	char *end;
+	long num;
+
+	if (argidx >= argc) {
+		fprintf(stderr, "%s: missing argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+
+	num = strtol(argv[argidx], &end, 0);
+
+	if (!end || *end!='\0') {
+		fprintf(stderr, "%s: expected integer argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+	return num;
+}
+
+static void process_options(int argc, char **argv)
+{
+	int i;
+
+	for (i=1; i<argc; ++i) {
+		if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
+			usage(EXIT_SUCCESS);
+		} else if (!strcmp(argv[i], "--peb") || !strcmp(argv[i], "-b")) {
+			if (peb >= 0)
+				goto failmulti;
+			peb = read_num(i, i+1, argc, argv);
+			if (peb < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--count") || !strcmp(argv[i], "-c")) {
+			if (count > 0)
+				goto failmulti;
+			count = read_num(i, i+1, argc, argv);
+			if (count <= 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--skip") || !strcmp(argv[i], "-s")) {
+			if (skip >= 0)
+				goto failmulti;
+			skip = read_num(i, i+1, argc, argv);
+			if (skip < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i],"--destructive")||!strcmp(argv[i],"-d")) {
+			if (flags & DESTRUCTIVE)
+				goto failmulti;
+			flags |= DESTRUCTIVE;
+		} else {
+			if (mtddev)
+				usage(EXIT_FAILURE);
+			mtddev = argv[i];
+		}
+	}
+
+	if (!mtddev)
+		errmsg_die("No device specified!\n");
+
+	if (peb < 0)
+		peb = 0;
+	if (skip < 0)
+		skip = 0;
+	if (count < 0)
+		count = 1;
+	return;
+failmulti:
+	fprintf(stderr, "'%s' specified more than once!\n", argv[i]);
+	exit(EXIT_FAILURE);
+failarg:
+	fprintf(stderr, "Invalid argument for '%s'!\n", argv[i]);
+	exit(EXIT_FAILURE);
+}
+
+static int write_eraseblock(int ebnum)
+{
+	int err = mtd_write(mtd_desc, &mtd, fd, ebnum, 0,
+				iobuf, mtd.eb_size, NULL, 0, 0);
+	if (err)
+		fprintf(stderr, "Error writing block %d!\n", ebnum);
+	return err;
+}
+
+static int read_eraseblock(int ebnum)
+{
+	int err = mtd_read(&mtd, fd, ebnum, 0, iobuf, mtd.eb_size);
+	if (err)
+		fprintf(stderr, "Error writing block %d!\n", ebnum);
+	return err;
+}
+
+static int write_eraseblock_by_page(int ebnum)
+{
+	void *buf = iobuf;
+	int i, err = 0;
+
+	for (i = 0; i < pgcnt; ++i) {
+		err = mtd_write(mtd_desc, &mtd, fd, ebnum, i * pgsize,
+						buf, pgsize, NULL, 0, 0);
+		if (err) {
+			fprintf(stderr, "Error writing block %d, page %d!\n",
+					ebnum, i);
+			break;
+		}
+		buf += pgsize;
+	}
+
+	return err;
+}
+
+static int write_eraseblock_by_2pages(int ebnum)
+{
+	int i, n = pgcnt / 2, err = 0;
+	size_t sz = pgsize * 2;
+	void *buf = iobuf;
+
+	for (i = 0; i < n; ++i) {
+		err = mtd_write(mtd_desc, &mtd, fd, ebnum, i * sz,
+						buf, sz, NULL, 0, 0);
+		if (err) {
+			fprintf(stderr, "Error writing block %d, page %d + %d!\n",
+					ebnum, i*2, i*2+1);
+			return err;
+		}
+		buf += sz;
+	}
+	if (pgcnt % 2) {
+		err = mtd_write(mtd_desc, &mtd, fd, ebnum, i * sz,
+						buf, pgsize, NULL, 0, 0);
+		if (err) {
+			fprintf(stderr, "Error reading block %d, page %d!\n",
+					ebnum, i*2);
+		}
+	}
+	return err;
+}
+
+static int read_eraseblock_by_page(int ebnum)
+{
+	void *buf = iobuf;
+	int i, err = 0;
+
+	for (i = 0; i < pgcnt; ++i) {
+		err = mtd_read(&mtd, fd, ebnum, i * pgsize, iobuf, pgsize);
+		if (err) {
+			fprintf(stderr, "Error reading block %d, page %d!\n",
+					ebnum, i);
+			break;
+		}
+		buf += pgsize;
+	}
+
+	return err;
+}
+
+static int read_eraseblock_by_2pages(int ebnum)
+{
+	int i, n = pgcnt / 2, err = 0;
+	size_t sz = pgsize * 2;
+	void *buf = iobuf;
+
+	for (i = 0; i < n; ++i) {
+		err = mtd_read(&mtd, fd, ebnum, i * sz, iobuf, sz);
+		if (err) {
+			fprintf(stderr, "Error reading block %d, page %d + %d!\n",
+					ebnum, i*2, i*2+1);
+			return err;
+		}
+		buf += sz;
+	}
+	if (pgcnt % 2) {
+		err = mtd_read(&mtd, fd, ebnum, i * sz, iobuf, pgsize);
+		if (err) {
+			fprintf(stderr, "Error reading block %d, page %d!\n",
+					ebnum, i*2);
+		}
+	}
+
+	return err;
+}
+
+static void start_timing(void)
+{
+	clock_gettime(CLOCK_MONOTONIC_RAW, &start);
+}
+
+static void stop_timing(void)
+{
+	clock_gettime(CLOCK_MONOTONIC_RAW, &finish);
+}
+
+static long calc_speed(void)
+{
+	long ms;
+
+	ms = (finish.tv_sec - start.tv_sec) * 1000L;
+	ms += (finish.tv_nsec - start.tv_nsec) / 1000000L;
+
+	if (ms <= 0)
+		return 0;
+
+	return ((long)goodebcnt * (mtd.eb_size / 1024L) * 1000L) / ms;
+}
+
+static void scan_for_bad_eraseblocks(unsigned int eb, int ebcnt, int ebskip)
+{
+	int i, bad = 0;
+
+	puts("scanning for bad eraseblocks");
+
+	for (i = 0; i < ebcnt; ++i) {
+		bbt[i] = mtd_is_bad(&mtd, fd, eb + i*(ebskip+1)) ? 1 : 0;
+		if (bbt[i])
+			bad += 1;
+	}
+
+	printf("scanned %d eraseblocks, %d are bad\n", ebcnt, bad);
+}
+
+static int erase_good_eraseblocks(unsigned int eb, int ebcnt, int ebskip)
+{
+	int err = 0, block;
+	unsigned int i;
+
+	for (i = 0; i < ebcnt; ++i) {
+		if (bbt[i])
+			continue;
+		block = eb + i*(ebskip+1);
+		err = mtd_erase(mtd_desc, &mtd, fd, block);
+		if (err)
+			fprintf(stderr, "Error erasing block %d!\n", block);
+	}
+
+	return err;
+}
+
+#define TIME_OP_PER_PEB( op )\
+		start_timing();\
+		for (i = 0; i < count; ++i) {\
+			if (bbt[i])\
+				continue;\
+			err = op(peb + i*(skip+1));\
+			if (err)\
+				goto out;\
+		}\
+		stop_timing();\
+		speed = calc_speed()
+
+int main(int argc, char **argv)
+{
+	int err, i, blocks, j, k, status = EXIT_FAILURE;
+	long speed;
+
+	process_options(argc, argv);
+
+	mtd_desc = libmtd_open();
+	if (!mtd_desc)
+		return errmsg("can't initialize libmtd");
+
+	if (mtd_get_dev_info(mtd_desc, mtddev, &mtd) < 0)
+		return errmsg("mtd_get_dev_info failed");
+
+	if (mtd.subpage_size == 1) {
+		puts("not NAND flash, assume page size is 512 bytes.");
+		pgsize = 512;
+	} else {
+		pgsize = mtd.subpage_size;
+	}
+
+	pgcnt = mtd.eb_size / pgsize;
+
+	if (count < 0)
+		count = mtd.eb_size;
+
+	if (peb >= mtd.eb_cnt)
+		return errmsg("Physical erase block %d is out of range!\n", peb);
+
+	if ((peb + (count - 1)*(skip + 1)) >= mtd.eb_cnt) {
+		return errmsg("Given block range exceeds block count of %d!\n",
+					mtd.eb_cnt);
+	}
+
+	iobuf = xmalloc(mtd.eb_size);
+	bbt = xzalloc(count);
+
+	if ((fd = open(mtddev, O_RDWR)) == -1) {
+		perror(mtddev);
+		goto outfree;
+	}
+
+	for (i = 0; i < mtd.eb_size; ++i)
+		iobuf[i] = rand();
+
+	scan_for_bad_eraseblocks(peb, count, skip);
+
+	for (i = 0; i < count; ++i) {
+		if (!bbt[i])
+			goodebcnt++;
+	}
+
+	/* Write all eraseblocks, 1 eraseblock at a time */
+	if (flags & DESTRUCTIVE) {
+		err = erase_good_eraseblocks(peb, count, skip);
+		if (err)
+			goto out;
+
+		puts("testing eraseblock write speed");
+		TIME_OP_PER_PEB(write_eraseblock);
+		printf("eraseblock write speed is %ld KiB/s\n", speed);
+	}
+
+	/* Read all eraseblocks, 1 eraseblock at a time */
+	puts("testing eraseblock read speed");
+	TIME_OP_PER_PEB(read_eraseblock);
+	printf("eraseblock read speed is %ld KiB/s\n", speed);
+
+	/* Write all eraseblocks, 1 page at a time */
+	if (flags & DESTRUCTIVE) {
+		err = erase_good_eraseblocks(peb, count, skip);
+		if (err)
+			goto out;
+
+		puts("testing page write speed");
+		TIME_OP_PER_PEB(write_eraseblock_by_page);
+		printf("page write speed is %ld KiB/s\n", speed);
+	}
+
+	/* Read all eraseblocks, 1 page at a time */
+	puts("testing page read speed");
+	TIME_OP_PER_PEB(read_eraseblock_by_page);
+	printf("page read speed is %ld KiB/s\n", speed);
+
+	/* Write all eraseblocks, 2 pages at a time */
+	if (flags & DESTRUCTIVE) {
+		err = erase_good_eraseblocks(peb, count, skip);
+		if (err)
+			goto out;
+
+		puts("testing 2 page write speed");
+		TIME_OP_PER_PEB(write_eraseblock_by_2pages);
+		printf("2 page write speed is %ld KiB/s\n", speed);
+	}
+
+	/* Read all eraseblocks, 2 pages at a time */
+	puts("testing 2 page read speed");
+	TIME_OP_PER_PEB(read_eraseblock_by_2pages);
+	printf("2 page read speed is %ld KiB/s\n", speed);
+
+	/* Erase all eraseblocks */
+	if (flags & DESTRUCTIVE) {
+		puts("Testing erase speed");
+		start_timing();
+		err = erase_good_eraseblocks(peb, count, skip);
+		if (err)
+			goto out;
+		stop_timing();
+		speed = calc_speed();
+		printf("erase speed is %ld KiB/s\n", speed);
+	}
+
+	/* Multi-block erase all eraseblocks */
+	if (!skip) {
+		for (k = 1; k < 7; ++k) {
+			blocks = 1 << k;
+			printf("Testing %dx multi-block erase speed\n", blocks);
+			start_timing();
+			for (i = 0; i < count; ) {
+				for (j = 0; j < blocks && (i + j) < count; ++j)
+					if (bbt[i + j])
+						break;
+				if (j < 1) {
+					++i;
+					continue;
+				}
+				err = mtd_erase_multi(mtd_desc, &mtd, fd, i, j);
+				if (err)
+					goto out;
+				i += j;
+			}
+			stop_timing();
+			speed = calc_speed();
+			printf("%dx multi-block erase speed is %ld KiB/s\n",
+					blocks, speed);
+		}
+	}
+
+	puts("finished");
+	status = EXIT_SUCCESS;
+out:
+	close(fd);
+outfree:
+	free(iobuf);
+	free(bbt);
+	return status;
+}
-- 
2.7.3

[PATCH 6/8] mtd-utils: Add nand flash bit errors test

[Richard Weinberger]

From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

Basically a user space port of the mtd speed test kernel module.
In addition to the module parameters, the utility can resture
the block contents after test and allows setting the maxium writes
for the test.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
---
 .gitignore               |   1 +
 Makefile                 |   2 +-
 nand-utils/nandbiterrs.c | 497 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 499 insertions(+), 1 deletion(-)
 create mode 100644 nand-utils/nandbiterrs.c

diff --git a/.gitignore b/.gitignore
index 6d98b3d..5f948e7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -43,6 +43,7 @@
 /nand-utils/nandwrite
 /nand-utils/nftl_format
 /nand-utils/nftldump
+/nand-utils/nandbiterrs
 /misc-utils/recv_image
 /nor-utils/rfddump
 /nor-utils/rfdformat
diff --git a/Makefile b/Makefile
index 2a7715d..2813f3a 100644
--- a/Makefile
+++ b/Makefile
@@ -29,7 +29,7 @@ UBIFS_BINS = \
 JFFSX_BINS = \
 	mkfs.jffs2 sumtool jffs2reader jffs2dump
 NAND_BINS = \
-	nanddump nandwrite nandtest nftldump nftl_format
+	nanddump nandwrite nandtest nftldump nftl_format nandbiterrs
 NOR_BINS = \
 	rfddump rfdformat
 
diff --git a/nand-utils/nandbiterrs.c b/nand-utils/nandbiterrs.c
new file mode 100644
index 0000000..c7424f5
--- /dev/null
+++ b/nand-utils/nandbiterrs.c
@@ -0,0 +1,497 @@
+/*
+ * Copyright © 2012 NetCommWireless
+ * Iwo Mergler <Iwo.Mergler@netcommwireless.com.au>
+ *
+ * Copyright © 2015 sigma star gmbh
+ * David Oberhollenzer <david.oberhollenzer@sigma-star.at>
+ *
+ * Test for multi-bit error recovery on a NAND page. This mostly tests the
+ * ECC controller / driver.
+ *
+ * There are two test modes:
+ *
+ *	0 - artificially inserting bit errors until the ECC fails
+ *	    This is the default method and fairly quick. It should
+ *	    be independent of the quality of the FLASH.
+ *
+ *	1 - re-writing the same pattern repeatedly until the ECC fails.
+ *	    This method relies on the physics of NAND FLASH to eventually
+ *	    generate '0' bits if '1' has been written sufficient times.
+ *	    Depending on the NAND, the first bit errors will appear after
+ *	    1000 or more writes and then will usually snowball, reaching the
+ *	    limits of the ECC quickly.
+ *
+ *	    The test stops after 10000 cycles, should your FLASH be
+ *	    exceptionally good and not generate bit errors before that. Try
+ *	    a different page in that case.
+ *
+ * Please note that neither of these tests will significantly 'use up' any
+ * FLASH endurance. Only a maximum of two erase operations will be performed.
+ *
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ */
+#define PROGRAM_NAME "nandbiterrs"
+
+#include <mtd/mtd-user.h>
+#include <sys/ioctl.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libmtd.h>
+#include <stdio.h>
+#include <fcntl.h>
+
+#include "common.h"
+
+/* We don't expect more than this many correctable bit errors per page. */
+#define MAXBITS 512
+
+#define KEEP_CONTENTS 0x01
+#define MODE_INCREMENTAL 0x02
+#define MODE_OVERWRITE 0x04
+
+static int peb = -1, page = -1, max_overwrite = -1, seed = -1;
+static const char *mtddev;
+
+static unsigned char *wbuffer, *rbuffer, *old_data;
+static int fd, pagesize, pagecount, flags;
+static struct mtd_dev_info mtd;
+static libmtd_t mtd_desc;
+
+static void usage(int status)
+{
+	fputs(
+	"Usage: "PROGRAM_NAME" [OPTIONS] <device>\n\n"
+	"Common options:\n"
+	"  -h, --help          Display this help output\n"
+	"  -k, --keep          Restore existing contents after test\n"
+	"  -b, --peb <num>     Use this physical erase block\n"
+	"  -p, --page <num>    Use this page within the erase block\n"
+	"  -s, --seed <num>    Specify seed for PRNG\n\n"
+	"Options controling test mode:\n"
+	"  -i, --incremental   Manually insert bit errors until ECC fails\n"
+	"  -o, --overwrite     Rewrite page until bits flip and ECC fails\n\n"
+	"Test mode specific options:\n"
+	"  -w, --writes <num>  Number of writes (default 10000)\n",
+	status==EXIT_SUCCESS ? stdout : stderr);
+	exit(status);
+}
+
+static long read_num(int idx, int argidx, int argc, char **argv)
+{
+	char *end;
+	long num;
+
+	if (argidx >= argc) {
+		fprintf(stderr, "%s: missing argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+
+	num = strtol(argv[argidx], &end, 0);
+
+	if (!end || *end!='\0') {
+		fprintf(stderr, "%s: expected integer argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+	return num;
+}
+
+static void process_options(int argc, char **argv)
+{
+	int i;
+
+	for (i=1; i<argc; ++i) {
+		if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
+			usage(EXIT_SUCCESS);
+		} else if (!strcmp(argv[i], "--keep") || !strcmp(argv[i], "-k")) {
+			if (flags & KEEP_CONTENTS)
+				goto failmulti;
+			flags |= KEEP_CONTENTS;
+		} else if (!strcmp(argv[i], "--peb") || !strcmp(argv[i], "-b")) {
+			if (peb >= 0)
+				goto failmulti;
+			peb = read_num(i, i+1, argc, argv);
+			if (peb < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i],"--incremental")||!strcmp(argv[i],"-i")) {
+			if (flags & (MODE_INCREMENTAL|MODE_OVERWRITE))
+				goto failmultimode;
+			flags |= MODE_INCREMENTAL;
+		} else if (!strcmp(argv[i],"--overwrite") || !strcmp(argv[i],"-o")) {
+			if (flags & (MODE_INCREMENTAL|MODE_OVERWRITE))
+				goto failmultimode;
+			flags |= MODE_OVERWRITE;
+		} else if (!strcmp(argv[i], "--writes") || !strcmp(argv[i], "-w")) {
+			if (max_overwrite > 0)
+				goto failmulti;
+			max_overwrite = read_num(i, i+1, argc, argv);
+			if (max_overwrite <= 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--seed") || !strcmp(argv[i], "-s")) {
+			if (seed >= 0)
+				goto failmulti;
+			seed = read_num(i, i+1, argc, argv);
+			if (seed < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--page") || !strcmp(argv[i], "-p")) {
+			if (page > 0)
+				goto failmulti;
+			page = read_num(i, i+1, argc, argv);
+			if (page < 0)
+				goto failarg;
+			++i;
+		} else {
+			if (mtddev)
+				usage(EXIT_FAILURE);
+			mtddev = argv[i];
+		}
+	}
+
+	if (!mtddev)
+		errmsg_die("No device specified!");
+
+	if (!(flags & (MODE_OVERWRITE|MODE_INCREMENTAL)))
+		errmsg_die("No test mode specified!");
+
+	if ((max_overwrite > 0) && !(flags & MODE_OVERWRITE))
+		errmsg_die("Write count specified but mode is not --overwrite!");
+
+	if (max_overwrite < 0)
+		max_overwrite = 10000;
+	if (peb < 0)
+		peb = 0;
+	if (page < 0)
+		page = 0;
+	if (seed < 0)
+		seed = 0;
+	return;
+failmultimode:
+	errmsg_die("Test mode specified more than once!");
+failmulti:
+	errmsg_die("'%s' specified more than once!", argv[i]);
+failarg:
+	errmsg_die("Invalid argument for '%s'!", argv[i]);
+}
+
+/* 'random' bytes from known offsets */
+static unsigned char hash(unsigned int offset)
+{
+	unsigned int v = offset;
+	unsigned char c;
+	v ^= 0x7f7edfd3;
+	v = v ^ (v >> 3);
+	v = v ^ (v >> 5);
+	v = v ^ (v >> 13);
+	c = v & 0xFF;
+	/* Reverse bits of result. */
+	c = (c & 0x0F) << 4 | (c & 0xF0) >> 4;
+	c = (c & 0x33) << 2 | (c & 0xCC) >> 2;
+	c = (c & 0x55) << 1 | (c & 0xAA) >> 1;
+	return c;
+}
+
+static int write_page(void)
+{
+	int err;
+
+	err = mtd_write(mtd_desc, &mtd, fd, peb, page*pagesize,
+					wbuffer, pagesize, NULL, 0, 0);
+
+	if (err)
+		fprintf(stderr, "Failed to write page %d in block %d\n", peb, page);
+
+	return err;
+}
+
+static int rewrite_page(void)
+{
+	if (ioctl(fd, MTDFILEMODE, MTD_FILE_MODE_RAW) != 0)
+		goto fail_mode;
+
+	if (write_page() != 0)
+		return -1;
+
+	if (ioctl(fd, MTDFILEMODE, MTD_FILE_MODE_NORMAL) != 0)
+		goto fail_mode;
+
+	return 0;
+fail_mode:
+	perror("MTDFILEMODE");
+	return -1;
+}
+
+static int read_page(void)
+{
+	struct mtd_ecc_stats old, new;
+	int err = 0;
+
+	if (ioctl(fd, ECCGETSTATS, &old) != 0)
+		goto failstats;
+
+	err = mtd_read(&mtd, fd, peb, page*pagesize, rbuffer, pagesize);
+	if (err) {
+		fputs("Read failed!\n", stderr);
+		return -1;
+	}
+
+	if (new.failed > old.failed) {
+		fprintf(stderr, "Failed to recover %d bitflips\n",
+				new.failed - old.failed);
+		return -1;
+	}
+
+	if (ioctl(fd, ECCGETSTATS, &new) != 0)
+		goto failstats;
+
+	return new.corrected - old.corrected;
+failstats:
+	perror("ECCGETSTATS");
+	return -1;
+}
+
+static int verify_page(void)
+{
+	unsigned int i, errs = 0;
+
+	for (i = 0; i < pagesize; ++i) {
+		if (rbuffer[i] != hash(i+seed))
+			++errs;
+	}
+
+	if (errs)
+		fputs("ECC failure, invalid data despite read success\n", stderr);
+
+	return errs;
+}
+
+/* Finds the first '1' bit in wbuffer and sets it to '0'. */
+static int insert_biterror(void)
+{
+	int bit, mask, byte;
+
+	for (byte = 0; byte < pagesize; ++byte) {
+		for (bit = 7, mask = 0x80; bit >= 0; bit--, mask>>=0) {
+			if (wbuffer[byte] & mask) {
+				wbuffer[byte] &= ~mask;
+				printf("Inserted biterror @ %u/%u\n", byte, bit);
+				return 0;
+			}
+		}
+		++byte;
+	}
+	fputs("biterror: Failed to find a '1' bit\n", stderr);
+	return -1;
+}
+
+/* Writes 'random' data to page and then introduces deliberate bit
+ * errors into the page, while verifying each step. */
+static int incremental_errors_test(void)
+{
+	unsigned int i, errs_per_subpage = 0;
+	int count = 0;
+
+	puts("incremental biterrors test");
+
+	for (i = 0; i < pagesize; ++i)
+		wbuffer[i] = hash(i+seed);
+
+	if (write_page() != 0)
+		return -1;
+
+	for (errs_per_subpage = 0; ; ++errs_per_subpage) {
+		if (rewrite_page() != 0)
+			return -1;
+
+		count = read_page();
+		if (count > 0)
+			printf("Read reported %d corrected bit errors\n", count);
+		if (count < 0) {
+			fprintf(stderr, "Read error after %d bit errors per page\n",
+				errs_per_subpage);
+			return 0;
+		}
+
+		if (verify_page() != 0)
+			return -1;
+
+		printf("Successfully corrected %d bit errors per subpage\n",
+			errs_per_subpage);
+
+		if (insert_biterror() != 0)
+			return -1;
+	}
+
+	return 0;
+}
+
+
+/* Writes 'random' data to page and then re-writes that same data repeatedly.
+   This eventually develops bit errors (bits written as '1' will slowly become
+   '0'), which are corrected as far as the ECC is capable of. */
+static int overwrite_test(void)
+{
+	unsigned int i, max_corrected = 0, opno;
+	unsigned int bitstats[MAXBITS]; /* bit error histogram. */
+	int err = 0;
+
+	memset(bitstats, 0, sizeof(bitstats));
+
+	puts("overwrite biterrors test");
+
+	for (i = 0; i < pagesize; ++i)
+		wbuffer[i] = hash(i+seed);
+
+	if (write_page() != 0)
+		return -1;
+
+	for (opno = 0; opno < max_overwrite; ++opno) {
+		err = rewrite_page();
+		if (err)
+			break;
+
+		err = read_page();
+		if (err >= 0) {
+			if (err >= MAXBITS) {
+				puts("Implausible number of bit errors corrected");
+				err = -1;
+				break;
+			}
+			bitstats[err]++;
+			if (err > max_corrected) {
+				max_corrected = err;
+				printf("Read reported %d corrected bit errors\n", err);
+			}
+		} else {
+			err = 0;
+			break;
+		}
+
+		err = verify_page();
+		if (err) {
+			bitstats[max_corrected] = opno;
+			break;
+		}
+	}
+
+	/* At this point bitstats[0] contains the number of ops with no bit
+	 * errors, bitstats[1] the number of ops with 1 bit error, etc. */
+	printf("Bit error histogram (%d operations total):\n", opno);
+	for (i = 0; i < max_corrected; ++i) {
+		printf("Page reads with %3d corrected bit errors: %d\n",
+			i, bitstats[i]);
+	}
+	return err;
+}
+
+int main(int argc, char **argv)
+{
+	int err = 0, status = EXIT_FAILURE;
+
+	process_options(argc, argv);
+
+	mtd_desc = libmtd_open();
+	if (!mtd_desc)
+		return errmsg("can't initialize libmtd");
+
+	if (mtd_get_dev_info(mtd_desc, mtddev, &mtd) < 0)
+		return errmsg("mtd_get_dev_info failed");
+
+	if (mtd.type!=MTD_MLCNANDFLASH && mtd.type!=MTD_NANDFLASH)
+		return errmsg("%s is not a NAND flash!", mtddev);
+
+	pagesize = mtd.subpage_size;
+	pagecount = mtd.eb_size / pagesize;
+
+	if (peb >= mtd.eb_cnt)
+		return errmsg("Physical erase block %d is out of range!", peb);
+	if (page >= pagecount)
+		return errmsg("Page number %d is out of range!", page);
+	if ((fd = open(mtddev, O_RDWR)) == -1) {
+		perror(mtddev);
+		return EXIT_FAILURE;
+	}
+
+	if (flags & KEEP_CONTENTS) {
+		old_data = malloc(mtd.eb_size);
+		if (!old_data) {
+			perror(NULL);
+			goto fail_dev;
+		}
+		if (mtd_read(&mtd, fd, peb, 0, old_data, mtd.eb_size)) {
+			fprintf(stderr, "Reading erase block %d failed!\n", peb);
+			goto fail_dev;
+		}
+	}
+
+	wbuffer = malloc(pagesize);
+	if (!wbuffer) {
+		perror(NULL);
+		goto fail_dev;
+	}
+
+	rbuffer = malloc(pagesize);
+	if (!rbuffer) {
+		perror(NULL);
+		goto fail_rbuffer;
+	}
+
+	if (mtd_erase(mtd_desc, &mtd, fd, peb)) {
+		fprintf(stderr, "Cannot erase block %d\n", peb);
+		goto fail_test;
+	}
+
+	if (flags & MODE_INCREMENTAL)
+		err = incremental_errors_test();
+	else if (flags & MODE_OVERWRITE)
+		err = overwrite_test();
+
+	status = err ? EXIT_FAILURE : EXIT_SUCCESS;
+
+	if (flags & KEEP_CONTENTS) {
+		if (mtd_erase(mtd_desc, &mtd, fd, peb)) {
+			fprintf(stderr, "Restoring: Cannot erase block %d\n", peb);
+			status = EXIT_FAILURE;
+			goto fail_test;
+		}
+
+		err = mtd_write(mtd_desc, &mtd, fd, peb, 0,
+					old_data, mtd.eb_size, NULL, 0, 0);
+
+		if (err) {
+			fputs("Failed restoring old block contents!\n", stderr);
+			status = EXIT_FAILURE;
+		}
+	} else {
+		/* We leave the block un-erased in case of test failure. */
+		if (err)
+			goto fail_test;
+
+		if (mtd_erase(mtd_desc, &mtd, fd, peb)) {
+			fprintf(stderr, "Cannot erase block %d\n", peb);
+			status = EXIT_FAILURE;
+		}
+	}
+fail_test:
+	free(rbuffer);
+fail_rbuffer:
+	free(wbuffer);
+fail_dev:
+	close(fd);
+	free(old_data);
+	return status;
+}
-- 
2.7.3

[PATCH 7/8] mtd-utils: Add flash read test utility

[Richard Weinberger]

From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

Basically a user space port of the mtd read test kernel module.
In addition to the module parameters, the utility can scan only
a sub-range of the flash erase block with a configurable stride.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
---
 .gitignore                  |   1 +
 Makefile                    |   3 +-
 misc-utils/flash_readtest.c | 263 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 266 insertions(+), 1 deletion(-)
 create mode 100644 misc-utils/flash_readtest.c

diff --git a/.gitignore b/.gitignore
index 5f948e7..8e240d5 100644
--- a/.gitignore
+++ b/.gitignore
@@ -30,6 +30,7 @@
 /misc-utils/flashcp
 /misc-utils/ftl_check
 /misc-utils/ftl_format
+/misc-utils/flash_readtest
 /jffsX-utils/jffs2dump
 /jffsX-utils/jffs2reader
 /jffsX-utils/mkfs.jffs2
diff --git a/Makefile b/Makefile
index 2813f3a..add8864 100644
--- a/Makefile
+++ b/Makefile
@@ -20,7 +20,8 @@ MISC_BINS = \
 	ftl_format doc_loadbios ftl_check mtd_debug docfdisk \
 	serve_image recv_image mtdpart flash_erase flash_lock \
 	flash_unlock flash_otp_info flash_otp_dump flash_otp_lock \
-	flash_otp_write flashcp flash_torture flash_stress flash_speed
+	flash_otp_write flashcp flash_torture flash_stress flash_speed \
+	flash_readtest
 UBI_BINS = \
 	ubiupdatevol ubimkvol ubirmvol ubicrc32 ubinfo ubiattach \
 	ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock
diff --git a/misc-utils/flash_readtest.c b/misc-utils/flash_readtest.c
new file mode 100644
index 0000000..9c15f67
--- /dev/null
+++ b/misc-utils/flash_readtest.c
@@ -0,0 +1,263 @@
+/*
+ * Copyright (C) 2006-2008 Nokia Corporation
+ * Copyright (C) 2015 sigma star gmbh
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Check MTD device read.
+ *
+ * Author: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
+ *
+ * Based on linux readtest.c
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ */
+#define PROGRAM_NAME "flash_readtest"
+
+#include <mtd/mtd-user.h>
+#include <sys/ioctl.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include <libmtd.h>
+#include <stdio.h>
+#include <fcntl.h>
+#include <time.h>
+
+#include "common.h"
+
+#define FLAG_VERBOSE 1
+
+static int peb=-1, skip=-1, count=-1, flags=0, pgcnt, pgsize, fd;
+static unsigned char *iobuf, *iobuf1;
+static struct mtd_dev_info mtd;
+static const char *mtddev;
+static libmtd_t mtd_desc;
+
+static void usage(int status)
+{
+	fputs(
+	"Usage: "PROGRAM_NAME" [OPTIONS] <device>\n\n"
+	"Common options:\n"
+	"  -h, --help          Display this help output\n"
+	"  -b, --peb <num>     Start from this physical erase block\n"
+	"  -c, --count <num>   Number of erase blocks to process (default: all)\n"
+	"  -s, --skip <num>    Number of blocks to skip\n"
+	"  -v, --verbose       Generate more verbose output\n\n",
+	status==EXIT_SUCCESS ? stdout : stderr);
+	exit(status);
+}
+
+static long read_num(int idx, int argidx, int argc, char **argv)
+{
+	char *end;
+	long num;
+
+	if (argidx >= argc) {
+		fprintf(stderr, "%s: missing argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+
+	num = strtol(argv[argidx], &end, 0);
+
+	if (!end || *end!='\0') {
+		fprintf(stderr, "%s: expected integer argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+	return num;
+}
+
+static void process_options(int argc, char **argv)
+{
+	int i;
+
+	for (i=1; i<argc; ++i) {
+		if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
+			usage(EXIT_SUCCESS);
+		} else if (!strcmp(argv[i], "--verbose") || !strcmp(argv[i], "-v")) {
+			if (flags & FLAG_VERBOSE)
+				goto failmulti;
+			flags |= FLAG_VERBOSE;
+		} else if (!strcmp(argv[i], "--peb") || !strcmp(argv[i], "-b")) {
+			if (peb >= 0)
+				goto failmulti;
+			peb = read_num(i, i+1, argc, argv);
+			if (peb < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--count") || !strcmp(argv[i], "-c")) {
+			if (count >= 0)
+				goto failmulti;
+			count = read_num(i, i+1, argc, argv);
+			if (count < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--skip") || !strcmp(argv[i], "-s")) {
+			if (skip >= 0)
+				goto failmulti;
+			skip = read_num(i, i+1, argc, argv);
+			if (skip < 0)
+				goto failarg;
+			++i;
+		} else {
+			if (mtddev)
+				usage(EXIT_FAILURE);
+			mtddev = argv[i];
+		}
+	}
+
+	if (!mtddev)
+		errmsg_die("No device specified!");
+
+	if (peb < 0)
+		peb = 0;
+	if (skip < 0)
+		skip = 0;
+	return;
+failmulti:
+	errmsg_die("'%s' specified more than once!", argv[i]);
+failarg:
+	errmsg_die("Invalid argument for '%s'!", argv[i]);
+}
+
+static int read_eraseblock_by_page(int ebnum)
+{
+	unsigned char *buf = iobuf, *oobbuf = iobuf1;
+	uint64_t addr = ((uint64_t)ebnum) * ((uint64_t)mtd.eb_size);
+	int i, ret;
+
+	for (i = 0; i < pgcnt; ++i) {
+		memset(buf, 0, pgsize);
+		ret = mtd_read(&mtd, fd, ebnum, i*pgsize, buf, pgsize);
+		if (ret) {
+			fprintf(stderr, "Error reading block %d, page %d\n", ebnum, i);
+			return -1;
+		}
+		if (mtd.oob_size) {
+			ret = mtd_read_oob(mtd_desc, &mtd, fd,
+							addr, mtd.oob_size, oobbuf);
+
+			if (ret) {
+				fprintf(stderr, "Error reading OOB in block %d, page %d\n",
+						ebnum, i);
+				return -1;
+			}
+			oobbuf += mtd.oob_size;
+		}
+		buf += pgsize;
+		addr += pgsize;
+	}
+
+	if (flags & FLAG_VERBOSE)
+		printf("Successfully read erase block %d\n", ebnum);
+
+	return 0;
+}
+
+static void dump_eraseblock(int ebnum)
+{
+	char line[128];
+	int i, j, n;
+	int pg, oob;
+
+	printf("dumping eraseblock %d\n", ebnum);
+	n = mtd.eb_size;
+	for (i = 0; i < n;) {
+		char *p = line;
+
+		p += sprintf(p, "%05x: ", i);
+		for (j = 0; j < 32 && i < n; j++, i++)
+			p += sprintf(p, "%02x", (unsigned int)iobuf[i]);
+		printf("%s\n", line);
+	}
+	if (!mtd.oob_size)
+		return;
+	printf("dumping oob from eraseblock %d\n", ebnum);
+	n = mtd.oob_size;
+	for (pg = 0, i = 0; pg < pgcnt; ++pg) {
+		for (oob = 0; oob < n;) {
+			char *p = line;
+
+			p += sprintf(p, "%05x: ", i);
+			for (j = 0; j < 32 && oob < n; ++j, ++oob, ++i)
+				p += sprintf(p, "%02x", (unsigned int)iobuf1[i]);
+			printf("%s\n", line);
+		}
+	}
+	putchar('\n');
+}
+
+int main(int argc, char **argv)
+{
+	int status = EXIT_SUCCESS, i, ret, blk;
+
+	process_options(argc, argv);
+
+	mtd_desc = libmtd_open();
+	if (!mtd_desc)
+		return errmsg("can't initialize libmtd");
+
+	if (mtd_get_dev_info(mtd_desc, mtddev, &mtd) < 0)
+		return errmsg("mtd_get_dev_info failed");
+
+	if (mtd.subpage_size == 1) {
+		puts("not NAND flash, assume page size is 512 bytes.");
+		pgsize = 512;
+	} else {
+		pgsize = mtd.subpage_size;
+	}
+
+	pgcnt = mtd.eb_size / pgsize;
+
+	if (count < 0)
+		count = mtd.eb_cnt;
+
+	if (peb >= mtd.eb_cnt)
+		return errmsg("Physical erase block %d is out of range!\n", peb);
+
+	if ((peb + (count - 1)*(skip + 1)) >= mtd.eb_cnt) {
+		return errmsg("Given block range exceeds block count of %d!\n",
+					mtd.eb_cnt);
+	}
+
+	iobuf = xmalloc(mtd.eb_size);
+	iobuf1 = xmalloc(mtd.eb_size);
+
+	if ((fd = open(mtddev, O_RDWR)) == -1) {
+		perror(mtddev);
+		status = EXIT_FAILURE;
+		goto out;
+	}
+
+	/* Read all eraseblocks 1 page at a time */
+	puts("testing page read");
+
+	for (i = 0; i < count; ++i) {
+		blk = peb + i*(skip+1);
+
+		if (mtd_is_bad(&mtd, fd, blk)) {
+			printf("Skipping bad block %d\n", blk);
+			continue;
+		}
+		ret = read_eraseblock_by_page(blk);
+		if (ret && (flags & FLAG_VERBOSE)) {
+			dump_eraseblock(blk);
+			status = EXIT_FAILURE;
+		}
+	}
+out:
+	free(iobuf);
+	free(iobuf1);
+	return status;
+}
+
-- 
2.7.3

[PATCH 8/8] mtd-utils: Add nand page test utility

[Richard Weinberger]

From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>

Basically a user space port of the mtd page test kernel module.
In addition to the module parameters, the utility supports using
only a sub-range of the flash erase blocks with a configurable stride.

Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
Signed-off-by: Richard Weinberger <richard@nod.at>
---
 .gitignore                |   1 +
 Makefile                  |   2 +-
 nand-utils/nandpagetest.c | 579 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 581 insertions(+), 1 deletion(-)
 create mode 100644 nand-utils/nandpagetest.c

diff --git a/.gitignore b/.gitignore
index 8e240d5..ff25458 100644
--- a/.gitignore
+++ b/.gitignore
@@ -45,6 +45,7 @@
 /nand-utils/nftl_format
 /nand-utils/nftldump
 /nand-utils/nandbiterrs
+/nand-utils/nandpagetest
 /misc-utils/recv_image
 /nor-utils/rfddump
 /nor-utils/rfdformat
diff --git a/Makefile b/Makefile
index add8864..76491ce 100644
--- a/Makefile
+++ b/Makefile
@@ -30,7 +30,7 @@ UBIFS_BINS = \
 JFFSX_BINS = \
 	mkfs.jffs2 sumtool jffs2reader jffs2dump
 NAND_BINS = \
-	nanddump nandwrite nandtest nftldump nftl_format nandbiterrs
+	nanddump nandwrite nandtest nftldump nftl_format nandbiterrs nandpagetest
 NOR_BINS = \
 	rfddump rfdformat
 
diff --git a/nand-utils/nandpagetest.c b/nand-utils/nandpagetest.c
new file mode 100644
index 0000000..11e2eef
--- /dev/null
+++ b/nand-utils/nandpagetest.c
@@ -0,0 +1,579 @@
+/*
+ * Copyright (C) 2006-2008 Nokia Corporation
+ * Copyright (C) 2016 sigma star gmbh
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; see the file COPYING. If not, write to the Free Software
+ * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Test page read and write on MTD device.
+ *
+ * Author: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
+ *
+ * Based on linux pagetest.c
+ * Author: Adrian Hunter <ext-adrian.hunter@nokia.com>
+ */
+#define PROGRAM_NAME "nandpagetest"
+
+#include <mtd/mtd-user.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <libmtd.h>
+#include <stdio.h>
+#include <fcntl.h>
+#include <time.h>
+
+#include "common.h"
+
+#define KEEP_CONTENTS 0x01
+#define SEED_SET 0x02
+
+static struct mtd_dev_info mtd;
+static const char *mtddev;
+static libmtd_t mtd_desc;
+
+static unsigned char *bbt=NULL, *writebuf=NULL, *backup=NULL;
+static unsigned char *twopages=NULL, *boundary=NULL;
+static int peb = -1, seed = -1, skip = -1, ebcnt = -1, flags = 0;
+static int fd, bufsize, pgsize, pgcnt;
+static unsigned int rnd_state;
+
+static void usage(int status)
+{
+	fputs(
+	"Usage: "PROGRAM_NAME" [OPTIONS] <device>\n\n"
+	"Options:\n"
+	"  -h, --help         Display this help output\n"
+	"  -p, --peb <num>    Index of the first erase block to use\n"
+	"  -c, --count <num>  Number of erase blocks to user (default all)\n"
+	"  -s, --skip <num>   Number of erase blocks to skip\n"
+	"  -S, --seed <num>   Seed for pseudor random number generator\n"
+	"  -k, --keep         Restore existing contents after test\n",
+	status==EXIT_SUCCESS ? stdout : stderr);
+	exit(status);
+}
+
+static long read_num(int idx, int argidx, int argc, char **argv)
+{
+	char *end;
+	long num;
+
+	if (argidx >= argc) {
+		fprintf(stderr, "%s: missing argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+
+	num = strtol(argv[argidx], &end, 0);
+
+	if (!end || *end!='\0') {
+		fprintf(stderr, "%s: expected integer argument\n", argv[idx]);
+		exit(EXIT_FAILURE);
+	}
+	return num;
+}
+
+static void process_options(int argc, char **argv)
+{
+	int i;
+
+	for (i=1; i<argc; ++i) {
+		if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
+			usage(EXIT_SUCCESS);
+		} else if (!strcmp(argv[i], "--peb") || !strcmp(argv[i], "-b")) {
+			if (peb >= 0)
+				goto failmulti;
+			peb = read_num(i, i+1, argc, argv);
+			if (peb < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--count") || !strcmp(argv[i], "-c")) {
+			if (ebcnt >= 0)
+				goto failmulti;
+			ebcnt = read_num(i, i+1, argc, argv);
+			if (ebcnt < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--skip") || !strcmp(argv[i], "-s")) {
+			if (skip >= 0)
+				goto failmulti;
+			skip = read_num(i, i+1, argc, argv);
+			if (skip < 0)
+				goto failarg;
+			++i;
+		} else if (!strcmp(argv[i], "--seed") || !strcmp(argv[i], "-S")) {
+			if (flags & SEED_SET)
+				goto failmulti;
+			seed = read_num(i, i+1, argc, argv);
+			flags |= SEED_SET;
+			++i;
+		} else if (!strcmp(argv[i], "--keep") || !strcmp(argv[i], "-k")) {
+			if (flags & KEEP_CONTENTS)
+				goto failmulti;
+			flags |= KEEP_CONTENTS;
+		} else {
+			if (mtddev)
+				usage(EXIT_FAILURE);
+			mtddev = argv[i];
+		}
+	}
+
+	if (!mtddev)
+		errmsg_die("No device specified!");
+
+	if (peb < 0)
+		peb = 0;
+	if (!(flags & SEED_SET))
+		seed = time(NULL);
+	if (skip < 0)
+		skip = 0;
+	return;
+failmulti:
+	errmsg_die("'%s' specified more than once!", argv[i]);
+failarg:
+	errmsg_die("Invalid argument for '%s'!", argv[i]);
+}
+
+static int write_eraseblock(int ebnum)
+{
+	int i;
+
+	for (i = 0; i < mtd.eb_size; ++i)
+		writebuf[i] = rand_r(&rnd_state);
+
+	return mtd_write(mtd_desc, &mtd, fd, ebnum, 0,
+				writebuf, mtd.eb_size, NULL, 0, 0);
+}
+
+static void get_first_and_last_block(int *first, int *last)
+{
+	int i;
+
+	*first = peb;
+	for (i = 0; i < ebcnt && bbt[i]; ++i)
+		*first += skip + 1;
+
+	*last = peb + (ebcnt - 1) * (skip + 1);
+	for (i = 0; i < ebcnt && bbt[ebcnt - i - 1]; ++i)
+		*last -= skip + 1;
+}
+
+/* Do a read to set the internal dataRAMs to different data */
+static int flush_data_rams(int eb0, int ebn)
+{
+	int err;
+	err = mtd_read(&mtd, fd, eb0, 0, twopages, bufsize);
+	if (err)
+		return err;
+	err = mtd_read(&mtd, fd, ebn, mtd.eb_size-bufsize,
+					twopages, bufsize);
+	if (err)
+		return err;
+	memset(twopages, 0, bufsize);
+	return 0;
+}
+
+static int verify_eraseblock(int ebnum)
+{
+	int err = 0, i, ret = 0, eb0, ebn, rd, diff;
+	loff_t  offset = 0, addr, addrn;
+	unsigned int j, old_state;
+
+	for (i = 0; i < mtd.eb_size; ++i)
+		writebuf[i] = rand_r(&rnd_state);
+
+	get_first_and_last_block(&eb0, &ebn);
+
+	for (j = 0; j < pgcnt - 1; ++j, offset += pgsize) {
+		err = flush_data_rams(eb0, ebn);
+		if (err)
+			return err;
+		err = mtd_read(&mtd, fd, ebnum, offset, twopages, bufsize);
+		if (err)
+			break;
+		if (memcmp(twopages, writebuf + (j * pgsize), bufsize)) {
+			fprintf(stderr, "error: verify failed at block %d, page %ld\n",
+					ebnum, ((long)offset) / pgsize );
+			ret = -1;
+		}
+	}
+	/* Check boundary between eraseblocks */
+	addr = (loff_t)ebnum*mtd.eb_size + offset;
+	addrn = (loff_t)ebn*mtd.eb_size + mtd.eb_size - 2*pgsize;
+
+	if (addr <= addrn && !mtd_is_bad(&mtd, fd, ebnum+1)) {
+		old_state = rnd_state;
+		err = flush_data_rams(eb0, ebn);
+		if (err)
+			return err;
+
+		if (lseek(fd, addr, SEEK_SET) != addr) {
+			fprintf(stderr, "cannot seek mtd%d to offset %"PRIdoff_t,
+	 				mtd.mtd_num, addr);
+			return -1;
+		}
+
+		for (rd = 0; rd < bufsize; rd += diff) {
+			diff = read(fd, twopages + rd, bufsize - rd);
+			if (diff < 0) {
+				fprintf(stderr, "cannot read %d bytes from mtd%d "
+								"(eraseblock %d, offset %d)",
+								bufsize-rd, mtd.mtd_num, ebnum,
+								(int)offset+rd);
+				return -1;
+		  	}
+		}
+
+		memcpy(boundary, writebuf + mtd.eb_size - pgsize, pgsize);
+
+		for (j = 0; j < pgsize; ++j)
+			(boundary + pgsize)[j] = rand_r(&rnd_state);
+
+		if (memcmp(twopages, boundary, bufsize)) {
+			fprintf(stderr, "error: verify failed at block %d, page %ld\n",
+					ebnum, ((long)offset) / pgsize );
+			ret = -1;
+		}
+		rnd_state = old_state;
+	}
+	return ret;
+}
+
+static int crosstest(void)
+{
+	unsigned char *pp1, *pp2, *pp3, *pp4;
+	int eb0, ebn, err = 0, offset;
+
+	puts("crosstest");
+	pp1 = xzalloc(pgsize * 4);
+	if (!pp1)
+		return -ENOMEM;
+	pp2 = pp1 + pgsize;
+	pp3 = pp2 + pgsize;
+	pp4 = pp3 + pgsize;
+
+	get_first_and_last_block(&eb0, &ebn);
+
+	/* Read 2nd-to-last page to pp1 */
+	err = mtd_read(&mtd, fd, ebn, mtd.eb_size - 2*pgsize, pp1, pgsize);
+	if (err)
+		goto out;
+
+	/* Read 3rd-to-last page to pp1 */
+	err = mtd_read(&mtd, fd, ebn, mtd.eb_size - 3*pgsize, pp1, pgsize);
+	if (err)
+		goto out;
+
+	/* Read first page to pp2 */
+	printf("reading page at block %d, page %d\n", eb0, 0);
+	err = mtd_read(&mtd, fd, eb0, 0, pp2, pgsize);
+	if (err)
+		goto out;
+
+	/* Read last page to pp3 */
+	offset = mtd.eb_size - pgsize;
+	printf("reading page at block %d, page %d\n", ebn, offset/pgsize);
+	err = mtd_read(&mtd, fd, ebn, offset, pp3, pgsize);
+	if (err)
+		goto out;
+
+	/* Read first page again to pp4 */
+	printf("reading page at block %d, page %d\n", eb0, 0);
+	err = mtd_read(&mtd, fd, eb0, 0, pp4, pgsize);
+	if (err)
+		goto out;
+
+	/* pp2 and pp4 should be the same */
+	printf("verifying pages read at block %d match\n", eb0);
+	if (memcmp(pp2, pp4, pgsize)) {
+		fputs("verify failed!\n", stderr);
+		err = -1;
+	} else {
+		puts("crosstest ok");
+	}
+out:
+	free(pp1);
+	return err;
+}
+
+static int erasecrosstest(void)
+{
+	unsigned char *readbuf = twopages;
+	int err = 0, i, eb0, ebn;
+
+	puts("erasecrosstest");
+
+	get_first_and_last_block(&eb0, &ebn);
+
+	printf("erasing block %d\n", eb0);
+	err = mtd_erase(mtd_desc, &mtd, fd, eb0);
+	if (err)
+		return err;
+
+	printf("writing 1st page of block %d\n", eb0);
+	for (i = 0; i < pgsize; ++i)
+		writebuf[i] = rand_r(&rnd_state);
+	strcpy((char*)writebuf, "There is no data like this!");
+	err = mtd_write(mtd_desc, &mtd, fd, eb0, 0, writebuf, pgsize, NULL, 0, 0);
+	if (err)
+		return err;
+
+	printf("reading 1st page of block %d\n", eb0);
+	memset(readbuf, 0, pgsize);
+	err = mtd_read(&mtd, fd, eb0, 0, readbuf, pgsize);
+	if (err)
+		return err;
+
+	printf("verifying 1st page of block %d\n", eb0);
+	if (memcmp(writebuf, readbuf, pgsize)) {
+		fputs("verify failed!\n", stderr);
+		return -1;
+	}
+
+	printf("erasing block %d\n", eb0);
+	err = mtd_erase(mtd_desc, &mtd, fd, eb0);
+	if (err)
+		return err;
+
+	printf("writing 1st page of block %d\n", eb0);
+	for (i = 0; i < pgsize; ++i)
+		writebuf[i] = rand_r(&rnd_state);
+	strcpy((char*)writebuf, "There is no data like this!");
+	err = mtd_write(mtd_desc, &mtd, fd, eb0, 0, writebuf, pgsize, NULL, 0, 0);
+	if (err)
+		return err;
+
+	printf("erasing block %d\n", ebn);
+	err = mtd_erase(mtd_desc, &mtd, fd, ebn);
+	if (err)
+		return err;
+
+	printf("reading 1st page of block %d\n", eb0);
+	memset(readbuf, 0, pgsize);
+	err = mtd_read(&mtd, fd, eb0, 0, readbuf, pgsize);
+	if (err)
+		return err;
+
+	printf("verifying 1st page of block %d\n", eb0);
+	if (memcmp(writebuf, readbuf, pgsize)) {
+		fputs("verify failed!\n", stderr);
+		return -1;
+	}
+
+	puts("erasecrosstest ok");
+	return 0;
+}
+
+static int erasetest(void)
+{
+	int err = 0, i, ebnum, ebn;
+
+	puts("erasetest");
+	get_first_and_last_block(&ebnum, &ebn);
+
+	printf("erasing block %d\n", ebnum);
+	err = mtd_erase(mtd_desc, &mtd, fd, ebnum);
+	if (err)
+		return err;
+
+	printf("writing 1st page of block %d\n", ebnum);
+	for (i = 0; i < pgsize; ++i)
+		writebuf[i] = rand_r(&rnd_state);
+	err = mtd_write(mtd_desc, &mtd, fd, ebnum, 0,
+					writebuf, pgsize, NULL, 0, 0);
+	if (err)
+		return err;
+
+	printf("erasing block %d\n", ebnum);
+	err = mtd_erase(mtd_desc, &mtd, fd, ebnum);
+	if (err)
+		return err;
+
+	printf("reading 1st page of block %d\n", ebnum);
+	err = mtd_read(&mtd, fd, ebnum, 0, twopages, pgsize);
+	if (err)
+		return err;
+
+	printf("verifying 1st page of block %d is all 0xff\n", ebnum);
+	for (i = 0; i < pgsize; ++i) {
+		if (twopages[i] != 0xff) {
+			fprintf(stderr, "verifying all 0xff failed at %d\n", i);
+			return -1;
+		}
+	}
+
+	puts("erasetest ok");
+	return 0;
+}
+
+int main(int argc, char **argv)
+{
+	int i, eb, err = 0, status = EXIT_FAILURE;
+	unsigned char *backupptr;
+
+	process_options(argc, argv);
+
+	mtd_desc = libmtd_open();
+	if (!mtd_desc)
+		return errmsg("can't initialize libmtd");
+
+	if (mtd_get_dev_info(mtd_desc, mtddev, &mtd) < 0)
+		return errmsg("mtd_get_dev_info failed");
+
+	if (mtd.type!=MTD_MLCNANDFLASH && mtd.type!=MTD_NANDFLASH)
+		return errmsg("%s is not a NAND flash!", mtddev);
+
+	pgsize = mtd.min_io_size;
+	pgcnt = mtd.eb_size / pgsize;
+	bufsize = pgsize * 2;
+
+	if (ebcnt < 0)
+		ebcnt = (mtd.eb_cnt - peb) / (skip + 1);
+
+	if (peb >= mtd.eb_cnt)
+		return errmsg("physical erase block %d is out of range!", peb);
+
+	eb = peb + (ebcnt - 1)*(skip + 1);
+
+	if (eb >= mtd.eb_cnt)
+		return errmsg("last physical erase block %d is out of range!", eb);
+
+	writebuf = xmalloc(mtd.eb_size);
+	twopages = xmalloc(bufsize);
+	boundary = xmalloc(bufsize);
+	bbt = xzalloc(ebcnt);
+
+	if ((fd = open(mtddev, O_RDWR)) == -1) {
+		perror(mtddev);
+		goto out_cleanup;
+	}
+
+	/* find bad blocks */
+	for (i = 0; i < ebcnt; ++i) {
+		eb = peb + i*(skip+1);
+		bbt[i] = mtd_is_bad(&mtd, fd, eb);
+
+		if (bbt[i])
+			printf("ignoring bad erase block %d\n", eb);
+	}
+
+	/* create block backup */
+	if (flags & KEEP_CONTENTS) {
+		eb = 0;
+		for (i = 0; i < ebcnt; ++i) {
+			if (!bbt[i])
+				++eb;
+		}
+		backup = malloc(mtd.eb_size * eb);
+		if (!backup) {
+			fprintf(stderr, "not enough memory to keep block contents!\n");
+			goto out_cleanup;
+		}
+		printf("reading %d blocks int memory\n", eb);
+		backupptr = backup;
+		for (i = 0; i < ebcnt; ++i) {
+			if (bbt[i])
+				continue;
+			eb = peb + i*(skip+1);
+			err = mtd_read(&mtd, fd, eb, 0, backupptr, mtd.eb_size);
+			if (err) {
+				fprintf(stderr, "error reading block %d!\n", eb);
+				goto out_cleanup;
+			}
+			backupptr += mtd.eb_size;
+		}
+	}
+
+	/* Erase all eraseblocks */
+	puts("erasing all blocks");
+	for (i = 0; i < ebcnt; ++i) {
+		if (bbt[i])
+			continue;
+		eb = peb + i*(skip+1);
+		if (mtd_erase(mtd_desc, &mtd, fd, eb)) {
+			fprintf(stderr, "error erasing block %d\n", eb);
+			goto out;
+		}
+	}
+	printf("erased %u eraseblocks\n", ebcnt);
+
+	/* Write all eraseblocks */
+	rnd_state = seed;
+	puts("writing all blocks");
+	for (i = 0; i < ebcnt; ++i) {
+		if (bbt[i])
+			continue;
+		eb = peb + i*(skip+1);
+		err = write_eraseblock(eb);
+		if (err)
+			goto out;
+		if (i % 256 == 0)
+			printf("written up to eraseblock %u\n", i);
+	}
+	printf("written %u eraseblocks\n", i);
+
+	/* Check all eraseblocks */
+	rnd_state = seed;
+	puts("verifying all eraseblocks");
+	for (i = 0; i < ebcnt; ++i) {
+		eb = peb + i*(skip+1);
+		if (bbt[i])
+			continue;
+		err = verify_eraseblock(eb);
+		if (err)
+			goto out;
+		if (i % 256 == 0)
+			printf("verified up to eraseblock %u\n", i);
+	}
+	printf("verified %u eraseblocks\n", i);
+
+	if (crosstest())
+		goto out;
+
+	if (erasecrosstest())
+		goto out;
+
+	if (erasetest())
+		goto out;
+
+	status = EXIT_SUCCESS;
+out:
+	/* restore block backup */
+	if (flags & KEEP_CONTENTS) {
+		puts("restoring original contents");
+		backupptr = backup;
+		for (i = 0; i < ebcnt; ++i) {
+			if (bbt[i])
+				continue;
+			eb = peb + i*(skip+1);
+			if (mtd_erase(mtd_desc, &mtd, fd, eb)) {
+				fprintf(stderr, "error erasing block %d!\n", eb);
+				status = EXIT_FAILURE;
+			}
+			err = mtd_write(mtd_desc, &mtd, fd, eb, 0,
+							backupptr, mtd.eb_size, NULL, 0, 0);
+			if (err) {
+				fprintf(stderr, "error restoring block %d!\n", eb);
+				status = EXIT_FAILURE;
+			}
+			backupptr += mtd.eb_size;
+		}
+	}
+out_cleanup:
+	free(bbt);
+	free(boundary);
+	free(twopages);
+	free(writebuf);
+	free(backup);
+	close(fd);
+	return status;
+}
+
-- 
2.7.3

Re: [RFC] Porting kernel MTD tests to user space

[Ezequiel Garcia]

On 25 April 2016 at 19:13, Richard Weinberger <richard@nod.at> wrote:
> David was so kind and did an initial port of Linux's MTD tests to user space.
> He took most tests as-is and massaged them to work in user space using libmtd.
> In the long run the goal is giving up the kernel tests as much as possible
> and improve the tests present in mtd-utils.
>
> Feedback is very welcome!
>

I think this is great, and it will be great to test NAND/NOR devices easier.

The autoconf patch looked also great, but for some reason it
stalled, so... what do we need to do to prevent this one from stalling as well?
-- 
Ezequiel García, VanguardiaSur
www.vanguardiasur.com.ar

Re: [RFC] Porting kernel MTD tests to user space

[Artem Bityutskiy]

On Tue, 2016-04-26 at 00:13 +0200, Richard Weinberger wrote:
> David was so kind and did an initial port of Linux's MTD tests to
> user space.
> He took most tests as-is and massaged them to work in user space
> using libmtd.
> In the long run the goal is giving up the kernel tests as much as
> possible
> and improve the tests present in mtd-utils.
> 
> Feedback is very welcome!

I would consider moving them, not just porting. Indeed, we probably do
not want to have the same code in 2 places, or is there a strong reason
why would we want that?

Artem.

Re: [RFC] Porting kernel MTD tests to user space

[Richard Weinberger]

Am 26.04.2016 um 07:17 schrieb Artem Bityutskiy:
> On Tue, 2016-04-26 at 00:13 +0200, Richard Weinberger wrote:
>> David was so kind and did an initial port of Linux's MTD tests to
>> user space.
>> He took most tests as-is and massaged them to work in user space
>> using libmtd.
>> In the long run the goal is giving up the kernel tests as much as
>> possible
>> and improve the tests present in mtd-utils.
>>
>> Feedback is very welcome!
> 
> I would consider moving them, not just porting. Indeed, we probably do
> not want to have the same code in 2 places, or is there a strong reason
> why would we want that?

I'd mark ported tests as obsolete but keep them to not break somebodies
setups.
But I think you are right, having two code bases is not what we want.

Brian, what is your opinion on that?

Thanks,
//richard

Re: [RFC] Porting kernel MTD tests to user space

[Richard Weinberger]

Am 26.04.2016 um 05:13 schrieb Ezequiel Garcia:
> On 25 April 2016 at 19:13, Richard Weinberger <richard@nod.at> wrote:
>> David was so kind and did an initial port of Linux's MTD tests to user space.
>> He took most tests as-is and massaged them to work in user space using libmtd.
>> In the long run the goal is giving up the kernel tests as much as possible
>> and improve the tests present in mtd-utils.
>>
>> Feedback is very welcome!
>>
> 
> I think this is great, and it will be great to test NAND/NOR devices easier.
> 
> The autoconf patch looked also great, but for some reason it
> stalled, so... what do we need to do to prevent this one from stalling as well?
> 

Please give it testing/review. :-)
Regarding my autoconf patch, IIRC nobody clearly stated whether the change
is welcomed or not.
But I can happily revive it.

Thanks,
//richard

Re: [RFC] Porting kernel MTD tests to user space

[Boris Brezillon]

Hi Richard,

On Tue, 26 Apr 2016 00:13:21 +0200
Richard Weinberger <richard@nod.at> wrote:

> David was so kind and did an initial port of Linux's MTD tests to user space.
> He took most tests as-is and massaged them to work in user space using libmtd.
> In the long run the goal is giving up the kernel tests as much as possible
> and improve the tests present in mtd-utils.
> 
> Feedback is very welcome!

Oh, nice move!
I'll try to review those patches.

BTW, I agree with Artem: we should remove in-kernel tests at some point
(maybe not immediately though).

Best Regards,

Boris

> 
> Thanks,
> //richard
> 
> [PATCH 1/8] mtd-utils: Fix return status in mtd_torture test function
> [PATCH 2/8] mtd-utils: Add multi-block erase function
> [PATCH 3/8] mtd-utils: Add flash torture test utility
> [PATCH 4/8] mtd-utils: Add flash stress test Utility
> [PATCH 5/8] mtd-utils: Add flash speed test Utility
> [PATCH 6/8] mtd-utils: Add nand flash bit errors test
> [PATCH 7/8] mtd-utils: Add flash read test utility
> [PATCH 8/8] mtd-utils: Add nand page test utility
> 
>  .gitignore                  |   6 +
>  Makefile                    |   7 +-
>  include/libmtd.h            |  14 ++
>  lib/libmtd.c                |  28 ++-
>  misc-utils/flash_readtest.c | 263 ++++++++++++++++++++
>  misc-utils/flash_speed.c    | 463 +++++++++++++++++++++++++++++++++++
>  misc-utils/flash_stress.c   | 276 +++++++++++++++++++++
>  misc-utils/flash_torture.c  | 240 ++++++++++++++++++
>  nand-utils/nandbiterrs.c    | 497 +++++++++++++++++++++++++++++++++++++
>  nand-utils/nandpagetest.c   | 579 ++++++++++++++++++++++++++++++++++++++++++++
>  10 files changed, 2362 insertions(+), 11 deletions(-)
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/



-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [PATCH 1/8] mtd-utils: Fix return status in mtd_torture test function

[Boris Brezillon]

On Tue, 26 Apr 2016 00:13:22 +0200
Richard Weinberger <richard@nod.at> wrote:

> From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> 
> This patch fixes the return status of the mtd_torture function
> in libmtd.
> 
> The torture test function is currently only used by the ubiformat
> utilitiy to check if a block is bad after a write fails (blocks are
> marked bad if the function returns an error status). However, the
> way the function was written, it ALWAYS returns an error value
> irregardless of whether it failed or not.
> 
> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> Signed-off-by: Richard Weinberger <richard@nod.at>

Reviewed-by: Boris Brezillon <boris.brezillon@free-electrons.com>

> ---
>  lib/libmtd.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/lib/libmtd.c b/lib/libmtd.c
> index bf6d71f..1717468 100644
> --- a/lib/libmtd.c
> +++ b/lib/libmtd.c
> @@ -939,7 +939,7 @@ static uint8_t patterns[] = {0xa5, 0x5a, 0x0};
>   * @patt: the pattern to check
>   * @size: buffer size in bytes
>   *
> - * This function returns %1 in there are only @patt bytes in @buf, and %0 if
> + * This function returns %0 if there are only @patt bytes in @buf, and %-1 if
>   * something else was also found.
>   */
>  static int check_pattern(const void *buf, uint8_t patt, int size)
> @@ -948,8 +948,8 @@ static int check_pattern(const void *buf, uint8_t patt, int size)
>  
>  	for (i = 0; i < size; i++)
>  		if (((const uint8_t *)buf)[i] != patt)
> -			return 0;
> -	return 1;
> +			return -1;
> +	return 0;
>  }
>  
>  int mtd_torture(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
> @@ -973,7 +973,7 @@ int mtd_torture(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
>  			goto out;
>  
>  		err = check_pattern(buf, 0xFF, mtd->eb_size);
> -		if (err == 0) {
> +		if (err) {
>  			errmsg("erased PEB %d, but a non-0xFF byte found", eb);
>  			errno = EIO;
>  			goto out;
> @@ -992,7 +992,7 @@ int mtd_torture(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
>  			goto out;
>  
>  		err = check_pattern(buf, patterns[i], mtd->eb_size);
> -		if (err == 0) {
> +		if (err) {
>  			errmsg("pattern %x checking failed for PEB %d",
>  				patterns[i], eb);
>  			errno = EIO;
> @@ -1005,7 +1005,7 @@ int mtd_torture(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
>  
>  out:
>  	free(buf);
> -	return -1;
> +	return err;
>  }
>  
>  int mtd_is_bad(const struct mtd_dev_info *mtd, int fd, int eb)



-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [PATCH 2/8] mtd-utils: Add multi-block erase function

[Boris Brezillon]

Richard, David,

On Tue, 26 Apr 2016 00:13:23 +0200
Richard Weinberger <richard@nod.at> wrote:

> From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> 
> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> Signed-off-by: Richard Weinberger <richard@nod.at>
> ---
>  include/libmtd.h | 14 ++++++++++++++
>  lib/libmtd.c     | 16 +++++++++++++---
>  2 files changed, 27 insertions(+), 3 deletions(-)
> 
> diff --git a/include/libmtd.h b/include/libmtd.h
> index a78c8cb..f3089d4 100644
> --- a/include/libmtd.h
> +++ b/include/libmtd.h
> @@ -174,6 +174,20 @@ int mtd_lock(const struct mtd_dev_info *mtd, int fd, int eb);
>  int mtd_unlock(const struct mtd_dev_info *mtd, int fd, int eb);
>  
>  /**
> + * mtd_erase - erase multiple eraseblocks.
> + * @desc: MTD library descriptor
> + * @mtd: MTD device description object
> + * @fd: MTD device node file descriptor
> + * @eb: index of first eraseblock to erase
> + * @blocks: the number of eraseblocks to erase
> + *
> + * This function erases eraseblock @eb of MTD device described by @fd. Returns
      
		    ^ erases @blocks starting at eraseblock @eb

> + * %0 in case of success and %-1 in case of failure.
> + */
> +int mtd_erase_multi(libmtd_t desc, const struct mtd_dev_info *mtd,
> +			int fd, int eb, int blocks);
> +
> +/**
>   * mtd_erase - erase an eraseblock.
>   * @desc: MTD library descriptor
>   * @mtd: MTD device description object
> diff --git a/lib/libmtd.c b/lib/libmtd.c
> index 1717468..6101238 100644
> --- a/lib/libmtd.c
> +++ b/lib/libmtd.c
> @@ -845,7 +845,8 @@ int mtd_unlock(const struct mtd_dev_info *mtd, int fd, int eb)
>  	return mtd_xlock(mtd, fd, eb, MEMUNLOCK);
>  }
>  
> -int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
> +int mtd_erase_multi(libmtd_t desc, const struct mtd_dev_info *mtd,
> +			int fd, int eb, int blocks)
>  {
>  	int ret;
>  	struct libmtd *lib = (struct libmtd *)desc;
> @@ -856,8 +857,12 @@ int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
>  	if (ret)
>  		return ret;
>  
> -	ei64.start = (__u64)eb * mtd->eb_size;
> -	ei64.length = mtd->eb_size;
> +	ret = mtd_valid_erase_block(mtd, eb + blocks - 1);
> +	if (ret)
> +		return ret;

Maybe you should also check if @eb is a valid block (what if @eb < 0,
but @eb + @blocks >= 0).

> +
> +	ei64.start = (__u64)eb * (__u64)mtd->eb_size;

I'm not sure you really need the second (__u64) cast, AFAIR, the first
one already guarantees that the intermediate result will be stored in a
__u64 variable.

> +	ei64.length = (__u64)mtd->eb_size * (__u64)blocks;

Ditto.

>  
>  	if (lib->offs64_ioctls == OFFS64_IOCTLS_SUPPORTED ||
>  	    lib->offs64_ioctls == OFFS64_IOCTLS_UNKNOWN) {
> @@ -892,6 +897,11 @@ int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
>  	return 0;
>  }
>  
> +int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
> +{
> +	return mtd_erase_multi(desc, mtd, fd, eb, 1);
> +}
> +
>  int mtd_regioninfo(int fd, int regidx, struct region_info_user *reginfo)
>  {
>  	int ret;



-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [PATCH 3/8] mtd-utils: Add flash torture test utility

[Boris Brezillon]

On Tue, 26 Apr 2016 00:13:24 +0200
Richard Weinberger <richard@nod.at> wrote:

> From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> 
> Basically a user space port of the mtd torture test kernel module. In
> addition to the block offset and count module parameters, the utility
> supports a block stride and can restore the block contents after test.
> 
> In contrast to the kernel module, the torture test is implemented by
> the libmtd mtd_toruture function and thus doesn't allow for similarly
> fine grained options on diagnostics.
> 
> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> Signed-off-by: Richard Weinberger <richard@nod.at>
> ---
>  .gitignore                 |   1 +
>  Makefile                   |   2 +-
>  misc-utils/flash_torture.c | 240 +++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 242 insertions(+), 1 deletion(-)
>  create mode 100644 misc-utils/flash_torture.c
> 
> diff --git a/.gitignore b/.gitignore
> index 2aac52c..5b529d1 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -35,6 +35,7 @@
>  /jffsX-utils/mkfs.jffs2
>  /misc-utils/mtd_debug
>  /misc-utils/mtdpart
> +/misc-utils/flash_torture
>  /nand-utils/nanddump
>  /nand-utils/nandtest
>  /nand-utils/nandwrite
> diff --git a/Makefile b/Makefile
> index 977c9c5..af3d1fd 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -20,7 +20,7 @@ MISC_BINS = \
>  	ftl_format doc_loadbios ftl_check mtd_debug docfdisk \
>  	serve_image recv_image mtdpart flash_erase flash_lock \
>  	flash_unlock flash_otp_info flash_otp_dump flash_otp_lock \
> -	flash_otp_write flashcp
> +	flash_otp_write flashcp flash_torture
>  UBI_BINS = \
>  	ubiupdatevol ubimkvol ubirmvol ubicrc32 ubinfo ubiattach \
>  	ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock
> diff --git a/misc-utils/flash_torture.c b/misc-utils/flash_torture.c
> new file mode 100644
> index 0000000..b5625c8
> --- /dev/null
> +++ b/misc-utils/flash_torture.c
> @@ -0,0 +1,240 @@
> +/*
> + * Copyright (C) 2006-2008 Artem Bityutskiy
> + * Copyright (C) 2006-2008 Jarkko Lavinen
> + * Copyright (C) 2006-2008 Adrian Hunter
> + * Copyright (C) 2015 sigma star gmbh
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published by
> + * the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; see the file COPYING. If not, write to the Free Software
> + * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
> + *
> + *
> + * WARNING: this test program may kill your flash and your device. Do not
> + * use it unless you know what you do. Authors are not responsible for any
> + * damage caused by this program.
> + *
> + * Author: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> + *
> + * Based on linux torturetest.c
> + * Authors: Artem Bityutskiy, Jarkko Lavinen, Adria Hunter
> + */
> +
> +#define PROGRAM_NAME "flash_torture"
> +
> +#define KEEP_CONTENTS 0x01
> +#define RUN_FOREVER 0x02
> +
> +#include <mtd/mtd-user.h>
> +#include <unistd.h>
> +#include <stdlib.h>
> +#include <libmtd.h>
> +#include <signal.h>
> +#include <stdio.h>
> +#include <fcntl.h>
> +
> +#include "common.h"
> +
> +static int peb=-1, blocks=-1, skip=-1;
> +static struct mtd_dev_info mtd;
> +static sig_atomic_t flags=0;
> +static const char *mtddev;
> +static libmtd_t mtd_desc;
> +static int mtdfd;
> +
> +static void sighandler(int sig)
> +{
> +	if (sig == SIGINT || sig == SIGTERM || sig == SIGHUP)
> +		flags &= ~RUN_FOREVER;
> +}
> +
> +static void usage(int status)
> +{
> +	fputs(
> +	"Usage: "PROGRAM_NAME" [OPTIONS] <device>\n\n"
> +	"Options:\n"
> +	"  -h, --help         Display this help output\n"
> +	"  -b, --peb <num>    Start from this physical erase block\n"
> +	"  -c, --blocks <num> Number of erase blocks to torture\n"
> +	"  -s, --skip <num>   Number of erase blocks to skip\n"
> +	"  -k, --keep         Try to restore existing contents after test\n"
> +	"  -r, --repeate      Repeate the torture test indefinitely\n",
> +	status==EXIT_SUCCESS ? stdout : stderr);
> +	exit(status);
> +}
> +
> +static long read_num(int idx, int argidx, int argc, char **argv)
> +{
> +	char *end;
> +	long num;
> +
> +	if (argidx >= argc) {
> +		fprintf(stderr, "%s: missing argument\n", argv[idx]);
> +		exit(EXIT_FAILURE);
> +	}
> +
> +	num = strtol(argv[argidx], &end, 0);
> +
> +	if (!end || *end!='\0') {
> +		fprintf(stderr, "%s: expected integer argument\n", argv[idx]);
> +		exit(EXIT_FAILURE);
> +	}
> +	return num;
> +}
> +
> +static void process_options(int argc, char **argv)
> +{
> +	int i;
> +
> +	for (i=1; i<argc; ++i) {
> +		if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
> +			usage(EXIT_SUCCESS);
> +		} else if (!strcmp(argv[i], "--peb") || !strcmp(argv[i], "-b")) {
> +			if (peb >= 0)
> +				goto failmulti;
> +			peb = read_num(i, i+1, argc, argv);
> +			if (peb < 0)
> +				goto failarg;
> +			++i;
> +		} else if (!strcmp(argv[i], "--blocks") || !strcmp(argv[i], "-c")) {
> +			if (blocks > 0)
> +				goto failmulti;
> +			blocks = read_num(i, i+1, argc, argv);
> +			if (blocks <= 0)
> +				goto failarg;
> +			++i;
> +		} else if (!strcmp(argv[i], "--skip") || !strcmp(argv[i], "-s")) {
> +			if (skip >= 0)
> +				goto failmulti;
> +			skip = read_num(i, i+1, argc, argv);
> +			if (skip < 0)
> +				goto failarg;
> +			++i;
> +		} else if (!strcmp(argv[i], "--keep") || !strcmp(argv[i], "-k")) {
> +			if (flags & KEEP_CONTENTS)
> +				goto failmulti;
> +			flags |= KEEP_CONTENTS;
> +		} else if (!strcmp(argv[i], "--repeate") || !strcmp(argv[i], "-r")) {
> +			if (flags & RUN_FOREVER)
> +				goto failmulti;
> +			flags |= RUN_FOREVER;
> +		} else {
> +			if (mtddev)
> +				usage(EXIT_FAILURE);
> +			mtddev = argv[i];
> +		}
> +	}

Hm, why not using getopt_long() to parse the command line?

> +
> +	if (!mtddev)
> +		errmsg_die("No device specified!\n");
> +	if (peb < 0)
> +		peb = 0;
> +	if (skip < 0)
> +		skip = 0;
> +	if (blocks < 0)
> +		blocks = 1;
> +	return;
> +failmulti:
> +	fprintf(stderr, "'%s' specified more than once!\n", argv[i]);
> +	exit(EXIT_FAILURE);
> +failarg:
> +	fprintf(stderr, "Invalid argument for '%s'!\n", argv[i]);
> +	exit(EXIT_FAILURE);
> +}
> +
> +int main(int argc, char **argv)
> +{
> +	int i, eb, err, count = 0;
> +	char* is_bad = NULL;

	char *is_bad = NULL;

> +	void *old=NULL;
> +
> +	process_options(argc, argv);
> +
> +	mtd_desc = libmtd_open();
> +	if (!mtd_desc)
> +		return errmsg("can't initialize libmtd");
> +
> +	if (mtd_get_dev_info(mtd_desc, mtddev, &mtd) < 0)
> +		return errmsg("mtd_get_dev_info failed");
> +
> +	if (peb >= mtd.eb_cnt)
> +		return errmsg("Physical erase block %d is out of range!\n", peb);
> +
> +	if ((peb + (blocks - 1)*(skip + 1)) >= mtd.eb_cnt) {
> +		return errmsg("Given block range exceeds block count of %d!\n",
> +				mtd.eb_cnt);
> +	}

You can drop the curly braces here.

> +
> +	signal(SIGINT, sighandler);
> +	signal(SIGTERM, sighandler);
> +	signal(SIGHUP, sighandler);
> +
> +	if (flags & KEEP_CONTENTS) {
> +		old = xmalloc(mtd.eb_size);
> +	}

Ditto.

> +
> +	is_bad = xmalloc(blocks);
> +
> +	if ((mtdfd = open(mtddev, O_RDWR)) == -1) {
> +		perror(mtddev);
> +		free(is_bad);
> +		free(old);
> +		return EXIT_FAILURE;
> +	}
> +
> +	for (i = 0; i < blocks; ++i) {
> +		eb = peb + i * (skip + 1);
> +		is_bad[i] = mtd_is_bad(&mtd, mtdfd, eb);
> +		if (is_bad[i]) {
> +			fprintf(stderr, "PEB %d marked bad, will be skipped\n", eb);
> +		}

Ditto.

> +	}
> +
> +	do {
> +		for (i = 0; i < blocks; ++i) {
> +			if (is_bad[i])
> +				continue;
> +
> +			eb = peb + i * (skip + 1);
> +
> +			if (flags & KEEP_CONTENTS) {
> +				err = mtd_read(&mtd, mtdfd, eb, 0, old, mtd.eb_size);
> +				if (err) {
> +					fprintf(stderr, "Failed to create backup copy "
> +							"of PEB %d, skipping!\n", eb);
> +					continue;
> +				}
> +			}
> +
> +			if (mtd_torture(mtd_desc, &mtd, mtdfd, eb))
> +				fprintf(stderr, "Block %d failed torture test!\n", eb);
> +
> +			if (flags & KEEP_CONTENTS) {
> +				err = mtd_erase(mtd_desc, &mtd, mtdfd, eb);
> +				if (err) {
> +					fprintf(stderr, "mtd_erase failed for block %d!\n", eb);
> +					continue;
> +				}
> +				err = mtd_write(mtd_desc, &mtd, mtdfd, eb, 0,
> +						old, mtd.eb_size, NULL, 0, 0);
> +				if (err)
> +					fprintf(stderr, "Failed to restore block %d!\n", eb);
> +			}
> +		}
> +
> +		printf("Torture test iterations done: %d\n", ++count);
> +	} while (flags & RUN_FOREVER);
> +
> +	free(old);
> +	free(is_bad);
> +	close(mtdfd);
> +	return EXIT_SUCCESS;
> +}



-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [PATCH 4/8] mtd-utils: Add flash stress test Utility

[Boris Brezillon]

On Tue, 26 Apr 2016 00:13:25 +0200
Richard Weinberger <richard@nod.at> wrote:

> diff --git a/misc-utils/flash_stress.c b/misc-utils/flash_stress.c
> new file mode 100644
> index 0000000..2dd2da1
> --- /dev/null
> +++ b/misc-utils/flash_stress.c

[...]

> +static void process_options(int argc, char **argv)
> +{
> +	int i;
> +
> +	for (i=1; i<argc; ++i) {
> +		if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
> +			usage(EXIT_SUCCESS);
> +		} else if (!strcmp(argv[i], "--keep") || !strcmp(argv[i], "-k")) {
> +			if (flags & KEEP_CONTENTS)
> +				goto failmulti;
> +			flags |= KEEP_CONTENTS;
> +		} else if (!strcmp(argv[i], "--seed") || !strcmp(argv[i], "-s")) {
> +			if (flags & SEED_SET)
> +				goto failmulti;
> +			srand(read_num(i, i+1, argc, argv));
> +			flags |= SEED_SET;
> +		} else if (!strcmp(argv[i], "--count") || !strcmp(argv[i], "-c")) {
> +			if (flags & COUNT_CHANGED)
> +				goto failmulti;
> +			count = read_num(i, i+1, argc, argv);
> +			if (count <= 0)
> +				goto failarg;
> +			++i;
> +			flags |= COUNT_CHANGED;
> +		} else {
> +			if (mtddev)
> +				usage(EXIT_FAILURE);
> +			mtddev = argv[i];
> +		}
> +	}

Please use getopt_long(). I guess this applies to all your patches, so
I'll stop mentioning it.

-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [PATCH 4/8] mtd-utils: Add flash stress test Utility

[Richard Weinberger]

Am 26.04.2016 um 10:18 schrieb Boris Brezillon:
> Please use getopt_long(). I guess this applies to all your patches, so
> I'll stop mentioning it.
> 

I agree, other mtd-utils programs use getopt too. Let's stay consistent.

BTW: How do you feel about switching globally over to argp?
http://www.gnu.org/software/libc/manual/html_node/Argp.html

IMHO it is much less pain to use and offers nice features such as auto generating
--help.
Although it is a GNU feature there is also a standalone package available which can be used
when building for musl or uclibc.

Thanks,
//richard

Re: [PATCH 4/8] mtd-utils: Add flash stress test Utility

[Boris Brezillon]

On Tue, 26 Apr 2016 11:22:28 +0200
Richard Weinberger <richard@nod.at> wrote:

> Am 26.04.2016 um 10:18 schrieb Boris Brezillon:
> > Please use getopt_long(). I guess this applies to all your patches, so
> > I'll stop mentioning it.
> >   
> 
> I agree, other mtd-utils programs use getopt too. Let's stay consistent.
> 
> BTW: How do you feel about switching globally over to argp?
> http://www.gnu.org/software/libc/manual/html_node/Argp.html
> 
> IMHO it is much less pain to use and offers nice features such as auto generating
> --help.
> Although it is a GNU feature there is also a standalone package available which can be used
> when building for musl or uclibc.

I don't have any objection to it. Let's see how Brian feels about it.


-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [PATCH 3/8] mtd-utils: Add flash torture test utility

[Ezequiel Garcia]

On 25 April 2016 at 19:13, Richard Weinberger <richard@nod.at> wrote:
> From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
>
> Basically a user space port of the mtd torture test kernel module. In
> addition to the block offset and count module parameters, the utility
> supports a block stride and can restore the block contents after test.
>
> In contrast to the kernel module, the torture test is implemented by
> the libmtd mtd_toruture function and thus doesn't allow for similarly
> fine grained options on diagnostics.
>
> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> Signed-off-by: Richard Weinberger <richard@nod.at>
> ---
>  .gitignore                 |   1 +
>  Makefile                   |   2 +-
>  misc-utils/flash_torture.c | 240 +++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 242 insertions(+), 1 deletion(-)
>  create mode 100644 misc-utils/flash_torture.c
>
> diff --git a/.gitignore b/.gitignore
> index 2aac52c..5b529d1 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -35,6 +35,7 @@
>  /jffsX-utils/mkfs.jffs2
>  /misc-utils/mtd_debug
>  /misc-utils/mtdpart
> +/misc-utils/flash_torture
>  /nand-utils/nanddump
>  /nand-utils/nandtest
>  /nand-utils/nandwrite
> diff --git a/Makefile b/Makefile
> index 977c9c5..af3d1fd 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -20,7 +20,7 @@ MISC_BINS = \
>         ftl_format doc_loadbios ftl_check mtd_debug docfdisk \
>         serve_image recv_image mtdpart flash_erase flash_lock \
>         flash_unlock flash_otp_info flash_otp_dump flash_otp_lock \
> -       flash_otp_write flashcp
> +       flash_otp_write flashcp flash_torture
>  UBI_BINS = \
>         ubiupdatevol ubimkvol ubirmvol ubicrc32 ubinfo ubiattach \
>         ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock
> diff --git a/misc-utils/flash_torture.c b/misc-utils/flash_torture.c
> new file mode 100644
> index 0000000..b5625c8
> --- /dev/null
> +++ b/misc-utils/flash_torture.c
> @@ -0,0 +1,240 @@
> +/*
> + * Copyright (C) 2006-2008 Artem Bityutskiy
> + * Copyright (C) 2006-2008 Jarkko Lavinen
> + * Copyright (C) 2006-2008 Adrian Hunter
> + * Copyright (C) 2015 sigma star gmbh
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published by
> + * the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; see the file COPYING. If not, write to the Free Software
> + * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
> + *
> + *
> + * WARNING: this test program may kill your flash and your device. Do not
> + * use it unless you know what you do. Authors are not responsible for any
> + * damage caused by this program.
> + *
> + * Author: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> + *
> + * Based on linux torturetest.c
> + * Authors: Artem Bityutskiy, Jarkko Lavinen, Adria Hunter
> + */
> +
> +#define PROGRAM_NAME "flash_torture"
> +
> +#define KEEP_CONTENTS 0x01
> +#define RUN_FOREVER 0x02
> +
> +#include <mtd/mtd-user.h>
> +#include <unistd.h>
> +#include <stdlib.h>
> +#include <libmtd.h>
> +#include <signal.h>
> +#include <stdio.h>
> +#include <fcntl.h>
> +
> +#include "common.h"
> +
> +static int peb=-1, blocks=-1, skip=-1;
> +static struct mtd_dev_info mtd;
> +static sig_atomic_t flags=0;
> +static const char *mtddev;
> +static libmtd_t mtd_desc;
> +static int mtdfd;
> +
> +static void sighandler(int sig)
> +{
> +       if (sig == SIGINT || sig == SIGTERM || sig == SIGHUP)
> +               flags &= ~RUN_FOREVER;
> +}
> +
> +static void usage(int status)
> +{
> +       fputs(
> +       "Usage: "PROGRAM_NAME" [OPTIONS] <device>\n\n"
> +       "Options:\n"
> +       "  -h, --help         Display this help output\n"
> +       "  -b, --peb <num>    Start from this physical erase block\n"
> +       "  -c, --blocks <num> Number of erase blocks to torture\n"
> +       "  -s, --skip <num>   Number of erase blocks to skip\n"
> +       "  -k, --keep         Try to restore existing contents after test\n"
> +       "  -r, --repeate      Repeate the torture test indefinitely\n",
> +       status==EXIT_SUCCESS ? stdout : stderr);
> +       exit(status);
> +}
> +
> +static long read_num(int idx, int argidx, int argc, char **argv)
> +{
> +       char *end;
> +       long num;
> +
> +       if (argidx >= argc) {
> +               fprintf(stderr, "%s: missing argument\n", argv[idx]);
> +               exit(EXIT_FAILURE);
> +       }
> +
> +       num = strtol(argv[argidx], &end, 0);
> +
> +       if (!end || *end!='\0') {
> +               fprintf(stderr, "%s: expected integer argument\n", argv[idx]);
> +               exit(EXIT_FAILURE);
> +       }
> +       return num;
> +}
> +
> +static void process_options(int argc, char **argv)
> +{
> +       int i;
> +
> +       for (i=1; i<argc; ++i) {
> +               if (!strcmp(argv[i], "--help") || !strcmp(argv[i], "-h")) {
> +                       usage(EXIT_SUCCESS);
> +               } else if (!strcmp(argv[i], "--peb") || !strcmp(argv[i], "-b")) {
> +                       if (peb >= 0)
> +                               goto failmulti;
> +                       peb = read_num(i, i+1, argc, argv);
> +                       if (peb < 0)
> +                               goto failarg;
> +                       ++i;
> +               } else if (!strcmp(argv[i], "--blocks") || !strcmp(argv[i], "-c")) {
> +                       if (blocks > 0)
> +                               goto failmulti;
> +                       blocks = read_num(i, i+1, argc, argv);
> +                       if (blocks <= 0)
> +                               goto failarg;
> +                       ++i;
> +               } else if (!strcmp(argv[i], "--skip") || !strcmp(argv[i], "-s")) {
> +                       if (skip >= 0)
> +                               goto failmulti;
> +                       skip = read_num(i, i+1, argc, argv);
> +                       if (skip < 0)
> +                               goto failarg;
> +                       ++i;
> +               } else if (!strcmp(argv[i], "--keep") || !strcmp(argv[i], "-k")) {
> +                       if (flags & KEEP_CONTENTS)
> +                               goto failmulti;
> +                       flags |= KEEP_CONTENTS;
> +               } else if (!strcmp(argv[i], "--repeate") || !strcmp(argv[i], "-r")) {
> +                       if (flags & RUN_FOREVER)
> +                               goto failmulti;
> +                       flags |= RUN_FOREVER;
> +               } else {
> +                       if (mtddev)
> +                               usage(EXIT_FAILURE);
> +                       mtddev = argv[i];
> +               }
> +       }
> +
> +       if (!mtddev)
> +               errmsg_die("No device specified!\n");
> +       if (peb < 0)
> +               peb = 0;
> +       if (skip < 0)
> +               skip = 0;
> +       if (blocks < 0)
> +               blocks = 1;
> +       return;
> +failmulti:
> +       fprintf(stderr, "'%s' specified more than once!\n", argv[i]);
> +       exit(EXIT_FAILURE);
> +failarg:
> +       fprintf(stderr, "Invalid argument for '%s'!\n", argv[i]);
> +       exit(EXIT_FAILURE);
> +}
> +
> +int main(int argc, char **argv)
> +{
> +       int i, eb, err, count = 0;
> +       char* is_bad = NULL;
> +       void *old=NULL;
> +
> +       process_options(argc, argv);
> +
> +       mtd_desc = libmtd_open();
> +       if (!mtd_desc)
> +               return errmsg("can't initialize libmtd");
> +
> +       if (mtd_get_dev_info(mtd_desc, mtddev, &mtd) < 0)
> +               return errmsg("mtd_get_dev_info failed");
> +
> +       if (peb >= mtd.eb_cnt)
> +               return errmsg("Physical erase block %d is out of range!\n", peb);
> +
> +       if ((peb + (blocks - 1)*(skip + 1)) >= mtd.eb_cnt) {
> +               return errmsg("Given block range exceeds block count of %d!\n",
> +                               mtd.eb_cnt);
> +       }
> +
> +       signal(SIGINT, sighandler);
> +       signal(SIGTERM, sighandler);
> +       signal(SIGHUP, sighandler);
> +
> +       if (flags & KEEP_CONTENTS) {
> +               old = xmalloc(mtd.eb_size);
> +       }
> +
> +       is_bad = xmalloc(blocks);
> +
> +       if ((mtdfd = open(mtddev, O_RDWR)) == -1) {
> +               perror(mtddev);
> +               free(is_bad);
> +               free(old);
> +               return EXIT_FAILURE;
> +       }
> +
> +       for (i = 0; i < blocks; ++i) {
> +               eb = peb + i * (skip + 1);
> +               is_bad[i] = mtd_is_bad(&mtd, mtdfd, eb);
> +               if (is_bad[i]) {
> +                       fprintf(stderr, "PEB %d marked bad, will be skipped\n", eb);
> +               }
> +       }
> +
> +       do {
> +               for (i = 0; i < blocks; ++i) {
> +                       if (is_bad[i])
> +                               continue;
> +
> +                       eb = peb + i * (skip + 1);
> +
> +                       if (flags & KEEP_CONTENTS) {
> +                               err = mtd_read(&mtd, mtdfd, eb, 0, old, mtd.eb_size);
> +                               if (err) {
> +                                       fprintf(stderr, "Failed to create backup copy "
> +                                                       "of PEB %d, skipping!\n", eb);
> +                                       continue;
> +                               }
> +                       }
> +
> +                       if (mtd_torture(mtd_desc, &mtd, mtdfd, eb))
> +                               fprintf(stderr, "Block %d failed torture test!\n", eb);
> +

mtd_torture seems to print its own message when the torture fails:

normsg("PEB %d passed torture test, do not mark it a bad", eb);

Isn't confusing to print "do not mark it a bad" (sic) in the torture test?
I mean: this would imply that the test would mark as bad if the torture fails?

-- 
Ezequiel García, VanguardiaSur
www.vanguardiasur.com.ar

Re: [PATCH 2/8] mtd-utils: Add multi-block erase function

[David Oberhollenzer]

On 04/26/2016 10:04 AM, Boris Brezillon wrote:
>> -int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
>> +int mtd_erase_multi(libmtd_t desc, const struct mtd_dev_info *mtd,
>> +			int fd, int eb, int blocks)
>>  {
>>  	int ret;
>>  	struct libmtd *lib = (struct libmtd *)desc;
>> @@ -856,8 +857,12 @@ int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
>>  	if (ret)
>>  		return ret;
>>  
>> -	ei64.start = (__u64)eb * mtd->eb_size;
>> -	ei64.length = mtd->eb_size;
>> +	ret = mtd_valid_erase_block(mtd, eb + blocks - 1);
>> +	if (ret)
>> +		return ret;
> 
> Maybe you should also check if @eb is a valid block (what if @eb < 0,
> but @eb + @blocks >= 0).
> 
This patch only renames the mtd_erase function to mtd_erase_multi and lets it
accept a block count. The existing code already checks @eb for validity.

A new implementation of mtd_erase is added below that calls mtd_erase_multi with a block
count of 1.

>> +
>> +	ei64.start = (__u64)eb * (__u64)mtd->eb_size;
>
> I'm not sure you really need the second (__u64) cast, AFAIR, the first
> one already guarantees that the intermediate result will be stored in a
> __u64 variable.

Yes, according to the C89 spec, section 3.2.1.5 it is technically not needed.
Since both are integer operands, casting one to something larger than int (and
of different signedness) causes the other operand to be converted to the same
type before the operation is performed.


David

Re: [PATCH 2/8] mtd-utils: Add multi-block erase function

[Boris Brezillon]

Hi David,

On Wed, 27 Apr 2016 11:21:31 +0200
David Oberhollenzer <david.oberhollenzer@sigma-star.at> wrote:

> On 04/26/2016 10:04 AM, Boris Brezillon wrote:
> >> -int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
> >> +int mtd_erase_multi(libmtd_t desc, const struct mtd_dev_info *mtd,
> >> +			int fd, int eb, int blocks)
> >>  {
> >>  	int ret;
> >>  	struct libmtd *lib = (struct libmtd *)desc;
> >> @@ -856,8 +857,12 @@ int mtd_erase(libmtd_t desc, const struct mtd_dev_info *mtd, int fd, int eb)
> >>  	if (ret)
> >>  		return ret;
> >>  
> >> -	ei64.start = (__u64)eb * mtd->eb_size;
> >> -	ei64.length = mtd->eb_size;
> >> +	ret = mtd_valid_erase_block(mtd, eb + blocks - 1);
> >> +	if (ret)
> >> +		return ret;  
> > 
> > Maybe you should also check if @eb is a valid block (what if @eb < 0,
> > but @eb + @blocks >= 0).
> >   
> This patch only renames the mtd_erase function to mtd_erase_multi and lets it
> accept a block count. The existing code already checks @eb for validity.

My bad, just didn't look at the original file before commenting.
Thanks for the clarification.

Boris

-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [PATCH 3/8] mtd-utils: Add flash torture test utility

[David Oberhollenzer]

On 04/26/2016 04:34 PM, Ezequiel Garcia wrote:
> mtd_torture seems to print its own message when the torture fails:
> 
> normsg("PEB %d passed torture test, do not mark it a bad", eb);
> 
> Isn't confusing to print "do not mark it a bad" (sic) in the torture test?
> I mean: this would imply that the test would mark as bad if the torture fails?
> 
Presumably mtd_torture was specifically written for ubiformat or moved over from
the ubiformat utility to libmtd, judging from the kind of diagnostics it generates
on its own.

It would probably be better to move these kinds of those diagnostics back/over to
ubiformat that uses the function to determine if it should mark a block as bad.


David

Re: [RFC] Porting kernel MTD tests to user space

[Brian Norris]

On Tue, Apr 26, 2016 at 09:00:22AM +0200, Richard Weinberger wrote:
> Am 26.04.2016 um 05:13 schrieb Ezequiel Garcia:
> > The autoconf patch looked also great, but for some reason it
> > stalled, so... what do we need to do to prevent this one from stalling as well?
> > 
> 
> Please give it testing/review. :-)
> Regarding my autoconf patch, IIRC nobody clearly stated whether the change
> is welcomed or not.

I reviewed it and noted a few missing pieces. But I was generally happy
with it, IIRC.

> But I can happily revive it.

Brian

Re: [PATCH 4/8] mtd-utils: Add flash stress test Utility

[Brian Norris]

On Tue, Apr 26, 2016 at 11:47:50AM +0200, Boris Brezillon wrote:
> On Tue, 26 Apr 2016 11:22:28 +0200
> Richard Weinberger <richard@nod.at> wrote:
> 
> > Am 26.04.2016 um 10:18 schrieb Boris Brezillon:
> > > Please use getopt_long(). I guess this applies to all your patches, so
> > > I'll stop mentioning it.
> > >   
> > 
> > I agree, other mtd-utils programs use getopt too. Let's stay consistent.
> > 
> > BTW: How do you feel about switching globally over to argp?
> > http://www.gnu.org/software/libc/manual/html_node/Argp.html
> > 
> > IMHO it is much less pain to use and offers nice features such as auto generating
> > --help.
> > Although it is a GNU feature there is also a standalone package available which can be used
> > when building for musl or uclibc.
> 
> I don't have any objection to it. Let's see how Brian feels about it.

I don't have strong opinions. My main question would have been about
build dependencies, but it sounds like it's not too onerous.

Brian

Re: [PATCH 1/8] mtd-utils: Fix return status in mtd_torture test function

[Brian Norris]

On Tue, Apr 26, 2016 at 12:13:22AM +0200, Richard Weinberger wrote:
> From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> 
> This patch fixes the return status of the mtd_torture function
> in libmtd.
> 
> The torture test function is currently only used by the ubiformat
> utilitiy to check if a block is bad after a write fails (blocks are
> marked bad if the function returns an error status). However, the
> way the function was written, it ALWAYS returns an error value
> irregardless of whether it failed or not.
> 
> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
> Signed-off-by: Richard Weinberger <richard@nod.at>

Seems like everyone's reviewed this, but no one applied it. (Richard,
did we get you push access to mtd-utils yet?)

Applied patch 1.

Brian

Re: [PATCH 1/8] mtd-utils: Fix return status in mtd_torture test function

[Richard Weinberger]

Brian,

Am 13.07.2016 um 19:30 schrieb Brian Norris:
> On Tue, Apr 26, 2016 at 12:13:22AM +0200, Richard Weinberger wrote:
>> From: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
>>
>> This patch fixes the return status of the mtd_torture function
>> in libmtd.
>>
>> The torture test function is currently only used by the ubiformat
>> utilitiy to check if a block is bad after a write fails (blocks are
>> marked bad if the function returns an error status). However, the
>> way the function was written, it ALWAYS returns an error value
>> irregardless of whether it failed or not.
>>
>> Signed-off-by: David Oberhollenzer <david.oberhollenzer@sigma-star.at>
>> Signed-off-by: Richard Weinberger <richard@nod.at>
> 
> Seems like everyone's reviewed this, but no one applied it. (Richard,
> did we get you push access to mtd-utils yet?)

I think I have write access.

But I'm still a bit unsure how the mtd-utils release cycle
works.
In fact, I fear there is none. ;-)

David was so kind and went through linux-mtd@ to located old but not merged
mtd-utils packages.
I'm currently rebasing them to 1.5.2 and would push them later into a temporary
branch.

Thanks,
//richard

Re: [PATCH 1/8] mtd-utils: Fix return status in mtd_torture test function

[Brian Norris]

On Wed, Jul 13, 2016 at 11:59:00PM +0200, Richard Weinberger wrote:
> But I'm still a bit unsure how the mtd-utils release cycle
> works.
> In fact, I fear there is none. ;-)

Yep, that's about correct. We can try to establish one if that'd help.
Previously, I think it was "whenever Artem decided there were enough
new things (fixes or features)".

> David was so kind and went through linux-mtd@ to located old but not merged
> mtd-utils packages.
> I'm currently rebasing them to 1.5.2 and would push them later into a temporary
> branch.

Cool.

Brian