video archive on a microSD card

video archive on a microSD card

[Alexander Gordeev]

Hi All,

I hope I'm writing to the right mailing list. If not please give me directions to the right place.
I'm trying to write video archive to a microSD card on an ARM based IP camera. The camera's SDK uses Linux 3.10.
The kernel is quite old and F2FS is there since 3.8 AFAIK so it's probably not mature yet. However, I decided to give it a try.
The idea is to write video continuously in 5 minute chunks. I also have an index file per each archive chunk file to for faster seeks and a single SQLite database.
When utilization is about 95%, the chunks and their indexes from the archive tail are deleted. So it's like a ring buffer. Also the overprovision ratio is the default 5%.
It worked quite good for several days with about 95% utilization, but then today it went bad. Writes are taking several seconds quite often as shown by strace.
vmstat shows that my process waits for IO most of the time:

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  2      0   6352      4  35924    0    0     8    88  562 1316 41  7  0 52
 1  2      0   6324      4  35928    0    0     4    44  553 1231 40  8  0 52
 1  2      0   6324      4  35928    0    0     0     0  690 1471 36 10  0 54
 1  3      0   6296      4  35932    0    0     0     0  530 1242 40  5  0 54
 2  2      0   6296      4  35936    0    0     4    48  545 1244 40  6  0 54
 1  2      0   6296      4  35940    0    0     4    44  549 1275 39  6  0 55
 2  2      0   6288      4  35944    0    0     4    44  563 1315 39  8  0 53
 3  2      0   6296      4  35944    0    0     0     0  502 1158 41  2  0 57
 1  3      0   6296      4  35952    0    0     8    88  700 1527 40  9  0 51
 1  2      0   6296      4  35952    0    0     0     0  482 1141 38  8  0 55
 1  2      0   6296      4  35956    0    0     4    44  594 1383 38 13  0 49
 1  2      0   6296      4  35956    0    0     0     0  489 1160 37  5  0 58
 5  1      0   6268      4  35980    0    0    12   132  704 1565 42  9  0 49
 2  1      0   6268      4  35984    0    0     4    44  531 1215 39 10  0 51
 3  2      0   6268      4  35992    0    0     8    92  714 1574 36  9  0 55
 1  1      0   6268      4  35992    0    0     0     0  485 1163 39  6  0 55
 1  1      0   6240      4  36000    0    0     8    92  553 1282 38  9  0 53
 1  2      0   6240      4  36000    0    0     0     0  488 1135 39  7  0 54
 1  1      0   6240      4  36000    0    0     0     0  552 1264 39  9  0 52
 3  1      0   6240      4  36000    0    0     0     0  510 1187 40  6  0 54
 1  2      0   6240      4  36004    0    0     4    44  674 1496 43  8  0 49
 1  1      0   6240      4  36012    0    0     8    88  572 1373 39  9  0 53
 4  1      0   6232      4  36016    0    0     4    48  549 1248 41  4  0 55
 3  1      0   6240      4  36016    0    0     0     0  520 1209 36  8  0 55

Here is also /sys/kernel/debug/f2fs/status for reference:
=====[ partition info(sda). #0 ]=====
[SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 Resv:50)]

Utilization: 94% (13597314 valid blocks)
  - Node: 16395 (Inode: 2913, Other: 13482)
  - Data: 13580919

Main area: 29646 segs, 14823 secs 14823 zones
  - COLD  data: 3468, 1734, 1734
  - WARM  data: 12954, 6477, 6477
  - HOT   data: 28105, 14052, 14052
  - Dir   dnode: 29204, 14602, 14602
  - File   dnode: 19960, 9980, 9980
  - Indir nodes: 29623, 14811, 14811

  - Valid: 13615
  - Dirty: 13309
  - Prefree: 0
  - Free: 2722 (763)

GC calls: 8622 (BG: 4311)
  - data segments : 8560
  - node segments : 62
Try to move 3552161 blocks
  - data blocks : 3540278
  - node blocks : 11883

Extent Hit Ratio: 49 / 4171

Balancing F2FS Async:
  - nodes    6 in  141
  - dents    0 in dirs:   0
  - meta   13 in  346
  - NATs 16983 > 29120
  - SITs:    17
  - free_nids:  1861

Distribution of User Blocks: [ valid | invalid | free ]
  [-----------------------------------------------|-|--]

SSR: 1230719 blocks in 14834 segments
LFS: 15150190 blocks in 29589 segments

BDF: 89, avg. vblocks: 949

Memory: 6754 KB = static: 4763 + cached: 1990


Please note that I tried to put all the archive and index files into the cold area using the file extensions mechanism.
And indeed I saw numbers close to the total amount of segments in the cold area a couple of days ago.
But now the cold area gets smaller very quickly. What am I doing wrong?

Can this be fixed by using a backport from git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git ?
Also I measured the SD card erase block size using flashbench. It seems it is 8MB, not 4MB, as I used here.
Can this lead to such serious problems? Is 8MB block safe to hardcode or should I use flashbench every time?

Please help. Thanks for reading!

-- 
 Alexander

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Alexander Gordeev]

Hi All,

12.08.2016, 14:52, "Alexander Gordeev" <alex@gordick.net>:
> Hi All,
>
> I hope I'm writing to the right mailing list. If not please give me directions to the right place.
> I'm trying to write video archive to a microSD card on an ARM based IP camera. The camera's SDK uses Linux 3.10.
> The kernel is quite old and F2FS is there since 3.8 AFAIK so it's probably not mature yet. However, I decided to give it a try.
> The idea is to write video continuously in 5 minute chunks. I also have an index file per each archive chunk file to for faster seeks and a single SQLite database.
> When utilization is about 95%, the chunks and their indexes from the archive tail are deleted. So it's like a ring buffer. Also the overprovision ratio is the default 5%.
> It worked quite good for several days with about 95% utilization, but then today it went bad. Writes are taking several seconds quite often as shown by strace.
> vmstat shows that my process waits for IO most of the time:
>
> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>  r b swpd free buff cache si so bi bo in cs us sy id wa
>  2 2 0 6352 4 35924 0 0 8 88 562 1316 41 7 0 52
>  1 2 0 6324 4 35928 0 0 4 44 553 1231 40 8 0 52
>  1 2 0 6324 4 35928 0 0 0 0 690 1471 36 10 0 54
>  1 3 0 6296 4 35932 0 0 0 0 530 1242 40 5 0 54
>  2 2 0 6296 4 35936 0 0 4 48 545 1244 40 6 0 54
>  1 2 0 6296 4 35940 0 0 4 44 549 1275 39 6 0 55
>  2 2 0 6288 4 35944 0 0 4 44 563 1315 39 8 0 53
>  3 2 0 6296 4 35944 0 0 0 0 502 1158 41 2 0 57
>  1 3 0 6296 4 35952 0 0 8 88 700 1527 40 9 0 51
>  1 2 0 6296 4 35952 0 0 0 0 482 1141 38 8 0 55
>  1 2 0 6296 4 35956 0 0 4 44 594 1383 38 13 0 49
>  1 2 0 6296 4 35956 0 0 0 0 489 1160 37 5 0 58
>  5 1 0 6268 4 35980 0 0 12 132 704 1565 42 9 0 49
>  2 1 0 6268 4 35984 0 0 4 44 531 1215 39 10 0 51
>  3 2 0 6268 4 35992 0 0 8 92 714 1574 36 9 0 55
>  1 1 0 6268 4 35992 0 0 0 0 485 1163 39 6 0 55
>  1 1 0 6240 4 36000 0 0 8 92 553 1282 38 9 0 53
>  1 2 0 6240 4 36000 0 0 0 0 488 1135 39 7 0 54
>  1 1 0 6240 4 36000 0 0 0 0 552 1264 39 9 0 52
>  3 1 0 6240 4 36000 0 0 0 0 510 1187 40 6 0 54
>  1 2 0 6240 4 36004 0 0 4 44 674 1496 43 8 0 49
>  1 1 0 6240 4 36012 0 0 8 88 572 1373 39 9 0 53
>  4 1 0 6232 4 36016 0 0 4 48 549 1248 41 4 0 55
>  3 1 0 6240 4 36016 0 0 0 0 520 1209 36 8 0 55
>
> Here is also /sys/kernel/debug/f2fs/status for reference:
> =====[ partition info(sda). #0 ]=====
> [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 Resv:50)]
>
> Utilization: 94% (13597314 valid blocks)
>   - Node: 16395 (Inode: 2913, Other: 13482)
>   - Data: 13580919
>
> Main area: 29646 segs, 14823 secs 14823 zones
>   - COLD data: 3468, 1734, 1734
>   - WARM data: 12954, 6477, 6477
>   - HOT data: 28105, 14052, 14052
>   - Dir dnode: 29204, 14602, 14602
>   - File dnode: 19960, 9980, 9980
>   - Indir nodes: 29623, 14811, 14811
>
>   - Valid: 13615
>   - Dirty: 13309
>   - Prefree: 0
>   - Free: 2722 (763)
>
> GC calls: 8622 (BG: 4311)
>   - data segments : 8560
>   - node segments : 62
> Try to move 3552161 blocks
>   - data blocks : 3540278
>   - node blocks : 11883
>
> Extent Hit Ratio: 49 / 4171
>
> Balancing F2FS Async:
>   - nodes 6 in 141
>   - dents 0 in dirs: 0
>   - meta 13 in 346
>   - NATs 16983 > 29120
>   - SITs: 17
>   - free_nids: 1861
>
> Distribution of User Blocks: [ valid | invalid | free ]
>   [-----------------------------------------------|-|--]
>
> SSR: 1230719 blocks in 14834 segments
> LFS: 15150190 blocks in 29589 segments
>
> BDF: 89, avg. vblocks: 949
>
> Memory: 6754 KB = static: 4763 + cached: 1990
>
> Please note that I tried to put all the archive and index files into the cold area using the file extensions mechanism.
> And indeed I saw numbers close to the total amount of segments in the cold area a couple of days ago.
> But now the cold area gets smaller very quickly. What am I doing wrong?
>
> Can this be fixed by using a backport from git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git ?
> Also I measured the SD card erase block size using flashbench. It seems it is 8MB, not 4MB, as I used here.
> Can this lead to such serious problems? Is 8MB block safe to hardcode or should I use flashbench every time?
>
> Please help. Thanks for reading!

I used backported version of f2fs above. It helps a bit, but not much. This is the new status:

=====[ partition info(sda). #0, RW]=====
[SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 Resv:50)]

Utilization: 94% (13608114 valid blocks)
  - Node: 17228 (Inode: 3793, Other: 13435)
  - Data: 13590886
  - Inline_xattr Inode: 0
  - Inline_data Inode: 52
  - Inline_dentry Inode: 0
  - Orphan Inode: 0

Main area: 29646 segs, 14823 secs 14823 zones
  - COLD  data: 11906, 5953, 5953
  - WARM  data: 11782, 5891, 5891
  - HOT   data: 11598, 5799, 5799
  - Dir   dnode: 9418, 4709, 4709
  - File   dnode: 11833, 5916, 5916
  - Indir nodes: 11382, 5691, 5691

  - Valid: 15671
  - Dirty: 12904
  - Prefree: 0
  - Free: 1071 (27)

CP calls: 3320 (BG: 0)
GC calls: 2240 (BG: 1)
  - data segments : 3866 (1236)
  - node segments : 243 (0)
Try to move 1123706 blocks (BG: 429924)
  - data blocks : 1012524 (429924)
  - node blocks : 111182 (0)

Extent Cache:
  - Hit Count: L1-1:55840 L1-2:14504 L2:1926
  - Hit Ratio: 3% (72270 / 2221766)
  - Inner Struct Count: tree: 570(0), node: 3

Balancing F2FS Async:
  - inmem:    0, wb_bios:    2
  - nodes:    0 in  239
  - dents:    0 in dirs:   0 (   0)
  - datas:  706 in files:   0
  - meta:    0 in  512
  - NATs:         0/      117
  - SITs:         0/    29646
  - free_nids:      1575

Distribution of User Blocks: [ valid | invalid | free ]
  [-----------------------------------------------|---|]

IPU: 1960 blocks
SSR: 18200 blocks in 71 segments
LFS: 2136736 blocks in 4173 segments

BDF: 87, avg. vblocks: 862

Memory: 9437 KB
  - static: 6385 KB
  - cached: 47 KB
  - paged : 3004 KB


I also have a patch for branch linux-3.10.y. I doesn't build without the patch.

-- 
 Alexander

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Chao Yu]

Hi Alexander,

On 2016/8/15 18:47, Alexander Gordeev wrote:
> Hi All,
> 
> 12.08.2016, 14:52, "Alexander Gordeev" <alex@gordick.net>:
>> Hi All,
>>
>> I hope I'm writing to the right mailing list. If not please give me directions to the right place.
>> I'm trying to write video archive to a microSD card on an ARM based IP camera. The camera's SDK uses Linux 3.10.
>> The kernel is quite old and F2FS is there since 3.8 AFAIK so it's probably not mature yet. However, I decided to give it a try.
>> The idea is to write video continuously in 5 minute chunks. I also have an index file per each archive chunk file to for faster seeks and a single SQLite database.
>> When utilization is about 95%, the chunks and their indexes from the archive tail are deleted. So it's like a ring buffer. Also the overprovision ratio is the default 5%.
>> It worked quite good for several days with about 95% utilization, but then today it went bad. Writes are taking several seconds quite often as shown by strace.
>> vmstat shows that my process waits for IO most of the time:
>>
>> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>>  r b swpd free buff cache si so bi bo in cs us sy id wa
>>  2 2 0 6352 4 35924 0 0 8 88 562 1316 41 7 0 52
>>  1 2 0 6324 4 35928 0 0 4 44 553 1231 40 8 0 52
>>  1 2 0 6324 4 35928 0 0 0 0 690 1471 36 10 0 54
>>  1 3 0 6296 4 35932 0 0 0 0 530 1242 40 5 0 54
>>  2 2 0 6296 4 35936 0 0 4 48 545 1244 40 6 0 54
>>  1 2 0 6296 4 35940 0 0 4 44 549 1275 39 6 0 55
>>  2 2 0 6288 4 35944 0 0 4 44 563 1315 39 8 0 53
>>  3 2 0 6296 4 35944 0 0 0 0 502 1158 41 2 0 57
>>  1 3 0 6296 4 35952 0 0 8 88 700 1527 40 9 0 51
>>  1 2 0 6296 4 35952 0 0 0 0 482 1141 38 8 0 55
>>  1 2 0 6296 4 35956 0 0 4 44 594 1383 38 13 0 49
>>  1 2 0 6296 4 35956 0 0 0 0 489 1160 37 5 0 58
>>  5 1 0 6268 4 35980 0 0 12 132 704 1565 42 9 0 49
>>  2 1 0 6268 4 35984 0 0 4 44 531 1215 39 10 0 51
>>  3 2 0 6268 4 35992 0 0 8 92 714 1574 36 9 0 55
>>  1 1 0 6268 4 35992 0 0 0 0 485 1163 39 6 0 55
>>  1 1 0 6240 4 36000 0 0 8 92 553 1282 38 9 0 53
>>  1 2 0 6240 4 36000 0 0 0 0 488 1135 39 7 0 54
>>  1 1 0 6240 4 36000 0 0 0 0 552 1264 39 9 0 52
>>  3 1 0 6240 4 36000 0 0 0 0 510 1187 40 6 0 54
>>  1 2 0 6240 4 36004 0 0 4 44 674 1496 43 8 0 49
>>  1 1 0 6240 4 36012 0 0 8 88 572 1373 39 9 0 53
>>  4 1 0 6232 4 36016 0 0 4 48 549 1248 41 4 0 55
>>  3 1 0 6240 4 36016 0 0 0 0 520 1209 36 8 0 55
>>
>> Here is also /sys/kernel/debug/f2fs/status for reference:
>> =====[ partition info(sda). #0 ]=====
>> [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 Resv:50)]
>>
>> Utilization: 94% (13597314 valid blocks)
>>   - Node: 16395 (Inode: 2913, Other: 13482)
>>   - Data: 13580919
>>
>> Main area: 29646 segs, 14823 secs 14823 zones
>>   - COLD data: 3468, 1734, 1734
>>   - WARM data: 12954, 6477, 6477
>>   - HOT data: 28105, 14052, 14052
>>   - Dir dnode: 29204, 14602, 14602
>>   - File dnode: 19960, 9980, 9980
>>   - Indir nodes: 29623, 14811, 14811
>>
>>   - Valid: 13615
>>   - Dirty: 13309
>>   - Prefree: 0
>>   - Free: 2722 (763)
>>
>> GC calls: 8622 (BG: 4311)
>>   - data segments : 8560
>>   - node segments : 62
>> Try to move 3552161 blocks
>>   - data blocks : 3540278
>>   - node blocks : 11883
>>
>> Extent Hit Ratio: 49 / 4171
>>
>> Balancing F2FS Async:
>>   - nodes 6 in 141
>>   - dents 0 in dirs: 0
>>   - meta 13 in 346
>>   - NATs 16983 > 29120
>>   - SITs: 17
>>   - free_nids: 1861
>>
>> Distribution of User Blocks: [ valid | invalid | free ]
>>   [-----------------------------------------------|-|--]
>>
>> SSR: 1230719 blocks in 14834 segments
>> LFS: 15150190 blocks in 29589 segments
>>
>> BDF: 89, avg. vblocks: 949
>>
>> Memory: 6754 KB = static: 4763 + cached: 1990
>>
>> Please note that I tried to put all the archive and index files into the cold area using the file extensions mechanism.
>> And indeed I saw numbers close to the total amount of segments in the cold area a couple of days ago.
>> But now the cold area gets smaller very quickly. What am I doing wrong?

How could you know the cold area is getting smaller? if it is looks as you said,
the behavior of f2fs seems not reasonable.

>>
>> Can this be fixed by using a backport from git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git ?
>> Also I measured the SD card erase block size using flashbench. It seems it is 8MB, not 4MB, as I used here.
>> Can this lead to such serious problems? Is 8MB block safe to hardcode or should I use flashbench every time?

I think 4MB is OK, if we set section size to 8MB, it will make us encountering
long latency of most operation due to foreground GC in where we may move more
blocks in one section.

>>
>> Please help. Thanks for reading!
> 
> I used backported version of f2fs above. It helps a bit, but not much. This is the new status:

Do you use discard mount option? and could you try using fstrim in range of
whole flash device to see whether performance is recoverable?

Thanks,

> 
> =====[ partition info(sda). #0, RW]=====
> [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 Resv:50)]
> 
> Utilization: 94% (13608114 valid blocks)
>   - Node: 17228 (Inode: 3793, Other: 13435)
>   - Data: 13590886
>   - Inline_xattr Inode: 0
>   - Inline_data Inode: 52
>   - Inline_dentry Inode: 0
>   - Orphan Inode: 0
> 
> Main area: 29646 segs, 14823 secs 14823 zones
>   - COLD  data: 11906, 5953, 5953
>   - WARM  data: 11782, 5891, 5891
>   - HOT   data: 11598, 5799, 5799
>   - Dir   dnode: 9418, 4709, 4709
>   - File   dnode: 11833, 5916, 5916
>   - Indir nodes: 11382, 5691, 5691
> 
>   - Valid: 15671
>   - Dirty: 12904
>   - Prefree: 0
>   - Free: 1071 (27)
> 
> CP calls: 3320 (BG: 0)
> GC calls: 2240 (BG: 1)
>   - data segments : 3866 (1236)
>   - node segments : 243 (0)
> Try to move 1123706 blocks (BG: 429924)
>   - data blocks : 1012524 (429924)
>   - node blocks : 111182 (0)
> 
> Extent Cache:
>   - Hit Count: L1-1:55840 L1-2:14504 L2:1926
>   - Hit Ratio: 3% (72270 / 2221766)
>   - Inner Struct Count: tree: 570(0), node: 3
> 
> Balancing F2FS Async:
>   - inmem:    0, wb_bios:    2
>   - nodes:    0 in  239
>   - dents:    0 in dirs:   0 (   0)
>   - datas:  706 in files:   0
>   - meta:    0 in  512
>   - NATs:         0/      117
>   - SITs:         0/    29646
>   - free_nids:      1575
> 
> Distribution of User Blocks: [ valid | invalid | free ]
>   [-----------------------------------------------|---|]
> 
> IPU: 1960 blocks
> SSR: 18200 blocks in 71 segments
> LFS: 2136736 blocks in 4173 segments
> 
> BDF: 87, avg. vblocks: 862
> 
> Memory: 9437 KB
>   - static: 6385 KB
>   - cached: 47 KB
>   - paged : 3004 KB
> 
> 
> I also have a patch for branch linux-3.10.y. I doesn't build without the patch.
> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity 
> planning reports. http://sdm.link/zohodev2dev
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> 


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. http://sdm.link/zohodev2dev

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Chao,

Thanks for your response!

15.08.2016, 14:58, "Chao Yu" <yuchao0@huawei.com>:
> Hi Alexander,
>
> On 2016/8/15 18:47, Alexander Gordeev wrote:
>>  Hi All,
>>
>>  12.08.2016, 14:52, "Alexander Gordeev" <alex@gordick.net>:
>>>  Hi All,
>>>
>>>  I hope I'm writing to the right mailing list. If not please give me directions to the right place.
>>>  I'm trying to write video archive to a microSD card on an ARM based IP camera. The camera's SDK uses Linux 3.10.
>>>  The kernel is quite old and F2FS is there since 3.8 AFAIK so it's probably not mature yet. However, I decided to give it a try.
>>>  The idea is to write video continuously in 5 minute chunks. I also have an index file per each archive chunk file to for faster seeks and a single SQLite database.
>>>  When utilization is about 95%, the chunks and their indexes from the archive tail are deleted. So it's like a ring buffer. Also the overprovision ratio is the default 5%.
>>>  It worked quite good for several days with about 95% utilization, but then today it went bad. Writes are taking several seconds quite often as shown by strace.
>>>  vmstat shows that my process waits for IO most of the time:
>>>
>>>  procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>>>   r b swpd free buff cache si so bi bo in cs us sy id wa
>>>   2 2 0 6352 4 35924 0 0 8 88 562 1316 41 7 0 52
>>>   1 2 0 6324 4 35928 0 0 4 44 553 1231 40 8 0 52
>>>   1 2 0 6324 4 35928 0 0 0 0 690 1471 36 10 0 54
>>>   1 3 0 6296 4 35932 0 0 0 0 530 1242 40 5 0 54
>>>   2 2 0 6296 4 35936 0 0 4 48 545 1244 40 6 0 54
>>>   1 2 0 6296 4 35940 0 0 4 44 549 1275 39 6 0 55
>>>   2 2 0 6288 4 35944 0 0 4 44 563 1315 39 8 0 53
>>>   3 2 0 6296 4 35944 0 0 0 0 502 1158 41 2 0 57
>>>   1 3 0 6296 4 35952 0 0 8 88 700 1527 40 9 0 51
>>>   1 2 0 6296 4 35952 0 0 0 0 482 1141 38 8 0 55
>>>   1 2 0 6296 4 35956 0 0 4 44 594 1383 38 13 0 49
>>>   1 2 0 6296 4 35956 0 0 0 0 489 1160 37 5 0 58
>>>   5 1 0 6268 4 35980 0 0 12 132 704 1565 42 9 0 49
>>>   2 1 0 6268 4 35984 0 0 4 44 531 1215 39 10 0 51
>>>   3 2 0 6268 4 35992 0 0 8 92 714 1574 36 9 0 55
>>>   1 1 0 6268 4 35992 0 0 0 0 485 1163 39 6 0 55
>>>   1 1 0 6240 4 36000 0 0 8 92 553 1282 38 9 0 53
>>>   1 2 0 6240 4 36000 0 0 0 0 488 1135 39 7 0 54
>>>   1 1 0 6240 4 36000 0 0 0 0 552 1264 39 9 0 52
>>>   3 1 0 6240 4 36000 0 0 0 0 510 1187 40 6 0 54
>>>   1 2 0 6240 4 36004 0 0 4 44 674 1496 43 8 0 49
>>>   1 1 0 6240 4 36012 0 0 8 88 572 1373 39 9 0 53
>>>   4 1 0 6232 4 36016 0 0 4 48 549 1248 41 4 0 55
>>>   3 1 0 6240 4 36016 0 0 0 0 520 1209 36 8 0 55
>>>
>>>  Here is also /sys/kernel/debug/f2fs/status for reference:
>>>  =====[ partition info(sda). #0 ]=====
>>>  [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 Resv:50)]
>>>
>>>  Utilization: 94% (13597314 valid blocks)
>>>    - Node: 16395 (Inode: 2913, Other: 13482)
>>>    - Data: 13580919
>>>
>>>  Main area: 29646 segs, 14823 secs 14823 zones
>>>    - COLD data: 3468, 1734, 1734
>>>    - WARM data: 12954, 6477, 6477
>>>    - HOT data: 28105, 14052, 14052
>>>    - Dir dnode: 29204, 14602, 14602
>>>    - File dnode: 19960, 9980, 9980
>>>    - Indir nodes: 29623, 14811, 14811
>>>
>>>    - Valid: 13615
>>>    - Dirty: 13309
>>>    - Prefree: 0
>>>    - Free: 2722 (763)
>>>
>>>  GC calls: 8622 (BG: 4311)
>>>    - data segments : 8560
>>>    - node segments : 62
>>>  Try to move 3552161 blocks
>>>    - data blocks : 3540278
>>>    - node blocks : 11883
>>>
>>>  Extent Hit Ratio: 49 / 4171
>>>
>>>  Balancing F2FS Async:
>>>    - nodes 6 in 141
>>>    - dents 0 in dirs: 0
>>>    - meta 13 in 346
>>>    - NATs 16983 > 29120
>>>    - SITs: 17
>>>    - free_nids: 1861
>>>
>>>  Distribution of User Blocks: [ valid | invalid | free ]
>>>    [-----------------------------------------------|-|--]
>>>
>>>  SSR: 1230719 blocks in 14834 segments
>>>  LFS: 15150190 blocks in 29589 segments
>>>
>>>  BDF: 89, avg. vblocks: 949
>>>
>>>  Memory: 6754 KB = static: 4763 + cached: 1990
>>>
>>>  Please note that I tried to put all the archive and index files into the cold area using the file extensions mechanism.
>>>  And indeed I saw numbers close to the total amount of segments in the cold area a couple of days ago.
>>>  But now the cold area gets smaller very quickly. What am I doing wrong?
>
> How could you know the cold area is getting smaller? if it is looks as you said,
> the behavior of f2fs seems not reasonable.

I'm not sure about this. I just saw numbers after "COLD data: "  above to be quite close to the total amount after "Main area:".
This is quite reasonable because I tried to the video files into the cold area. So the cold area should take almost all the device.
The idea is to separate video files from SQLite database writes. I got to this idea while reading some docs on f2fs.

>>>  Can this be fixed by using a backport from git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git ?
>>>  Also I measured the SD card erase block size using flashbench. It seems it is 8MB, not 4MB, as I used here.
>>>  Can this lead to such serious problems? Is 8MB block safe to hardcode or should I use flashbench every time?
>
> I think 4MB is OK, if we set section size to 8MB, it will make us encountering
> long latency of most operation due to foreground GC in where we may move more
> blocks in one section.

I see. I thought I have to align the section to the internal flash erase block size.
Actually, my problem is the increased latency after several days of rotating the archive a 95% utilization.

>>>  Please help. Thanks for reading!
>>
>>  I used backported version of f2fs above. It helps a bit, but not much. This is the new status:
>
> Do you use discard mount option? and could you try using fstrim in range of
> whole flash device to see whether performance is recoverable?

Well, it seems my SD card or SD controoler don't support discard. First I've seen this when formatting the SD card.
mkfs told me, that the device doesn't support discard. Then I tried to run blkdiscard manually. Unfortunately, it failed.

By the way, I create the FS like this: mkfs.f2fs -s 2 -e "arc,idx" /dev/sda1
And mount it like this: mount -t f2fs /dev/sda1 /storage/sda1 -o nodev,noexec,nosuid,noatime,nodiratime,nouser_xattr,noacl
I'd be happy to provide any other information for you.

-- 
 Alexander

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

[PATCH] f2fs: fix build for v3.10

[Alexander Gordeev]

kvfree() first appears in v3.15. f2fs_kvfree() should be used in the backport.

Signed-off-by: Alexander Gordeev <alex@gordick.net>
---
 fs/f2fs/file.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index 100594e..74d0523 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1036,7 +1036,7 @@ static int __exchange_data_block(struct inode *src_inode,
 
 		do_replace = f2fs_kvzalloc(sizeof(int) * olen, GFP_KERNEL);
 		if (!do_replace) {
-			kvfree(src_blkaddr);
+			f2fs_kvfree(src_blkaddr);
 			return -ENOMEM;
 		}
 
@@ -1054,15 +1054,15 @@ static int __exchange_data_block(struct inode *src_inode,
 		dst += olen;
 		len -= olen;
 
-		kvfree(src_blkaddr);
-		kvfree(do_replace);
+		f2fs_kvfree(src_blkaddr);
+		f2fs_kvfree(do_replace);
 	}
 	return 0;
 
 roll_back:
 	__roll_back_blkaddrs(src_inode, src_blkaddr, do_replace, src, len);
-	kvfree(src_blkaddr);
-	kvfree(do_replace);
+	f2fs_kvfree(src_blkaddr);
+	f2fs_kvfree(do_replace);
 	return ret;
 }
 
-- 
2.5.0


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. http://sdm.link/zohodev2dev

Re: video archive on a microSD card

[Chao Yu]

Hi Alexander,

On 2016/8/15 20:22, Alexander Gordeev wrote:
> Hi Chao,
> 
> Thanks for your response!
> 
> 15.08.2016, 14:58, "Chao Yu" <yuchao0@huawei.com>:
>> Hi Alexander,
>>
>> On 2016/8/15 18:47, Alexander Gordeev wrote:
>>>  Hi All,
>>>
>>>  12.08.2016, 14:52, "Alexander Gordeev" <alex@gordick.net>:
>>>>  Hi All,
>>>>
>>>>  I hope I'm writing to the right mailing list. If not please give me directions to the right place.
>>>>  I'm trying to write video archive to a microSD card on an ARM based IP camera. The camera's SDK uses Linux 3.10.
>>>>  The kernel is quite old and F2FS is there since 3.8 AFAIK so it's probably not mature yet. However, I decided to give it a try.
>>>>  The idea is to write video continuously in 5 minute chunks. I also have an index file per each archive chunk file to for faster seeks and a single SQLite database.
>>>>  When utilization is about 95%, the chunks and their indexes from the archive tail are deleted. So it's like a ring buffer. Also the overprovision ratio is the default 5%.
>>>>  It worked quite good for several days with about 95% utilization, but then today it went bad. Writes are taking several seconds quite often as shown by strace.
>>>>  vmstat shows that my process waits for IO most of the time:
>>>>
>>>>  procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>>>>   r b swpd free buff cache si so bi bo in cs us sy id wa
>>>>   2 2 0 6352 4 35924 0 0 8 88 562 1316 41 7 0 52
>>>>   1 2 0 6324 4 35928 0 0 4 44 553 1231 40 8 0 52
>>>>   1 2 0 6324 4 35928 0 0 0 0 690 1471 36 10 0 54
>>>>   1 3 0 6296 4 35932 0 0 0 0 530 1242 40 5 0 54
>>>>   2 2 0 6296 4 35936 0 0 4 48 545 1244 40 6 0 54
>>>>   1 2 0 6296 4 35940 0 0 4 44 549 1275 39 6 0 55
>>>>   2 2 0 6288 4 35944 0 0 4 44 563 1315 39 8 0 53
>>>>   3 2 0 6296 4 35944 0 0 0 0 502 1158 41 2 0 57
>>>>   1 3 0 6296 4 35952 0 0 8 88 700 1527 40 9 0 51
>>>>   1 2 0 6296 4 35952 0 0 0 0 482 1141 38 8 0 55
>>>>   1 2 0 6296 4 35956 0 0 4 44 594 1383 38 13 0 49
>>>>   1 2 0 6296 4 35956 0 0 0 0 489 1160 37 5 0 58
>>>>   5 1 0 6268 4 35980 0 0 12 132 704 1565 42 9 0 49
>>>>   2 1 0 6268 4 35984 0 0 4 44 531 1215 39 10 0 51
>>>>   3 2 0 6268 4 35992 0 0 8 92 714 1574 36 9 0 55
>>>>   1 1 0 6268 4 35992 0 0 0 0 485 1163 39 6 0 55
>>>>   1 1 0 6240 4 36000 0 0 8 92 553 1282 38 9 0 53
>>>>   1 2 0 6240 4 36000 0 0 0 0 488 1135 39 7 0 54
>>>>   1 1 0 6240 4 36000 0 0 0 0 552 1264 39 9 0 52
>>>>   3 1 0 6240 4 36000 0 0 0 0 510 1187 40 6 0 54
>>>>   1 2 0 6240 4 36004 0 0 4 44 674 1496 43 8 0 49
>>>>   1 1 0 6240 4 36012 0 0 8 88 572 1373 39 9 0 53
>>>>   4 1 0 6232 4 36016 0 0 4 48 549 1248 41 4 0 55
>>>>   3 1 0 6240 4 36016 0 0 0 0 520 1209 36 8 0 55
>>>>
>>>>  Here is also /sys/kernel/debug/f2fs/status for reference:
>>>>  =====[ partition info(sda). #0 ]=====
>>>>  [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 Resv:50)]
>>>>
>>>>  Utilization: 94% (13597314 valid blocks)
>>>>    - Node: 16395 (Inode: 2913, Other: 13482)
>>>>    - Data: 13580919
>>>>
>>>>  Main area: 29646 segs, 14823 secs 14823 zones
>>>>    - COLD data: 3468, 1734, 1734
>>>>    - WARM data: 12954, 6477, 6477
>>>>    - HOT data: 28105, 14052, 14052
>>>>    - Dir dnode: 29204, 14602, 14602
>>>>    - File dnode: 19960, 9980, 9980
>>>>    - Indir nodes: 29623, 14811, 14811
>>>>
>>>>    - Valid: 13615
>>>>    - Dirty: 13309
>>>>    - Prefree: 0
>>>>    - Free: 2722 (763)
>>>>
>>>>  GC calls: 8622 (BG: 4311)
>>>>    - data segments : 8560
>>>>    - node segments : 62
>>>>  Try to move 3552161 blocks
>>>>    - data blocks : 3540278
>>>>    - node blocks : 11883
>>>>
>>>>  Extent Hit Ratio: 49 / 4171
>>>>
>>>>  Balancing F2FS Async:
>>>>    - nodes 6 in 141
>>>>    - dents 0 in dirs: 0
>>>>    - meta 13 in 346
>>>>    - NATs 16983 > 29120
>>>>    - SITs: 17
>>>>    - free_nids: 1861
>>>>
>>>>  Distribution of User Blocks: [ valid | invalid | free ]
>>>>    [-----------------------------------------------|-|--]
>>>>
>>>>  SSR: 1230719 blocks in 14834 segments
>>>>  LFS: 15150190 blocks in 29589 segments
>>>>
>>>>  BDF: 89, avg. vblocks: 949
>>>>
>>>>  Memory: 6754 KB = static: 4763 + cached: 1990
>>>>
>>>>  Please note that I tried to put all the archive and index files into the cold area using the file extensions mechanism.
>>>>  And indeed I saw numbers close to the total amount of segments in the cold area a couple of days ago.
>>>>  But now the cold area gets smaller very quickly. What am I doing wrong?
>>
>> How could you know the cold area is getting smaller? if it is looks as you said,
>> the behavior of f2fs seems not reasonable.
> 
> I'm not sure about this. I just saw numbers after "COLD data: "  above to be quite close to the total amount after "Main area:".

Hmm.. number after "COLD data:" means No. of current cold segment, section and zone.

> This is quite reasonable because I tried to the video files into the cold area. So the cold area should take almost all the device.
> The idea is to separate video files from SQLite database writes. I got to this idea while reading some docs on f2fs.

I think this would be a good idea if SQLite db files and idx&video file have
different updating frequency.

> 
>>>>  Can this be fixed by using a backport from git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git ?
>>>>  Also I measured the SD card erase block size using flashbench. It seems it is 8MB, not 4MB, as I used here.
>>>>  Can this lead to such serious problems? Is 8MB block safe to hardcode or should I use flashbench every time?
>>
>> I think 4MB is OK, if we set section size to 8MB, it will make us encountering
>> long latency of most operation due to foreground GC in where we may move more
>> blocks in one section.
> 
> I see. I thought I have to align the section to the internal flash erase block size.
> Actually, my problem is the increased latency after several days of rotating the archive a 95% utilization.

  - Valid: 15671
  - Dirty: 12904
  - Prefree: 0
  - Free: 1071 (27)

CP calls: 3320 (BG: 0)
GC calls: 2240 (BG: 1)
  - data segments : 3866 (1236)
  - node segments : 243 (0)

I can see that in your partition there are lots of dirty segments, and few free
segment, in this statement, we will be more likely to trigger synchronously
foreground garbage collection which may lead to long latency. Also, it is
indicated in you "GC calls" statement.

Can you do synchronously GC through F2FS_IOC_GARBAGE_COLLECT ioctl with
parameter @sync which value is set as 1 until free segments size is more close
to free space FS show to user? I expect this can help to recover performance in
your enviornment.

If this does not help, I think we should do some trace in order to find out the
root cause of write delay.

Thanks,

> 
>>>>  Please help. Thanks for reading!
>>>
>>>  I used backported version of f2fs above. It helps a bit, but not much. This is the new status:
>>
>> Do you use discard mount option? and could you try using fstrim in range of
>> whole flash device to see whether performance is recoverable?
> 
> Well, it seems my SD card or SD controoler don't support discard. First I've seen this when formatting the SD card.
> mkfs told me, that the device doesn't support discard. Then I tried to run blkdiscard manually. Unfortunately, it failed.
> 
> By the way, I create the FS like this: mkfs.f2fs -s 2 -e "arc,idx" /dev/sda1
> And mount it like this: mount -t f2fs /dev/sda1 /storage/sda1 -o nodev,noexec,nosuid,noatime,nodiratime,nouser_xattr,noacl
> I'd be happy to provide any other information for you.
> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are 
> consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
> J-Flow, sFlow and other flows. Make informed decisions using capacity 
> planning reports. http://sdm.link/zohodev2dev
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> 

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Chao,

> On 2016/8/15 20:22, Alexander Gordeev wrote:
> > 15.08.2016, 14:58, "Chao Yu" <yuch...@huawei.com>:
> >> Hi Alexander,
> >>
> >> On 2016/8/15 18:47, Alexander Gordeev wrote:
> >>>  Hi All,
> >>>
> >>>  12.08.2016, 14:52, "Alexander Gordeev" <a...@gordick.net>:
> >>>>  Hi All,
> >>>>
> >>>>  I hope I'm writing to the right mailing list. If not please give me 
> >>>> directions to the right place.
> >>>>  I'm trying to write video archive to a microSD card on an ARM based IP 
> >>>> camera. The camera's SDK uses Linux 3.10.
> >>>>  The kernel is quite old and F2FS is there since 3.8 AFAIK so it's 
> >>>> probably not mature yet. However, I decided to give it a try.
> >>>>  The idea is to write video continuously in 5 minute chunks. I also have 
> >>>> an index file per each archive chunk file to for faster seeks and a single 
> >>>> SQLite database.
> >>>>  When utilization is about 95%, the chunks and their indexes from the 
> >>>> archive tail are deleted. So it's like a ring buffer. Also the 
> >>>> overprovision ratio is the default 5%.
> >>>>  It worked quite good for several days with about 95% utilization, but 
> >>>> then today it went bad. Writes are taking several seconds quite often as 
> >>>> shown by strace.
> >>>>  vmstat shows that my process waits for IO most of the time:
> >>>>
> >>>>  procs -----------memory---------- ---swap-- -----io---- -system-- 
> >>>> ----cpu----
> >>>>   r b swpd free buff cache si so bi bo in cs us sy id wa
> >>>>   2 2 0 6352 4 35924 0 0 8 88 562 1316 41 7 0 52
> >>>>   1 2 0 6324 4 35928 0 0 4 44 553 1231 40 8 0 52
> >>>>   1 2 0 6324 4 35928 0 0 0 0 690 1471 36 10 0 54
> >>>>   1 3 0 6296 4 35932 0 0 0 0 530 1242 40 5 0 54
> >>>>   2 2 0 6296 4 35936 0 0 4 48 545 1244 40 6 0 54
> >>>>   1 2 0 6296 4 35940 0 0 4 44 549 1275 39 6 0 55
> >>>>   2 2 0 6288 4 35944 0 0 4 44 563 1315 39 8 0 53
> >>>>   3 2 0 6296 4 35944 0 0 0 0 502 1158 41 2 0 57
> >>>>   1 3 0 6296 4 35952 0 0 8 88 700 1527 40 9 0 51
> >>>>   1 2 0 6296 4 35952 0 0 0 0 482 1141 38 8 0 55
> >>>>   1 2 0 6296 4 35956 0 0 4 44 594 1383 38 13 0 49
> >>>>   1 2 0 6296 4 35956 0 0 0 0 489 1160 37 5 0 58
> >>>>   5 1 0 6268 4 35980 0 0 12 132 704 1565 42 9 0 49
> >>>>   2 1 0 6268 4 35984 0 0 4 44 531 1215 39 10 0 51
> >>>>   3 2 0 6268 4 35992 0 0 8 92 714 1574 36 9 0 55
> >>>>   1 1 0 6268 4 35992 0 0 0 0 485 1163 39 6 0 55
> >>>>   1 1 0 6240 4 36000 0 0 8 92 553 1282 38 9 0 53
> >>>>   1 2 0 6240 4 36000 0 0 0 0 488 1135 39 7 0 54
> >>>>   1 1 0 6240 4 36000 0 0 0 0 552 1264 39 9 0 52
> >>>>   3 1 0 6240 4 36000 0 0 0 0 510 1187 40 6 0 54
> >>>>   1 2 0 6240 4 36004 0 0 4 44 674 1496 43 8 0 49
> >>>>   1 1 0 6240 4 36012 0 0 8 88 572 1373 39 9 0 53
> >>>>   4 1 0 6232 4 36016 0 0 4 48 549 1248 41 4 0 55
> >>>>   3 1 0 6240 4 36016 0 0 0 0 520 1209 36 8 0 55
> >>>>
> >>>>  Here is also /sys/kernel/debug/f2fs/status for reference:
> >>>>  =====[ partition info(sda). #0 ]=====
> >>>>  [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 
> >>>> Resv:50)]
> >>>>
> >>>>  Utilization: 94% (13597314 valid blocks)
> >>>>    - Node: 16395 (Inode: 2913, Other: 13482)
> >>>>    - Data: 13580919
> >>>>
> >>>>  Main area: 29646 segs, 14823 secs 14823 zones
> >>>>    - COLD data: 3468, 1734, 1734
> >>>>    - WARM data: 12954, 6477, 6477
> >>>>    - HOT data: 28105, 14052, 14052
> >>>>    - Dir dnode: 29204, 14602, 14602
> >>>>    - File dnode: 19960, 9980, 9980
> >>>>    - Indir nodes: 29623, 14811, 14811
> >>>>
> >>>>    - Valid: 13615
> >>>>    - Dirty: 13309
> >>>>    - Prefree: 0
> >>>>    - Free: 2722 (763)
> >>>>
> >>>>  GC calls: 8622 (BG: 4311)
> >>>>    - data segments : 8560
> >>>>    - node segments : 62
> >>>>  Try to move 3552161 blocks
> >>>>    - data blocks : 3540278
> >>>>    - node blocks : 11883
> >>>>
> >>>>  Extent Hit Ratio: 49 / 4171
> >>>>
> >>>>  Balancing F2FS Async:
> >>>>    - nodes 6 in 141
> >>>>    - dents 0 in dirs: 0
> >>>>    - meta 13 in 346
> >>>>    - NATs 16983 > 29120
> >>>>    - SITs: 17
> >>>>    - free_nids: 1861
> >>>>
> >>>>  Distribution of User Blocks: [ valid | invalid | free ]
> >>>>    [-----------------------------------------------|-|--]
> >>>>
> >>>>  SSR: 1230719 blocks in 14834 segments
> >>>>  LFS: 15150190 blocks in 29589 segments
> >>>>
> >>>>  BDF: 89, avg. vblocks: 949
> >>>>
> >>>>  Memory: 6754 KB = static: 4763 + cached: 1990
> >>>>
> >>>>  Please note that I tried to put all the archive and index files into the 
> >>>> cold area using the file extensions mechanism.
> >>>>  And indeed I saw numbers close to the total amount of segments in the 
> >>>> cold area a couple of days ago.
> >>>>  But now the cold area gets smaller very quickly. What am I doing wrong?
> >>
> >> How could you know the cold area is getting smaller? if it is looks as you 
> >> said,
> >> the behavior of f2fs seems not reasonable.
> > 
> > I'm not sure about this. I just saw numbers after "COLD data: "  above to be 
> > quite close to the total amount after "Main area:".
> 
> Hmm.. number after "COLD data:" means No. of current cold segment, section and 
> zone.
> 
> > This is quite reasonable because I tried to the video files into the cold 
> > area. So the cold area should take almost all the device.
> > The idea is to separate video files from SQLite database writes. I got to 
> > this idea while reading some docs on f2fs.
> 
> I think this would be a good idea if SQLite db files and idx&video file have
> different updating frequency.

Yes, their times of life are quite different probably. I don't know exactly how
SQLite manages its db files. I think it's not append only. So I thought that it
would be a right idea to separate the other files, that I'm in total control
of, from SQLite db file.

My video archive and index files are written in append-only manner. They are
never overwritten. They can only be deleted when the archive is rotated.
Per my understanding of f2fs internals, it should write these "cold" files and
usual "hot" files to different sections (that should map internally to
different allocation units). So the sections used by "cold" data should almost
never get "dirty" because most of the time all their blocks become free at
the same time. Of course, the files are not exactly 4MB in size so the last
section of the deleted file will become dirty. If it is moved by garbage
collector and becomes mixed with fresh "cold" data, then indeed it might cause
some problems, I think. What is your opinion?

Maybe disabling garbage collection for "cold" data may help?

> >>>>  Can this be fixed by using a backport from 
> >>>> git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git ?
> >>>>  Also I measured the SD card erase block size using flashbench. It seems 
> >>>> it is 8MB, not 4MB, as I used here.
> >>>>  Can this lead to such serious problems? Is 8MB block safe to hardcode or 
> >>>> should I use flashbench every time?
> >>
> >> I think 4MB is OK, if we set section size to 8MB, it will make us 
> >> encountering
> >> long latency of most operation due to foreground GC in where we may move more
> >> blocks in one section.
> > 
> > I see. I thought I have to align the section to the internal flash erase 
> > block size.
> > Actually, my problem is the increased latency after several days of rotating 
> > the archive a 95% utilization.
> 
>   - Valid: 15671
>   - Dirty: 12904
>   - Prefree: 0
>   - Free: 1071 (27)
> 
> CP calls: 3320 (BG: 0)
> GC calls: 2240 (BG: 1)
>   - data segments : 3866 (1236)
>   - node segments : 243 (0)
> 
> I can see that in your partition there are lots of dirty segments, and few free
> segment, in this statement, we will be more likely to trigger synchronously
> foreground garbage collection which may lead to long latency. Also, it is
> indicated in you "GC calls" statement.

Yes, I see.

> Can you do synchronously GC through F2FS_IOC_GARBAGE_COLLECT ioctl with
> parameter @sync which value is set as 1 until free segments size is more close
> to free space FS show to user? I expect this can help to recover performance in
> your enviornment.
> 
> If this does not help, I think we should do some trace in order to find out the
> root cause of write delay.

I'll try the ioctl, thanks!

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Chao Yu]

Hi Alexander,

On 2016/8/17 17:47, Alexander Gordeev wrote:
> Hi Chao,
> 
>> On 2016/8/15 20:22, Alexander Gordeev wrote:
>>> 15.08.2016, 14:58, "Chao Yu" <yuch...@huawei.com>:
>>>> Hi Alexander,
>>>>
>>>> On 2016/8/15 18:47, Alexander Gordeev wrote:
>>>>>  Hi All,
>>>>>
>>>>>  12.08.2016, 14:52, "Alexander Gordeev" <a...@gordick.net>:
>>>>>>  Hi All,
>>>>>>
>>>>>>  I hope I'm writing to the right mailing list. If not please give me 
>>>>>> directions to the right place.
>>>>>>  I'm trying to write video archive to a microSD card on an ARM based IP 
>>>>>> camera. The camera's SDK uses Linux 3.10.
>>>>>>  The kernel is quite old and F2FS is there since 3.8 AFAIK so it's 
>>>>>> probably not mature yet. However, I decided to give it a try.
>>>>>>  The idea is to write video continuously in 5 minute chunks. I also have 
>>>>>> an index file per each archive chunk file to for faster seeks and a single 
>>>>>> SQLite database.
>>>>>>  When utilization is about 95%, the chunks and their indexes from the 
>>>>>> archive tail are deleted. So it's like a ring buffer. Also the 
>>>>>> overprovision ratio is the default 5%.
>>>>>>  It worked quite good for several days with about 95% utilization, but 
>>>>>> then today it went bad. Writes are taking several seconds quite often as 
>>>>>> shown by strace.
>>>>>>  vmstat shows that my process waits for IO most of the time:
>>>>>>
>>>>>>  procs -----------memory---------- ---swap-- -----io---- -system-- 
>>>>>> ----cpu----
>>>>>>   r b swpd free buff cache si so bi bo in cs us sy id wa
>>>>>>   2 2 0 6352 4 35924 0 0 8 88 562 1316 41 7 0 52
>>>>>>   1 2 0 6324 4 35928 0 0 4 44 553 1231 40 8 0 52
>>>>>>   1 2 0 6324 4 35928 0 0 0 0 690 1471 36 10 0 54
>>>>>>   1 3 0 6296 4 35932 0 0 0 0 530 1242 40 5 0 54
>>>>>>   2 2 0 6296 4 35936 0 0 4 48 545 1244 40 6 0 54
>>>>>>   1 2 0 6296 4 35940 0 0 4 44 549 1275 39 6 0 55
>>>>>>   2 2 0 6288 4 35944 0 0 4 44 563 1315 39 8 0 53
>>>>>>   3 2 0 6296 4 35944 0 0 0 0 502 1158 41 2 0 57
>>>>>>   1 3 0 6296 4 35952 0 0 8 88 700 1527 40 9 0 51
>>>>>>   1 2 0 6296 4 35952 0 0 0 0 482 1141 38 8 0 55
>>>>>>   1 2 0 6296 4 35956 0 0 4 44 594 1383 38 13 0 49
>>>>>>   1 2 0 6296 4 35956 0 0 0 0 489 1160 37 5 0 58
>>>>>>   5 1 0 6268 4 35980 0 0 12 132 704 1565 42 9 0 49
>>>>>>   2 1 0 6268 4 35984 0 0 4 44 531 1215 39 10 0 51
>>>>>>   3 2 0 6268 4 35992 0 0 8 92 714 1574 36 9 0 55
>>>>>>   1 1 0 6268 4 35992 0 0 0 0 485 1163 39 6 0 55
>>>>>>   1 1 0 6240 4 36000 0 0 8 92 553 1282 38 9 0 53
>>>>>>   1 2 0 6240 4 36000 0 0 0 0 488 1135 39 7 0 54
>>>>>>   1 1 0 6240 4 36000 0 0 0 0 552 1264 39 9 0 52
>>>>>>   3 1 0 6240 4 36000 0 0 0 0 510 1187 40 6 0 54
>>>>>>   1 2 0 6240 4 36004 0 0 4 44 674 1496 43 8 0 49
>>>>>>   1 1 0 6240 4 36012 0 0 8 88 572 1373 39 9 0 53
>>>>>>   4 1 0 6232 4 36016 0 0 4 48 549 1248 41 4 0 55
>>>>>>   3 1 0 6240 4 36016 0 0 0 0 520 1209 36 8 0 55
>>>>>>
>>>>>>  Here is also /sys/kernel/debug/f2fs/status for reference:
>>>>>>  =====[ partition info(sda). #0 ]=====
>>>>>>  [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529 
>>>>>> Resv:50)]
>>>>>>
>>>>>>  Utilization: 94% (13597314 valid blocks)
>>>>>>    - Node: 16395 (Inode: 2913, Other: 13482)
>>>>>>    - Data: 13580919
>>>>>>
>>>>>>  Main area: 29646 segs, 14823 secs 14823 zones
>>>>>>    - COLD data: 3468, 1734, 1734
>>>>>>    - WARM data: 12954, 6477, 6477
>>>>>>    - HOT data: 28105, 14052, 14052
>>>>>>    - Dir dnode: 29204, 14602, 14602
>>>>>>    - File dnode: 19960, 9980, 9980
>>>>>>    - Indir nodes: 29623, 14811, 14811
>>>>>>
>>>>>>    - Valid: 13615
>>>>>>    - Dirty: 13309
>>>>>>    - Prefree: 0
>>>>>>    - Free: 2722 (763)
>>>>>>
>>>>>>  GC calls: 8622 (BG: 4311)
>>>>>>    - data segments : 8560
>>>>>>    - node segments : 62
>>>>>>  Try to move 3552161 blocks
>>>>>>    - data blocks : 3540278
>>>>>>    - node blocks : 11883
>>>>>>
>>>>>>  Extent Hit Ratio: 49 / 4171
>>>>>>
>>>>>>  Balancing F2FS Async:
>>>>>>    - nodes 6 in 141
>>>>>>    - dents 0 in dirs: 0
>>>>>>    - meta 13 in 346
>>>>>>    - NATs 16983 > 29120
>>>>>>    - SITs: 17
>>>>>>    - free_nids: 1861
>>>>>>
>>>>>>  Distribution of User Blocks: [ valid | invalid | free ]
>>>>>>    [-----------------------------------------------|-|--]
>>>>>>
>>>>>>  SSR: 1230719 blocks in 14834 segments
>>>>>>  LFS: 15150190 blocks in 29589 segments
>>>>>>
>>>>>>  BDF: 89, avg. vblocks: 949
>>>>>>
>>>>>>  Memory: 6754 KB = static: 4763 + cached: 1990
>>>>>>
>>>>>>  Please note that I tried to put all the archive and index files into the 
>>>>>> cold area using the file extensions mechanism.
>>>>>>  And indeed I saw numbers close to the total amount of segments in the 
>>>>>> cold area a couple of days ago.
>>>>>>  But now the cold area gets smaller very quickly. What am I doing wrong?
>>>>
>>>> How could you know the cold area is getting smaller? if it is looks as you 
>>>> said,
>>>> the behavior of f2fs seems not reasonable.
>>>
>>> I'm not sure about this. I just saw numbers after "COLD data: "  above to be 
>>> quite close to the total amount after "Main area:".
>>
>> Hmm.. number after "COLD data:" means No. of current cold segment, section and 
>> zone.
>>
>>> This is quite reasonable because I tried to the video files into the cold 
>>> area. So the cold area should take almost all the device.
>>> The idea is to separate video files from SQLite database writes. I got to 
>>> this idea while reading some docs on f2fs.
>>
>> I think this would be a good idea if SQLite db files and idx&video file have
>> different updating frequency.
> 
> Yes, their times of life are quite different probably. I don't know exactly how
> SQLite manages its db files. I think it's not append only. So I thought that it
> would be a right idea to separate the other files, that I'm in total control
> of, from SQLite db file.
> 
> My video archive and index files are written in append-only manner. They are
> never overwritten. They can only be deleted when the archive is rotated.

Will video archive and index file be appended to size which is unaligned to 4k,
and later, be appended continuously?

> Per my understanding of f2fs internals, it should write these "cold" files and
> usual "hot" files to different sections (that should map internally to
> different allocation units). So the sections used by "cold" data should almost
> never get "dirty" because most of the time all their blocks become free at
> the same time. Of course, the files are not exactly 4MB in size so the last
> section of the deleted file will become dirty. If it is moved by garbage
> collector and becomes mixed with fresh "cold" data, then indeed it might cause
> some problems, I think. What is your opinion?

If your fs is not fragmented, it may be as what you said, otherwise, SSR will
still try to reuse invalid block of other temperture segments, then your cold
data will be fixed with warm data too.

I guess, what you are facing is the latter one:
SSR: 1230719 blocks in 14834 segments

Maybe we can try to alter updating policy from OPU to IPU for your case to avoid
performance regression of SSR and more frequently FG-GC:

echo 1 > /sys/fs/f2fs/"yourdevicename"/ipu_policy

Thanks,

> 
> Maybe disabling garbage collection for "cold" data may help?
> 
>>>>>>  Can this be fixed by using a backport from 
>>>>>> git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-stable.git ?
>>>>>>  Also I measured the SD card erase block size using flashbench. It seems 
>>>>>> it is 8MB, not 4MB, as I used here.
>>>>>>  Can this lead to such serious problems? Is 8MB block safe to hardcode or 
>>>>>> should I use flashbench every time?
>>>>
>>>> I think 4MB is OK, if we set section size to 8MB, it will make us 
>>>> encountering
>>>> long latency of most operation due to foreground GC in where we may move more
>>>> blocks in one section.
>>>
>>> I see. I thought I have to align the section to the internal flash erase 
>>> block size.
>>> Actually, my problem is the increased latency after several days of rotating 
>>> the archive a 95% utilization.
>>
>>   - Valid: 15671
>>   - Dirty: 12904
>>   - Prefree: 0
>>   - Free: 1071 (27)
>>
>> CP calls: 3320 (BG: 0)
>> GC calls: 2240 (BG: 1)
>>   - data segments : 3866 (1236)
>>   - node segments : 243 (0)
>>
>> I can see that in your partition there are lots of dirty segments, and few free
>> segment, in this statement, we will be more likely to trigger synchronously
>> foreground garbage collection which may lead to long latency. Also, it is
>> indicated in you "GC calls" statement.
> 
> Yes, I see.
> 
>> Can you do synchronously GC through F2FS_IOC_GARBAGE_COLLECT ioctl with
>> parameter @sync which value is set as 1 until free segments size is more close
>> to free space FS show to user? I expect this can help to recover performance in
>> your enviornment.
>>
>> If this does not help, I think we should do some trace in order to find out the
>> root cause of write delay.
> 
> I'll try the ioctl, thanks!
> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> 

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Chao,

Thanks for your continued interest!

17.08.2016, 18:54, "Chao Yu" <chao@kernel.org>:
> Hi Alexander,
>
> On 2016/8/17 17:47, Alexander Gordeev wrote:
>>  Hi Chao,
>>
>>>  On 2016/8/15 20:22, Alexander Gordeev wrote:
>>>>  15.08.2016, 14:58, "Chao Yu" <yuch...@huawei.com>:
>>>>>  Hi Alexander,
>>>>>
>>>>>  On 2016/8/15 18:47, Alexander Gordeev wrote:
>>>>>>   Hi All,
>>>>>>
>>>>>>   12.08.2016, 14:52, "Alexander Gordeev" <a...@gordick.net>:
>>>>>>>   Hi All,
>>>>>>>
>>>>>>>   I hope I'm writing to the right mailing list. If not please give me
>>>>>>>  directions to the right place.
>>>>>>>   I'm trying to write video archive to a microSD card on an ARM based IP
>>>>>>>  camera. The camera's SDK uses Linux 3.10.
>>>>>>>   The kernel is quite old and F2FS is there since 3.8 AFAIK so it's
>>>>>>>  probably not mature yet. However, I decided to give it a try.
>>>>>>>   The idea is to write video continuously in 5 minute chunks. I also have
>>>>>>>  an index file per each archive chunk file to for faster seeks and a single
>>>>>>>  SQLite database.
>>>>>>>   When utilization is about 95%, the chunks and their indexes from the
>>>>>>>  archive tail are deleted. So it's like a ring buffer. Also the
>>>>>>>  overprovision ratio is the default 5%.
>>>>>>>   It worked quite good for several days with about 95% utilization, but
>>>>>>>  then today it went bad. Writes are taking several seconds quite often as
>>>>>>>  shown by strace.
>>>>>>>   vmstat shows that my process waits for IO most of the time:
>>>>>>>
>>>>>>>   procs -----------memory---------- ---swap-- -----io---- -system--
>>>>>>>  ----cpu----
>>>>>>>    r b swpd free buff cache si so bi bo in cs us sy id wa
>>>>>>>    2 2 0 6352 4 35924 0 0 8 88 562 1316 41 7 0 52
>>>>>>>    1 2 0 6324 4 35928 0 0 4 44 553 1231 40 8 0 52
>>>>>>>    1 2 0 6324 4 35928 0 0 0 0 690 1471 36 10 0 54
>>>>>>>    1 3 0 6296 4 35932 0 0 0 0 530 1242 40 5 0 54
>>>>>>>    2 2 0 6296 4 35936 0 0 4 48 545 1244 40 6 0 54
>>>>>>>    1 2 0 6296 4 35940 0 0 4 44 549 1275 39 6 0 55
>>>>>>>    2 2 0 6288 4 35944 0 0 4 44 563 1315 39 8 0 53
>>>>>>>    3 2 0 6296 4 35944 0 0 0 0 502 1158 41 2 0 57
>>>>>>>    1 3 0 6296 4 35952 0 0 8 88 700 1527 40 9 0 51
>>>>>>>    1 2 0 6296 4 35952 0 0 0 0 482 1141 38 8 0 55
>>>>>>>    1 2 0 6296 4 35956 0 0 4 44 594 1383 38 13 0 49
>>>>>>>    1 2 0 6296 4 35956 0 0 0 0 489 1160 37 5 0 58
>>>>>>>    5 1 0 6268 4 35980 0 0 12 132 704 1565 42 9 0 49
>>>>>>>    2 1 0 6268 4 35984 0 0 4 44 531 1215 39 10 0 51
>>>>>>>    3 2 0 6268 4 35992 0 0 8 92 714 1574 36 9 0 55
>>>>>>>    1 1 0 6268 4 35992 0 0 0 0 485 1163 39 6 0 55
>>>>>>>    1 1 0 6240 4 36000 0 0 8 92 553 1282 38 9 0 53
>>>>>>>    1 2 0 6240 4 36000 0 0 0 0 488 1135 39 7 0 54
>>>>>>>    1 1 0 6240 4 36000 0 0 0 0 552 1264 39 9 0 52
>>>>>>>    3 1 0 6240 4 36000 0 0 0 0 510 1187 40 6 0 54
>>>>>>>    1 2 0 6240 4 36004 0 0 4 44 674 1496 43 8 0 49
>>>>>>>    1 1 0 6240 4 36012 0 0 8 88 572 1373 39 9 0 53
>>>>>>>    4 1 0 6232 4 36016 0 0 4 48 549 1248 41 4 0 55
>>>>>>>    3 1 0 6240 4 36016 0 0 0 0 520 1209 36 8 0 55
>>>>>>>
>>>>>>>   Here is also /sys/kernel/debug/f2fs/status for reference:
>>>>>>>   =====[ partition info(sda). #0 ]=====
>>>>>>>   [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529
>>>>>>>  Resv:50)]
>>>>>>>
>>>>>>>   Utilization: 94% (13597314 valid blocks)
>>>>>>>     - Node: 16395 (Inode: 2913, Other: 13482)
>>>>>>>     - Data: 13580919
>>>>>>>
>>>>>>>   Main area: 29646 segs, 14823 secs 14823 zones
>>>>>>>     - COLD data: 3468, 1734, 1734
>>>>>>>     - WARM data: 12954, 6477, 6477
>>>>>>>     - HOT data: 28105, 14052, 14052
>>>>>>>     - Dir dnode: 29204, 14602, 14602
>>>>>>>     - File dnode: 19960, 9980, 9980
>>>>>>>     - Indir nodes: 29623, 14811, 14811
>>>>>>>
>>>>>>>     - Valid: 13615
>>>>>>>     - Dirty: 13309
>>>>>>>     - Prefree: 0
>>>>>>>     - Free: 2722 (763)
>>>>>>>
>>>>>>>   GC calls: 8622 (BG: 4311)
>>>>>>>     - data segments : 8560
>>>>>>>     - node segments : 62
>>>>>>>   Try to move 3552161 blocks
>>>>>>>     - data blocks : 3540278
>>>>>>>     - node blocks : 11883
>>>>>>>
>>>>>>>   Extent Hit Ratio: 49 / 4171
>>>>>>>
>>>>>>>   Balancing F2FS Async:
>>>>>>>     - nodes 6 in 141
>>>>>>>     - dents 0 in dirs: 0
>>>>>>>     - meta 13 in 346
>>>>>>>     - NATs 16983 > 29120
>>>>>>>     - SITs: 17
>>>>>>>     - free_nids: 1861
>>>>>>>
>>>>>>>   Distribution of User Blocks: [ valid | invalid | free ]
>>>>>>>     [-----------------------------------------------|-|--]
>>>>>>>
>>>>>>>   SSR: 1230719 blocks in 14834 segments
>>>>>>>   LFS: 15150190 blocks in 29589 segments
>>>>>>>
>>>>>>>   BDF: 89, avg. vblocks: 949
>>>>>>>
>>>>>>>   Memory: 6754 KB = static: 4763 + cached: 1990
>>>>>>>
>>>>>>>   Please note that I tried to put all the archive and index files into the
>>>>>>>  cold area using the file extensions mechanism.
>>>>>>>   And indeed I saw numbers close to the total amount of segments in the
>>>>>>>  cold area a couple of days ago.
>>>>>>>   But now the cold area gets smaller very quickly. What am I doing wrong?
>>>>>
>>>>>  How could you know the cold area is getting smaller? if it is looks as you
>>>>>  said,
>>>>>  the behavior of f2fs seems not reasonable.
>>>>
>>>>  I'm not sure about this. I just saw numbers after "COLD data: " above to be
>>>>  quite close to the total amount after "Main area:".
>>>
>>>  Hmm.. number after "COLD data:" means No. of current cold segment, section and
>>>  zone.
>>>
>>>>  This is quite reasonable because I tried to the video files into the cold
>>>>  area. So the cold area should take almost all the device.
>>>>  The idea is to separate video files from SQLite database writes. I got to
>>>>  this idea while reading some docs on f2fs.
>>>
>>>  I think this would be a good idea if SQLite db files and idx&video file have
>>>  different updating frequency.
>>
>>  Yes, their times of life are quite different probably. I don't know exactly how
>>  SQLite manages its db files. I think it's not append only. So I thought that it
>>  would be a right idea to separate the other files, that I'm in total control
>>  of, from SQLite db file.
>>
>>  My video archive and index files are written in append-only manner. They are
>>  never overwritten. They can only be deleted when the archive is rotated.
>
> Will video archive and index file be appended to size which is unaligned to 4k,
> and later, be appended continuously?

Well, the actual write() system calls are not aligned to 4k and the size can be quite
random. But I think the job of making aligned writes to the hardware is done by FS
buffers. The archive files are written at a steady rate of about 1.2Mb/s so in my
understanding all their blocks should be ready before they are written to the card.
The index files are written at a much lower rate of about 8Kb/s. But this should be
still enough to always fill their blocks before they are written to the card.

Some sysctl settings from my embedded system:
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 20
vm.dirty_writeback_centisecs = 500
Total amount of RAM is 106736 kB.

>>  Per my understanding of f2fs internals, it should write these "cold" files and
>>  usual "hot" files to different sections (that should map internally to
>>  different allocation units). So the sections used by "cold" data should almost
>>  never get "dirty" because most of the time all their blocks become free at
>>  the same time. Of course, the files are not exactly 4MB in size so the last
>>  section of the deleted file will become dirty. If it is moved by garbage
>>  collector and becomes mixed with fresh "cold" data, then indeed it might cause
>>  some problems, I think. What is your opinion?
>
> If your fs is not fragmented, it may be as what you said, otherwise, SSR will
> still try to reuse invalid block of other temperture segments, then your cold
> data will be fixed with warm data too.
>
> I guess, what you are facing is the latter one:
> SSR: 1230719 blocks in 14834 segments

I guess, I need to somehow disable any cleaning or SSR for my archive and index
files. But keep the cleaning for other data and nodes.
I think the FS can get fragmented quite easily otherwise. The status above is
captured when the FS already has problems. I think it can become fragmented
this way:
1. The archive is written until the utilization is 95%. It is written separately from other
data and nodes thanks to the "cold" data feature.
2. After hitting 95% the archive my program starts to rotate the archive. The rotation
routine checks the free space, reported by statfs(), once a minute. If it is below 5%
of total, then it deletes several oldest records in the archive.
3. The last deleted record leaves a dirty section. This section holds several blocks
from a record, which now becomes the oldest one.
4. This section is merged with fresh "cold" or even warmer data by either GC, or
SSR in one or more newly used sections.
5. Then very soon the new oldest record is again deleted. And now we have one
or even several dirty sections filled with blocks from a not so old record. Which are
again merged with other records.
6. All the records get fragmented after one full rotation. The fragmentation gets
worse and worse.

So I think the best thing to do is to have sections with "cold" data be completely
out of all the cleaning schemes. It will clean itself by rotating.
Still other data and nodes might need to use some cleaning schemes.
Please correct me if I don't get it right.

> Maybe we can try to alter updating policy from OPU to IPU for your case to avoid
> performance regression of SSR and more frequently FG-GC:
>
> echo 1 > /sys/fs/f2fs/"yourdevicename"/ipu_policy

Thanks, I'll try it!

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Jaegeuk Kim]

Hello,

On Thu, Aug 18, 2016 at 02:04:55PM +0300, Alexander Gordeev wrote:

...

> >>>>>>>   Here is also /sys/kernel/debug/f2fs/status for reference:
> >>>>>>>   =====[ partition info(sda). #0 ]=====
> >>>>>>>   [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529
> >>>>>>>  Resv:50)]
> >>>>>>>
> >>>>>>>   Utilization: 94% (13597314 valid blocks)
> >>>>>>>     - Node: 16395 (Inode: 2913, Other: 13482)
> >>>>>>>     - Data: 13580919
> >>>>>>>
> >>>>>>>   Main area: 29646 segs, 14823 secs 14823 zones
> >>>>>>>     - COLD data: 3468, 1734, 1734
> >>>>>>>     - WARM data: 12954, 6477, 6477
> >>>>>>>     - HOT data: 28105, 14052, 14052
> >>>>>>>     - Dir dnode: 29204, 14602, 14602
> >>>>>>>     - File dnode: 19960, 9980, 9980
> >>>>>>>     - Indir nodes: 29623, 14811, 14811
> >>>>>>>
> >>>>>>>     - Valid: 13615
> >>>>>>>     - Dirty: 13309
> >>>>>>>     - Prefree: 0
> >>>>>>>     - Free: 2722 (763)
> >>>>>>>
> >>>>>>>   GC calls: 8622 (BG: 4311)
> >>>>>>>     - data segments : 8560
> >>>>>>>     - node segments : 62
> >>>>>>>   Try to move 3552161 blocks
> >>>>>>>     - data blocks : 3540278
> >>>>>>>     - node blocks : 11883
> >>>>>>>
> >>>>>>>   Extent Hit Ratio: 49 / 4171
> >>>>>>>
> >>>>>>>   Balancing F2FS Async:
> >>>>>>>     - nodes 6 in 141
> >>>>>>>     - dents 0 in dirs: 0
> >>>>>>>     - meta 13 in 346
> >>>>>>>     - NATs 16983 > 29120
> >>>>>>>     - SITs: 17
> >>>>>>>     - free_nids: 1861
> >>>>>>>
> >>>>>>>   Distribution of User Blocks: [ valid | invalid | free ]
> >>>>>>>     [-----------------------------------------------|-|--]
> >>>>>>>
> >>>>>>>   SSR: 1230719 blocks in 14834 segments
> >>>>>>>   LFS: 15150190 blocks in 29589 segments
> >>>>>>>
> >>>>>>>   BDF: 89, avg. vblocks: 949
> >>>>>>>
> >>>>>>>   Memory: 6754 KB = static: 4763 + cached: 1990

...

> >>  Per my understanding of f2fs internals, it should write these "cold" files and
> >>  usual "hot" files to different sections (that should map internally to
> >>  different allocation units). So the sections used by "cold" data should almost
> >>  never get "dirty" because most of the time all their blocks become free at
> >>  the same time. Of course, the files are not exactly 4MB in size so the last
> >>  section of the deleted file will become dirty. If it is moved by garbage
> >>  collector and becomes mixed with fresh "cold" data, then indeed it might cause
> >>  some problems, I think. What is your opinion?
> >
> > If your fs is not fragmented, it may be as what you said, otherwise, SSR will
> > still try to reuse invalid block of other temperture segments, then your cold
> > data will be fixed with warm data too.
> >
> > I guess, what you are facing is the latter one:
> > SSR: 1230719 blocks in 14834 segments
> 
> I guess, I need to somehow disable any cleaning or SSR for my archive and index
> files. But keep the cleaning for other data and nodes.

Could you test a mount option, "mode=lfs", to disable SSR?
(I guess sqlite may suffer from logner latency due to GC though.)

Seems like it's caused by SSR starting to make worse before 95% as you described
below.

Thanks,

> I think the FS can get fragmented quite easily otherwise. The status above is
> captured when the FS already has problems. I think it can become fragmented
> this way:
> 1. The archive is written until the utilization is 95%. It is written separately from other
> data and nodes thanks to the "cold" data feature.
> 2. After hitting 95% the archive my program starts to rotate the archive. The rotation
> routine checks the free space, reported by statfs(), once a minute. If it is below 5%
> of total, then it deletes several oldest records in the archive.
> 3. The last deleted record leaves a dirty section. This section holds several blocks
> from a record, which now becomes the oldest one.
> 4. This section is merged with fresh "cold" or even warmer data by either GC, or
> SSR in one or more newly used sections.
> 5. Then very soon the new oldest record is again deleted. And now we have one
> or even several dirty sections filled with blocks from a not so old record. Which are
> again merged with other records.
> 6. All the records get fragmented after one full rotation. The fragmentation gets
> worse and worse.
> 
> So I think the best thing to do is to have sections with "cold" data be completely
> out of all the cleaning schemes. It will clean itself by rotating.
> Still other data and nodes might need to use some cleaning schemes.
> Please correct me if I don't get it right.
> 
> > Maybe we can try to alter updating policy from OPU to IPU for your case to avoid
> > performance regression of SSR and more frequently FG-GC:
> >
> > echo 1 > /sys/fs/f2fs/"yourdevicename"/ipu_policy
> 
> Thanks, I'll try it!
> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

19.08.2016, 05:41, "Jaegeuk Kim" <jaegeuk@kernel.org>:
>  Hello,
>
>  On Thu, Aug 18, 2016 at 02:04:55PM +0300, Alexander Gordeev wrote:
>
>  ...
>
>>   >>>>>>>   Here is also /sys/kernel/debug/f2fs/status for reference:
>>   >>>>>>>   =====[ partition info(sda). #0 ]=====
>>   >>>>>>>   [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529
>>   >>>>>>>  Resv:50)]
>>   >>>>>>>
>>   >>>>>>>   Utilization: 94% (13597314 valid blocks)
>>   >>>>>>>     - Node: 16395 (Inode: 2913, Other: 13482)
>>   >>>>>>>     - Data: 13580919
>>   >>>>>>>
>>   >>>>>>>   Main area: 29646 segs, 14823 secs 14823 zones
>>   >>>>>>>     - COLD data: 3468, 1734, 1734
>>   >>>>>>>     - WARM data: 12954, 6477, 6477
>>   >>>>>>>     - HOT data: 28105, 14052, 14052
>>   >>>>>>>     - Dir dnode: 29204, 14602, 14602
>>   >>>>>>>     - File dnode: 19960, 9980, 9980
>>   >>>>>>>     - Indir nodes: 29623, 14811, 14811
>>   >>>>>>>
>>   >>>>>>>     - Valid: 13615
>>   >>>>>>>     - Dirty: 13309
>>   >>>>>>>     - Prefree: 0
>>   >>>>>>>     - Free: 2722 (763)
>>   >>>>>>>
>>   >>>>>>>   GC calls: 8622 (BG: 4311)
>>   >>>>>>>     - data segments : 8560
>>   >>>>>>>     - node segments : 62
>>   >>>>>>>   Try to move 3552161 blocks
>>   >>>>>>>     - data blocks : 3540278
>>   >>>>>>>     - node blocks : 11883
>>   >>>>>>>
>>   >>>>>>>   Extent Hit Ratio: 49 / 4171
>>   >>>>>>>
>>   >>>>>>>   Balancing F2FS Async:
>>   >>>>>>>     - nodes 6 in 141
>>   >>>>>>>     - dents 0 in dirs: 0
>>   >>>>>>>     - meta 13 in 346
>>   >>>>>>>     - NATs 16983 > 29120
>>   >>>>>>>     - SITs: 17
>>   >>>>>>>     - free_nids: 1861
>>   >>>>>>>
>>   >>>>>>>   Distribution of User Blocks: [ valid | invalid | free ]
>>   >>>>>>>     [-----------------------------------------------|-|--]
>>   >>>>>>>
>>   >>>>>>>   SSR: 1230719 blocks in 14834 segments
>>   >>>>>>>   LFS: 15150190 blocks in 29589 segments
>>   >>>>>>>
>>   >>>>>>>   BDF: 89, avg. vblocks: 949
>>   >>>>>>>
>>   >>>>>>>   Memory: 6754 KB = static: 4763 + cached: 1990
>
>  ...
>
>>   >>  Per my understanding of f2fs internals, it should write these "cold" files and
>>   >>  usual "hot" files to different sections (that should map internally to
>>   >>  different allocation units). So the sections used by "cold" data should almost
>>   >>  never get "dirty" because most of the time all their blocks become free at
>>   >>  the same time. Of course, the files are not exactly 4MB in size so the last
>>   >>  section of the deleted file will become dirty. If it is moved by garbage
>>   >>  collector and becomes mixed with fresh "cold" data, then indeed it might cause
>>   >>  some problems, I think. What is your opinion?
>>   >
>>   > If your fs is not fragmented, it may be as what you said, otherwise, SSR will
>>   > still try to reuse invalid block of other temperture segments, then your cold
>>   > data will be fixed with warm data too.
>>   >
>>   > I guess, what you are facing is the latter one:
>>   > SSR: 1230719 blocks in 14834 segments
>>
>>   I guess, I need to somehow disable any cleaning or SSR for my archive and index
>>   files. But keep the cleaning for other data and nodes.
>
>  Could you test a mount option, "mode=lfs", to disable SSR?
>  (I guess sqlite may suffer from logner latency due to GC though.)
>
>  Seems like it's caused by SSR starting to make worse before 95% as you described
>  below.

Thanks, I'll run a test with a couple of SD cards over weekend.
So if I understand it correctly, GC will not cause the problems described below, right?
I.e. it will not mix the new data with old data from dirty sections?
Longer SQLite latencies should not be a problem because the database is written not
frequently and also it is about 200-250KB in size usually. Maybe forcing IPU as
suggested by Chao would help sqlite, no?
However looks like setting ipu_policy to 1 has no effect when mode=lfs.
The IPU counter is still zero on my test system.

>>   I think the FS can get fragmented quite easily otherwise. The status above is
>>   captured when the FS already has problems. I think it can become fragmented
>>   this way:
>>   1. The archive is written until the utilization is 95%. It is written separately from other
>>   data and nodes thanks to the "cold" data feature.
>>   2. After hitting 95% the archive my program starts to rotate the archive. The rotation
>>   routine checks the free space, reported by statfs(), once a minute. If it is below 5%
>>   of total, then it deletes several oldest records in the archive.
>>   3. The last deleted record leaves a dirty section. This section holds several blocks
>>   from a record, which now becomes the oldest one.
>>   4. This section is merged with fresh "cold" or even warmer data by either GC, or
>>   SSR in one or more newly used sections.
>>   5. Then very soon the new oldest record is again deleted. And now we have one
>>   or even several dirty sections filled with blocks from a not so old record. Which are
>>   again merged with other records.
>>   6. All the records get fragmented after one full rotation. The fragmentation gets
>>   worse and worse.
>>
>>   So I think the best thing to do is to have sections with "cold" data be completely
>>   out of all the cleaning schemes. It will clean itself by rotating.
>>   Still other data and nodes might need to use some cleaning schemes.
>>   Please correct me if I don't get it right.
>>
>>   > Maybe we can try to alter updating policy from OPU to IPU for your case to avoid
>>   > performance regression of SSR and more frequently FG-GC:
>>   >
>>   > echo 1 > /sys/fs/f2fs/"yourdevicename"/ipu_policy
>>
>>   Thanks, I'll try it!

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Alexander Gordeev]

Hi,

I'd also like to report two bugs. I used f2fs-stable branch linux-3.10.y.
I'm using some proprietary modules also, but in my understanding they shouldn't affect the fs.

1. I have a Sandisk 16GB microSDHC card, that is quite old and may be at its end of life.
The bug is produced every time I try to delete any file on it.

[   28.805413] ------------[ cut here ]------------
[   28.810327] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1103 new_curseg+0x330/0x40c [f2fs]()
[   28.824971] Modules linked in:
[   28.828128]  ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block evdev bcmdhd(O) amb
arella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc_core ohci_hcd ehc
i_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regmap_i2c snd_pcm snd
_page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
[   28.891640] CPU: 0 PID: 489 Comm: videoserverd Tainted: P           O 3.10.93 #1
[   28.899128] [<8001338c>] (unwind_backtrace+0x0/0x12c) from [<80011a0c>] (show_stack+0x10/0x14)
[   28.907795] [<80011a0c>] (show_stack+0x10/0x14) from [<8001effc>] (warn_slowpath_common+0x54/0x68)
[   28.916777] [<8001effc>] (warn_slowpath_common+0x54/0x68) from [<8001f0ac>] (warn_slowpath_null+0x1c/0x24)
[   28.926531] [<8001f0ac>] (warn_slowpath_null+0x1c/0x24) from [<7f51c220>] (new_curseg+0x330/0x40c [f2fs])
[   28.936364] [<7f51c220>] (new_curseg+0x330/0x40c [f2fs]) from [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs])
[   28.947594] [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs]) from [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs])
[   28.959624] [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
[   28.970374] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
[   28.980595] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
[   28.991328] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
[   29.002072] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
[   29.012652] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
[   29.022684] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
[   29.032013] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
[   29.040940] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
[   29.049402] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
[   29.058186] ---[ end trace d249769884543c44 ]---
[   29.062825] ------------[ cut here ]------------
[   29.067523] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1142 new_curseg+0x3e4/0x40c [f2fs]()
[   29.082113] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block e
vdev bcmdhd(O) ambarella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc
_core ohci_hcd ehci_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regm
ap_i2c snd_pcm snd_page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
[   29.147142] CPU: 0 PID: 489 Comm: videoserverd Tainted: P        W  O 3.10.93 #1
[   29.154564] [<8001338c>] (unwind_backtrace+0x0/0x12c) from [<80011a0c>] (show_stack+0x10/0x14)
[   29.163211] [<80011a0c>] (show_stack+0x10/0x14) from [<8001effc>] (warn_slowpath_common+0x54/0x68)
[   29.172184] [<8001effc>] (warn_slowpath_common+0x54/0x68) from [<8001f0ac>] (warn_slowpath_null+0x1c/0x24)
[   29.181887] [<8001f0ac>] (warn_slowpath_null+0x1c/0x24) from [<7f51c2d4>] (new_curseg+0x3e4/0x40c [f2fs])
[   29.191508] [<7f51c2d4>] (new_curseg+0x3e4/0x40c [f2fs]) from [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs])
[   29.202785] [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs]) from [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs])
[   29.214742] [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
[   29.225477] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
[   29.235822] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
[   29.246575] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
[   29.257220] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
[   29.267850] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
[   29.277887] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
[   29.287127] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
[   29.296045] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
[   29.304504] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
[   29.313271] ---[ end trace d249769884543c45 ]---
[   29.371882] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[   29.396142] pgd = 85644000
[   29.399051] [00000000] *pgd=0357b831, *pte=00000000, *ppte=00000000
[   29.407041] Internal error: Oops: 17 [#1] PREEMPT ARM
[   29.412085] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block e
vdev bcmdhd(O) ambarella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc
_core ohci_hcd ehci_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regm
ap_i2c snd_pcm snd_page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
[   29.476936] CPU: 0 PID: 489 Comm: videoserverd Tainted: P        W  O 3.10.93 #1
[   29.484309] task: 869fdc80 ti: 83350000 task.ti: 83350000
[   29.489728] PC is at update_sit_entry+0xc4/0x250 [f2fs]
[   29.494949] LR is at update_sit_entry+0x78/0x250 [f2fs]
[   29.500154] pc : [<7f51a74c>]    lr : [<7f51a700>]    psr: 20060013
sp : 83351c48  ip : 00000000  fp : 84a0f740
[   29.511600] r10: 00000000  r9 : 00001d74  r8 : 00000001
[   29.516800] r7 : 00000000  r6 : 00000080  r5 : 859ac2e0  r4 : 857e6800
[   29.523301] r3 : 00000007  r2 : 00153fae  r1 : 00000000  r0 : 00153fae
[   29.529805] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   29.536914] Control: 10c53c7d  Table: 05844059  DAC: 00000015
[   29.542639] Process videoserverd (pid: 489, stack limit = 0x83350238)
[   29.549054] Stack: (0x83351c48 to 0x83352000)
[   29.553394] 1c40:                   00000001 857e6800 003b5800 0034c075 00000004 00000001
[   29.561547] 1c60: 00000000 83351d38 80550080 7f51baf0 857e6800 86974cd0 86974cd0 7f51ccd8
[   29.569701] 1c80: 83351d2c 84a0f740 00000000 84a0f768 0034c075 83351d01 00000000 800d0d94
[   29.577853] 1ca0: 00000000 000000f0 60060013 7f5103cc 86455e00 83351d2c 00000004 80550080
[   29.586005] 1cc0: 0000010c 83351d01 857e6938 000006b0 857e6800 7f51cf84 83351d01 00000004
[   29.594156] 1ce0: 857e6800 00000000 83351d70 80550080 804ce684 000006b0 857e6800 7f51d128
[   29.602309] 1d00: 0006b080 00000000 857e6938 7f515560 00000064 8006ddd8 8054b8f4 000006b0
[   29.610461] 1d20: 000006b0 0034c075 86390d08 857e6800 00000001 00000411 003b5800 0034c075
[   29.618612] 1d40: 80550080 00000000 80550080 857e6800 00000200 857e6a80 861c2333 00000075
[   29.626765] 1d60: 857e6800 7f516af0 000006b0 00000001 00000001 00000000 00000000 00000000
[   29.634916] 1d80: 00000000 00000000 00000001 00000000 861c2333 80550080 00000000 7f50e33c
[   29.643069] 1da0: 00000001 83351e00 83351eb0 00001a28 000006b0 85795a50 83350000 00000000
[   29.651220] 1dc0: 000273c0 00000000 00001a2a 805a0840 00000001 861c2000 00000001 83351e8c
[   29.659372] 1de0: 0034c000 857e6804 00001a28 83350000 83351e08 000273c0 000003fc 83351e88
[   29.667525] 1e00: 84a0f6c0 84a0f7c0 91827364 83351e0c 83351e0c 83351e14 83351e14 000006b0
[   29.675676] 1e20: 000006b0 0034c075 00000008 00000400 00000000 857e6800 83351e8c 00000000
[   29.683828] 1e40: 00000000 00000000 857e6a80 00000400 00000000 857e6800 83351e8c 7f50e7ec
[   29.691979] 1e60: 00000000 ffffffea 00000000 83351ef4 00000000 00000000 00000000 00000000
[   29.700131] 1e80: 857e6a50 857e6a60 00001a28 83351e8c 83351e8c 00000000 00000050 00000000
[   29.708282] 1ea0: 00000002 84d43950 000000d6 863d83d8 863d83ec 00000004 00000000 00000000
[   29.716434] 1ec0: 00000000 00000000 00000000 857e6800 86390568 00000000 857e692c 84d43034
[   29.724586] 1ee0: 863dc720 863dc720 01da6a24 7f503e70 863d83c0 80577860 00000000 86390568
[   29.732738] 1f00: 863d83c0 00000000 01da9a34 ffffff9c 863d83c0 800ae7f8 01da6a24 000bda58
[   29.740890] 1f20: 00000000 00000000 8699e000 800ae9b0 85795a50 863acaa8 6fe7deef 00000004
[   29.749042] 1f40: 8699e027 ffffff9c 00000000 86214d50 86390568 00000000 00000002 00000000
[   29.757194] 1f60: 00000000 00000000 00000000 00000000 00000000 00000000 57b1c919 3ace5a0f
[   29.765346] 1f80: 57b1c919 3ace5a0f 01da9a34 00418a80 7eee3978 0000000a 8000ee44 83350000
[   29.773498] 1fa0: 00000000 8000ecc0 01da9a34 00418a80 01da9a34 7eee3838 81b40000 000081b4
[   29.781650] 1fc0: 01da9a34 00418a80 7eee3978 0000000a 01da6ef0 01dac120 7eee3978 01da6a24
[   29.789802] 1fe0: 76c49a71 7eee382c 76c49a79 76ca6926 800d0030 01da9a34 00000000 00000000
[   29.798018] [<7f51a74c>] (update_sit_entry+0xc4/0x250 [f2fs]) from [<7f51baf0>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
[   29.808554] [<7f51baf0>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f51ccd8>] (allocate_data_block+0x2e8/0x3d8 [f2fs])
[   29.819429] [<7f51ccd8>] (allocate_data_block+0x2e8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
[   29.830129] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
[   29.840312] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
[   29.851017] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
[   29.861630] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
[   29.872157] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
[   29.882157] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
[   29.891373] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
[   29.900231] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
[   29.908672] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
[   29.917434] Code: e2063007 e3a06001 e1a06316 e1a071aa (e7d121aa) 
[   29.974780] ---[ end trace d249769884543c46 ]---


2. I tried to run manual defragmentation of all the files on an almost new
Transcend 128GB microSDXC card to fix the archive. Unfortunately this bug
appeared after successful defragmentation of several files:

[ 5819.169892] ------------[ cut here ]------------
[ 5819.174668] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1087 new_curseg+0x150/0x3b0 [f2fs]()
[ 5819.189524] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage gpio_keys v
[ 5819.251573] CPU: 0 PID: 1363 Comm: f2fstool Tainted: P           O 3.10.93 #1
[ 5819.258793] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
[ 5819.267471] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
[ 5819.276445] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
[ 5819.286294] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f47f530>] (new_curseg+0x150/0x3b0 [f2fs])
[ 5819.296041] [<7f47f530>] (new_curseg+0x150/0x3b0 [f2fs]) from [<7f47f9dc>] (allocate_segment_by_default+0xa4/0x270 [f2fs])
[ 5819.307317] [<7f47f9dc>] (allocate_segment_by_default+0xa4/0x270 [f2fs]) from [<7f480114>] (allocate_data_block+0x310/0x318 [f2fs])
[ 5819.319257] [<7f480114>] (allocate_data_block+0x310/0x318 [f2fs]) from [<7f480340>] (do_write_page+0x224/0x270 [f2fs])
[ 5819.330071] [<7f480340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs])
[ 5819.340372] [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs])
[ 5819.351070] [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs])
[ 5819.362209] [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs]) from [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs])
[ 5819.372382] [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs]) from [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs])
[ 5819.382756] [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs]) from [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
[ 5819.394361] [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
[ 5819.405599] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<80066620>] (filemap_fdatawrite+0x24/0x2c)
[ 5819.415810] [<80066620>] (filemap_fdatawrite+0x24/0x2c) from [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs])
[ 5819.426445] [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs]) from [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs])
[ 5819.437202] [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs]) from [<8009fb64>] (vfs_ioctl+0x28/0x3c)
[ 5819.446105] [<8009fb64>] (vfs_ioctl+0x28/0x3c) from [<800a058c>] (do_vfs_ioctl+0x484/0x56c)
[ 5819.454463] [<800a058c>] (do_vfs_ioctl+0x484/0x56c) from [<800a06ac>] (SyS_ioctl+0x38/0x58)
[ 5819.462882] [<800a06ac>] (SyS_ioctl+0x38/0x58) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
[ 5819.471311] ---[ end trace 7e587b920f507a04 ]---
[ 5819.475939] ------------[ cut here ]------------
[ 5819.480611] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1142 new_curseg+0x300/0x3b0 [f2fs]()
[ 5819.495227] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage gpio_keys v
[ 5819.557163] CPU: 0 PID: 1363 Comm: f2fstool Tainted: P        W  O 3.10.93 #1
[ 5819.564332] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
[ 5819.572959] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
[ 5819.581946] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
[ 5819.591707] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f47f6e0>] (new_curseg+0x300/0x3b0 [f2fs])
[ 5819.601375] [<7f47f6e0>] (new_curseg+0x300/0x3b0 [f2fs]) from [<7f47f9dc>] (allocate_segment_by_default+0xa4/0x270 [f2fs])
[ 5819.612548] [<7f47f9dc>] (allocate_segment_by_default+0xa4/0x270 [f2fs]) from [<7f480114>] (allocate_data_block+0x310/0x318 [f2fs])
[ 5819.624523] [<7f480114>] (allocate_data_block+0x310/0x318 [f2fs]) from [<7f480340>] (do_write_page+0x224/0x270 [f2fs])
[ 5819.635354] [<7f480340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs])
[ 5819.645642] [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs])
[ 5819.656337] [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs])
[ 5819.667505] [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs]) from [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs])
[ 5819.677682] [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs]) from [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs])
[ 5819.688086] [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs]) from [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
[ 5819.699547] irq_sync_frame_count(388): Add missing 2 frames to the frame queue! DSP(H264: 247189; MJEPG: 0), IAV(Total: 247187)!
[ 5819.711303] [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
[ 5819.722548] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<80066620>] (filemap_fdatawrite+0x24/0x2c)
[ 5819.732771] [<80066620>] (filemap_fdatawrite+0x24/0x2c) from [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs])
[ 5819.743389] [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs]) from [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs])
[ 5819.754139] [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs]) from [<8009fb64>] (vfs_ioctl+0x28/0x3c)
[ 5819.763037] [<8009fb64>] (vfs_ioctl+0x28/0x3c) from [<800a058c>] (do_vfs_ioctl+0x484/0x56c)
[ 5819.771374] [<800a058c>] (do_vfs_ioctl+0x484/0x56c) from [<800a06ac>] (SyS_ioctl+0x38/0x58)
[ 5819.779759] [<800a06ac>] (SyS_ioctl+0x38/0x58) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
[ 5819.788228] ---[ end trace 7e587b920f507a05 ]---
[ 5819.816478] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[ 5819.839179] pgd = 82960000
[ 5819.842605] [00000000] *pgd=0084f831, *pte=00000000, *ppte=00000000
[ 5819.849403] Internal error: Oops: 17 [#1] PREEMPT ARM
[ 5819.854439] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage gpio_keys v
[ 5819.916258] CPU: 0 PID: 1363 Comm: f2fstool Tainted: P        W  O 3.10.93 #1
[ 5819.923373] task: 81d6f800 ti: 83350000 task.ti: 83350000
[ 5819.928824] PC is at update_sit_entry+0xf0/0x218 [f2fs]
[ 5819.934088] LR is at update_sit_entry+0xac/0x218 [f2fs]
[ 5819.939295] pc : [<7f47d454>]    lr : [<7f47d410>]    psr: 200f0013
[ 5819.939295] sp : 83351a20  ip : 00000001  fp : 8515a8c0
[ 5819.950741] r10: 00000001  r9 : ffffffff  r8 : a86023f8
[ 5819.955944] r7 : 0000e708  r6 : 00000080  r5 : 00000000  r4 : 8587cc00
[ 5819.962446] r3 : 00000001  r2 : 00000000  r1 : 00000000  r0 : 00174241
[ 5819.968951] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[ 5819.976062] Control: 10c53c7d  Table: 02b60059  DAC: 00000015
[ 5819.981785] Process f2fstool (pid: 1363, stack limit = 0x83350238)
[ 5819.987941] Stack: (0x83351a20 to 0x83352000)
[ 5819.992285] 1a20: 83351bd0 8587cc00 0040b455 01cff800 00000000 83351bc4 869ec468 8052e420
[ 5820.000443] 1a40: 861d36a8 7f47f00c 8587cc00 869ec468 00000002 7f48003c 8046a4a0 83351b14
[ 5820.008598] 1a60: 861d3680 83351bd0 0040b455 83351ac1 833936b4 000024c7 83351ac8 00000002
[ 5820.016752] 1a80: 83351bc4 83351ac1 00000000 83351bc4 00000001 00000001 00000000 7f480340
[ 5820.024907] 1aa0: 83351ac1 00000002 00000001 83351b14 83351bc4 8587cc00 8052e420 7f4804b8
[ 5820.033062] 1ac0: 0024c790 01520100 000024c7 0000249a 019c12dd 8052e401 8587cc00 8052e420
[ 5820.041216] 1ae0: 86540528 7f476ff4 8019cf50 680d0013 86540528 00000000 00000000 000024c7
[ 5820.049371] 1b00: 00000152 00000000 0040b455 8587cc00 8052e420 86540528 804a6380 8049ae20
[ 5820.057525] 1b20: 000024c7 00000152 00000000 0040b455 8285f253 8587cc00 8052e420 86540528
[ 5820.065679] 1b40: 8285f253 83350000 00000001 00000001 00000000 7f472bdc 00000001 00000210
[ 5820.073834] 1b60: 00001f66 00001f64 00000003 83351b80 804a9be0 00000000 0002f160 00000055
[ 5820.081989] 1b80: 8285f000 00001f68 8587cc04 00000000 0002f190 83351c24 85e71c00 0040b400
[ 5820.090144] 1ba0: 000024c7 0000249a 019c12dd 8272c001 91827364 83351bb4 83351bb4 83351bbc
[ 5820.098298] 1bc0: 83351bbc 8587cc00 00000000 00000411 01cff800 0040b455 8052e420 00000000
[ 5820.106451] 1be0: 00000004 8587cc00 0000001b 00000001 00000000 00000002 0000001b 00000000
[ 5820.114605] 1c00: 00000004 7f472fd4 00000001 00000000 00000000 00000000 0000001a 00000009
[ 5820.122759] 1c20: 00001f64 8658d260 8658d1f0 00000003 00000050 864395f1 00000002 00000001
[ 5820.130913] 1c40: 00001c2b 00000000 804c2d80 804c47c0 00001a94 00000206 00000000 003a3fff
[ 5820.139067] 1c60: 60060013 8587cc00 8052dd60 864536e0 00000001 83351da0 00000000 02572000
[ 5820.147221] 1c80: 00000000 7f47779c 0000000e 00000102 83351d20 83350008 8587cd2c 00000000
[ 5820.155376] 1ca0: 864c75b4 80066130 00000002 8587cc00 00000000 00000411 003a3fff 01324440
[ 5820.163530] 1cc0: 8052dd60 00000000 00002570 8645379c ffffffff 864536e0 00002570 00000001
[ 5820.171684] 1ce0: 8587cc00 8052dd60 83351da0 7f47424c 0000000e 00000002 81c5c000 ffffffff
[ 5820.179837] 1d00: 00000005 00000000 00002571 0000000e 83351d54 00000000 00000001 00000002
[ 5820.187992] 1d20: 0000257a 91827364 83351d28 83351d28 83351d30 83351d30 0000000e 00000000
[ 5820.196148] 1d40: 804e4840 8049eca0 804d7e80 8046db00 80463d80 8052dd60 8048b480 804ff940
[ 5820.204302] 1d60: 804e2ea0 804d8840 80471960 804777c0 805111a0 80475fc0 00000000 8645379c
[ 5820.212456] 1d80: 864536e0 00000000 00000000 00002600 00000200 00002400 00000000 800665f4
[ 5820.220609] 1da0: 7ffffe8e 00000000 00000000 00000000 ffffffff 7fffffff 00000001 00000000
[ 5820.228766] 1dc0: 8047b6e0 00002b0a 00000200 80066620 ffffffff 7fffffff 00000001 00021000
[ 5820.236921] 1de0: 8047b6e0 7f4639fc 02b09fff 00000000 83351ec0 00000200 00000800 00000000
[ 5820.245075] 1e00: 86453744 8006bd7c 00000006 000000a8 00000000 8008a218 00000012 80013f04
[ 5820.253229] 1e20: 804edd20 0132442e 00002600 000001e4 00000020 00000000 82748c00 864536e0
[ 5820.261383] 1e40: 82748c00 7e8a6a90 8587cc00 00000fff 00000000 83350000 00000000 7f4668f4
[ 5820.269537] 1e60: 76e94000 000000a8 82961db8 81d2e4d8 83350018 8007ee74 000000a2 000000a8
[ 5820.277692] 1e80: 00000000 600d0093 80450720 76f36000 000003b7 81dc8d80 8585f580 000000a8
[ 5820.285846] 1ea0: 82961db8 82960000 83350018 8007f054 82961db8 000000a8 83351fb0 000000a8
[ 5820.294000] 1ec0: 00000000 00000000 02b0a000 00000000 000000a8 800138e4 00000800 00000000
[ 5820.302154] 1ee0: 00000200 00001a88 000081b4 6eadc2c5 80403434 7e8a6a90 864536e0 82748c00
[ 5820.310308] 1f00: 0000f508 7e8a6a90 83350000 00000000 00000000 8009fb64 4030582a 800a058c
[ 5820.318464] 1f20: 57a7001f 3982fe52 57a7014a 1eb08998 57a7014a 1eb08998 00001a88 00000000
[ 5820.326618] 1f40: 7e8a6a18 7e8a6a18 00000000 80095854 00001a88 00000000 00800000 000081b4
[ 5820.334771] 1f60: 00000001 00000003 82748c00 00000000 0000f508 7e8a6a90 83350000 00000000
[ 5820.342924] 1f80: 00000000 800a06ac 00000003 00000000 00000000 00000000 00000000 00000036
[ 5820.351078] 1fa0: 8000e344 8000e1c0 00000000 00000000 00000003 0000f508 7e8a6a90 7e8a6a90
[ 5820.359231] 1fc0: 00000000 00000000 00000000 00000036 00000000 00000000 76fba000 00000000
[ 5820.367385] 1fe0: 76f37d91 7e8a6a84 000086f1 76f37d96 800d0030 00000003 5bb15ad7 ae252a72
[ 5820.375663] [<7f47d454>] (update_sit_entry+0xf0/0x218 [f2fs]) from [<7f47f00c>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
[ 5820.386266] [<7f47f00c>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f48003c>] (allocate_data_block+0x238/0x318 [f2fs])
[ 5820.397214] [<7f48003c>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f480340>] (do_write_page+0x224/0x270 [f2fs])
[ 5820.407981] [<7f480340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs])
[ 5820.418219] [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs])
[ 5820.428885] [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs])
[ 5820.439983] [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs]) from [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs])
[ 5820.450124] [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs]) from [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs])
[ 5820.460443] [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs]) from [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
[ 5820.471952] [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
[ 5820.483156] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<80066620>] (filemap_fdatawrite+0x24/0x2c)
[ 5820.493342] [<80066620>] (filemap_fdatawrite+0x24/0x2c) from [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs])
[ 5820.503901] [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs]) from [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs])
[ 5820.514626] [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs]) from [<8009fb64>] (vfs_ioctl+0x28/0x3c)
[ 5820.523485] [<8009fb64>] (vfs_ioctl+0x28/0x3c) from [<800a058c>] (do_vfs_ioctl+0x484/0x56c)
[ 5820.531820] [<800a058c>] (do_vfs_ioctl+0x484/0x56c) from [<800a06ac>] (SyS_ioctl+0x38/0x58)
[ 5820.540161] [<800a06ac>] (SyS_ioctl+0x38/0x58) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
[ 5820.548585] Code: e1c305f8 e3a03001 e59b2004 e1a06613 (e7d231a5) 
[ 5820.674046] ---[ end trace 7e587b920f507a06 ]---


I'll be happy to provide any additional information.

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Alexander Gordeev]

Hi,

I ran the test over weekend and I think I have some interesting results.
I used 1 new SD card in one device and two fully utilized SD cards,
that have problems with write latency, on two oter devices.
I mounted all the cards with mode=lfs. The new SD card got utilized at 95% after some time.
Here is the current status after the archive is rotated for some time:

=====[ partition info(sda1). #0, RW]=====
[SB: 1] [CP: 2] [SIT: 2] [NAT: 68] [SSA: 30] [MAIN: 15112(OverProv:803 Resv:50)]

Utilization: 94% (6929763 valid blocks)
  - Node: 8113 (Inode: 1255, Other: 6858)
  - Data: 6921650
  - Inline_xattr Inode: 0
  - Inline_data Inode: 1
  - Inline_dentry Inode: 0
  - Orphan Inode: 0

Main area: 15112 segs, 7556 secs 7556 zones
  - COLD  data: 5306, 2653, 2653
  - WARM  data: 5233, 2616, 2616
  - HOT   data: 15100, 7550, 7550
  - Dir   dnode: 15097, 7548, 7548
  - File   dnode: 4701, 2350, 2350
  - Indir nodes: 15105, 7552, 7552

  - Valid: 97
  - Dirty: 13798
  - Prefree: 0
  - Free: 1217 (604)

CP calls: 282 (BG: 0)
GC calls: 0 (BG: 0)
  - data segments : 0 (0)
  - node segments : 0 (0)
Try to move 0 blocks (BG: 0)
  - data blocks : 0 (0)
  - node blocks : 0 (0)

Extent Cache:
  - Hit Count: L1-1:3084 L1-2:456 L2:0
  - Hit Ratio: 4% (3540 / 84026)
  - Inner Struct Count: tree: 1252(0), node: 0

Balancing F2FS Async:
  - inmem:    0, wb_bios:    0
  - nodes:   12 in   30
  - dents:    3 in dirs:   2 (   2)
  - datas:   48 in files:   0
  - meta:    9 in   34
  - NATs:        10/      249
  - SITs:        13/    15112
  - free_nids:      1797

Distribution of User Blocks: [ valid | invalid | free ]
  [-----------------------------------------------|--|-]

IPU: 0 blocks
SSR: 0 blocks in 0 segments
LFS: 912188 blocks in 1781 segments

BDF: 94, avg. vblocks: 996

Memory: 3604 KB
  - static: 3270 KB
  - cached: 78 KB
  - paged : 256 KB


The interesting thing here is the very small number of valid and
a huge number of dirty sections. I don't understand this at all.
Still the archive is working perfectly. Also I see, that there GC is never
called, which is probably an indication of FS working exactly as
we expected.
Also the small number of cold sections does not make problems.
So, well, it works perfect so fat. But I don't understand everything here.
Is this expected?

The other two SD cards were tested differently. On one of them I called
ioctl(F2FS_IOC_GARBAGE_COLLECT) for several hours. And indeed the number
of dirty sectoins dropped considerably. It works fine so far.

On the other SD card I called ioctl(F2FS_IOC_DEFRAGMENT) for every
file in the archive. It works fine as well now. But the number of dirty sections
was still very high at the end of defragmentation. I don't understand this
as well.

Thanks!

19.08.2016, 14:56, "Alexander Gordeev" <alex@gordick.net>:
>  Hi Jaegeuk,
>
>  19.08.2016, 05:41, "Jaegeuk Kim" <jaegeuk@kernel.org>:
>>    Hello,
>>
>>    On Thu, Aug 18, 2016 at 02:04:55PM +0300, Alexander Gordeev wrote:
>>
>>    ...
>>
>>>     >>>>>>>   Here is also /sys/kernel/debug/f2fs/status for reference:
>>>     >>>>>>>   =====[ partition info(sda). #0 ]=====
>>>     >>>>>>>   [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529
>>>     >>>>>>>  Resv:50)]
>>>     >>>>>>>
>>>     >>>>>>>   Utilization: 94% (13597314 valid blocks)
>>>     >>>>>>>     - Node: 16395 (Inode: 2913, Other: 13482)
>>>     >>>>>>>     - Data: 13580919
>>>     >>>>>>>
>>>     >>>>>>>   Main area: 29646 segs, 14823 secs 14823 zones
>>>     >>>>>>>     - COLD data: 3468, 1734, 1734
>>>     >>>>>>>     - WARM data: 12954, 6477, 6477
>>>     >>>>>>>     - HOT data: 28105, 14052, 14052
>>>     >>>>>>>     - Dir dnode: 29204, 14602, 14602
>>>     >>>>>>>     - File dnode: 19960, 9980, 9980
>>>     >>>>>>>     - Indir nodes: 29623, 14811, 14811
>>>     >>>>>>>
>>>     >>>>>>>     - Valid: 13615
>>>     >>>>>>>     - Dirty: 13309
>>>     >>>>>>>     - Prefree: 0
>>>     >>>>>>>     - Free: 2722 (763)
>>>     >>>>>>>
>>>     >>>>>>>   GC calls: 8622 (BG: 4311)
>>>     >>>>>>>     - data segments : 8560
>>>     >>>>>>>     - node segments : 62
>>>     >>>>>>>   Try to move 3552161 blocks
>>>     >>>>>>>     - data blocks : 3540278
>>>     >>>>>>>     - node blocks : 11883
>>>     >>>>>>>
>>>     >>>>>>>   Extent Hit Ratio: 49 / 4171
>>>     >>>>>>>
>>>     >>>>>>>   Balancing F2FS Async:
>>>     >>>>>>>     - nodes 6 in 141
>>>     >>>>>>>     - dents 0 in dirs: 0
>>>     >>>>>>>     - meta 13 in 346
>>>     >>>>>>>     - NATs 16983 > 29120
>>>     >>>>>>>     - SITs: 17
>>>     >>>>>>>     - free_nids: 1861
>>>     >>>>>>>
>>>     >>>>>>>   Distribution of User Blocks: [ valid | invalid | free ]
>>>     >>>>>>>     [-----------------------------------------------|-|--]
>>>     >>>>>>>
>>>     >>>>>>>   SSR: 1230719 blocks in 14834 segments
>>>     >>>>>>>   LFS: 15150190 blocks in 29589 segments
>>>     >>>>>>>
>>>     >>>>>>>   BDF: 89, avg. vblocks: 949
>>>     >>>>>>>
>>>     >>>>>>>   Memory: 6754 KB = static: 4763 + cached: 1990
>>
>>    ...
>>
>>>     >>  Per my understanding of f2fs internals, it should write these "cold" files and
>>>     >>  usual "hot" files to different sections (that should map internally to
>>>     >>  different allocation units). So the sections used by "cold" data should almost
>>>     >>  never get "dirty" because most of the time all their blocks become free at
>>>     >>  the same time. Of course, the files are not exactly 4MB in size so the last
>>>     >>  section of the deleted file will become dirty. If it is moved by garbage
>>>     >>  collector and becomes mixed with fresh "cold" data, then indeed it might cause
>>>     >>  some problems, I think. What is your opinion?
>>>     >
>>>     > If your fs is not fragmented, it may be as what you said, otherwise, SSR will
>>>     > still try to reuse invalid block of other temperture segments, then your cold
>>>     > data will be fixed with warm data too.
>>>     >
>>>     > I guess, what you are facing is the latter one:
>>>     > SSR: 1230719 blocks in 14834 segments
>>>
>>>     I guess, I need to somehow disable any cleaning or SSR for my archive and index
>>>     files. But keep the cleaning for other data and nodes.
>>
>>    Could you test a mount option, "mode=lfs", to disable SSR?
>>    (I guess sqlite may suffer from logner latency due to GC though.)
>>
>>    Seems like it's caused by SSR starting to make worse before 95% as you described
>>    below.
>
>  Thanks, I'll run a test with a couple of SD cards over weekend.
>  So if I understand it correctly, GC will not cause the problems described below, right?
>  I.e. it will not mix the new data with old data from dirty sections?
>  Longer SQLite latencies should not be a problem because the database is written not
>  frequently and also it is about 200-250KB in size usually. Maybe forcing IPU as
>  suggested by Chao would help sqlite, no?
>  However looks like setting ipu_policy to 1 has no effect when mode=lfs.
>  The IPU counter is still zero on my test system.
> ...
-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Jaegeuk Kim]

Hi Alexander,

On Fri, Aug 19, 2016 at 02:56:12PM +0300, Alexander Gordeev wrote:
> Hi Jaegeuk,
> 
> 19.08.2016, 05:41, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> >  Hello,
> >
> >  On Thu, Aug 18, 2016 at 02:04:55PM +0300, Alexander Gordeev wrote:
> >
> >  ...
> >
> >>   >>>>>>>   Here is also /sys/kernel/debug/f2fs/status for reference:
> >>   >>>>>>>   =====[ partition info(sda). #0 ]=====
> >>   >>>>>>>   [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529
> >>   >>>>>>>  Resv:50)]
> >>   >>>>>>>
> >>   >>>>>>>   Utilization: 94% (13597314 valid blocks)
> >>   >>>>>>>     - Node: 16395 (Inode: 2913, Other: 13482)
> >>   >>>>>>>     - Data: 13580919
> >>   >>>>>>>
> >>   >>>>>>>   Main area: 29646 segs, 14823 secs 14823 zones
> >>   >>>>>>>     - COLD data: 3468, 1734, 1734
> >>   >>>>>>>     - WARM data: 12954, 6477, 6477
> >>   >>>>>>>     - HOT data: 28105, 14052, 14052
> >>   >>>>>>>     - Dir dnode: 29204, 14602, 14602
> >>   >>>>>>>     - File dnode: 19960, 9980, 9980
> >>   >>>>>>>     - Indir nodes: 29623, 14811, 14811
> >>   >>>>>>>
> >>   >>>>>>>     - Valid: 13615
> >>   >>>>>>>     - Dirty: 13309
> >>   >>>>>>>     - Prefree: 0
> >>   >>>>>>>     - Free: 2722 (763)
> >>   >>>>>>>
> >>   >>>>>>>   GC calls: 8622 (BG: 4311)
> >>   >>>>>>>     - data segments : 8560
> >>   >>>>>>>     - node segments : 62
> >>   >>>>>>>   Try to move 3552161 blocks
> >>   >>>>>>>     - data blocks : 3540278
> >>   >>>>>>>     - node blocks : 11883
> >>   >>>>>>>
> >>   >>>>>>>   Extent Hit Ratio: 49 / 4171
> >>   >>>>>>>
> >>   >>>>>>>   Balancing F2FS Async:
> >>   >>>>>>>     - nodes 6 in 141
> >>   >>>>>>>     - dents 0 in dirs: 0
> >>   >>>>>>>     - meta 13 in 346
> >>   >>>>>>>     - NATs 16983 > 29120
> >>   >>>>>>>     - SITs: 17
> >>   >>>>>>>     - free_nids: 1861
> >>   >>>>>>>
> >>   >>>>>>>   Distribution of User Blocks: [ valid | invalid | free ]
> >>   >>>>>>>     [-----------------------------------------------|-|--]
> >>   >>>>>>>
> >>   >>>>>>>   SSR: 1230719 blocks in 14834 segments
> >>   >>>>>>>   LFS: 15150190 blocks in 29589 segments
> >>   >>>>>>>
> >>   >>>>>>>   BDF: 89, avg. vblocks: 949
> >>   >>>>>>>
> >>   >>>>>>>   Memory: 6754 KB = static: 4763 + cached: 1990
> >
> >  ...
> >
> >>   >>  Per my understanding of f2fs internals, it should write these "cold" files and
> >>   >>  usual "hot" files to different sections (that should map internally to
> >>   >>  different allocation units). So the sections used by "cold" data should almost
> >>   >>  never get "dirty" because most of the time all their blocks become free at
> >>   >>  the same time. Of course, the files are not exactly 4MB in size so the last
> >>   >>  section of the deleted file will become dirty. If it is moved by garbage
> >>   >>  collector and becomes mixed with fresh "cold" data, then indeed it might cause
> >>   >>  some problems, I think. What is your opinion?
> >>   >
> >>   > If your fs is not fragmented, it may be as what you said, otherwise, SSR will
> >>   > still try to reuse invalid block of other temperture segments, then your cold
> >>   > data will be fixed with warm data too.
> >>   >
> >>   > I guess, what you are facing is the latter one:
> >>   > SSR: 1230719 blocks in 14834 segments
> >>
> >>   I guess, I need to somehow disable any cleaning or SSR for my archive and index
> >>   files. But keep the cleaning for other data and nodes.
> >
> >  Could you test a mount option, "mode=lfs", to disable SSR?
> >  (I guess sqlite may suffer from logner latency due to GC though.)
> >
> >  Seems like it's caused by SSR starting to make worse before 95% as you described
> >  below.
> 
> Thanks, I'll run a test with a couple of SD cards over weekend.
> So if I understand it correctly, GC will not cause the problems described below, right?
> I.e. it will not mix the new data with old data from dirty sections?
> Longer SQLite latencies should not be a problem because the database is written not
> frequently and also it is about 200-250KB in size usually. Maybe forcing IPU as
> suggested by Chao would help sqlite, no?
> However looks like setting ipu_policy to 1 has no effect when mode=lfs.
> The IPU counter is still zero on my test system.

Yup, in mode=lfs, ipu will be disabled automatically.

Thanks,

> 
> >>   I think the FS can get fragmented quite easily otherwise. The status above is
> >>   captured when the FS already has problems. I think it can become fragmented
> >>   this way:
> >>   1. The archive is written until the utilization is 95%. It is written separately from other
> >>   data and nodes thanks to the "cold" data feature.
> >>   2. After hitting 95% the archive my program starts to rotate the archive. The rotation
> >>   routine checks the free space, reported by statfs(), once a minute. If it is below 5%
> >>   of total, then it deletes several oldest records in the archive.
> >>   3. The last deleted record leaves a dirty section. This section holds several blocks
> >>   from a record, which now becomes the oldest one.
> >>   4. This section is merged with fresh "cold" or even warmer data by either GC, or
> >>   SSR in one or more newly used sections.
> >>   5. Then very soon the new oldest record is again deleted. And now we have one
> >>   or even several dirty sections filled with blocks from a not so old record. Which are
> >>   again merged with other records.
> >>   6. All the records get fragmented after one full rotation. The fragmentation gets
> >>   worse and worse.
> >>
> >>   So I think the best thing to do is to have sections with "cold" data be completely
> >>   out of all the cleaning schemes. It will clean itself by rotating.
> >>   Still other data and nodes might need to use some cleaning schemes.
> >>   Please correct me if I don't get it right.
> >>
> >>   > Maybe we can try to alter updating policy from OPU to IPU for your case to avoid
> >>   > performance regression of SSR and more frequently FG-GC:
> >>   >
> >>   > echo 1 > /sys/fs/f2fs/"yourdevicename"/ipu_policy
> >>
> >>   Thanks, I'll try it!
> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Jaegeuk Kim]

On Mon, Aug 22, 2016 at 11:52:12PM +0300, Alexander Gordeev wrote:
> Hi,
> 
> I ran the test over weekend and I think I have some interesting results.
> I used 1 new SD card in one device and two fully utilized SD cards,
> that have problems with write latency, on two oter devices.
> I mounted all the cards with mode=lfs. The new SD card got utilized at 95% after some time.
> Here is the current status after the archive is rotated for some time:
> 
> =====[ partition info(sda1). #0, RW]=====
> [SB: 1] [CP: 2] [SIT: 2] [NAT: 68] [SSA: 30] [MAIN: 15112(OverProv:803 Resv:50)]
> 
> Utilization: 94% (6929763 valid blocks)
>   - Node: 8113 (Inode: 1255, Other: 6858)
>   - Data: 6921650
>   - Inline_xattr Inode: 0
>   - Inline_data Inode: 1
>   - Inline_dentry Inode: 0
>   - Orphan Inode: 0
> 
> Main area: 15112 segs, 7556 secs 7556 zones
>   - COLD  data: 5306, 2653, 2653
>   - WARM  data: 5233, 2616, 2616
>   - HOT   data: 15100, 7550, 7550
>   - Dir   dnode: 15097, 7548, 7548
>   - File   dnode: 4701, 2350, 2350
>   - Indir nodes: 15105, 7552, 7552
> 
>   - Valid: 97
>   - Dirty: 13798
>   - Prefree: 0
>   - Free: 1217 (604)
> 
> CP calls: 282 (BG: 0)
> GC calls: 0 (BG: 0)
>   - data segments : 0 (0)
>   - node segments : 0 (0)
> Try to move 0 blocks (BG: 0)
>   - data blocks : 0 (0)
>   - node blocks : 0 (0)
> 
> Extent Cache:
>   - Hit Count: L1-1:3084 L1-2:456 L2:0
>   - Hit Ratio: 4% (3540 / 84026)
>   - Inner Struct Count: tree: 1252(0), node: 0
> 
> Balancing F2FS Async:
>   - inmem:    0, wb_bios:    0
>   - nodes:   12 in   30
>   - dents:    3 in dirs:   2 (   2)
>   - datas:   48 in files:   0
>   - meta:    9 in   34
>   - NATs:        10/      249
>   - SITs:        13/    15112
>   - free_nids:      1797
> 
> Distribution of User Blocks: [ valid | invalid | free ]
>   [-----------------------------------------------|--|-]
> 
> IPU: 0 blocks
> SSR: 0 blocks in 0 segments
> LFS: 912188 blocks in 1781 segments
> 
> BDF: 94, avg. vblocks: 996
> 
> Memory: 3604 KB
>   - static: 3270 KB
>   - cached: 78 KB
>   - paged : 256 KB
> 
> 
> The interesting thing here is the very small number of valid and
> a huge number of dirty sections. I don't understand this at all.

This is the number of dirty segments, so it needs to consider section and
segment at the same time; a dirty section can consist of valid and free
segments.
How abouting testing 2MB-sized section which is the default option?

> Still the archive is working perfectly. Also I see, that there GC is never
> called, which is probably an indication of FS working exactly as
> we expected.
> Also the small number of cold sections does not make problems.
> So, well, it works perfect so fat. But I don't understand everything here.
> Is this expected?

So, I'm in doubt that dirty sections consist of entirely valid or free segments.

> 
> The other two SD cards were tested differently. On one of them I called
> ioctl(F2FS_IOC_GARBAGE_COLLECT) for several hours. And indeed the number
> of dirty sectoins dropped considerably. It works fine so far.

It makes sense that valid segments in dirty sections would be migrated to
different free sections.

> On the other SD card I called ioctl(F2FS_IOC_DEFRAGMENT) for every
> file in the archive. It works fine as well now. But the number of dirty sections
> was still very high at the end of defragmentation. I don't understand this
> as well.

This is for defragementation to the given file, which would not move blocks in
order to decrease the number of dirty sections.
I think it's not necessary for this workload.

Thanks,

> 
> Thanks!
> 
> 19.08.2016, 14:56, "Alexander Gordeev" <alex@gordick.net>:
> >  Hi Jaegeuk,
> >
> >  19.08.2016, 05:41, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> >>    Hello,
> >>
> >>    On Thu, Aug 18, 2016 at 02:04:55PM +0300, Alexander Gordeev wrote:
> >>
> >>    ...
> >>
> >>>     >>>>>>>   Here is also /sys/kernel/debug/f2fs/status for reference:
> >>>     >>>>>>>   =====[ partition info(sda). #0 ]=====
> >>>     >>>>>>>   [SB: 1] [CP: 2] [SIT: 4] [NAT: 118] [SSA: 60] [MAIN: 29646(OverProv:1529
> >>>     >>>>>>>  Resv:50)]
> >>>     >>>>>>>
> >>>     >>>>>>>   Utilization: 94% (13597314 valid blocks)
> >>>     >>>>>>>     - Node: 16395 (Inode: 2913, Other: 13482)
> >>>     >>>>>>>     - Data: 13580919
> >>>     >>>>>>>
> >>>     >>>>>>>   Main area: 29646 segs, 14823 secs 14823 zones
> >>>     >>>>>>>     - COLD data: 3468, 1734, 1734
> >>>     >>>>>>>     - WARM data: 12954, 6477, 6477
> >>>     >>>>>>>     - HOT data: 28105, 14052, 14052
> >>>     >>>>>>>     - Dir dnode: 29204, 14602, 14602
> >>>     >>>>>>>     - File dnode: 19960, 9980, 9980
> >>>     >>>>>>>     - Indir nodes: 29623, 14811, 14811
> >>>     >>>>>>>
> >>>     >>>>>>>     - Valid: 13615
> >>>     >>>>>>>     - Dirty: 13309
> >>>     >>>>>>>     - Prefree: 0
> >>>     >>>>>>>     - Free: 2722 (763)
> >>>     >>>>>>>
> >>>     >>>>>>>   GC calls: 8622 (BG: 4311)
> >>>     >>>>>>>     - data segments : 8560
> >>>     >>>>>>>     - node segments : 62
> >>>     >>>>>>>   Try to move 3552161 blocks
> >>>     >>>>>>>     - data blocks : 3540278
> >>>     >>>>>>>     - node blocks : 11883
> >>>     >>>>>>>
> >>>     >>>>>>>   Extent Hit Ratio: 49 / 4171
> >>>     >>>>>>>
> >>>     >>>>>>>   Balancing F2FS Async:
> >>>     >>>>>>>     - nodes 6 in 141
> >>>     >>>>>>>     - dents 0 in dirs: 0
> >>>     >>>>>>>     - meta 13 in 346
> >>>     >>>>>>>     - NATs 16983 > 29120
> >>>     >>>>>>>     - SITs: 17
> >>>     >>>>>>>     - free_nids: 1861
> >>>     >>>>>>>
> >>>     >>>>>>>   Distribution of User Blocks: [ valid | invalid | free ]
> >>>     >>>>>>>     [-----------------------------------------------|-|--]
> >>>     >>>>>>>
> >>>     >>>>>>>   SSR: 1230719 blocks in 14834 segments
> >>>     >>>>>>>   LFS: 15150190 blocks in 29589 segments
> >>>     >>>>>>>
> >>>     >>>>>>>   BDF: 89, avg. vblocks: 949
> >>>     >>>>>>>
> >>>     >>>>>>>   Memory: 6754 KB = static: 4763 + cached: 1990
> >>
> >>    ...
> >>
> >>>     >>  Per my understanding of f2fs internals, it should write these "cold" files and
> >>>     >>  usual "hot" files to different sections (that should map internally to
> >>>     >>  different allocation units). So the sections used by "cold" data should almost
> >>>     >>  never get "dirty" because most of the time all their blocks become free at
> >>>     >>  the same time. Of course, the files are not exactly 4MB in size so the last
> >>>     >>  section of the deleted file will become dirty. If it is moved by garbage
> >>>     >>  collector and becomes mixed with fresh "cold" data, then indeed it might cause
> >>>     >>  some problems, I think. What is your opinion?
> >>>     >
> >>>     > If your fs is not fragmented, it may be as what you said, otherwise, SSR will
> >>>     > still try to reuse invalid block of other temperture segments, then your cold
> >>>     > data will be fixed with warm data too.
> >>>     >
> >>>     > I guess, what you are facing is the latter one:
> >>>     > SSR: 1230719 blocks in 14834 segments
> >>>
> >>>     I guess, I need to somehow disable any cleaning or SSR for my archive and index
> >>>     files. But keep the cleaning for other data and nodes.
> >>
> >>    Could you test a mount option, "mode=lfs", to disable SSR?
> >>    (I guess sqlite may suffer from logner latency due to GC though.)
> >>
> >>    Seems like it's caused by SSR starting to make worse before 95% as you described
> >>    below.
> >
> >  Thanks, I'll run a test with a couple of SD cards over weekend.
> >  So if I understand it correctly, GC will not cause the problems described below, right?
> >  I.e. it will not mix the new data with old data from dirty sections?
> >  Longer SQLite latencies should not be a problem because the database is written not
> >  frequently and also it is about 200-250KB in size usually. Maybe forcing IPU as
> >  suggested by Chao would help sqlite, no?
> >  However looks like setting ipu_policy to 1 has no effect when mode=lfs.
> >  The IPU counter is still zero on my test system.
> > ...
> -- 
>  Alexander

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Jaegeuk Kim]

Hi,

This is caused by no free segment to write data or node blocks.
In both of cases, have you tried file defragmentation?
I'd like to know whether this bug happens in normal cases as well.

Can you provide fsck.f2fs messages?

Thanks,

On Fri, Aug 19, 2016 at 08:22:39PM +0300, Alexander Gordeev wrote:
> Hi,
> 
> I'd also like to report two bugs. I used f2fs-stable branch linux-3.10.y.
> I'm using some proprietary modules also, but in my understanding they shouldn't affect the fs.
> 
> 1. I have a Sandisk 16GB microSDHC card, that is quite old and may be at its end of life.
> The bug is produced every time I try to delete any file on it.
> 
> [   28.805413] ------------[ cut here ]------------
> [   28.810327] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1103 new_curseg+0x330/0x40c [f2fs]()
> [   28.824971] Modules linked in:
> [   28.828128]  ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block evdev bcmdhd(O) amb
> arella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc_core ohci_hcd ehc
> i_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regmap_i2c snd_pcm snd
> _page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
> [   28.891640] CPU: 0 PID: 489 Comm: videoserverd Tainted: P           O 3.10.93 #1
> [   28.899128] [<8001338c>] (unwind_backtrace+0x0/0x12c) from [<80011a0c>] (show_stack+0x10/0x14)
> [   28.907795] [<80011a0c>] (show_stack+0x10/0x14) from [<8001effc>] (warn_slowpath_common+0x54/0x68)
> [   28.916777] [<8001effc>] (warn_slowpath_common+0x54/0x68) from [<8001f0ac>] (warn_slowpath_null+0x1c/0x24)
> [   28.926531] [<8001f0ac>] (warn_slowpath_null+0x1c/0x24) from [<7f51c220>] (new_curseg+0x330/0x40c [f2fs])
> [   28.936364] [<7f51c220>] (new_curseg+0x330/0x40c [f2fs]) from [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs])
> [   28.947594] [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs]) from [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs])
> [   28.959624] [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
> [   28.970374] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
> [   28.980595] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
> [   28.991328] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
> [   29.002072] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
> [   29.012652] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
> [   29.022684] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
> [   29.032013] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
> [   29.040940] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
> [   29.049402] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
> [   29.058186] ---[ end trace d249769884543c44 ]---
> [   29.062825] ------------[ cut here ]------------
> [   29.067523] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1142 new_curseg+0x3e4/0x40c [f2fs]()
> [   29.082113] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block e
> vdev bcmdhd(O) ambarella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc
> _core ohci_hcd ehci_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regm
> ap_i2c snd_pcm snd_page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
> [   29.147142] CPU: 0 PID: 489 Comm: videoserverd Tainted: P        W  O 3.10.93 #1
> [   29.154564] [<8001338c>] (unwind_backtrace+0x0/0x12c) from [<80011a0c>] (show_stack+0x10/0x14)
> [   29.163211] [<80011a0c>] (show_stack+0x10/0x14) from [<8001effc>] (warn_slowpath_common+0x54/0x68)
> [   29.172184] [<8001effc>] (warn_slowpath_common+0x54/0x68) from [<8001f0ac>] (warn_slowpath_null+0x1c/0x24)
> [   29.181887] [<8001f0ac>] (warn_slowpath_null+0x1c/0x24) from [<7f51c2d4>] (new_curseg+0x3e4/0x40c [f2fs])
> [   29.191508] [<7f51c2d4>] (new_curseg+0x3e4/0x40c [f2fs]) from [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs])
> [   29.202785] [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs]) from [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs])
> [   29.214742] [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
> [   29.225477] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
> [   29.235822] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
> [   29.246575] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
> [   29.257220] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
> [   29.267850] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
> [   29.277887] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
> [   29.287127] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
> [   29.296045] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
> [   29.304504] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
> [   29.313271] ---[ end trace d249769884543c45 ]---
> [   29.371882] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> [   29.396142] pgd = 85644000
> [   29.399051] [00000000] *pgd=0357b831, *pte=00000000, *ppte=00000000
> [   29.407041] Internal error: Oops: 17 [#1] PREEMPT ARM
> [   29.412085] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block e
> vdev bcmdhd(O) ambarella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc
> _core ohci_hcd ehci_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regm
> ap_i2c snd_pcm snd_page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
> [   29.476936] CPU: 0 PID: 489 Comm: videoserverd Tainted: P        W  O 3.10.93 #1
> [   29.484309] task: 869fdc80 ti: 83350000 task.ti: 83350000
> [   29.489728] PC is at update_sit_entry+0xc4/0x250 [f2fs]
> [   29.494949] LR is at update_sit_entry+0x78/0x250 [f2fs]
> [   29.500154] pc : [<7f51a74c>]    lr : [<7f51a700>]    psr: 20060013
> sp : 83351c48  ip : 00000000  fp : 84a0f740
> [   29.511600] r10: 00000000  r9 : 00001d74  r8 : 00000001
> [   29.516800] r7 : 00000000  r6 : 00000080  r5 : 859ac2e0  r4 : 857e6800
> [   29.523301] r3 : 00000007  r2 : 00153fae  r1 : 00000000  r0 : 00153fae
> [   29.529805] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
> [   29.536914] Control: 10c53c7d  Table: 05844059  DAC: 00000015
> [   29.542639] Process videoserverd (pid: 489, stack limit = 0x83350238)
> [   29.549054] Stack: (0x83351c48 to 0x83352000)
> [   29.553394] 1c40:                   00000001 857e6800 003b5800 0034c075 00000004 00000001
> [   29.561547] 1c60: 00000000 83351d38 80550080 7f51baf0 857e6800 86974cd0 86974cd0 7f51ccd8
> [   29.569701] 1c80: 83351d2c 84a0f740 00000000 84a0f768 0034c075 83351d01 00000000 800d0d94
> [   29.577853] 1ca0: 00000000 000000f0 60060013 7f5103cc 86455e00 83351d2c 00000004 80550080
> [   29.586005] 1cc0: 0000010c 83351d01 857e6938 000006b0 857e6800 7f51cf84 83351d01 00000004
> [   29.594156] 1ce0: 857e6800 00000000 83351d70 80550080 804ce684 000006b0 857e6800 7f51d128
> [   29.602309] 1d00: 0006b080 00000000 857e6938 7f515560 00000064 8006ddd8 8054b8f4 000006b0
> [   29.610461] 1d20: 000006b0 0034c075 86390d08 857e6800 00000001 00000411 003b5800 0034c075
> [   29.618612] 1d40: 80550080 00000000 80550080 857e6800 00000200 857e6a80 861c2333 00000075
> [   29.626765] 1d60: 857e6800 7f516af0 000006b0 00000001 00000001 00000000 00000000 00000000
> [   29.634916] 1d80: 00000000 00000000 00000001 00000000 861c2333 80550080 00000000 7f50e33c
> [   29.643069] 1da0: 00000001 83351e00 83351eb0 00001a28 000006b0 85795a50 83350000 00000000
> [   29.651220] 1dc0: 000273c0 00000000 00001a2a 805a0840 00000001 861c2000 00000001 83351e8c
> [   29.659372] 1de0: 0034c000 857e6804 00001a28 83350000 83351e08 000273c0 000003fc 83351e88
> [   29.667525] 1e00: 84a0f6c0 84a0f7c0 91827364 83351e0c 83351e0c 83351e14 83351e14 000006b0
> [   29.675676] 1e20: 000006b0 0034c075 00000008 00000400 00000000 857e6800 83351e8c 00000000
> [   29.683828] 1e40: 00000000 00000000 857e6a80 00000400 00000000 857e6800 83351e8c 7f50e7ec
> [   29.691979] 1e60: 00000000 ffffffea 00000000 83351ef4 00000000 00000000 00000000 00000000
> [   29.700131] 1e80: 857e6a50 857e6a60 00001a28 83351e8c 83351e8c 00000000 00000050 00000000
> [   29.708282] 1ea0: 00000002 84d43950 000000d6 863d83d8 863d83ec 00000004 00000000 00000000
> [   29.716434] 1ec0: 00000000 00000000 00000000 857e6800 86390568 00000000 857e692c 84d43034
> [   29.724586] 1ee0: 863dc720 863dc720 01da6a24 7f503e70 863d83c0 80577860 00000000 86390568
> [   29.732738] 1f00: 863d83c0 00000000 01da9a34 ffffff9c 863d83c0 800ae7f8 01da6a24 000bda58
> [   29.740890] 1f20: 00000000 00000000 8699e000 800ae9b0 85795a50 863acaa8 6fe7deef 00000004
> [   29.749042] 1f40: 8699e027 ffffff9c 00000000 86214d50 86390568 00000000 00000002 00000000
> [   29.757194] 1f60: 00000000 00000000 00000000 00000000 00000000 00000000 57b1c919 3ace5a0f
> [   29.765346] 1f80: 57b1c919 3ace5a0f 01da9a34 00418a80 7eee3978 0000000a 8000ee44 83350000
> [   29.773498] 1fa0: 00000000 8000ecc0 01da9a34 00418a80 01da9a34 7eee3838 81b40000 000081b4
> [   29.781650] 1fc0: 01da9a34 00418a80 7eee3978 0000000a 01da6ef0 01dac120 7eee3978 01da6a24
> [   29.789802] 1fe0: 76c49a71 7eee382c 76c49a79 76ca6926 800d0030 01da9a34 00000000 00000000
> [   29.798018] [<7f51a74c>] (update_sit_entry+0xc4/0x250 [f2fs]) from [<7f51baf0>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
> [   29.808554] [<7f51baf0>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f51ccd8>] (allocate_data_block+0x2e8/0x3d8 [f2fs])
> [   29.819429] [<7f51ccd8>] (allocate_data_block+0x2e8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
> [   29.830129] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
> [   29.840312] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
> [   29.851017] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
> [   29.861630] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
> [   29.872157] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
> [   29.882157] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
> [   29.891373] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
> [   29.900231] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
> [   29.908672] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
> [   29.917434] Code: e2063007 e3a06001 e1a06316 e1a071aa (e7d121aa) 
> [   29.974780] ---[ end trace d249769884543c46 ]---
> 
> 
> 2. I tried to run manual defragmentation of all the files on an almost new
> Transcend 128GB microSDXC card to fix the archive. Unfortunately this bug
> appeared after successful defragmentation of several files:
> 
> [ 5819.169892] ------------[ cut here ]------------
> [ 5819.174668] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1087 new_curseg+0x150/0x3b0 [f2fs]()
> [ 5819.189524] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage gpio_keys v
> [ 5819.251573] CPU: 0 PID: 1363 Comm: f2fstool Tainted: P           O 3.10.93 #1
> [ 5819.258793] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
> [ 5819.267471] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
> [ 5819.276445] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
> [ 5819.286294] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f47f530>] (new_curseg+0x150/0x3b0 [f2fs])
> [ 5819.296041] [<7f47f530>] (new_curseg+0x150/0x3b0 [f2fs]) from [<7f47f9dc>] (allocate_segment_by_default+0xa4/0x270 [f2fs])
> [ 5819.307317] [<7f47f9dc>] (allocate_segment_by_default+0xa4/0x270 [f2fs]) from [<7f480114>] (allocate_data_block+0x310/0x318 [f2fs])
> [ 5819.319257] [<7f480114>] (allocate_data_block+0x310/0x318 [f2fs]) from [<7f480340>] (do_write_page+0x224/0x270 [f2fs])
> [ 5819.330071] [<7f480340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs])
> [ 5819.340372] [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs])
> [ 5819.351070] [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs])
> [ 5819.362209] [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs]) from [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs])
> [ 5819.372382] [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs]) from [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs])
> [ 5819.382756] [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs]) from [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
> [ 5819.394361] [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
> [ 5819.405599] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<80066620>] (filemap_fdatawrite+0x24/0x2c)
> [ 5819.415810] [<80066620>] (filemap_fdatawrite+0x24/0x2c) from [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs])
> [ 5819.426445] [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs]) from [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs])
> [ 5819.437202] [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs]) from [<8009fb64>] (vfs_ioctl+0x28/0x3c)
> [ 5819.446105] [<8009fb64>] (vfs_ioctl+0x28/0x3c) from [<800a058c>] (do_vfs_ioctl+0x484/0x56c)
> [ 5819.454463] [<800a058c>] (do_vfs_ioctl+0x484/0x56c) from [<800a06ac>] (SyS_ioctl+0x38/0x58)
> [ 5819.462882] [<800a06ac>] (SyS_ioctl+0x38/0x58) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
> [ 5819.471311] ---[ end trace 7e587b920f507a04 ]---
> [ 5819.475939] ------------[ cut here ]------------
> [ 5819.480611] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1142 new_curseg+0x300/0x3b0 [f2fs]()
> [ 5819.495227] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage gpio_keys v
> [ 5819.557163] CPU: 0 PID: 1363 Comm: f2fstool Tainted: P        W  O 3.10.93 #1
> [ 5819.564332] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
> [ 5819.572959] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
> [ 5819.581946] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
> [ 5819.591707] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f47f6e0>] (new_curseg+0x300/0x3b0 [f2fs])
> [ 5819.601375] [<7f47f6e0>] (new_curseg+0x300/0x3b0 [f2fs]) from [<7f47f9dc>] (allocate_segment_by_default+0xa4/0x270 [f2fs])
> [ 5819.612548] [<7f47f9dc>] (allocate_segment_by_default+0xa4/0x270 [f2fs]) from [<7f480114>] (allocate_data_block+0x310/0x318 [f2fs])
> [ 5819.624523] [<7f480114>] (allocate_data_block+0x310/0x318 [f2fs]) from [<7f480340>] (do_write_page+0x224/0x270 [f2fs])
> [ 5819.635354] [<7f480340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs])
> [ 5819.645642] [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs])
> [ 5819.656337] [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs])
> [ 5819.667505] [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs]) from [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs])
> [ 5819.677682] [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs]) from [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs])
> [ 5819.688086] [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs]) from [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
> [ 5819.699547] irq_sync_frame_count(388): Add missing 2 frames to the frame queue! DSP(H264: 247189; MJEPG: 0), IAV(Total: 247187)!
> [ 5819.711303] [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
> [ 5819.722548] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<80066620>] (filemap_fdatawrite+0x24/0x2c)
> [ 5819.732771] [<80066620>] (filemap_fdatawrite+0x24/0x2c) from [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs])
> [ 5819.743389] [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs]) from [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs])
> [ 5819.754139] [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs]) from [<8009fb64>] (vfs_ioctl+0x28/0x3c)
> [ 5819.763037] [<8009fb64>] (vfs_ioctl+0x28/0x3c) from [<800a058c>] (do_vfs_ioctl+0x484/0x56c)
> [ 5819.771374] [<800a058c>] (do_vfs_ioctl+0x484/0x56c) from [<800a06ac>] (SyS_ioctl+0x38/0x58)
> [ 5819.779759] [<800a06ac>] (SyS_ioctl+0x38/0x58) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
> [ 5819.788228] ---[ end trace 7e587b920f507a05 ]---
> [ 5819.816478] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> [ 5819.839179] pgd = 82960000
> [ 5819.842605] [00000000] *pgd=0084f831, *pte=00000000, *ppte=00000000
> [ 5819.849403] Internal error: Oops: 17 [#1] PREEMPT ARM
> [ 5819.854439] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage gpio_keys v
> [ 5819.916258] CPU: 0 PID: 1363 Comm: f2fstool Tainted: P        W  O 3.10.93 #1
> [ 5819.923373] task: 81d6f800 ti: 83350000 task.ti: 83350000
> [ 5819.928824] PC is at update_sit_entry+0xf0/0x218 [f2fs]
> [ 5819.934088] LR is at update_sit_entry+0xac/0x218 [f2fs]
> [ 5819.939295] pc : [<7f47d454>]    lr : [<7f47d410>]    psr: 200f0013
> [ 5819.939295] sp : 83351a20  ip : 00000001  fp : 8515a8c0
> [ 5819.950741] r10: 00000001  r9 : ffffffff  r8 : a86023f8
> [ 5819.955944] r7 : 0000e708  r6 : 00000080  r5 : 00000000  r4 : 8587cc00
> [ 5819.962446] r3 : 00000001  r2 : 00000000  r1 : 00000000  r0 : 00174241
> [ 5819.968951] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
> [ 5819.976062] Control: 10c53c7d  Table: 02b60059  DAC: 00000015
> [ 5819.981785] Process f2fstool (pid: 1363, stack limit = 0x83350238)
> [ 5819.987941] Stack: (0x83351a20 to 0x83352000)
> [ 5819.992285] 1a20: 83351bd0 8587cc00 0040b455 01cff800 00000000 83351bc4 869ec468 8052e420
> [ 5820.000443] 1a40: 861d36a8 7f47f00c 8587cc00 869ec468 00000002 7f48003c 8046a4a0 83351b14
> [ 5820.008598] 1a60: 861d3680 83351bd0 0040b455 83351ac1 833936b4 000024c7 83351ac8 00000002
> [ 5820.016752] 1a80: 83351bc4 83351ac1 00000000 83351bc4 00000001 00000001 00000000 7f480340
> [ 5820.024907] 1aa0: 83351ac1 00000002 00000001 83351b14 83351bc4 8587cc00 8052e420 7f4804b8
> [ 5820.033062] 1ac0: 0024c790 01520100 000024c7 0000249a 019c12dd 8052e401 8587cc00 8052e420
> [ 5820.041216] 1ae0: 86540528 7f476ff4 8019cf50 680d0013 86540528 00000000 00000000 000024c7
> [ 5820.049371] 1b00: 00000152 00000000 0040b455 8587cc00 8052e420 86540528 804a6380 8049ae20
> [ 5820.057525] 1b20: 000024c7 00000152 00000000 0040b455 8285f253 8587cc00 8052e420 86540528
> [ 5820.065679] 1b40: 8285f253 83350000 00000001 00000001 00000000 7f472bdc 00000001 00000210
> [ 5820.073834] 1b60: 00001f66 00001f64 00000003 83351b80 804a9be0 00000000 0002f160 00000055
> [ 5820.081989] 1b80: 8285f000 00001f68 8587cc04 00000000 0002f190 83351c24 85e71c00 0040b400
> [ 5820.090144] 1ba0: 000024c7 0000249a 019c12dd 8272c001 91827364 83351bb4 83351bb4 83351bbc
> [ 5820.098298] 1bc0: 83351bbc 8587cc00 00000000 00000411 01cff800 0040b455 8052e420 00000000
> [ 5820.106451] 1be0: 00000004 8587cc00 0000001b 00000001 00000000 00000002 0000001b 00000000
> [ 5820.114605] 1c00: 00000004 7f472fd4 00000001 00000000 00000000 00000000 0000001a 00000009
> [ 5820.122759] 1c20: 00001f64 8658d260 8658d1f0 00000003 00000050 864395f1 00000002 00000001
> [ 5820.130913] 1c40: 00001c2b 00000000 804c2d80 804c47c0 00001a94 00000206 00000000 003a3fff
> [ 5820.139067] 1c60: 60060013 8587cc00 8052dd60 864536e0 00000001 83351da0 00000000 02572000
> [ 5820.147221] 1c80: 00000000 7f47779c 0000000e 00000102 83351d20 83350008 8587cd2c 00000000
> [ 5820.155376] 1ca0: 864c75b4 80066130 00000002 8587cc00 00000000 00000411 003a3fff 01324440
> [ 5820.163530] 1cc0: 8052dd60 00000000 00002570 8645379c ffffffff 864536e0 00002570 00000001
> [ 5820.171684] 1ce0: 8587cc00 8052dd60 83351da0 7f47424c 0000000e 00000002 81c5c000 ffffffff
> [ 5820.179837] 1d00: 00000005 00000000 00002571 0000000e 83351d54 00000000 00000001 00000002
> [ 5820.187992] 1d20: 0000257a 91827364 83351d28 83351d28 83351d30 83351d30 0000000e 00000000
> [ 5820.196148] 1d40: 804e4840 8049eca0 804d7e80 8046db00 80463d80 8052dd60 8048b480 804ff940
> [ 5820.204302] 1d60: 804e2ea0 804d8840 80471960 804777c0 805111a0 80475fc0 00000000 8645379c
> [ 5820.212456] 1d80: 864536e0 00000000 00000000 00002600 00000200 00002400 00000000 800665f4
> [ 5820.220609] 1da0: 7ffffe8e 00000000 00000000 00000000 ffffffff 7fffffff 00000001 00000000
> [ 5820.228766] 1dc0: 8047b6e0 00002b0a 00000200 80066620 ffffffff 7fffffff 00000001 00021000
> [ 5820.236921] 1de0: 8047b6e0 7f4639fc 02b09fff 00000000 83351ec0 00000200 00000800 00000000
> [ 5820.245075] 1e00: 86453744 8006bd7c 00000006 000000a8 00000000 8008a218 00000012 80013f04
> [ 5820.253229] 1e20: 804edd20 0132442e 00002600 000001e4 00000020 00000000 82748c00 864536e0
> [ 5820.261383] 1e40: 82748c00 7e8a6a90 8587cc00 00000fff 00000000 83350000 00000000 7f4668f4
> [ 5820.269537] 1e60: 76e94000 000000a8 82961db8 81d2e4d8 83350018 8007ee74 000000a2 000000a8
> [ 5820.277692] 1e80: 00000000 600d0093 80450720 76f36000 000003b7 81dc8d80 8585f580 000000a8
> [ 5820.285846] 1ea0: 82961db8 82960000 83350018 8007f054 82961db8 000000a8 83351fb0 000000a8
> [ 5820.294000] 1ec0: 00000000 00000000 02b0a000 00000000 000000a8 800138e4 00000800 00000000
> [ 5820.302154] 1ee0: 00000200 00001a88 000081b4 6eadc2c5 80403434 7e8a6a90 864536e0 82748c00
> [ 5820.310308] 1f00: 0000f508 7e8a6a90 83350000 00000000 00000000 8009fb64 4030582a 800a058c
> [ 5820.318464] 1f20: 57a7001f 3982fe52 57a7014a 1eb08998 57a7014a 1eb08998 00001a88 00000000
> [ 5820.326618] 1f40: 7e8a6a18 7e8a6a18 00000000 80095854 00001a88 00000000 00800000 000081b4
> [ 5820.334771] 1f60: 00000001 00000003 82748c00 00000000 0000f508 7e8a6a90 83350000 00000000
> [ 5820.342924] 1f80: 00000000 800a06ac 00000003 00000000 00000000 00000000 00000000 00000036
> [ 5820.351078] 1fa0: 8000e344 8000e1c0 00000000 00000000 00000003 0000f508 7e8a6a90 7e8a6a90
> [ 5820.359231] 1fc0: 00000000 00000000 00000000 00000036 00000000 00000000 76fba000 00000000
> [ 5820.367385] 1fe0: 76f37d91 7e8a6a84 000086f1 76f37d96 800d0030 00000003 5bb15ad7 ae252a72
> [ 5820.375663] [<7f47d454>] (update_sit_entry+0xf0/0x218 [f2fs]) from [<7f47f00c>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
> [ 5820.386266] [<7f47f00c>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f48003c>] (allocate_data_block+0x238/0x318 [f2fs])
> [ 5820.397214] [<7f48003c>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f480340>] (do_write_page+0x224/0x270 [f2fs])
> [ 5820.407981] [<7f480340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs])
> [ 5820.418219] [<7f4804b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs])
> [ 5820.428885] [<7f476ff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs])
> [ 5820.439983] [<7f472bdc>] (do_garbage_collect+0xad8/0xbcc [f2fs]) from [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs])
> [ 5820.450124] [<7f472fd4>] (f2fs_gc+0x304/0x4e8 [f2fs]) from [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs])
> [ 5820.460443] [<7f47779c>] (f2fs_write_data_page+0x484/0x4d4 [f2fs]) from [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
> [ 5820.471952] [<7f47424c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
> [ 5820.483156] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<80066620>] (filemap_fdatawrite+0x24/0x2c)
> [ 5820.493342] [<80066620>] (filemap_fdatawrite+0x24/0x2c) from [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs])
> [ 5820.503901] [<7f4639fc>] (f2fs_defragment_range+0x664/0x6b4 [f2fs]) from [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs])
> [ 5820.514626] [<7f4668f4>] (f2fs_ioctl+0xae0/0x117c [f2fs]) from [<8009fb64>] (vfs_ioctl+0x28/0x3c)
> [ 5820.523485] [<8009fb64>] (vfs_ioctl+0x28/0x3c) from [<800a058c>] (do_vfs_ioctl+0x484/0x56c)
> [ 5820.531820] [<800a058c>] (do_vfs_ioctl+0x484/0x56c) from [<800a06ac>] (SyS_ioctl+0x38/0x58)
> [ 5820.540161] [<800a06ac>] (SyS_ioctl+0x38/0x58) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
> [ 5820.548585] Code: e1c305f8 e3a03001 e59b2004 e1a06613 (e7d231a5) 
> [ 5820.674046] ---[ end trace 7e587b920f507a06 ]---
> 
> 
> I'll be happy to provide any additional information.
> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

Thanks for all the help!

24.08.2016, 00:12, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> On Mon, Aug 22, 2016 at 11:52:12PM +0300, Alexander Gordeev wrote:
>>  Hi,
>>
>>  I ran the test over weekend and I think I have some interesting results.
>>  I used 1 new SD card in one device and two fully utilized SD cards,
>>  that have problems with write latency, on two oter devices.
>>  I mounted all the cards with mode=lfs. The new SD card got utilized at 95% after some time.
>>  Here is the current status after the archive is rotated for some time:
>>
>>  =====[ partition info(sda1). #0, RW]=====
>>  [SB: 1] [CP: 2] [SIT: 2] [NAT: 68] [SSA: 30] [MAIN: 15112(OverProv:803 Resv:50)]
>>
>>  Utilization: 94% (6929763 valid blocks)
>>    - Node: 8113 (Inode: 1255, Other: 6858)
>>    - Data: 6921650
>>    - Inline_xattr Inode: 0
>>    - Inline_data Inode: 1
>>    - Inline_dentry Inode: 0
>>    - Orphan Inode: 0
>>
>>  Main area: 15112 segs, 7556 secs 7556 zones
>>    - COLD data: 5306, 2653, 2653
>>    - WARM data: 5233, 2616, 2616
>>    - HOT data: 15100, 7550, 7550
>>    - Dir dnode: 15097, 7548, 7548
>>    - File dnode: 4701, 2350, 2350
>>    - Indir nodes: 15105, 7552, 7552
>>
>>    - Valid: 97
>>    - Dirty: 13798
>>    - Prefree: 0
>>    - Free: 1217 (604)
>>
>>  CP calls: 282 (BG: 0)
>>  GC calls: 0 (BG: 0)
>>    - data segments : 0 (0)
>>    - node segments : 0 (0)
>>  Try to move 0 blocks (BG: 0)
>>    - data blocks : 0 (0)
>>    - node blocks : 0 (0)
>>
>>  Extent Cache:
>>    - Hit Count: L1-1:3084 L1-2:456 L2:0
>>    - Hit Ratio: 4% (3540 / 84026)
>>    - Inner Struct Count: tree: 1252(0), node: 0
>>
>>  Balancing F2FS Async:
>>    - inmem: 0, wb_bios: 0
>>    - nodes: 12 in 30
>>    - dents: 3 in dirs: 2 ( 2)
>>    - datas: 48 in files: 0
>>    - meta: 9 in 34
>>    - NATs: 10/ 249
>>    - SITs: 13/ 15112
>>    - free_nids: 1797
>>
>>  Distribution of User Blocks: [ valid | invalid | free ]
>>    [-----------------------------------------------|--|-]
>>
>>  IPU: 0 blocks
>>  SSR: 0 blocks in 0 segments
>>  LFS: 912188 blocks in 1781 segments
>>
>>  BDF: 94, avg. vblocks: 996
>>
>>  Memory: 3604 KB
>>    - static: 3270 KB
>>    - cached: 78 KB
>>    - paged : 256 KB
>>
>>  The interesting thing here is the very small number of valid and
>>  a huge number of dirty sections. I don't understand this at all.
>
> This is the number of dirty segments, so it needs to consider section and
> segment at the same time; a dirty section can consist of valid and free
> segments.
> How abouting testing 2MB-sized section which is the default option?

I tried what you said. Still the majority of segments are dirty for some reason:

=====[ partition info(sda). #0, RW]=====
[SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)]

Utilization: 10% (3013093 valid blocks)
  - Node: 3528 (Inode: 548, Other: 2980)
  - Data: 3009565
  - Inline_xattr Inode: 0
  - Inline_data Inode: 0
  - Inline_dentry Inode: 0
  - Orphan Inode: 0

Main area: 59149 segs, 59149 secs 59149 zones
  - COLD  data: 7183, 7183, 7183
  - WARM  data: 6654, 6654, 6654
  - HOT   data: 59134, 59134, 59134
  - Dir   dnode: 59127, 59127, 59127
  - File   dnode: 59125, 59125, 59125
  - Indir nodes: 59129, 59129, 59129

  - Valid: 300
  - Dirty: 6438
  - Prefree: 0
  - Free: 52411 (52411)

CP calls: 1023 (BG: 473)
GC calls: 470 (BG: 470)
  - data segments : 466 (466)
  - node segments : 4 (4)
Try to move 152221 blocks (BG: 152221)
  - data blocks : 151417 (151417)
  - node blocks : 804 (804)

Extent Cache:
  - Hit Count: L1-1:6262 L1-2:0 L2:0
  - Hit Ratio: 2% (6262 / 273606)
  - Inner Struct Count: tree: 292(0), node: 8

Balancing F2FS Async:
  - inmem:    0, wb_bios:    0
  - nodes:    0 in    0
  - dents:    0 in dirs:   0 (   0)
  - datas:    0 in files:   0
  - meta:    0 in    0
  - NATs:         0/       43
  - SITs:         0/    59149
  - free_nids:      3414

Distribution of User Blocks: [ valid | invalid | free ]
  [-----|--|-------------------------------------------]

IPU: 0 blocks
SSR: 0 blocks in 0 segments
LFS: 3691542 blocks in 7208 segments

BDF: 95, avg. vblocks: 444

Memory: 12662 KB
  - static: 12597 KB
  - cached: 64 KB
  - paged : 0 KB


But the archive is working perfectly as before.

>>  Still the archive is working perfectly. Also I see, that there GC is never
>>  called, which is probably an indication of FS working exactly as
>>  we expected.
>>  Also the small number of cold sections does not make problems.
>>  So, well, it works perfect so fat. But I don't understand everything here.
>>  Is this expected?
>
> So, I'm in doubt that dirty sections consist of entirely valid or free segments.
>
>>  The other two SD cards were tested differently. On one of them I called
>>  ioctl(F2FS_IOC_GARBAGE_COLLECT) for several hours. And indeed the number
>>  of dirty sectoins dropped considerably. It works fine so far.
>
> It makes sense that valid segments in dirty sections would be migrated to
> different free sections.
>
>>  On the other SD card I called ioctl(F2FS_IOC_DEFRAGMENT) for every
>>  file in the archive. It works fine as well now. But the number of dirty sections
>>  was still very high at the end of defragmentation. I don't understand this
>>  as well.
>
> This is for defragementation to the given file, which would not move blocks in
> order to decrease the number of dirty sections.
> I think it's not necessary for this workload.

Yes, I'd prefer to not run manual GC or defragmentation. This was just a test.

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

24.08.2016, 00:27, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> Hi,
>
> This is caused by no free segment to write data or node blocks.
> In both of cases, have you tried file defragmentation?

I see, thanks!

In the first case I didn't try file defragmentation or manual GC.
The problem happens when a try to delete a file. Actually, this
might be a hardware failure. I can try to reformat and test it, if there
is no more sensible data, that we can get from it.

In the second case it happened exactly during defragmentation.

> I'd like to know whether this bug happens in normal cases as well.
>
> Can you provide fsck.f2fs messages?



> On Fri, Aug 19, 2016 at 08:22:39PM +0300, Alexander Gordeev wrote:
>> ...

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

24.08.2016, 00:27, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> Hi,
>
> This is caused by no free segment to write data or node blocks.
> In both of cases, have you tried file defragmentation?
> I'd like to know whether this bug happens in normal cases as well.

Just got this after my main process, that writes the video archive, was
killed by OOM killer, and the system restarted thanks to
vm.panic_on_oom + kernel.panic. Then right after the restart I got this:

[   26.845556] ------------[ cut here ]------------
[   26.861986] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:876 update_sit_entry+0x118/0x218 [f2fs]()
[   26.881959] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage evdev bcmdv
[   26.971301] CPU: 0 PID: 525 Comm: videoserverd Tainted: P           O 3.10.93 #1
[   26.996438] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
[   27.021738] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
[   27.041226] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
[   27.066835] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f4b647c>] (update_sit_entry+0x118/0x218 [f2fs])
[   27.133465] [<7f4b647c>] (update_sit_entry+0x118/0x218 [f2fs]) from [<7f4b800c>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
[   27.188132] [<7f4b800c>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs])
[   27.209063] [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs])
[   27.242807] [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs])
[   27.260651] [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs])
[   27.281858] [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs])
[   27.302545] [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs]) from [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
[   27.315031] [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
[   27.333922] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<800666f4>] (filemap_write_and_wait_range+0x34/0x78)
[   27.369185] [<800666f4>] (filemap_write_and_wait_range+0x34/0x78) from [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs])
[   27.384766] [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs]) from [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs])
[   27.396348] [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs]) from [<800b5fdc>] (vfs_fsync+0x20/0x28)
[   27.406225] [<800b5fdc>] (vfs_fsync+0x20/0x28) from [<800b6140>] (do_fsync+0x28/0x48)
[   27.414893] [<800b6140>] (do_fsync+0x28/0x48) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
[   27.428760] ---[ end trace d9a54d63707dc19e ]---
[   34.177477] ------------[ cut here ]------------
[   34.204559] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:881 update_sit_entry+0x16c/0x218 [f2fs]()
[   34.314185] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage evdev bcmdv
[   34.409462] CPU: 0 PID: 613 Comm: videoserverd Tainted: P        W  O 3.10.93 #1
[   34.472011] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
[   34.509232] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
[   34.541215] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
[   34.621879] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f4b64d0>] (update_sit_entry+0x16c/0x218 [f2fs])
[   34.662256] [<7f4b64d0>] (update_sit_entry+0x16c/0x218 [f2fs]) from [<7f4b8050>] (refresh_sit_entry+0x60/0xb8 [f2fs])
[   34.702123] [<7f4b8050>] (refresh_sit_entry+0x60/0xb8 [f2fs]) from [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs])
[   34.741857] [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs])
[   34.805079] [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs])
[   34.841321] [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs])
[   34.906465] [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs])
[   34.979779] [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs]) from [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
[   35.009256] [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
[   35.069102] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<800666f4>] (filemap_write_and_wait_range+0x34/0x78)
[   35.096669] [<800666f4>] (filemap_write_and_wait_range+0x34/0x78) from [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs])
[   35.119298] [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs]) from [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs])
[   35.142967] [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs]) from [<800b5fdc>] (vfs_fsync+0x20/0x28)
[   35.194647] [<800b5fdc>] (vfs_fsync+0x20/0x28) from [<800b6140>] (do_fsync+0x28/0x48)
[   35.213048] [<800b6140>] (do_fsync+0x28/0x48) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
[   35.239412] ---[ end trace d9a54d63707dc19f ]---


I consider this to be a normal situation. The filesystem was created
several days ago with the default section size and mounted with these
options: nodev,noexec,nosuid,noatime,nodiratime,nouser_xattr,noacl,mode=lfs

Here is also the status right after the warnings:

=====[ partition info(sda). #0, RW]=====
[SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)]

Utilization: 18% (5357903 valid blocks)
  - Node: 6279 (Inode: 976, Other: 5303)
  - Data: 5351624
  - Inline_xattr Inode: 0
  - Inline_data Inode: 0
  - Inline_dentry Inode: 0
  - Orphan Inode: 0

Main area: 59149 segs, 59149 secs 59149 zones
  - COLD  data: 12160, 12160, 12160
  - WARM  data: 11878, 11878, 11878
  - HOT   data: 59117, 59117, 59117
  - Dir   dnode: 59118, 59118, 59118
  - File   dnode: 59115, 59115, 59115
  - Indir nodes: 59129, 59129, 59129

  - Valid: 380
  - Dirty: 11221
  - Prefree: 0
  - Free: 47548 (47548)

CP calls: 4 (BG: 0)
GC calls: 0 (BG: 0)
  - data segments : 0 (0)
  - node segments : 0 (0)
Try to move 0 blocks (BG: 0)
  - data blocks : 0 (0)
  - node blocks : 0 (0)

Extent Cache:
  - Hit Count: L1-1:26 L1-2:2 L2:1
  - Hit Ratio: 5% (29 / 574)
  - Inner Struct Count: tree: 973(0), node: 0

Balancing F2FS Async:
  - inmem:    0, wb_bios:    0
  - nodes:    4 in    9
  - dents:    2 in dirs:   2 (   0)
  - datas:    0 in files:   0
  - meta:    0 in   23
  - NATs:         3/      167
  - SITs:         3/    59149
  - free_nids:      3587

Distribution of User Blocks: [ valid | invalid | free ]
  [---------|--|---------------------------------------]

IPU: 0 blocks
SSR: 0 blocks in 0 segments
LFS: 3394 blocks in 7 segments

BDF: 93, avg. vblocks: 460

Memory: 12820 KB
  - static: 12597 KB
  - cached: 94 KB
  - paged : 128 KB


> Can you provide fsck.f2fs messages?

Sure:

Info: sector size = 512
Info: total sectors = 243253248 (in 512 bytes)
Info: MKFS version
  "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"
Info: FSCK version
  from "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"
    to "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"

[FSCK] Unreachable nat entries                        [Ok..] [0x0]
[FSCK] SIT valid block bitmap checking                [Ok..]
[FSCK] Hard link checking for regular file            [Ok..] [0x0]
[FSCK] valid_block_count matching with CP             [Ok..] [0x51c14f]
[FSCK] valid_node_count matcing with CP (de lookup)   [Ok..] [0x1887]
[FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x1887]
[FSCK] valid_inode_count matched with CP              [Ok..] [0x3d0]
[FSCK] free segment_count matched with CP             [Ok..] [0xb9bc]
[FSCK] next block offset is free                      [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs                           [Ok..]

Done.

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Jaegeuk Kim]

Hi Alexander,

On Fri, Aug 26, 2016 at 07:04:28PM +0300, Alexander Gordeev wrote:
> Hi Jaegeuk,
> 
> 24.08.2016, 00:27, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> > Hi,
> >
> > This is caused by no free segment to write data or node blocks.
> > In both of cases, have you tried file defragmentation?
> > I'd like to know whether this bug happens in normal cases as well.
> 
> Just got this after my main process, that writes the video archive, was
> killed by OOM killer, and the system restarted thanks to
> vm.panic_on_oom + kernel.panic. Then right after the restart I got this:
> 
> [   26.845556] ------------[ cut here ]------------
> [   26.861986] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:876 update_sit_entry+0x118/0x218 [f2fs]()
> [   26.881959] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage evdev bcmdv
> [   26.971301] CPU: 0 PID: 525 Comm: videoserverd Tainted: P           O 3.10.93 #1
> [   26.996438] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
> [   27.021738] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
> [   27.041226] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
> [   27.066835] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f4b647c>] (update_sit_entry+0x118/0x218 [f2fs])
> [   27.133465] [<7f4b647c>] (update_sit_entry+0x118/0x218 [f2fs]) from [<7f4b800c>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
> [   27.188132] [<7f4b800c>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs])
> [   27.209063] [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs])
> [   27.242807] [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs])
> [   27.260651] [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs])
> [   27.281858] [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs])
> [   27.302545] [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs]) from [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
> [   27.315031] [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
> [   27.333922] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<800666f4>] (filemap_write_and_wait_range+0x34/0x78)
> [   27.369185] [<800666f4>] (filemap_write_and_wait_range+0x34/0x78) from [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs])
> [   27.384766] [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs]) from [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs])
> [   27.396348] [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs]) from [<800b5fdc>] (vfs_fsync+0x20/0x28)
> [   27.406225] [<800b5fdc>] (vfs_fsync+0x20/0x28) from [<800b6140>] (do_fsync+0x28/0x48)
> [   27.414893] [<800b6140>] (do_fsync+0x28/0x48) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
> [   27.428760] ---[ end trace d9a54d63707dc19e ]---
> [   34.177477] ------------[ cut here ]------------
> [   34.204559] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:881 update_sit_entry+0x16c/0x218 [f2fs]()
> [   34.314185] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage evdev bcmdv
> [   34.409462] CPU: 0 PID: 613 Comm: videoserverd Tainted: P        W  O 3.10.93 #1
> [   34.472011] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
> [   34.509232] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
> [   34.541215] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
> [   34.621879] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f4b64d0>] (update_sit_entry+0x16c/0x218 [f2fs])
> [   34.662256] [<7f4b64d0>] (update_sit_entry+0x16c/0x218 [f2fs]) from [<7f4b8050>] (refresh_sit_entry+0x60/0xb8 [f2fs])
> [   34.702123] [<7f4b8050>] (refresh_sit_entry+0x60/0xb8 [f2fs]) from [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs])
> [   34.741857] [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs])
> [   34.805079] [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs])
> [   34.841321] [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs])
> [   34.906465] [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs])
> [   34.979779] [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs]) from [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
> [   35.009256] [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
> [   35.069102] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<800666f4>] (filemap_write_and_wait_range+0x34/0x78)
> [   35.096669] [<800666f4>] (filemap_write_and_wait_range+0x34/0x78) from [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs])
> [   35.119298] [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs]) from [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs])
> [   35.142967] [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs]) from [<800b5fdc>] (vfs_fsync+0x20/0x28)
> [   35.194647] [<800b5fdc>] (vfs_fsync+0x20/0x28) from [<800b6140>] (do_fsync+0x28/0x48)
> [   35.213048] [<800b6140>] (do_fsync+0x28/0x48) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
> [   35.239412] ---[ end trace d9a54d63707dc19f ]---
> 
> 
> I consider this to be a normal situation. The filesystem was created
> several days ago with the default section size and mounted with these
> options: nodev,noexec,nosuid,noatime,nodiratime,nouser_xattr,noacl,mode=lfs
> 
> Here is also the status right after the warnings:
> 
> =====[ partition info(sda). #0, RW]=====
> [SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)]
> 
> Utilization: 18% (5357903 valid blocks)
>   - Node: 6279 (Inode: 976, Other: 5303)
>   - Data: 5351624
>   - Inline_xattr Inode: 0
>   - Inline_data Inode: 0
>   - Inline_dentry Inode: 0
>   - Orphan Inode: 0
> 
> Main area: 59149 segs, 59149 secs 59149 zones
>   - COLD  data: 12160, 12160, 12160
>   - WARM  data: 11878, 11878, 11878
>   - HOT   data: 59117, 59117, 59117
>   - Dir   dnode: 59118, 59118, 59118
>   - File   dnode: 59115, 59115, 59115
>   - Indir nodes: 59129, 59129, 59129
> 
>   - Valid: 380
>   - Dirty: 11221
>   - Prefree: 0
>   - Free: 47548 (47548)
> 
> CP calls: 4 (BG: 0)
> GC calls: 0 (BG: 0)
>   - data segments : 0 (0)
>   - node segments : 0 (0)
> Try to move 0 blocks (BG: 0)
>   - data blocks : 0 (0)
>   - node blocks : 0 (0)
> 
> Extent Cache:
>   - Hit Count: L1-1:26 L1-2:2 L2:1
>   - Hit Ratio: 5% (29 / 574)
>   - Inner Struct Count: tree: 973(0), node: 0
> 
> Balancing F2FS Async:
>   - inmem:    0, wb_bios:    0
>   - nodes:    4 in    9
>   - dents:    2 in dirs:   2 (   0)
>   - datas:    0 in files:   0
>   - meta:    0 in   23
>   - NATs:         3/      167
>   - SITs:         3/    59149
>   - free_nids:      3587
> 
> Distribution of User Blocks: [ valid | invalid | free ]
>   [---------|--|---------------------------------------]
> 
> IPU: 0 blocks
> SSR: 0 blocks in 0 segments
> LFS: 3394 blocks in 7 segments
> 
> BDF: 93, avg. vblocks: 460
> 
> Memory: 12820 KB
>   - static: 12597 KB
>   - cached: 94 KB
>   - paged : 128 KB
> 
> 
> > Can you provide fsck.f2fs messages?
> 
> Sure:
> 
> Info: sector size = 512
> Info: total sectors = 243253248 (in 512 bytes)
> Info: MKFS version
>   "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"
> Info: FSCK version
>   from "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"
>     to "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"
> 
> [FSCK] Unreachable nat entries                        [Ok..] [0x0]
> [FSCK] SIT valid block bitmap checking                [Ok..]
> [FSCK] Hard link checking for regular file            [Ok..] [0x0]
> [FSCK] valid_block_count matching with CP             [Ok..] [0x51c14f]
> [FSCK] valid_node_count matcing with CP (de lookup)   [Ok..] [0x1887]
> [FSCK] valid_node_count matcing with CP (nat lookup)  [Ok..] [0x1887]
> [FSCK] valid_inode_count matched with CP              [Ok..] [0x3d0]
> [FSCK] free segment_count matched with CP             [Ok..] [0xb9bc]
> [FSCK] next block offset is free                      [Ok..]
> [FSCK] fixing SIT types
> [FSCK] other corrupted bugs                           [Ok..]
> 
> Done.

This means there is no inconsistency in the image.
Does WARN_ON occur all the time after mouting this device?

Thanks,

> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Jaegeuk Kim]

On Thu, Aug 25, 2016 at 11:14:03PM +0300, Alexander Gordeev wrote:
> Hi Jaegeuk,
> 
> Thanks for all the help!
> 
> 24.08.2016, 00:12, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> > On Mon, Aug 22, 2016 at 11:52:12PM +0300, Alexander Gordeev wrote:
> >>  Hi,
> >>
> >>  I ran the test over weekend and I think I have some interesting results.
> >>  I used 1 new SD card in one device and two fully utilized SD cards,
> >>  that have problems with write latency, on two oter devices.
> >>  I mounted all the cards with mode=lfs. The new SD card got utilized at 95% after some time.
> >>  Here is the current status after the archive is rotated for some time:
> >>
> >>  =====[ partition info(sda1). #0, RW]=====
> >>  [SB: 1] [CP: 2] [SIT: 2] [NAT: 68] [SSA: 30] [MAIN: 15112(OverProv:803 Resv:50)]
> >>
> >>  Utilization: 94% (6929763 valid blocks)
> >>    - Node: 8113 (Inode: 1255, Other: 6858)
> >>    - Data: 6921650
> >>    - Inline_xattr Inode: 0
> >>    - Inline_data Inode: 1
> >>    - Inline_dentry Inode: 0
> >>    - Orphan Inode: 0
> >>
> >>  Main area: 15112 segs, 7556 secs 7556 zones
> >>    - COLD data: 5306, 2653, 2653
> >>    - WARM data: 5233, 2616, 2616
> >>    - HOT data: 15100, 7550, 7550
> >>    - Dir dnode: 15097, 7548, 7548
> >>    - File dnode: 4701, 2350, 2350
> >>    - Indir nodes: 15105, 7552, 7552
> >>
> >>    - Valid: 97
> >>    - Dirty: 13798
> >>    - Prefree: 0
> >>    - Free: 1217 (604)
> >>
> >>  CP calls: 282 (BG: 0)
> >>  GC calls: 0 (BG: 0)
> >>    - data segments : 0 (0)
> >>    - node segments : 0 (0)
> >>  Try to move 0 blocks (BG: 0)
> >>    - data blocks : 0 (0)
> >>    - node blocks : 0 (0)
> >>
> >>  Extent Cache:
> >>    - Hit Count: L1-1:3084 L1-2:456 L2:0
> >>    - Hit Ratio: 4% (3540 / 84026)
> >>    - Inner Struct Count: tree: 1252(0), node: 0
> >>
> >>  Balancing F2FS Async:
> >>    - inmem: 0, wb_bios: 0
> >>    - nodes: 12 in 30
> >>    - dents: 3 in dirs: 2 ( 2)
> >>    - datas: 48 in files: 0
> >>    - meta: 9 in 34
> >>    - NATs: 10/ 249
> >>    - SITs: 13/ 15112
> >>    - free_nids: 1797
> >>
> >>  Distribution of User Blocks: [ valid | invalid | free ]
> >>    [-----------------------------------------------|--|-]
> >>
> >>  IPU: 0 blocks
> >>  SSR: 0 blocks in 0 segments
> >>  LFS: 912188 blocks in 1781 segments
> >>
> >>  BDF: 94, avg. vblocks: 996
> >>
> >>  Memory: 3604 KB
> >>    - static: 3270 KB
> >>    - cached: 78 KB
> >>    - paged : 256 KB
> >>
> >>  The interesting thing here is the very small number of valid and
> >>  a huge number of dirty sections. I don't understand this at all.
> >
> > This is the number of dirty segments, so it needs to consider section and
> > segment at the same time; a dirty section can consist of valid and free
> > segments.
> > How abouting testing 2MB-sized section which is the default option?
> 
> I tried what you said. Still the majority of segments are dirty for some reason:
> 
> =====[ partition info(sda). #0, RW]=====
> [SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)]
> 
> Utilization: 10% (3013093 valid blocks)
>   - Node: 3528 (Inode: 548, Other: 2980)
>   - Data: 3009565
>   - Inline_xattr Inode: 0
>   - Inline_data Inode: 0
>   - Inline_dentry Inode: 0
>   - Orphan Inode: 0
> 
> Main area: 59149 segs, 59149 secs 59149 zones
>   - COLD  data: 7183, 7183, 7183
>   - WARM  data: 6654, 6654, 6654
>   - HOT   data: 59134, 59134, 59134
>   - Dir   dnode: 59127, 59127, 59127
>   - File   dnode: 59125, 59125, 59125
>   - Indir nodes: 59129, 59129, 59129
> 
>   - Valid: 300
>   - Dirty: 6438
>   - Prefree: 0
>   - Free: 52411 (52411)
> 
> CP calls: 1023 (BG: 473)
> GC calls: 470 (BG: 470)
>   - data segments : 466 (466)
>   - node segments : 4 (4)
> Try to move 152221 blocks (BG: 152221)
>   - data blocks : 151417 (151417)
>   - node blocks : 804 (804)
> 
> Extent Cache:
>   - Hit Count: L1-1:6262 L1-2:0 L2:0
>   - Hit Ratio: 2% (6262 / 273606)
>   - Inner Struct Count: tree: 292(0), node: 8
> 
> Balancing F2FS Async:
>   - inmem:    0, wb_bios:    0
>   - nodes:    0 in    0
>   - dents:    0 in dirs:   0 (   0)
>   - datas:    0 in files:   0
>   - meta:    0 in    0
>   - NATs:         0/       43
>   - SITs:         0/    59149
>   - free_nids:      3414
> 
> Distribution of User Blocks: [ valid | invalid | free ]
>   [-----|--|-------------------------------------------]
> 
> IPU: 0 blocks
> SSR: 0 blocks in 0 segments
> LFS: 3691542 blocks in 7208 segments
> 
> BDF: 95, avg. vblocks: 444
> 
> Memory: 12662 KB
>   - static: 12597 KB
>   - cached: 64 KB
>   - paged : 0 KB
> 
> 
> But the archive is working perfectly as before.

Okay, so we need to gather more information about IO traces. :)

Could you get them by:

echo 1 > /sys/kernel/debug/tracing/events/f2fs/f2fs_submit_write_bio/enable
echo 1 > /sys/kernel/debug/tracing/events/f2fs/f2fs_submit_page_mbio/enable
echo 1 > /sys/kernel/debug/tracing/tracing_on
cat /sys/kernel/debug/tracing/trace_pipe

You can get a script in f2fs-tools.git/scripts/tracepoint.sh

Thanks,

> 
> >>  Still the archive is working perfectly. Also I see, that there GC is never
> >>  called, which is probably an indication of FS working exactly as
> >>  we expected.
> >>  Also the small number of cold sections does not make problems.
> >>  So, well, it works perfect so fat. But I don't understand everything here.
> >>  Is this expected?
> >
> > So, I'm in doubt that dirty sections consist of entirely valid or free segments.
> >
> >>  The other two SD cards were tested differently. On one of them I called
> >>  ioctl(F2FS_IOC_GARBAGE_COLLECT) for several hours. And indeed the number
> >>  of dirty sectoins dropped considerably. It works fine so far.
> >
> > It makes sense that valid segments in dirty sections would be migrated to
> > different free sections.
> >
> >>  On the other SD card I called ioctl(F2FS_IOC_DEFRAGMENT) for every
> >>  file in the archive. It works fine as well now. But the number of dirty sections
> >>  was still very high at the end of defragmentation. I don't understand this
> >>  as well.
> >
> > This is for defragementation to the given file, which would not move blocks in
> > order to decrease the number of dirty sections.
> > I think it's not necessary for this workload.
> 
> Yes, I'd prefer to not run manual GC or defragmentation. This was just a test.
> 
> -- 
>  Alexander
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

27.08.2016, 04:15, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> Hi Alexander,
>
> On Fri, Aug 26, 2016 at 07:04:28PM +0300, Alexander Gordeev wrote:
>>  Hi Jaegeuk,
>>
>>  24.08.2016, 00:27, "Jaegeuk Kim" <jaegeuk@kernel.org>:
>>  > Hi,
>>  >
>>  > This is caused by no free segment to write data or node blocks.
>>  > In both of cases, have you tried file defragmentation?
>>  > I'd like to know whether this bug happens in normal cases as well.
>>
>>  Just got this after my main process, that writes the video archive, was
>>  killed by OOM killer, and the system restarted thanks to
>>  vm.panic_on_oom + kernel.panic. Then right after the restart I got this:
>>
>>  [ 26.845556] ------------[ cut here ]------------
>>  [ 26.861986] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:876 update_sit_entry+0x118/0x218 [f2fs]()
>>  [ 26.881959] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage evdev bcmdv
>>  [ 26.971301] CPU: 0 PID: 525 Comm: videoserverd Tainted: P O 3.10.93 #1
>>  [ 26.996438] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
>>  [ 27.021738] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
>>  [ 27.041226] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
>>  [ 27.066835] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f4b647c>] (update_sit_entry+0x118/0x218 [f2fs])
>>  [ 27.133465] [<7f4b647c>] (update_sit_entry+0x118/0x218 [f2fs]) from [<7f4b800c>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
>>  [ 27.188132] [<7f4b800c>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs])
>>  [ 27.209063] [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs])
>>  [ 27.242807] [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs])
>>  [ 27.260651] [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs])
>>  [ 27.281858] [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs])
>>  [ 27.302545] [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs]) from [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
>>  [ 27.315031] [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
>>  [ 27.333922] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<800666f4>] (filemap_write_and_wait_range+0x34/0x78)
>>  [ 27.369185] [<800666f4>] (filemap_write_and_wait_range+0x34/0x78) from [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs])
>>  [ 27.384766] [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs]) from [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs])
>>  [ 27.396348] [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs]) from [<800b5fdc>] (vfs_fsync+0x20/0x28)
>>  [ 27.406225] [<800b5fdc>] (vfs_fsync+0x20/0x28) from [<800b6140>] (do_fsync+0x28/0x48)
>>  [ 27.414893] [<800b6140>] (do_fsync+0x28/0x48) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
>>  [ 27.428760] ---[ end trace d9a54d63707dc19e ]---
>>  [ 34.177477] ------------[ cut here ]------------
>>  [ 34.204559] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:881 update_sit_entry+0x16c/0x218 [f2fs]()
>>  [ 34.314185] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod mmc_block usb_storage evdev bcmdv
>>  [ 34.409462] CPU: 0 PID: 613 Comm: videoserverd Tainted: P W O 3.10.93 #1
>>  [ 34.472011] [<80012444>] (unwind_backtrace+0x0/0x118) from [<80010cdc>] (show_stack+0x10/0x14)
>>  [ 34.509232] [<80010cdc>] (show_stack+0x10/0x14) from [<8001d3c8>] (warn_slowpath_common+0x4c/0x68)
>>  [ 34.541215] [<8001d3c8>] (warn_slowpath_common+0x4c/0x68) from [<8001d474>] (warn_slowpath_null+0x18/0x20)
>>  [ 34.621879] [<8001d474>] (warn_slowpath_null+0x18/0x20) from [<7f4b64d0>] (update_sit_entry+0x16c/0x218 [f2fs])
>>  [ 34.662256] [<7f4b64d0>] (update_sit_entry+0x16c/0x218 [f2fs]) from [<7f4b8050>] (refresh_sit_entry+0x60/0xb8 [f2fs])
>>  [ 34.702123] [<7f4b8050>] (refresh_sit_entry+0x60/0xb8 [f2fs]) from [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs])
>>  [ 34.741857] [<7f4b903c>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs])
>>  [ 34.805079] [<7f4b9340>] (do_write_page+0x224/0x270 [f2fs]) from [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs])
>>  [ 34.841321] [<7f4b94b8>] (write_data_page+0x8c/0xa4 [f2fs]) from [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs])
>>  [ 34.906465] [<7f4afff4>] (do_write_data_page+0x10c/0x430 [f2fs]) from [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs])
>>  [ 34.979779] [<7f4b060c>] (f2fs_write_data_page+0x2f4/0x4d4 [f2fs]) from [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs])
>>  [ 35.009256] [<7f4ad24c>] (f2fs_write_data_pages+0x2d8/0x3dc [f2fs]) from [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68)
>>  [ 35.069102] [<800665f4>] (__filemap_fdatawrite_range+0x60/0x68) from [<800666f4>] (filemap_write_and_wait_range+0x34/0x78)
>>  [ 35.096669] [<800666f4>] (filemap_write_and_wait_range+0x34/0x78) from [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs])
>>  [ 35.119298] [<7f49bbc8>] (f2fs_do_sync_file+0xc8/0x56c [f2fs]) from [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs])
>>  [ 35.142967] [<7f49c090>] (f2fs_sync_file+0x24/0x2c [f2fs]) from [<800b5fdc>] (vfs_fsync+0x20/0x28)
>>  [ 35.194647] [<800b5fdc>] (vfs_fsync+0x20/0x28) from [<800b6140>] (do_fsync+0x28/0x48)
>>  [ 35.213048] [<800b6140>] (do_fsync+0x28/0x48) from [<8000e1c0>] (ret_fast_syscall+0x0/0x38)
>>  [ 35.239412] ---[ end trace d9a54d63707dc19f ]---
>>
>>  I consider this to be a normal situation. The filesystem was created
>>  several days ago with the default section size and mounted with these
>>  options: nodev,noexec,nosuid,noatime,nodiratime,nouser_xattr,noacl,mode=lfs
>>
>>  Here is also the status right after the warnings:
>>
>>  =====[ partition info(sda). #0, RW]=====
>>  [SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)]
>>
>>  Utilization: 18% (5357903 valid blocks)
>>    - Node: 6279 (Inode: 976, Other: 5303)
>>    - Data: 5351624
>>    - Inline_xattr Inode: 0
>>    - Inline_data Inode: 0
>>    - Inline_dentry Inode: 0
>>    - Orphan Inode: 0
>>
>>  Main area: 59149 segs, 59149 secs 59149 zones
>>    - COLD data: 12160, 12160, 12160
>>    - WARM data: 11878, 11878, 11878
>>    - HOT data: 59117, 59117, 59117
>>    - Dir dnode: 59118, 59118, 59118
>>    - File dnode: 59115, 59115, 59115
>>    - Indir nodes: 59129, 59129, 59129
>>
>>    - Valid: 380
>>    - Dirty: 11221
>>    - Prefree: 0
>>    - Free: 47548 (47548)
>>
>>  CP calls: 4 (BG: 0)
>>  GC calls: 0 (BG: 0)
>>    - data segments : 0 (0)
>>    - node segments : 0 (0)
>>  Try to move 0 blocks (BG: 0)
>>    - data blocks : 0 (0)
>>    - node blocks : 0 (0)
>>
>>  Extent Cache:
>>    - Hit Count: L1-1:26 L1-2:2 L2:1
>>    - Hit Ratio: 5% (29 / 574)
>>    - Inner Struct Count: tree: 973(0), node: 0
>>
>>  Balancing F2FS Async:
>>    - inmem: 0, wb_bios: 0
>>    - nodes: 4 in 9
>>    - dents: 2 in dirs: 2 ( 0)
>>    - datas: 0 in files: 0
>>    - meta: 0 in 23
>>    - NATs: 3/ 167
>>    - SITs: 3/ 59149
>>    - free_nids: 3587
>>
>>  Distribution of User Blocks: [ valid | invalid | free ]
>>    [---------|--|---------------------------------------]
>>
>>  IPU: 0 blocks
>>  SSR: 0 blocks in 0 segments
>>  LFS: 3394 blocks in 7 segments
>>
>>  BDF: 93, avg. vblocks: 460
>>
>>  Memory: 12820 KB
>>    - static: 12597 KB
>>    - cached: 94 KB
>>    - paged : 128 KB
>>
>>  > Can you provide fsck.f2fs messages?
>>
>>  Sure:
>>
>>  Info: sector size = 512
>>  Info: total sectors = 243253248 (in 512 bytes)
>>  Info: MKFS version
>>    "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"
>>  Info: FSCK version
>>    from "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"
>>      to "Linux version 3.10.93 (alex@deb-i386) (gcc version 4.9.1 20140625 (prerelease) (crosstool-NG - Ambarella Linaro Multilib GCC [CortexA9 & ARMv6k] 2014.06) ) #1 PREEMPT Mon Aug 22 12:09:13 MSK 2016"
>>
>>  [FSCK] Unreachable nat entries [Ok..] [0x0]
>>  [FSCK] SIT valid block bitmap checking [Ok..]
>>  [FSCK] Hard link checking for regular file [Ok..] [0x0]
>>  [FSCK] valid_block_count matching with CP [Ok..] [0x51c14f]
>>  [FSCK] valid_node_count matcing with CP (de lookup) [Ok..] [0x1887]
>>  [FSCK] valid_node_count matcing with CP (nat lookup) [Ok..] [0x1887]
>>  [FSCK] valid_inode_count matched with CP [Ok..] [0x3d0]
>>  [FSCK] free segment_count matched with CP [Ok..] [0xb9bc]
>>  [FSCK] next block offset is free [Ok..]
>>  [FSCK] fixing SIT types
>>  [FSCK] other corrupted bugs [Ok..]
>>
>>  Done.
>
> This means there is no inconsistency in the image.
> Does WARN_ON occur all the time after mouting this device?

No, it occurs rarely. I've seen this 2 times so far for at least a dozen of reboots.

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Alexander Gordeev]

Hi,

I have an update about the first of the bugs, that I reported.

19.08.2016, 20:23, "Alexander Gordeev" <alex@gordick.net>:
> Hi,
>
> I'd also like to report two bugs. I used f2fs-stable branch linux-3.10.y.
> I'm using some proprietary modules also, but in my understanding they shouldn't affect the fs.
>
> 1. I have a Sandisk 16GB microSDHC card, that is quite old and may be at its end of life.
> The bug is produced every time I try to delete any file on it.
>
> [ 28.805413] ------------[ cut here ]------------
> [ 28.810327] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1103 new_curseg+0x330/0x40c [f2fs]()
> [ 28.824971] Modules linked in:
> [ 28.828128] ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block evdev bcmdhd(O) amb
> arella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc_core ohci_hcd ehc
> i_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regmap_i2c snd_pcm snd
> _page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
> [ 28.891640] CPU: 0 PID: 489 Comm: videoserverd Tainted: P O 3.10.93 #1
> [ 28.899128] [<8001338c>] (unwind_backtrace+0x0/0x12c) from [<80011a0c>] (show_stack+0x10/0x14)
> [ 28.907795] [<80011a0c>] (show_stack+0x10/0x14) from [<8001effc>] (warn_slowpath_common+0x54/0x68)
> [ 28.916777] [<8001effc>] (warn_slowpath_common+0x54/0x68) from [<8001f0ac>] (warn_slowpath_null+0x1c/0x24)
> [ 28.926531] [<8001f0ac>] (warn_slowpath_null+0x1c/0x24) from [<7f51c220>] (new_curseg+0x330/0x40c [f2fs])
> [ 28.936364] [<7f51c220>] (new_curseg+0x330/0x40c [f2fs]) from [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs])
> [ 28.947594] [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs]) from [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs])
> [ 28.959624] [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
> [ 28.970374] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
> [ 28.980595] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
> [ 28.991328] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
> [ 29.002072] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
> [ 29.012652] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
> [ 29.022684] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
> [ 29.032013] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
> [ 29.040940] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
> [ 29.049402] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
> [ 29.058186] ---[ end trace d249769884543c44 ]---
> [ 29.062825] ------------[ cut here ]------------
> [ 29.067523] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1142 new_curseg+0x3e4/0x40c [f2fs]()
> [ 29.082113] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block e
> vdev bcmdhd(O) ambarella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc
> _core ohci_hcd ehci_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regm
> ap_i2c snd_pcm snd_page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
> [ 29.147142] CPU: 0 PID: 489 Comm: videoserverd Tainted: P W O 3.10.93 #1
> [ 29.154564] [<8001338c>] (unwind_backtrace+0x0/0x12c) from [<80011a0c>] (show_stack+0x10/0x14)
> [ 29.163211] [<80011a0c>] (show_stack+0x10/0x14) from [<8001effc>] (warn_slowpath_common+0x54/0x68)
> [ 29.172184] [<8001effc>] (warn_slowpath_common+0x54/0x68) from [<8001f0ac>] (warn_slowpath_null+0x1c/0x24)
> [ 29.181887] [<8001f0ac>] (warn_slowpath_null+0x1c/0x24) from [<7f51c2d4>] (new_curseg+0x3e4/0x40c [f2fs])
> [ 29.191508] [<7f51c2d4>] (new_curseg+0x3e4/0x40c [f2fs]) from [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs])
> [ 29.202785] [<7f51c6f0>] (allocate_segment_by_default+0x230/0x2b4 [f2fs]) from [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs])
> [ 29.214742] [<7f51ccc8>] (allocate_data_block+0x2d8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
> [ 29.225477] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
> [ 29.235822] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
> [ 29.246575] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
> [ 29.257220] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
> [ 29.267850] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
> [ 29.277887] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
> [ 29.287127] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
> [ 29.296045] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
> [ 29.304504] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
> [ 29.313271] ---[ end trace d249769884543c45 ]---
> [ 29.371882] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> [ 29.396142] pgd = 85644000
> [ 29.399051] [00000000] *pgd=0357b831, *pte=00000000, *ppte=00000000
> [ 29.407041] Internal error: Oops: 17 [#1] PREEMPT ARM
> [ 29.412085] Modules linked in: ov4689_mipi(PO) crc32 f2fs xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod usb_storage scsi_mod mmc_block e
> vdev bcmdhd(O) ambarella_crypto gpio_keys pwm_ambarella snd_soc_wm8974_amb spi_slave_ambarella snd_soc_ambarella_i2s snd_soc_ambarella ambarella_eth spi_ambarella of_mdio ambarella_sd cfg80211 rfkill mmc
> _core ohci_hcd ehci_hcd usbcore ambarella_udc ambarella_input_ir ambarella_input_adc ambdbus(PO) ambtve(PO) ambad(PO) dsplog(PO) iav(PO) snd_soc_amba_board snd_soc_ambdummy snd_soc_core snd_compress regm
> ap_i2c snd_pcm snd_page_alloc snd_timer snd soundcore regmap_spi imgproc(PO) hw_timer(PO) vout(PO) dsp(PO) ksz80x1r libphy spidev i2c_dev
> [ 29.476936] CPU: 0 PID: 489 Comm: videoserverd Tainted: P W O 3.10.93 #1
> [ 29.484309] task: 869fdc80 ti: 83350000 task.ti: 83350000
> [ 29.489728] PC is at update_sit_entry+0xc4/0x250 [f2fs]
> [ 29.494949] LR is at update_sit_entry+0x78/0x250 [f2fs]
> [ 29.500154] pc : [<7f51a74c>] lr : [<7f51a700>] psr: 20060013
> sp : 83351c48 ip : 00000000 fp : 84a0f740
> [ 29.511600] r10: 00000000 r9 : 00001d74 r8 : 00000001
> [ 29.516800] r7 : 00000000 r6 : 00000080 r5 : 859ac2e0 r4 : 857e6800
> [ 29.523301] r3 : 00000007 r2 : 00153fae r1 : 00000000 r0 : 00153fae
> [ 29.529805] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
> [ 29.536914] Control: 10c53c7d Table: 05844059 DAC: 00000015
> [ 29.542639] Process videoserverd (pid: 489, stack limit = 0x83350238)
> [ 29.549054] Stack: (0x83351c48 to 0x83352000)
> [ 29.553394] 1c40: 00000001 857e6800 003b5800 0034c075 00000004 00000001
> [ 29.561547] 1c60: 00000000 83351d38 80550080 7f51baf0 857e6800 86974cd0 86974cd0 7f51ccd8
> [ 29.569701] 1c80: 83351d2c 84a0f740 00000000 84a0f768 0034c075 83351d01 00000000 800d0d94
> [ 29.577853] 1ca0: 00000000 000000f0 60060013 7f5103cc 86455e00 83351d2c 00000004 80550080
> [ 29.586005] 1cc0: 0000010c 83351d01 857e6938 000006b0 857e6800 7f51cf84 83351d01 00000004
> [ 29.594156] 1ce0: 857e6800 00000000 83351d70 80550080 804ce684 000006b0 857e6800 7f51d128
> [ 29.602309] 1d00: 0006b080 00000000 857e6938 7f515560 00000064 8006ddd8 8054b8f4 000006b0
> [ 29.610461] 1d20: 000006b0 0034c075 86390d08 857e6800 00000001 00000411 003b5800 0034c075
> [ 29.618612] 1d40: 80550080 00000000 80550080 857e6800 00000200 857e6a80 861c2333 00000075
> [ 29.626765] 1d60: 857e6800 7f516af0 000006b0 00000001 00000001 00000000 00000000 00000000
> [ 29.634916] 1d80: 00000000 00000000 00000001 00000000 861c2333 80550080 00000000 7f50e33c
> [ 29.643069] 1da0: 00000001 83351e00 83351eb0 00001a28 000006b0 85795a50 83350000 00000000
> [ 29.651220] 1dc0: 000273c0 00000000 00001a2a 805a0840 00000001 861c2000 00000001 83351e8c
> [ 29.659372] 1de0: 0034c000 857e6804 00001a28 83350000 83351e08 000273c0 000003fc 83351e88
> [ 29.667525] 1e00: 84a0f6c0 84a0f7c0 91827364 83351e0c 83351e0c 83351e14 83351e14 000006b0
> [ 29.675676] 1e20: 000006b0 0034c075 00000008 00000400 00000000 857e6800 83351e8c 00000000
> [ 29.683828] 1e40: 00000000 00000000 857e6a80 00000400 00000000 857e6800 83351e8c 7f50e7ec
> [ 29.691979] 1e60: 00000000 ffffffea 00000000 83351ef4 00000000 00000000 00000000 00000000
> [ 29.700131] 1e80: 857e6a50 857e6a60 00001a28 83351e8c 83351e8c 00000000 00000050 00000000
> [ 29.708282] 1ea0: 00000002 84d43950 000000d6 863d83d8 863d83ec 00000004 00000000 00000000
> [ 29.716434] 1ec0: 00000000 00000000 00000000 857e6800 86390568 00000000 857e692c 84d43034
> [ 29.724586] 1ee0: 863dc720 863dc720 01da6a24 7f503e70 863d83c0 80577860 00000000 86390568
> [ 29.732738] 1f00: 863d83c0 00000000 01da9a34 ffffff9c 863d83c0 800ae7f8 01da6a24 000bda58
> [ 29.740890] 1f20: 00000000 00000000 8699e000 800ae9b0 85795a50 863acaa8 6fe7deef 00000004
> [ 29.749042] 1f40: 8699e027 ffffff9c 00000000 86214d50 86390568 00000000 00000002 00000000
> [ 29.757194] 1f60: 00000000 00000000 00000000 00000000 00000000 00000000 57b1c919 3ace5a0f
> [ 29.765346] 1f80: 57b1c919 3ace5a0f 01da9a34 00418a80 7eee3978 0000000a 8000ee44 83350000
> [ 29.773498] 1fa0: 00000000 8000ecc0 01da9a34 00418a80 01da9a34 7eee3838 81b40000 000081b4
> [ 29.781650] 1fc0: 01da9a34 00418a80 7eee3978 0000000a 01da6ef0 01dac120 7eee3978 01da6a24
> [ 29.789802] 1fe0: 76c49a71 7eee382c 76c49a79 76ca6926 800d0030 01da9a34 00000000 00000000
> [ 29.798018] [<7f51a74c>] (update_sit_entry+0xc4/0x250 [f2fs]) from [<7f51baf0>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
> [ 29.808554] [<7f51baf0>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f51ccd8>] (allocate_data_block+0x2e8/0x3d8 [f2fs])
> [ 29.819429] [<7f51ccd8>] (allocate_data_block+0x2e8/0x3d8 [f2fs]) from [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs])
> [ 29.830129] [<7f51cf84>] (do_write_page+0x1bc/0x2c8 [f2fs]) from [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs])
> [ 29.840312] [<7f51d128>] (write_node_page+0x3c/0x44 [f2fs]) from [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs])
> [ 29.851017] [<7f515560>] (f2fs_write_node_page+0xec/0x294 [f2fs]) from [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs])
> [ 29.861630] [<7f516af0>] (move_node_page+0xcc/0xf8 [f2fs]) from [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs])
> [ 29.872157] [<7f50e33c>] (do_garbage_collect+0xa24/0xcdc [f2fs]) from [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs])
> [ 29.882157] [<7f50e7ec>] (f2fs_gc+0xc8/0x618 [f2fs]) from [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs])
> [ 29.891373] [<7f503e70>] (f2fs_unlink+0x40/0xe8 [f2fs]) from [<800ae7f8>] (vfs_unlink+0x78/0x114)
> [ 29.900231] [<800ae7f8>] (vfs_unlink+0x78/0x114) from [<800ae9b0>] (do_unlinkat+0x11c/0x1a8)
> [ 29.908672] [<800ae9b0>] (do_unlinkat+0x11c/0x1a8) from [<8000ecc0>] (ret_fast_syscall+0x0/0x38)
> [ 29.917434] Code: e2063007 e3a06001 e1a06316 e1a071aa (e7d121aa)
> [ 29.974780] ---[ end trace d249769884543c46 ]---

As you can see, after to warnings goes an oops. I think it was overlooked.
But it makes the most trouble to me. 

I tried to understand what's going on.
It turns out that se->cur_valid_map is NULL in update_sit_entry().
Probably, this is not expected.

I made this patch:

diff --git a/ambarella/kernel/linux-3.10/fs/f2fs/segment.c b/ambarella/kernel/linux-3.10/fs/f2fs/segment.c
index 20eb067..767394e 100644
--- a/ambarella/kernel/linux-3.10/fs/f2fs/segment.c
+++ b/ambarella/kernel/linux-3.10/fs/f2fs/segment.c
@@ -872,6 +872,7 @@ static void update_sit_entry(struct f2fs_sb_info *sbi, block_t blkaddr, int del)
 
        /* Update valid block bitmap */
        if (del > 0) {
+               f2fs_bug_on(sbi, !se->cur_valid_map);
                if (f2fs_test_and_set_bit(offset, se->cur_valid_map))
                        f2fs_bug_on(sbi, 1);
                if (!f2fs_test_and_set_bit(offset, se->discard_map))

And got this new warning:

[   49.878738] ------------[ cut here ]------------
[   49.913650] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:875 update_sit_entry+0xfc/0x274 [f2fs]()
[   49.956951] Modules linked in: crc32 f2fs ov4689_mipi(PO) xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod bcmdhd(O) cfg80v
[   50.047314] CPU: 0 PID: 421 Comm: f2fs_gc-8:0 Tainted: P        W  O 3.10.93 #1
[   50.066051] [<800128d0>] (unwind_backtrace+0x0/0x118) from [<80011180>] (show_stack+0x10/0x14)
[   50.097576] [<80011180>] (show_stack+0x10/0x14) from [<80020200>] (warn_slowpath_common+0x4c/0x68)
[   50.117019] [<80020200>] (warn_slowpath_common+0x4c/0x68) from [<800202ac>] (warn_slowpath_null+0x18/0x20)
[   50.127578] [<800202ac>] (warn_slowpath_null+0x18/0x20) from [<7f536b08>] (update_sit_entry+0xfc/0x274 [f2fs])
[   50.145997] [<7f536b08>] (update_sit_entry+0xfc/0x274 [f2fs]) from [<7f538838>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
[   50.157110] [<7f538838>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f539840>] (allocate_data_block+0x238/0x318 [f2fs])
[   50.170414] [<7f539840>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f539b44>] (do_write_page+0x224/0x270 [f2fs])
[   50.211114] [<7f539b44>] (do_write_page+0x224/0x270 [f2fs]) from [<7f539c28>] (write_node_page+0x38/0x40 [f2fs])
[   50.235535] [<7f539c28>] (write_node_page+0x38/0x40 [f2fs]) from [<7f5322fc>] (f2fs_write_node_page+0x1f8/0x2e4 [f2fs])
[   50.262378] [<7f5322fc>] (f2fs_write_node_page+0x1f8/0x2e4 [f2fs]) from [<7f533a48>] (move_node_page+0xa8/0xe8 [f2fs])
[   50.303626] [<7f533a48>] (move_node_page+0xa8/0xe8 [f2fs]) from [<7f52ac60>] (do_garbage_collect+0x3b8/0xbbc [f2fs])
[   50.316456] [<7f52ac60>] (do_garbage_collect+0x3b8/0xbbc [f2fs]) from [<7f52b768>] (f2fs_gc+0x304/0x4e8 [f2fs])
[   50.327530] [<7f52b768>] (f2fs_gc+0x304/0x4e8 [f2fs]) from [<7f52bbc0>] (gc_thread_func+0x274/0x368 [f2fs])
[   50.340949] [<7f52bbc0>] (gc_thread_func+0x274/0x368 [f2fs]) from [<8003eb28>] (kthread+0xa0/0xac)
[   50.350623] [<8003eb28>] (kthread+0xa0/0xac) from [<8000dd60>] (ret_from_fork+0x14/0x34)
[   50.359590] ---[ end trace 2e2fd9e8fd78501a ]---

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Jaegeuk Kim]

On Mon, Aug 29, 2016 at 07:50:24PM +0300, Alexander Gordeev wrote:
> Hi,
> 
> I have an update about the first of the bugs, that I reported.
> 
> 19.08.2016, 20:23, "Alexander Gordeev" <alex@gordick.net>:
> > Hi,
> >
> > I'd also like to report two bugs. I used f2fs-stable branch linux-3.10.y.
> > I'm using some proprietary modules also, but in my understanding they shouldn't affect the fs.
> >
> > 1. I have a Sandisk 16GB microSDHC card, that is quite old and may be at its end of life.
> > The bug is produced every time I try to delete any file on it.
> >

...

> As you can see, after to warnings goes an oops. I think it was overlooked.
> But it makes the most trouble to me. 
> 
> I tried to understand what's going on.
> It turns out that se->cur_valid_map is NULL in update_sit_entry().
> Probably, this is not expected.

Thank you for the analysis. :)
It seems the blkaddr is out-of-range.
Can you print out its blkaddr?

If possible,  could you print out:
  curseg->segno, curseg->next_blkoff,a curseg->next_segno
  at the end of allocate_segment_by_default()?

Thanks,

> 
> I made this patch:
> 
> diff --git a/ambarella/kernel/linux-3.10/fs/f2fs/segment.c b/ambarella/kernel/linux-3.10/fs/f2fs/segment.c
> index 20eb067..767394e 100644
> --- a/ambarella/kernel/linux-3.10/fs/f2fs/segment.c
> +++ b/ambarella/kernel/linux-3.10/fs/f2fs/segment.c
> @@ -872,6 +872,7 @@ static void update_sit_entry(struct f2fs_sb_info *sbi, block_t blkaddr, int del)
>  
>         /* Update valid block bitmap */
>         if (del > 0) {
> +               f2fs_bug_on(sbi, !se->cur_valid_map);
>                 if (f2fs_test_and_set_bit(offset, se->cur_valid_map))
>                         f2fs_bug_on(sbi, 1);
>                 if (!f2fs_test_and_set_bit(offset, se->discard_map))
> 
> And got this new warning:
> 
> [   49.878738] ------------[ cut here ]------------
> [   49.913650] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:875 update_sit_entry+0xfc/0x274 [f2fs]()
> [   49.956951] Modules linked in: crc32 f2fs ov4689_mipi(PO) xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter ip_tables x_tables sd_mod bcmdhd(O) cfg80v
> [   50.047314] CPU: 0 PID: 421 Comm: f2fs_gc-8:0 Tainted: P        W  O 3.10.93 #1
> [   50.066051] [<800128d0>] (unwind_backtrace+0x0/0x118) from [<80011180>] (show_stack+0x10/0x14)
> [   50.097576] [<80011180>] (show_stack+0x10/0x14) from [<80020200>] (warn_slowpath_common+0x4c/0x68)
> [   50.117019] [<80020200>] (warn_slowpath_common+0x4c/0x68) from [<800202ac>] (warn_slowpath_null+0x18/0x20)
> [   50.127578] [<800202ac>] (warn_slowpath_null+0x18/0x20) from [<7f536b08>] (update_sit_entry+0xfc/0x274 [f2fs])
> [   50.145997] [<7f536b08>] (update_sit_entry+0xfc/0x274 [f2fs]) from [<7f538838>] (refresh_sit_entry+0x1c/0xb8 [f2fs])
> [   50.157110] [<7f538838>] (refresh_sit_entry+0x1c/0xb8 [f2fs]) from [<7f539840>] (allocate_data_block+0x238/0x318 [f2fs])
> [   50.170414] [<7f539840>] (allocate_data_block+0x238/0x318 [f2fs]) from [<7f539b44>] (do_write_page+0x224/0x270 [f2fs])
> [   50.211114] [<7f539b44>] (do_write_page+0x224/0x270 [f2fs]) from [<7f539c28>] (write_node_page+0x38/0x40 [f2fs])
> [   50.235535] [<7f539c28>] (write_node_page+0x38/0x40 [f2fs]) from [<7f5322fc>] (f2fs_write_node_page+0x1f8/0x2e4 [f2fs])
> [   50.262378] [<7f5322fc>] (f2fs_write_node_page+0x1f8/0x2e4 [f2fs]) from [<7f533a48>] (move_node_page+0xa8/0xe8 [f2fs])
> [   50.303626] [<7f533a48>] (move_node_page+0xa8/0xe8 [f2fs]) from [<7f52ac60>] (do_garbage_collect+0x3b8/0xbbc [f2fs])
> [   50.316456] [<7f52ac60>] (do_garbage_collect+0x3b8/0xbbc [f2fs]) from [<7f52b768>] (f2fs_gc+0x304/0x4e8 [f2fs])
> [   50.327530] [<7f52b768>] (f2fs_gc+0x304/0x4e8 [f2fs]) from [<7f52bbc0>] (gc_thread_func+0x274/0x368 [f2fs])
> [   50.340949] [<7f52bbc0>] (gc_thread_func+0x274/0x368 [f2fs]) from [<8003eb28>] (kthread+0xa0/0xac)
> [   50.350623] [<8003eb28>] (kthread+0xa0/0xac) from [<8000dd60>] (ret_from_fork+0x14/0x34)
> [   50.359590] ---[ end trace 2e2fd9e8fd78501a ]---
> 
> -- 
>  Alexander

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Jaegeuk Kim]

On Mon, Aug 29, 2016 at 03:23:06PM +0300, Alexander Gordeev wrote:
> Hi Jaegeuk,
> 
> 27.08.2016, 04:20, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> > On Thu, Aug 25, 2016 at 11:14:03PM +0300, Alexander Gordeev wrote:
> >>  Hi Jaegeuk,
> >>
> >>  Thanks for all the help!
> >>...
> >>  > This is the number of dirty segments, so it needs to consider section and
> >>  > segment at the same time; a dirty section can consist of valid and free
> >>  > segments.
> >>  > How abouting testing 2MB-sized section which is the default option?
> >>
> >>  I tried what you said. Still the majority of segments are dirty for some reason:
> >>
> >>  =====[ partition info(sda). #0, RW]=====
> >>  [SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)]
> >>
> >>  Utilization: 10% (3013093 valid blocks)
> >>    - Node: 3528 (Inode: 548, Other: 2980)
> >>    - Data: 3009565
> >>    - Inline_xattr Inode: 0
> >>    - Inline_data Inode: 0
> >>    - Inline_dentry Inode: 0
> >>    - Orphan Inode: 0
> >>
> >>  Main area: 59149 segs, 59149 secs 59149 zones
> >>    - COLD data: 7183, 7183, 7183
> >>    - WARM data: 6654, 6654, 6654
> >>    - HOT data: 59134, 59134, 59134
> >>    - Dir dnode: 59127, 59127, 59127
> >>    - File dnode: 59125, 59125, 59125
> >>    - Indir nodes: 59129, 59129, 59129
> >>
> >>    - Valid: 300
> >>    - Dirty: 6438
> >>    - Prefree: 0
> >>    - Free: 52411 (52411)
> >>
> >>  CP calls: 1023 (BG: 473)
> >>  GC calls: 470 (BG: 470)
> >>    - data segments : 466 (466)
> >>    - node segments : 4 (4)
> >>  Try to move 152221 blocks (BG: 152221)
> >>    - data blocks : 151417 (151417)
> >>    - node blocks : 804 (804)
> >>
> >>  Extent Cache:
> >>    - Hit Count: L1-1:6262 L1-2:0 L2:0
> >>    - Hit Ratio: 2% (6262 / 273606)
> >>    - Inner Struct Count: tree: 292(0), node: 8
> >>
> >>  Balancing F2FS Async:
> >>    - inmem: 0, wb_bios: 0
> >>    - nodes: 0 in 0
> >>    - dents: 0 in dirs: 0 ( 0)
> >>    - datas: 0 in files: 0
> >>    - meta: 0 in 0
> >>    - NATs: 0/ 43
> >>    - SITs: 0/ 59149
> >>    - free_nids: 3414
> >>
> >>  Distribution of User Blocks: [ valid | invalid | free ]
> >>    [-----|--|-------------------------------------------]
> >>
> >>  IPU: 0 blocks
> >>  SSR: 0 blocks in 0 segments
> >>  LFS: 3691542 blocks in 7208 segments
> >>
> >>  BDF: 95, avg. vblocks: 444
> >>
> >>  Memory: 12662 KB
> >>    - static: 12597 KB
> >>    - cached: 64 KB
> >>    - paged : 0 KB
> >>
> >>  But the archive is working perfectly as before.
> >
> > Okay, so we need to gather more information about IO traces. :)
> >
> > Could you get them by:
> >
> > echo 1 > /sys/kernel/debug/tracing/events/f2fs/f2fs_submit_write_bio/enable
> > echo 1 > /sys/kernel/debug/tracing/events/f2fs/f2fs_submit_page_mbio/enable
> > echo 1 > /sys/kernel/debug/tracing/tracing_on
> > cat /sys/kernel/debug/tracing/trace_pipe
> >
> > You can get a script in f2fs-tools.git/scripts/tracepoint.sh
> 
> I collected the trace. It is attached. Thanks!

Thanks.

What I've found from your trace are:
- there are two files (ino=17690, ino=17691) which shared the data log.
- ino=17690 writes data sequentiallly, and ino=17691 writes small data randomly.
- ino=17690 writes misaligned 4KB blocks at every around 296KB which produces
 dirty segments.

Could you check all the writes and truncation in your app are aligned to 4KB?
And, if ino=17691 is sqlite, it needs to check whether it is reaaly using other
data log.

Thanks,

> 
> -- 
>  Alexander



------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

29.08.2016, 21:00, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> On Mon, Aug 29, 2016 at 07:50:24PM +0300, Alexander Gordeev wrote:
>>  Hi,
>>
>>  I have an update about the first of the bugs, that I reported.
>>
>>  19.08.2016, 20:23, "Alexander Gordeev" <alex@gordick.net>:
>>  > Hi,
>>  >
>>  > I'd also like to report two bugs. I used f2fs-stable branch linux-3.10.y.
>>  > I'm using some proprietary modules also, but in my understanding they shouldn't affect the fs.
>>  >
>>  > 1. I have a Sandisk 16GB microSDHC card, that is quite old and may be at its end of life.
>>  > The bug is produced every time I try to delete any file on it.
>>  >
>
> ...
>
>>  As you can see, after to warnings goes an oops. I think it was overlooked.
>>  But it makes the most trouble to me.
>>
>>  I tried to understand what's going on.
>>  It turns out that se->cur_valid_map is NULL in update_sit_entry().
>>  Probably, this is not expected.
>
> Thank you for the analysis. :)
> It seems the blkaddr is out-of-range.
> Can you print out its blkaddr?
>
> If possible, could you print out:
>   curseg->segno, curseg->next_blkoff,a curseg->next_segno
>   at the end of allocate_segment_by_default()?

Thank you for the help!
Here are the new traces:

[   49.274678] ------------[ cut here ]------------
[   49.280216] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x268/0x394 [f2fs]()
...
[   49.518054] ---[ end trace 49dca462b4f988ff ]---
[   49.522689] ------------[ cut here ]------------
[   49.527375] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x2f0/0x394 [f2fs]()
...
[   49.764853] ---[ end trace 49dca462b4f98900 ]---
[   49.857344] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
[   49.913664] ------ blkaddr = 3889152 --------
[   49.918173] Unable to handle kernel NULL pointer dereference at virtual address 00000000
...

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Jaegeuk Kim]

On Wed, Aug 31, 2016 at 11:52:00AM +0300, Alexander Gordeev wrote:
> Hi Jaegeuk,
> 
> 29.08.2016, 21:00, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> > On Mon, Aug 29, 2016 at 07:50:24PM +0300, Alexander Gordeev wrote:
> >>  Hi,
> >>
> >>  I have an update about the first of the bugs, that I reported.
> >>
> >>  19.08.2016, 20:23, "Alexander Gordeev" <alex@gordick.net>:
> >>  > Hi,
> >>  >
> >>  > I'd also like to report two bugs. I used f2fs-stable branch linux-3.10.y.
> >>  > I'm using some proprietary modules also, but in my understanding they shouldn't affect the fs.
> >>  >
> >>  > 1. I have a Sandisk 16GB microSDHC card, that is quite old and may be at its end of life.
> >>  > The bug is produced every time I try to delete any file on it.
> >>  >
> >
> > ...
> >
> >>  As you can see, after to warnings goes an oops. I think it was overlooked.
> >>  But it makes the most trouble to me.
> >>
> >>  I tried to understand what's going on.
> >>  It turns out that se->cur_valid_map is NULL in update_sit_entry().
> >>  Probably, this is not expected.
> >
> > Thank you for the analysis. :)
> > It seems the blkaddr is out-of-range.
> > Can you print out its blkaddr?
> >
> > If possible, could you print out:
> >   curseg->segno, curseg->next_blkoff,a curseg->next_segno
> >   at the end of allocate_segment_by_default()?
> 
> Thank you for the help!
> Here are the new traces:

Thank you.

> 
> [   49.274678] ------------[ cut here ]------------
> [   49.280216] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x268/0x394 [f2fs]()

This means there is no free segment.
Could you print out the below information before this f2fs_bug_on?
- prefree_segments(sbi)
- free_segments(sbi)

> ...
> [   49.518054] ---[ end trace 49dca462b4f988ff ]---
> [   49.522689] ------------[ cut here ]------------
> [   49.527375] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x2f0/0x394 [f2fs]()

This is caused by the above segno which was -1.

> ...
> [   49.764853] ---[ end trace 49dca462b4f98900 ]---
> [   49.857344] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
> [   49.913664] ------ blkaddr = 3889152 --------
> [   49.918173] Unable to handle kernel NULL pointer dereference at virtual address 00000000

Also, we can see next_segno was -1, which incur no se entry for this segment.
At this moment, I'd really like to see its /sys/kernel/debug/f2fs/status whether
there is really not enough free segments.

Thanks,

> ...
> 
> -- 
>  Alexander

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

01.09.2016, 02:46, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> On Wed, Aug 31, 2016 at 11:52:00AM +0300, Alexander Gordeev wrote:
>>  Hi Jaegeuk,
>>
>>  29.08.2016, 21:00, "Jaegeuk Kim" <jaegeuk@kernel.org>:
>>  > On Mon, Aug 29, 2016 at 07:50:24PM +0300, Alexander Gordeev wrote:
>>  >>  Hi,
>>  >>
>>  >>  I have an update about the first of the bugs, that I reported.
>>  >>
>>  >>  19.08.2016, 20:23, "Alexander Gordeev" <alex@gordick.net>:
>>  >>  > Hi,
>>  >>  >
>>  >>  > I'd also like to report two bugs. I used f2fs-stable branch linux-3.10.y.
>>  >>  > I'm using some proprietary modules also, but in my understanding they shouldn't affect the fs.
>>  >>  >
>>  >>  > 1. I have a Sandisk 16GB microSDHC card, that is quite old and may be at its end of life.
>>  >>  > The bug is produced every time I try to delete any file on it.
>>  >>  >
>>  >
>>  > ...
>>  >
>>  >>  As you can see, after to warnings goes an oops. I think it was overlooked.
>>  >>  But it makes the most trouble to me.
>>  >>
>>  >>  I tried to understand what's going on.
>>  >>  It turns out that se->cur_valid_map is NULL in update_sit_entry().
>>  >>  Probably, this is not expected.
>>  >
>>  > Thank you for the analysis. :)
>>  > It seems the blkaddr is out-of-range.
>>  > Can you print out its blkaddr?
>>  >
>>  > If possible, could you print out:
>>  >   curseg->segno, curseg->next_blkoff,a curseg->next_segno
>>  >   at the end of allocate_segment_by_default()?
>>
>>  Thank you for the help!
>>  Here are the new traces:
>
> Thank you.
>
>>  [ 49.274678] ------------[ cut here ]------------
>>  [ 49.280216] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x268/0x394 [f2fs]()
>
> This means there is no free segment.
> Could you print out the below information before this f2fs_bug_on?
> - prefree_segments(sbi)
> - free_segments(sbi)
>
>>  ...
>>  [ 49.518054] ---[ end trace 49dca462b4f988ff ]---
>>  [ 49.522689] ------------[ cut here ]------------
>>  [ 49.527375] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x2f0/0x394 [f2fs]()
>
> This is caused by the above segno which was -1.
>
>>  ...
>>  [ 49.764853] ---[ end trace 49dca462b4f98900 ]---
>>  [ 49.857344] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
>>  [ 49.913664] ------ blkaddr = 3889152 --------
>>  [ 49.918173] Unable to handle kernel NULL pointer dereference at virtual address 00000000
>
> Also, we can see next_segno was -1, which incur no se entry for this segment.
> At this moment, I'd really like to see its /sys/kernel/debug/f2fs/status whether
> there is really not enough free segments.

I collected both /sys/kernel/debug/f2fs/status and numbers of blocks.
The oops happens only when I try to delete anything. Also it can be caused
by GC, but I turned background GC off. So I mounted the fs, dumped the status
and then tried to remove a file. Here is what I got:

=====[ partition info(sda). #0, RW]=====
[SB: 1] [CP: 2] [SIT: 2] [NAT: 34] [SSA: 16] [MAIN: 7540(OverProv:424 Resv:50)]

Utilization: 94% (3443739 valid blocks)
  - Node: 4458 (Inode: 1115, Other: 3343)
  - Data: 3439281
  - Inline_xattr Inode: 0
  - Inline_data Inode: 0
  - Inline_dentry Inode: 0
  - Orphan Inode: 0

Main area: 7540 segs, 3770 secs 3770 zones
  - COLD  data: 6192, 3096, 3096
  - WARM  data: 6186, 3093, 3093
  - HOT   data: 6198, 3099, 3099
  - Dir   dnode: 4836, 2418, 2418
  - File   dnode: 5003, 2501, 2501
  - Indir nodes: 7533, 3766, 3766

  - Valid: 6728
  - Dirty: 5
  - Prefree: 0
  - Free: 807 (0)

CP calls: 0 (BG: 0)
GC calls: 0 (BG: 0)
  - data segments : 0 (0)
  - node segments : 0 (0)
Try to move 0 blocks (BG: 0)
  - data blocks : 0 (0)
  - node blocks : 0 (0)

Extent Cache:
  - Hit Count: L1-1:0 L1-2:0 L2:0
  - Hit Ratio: 0% (0 / 0)
  - Inner Struct Count: tree: 0(0), node: 0

Balancing F2FS Async:
  - inmem:    0, wb_bios:    0
  - nodes:    0 in    1
  - dents:    0 in dirs:   0 (   0)
  - datas:    0 in files:   0
  - meta:    0 in  155
  - NATs:         0/        1
  - SITs:         0/     7540
  - free_nids:      3496

Distribution of User Blocks: [ valid | invalid | free ]
  [-----------------------------------------------|-|--]

IPU: 0 blocks
SSR: 0 blocks in 0 segments
LFS: 0 blocks in 0 segments

BDF: 78, avg. vblocks: 510

Memory: 2325 KB
  - static: 1647 KB
  - cached: 54 KB
  - paged : 624 KB

[  107.562070] ------ blkaddr = 2590606, se->cur_valid_map = 8477f040 --------
[  107.570415] ------ blkaddr = 2590607, se->cur_valid_map = 8477f040 --------
[  107.577537] ------ blkaddr = 2590608, se->cur_valid_map = 8477f040 --------
[  107.585093] ------ blkaddr = 2590609, se->cur_valid_map = 8477f040 --------
[  107.592470] ------ blkaddr = 2590610, se->cur_valid_map = 8477f040 --------
[  107.599760] ------ blkaddr = 2590611, se->cur_valid_map = 8477f040 --------
[  107.606907] ------ blkaddr = 2590612, se->cur_valid_map = 8477f040 --------
[  107.614333] ------ blkaddr = 2590613, se->cur_valid_map = 8477f040 --------
[  107.621706] ------ blkaddr = 2590614, se->cur_valid_map = 8477f040 --------
[  107.628913] ------ blkaddr = 2590615, se->cur_valid_map = 8477f040 --------
[  107.636142] ------ blkaddr = 2590616, se->cur_valid_map = 8477f040 --------
[  107.643401] ------ blkaddr = 2590617, se->cur_valid_map = 8477f040 --------
[  107.650785] ------ blkaddr = 2590618, se->cur_valid_map = 8477f040 --------
[  107.657863] ------ blkaddr = 2590619, se->cur_valid_map = 8477f040 --------
[  107.665287] ------ blkaddr = 2590620, se->cur_valid_map = 8477f040 --------
[  107.672514] ------ blkaddr = 2590621, se->cur_valid_map = 8477f040 --------
[  107.679892] ------ blkaddr = 2590622, se->cur_valid_map = 8477f040 --------
[  107.686979] ------ blkaddr = 2590623, se->cur_valid_map = 8477f040 --------
[  107.694408] ------ blkaddr = 2590624, se->cur_valid_map = 8477f040 --------
[  107.701613] ------ blkaddr = 2590625, se->cur_valid_map = 8477f040 --------
[  107.708976] ------ blkaddr = 2590626, se->cur_valid_map = 8477f040 --------
[  107.716036] ------ blkaddr = 2590627, se->cur_valid_map = 8477f040 --------
[  107.723434] ------ blkaddr = 2590628, se->cur_valid_map = 8477f040 --------
[  107.730638] ------ blkaddr = 2590629, se->cur_valid_map = 8477f040 --------
[  107.737701] ------ blkaddr = 2590630, se->cur_valid_map = 8477f040 --------
[  107.745034] ------ blkaddr = 2590631, se->cur_valid_map = 8477f040 --------
[  107.752257] ------ blkaddr = 2590632, se->cur_valid_map = 8477f040 --------
[  107.759695] ------ blkaddr = 2590633, se->cur_valid_map = 8477f040 --------
[  107.766755] ------ blkaddr = 2590634, se->cur_valid_map = 8477f040 --------
[  107.774133] ------ blkaddr = 2590635, se->cur_valid_map = 8477f040 --------
[  107.781378] ------ blkaddr = 2590636, se->cur_valid_map = 8477f040 --------
[  107.788831] ------ blkaddr = 2590637, se->cur_valid_map = 8477f040 --------
[  107.795915] ------ blkaddr = 2590638, se->cur_valid_map = 8477f040 --------
[  107.803334] ------ blkaddr = 2590639, se->cur_valid_map = 8477f040 --------
[  107.810548] ------ blkaddr = 2590640, se->cur_valid_map = 8477f040 --------
[  107.817776] ------ blkaddr = 2590641, se->cur_valid_map = 8477f040 --------
[  107.825163] ------ blkaddr = 2590642, se->cur_valid_map = 8477f040 --------
[  107.832667] ------ blkaddr = 2590643, se->cur_valid_map = 8477f040 --------
[  107.840013] ------ blkaddr = 2590644, se->cur_valid_map = 8477f040 --------
[  107.847204] ------ blkaddr = 2590645, se->cur_valid_map = 8477f040 --------
[  107.854562] ------ blkaddr = 2590646, se->cur_valid_map = 8477f040 --------
[  107.861929] ------ blkaddr = 2590647, se->cur_valid_map = 8477f040 --------
[  107.869446] ------ blkaddr = 2590648, se->cur_valid_map = 8477f040 --------
[  107.876609] ------ blkaddr = 2590649, se->cur_valid_map = 8477f040 --------
[  107.884136] ------ blkaddr = 2590650, se->cur_valid_map = 8477f040 --------
[  107.891471] ------ blkaddr = 2590651, se->cur_valid_map = 8477f040 --------
[  107.898956] ------ blkaddr = 2590652, se->cur_valid_map = 8477f040 --------
[  107.906147] ------ blkaddr = 2590653, se->cur_valid_map = 8477f040 --------
[  107.913694] ------ blkaddr = 2590654, se->cur_valid_map = 8477f040 --------
[  107.920974] ------ blkaddr = 2590655, se->cur_valid_map = 8477f040 --------
[  107.928101] ------ blkaddr = 2590656, se->cur_valid_map = 8477f040 --------
[  107.935448] ------ blkaddr = 2590657, se->cur_valid_map = 8477f040 --------
[  107.942806] ------ blkaddr = 2590658, se->cur_valid_map = 8477f040 --------
[  107.950291] ------ blkaddr = 2590659, se->cur_valid_map = 8477f040 --------
[  107.957491] ------ blkaddr = 2590660, se->cur_valid_map = 8477f040 --------
[  107.965063] ------ blkaddr = 2590661, se->cur_valid_map = 8477f040 --------
[  107.972400] ------ blkaddr = 2590662, se->cur_valid_map = 8477f040 --------
[  107.980144] ------ blkaddr = 2590663, se->cur_valid_map = 8477f040 --------
[  107.987381] ------ blkaddr = 2590664, se->cur_valid_map = 8477f040 --------
[  107.994762] ------ blkaddr = 2590665, se->cur_valid_map = 8477f040 --------
[  108.001921] ------ blkaddr = 2590666, se->cur_valid_map = 8477f040 --------
[  108.009080] ------ blkaddr = 2590667, se->cur_valid_map = 8477f040 --------
[  108.016066] ------ blkaddr = 2590668, se->cur_valid_map = 8477f040 --------
[  108.023221] ------ blkaddr = 2590669, se->cur_valid_map = 8477f040 --------
[  108.030320] ------ blkaddr = 2590670, se->cur_valid_map = 8477f040 --------
[  108.037281] ------ blkaddr = 2590671, se->cur_valid_map = 8477f040 --------
[  108.044410] ------ blkaddr = 2590672, se->cur_valid_map = 8477f040 --------
[  108.051493] ------ blkaddr = 2590673, se->cur_valid_map = 8477f040 --------
[  108.058450] ------ blkaddr = 2590674, se->cur_valid_map = 8477f040 --------
[  108.065579] ------ blkaddr = 2590675, se->cur_valid_map = 8477f040 --------
[  108.072669] ------ blkaddr = 2590676, se->cur_valid_map = 8477f040 --------
[  108.079785] ------ blkaddr = 2590677, se->cur_valid_map = 8477f040 --------
[  108.086742] ------ blkaddr = 2590678, se->cur_valid_map = 8477f040 --------
[  108.093880] ------ blkaddr = 2590679, se->cur_valid_map = 8477f040 --------
[  108.100968] ------ blkaddr = 2590680, se->cur_valid_map = 8477f040 --------
[  108.107925] ------ blkaddr = 2590681, se->cur_valid_map = 8477f040 --------
[  108.115050] ------ blkaddr = 2590682, se->cur_valid_map = 8477f040 --------
[  108.122131] ------ blkaddr = 2590683, se->cur_valid_map = 8477f040 --------
[  108.129212] ------ blkaddr = 2590684, se->cur_valid_map = 8477f040 --------
[  108.136168] ------ blkaddr = 2590685, se->cur_valid_map = 8477f040 --------
[  108.143287] ------ blkaddr = 2590686, se->cur_valid_map = 8477f040 --------
[  108.150366] ------ blkaddr = 2590687, se->cur_valid_map = 8477f040 --------
[  108.157324] ------ blkaddr = 2590688, se->cur_valid_map = 8477f040 --------
[  108.164445] ------ blkaddr = 2590689, se->cur_valid_map = 8477f040 --------
[  108.171525] ------ blkaddr = 2590690, se->cur_valid_map = 8477f040 --------
[  108.178484] ------ blkaddr = 2590691, se->cur_valid_map = 8477f040 --------
[  108.185609] ------ blkaddr = 2590692, se->cur_valid_map = 8477f040 --------
[  108.192687] ------ blkaddr = 2590693, se->cur_valid_map = 8477f040 --------
[  108.199764] ------ blkaddr = 2590694, se->cur_valid_map = 8477f040 --------
[  108.206719] ------ blkaddr = 2590695, se->cur_valid_map = 8477f040 --------
[  108.213837] ------ blkaddr = 2590696, se->cur_valid_map = 8477f040 --------
[  108.221095] ------ blkaddr = 2590697, se->cur_valid_map = 8477f040 --------
[  108.228540] ------ blkaddr = 2590698, se->cur_valid_map = 8477f040 --------
[  108.235771] ------ blkaddr = 2590699, se->cur_valid_map = 8477f040 --------
[  108.242902] ------ blkaddr = 2590700, se->cur_valid_map = 8477f040 --------
[  108.250014] ------ blkaddr = 2590701, se->cur_valid_map = 8477f040 --------
[  108.256973] ------ blkaddr = 2590702, se->cur_valid_map = 8477f040 --------
[  108.264124] ------ blkaddr = 2590703, se->cur_valid_map = 8477f040 --------
[  108.271229] ------ blkaddr = 2590704, se->cur_valid_map = 8477f040 --------
[  108.278188] ------ blkaddr = 2590705, se->cur_valid_map = 8477f040 --------
[  108.285433] ------ blkaddr = 2590706, se->cur_valid_map = 8477f040 --------
[  108.292539] ------ blkaddr = 2590707, se->cur_valid_map = 8477f040 --------
[  108.299629] ------ blkaddr = 2590708, se->cur_valid_map = 8477f040 --------
[  108.306587] ------ blkaddr = 2590709, se->cur_valid_map = 8477f040 --------
[  108.313718] ------ blkaddr = 2590710, se->cur_valid_map = 8477f040 --------
[  108.320803] ------ blkaddr = 2590711, se->cur_valid_map = 8477f040 --------
[  108.327760] ------ blkaddr = 2590712, se->cur_valid_map = 8477f040 --------
[  108.334883] ------ blkaddr = 2590713, se->cur_valid_map = 8477f040 --------
[  108.341966] ------ blkaddr = 2590714, se->cur_valid_map = 8477f040 --------
[  108.349047] ------ blkaddr = 2590715, se->cur_valid_map = 8477f040 --------
[  108.356004] ------ blkaddr = 2590716, se->cur_valid_map = 8477f040 --------
[  108.363130] ------ blkaddr = 2590717, se->cur_valid_map = 8477f040 --------
[  108.370210] ------ blkaddr = 2590718, se->cur_valid_map = 8477f040 --------
[  108.377244] ------ prefree_segments = 0, free_segments = 807 ------
[  108.383526] ------------[ cut here ]------------
[  108.388231] WARNING: at /home/alex/work/s2l/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x280/0x3b8 [f2fs]()
...
[  108.628965] ---[ end trace c59ce5906bb6d95e ]---
[  108.633563] ------------[ cut here ]------------
[  108.638240] WARNING: at /home/alex/work/s2l/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x308/0x3b8 [f2fs]()
...
[  108.878788] ---[ end trace c59ce5906bb6d95f ]---
[  108.885201] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
[  108.892808] ------ blkaddr = 2590719, se->cur_valid_map = 8477f040 --------
[  108.899825] ------ blkaddr = 3889152, se->cur_valid_map =   (null) --------
[  108.906800] Unable to handle kernel NULL pointer dereference at virtual address 00000000
...
[  109.429454] ---[ end trace c59ce5906bb6d960 ]---


Thanks!

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Jaegeuk Kim]

Hello,

On Thu, Sep 01, 2016 at 08:40:07PM +0300, Alexander Gordeev wrote:
...
> >>  [ 49.274678] ------------[ cut here ]------------
> >>  [ 49.280216] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x268/0x394 [f2fs]()
> >
> > This means there is no free segment.
> > Could you print out the below information before this f2fs_bug_on?
> > - prefree_segments(sbi)
> > - free_segments(sbi)
> >
> >>  ...
> >>  [ 49.518054] ---[ end trace 49dca462b4f988ff ]---
> >>  [ 49.522689] ------------[ cut here ]------------
> >>  [ 49.527375] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x2f0/0x394 [f2fs]()
> >
> > This is caused by the above segno which was -1.
> >
> >>  ...
> >>  [ 49.764853] ---[ end trace 49dca462b4f98900 ]---
> >>  [ 49.857344] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
> >>  [ 49.913664] ------ blkaddr = 3889152 --------
> >>  [ 49.918173] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> >
> > Also, we can see next_segno was -1, which incur no se entry for this segment.
> > At this moment, I'd really like to see its /sys/kernel/debug/f2fs/status whether
> > there is really not enough free segments.
> 
> I collected both /sys/kernel/debug/f2fs/status and numbers of blocks.
> The oops happens only when I try to delete anything. Also it can be caused
> by GC, but I turned background GC off. So I mounted the fs, dumped the status
> and then tried to remove a file. Here is what I got:
> 
> =====[ partition info(sda). #0, RW]=====
> [SB: 1] [CP: 2] [SIT: 2] [NAT: 34] [SSA: 16] [MAIN: 7540(OverProv:424 Resv:50)]

Could you check your mkfs.f2fs version?
I want to check ovp and resv numbers.

You can get the latest one from:
http://git.kernel.org/cgit/linux/kernel/git/jaegeuk/f2fs-tools.git

> Utilization: 94% (3443739 valid blocks)
>   - Node: 4458 (Inode: 1115, Other: 3343)
>   - Data: 3439281
>   - Inline_xattr Inode: 0
>   - Inline_data Inode: 0
>   - Inline_dentry Inode: 0
>   - Orphan Inode: 0
> 
> Main area: 7540 segs, 3770 secs 3770 zones
>   - COLD  data: 6192, 3096, 3096
>   - WARM  data: 6186, 3093, 3093
>   - HOT   data: 6198, 3099, 3099
>   - Dir   dnode: 4836, 2418, 2418
>   - File   dnode: 5003, 2501, 2501
>   - Indir nodes: 7533, 3766, 3766
> 
>   - Valid: 6728
>   - Dirty: 5
>   - Prefree: 0
>   - Free: 807 (0)

Here, we have 807 free segments, but no section that we can use right now.

> CP calls: 0 (BG: 0)
> GC calls: 0 (BG: 0)
>   - data segments : 0 (0)
>   - node segments : 0 (0)
> Try to move 0 blocks (BG: 0)
>   - data blocks : 0 (0)
>   - node blocks : 0 (0)
> 
> Extent Cache:
>   - Hit Count: L1-1:0 L1-2:0 L2:0
>   - Hit Ratio: 0% (0 / 0)
>   - Inner Struct Count: tree: 0(0), node: 0
> 
> Balancing F2FS Async:
>   - inmem:    0, wb_bios:    0
>   - nodes:    0 in    1
>   - dents:    0 in dirs:   0 (   0)
>   - datas:    0 in files:   0
>   - meta:    0 in  155
>   - NATs:         0/        1
>   - SITs:         0/     7540
>   - free_nids:      3496
> 
> Distribution of User Blocks: [ valid | invalid | free ]
>   [-----------------------------------------------|-|--]
> 
> IPU: 0 blocks
> SSR: 0 blocks in 0 segments
> LFS: 0 blocks in 0 segments
> 
> BDF: 78, avg. vblocks: 510
> 
> Memory: 2325 KB
>   - static: 1647 KB
>   - cached: 54 KB
>   - paged : 624 KB
> 
> [  107.562070] ------ blkaddr = 2590606, se->cur_valid_map = 8477f040 --------
> [  107.570415] ------ blkaddr = 2590607, se->cur_valid_map = 8477f040 --------
...
> [  108.334883] ------ blkaddr = 2590713, se->cur_valid_map = 8477f040 --------
> [  108.341966] ------ blkaddr = 2590714, se->cur_valid_map = 8477f040 --------
> [  108.349047] ------ blkaddr = 2590715, se->cur_valid_map = 8477f040 --------
> [  108.356004] ------ blkaddr = 2590716, se->cur_valid_map = 8477f040 --------
> [  108.363130] ------ blkaddr = 2590717, se->cur_valid_map = 8477f040 --------
> [  108.370210] ------ blkaddr = 2590718, se->cur_valid_map = 8477f040 --------
> [  108.377244] ------ prefree_segments = 0, free_segments = 807 ------
> [  108.383526] ------------[ cut here ]------------
> [  108.388231] WARNING: at /home/alex/work/s2l/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x280/0x3b8 [f2fs]()
> ...
> [  108.628965] ---[ end trace c59ce5906bb6d95e ]---
> [  108.633563] ------------[ cut here ]------------
> [  108.638240] WARNING: at /home/alex/work/s2l/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x308/0x3b8 [f2fs]()
> ...
> [  108.878788] ---[ end trace c59ce5906bb6d95f ]---
> [  108.885201] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------

So, new_curseg failed to find a free section, and gave 7540 which is the end
of main area. And then, it tried to allocate that LBA and got panic due to wrong
bitmap position when setting its SIT bitmap.

For now, the root cause is due to zero free section, and I'd like to check
whether ovp and reserved space were correctly assigned or not.
Or, there was a missing flow which consumed reserved space entirely without
any cleaning.

Thanks,

> [  108.892808] ------ blkaddr = 2590719, se->cur_valid_map = 8477f040 --------
> [  108.899825] ------ blkaddr = 3889152, se->cur_valid_map =   (null) --------
> [  108.906800] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> ...
> [  109.429454] ---[ end trace c59ce5906bb6d960 ]---
> 
> 
> Thanks!
> 
> -- 
>  Alexander

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi ,

01.09.2016, 21:25, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> Hello,
>
> On Thu, Sep 01, 2016 at 08:40:07PM +0300, Alexander Gordeev wrote:
> ...
>>  >>  [ 49.274678] ------------[ cut here ]------------
>>  >>  [ 49.280216] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x268/0x394 [f2fs]()
>>  >
>>  > This means there is no free segment.
>>  > Could you print out the below information before this f2fs_bug_on?
>>  > - prefree_segments(sbi)
>>  > - free_segments(sbi)
>>  >
>>  >>  ...
>>  >>  [ 49.518054] ---[ end trace 49dca462b4f988ff ]---
>>  >>  [ 49.522689] ------------[ cut here ]------------
>>  >>  [ 49.527375] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x2f0/0x394 [f2fs]()
>>  >
>>  > This is caused by the above segno which was -1.
>>  >
>>  >>  ...
>>  >>  [ 49.764853] ---[ end trace 49dca462b4f98900 ]---
>>  >>  [ 49.857344] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
>>  >>  [ 49.913664] ------ blkaddr = 3889152 --------
>>  >>  [ 49.918173] Unable to handle kernel NULL pointer dereference at virtual address 00000000
>>  >
>>  > Also, we can see next_segno was -1, which incur no se entry for this segment.
>>  > At this moment, I'd really like to see its /sys/kernel/debug/f2fs/status whether
>>  > there is really not enough free segments.
>>
>>  I collected both /sys/kernel/debug/f2fs/status and numbers of blocks.
>>  The oops happens only when I try to delete anything. Also it can be caused
>>  by GC, but I turned background GC off. So I mounted the fs, dumped the status
>>  and then tried to remove a file. Here is what I got:
>>
>>  =====[ partition info(sda). #0, RW]=====
>>  [SB: 1] [CP: 2] [SIT: 2] [NAT: 34] [SSA: 16] [MAIN: 7540(OverProv:424 Resv:50)]
>
> Could you check your mkfs.f2fs version?
> I want to check ovp and resv numbers.

I currently used mkfs.f2fs from my board's SDK. It prints this:
F2FS-tools: mkfs.f2fs Ver: 1.4.1 (2015-03-04)

Should I upgrade it?
I think the problem will go away if I recreate the FS. At the moment
it reproduces every time. I wonder if we need it again.

> You can get the latest one from:
> http://git.kernel.org/cgit/linux/kernel/git/jaegeuk/f2fs-tools.git
>
>>  Utilization: 94% (3443739 valid blocks)
>>    - Node: 4458 (Inode: 1115, Other: 3343)
>>    - Data: 3439281
>>    - Inline_xattr Inode: 0
>>    - Inline_data Inode: 0
>>    - Inline_dentry Inode: 0
>>    - Orphan Inode: 0
>>
>>  Main area: 7540 segs, 3770 secs 3770 zones
>>    - COLD data: 6192, 3096, 3096
>>    - WARM data: 6186, 3093, 3093
>>    - HOT data: 6198, 3099, 3099
>>    - Dir dnode: 4836, 2418, 2418
>>    - File dnode: 5003, 2501, 2501
>>    - Indir nodes: 7533, 3766, 3766
>>
>>    - Valid: 6728
>>    - Dirty: 5
>>    - Prefree: 0
>>    - Free: 807 (0)
>
> Here, we have 807 free segments, but no section that we can use right now.

Ah, I see now.

>>  CP calls: 0 (BG: 0)
>>  GC calls: 0 (BG: 0)
>>    - data segments : 0 (0)
>>    - node segments : 0 (0)
>>  Try to move 0 blocks (BG: 0)
>>    - data blocks : 0 (0)
>>    - node blocks : 0 (0)
>>
>>  Extent Cache:
>>    - Hit Count: L1-1:0 L1-2:0 L2:0
>>    - Hit Ratio: 0% (0 / 0)
>>    - Inner Struct Count: tree: 0(0), node: 0
>>
>>  Balancing F2FS Async:
>>    - inmem: 0, wb_bios: 0
>>    - nodes: 0 in 1
>>    - dents: 0 in dirs: 0 ( 0)
>>    - datas: 0 in files: 0
>>    - meta: 0 in 155
>>    - NATs: 0/ 1
>>    - SITs: 0/ 7540
>>    - free_nids: 3496
>>
>>  Distribution of User Blocks: [ valid | invalid | free ]
>>    [-----------------------------------------------|-|--]
>>
>>  IPU: 0 blocks
>>  SSR: 0 blocks in 0 segments
>>  LFS: 0 blocks in 0 segments
>>
>>  BDF: 78, avg. vblocks: 510
>>
>>  Memory: 2325 KB
>>    - static: 1647 KB
>>    - cached: 54 KB
>>    - paged : 624 KB
>>
>>  [ 107.562070] ------ blkaddr = 2590606, se->cur_valid_map = 8477f040 --------
>>  [ 107.570415] ------ blkaddr = 2590607, se->cur_valid_map = 8477f040 --------
>
> ...
>>  [ 108.334883] ------ blkaddr = 2590713, se->cur_valid_map = 8477f040 --------
>>  [ 108.341966] ------ blkaddr = 2590714, se->cur_valid_map = 8477f040 --------
>>  [ 108.349047] ------ blkaddr = 2590715, se->cur_valid_map = 8477f040 --------
>>  [ 108.356004] ------ blkaddr = 2590716, se->cur_valid_map = 8477f040 --------
>>  [ 108.363130] ------ blkaddr = 2590717, se->cur_valid_map = 8477f040 --------
>>  [ 108.370210] ------ blkaddr = 2590718, se->cur_valid_map = 8477f040 --------
>>  [ 108.377244] ------ prefree_segments = 0, free_segments = 807 ------
>>  [ 108.383526] ------------[ cut here ]------------
>>  [ 108.388231] WARNING: at /home/alex/work/s2l/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x280/0x3b8 [f2fs]()
>>  ...
>>  [ 108.628965] ---[ end trace c59ce5906bb6d95e ]---
>>  [ 108.633563] ------------[ cut here ]------------
>>  [ 108.638240] WARNING: at /home/alex/work/s2l/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x308/0x3b8 [f2fs]()
>>  ...
>>  [ 108.878788] ---[ end trace c59ce5906bb6d95f ]---
>>  [ 108.885201] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
>
> So, new_curseg failed to find a free section, and gave 7540 which is the end
> of main area. And then, it tried to allocate that LBA and got panic due to wrong
> bitmap position when setting its SIT bitmap.
>
> For now, the root cause is due to zero free section, and I'd like to check
> whether ovp and reserved space were correctly assigned or not.
> Or, there was a missing flow which consumed reserved space entirely without
> any cleaning.

I think, this is the SD card, I tried to defragment some time ago.
Probably I shouldn't ever do it.
Still IMHO the code should handle even such cases. For example, someone
can craft a FS, and it might make bad things...

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Jaegeuk Kim]

On Thu, Sep 01, 2016 at 08:04:31PM +0300, Alexander Gordeev wrote:
> Hi Jaegeuk,
> 
> 29.08.2016, 21:24, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> > On Mon, Aug 29, 2016 at 03:23:06PM +0300, Alexander Gordeev wrote:
> >>  Hi Jaegeuk,
> >>
> >>  27.08.2016, 04:20, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> >>  > On Thu, Aug 25, 2016 at 11:14:03PM +0300, Alexander Gordeev wrote:
> >>  >>  Hi Jaegeuk,
> >>  >>
> >>  >>  Thanks for all the help!
> >>  >>...
> >>  >>  > This is the number of dirty segments, so it needs to consider section and
> >>  >>  > segment at the same time; a dirty section can consist of valid and free
> >>  >>  > segments.
> >>  >>  > How abouting testing 2MB-sized section which is the default option?
> >>  >>
> >>  >>  I tried what you said. Still the majority of segments are dirty for some reason:
> >>  >>
> >>  >>  =====[ partition info(sda). #0, RW]=====
> >>  >>  [SB: 1] [CP: 2] [SIT: 6] [NAT: 114] [SSA: 116] [MAIN: 59149(OverProv:3003 Resv:48)]
> >>  >>
> >>  >>  Utilization: 10% (3013093 valid blocks)
> >>  >>    - Node: 3528 (Inode: 548, Other: 2980)
> >>  >>    - Data: 3009565
> >>  >>    - Inline_xattr Inode: 0
> >>  >>    - Inline_data Inode: 0
> >>  >>    - Inline_dentry Inode: 0
> >>  >>    - Orphan Inode: 0
> >>  >>
> >>  >>  Main area: 59149 segs, 59149 secs 59149 zones
> >>  >>    - COLD data: 7183, 7183, 7183
> >>  >>    - WARM data: 6654, 6654, 6654
> >>  >>    - HOT data: 59134, 59134, 59134
> >>  >>    - Dir dnode: 59127, 59127, 59127
> >>  >>    - File dnode: 59125, 59125, 59125
> >>  >>    - Indir nodes: 59129, 59129, 59129
> >>  >>
> >>  >>    - Valid: 300
> >>  >>    - Dirty: 6438
> >>  >>    - Prefree: 0
> >>  >>    - Free: 52411 (52411)
> >>  >>
> >>  >>  CP calls: 1023 (BG: 473)
> >>  >>  GC calls: 470 (BG: 470)
> >>  >>    - data segments : 466 (466)
> >>  >>    - node segments : 4 (4)
> >>  >>  Try to move 152221 blocks (BG: 152221)
> >>  >>    - data blocks : 151417 (151417)
> >>  >>    - node blocks : 804 (804)
> >>  >>
> >>  >>  Extent Cache:
> >>  >>    - Hit Count: L1-1:6262 L1-2:0 L2:0
> >>  >>    - Hit Ratio: 2% (6262 / 273606)
> >>  >>    - Inner Struct Count: tree: 292(0), node: 8
> >>  >>
> >>  >>  Balancing F2FS Async:
> >>  >>    - inmem: 0, wb_bios: 0
> >>  >>    - nodes: 0 in 0
> >>  >>    - dents: 0 in dirs: 0 ( 0)
> >>  >>    - datas: 0 in files: 0
> >>  >>    - meta: 0 in 0
> >>  >>    - NATs: 0/ 43
> >>  >>    - SITs: 0/ 59149
> >>  >>    - free_nids: 3414
> >>  >>
> >>  >>  Distribution of User Blocks: [ valid | invalid | free ]
> >>  >>    [-----|--|-------------------------------------------]
> >>  >>
> >>  >>  IPU: 0 blocks
> >>  >>  SSR: 0 blocks in 0 segments
> >>  >>  LFS: 3691542 blocks in 7208 segments
> >>  >>
> >>  >>  BDF: 95, avg. vblocks: 444
> >>  >>
> >>  >>  Memory: 12662 KB
> >>  >>    - static: 12597 KB
> >>  >>    - cached: 64 KB
> >>  >>    - paged : 0 KB
> >>  >>
> >>  >>  But the archive is working perfectly as before.
> >>  >
> >>  > Okay, so we need to gather more information about IO traces. :)
> >>  >
> >>  > Could you get them by:
> >>  >
> >>  > echo 1 > /sys/kernel/debug/tracing/events/f2fs/f2fs_submit_write_bio/enable
> >>  > echo 1 > /sys/kernel/debug/tracing/events/f2fs/f2fs_submit_page_mbio/enable
> >>  > echo 1 > /sys/kernel/debug/tracing/tracing_on
> >>  > cat /sys/kernel/debug/tracing/trace_pipe
> >>  >
> >>  > You can get a script in f2fs-tools.git/scripts/tracepoint.sh
> >>
> >>  I collected the trace. It is attached. Thanks!
> >
> > Thanks.
> >
> > What I've found from your trace are:
> > - there are two files (ino=17690, ino=17691) which shared the data log.
> > - ino=17690 writes data sequentiallly, and ino=17691 writes small data randomly.
> > - ino=17690 writes misaligned 4KB blocks at every around 296KB which produces
> >  dirty segments.
> >
> > Could you check all the writes and truncation in your app are aligned to 4KB?
> > And, if ino=17691 is sqlite, it needs to check whether it is reaaly using other
> > data log.
> 
> I collected more logs from both kernel tracing and strace and tried to get more
> understanding of this. I think, I get what's wrong now.
> 
> ino=17690 is a video file. ino=17691 is not SQLite, it is an index file. It is written
> 24 bytes per frame. Here is a small piece of strace log for writing a single frame:
> 
> write(19, "...", 4) = 4
> write(19, "...", 4) = 4
> write(19, "...", 2432) = 2432
> write(20, "...", 24) = 24
> 
> First three writes are writing to a video file (4 byte stream id, then 4 byte length
> and then the actual frame), then the fourth one writes to and index file. Yes, I know,
> this looks ugly. :)
> All the writes are not aligned to 4096, but there are no truncations, only
> appending.
> 
> Then, I think, I see f2fs worker thread wakes up about every two seconds to
> write dirty pages. Unfortunately it seems to write everything collected so far, even
> the most recent pages, which are not fully filled yet. I'd say that can not be
> expected, that every app will write data aligned to 4096 bytes. So this means
> more overhead and overwrites even in a more general case. Is it different in
> mode=adaptive?

No, the flushing time is controlled by vm, and you can tune that through proc.
And, IMO, even if those are append-only, it'd be worth to split index and media
files into different logs; it seems using the cold log for media file only would
be recommendable.

> The 296KB size, probably, comes from my bitrate, which is about 142KB/s, times
> 2 seconds. It is roughly the right size.
> My video FPS is about 30, so the size of data, written to an index, is about 1440
> in two seconds. This is why it looks like randow writes, I think.
> 
> Also I see from my new traces, that f2fs_submit_write_bio for other inodes
> are writing to completely different sectors. Looks like the "cold" data feature
> is working good.
> 
> To conclude:
> 1. I think I can leave everything as is because (1) there is a small number of
> rewrites and (2) I start rotating the archive at 95% utilization so given the tiny
> amount of data in index and sqlite files, this should be ok, I hope.

If both of index and media files are deleted before suffering from cleaning,
IMO, it'd be fine. You can check the cleaning information in status file.

> 2. But I'd better write both video and index files at 4096 boundary.
> 3. Or this should be fixed in f2fs. I think, there should be a configurable amount
> of time to wait for dirty page to expire. It should be written only after expiration.
> Unless a user calls fsync() of course. Is there such a tunable?
> 
> Does this make sense?

Yeah, I think you can tune flushing timing through proc entries.
(e.g., /proc/sys/vm/dirty_writeback_centisecs)

Thanks,

> 
> The new traces are attached for reference. They are filtered somewhat
> for easier side-by-side viewing.
> 
> Thanks!
> 
> -- 
>  Alexander



------------------------------------------------------------------------------

Re: video archive on a microSD card

[Jaegeuk Kim]

On Thu, Sep 01, 2016 at 10:37:36PM +0300, Alexander Gordeev wrote:
> Hi ,
> 
> 01.09.2016, 21:25, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> > Hello,
> >
> > On Thu, Sep 01, 2016 at 08:40:07PM +0300, Alexander Gordeev wrote:
> > ...
> >>  >>  [ 49.274678] ------------[ cut here ]------------
> >>  >>  [ 49.280216] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x268/0x394 [f2fs]()
> >>  >
> >>  > This means there is no free segment.
> >>  > Could you print out the below information before this f2fs_bug_on?
> >>  > - prefree_segments(sbi)
> >>  > - free_segments(sbi)
> >>  >
> >>  >>  ...
> >>  >>  [ 49.518054] ---[ end trace 49dca462b4f988ff ]---
> >>  >>  [ 49.522689] ------------[ cut here ]------------
> >>  >>  [ 49.527375] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x2f0/0x394 [f2fs]()
> >>  >
> >>  > This is caused by the above segno which was -1.
> >>  >
> >>  >>  ...
> >>  >>  [ 49.764853] ---[ end trace 49dca462b4f98900 ]---
> >>  >>  [ 49.857344] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
> >>  >>  [ 49.913664] ------ blkaddr = 3889152 --------
> >>  >>  [ 49.918173] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> >>  >
> >>  > Also, we can see next_segno was -1, which incur no se entry for this segment.
> >>  > At this moment, I'd really like to see its /sys/kernel/debug/f2fs/status whether
> >>  > there is really not enough free segments.
> >>
> >>  I collected both /sys/kernel/debug/f2fs/status and numbers of blocks.
> >>  The oops happens only when I try to delete anything. Also it can be caused
> >>  by GC, but I turned background GC off. So I mounted the fs, dumped the status
> >>  and then tried to remove a file. Here is what I got:
> >>
> >>  =====[ partition info(sda). #0, RW]=====
> >>  [SB: 1] [CP: 2] [SIT: 2] [NAT: 34] [SSA: 16] [MAIN: 7540(OverProv:424 Resv:50)]
> >
> > Could you check your mkfs.f2fs version?
> > I want to check ovp and resv numbers.
> 
> I currently used mkfs.f2fs from my board's SDK. It prints this:
> F2FS-tools: mkfs.f2fs Ver: 1.4.1 (2015-03-04)
> 
> Should I upgrade it?
> I think the problem will go away if I recreate the FS. At the moment
> it reproduces every time. I wonder if we need it again.

Yes, please upgrade to the latest version to make sure.
And, it needs to format it again.
Seems there is no further information that we can use from the old partition.

Thanks,

> 
> > You can get the latest one from:
> > http://git.kernel.org/cgit/linux/kernel/git/jaegeuk/f2fs-tools.git
> >
> >>  Utilization: 94% (3443739 valid blocks)
> >>    - Node: 4458 (Inode: 1115, Other: 3343)
> >>    - Data: 3439281
> >>    - Inline_xattr Inode: 0
> >>    - Inline_data Inode: 0
> >>    - Inline_dentry Inode: 0
> >>    - Orphan Inode: 0
> >>
> >>  Main area: 7540 segs, 3770 secs 3770 zones
> >>    - COLD data: 6192, 3096, 3096
> >>    - WARM data: 6186, 3093, 3093
> >>    - HOT data: 6198, 3099, 3099
> >>    - Dir dnode: 4836, 2418, 2418
> >>    - File dnode: 5003, 2501, 2501
> >>    - Indir nodes: 7533, 3766, 3766
> >>
> >>    - Valid: 6728
> >>    - Dirty: 5
> >>    - Prefree: 0
> >>    - Free: 807 (0)
> >
> > Here, we have 807 free segments, but no section that we can use right now.
> 
> Ah, I see now.
> 
> >>  CP calls: 0 (BG: 0)
> >>  GC calls: 0 (BG: 0)
> >>    - data segments : 0 (0)
> >>    - node segments : 0 (0)
> >>  Try to move 0 blocks (BG: 0)
> >>    - data blocks : 0 (0)
> >>    - node blocks : 0 (0)
> >>
> >>  Extent Cache:
> >>    - Hit Count: L1-1:0 L1-2:0 L2:0
> >>    - Hit Ratio: 0% (0 / 0)
> >>    - Inner Struct Count: tree: 0(0), node: 0
> >>
> >>  Balancing F2FS Async:
> >>    - inmem: 0, wb_bios: 0
> >>    - nodes: 0 in 1
> >>    - dents: 0 in dirs: 0 ( 0)
> >>    - datas: 0 in files: 0
> >>    - meta: 0 in 155
> >>    - NATs: 0/ 1
> >>    - SITs: 0/ 7540
> >>    - free_nids: 3496
> >>
> >>  Distribution of User Blocks: [ valid | invalid | free ]
> >>    [-----------------------------------------------|-|--]
> >>
> >>  IPU: 0 blocks
> >>  SSR: 0 blocks in 0 segments
> >>  LFS: 0 blocks in 0 segments
> >>
> >>  BDF: 78, avg. vblocks: 510
> >>
> >>  Memory: 2325 KB
> >>    - static: 1647 KB
> >>    - cached: 54 KB
> >>    - paged : 624 KB
> >>
> >>  [ 107.562070] ------ blkaddr = 2590606, se->cur_valid_map = 8477f040 --------
> >>  [ 107.570415] ------ blkaddr = 2590607, se->cur_valid_map = 8477f040 --------
> >
> > ...
> >>  [ 108.334883] ------ blkaddr = 2590713, se->cur_valid_map = 8477f040 --------
> >>  [ 108.341966] ------ blkaddr = 2590714, se->cur_valid_map = 8477f040 --------
> >>  [ 108.349047] ------ blkaddr = 2590715, se->cur_valid_map = 8477f040 --------
> >>  [ 108.356004] ------ blkaddr = 2590716, se->cur_valid_map = 8477f040 --------
> >>  [ 108.363130] ------ blkaddr = 2590717, se->cur_valid_map = 8477f040 --------
> >>  [ 108.370210] ------ blkaddr = 2590718, se->cur_valid_map = 8477f040 --------
> >>  [ 108.377244] ------ prefree_segments = 0, free_segments = 807 ------
> >>  [ 108.383526] ------------[ cut here ]------------
> >>  [ 108.388231] WARNING: at /home/alex/work/s2l/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x280/0x3b8 [f2fs]()
> >>  ...
> >>  [ 108.628965] ---[ end trace c59ce5906bb6d95e ]---
> >>  [ 108.633563] ------------[ cut here ]------------
> >>  [ 108.638240] WARNING: at /home/alex/work/s2l/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x308/0x3b8 [f2fs]()
> >>  ...
> >>  [ 108.878788] ---[ end trace c59ce5906bb6d95f ]---
> >>  [ 108.885201] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
> >
> > So, new_curseg failed to find a free section, and gave 7540 which is the end
> > of main area. And then, it tried to allocate that LBA and got panic due to wrong
> > bitmap position when setting its SIT bitmap.
> >
> > For now, the root cause is due to zero free section, and I'd like to check
> > whether ovp and reserved space were correctly assigned or not.
> > Or, there was a missing flow which consumed reserved space entirely without
> > any cleaning.
> 
> I think, this is the SD card, I tried to defragment some time ago.
> Probably I shouldn't ever do it.
> Still IMHO the code should handle even such cases. For example, someone
> can craft a FS, and it might make bad things...
> 
> -- 
>  Alexander

------------------------------------------------------------------------------

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

01.09.2016, 23:16, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> On Thu, Sep 01, 2016 at 10:37:36PM +0300, Alexander Gordeev wrote:
>>  Hi ,
>>
>>  01.09.2016, 21:25, "Jaegeuk Kim" <jaegeuk@kernel.org>:
>>  > Hello,
>>  >
>>  > On Thu, Sep 01, 2016 at 08:40:07PM +0300, Alexander Gordeev wrote:
>>  > ...
>>  >>  >>  [ 49.274678] ------------[ cut here ]------------
>>  >>  >>  [ 49.280216] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x268/0x394 [f2fs]()
>>  >>  >
>>  >>  > This means there is no free segment.
>>  >>  > Could you print out the below information before this f2fs_bug_on?
>>  >>  > - prefree_segments(sbi)
>>  >>  > - free_segments(sbi)
>>  >>  >
>>  >>  >>  ...
>>  >>  >>  [ 49.518054] ---[ end trace 49dca462b4f988ff ]---
>>  >>  >>  [ 49.522689] ------------[ cut here ]------------
>>  >>  >>  [ 49.527375] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x2f0/0x394 [f2fs]()
>>  >>  >
>>  >>  > This is caused by the above segno which was -1.
>>  >>  >
>>  >>  >>  ...
>>  >>  >>  [ 49.764853] ---[ end trace 49dca462b4f98900 ]---
>>  >>  >>  [ 49.857344] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
>>  >>  >>  [ 49.913664] ------ blkaddr = 3889152 --------
>>  >>  >>  [ 49.918173] Unable to handle kernel NULL pointer dereference at virtual address 00000000
>>  >>  >
>>  >>  > Also, we can see next_segno was -1, which incur no se entry for this segment.
>>  >>  > At this moment, I'd really like to see its /sys/kernel/debug/f2fs/status whether
>>  >>  > there is really not enough free segments.
>>  >>
>>  >>  I collected both /sys/kernel/debug/f2fs/status and numbers of blocks.
>>  >>  The oops happens only when I try to delete anything. Also it can be caused
>>  >>  by GC, but I turned background GC off. So I mounted the fs, dumped the status
>>  >>  and then tried to remove a file. Here is what I got:
>>  >>
>>  >>  =====[ partition info(sda). #0, RW]=====
>>  >>  [SB: 1] [CP: 2] [SIT: 2] [NAT: 34] [SSA: 16] [MAIN: 7540(OverProv:424 Resv:50)]
>>  >
>>  > Could you check your mkfs.f2fs version?
>>  > I want to check ovp and resv numbers.
>>
>>  I currently used mkfs.f2fs from my board's SDK. It prints this:
>>  F2FS-tools: mkfs.f2fs Ver: 1.4.1 (2015-03-04)
>>
>>  Should I upgrade it?
>>  I think the problem will go away if I recreate the FS. At the moment
>>  it reproduces every time. I wonder if we need it again.
>
> Yes, please upgrade to the latest version to make sure.
> And, it needs to format it again.
> Seems there is no further information that we can use from the old partition.

Ok, thanks! I upgraded f2fs-tools and recreated the FS. Works fine so far!
Still IMHO this might be a problem, if someone with a specially crafted FS on a USB
stick can cause a kernel oops on any modern Linux system. Isn't it?

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Alexander Gordeev]

Hi Jaegeuk,

01.09.2016, 23:07, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> On Thu, Sep 01, 2016 at 08:04:31PM +0300, Alexander Gordeev wrote:
>>  Hi Jaegeuk,
>>
>>  29.08.2016, 21:24, "Jaegeuk Kim" <jaegeuk@kernel.org>:
>>  > What I've found from your trace are:
>>  > - there are two files (ino=17690, ino=17691) which shared the data log.
>>  > - ino=17690 writes data sequentiallly, and ino=17691 writes small data randomly.
>>  > - ino=17690 writes misaligned 4KB blocks at every around 296KB which produces
>>  >  dirty segments.
>>  >
>>  > Could you check all the writes and truncation in your app are aligned to 4KB?
>>  > And, if ino=17691 is sqlite, it needs to check whether it is reaaly using other
>>  > data log.
>>
>>  I collected more logs from both kernel tracing and strace and tried to get more
>>  understanding of this. I think, I get what's wrong now.
>>
>>  ino=17690 is a video file. ino=17691 is not SQLite, it is an index file. It is written
>>  24 bytes per frame. Here is a small piece of strace log for writing a single frame:
>>
>>  write(19, "...", 4) = 4
>>  write(19, "...", 4) = 4
>>  write(19, "...", 2432) = 2432
>>  write(20, "...", 24) = 24
>>
>>  First three writes are writing to a video file (4 byte stream id, then 4 byte length
>>  and then the actual frame), then the fourth one writes to and index file. Yes, I know,
>>  this looks ugly. :)
>>  All the writes are not aligned to 4096, but there are no truncations, only
>>  appending.
>>
>>  Then, I think, I see f2fs worker thread wakes up about every two seconds to
>>  write dirty pages. Unfortunately it seems to write everything collected so far, even
>>  the most recent pages, which are not fully filled yet. I'd say that can not be
>>  expected, that every app will write data aligned to 4096 bytes. So this means
>>  more overhead and overwrites even in a more general case. Is it different in
>>  mode=adaptive?
>
> No, the flushing time is controlled by vm, and you can tune that through proc.
> And, IMO, even if those are append-only, it'd be worth to split index and media
> files into different logs; it seems using the cold log for media file only would
> be recommendable.
>
>>  The 296KB size, probably, comes from my bitrate, which is about 142KB/s, times
>>  2 seconds. It is roughly the right size.
>>  My video FPS is about 30, so the size of data, written to an index, is about 1440
>>  in two seconds. This is why it looks like randow writes, I think.
>>
>>  Also I see from my new traces, that f2fs_submit_write_bio for other inodes
>>  are writing to completely different sectors. Looks like the "cold" data feature
>>  is working good.
>>
>>  To conclude:
>>  1. I think I can leave everything as is because (1) there is a small number of
>>  rewrites and (2) I start rotating the archive at 95% utilization so given the tiny
>>  amount of data in index and sqlite files, this should be ok, I hope.
>
> If both of index and media files are deleted before suffering from cleaning,
> IMO, it'd be fine. You can check the cleaning information in status file.
>
>>  2. But I'd better write both video and index files at 4096 boundary.
>>  3. Or this should be fixed in f2fs. I think, there should be a configurable amount
>>  of time to wait for dirty page to expire. It should be written only after expiration.
>>  Unless a user calls fsync() of course. Is there such a tunable?
>>
>>  Does this make sense?
>
> Yeah, I think you can tune flushing timing through proc entries.
> (e.g., /proc/sys/vm/dirty_writeback_centisecs)

After searching for more information about what /proc/sys/vm/dirty_*
options do, I found this email: https://lkml.org/lkml/2013/9/10/603
Now I understand why the flush thread writes even very recent pages.
I was under a wrong impression, that it checks timestamps on a per page
basis, not per inode. So I thought, that f2fs does it differently. :) Sorry.

Well, looks like this case is now completely clear to everyone. Probably,
I should write an article about tuning f2fs for this type of workload. :)

Thank you very much for all the help!

-- 
 Alexander

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: video archive on a microSD card

[Jaegeuk Kim]

On Fri, Sep 02, 2016 at 03:05:10PM +0300, Alexander Gordeev wrote:
> Hi Jaegeuk,
> 
> 01.09.2016, 23:16, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> > On Thu, Sep 01, 2016 at 10:37:36PM +0300, Alexander Gordeev wrote:
> >>  Hi ,
> >>
> >>  01.09.2016, 21:25, "Jaegeuk Kim" <jaegeuk@kernel.org>:
> >>  > Hello,
> >>  >
> >>  > On Thu, Sep 01, 2016 at 08:40:07PM +0300, Alexander Gordeev wrote:
> >>  > ...
> >>  >>  >>  [ 49.274678] ------------[ cut here ]------------
> >>  >>  >>  [ 49.280216] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1105 new_curseg+0x268/0x394 [f2fs]()
> >>  >>  >
> >>  >>  > This means there is no free segment.
> >>  >>  > Could you print out the below information before this f2fs_bug_on?
> >>  >>  > - prefree_segments(sbi)
> >>  >>  > - free_segments(sbi)
> >>  >>  >
> >>  >>  >>  ...
> >>  >>  >>  [ 49.518054] ---[ end trace 49dca462b4f988ff ]---
> >>  >>  >>  [ 49.522689] ------------[ cut here ]------------
> >>  >>  >>  [ 49.527375] WARNING: at /home/alex/work/s2l/amb_S2l_SDK_2.5/SDK2.5/s2l_linux_sdk/ambarella/kernel/linux-3.10/fs/f2fs/segment.c:1144 new_curseg+0x2f0/0x394 [f2fs]()
> >>  >>  >
> >>  >>  > This is caused by the above segno which was -1.
> >>  >>  >
> >>  >>  >>  ...
> >>  >>  >>  [ 49.764853] ---[ end trace 49dca462b4f98900 ]---
> >>  >>  >>  [ 49.857344] ------ segno = 7540, next_blkoff = 0, next_segno = 4294967295 --------
> >>  >>  >>  [ 49.913664] ------ blkaddr = 3889152 --------
> >>  >>  >>  [ 49.918173] Unable to handle kernel NULL pointer dereference at virtual address 00000000
> >>  >>  >
> >>  >>  > Also, we can see next_segno was -1, which incur no se entry for this segment.
> >>  >>  > At this moment, I'd really like to see its /sys/kernel/debug/f2fs/status whether
> >>  >>  > there is really not enough free segments.
> >>  >>
> >>  >>  I collected both /sys/kernel/debug/f2fs/status and numbers of blocks.
> >>  >>  The oops happens only when I try to delete anything. Also it can be caused
> >>  >>  by GC, but I turned background GC off. So I mounted the fs, dumped the status
> >>  >>  and then tried to remove a file. Here is what I got:
> >>  >>
> >>  >>  =====[ partition info(sda). #0, RW]=====
> >>  >>  [SB: 1] [CP: 2] [SIT: 2] [NAT: 34] [SSA: 16] [MAIN: 7540(OverProv:424 Resv:50)]
> >>  >
> >>  > Could you check your mkfs.f2fs version?
> >>  > I want to check ovp and resv numbers.
> >>
> >>  I currently used mkfs.f2fs from my board's SDK. It prints this:
> >>  F2FS-tools: mkfs.f2fs Ver: 1.4.1 (2015-03-04)
> >>
> >>  Should I upgrade it?
> >>  I think the problem will go away if I recreate the FS. At the moment
> >>  it reproduces every time. I wonder if we need it again.
> >
> > Yes, please upgrade to the latest version to make sure.
> > And, it needs to format it again.
> > Seems there is no further information that we can use from the old partition.
> 
> Ok, thanks! I upgraded f2fs-tools and recreated the FS. Works fine so far!
> Still IMHO this might be a problem, if someone with a specially crafted FS on a USB
> stick can cause a kernel oops on any modern Linux system. Isn't it?

Yes, it is a problem. :)
But for now, we rely on any reported case that we can get.
If we can confirm this is a root-cause, hopufully I need to consider something
in kernel-- online fixing or messages whatever.

Thanks,

> 
> -- 
>  Alexander

------------------------------------------------------------------------------