* [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 @ 2021-06-08 15:24 Seongyong Park 2021-06-08 15:24 ` [PATCH v3 1/2] media: video-i2c: more precise intervals between frames Seongyong Park ` (2 more replies) 0 siblings, 3 replies; 12+ messages in thread From: Seongyong Park @ 2021-06-08 15:24 UTC (permalink / raw) To: linux-media; +Cc: Matt Ranostay, Mauro Carvalho Chehab, Seongyong Park Thanks to Matt Ranostay and Mauro Carvalho Chehab (mchehab) for guidance Seongyong Park (2): media: video-i2c: more precise intervals between frames media: video-i2c: append register data on MLX90640's frame drivers/media/i2c/video-i2c.c | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-) -- 2.31.1 ^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v3 1/2] media: video-i2c: more precise intervals between frames 2021-06-08 15:24 [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 Seongyong Park @ 2021-06-08 15:24 ` Seongyong Park 2021-06-09 7:22 ` Matt Ranostay 2021-06-08 15:24 ` [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame Seongyong Park 2021-06-09 7:08 ` [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 Matt Ranostay 2 siblings, 1 reply; 12+ messages in thread From: Seongyong Park @ 2021-06-08 15:24 UTC (permalink / raw) To: linux-media; +Cc: Matt Ranostay, Mauro Carvalho Chehab, Seongyong Park MLX90640 should ideally be working without a frame skip. In short, if a frame is skipped, then half of a frame loses correction information, having no way to retrieve its original compensation. This patch improves the timing in three ways: 1) Replaced schedule_timeout_interruptible() to usleep_range() The former "only ensures that it will sleep for at least schedule_delay (if not interrupted)", as pointed out by mchehab. As a result, the frame rate could lag behind than the actual capability of the hardware (Raspberry Pi would show a few Hz slower than set value) 2) Calculation based on us, not jiffies Jiffies usually has resolution of 100Hz, and possibly even cruder. MLX90640 can go up to 64Hz frame rate, which does not make sense to calculate the interval with aforementioned resolution. 3) Interval calculation based on the last frame's end time Using the start time of the current frame will probably make tiny bit of drift every time. This made more sense when I didn't realize 1), but it still makes sense without adding virtually any complexity, so this stays in. Signed-off-by: Seongyong Park <euphoriccatface@gmail.com> --- drivers/media/i2c/video-i2c.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c index 0465832a4..64ba96329 100644 --- a/drivers/media/i2c/video-i2c.c +++ b/drivers/media/i2c/video-i2c.c @@ -443,14 +443,15 @@ static void buffer_queue(struct vb2_buffer *vb) static int video_i2c_thread_vid_cap(void *priv) { struct video_i2c_data *data = priv; - unsigned int delay = mult_frac(HZ, data->frame_interval.numerator, - data->frame_interval.denominator); + u32 delay = mult_frac(1000000UL, data->frame_interval.numerator, + data->frame_interval.denominator); + s64 end_us = ktime_to_us(ktime_get()); set_freezable(); do { - unsigned long start_jiffies = jiffies; struct video_i2c_buffer *vid_cap_buf = NULL; + s64 current_us; int schedule_delay; try_to_freeze(); @@ -477,12 +478,14 @@ static int video_i2c_thread_vid_cap(void *priv) VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE); } - schedule_delay = delay - (jiffies - start_jiffies); - - if (time_after(jiffies, start_jiffies + delay)) - schedule_delay = delay; - - schedule_timeout_interruptible(schedule_delay); + end_us += delay; + current_us = ktime_to_us(ktime_get()); + if (current_us < end_us) { + schedule_delay = end_us - current_us; + usleep_range(schedule_delay * 3/4, schedule_delay); + } else { + end_us = current_us; + } } while (!kthread_should_stop()); return 0; -- 2.31.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v3 1/2] media: video-i2c: more precise intervals between frames 2021-06-08 15:24 ` [PATCH v3 1/2] media: video-i2c: more precise intervals between frames Seongyong Park @ 2021-06-09 7:22 ` Matt Ranostay 0 siblings, 0 replies; 12+ messages in thread From: Matt Ranostay @ 2021-06-09 7:22 UTC (permalink / raw) To: Seongyong Park; +Cc: linux-media, Mauro Carvalho Chehab On Tue, Jun 8, 2021 at 8:25 AM Seongyong Park <euphoriccatface@gmail.com> wrote: > > MLX90640 should ideally be working without a frame skip. > In short, if a frame is skipped, then half of a frame loses correction > information, having no way to retrieve its original compensation. > > This patch improves the timing in three ways: > > 1) Replaced schedule_timeout_interruptible() to usleep_range() > The former "only ensures that it will sleep for at least > schedule_delay (if not interrupted)", as pointed out by mchehab. > As a result, the frame rate could lag behind than the actual capability > of the hardware > (Raspberry Pi would show a few Hz slower than set value) > > 2) Calculation based on us, not jiffies > Jiffies usually has resolution of 100Hz, and possibly even cruder. > MLX90640 can go up to 64Hz frame rate, which does not make sense to > calculate the interval with aforementioned resolution. > > 3) Interval calculation based on the last frame's end time > Using the start time of the current frame will probably make tiny bit > of drift every time. This made more sense when I didn't realize 1), > but it still makes sense without adding virtually any complexity, > so this stays in. > > Signed-off-by: Seongyong Park <euphoriccatface@gmail.com> > --- > drivers/media/i2c/video-i2c.c | 21 ++++++++++++--------- > 1 file changed, 12 insertions(+), 9 deletions(-) > > diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c > index 0465832a4..64ba96329 100644 > --- a/drivers/media/i2c/video-i2c.c > +++ b/drivers/media/i2c/video-i2c.c > @@ -443,14 +443,15 @@ static void buffer_queue(struct vb2_buffer *vb) > static int video_i2c_thread_vid_cap(void *priv) > { > struct video_i2c_data *data = priv; > - unsigned int delay = mult_frac(HZ, data->frame_interval.numerator, > - data->frame_interval.denominator); > + u32 delay = mult_frac(1000000UL, data->frame_interval.numerator, > + data->frame_interval.denominator); > + s64 end_us = ktime_to_us(ktime_get()); > > set_freezable(); > > do { > - unsigned long start_jiffies = jiffies; > struct video_i2c_buffer *vid_cap_buf = NULL; > + s64 current_us; > int schedule_delay; > > try_to_freeze(); > @@ -477,12 +478,14 @@ static int video_i2c_thread_vid_cap(void *priv) > VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE); > } > > - schedule_delay = delay - (jiffies - start_jiffies); > - > - if (time_after(jiffies, start_jiffies + delay)) > - schedule_delay = delay; > - > - schedule_timeout_interruptible(schedule_delay); > + end_us += delay; > + current_us = ktime_to_us(ktime_get()); > + if (current_us < end_us) { > + schedule_delay = end_us - current_us; > + usleep_range(schedule_delay * 3/4, schedule_delay); Looks fine to me.. not sure I like the 3/4 but the only way to do it without a floating point of 0.75 which is evil and would kick off a compiler warning anyway. Acked-by: Matt Ranostay <matt.ranostay@konsulko.com> > + } else { > + end_us = current_us; > + } > } while (!kthread_should_stop()); > > return 0; > -- > 2.31.1 > ^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame 2021-06-08 15:24 [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 Seongyong Park 2021-06-08 15:24 ` [PATCH v3 1/2] media: video-i2c: more precise intervals between frames Seongyong Park @ 2021-06-08 15:24 ` Seongyong Park 2021-06-09 7:13 ` Matt Ranostay 2021-08-05 14:55 ` Sakari Ailus 2021-06-09 7:08 ` [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 Matt Ranostay 2 siblings, 2 replies; 12+ messages in thread From: Seongyong Park @ 2021-06-08 15:24 UTC (permalink / raw) To: linux-media; +Cc: Matt Ranostay, Mauro Carvalho Chehab, Seongyong Park On MLX90640, Each measurement step updates half of the pixels in the frame (every other pixel in default "chess mode", and every other row in "interleave mode"), while additional coefficient data (25th & 26th row) updates every step. The compensational coefficient data only corresponds with the pixels updated in the same step. Only way to know which "subpage" was updated on the last step is to read "status register" on address 0x8000. Without this data, compensation calculation may be able to detect which sets of pixels have been updated, but it will have to make assumptions when frame skip happens, and there is no way to do it correctly when the host simply cannot keep up with refresh rate. Signed-off-by: Seongyong Park <euphoriccatface@gmail.com> --- drivers/media/i2c/video-i2c.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c index 64ba96329..2b50a76f3 100644 --- a/drivers/media/i2c/video-i2c.c +++ b/drivers/media/i2c/video-i2c.c @@ -74,7 +74,8 @@ static const struct v4l2_fmtdesc mlx90640_format = { static const struct v4l2_frmsize_discrete mlx90640_size = { .width = 32, - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ + .height = 27, + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ }; static const struct regmap_config amg88xx_regmap_config = { @@ -168,8 +169,12 @@ static int amg88xx_xfer(struct video_i2c_data *data, char *buf) static int mlx90640_xfer(struct video_i2c_data *data, char *buf) { - return regmap_bulk_read(data->regmap, 0x400, buf, - data->chip->buffer_size); + int ret = regmap_bulk_read(data->regmap, 0x400, buf, + data->chip->buffer_size - 64); + if (ret) + return ret; + return regmap_bulk_read(data->regmap, 0x8000, buf + (data->chip->buffer_size - 64), + 64); } static int amg88xx_setup(struct video_i2c_data *data) @@ -375,7 +380,7 @@ static const struct video_i2c_chip video_i2c_chip[] = { .format = &mlx90640_format, .frame_intervals = mlx90640_frame_intervals, .num_frame_intervals = ARRAY_SIZE(mlx90640_frame_intervals), - .buffer_size = 1664, + .buffer_size = 1728, .bpp = 16, .regmap_config = &mlx90640_regmap_config, .nvmem_config = &mlx90640_nvram_config, -- 2.31.1 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame 2021-06-08 15:24 ` [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame Seongyong Park @ 2021-06-09 7:13 ` Matt Ranostay 2021-06-12 5:55 ` Seongyong Park 2021-08-05 14:55 ` Sakari Ailus 1 sibling, 1 reply; 12+ messages in thread From: Matt Ranostay @ 2021-06-09 7:13 UTC (permalink / raw) To: Seongyong Park; +Cc: linux-media, Mauro Carvalho Chehab On Tue, Jun 8, 2021 at 8:25 AM Seongyong Park <euphoriccatface@gmail.com> wrote: > > On MLX90640, Each measurement step updates half of the pixels in the frame > (every other pixel in default "chess mode", and every other row > in "interleave mode"), while additional coefficient data (25th & 26th row) > updates every step. The compensational coefficient data only corresponds > with the pixels updated in the same step. > > Only way to know which "subpage" was updated on the last step is to read > "status register" on address 0x8000. Without this data, > compensation calculation may be able to detect which sets of pixels have > been updated, but it will have to make assumptions when frame skip happens, > and there is no way to do it correctly when the host simply cannot > keep up with refresh rate. > > Signed-off-by: Seongyong Park <euphoriccatface@gmail.com> > --- > drivers/media/i2c/video-i2c.c | 13 +++++++++---- > 1 file changed, 9 insertions(+), 4 deletions(-) > > diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c > index 64ba96329..2b50a76f3 100644 > --- a/drivers/media/i2c/video-i2c.c > +++ b/drivers/media/i2c/video-i2c.c > @@ -74,7 +74,8 @@ static const struct v4l2_fmtdesc mlx90640_format = { > > static const struct v4l2_frmsize_discrete mlx90640_size = { > .width = 32, > - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ > + .height = 27, > + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ Guess you hit the 80 character line here and checkpatch.pl complained .. But should all be one line since it is much more clear on one line. > }; > > static const struct regmap_config amg88xx_regmap_config = { > @@ -168,8 +169,12 @@ static int amg88xx_xfer(struct video_i2c_data *data, char *buf) > > static int mlx90640_xfer(struct video_i2c_data *data, char *buf) > { > - return regmap_bulk_read(data->regmap, 0x400, buf, > - data->chip->buffer_size); > + int ret = regmap_bulk_read(data->regmap, 0x400, buf, > + data->chip->buffer_size - 64); > + if (ret) > + return ret; > + return regmap_bulk_read(data->regmap, 0x8000, buf + (data->chip->buffer_size - 64), > + 64); > } > > static int amg88xx_setup(struct video_i2c_data *data) > @@ -375,7 +380,7 @@ static const struct video_i2c_chip video_i2c_chip[] = { > .format = &mlx90640_format, > .frame_intervals = mlx90640_frame_intervals, > .num_frame_intervals = ARRAY_SIZE(mlx90640_frame_intervals), > - .buffer_size = 1664, > + .buffer_size = 1728, Minus nitpick above looks good to me. You can keep the acked-by if that is only change Acked-by: Matt Ranostay <matt.ranostay@konsulko.com> > .bpp = 16, > .regmap_config = &mlx90640_regmap_config, > .nvmem_config = &mlx90640_nvram_config, > -- > 2.31.1 > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame 2021-06-09 7:13 ` Matt Ranostay @ 2021-06-12 5:55 ` Seongyong Park 2021-06-12 7:10 ` Matt Ranostay 0 siblings, 1 reply; 12+ messages in thread From: Seongyong Park @ 2021-06-12 5:55 UTC (permalink / raw) To: Matt Ranostay; +Cc: linux-media, Mauro Carvalho Chehab 2021년 6월 10일 (목) 오전 8:13, Seongyong Park <euphoriccatface@gmail.com>님이 작성: > > 2021년 6월 9일 (수) 오후 4:14, Matt Ranostay <matt.ranostay@konsulko.com>님이 작성: > > > > On Tue, Jun 8, 2021 at 8:25 AM Seongyong Park <euphoriccatface@gmail.com> wrote: > > > > > - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ > > > + .height = 27, > > > + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ > > > > Guess you hit the 80 character line here and checkpatch.pl complained > > .. But should all be one line since it is > > much more clear on one line. > > > > > }; > > > > > > static const struct regmap_config amg88xx_regmap_config = { > > > @@ -168,8 +169,12 @@ static int amg88xx_xfer(struct video_i2c_data *data, char *buf) > > > > > > static int mlx90640_xfer(struct video_i2c_data *data, char *buf) > > > { > > > - return regmap_bulk_read(data->regmap, 0x400, buf, > > > - data->chip->buffer_size); > > > + int ret = regmap_bulk_read(data->regmap, 0x400, buf, > > > + data->chip->buffer_size - 64); > > > + if (ret) > > > + return ret; > > > + return regmap_bulk_read(data->regmap, 0x8000, buf + (data->chip->buffer_size - 64), > > > + 64); > > > } > > > > > > static int amg88xx_setup(struct video_i2c_data *data) > > > @@ -375,7 +380,7 @@ static const struct video_i2c_chip video_i2c_chip[] = { > > > .format = &mlx90640_format, > > > .frame_intervals = mlx90640_frame_intervals, > > > .num_frame_intervals = ARRAY_SIZE(mlx90640_frame_intervals), > > > - .buffer_size = 1664, > > > + .buffer_size = 1728, > > > > Minus nitpick above looks good to me. You can keep the acked-by if > > that is only change > > > > Acked-by: Matt Ranostay <matt.ranostay@konsulko.com> > > > Yes, other than your suggestion, indeed that is the only change I made > for this commit between v1 and v2 (and v3 is the same as v2.) > and that is because of the checkpatch.pl indeed :) > > Thanks, > Seongyong Park Re-sending this mail mainly because I have made a direct reply, rather than a reply to all. Sorry about that. Matt, while we're at it, if I happen to make another revision of this patchset, would you find it looking okay to make a line break after the second parameter? The odd arrangement was partly because I wanted to make a line break after the same number of parameters. The lines will look like this: + int ret = regmap_bulk_read(data->regmap, 0x400, + buf, data->chip->buffer_size - 64); ... + return regmap_bulk_read(data->regmap, 0x8000, + buf + (data->chip->buffer_size - 64), 64); Thanks, - Seongyong Park ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame 2021-06-12 5:55 ` Seongyong Park @ 2021-06-12 7:10 ` Matt Ranostay 0 siblings, 0 replies; 12+ messages in thread From: Matt Ranostay @ 2021-06-12 7:10 UTC (permalink / raw) To: Seongyong Park; +Cc: linux-media, Mauro Carvalho Chehab On Fri, Jun 11, 2021 at 10:55 PM Seongyong Park <euphoriccatface@gmail.com> wrote: > > 2021년 6월 10일 (목) 오전 8:13, Seongyong Park <euphoriccatface@gmail.com>님이 작성: > > > > 2021년 6월 9일 (수) 오후 4:14, Matt Ranostay <matt.ranostay@konsulko.com>님이 작성: > > > > > > On Tue, Jun 8, 2021 at 8:25 AM Seongyong Park <euphoriccatface@gmail.com> wrote: > > > > > > > - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ > > > > + .height = 27, > > > > + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ > > > > > > Guess you hit the 80 character line here and checkpatch.pl complained > > > .. But should all be one line since it is > > > much more clear on one line. > > > > > > > }; > > > > > > > > static const struct regmap_config amg88xx_regmap_config = { > > > > @@ -168,8 +169,12 @@ static int amg88xx_xfer(struct video_i2c_data *data, char *buf) > > > > > > > > static int mlx90640_xfer(struct video_i2c_data *data, char *buf) > > > > { > > > > - return regmap_bulk_read(data->regmap, 0x400, buf, > > > > - data->chip->buffer_size); > > > > + int ret = regmap_bulk_read(data->regmap, 0x400, buf, > > > > + data->chip->buffer_size - 64); > > > > + if (ret) > > > > + return ret; > > > > + return regmap_bulk_read(data->regmap, 0x8000, buf + (data->chip->buffer_size - 64), > > > > + 64); > > > > } > > > > > > > > static int amg88xx_setup(struct video_i2c_data *data) > > > > @@ -375,7 +380,7 @@ static const struct video_i2c_chip video_i2c_chip[] = { > > > > .format = &mlx90640_format, > > > > .frame_intervals = mlx90640_frame_intervals, > > > > .num_frame_intervals = ARRAY_SIZE(mlx90640_frame_intervals), > > > > - .buffer_size = 1664, > > > > + .buffer_size = 1728, > > > > > > Minus nitpick above looks good to me. You can keep the acked-by if > > > that is only change > > > > > > Acked-by: Matt Ranostay <matt.ranostay@konsulko.com> > > > > > Yes, other than your suggestion, indeed that is the only change I made > > for this commit between v1 and v2 (and v3 is the same as v2.) > > and that is because of the checkpatch.pl indeed :) > > > > Thanks, > > Seongyong Park > > Re-sending this mail mainly because I have made a direct reply, rather > than a reply to all. > Sorry about that. > > Matt, while we're at it, if I happen to make another revision of this patchset, > would you find it looking okay to make a line break after the second parameter? > The odd arrangement was partly because I wanted to make a line break > after the same number of parameters. > > The lines will look like this: > + int ret = regmap_bulk_read(data->regmap, 0x400, > + buf, data->chip->buffer_size - 64); > ... > + return regmap_bulk_read(data->regmap, 0x8000, > + buf + (data->chip->buffer_size - 64), 64); > Yeah this is fine by me. Keep the signed off if this is the only change. - Matt > Thanks, > - Seongyong Park ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame 2021-06-08 15:24 ` [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame Seongyong Park 2021-06-09 7:13 ` Matt Ranostay @ 2021-08-05 14:55 ` Sakari Ailus 2021-09-13 8:57 ` Hans Verkuil 1 sibling, 1 reply; 12+ messages in thread From: Sakari Ailus @ 2021-08-05 14:55 UTC (permalink / raw) To: Seongyong Park Cc: linux-media, Matt Ranostay, Mauro Carvalho Chehab, hverkuil Hi Seongyong, On Wed, Jun 09, 2021 at 12:24:51AM +0900, Seongyong Park wrote: > On MLX90640, Each measurement step updates half of the pixels in the frame > (every other pixel in default "chess mode", and every other row > in "interleave mode"), while additional coefficient data (25th & 26th row) > updates every step. The compensational coefficient data only corresponds > with the pixels updated in the same step. > > Only way to know which "subpage" was updated on the last step is to read > "status register" on address 0x8000. Without this data, > compensation calculation may be able to detect which sets of pixels have > been updated, but it will have to make assumptions when frame skip happens, > and there is no way to do it correctly when the host simply cannot > keep up with refresh rate. > > Signed-off-by: Seongyong Park <euphoriccatface@gmail.com> > --- > drivers/media/i2c/video-i2c.c | 13 +++++++++---- > 1 file changed, 9 insertions(+), 4 deletions(-) > > diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c > index 64ba96329..2b50a76f3 100644 > --- a/drivers/media/i2c/video-i2c.c > +++ b/drivers/media/i2c/video-i2c.c > @@ -74,7 +74,8 @@ static const struct v4l2_fmtdesc mlx90640_format = { > > static const struct v4l2_frmsize_discrete mlx90640_size = { > .width = 32, > - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ > + .height = 27, > + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ This device should actually use a multi-plane format as the data isn't Y16_BE as the driver declares. That said, the format would be device specific and using one would require specific support from applications. I might just declare it produces fewer lines while an application that knows the device could access the extra data close to the end of the buffer. An alternative would be to use that multi-plane format and keep driver support for the plain Y16_BE as well. But I think this would be a bit heavy-weight solution. I wonder what Hans thinks. > }; > > static const struct regmap_config amg88xx_regmap_config = { > @@ -168,8 +169,12 @@ static int amg88xx_xfer(struct video_i2c_data *data, char *buf) > > static int mlx90640_xfer(struct video_i2c_data *data, char *buf) > { > - return regmap_bulk_read(data->regmap, 0x400, buf, > - data->chip->buffer_size); > + int ret = regmap_bulk_read(data->regmap, 0x400, buf, > + data->chip->buffer_size - 64); > + if (ret) > + return ret; > + return regmap_bulk_read(data->regmap, 0x8000, buf + (data->chip->buffer_size - 64), Please wrap before 80 (unless there's a reason to do otherwise). > + 64); > } > -- Kind regards, Sakari Ailus ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame 2021-08-05 14:55 ` Sakari Ailus @ 2021-09-13 8:57 ` Hans Verkuil 2021-09-13 10:05 ` Sakari Ailus 0 siblings, 1 reply; 12+ messages in thread From: Hans Verkuil @ 2021-09-13 8:57 UTC (permalink / raw) To: Sakari Ailus, Seongyong Park Cc: linux-media, Matt Ranostay, Mauro Carvalho Chehab On 05/08/2021 16:55, Sakari Ailus wrote: > Hi Seongyong, > > On Wed, Jun 09, 2021 at 12:24:51AM +0900, Seongyong Park wrote: >> On MLX90640, Each measurement step updates half of the pixels in the frame >> (every other pixel in default "chess mode", and every other row >> in "interleave mode"), while additional coefficient data (25th & 26th row) >> updates every step. The compensational coefficient data only corresponds >> with the pixels updated in the same step. >> >> Only way to know which "subpage" was updated on the last step is to read >> "status register" on address 0x8000. Without this data, >> compensation calculation may be able to detect which sets of pixels have >> been updated, but it will have to make assumptions when frame skip happens, >> and there is no way to do it correctly when the host simply cannot >> keep up with refresh rate. >> >> Signed-off-by: Seongyong Park <euphoriccatface@gmail.com> >> --- >> drivers/media/i2c/video-i2c.c | 13 +++++++++---- >> 1 file changed, 9 insertions(+), 4 deletions(-) >> >> diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c >> index 64ba96329..2b50a76f3 100644 >> --- a/drivers/media/i2c/video-i2c.c >> +++ b/drivers/media/i2c/video-i2c.c >> @@ -74,7 +74,8 @@ static const struct v4l2_fmtdesc mlx90640_format = { >> >> static const struct v4l2_frmsize_discrete mlx90640_size = { >> .width = 32, >> - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ >> + .height = 27, >> + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ > > This device should actually use a multi-plane format as the data isn't > Y16_BE as the driver declares. That said, the format would be device > specific and using one would require specific support from applications. > > I might just declare it produces fewer lines while an application that > knows the device could access the extra data close to the end of the > buffer. > > An alternative would be to use that multi-plane format and keep driver > support for the plain Y16_BE as well. But I think this would be a bit > heavy-weight solution. > > I wonder what Hans thinks. I think it is a bit overkill as well. Wouldn't it be better to just add a new pixel format for this device, and document it properly? I would keep the existing Y16_BE as it was to avoid breaking userspace (since adding the extra line breaks the ABI IMHO), and add a new format that properly documents how to parse the contents of the buffer, which includes the last line with the status register. Sorry for the late reply, it looks like an attempt was made to CC me, but the email address was wrong, so I never received it. I only noticed this when I started to review this patch. Regards, Hans > >> }; >> >> static const struct regmap_config amg88xx_regmap_config = { >> @@ -168,8 +169,12 @@ static int amg88xx_xfer(struct video_i2c_data *data, char *buf) >> >> static int mlx90640_xfer(struct video_i2c_data *data, char *buf) >> { >> - return regmap_bulk_read(data->regmap, 0x400, buf, >> - data->chip->buffer_size); >> + int ret = regmap_bulk_read(data->regmap, 0x400, buf, >> + data->chip->buffer_size - 64); >> + if (ret) >> + return ret; >> + return regmap_bulk_read(data->regmap, 0x8000, buf + (data->chip->buffer_size - 64), > > Please wrap before 80 (unless there's a reason to do otherwise). > >> + 64); >> } >> > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame 2021-09-13 8:57 ` Hans Verkuil @ 2021-09-13 10:05 ` Sakari Ailus 2021-10-09 11:14 ` Seongyong Park 0 siblings, 1 reply; 12+ messages in thread From: Sakari Ailus @ 2021-09-13 10:05 UTC (permalink / raw) To: Hans Verkuil Cc: Seongyong Park, linux-media, Matt Ranostay, Mauro Carvalho Chehab Hi Hans, Seongyong, On Mon, Sep 13, 2021 at 10:57:04AM +0200, Hans Verkuil wrote: > On 05/08/2021 16:55, Sakari Ailus wrote: > > Hi Seongyong, > > > > On Wed, Jun 09, 2021 at 12:24:51AM +0900, Seongyong Park wrote: > >> On MLX90640, Each measurement step updates half of the pixels in the frame > >> (every other pixel in default "chess mode", and every other row > >> in "interleave mode"), while additional coefficient data (25th & 26th row) > >> updates every step. The compensational coefficient data only corresponds > >> with the pixels updated in the same step. > >> > >> Only way to know which "subpage" was updated on the last step is to read > >> "status register" on address 0x8000. Without this data, > >> compensation calculation may be able to detect which sets of pixels have > >> been updated, but it will have to make assumptions when frame skip happens, > >> and there is no way to do it correctly when the host simply cannot > >> keep up with refresh rate. > >> > >> Signed-off-by: Seongyong Park <euphoriccatface@gmail.com> > >> --- > >> drivers/media/i2c/video-i2c.c | 13 +++++++++---- > >> 1 file changed, 9 insertions(+), 4 deletions(-) > >> > >> diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c > >> index 64ba96329..2b50a76f3 100644 > >> --- a/drivers/media/i2c/video-i2c.c > >> +++ b/drivers/media/i2c/video-i2c.c > >> @@ -74,7 +74,8 @@ static const struct v4l2_fmtdesc mlx90640_format = { > >> > >> static const struct v4l2_frmsize_discrete mlx90640_size = { > >> .width = 32, > >> - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ > >> + .height = 27, > >> + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ > > > > This device should actually use a multi-plane format as the data isn't > > Y16_BE as the driver declares. That said, the format would be device > > specific and using one would require specific support from applications. > > > > I might just declare it produces fewer lines while an application that > > knows the device could access the extra data close to the end of the > > buffer. > > > > An alternative would be to use that multi-plane format and keep driver > > support for the plain Y16_BE as well. But I think this would be a bit > > heavy-weight solution. > > > > I wonder what Hans thinks. > > I think it is a bit overkill as well. > > Wouldn't it be better to just add a new pixel format for this device, and > document it properly? I would keep the existing Y16_BE as it was to avoid > breaking userspace (since adding the extra line breaks the ABI IMHO), and > add a new format that properly documents how to parse the contents of the > buffer, which includes the last line with the status register. > > Sorry for the late reply, it looks like an attempt was made to CC me, but > the email address was wrong, so I never received it. I only noticed this > when I started to review this patch. Yeah, my mistake. I thought I had an alias for you but apparently not. I do now. I think that'd be fine, albeit equally heavy-weight. But also correct. Documenting what's there is also important. -- Regards, Sakari Ailus ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame 2021-09-13 10:05 ` Sakari Ailus @ 2021-10-09 11:14 ` Seongyong Park 0 siblings, 0 replies; 12+ messages in thread From: Seongyong Park @ 2021-10-09 11:14 UTC (permalink / raw) To: Sakari Ailus Cc: Hans Verkuil, linux-media, Matt Ranostay, Mauro Carvalho Chehab Thank you to everyone who gave some feedback. I finally have found the time to get back to this. Sorry for the delay. I tried gathering what I'd need to do if I wanted to add the suggested new pixel format. Since I'm learning a ton with this project on my own, I'd like some help to figure things out. Much appreciated if someone could confirm whether this looks right: 1. Modify any part in video-i2c that assume capability / stream parameter / buffer type of *VIDEO_CAPTURE only For example, the following part of `video_i2c_g_parm()` ``` if (parm->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) return -EINVAL; ``` 2. Make it possible for the videobuf2 queue to be configured for BUF_TYPE_VIDEO_CAPTURE_MPLANE as well Since the `vb2_queue_init()` is currently called inside `video_i2c_probe()`, I assume the setup and initialization will have to be moved to another place. Maybe into `start_streaming()`. 3. Add an entry to `videodev2.h` I assume so because, with quick grep-ing, I couldn't find any pixel format definitions under `drivers/media` but one (drivers/media/usb/msi2500) The entry would be named like V4L2_PIX_FMT_MLX9064X (MLX90641 seems to have similar format) 4. Add a flag in `video_i2c_data` for the multiplane output, and add appropriate code to fill in the second plane ...and then the rest will be adjusting the code, like getting the size of data correct and so on. Thanks in advance. Seongyong Park 2021년 9월 13일 (월) 오후 7:05, Sakari Ailus <sakari.ailus@iki.fi>님이 작성: > > Hi Hans, Seongyong, > > On Mon, Sep 13, 2021 at 10:57:04AM +0200, Hans Verkuil wrote: > > On 05/08/2021 16:55, Sakari Ailus wrote: > > > Hi Seongyong, > > > > > > On Wed, Jun 09, 2021 at 12:24:51AM +0900, Seongyong Park wrote: > > >> On MLX90640, Each measurement step updates half of the pixels in the frame > > >> (every other pixel in default "chess mode", and every other row > > >> in "interleave mode"), while additional coefficient data (25th & 26th row) > > >> updates every step. The compensational coefficient data only corresponds > > >> with the pixels updated in the same step. > > >> > > >> Only way to know which "subpage" was updated on the last step is to read > > >> "status register" on address 0x8000. Without this data, > > >> compensation calculation may be able to detect which sets of pixels have > > >> been updated, but it will have to make assumptions when frame skip happens, > > >> and there is no way to do it correctly when the host simply cannot > > >> keep up with refresh rate. > > >> > > >> Signed-off-by: Seongyong Park <euphoriccatface@gmail.com> > > >> --- > > >> drivers/media/i2c/video-i2c.c | 13 +++++++++---- > > >> 1 file changed, 9 insertions(+), 4 deletions(-) > > >> > > >> diff --git a/drivers/media/i2c/video-i2c.c b/drivers/media/i2c/video-i2c.c > > >> index 64ba96329..2b50a76f3 100644 > > >> --- a/drivers/media/i2c/video-i2c.c > > >> +++ b/drivers/media/i2c/video-i2c.c > > >> @@ -74,7 +74,8 @@ static const struct v4l2_fmtdesc mlx90640_format = { > > >> > > >> static const struct v4l2_frmsize_discrete mlx90640_size = { > > >> .width = 32, > > >> - .height = 26, /* 24 lines of pixel data + 2 lines of processing data */ > > >> + .height = 27, > > >> + /* 24 lines of pixel data + 2 lines of processing data + 1 line of registers */ > > > > > > This device should actually use a multi-plane format as the data isn't > > > Y16_BE as the driver declares. That said, the format would be device > > > specific and using one would require specific support from applications. > > > > > > I might just declare it produces fewer lines while an application that > > > knows the device could access the extra data close to the end of the > > > buffer. > > > > > > An alternative would be to use that multi-plane format and keep driver > > > support for the plain Y16_BE as well. But I think this would be a bit > > > heavy-weight solution. > > > > > > I wonder what Hans thinks. > > > > I think it is a bit overkill as well. > > > > Wouldn't it be better to just add a new pixel format for this device, and > > document it properly? I would keep the existing Y16_BE as it was to avoid > > breaking userspace (since adding the extra line breaks the ABI IMHO), and > > add a new format that properly documents how to parse the contents of the > > buffer, which includes the last line with the status register. > > > > Sorry for the late reply, it looks like an attempt was made to CC me, but > > the email address was wrong, so I never received it. I only noticed this > > when I started to review this patch. > > Yeah, my mistake. I thought I had an alias for you but apparently not. I do > now. > > I think that'd be fine, albeit equally heavy-weight. But also correct. > Documenting what's there is also important. > > -- > Regards, > > Sakari Ailus ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 2021-06-08 15:24 [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 Seongyong Park 2021-06-08 15:24 ` [PATCH v3 1/2] media: video-i2c: more precise intervals between frames Seongyong Park 2021-06-08 15:24 ` [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame Seongyong Park @ 2021-06-09 7:08 ` Matt Ranostay 2 siblings, 0 replies; 12+ messages in thread From: Matt Ranostay @ 2021-06-09 7:08 UTC (permalink / raw) To: Seongyong Park; +Cc: linux-media, Mauro Carvalho Chehab On Tue, Jun 8, 2021 at 8:25 AM Seongyong Park <euphoriccatface@gmail.com> wrote: > > Thanks to Matt Ranostay and Mauro Carvalho Chehab (mchehab) for guidance > Just one nitpick on future revisions. It is helpful to mention the changes from from v1 to v2 makes it easier for a quicker review. - Matt > Seongyong Park (2): > media: video-i2c: more precise intervals between frames > media: video-i2c: append register data on MLX90640's frame > > drivers/media/i2c/video-i2c.c | 34 +++++++++++++++++++++------------- > 1 file changed, 21 insertions(+), 13 deletions(-) > > -- > 2.31.1 > ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2021-10-09 11:14 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-06-08 15:24 [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 Seongyong Park 2021-06-08 15:24 ` [PATCH v3 1/2] media: video-i2c: more precise intervals between frames Seongyong Park 2021-06-09 7:22 ` Matt Ranostay 2021-06-08 15:24 ` [PATCH v3 2/2] media: video-i2c: append register data on MLX90640's frame Seongyong Park 2021-06-09 7:13 ` Matt Ranostay 2021-06-12 5:55 ` Seongyong Park 2021-06-12 7:10 ` Matt Ranostay 2021-08-05 14:55 ` Sakari Ailus 2021-09-13 8:57 ` Hans Verkuil 2021-09-13 10:05 ` Sakari Ailus 2021-10-09 11:14 ` Seongyong Park 2021-06-09 7:08 ` [PATCH v3 0/2] media: video-i2c: additional support for Melexis MLX90640 Matt Ranostay
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).