why does git set X in LESS env var?

why does git set X in LESS env var?

[Christoph Anton Mitterer]

Hey.


I recently stumbled over the problem that mouse wheel scrolling doesn't
work with (git-)delta[0], a problem[1][2], which apparently numerous
people had before me.

Numerous solutions were given in [1] and [2], for example using --mouse
as less option.
I noticed however, that this causes a somewhat different mouse
scrolling behaviour than my less usually gives me and dug a bit
further, noticing that the problem was that `X` was set in the `LESS`
env var for the less process, wrongly assuming[3] first that delta
would set it.

After delta's upstream noticed that this must be wrong, I looked
further and found that git set it since commit
0abc0260fa3419de649fcc1444e3d256a17ca6c7, which gives however no
indication why it was added.

A somewhat later commit, b3275838d969b7ecb91aae584226fccbeb046aca,
which removes `S` from being set, mentions:
> … The FRX flags actually make sense for Git (F and X because
> sometimes the output Git pipes to less is short, and R because Git
> pipes colored output).

But I still don't get from that why X would be needed?

My less manpage documents it as:
> -X or --no‐init
>     Disables sending the termcap initialization and deinitialization
>     strings to the terminal.  This is sometimes desirable if the
>     deinitialization string does something unnecessary, like clearing
>     the screen.

Is it to avoid clearing the screen?


Not sure whether git should really do something here... I mean it could
add --mouse per default, but then users who don't want that would need
to work around it (and there is no negative option for --mouse, it
seems).


Oh, I should add that scrolling without `X` and without `--mouse` sees
to only work on some terminals (e.g. VTE based ones, but not xterm).

Thanks,
Chris.


[0] https://github.com/dandavison/delta/
[1] https://github.com/dandavison/delta/issues/58
[2] https://github.com/dandavison/delta/issues/630
[3] https://github.com/dandavison/delta/issues/58#issuecomment-1756542986

Re: why does git set X in LESS env var?

[Junio C Hamano]

Christoph Anton Mitterer <calestyo@scientia.org> writes:

> But I still don't get from that why X would be needed?
>
> My less manpage documents it as:
>> -X or --no‐init
>>     Disables sending the termcap initialization and deinitialization
>>     strings to the terminal.  This is sometimes desirable if the
>>     deinitialization string does something unnecessary, like clearing
>>     the screen.
>
> Is it to avoid clearing the screen?

I think that was the reason we added it back in 2005.  In any case,
asking "why" is not a useful use of anybody's time, because it is
very unlikely to change in the official version we ship, and because
it is so easy for any individual who does not like it to drop by
exporting the $LESS environment variable.

Thanks.

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Wed, 2023-10-11 at 15:23 -0700, Junio C Hamano wrote:
> I think that was the reason we added it back in 2005.  In any case,
> asking "why" is not a useful use of anybody's time, because it is
> very unlikely to change in the official version we ship, and because
> it is so easy for any individual who does not like it to drop by
> exporting the $LESS environment variable.


Well the other commit I've mentioned kinda read as if it was thought
that either X or both F and X were needed for the effect to exit less
immediately if the output is too short ("F and X because
> sometimes the output Git pipes to less is short").

So I thought maybe that was intended, and the no-clear was just a side-
effect no one ever really thought about.


Anyway, thanks,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 00:26, Christoph Anton Mitterer wrote:
> On Wed, 2023-10-11 at 15:23 -0700, Junio C Hamano wrote:
>> I think that was the reason we added it back in 2005.  In any case,
>> asking "why" is not a useful use of anybody's time, because it is
>> very unlikely to change in the official version we ship, and because
>> it is so easy for any individual who does not like it to drop by
>> exporting the $LESS environment variable.
> 
> Well the other commit I've mentioned kinda read as if it was thought
> that either X or both F and X were needed for the effect to exit less
> immediately if the output is too short ("F and X because
> sometimes the output Git pipes to less is short").

In general, not clearing the screen (i.e. "-X") is there so the 
displayed contents is still visible in the terminal after exiting the 
pager.  That wouldn't be the case if the screen was cleared, making it 
less usable for most users.

Exiting if less contents than one full screen was displayed (i.e. "-F") 
is there to save people from the frustration of quitting a pager that 
actually wasn't needed to be executed.

When it comes to "-R", it has to be there, otherwise no coloring of the 
paginated output could be possible.

> So I thought maybe that was intended, and the no-clear was just a side-
> effect no one ever really thought about.
> 
> 
> Anyway, thanks,
> Chris.

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Thu, 2023-10-12 at 00:51 +0200, Dragan Simic wrote:
> In general, not clearing the screen (i.e. "-X") is there so the 
> displayed contents is still visible in the terminal after exiting the
> pager.  That wouldn't be the case if the screen was cleared, making
> it 
> less usable for most users.

Well, I personally, also prefer it that way... but I'd also say that
just like in the case of `S`, this is not really needed from the git
side, but rather simply a user choice.

And since, if the output did not fit one one screen, the non-cleared
remains may likely be chopped off,... one could argue that some users
would actually prefer to have it cleared.


> Exiting if less contents than one full screen was displayed (i.e. "-
> F") 
> is there to save people from the frustration of quitting a pager that
> actually wasn't needed to be executed.

Same actually here, at least strictly speaking, ... though I (and
probably everybody else?) would really hate it, if that was removed. ^^


Anyway... that's no request from my side to change the default. I just
wanted to know whether that don't-clear-the-screen part was the
motivation for the `X`.


In case someone cares, I've asked less upstream whether there's a way
to have VTE scrolling work with -X:
https://github.com/gwsw/less/issues/445


Thanks,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 01:16, Christoph Anton Mitterer wrote:
> On Thu, 2023-10-12 at 00:51 +0200, Dragan Simic wrote:
>> In general, not clearing the screen (i.e. "-X") is there so the
>> displayed contents is still visible in the terminal after exiting the
>> pager.  That wouldn't be the case if the screen was cleared, making
>> it
>> less usable for most users.
> 
> Well, I personally, also prefer it that way... but I'd also say that
> just like in the case of `S`, this is not really needed from the git
> side, but rather simply a user choice.

It's about providing a set of sane defaults for less(1), which other 
utilities also do, including dmesg, for example.  Of course, everyone 
can set $PAGER or $GIT_PAGER to fit their own prereferences.

> And since, if the output did not fit one one screen, the non-cleared
> remains may likely be chopped off,... one could argue that some users
> would actually prefer to have it cleared.

I'm not sure what do you mean by the non-cleared remains being chopped 
off...  Could you clarify that a bit, please?

As I already mentioned above, everyone is free to configure the pager 
behavior in any way they like.

>> Exiting if less contents than one full screen was displayed (i.e. "-
>> F")
>> is there to save people from the frustration of quitting a pager that
>> actually wasn't needed to be executed.
> 
> Same actually here, at least strictly speaking, ... though I (and
> probably everybody else?) would really hate it, if that was removed. ^^

I'm afraid that I don't understand very well are you complaining about 
the presence of "-F" or not?

> Anyway... that's no request from my side to change the default. I just
> wanted to know whether that don't-clear-the-screen part was the
> motivation for the `X`.

AFAIK, there should be no motivation other than not clearing the screen. 
  Other utilities that invoke the pager internally configure the default 
pager options in a very similar way.

> In case someone cares, I've asked less upstream whether there's a way
> to have VTE scrolling work with -X:
> https://github.com/gwsw/less/issues/445

Quite frankly, I can't stand scrolling in less(1) using the mouse wheel, 
but I do understand why some people like it.

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Thu, 2023-10-12 at 01:29 +0200, Dragan Simic wrote:
> I'm not sure what do you mean by the non-cleared remains being
> chopped 
> off...  Could you clarify that a bit, please?

Well if I do say:
$ reset
$ git diff HEAD~10

and from there scroll down a bit and then q to exit less (and the
screen is not cleared), I see the output only so far as I've had
previously scrolled down in less.

Everything that would have come after that is of course not visible.
The place where I exited may be some "well defined" border, like the
end of a commit... or anywhere it the middle of a patch (making the
left over remains on the terminal perhaps even ambiguous).

What's worse, when (in less) I scroll down and up again, perhaps
repeating several times, and then quit... I see (at least in my
less/terminal combination) things twice and mangled up (i.e. when I
scroll up the terminal (outside of less)).

So AFAICS, not clearing the screen only works properly when never
scrolling up again (in less).


> As I already mentioned above, everyone is free to configure the pager
> behavior in any way they like.

Sure :-)

> 
> > > Exiting if less contents than one full screen was displayed (i.e.
> > > "-
> > > F")
> > > is there to save people from the frustration of quitting a pager
> > > that
> > > actually wasn't needed to be executed.
> > 
> > Same actually here, at least strictly speaking, ... though I (and
> > probably everybody else?) would really hate it, if that was
> > removed. ^^
> 
> I'm afraid that I don't understand very well are you complaining
> about 
> the presence of "-F" or not?

No :-) As I've said, I like it that way and I and probably everyone
else would be annoyed, if -F was not present.

I just meant that strictly speaking the same reason why "S" was
removed, could be applied to "F" as well.

It is - like -R - not necessary for less to work with git.

But it is, of course, what virtually everyone will want in practise.


> Quite frankly, I can't stand scrolling in less(1) using the mouse
> wheel, 
> but I do understand why some people like it.

The main reason I want it is, that things don't get messy, when I
forget being in less and mouse scroll. ;-)


Thanks,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 01:43, Christoph Anton Mitterer wrote:
> On Thu, 2023-10-12 at 01:29 +0200, Dragan Simic wrote:
>> I'm not sure what do you mean by the non-cleared remains being
>> chopped
>> off...  Could you clarify that a bit, please?
> 
> Well if I do say:
> $ reset
> $ git diff HEAD~10
> 
> and from there scroll down a bit and then q to exit less (and the
> screen is not cleared), I see the output only so far as I've had
> previously scrolled down in less.

There's also scrollback in the terminal, which can be used to show more 
of the contents that was displayed before exiting the pager.

> Everything that would have come after that is of course not visible.
> The place where I exited may be some "well defined" border, like the
> end of a commit... or anywhere it the middle of a patch (making the
> left over remains on the terminal perhaps even ambiguous).

If you didn't select some line or page to be displayed, by scrolling 
within the pager, it obviously isn't going to be displayed, which is the 
whole point of using a pager instead of "spitting" the whole contents 
out at once.

Where and when you exit the pager is up to you only, and you can decide 
what will be left on the screen at that point.

> What's worse, when (in less) I scroll down and up again, perhaps
> repeating several times, and then quit... I see (at least in my
> less/terminal combination) things twice and mangled up (i.e. when I
> scroll up the terminal (outside of less)).

That sounds like some issue with your terminal or terminal emulator, 
which should be debugged and fixed separately.  Such misbehavior isn't 
supposed to happen at all.

> So AFAICS, not clearing the screen only works properly when never
> scrolling up again (in less).

It works just fine for me, for example.  You're obviously having some 
unrelated issues with your terminal emulator.

>> As I already mentioned above, everyone is free to configure the pager
>> behavior in any way they like.
> 
> Sure :-)
> 
>> 
>> > > Exiting if less contents than one full screen was displayed (i.e.
>> > > "-
>> > > F")
>> > > is there to save people from the frustration of quitting a pager
>> > > that
>> > > actually wasn't needed to be executed.
>> >
>> > Same actually here, at least strictly speaking, ... though I (and
>> > probably everybody else?) would really hate it, if that was
>> > removed. ^^
>> 
>> I'm afraid that I don't understand very well are you complaining
>> about
>> the presence of "-F" or not?
> 
> No :-) As I've said, I like it that way and I and probably everyone
> else would be annoyed, if -F was not present.
> 
> I just meant that strictly speaking the same reason why "S" was
> removed, could be applied to "F" as well.

I see.  Actually, removing "-S" was a good decision, IMHO, because 
chopping long lines isn't something that a sane set of defaults should 
do.  Many users would probably be confused with the need to use the 
right arrow to see long lines in their entirety.

> It is - like -R - not necessary for less to work with git.
> 
> But it is, of course, what virtually everyone will want in practise.

Well, "-R" is pretty much mandatory, because coloring the outputs has 
become some kind of defacto standard, and it's very useful when viewing 
diffs, for example.

>> Quite frankly, I can't stand scrolling in less(1) using the mouse
>> wheel,
>> but I do understand why some people like it.
> 
> The main reason I want it is, that things don't get messy, when I
> forget being in less and mouse scroll. ;-)
> 
> 
> Thanks,
> Chris.

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Thu, 2023-10-12 at 02:06 +0200, Dragan Simic wrote:
> There's also scrollback in the terminal, which can be used to show
> more 
> of the contents that was displayed before exiting the pager.

Sure.


> > Everything that would have come after that is of course not
> > visible.
> > The place where I exited may be some "well defined" border, like
> > the
> > end of a commit... or anywhere it the middle of a patch (making the
> > left over remains on the terminal perhaps even ambiguous).
> 
> If you didn't select some line or page to be displayed, by scrolling 
> within the pager, it obviously isn't going to be displayed, which is
> the 
> whole point of using a pager instead of "spitting" the whole contents
> out at once.

It's also clear that it's one point of a pager :-)

But that doesn't change that it's rather a user decision, whether or
not it makes sense to leave that, what's already been shown by the
pager, on the terminal after exiting the pager or not.

I don't think people always select the lines in the pager to some
reasonable border (e.g. end of a commit, end of a hunk, whatever).
So it's likely that after leaving the pager, the terminal's scrollback
buffer will contain something that is not complete and may thus be
ambiguous.


> 
> That sounds like some issue with your terminal or terminal emulator, 
> which should be debugged and fixed separately.  Such misbehavior
> isn't 
> supposed to happen at all.

Are you sure about that?

Well it happens at least in gnome-terminal, xterm and (KDE) konsole.


> I see.  Actually, removing "-S" was a good decision, IMHO, because 
> chopping long lines isn't something that a sane set of defaults
> should 
> do.  Many users would probably be confused with the need to use the 
> right arrow to see long lines in their entirety.

Sure.

And having -F is IMO a good default (that virtually everyone would
want), too.

With respect to -X, I'm less sure whether it's that clear.


Cheers,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 02:22, Christoph Anton Mitterer wrote:
> On Thu, 2023-10-12 at 02:06 +0200, Dragan Simic wrote:
>> There's also scrollback in the terminal, which can be used to show
>> more
>> of the contents that was displayed before exiting the pager.
> 
> Sure.
> 
>> > Everything that would have come after that is of course not
>> > visible.
>> > The place where I exited may be some "well defined" border, like
>> > the
>> > end of a commit... or anywhere it the middle of a patch (making the
>> > left over remains on the terminal perhaps even ambiguous).
>> 
>> If you didn't select some line or page to be displayed, by scrolling
>> within the pager, it obviously isn't going to be displayed, which is
>> the
>> whole point of using a pager instead of "spitting" the whole contents
>> out at once.
> 
> It's also clear that it's one point of a pager :-)
> 
> But that doesn't change that it's rather a user decision, whether or
> not it makes sense to leave that, what's already been shown by the
> pager, on the terminal after exiting the pager or not.
> 
> I don't think people always select the lines in the pager to some
> reasonable border (e.g. end of a commit, end of a hunk, whatever).
> So it's likely that after leaving the pager, the terminal's scrollback
> buffer will contain something that is not complete and may thus be
> ambiguous.

Makes sense, but please see also my other reply on the list.  To sum it 
up, we can have either the current behavior, the inconsistent behavior, 
or an even more annoying behavior.  I believe that the current behavior 
is the best choice among these three options.

>> That sounds like some issue with your terminal or terminal emulator,
>> which should be debugged and fixed separately.  Such misbehavior
>> isn't
>> supposed to happen at all.
> 
> Are you sure about that?
> 
> Well it happens at least in gnome-terminal, xterm and (KDE) konsole.

Yes, I'm sure, because I'd be fixing that already if that were the case 
in my environment. :)  I use Xfce and its default terminal emulator, 
though, and I don't know what it's like in other desktop environments 
and their terminal emulators.

>> I see.  Actually, removing "-S" was a good decision, IMHO, because
>> chopping long lines isn't something that a sane set of defaults
>> should
>> do.  Many users would probably be confused with the need to use the
>> right arrow to see long lines in their entirety.
> 
> Sure.
> 
> And having -F is IMO a good default (that virtually everyone would
> want), too.
> 
> With respect to -X, I'm less sure whether it's that clear.

Please see my other response, which explains why having "-FX" is 
actually a good thing.

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Thu, 2023-10-12 at 02:31 +0200, Dragan Simic wrote:
> 
> Makes sense, but please see also my other reply on the list.  To sum
> it 
> up, we can have either the current behavior, the inconsistent
> behavior, 
> or an even more annoying behavior.  I believe that the current
> behavior 
> is the best choice among these three options.

Well as I've said... I don't demand that it's changed, but I simply
think it's a wrong assumption that it's in any way better or worse.

Leaving back partial output, or when scrolling up&down completely
messed up output, is surely not per se more annoying or a bigger
problem than leaving back no output at all in one case (when it doesn't
fit on one screen) or leaving back output (when it fits).



> Yes, I'm sure, because I'd be fixing that already if that were the
> case 
> in my environment. :)  I use Xfce and its default terminal emulator, 
> though, and I don't know what it's like in other desktop environments
> and their terminal emulators.

I just tried it with xfce4-terminal 1.1.0 (which AFAICS is the most
recent version) in Debian, and unless they break anything with custom
patches, or you distro fixes anything with custom patches... I'd say
you must suffer from the same issue and probably just try something
different.

Since Debian's less is pretty outdated, I've even compiled a quite
recent less 643 (there's not even a tarball yet for 644, only a git
tag).


A made a screen recording... it's not 8K ;-) but I guess you can see
what I do:
https://youtu.be/KMs3sLk9nXY



Cheers,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 03:39, Christoph Anton Mitterer wrote:
> On Thu, 2023-10-12 at 02:31 +0200, Dragan Simic wrote:
>> 
>> Makes sense, but please see also my other reply on the list.  To sum
>> it
>> up, we can have either the current behavior, the inconsistent
>> behavior,
>> or an even more annoying behavior.  I believe that the current
>> behavior
>> is the best choice among these three options.
> 
> Well as I've said... I don't demand that it's changed, but I simply
> think it's a wrong assumption that it's in any way better or worse.
> 
> Leaving back partial output, or when scrolling up&down completely
> messed up output, is surely not per se more annoying or a bigger
> problem than leaving back no output at all in one case (when it doesn't
> fit on one screen) or leaving back output (when it fits).

Let me repeat that the messed up output you're experiencing isn't normal 
and has nothing to do with the arguments passed to less(1).  That's a 
separate issue of the terminal emulator(s) you're using, or in issue of 
your specific environment, and should be debugged and addressed as a 
separate issue.

To me, having inconsistent displaying of the short and long outputs is 
simply not acceptable.

>> Yes, I'm sure, because I'd be fixing that already if that were the
>> case
>> in my environment. :)  I use Xfce and its default terminal emulator,
>> though, and I don't know what it's like in other desktop environments
>> and their terminal emulators.
> 
> I just tried it with xfce4-terminal 1.1.0 (which AFAICS is the most
> recent version) in Debian, and unless they break anything with custom
> patches, or you distro fixes anything with custom patches... I'd say
> you must suffer from the same issue and probably just try something
> different.

Let me repeat that I don't see those issues, and actually, IIRC, have 
never seen them in my 25+ years of using various Linux distributions.  
If I've ever seen that, it would've already motivated me to have it 
debugged and fixed.

Perhaps something is wrong with your specific environment, because I see 
no other reason for this issue.

> Since Debian's less is pretty outdated, I've even compiled a quite
> recent less 643 (there's not even a tarball yet for 644, only a git
> tag).
> 
> A made a screen recording... it's not 8K ;-) but I guess you can see
> what I do:
> https://youtu.be/KMs3sLk9nXY

For some reason, I can see it with 360p as the highest available 
resolution, so I really can't read what's displayed on the recorded 
screen.  Strange.  Could you, please, upload the video in higher 
resolution, perhaps to a file sharing service such as 
https://easyupload.io/ ?

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Thu, 2023-10-12 at 07:46 +0200, Dragan Simic wrote:
> Let me repeat that the messed up output you're experiencing isn't
> normal 
> and has nothing to do with the arguments passed to less(1).  That's a
> separate issue of the terminal emulator(s) you're using, or in issue
> of 
> your specific environment, and should be debugged and addressed as a 
> separate issue.

Be it as it may...

As I've told you before it happens at least in gnome-terminal (and thus
presumably and VTE based terminal), xterm, xfce4-terminal and konsole
(all current versions of Debian unstable)... with less as of Debian
unstable as well as 643.

That affects at least on major distro, and there's a good chance that
it affects any other distro based on Debian (*buntu, etc.).


I further tried on SLES 15 with both gnome-terminal 3.42.2 and xterm
330 as well as less 530.

Even tried with the terminal emulator started via env -i and only TERM
set manually.


*All* cases affected by the same problem I've described before.


Same with the command you've used in your follow-up post, here a video
of it in HD:
https://youtu.be/MsxtQgrKM50


> To me, having inconsistent displaying of the short and long outputs
> is 
> simply not acceptable.

Which is fine - and as I've said: I personally also tend to prefer it
like that - but even if the above would be just some bug (which however
seems to affect all systems I could test on a short notice, except
yours)... one can IMO still not generally say whether on or the other
behaviour is generally accepted to be the better one.

Even if output may be just chopped of and thus ambiguously incomplete,
some people may still prefer to have rather no output at all.

And in fact:
This is the default mode of less alone.



> > 
> Perhaps something is wrong with your specific environment, because I
> see 
> no other reason for this issue.

Well may be, but seems unlikely from my PoV, given that I've now tested
even on other distros and systems not under my control.



Anyway... I think this got a bit too off-topic here :-D


Cheers,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 22:23, Christoph Anton Mitterer wrote:
> On Thu, 2023-10-12 at 07:46 +0200, Dragan Simic wrote:
>> Let me repeat that the messed up output you're experiencing isn't 
>> normal
>> and has nothing to do with the arguments passed to less(1).  That's a
>> separate issue of the terminal emulator(s) you're using, or in issue 
>> of
>> your specific environment, and should be debugged and addressed as a
>> separate issue.
> 
> As I've told you before it happens at least in gnome-terminal (and thus
> presumably and VTE based terminal), xterm, xfce4-terminal and konsole
> (all current versions of Debian unstable)... with less as of Debian
> unstable as well as 643.
> 
> That affects at least on major distro, and there's a good chance that
> it affects any other distro based on Debian (*buntu, etc.).
> 
> I further tried on SLES 15 with both gnome-terminal 3.42.2 and xterm
> 330 as well as less 530.
> 
> Even tried with the terminal emulator started via env -i and only TERM
> set manually.
> 
> *All* cases affected by the same problem I've described before.
> 
> Same with the command you've used in your follow-up post, here a video
> of it in HD:
> https://youtu.be/MsxtQgrKM50

Ah, I can finally see what are you talking about...  Thank you very much 
for all the testing you've performed and for supplying this screen 
recording!  I can confirm that my environment is also affected, but for 
some reason I haven't observed it this way before.

Huh, that's really worrisome and I'm willing to help you with debugging 
and fixing this issue.  Please, let me perform some debugging and 
digging around, and I'll come back to you with some further insights,

>> To me, having inconsistent displaying of the short and long outputs
>> is simply not acceptable.
> 
> Which is fine - and as I've said: I personally also tend to prefer it
> like that - but even if the above would be just some bug (which however
> seems to affect all systems I could test on a short notice, except
> yours)... one can IMO still not generally say whether on or the other
> behaviour is generally accepted to be the better one.
> 
> Even if output may be just chopped of and thus ambiguously incomplete,
> some people may still prefer to have rather no output at all.

Those people, just as anyone else, can use $PAGER or $GIT_PAGER to 
configure the pagination the way they like it.  In the end, that's also 
what I do in my environment.

>> Perhaps something is wrong with your specific environment, because
>> I see no other reason for this issue.
> 
> Well may be, but seems unlikely from my PoV, given that I've now tested
> even on other distros and systems not under my control.
> 
> Anyway... I think this got a bit too off-topic here :-D

Well, yes and no.  This scrolling-related issue is obviously affecting 
numerous git users, which makes it quite relevant for git as a project.  
Of course, not directly relevant, but indirectly, yes.

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Thu, 2023-10-12 at 23:15 +0200, Dragan Simic wrote:
> 
> Ah, I can finally see what are you talking about...  Thank you very
> much 
> for all the testing you've performed and for supplying this screen 
> recording!  I can confirm that my environment is also affected, but
> for 
> some reason I haven't observed it this way before.

Well... perhaps because it's not really "easy" to spot unless one
carefully reads through the lines (which I guess, one does not that
often in the terminal "history").

Have a look at my ticket at less, especially:
https://github.com/gwsw/less/issues/445#issuecomment-1758887183

Where it was confirmed that the issue I describe might happen (and I
guess is non-fixable?

The whole issue also contains an explanation on why scrolling doesn't
work when -X is used (but --mouse is not) on VTE terminals (and maybe
others, though not xterm).
And it's basically not "fixable" but simply "by design".


> Huh, that's really worrisome and I'm willing to help you with
> debugging 
> and fixing this issue.  Please, let me perform some debugging and 
> digging around, and I'll come back to you with some further insights,

Well, my assumption (though I'm really not a terminal expert) would be
that it's not fixable... because less would somehow make sure that
everything it prints (in the alt screen buffer) is properly
concatenated in the the regular one.


less upstream made some suggestions:
https://github.com/gwsw/less/issues/445#issuecomment-1759986293

One would be to change the terminfo entry, which I guess is not really
feasible as a general solution.

The other would be less’ --redraw-on-quit option.


Maybe that would be an even better solution for git?
AFAICS, we could have -F, the VTE mouse scrolling out of the box, plus
(via that option) the final screen buffer of less printed when exiting.

Would give some context (what one did in the pager, where one was) on
the regular screen buffer, but (presumably) avoid that mess up... and
perhaps even prevent unneeded pages of output from the pager, if one
scrolled down a lot, which maybe aren't even needed?


(Also note, that less` upstream calls `-X` "a risky flag" ;-) )


> Those people, just as anyone else, can use $PAGER or $GIT_PAGER to 
> configure the pagination the way they like it.  In the end, that's
> also 
> what I do in my environment.

Sure... all I said, that (IMO) in the case of `-X` it's not like with
`-R` but rater like with `-S`... i.e. neither mode has more right to be
the default than the other.



Cheers,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 23:48, Christoph Anton Mitterer wrote:
> On Thu, 2023-10-12 at 23:15 +0200, Dragan Simic wrote:
>> 
>> Ah, I can finally see what are you talking about...  Thank you very 
>> much
>> for all the testing you've performed and for supplying this screen
>> recording!  I can confirm that my environment is also affected, but 
>> for
>> some reason I haven't observed it this way before.
> 
> Well... perhaps because it's not really "easy" to spot unless one
> carefully reads through the lines (which I guess, one does not that
> often in the terminal "history").
> 
> Have a look at my ticket at less, especially:
> https://github.com/gwsw/less/issues/445#issuecomment-1758887183

Great, thanks, it's full of very useful information.

> Where it was confirmed that the issue I describe might happen (and I
> guess is non-fixable?
> 
> The whole issue also contains an explanation on why scrolling doesn't
> work when -X is used (but --mouse is not) on VTE terminals (and maybe
> others, though not xterm).
> And it's basically not "fixable" but simply "by design".

It seems that "--redraw-on-quit" is a possible candidate for replacing 
"-X" in the set of default options for less(1), and also in other CLI 
utilities, but I still need to test it in detail.

>> Huh, that's really worrisome and I'm willing to help you with
>> debugging
>> and fixing this issue.  Please, let me perform some debugging and
>> digging around, and I'll come back to you with some further insights,
> 
> Well, my assumption (though I'm really not a terminal expert) would be
> that it's not fixable... because less would somehow make sure that
> everything it prints (in the alt screen buffer) is properly
> concatenated in the the regular one.

I did some work on terminal emulators, but I'm also not very much of an 
expert when it comes to terminal emulators.

> less upstream made some suggestions:
> https://github.com/gwsw/less/issues/445#issuecomment-1759986293
> 
> One would be to change the terminfo entry, which I guess is not really
> feasible as a general solution.
> 
> The other would be less’ --redraw-on-quit option.

I agree that the latter seems like a viable solution, but as I already 
wrote above, I need to test it first, together with doing some digging 
through the less(1) source code.

> Maybe that would be an even better solution for git?
> AFAICS, we could have -F, the VTE mouse scrolling out of the box, plus
> (via that option) the final screen buffer of less printed when exiting.

Exactly, but still needs testing and a detailed insight.

> Would give some context (what one did in the pager, where one was) on
> the regular screen buffer, but (presumably) avoid that mess up... and
> perhaps even prevent unneeded pages of output from the pager, if one
> scrolled down a lot, which maybe aren't even needed?

Yes, it seems to tick all the boxes.

> (Also note, that less` upstream calls `-X` "a risky flag" ;-) )

In general, "-X" seems to be more of a bandaid option.

>> Those people, just as anyone else, can use $PAGER or $GIT_PAGER to
>> configure the pagination the way they like it.  In the end, that's
>> also
>> what I do in my environment.
> 
> Sure... all I said, that (IMO) in the case of `-X` it's not like with
> `-R` but rater like with `-S`... i.e. neither mode has more right to be
> the default than the other.

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Fri, 2023-10-13 at 00:36 +0200, Dragan Simic wrote:
> It seems that "--redraw-on-quit" is a possible candidate for
> replacing 
> "-X" in the set of default options for less(1)

*If* some changes were made to how git handles this, it might perhaps
be worth to consider not to touch LESS at all, but only add the
required settings via command line arguments (i.e. -F -R ...).

Or perhaps only remove options from it, if they're known to break the
behaviour with git (like -+R might).


I always feel configuration via env vars is a bit fragile:
- especially when one has generic names like POSIXLY_CORRECT there's
  some chance that by exporting it to one program, where one wants the
  effect, another program started off by that also gets it
  unintentionally
- generic terms may be used by multiple programs, causing problems

Also, if one can set only one LESS var in the environment, not one for
less "alone", one for less with git, etc. - that is unless for programs
like bat/delta which have specific own env vars to set the pager.

So if I set e.g. LESS to something, than typically only to stuff from
which I believe it works as expected for any possible users.
E.g. -F might be such a case.

But if I do that, git won't touch LESS and set the required -R, so I
have to do that manually for git, e.g. either via git_config or by
defining an alias git='LESS=FRX git'.
But in both cases it would "break" again, should ever another option be
needed and added by git to the default LESS (which is however only set
when it's unset).
And in case of an alias, there would be the additional problem, that
it's typically not picked up in non-interactive shells.


Long story short, it might make sense for git, to (mostly) ignore LESS
and rather invoke less with -F -R.

The problem with that in turn would of course be that it doesn't
automatically propagate down, if e.g. git's pager is set to detla and
delta in turn runs less.
However, that's IMO litte concern, since then it's delta's duty to set
-R (if it think it needs to do so), which it actually does.


Cheers,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-13 01:06, Christoph Anton Mitterer wrote:
> On Fri, 2023-10-13 at 00:36 +0200, Dragan Simic wrote:
>> It seems that "--redraw-on-quit" is a possible candidate for
>> replacing "-X" in the set of default options for less(1)
> 
> *If* some changes were made to how git handles this, it might perhaps
> be worth to consider not to touch LESS at all, but only add the
> required settings via command line arguments (i.e. -F -R ...).

Actually, that would be wrong.  If someone sets $LESS or $PAGER (or 
$GIT_PAGER, more specifically), it's up to the utility that invokes the 
pager internally not to override the user preferences configured through 
these environment variables.  That's how everyone can customize the 
pager behavior.

> Or perhaps only remove options from it, if they're known to break the
> behaviour with git (like -+R might).

Again, not the way the whole thing with pagination works.  If someone 
sets their environment variables wrong, it's simply the way they want 
it, and it isn't anyone else's business to attempt fixing it 
automatically.

> I always feel configuration via env vars is a bit fragile:
> - especially when one has generic names like POSIXLY_CORRECT there's
>   some chance that by exporting it to one program, where one wants the
>   effect, another program started off by that also gets it
>   unintentionally
> - generic terms may be used by multiple programs, causing problems

Well, fragile or not, that's the way it works.  It has its downsides for 
sure, but it's all about having each utility handle the environment 
carefully and document it in its man page(s), so the users can also 
carefully craft the values of their customized environment variables.

> Also, if one can set only one LESS var in the environment, not one for
> less "alone", one for less with git, etc. - that is unless for programs
> like bat/delta which have specific own env vars to set the pager.

$LESS can be seen as a global set of the common options for less(1), 
which may include the coloring configuration or the enablement of 
case-insensitive search, for example, while $MANPAGER, $GIT_PAGER and 
$BAT_PAGER may contain utility-specific options for less(1).  That's 
actually very good, because it makes possible to avoid duplication of 
the common options.

> So if I set e.g. LESS to something, than typically only to stuff from
> which I believe it works as expected for any possible users.
> E.g. -F might be such a case.

It's up to everyone to decide what are the common options for less(1) 
that they want to set in $LESS.

> But if I do that, git won't touch LESS and set the required -R, so I
> have to do that manually for git, e.g. either via git_config or by
> defining an alias git='LESS=FRX git'.

You don't have to define an alias, there's $GIT_PAGER for that purpose, 
as I already explained above.

Moreover, the whole idea of the various utilities touching the $LESS 
variable internally is to provide sane defaults to the users that don't 
configure $LESS, $PAGER, etc. on their own.  Once the user starts to 
provide their own environment variable(s), it's no longer up to the 
utility to help the user by altering their environment configuration.

> But in both cases it would "break" again, should ever another option be
> needed and added by git to the default LESS (which is however only set
> when it's unset).
> And in case of an alias, there would be the additional problem, that
> it's typically not picked up in non-interactive shells.

As you can see in my replies, it isn't about taking care of the users 
who provide their own environment configuration.  It's all about 
providing the set of sane defaults, for the users with no custom 
configuration.

> Long story short, it might make sense for git, to (mostly) ignore LESS
> and rather invoke less with -F -R.

No, that would be wrong on multiple levels, as I already explained in 
detail.

> The problem with that in turn would of course be that it doesn't
> automatically propagate down, if e.g. git's pager is set to detla and
> delta in turn runs less.
> However, that's IMO litte concern, since then it's delta's duty to set
> -R (if it think it needs to do so), which it actually does.

I don't know what delta is and how it actually paginates its outputs, 
but it should follow the rules of the environment-based pager 
configuration that I described in detail above.

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

On Fri, 2023-10-13 at 06:43 +0200, Dragan Simic wrote:
> > *If* some changes were made to how git handles this, it might
> > perhaps
> > be worth to consider not to touch LESS at all, but only add the
> > required settings via command line arguments (i.e. -F -R ...).
> 
> Actually, that would be wrong.  If someone sets $LESS or $PAGER (or 
> $GIT_PAGER, more specifically), it's up to the utility that invokes
> the 
> pager internally not to override the user preferences configured
> through 
> these environment variables.  That's how everyone can customize the 
> pager behavior.

Well, but if its clear that the output would otherwise be garbage (e.g.
because -R is missing).

In any case right now we have the situation that a user cannot just
easily set LESS in his environment, with a minimum set of options, and
git's use of less will continue flawlessly out of the box, as the -R
would be missing.


> > Or perhaps only remove options from it, if they're known to break
> > the
> > behaviour with git (like -+R might).
> 
> Again, not the way the whole thing with pagination works.  If someone
> sets their environment variables wrong, it's simply the way they want
> it, and it isn't anyone else's business to attempt fixing it 
> automatically.

Well, I wouldn't agree with that.
LESS foremost a env var to configure less (surprise ^^).

If git (or anyone else) uses less internally, e.g. because they don't
want to implement their own pager, fine... but then they cannot just
blindly assume that LESS is set only for git's (or any other tool's
needs).

So I'd say the proper way is rather that any such tool makes sure, that
any options strictly required as set no matter what. Just as e.g. delta
does.


> Well, fragile or not, that's the way it works.  It has its downsides
> for 
> sure, but it's all about having each utility handle the environment 
> carefully and document it in its man page(s), so the users can also 
> carefully craft the values of their customized environment variables.

Sure, but from a user's view, the use of less (or anything else) within
git is conceptually completely opaque.

In less' manpage LESS isn't documented as "oh and you must make sure -R
is included or otherwise git will break"...



> $LESS can be seen as a global set of the common options for less(1), 

o.O ... but, as I've described, one cannot really use it as that:

If I globally set e.g. LESS="F" because my desire is to make less
always exit as soon as the file fits on a screen, which I think is a
reasonable thing to do, git would no longer add "R" and output would
break.


> You don't have to define an alias, there's $GIT_PAGER for that
> purpose, 
> as I already explained above.

Well, yes... and as I've said before, one could also solve it via
git_config... but the problem stays the same... as soon as someone
wants to use LESS as global less options just as you described it
yourself, git will no longer worker properly because of the missing -R.

And actually if one would use GIT_PAGER one would again defeat the
purpose of a allegedly global options LESS, because unless one does
something like GIT_PAGER="${LESS}R" it wouldn't see any changes made to
LESS.


> Moreover, the whole idea of the various utilities touching the $LESS 
> variable internally is to provide sane defaults to the users that
> don't 
> configure $LESS, $PAGER, etc. on their own.

Then I don't see what the big problem would be to just do it via a
command argument - if someone really has ever some reasons to remove --
RAW‐CONTROL‐CHARS from the command options when less is invoked via git
... then he could still go into git_config and set that manually.

But it would seem to me that the overall handling would be much more
what one expects, than when doing the same via LESS.



> I don't know what delta is and how it actually paginates its outputs,
> but it should follow the rules of the environment-based pager 
> configuration that I described in detail above.

Well, AFAIU, it doesn't and for good reasons :-)


Anyway... I think all necessary things have been said and this thread
has grown far to large with only semi-related stuff... so thanks for
all the replies why git uses "-X".


Cheers,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-13 15:45, Christoph Anton Mitterer wrote:
> On Fri, 2023-10-13 at 06:43 +0200, Dragan Simic wrote:
>> Actually, that would be wrong.  If someone sets $LESS or $PAGER (or
>> $GIT_PAGER, more specifically), it's up to the utility that invokes
>> the pager internally not to override the user preferences configured
>> through these environment variables.  That's how everyone can 
>> customize
>> the pager behavior.
> 
> Well, but if its clear that the output would otherwise be garbage (e.g.
> because -R is missing).

Well, it's the basic principle of "garbage in, garbage out".  If there's 
something wrong with the contents of the environment variables, that's 
simply the way the user configured it, and it's only their job to get it 
right.

> In any case right now we have the situation that a user cannot just
> easily set LESS in his environment, with a minimum set of options, and
> git's use of less will continue flawlessly out of the box, as the -R
> would be missing.

Let me repeat that it isn't the job of git or any other pager-enabled 
utility to fix the user-defined environment.  Otherwise, the user 
actually wouldn't be able to make their choice freely.

>> Again, not the way the whole thing with pagination works.  If someone
>> sets their environment variables wrong, it's simply the way they want
>> it, and it isn't anyone else's business to attempt fixing it
>> automatically.
> 
> Well, I wouldn't agree with that.
> LESS foremost a env var to configure less (surprise ^^).
> 
> If git (or anyone else) uses less internally, e.g. because they don't
> want to implement their own pager, fine... but then they cannot just
> blindly assume that LESS is set only for git's (or any other tool's
> needs).

You seem to be missing the presence of other enviroment variables, 
namely $GIT_PAGER, which I already described in detail in my previous 
reply.  I'd appreciate if you'd read that description in detail, and 
possibly test it a bit.

> So I'd say the proper way is rather that any such tool makes sure, that
> any options strictly required as set no matter what. Just as e.g. delta
> does.

Again, that's simply wrong and defeats the user's freedom of choice.

>> Well, fragile or not, that's the way it works.  It has its downsides
>> for
>> sure, but it's all about having each utility handle the environment
>> carefully and document it in its man page(s), so the users can also
>> carefully craft the values of their customized environment variables.
> 
> Sure, but from a user's view, the use of less (or anything else) within
> git is conceptually completely opaque.

Actually, it isn't, because there are $LESS, $PAGER and $GIT_PAGER 
environment variables to customize the behavior.

> In less' manpage LESS isn't documented as "oh and you must make sure -R
> is included or otherwise git will break"...

Quite frankly, it would be silly to expect the less(1) man page to 
mention something about git(1).

>> $LESS can be seen as a global set of the common options for less(1),
> 
> o.O ... but, as I've described, one cannot really use it as that:
> 
> If I globally set e.g. LESS="F" because my desire is to make less
> always exit as soon as the file fits on a screen, which I think is a
> reasonable thing to do, git would no longer add "R" and output would
> break.

Again, you seem not to understand well the distinction between the 
global settings (i.e. $LESS and $PAGER) and the utility-specific 
settings (e.g. $GIT_PAGER).  Or you maybe simply refuse to understand 
it, I don't know.

>> You don't have to define an alias, there's $GIT_PAGER for that
>> purpose, as I already explained above.
> 
> Well, yes... and as I've said before, one could also solve it via
> git_config... but the problem stays the same... as soon as someone
> wants to use LESS as global less options just as you described it
> yourself, git will no longer worker properly because of the missing -R.
> 
> And actually if one would use GIT_PAGER one would again defeat the
> purpose of a allegedly global options LESS, because unless one does
> something like GIT_PAGER="${LESS}R" it wouldn't see any changes made to
> LESS.

Let me clarify that the contents of $LESS is applied by less(1) 
internally, so the final runtime configuration for less(1) is a sum of 
the configurations made available through the $LESS and $PAGER (or 
$GIT_PAGER) environment variables.  It's a rather powerful approach, if 
used properly.

>> Moreover, the whole idea of the various utilities touching the $LESS
>> variable internally is to provide sane defaults to the users that
>> don't
>> configure $LESS, $PAGER, etc. on their own.
> 
> Then I don't see what the big problem would be to just do it via a
> command argument - if someone really has ever some reasons to remove --
> RAW‐CONTROL‐CHARS from the command options when less is invoked via git
> ... then he could still go into git_config and set that manually.
> 
> But it would seem to me that the overall handling would be much more
> what one expects, than when doing the same via LESS.

Again, adding or modifying any command-line arguments by git itself 
would defeat the purpose of the environment variables and prevent the 
users from making their choice of the pagination configuration freely.

>> I don't know what delta is and how it actually paginates its outputs,
>> but it should follow the rules of the environment-based pager
>> configuration that I described in detail above.
> 
> Well, AFAIU, it doesn't and for good reasons :-)

In that case, delta does it wrong, and I hope you understand why.

> Anyway... I think all necessary things have been said and this thread
> has grown far to large with only semi-related stuff... so thanks for
> all the replies why git uses "-X".

I hope all this was useful to you.  It was useful to me. :)

Re: why does git set X in LESS env var?

[Jeff King]

On Wed, Oct 11, 2023 at 03:23:20PM -0700, Junio C Hamano wrote:

> Christoph Anton Mitterer <calestyo@scientia.org> writes:
> 
> > But I still don't get from that why X would be needed?
> >
> > My less manpage documents it as:
> >> -X or --no‐init
> >>     Disables sending the termcap initialization and deinitialization
> >>     strings to the terminal.  This is sometimes desirable if the
> >>     deinitialization string does something unnecessary, like clearing
> >>     the screen.
> >
> > Is it to avoid clearing the screen?
> 
> I think that was the reason we added it back in 2005.  In any case,
> asking "why" is not a useful use of anybody's time, because it is
> very unlikely to change in the official version we ship, and because
> it is so easy for any individual who does not like it to drop by
> exporting the $LESS environment variable.

I agree it is probably not worth changing now, but I think the history
here is a little interesting.

Yes, I think "X" was added because less would clear the screen after
exiting, and with "F" this meant you'd see nothing. Here's a thread from
the same time period discussing it:

  https://lore.kernel.org/git/cc723f590610210623sbee2075i5f2fd441cceb84ae@mail.gmail.com/

But I also think this was a pretty well-known annoyance with "less" back
then.

However, I can't seem to reproduce it now! Digging into the history and
the changelog, this note is in "changes between less versions 487 and
530":

  Don't output terminal init sequence if using -F and file fits on one
  screen.

So it seems like the problem has been fixed inside less for recent
versions. And in theory we _could_ drop "-X" if it is causing problems.
That version of less is ~5 years old. It does seem a little premature to
assume everybody has it. And as you say, if there are people who really
care about their LESS options, it is easy for them to override it.

-Peff

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 02:04, Jeff King wrote:
> On Wed, Oct 11, 2023 at 03:23:20PM -0700, Junio C Hamano wrote:
> 
>> Christoph Anton Mitterer <calestyo@scientia.org> writes:
>> 
>> > But I still don't get from that why X would be needed?
>> >
>> > My less manpage documents it as:
>> >> -X or --no‐init
>> >>     Disables sending the termcap initialization and deinitialization
>> >>     strings to the terminal.  This is sometimes desirable if the
>> >>     deinitialization string does something unnecessary, like clearing
>> >>     the screen.
>> >
>> > Is it to avoid clearing the screen?
>> 
>> I think that was the reason we added it back in 2005.  In any case,
>> asking "why" is not a useful use of anybody's time, because it is
>> very unlikely to change in the official version we ship, and because
>> it is so easy for any individual who does not like it to drop by
>> exporting the $LESS environment variable.
> 
> I agree it is probably not worth changing now, but I think the history
> here is a little interesting.
> 
> Yes, I think "X" was added because less would clear the screen after
> exiting, and with "F" this meant you'd see nothing. Here's a thread 
> from
> the same time period discussing it:
> 
> 
> https://lore.kernel.org/git/cc723f590610210623sbee2075i5f2fd441cceb84ae@mail.gmail.com/
> 
> But I also think this was a pretty well-known annoyance with "less" 
> back
> then.
> 
> However, I can't seem to reproduce it now! Digging into the history and
> the changelog, this note is in "changes between less versions 487 and
> 530":
> 
>   Don't output terminal init sequence if using -F and file fits on one
>   screen.
> 
> So it seems like the problem has been fixed inside less for recent
> versions. And in theory we _could_ drop "-X" if it is causing problems.
> That version of less is ~5 years old. It does seem a little premature 
> to
> assume everybody has it. And as you say, if there are people who really
> care about their LESS options, it is easy for them to override it.

Thanks for this detailed analysis!

It's important to keep in mind that removing "-X" and leaving "-F" would 
introduce an inconsistency in the displaying of outputs, by having the 
long outputs disappear after exiting the pager manually, and by having 
the short outputs remain displayed after the pager exits on its own.

On the other hand, removing both "-X" and "-F" would make people even 
more annoyed by requiring them to exit the pager manually even for short 
outputs.

Quite frankly, "-FRX" is a just fine set of defaults.

Re: why does git set X in LESS env var?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> ... Digging into the history and
> the changelog, this note is in "changes between less versions 487 and
> 530":
>
>   Don't output terminal init sequence if using -F and file fits on one
>   screen.

;-)

That is really a good one to dig out.  So in short, X was needed
because we wanted to use F, and we could drop it if everybody is
using recent versions of less, but the default to use FX at the same
time gives us the same behaviour between both newer and older
versions of less.

> ... And as you say, if there are people who really
> care about their LESS options, it is easy for them to override it.

Yup.  I really like the discovery of that changelog entry.

Thanks.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 03:39, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
> 
>> ... Digging into the history and
>> the changelog, this note is in "changes between less versions 487 and
>> 530":
>> 
>>   Don't output terminal init sequence if using -F and file fits on one
>>   screen.
> 
> ;-)
> 
> That is really a good one to dig out.  So in short, X was needed
> because we wanted to use F, and we could drop it if everybody is
> using recent versions of less, but the default to use FX at the same
> time gives us the same behaviour between both newer and older
> versions of less.

Please note that dropping "-X" and leaving "-F" would actually introduce 
the inconsistency that I already mentioned.  To reiterate, short outputs 
would then remain displayed on screen, while long outputs would 
disappear after exiting less(1).  I don't think that's the desired 
behavior.

>> ... And as you say, if there are people who really
>> care about their LESS options, it is easy for them to override it.
> 
> Yup.  I really like the discovery of that changelog entry.
> 
> Thanks.

Re: why does git set X in LESS env var?

[Junio C Hamano]

Dragan Simic <dsimic@manjaro.org> writes:

> Please note that dropping "-X" and leaving "-F" would actually
> introduce the inconsistency that I already mentioned.  To reiterate,
> short outputs would then remain displayed on screen, while long
> outputs would disappear after exiting less(1).

Good point.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 18:19, Junio C Hamano wrote:
> Dragan Simic <dsimic@manjaro.org> writes:
> 
>> Please note that dropping "-X" and leaving "-F" would actually
>> introduce the inconsistency that I already mentioned.  To reiterate,
>> short outputs would then remain displayed on screen, while long
>> outputs would disappear after exiting less(1).
> 
> Good point.

I've been thinking about this, and a rather elegant, backward-compatible 
solution is possible, but it requires some improvements to be made to 
less(1) first.  I'll reach out to the author of less(1) and propose that 
new feature, and I'll let you know his opinion about it.

Re: why does git set X in LESS env var?

[Thomas Guyot]

On 2023-11-02 01:48, Thomas Guyot wrote:
> Hey there...

I obviously thought about checking this client's setting the moment I 
hit the send button - if the response is garbled I'll resend it properly.

--
Thomas

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-11-02 07:01, Thomas Guyot wrote:
> On 2023-11-02 01:48, Thomas Guyot wrote:
>> Hey there...
> 
> I obviously thought about checking this client's setting the moment I
> hit the send button - if the response is garbled I'll resend it
> properly.

It's in HTML format, which isn't the preferred way, but it's still 
readable.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-11-02 06:48, Thomas Guyot wrote:
> On 2023-10-13 16:12, Dragan Simic wrote:
>> On 2023-10-12 18:19, Junio C Hamano wrote:
>>> Dragan Simic <dsimic@manjaro.org> writes:
>>>> Please note that dropping "-X" and leaving "-F" would actually
>>>> introduce the inconsistency that I already mentioned.  To reiterate,
>>>> short outputs would then remain displayed on screen, while long
>>>> outputs would disappear after exiting less(1).
>>> 
>>> Good point.
>> 
>> I've been thinking about this, and a rather elegant, 
>> backward-compatible
>> solution is possible, but it requires some improvements to be made to
>> less(1) first.  I'll reach out to the author of less(1) and propose 
>> that
>> new feature, and I'll let you know his opinion about it.
> 
>  Hey there...

Hello!

> I'm clearly late to the party but I'm wondering, has anyone tested
> adding -cy0 ? From the manpage (slightly edited):
> 
>        -c or --clear-screen ( and backward compat. -C or
> --CLEAR-SCREEN )
>               Causes full screen repaints to be painted from the top
> line down.  By default, full screen repaints are done by scrolling
> from the  bottom  of the screen.

AFAIK, the "-c" option is about the way screen contents is updated when 
scrolled, and it exists to aid in resolving possible issues with some 
terminal emulators.  To make sure, I just tested it, and "-c" doesn't 
replace "-X".

>        -yn or --max-forw-scroll=n
>               Specifies  a  maximum  number of lines to scroll
> forward.  If it is necessary to scroll forward more than n lines, the
> screen is repainted in‐
>               stead.  The -c or -C option may be used to repaint from
> the top of the screen if desired.  By default, any forward movement
> causes scrolling.

This option is, I'd guess, also about aiding in resolving possible 
issues with some terminal emulators.  Or maybe even with some actual 
terminals as pieces of hardware, who knows, which may be too slow to 
scroll many lines at once.

> I actually have one major issue with it, it's that displaying anything
> less than a full page will fill the screen with ~ on the bottom, just
> like when scrolling up on a partial page  without -F. I can see this
> being a major annoyance when using for ex. git log -1, git show --stat
> or --name-only, etc. as I  usually do it to keep the latest history
> within the current screen (and there's likely even commands that I
> never seen using the pager because I never exceeded the page height).

Huh, this confuses me a bit, quite frankly.  Isn't the "-F" option used 
specifically to make pagination invisible in case fewer lines than one 
full screen are displayed?

> OTOH by repainting from the top, the scrollback buffer is never
> affected. only the last displayed page remains on the terminal.

Just to clarify, it's the "-X" option that creates all the issues, and 
the "--redraw-on-quit" option is already there to replace it with no 
associated issues, but the trouble is that only newer versions of 
less(1) support the "--redraw-on-quit" option.  IOW, it's all about 
improving less(1) to avoid complex workarounds required to handle 
different versions, such as the workarounds used in bat(1).

> If less could only enable this behavior after the first full page
> draw, that would be perfect!

Could you, please, elaborate a bit on that?

> Dragan, that may be useful if you're discussing with less
> developers...

We've basically reached some kind of an agreement about the need for a 
good solution, which turned out to be rather complex as a result of 
being quite universal and extensible, which was required for it to, 
hopefully, be accepted into less(1).  Also, the author of less(1) seems 
to be quite busy with some other things, and he prefers to implement new 
features himself.

We've also agreed on another new feature for less(1), hopefully, which 
isn't exactly related, but should be quite useful.  It's about the 
secure mode for less(1).

Re: why does git set X in LESS env var?

[Thomas Guyot]

On 2023-11-02 02:48, Dragan Simic wrote:
> On 2023-11-02 06:48, Thomas Guyot wrote:
>>         -c or --clear-screen ( and backward compat. -C or
>> --CLEAR-SCREEN )
>>                Causes full screen repaints to be painted from the top
>> line down.  By default, full screen repaints are done by scrolling
>> from the  bottom  of the screen.
> AFAIK, the "-c" option is about the way screen contents is updated when
> scrolled, and it exists to aid in resolving possible issues with some
> terminal emulators.  To make sure, I just tested it, and "-c" doesn't
> replace "-X".
That's correct, you need both and also -y0

>>         -yn or --max-forw-scroll=n
>>                Specifies  a  maximum  number of lines to scroll
>> forward.  If it is necessary to scroll forward more than n lines, the
>> screen is repainted in‐
>>                stead.  The -c or -C option may be used to repaint from
>> the top of the screen if desired.  By default, any forward movement
>> causes scrolling.
> This option is, I'd guess, also about aiding in resolving possible
> issues with some terminal emulators.  Or maybe even with some actual
> terminals as pieces of hardware, who knows, which may be too slow to
> scroll many lines at once.

With a value of 0, it effectively redraw the screen on scroll. This 
could have a potential impact on slow connections.

>> I actually have one major issue with it, it's that displaying anything
>> less than a full page will fill the screen with ~ on the bottom, just
>> like when scrolling up on a partial page  without -F. I can see this
>> being a major annoyance when using for ex. git log -1, git show --stat
>> or --name-only, etc. as I  usually do it to keep the latest history
>> within the current screen (and there's likely even commands that I
>> never seen using the pager because I never exceeded the page height).
> Huh, this confuses me a bit, quite frankly.  Isn't the "-F" option used
> specifically to make pagination invisible in case fewer lines than one
> full screen are displayed?

Indeed, but when less update from the bottom, it can add new lines and 
let the overflow lines scroll up into the scrollback buffer.

Then updating it from the top, it draws the whole page, top to bottom. 
That's fine for a full page but not desired for a partial one. Also note 
that on my terminal (rxvt-unicode) when less clears the screen to draw 
the first page the current screen is rolled up into scrollback - iirc 
that's a configurable option, it would be worth testing other terminal's 
behavior on that. IIRC it may also erase it when using the wrong termcap 
file.

I haven't looked at the code, but I think it could be possibly to start 
the -c behavior only after a full page is drawn, after exiting on 
partial pages, which would give us the best of both worlds.

>> OTOH by repainting from the top, the scrollback buffer is never
>> affected. only the last displayed page remains on the terminal.
> Just to clarify, it's the "-X" option that creates all the issues, and
> the "--redraw-on-quit" option is already there to replace it with no
> associated issues, but the trouble is that only newer versions of
> less(1) support the "--redraw-on-quit" option.  IOW, it's all about
> improving less(1) to avoid complex workarounds required to handle
> different versions, such as the workarounds used in bat(1).

TBH I haven't tested --redraw-on-quit, even on Debian Bookworm which was 
just released a couple months ago this option isn't available. I suspect 
that the issue isn't -X, but the scrolling behavior controlled by -y and 
the full redraw controlled by -c.Actually I just tested my solution on 
xfce4-terminal and it doesn't work, the terminal still push up stuff 
above on redraw (noteworthy is with rxvt-unicode the first draw pushes 
the current screen contents up but no other redraw does, which is what 
makes it work so well - I haven't tried to find out what is being done 
exactly... OTOH the redraw on scroll down is slightly noticeable there, 
while impossible to see on xfce4-terminal. I'll install the latest less 
and see what happens with --redraw on
>> If less could only enable this behavior after the first full page
>> draw, that would be perfect!
> Could you, please, elaborate a bit on that?

I mentioned it slightly above, to be clear it would mean that:

1. less starts by just writing lined down as usual, making any lines 
above scroll up and overflow into the scrollback buffer as usual
2.  If less draws less than a page, exits as before - the effective 
result is as if pager was cat
3. If less reaches a full page and still has lines to write, it turns on 
-c's behavior and further updates happen from the top of the screen, 
preventing scroll up (at least on rxvt-unicode)

Now, if all other terms misbehave here, that's an issue, making this 
suggestion mostly useless. And considering the number of Windows users 
we absolutely need to test Windows Terminal, and should probably test 
MacOS's term too (whatever that is).
>> Dragan, that may be useful if you're discussing with less
>> developers...
> We've basically reached some kind of an agreement about the need for a
> good solution, which turned out to be rather complex as a result of
> being quite universal and extensible, which was required for it to,
> hopefully, be accepted into less(1).  Also, the author of less(1) seems
> to be quite busy with some other things, and he prefers to implement new
> features himself.
>
> We've also agreed on another new feature for less(1), hopefully, which
> isn't exactly related, but should be quite useful.  It's about the
> secure mode for less(1).

Feel free to cc me on your next correspondence. If there are mailing 
lists archives for the thread I'll fetch them as needed. We have at 
least one working term/switch combination, which IMO is a better start 
than nothing :)

Regards,

--
Thomas

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-11-02 14:19, Thomas Guyot wrote:
> On 2023-11-02 02:48, Dragan Simic wrote:
>> On 2023-11-02 06:48, Thomas Guyot wrote:
>>>         -c or --clear-screen ( and backward compat. -C or
>>> --CLEAR-SCREEN )
>>>                Causes full screen repaints to be painted from the top
>>> line down.  By default, full screen repaints are done by scrolling
>>> from the  bottom  of the screen.
>> 
>> AFAIK, the "-c" option is about the way screen contents is updated 
>> when
>> scrolled, and it exists to aid in resolving possible issues with some
>> terminal emulators.  To make sure, I just tested it, and "-c" doesn't
>> replace "-X".
> 
> That's correct, you need both and also -y0

Hmm, I tried the following:

     GIT_PAGER='less -R -F -X -c -y0'

In my environment (Xfce), the result after scrolling the output of "git 
log -p" up and down a bit was about 20 copies of the same screen "page" 
in the scrollback, plus a couple of blank "pages".  Not good, 
unfortunately, and actually much worse than having just "-R -F -X".

>> Huh, this confuses me a bit, quite frankly.  Isn't the "-F" option 
>> used
>> specifically to make pagination invisible in case fewer lines than one
>> full screen are displayed?
> 
> Indeed, but when less update from the bottom, it can add new lines and
> let the overflow lines scroll up into the scrollback buffer.
> 
> Then updating it from the top, it draws the whole page, top to bottom.
> That's fine for a full page but not desired for a partial one. Also
> note that on my terminal (rxvt-unicode) when less clears the screen to
> draw the first page the current screen is rolled up into scrollback -
> iirc that's a configurable option, it would be worth testing other
> terminal's behavior on that. IIRC it may also erase it when using the
> wrong termcap file.
> 
> I haven't looked at the code, but I think it could be possibly to
> start the -c behavior only after a full page is drawn, after exiting
> on partial pages, which would give us the best of both worlds.

Does the GIT_PAGER setup, as I described it above, work for you without 
the described artifacts, in any of the environments you have access to?

>>> OTOH by repainting from the top, the scrollback buffer is never
>>> affected. only the last displayed page remains on the terminal.
>> 
>> Just to clarify, it's the "-X" option that creates all the issues, and
>> the "--redraw-on-quit" option is already there to replace it with no
>> associated issues, but the trouble is that only newer versions of
>> less(1) support the "--redraw-on-quit" option.  IOW, it's all about
>> improving less(1) to avoid complex workarounds required to handle
>> different versions, such as the workarounds used in bat(1).
> 
> TBH I haven't tested --redraw-on-quit, even on Debian Bookworm which
> was just released a couple months ago this option isn't available. I
> suspect that the issue isn't -X, but the scrolling behavior controlled
> by -y and the full redraw controlled by -c.

When you get into the terminfo entry definitions, the root cause is that 
the terminal initialization sequences contain switching to alternate 
screen, which causes screen contents to be lost when less(1) exits.  
Thus, "-X" has been actually abused in the pager setups to skip the 
terminal initialization sequences, which may also result in other 
issues.

One of the solutions is to edit the terminfo entry manually and remove 
the escape codes that cause the switching to and from alternate screen, 
which I tested, but that also introduced another issue -- the screen 
contents was always present after less(1) exited, which isn't always the 
desired behavior.

> Actually I just tested my
> solution on xfce4-terminal and it doesn't work, the terminal still
> push up stuff above on redraw (noteworthy is with rxvt-unicode the
> first draw pushes the current screen contents up but no other redraw
> does, which is what makes it work so well - I haven't tried to find
> out what is being done exactly... OTOH the redraw on scroll down is
> slightly noticeable there, while impossible to see on xfce4-terminal.
> I'll install the latest less and see what happens with --redraw on

Please test the "--redraw-on-quit" option, so far it's the best 
available solution, IMHO.

>>> If less could only enable this behavior after the first full page
>>> draw, that would be perfect!
>> 
>> Could you, please, elaborate a bit on that?
> 
> I mentioned it slightly above, to be clear it would mean that:
> 
> 1. less starts by just writing lined down as usual, making any lines
> above scroll up and overflow into the scrollback buffer as usual
> 2.  If less draws less than a page, exits as before - the effective
> result is as if pager was cat
> 3. If less reaches a full page and still has lines to write, it turns
> on -c's behavior and further updates happen from the top of the
> screen, preventing scroll up (at least on rxvt-unicode)
> 
> Now, if all other terms misbehave here, that's an issue, making this
> suggestion mostly useless. And considering the number of Windows users
> we absolutely need to test Windows Terminal, and should probably test
> MacOS's term too (whatever that is).

Quite frankly, I think that such a solution would be like "fixing the 
fix, which is actually an abuse", as I described it above, eventually 
introducing even more issues, instead of solving the original issue.

>>> Dragan, that may be useful if you're discussing with less
>>> developers...
>> 
>> We've basically reached some kind of an agreement about the need for a
>> good solution, which turned out to be rather complex as a result of
>> being quite universal and extensible, which was required for it to,
>> hopefully, be accepted into less(1).  Also, the author of less(1) 
>> seems
>> to be quite busy with some other things, and he prefers to implement 
>> new
>> features himself.
>> 
>> We've also agreed on another new feature for less(1), hopefully, which
>> isn't exactly related, but should be quite useful.  It's about the
>> secure mode for less(1).
> 
> Feel free to cc me on your next correspondence. If there are mailing
> lists archives for the thread I'll fetch them as needed. We have at
> least one working term/switch combination, which IMO is a better start
> than nothing :)

Please test the "--redraw-on-quit" option, AFAICT that's all we need 
(plus the already mentioned other improvements to less(1), to avoid the 
version-dependent workarounds), and the distributions will eventually 
catch up with the newer versions of less(1).  If the whole thing has 
worked for decades as-is, it can continue working that way for a year or 
two until the packages get updated.

There's actually no two-way mailing list for less(1), the entire project 
is pretty much a one-man show, so to speak.  There's a GitHub page that 
allows issues to be submitted, but I didn't use that, so I exchanged a 
few private email messages instead with the author.  I've already summed 
up the important parts of those messages.

Re: why does git set X in LESS env var?

[Thomas Guyot]

On 2023-11-02 10:19, Dragan Simic wrote:
> On 2023-11-02 14:19, Thomas Guyot wrote:
>> That's correct, you need both and also -y0
> Hmm, I tried the following:
>
>       GIT_PAGER='less -R -F -X -c -y0'
>
> In my environment (Xfce), the result after scrolling the output of "git
> log -p" up and down a bit was about 20 copies of the same screen "page"
> in the scrollback, plus a couple of blank "pages".  Not good,
> unfortunately, and actually much worse than having just "-R -F -X".

Indeed, I did notice it with xfce4-term - I suspect different controls 
are used to clear the screen, with rxvt-unicode the initial one scrolls 
anything in the current display to the scrollback (which is important to 
avoid clearing the last commands/output form the scrollback), but 
afterward screen updates do not update the scrollback, it remains within 
the screen.

These options may actually affect the behavior (I have put my current 
values there)

     secondaryScreen: True
         Turn on/off secondary screen (default enabled).

     secondaryScroll: True
         Turn on/off secondary screen scroll (default enabled). If this
         option is enabled, scrolls on the secondary screen will change the
         scrollback buffer and, when secondaryScreen is off, switching
         to/from the secondary screen will instead scroll the screen up.

So it appears I could disable secondaryScroll to avoid getting scrolled 
lines into the scrollback buffer. It's also noteworthy that with 
secondaryScreen disabled, rxvt-unicode instead scroll things up to avoid 
the current screen form being wiped out.
>> Indeed, but when less update from the bottom, it can add new lines and
>> let the overflow lines scroll up into the scrollback buffer.
>>
>> Then updating it from the top, it draws the whole page, top to bottom.
>> That's fine for a full page but not desired for a partial one. Also
>> note that on my terminal (rxvt-unicode) when less clears the screen to
>> draw the first page the current screen is rolled up into scrollback -
>> iirc that's a configurable option, it would be worth testing other
>> terminal's behavior on that. IIRC it may also erase it when using the
>> wrong termcap file.
>>
>> I haven't looked at the code, but I think it could be possibly to
>> start the -c behavior only after a full page is drawn, after exiting
>> on partial pages, which would give us the best of both worlds.
> Does the GIT_PAGER setup, as I described it above, work for you without
> the described artifacts, in any of the environments you have access to?

Yes, rxvt-unicode, with the settings mentioned above.

>>> Just to clarify, it's the "-X" option that creates all the issues, and
>>> the "--redraw-on-quit" option is already there to replace it with no
>>> associated issues, but the trouble is that only newer versions of
>>> less(1) support the "--redraw-on-quit" option.  IOW, it's all about
>>> improving less(1) to avoid complex workarounds required to handle
>>> different versions, such as the workarounds used in bat(1).
>> TBH I haven't tested --redraw-on-quit, even on Debian Bookworm which
>> was just released a couple months ago this option isn't available. I
>> suspect that the issue isn't -X, but the scrolling behavior controlled
>> by -y and the full redraw controlled by -c.
> When you get into the terminfo entry definitions, the root cause is that
> the terminal initialization sequences contain switching to alternate
> screen, which causes screen contents to be lost when less(1) exits.
> Thus, "-X" has been actually abused in the pager setups to skip the
> terminal initialization sequences, which may also result in other
> issues.

Right - this is something that rxvt-unicode addresses with the correct 
settings.
IIRC I disabled the secondasyScreen to be able to use the scrollback 
buffer (unavailable otherwise), which probably also means that with it 
enabled I would also have no issue without -X, but that would need 
testing. Also note that rxvt-unicode has its own terminfo file 
(xfce4-term uses xterm), and in some places I'm forced to use xterm as a 
fallback I get issues like current screen being wiped when switching 
to/from secondary screen.
> One of the solutions is to edit the terminfo entry manually and remove
> the escape codes that cause the switching to and from alternate screen,
> which I tested, but that also introduced another issue -- the screen
> contents was always present after less(1) exited, which isn't always the
> desired behavior.

But when less scrolls down (line by line, not page by page), it always 
append lines and let them scroll up. Won't you see these? (page-by-page 
otoh will redraw the full screen without scrolling).

Also won't it wipe the *current* screen so that you won't see the 
commands/output you had *before* running less?

This is why rxvt-unicode has an option to disable secondary screen, and 
scroll contents all the way to the buffer on switch to/from secondary 
screen.
>> Actually I just tested my
>> solution on xfce4-terminal and it doesn't work, the terminal still
>> push up stuff above on redraw (noteworthy is with rxvt-unicode the
>> first draw pushes the current screen contents up but no other redraw
>> does, which is what makes it work so well - I haven't tried to find
>> out what is being done exactly... OTOH the redraw on scroll down is
>> slightly noticeable there, while impossible to see on xfce4-terminal.
>> I'll install the latest less and see what happens with --redraw on
> Please test the "--redraw-on-quit" option, so far it's the best
> available solution, IMHO.

I will, not now though - need it compile less form source I guess...
>>>> If less could only enable this behavior after the first full page
>>>> draw, that would be perfect!
>>> Could you, please, elaborate a bit on that?
>> I mentioned it slightly above, to be clear it would mean that:
>>
>> 1. less starts by just writing lined down as usual, making any lines
>> above scroll up and overflow into the scrollback buffer as usual
>> 2.  If less draws less than a page, exits as before - the effective
>> result is as if pager was cat
>> 3. If less reaches a full page and still has lines to write, it turns
>> on -c's behavior and further updates happen from the top of the
>> screen, preventing scroll up (at least on rxvt-unicode)
>>
>> Now, if all other terms misbehave here, that's an issue, making this
>> suggestion mostly useless. And considering the number of Windows users
>> we absolutely need to test Windows Terminal, and should probably test
>> MacOS's term too (whatever that is).
> Quite frankly, I think that such a solution would be like "fixing the
> fix, which is actually an abuse", as I described it above, eventually
> introducing even more issues, instead of solving the original issue.

I'll let you know my findings... I'm not convinced --redraw-on-quit is 
actually going to fix it for all unless this does a lot more than the 
option name implies (but quite happy if it does).

Regards.
--
Thomas

Re: why does git set X in LESS env var?

[Andy Koppe]

Thomas Guyot wrote:
> >> I actually have one major issue with it, it's that displaying anything
> >> less than a full page will fill the screen with ~ on the bottom, just
> >> like when scrolling up on a partial page  without -F.

'less' has the '-~' (or --tilde) option to suppress that.

> I mentioned it slightly above, to be clear it would mean that:
>
> 1. less starts by just writing lined down as usual, making any lines
> above scroll up and overflow into the scrollback buffer as usual
> 2.  If less draws less than a page, exits as before - the effective
> result is as if pager was cat
> 3. If less reaches a full page and still has lines to write, it turns on
> -c's behavior and further updates happen from the top of the screen,
> preventing scroll up (at least on rxvt-unicode)
>
> Now, if all other terms misbehave here, that's an issue, making this
> suggestion mostly useless. And considering the number of Windows users
> we absolutely need to test Windows Terminal, and should probably test
> MacOS's term too (whatever that is).

For what it's worth, the 'mintty' terminal used by default for Git for
Windows as well as MSYS and Cygwin has another approach to the whole
problem. Its rather flippantly named 'Flip Screen' context menu
command with Alt+F12 or Ctrl+Shift+S shortcut lets users temporarily
look at the alternate screen buffer while the main screen buffer is
active, and vice versa.

If 'less' is invoked without the -X option, it will switch to the
alternate screen, where mousewheel scrolling works by sending cursor
up/down keycodes. While in 'less', you can temporarily flip to the
main screen to look up something in the shell session there or copy
something for searching in 'less'. While looking at the main screen,
the mousewheel will scroll the scrollback buffer. Keyboard input
that's sent to 'less' will flip back to the alternate screen.

Quitting 'less' switches back to the main screen, so the 'less' output
disappears and you're back in the shell session with the command that
invoked 'less' as the last thing shown. But again, the 'Flip Screen'
command or shortcuts can be used to temporarily look at or copy from
the alternate screen, which will contain the last page displayed by
'less'. (The alternate screen does not have a scrollback buffer.)

The 'Flip Screen' feature of course also works with other
alternate-screen applications, for example editors.

Apparently the Mac terminal has such a feature as well:
https://support.apple.com/en-ie/guide/terminal/trmld1f46097/mac

(Full disclosure: I originally made mintty, from PuTTY.)

Kind regards,
Andy

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-11-03 16:28, Andy Koppe wrote:
> Thomas Guyot wrote:
>> >> I actually have one major issue with it, it's that displaying anything
>> >> less than a full page will fill the screen with ~ on the bottom, just
>> >> like when scrolling up on a partial page  without -F.
> 
> 'less' has the '-~' (or --tilde) option to suppress that.

Good to know, thanks.  However, with the "-F" option in place, the 
situation in which less than a full page is displayed within less(1) 
shouldn't be encountered.

> For what it's worth, the 'mintty' terminal used by default for Git for
> Windows as well as MSYS and Cygwin has another approach to the whole
> problem. Its rather flippantly named 'Flip Screen' context menu
> command with Alt+F12 or Ctrl+Shift+S shortcut lets users temporarily
> look at the alternate screen buffer while the main screen buffer is
> active, and vice versa.

This is a rather neat feature.  Though, I wonder how many users are 
actually aware of this feature, and how frequently is it used.  I wasn't 
aware of it, and I used Git CLI on Windows for some time.

> If 'less' is invoked without the -X option, it will switch to the
> alternate screen, where mousewheel scrolling works by sending cursor
> up/down keycodes. While in 'less', you can temporarily flip to the
> main screen to look up something in the shell session there or copy
> something for searching in 'less'. While looking at the main screen,
> the mousewheel will scroll the scrollback buffer. Keyboard input
> that's sent to 'less' will flip back to the alternate screen.

To me, this is another confirmation that (ab)using the "-X" option is 
something that we need to get rid of, as I already described earlier in 
this email thread.

> Quitting 'less' switches back to the main screen, so the 'less' output
> disappears and you're back in the shell session with the command that
> invoked 'less' as the last thing shown. But again, the 'Flip Screen'
> command or shortcuts can be used to temporarily look at or copy from
> the alternate screen, which will contain the last page displayed by
> 'less'. (The alternate screen does not have a scrollback buffer.)
> 
> The 'Flip Screen' feature of course also works with other
> alternate-screen applications, for example editors.
> 
> Apparently the Mac terminal has such a feature as well:
> https://support.apple.com/en-ie/guide/terminal/trmld1f46097/mac

Maybe there is some data available about how frequently this neat 
feature is used?  It would be really good to know how much is it 
actually used.

> (Full disclosure: I originally made mintty, from PuTTY.)

Thanks for your work!

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-11-03 12:47, Thomas Guyot wrote:
> On 2023-11-02 10:19, Dragan Simic wrote:
>> On 2023-11-02 14:19, Thomas Guyot wrote:
>>> That's correct, you need both and also -y0
>> 
>> Hmm, I tried the following:
>> 
>>       GIT_PAGER='less -R -F -X -c -y0'
>> 
>> In my environment (Xfce), the result after scrolling the output of 
>> "git
>> log -p" up and down a bit was about 20 copies of the same screen 
>> "page"
>> in the scrollback, plus a couple of blank "pages".  Not good,
>> unfortunately, and actually much worse than having just "-R -F -X".
> 
> Indeed, I did notice it with xfce4-term - I suspect different controls
> are used to clear the screen, with rxvt-unicode the initial one
> scrolls anything in the current display to the scrollback (which is
> important to avoid clearing the last commands/output form the
> scrollback), but afterward screen updates do not update the
> scrollback, it remains within the screen.
> 
> These options may actually affect the behavior (I have put my current
> values there)
> 
>     secondaryScreen: True
>         Turn on/off secondary screen (default enabled).
> 
>     secondaryScroll: True
>         Turn on/off secondary screen scroll (default enabled). If this
>         option is enabled, scrolls on the secondary screen will change 
> the
>         scrollback buffer and, when secondaryScreen is off, switching
>         to/from the secondary screen will instead scroll the screen up.
> 
> So it appears I could disable secondaryScroll to avoid getting
> scrolled lines into the scrollback buffer. It's also noteworthy that
> with secondaryScreen disabled, rxvt-unicode instead scroll things up
> to avoid the current screen form being wiped out.

This is quite interesting.  However, we can't expect that the users 
perform such adjustments to their environments, if you agree.  Tweaking 
one's own environment is fine, but the shipped solution should "just 
work" in most environments, or ideally in all environments in their 
original form.

>>> Indeed, but when less update from the bottom, it can add new lines 
>>> and
>>> let the overflow lines scroll up into the scrollback buffer.
>>> 
>>> Then updating it from the top, it draws the whole page, top to 
>>> bottom.
>>> That's fine for a full page but not desired for a partial one. Also
>>> note that on my terminal (rxvt-unicode) when less clears the screen 
>>> to
>>> draw the first page the current screen is rolled up into scrollback -
>>> iirc that's a configurable option, it would be worth testing other
>>> terminal's behavior on that. IIRC it may also erase it when using the
>>> wrong termcap file.
>>> 
>>> I haven't looked at the code, but I think it could be possibly to
>>> start the -c behavior only after a full page is drawn, after exiting
>>> on partial pages, which would give us the best of both worlds.
>> 
>> Does the GIT_PAGER setup, as I described it above, work for you 
>> without
>> the described artifacts, in any of the environments you have access 
>> to?
> 
> Yes, rxvt-unicode, with the settings mentioned above.

Good to know, thanks.  Looking forward to see will the 
"--redraw-on-quit" option work as expected in your rxvt-unicode 
terminal.

>> One of the solutions is to edit the terminfo entry manually and remove
>> the escape codes that cause the switching to and from alternate 
>> screen,
>> which I tested, but that also introduced another issue -- the screen
>> contents was always present after less(1) exited, which isn't always 
>> the
>> desired behavior.
> 
> But when less scrolls down (line by line, not page by page), it always
> append lines and let them scroll up. Won't you see these?
> (page-by-page otoh will redraw the full screen without scrolling).
> 
> Also won't it wipe the *current* screen so that you won't see the
> commands/output you had *before* running less?
> 
> This is why rxvt-unicode has an option to disable secondary screen,
> and scroll contents all the way to the buffer on switch to/from
> secondary screen.

IIRC, when I tested by removing the escape codes manually from the 
terminfo entry, it all worked fine, but the screen contents always 
remained displayed after less(1) exited.  That isn't always the desired 
behavior.

>>> Actually I just tested my
>>> solution on xfce4-terminal and it doesn't work, the terminal still
>>> push up stuff above on redraw (noteworthy is with rxvt-unicode the
>>> first draw pushes the current screen contents up but no other redraw
>>> does, which is what makes it work so well - I haven't tried to find
>>> out what is being done exactly... OTOH the redraw on scroll down is
>>> slightly noticeable there, while impossible to see on xfce4-terminal.
>>> I'll install the latest less and see what happens with --redraw on
>> 
>> Please test the "--redraw-on-quit" option, so far it's the best
>> available solution, IMHO.
> 
> I will, not now though - need it compile less form source I guess...

Great, thanks.  Looking forward to the results of your testing.  As a 
reminder, this is what I use:

     GIT_PAGER='less -R -F --redraw-on-quit'

My LESS environment variable contains only some coloring-related 
options, which don't matter in this case.

>>> Now, if all other terms misbehave here, that's an issue, making this
>>> suggestion mostly useless. And considering the number of Windows 
>>> users
>>> we absolutely need to test Windows Terminal, and should probably test
>>> MacOS's term too (whatever that is).
>> 
>> Quite frankly, I think that such a solution would be like "fixing the
>> fix, which is actually an abuse", as I described it above, eventually
>> introducing even more issues, instead of solving the original issue.
> 
> I'll let you know my findings... I'm not convinced --redraw-on-quit is
> actually going to fix it for all unless this does a lot more than the
> option name implies (but quite happy if it does).

Actually, it's more about not (ab)using the "-X" option, because 
skipping the terminal initialization may cause various issues.  The 
"--redraw-on-quit" option is there just to have the screen contents 
preserved after less(1) exits.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-11-02 15:19, Dragan Simic wrote:
> On 2023-11-02 14:19, Thomas Guyot wrote:
>> On 2023-11-02 02:48, Dragan Simic wrote:
>>> We've basically reached some kind of an agreement about the need for 
>>> a
>>> good solution, which turned out to be rather complex as a result of
>>> being quite universal and extensible, which was required for it to,
>>> hopefully, be accepted into less(1).  Also, the author of less(1) 
>>> seems
>>> to be quite busy with some other things, and he prefers to implement 
>>> new
>>> features himself.
>>> 
>>> We've also agreed on another new feature for less(1), hopefully, 
>>> which
>>> isn't exactly related, but should be quite useful.  It's about the
>>> secure mode for less(1).
>> 
>> Feel free to cc me on your next correspondence. If there are mailing
>> lists archives for the thread I'll fetch them as needed. We have at
>> least one working term/switch combination, which IMO is a better start
>> than nothing :)
> 
> Please test the "--redraw-on-quit" option, AFAICT that's all we need
> (plus the already mentioned other improvements to less(1), to avoid
> the version-dependent workarounds), and the distributions will
> eventually catch up with the newer versions of less(1).  If the whole
> thing has worked for decades as-is, it can continue working that way
> for a year or two until the packages get updated.
> 
> There's actually no two-way mailing list for less(1), the entire
> project is pretty much a one-man show, so to speak.  There's a GitHub
> page that allows issues to be submitted, but I didn't use that, so I
> exchanged a few private email messages instead with the author.  I've
> already summed up the important parts of those messages.

Good news! :)  The author of less(1) has implemented a couple of new 
features that should resolve our issues with the pagination.  The 
improvements for the secure mode of less(1) have also been implemented.  
I'll test all that in detail, and I'll move forward with implementing 
the required changes in Git.

It seems that a new version of less(1) may also be released rather soon, 
so we might be on a good way to have these longstanding issues resolved 
in the upcoming releases of Git and less(1).  It will take time for the 
Linux distributions to catch up with their package versions, but also 
the rolling-release distributions will get the new versions with no 
delays.

Re: why does git set X in LESS env var?

[Dragan Simic]

Hello all,

On 2023-11-06 04:47, Dragan Simic wrote:
> On 2023-11-02 15:19, Dragan Simic wrote:
>> On 2023-11-02 14:19, Thomas Guyot wrote:
>>> On 2023-11-02 02:48, Dragan Simic wrote:
>>>> We've basically reached some kind of an agreement about the need for 
>>>> a
>>>> good solution, which turned out to be rather complex as a result of
>>>> being quite universal and extensible, which was required for it to,
>>>> hopefully, be accepted into less(1).  Also, the author of less(1) 
>>>> seems
>>>> to be quite busy with some other things, and he prefers to implement 
>>>> new
>>>> features himself.
>>>> 
>>>> We've also agreed on another new feature for less(1), hopefully, 
>>>> which
>>>> isn't exactly related, but should be quite useful.  It's about the
>>>> secure mode for less(1).
>>> 
>>> Feel free to cc me on your next correspondence. If there are mailing
>>> lists archives for the thread I'll fetch them as needed. We have at
>>> least one working term/switch combination, which IMO is a better 
>>> start
>>> than nothing :)
>> 
>> Please test the "--redraw-on-quit" option, AFAICT that's all we need
>> (plus the already mentioned other improvements to less(1), to avoid
>> the version-dependent workarounds), and the distributions will
>> eventually catch up with the newer versions of less(1).  If the whole
>> thing has worked for decades as-is, it can continue working that way
>> for a year or two until the packages get updated.
>> 
>> There's actually no two-way mailing list for less(1), the entire
>> project is pretty much a one-man show, so to speak.  There's a GitHub
>> page that allows issues to be submitted, but I didn't use that, so I
>> exchanged a few private email messages instead with the author.  I've
>> already summed up the important parts of those messages.
> 
> Good news! :)  The author of less(1) has implemented a couple of new
> features that should resolve our issues with the pagination.  The
> improvements for the secure mode of less(1) have also been
> implemented.  I'll test all that in detail, and I'll move forward with
> implementing the required changes in Git.
> 
> It seems that a new version of less(1) may also be released rather
> soon, so we might be on a good way to have these longstanding issues
> resolved in the upcoming releases of Git and less(1).  It will take
> time for the Linux distributions to catch up with their package
> versions, but also the rolling-release distributions will get the new
> versions with no delays.

Good news, new beta version 653 of less(1) has been released! [1]

Version 653 contains new pagination-related features (in particular,
LESSKEY_CONTENT and LESS_UNSUPPORT) I asked the less(1) author for,
which will finally resolve age-old pagination issues in git and
a few other upstream projects.

I'll test the new beta version, after which I'll start working on
the required patches for git and a few other upstream projects.

Looking forward to resolving those age-old pagination issues! :)

[1] https://greenwoodsoftware.com/less/news.653.html

Re: why does git set X in LESS env var?

[Christoph Anton Mitterer]

Hey.

Just noted that the popular bat utility apparently also uses -X to make
-F work (but also mention that this break scrolling).

But it seem they have a check, an if less is version 530 or newer they
don't set -X.

https://github.com/sharkdp/bat#using-a-different-pager

Could be a way to go for git.

Cheers,
Chris.

Re: why does git set X in LESS env var?

[Dragan Simic]

On 2023-10-12 05:54, Christoph Anton Mitterer wrote:
> Hey.
> 
> Just noted that the popular bat utility apparently also uses -X to make
> -F work (but also mention that this break scrolling).
> 
> But it seem they have a check, an if less is version 530 or newer they
> don't set -X.
> 
> https://github.com/sharkdp/bat#using-a-different-pager
> 
> Could be a way to go for git.

Let me repeat that the described scrolling-related issues are 
misleading.  I use this, configured through $BAT_PAGER, and have never 
experienced such issues in a few years of using bat(1):

       ├─bash
       │   └─bat -n --tabs 8 --wrap never message.txt
       │       └─less -R -F -X