RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Ed Vance]

Hi Robert,

On Sat, June 15, 2002, Robert White wrote:
> 
> The n_tty line discipline module contains a "semantic error" 
> that limits its speed and usefullness in many uses.  The 
> attached patch directly addresses serial performance in a 
> completely backwards-compatable way.
> 
> In particular, the current handling of VMIN hugely limits, 
> complicates, and/or slows down optimal serial use.  The most 
> obvius example is that if you call read(2) with a buffer size 
> less than the current value of VMIN, the line discipline will 
> insist that the read call wait for characters that can not be 
> returned to that call.  The POSIX standard is silent on the 
> subject of whether this is right or wrong.  Common sense says 
> it is wrong.
> 

Ten years ago, I would have agreed with you. I suggested a very 
similar change for VTIME=0;VMIN>0 behavior back then, while we 
were porting SVR4 to proprietary hardware. This was for 
compatibility with the way reads were handled on a previous 
non-un*x product. It was deemed a spec violation, so we added an 
ioctl to implement the compatible behavior. 

Does the spec say that VMIN behavior depends on the size of a 
blocking read? No, it says that the read completes when VMIN or 
more characters have been received. If VMIN is three and two 
characters have been received, completing a blocking read of any 
size is a violation. Should we also add a "read buffer size" 
parameter to select and poll, etc. so they can report that read 
data is available before satisfying VMIN, too? 

Ted, Russell, please weigh in on this. 

Best regards,
Ed

---------------------------------------------------------------- 
Ed Vance              edv@macrolink.com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Russell King]

On Wed, Jun 26, 2002 at 10:48:30AM -0700, Ed Vance wrote:
> Ted, Russell, please weigh in on this. 

Remind me after OLS.  My connectivity is subject to wireless cards
sporadically loaned by other people at the moment, so I'm not always
around.

-- 
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

Howdy,

Actualy you are wrong about the "Standard".  The Linux manual pages only say 
the read will return when so many characters have been received.  
Alternately, the System V Interface Definition (in every version I have been 
able to find on the net) speaks in terms of "satisfying the read" .  Both are 
silent on the issue of what to do if the read is asking for less than VMIN 
character.

That is, since theVMIN/VIME mechanisim exists only with respect to each 
individual read request and not with respect to any kind of driver state, 
there is no inferrance or onus that says the state applies to anything other 
than the receipt of charactersfrom the driver too the read itself.

That is, the context for the word "received" MUST BE "received into the read 
buffer."  [or perhaps "received into the user context"].  How so, you may 
ask?

If "received" is in respect to the UART (e.g. the machine/hardware boundry) 
then it would be ture that if there were 100 characters in the driver's 
internal buffer, VMIN were 1, VTIM were 1, and a read were issued but nomore 
characters were "received" by the UART then the read would never return.  
That is, by the standard, "one character must be received to start the timer, 
and one character must berceived minimum" so if "receied" means"into the 
driver" instedof "into the user context" the 100 characters in the buffer, 
having been received in the past would not relate the read (and so on od 
nasium 8-)   That would be rediculous.

Since in every stated and useful sense the VMIN and VTIME mechanisims exist 
spesfically and expressley for the purpose of satisfying individual read 
requests, that is in terms of the user-context/program "receiving" data from 
the abstract "device" at the file interface, data that can not possibly 
relate to a read because it is "out of bounds"  is immaterial to the act by 
defintion.  So waiting for the user context to receive characters it has not 
asked for and can not possibly receive is a "bad thing".

The "waiting for data that can't be returned" and/or the inconsistent 
application of the concept of "received" serves nobodies interests.

I was going to simply dismiss that bit of hyperboli (sp?) about poll as a 
cheap bit of bad "debate technique" but I'v decided to respond.

Poll aready and explicitly addresses the size of the read and/or write as one 
character, so tossing it in as if it were an example was logcally junct.  
Addintionally, to make the example work you "threw in" with a comment about 
"adding an argument to poll."  The reason that this was fatuous is that the 
parameter you would "add"  to poll would be disruptive (which is why you put 
it in 8-) but the information is already provided in the read, the buffer 
length IS ALREADY being passed.  The analogy and the allegation of dificulty 
or disruption is inapplicable.

Rob.

PS I am using a horribly lossy link and an almost unreadable font (I love 
traveling on business 8-) so please excuse the particularly bad spelling, 
grammar, and puncuation... 8-)

On Wednesday 26 June 2002 10:48, Ed Vance wrote:
> Hi Robert,
>
> On Sat, June 15, 2002, Robert White wrote:
> > The n_tty line discipline module contains a "semantic error"
> > that limits its speed and usefullness in many uses.  The
> > attached patch directly addresses serial performance in a
> > completely backwards-compatable way.
> >
> > In particular, the current handling of VMIN hugely limits,
> > complicates, and/or slows down optimal serial use.  The most
> > obvius example is that if you call read(2) with a buffer size
> > less than the current value of VMIN, the line discipline will
> > insist that the read call wait for characters that can not be
> > returned to that call.  The POSIX standard is silent on the
> > subject of whether this is right or wrong.  Common sense says
> > it is wrong.
>
> Ten years ago, I would have agreed with you. I suggested a very
> similar change for VTIME=0;VMIN>0 behavior back then, while we
> were porting SVR4 to proprietary hardware. This was for
> compatibility with the way reads were handled on a previous
> non-un*x product. It was deemed a spec violation, so we added an
> ioctl to implement the compatible behavior.
>
> Does the spec say that VMIN behavior depends on the size of a
> blocking read? No, it says that the read completes when VMIN or
> more characters have been received. If VMIN is three and two
> characters have been received, completing a blocking read of any
> size is a violation. Should we also add a "read buffer size"
> parameter to select and poll, etc. so they can report that read
> data is available before satisfying VMIN, too?
>
> Ted, Russell, please weigh in on this.
>
> Best regards,
> Ed
>
> ----------------------------------------------------------------
> Ed Vance              edv@macrolink.com
> Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
> ----------------------------------------------------------------

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

Howdy,

Actualy you are wrong about the "Standard".  The Linux manual pages only say 
the read will return when so many characters have been received.  
Alternately, the System V Interface Definition (in every version I have been 
able to find on the net) speaks in terms of "satisfying the read" .  Both are 
silent on the issue of what to do if the read is asking for less than VMIN 
character.

That is, since theVMIN/VIME mechanisim exists only with respect to each 
individual read request and not with respect to any kind of driver state, 
there is no inferrance or onus that says the state applies to anything other 
than the receipt of charactersfrom the driver too the read itself.

That is, the context for the word "received" MUST BE "received into the read 
buffer."  [or perhaps "received into the user context"].  How so, you may 
ask?

If "received" is in respect to the UART (e.g. the machine/hardware boundry) 
then it would be ture that if there were 100 characters in the driver's 
internal buffer, VMIN were 1, VTIM were 1, and a read were issued but nomore 
characters were "received" by the UART then the read would never return.  
That is, by the standard, "one character must be received to start the timer, 
and one character must berceived minimum" so if "receied" means"into the 
driver" instedof "into the user context" the 100 characters in the buffer, 
having been received in the past would not relate the read (and so on od 
nasium 8-)   That would be rediculous.

Since in every stated and useful sense the VMIN and VTIME mechanisims exist 
spesfically and expressley for the purpose of satisfying individual read 
requests, that is in terms of the user-context/program "receiving" data from 
the abstract "device" at the file interface, data that can not possibly 
relate to a read because it is "out of bounds"  is immaterial to the act by 
defintion.  So waiting for the user context to receive characters it has not 
asked for and can not possibly receive is a "bad thing".

The "waiting for data that can't be returned" and/or the inconsistent 
application of the concept of "received" serves nobodies interests.

I was going to simply dismiss that bit of hyperboli (sp?) about poll as a 
cheap bit of bad "debate technique" but I'v decided to respond.

Poll aready and explicitly addresses the size of the read and/or write as one 
character, so tossing it in as if it were an example was logcally junct.  
Addintionally, to make the example work you "threw in" with a comment about 
"adding an argument to poll."  The reason that this was fatuous is that the 
parameter you would "add"  to poll would be disruptive (which is why you put 
it in 8-) but the information is already provided in the read, the buffer 
length IS ALREADY being passed.  The analogy and the allegation of dificulty 
or disruption is inapplicable.

Rob.

PS I am using a horribly lossy link and an almost unreadable font (I love 
traveling on business 8-) so please excuse the particularly bad spelling, 
grammar, and puncuation... 8-)

On Wednesday 26 June 2002 10:48, Ed Vance wrote:
> Hi Robert,
>
> On Sat, June 15, 2002, Robert White wrote:
> > The n_tty line discipline module contains a "semantic error"
> > that limits its speed and usefullness in many uses.  The
> > attached patch directly addresses serial performance in a
> > completely backwards-compatable way.
> >
> > In particular, the current handling of VMIN hugely limits,
> > complicates, and/or slows down optimal serial use.  The most
> > obvius example is that if you call read(2) with a buffer size
> > less than the current value of VMIN, the line discipline will
> > insist that the read call wait for characters that can not be
> > returned to that call.  The POSIX standard is silent on the
> > subject of whether this is right or wrong.  Common sense says
> > it is wrong.
>
> Ten years ago, I would have agreed with you. I suggested a very
> similar change for VTIME=0;VMIN>0 behavior back then, while we
> were porting SVR4 to proprietary hardware. This was for
> compatibility with the way reads were handled on a previous
> non-un*x product. It was deemed a spec violation, so we added an
> ioctl to implement the compatible behavior.
>
> Does the spec say that VMIN behavior depends on the size of a
> blocking read? No, it says that the read completes when VMIN or
> more characters have been received. If VMIN is three and two
> characters have been received, completing a blocking read of any
> size is a violation. Should we also add a "read buffer size"
> parameter to select and poll, etc. so they can report that read
> data is available before satisfying VMIN, too?
>
> Ted, Russell, please weigh in on this.
>
> Best regards,
> Ed
>
> ----------------------------------------------------------------
> Ed Vance              edv@macrolink.com
> Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
> ----------------------------------------------------------------

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Russell King]

On Wed, Jun 26, 2002 at 10:48:30AM -0700, Ed Vance wrote:
> Does the spec say that VMIN behavior depends on the size of a 
> blocking read? No, it says that the read completes when VMIN or 
> more characters have been received. If VMIN is three and two 
> characters have been received, completing a blocking read of any 
> size is a violation. Should we also add a "read buffer size" 
> parameter to select and poll, etc. so they can report that read 
> data is available before satisfying VMIN, too? 
> 
> Ted, Russell, please weigh in on this. 

I just found this mail again.  Yes, I agree with your interpretation,
which reflects the code we presently have, as well as my reading of
SuS.  The SuS is quite clear that "A pending read shall not be
satisfied until MIN bytes are received".  It doesn't say "A pending
read shall not be satisfied until MIN bytes or the number of requested
bytes in read() are received"

In addition, it also says "MIN represents the minimum number of bytes
that should be received when the read() function returns successfully."
Successful completion for read() is defined as "Upon successful
completion, read() shall return a non-negative integer indicating the
number of bytes actually read."

So, for _any_ read() to a terminal with MIN set, for this call to
return data (ie, success) MIN bytes must have been received.

(Note that the behaviour where the number of bytes > MIN seems to be
a little vague, SuS just talks about "block the calling thread until
_some_ data becomes available" for the blocking case.)

I'd be interested to know if Ted agrees with my position here; he is
the author of the tty code, and is presently looking at that area.

-- 
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

I agree that that is what that line of the text says, my position is that the 
entire statement was was written nieavely, and proveably so.  Throughout the 
entire section the standard (not the linux manual page) discusses "satisfying 
a read" (singular).  The text was written with an "everybody will know 
basically what I mean" aditude that leaves it flawed for strict 
interpretation.  And the linux manual pages still show through enough to use 
as the bases of my argument.

Pre-Summary: My entire position rests on what the original author meant by 
"receive".  That is, is it "receive into the driver" or "receive into the 
user context".  He proveably meant receive into the user context (e.g. the 
read buffer).

The best example to use for the "sloppyness" of writing in the standard and 
manual page is that technically, if it is "receive into the driver", if you 
have both VMIN and VTIME set non-zero, VTIME will not start until "a 
character is received" which, by strict intrepertation of "received (by the 
driver)", discounts the presence of any characters in the buffer.  Restated, 
if it is "receive into the driver" this read will wait for a new character 
even if there are hundreds of characters already buffered.

(CITATION: from the linux manual page: "When both are set, a read will wait 
until at least one character has been received, and then return  as  soon  as 
either  MIN  characters  have been received or time TIME has passed since the 
last character was received.")

This "clearly states" that if you have a meg in the buffer (passed-tense 
received) the driver "will  wait for a character to be received" (imparative 
future tense) but we all know that is dumb...

Even the existing implementation for VMIN and VTIME has them in the user read 
context already, which is why the round-down is only the one compound 
statement.  That clearly shows that "received" is interpreted as "received 
into the user context (read buffer)."

IF the context of "to receive", that is, the boundry between here and not-here 
is fixed, by definition, to the read call itself, the technicalities all 
disapear except one, that one being what to do if VMIN is greater than the 
receive (read) buffer size since, in this more-consistent boundry point "to 
receive" across, the more-than-buffer-sized characters can't ever then be 
received by definition.  The obvious interpretation is then not to wait for 
the unreceivable characters.

There are more examples of the "received by" being the read call, and the 
"received from" being the driver instead of the hardware.  Look at when 
non-raw processing take place.

Having only the raw-mode processing "receive" at the hardware level (with the 
buffer tossed in halfway because we all know that's what it means 8-) doesn't 
make consistent sense.  Especially if it is to preserve a conception that 
isn't ever used "in the wild".

Rob.


On Friday 26 July 2002 07:17, Russell King wrote:
> On Wed, Jun 26, 2002 at 10:48:30AM -0700, Ed Vance wrote:
> > Does the spec say that VMIN behavior depends on the size of a
> > blocking read? No, it says that the read completes when VMIN or
> > more characters have been received. If VMIN is three and two
> > characters have been received, completing a blocking read of any
> > size is a violation. Should we also add a "read buffer size"
> > parameter to select and poll, etc. so they can report that read
> > data is available before satisfying VMIN, too?
> >
> > Ted, Russell, please weigh in on this.
>
> I just found this mail again.  Yes, I agree with your interpretation,
> which reflects the code we presently have, as well as my reading of
> SuS.  The SuS is quite clear that "A pending read shall not be
> satisfied until MIN bytes are received".  It doesn't say "A pending
> read shall not be satisfied until MIN bytes or the number of requested
> bytes in read() are received"
>
> In addition, it also says "MIN represents the minimum number of bytes
> that should be received when the read() function returns successfully."
> Successful completion for read() is defined as "Upon successful
> completion, read() shall return a non-negative integer indicating the
> number of bytes actually read."
>
> So, for _any_ read() to a terminal with MIN set, for this call to
> return data (ie, success) MIN bytes must have been received.
>
> (Note that the behaviour where the number of bytes > MIN seems to be
> a little vague, SuS just talks about "block the calling thread until
> _some_ data becomes available" for the blocking case.)
>
> I'd be interested to know if Ted agrees with my position here; he is
> the author of the tty code, and is presently looking at that area.

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Russell King]

On Sat, Jul 27, 2002 at 03:07:56PM -0700, Robert White wrote:
> I agree that that is what that line of the text says, my position is that
> the entire statement was was written nieavely, and proveably so.

Ok, so lets go through your claims.

> Throughout the entire section the standard (not the linux manual page)
> discusses "satisfying a read" (singular).

The text refers you, via a hyperlink to the read() function.  Which is
singular, because the interface is singular.  I fail to see what point
you're making here.

> The text was written with an
> "everybody will know basically what I mean" aditude that leaves it flawed
> for strict interpretation.

This sentence does not allow any misinterpretation:

   A pending read shall not be satisfied until MIN bytes are received
   (that is, the pending read shall block until MIN bytes are received),
   or a signal is received.

You call a read() on a blocking file descriptor.  It blocks until MIN bytes
are received.  End of story.  If you have MIN set to 10, and you've called
read(,,1), the pending read is blocked, for 1 byte and "shall block until
MIN bytes are received".  Crystal clear.

> And the linux manual pages still show through
> enough to use as the bases of my argument.

Who cares about the Linux manual pages when discussing a standard?  The
manual pages discuss an implementation of a standard that may very well
be buggy.  They are not a standard unto themselves.

As Alan Cox just confirmed to me - "Read SuSv3 , nothing else matters".

> Pre-Summary: My entire position rests on what the original author meant by 
> "receive".  That is, is it "receive into the driver" or "receive into the 
> user context".  He proveably meant receive into the user context (e.g. the 
> read buffer).

I disagree.  Firstly, this standard isn't describing your user space
program.  It's describing the behaviour of the read() system call with
respect to blocking.  Secondly, the third sentence of this paragraph
gives some insight into the purpose of VMIN:

  A program that uses case B to read record-based terminal I/O may block
  indefinitely in the read operation.

but that's only insight, and you can't hang too much on that.

> The best example to use for the "sloppyness" of writing in the standard and 
> manual page is that technically, if it is "receive into the driver", if you 
> have both VMIN and VTIME set non-zero, VTIME will not start until "a 
> character is received" which, by strict intrepertation of "received (by the 
> driver)", discounts the presence of any characters in the buffer.  Restated, 
> if it is "receive into the driver" this read will wait for a new character 
> even if there are hundreds of characters already buffered.

The standard covers that case:

  If data is in the buffer at the time of the read(), the result shall be
  as if data has been received immediately after the read().

So pre-existing data in the driver waiting to be read is treated as if
it were received _after_ you called read().  Which is sensible.

> (CITATION: from the linux manual page: "When both are set, a read will wait 
> until at least one character has been received, and then return  as  soon  as 
> either  MIN  characters  have been received or time TIME has passed since the 
> last character was received.")

The Linux manual page is not a standard.  It is therefore meaningless to
this discussion.

> [rest cut]

-- 
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Andries Brouwer]

On Sat, Jul 27, 2002 at 03:07:56PM -0700, Robert White wrote:

> I agree that that is what that line of the text says, my position is that the 
> entire statement was was written nieavely, and proveably so.  Throughout the 
> entire section the standard (not the linux manual page) discusses "satisfying 
> a read" (singular).  The text was written with an "everybody will know 
> basically what I mean" aditude that leaves it flawed for strict 
> interpretation.  And the linux manual pages still show through enough to use 
> as the bases of my argument.

I followed this discussion with half an eye, and do not really want
to spend the time figuring out what my point of view would be.

But.

In such discussions the Linux man pages carry hardly any weight.
Of interest are the original man pages of the system where the
feature was introduced. And of interest is the original implementation.
And of interest is the POSIX standard.

Sometimes the POSIX author misunderstands something and writes some
silly text into the standard. There are technical committees that
one can approach and try to get a correction. Until such a time,
one should read the POSIX text as written, and not as intended.

Generally, Linux follows POSIX. That is good: since everybody else
also does that, we can exchange programs. Everybody the same silliness
has advantages over each his own correct solution.

Sometimes, when following POSIX is too silly or too painful, Linux
chooses its own way. But in such cases Linux does not follow the Linux
man pages, it is just the other way around. So, I am afraid citing
the Linux man pages will never give you a powerful argument.

Andries
aeb@cwi.nl

[yes, I maintain the man pages; of course they are almost perfect,
since few people have corrections; but corrections are always welcome]

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

I don't have access to the truely official posix standards document (posix.org 
want's a login and ieee want's dues/subscriptions).  The manual page citation 
is common to linux and sun and was accessable to all readers, between the two 
of them, so I cited it.  Besides, I am citing back the same source as the 
quotes I was getting cited at me... 8-)

Blocking a system call for something that can't be used by the call is dumb.

Repeatedly changing the value of VMIN before each read (doubling your system 
calls/context switches) is dumb.

Having virtually every user on the planet realize this and just set VMIN == 1 
is an fairly telling indicator.

Repeatedly calling the kernel to assemble an input buffer which is necessary 
if VMIN ==1, is dumb.

Having an upper limit of 255 on the number of characters you can wait for in a 
single read, now that communications speeds are usefully large, is dumb.

The lay-programmers I have shown this too so far (a small and unprovable set, 
I know 8-) think blocking for the extra characters is an error.  This is the 
tried-and-true "rule of least astonishment".  It is "astonishing" to make a 
call read(fd,buffer,N), when you know N characters are already buffered, and 
not have it return because VMIN is bigger than N.

The fix for rounding down is one line and breaks nothing on the planet that 
anybody can find (mostly because of the default VMIN==1 everybody uses).

The rounding up part (adds one condition to the afore mentioned and one other 
conditional assignment) is more arguable but still quite a win since there 
isn't a modern serial protocol existant that uses a prefered maximum buffer 
size of less that 256 characters.

So far all I am getting back is one non-public "that's a good idea, but I want 
to see what everybody else thinks" and a bunch of "but the standard says"...

The standard is OLD and UNREALISTIC (and if you get all lawyer-ish, somewhat 
inconsitant and vague.) and the modification is one hundred percent 
compatable with what is in the field and what "more than half" of the users 
expect.

Rob.

On Saturday 27 July 2002 16:21, Andries Brouwer wrote:
> On Sat, Jul 27, 2002 at 03:07:56PM -0700, Robert White wrote:
> > I agree that that is what that line of the text says, my position is that
> > the entire statement was was written nieavely, and proveably so. 
> > Throughout the entire section the standard (not the linux manual page)
> > discusses "satisfying a read" (singular).  The text was written with an
> > "everybody will know basically what I mean" aditude that leaves it flawed
> > for strict
> > interpretation.  And the linux manual pages still show through enough to
> > use as the bases of my argument.
>
> I followed this discussion with half an eye, and do not really want
> to spend the time figuring out what my point of view would be.
>
> But.
>
> In such discussions the Linux man pages carry hardly any weight.
> Of interest are the original man pages of the system where the
> feature was introduced. And of interest is the original implementation.
> And of interest is the POSIX standard.
>
> Sometimes the POSIX author misunderstands something and writes some
> silly text into the standard. There are technical committees that
> one can approach and try to get a correction. Until such a time,
> one should read the POSIX text as written, and not as intended.
>
> Generally, Linux follows POSIX. That is good: since everybody else
> also does that, we can exchange programs. Everybody the same silliness
> has advantages over each his own correct solution.
>
> Sometimes, when following POSIX is too silly or too painful, Linux
> chooses its own way. But in such cases Linux does not follow the Linux
> man pages, it is just the other way around. So, I am afraid citing
> the Linux man pages will never give you a powerful argument.
>
> Andries
> aeb@cwi.nl
>
> [yes, I maintain the man pages; of course they are almost perfect,
> since few people have corrections; but corrections are always welcome]

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Stevie O]

At 07:34 PM 7/27/2002 -0700, Robert White wrote:
>So far all I am getting back is one non-public "that's a good idea, but I want 
>to see what everybody else thinks" and a bunch of "but the standard says"...
>
>The standard is OLD and UNREALISTIC (and if you get all lawyer-ish, somewhat 
>inconsitant and vague.) and the modification is one hundred percent 
>compatable with what is in the field and what "more than half" of the users 
>expect.


But... but... the standard says...

   A pending read shall not be satisfied until MIN bytes are received
   (that is, the pending read shall block until MIN bytes are received),
   or a signal is received.

And because I'm too dead-set on doing it that way solely because that's how it's always been done, I won't ever consider changing it.  I'll blindly ignore how much xmodem transfers suck, and the fact that I can come up with no practical purpose at all for this feature, and just repeat what the standard says.  Why should we obey what Linux man pages say? What do the Linux man pages have to do with Linux?

Remember: Computers and their programs aren't here to make our lives easier, or to make tasks simpler. They are here to follow standards.


--
Stevie-O

Real programmers use COPY CON PROGRAM.EXE

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Andries Brouwer]

On Sat, Jul 27, 2002 at 07:34:27PM -0700, Robert White wrote:

> I don't have access to the truely official posix standards document

Then before you continue, you should perhaps get hold of some
such text.

SUSv3 is visible at
http://www.UNIX-systems.org/version3/

POSIX 1003.1-2001 is visible at
http://www.opengroup.org/onlinepubs/007904975/toc.htm

The page of interest for you is perhaps
http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap11.html

Andries

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Alan Cox]

On Sun, 2002-07-28 at 03:34, Robert White wrote:
> Having virtually every user on the planet realize this and just set VMIN == 1 
> is an fairly telling indicator.
> 
> Repeatedly calling the kernel to assemble an input buffer which is necessary 
> if VMIN ==1, is dumb.

VMIN was basically invented for communication protocols when you know
the block length that should arrive within a given timeout. Its pretty
much essential on old old boxes and was very important for
interrupt/context switch reduction when doing block transfers. In that
world the read blocks or in O_NDELAY returns -EAGAIN (0 in old SYS5)
until the data block is big enough to warrant its copying. Similarly
poll has no business saying data is ready until a large enough block is.

When talking to a human setting VMIN > 1 makes no sense anyway. In fact
nowdays it makes even less sense than it did before because of the use
of UTF8 encodings for unicode characters.

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

Um... "no duh"  (I have been programming Unix boxen for twenty years)

In point of fact I specifically AND repeatedly stated that there would 
probably be no change to human-interactive applications.

On the other hand there are a wide number of applications that do have need 
for variable packet lengths, VMIN/VTIME style timeouts and known, but 
variable sized or larger than 256 byte packet sizes.

The entire "smart card" arena is one such set of applications.  Also test 
gear, and simple X/Y/Z modem type transfers.

If it talks at serial speeds, if it has any kind of TIMEOUT feature and/or 
if it doesn't makes sense to strobe it out with no-delay reads (e.g. 
perhaps because there isn't any background-worthy in-application task 
n'cest pa?)  Then having the VMIN and VTIME behaviors, but not having them 
block for un-asked for characters or unconscionably short packets makes 
lots of sense.

As I said, repeatedly calling the kernel TO ASSEMBLE AN INPUT BUFFER if 
VMIN == 1 IS DUMB.

Reading out single (raw) keystrokes isn't "ASSEMBLING AN INPUT BUFFER", it 
is INTERACTING WITH A USER.  [For the audience that is reading along: (in 
cooked mode the buffer assembly happens in... gasp... one read call...) (in 
an editor, for example, or a similar application where VMIN==1 us used, the 
task of aggregating the input into storage is not so straight forward as 
assembling an input buffer as the nature of each keystroke may impact the 
larger state of the program)

When reading data from a non-human source the predictability goes up and 
the buffered data can be more efficiently blocked out by known predicates, 
etc.  This is the audience for the patch.

Rob.


At 21:04 7/28/2002 +0100, Alan Cox wrote:
>On Sun, 2002-07-28 at 03:34, Robert White wrote:
> > Having virtually every user on the planet realize this and just set 
> VMIN == 1
> > is an fairly telling indicator.
> >
> > Repeatedly calling the kernel to assemble an input buffer which is 
> necessary
> > if VMIN ==1, is dumb.
>
>VMIN was basically invented for communication protocols when you know
>the block length that should arrive within a given timeout. Its pretty
>much essential on old old boxes and was very important for
>interrupt/context switch reduction when doing block transfers. In that
>world the read blocks or in O_NDELAY returns -EAGAIN (0 in old SYS5)
>until the data block is big enough to warrant its copying. Similarly
>poll has no business saying data is ready until a large enough block is.
>
>When talking to a human setting VMIN > 1 makes no sense anyway. In fact
>nowdays it makes even less sense than it did before because of the use
>of UTF8 encodings for unicode characters.

----------------------------------------------------------------------
-- Rob White         --  Consider: for all vectors v, Cv - Cv = 0   --
-- rwhite@pobox.com  --    general relativity is thus disproved.    --
----------------------------------------------------------------------
-- There once was a man who claimed nothing was true,               --
--   he was later, of course, found to be lying.                    --
----------------------------------------------------------------------
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

mQGiBDsFXZsRBAD3vGQFZV1Noe8WeKVQwZ+0GK7Z1PMkD3Tu++cZuuxKyqrdHnd9
ayXefnrtjvWOanYk04LCw4MNV7jDYUefD/WtBf42NXhXcPUgBw+D+AhlSSPzcaZI
1xOH/hZABu2f8fOf2LeB3lIEvxZk3lEbNop8ssAO28nU76cPBJe947IpDQCg/9q6
0hxapeW7nwkW511jiYH64+kEAMSBhNkesVnh6ZhMBQIrERF+PkZBE6LWBfZzUned
DrJjbFMkt7QGMlqqZcKUQWLLjvVnD2a6WJ9W7sif82Y2svyz0d2dEW3BK9EuTMdV
W25XfIKk19gMn6mBE0LoOESlsHNhU4HRmUO0mhjhF861mVSRq71Oxzp3xth9LKGz
yjwWBACP/pTS7qVaJHxvHRPfQKzZeBQObCHUjhbelQ4qx2vNkMgnb3khjzyFd7Tf
5CmMLtXn9GyDv9osY2f9HGw+7Kypg/eumFy/kgdVq2bUIkBfM2eD1CnruKzJ0wYv
6ppxQG1U/g9MY9q3pUsfcn2tXlvvMuKoMPlVwIaKjZPcLNjImLQfUm9iZXJ0IFdo
aXRlIDxyd2hpdGVAcG9ib3guY29tPokAWAQQEQIAGAUCOwVdmwgLAwkIBwIBCgIZ
AQUbAwAAAAAKCRC1VByAmCYfhULxAJ918uUYlrNwB6XRlj0fQCmmoRC39ACeKDfr
k5Cf5SF/NYsE91/tK6TTWW+5BA0EOwVdnBAQAPkYoH5aBmF6Q5CV3AVsh4bsYezN
RR8O2OCjecbJ3HoLrOQ/40aUtjBKU9d8AhZIgLUV5SmZqZ8HdNP/46HFliBOmGW4
2A3uEF2rthccUdhQyiJXQym+lehWKzh4XAvb+ExN1eOqRsz7zhfoKp0UYeOEqU/R
g4Soebbvj6dDRgjGzB13VyQ4SuLE8OiOE2eXTpITYfbb6yUOF/32mPfIfHmwch04
dfv2wXPEgxEmK0Ngw+Po1gr9oSgmC66prrNlD6IAUwGgfNaroxIe+g8qzh90hE/K
8xfzpEDp19J3tkItAjbBJstoXp18mAkKjX4t7eRdefXUkk+bGI78KqdLfDL2Qle3
CH8IF3KiutapQvMF6PlTETlPtvFuuUs4INoBp1ajFOmPQFXz0AfGy0OplK33TGSG
SfgMg71l6RfUodNQ+PVZX9x2Uk89PY3bzpnhV5JZzf24rnRPxfx2vIPFRzBhznzJ
Zv8V+bv9kV7HAarTW56NoKVyOtQa8L9GAFgr5fSI/VhOSdvNILSd5JEHNmszbDgN
RR0PfIizHHxbLY7288kjwEPwpVsYjY67VYy4XTjTNP18F1dDox0YbN4zISy1Kv88
4bEpQBgRjXyEpwpy1obEAxnIByl6ypUM2Zafq9AKUJsCRtMIPWakXUGfnHy9iUsi
GSa6q6Jew1XrPdYXAAICD/9YtZC+8OMCshWnlD1LtdAjl8i/E8nsJ5oDlfNl447k
roxLvjLf5WAqUsw5ym8iqAdVDwT5o0OMcTc9zLgM3CFO70oTpE+Rzw6Va5fEJpYQ
1+525rd1ORbVWzDQmOfvo0jC0Z6eMxKuwvKrPdCPaC7gd3FWltPMOO+GX6i0yLPx
0rCsQCsGcWZyAf4Epg/4W0O/I8IGHJOXBddhexB865WU7HbXXIsMiJzShE+y23Vw
sRwTEcmQIM+3fxKnA8/ou2WAKx/SHx9actQevimWL3tV8dTTdsIbu4xfrWSRPUht
7Hgq5OhJXRP1tjZ/gE57MnsqT5AzwJ2m5l5yyx3YWR5znHe8exUFTK0PekkO0gPl
T7BwWn86itLk+Ozd3Y30Y7buWtUiEEeZZpYH2BL54D46DxMMoVwxc22y7sY+GVxB
ADw+fvVoeV5na7pXxIGfzOMZMJikG4fYEVsfsYz8WJUyWR2qIEoRJnTWYTWIoNGn
FYNr6y9HXSMYQF9XIRtRKeo3OaYVQia+NqvyyLGeuM9fYqDkaBU4Gh7bfttLswZ1
fNZvj+2GfHJTxMl9F6TA5S+2OEFIQpX+aGWYdQdrr0mO7EJR1jOi1AYvqtSkLh0y
2pe1r2RiZMqu9PYccbYygE3RTZdxeNKO+x3mdRg0gbpTIFNO0MacvMvzpYMDqEtF
fYkATAQYEQIADAUCOwVdnAUbDAAAAAAKCRC1VByAmCYfhU4oAJ9fftoWL+V1DkxQ
X+SfvAHwOvdhqwCg2TVU2ss5LYnxyLfarAUs0cPlYSc=
=Zx5A
-----END PGP PUBLIC KEY BLOCK-----

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Alan Cox]

On Sun, 2002-07-28 at 03:34, Robert White wrote:
> Having virtually every user on the planet realize this and just set VMIN == 1 
> is an fairly telling indicator.
> 
> Repeatedly calling the kernel to assemble an input buffer which is necessary 
> if VMIN ==1, is dumb.

VMIN was basically invented for communication protocols when you know
the block length that should arrive within a given timeout. Its pretty
much essential on old old boxes and was very important for
interrupt/context switch reduction when doing block transfers. In that
world the read blocks or in O_NDELAY returns -EAGAIN (0 in old SYS5)
until the data block is big enough to warrant its copying. Similarly
poll has no business saying data is ready until a large enough block is.

When talking to a human setting VMIN > 1 makes no sense anyway. In fact
nowdays it makes even less sense than it did before because of the use
of UTF8 encodings for unicode characters.

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

And if you could show me ONE example of wild code that would be harmed by the 
mod I would shut up and go away.

(I say wild code, but even contrived code would be interesting.)

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Ed Vance]

Hi David,

On Tue, July 30, 2002 at 10:32 PM, David Lawyer wrote:
> On Tue, Jul 30, 2002 at 10:07:51AM -0700, Ed Vance wrote:
> > On Sat, July 27, 2002 at 8:02 PM, stevie@qrpff.net wrote:
> > > 
> > > But... but... the standard says...
> > > 
> > >    A pending read shall not be satisfied until MIN bytes are 
> > >    received (that is, the pending read shall block until MIN 
> > >    bytes are received), or a signal is received.
> > > 
> > > And because I'm too dead-set on doing it that way solely because 
> > > that's how it's always been done, I won't ever consider changing it. 

 [ snip ] 

> > Stevie O gets to the central issue here. Why _not_ change long-existing,
> > widely used interfaces in subtle ways because the old way makes no 
> > sense to us and the new way does? Is standards-based programming a 
> > lemming behavior? 

 [ snip ]

> > As I said before, innovation is fine. Just don't pollute the existing
> > interfaces. If even one real customer running real work has bad day 
> > because of exposure of an old app bug or an unanticipated consequence 
> > of the change, then it wasn't worth ignoring safer implementation 
> > practices. One can't attain 100% safety, but one _can_ minimize the 
> > risk. 
> 
> I think it's very important for Linux to be efficient so that it will
> work well on old hardware, etc.  

I agree 100% so far.

>                                 What's being proposed seems to help.
> So I think that if there is a low probability that it will break any
> existing application where non-trivial harm will be caused, then I would
> favor innovation (actually common sense in this case).  

I don't agree that we have a trade off, here. (big surprise? :) We are
simply asking the wrong question in the quest to achieve the desired
benefit. We are asking how to deviate from the standard to get better
performance. We should be asking how to change the kernel to get better
performance without changing the standard app interface. Of course, making
the standard (but rather uninspired) app interface work efficiently is a
_lot_ more technically challenging. 

>                                                        Also someone
> should try to get the standards modified to clearly allow what's
> proposed.

Good luck with that. ;)

BTW, I am hypersensitive to this issue because I got bit by Mot V/88 that
had a buggy MIN and TIME implementation. My phone ran flames for about a
month because of my initial decision to match their quirky interface
behavior in my serial card's driver. 

Best regards,
Ed

---------------------------------------------------------------- 
Ed Vance              edv@macrolink.com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Ed Vance]

Hi David,

On Tue, July 30, 2002 at 10:32 PM, David Lawyer wrote:
> On Tue, Jul 30, 2002 at 10:07:51AM -0700, Ed Vance wrote:
> > On Sat, July 27, 2002 at 8:02 PM, stevie@qrpff.net wrote:
> > > 
> > > But... but... the standard says...
> > > 
> > >    A pending read shall not be satisfied until MIN bytes are 
> > >    received (that is, the pending read shall block until MIN 
> > >    bytes are received), or a signal is received.
> > > 
> > > And because I'm too dead-set on doing it that way solely because 
> > > that's how it's always been done, I won't ever consider changing it. 

 [ snip ] 

> > Stevie O gets to the central issue here. Why _not_ change long-existing,
> > widely used interfaces in subtle ways because the old way makes no 
> > sense to us and the new way does? Is standards-based programming a 
> > lemming behavior? 

 [ snip ]

> > As I said before, innovation is fine. Just don't pollute the existing
> > interfaces. If even one real customer running real work has bad day 
> > because of exposure of an old app bug or an unanticipated consequence 
> > of the change, then it wasn't worth ignoring safer implementation 
> > practices. One can't attain 100% safety, but one _can_ minimize the 
> > risk. 
> 
> I think it's very important for Linux to be efficient so that it will
> work well on old hardware, etc.  

I agree 100% so far.

>                                 What's being proposed seems to help.
> So I think that if there is a low probability that it will break any
> existing application where non-trivial harm will be caused, then I would
> favor innovation (actually common sense in this case).  

I don't agree that we have a trade off, here. (big surprise? :) We are
simply asking the wrong question in the quest to achieve the desired
benefit. We are asking how to deviate from the standard to get better
performance. We should be asking how to change the kernel to get better
performance without changing the standard app interface. Of course, making
the standard (but rather uninspired) app interface work efficiently is a
_lot_ more technically challenging. 

>                                                        Also someone
> should try to get the standards modified to clearly allow what's
> proposed.

Good luck with that. ;)

BTW, I am hypersensitive to this issue because I got bit by Mot V/88 that
had a buggy MIN and TIME implementation. My phone ran flames for about a
month because of my initial decision to match their quirky interface
behavior in my serial card's driver. 

Best regards,
Ed

---------------------------------------------------------------- 
Ed Vance              edv@macrolink.com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Ed Vance]

On Sat, July 27, 2002 at 8:02 PM, stevie@qrpff.net wrote:
> 
> But... but... the standard says...
> 
>    A pending read shall not be satisfied until MIN bytes are 
>    received (that is, the pending read shall block until MIN 
>    bytes are received), or a signal is received.
> 
> And because I'm too dead-set on doing it that way solely because 
> that's how it's always been done, I won't ever consider changing it.  
> I'll blindly ignore how much xmodem transfers suck, and the fact 
> that I can come up with no practical purpose at all for this feature, 
> and just repeat what the standard says.  Why should we obey what 
> Linux man pages say? What do the Linux man pages have to do with 
> Linux?
> 
> Remember: Computers and their programs aren't here to make our lives 
> easier, or to make tasks simpler. They are here to follow standards.

Hi Rob, 

Stevie O gets to the central issue here. Why _not_ change long-existing,
widely used interfaces in subtle ways because the old way makes no sense to
us and the new way does? Is standards-based programming a lemming behavior? 

My answer is that but-I-think-it-would-be-better, alone, is not a sufficient
reason to risk exposure of old application bugs (if you don't actually have
to) and to bring down apps that ran just fine with the bugs for years. In
this case, the proposed new functionality is triggered by use of interface
space that already had a specified behavior. 

When new functionality is added, at a minimum, its interface should be
outside of the previous valid use set. If one simply must attach the
tendrils of cleverness to the vines of an existing interface, the new
functionality should only appear upon app behaviors that would previously
have been invalid enough to reject the request and burp up an error code. 

As I said before, innovation is fine. Just don't pollute the existing
interfaces. If even one real customer running real work has bad day because
of exposure of an old app bug or an unanticipated consequence of the change,
then it wasn't worth ignoring safer implementation practices. One can't
attain 100% safety, but one _can_ minimize the risk. 

Best regards,
Ed 

---------------------------------------------------------------- 
Ed Vance              edv@macrolink.com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Ed Vance]

On Sat, July 27, 2002 at 8:02 PM, stevie@qrpff.net wrote:
> 
> But... but... the standard says...
> 
>    A pending read shall not be satisfied until MIN bytes are 
>    received (that is, the pending read shall block until MIN 
>    bytes are received), or a signal is received.
> 
> And because I'm too dead-set on doing it that way solely because 
> that's how it's always been done, I won't ever consider changing it.  
> I'll blindly ignore how much xmodem transfers suck, and the fact 
> that I can come up with no practical purpose at all for this feature, 
> and just repeat what the standard says.  Why should we obey what 
> Linux man pages say? What do the Linux man pages have to do with 
> Linux?
> 
> Remember: Computers and their programs aren't here to make our lives 
> easier, or to make tasks simpler. They are here to follow standards.

Hi Rob, 

Stevie O gets to the central issue here. Why _not_ change long-existing,
widely used interfaces in subtle ways because the old way makes no sense to
us and the new way does? Is standards-based programming a lemming behavior? 

My answer is that but-I-think-it-would-be-better, alone, is not a sufficient
reason to risk exposure of old application bugs (if you don't actually have
to) and to bring down apps that ran just fine with the bugs for years. In
this case, the proposed new functionality is triggered by use of interface
space that already had a specified behavior. 

When new functionality is added, at a minimum, its interface should be
outside of the previous valid use set. If one simply must attach the
tendrils of cleverness to the vines of an existing interface, the new
functionality should only appear upon app behaviors that would previously
have been invalid enough to reject the request and burp up an error code. 

As I said before, innovation is fine. Just don't pollute the existing
interfaces. If even one real customer running real work has bad day because
of exposure of an old app bug or an unanticipated consequence of the change,
then it wasn't worth ignoring safer implementation practices. One can't
attain 100% safety, but one _can_ minimize the risk. 

Best regards,
Ed 

---------------------------------------------------------------- 
Ed Vance              edv@macrolink.com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[David Lawyer]

On Tue, Jul 30, 2002 at 10:07:51AM -0700, Ed Vance wrote:
> On Sat, July 27, 2002 at 8:02 PM, stevie@qrpff.net wrote:
> > 
> > But... but... the standard says...
> > 
> >    A pending read shall not be satisfied until MIN bytes are 
> >    received (that is, the pending read shall block until MIN 
> >    bytes are received), or a signal is received.
> > 
> > And because I'm too dead-set on doing it that way solely because 
> > that's how it's always been done, I won't ever consider changing it.  
> > I'll blindly ignore how much xmodem transfers suck, and the fact 
> > that I can come up with no practical purpose at all for this feature, 
> > and just repeat what the standard says.  Why should we obey what 
> > Linux man pages say? What do the Linux man pages have to do with 
> > Linux?
> > 
> > Remember: Computers and their programs aren't here to make our lives 
> > easier, or to make tasks simpler. They are here to follow standards.
> 
> Hi Rob, 
> 
> Stevie O gets to the central issue here. Why _not_ change long-existing,
> widely used interfaces in subtle ways because the old way makes no sense to
> us and the new way does? Is standards-based programming a lemming behavior? 
> 
> My answer is that but-I-think-it-would-be-better, alone, is not a sufficient
> reason to risk exposure of old application bugs (if you don't actually have
> to) and to bring down apps that ran just fine with the bugs for years. In
> this case, the proposed new functionality is triggered by use of interface
> space that already had a specified behavior. 
> 
> When new functionality is added, at a minimum, its interface should be
> outside of the previous valid use set. If one simply must attach the
> tendrils of cleverness to the vines of an existing interface, the new
> functionality should only appear upon app behaviors that would previously
> have been invalid enough to reject the request and burp up an error code. 
> 
> As I said before, innovation is fine. Just don't pollute the existing
> interfaces. If even one real customer running real work has bad day because
> of exposure of an old app bug or an unanticipated consequence of the change,
> then it wasn't worth ignoring safer implementation practices. One can't
> attain 100% safety, but one _can_ minimize the risk. 

I think it's very important for Linux to be efficient so that it will
work well on old hardware, etc.  What's being proposed seems to help.
So I think that if there is a low probability that it will break any
existing application where non-trivial harm will be caused, then I would
favor innovation (actually common sense in this case).  Also someone
should try to get the standards modified to clearly allow what's
proposed.

			David Lawyer (Maintainer of Serial-HOWTO, but I
			           try to avoid Serial-Programming :-)

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Ed Vance]

On Sat, July 27, 2002 at 7:36 PM, Robert White wrote:
> 
> And if you could show me ONE example of wild code that would be 
> harmed by the mod I would shut up and go away.
> 
> (I say wild code, but even contrived code would be interesting.)

>From the archive ...
http://marc.theaimsgroup.com/?l=linux-kernel&m=102440556723250&w=2

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

Yep, this is an example of "precaching" and was the example I said I have 
seen.  Since there is no USE of the precached data during the read, the 
there wasn't any demonstrable purpose to the action.

That is, this is NOT CODE "IN THE WILD", it is the contrived example of 
use, it doesn't meet the requested requirement that the example be "harmed" 
by the patch as there is no actual algorithm attached.

Any code that did this would then read this data out of the buffer by 
another request after resetting VMIN which *IS* a dependence on "magic 
state" since the current read is blocking on state that cannot be returned 
to the user.

Note that the author of the linked message says "having said that, I don't 
know of any instances where the above is employed." and then agrees the 
patch is worthwhile...

So it doesn't fulfill my request/requirement, the cited author explicitly 
supports my position, and there isn't any implication of how the rest of 
the supposed program might use this magic state...

hmmm....

Not the best argument Ed.

Rob.

At 14:46 7/29/2002 -0700, Ed Vance wrote:
>On Sat, July 27, 2002 at 7:36 PM, Robert White wrote:
> >
> > And if you could show me ONE example of wild code that would be
> > harmed by the mod I would shut up and go away.
> >
> > (I say wild code, but even contrived code would be interesting.)
>
> From the archive ...
>http://marc.theaimsgroup.com/?l=linux-kernel&m=102440556723250&w=2

----------------------------------------------------------------------
-- Rob White         --  Consider: for all vectors v, Cv - Cv = 0   --
-- rwhite@pobox.com  --    general relativity is thus disproved.    --
----------------------------------------------------------------------
-- There once was a man who claimed nothing was true,               --
--   he was later, of course, found to be lying.                    --
----------------------------------------------------------------------
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

mQGiBDsFXZsRBAD3vGQFZV1Noe8WeKVQwZ+0GK7Z1PMkD3Tu++cZuuxKyqrdHnd9
ayXefnrtjvWOanYk04LCw4MNV7jDYUefD/WtBf42NXhXcPUgBw+D+AhlSSPzcaZI
1xOH/hZABu2f8fOf2LeB3lIEvxZk3lEbNop8ssAO28nU76cPBJe947IpDQCg/9q6
0hxapeW7nwkW511jiYH64+kEAMSBhNkesVnh6ZhMBQIrERF+PkZBE6LWBfZzUned
DrJjbFMkt7QGMlqqZcKUQWLLjvVnD2a6WJ9W7sif82Y2svyz0d2dEW3BK9EuTMdV
W25XfIKk19gMn6mBE0LoOESlsHNhU4HRmUO0mhjhF861mVSRq71Oxzp3xth9LKGz
yjwWBACP/pTS7qVaJHxvHRPfQKzZeBQObCHUjhbelQ4qx2vNkMgnb3khjzyFd7Tf
5CmMLtXn9GyDv9osY2f9HGw+7Kypg/eumFy/kgdVq2bUIkBfM2eD1CnruKzJ0wYv
6ppxQG1U/g9MY9q3pUsfcn2tXlvvMuKoMPlVwIaKjZPcLNjImLQfUm9iZXJ0IFdo
aXRlIDxyd2hpdGVAcG9ib3guY29tPokAWAQQEQIAGAUCOwVdmwgLAwkIBwIBCgIZ
AQUbAwAAAAAKCRC1VByAmCYfhULxAJ918uUYlrNwB6XRlj0fQCmmoRC39ACeKDfr
k5Cf5SF/NYsE91/tK6TTWW+5BA0EOwVdnBAQAPkYoH5aBmF6Q5CV3AVsh4bsYezN
RR8O2OCjecbJ3HoLrOQ/40aUtjBKU9d8AhZIgLUV5SmZqZ8HdNP/46HFliBOmGW4
2A3uEF2rthccUdhQyiJXQym+lehWKzh4XAvb+ExN1eOqRsz7zhfoKp0UYeOEqU/R
g4Soebbvj6dDRgjGzB13VyQ4SuLE8OiOE2eXTpITYfbb6yUOF/32mPfIfHmwch04
dfv2wXPEgxEmK0Ngw+Po1gr9oSgmC66prrNlD6IAUwGgfNaroxIe+g8qzh90hE/K
8xfzpEDp19J3tkItAjbBJstoXp18mAkKjX4t7eRdefXUkk+bGI78KqdLfDL2Qle3
CH8IF3KiutapQvMF6PlTETlPtvFuuUs4INoBp1ajFOmPQFXz0AfGy0OplK33TGSG
SfgMg71l6RfUodNQ+PVZX9x2Uk89PY3bzpnhV5JZzf24rnRPxfx2vIPFRzBhznzJ
Zv8V+bv9kV7HAarTW56NoKVyOtQa8L9GAFgr5fSI/VhOSdvNILSd5JEHNmszbDgN
RR0PfIizHHxbLY7288kjwEPwpVsYjY67VYy4XTjTNP18F1dDox0YbN4zISy1Kv88
4bEpQBgRjXyEpwpy1obEAxnIByl6ypUM2Zafq9AKUJsCRtMIPWakXUGfnHy9iUsi
GSa6q6Jew1XrPdYXAAICD/9YtZC+8OMCshWnlD1LtdAjl8i/E8nsJ5oDlfNl447k
roxLvjLf5WAqUsw5ym8iqAdVDwT5o0OMcTc9zLgM3CFO70oTpE+Rzw6Va5fEJpYQ
1+525rd1ORbVWzDQmOfvo0jC0Z6eMxKuwvKrPdCPaC7gd3FWltPMOO+GX6i0yLPx
0rCsQCsGcWZyAf4Epg/4W0O/I8IGHJOXBddhexB865WU7HbXXIsMiJzShE+y23Vw
sRwTEcmQIM+3fxKnA8/ou2WAKx/SHx9actQevimWL3tV8dTTdsIbu4xfrWSRPUht
7Hgq5OhJXRP1tjZ/gE57MnsqT5AzwJ2m5l5yyx3YWR5znHe8exUFTK0PekkO0gPl
T7BwWn86itLk+Ozd3Y30Y7buWtUiEEeZZpYH2BL54D46DxMMoVwxc22y7sY+GVxB
ADw+fvVoeV5na7pXxIGfzOMZMJikG4fYEVsfsYz8WJUyWR2qIEoRJnTWYTWIoNGn
FYNr6y9HXSMYQF9XIRtRKeo3OaYVQia+NqvyyLGeuM9fYqDkaBU4Gh7bfttLswZ1
fNZvj+2GfHJTxMl9F6TA5S+2OEFIQpX+aGWYdQdrr0mO7EJR1jOi1AYvqtSkLh0y
2pe1r2RiZMqu9PYccbYygE3RTZdxeNKO+x3mdRg0gbpTIFNO0MacvMvzpYMDqEtF
fYkATAQYEQIADAUCOwVdnAUbDAAAAAAKCRC1VByAmCYfhU4oAJ9fftoWL+V1DkxQ
X+SfvAHwOvdhqwCg2TVU2ss5LYnxyLfarAUs0cPlYSc=
=Zx5A
-----END PGP PUBLIC KEY BLOCK-----

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Ed Vance]

Hi Rob,

On Thu, June 27, 2002 at 9:37 AM, Robert White wrote:
> 
> Actually you are wrong about the "Standard".  The Linux manual 
> pages only say the read will return when so many characters 
> have been received.  Alternately, the System V Interface 
> Definition (in every version I have been able to find on the 
> net) speaks in terms of "satisfying the read" .  Both are 
> silent on the issue of what to do if the read is asking for 
> less than VMIN characters.

The Linux termios man page says rather clearly that "If only MIN 
is set, the read will not return before MIN characters have been 
received". Your change would create cases where the read _will_ 
return before MIN characters have been received. That is a conflict. 

> That is, since the VMIN/VTIME mechanism exists only with respect 
> to each individual read request and not with respect to any 
> kind of driver state, there is no inference or onus that says 
> the state applies to anything other than the receipt of 
> characters from the driver to the read itself.
> 
> That is, the context for the word "received" MUST BE "received 
> into the read buffer."  [or perhaps "received into the user 
> context"].  How so, you may ask?
> 
> If "received" is in respect to the UART (e.g. the machine/
> hardware boundary) then it would be true that if there were 100 
> characters in the driver's internal buffer, VMIN were 1, VTIME 
> were 1, and a read were issued but no more characters were 
> "received" by the UART then the read would never return.  
> That is, by the standard, "one character must be received to 
> start the timer, and one character must perceived minimum" so 
> if "received" means "into the driver" instead of "into the user 
> context" the 100 characters in the buffer, having been received 
> in the past would not relate the read (and so on ad nauseum 8-)   
> That would be ridiculous.

Termio, etc. specify the API, not the kernel's implementation of the 
API. All such driver software implements at least two levels of 
buffering to deal with the raw vs. canonical mode. The received data 
does not go directly to the user buffer. The data moves into the user 
buffer only when the read completes. Usually, the data received by 
the UART is handed up to a "line discipline" layer. In Streams 
implementations, there are (at least) three levels of buffering below 
the user's buffer: driver raw queue, line discipline private line 
assembly queue, and the stream head's queue. 

> Since in every stated and useful sense the VMIN and VTIME 
> mechanisms exist specifically and expressly for the purpose of 
> satisfying individual read requests, that is in terms of the 
> user-context/program "receiving" data from the abstract "device" 
> at the file interface, data that can not possibly relate to a 
> read because it is "out of bounds"  is immaterial to the act by 
> definition.  So waiting for the user context to receive 
> characters it has not asked for and cannot possibly receive is 
> a "bad thing".

It operates as specified and does what you tell it to through the API. 
That is neither good nor bad, per se. That's just compliance. If you 
tell it to do something that you don't want, that's not a driver bug. 

> The "waiting for data that can't be returned" and/or the 
> inconsistent application of the concept of "received" serves 
> nobody's interests.

Please make your good changes in a way that does not conflict with the 
existing API. Ioctl is your friend. 

> 
> I was going to simply dismiss that bit of hyperbole about 
> poll as a cheap bit of bad "debate technique" but I've decided 
> to respond.

Surprise. 

> Poll already and explicitly addresses the size of the read and/
> or write as one character, so tossing it in as if it were an 
> example was logically junct.  Additionally, to make the example 
> work you "threw in" with a comment about "adding an argument to 
> poll."  The reason that this was fatuous is that the parameter 
> you would "add"  to poll would be disruptive (which is why you 
> put it in 8-) but the information is already provided in the 
> read, the buffer length IS ALREADY being passed.  The analogy 
> and the allegation of difficulty or disruption is inapplicable.

That's not how poll and select are used. For the read case, Poll and 
Select are active when waiting for the availability of data. When data 
becomes available, they return and you post a read to scoop up the new 
data. There is no pending read during poll and select to supply a read 
buffer length to the driver.

I have no problem with innovation. Just don't pollute the existing API. 

Regards,
Ed

----------------------------------------------------------------
Ed Vance              edv@macrolink.com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

RE: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Ed Vance]

On Sat, June 15, 2002 at 9:01 PM, Robert White wrote:
> Kernel Versions: 2.2, 2.4, 2.5 (all of them since 1996 really 8-)
> 
> The n_tty line discipline module contains a "semantic error" that 
> limits its speed and usefulness in many uses.  The attached patch 
> directly addresses serial performance in a completely backwards-
> compatible way.
> 
> In particular, the current handling of VMIN hugely limits, 
> complicates, and/or slows down optimal serial use.  The most 
> obvious example is that if you call read(2) with a buffer size less 
> than the current value of VMIN, the line discipline will insist 
> that the read call wait for characters that can not be returned to 
> that call.  The POSIX standard is silent on the subject of whether 
> this is right or wrong.  Common sense says it is wrong.

Hi,

IIRC, the way VMIN>0,VTIME=0 is supposed to work is to make characters 
available to the top level queue to be read when the low level input 
queue contains VMIN or more characters. Until that moment, there are 
no characters available to a read of any buffer size regardless of how 
many characters have been received at the low level. This is why a 
single character read blocks when at least one character has been 
received but not yet VMIN characters. Only data in the top level queue 
can be read. If the line discipline has not yet released data to the 
top level queue because of VMIN, then no data can be read, but this is 
not an error. 

Many have been tempted to change the behavior of this part of the 
system. IMHO, it is not worth tossing away application portability. 

Standards compliance can feel a bit like vertigo while instrument 
flying. Sometimes one has to just stare at the artificial horizon and 
say "I believe it" to one's self until the gut is convinced. 

Best regards,
Ed

---------------------------------------------------------------- 
Ed Vance              edv@macrolink.com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

Can you cite a single example of where the standard explicitly states that the 
a read of fewer bytes than VMIN should, none the less, block for the receipt 
of characters that can not be returned by the read?

You will find the standard is silent on the matter.  While it does say that 
the read will block until VMIN characters have been received it has no 
statement on what to do if less than VMIN characters have been requested in 
the particular read.  If I recall correctly (but it has been a while), Sun's 
implementation will return less than VMIN characters without waiting for the 
"extra" characters to be received.

Reguardless, the revise-down part can not possibly affect application 
portability.

Simply put, the patch provided is vanishingly close to* 100% user-space 
compatable with the stated intent of VMIN (and VTIME) as an upper limit on 
how long to wait for a read.  In particular, the downward revision behaviour 
will break absolutely no code.  (I haven't even been able to find an example 
of code where VMIN is greater than 1 and not explicitly reset to the expected 
packet read size).  [ *I use "vanishingly close to" instead of "is" because 
of a single case where VMIN is 255 and VTIME is 0, see below. ]

Since the entire purpose of the VMIN and VTIME is to allow reads to complete 
based on certain minimums.  If the minimum to receive is greater than the 
minimum that *can be returned" waiting for the extra, returnable characters 
is pointless.

When you read the manual pages it is clear the presumption is that the 
read-buffer-size is/would be bigger than the value of MIN.

===== Begin Quote (from sun termio(7i) since that's what I can cite ===== 

     MIN represents the minimum number of characters that  should
     be  received  when  the read is satisfied (that is, when the
     characters are returned to the user).  TIME is  a  timer  of
     0.10-second  granularity  that is used to timeout bursty and
     short-term data transmissions.  The four possible values for
     MIN and TIME and their interactions are described below.

     Case A: MIN > 0, TIME > 0
          In this case, TIME serves as  an  intercharacter  timer
          and is activated after the first character is received.
          Since it is an intercharacter timer, it is reset  after
          a  character  is received.  The interaction between MIN
          and TIME is as follows:  as soon as  one  character  is
          received,  the intercharacter timer is started.  If MIN
          characters are received before the intercharacter timer
          expires  (note  that the timer is reset upon receipt of
          each character), the read is satisfied.  If  the  timer
          expires before MIN characters are received, the charac-
          ters received to that point are returned to  the  user.
          Note  that if TIME expires, at least one character will
          be returned because  the  timer  would  not  have  been
          enabled  unless a character was received.  In this case
          (MIN > 0, TIME > 0), the read sleeps until the MIN  and
          TIME  mechanisms  are  activated  by the receipt of the
          first character.  If the number of characters  read  is
          less than the number of characters available, the timer
          is not reactivated and the subsequent read is satisfied
          immediately.
 
     Case B: MIN > 0, TIME = 0
          In this case, since the value  of  TIME  is  zero,  the
          timer  plays  no  role  and  only MIN is significant. A
          pending read is not satisfied until MIN characters  are
          received  (the pending read sleeps until MIN characters
          are received).  A program that uses this case  to  read
          record based terminal I/O may block indefinitely in the
          read operation.

     Case C: MIN = 0, TIME > 0
          In this case, since MIN = 0, TIME no longer  represents
          an intercharacter timer:  it now serves as a read timer
          that is activated as soon as a read is done.  A read is
          satisfied  as soon as a single character is received or
          the read timer expires.  Note that, in  this  case,  if
          the  timer  expires,  no character is returned.  If the
          timer does not expire, the only way  the  read  can  be
          satisfied is if a character is received.  In this case,
          the read will not  block  indefinitely  waiting  for  a
          character;  if no character is received within TIME*.10
          seconds after the read is initiated, the  read  returns
          with zero characters.

     Case D: MIN = 0, TIME = 0
          In this case, return  is  immediate.   The  minimum  of
          either the number of characters requested or the number
          of characters currently available is  returned  without
          waiting for more characters to be input.

  Comparison of the different cases of MIN, TIME interaction
     Some points to note about MIN and TIME:

     1.   In the following explanations, note that  the  interac-
          tions  of  MIN and TIME are not symmetric. For example,
          when MIN > 0 and TIME = 0, TIME has  no  effect.   How-
          ever, in the opposite case, where MIN = 0 and TIME > 0,
          both MIN and TIME play a role in that MIN is  satisfied
          with the receipt of a single character.

     2.   Also note that in case A (MIN >  0,  TIME  >  0),  TIME
          represents  an  intercharacter timer, whereas in case C
          (MIN = 0, TIME > 0), TIME represents a read timer.

     These two points highlight the dual purpose of the  MIN/TIME
     feature.   Cases  A  and  B,  where MIN > 0, exist to handle
     burst mode activity (for example, file  transfer  programs),
     where  a  program would like to process at least MIN charac-
     ters at a time.  In case  A,  the  intercharacter  timer  is
     activated  by  a  user  as  a safety measure; in case B, the
     timer is turned off.

     Cases C and  D  exist  to  handle  single  character,  timed
     transfers.   These  cases  are  readily adaptable to screen-
     based applications that need  to  know  if  a  character  is
     present in the input queue before refreshing the screen.  In
     case C, the read is timed, whereas in case D, it is not.

     Another important note is that MIN is always just a minimum.
     It  does not denote a record length.  For example, if a pro-
     gram does a read of 20 bytes, MIN is 10, and  25  characters
     are  present,  then  20  characters  will be returned to the
     user.

===== End Quote =====

Once again, notice the presumption that read would always supply a buffer 
equal to or larger than VMIN.  Nowhere does it mention anything about "upper" 
and "lower" parts of drivers it only addresses single reads and timing and 
return conditions to that read.  Since it is "silent" on the matter of what 
to do when the read size in less than MIN and it is "kind of dumb" to wait to 
return characters that can't be returned.  The downward revision code is 
completely within the spirit and structure of the standard.  It is also 
essentially impossible to break existing code by adding this fix.  (Existing 
code would already be hanging, or already be in bounds, or be part of a 
streaming dataset, and so on.)

The "bad thing" in the way it has been implemented currently is that when the 
packet sizes varry it takes a lot of time repeatedly reseting VMIN or 
bouncing in and out of kernel context and copying buffers because VMIN is 
artifically small (see pppd etc).  I am "fairly certain" that no other 
implementation would wait for unreturnable characters anyway since there is 
no real way to deal with the unreturnd characters that would result in 
predictable code.

As for the enhancement part.  The mod "could" jam up code in the single 
spesific case where VMIN==255 and VTIME==0 and the read size was larger than 
the received data could ever be.  I have found zero such instances.  Granted 
I do not have a psychic line into every piece of code everywhere.  I havn't 
even found a single instance of VMIN==255 and VTIME==0 (which, I reiterate, 
is the only case where the enhancement could possibly break code that wasn't 
already broken).

Find me one instance of code in the wild where:
 VMIN==255 and VTIME==0;
or
 VMIN=255 and VTIME>0 and performance isn't improved by the patch;
or
 VMIN < 255 and performance isn't unchanged-or-improved by the patch
and I'll bow my head in shame and go away.  8-)

So the floating down is a hard fix, and is actually hugely useful when dealing 
with small-event protocols.  The floating up is also hugely useful but there 
is a diminishingly miniscule chance of boning something.  The float-down fix 
doesn't violate a single standard (unless I am missing something).  and 
finally, the utility of the floating-up enhancement greatly outweighs the 
chance of it messing something up.

As I type this I can see that a guardian case of not floating up if TIME==0 
might be a conservatively useful case, but I dout it has any real-world 
value.

(in case you care for an example)
The real-world project that started me down this road was dealing with SIM 
cards (a.k.a. "Java Cards", "Smart Cards" etc) which (for protocol "T=0") 
operate at about 9600 baud and which return frames (who's variable lengths 
are known before the read) of data between 2 and 258 bytes, which can be 
paded with any number of 0x60(s), and which signal an error by "becomming 
unresponsive."  Lets face it, any protocol with a frame that includes a 
length component with one-byte encoding will, perforce, tend to overflow the 
255 value inherant in the VMIN data type because any framing on the payload 
will be longer thant the maximum payload of 255.

Also consider that virutally every serial protocol in use today has a larger 
than 255 character maximum frame.

I have been using the patch on several production systems (actually full time 
development boxes, so "production in development environment" whatever the 
distinction may be worth) and have found zero cases where there was any 
problems and the applications being developed with the fixed behavior have 
had a several-fold improvement in performance.

I think you will find the patch worth it in the long run and I doubt that, 
should you use your greater-access to the code and user base, you would find  
a single case where it breaks, invalidates, or confuses a single person or 
program on the planet.

Rob White.


On Monday 17 June 2002 10:27, Ed Vance wrote:
> On Sat, June 15, 2002 at 9:01 PM, Robert White wrote:
> > Kernel Versions: 2.2, 2.4, 2.5 (all of them since 1996 really 8-)
> >
> > The n_tty line discipline module contains a "semantic error" that
> > limits its speed and usefulness in many uses.  The attached patch
> > directly addresses serial performance in a completely backwards-
> > compatible way.
> >
> > In particular, the current handling of VMIN hugely limits,
> > complicates, and/or slows down optimal serial use.  The most
> > obvious example is that if you call read(2) with a buffer size less
> > than the current value of VMIN, the line discipline will insist
> > that the read call wait for characters that can not be returned to
> > that call.  The POSIX standard is silent on the subject of whether
> > this is right or wrong.  Common sense says it is wrong.
>
> Hi,
>
> IIRC, the way VMIN>0,VTIME=0 is supposed to work is to make characters
> available to the top level queue to be read when the low level input
> queue contains VMIN or more characters. Until that moment, there are
> no characters available to a read of any buffer size regardless of how
> many characters have been received at the low level. This is why a
> single character read blocks when at least one character has been
> received but not yet VMIN characters. Only data in the top level queue
> can be read. If the line discipline has not yet released data to the
> top level queue because of VMIN, then no data can be read, but this is
> not an error.
>
> Many have been tempted to change the behavior of this part of the
> system. IMHO, it is not worth tossing away application portability.
>
> Standards compliance can feel a bit like vertigo while instrument
> flying. Sometimes one has to just stare at the artificial horizon and
> say "I believe it" to one's self until the gut is convinced.
>
> Best regards,
> Ed
>
> ----------------------------------------------------------------
> Ed Vance              edv@macrolink.com
> Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
> ----------------------------------------------------------------

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Stuart MacDonald]

From: "Robert White" <rwhite@pobox.com>
> should you use your greater-access to the code and user base, you would
find
> a single case where it breaks, invalidates, or confuses a single person or
> program on the planet.

I can see one easy case: protocol has frame size of N. VMIN is set to
say 3 * N. read() supplies a buffer of N. The purpose being that the
first read will block until 3 frames are ready to process, then return
the first one. Your patch would break that.

Having said that, I don't know of any instances where the above is
employed. I'd say your patch is worthwhile.

..Stu

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Stuart MacDonald]

From: "Robert White" <rwhite@pobox.com>
> should you use your greater-access to the code and user base, you would
find
> a single case where it breaks, invalidates, or confuses a single person or
> program on the planet.

I can see one easy case: protocol has frame size of N. VMIN is set to
say 3 * N. read() supplies a buffer of N. The purpose being that the
first read will block until 3 frames are ready to process, then return
the first one. Your patch would break that.

Having said that, I don't know of any instances where the above is
employed. I'd say your patch is worthwhile.

..Stu

Re: n_tty.c driver patch (semantic and performance correction) (a ll recent versions)

[Robert White]

Can you cite a single example of where the standard explicitly states that the 
a read of fewer bytes than VMIN should, none the less, block for the receipt 
of characters that can not be returned by the read?

You will find the standard is silent on the matter.  While it does say that 
the read will block until VMIN characters have been received it has no 
statement on what to do if less than VMIN characters have been requested in 
the particular read.  If I recall correctly (but it has been a while), Sun's 
implementation will return less than VMIN characters without waiting for the 
"extra" characters to be received.

Reguardless, the revise-down part can not possibly affect application 
portability.

Simply put, the patch provided is vanishingly close to* 100% user-space 
compatable with the stated intent of VMIN (and VTIME) as an upper limit on 
how long to wait for a read.  In particular, the downward revision behaviour 
will break absolutely no code.  (I haven't even been able to find an example 
of code where VMIN is greater than 1 and not explicitly reset to the expected 
packet read size).  [ *I use "vanishingly close to" instead of "is" because 
of a single case where VMIN is 255 and VTIME is 0, see below. ]

Since the entire purpose of the VMIN and VTIME is to allow reads to complete 
based on certain minimums.  If the minimum to receive is greater than the 
minimum that *can be returned" waiting for the extra, returnable characters 
is pointless.

When you read the manual pages it is clear the presumption is that the 
read-buffer-size is/would be bigger than the value of MIN.

===== Begin Quote (from sun termio(7i) since that's what I can cite ===== 

     MIN represents the minimum number of characters that  should
     be  received  when  the read is satisfied (that is, when the
     characters are returned to the user).  TIME is  a  timer  of
     0.10-second  granularity  that is used to timeout bursty and
     short-term data transmissions.  The four possible values for
     MIN and TIME and their interactions are described below.

     Case A: MIN > 0, TIME > 0
          In this case, TIME serves as  an  intercharacter  timer
          and is activated after the first character is received.
          Since it is an intercharacter timer, it is reset  after
          a  character  is received.  The interaction between MIN
          and TIME is as follows:  as soon as  one  character  is
          received,  the intercharacter timer is started.  If MIN
          characters are received before the intercharacter timer
          expires  (note  that the timer is reset upon receipt of
          each character), the read is satisfied.  If  the  timer
          expires before MIN characters are received, the charac-
          ters received to that point are returned to  the  user.
          Note  that if TIME expires, at least one character will
          be returned because  the  timer  would  not  have  been
          enabled  unless a character was received.  In this case
          (MIN > 0, TIME > 0), the read sleeps until the MIN  and
          TIME  mechanisms  are  activated  by the receipt of the
          first character.  If the number of characters  read  is
          less than the number of characters available, the timer
          is not reactivated and the subsequent read is satisfied
          immediately.
 
     Case B: MIN > 0, TIME = 0
          In this case, since the value  of  TIME  is  zero,  the
          timer  plays  no  role  and  only MIN is significant. A
          pending read is not satisfied until MIN characters  are
          received  (the pending read sleeps until MIN characters
          are received).  A program that uses this case  to  read
          record based terminal I/O may block indefinitely in the
          read operation.

     Case C: MIN = 0, TIME > 0
          In this case, since MIN = 0, TIME no longer  represents
          an intercharacter timer:  it now serves as a read timer
          that is activated as soon as a read is done.  A read is
          satisfied  as soon as a single character is received or
          the read timer expires.  Note that, in  this  case,  if
          the  timer  expires,  no character is returned.  If the
          timer does not expire, the only way  the  read  can  be
          satisfied is if a character is received.  In this case,
          the read will not  block  indefinitely  waiting  for  a
          character;  if no character is received within TIME*.10
          seconds after the read is initiated, the  read  returns
          with zero characters.

     Case D: MIN = 0, TIME = 0
          In this case, return  is  immediate.   The  minimum  of
          either the number of characters requested or the number
          of characters currently available is  returned  without
          waiting for more characters to be input.

  Comparison of the different cases of MIN, TIME interaction
     Some points to note about MIN and TIME:

     1.   In the following explanations, note that  the  interac-
          tions  of  MIN and TIME are not symmetric. For example,
          when MIN > 0 and TIME = 0, TIME has  no  effect.   How-
          ever, in the opposite case, where MIN = 0 and TIME > 0,
          both MIN and TIME play a role in that MIN is  satisfied
          with the receipt of a single character.

     2.   Also note that in case A (MIN >  0,  TIME  >  0),  TIME
          represents  an  intercharacter timer, whereas in case C
          (MIN = 0, TIME > 0), TIME represents a read timer.

     These two points highlight the dual purpose of the  MIN/TIME
     feature.   Cases  A  and  B,  where MIN > 0, exist to handle
     burst mode activity (for example, file  transfer  programs),
     where  a  program would like to process at least MIN charac-
     ters at a time.  In case  A,  the  intercharacter  timer  is
     activated  by  a  user  as  a safety measure; in case B, the
     timer is turned off.

     Cases C and  D  exist  to  handle  single  character,  timed
     transfers.   These  cases  are  readily adaptable to screen-
     based applications that need  to  know  if  a  character  is
     present in the input queue before refreshing the screen.  In
     case C, the read is timed, whereas in case D, it is not.

     Another important note is that MIN is always just a minimum.
     It  does not denote a record length.  For example, if a pro-
     gram does a read of 20 bytes, MIN is 10, and  25  characters
     are  present,  then  20  characters  will be returned to the
     user.

===== End Quote =====

Once again, notice the presumption that read would always supply a buffer 
equal to or larger than VMIN.  Nowhere does it mention anything about "upper" 
and "lower" parts of drivers it only addresses single reads and timing and 
return conditions to that read.  Since it is "silent" on the matter of what 
to do when the read size in less than MIN and it is "kind of dumb" to wait to 
return characters that can't be returned.  The downward revision code is 
completely within the spirit and structure of the standard.  It is also 
essentially impossible to break existing code by adding this fix.  (Existing 
code would already be hanging, or already be in bounds, or be part of a 
streaming dataset, and so on.)

The "bad thing" in the way it has been implemented currently is that when the 
packet sizes varry it takes a lot of time repeatedly reseting VMIN or 
bouncing in and out of kernel context and copying buffers because VMIN is 
artifically small (see pppd etc).  I am "fairly certain" that no other 
implementation would wait for unreturnable characters anyway since there is 
no real way to deal with the unreturnd characters that would result in 
predictable code.

As for the enhancement part.  The mod "could" jam up code in the single 
spesific case where VMIN==255 and VTIME==0 and the read size was larger than 
the received data could ever be.  I have found zero such instances.  Granted 
I do not have a psychic line into every piece of code everywhere.  I havn't 
even found a single instance of VMIN==255 and VTIME==0 (which, I reiterate, 
is the only case where the enhancement could possibly break code that wasn't 
already broken).

Find me one instance of code in the wild where:
 VMIN==255 and VTIME==0;
or
 VMIN=255 and VTIME>0 and performance isn't improved by the patch;
or
 VMIN < 255 and performance isn't unchanged-or-improved by the patch
and I'll bow my head in shame and go away.  8-)

So the floating down is a hard fix, and is actually hugely useful when dealing 
with small-event protocols.  The floating up is also hugely useful but there 
is a diminishingly miniscule chance of boning something.  The float-down fix 
doesn't violate a single standard (unless I am missing something).  and 
finally, the utility of the floating-up enhancement greatly outweighs the 
chance of it messing something up.

As I type this I can see that a guardian case of not floating up if TIME==0 
might be a conservatively useful case, but I dout it has any real-world 
value.

(in case you care for an example)
The real-world project that started me down this road was dealing with SIM 
cards (a.k.a. "Java Cards", "Smart Cards" etc) which (for protocol "T=0") 
operate at about 9600 baud and which return frames (who's variable lengths 
are known before the read) of data between 2 and 258 bytes, which can be 
paded with any number of 0x60(s), and which signal an error by "becomming 
unresponsive."  Lets face it, any protocol with a frame that includes a 
length component with one-byte encoding will, perforce, tend to overflow the 
255 value inherant in the VMIN data type because any framing on the payload 
will be longer thant the maximum payload of 255.

Also consider that virutally every serial protocol in use today has a larger 
than 255 character maximum frame.

I have been using the patch on several production systems (actually full time 
development boxes, so "production in development environment" whatever the 
distinction may be worth) and have found zero cases where there was any 
problems and the applications being developed with the fixed behavior have 
had a several-fold improvement in performance.

I think you will find the patch worth it in the long run and I doubt that, 
should you use your greater-access to the code and user base, you would find  
a single case where it breaks, invalidates, or confuses a single person or 
program on the planet.

Rob White.


On Monday 17 June 2002 10:27, Ed Vance wrote:
> On Sat, June 15, 2002 at 9:01 PM, Robert White wrote:
> > Kernel Versions: 2.2, 2.4, 2.5 (all of them since 1996 really 8-)
> >
> > The n_tty line discipline module contains a "semantic error" that
> > limits its speed and usefulness in many uses.  The attached patch
> > directly addresses serial performance in a completely backwards-
> > compatible way.
> >
> > In particular, the current handling of VMIN hugely limits,
> > complicates, and/or slows down optimal serial use.  The most
> > obvious example is that if you call read(2) with a buffer size less
> > than the current value of VMIN, the line discipline will insist
> > that the read call wait for characters that can not be returned to
> > that call.  The POSIX standard is silent on the subject of whether
> > this is right or wrong.  Common sense says it is wrong.
>
> Hi,
>
> IIRC, the way VMIN>0,VTIME=0 is supposed to work is to make characters
> available to the top level queue to be read when the low level input
> queue contains VMIN or more characters. Until that moment, there are
> no characters available to a read of any buffer size regardless of how
> many characters have been received at the low level. This is why a
> single character read blocks when at least one character has been
> received but not yet VMIN characters. Only data in the top level queue
> can be read. If the line discipline has not yet released data to the
> top level queue because of VMIN, then no data can be read, but this is
> not an error.
>
> Many have been tempted to change the behavior of this part of the
> system. IMHO, it is not worth tossing away application portability.
>
> Standards compliance can feel a bit like vertigo while instrument
> flying. Sometimes one has to just stare at the artificial horizon and
> say "I believe it" to one's self until the gut is convinced.
>
> Best regards,
> Ed
>
> ----------------------------------------------------------------
> Ed Vance              edv@macrolink.com
> Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
> ----------------------------------------------------------------