Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Henrik Rydberg]

>> Ping has touched upon this subject as well, from the pen & touch perspective.
>> Generally, some ABS axes are actually enumerations, for which we have no
>> direct abstraction. If we had a way to declare the used values for such
>> enumerations, it would resolve these and possibly other issues.

> I think that presence of pen/touch can be detected by having
> BTN_TOOL_PEN and BTN_TOOL_FINGER. However in this case the tool is
> finger, so I do not think we should introduce BTN_TOOL_ENVELOPE. Maybe
> this is another case where we should employ the proposed device flags?

Yes. Having something like INPUT_QUIRK_SEMI_MT might be enough, and we could
drop the whole MT_TOOL_ENVELOPE circus. Chase, Peter, Chris, would you be
comfortable with such a solution?

> Anyway, it looks like we have a few concerns with current
> MT_TOOL_ENVELOPE so I want to rewind my 'next' branch.

Yep. Should I also take the opportunity to sync from -rc1 instead, and fold the
cleanup patches into the appropriate places?

Thanks,
Henrik

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chase Douglas]

On 12/15/2010 05:26 AM, Henrik Rydberg wrote:
>>> Ping has touched upon this subject as well, from the pen & touch perspective.
>>> Generally, some ABS axes are actually enumerations, for which we have no
>>> direct abstraction. If we had a way to declare the used values for such
>>> enumerations, it would resolve these and possibly other issues.
> 
>> I think that presence of pen/touch can be detected by having
>> BTN_TOOL_PEN and BTN_TOOL_FINGER. However in this case the tool is
>> finger, so I do not think we should introduce BTN_TOOL_ENVELOPE. Maybe
>> this is another case where we should employ the proposed device flags?
> 
> Yes. Having something like INPUT_QUIRK_SEMI_MT might be enough, and we could
> drop the whole MT_TOOL_ENVELOPE circus. Chase, Peter, Chris, would you be
> comfortable with such a solution?

That sounds like a good solution to me. I believe it would resolve all
the issues I had.

As I attempted to write up more documentation, I thought of the
following. What do you think?

With regards to partial MT devices, if the device provides a single
valued property, such as pressure and tool type for synaptics, it may
only be provided through the traditional property semantics, i.e.
ABS_PRESSURE and BTN_TOOL_*. If the device provides multiple values for
a property, then ABS_MT_* types may be used as well to provide up to two
values, though the client should understand there's no direct
correlation between the slot's coordinates and the property. I could see
this being used to provide info on multiple tool types or a high and low
pressure.

Enforcing the above behaviour provides even more information about the
capabilities of the device based solely on the evdev codes published.

Thanks,

-- Chase

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Henrik Rydberg]

>>> I think that presence of pen/touch can be detected by having

>>> BTN_TOOL_PEN and BTN_TOOL_FINGER. However in this case the tool is
>>> finger, so I do not think we should introduce BTN_TOOL_ENVELOPE. Maybe
>>> this is another case where we should employ the proposed device flags?
>>
>> Yes. Having something like INPUT_QUIRK_SEMI_MT might be enough, and we could
>> drop the whole MT_TOOL_ENVELOPE circus. Chase, Peter, Chris, would you be
>> comfortable with such a solution?
> 
> That sounds like a good solution to me. I believe it would resolve all
> the issues I had.


Sounds good.


> As I attempted to write up more documentation, I thought of the
> following. What do you think?
> 
> With regards to partial MT devices, if the device provides a single
> valued property, such as pressure and tool type for synaptics, it may
> only be provided through the traditional property semantics, i.e.
> ABS_PRESSURE and BTN_TOOL_*. If the device provides multiple values for
> a property, then ABS_MT_* types may be used as well to provide up to two
> values, though the client should understand there's no direct
> correlation between the slot's coordinates and the property. I could see
> this being used to provide info on multiple tool types or a high and low
> pressure.
> 
> Enforcing the above behaviour provides even more information about the
> capabilities of the device based solely on the evdev codes published.


Looks good, but I do not think we need to formalize all possibilities here, only
the usage of MT data for bounding rectangle and ST data for finger count.
Referring to the patch just sent: whenever INPUT_PROP_SEMI_MT is true, this
behavior is expected. In the event of new odd hardware, the combination of a new
property quirk and a documented recipe should do the trick.

Thanks,
Henrik

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chris Bagwell]

On Wed, Dec 15, 2010 at 11:40 AM, Chase Douglas
<chase.douglas@canonical.com> wrote:
> On 12/15/2010 05:26 AM, Henrik Rydberg wrote:
>>>> Ping has touched upon this subject as well, from the pen & touch perspective.
>>>> Generally, some ABS axes are actually enumerations, for which we have no
>>>> direct abstraction. If we had a way to declare the used values for such
>>>> enumerations, it would resolve these and possibly other issues.
>>
>>> I think that presence of pen/touch can be detected by having
>>> BTN_TOOL_PEN and BTN_TOOL_FINGER. However in this case the tool is
>>> finger, so I do not think we should introduce BTN_TOOL_ENVELOPE. Maybe
>>> this is another case where we should employ the proposed device flags?
>>
>> Yes. Having something like INPUT_QUIRK_SEMI_MT might be enough, and we could
>> drop the whole MT_TOOL_ENVELOPE circus. Chase, Peter, Chris, would you be
>> comfortable with such a solution?
>
> That sounds like a good solution to me. I believe it would resolve all
> the issues I had.

Works for me as well.

>
> As I attempted to write up more documentation, I thought of the
> following. What do you think?
>
> With regards to partial MT devices, if the device provides a single
> valued property, such as pressure and tool type for synaptics, it may
> only be provided through the traditional property semantics, i.e.
> ABS_PRESSURE and BTN_TOOL_*. If the device provides multiple values for
> a property, then ABS_MT_* types may be used as well to provide up to two
> values, though the client should understand there's no direct
> correlation between the slot's coordinates and the property. I could see
> this being used to provide info on multiple tool types or a high and low
> pressure.
>
> Enforcing the above behaviour provides even more information about the
> capabilities of the device based solely on the evdev codes published.
>

I meant to mention: once your initial draft gets committed I would
love to help update it some.  I specifically want to show two example
usage.  1) touchpad as 1 to 3 touchs occur and show special
considerations to ABS_* that apps should handle.  2) a touchscreen
that supports a pen as well and show how tool change (finger to pen)
should work.  For both those examples, it would be interesting to
discuss how MT can be used concurrently (does pen in slot 0 and touch
in slot 1 make sense for example).

I think it will be invaluable to document this stuff for driver
writers and apps but I'm not sure yet what level needs to be enforced.

Chris

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chase Douglas]

On 12/15/2010 03:41 PM, Chris Bagwell wrote:
> I meant to mention: once your initial draft gets committed I would
> love to help update it some.  I specifically want to show two example
> usage.  1) touchpad as 1 to 3 touchs occur and show special
> considerations to ABS_* that apps should handle.  2) a touchscreen
> that supports a pen as well and show how tool change (finger to pen)
> should work.  For both those examples, it would be interesting to
> discuss how MT can be used concurrently (does pen in slot 0 and touch
> in slot 1 make sense for example).

This is the other main reason I wanted to make the document. Having a
place where best practices are detailed will help future driver writers
and hopefully reduce errors in evdev code usage. I'd love to see this
added to the document.

I do think that MT is complex enough that related documentation should
be in multi-touch-protocol.txt, though. Anywhere I discussed MT in
evdev-codes.txt I referred the reader to the other file. Henrik, does
that sound good to you?

> I think it will be invaluable to document this stuff for driver
> writers and apps but I'm not sure yet what level needs to be enforced.

That's the biggest issue I see right now. Do we want black and white
specificity? For example, using terms like "must" and "may not" etc. Or
do we want the document to merely hold best practices while not
proscribing exact details? I think even with exact details we can loosen
them if needed, but that has its own can of worms.

Dmitry, what are your thoughts on this?

Thanks,

-- Chase

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chase Douglas]

On 12/15/2010 02:36 PM, Henrik Rydberg wrote:
>>>> I think that presence of pen/touch can be detected by having
> 
>>>> BTN_TOOL_PEN and BTN_TOOL_FINGER. However in this case the tool is
>>>> finger, so I do not think we should introduce BTN_TOOL_ENVELOPE. Maybe
>>>> this is another case where we should employ the proposed device flags?
>>>
>>> Yes. Having something like INPUT_QUIRK_SEMI_MT might be enough, and we could
>>> drop the whole MT_TOOL_ENVELOPE circus. Chase, Peter, Chris, would you be
>>> comfortable with such a solution?
>>
>> That sounds like a good solution to me. I believe it would resolve all
>> the issues I had.
> 
> 
> Sounds good.
> 
> 
>> As I attempted to write up more documentation, I thought of the
>> following. What do you think?
>>
>> With regards to partial MT devices, if the device provides a single
>> valued property, such as pressure and tool type for synaptics, it may
>> only be provided through the traditional property semantics, i.e.
>> ABS_PRESSURE and BTN_TOOL_*. If the device provides multiple values for
>> a property, then ABS_MT_* types may be used as well to provide up to two
>> values, though the client should understand there's no direct
>> correlation between the slot's coordinates and the property. I could see
>> this being used to provide info on multiple tool types or a high and low
>> pressure.
>>
>> Enforcing the above behaviour provides even more information about the
>> capabilities of the device based solely on the evdev codes published.
> 
> 
> Looks good, but I do not think we need to formalize all possibilities here, only
> the usage of MT data for bounding rectangle and ST data for finger count.
> Referring to the patch just sent: whenever INPUT_PROP_SEMI_MT is true, this
> behavior is expected. In the event of new odd hardware, the combination of a new
> property quirk and a documented recipe should do the trick.

Would you feel comfortable stating the above in less concrete terms, as
sort of a best practices guide? I'd like to know for this specific case
if you agree beyond ST finger count data, or if you feel we should do
something else like always provide as much data as possible in MT
properties? It's a real corner case, and I don't care too much one way
or another. I just don't want synaptics implemented one way, elantech
another, etc.

Thanks,

-- Chase

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Peter Hutterer]

On Wed, Dec 15, 2010 at 11:26:59AM +0100, Henrik Rydberg wrote:
> >> Ping has touched upon this subject as well, from the pen & touch perspective.
> >> Generally, some ABS axes are actually enumerations, for which we have no
> >> direct abstraction. If we had a way to declare the used values for such
> >> enumerations, it would resolve these and possibly other issues.
> 
> > I think that presence of pen/touch can be detected by having
> > BTN_TOOL_PEN and BTN_TOOL_FINGER. However in this case the tool is
> > finger, so I do not think we should introduce BTN_TOOL_ENVELOPE. Maybe
> > this is another case where we should employ the proposed device flags?
> 
> Yes. Having something like INPUT_QUIRK_SEMI_MT might be enough, and we could
> drop the whole MT_TOOL_ENVELOPE circus. Chase, Peter, Chris, would you be
> comfortable with such a solution?

sounds good to me, thanks.

Cheers,
  Peter

> > Anyway, it looks like we have a few concerns with current
> > MT_TOOL_ENVELOPE so I want to rewind my 'next' branch.
> 
> Yep. Should I also take the opportunity to sync from -rc1 instead, and fold the
> cleanup patches into the appropriate places?
> 
> Thanks,
> Henrik
> 

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Henrik Rydberg]

>

> I do think that MT is complex enough that related documentation should
> be in multi-touch-protocol.txt, though. Anywhere I discussed MT in
> evdev-codes.txt I referred the reader to the other file. Henrik, does
> that sound good to you?


Yep, thanks.

>> I think it will be invaluable to document this stuff for driver
>> writers and apps but I'm not sure yet what level needs to be enforced.
> 
> That's the biggest issue I see right now. Do we want black and white
> specificity? For example, using terms like "must" and "may not" etc. Or
> do we want the document to merely hold best practices while not
> proscribing exact details? I think even with exact details we can loosen
> them if needed, but that has its own can of worms.


It will most likely need to be judged on a case-by-case basis.

Thanks,
Henrik

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Henrik Rydberg]

>>>

>>> With regards to partial MT devices, if the device provides a single
>>> valued property, such as pressure and tool type for synaptics, it may
>>> only be provided through the traditional property semantics, i.e.
>>> ABS_PRESSURE and BTN_TOOL_*. If the device provides multiple values for
>>> a property, then ABS_MT_* types may be used as well to provide up to two
>>> values, though the client should understand there's no direct
>>> correlation between the slot's coordinates and the property. I could see
>>> this being used to provide info on multiple tool types or a high and low
>>> pressure.
>>>
>>> Enforcing the above behaviour provides even more information about the
>>> capabilities of the device based solely on the evdev codes published.
>>
>>
>> Looks good, but I do not think we need to formalize all possibilities here, only
>> the usage of MT data for bounding rectangle and ST data for finger count.
>> Referring to the patch just sent: whenever INPUT_PROP_SEMI_MT is true, this
>> behavior is expected. In the event of new odd hardware, the combination of a new
>> property quirk and a documented recipe should do the trick.
> 
> Would you feel comfortable stating the above in less concrete terms, as
> sort of a best practices guide? I'd like to know for this specific case
> if you agree beyond ST finger count data, or if you feel we should do
> something else like always provide as much data as possible in MT
> properties? It's a real corner case, and I don't care too much one way
> or another. I just don't want synaptics implemented one way, elantech
> another, etc.


A driver can still choose to report ABS_MT_PRESSURE for instance, in which case
it is assumed to make sense for individual fingers/corners. For semi-mt devices,
it seems reasonable to go to the ST variants to collect information not provided
via the MT protocol. I see no immediate reason to specify beyond that point.

Thanks,
Henrik

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chris Bagwell]

On Wed, Dec 15, 2010 at 11:31 AM, Chase Douglas
<chase.douglas@canonical.com> wrote:
> On 12/15/2010 02:25 AM, Dmitry Torokhov wrote:
>> On Wed, Dec 15, 2010 at 02:37:54AM +0100, Henrik Rydberg wrote:
>>> Hi Chase,
>>>
>>>>
>>>
>>>> I gave this some more thought, and I was close to accepting it with
>>>> documentation of the above restrictions. Then I thought of how the
>>>> following two devices would be presented to userspace:
>>>>
>>>> 1. A real MT device supporting up to 2 touches (e.g. a bamboo touch)
>>>>   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
>>>>   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
>>>>
>>>> 2. A partial MT device using MT_TOOL_ENVELOPE (e.g. synaptics MT)
>>>>   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
>>>>   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
>>>>
>>>> Note that they are identical! The range of values for each axis would be
>>>> identical too. The only way to tell the two apart would be to watch for
>>>> the ABS_MT_TOOL_TYPE axis.
>>
>> Question: does the driver really need to know this data beforehand?  I'd
>> expect MT-aware driver simply having handlers for both styles and then
>> doing the best it can with the data stream it gets... There should not
>> be ambiguity as to what event is - I believe we should be sending
>> BTN_TOOL_ENVELOPE even for the single/first contact for devices that do
>> not do full MT tracking.
>
> In XI 2.1 with MT, I would envision a partial MT device having different
> axis labels. We don't want to push an implementation specific
> abstraction, as MT_TOOL_ENVELOPE is, through the X protocol and require
> clients to watch the tool type. It should be readily apparent by the
> axis labels of the position valuators whether they represent true MT
> coordinates or a bounding rectangle of touches.
>
> As for whether clients will want to know if the device is real or
> partial MT, I think we should design a protocol that allows for a client
> to know. Maybe the application gives visual feedback on how to perform
> gestures depending on the device type?
>

Agree with these points.  From a configuration GUI perspective, I'd
want to show a subset of gestures can be performed on these partial
MT... or at least show a different video on how gesture must be
performed to be recognized.

So I think we do need to expose up front.

Chris

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chris Bagwell]

On Wed, Dec 15, 2010 at 11:31 AM, Chase Douglas
<chase.douglas@canonical.com> wrote:
> On 12/15/2010 02:25 AM, Dmitry Torokhov wrote:
>> On Wed, Dec 15, 2010 at 02:37:54AM +0100, Henrik Rydberg wrote:
>>> Hi Chase,
>>>
>>>>
>>>
>>>> I gave this some more thought, and I was close to accepting it with
>>>> documentation of the above restrictions. Then I thought of how the
>>>> following two devices would be presented to userspace:
>>>>
>>>> 1. A real MT device supporting up to 2 touches (e.g. a bamboo touch)
>>>>   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
>>>>   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
>>>>
>>>> 2. A partial MT device using MT_TOOL_ENVELOPE (e.g. synaptics MT)
>>>>   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
>>>>   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
>>>>
>>>> Note that they are identical! The range of values for each axis would be
>>>> identical too. The only way to tell the two apart would be to watch for
>>>> the ABS_MT_TOOL_TYPE axis.
>>
>> Question: does the driver really need to know this data beforehand?  I'd
>> expect MT-aware driver simply having handlers for both styles and then
>> doing the best it can with the data stream it gets... There should not
>> be ambiguity as to what event is - I believe we should be sending
>> BTN_TOOL_ENVELOPE even for the single/first contact for devices that do
>> not do full MT tracking.
>
> In XI 2.1 with MT, I would envision a partial MT device having different
> axis labels. We don't want to push an implementation specific
> abstraction, as MT_TOOL_ENVELOPE is, through the X protocol and require
> clients to watch the tool type. It should be readily apparent by the
> axis labels of the position valuators whether they represent true MT
> coordinates or a bounding rectangle of touches.
>
> As for whether clients will want to know if the device is real or
> partial MT, I think we should design a protocol that allows for a client
> to know. Maybe the application gives visual feedback on how to perform
> gestures depending on the device type?
>

Agree with these points.  From a configuration GUI perspective, I'd
want to show a subset of gestures can be performed on these partial
MT... or at least show a different video on how gesture must be
performed to be recognized.

So I think we do need to expose up front.

Chris
--
To unsubscribe from this list: send the line "unsubscribe linux-input" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chase Douglas]

On 12/15/2010 02:25 AM, Dmitry Torokhov wrote:
> On Wed, Dec 15, 2010 at 02:37:54AM +0100, Henrik Rydberg wrote:
>> Hi Chase,
>>
>>>
>>
>>> I gave this some more thought, and I was close to accepting it with
>>> documentation of the above restrictions. Then I thought of how the
>>> following two devices would be presented to userspace:
>>>
>>> 1. A real MT device supporting up to 2 touches (e.g. a bamboo touch)
>>>   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
>>>   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
>>>
>>> 2. A partial MT device using MT_TOOL_ENVELOPE (e.g. synaptics MT)
>>>   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
>>>   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
>>>
>>> Note that they are identical! The range of values for each axis would be
>>> identical too. The only way to tell the two apart would be to watch for
>>> the ABS_MT_TOOL_TYPE axis.
> 
> Question: does the driver really need to know this data beforehand?  I'd
> expect MT-aware driver simply having handlers for both styles and then
> doing the best it can with the data stream it gets... There should not
> be ambiguity as to what event is - I believe we should be sending
> BTN_TOOL_ENVELOPE even for the single/first contact for devices that do
> not do full MT tracking.

In XI 2.1 with MT, I would envision a partial MT device having different
axis labels. We don't want to push an implementation specific
abstraction, as MT_TOOL_ENVELOPE is, through the X protocol and require
clients to watch the tool type. It should be readily apparent by the
axis labels of the position valuators whether they represent true MT
coordinates or a bounding rectangle of touches.

As for whether clients will want to know if the device is real or
partial MT, I think we should design a protocol that allows for a client
to know. Maybe the application gives visual feedback on how to perform
gestures depending on the device type?

Thanks,

-- Chase

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Dmitry Torokhov]

On Wed, Dec 15, 2010 at 02:37:54AM +0100, Henrik Rydberg wrote:
> Hi Chase,
> 
> >
> 
> > I gave this some more thought, and I was close to accepting it with
> > documentation of the above restrictions. Then I thought of how the
> > following two devices would be presented to userspace:
> > 
> > 1. A real MT device supporting up to 2 touches (e.g. a bamboo touch)
> >   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
> >   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
> > 
> > 2. A partial MT device using MT_TOOL_ENVELOPE (e.g. synaptics MT)
> >   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
> >   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
> > 
> > Note that they are identical! The range of values for each axis would be
> > identical too. The only way to tell the two apart would be to watch for
> > the ABS_MT_TOOL_TYPE axis.

Question: does the driver really need to know this data beforehand?  I'd
expect MT-aware driver simply having handlers for both styles and then
doing the best it can with the data stream it gets... There should not
be ambiguity as to what event is - I believe we should be sending
BTN_TOOL_ENVELOPE even for the single/first contact for devices that do
not do full MT tracking.

> 
> 
> Ping has touched upon this subject as well, from the pen & touch perspective.
> Generally, some ABS axes are actually enumerations, for which we have no direct
> abstraction. If we had a way to declare the used values for such enumerations,
> it would resolve these and possibly other issues.

I think that presence of pen/touch can be detected by having
BTN_TOOL_PEN and BTN_TOOL_FINGER. However in this case the tool is
finger, so I do not think we should introduce BTN_TOOL_ENVELOPE. Maybe
this is another case where we should employ the proposed device flags?

Anyway, it looks like we have a few concerns with current
MT_TOOL_ENVELOPE so I want to rewind my 'next' branch.

-- 
Dmitry

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Henrik Rydberg]

Hi Chase,

>

> I gave this some more thought, and I was close to accepting it with
> documentation of the above restrictions. Then I thought of how the
> following two devices would be presented to userspace:
> 
> 1. A real MT device supporting up to 2 touches (e.g. a bamboo touch)
>   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
>   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
> 
> 2. A partial MT device using MT_TOOL_ENVELOPE (e.g. synaptics MT)
>   - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
>   - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE
> 
> Note that they are identical! The range of values for each axis would be
> identical too. The only way to tell the two apart would be to watch for
> the ABS_MT_TOOL_TYPE axis.


Ping has touched upon this subject as well, from the pen & touch perspective.
Generally, some ABS axes are actually enumerations, for which we have no direct
abstraction. If we had a way to declare the used values for such enumerations,
it would resolve these and possibly other issues.

Thanks,
Henrik

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chase Douglas]

On 12/14/2010 02:12 PM, Dmitry Torokhov wrote:
> Hi Chase,
> 
> On Tue, Dec 14, 2010 at 01:21:08PM -0800, Chase Douglas wrote:
>> Hello all,
>>
>> I was left somewhat dissatisfied by the MT_TOOL_ENVELOPE addition, so I decided
>> to try to propose a different solution for the problem. As a recap, the problem
>> can best be defined by Synaptics hardware that provide two touch coordinates
>> (X1, Y1) and (X2, Y2). Unfortunately, the real touches may be at (X1, Y2) and
>> (X2, Y1). Further, three touches may be recognized, but only two coordinates are
>> reported.
>>
>> Following this are four patches. The first merely reverts the MT_TOOL_ENVELOPE
>> addition. The second adds documentation for evdev codes to the Documentation
>> directory. It was hastily created, so it has some ommissions and may have some
>> mistakes. My hope is that we keep this or a similar document up to date whenever
>> non-obvious codes are added to evdev.
> 
> This is great, thank you.

I've found a few formatting issues, so don't apply it quite yet.

>> The third patch adds the following EV_ABS
>> codes:
>>
>> ABS_RECT_MIN_X
>> ABS_RECT_MIN_Y
>> ABS_RECT_MAX_X
>> ABS_RECT_MAX_Y
>>
>> The purpose of these codes is to provide for devices that at best report a
>> rectangular bounding box of touches. Instead of using the MT evdev protocol,
>> this approach uses ST protocol semantics.
>>
>> Finally, the last patch adds support for the above codes to the synaptics
>> driver.
>>
>> It is my belief that this is a better interface than MT_TOOL_ENVELOPE for the
>> following reasons:
>>
>> * The code meanings are more readily graspable from the names
>> * The codes behave with ST semantics, which is useful because we likely cannot
>>   split properties like pressure and orientation among the touches
>> * ST semantics are easier to comprehend than MT semantics, and MT isn't required
>>   to solve this problem
>> * The codes provide only the max and min values, none in between. This is in
>>   contrast with MT_TOOL_ENVELOPE, which provides all touches as presented by the
>>   hardware. However, we don't trust all the coordinate pairings, so providing
>>   faulty pairings may induce incorrect userspace usage of the events.
>> * A clear distinction is made here that full multitouch devices should use the
>>   MT protocol, while lesser devices should use the ST protocol.
>>
> 
> No, I do not agree with this proposal. This would introduce new _axes_
> (with potentially different min, max, resolution, etc) for the working
> surface of the devices instead of merely saying that the device may
> report more than simple singular contact within the standard axes.
> 
> I believe that MT protocol _is_ the correct vehicle to transmit semi-MT
> device state to userspace.
> 
> As to reporting more than 2 contacts with MT_TOOL_ENVELOPE - I think we
> should not do that. Instead we should only report 2 outermost points of
> the bounding rectangle as MT_TOOL_ENVELOPE and convey number of contacts
> with BTN_TOOL_xxxTAP (so up to 4 contacts, at least for now).

I gave this some more thought, and I was close to accepting it with
documentation of the above restrictions. Then I thought of how the
following two devices would be presented to userspace:

1. A real MT device supporting up to 2 touches (e.g. a bamboo touch)
  - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
  - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE

2. A partial MT device using MT_TOOL_ENVELOPE (e.g. synaptics MT)
  - ABS_{X,Y}, BTN_TOUCH, BTN_TOOL_FINGER, and BTN_TOOL_DOUBLETAP for ST
  - ABS_MT_SLOT, ABS_MT_TRACKING_ID, ABS_MT_POSITION_{X,Y}, ABS_MT_TOOL_TYPE

Note that they are identical! The range of values for each axis would be
identical too. The only way to tell the two apart would be to watch for
the ABS_MT_TOOL_TYPE axis. If the tool type stays MT_TOOL_FINGER when
both slots are active, it's a real MT device. If the tool type switches
to MT_TOOL_ENVELOPE, it's a partial MT device.

This means in userspace we cannot determine the full capabilities of the
device until at least two fingers touch it. This presents a challenge to
user space software like X and a gesture recognizer. In the former case,
the device properties would need to be changed, and X clients may need
to watch the device properties to ensure they are handling devices
correctly. No one does this yet because no input devices have ever
needed to be mutable. In the recognizer case, a partial MT device cannot
perform some rotation gestures. Gesture clients will need to know
whether a rotation gesture is possible on a given device.

Further, another issue I see is that this conflates boundary boxes of
input touches with tool type. How do we denote the tool type of a
partial MT device when MT_TOOL_ENVELOPE is taking up the
ABS_MT_TOOL_TYPE property? Or do we want to codify MT_TOOL_ENVELOPE to
mean a bounding box of *only* finger touches? If the latter, that's just
piling even more constraints on an evdev code name that doesn't give a
hint of any of these complexities. We can document this all, but I have
a hard time thinking this is the best way forward.

I understand that more new ABS_* axes may be undesirable, but it feels
the best approach to me. However, I would be ok with MT slots approaches
as long as they are well defined and do not introduce problems
highlighted above.

Thanks,

-- Chase

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Dmitry Torokhov]

Hi Chase,

On Tue, Dec 14, 2010 at 01:21:08PM -0800, Chase Douglas wrote:
> Hello all,
> 
> I was left somewhat dissatisfied by the MT_TOOL_ENVELOPE addition, so I decided
> to try to propose a different solution for the problem. As a recap, the problem
> can best be defined by Synaptics hardware that provide two touch coordinates
> (X1, Y1) and (X2, Y2). Unfortunately, the real touches may be at (X1, Y2) and
> (X2, Y1). Further, three touches may be recognized, but only two coordinates are
> reported.
> 
> Following this are four patches. The first merely reverts the MT_TOOL_ENVELOPE
> addition. The second adds documentation for evdev codes to the Documentation
> directory. It was hastily created, so it has some ommissions and may have some
> mistakes. My hope is that we keep this or a similar document up to date whenever
> non-obvious codes are added to evdev.

This is great, thank you.

> The third patch adds the following EV_ABS
> codes:
> 
> ABS_RECT_MIN_X
> ABS_RECT_MIN_Y
> ABS_RECT_MAX_X
> ABS_RECT_MAX_Y
> 
> The purpose of these codes is to provide for devices that at best report a
> rectangular bounding box of touches. Instead of using the MT evdev protocol,
> this approach uses ST protocol semantics.
> 
> Finally, the last patch adds support for the above codes to the synaptics
> driver.
> 
> It is my belief that this is a better interface than MT_TOOL_ENVELOPE for the
> following reasons:
> 
> * The code meanings are more readily graspable from the names
> * The codes behave with ST semantics, which is useful because we likely cannot
>   split properties like pressure and orientation among the touches
> * ST semantics are easier to comprehend than MT semantics, and MT isn't required
>   to solve this problem
> * The codes provide only the max and min values, none in between. This is in
>   contrast with MT_TOOL_ENVELOPE, which provides all touches as presented by the
>   hardware. However, we don't trust all the coordinate pairings, so providing
>   faulty pairings may induce incorrect userspace usage of the events.
> * A clear distinction is made here that full multitouch devices should use the
>   MT protocol, while lesser devices should use the ST protocol.
> 

No, I do not agree with this proposal. This would introduce new _axes_
(with potentially different min, max, resolution, etc) for the working
surface of the devices instead of merely saying that the device may
report more than simple singular contact within the standard axes.

I believe that MT protocol _is_ the correct vehicle to transmit semi-MT
device state to userspace.

As to reporting more than 2 contacts with MT_TOOL_ENVELOPE - I think we
should not do that. Instead we should only report 2 outermost points of
the bounding rectangle as MT_TOOL_ENVELOPE and convey number of contacts
with BTN_TOOL_xxxTAP (so up to 4 contacts, at least for now).

Thanks.

-- 
Dmitry

Re: [PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chase Douglas]

On 12/14/2010 01:21 PM, Chase Douglas wrote:
> Hello all,
> 
> I was left somewhat dissatisfied by the MT_TOOL_ENVELOPE addition, so I decided
> to try to propose a different solution for the problem. As a recap, the problem
> can best be defined by Synaptics hardware that provide two touch coordinates
> (X1, Y1) and (X2, Y2). Unfortunately, the real touches may be at (X1, Y2) and
> (X2, Y1). Further, three touches may be recognized, but only two coordinates are
> reported.
> 
> Following this are four patches.

[...]

I forgot to mention that these apply on top of Dmitry's next tree plus
the first synaptics multitouch patch. My bad :(.

-- Chase

[PATCH 0/4] Alternative approach to MT_TOOL_ENVELOPE

[Chase Douglas]

Hello all,

I was left somewhat dissatisfied by the MT_TOOL_ENVELOPE addition, so I decided
to try to propose a different solution for the problem. As a recap, the problem
can best be defined by Synaptics hardware that provide two touch coordinates
(X1, Y1) and (X2, Y2). Unfortunately, the real touches may be at (X1, Y2) and
(X2, Y1). Further, three touches may be recognized, but only two coordinates are
reported.

Following this are four patches. The first merely reverts the MT_TOOL_ENVELOPE
addition. The second adds documentation for evdev codes to the Documentation
directory. It was hastily created, so it has some ommissions and may have some
mistakes. My hope is that we keep this or a similar document up to date whenever
non-obvious codes are added to evdev. The third patch adds the following EV_ABS
codes:

ABS_RECT_MIN_X
ABS_RECT_MIN_Y
ABS_RECT_MAX_X
ABS_RECT_MAX_Y

The purpose of these codes is to provide for devices that at best report a
rectangular bounding box of touches. Instead of using the MT evdev protocol,
this approach uses ST protocol semantics.

Finally, the last patch adds support for the above codes to the synaptics
driver.

It is my belief that this is a better interface than MT_TOOL_ENVELOPE for the
following reasons:

* The code meanings are more readily graspable from the names
* The codes behave with ST semantics, which is useful because we likely cannot
  split properties like pressure and orientation among the touches
* ST semantics are easier to comprehend than MT semantics, and MT isn't required
  to solve this problem
* The codes provide only the max and min values, none in between. This is in
  contrast with MT_TOOL_ENVELOPE, which provides all touches as presented by the
  hardware. However, we don't trust all the coordinate pairings, so providing
  faulty pairings may induce incorrect userspace usage of the events.
* A clear distinction is made here that full multitouch devices should use the
  MT protocol, while lesser devices should use the ST protocol.

Thanks!

-- Chase