How can I help?

How can I help?

[Chris Stromblad]

Hi there,

Just wanted to introduce myself to the list and also ask a few
questions. My name is Chris, currently living in the UK, nerd by day and
even more so at night.

Today I thought about playing with KVM for the first time, but without
much luck. Reading the archives I noticed someone else was experiencing
the same problem, it's the "exception 12" problem in qemu/kvm.

While I am an okay C-programmer, I've got little experience with
kernel-development but would be happy to learn or assist in anyway I
can.

I'm currently working on an Intel Core 2 CPU, with Archlinux as choice
of distribution. If there is anything I can do to help, please let me
know!

/ Chris

PS: Ops, almost forgot. Are there any solutions for the "exception 12"
problem, or something I can do?



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: How can I help?

[Avi Kivity]

Chris Stromblad wrote:
> Hi there,
>
> Just wanted to introduce myself to the list and also ask a few
> questions. My name is Chris, currently living in the UK, nerd by day and
> even more so at night.
>
> Today I thought about playing with KVM for the first time, but without
> much luck. Reading the archives I noticed someone else was experiencing
> the same problem, it's the "exception 12" problem in qemu/kvm.
>
> While I am an okay C-programmer, I've got little experience with
> kernel-development but would be happy to learn or assist in anyway I
> can.
>
>   

Unfortunately hacking on the code requires much poring over ancient 
manuscripts, a thorough understanding of the glorious history of the x86 
architecture, and much banging of heads against walls.  So unless you're 
prepared to spend quite a bit of time on kvm, your best bet is in testing.

Having a regression test of a real workload with timing measurements 
always helps, prefereably with multiple virtual machines running 
concurrently.


> I'm currently working on an Intel Core 2 CPU, with Archlinux as choice
> of distribution. If there is anything I can do to help, please let me
> know!
>
> / Chris
>
> PS: Ops, almost forgot. Are there any solutions for the "exception 12"
> problem, or something I can do?
>
>   

What guest are you running?  At what stage are you seeing the problem?  
Usually the workaround involves disabling the bootloader splashscreen or 
switching bootloaders.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: How can I help?

[Chris Stromblad]

Few years ago I did a fair bit of x86 programming, but with the more
recent CPUs and all the new instructions, I'd probably be bit out of it.
Then again, wouldn't hurt having a look at it... who knows, maybe in a
few years I could actually do something ;)

The problem.

Basically I get the problem immediately when attempting to boot from an
ISO image (the installation step). The image is a UBUNTU x86_64
installation disk. Pretty much instantly when I attempt to boot the
window flashes by and in the original window I get the CPU reg dump with
the exception 12.

It works when using the unmodified version of QEMU.

/ Chris

On Sun, 2007-02-25 at 07:25 +0200, Avi Kivity wrote:
> Chris Stromblad wrote:
> > Hi there,
> >
> > Just wanted to introduce myself to the list and also ask a few
> > questions. My name is Chris, currently living in the UK, nerd by day and
> > even more so at night.
> >
> > Today I thought about playing with KVM for the first time, but without
> > much luck. Reading the archives I noticed someone else was experiencing
> > the same problem, it's the "exception 12" problem in qemu/kvm.
> >
> > While I am an okay C-programmer, I've got little experience with
> > kernel-development but would be happy to learn or assist in anyway I
> > can.
> >
> >   
> 
> Unfortunately hacking on the code requires much poring over ancient 
> manuscripts, a thorough understanding of the glorious history of the x86 
> architecture, and much banging of heads against walls.  So unless you're 
> prepared to spend quite a bit of time on kvm, your best bet is in testing.
> 
> Having a regression test of a real workload with timing measurements 
> always helps, prefereably with multiple virtual machines running 
> concurrently.
> 
> 
> > I'm currently working on an Intel Core 2 CPU, with Archlinux as choice
> > of distribution. If there is anything I can do to help, please let me
> > know!
> >
> > / Chris
> >
> > PS: Ops, almost forgot. Are there any solutions for the "exception 12"
> > problem, or something I can do?
> >
> >   
> 
> What guest are you running?  At what stage are you seeing the problem?  
> Usually the workaround involves disabling the bootloader splashscreen or 
> switching bootloaders.
> 



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: How can I help?

[Avi Kivity]

Chris Stromblad wrote:
> Few years ago I did a fair bit of x86 programming, but with the more
> recent CPUs and all the new instructions, I'd probably be bit out of it.
> Then again, wouldn't hurt having a look at it... who knows, maybe in a
> few years I could actually do something ;)
>
>   

x86 experience definitely helps.  You might want to download the latest 
Intel and AMD documentation and check out the relevant chapters.  For 
Intel, that's chapters 20-25 in the software developer's manual, volume 3B.

Actually knowing all the instructions isn't that important; it's the 
general quirkiness of the arch that needs to be understood.


> The problem.
>
> Basically I get the problem immediately when attempting to boot from an
> ISO image (the installation step). The image is a UBUNTU x86_64
> installation disk. Pretty much instantly when I attempt to boot the
> window flashes by and in the original window I get the CPU reg dump with
> the exception 12.
>   

Ok.  That's definitely the real-mode emulation problem we have on 
Intel.  You might try installing using qemu, and disabling any boot 
loader splashscreens (I don't know the exact syntax required), and 
restarting under kvm.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: How can I help?

[Chris Stromblad]

Sure, I'll have a look at the documentation. Need some evening reading
anyways ;)

Thanks for the suggestion I will try and install using QEMU and then
restart under KVM and the modified version of qemu, cheers!

/ Chris

On Sun, 2007-02-25 at 11:39 +0200, Avi Kivity wrote:
> Chris Stromblad wrote:
> > Few years ago I did a fair bit of x86 programming, but with the more
> > recent CPUs and all the new instructions, I'd probably be bit out of it.
> > Then again, wouldn't hurt having a look at it... who knows, maybe in a
> > few years I could actually do something ;)
> >
> >   
> 
> x86 experience definitely helps.  You might want to download the latest 
> Intel and AMD documentation and check out the relevant chapters.  For 
> Intel, that's chapters 20-25 in the software developer's manual, volume 3B.
> 
> Actually knowing all the instructions isn't that important; it's the 
> general quirkiness of the arch that needs to be understood.
> 
> 
> > The problem.
> >
> > Basically I get the problem immediately when attempting to boot from an
> > ISO image (the installation step). The image is a UBUNTU x86_64
> > installation disk. Pretty much instantly when I attempt to boot the
> > window flashes by and in the original window I get the CPU reg dump with
> > the exception 12.
> >   
> 
> Ok.  That's definitely the real-mode emulation problem we have on 
> Intel.  You might try installing using qemu, and disabling any boot 
> loader splashscreens (I don't know the exact syntax required), and 
> restarting under kvm.
> 



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: How can I help?

[Chris Stromblad]

Hi again,

Exactly how can I disable the bootsplash? And out of curiosity, why is
that preventing KVM/QEMU from working properly?

/ Chris

On Sun, 2007-02-25 at 11:39 +0200, Avi Kivity wrote:
> Chris Stromblad wrote:
> > Few years ago I did a fair bit of x86 programming, but with the more
> > recent CPUs and all the new instructions, I'd probably be bit out of it.
> > Then again, wouldn't hurt having a look at it... who knows, maybe in a
> > few years I could actually do something ;)
> >
> >   
> 
> x86 experience definitely helps.  You might want to download the latest 
> Intel and AMD documentation and check out the relevant chapters.  For 
> Intel, that's chapters 20-25 in the software developer's manual, volume 3B.
> 
> Actually knowing all the instructions isn't that important; it's the 
> general quirkiness of the arch that needs to be understood.
> 
> 
> > The problem.
> >
> > Basically I get the problem immediately when attempting to boot from an
> > ISO image (the installation step). The image is a UBUNTU x86_64
> > installation disk. Pretty much instantly when I attempt to boot the
> > window flashes by and in the original window I get the CPU reg dump with
> > the exception 12.
> >   
> 
> Ok.  That's definitely the real-mode emulation problem we have on 
> Intel.  You might try installing using qemu, and disabling any boot 
> loader splashscreens (I don't know the exact syntax required), and 
> restarting under kvm.
> 



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: How can I help?

[Avi Kivity]

Chris Stromblad wrote:
> Hi again,
>
> Exactly how can I disable the bootsplash? 

Install/boot with -no-kvm, then edit the grub configuration file.  The 
details vary, and I'm not sure how to do it on Ubuntu.


> And out of curiosity, why is
> that preventing KVM/QEMU from working properly?
>   

kvm uses vm86 mode to virtualize real mode.  Some boot loaders use 
features of real mode which are not present (or not identical to) vm86 mode.

One day we will emulate real mode (or use a combination of vm86 and 
emulation) which will fix the problem.

> / Chris
>
> On Sun, 2007-02-25 at 11:39 +0200, Avi Kivity wrote:
>   
>> Chris Stromblad wrote:
>>     
>>> Few years ago I did a fair bit of x86 programming, but with the more
>>> recent CPUs and all the new instructions, I'd probably be bit out of it.
>>> Then again, wouldn't hurt having a look at it... who knows, maybe in a
>>> few years I could actually do something ;)
>>>
>>>   
>>>       
>> x86 experience definitely helps.  You might want to download the latest 
>> Intel and AMD documentation and check out the relevant chapters.  For 
>> Intel, that's chapters 20-25 in the software developer's manual, volume 3B.
>>
>> Actually knowing all the instructions isn't that important; it's the 
>> general quirkiness of the arch that needs to be understood.
>>
>>
>>     
>>> The problem.
>>>
>>> Basically I get the problem immediately when attempting to boot from an
>>> ISO image (the installation step). The image is a UBUNTU x86_64
>>> installation disk. Pretty much instantly when I attempt to boot the
>>> window flashes by and in the original window I get the CPU reg dump with
>>> the exception 12.
>>>   
>>>       
>> Ok.  That's definitely the real-mode emulation problem we have on 
>> Intel.  You might try installing using qemu, and disabling any boot 
>> loader splashscreens (I don't know the exact syntax required), and 
>> restarting under kvm.
>>
>>     
>
>
>   


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: How can I help?

[Larry Baker]

Greg,

> I'd go buy a few different types, of different cores, and try to find
> one that works for your workload.  That's going to be the simplest, and
> probably cheapest, way to do this.

Thanks a lot for your help.  I have to pick my poison.  The ARM SoC box I'm using has a UART, but their RTS/CTS is broken (as in, stty crtscts rejected; they fixed that, but I still get no data).  I can work on that driver or board configuration, and have a fix for a single platform.  Or, I can find a working or fix (questionable) a broken USB serial driver and have a solution good for multiple platforms.  I'd really like to give back and fix both.  Unfortunately, I don't have enough lifetimes.  As challenging and interesting as that may be, I have an actual job I have to do.  And my boss likes it better when I spend time on that. :)

I'll collect more data using adapters with different chips as well as a more recent kernel.  Then I can provide more useful information to the linux-usb@vger.kernel.org email list.

Larry Baker
US Geological Survey
650-329-5608
baker@usgs.gov

Re: How can I help?

[Greg KH]

On Fri, Apr 18, 2014 at 04:11:26PM -0700, Larry Baker wrote:
> Greg,
> 
> On 18 Apr 2014, at 3:43 PM, Greg KH wrote:
> 
> > On Fri, Apr 18, 2014 at 03:37:53PM -0700, Larry Baker wrote:
> >> Greg,
> >> 
> >> Any suggestions how to trace what is going on?
> > 
> > Going on with what?  Was there a bug report in that wall of text that I
> > missed?  :)
> 
> "It sort of works.  What happens is, after about 8 or 9 hours, the RX
> to the TTY port times out.  Sometimes it happens in only an hour,
> other times it takes about 14 hours."
> 
> "I am suspicious, for example, that the CTS grant is getting lost."
> 
> Should I report this on the linux-usb@vger.kernel.org mailing list?

Yes, that would be good.

But failures like this is going to be hard to track down, as you said,
how do you know it's not just the uart in the chip that drops this?

These chips are really not meant to be run all the time like this, with
constant data streams.  They are really cheep chips.

I'd go buy a few different types, of different cores, and try to find
one that works for your workload.  That's going to be the simplest, and
probably cheapest, way to do this.

best of luck,

greg k-h

Re: How can I help?

[Larry Baker]

Greg,

On 18 Apr 2014, at 3:43 PM, Greg KH wrote:

> On Fri, Apr 18, 2014 at 03:37:53PM -0700, Larry Baker wrote:
>> Greg,
>> 
>> Any suggestions how to trace what is going on?
> 
> Going on with what?  Was there a bug report in that wall of text that I
> missed?  :)

"It sort of works.  What happens is, after about 8 or 9 hours, the RX to the TTY port times out.  Sometimes it happens in only an hour, other times it takes about 14 hours."

"I am suspicious, for example, that the CTS grant is getting lost."

Should I report this on the linux-usb@vger.kernel.org mailing list?

>> I see all the tracing debug code has been removed in the latest
>> ftdi_sio driver.
> 
> That's because the kernel provides tracing functionality built-in, no
> need for it to be on a driver-by-driver basis.  Use it if you think it
> is necessary.

Can you point me to a web page or kernel doc for how to use it?

> Everything else are very cheap devices, with questionable UART silicon.

That's what I'm afraid I may never be able to determine -- whether the problem I have is between the driver and the chip (USB protocol), or is in the UART chip (could be USB protocol or the UART itself).  This is basically a side project, and I don't have the proper tools anyway to do that kind of trouble-shooting.  I will do what I can and report any findings, especially if a different USB serial driver works.

Thanks,

Larry Baker
US Geological Survey
650-329-5608
baker@usgs.gov

Re: How can I help?

[Theodore Ts'o]

On Fri, Apr 18, 2014 at 03:37:53PM -0700, Larry Baker wrote:
> 
> What about my question of vendor supported vs. reverse engineered
> USB serial drivers.  If there is a chip with (better) vendor Linux
> driver support, I can complain to them and they can fix it instead
> of us.

So here's the problem with vendor supported drivers.  More often than
not, it will be for some distribution kernel, and not something which
can be contributed to the upstream kernel.  Very often, it was
whatever kernel version the last Major Customer of said vendor was
using, and the work was done by a contractor (or by an engineer who is
no longer with the vendor), and the driver can't be easily ported to
some other kernel.

You yourself mentioned using a 2.6.32 CentOS kernel.  The 2.6.32
kernel was released some four years ago, in 2010.  Most of the
community developers work on the upstream kernel; we have trouble
remembering what was in a kernel from your years ago.  (``In the
internet, two years is infinity'')

The more enlightened companies these days will insist that the
peripheral vendor will provide if there is a open source driver which
has been contributed to the upstream kernel.  That way, eventually all
of the distributions can get the same driver, and we don't have to
worry about multiple out-of-tree drivers only barely supported by the
vendors (because the contractor or the employee has moved on).

This works well if you are an IBM or a HP, or if you are Samsung or a
LG, and you can say, "we're only going to buy millions of dollars of
your serial chip / scsi controller / whatever if your driver gets
submitted upstream.

> If you were to buy a USB-to-Serial adapter, which chip would
> you buy?

So it's been a long time since I've had to use an RS-232 device (which
is why I stepped down as the serial maintainer years ago), but
speaking generally, what you want is a chip which is actively vendor
supported, and in the mainline source tree.  So if I were looking, I'd
probably do some looking at git commits for the USB serial drivers,
and see which ones seem to be actively maintained by someone who is
clearly either working for one of the vendors, or is actively being
supported by one the vendors.

Good luck,

					- Ted

Re: How can I help?

[Greg KH]

On Fri, Apr 18, 2014 at 03:37:53PM -0700, Larry Baker wrote:
> Greg,
> 
> Any suggestions how to trace what is going on?

Going on with what?  Was there a bug report in that wall of text that I
missed?  :)

> I see all the tracing debug code has been removed in the latest
> ftdi_sio driver.

That's because the kernel provides tracing functionality built-in, no
need for it to be on a driver-by-driver basis.  Use it if you think it
is necessary.

> I think it will be easier to first try upgrading the kernel/drivers on
> the AtomPC, even though I want to run on an ARM SoC.  The ARM SoC I'm
> currently using requires some ARM ERRATA in the kernel; exactly which
> ones still seems to be a moving target.  As you say, the drivers
> should be architecture neutral.  If a fix for this problem can be
> found using an AtomPC, I am confident it will also apply to an ARM
> SoC.
> 
> The drivers I'm looking at are all in the usb/serial branch of the
> Linux kernel.  You recommend discussing this on the
> linux-usb@vger.kernel.org mailing list instead of this one?  CC this
> one?
> 
> What about my question of vendor supported vs. reverse engineered USB
> serial drivers.  If there is a chip with (better) vendor Linux driver
> support, I can complain to them and they can fix it instead of us.  If
> you were to buy a USB-to-Serial adapter, which chip would you buy?

I would buy an io_edgeport device, but they are really expensive, and no
longer in production from what I can tell.  Everything else are very
cheap devices, with questionable UART silicon.

In the end, it all depends on what you want to use the device for, some
are much better than others.

thanks,

greg k-h

Re: How can I help?

[Larry Baker]

Greg,

Any suggestions how to trace what is going on?  I see all the tracing debug code has been removed in the latest ftdi_sio driver.

I think it will be easier to first try upgrading the kernel/drivers on the AtomPC, even though I want to run on an ARM SoC.  The ARM SoC I'm currently using requires some ARM ERRATA in the kernel; exactly which ones still seems to be a moving target.  As you say, the drivers should be architecture neutral.  If a fix for this problem can be found using an AtomPC, I am confident it will also apply to an ARM SoC.

The drivers I'm looking at are all in the usb/serial branch of the Linux kernel.  You recommend discussing this on the linux-usb@vger.kernel.org mailing list instead of this one?  CC this one?

What about my question of vendor supported vs. reverse engineered USB serial drivers.  If there is a chip with (better) vendor Linux driver support, I can complain to them and they can fix it instead of us.  If you were to buy a USB-to-Serial adapter, which chip would you buy?

Thank you for your prompt reply.

Larry Baker
US Geological Survey
650-329-5608
baker@usgs.gov



On 18 Apr 2014, at 3:09 PM, Greg KH wrote:

> On Fri, Apr 18, 2014 at 02:53:35PM -0700, Larry Baker wrote:
>> Greg and Johan (via linux-serial@vger.kernel.org),
>> 
>> I am hoping you will have some good advice for me, and maybe I can help you.
>> 
>> I am migrating a serial data acquisition app from a PC platform to a lower cost/lower power ARM SoC platform.  The data come from the instrument over an RS-232 link that requires RTS/CTS handshaking.  It also uses a packet-based handshaking protocol in software.  Serial ports with hardware handshaking have been difficult to find on the SoC platforms I'm experimenting with, so I bought a USB-to-Serial adapter.  The one I bought is identified as an FTDI FT232RL.  It sort of works.  What happens is, after about 8 or 9 hours, the RX to the TTY port times out.  Sometimes it happens in only an hour, other times it takes about 14 hours.  After about 10 seconds, it seems to recover.  Too late, though, for the application; it has already started its abort sequence by then.
>> 
>> On the ARM SoC platform (CompuLab Utilite, Linux utilite
>> 3.0.35-cm-fx6-5.5-rtscts-00002-g39389d3 #102 SMP Tue Apr 1 18:52:11
>> IDT 2014 armv7l armv7l armv7l GNU/Linux), the ftdi_sio driver is
>> v1.6.0. To eliminate the possibility it was a Linux ARM bug, I ran the
>> same setup on an AtomPC.  I got the same result.  The AtomPC runs
>> CentOS 6.5 (CompuLab fit-PC2i, Linux fit-pc2i.wr.usgs.gov
>> 2.6.32-431.5.1.el6.i686 #1 SMP Tue Feb 11 21:56:33 UTC 2014 i686 i686
>> i386 GNU/Linux), which has ftdi_sio driver v1.5.0.  I was blaming the
>> instrument, until I hooked it up to the COM1 port on the AtomPC and
>> everything ran fine.  (The full software handshaking protocol handling
>> in my app is new; the other mode of operation is the device
>> continuously streams its data.  This last test confirmed that my
>> software handshaking was not at fault.)
> 
> The driver "version" number means nothing when it is within the kernel
> tree itself.  Just go by the kernel version number instead.
> 
> All of the above listed kernels are very old, please try something
> "modern", like the 3.14 kernel release, as there is nothing we can do
> about these obsolete kernel versions, other than point you at the
> company that is forcing you to use them to get support from.
> 
>> I have seen people complain about problems using FTDI chips and
>> questions about ARM compatibility.  I think those were either pilot
>> error or have been fixed by now.  (Though, I was surprised by a very
>> recent post and a patch as a result to add parity support to one of
>> the non-FTDI drivers.  I would have taken parity support for granted
>> -- it seems pretty fundamental to serial data comm.)  The FTDI-based
>> adapters seem to be the ones that most people like.  I just ordered a
>> couple adapters with Prolific chips to try.
> 
> All of the usb-serial drivers should work on any platform that Linux
> works on, there should not be any issues.  If there are, please let us
> know.
> 
> And yes, some drivers aren't as "feature rich" as others are, that's
> just the way it goes.  Parity settings were never used in the majority
> of the places where the device you are referring to was used in
> (embedded systems talking to specific hardware devices.)  So it was
> never added, that doesn't mean the driver doesn't work well at all.
> 
>> If you would kindly point me to the correct web page or e-mail address
>> to participate in debugging the FTDI or other USB-to-Serial drivers, I
>> will help a much as I can.  I am suspicious, for example, that the CTS
>> grant is getting lost.  When data starts to flow again after the
>> timeout (there are shutdown commands being sent to the instrument), I
>> see correctly formed packets (the ones that timed out earlier -- they
>> were already in the device transmit queue); no runts, no checksum
>> errors.  That fits with the scenario that CTS is not always being seen
>> by the requester.  I don't have hardware like a logic analyzer; just
>> an RS-232 break-out box.  I plan to try wiring RTS to CTS on the
>> sender side, disable RTS/CTS flow control on the app side (I don't
>> know how much the device driver vs. the chip is involved in the
>> RTS/CTS handshaking; I want to disable any RTS/CTS state transitions
>> all the way out to the chip, if the driver will do that), and see if
>> the RX timeouts go away.  If so, that is a valuable data point and
>> gives me a place to start probing in the driver (assuming it is not
>> the chip itself that is the problem).
> 
> The linux-usb@vger.kernel.org place is usually the best for Linux USB
> serial driver issues, unless it is serial/tty core related, and then
> linux-serial like you used here should be fine.
> 
> thanks,
> 
> greg k-h

Re: How can I help?

[Greg KH]

On Fri, Apr 18, 2014 at 02:53:35PM -0700, Larry Baker wrote:
> Greg and Johan (via linux-serial@vger.kernel.org),
> 
> I am hoping you will have some good advice for me, and maybe I can help you.
> 
> I am migrating a serial data acquisition app from a PC platform to a lower cost/lower power ARM SoC platform.  The data come from the instrument over an RS-232 link that requires RTS/CTS handshaking.  It also uses a packet-based handshaking protocol in software.  Serial ports with hardware handshaking have been difficult to find on the SoC platforms I'm experimenting with, so I bought a USB-to-Serial adapter.  The one I bought is identified as an FTDI FT232RL.  It sort of works.  What happens is, after about 8 or 9 hours, the RX to the TTY port times out.  Sometimes it happens in only an hour, other times it takes about 14 hours.  After about 10 seconds, it seems to recover.  Too late, though, for the application; it has already started its abort sequence by then.
> 
> On the ARM SoC platform (CompuLab Utilite, Linux utilite
> 3.0.35-cm-fx6-5.5-rtscts-00002-g39389d3 #102 SMP Tue Apr 1 18:52:11
> IDT 2014 armv7l armv7l armv7l GNU/Linux), the ftdi_sio driver is
> v1.6.0. To eliminate the possibility it was a Linux ARM bug, I ran the
> same setup on an AtomPC.  I got the same result.  The AtomPC runs
> CentOS 6.5 (CompuLab fit-PC2i, Linux fit-pc2i.wr.usgs.gov
> 2.6.32-431.5.1.el6.i686 #1 SMP Tue Feb 11 21:56:33 UTC 2014 i686 i686
> i386 GNU/Linux), which has ftdi_sio driver v1.5.0.  I was blaming the
> instrument, until I hooked it up to the COM1 port on the AtomPC and
> everything ran fine.  (The full software handshaking protocol handling
> in my app is new; the other mode of operation is the device
> continuously streams its data.  This last test confirmed that my
> software handshaking was not at fault.)

The driver "version" number means nothing when it is within the kernel
tree itself.  Just go by the kernel version number instead.

All of the above listed kernels are very old, please try something
"modern", like the 3.14 kernel release, as there is nothing we can do
about these obsolete kernel versions, other than point you at the
company that is forcing you to use them to get support from.

> I have seen people complain about problems using FTDI chips and
> questions about ARM compatibility.  I think those were either pilot
> error or have been fixed by now.  (Though, I was surprised by a very
> recent post and a patch as a result to add parity support to one of
> the non-FTDI drivers.  I would have taken parity support for granted
> -- it seems pretty fundamental to serial data comm.)  The FTDI-based
> adapters seem to be the ones that most people like.  I just ordered a
> couple adapters with Prolific chips to try.

All of the usb-serial drivers should work on any platform that Linux
works on, there should not be any issues.  If there are, please let us
know.

And yes, some drivers aren't as "feature rich" as others are, that's
just the way it goes.  Parity settings were never used in the majority
of the places where the device you are referring to was used in
(embedded systems talking to specific hardware devices.)  So it was
never added, that doesn't mean the driver doesn't work well at all.

> If you would kindly point me to the correct web page or e-mail address
> to participate in debugging the FTDI or other USB-to-Serial drivers, I
> will help a much as I can.  I am suspicious, for example, that the CTS
> grant is getting lost.  When data starts to flow again after the
> timeout (there are shutdown commands being sent to the instrument), I
> see correctly formed packets (the ones that timed out earlier -- they
> were already in the device transmit queue); no runts, no checksum
> errors.  That fits with the scenario that CTS is not always being seen
> by the requester.  I don't have hardware like a logic analyzer; just
> an RS-232 break-out box.  I plan to try wiring RTS to CTS on the
> sender side, disable RTS/CTS flow control on the app side (I don't
> know how much the device driver vs. the chip is involved in the
> RTS/CTS handshaking; I want to disable any RTS/CTS state transitions
> all the way out to the chip, if the driver will do that), and see if
> the RX timeouts go away.  If so, that is a valuable data point and
> gives me a place to start probing in the driver (assuming it is not
> the chip itself that is the problem).

The linux-usb@vger.kernel.org place is usually the best for Linux USB
serial driver issues, unless it is serial/tty core related, and then
linux-serial like you used here should be fine.

thanks,

greg k-h

How can I help?

[Larry Baker]

Greg and Johan (via linux-serial@vger.kernel.org),

I am hoping you will have some good advice for me, and maybe I can help you.

I am migrating a serial data acquisition app from a PC platform to a lower cost/lower power ARM SoC platform.  The data come from the instrument over an RS-232 link that requires RTS/CTS handshaking.  It also uses a packet-based handshaking protocol in software.  Serial ports with hardware handshaking have been difficult to find on the SoC platforms I'm experimenting with, so I bought a USB-to-Serial adapter.  The one I bought is identified as an FTDI FT232RL.  It sort of works.  What happens is, after about 8 or 9 hours, the RX to the TTY port times out.  Sometimes it happens in only an hour, other times it takes about 14 hours.  After about 10 seconds, it seems to recover.  Too late, though, for the application; it has already started its abort sequence by then.

On the ARM SoC platform (CompuLab Utilite, Linux utilite 3.0.35-cm-fx6-5.5-rtscts-00002-g39389d3 #102 SMP Tue Apr 1 18:52:11 IDT 2014 armv7l armv7l armv7l GNU/Linux), the ftdi_sio driver is v1.6.0. To eliminate the possibility it was a Linux ARM bug, I ran the same setup on an AtomPC.  I got the same result.  The AtomPC runs CentOS 6.5 (CompuLab fit-PC2i, Linux fit-pc2i.wr.usgs.gov 2.6.32-431.5.1.el6.i686 #1 SMP Tue Feb 11 21:56:33 UTC 2014 i686 i686 i386 GNU/Linux), which has ftdi_sio driver v1.5.0.  I was blaming the instrument, until I hooked it up to the COM1 port on the AtomPC and everything ran fine.  (The full software handshaking protocol handling in my app is new; the other mode of operation is the device continuously streams its data.  This last test confirmed that my software ha
 ndshaking was not at fault.)

I have seen people complain about problems using FTDI chips and questions about ARM compatibility.  I think those were either pilot error or have been fixed by now.  (Though, I was surprised by a very recent post and a patch as a result to add parity support to one of the non-FTDI drivers.  I would have taken parity support for granted -- it seems pretty fundamental to serial data comm.)  The FTDI-based adapters seem to be the ones that most people like.  I just ordered a couple adapters with Prolific chips to try.

Your names are at the top of the ftdi_sio driver source in the latest Linux kernel source tree, and I've seen Greg's name in several e-mail exchanges about Linux USB-to-Serial drivers, e.g., the Prolific driver.  I'd like your advice about the best choice of chip on Linux.  From what Greg has written, some of the drivers were written based on reverse engineering of the vendor's Windows drivers and trial-and-error.  Are there any Linux drivers supplied by and/or supported by the chip vendors?  Are those any better?  I've had the general impression that FTDI is supposed to be a good choice.  If that it what you would recommend, then I should help you figure out how to fix the bug I'm running into.  I see the debugging code removed in the latest Linux ftdi_sio drivers.  But, I think the v1.5.
 0 and v1.6.0 drivers still support debugging.  I could enable that and hope my little 4GB SD card rootfs does not overflow.  I also added a TX/RX packet header capture queue with time-stamps to my app.  When I get a failure, I print out the last 20 TTY transactions.

If you would kindly point me to the correct web page or e-mail address to participate in debugging the FTDI or other USB-to-Serial drivers, I will help a much as I can.  I am suspicious, for example, that the CTS grant is getting lost.  When data starts to flow again after the timeout (there are shutdown commands being sent to the instrument), I see correctly formed packets (the ones that timed out earlier -- they were already in the device transmit queue); no runts, no checksum errors.  That fits with the scenario that CTS is not always being seen by the requester.  I don't have hardware like a logic analyzer; just an RS-232 break-out box.  I plan to try wiring RTS to CTS on the sender side, disable RTS/CTS flow control on the app side (I don't know how much the device driver vs. the chip
  is involved in the RTS/CTS handshaking; I want to disable any RTS/CTS state transitions all the way out to the chip, if the driver will do that), and see if the RX timeouts go away.  If so, that is a valuable data point and gives me a place to start probing in the driver (assuming it is not the chip itself that is the problem).

Thank you in advance for your advice.

Larry Baker
US Geological Survey
650-329-5608
baker@usgs.gov