Trouble using 4 of the 8 RS232 onboard ports of an x86 SBC

Hello,

We're using an Acrosser AR-B5495 mini-itx Atom board with 8 RS232 ports with debian lenny, vanilla debian kernel, 2.6.26-2-686 #1 SMP Thu Nov 25 01:53:57 UTC 2010 i686 GNU/Linux.

All serial ports are correctly recognized & configured by setserial, and the addrs/irqs match the ones set on the BIOS:

sat2000a:~# dmesg | egrep "Serial|tty"
[ 0.004000] console [tty0] enabled
[ 1.552822] Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing enabled
[ 1.552823] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 1.552823] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
[ 1.552823] serial8250: ttyS2 at I/O 0x3e8 (irq = 4) is a 16550A
[ 1.552823] serial8250: ttyS3 at I/O 0x2e8 (irq = 3) is a 16550A
[ 1.556834] 00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 1.556846] 00:07: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
[ 1.556846] 00:08: ttyS2 at I/O 0x3e8 (irq = 4) is a 16550A
[ 1.556846] 00:09: ttyS3 at I/O 0x2e8 (irq = 3) is a 16550A
[ 1.556846] 00:0a: ttyS4 at I/O 0x2f0 (irq = 11) is a 16550A
[ 1.556846] 00:0b: ttyS5 at I/O 0x2e0 (irq = 10) is a 16550A
[ 1.556846] 00:0c: ttyS6 at I/O 0x2d8 (irq = 11) is a 16550A
[ 1.556846] 00:0d: ttyS7 at I/O 0x2d0 (irq = 10) is a 16550A


sat2000a:~# setserial -g /dev/ttyS*
/dev/ttyS0, UART: 16550A, Port: 0x03f8, IRQ: 4
/dev/ttyS1, UART: 16550A, Port: 0x02f8, IRQ: 3
/dev/ttyS2, UART: 16550A, Port: 0x03e8, IRQ: 4
/dev/ttyS3, UART: 16550A, Port: 0x02e8, IRQ: 3
/dev/ttyS4, UART: 16550A, Port: 0x02f0, IRQ: 11
/dev/ttyS5, UART: 16550A, Port: 0x02e0, IRQ: 10
/dev/ttyS6, UART: 16550A, Port: 0x02d8, IRQ: 11
/dev/ttyS7, UART: 16550A, Port: 0x02d0, IRQ: 10
sat2000a:~# cat /proc/tty/driver/serial
serinfo:1.0 driver revision:
0: uart:16550A port:000003F8 irq:4 tx:0 rx:450323 fe:188
1: uart:16550A port:000002F8 irq:3 tx:0 rx:51604
2: uart:16550A port:000003E8 irq:4 tx:0 rx:0
3: uart:16550A port:000002E8 irq:3 tx:0 rx:0
4: uart:16550A port:000002F0 irq:11 tx:0 rx:0
5: uart:16550A port:000002E0 irq:10 tx:362520 rx:688087
6: uart:16550A port:000002D8 irq:11 tx:0 rx:0
7: uart:16550A port:000002D0 irq:10 tx:0 rx:0

Nevertheless, ttyS2, 3, 6 and 7 do not work. /proc/tty/driver/serial shows tx: bytes when we write to the port, but nothing comes out on the RS232 connector, and rx is always 0.

Although we only need 4 of the RS232 for now, we're getting poor performance on these (slow, sw hangs), and the reason is probably the same; which I suppose is related to IRQ sharing (as these are the ports using the irqs for the "second time"). Making the sw which is using the 4 working ports hang is specially easy to reproduce by opening one of the non-working ports with minicom, for example - that, I think points to the irq sharing.

Using ports ttyS0, 1, 4, and 5 works (with the occasional sw hang, and slowliness), and using USB-RS232 dongles works perfectly (so our sw and attached communicating hardware are apparently not to blame).

The standard debian kernel seems to have the necessary config options (from /boot/config-2.6.26-2-686):
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_CS=m
CONFIG_SERIAL_8250_NR_UARTS=32
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_FOURPORT=m
CONFIG_SERIAL_8250_ACCENT=m
CONFIG_SERIAL_8250_BOCA=m
CONFIG_SERIAL_8250_EXAR_ST16C554=m
CONFIG_SERIAL_8250_HUB6=m
CONFIG_SERIAL_8250_SHARE_IRQ=y
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
CONFIG_SERIAL_8250_RSA=y

Is anyone using multiple onboard RS232 ports with linux? Is there any additional magic required, or can someone hint me where to look for debugging this?

I'm specially worried that this may be a linux glitch, as we'll soon integrate 10 systems with 8-port serial cards (pcie, startech PEX8S952, 16C950 UART; haven't arrived at our shop yet).

Thanks!
Joao S Veiga

This might be useful. It seems to indicate that either you need to configure CONFIG_SERIAL_8250_RUNTIME_UARTS to 8 or set the module or boot parameter.

Thank you;
I do have 8250.nr_uarts=8 in the boot kernel parms. Apart from that, the only other parm is ro (the flash ide disk is mounted read only, required rw stuff is in a ram fs).
Sorry, I did not include this in the original post.
Joao S Veiga

In that case, my only suggestion would be to try a later kernel. I checked the kernel mailing list, and there have been many patches and new chipsets supported in the three years since 2.6.26 was introduced. I don't know what (if anything) has been backported in the kernel you're using.

Did it ever work correctly? Can you use some live media to re-check or double check that it should work?

Are all the ports fully wired in? Sometimes rs-232 lines are not fully wired.

In a very odd case, there is actually a second port on each connection. Might have to look into that. I know it is there but, I know of no instance where it was ever used.

Thank you macemoneta,
I'll try a debian backport of a newer kernel, or as jefro suggested, a fresh kubuntu on Monday (the hardware is at the office); I did not expect the need of the newest kernel, because RS232 16550 UART support should be older than all of us, but I'll try it. When the project started, lenny was the stable release.

Thank you jefro,
No, it never worked. We've been developing the building blocks of the application for a while, but only now we tried to make all the hardware to work together (a GPS, an electronic compass, a touchscreen LCD, and an IO board, all connected to the mobo via RS232). Then we found only ttyS0,1,4,5 were working.

We also tried disabling 0 and 1 in the BIOS settings, but 2 and 3 still wouldn't work. We also tried assigning IRQ 4 to 0 and 1, but it didn't help.

ttyS0 has its own DB9 connector in the mobo. The others have connectors in the motherboard which goes to a connector-flatcable-db9 kit, supplied by the mobo manufacturer. The pinouts of the 7 connectors are the same, and we swapped cables around.

Also tried changing the mobo config jumpers to switch the RS232 levels from 12V to TTL.

Also could not find a support contact email on Acrosser's web site; we emailed the Acrosser USA salesperson who sold us the sample asking for a support contact, but got no response

We'll use a bunch of USB-RS232 dongles to move on with testing, but I'd really like to know if this is a problem with this mobo model (which can be easily solved by moving to another one), or a linux kernel support problem (which will be a nightmare, because as I mentioned, we're waiting for the arrival of 10 8-port pcie cards to be used on another project with linux, with a tight schedule!).

best regards,

Joao S Veiga

Hello, here's an update:

I booted on debian testing (kernel 2.6.39), where I had to add acpi=off (or it wouldn't boot, which resulted in seeing the atom N270 as a single pipe CPU, but this is unrelated).

I did not go further with this test for now, because I found the cause of part of the problem:

ttyS2, 3, 6, 7 were not working because they are in the wrong connectors.

What the motherboard labels and motherboard manual show as COM3 and COM4 are actually COM7 and 8, and what they show as COM7 and 8 are COM3 and 4. The kernel correctly ids the ttyS0-7 in the same sequence/addr/irqs configured in the BIOS for COM1-8, but the physical connectors in the motherboard have wrong identifications! It's not a matter of having the labels in a dubious position between connectors; they are really misidentified.

We still have the lockups though (program works fine with USB-RS232 dongles, but very slow/locks with the native RS232 ports). We're trying different acpi parameters for the kernel (still on lenny), and will check if the problem persists. Then on to 2.6.39, and if we still have the problems, I'll open another thread, as the problem from the subject of this one is already solved.

Thanks!

Joao S Veiga

Hi,

I'm browsing my old posts, and found out I never posted the final of this. Here it is, if someone has the same troubles.

Besides being misidentified in the motherboard, we ended up finding out that some ports "interfere" with others.

I don't know if this is a hardware problem on our mobo, or a design flaw, but depending on the port "pair", if you open both ports on minicom, getting data on one terminal makes the other one jump a line. For example, connecting a NMEA (GPS) device in one port and looking at the ascii output in both minicoms produces one empty line on the "empty" minicom for each NMEA line received and shown in the correct minicom.

I did not investigate further, but it looks like when the data is ready to be read in one port, the signaling/interrupt shows up for two ports. I ended up selecting the "right" port combination for my application (which uses only 3 out of the board's 8 ports).

(the debian on the board is now squeeze)

BR,

Joao S Veiga