[SOLVED] Python serial and struct on 64 bit

CoderMan · 05-14-2010, 08:53 PM

For a work project, I've got a bunch of python code from about a year ago that controls the movement of our EVI-D30 camera over a ttyUSB connection. It used to work fine on a 32-bit Fedora box, but recently we moved our whole project over to a 64-bit Gentoo server, and the same code seems to be worthless on the new platform. I didn't write the code, so I'm have trouble figuring out how to fix it.

Error messages usually look like this:

Code:

  File "./CameraController.py", line 172, in pan
    turn_callback(cmdStruct[0], cmdStruct[1])
  File "./CameraController.py", line 147, in turn_callback
    cameras[camera].TiltUp()
  File "/rcdp/developers/choward/rcdp-repo/CameraController/VISCA.py", line 377, in TiltUp
    self.moveRelative(0,self.movement_quanta)
  File "/rcdp/developers/choward/rcdp-repo/CameraController/VISCA.py", line 405, in moveRelative
    tmp = self.getPosition()
  File "/rcdp/developers/choward/rcdp-repo/CameraController/VISCA.py", line 176, in getPosition
    x = self._convert_position(x)
  File "/rcdp/developers/choward/rcdp-repo/CameraController/VISCA.py", line 437, in _convert_position
    x = struct.unpack('4b',x)
error: unpack requires a string argument of length 4

I don't want to post all the source code (there is quite a bit of it), but the program imports the "serial" module for communication with the camera, and the "struct" module for packing/unpacking bytes sent along the serial connection. But here is the offending code mentioned:

Code:

    def getPosition(self):
        """Returns Current Camera Angles (x,y)"""
        command = struct.pack('B',0x12)
        line = self._send_move_inq(command)

        line = line[2:-1]

        x = line[:4]
        y = line[4:]

        x = self._convert_position(x) # line 176
        y = self._convert_position(y)

        return (int((x/862.0)*90),int((y/280.0)*30))

Code:

    def _convert_position(self,x):
        """turn a struct from the serial port to an integer"""
        x = struct.unpack('4b',x) # line 437

        ## account for negative 16 bit bit-patterns!                                                                                                         
        tmp = ''
        for i in range(0,4):
            tmp = tmp +  hex(x[i])[2]

        tmp = int(tmp,16)
        tmp = (tmp & 32767) - (tmp  & 32768)
        return tmp

Anyway, I was hoping that perhaps somebody who understood how these modules worked ("serial" and "struct") might also have some insight into how they would be affected by a switch to a 64-bit platform.

Sergei Steshenko · 05-14-2010, 09:32 PM

Quote:

Originally Posted by CoderMan

...
Anyway, I was hoping that perhaps somebody who understood how these modules worked ("serial" and "struct") might also have some insight into how they would be affected by a switch to a 64-bit platform.

First of all, before making guesses about 32 -> 64 switch, why won't you simply print the offending 'x' in a manner that clearly shows what its length is ?

I am not a Python guy (a Perl one), but I would do the suggested action myself in order to see what "semantically" is wrong with the input argument.

I have a suspicion that it's something else (than 32 -> 64 switch), but I do not want to make my (stupid) guesses public until I see actual input data.

CoderMan · 05-15-2010, 02:14 AM

Quote:

Originally Posted by Sergei Steshenko

First of all, before making guesses about 32 -> 64 switch, why won't you simply print the offending 'x' in a manner that clearly shows what its length is ?

I am not a Python guy (a Perl one), but I would do the suggested action myself in order to see what "semantically" is wrong with the input argument.

I have a suspicion that it's something else (than 32 -> 64 switch), but I do not want to make my (stupid) guesses public until I see actual input data.

Actually, I did. It printed nothing. 'x' was empty. I couldn't figure out anything from that, so I didn't mention it, though in retrospect I should have.

I tried tracing the input back through _send_move_inqury. I don't have the code with me now (it is the weekend), but I remember it ultimately went back to a read command from the serial module.

Sergei Steshenko · 05-15-2010, 10:25 AM

Quote:

Originally Posted by CoderMan

Actually, I did. It printed nothing. 'x' was empty. I couldn't figure out anything from that, so I didn't mention it, though in retrospect I should have.
...

So you withheld the most important piece of info.

The code fails correctly - input item length is 0 though it is expected to be 4. You need to first get the right data.

...

On a second thought - how did you print 'x' ? I mean what looked empty to you could be 100 whitespaces. Anyway, the point remains - the program fails because it gets wrong data from your device, and that should be debugged first.

smeezekitty · 05-15-2010, 11:22 AM

Although python sucks, i thought it was intended to be portable(?)
I'd say python is a major fail.

Sergei Steshenko · 05-15-2010, 11:37 AM

Quote:

Originally Posted by smeezekitty

Although python sucks, i thought it was intended to be portable(?)
I'd say python is a major fail.

You can't know. Probably the driver lower level is written in "C" in the first place. And if you look into the already published code, you'll see constants like 32767 and 32768 which are clearly related to 16 bits world and tell about the programmer much more than about the language.

No, don't get me wrong - I'm still much more fond of Perl than of Python, and so far I have no need to write in Python, but in this case there is yet no evidence Python is at fault.

grail · 05-15-2010, 09:40 PM

I agree with Sergei that you need to find out how the value of 'x' is being created.

If it is indeed empty, you would need to traceback through this line - line = self._send_move_inq(command)
As it is the only non-standard part of the function - def getPosition(self):
prior to the error.

CoderMan · 05-16-2010, 12:23 AM

Quote:

Originally Posted by Sergei Steshenko

So you withheld the most important piece of info.

The code fails correctly - input item length is 0 though it is expected to be 4. You need to first get the right data.

...

On a second thought - how did you print 'x' ? I mean what looked empty to you could be 100 whitespaces. Anyway, the point remains - the program fails because it gets wrong data from your device, and that should be debugged first.

Well, to be honest, that is the other reason I didn't mention it... because I really wasn't sure it was empty. Like you say... I printed it out, but for all I know it could have been invisible escape characters.

It looks like this thread isn't going to go much further until I have time to do an in-depth back-trace analysis, which was what I was hoping to avoid, since I'm not familiar with the software. Well, thanks anyway.

grail · 05-16-2010, 04:52 AM

Well I think you gave up a little easy, but that's okay. I would make a suggestion that if you are unsure about the output, maybe place delimeters either side
and this will see if whitespace present.
Also, are you perhaps able to show code for the function I mentioned? Might help the Python gurus.

Sergei Steshenko · 05-17-2010, 07:04 AM

Quote:

Originally Posted by CoderMan

Well, to be honest, that is the other reason I didn't mention it... because I really wasn't sure it was empty.
...

As another poster has already mentioned, use a delimiter.

In Perl I would use something like this:

Code:

warn "\$x:$x:";

- in this case the two ':' being the delimiter. I don't know the equivalent syntax in Python.

CoderMan · 05-17-2010, 01:15 PM

Quote:

Originally Posted by grail

I agree with Sergei that you need to find out how the value of 'x' is being created.

If it is indeed empty, you would need to traceback through this line - line = self._send_move_inq(command)
As it is the only non-standard part of the function - def getPosition(self):
prior to the error.

Code:

    def _send_move_inq(self,cmd):
        "Pedastal inquiry"
        self.sport.write(
            self._header +
            self._mode_info +
            self._category_move +
            cmd +
            self._terminator)
        return self._read()

Code:

        # set up the serial port
                                                    
        self.sport = serial.Serial(self.port,
                                   baudrate = 9600,
                                   bytesize = serial.EIGHTBITS,
                                   parity = serial.PARITY_NONE,
                                   stopbits = serial.STOPBITS_ONE,
                                   timeout = 0.1,
                                   xonxoff = 0,
                                   rtscts = 1)

Code:

        self.port = portName
        self.daisychain = cameraNumber

        self.daisychain = int(self.daisychain)

        # Header, which camera we're talking to                                                                                                              
        self._header = struct.pack('B',0x80+self.daisychain)

        #modes                                                                                                                                               
        self._mode_control = struct.pack('B',0x01)
        self._mode_info    = struct.pack('B',0x09)
        self._mode_net     = struct.pack('B',0x00)

        #categorys (not rigidly defined in spec                                                                                                              
        self._category_move = struct.pack('B',0x06)
        self._category_cam = struct.pack('B',0x04)

        # End of command                                                                                                                                     
        self._terminator = struct.pack('B',0xFF)

Code:

    def _read(self):
        return self.sport.read(30) + self.sport.read(30)

CoderMan · 05-17-2010, 01:29 PM

If I add

Code:

print "length of line = " + str(len(line))

and

Code:

print "length of x = " + str(len(x))

I get this during program output:

Code:

length of line = 0
length of x = 0

grail · 05-17-2010, 09:22 PM

Code:

self.sport.write(
            self._header +
            self._mode_info +
            self._category_move +
            cmd +
            self._terminator)

This creates an item of size 5 but you are unpacking one of size 4 with command:

Code:

x = struct.unpack('4b',x)

CoderMan · 06-02-2010, 01:33 PM

Further testing suggests that there is nothing wrong with the camera control library itself, but rather there was something wrong with the code employing the library. I rewrote the wrapper code, and everything seems to be working fine.