[SOLVED] TCL AND operator (&) on a binary value

jlinkels · 05-01-2010, 09:20 PM

I have a binary value which I receive from a controller. Say this binary value is 42. Just plain hex 42. If you would look at that byte in a debugger you would see 42.

Now this value hold 8 bits each indicating a high or a low output. So 0x42 = 01000010b. Which means bits 1 and 6 are '1'.

When I would want to find out which bits are set and which are not in a language like C, I simply do:

Code:

mask = 0x80;
if (binval & mask) {...}
etc..

However I am programming in TCL, and I try to do:

Code:

set mask 0x80
if { [expr ($binval & $mask) > 0] } {...}
etc...

this fails. At the moment the expr is executed, $binval is evaluated and substituted so the expression I am evaluating is

Code:

set mask 0x80
if { [expr ('B' & '0x80') > 0] } {...}
etc...

Eventually I got it working by converting the $binval into a '0x42' string value, like this:

Code:

binary scan $binval c byte
set byte [format "0x%0x" $byte]
set mask 0x80
if { [expr ($byte & $mask) > 0] } {...}
etc...[

Then the expression yields what I want. But this seems so stupid and clumsy. Isn't there a better way where I can compare two binary values without conversions?

jlinkels

Sergei Steshenko · 05-01-2010, 09:31 PM

Quote:

Originally Posted by jlinkels

I have a binary value which I receive from a controller. Say this binary value is 42. Just plain hex 42. If you would look at that byte in a debugger you would see 42.

Now this value hold 8 bits each indicating a high or a low output. So 0x42 = 01000010b. Which means bits 1 and 6 are '1'.

When I would want to find out which bits are set and which are not in a language like C, I simply do:

Code:

mask = 0x80;
if (binval & mask) {...}
etc..

However I am programming in TCL, and I try to do:

Code:

set mask 0x80
if { [expr ($binval & $mask) > 0] } {...}
etc...

this fails. At the moment the expr is executed, $binval is evaluated and substituted so the expression I am evaluating is

Code:

set mask 0x80
if { [expr ('B' & '0x80') > 0] } {...}
etc...

Eventually I got it working by converting the $binval into a '0x42' string value, like this:

Code:

binary scan $binval c byte
set byte [format "0x%0x" $byte]
set mask 0x80
if { [expr ($byte & $mask) > 0] } {...}
etc...[

Then the expression yields what I want. But this seems so stupid and clumsy. Isn't there a better way where I can compare two binary values without conversions?

jlinkels

http://wiki.tcl.tk/17464 ?

jlinkels · 05-02-2010, 07:08 AM

Sergei, on that page I don't see anything which is remotely connected to my question/problem. What paragraph on that page should explain on this?

Maybe it is the wrong page you are referring to, or my question was completley unclear?

My problem is not the '&' operator as such, but that the expression evaluates the string value of my byte, not the binary value. And now I create a string representation of my byte in order to perform the operation.

jlinkels

Sergei Steshenko · 05-02-2010, 10:16 AM

Quote:

Originally Posted by jlinkels

Sergei, on that page I don't see anything which is remotely connected to my question/problem. What paragraph on that page should explain on this?

Maybe it is the wrong page you are referring to, or my question was completley unclear?

My problem is not the '&' operator as such, but that the expression evaluates the string value of my byte, not the binary value. And now I create a string representation of my byte in order to perform the operation.

jlinkels

The page contains a link to tcl::mathop page: http://wiki.tcl.tk/20280 and on it, among other things:

Code:

tcl::mathop is a Tcl 8.5 namespace  that contains commands equivalent to the expr operators. These commands are all bytecode compiled. Note that changing or adding to these commands does not  change the expr operators themselves.
...
Bitwise and	 &	 -1	 a	 a&b	 ((a&b)&c)&...

I.e. it looks like using tcl::mathop it is possible to conduct bitwise operation directly.

...

Examples: http://wiki.tcl.tk/20258

jlinkels · 05-02-2010, 07:37 PM

Not really. It does implement a '&' operator you can use without expression, but if an operand contains a binary value it still fails. Look at this example below:

Code:

% set a [binary format c 0x42]
B
% puts [& $a 0xfe]  #fails because $a is evaluated to 'B'
can't use non-numeric string as operand of "&"
% binary scan $a H2 b
1
% puts $b
42                   #makes decimal 42
% puts [& $b 0x40]
0                    #not the intention
% puts [& 0x$b 0x40] #so we put '0x'in front of b
64                   #0x42 & 0x40 = 0x40 = 64d, correct
% binary scan $a c b #convert to decimal
1
% puts $b
66                   #0x42 = 66d
% puts [& $b 0x40]
64                   #also correct
%

The use of [& $a $b] instead of [expr $a & $b] is irrelevant for the example, it is the same.
There is no need to format a hex string from a binary value as I did in my original post. Just I have to convert the binary value to a representation TCL understands.
It is a *bit* weird, but not so hard to understand anymore. There are more counter-intuitive things you can do with TCL.

Eventually I think this is the normal way to perform a bit-wise AND in TCL, but if there are more suggestions I am happy to try them.

Sergei, thanks for you replies as well.

jlinkels

Sergei Steshenko · 05-02-2010, 08:03 PM

Quote:

Originally Posted by jlinkels

Not really. It does implement a '&' operator you can use without expression, but if an operand contains a binary value it still fails. Look at this example below:

Code:

% set a [binary format c 0x42]
B
% puts [& $a 0xfe]  #fails because $a is evaluated to 'B'
can't use non-numeric string as operand of "&"
% binary scan $a H2 b
1
% puts $b
42                   #makes decimal 42
% puts [& $b 0x40]
0                    #not the intention
% puts [& 0x$b 0x40] #so we put '0x'in front of b
64                   #0x42 & 0x40 = 0x40 = 64d, correct
% binary scan $a c b #convert to decimal
1
% puts $b
66                   #0x42 = 66d
% puts [& $b 0x40]
64                   #also correct
%

The use of [& $a $b] instead of [expr $a & $b] is irrelevant for the example, it is the same.
There is no need to format a hex string from a binary value as I did in my original post. Just I have to convert the binary value to a representation TCL understands.
It is a *bit* weird, but not so hard to understand anymore. There are more counter-intuitive things you can do with TCL.

Eventually I think this is the normal way to perform a bit-wise AND in TCL, but if there are more suggestions I am happy to try them.

Sergei, thanks for you replies as well.

jlinkels

My golden rule is not to program in TCL whenever possible. If a tool supports only TCL, I am using TCL to only get values and set values, all the programming logic and calculations I implement in Perl. Typically I organize connection from Perl to the tool (with its TCL) through pipe, and from the tool to the Perl part trough log file.

Substitute Perl with your favorite normal programming/scripting language.

jlinkels · 05-02-2010, 09:39 PM

Quote:

Originally Posted by Sergei Steshenko

Substitute Perl with your favorite normal programming/scripting language.

Frankly, it is TCL, so what have we got, an infinite substitution loop?

I used to use Tcl/Tk for implementing GUI programs extremely fast. But there is Perl/Tk as well so that is no excuse.

For this project one of my devices used Telnet communication so Expect was the logical choice. Then another Telnet device. Then some SNMP devices. But the snmp stack in PHP (the main language of this project) is broken so to use it you have to to execute net-snmp thru a shell. Since I have most of the framework for device communication set up in Tcl, why not spawning net-snmp instead of Telnet and write the driver in Tcl?

Eventually, I have set up all my drivers for the various devices in Tcl. Only this last program addresses a new device which speaks Modbus. I have been programming using binary communication over sockets before, but I never had to do bit-wise operations, and that was where I got stuck now.

jlinkels

Sergei Steshenko · 05-02-2010, 09:52 PM

Quote:

Originally Posted by jlinkels

Frankly, it is TCL, so what have we got, an infinite substitution loop?

I used to use Tcl/Tk for implementing GUI programs extremely fast. But there is Perl/Tk as well so that is no excuse.

For this project one of my devices used Telnet communication so Expect was the logical choice. Then another Telnet device. Then some SNMP devices. But the snmp stack in PHP (the main language of this project) is broken so to use it you have to to execute net-snmp thru a shell. Since I have most of the framework for device communication set up in Tcl, why not spawning net-snmp instead of Telnet and write the driver in Tcl?

Eventually, I have set up all my drivers for the various devices in Tcl. Only this last program addresses a new device which speaks Modbus. I have been programming using binary communication over sockets before, but I never had to do bit-wise operations, and that was where I got stuck now.

jlinkels

Well, TCL is an objectively bad language because it is (used to be ?) a true source code interpreter. So, syntax errors are not found until containing them code is executed.

For example, in VLSI synthesis may take several days, and it's quite a pity if the whole process ultimately fails because of a silly syntax error at the script end.

That's why static TCL checkers have been invented ...

And TCL error messages sometimes are (used to be ?) very misleading.

Regarding GUI stuff - not only there are Tk Perl bindings, but also gtk+ ones.

So, my point is not Perl specifically, but any decent "scripting" language which first compiles and only then executes, and with more familiar expressions and stuff syntax.

salasi · 05-03-2010, 12:27 PM

Warning: IANAE, but I am someone with an ancient book on a related subject.

Quote:

Originally Posted by jlinkels

I have a binary value which I receive from a controller. Say this binary value is 42. Just plain hex 42. If you would look at that byte in a debugger you would see 42.

Now this value hold 8 bits each indicating a high or a low output. So 0x42 = 01000010b. Which means bits 1 and 6 are '1'.

Quote:

Frequently made mistake: Be careful when formatting numbers with leading zeros in Tcl. A "feature" of Tcl since its inception assumes that a number with leading zeros is written in octal...Note that there is no inherent problem with leading zeroes as long as you don't try to do arithmetic with them.

Handbook of Programming Languages, Vol III, LL&T, P 363

err, helpful, or what?

jlinkels · 05-03-2010, 06:17 PM

That is right. A number with a leading zero is considered octal, but a number with a leading '0x' is considered hex. That is the only way to enter hex numbers.

It could be a pitfall sometimes, so thanks for the contribution.

jlinkels

salasi · 05-04-2010, 12:16 PM

Quote:

a number with a leading '0x' is considered hex

that wasn't apparent from the book that I had, and does make some more sense.

Tcl has rather more of these features than is strictly comfortable, doesn't it?

Quote:

Note that the octal and hexadecimal conversion takes place differently in the expr command than in the Tcl substitution phase. In the substitution phase, a \x32 would be converted to an ascii "2", while expr would convert 0x32 to a decimal 50.

some of these things seem like devious ways of hiding obfuscated code, but, in practice, more likely to contribute to buggy code. Ugh!

jlinkels · 05-04-2010, 10:01 PM

Quote:

Originally Posted by salasi

Tcl has rather more of these features than is strictly comfortable, doesn't it?

It has some, but I found the rules not hard to learn. I hardly make mistakes, and writing TCL code is surprisingly bug free. Lately I wrote a TCL program controlling some hardware. It mainly consisted of a very complicated state machine with 12 states and 4 different ways to proceed thru those states with various bailouts at numerous states. It also included SNMP communication and error handling and MySQL database access. The state transition conditions were defined in arrays of lists and many states share the same functions, but with other parameters from this state transition matrix. The program comprises 600+ lines of TCL code. When I started testing, I used half a day to clean up the syntax errors flagged by the compiler, and then I solved 2 (two) bugs in a program of 600+ lines code which amazes me until today. I am far from a genious programmer, but generally I spend much less time debugging TCL programs than PHP. The absence of errors was not typical in this program, it happens more often in my TCL programs.

Quote:

Originally Posted by salasi

Quote:Note that the octal and hexadecimal conversion takes place differently in the expr command than in the Tcl substitution phase. In the substitution phase, a \x32 would be converted to an ascii "2", while expr would convert 0x32 to a decimal 50
some of these things seem like devious ways of hiding obfuscated code, but, in practice, more likely to contribute to buggy code. Ugh!

This is not really a good example. \x32 is simply different from 0x32 and it has not anything to do with substitution.

Code:

% set a 0x32
0x32
% puts [expr $a]
50
% set a \x32
2
% puts [expr $a]
2

If you look at this site: http://wiki.tcl.tk you see numerous examples of the extreme incomprehensibility and obfuscation of TCL. It looks as if the contributors stretch TCL to the extremes of the specification to show how unexpected TCL can operate. But it might not be the best way to use TCL. C is at least as obfuscatble as TCL but less powerful in terms of needed statements per action.

Not that it is all good with TCL. Use the MySQL API and you'll surprised about the {} hell you enter when trying to access query results when columns containing NULL have to be processed. Or try to exec a program with parameters. But both examples require only an understanding of the language, once understood it doesn't necessarily lead to bugs.

jlinkels

salasi · 05-05-2010, 02:44 AM

Sorry, I get the impression that I am continuing this thread further than makes any sense for you, or your original problem, but:

Quote:

Originally Posted by jlinkels

This is not really a good example. \x32 is simply different from 0x32 and it has not anything to do with substitution.

Code:

% set a 0x32
0x32
% puts [expr $a]
50
% set a \x32
2
% puts [expr $a]
2

If you look at this site: http://wiki.tcl.tk you see numerous examples of the extreme incomprehensibility and obfuscation of TCL.

What I should have made clear (and this was an oversight on my part) was that this was not originally my example, but came from here. Of course, some examples of documentation are better than others...

Quote:

It looks as if the contributors stretch TCL to the extremes of the specification to show how unexpected TCL can operate. But it might not be the best way to use TCL. C is at least as obfuscatble as TCL but less powerful in terms of needed statements per action.

Yeah, deliberate obfuscation is probably always going to be possible, short of something like Spark/ADA or MISRA, at least as a game. The problem comes when you find that you are writing obfuscated or semi-obfuscated code as the simplest/cleanest/compactest code to solve your problem because more obvious/clear/clean structures are somehow unavailable to you.

Anyway, thanks for taking the time to increase my understanding of tcl, with which, previously, I had only had very glancing contact.

jlinkels · 05-05-2010, 08:12 PM

Quote:

Originally Posted by salasi

Sorry, I get the impression that I am continuing this thread further than makes any sense for you, or your original problem, but:
<snip>What I should have made clear (and this was an oversight on my part) was that this was not originally my example, but came from here. Of course, some examples of documentation are better than others...

It's an interesting discussion anyway, and I consider my problem as solved. Not fully elegant, but bearable.
That website you are referring to is a very useful website in most aspects, and often I use it as reference. Not your fault if you were quoting an incorrect example.

jlinkels

Another Kevin · 05-08-2010, 12:57 PM

I started replying to the screeds that Tcl is objectively bad, but decided that the posting was getting far too long for the forum, and dropped it over at http://dftscript.blogspot.com/2010/0...different.html instead.

The elevator-speech summary of the posting might be, "I happen to like Tcl, because it suits the way I think. If you find it unfit for purpose, I'd be happy to recommend other languages that I also use. Don't go calling it 'objectively bad' without data to support your claims."