Numeric Variable long in shell script

frambau · 02-17-2012, 01:25 AM

Hi to all,

I need to use a numeric incremental variable in a for stament in shell script.

now I'm using this,

declare -i MAX
MAX=1

for ....

MAX=MAX+1

My problem is with the size of the variable. when Max is upper than 33000 take again the value 1 and restart the assignement.

I need a couter until 1000000000 aprox.

Any solve????

THANKS

crulat · 02-17-2012, 02:04 AM

Code:

for counter in {1..1000000000}; do
    if (( ++i < 33000 )); then
        your command here
    else
            i=1
    fi    
done

frambau · 02-17-2012, 02:15 AM

Thanks Crulat

But this is not my problem.

Sorry for not having explained well.
I need is to use a variable that can take values from 1 to 1000000000 one at a time, and print this variable

grail · 02-17-2012, 02:42 AM

Well according to this snippet:

Quote:

Integer variables in older versions of Bash were signed long (32-bit) integers, in the range of -2147483648 to 2147483647. An operation that took a variable outside these limits gave an erroneous result.

this alone would seem to mean you should have no issue, but there is an addendum:

Quote:

As of version >= 2.05b, Bash supports 64-bit integers.

frambau · 02-17-2012, 02:53 AM

Thanks Grail,

However, this number must be an incremental to insert into a database. How can I do?

If you remember, I was doing a program with awk to parse an xml.

Now these data csv I have to insert into the database, but one of the fields must be incremental.

I use a txt file, suppose that is incremental.txt, which contains the value 1.

I make a cat inside the shell, and then increase in the awk (+ + var) to resave the txt.

when exactly exceeds 44024, it becomes again 1.

What I can do?

thank you very much

grail · 02-17-2012, 03:28 AM

Well although it takes time, the following returns for me:

Code:

awk 'BEGIN{for(i=1;i<=1000000000;i++)j++;print j}'

I believe a bash one will return as well just it may take longer. So I would guess that something else is actually stopping your code from progressing past the figure you have found.

frambau · 02-17-2012, 03:32 AM

ok thanks.

catkin · 02-17-2012, 03:35 AM

Quote:

Originally Posted by crulat

Code:

for counter in {1..1000000000}; do
    if (( ++i < 33000 )); then
        your command here
    else
            i=1
    fi    
done

When tested as ...

Code:

for counter in {1..1000000000}; do
    echo $counter    
done

..., in both a terminal emulator (urxvt) and in a virtual terminal it crashed bash itself. The terminal emulator exited. The virtual terminal's bash session crashed and the login prompt re-appeared. A real bug?!

millgates · 02-17-2012, 03:41 AM

that's probably because bash expands braces into list of numbers in the range {1..bilion} which would be a very long string (my guess is about ~8--9 GB) and probably runs out of memory in the process

crulat · 02-17-2012, 07:06 AM

Try this, it works well, i have tested:

Code:

for((i=1; i<1000000000; ++i)); do 
     your_command_here
done

grail · 02-17-2012, 10:22 AM

Last worked for me to, although I did manage to time it at over 107minutes

catkin · 02-17-2012, 10:29 AM

Quote:

Originally Posted by frambau

Thanks Grail,

However, this number must be an incremental to insert into a database. How can I do?

If you remember, I was doing a program with awk to parse an xml.

Now these data csv I have to insert into the database, but one of the fields must be incremental.

I use a txt file, suppose that is incremental.txt, which contains the value 1.

I make a cat inside the shell, and then increase in the awk (+ + var) to resave the txt.

when exactly exceeds 44024, it becomes again 1.

Can you post the awk code to illustrate what you mean?

AIUI you are saying that awk rolls over an incrementing integer on processing 44024+1. That sounds improbable.

Nominal Animal · 02-17-2012, 11:22 AM

This is a bit off topic, but you can do arbitrarily large counters by using bc:

Code:

# Initialize to a really large, >128 bit value, just for illustration
counter=999999999999999999999999999999999999

# Increment counter by one.
counter=`echo "$counter + 1;" | bc`

# Show counter.
echo $counter

Since bc is a traditional utility also standardized in POSIX, it should be always available on all Unix/Linux/BSD systems, even the older ones.

David the H. · 02-18-2012, 10:07 AM

What's wrong with good old seq? According to the info page, it's good for up to at least 2^53 (gnu version).

To avoid system limitations on the generated string, use a while loop instead of a for loop, or xargs.

Code:

while read int; do
	echo "$int"
done < <( seq 1 1000000000 )

( The above uses bash's process substitution. )

No matter what technique you use, however, doing this through the shell is going to be slooooow. For faster results you really need to switch to a more efficient language like perl or C.

PTrenholme · 02-18-2012, 02:09 PM

And, if you're using a data base, and using those numbers as a unique identifier, check your db documentation. Most db system include a provision for automatically generating unique identifiers. And, of course, if that number is intended for use as a record number, that would, for most db, be redundant, since record numbers are usually automatically generated.

Perhaps you could rephrase your question as a data base specific one.