[SOLVED] Convert list of timestamps to UTC

hattori.hanzo · 03-05-2009, 05:45 AM

How would one convert a list of timestamps to UTC?

Code:

2009-01-25 20:57:22

I had a look at Perl's Time::Local module but its not working for me.

thanks & regards

Hko · 03-05-2009, 06:59 AM

With the shell-util "date", on my machine in the Netherlands timezone:

Code:

hko:~$ cat /etc/timezone 
Europe/Amsterdam

hko:~$ date --date='2009-01-25 20:57:22'
Sun Jan 25 20:57:22 CET 2009

hko:~$ date --date='TZ="UTC" 2009-01-25 20:57:22'
Sun Jan 25 21:57:22 CET 2009

wje_lq · 03-05-2009, 07:45 AM

Quote:

Originally Posted by Hko

With the shell-util "date", on my machine in the Netherlands timezone:

Code:

hko:~$ cat /etc/timezone 
Europe/Amsterdam

hko:~$ date --date='2009-01-25 20:57:22'
Sun Jan 25 20:57:22 CET 2009

hko:~$ date --date='TZ="UTC" 2009-01-25 20:57:22'
Sun Jan 25 21:57:22 CET 2009

No, that converts from UTC to local time. He wants to convert from local time to UTC. I'm in California; when it's noon UTC, I'm running eight hours earlier, at 4:00am. When we run the proposed (almost correct) solution, we get:

Code:

wally:~/thursday/2$ date --date='2009-01-25 20:57:22'
Sun Jan 25 20:57:22 PST 2009
wally:~/thursday/2$ date --date='TZ="UTC" 2009-01-25 20:57:22'
Sun Jan 25 12:57:22 PST 2009

We would have wanted:

Code:

Mon Jan 26 04:57:22 UTC 2009

I've played with the date command to find a correct solution, but I couldn't get it do work for this.

wje_lq · 03-05-2009, 08:07 AM

Owe. Weight. I no!

You're using Perl, right? Duh. Here ya go:

Code:

#!/usr/bin/perl

sub convert_to_utc
{
  my($input)=@_;

  my $seconds;
  my $tm_year;
  my $tm_mon;
  my $tm_mday;
  my $tm_hour;
  my $tm_min;
  my $tm_sec;

  $seconds=`date '+%s' "--date=$input"`;

  ($tm_sec,
   $tm_min,
   $tm_hour,
   $tm_mday,
   $tm_mon,
   $tm_year
  )=gmtime($seconds);

  return sprintf("%04d-%02d-%02d %02d:%02d:%02d",
                 $tm_year+1900,
                 $tm_mon +1,
                 $tm_mday,
                 $tm_hour,
                 $tm_min,
                 $tm_sec
                );
}

print convert_to_utc('2009-01-25 20:57:22')."\n";

When I run it, I get:

Code:

2009-01-26 04:57:22

... which is what you want if you're in, say, California.

Hope this helps.

hattori.hanzo · 03-05-2009, 04:49 PM

Hey wje_lq

Thanks for the input. When I tried to run it on my system I get the same as the input:

Quote:

2009-01-25 20:57:22

wje_lq · 03-05-2009, 06:45 PM

There's a good possibility that this is not the whole story.

The script as I posted started with this line:

Code:

#!/usr/bin/perl

I was able to duplicate your error messages exactly by removing that line from the script. So I'm guessing that line is not in your copy of the script.

If you removed that line because you got an error message like this:

Code:

-bash: ./2.pl: /usr/bin/perl: bad interpreter: No such file or directory

then do this at the bash command prompt:

Code:

which perl

Then put a #! at the beginning of your output from that command and make that the first line of the script.

Edit:
The above is based on what you posted before you edited it. I'll have to take a look at your new response and see what goes, in my next post.

hattori.hanzo · 03-05-2009, 06:53 PM

Thanks. I just tried printing all the variables out and noticed gmtime is not converting the seconds correctly on my system;

Code:

 ($tm_sec,$tm_min,$tm_hour,$tm_mday,$tm_mon,$tm_year) = gmtime ( $seconds );

Code:

input: 2009-01-25 20:57:22
seconds: 1232917042

22,57,20,25,0,109
output: 2009-01-25 20:57:22

wje_lq · 03-05-2009, 06:56 PM

Quote:

When I tried to run it on my system I get the same as the input

That is quite bizarre. What happens when you run this Perl script? It might help narrow down the problem.

Code:

#!/usr/bin/perl

sub convert_to_utc
{
  my($input)=@_;

  my $seconds;
  my $tm_year;
  my $tm_mon;
  my $tm_mday;
  my $tm_hour;
  my $tm_min;
  my $tm_sec;

  $seconds=`date '+%s' "--date=$input"`;
  system("echo TZ is \$TZ\n");
  system("strings /etc/localtime | tail -5");
  print("seconds is $seconds\n");

  ($tm_sec,
   $tm_min,
   $tm_hour,
   $tm_mday,
   $tm_mon,
   $tm_year
  )=gmtime($seconds);

  return sprintf("%04d-%02d-%02d %02d:%02d:%02d",
                 $tm_year+1900,
                 $tm_mon +1,
                 $tm_mday,
                 $tm_hour,
                 $tm_min,
                 $tm_sec
                );
}

print convert_to_utc('2009-01-25 20:57:22')."\n";

wje_lq · 03-05-2009, 07:02 PM

Quote:

gmtime is not converting the seconds correctly on my system

No, it's converting it correctly. Given a $seconds value of 1232917042, those are the numbers you're supposed to get.

The problem is the conversion from the original data to $seconds in the first place.

Try running that second script of mine and see what you get.

hattori.hanzo · 03-05-2009, 07:21 PM

Quote:

Originally Posted by wje_lq

No, it's converting it correctly. Given a $seconds value of 1232917042, those are the numbers you're supposed to get.

The problem is the conversion from the original data to $seconds in the first place.

Try running that second script of mine and see what you get.

This is what I get. Thanks.

Quote:

TZ is
TZif2
TZif2
UTC0
seconds is 1232917042

2009-01-25 20:57:22

wje_lq · 03-05-2009, 07:34 PM

Well, there's your problem.

According to your /etc/localtime, your local time zone is UTC. So converting time from local time to UTC shouldn't make it any different at all, and sure enough, in your experience, that's exactly what's happening.

hattori.hanzo · 03-05-2009, 07:41 PM

Quote:

Originally Posted by wje_lq

Well, there's your problem.

According to your /etc/localtime, your local time zone is UTC. So converting time from local time to UTC shouldn't make it any different at all, and sure enough, in your experience, that's exactly what's happening.

Hmmm.. but the input is '2009-01-25 20:57:22' is not in UTC (timestamps are from a system in another timezone), so I was expecting the output to show the equivalent in UTC.

Basically, I have a list of timestamps from a server not in UTC but in a timezone (GMT +10) which compiles with DST, and need to convert those timestamps back to UTC for correlation and storage in a database. Apologies if I did not make this clear in my first post but I was just trying to keep it simple.

wje_lq · 03-05-2009, 08:34 PM

Quote:

the input is '2009-01-25 20:57:22' is not in UTC

The input doesn't say what time zone it is, and if your system is UTC, then as far as your system is concerned, that time is UTC.

Quote:

in a timezone (GMT +10) which compiles with DST, and need to convert those timestamps back to UTC

This is a horse of a varying hue.

Try this:

Code:

#!/usr/bin/perl

sub convert_to_utc
{
  my($input)=@_;

  my $seconds;
  my $tm_year;
  my $tm_mon;
  my $tm_mday;
  my $tm_hour;
  my $tm_min;
  my $tm_sec;

  $seconds=`date '+%s' "--date=$input"`;

  ($tm_sec,
   $tm_min,
   $tm_hour,
   $tm_mday,
   $tm_mon,
   $tm_year
  )=gmtime($seconds);

  return sprintf("%04d-%02d-%02d %02d:%02d:%02d",
                 $tm_year+1900,
                 $tm_mon +1,
                 $tm_mday,
                 $tm_hour,
                 $tm_min,
                 $tm_sec
                );
}

$ENV{'TZ'}="GMT+10";

print convert_to_utc('2009-01-25 20:57:22')."\n";

hattori.hanzo · 03-05-2009, 09:17 PM

Quote:

Originally Posted by wje_lq

Try this:

Code:

#!/usr/bin/perl

sub convert_to_utc
{
  my($input)=@_;

  my $seconds;
  my $tm_year;
  my $tm_mon;
  my $tm_mday;
  my $tm_hour;
  my $tm_min;
  my $tm_sec;

  $seconds=`date '+%s' "--date=$input"`;

  ($tm_sec,
   $tm_min,
   $tm_hour,
   $tm_mday,
   $tm_mon,
   $tm_year
  )=gmtime($seconds);

  return sprintf("%04d-%02d-%02d %02d:%02d:%02d",
                 $tm_year+1900,
                 $tm_mon +1,
                 $tm_mday,
                 $tm_hour,
                 $tm_min,
                 $tm_sec
                );
}

$ENV{'TZ'}="GMT+10";

print convert_to_utc('2009-01-25 20:57:22')."\n";

Thank you very much. This works perfectly. I was actually researching to see if we could utilise the TZ variable in your script without changing the timezone on the system, then I saw your latest post. :-)

Appreciate you help...

sundialsvcs · 03-06-2009, 10:40 AM

Make very aggressive use of CPAN modules when you use Perl... the DateTime module only scratches the surface of what "has already been done."