LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 05-02-2017, 09:27 AM   #1
iconmani
LQ Newbie
 
Registered: May 2017
Location: Belfast, UK
Distribution: solaris, centos, linux, rhel, suse, ubuntu, fedora, rhev-h, oracle linux, OVS
Posts: 3

Rep: Reputation: Disabled
Smile Leap secoud issue | rhel


Hi Everyone,

Did anyone encountered leap second issue in rhel 6.3?
We recently had a production outage due to this issue which occurred on 1st Jan 2017 00:00 AM.
It literally spoiled my new-year party and i don't want to encounter this again next year.
Due to product obligations, we can't update the OS and hence need to find out a workaround.

Red-hat documentations says to run ntp in skew mode, i just wanted to know from the community that did anyone encountered this issue and if yes, what workarounds actually worked apart form upgrading the OS?

Thanks,
Mani Prakash
 
Old 05-02-2017, 10:51 AM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
No.

After the RedHat announcement about this past leap second before it occurred I had done reading that suggested we really only have to worry if we do an absolute time update at exactly midinight UTC when the leap second occurs. Since we had a leap second in 2015 without any noticeable ill effects it seemed likely we wouldn't this time either.

We in fact saw no issues on any RHEL5/RHEL6/RHEL7 server due to this leap second.

If you saw an issue you may want to look into what might be doing an absolute time update at exactly midnight UTC on your system.

I don't think there is much benefit to addressing it now, after the fact. You may need to address it if they announce another coming leap second in future.
 
1 members found this post helpful.
Old 05-02-2017, 11:18 AM   #3
iconmani
LQ Newbie
 
Registered: May 2017
Location: Belfast, UK
Distribution: solaris, centos, linux, rhel, suse, ubuntu, fedora, rhev-h, oracle linux, OVS
Posts: 3

Original Poster
Rep: Reputation: Disabled
rhl 6.4 and above has this issue fixed, we encountered it specifically on rhel 6.3 servers.
This resulted in high CPU utilisation caused because of the java processes and we had to reboot the vms.
 
Old 05-02-2017, 11:35 AM   #4
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669Reputation: 1669
We have a mix of releases including 4 that are on RHEL6.3 and others that are on RHEL6.0 through RHEL6.2. None of them experienced the issue.
 
Old 05-02-2017, 03:45 PM   #5
bathory
LQ Guru
 
Registered: Jun 2004
Location: Piraeus
Distribution: Slackware
Posts: 13,163
Blog Entries: 1

Rep: Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032
Quote:
Originally Posted by iconmani View Post
Red-hat documentations says to run ntp in skew mode, i just wanted to know from the community that did anyone encountered this issue and if yes, what workarounds actually worked apart form upgrading the OS?
Quote:
Originally Posted by iconmani View Post
rhl 6.4 and above has this issue fixed, we encountered it specifically on rhel 6.3 servers.
This resulted in high CPU utilisation caused because of the java processes and we had to reboot the vms.
I've seen it back on 2015 on a Centos-6.2 running zimbra (that is a java based mailserver). It's actually a java problem, so updating just java (not the whole OS) will solve it.

Anyway at that time there was no java without the leap second issue available, so I had to run the following commands (from here):
Code:
/etc/init.d/ntpd stop; date; `date +"%m%d%H%M%C%y.%S"`; date; /etc/init.d/ntpd start
Then restarting the problematic application (zimbra), everything turned back to normal

HTH

Last edited by bathory; 05-03-2017 at 05:14 AM. Reason: typo
 
Old 05-03-2017, 09:38 PM   #6
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,792

Rep: Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201
The most simple command to clear the leap second flag is
Code:
date -s now
I would simply upgrade the OS to 6.9 or at least 6.4
If this is not an option you can install a 6.4 kernel on 6.3
 
Old 05-04-2017, 12:35 AM   #7
bathory
LQ Guru
 
Registered: Jun 2004
Location: Piraeus
Distribution: Slackware
Posts: 13,163
Blog Entries: 1

Rep: Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032Reputation: 2032
Quote:
Originally Posted by MadeInGermany View Post
The most simple command to clear the leap second flag is
Code:
date -s now
I would simply upgrade the OS to 6.9 or at least 6.4
If this is not an option you can install a 6.4 kernel on 6.3
Hi,

The leap second problem was due to the java version shipped with RHEL-6.2/6.3.
So OP can just update java if upgrading OS is not an option, or else change the time and restart ntpd if needed

Regards
 
Old 05-09-2017, 01:08 AM   #8
ssetia
LQ Newbie
 
Registered: May 2017
Posts: 1

Rep: Reputation: Disabled
Leap second has become a pretty common problem and some of the common issues can be observed during this time are:
- High CPU usage for processes like Java
- Sometimes even VM becomes unresponsive

Resolution:
- To resolve this either we can go ahead with Patching process and if we don't want to go with that then there is another way also.
- Manually resetting the date using below command can also fix this issue.
date -s "`LC_ALL=C date`"
OR
date `date +'%m%d%H%M%C%y.%S'`

More details can be found here (may ask for login)
https://access.redhat.com/solutions/154793
 
Old 05-09-2017, 11:15 PM   #9
pantdk
Member
 
Registered: Oct 2011
Location: New Delhi
Posts: 248
Blog Entries: 3

Rep: Reputation: 17
get the below link to resolve the issue
https://access.redhat.com/labs/leapsecond/
https://access.redhat.com/solutions/1471933
https://access.redhat.com/solutions/1530053
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: openSUSE Leap 42.2 is out, how to upgrade safely from openSUSE Leap 42.1 LXer Syndicated Linux News 0 11-17-2016 07:18 AM
Palo Alto and Leap Year issue Linux_Kidd General 3 03-03-2016 03:20 PM
[SOLVED] GCC/G++ 4.4.7 issue on RHEL 6.3 bluforce Linux - Software 6 03-06-2013 10:28 PM
LXer: Kernel Developers Tracking Down New Year's Eve Leap Second Issue LXer Syndicated Linux News 0 01-05-2009 05:21 PM
RHEL 3 AS U4 Issue alexander_goron LinuxQuestions.org Member Success Stories 0 04-12-2005 09:52 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 04:21 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration