LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-06-2014, 04:31 PM   #1
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
logstash-1.4.0-1 on CentOS 5.10


I installed logstash 1.4.0 from elasticsearch HQ

and I can start it, but within 1 minute it dies and status shows "Logstash Daemon dead but pid file exists"

Nothing is written to the /etc/logstash/logstash.log or any other log file in /var/log about the "event"

/opt/logstash/bin/logstash -vv barfs with
Code:
+---------------------------------------------------------+
| An unexpected error occurred. This is probably a bug.   |
| You can find help with this problem in a few places:    |
|                                                         |
| * chat: #logstash IRC channel on freenode irc.          |
|     IRC via the web: http://goo.gl/TI4Ro                |
| * email: logstash-users@googlegroups.com                |
| * bug system: https://logstash.jira.com/                |
|                                                         |
+---------------------------------------------------------+
The error reported is: 
  undefined method `+' for nil:NilClass
/opt/logstash/lib/logstash/agent.rb:100:in `execute'
/opt/logstash/lib/logstash/runner.rb:190:in `run'
org/jruby/RubyProc.java:271:in `call'
/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.17/lib/stud/task.rb:12:in `initialize'
Guys on #logstash@Freenode said the word 'bug'...and one person said OpenJDK-7 is not the 'fix'

I'm wondering if I need to install rubygems...?

The strange part is, I had ElasticSearch+kibana+rsyslog data in Kibana, but then I tried to be 'clever' and change the rsyslog>logstash config and now, zilch data.

I had success earlier using just plain ol' rsyslog as an input but now, zilch.

Do I even need logstash for regular rsyslogd files > ES?

References for the initial install:
http://blog.basefarm.com/blog/how-to...rface-on-rhel/ and
http://sharadchhetri.com/2014/03/01/...el-6-centos-6/

Anyone able to help me get rsyslog data into ES/Kibana?

Thanks for your time.

Last edited by Habitual; 05-07-2014 at 09:23 AM. Reason: fixed subject
 
Old 05-07-2014, 09:29 AM   #2
kirukan
Senior Member
 
Registered: Jun 2008
Location: Eelam
Distribution: Redhat, Solaris, Suse
Posts: 1,278

Rep: Reputation: 148Reputation: 148
Quote:
I'm wondering if I need to install rubygems...?
http://logstash.net/docs/1.4.0/tutor...-with-logstash
The only requirement is JAVA
Let's try the basic testing as it is mentioned in the above link before going to use your own logstash configuration
 
Old 05-07-2014, 09:46 AM   #3
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374

Original Poster
Blog Entries: 37

Rep: Reputation: Disabled
Tests with
Code:
bin/logstash -e 'input { stdin { } } output { elasticsearch { host => localhost } }'
work and populate the interface.

Code:
/opt/logstash/bin/logstash agent -f /etc/logstash/conf.d/logstash.conf --configtest
Configuration OK
Code:
/usr/src/logstash-1.4.0/bin/logstash -f /usr/src/logstash-1.4.0/apache.conf web &
works (or did work) to populate the interface
I even did the shakespeare.json import but that never showed up in the Kibana interface either.

Starting the 'service' is where it chokes after about 1 minute and dies with mentioned output.

Thanks.
 
Old 05-07-2014, 11:33 AM   #4
kirukan
Senior Member
 
Registered: Jun 2008
Location: Eelam
Distribution: Redhat, Solaris, Suse
Posts: 1,278

Rep: Reputation: 148Reputation: 148
can share your logstash configuration file here?
 
Old 05-07-2014, 11:59 AM   #5
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374

Original Poster
Blog Entries: 37

Rep: Reputation: Disabled
from the terrible notes I took on it, I believe I tried this one last in etc/logstash/conf.d/logstash.conf:
Code:
input {
syslog {
type => syslog
port => 5544 }
  file {
    	path => "/var/log/rsyslog_custom/web/httpd.log"
	start_position => beginning
  }
}

filter {
  if [path] =~ "access" {
    mutate { replace => { "type" => "apache_access" } }
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
}

output {
  elasticsearch {
    host => localhost
  }
  stdout { codec => rubydebug }
}


I have since disabled rsyslog on my 3 hosts since my /var/log/messages file on the rsyslog-server filled up hda1 at 11G
Oh the horror.
I think entries in /etc/rsyslog.conf can suppress the 'sending' to /var/log/messages with such as
Code:
$ModLoad imfile

$InputFileName /var/log/rsyslog_custom/web/httpd.log
$InputFileTag c9web-httpd-log:
$InputFileStateFile state-c9web-httpd-log
$InputRunFileMonitor

$InputFilePollInterval 1

if $programname == "c9web-httpd-log" then @fqdn:5544
if $programname == "c9web-httpd-log" then stop
I'm still very much interested in a solution, and I'm will to turn one back "on" for testing a solution.

Thank you for your time. I really appreciate the help.
 
Old 05-14-2014, 03:06 PM   #6
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374

Original Poster
Blog Entries: 37

Rep: Reputation: Disabled
If you're downloading and hoping to use Kibana3, like me, you can't use RPMs for elasticsearch and logstash.
All repos I found in howtos on the net for logstash > rsyslog > elasticsearch > kibana DON'T WORK, at least not on CentOS5

Since these are included with https://download.elasticsearch.org/k...-latest.tar.gz or https://download.elasticsearch.org/k...a-3.0.1.tar.gz

You can install a logstash*.rpm, you just can't use
Code:
service logstash start
to get it going. At least, I couldn't. It'd start, then die about 1 minute later as mentioned in 1st post.

The only command that works is:
Code:
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
howtos:
http://blog.basefarm.com/blog/how-to...rface-on-rhel/
http://www.denniskanbier.nl/blog/log...-and-centos-6/

Enjoy the Goodness!
 
Old 05-15-2014, 05:59 AM   #7
dkanbier
LQ Newbie
 
Registered: May 2014
Distribution: Fedora
Posts: 13

Rep: Reputation: Disabled
Quote:
Originally Posted by Habitual View Post
If you're downloading and hoping to use Kibana3, like me, you can't use RPMs for elasticsearch and logstash.
All repos I found in howtos on the net for logstash > rsyslog > elasticsearch > kibana DON'T WORK, at least not on CentOS5

Since these are included with or

You can install a logstash*.rpm, you just can't use
Code:
service logstash start
to get it going. At least, I couldn't. It'd start, then die about 1 minute later as mentioned in 1st post.

The only command that works is:
Code:
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf

Enjoy the Goodness!
I'm wondering what the permissions are on the config file /etc/logstash/conf.d/logstash.conf.

Could you give the output of "ls -l /etc/logstash/conf.d/logstash.conf"?

And as which user do you start logstash when using the command line? If you're not doing so already you could try starting it by hand with the "-v" option as the logstash user. The -vv option also gives me the same error.
 
Old 05-15-2014, 08:33 AM   #8
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374

Original Poster
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by dkanbier View Post
I'm wondering what the permissions are on the config file /etc/logstash/conf.d/logstash.conf.

Could you give the output of "ls -l /etc/logstash/conf.d/logstash.conf"?
Code:
-rw-r--r-- 1 root root 364 May 14 07:57 /etc/logstash/conf.d/logstash.conf
Quote:
And as which user do you start logstash when using the command line? If you're not doing so already you could try starting it by hand with the "-v" option as the logstash user. The -vv option also gives me the same error.
as root.

A result of the rpm install:
Code:
logstash:x:104:164:logstash:/opt/logstash:/sbin/nologin
and trying a forced shell:
Code:
sudo -u logstash bash
bash: /root/.bashrc: Permission denied
bash-3.2$ id
uid=104(logstash) gid=164(logstash) groups=164(logstash)
bash-3.2$ pwd
/root
$ ls -dl /opt/logstash
drwxr-xr-x 9 logstash logstash 4096 May 13 13:36 /opt/logstash
It gets really ugly using the -v switch as the logstash user:
Code:
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf -v
Using milestone 2 input plugin 'file'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
Registering file input {:path=>["/var/log/remotes/web/*.log"], :level=>:info}
No sincedb_path set, generating one based on the file path {:sincedb_path=>"/root/.sincedb_38f4dc076e48e774d7872c63b279d079", :path=>["/var/log/remotes/web/*.log"], :level=>:info}
Registering file input {:path=>["/var/log/remotes/cirrhus9a/*.log"], :level=>:info}
No sincedb_path set, generating one based on the file path {:sincedb_path=>"/root/.sincedb_f3f1a09b7195f62d15e9bfe6d07044c7", :path=>["/var/log/remotes/cirrhus9a/*.log"], :level=>:info}
Registering file input {:path=>["/var/log/remotes/cirrhus9b/*.log"], :level=>:info}
No sincedb_path set, generating one based on the file path {:sincedb_path=>"/root/.sincedb_42f4e99991f750cf42bd5d2e154ef9de", :path=>["/var/log/remotes/cirrhus9b/*.log"], :level=>:info}
Pipeline started {:level=>:info}
Starting embedded Elasticsearch local node. {:level=>:info}
log4j, [2014-05-15T06:23:28.585]  INFO: org.elasticsearch.node: [Raa of the Caves] version[1.1.1], pid[26441], build[f1585f0/2014-04-16T14:27:12Z]
log4j, [2014-05-15T06:23:28.586]  INFO: org.elasticsearch.node: [Raa of the Caves] initializing ...
log4j, [2014-05-15T06:23:28.604]  INFO: org.elasticsearch.plugins: [Raa of the Caves] loaded [], sites []
Exception in thread ">output" org.elasticsearch.ElasticsearchIllegalStateException: Failed to obtain node lock, is the following location writable?: [/root/data/elasticsearch]
        at org.elasticsearch.env.NodeEnvironment.<init>(org/elasticsearch/env/NodeEnvironment.java:114)
        at org.elasticsearch.node.internal.InternalNode.<init>(org/elasticsearch/node/internal/InternalNode.java:150)
        at org.elasticsearch.node.NodeBuilder.build(org/elasticsearch/node/NodeBuilder.java:159)
        at org.elasticsearch.node.NodeBuilder.node(org/elasticsearch/node/NodeBuilder.java:166)
        at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:622)
        at RUBY.start_local_elasticsearch(/opt/logstash/lib/logstash/outputs/elasticsearch.rb:299)
        at RUBY.register(/opt/logstash/lib/logstash/outputs/elasticsearch.rb:247)
        at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
        at RUBY.outputworker(/opt/logstash/lib/logstash/pipeline.rb:220)
        at RUBY.start_outputs(/opt/logstash/lib/logstash/pipeline.rb:152)
        at java.lang.Thread.run(java/lang/Thread.java:701)
Caused by: java.io.IOException: failed to obtain lock on /root/data/elasticsearch/nodes/49
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:99)
        at org.elasticsearch.node.internal.InternalNode.<init>(InternalNode.java:150)
        at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
        at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:166)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:622)
        at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:440)
        at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:304)
        at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:52)
        at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
        at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
        at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95)
        at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
        at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
        at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
        at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139)
        at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182)
        at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
        at org.jruby.ast.VCallNode.interpret(VCallNode.java:88)
        at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
        at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
        at org.jruby.ast.IfNode.interpret(IfNode.java:116)
        at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
        at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
        at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
        at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:112)
        at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:164)
        at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)
        at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:85)
        at org.jruby.RubySymbol$1.yieldInner(RubySymbol.java:445)
        at org.jruby.RubySymbol$1.yield(RubySymbol.java:465)
        at org.jruby.runtime.Block.yield(Block.java:142)
        at org.jruby.RubyArray.eachCommon(RubyArray.java:1606)
        at org.jruby.RubyArray.each(RubyArray.java:1613)
        at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
        at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:316)
        at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:145)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:149)
        at org.jruby.ast.CallNoArgBlockPassNode.interpret(CallNoArgBlockPassNode.java:53)
        at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
        at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
        at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
        at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139)
        at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182)
        at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
        at org.jruby.ast.VCallNode.interpret(VCallNode.java:88)
        at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
        at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
        at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
        at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194)
        at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:125)
        at org.jruby.runtime.Block.call(Block.java:101)
        at org.jruby.RubyProc.call(RubyProc.java:290)
        at org.jruby.RubyProc.call(RubyProc.java:228)
        at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:99)
        at java.lang.Thread.run(Thread.java:701)
Caused by: java.io.IOException: Cannot create directory: /root/data/elasticsearch/nodes/49
        at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:171)
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:81)
        ... 61 more
Interrupt received. Shutting down the pipeline. {:level=>:warn}
Sending shutdown signal to input thread {:thread=>#<Thread:0x6b211be8 run>, :level=>:info}
Plugin is finished {:plugin=><LogStash::Inputs::Stdin type=>"stdin-type">, :level=>:info}
Sending shutdown signal to input thread {:thread=>#<Thread:0x38d4d428 sleep>, :level=>:info}
Sending shutdown signal to input thread {:thread=>#<Thread:0x7e7be20b sleep>, :level=>:info}
Sending shutdown signal to input thread {:thread=>#<Thread:0x85168c6 sleep>, :level=>:info}
probably since I started it as root in the first place.
Code:
# ls -dl /root/data/elasticsearch
drwxr-xr-x 3 root root 4096 May 14 06:41 /root/data/elasticsearch
I have no problem running
Code:
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
as root in screen to keep it running, if that's what it takes.

Thanks.

Last edited by Habitual; 05-15-2014 at 08:35 AM.
 
Old 05-15-2014, 09:20 AM   #9
dkanbier
LQ Newbie
 
Registered: May 2014
Distribution: Fedora
Posts: 13

Rep: Reputation: Disabled
I'm just trying to help you troubleshoot, if you're done with it I'm fine with that too just let me know! But I like to discover what goes wrong

I think the error code we get is valid when we run logstash as the logstash user:

Code:
Exception in thread ">output" org.elasticsearch.ElasticsearchIllegalStateException: Failed to obtain node lock, is the following location writable?: [/root/data/elasticsearch]
The user logstash shouldn't be able to access anything in /root, so that's good!

Now before jumping to conclusions I'd also like to know if you installed a separate elasticsearch server or if you'd like to use the embedded one in Logstash. If I look at the logstash configuration you use, you're trying to use a separate elasticsearch server. If so, could we get the configuration on that?

Also, I think you can get some more output from logstash by modifying it's init script /etc/init.d/logstash. I simply added a --debug option in the DAEMON_OPTS line:

Code:
DAEMON_OPTS="agent -f ${LS_CONF_DIR} -l ${LS_LOG_FILE} ${LS_OPTS} --debug"
It should log to /var/log/logstash now.

One last thing, this is how I change to users without a shell. A bit less messy

Code:
[root@dev var]# su - logstash -s /bin/bash
-bash-4.1$
Cheers!
 
Old 05-15-2014, 09:30 AM   #10
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374

Original Poster
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by dkanbier View Post
Now before jumping to conclusions I'd also like to know if you installed a separate elasticsearch server or if you'd like to use the embedded one in Logstash. If I look at the logstash configuration you use, you're trying to use a separate elasticsearch server. If so, could we get the configuration on that?
since EL is provided by the logstash.rpm, I used no other. I had previously installed one but removed it when I found the logstash.rpm has one included.

Quote:
Originally Posted by dkanbier View Post
Also, I think you can get some more output from logstash by modifying it's init script /etc/init.d/logstash. I simply added a --debug option in the DAEMON_OPTS line:

Code:
DAEMON_OPTS="agent -f ${LS_CONF_DIR} -l ${LS_LOG_FILE} ${LS_OPTS} --debug"
Thanks, I'll try this after I fix an "InputFileName" issue I'm having with rsyslog.conf on a client.

It should log to /var/log/logstash now.

Quote:
Originally Posted by dkanbier View Post
A bit less messy

Code:
[root@dev var]# su - logstash -s /bin/bash
-bash-4.1$
It is.

Thanks for the feedback.

Edit: the rpm-provided /etc/init.d/logstash doesn't have a DAEMON_OPTS
It does have
Code:
args="agent -f ${LS_CONF_DIR} -l ${LS_LOG_FILE} ${LS_OPTS}"
so I stuck --debug there.

Edit: Didn't work. only produced 2 additional files in /var/log/logstash
logstash.err - 0 bytes
logstash.stderr - barking about "-- -" not allowed. There is/was no "-- -" just --debug" added

Last edited by Habitual; 05-15-2014 at 10:05 AM.
 
Old 05-15-2014, 09:54 AM   #11
dkanbier
LQ Newbie
 
Registered: May 2014
Distribution: Fedora
Posts: 13

Rep: Reputation: Disabled
No problem!

I'm actually surprised Logstash keeps running if you run it by hand with the logstash configuration you posted. I always thought you needed to specify the "embedded" option in your output section if you want to log to the embedded elasticsearch.

I'm very curious if there is process listening on the elasticsearch port (default 9200 and 9300 I think) if you're running logstash in screen:

Code:
netstat -tulpn | grep LISTEN
If there is nothing listening, you could try this for the output section:

Code:
output {
  elasticsearch { embedded => true }
  stdout { codec => rubydebug }
}
If there is something listening, is it the logstash pid or something else?

Quote:
Originally Posted by Habitual View Post
Edit: the rpm-provided /etc/init.d/logstash doesn't have a DAEMON_OPTS
It does have
Code:
args="agent -f ${LS_CONF_DIR} -l ${LS_LOG_FILE} ${LS_OPTS}"
so I stuck --debug there.
That's weird, what version of the RPM are you using?

Last edited by dkanbier; 05-15-2014 at 09:56 AM.
 
Old 05-15-2014, 10:17 AM   #12
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374

Original Poster
Blog Entries: 37

Rep: Reputation: Disabled
Code:
netstat -plaunt | grep 92 | grep java
tcp        0      0 :::9200                     :::*                        LISTEN      23399/java
...
/opt/logstash/vendor/jar/elasticsearch-1.1.1/lib/elasticsearch-1.1.1.jar
logstash.conf has changed since http://www.linuxquestions.org/questi...1/#post5166442 and is now simply:
Code:
cat /etc/logstash/conf.d/logstash.conf
input {
  stdin {
    type => "stdin-type"
  }

  file {
    type => "syslog"
    path => [ "/var/log/remotes/web/*.log" ]
  }

  file {
    type => "syslog"
    path => [ "/var/log/remotes/cirrhus9a/*.log" ]
  }

  file {
    type => "syslog"
    path => [ "/var/log/remotes/cirrhus9b/*.log" ]
  }
}

output {
  stdout { }
  elasticsearch { embedded => true }
}
logstash-1.4.1-1_bd507eb

I've tried all of these:
http://download.elasticsearch.org/lo...tos.noarch.rpm
https://download.elasticsearch.org/l...c09.noarch.rpm
https://download.elasticsearch.org/l...7eb.noarch.rpm
 
Old 05-16-2014, 03:02 AM   #13
dkanbier
LQ Newbie
 
Registered: May 2014
Distribution: Fedora
Posts: 13

Rep: Reputation: Disabled
Quote:
Originally Posted by Habitual View Post
Code:
netstat -plaunt | grep 92 | grep java
tcp        0      0 :::9200                     :::*                        LISTEN      23399/java
...
/opt/logstash/vendor/jar/elasticsearch-1.1.1/lib/elasticsearch-1.1.1.jar
logstash.conf has changed since http://www.linuxquestions.org/questi...1/#post5166442 and is now simply:
Code:
cat /etc/logstash/conf.d/logstash.conf
input {
  stdin {
    type => "stdin-type"
  }

  file {
    type => "syslog"
    path => [ "/var/log/remotes/web/*.log" ]
  }

  file {
    type => "syslog"
    path => [ "/var/log/remotes/cirrhus9a/*.log" ]
  }

  file {
    type => "syslog"
    path => [ "/var/log/remotes/cirrhus9b/*.log" ]
  }
}

output {
  stdout { }
  elasticsearch { embedded => true }
}
logstash-1.4.1-1_bd507eb

I've tried all of these:
http://download.elasticsearch.org/lo...tos.noarch.rpm
https://download.elasticsearch.org/l...c09.noarch.rpm
https://download.elasticsearch.org/l...7eb.noarch.rpm
It seems I'm a bit behind with my version: logstash-1.4.0-1_c82dc09.noarch

Did you try to run logstash as a service when you added the --debug option? Did it log anything?
 
Old 05-16-2014, 08:24 AM   #14
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374

Original Poster
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by dkanbier View Post
It seems I'm a bit behind with my version: logstash-1.4.0-1_c82dc09.noarch

Did you try to run logstash as a service when you added the --debug option? Did it log anything?
I reported the output of using --debug in http://www.linuxquestions.org/questi...1/#post5171391

I'd be really interested in is the /etc/init.d/logstash provided by logstash-1.4.0-1_c82dc09.noarch works on your version of CentOS or Redhat/SuSE/other rpm-based OS as logstash-1.4.1-1_bd507eb's version of it does not have a "DAEMON_OPTS=" line in that rpm's /etc/init.d/logstash.

Perhaps it will "just work" when I reboot my host later today, after a zabbix upgrade since
Code:
logstash       	0:off	1:off	2:on	3:on	4:on	5:on	6:off
?

Anyway, logstash 1.4.1 with Kibana3.x and rsyslogd 7.6.3 are all working in conjunction over here. So that idiosyncrasy is the only thing left.

Have a Great Day!
 
Old 05-16-2014, 11:18 AM   #15
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374

Original Poster
Blog Entries: 37

Rep: Reputation: Disabled
Aha!

Fixed service logstash (start|status) using
Code:
mkdir /tmp/test
cp logstash-1.4.0-1_c82dc09.noarch.rpm /tmp/test/
cp /etc/rc.d/init.d/logstash /etc/rc.d/init.d/logstash.org
cd /tmp/test
rpm2cpio logstash-1.4.0-1_c82dc09.noarch.rpm  | cpio -idmv
cp etc/init.d/logstash /etc/rc.d/init.d/
service logstash start
and several minutes later, I ran
Code:
service logstash status
and it's still running!

My current indexes are gone using
Code:
curl http://localhost:9200/_aliases?pretty=1
but it's progress!

I restarted using
Code:
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
manually and my indexes "came back", soooo..........how to get service logstash start "use" /etc/logstash/conf.d/logstash.conf ?

I suppose I could nuke logstash-1.4.1-1_bd507eb
and install logstash-1.4.0-1_c82dc09.noarch.rpm

To be continued...

Have a Great Day.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Centralized log setup awesant elasticsearch logstash and kibana3 LXer Syndicated Linux News 0 01-26-2014 06:12 PM
CentOS centralised logging, syslogd, rsyslog, syslog-ng, logstash sender? batfastad Linux - Server 4 11-29-2012 03:56 AM
Centos5 Ramdisk help cf500 Linux - General 8 02-22-2011 01:59 AM
centOS5.2 ekac Linux - Newbie 4 06-05-2009 07:53 AM
XEN on centos5 hackintosh Linux - Server 2 10-19-2007 11:11 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 03:01 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration