Logstash stopped processing because of an error systemexit exit

Logstash : logstash-oss-6.5.1.tar.gz OS: Red Hat Enterprise Linux Server release 6.9 (Santiago) config： slack { url => "xxxxxxxxx" channel => "alert" username => logsta...

Logstash : logstash-oss-6.5.1.tar.gz
OS: Red Hat Enterprise Linux Server release 6.9 (Santiago)
config： slack {
url => «xxxxxxxxx»
channel => «alert»
username => logstash
icon_emoji => «:redsiren:»
attachments => [{text => «%{message}»}]
format => «OOM occurred on %{host}»
}

ERROR message ：
[2019-01-08T07:41:19,207][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<JSON::ParserError: unexpected token at ‘{«text»:»2018-12-17 13:40:29 ERROR [com.common.system.filter.TLThreadContextFilter] -f4363092-936e-4465-80e2-bc2edb9f61df- catching
javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
af.common.system.dao.ldap.ApplicationDaoImpl] -94ef0b94-13e7-417e-9a75-f988923e8e4b- App: 749, urlmapping: , not retrieved from cache»}’>, :backtrace=>[«json/ext/Parser.java:250:in parse'", "/opt/logstash-6.5.1/vendor/bundle/jruby/2.3.0/gems/json-1.8.6-java/lib/json/common.rb:155:in parse'», «/opt/logstash-6.5.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-slack-2.1.1/lib/logstash/outputs/slack.rb:64:in block in receive'", "org/jruby/RubyArray.java:2486:in map'», «/opt/logstash-6.5.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-slack-2.1.1/lib/logstash/outputs/slack.rb:64:in receive'", "/opt/logstash-6.5.1/logstash-core/lib/logstash/outputs/base.rb:89:in block in multi_receive'», «org/jruby/RubyArray.java:1734:in each'", "/opt/ibm/logstash-6.5.1/logstash-core/lib/logstash/outputs/base.rb:89:in multi_receive'», «org/logstash/config/ir/compiler/OutputStrategyExt.java:114:in multi_receive'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:97:in multi_receive'», «/opt/logstash-6.5.1/logstash-core/lib/logstash/pipeline.rb:373:in block in output_batch'", "org/jruby/RubyHash.java:1343:in each'», «/opt/logstash-6.5.1/logstash-core/lib/logstash/pipeline.rb:372:in output_batch'", "/opt/logstash-6.5.1/logstash-core/lib/logstash/pipeline.rb:324:in worker_loop'», «/opt/logstash-6.5.1/logstash-core/lib/logstash/pipeline.rb:286:in `block in start_workers'»]}
[2019-01-08T07:41:19,430][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

Thanks

Источник

Issue

How can logstash be executed? (error occured)

run logstash on AWS Linux.
but error occurred while executing.

  • my Linux(Ubuntu Version: 20.04)
  • elasticsearch installed.(execute successfully)
  • kibana installed.(execute successfully)
  • logstash installed. (execute error occured)

my .conf file code

input {
    jdbc {
        clean_run => true
        jdbc_driver_library => "/usr/share/java/mysql-connector-java-8.0.23.jar"
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://AWSLINK:3306/schema_name?useSSL=false&user=root&password=1234"
        jdbc_user => "root"
        jdbc_password => "1234"
        schedule => "* * * * *"
        statement => "select * from schema_name"
    }
}

output {
    elasticsearch {
        hosts => 52.188.20.167:9200"
        index => "AWS_DB_0514"
    }
    stdout {
        codec => rubydebug
    }
}

I execute logstash in linux(command)

./logstash -f test.conf --path.settings /etc/logstash/

I execute above code but error occured.(attempt)

Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2021-05-14T08:37:16,025][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2021-05-14T08:37:16,039][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.12.0", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.10+9 on 11.0.10+9 +indy +jit [linux-x86_64]"}
[2021-05-14T08:37:16,466][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-05-14T08:37:17,524][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2021-05-14T08:37:18,048][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [0-9], [ \t\r\n], "#", "}" at line 16, column 24 (byte 608) after output {n    elasticsearch {n        hosts => 52.188", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:184:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:389:in `block in converge_state'"]}
[2021-05-14T08:37:18,165][INFO ][logstash.runner          ] Logstash shut down.
[2021-05-14T08:37:18,177][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:747) ~[jruby-complete-9.2.13.0.jar:?]
        at org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:710) ~[jruby-complete-9.2.13.0.jar:?]
        at usr.share.logstash.lib.bootstrap.environment.<main>(/usr/share/logstash/lib/bootstrap/environment.rb:89) ~[?:?]

Solution

There is an error on line 16.

incorrect code

hosts => 52.188.20.167:9200"

correct code

hosts => "52.188.20.167:9200"

Answered By – Python-97

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Источник

We are trying to index Nginx access and error log separately in Elasticsearch. for that we have created Filbeat and Logstash config as below.

Below is our /etc/filebeat/filebeat.yml configuration

filebeat.inputs:
- type: log
  paths:
    - /var/log/nginx/*access*.log
  exclude_files: ['.gz$']
  exclude_lines: ['*ELB-HealthChecker*']
  fields:
    log_type: type1 
- type: log
  paths:
    - /var/log/nginx/*error*.log
  exclude_files: ['.gz$']
  exclude_lines: ['*ELB-HealthChecker*']
  fields:
    log_type: type2

output.logstash:
  hosts: ["10.227.XXX.XXX:5400"]

Our logstash file /etc/logstash/conf.d/logstash-nginx-es.conf config is as below

input {
    beats {
        port => 5400
    }
}

filter {
  if ([fields][log_type] == "type1") {
    grok {
      match => [ "message" , "%{NGINXACCESS}+%{GREEDYDATA:extra_fields}"]
      overwrite => [ "message" ]
    }
    mutate {
      convert => ["response", "integer"]
      convert => ["bytes", "integer"]
      convert => ["responsetime", "float"]
    }
    geoip {
      source => "clientip"
      target => "geoip"
      add_tag => [ "nginx-geoip" ]
    }
    date {
      match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
      remove_field => [ "timestamp" ]
    }
    useragent {
      source => "user_agent"
    }
  } else {
      grok {
        match => [ "message" , "(?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) [%{LOGLEVEL:severity}] %{POSINT:pid}#%{NUMBER:threadid}: *%{NUMBER:connectionid} %{GREEDYDATA:message}, client: %{IP:client}, server: %{GREEDYDATA:server}, request: "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion}))"(, upstream: "%{GREEDYDATA:upstream}")?, host: "%{DATA:host}"(, referrer: "%{GREEDYDATA:referrer}")?"]
        overwrite => [ "message" ]
      }
      mutate {
        convert => ["response", "integer"]
        convert => ["bytes", "integer"]
        convert => ["responsetime", "float"]
      }
      geoip {
        source => "clientip"
        target => "geoip"
        add_tag => [ "nginx-geoip" ]
      }
      date {
        match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
        remove_field => [ "timestamp" ]
      }
      useragent {
        source => "user_agent"
      }
    }
}

output {
  if ([fields][log_type] == "type1") {
    amazon_es {
      hosts => ["vpc-XXXX-XXXX.ap-southeast-1.es.amazonaws.com"]
      region => "ap-southeast-1"
      aws_access_key_id => 'XXXX'
      aws_secret_access_key => 'XXXX'
      index => "nginx-access-logs-%{+YYYY.MM.dd}"
    }
} else {
    amazon_es {
      hosts => ["vpc-XXXX-XXXX.ap-southeast-1.es.amazonaws.com"]
      region => "ap-southeast-1"
      aws_access_key_id => 'XXXX'
      aws_secret_access_key => 'XXXX'
      index => "nginx-error-logs-%{+YYYY.MM.dd}"
    }
  }
    stdout { 
      codec => rubydebug 
    }
}

And we are receiving below error while starting logstash.

[2020-10-12T06:05:39,183][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.9.2", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 25.265-b01 on 1.8.0_265-b01 +indy +jit [linux-x86_64]"}
[2020-10-12T06:05:39,861][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-10-12T06:05:41,454][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \t\r\n], "#", "{", ",", "]" at line 32, column 263 (byte 918) after filter {n  if ([fields][log_type] == "type1") {n    grok {n      match => [ "message" , "%{NGINXACCESS}+%{GREEDYDATA:extra_fields}"]n      overwrite => [ "message" ]n    }n    mutate {n      convert => ["response", "integer"]n      convert => ["bytes", "integer"]n      convert => ["responsetime", "float"]n    }n    geoip {n      source => "clientip"n      target => "geoip"n      add_tag => [ "nginx-geoip" ]n    }n    date {n      match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]n      remove_field => [ "timestamp" ]n    }n    useragent {n      source => "user_agent"n    }n  } else {n      grok {n        match => [ "message" , "(?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER:threadid}\: \*%{NUMBER:connectionid} %{GREEDYDATA:message}, client: %{IP:client}, server: %{GREEDYDATA:server}, request: "", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:183:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:44:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:357:in `block in converge_state'"]}
[2020-10-12T06:05:41,795][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-10-12T06:05:46,685][INFO ][logstash.runner          ] Logstash shut down.
[2020-10-12T06:05:46,706][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

There seems to be some formatting issue. Please help what is the problem

=================================UPDATE===================================

For all those who are looking for a robust grok filter for nginx access and error logs … please try below filter patterns.

Access_Logs — %{IPORHOST:remote_ip} - %{DATA:user_name} [%{HTTPDATE:access_time}] "%{WORD:http_method} %{URIPATHPARAM:url} HTTP/%{NUMBER:http_version}" %{NUMBER:response_code} %{NUMBER:body_sent_bytes} "%{SPACE:referrer}" "%{DATA:agent}" %{NUMBER:duration} req_header:"%{DATA:req_header}" req_body:"%{DATA:req_body}" resp_header:"%{DATA:resp_header}" resp_body:"%{GREEDYDATA:resp_body}"

Error_Logs — (?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) [%{LOGLEVEL:severity}] %{POSINT:pid}#%{NUMBER:threadid}: *%{NUMBER:connectionid} %{DATA:errormessage}, client: %{IP:client}, server: %{IP:server}, request: "(?<httprequest>%{WORD:httpcommand} %{NOTSPACE:httpfile} HTTP/(?<httpversion>[0-9.]*))", host: "%{NOTSPACE:host}"(, referrer: "%{NOTSPACE:referrer}")?

Источник
[2023-01-12T15:39:37,782][INFO ][logstash.runner          ] Log4j configuration path used is:  ссылка удалена 
[2023-01-12T15:39:37,815][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.5.3", "jruby.version"=>"jruby 9.3.9.0 (2.6.8) 2022-10-24 537cd1f8bc OpenJDK 64-Bit Server VM 17.0.5+8 on 17.0.5+8 +indy +jit [x86_64-linux]"}
[2023-01-12T15:39:37,818][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.management ссылка удалена ]
[2023-01-12T15:39:39,165][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>" ссылка удалена "}
[2023-01-12T15:39:39,172][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2023-01-12T15:39:39,346][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2023-01-12T15:39:39,394][INFO ][logstash.runner          ] Logstash shut down.
[2023-01-12T15:39:39,400][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
	at org.jruby.RubyKernel.exit(org ссылка удалена ) ~[jruby.jar:?]
	at org.jruby.RubyKernel.exit(org ссылка удалена ) ~[jruby.jar:?]
	at usr.share.logstash.lib.bootstrap.environment.<main>( ссылка удалена ) ~[?:?]
[2023-01-12T15:39:52,838][INFO ][logstash.runner          ] Log4j configuration path used is:  ссылка удалена 
[2023-01-12T15:39:52,844][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.5.3", "jruby.version"=>"jruby 9.3.9.0 (2.6.8) 2022-10-24 537cd1f8bc OpenJDK 64-Bit Server VM 17.0.5+8 on 17.0.5+8 +indy +jit [x86_64-linux]"}
[2023-01-12T15:39:52,846][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.management ссылка удалена ]
[2023-01-12T15:39:54,063][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>" ссылка удалена "}
[2023-01-12T15:39:54,074][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2023-01-12T15:39:54,192][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2023-01-12T15:39:54,251][INFO ][logstash.runner          ] Logstash shut down.
[2023-01-12T15:39:54,259][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
	at org.jruby.RubyKernel.exit(org ссылка удалена ) ~[jruby.jar:?]
	at org.jruby.RubyKernel.exit(org ссылка удалена ) ~[jruby.jar:?]
	at usr.share.logstash.lib.bootstrap.environment.<main>( ссылка удалена ) ~[?:?]
[2023-01-12T15:40:07,412][INFO ][logstash.runner          ] Log4j configuration path used is:  ссылка удалена 
[2023-01-12T15:40:07,418][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.5.3", "jruby.version"=>"jruby 9.3.9.0 (2.6.8) 2022-10-24 537cd1f8bc OpenJDK 64-Bit Server VM 17.0.5+8 on 17.0.5+8 +indy +jit [x86_64-linux]"}
[2023-01-12T15:40:07,420][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.management ссылка удалена ]
[2023-01-12T15:40:08,392][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>" ссылка удалена "}
[2023-01-12T15:40:08,397][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2023-01-12T15:40:08,496][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2023-01-12T15:40:08,542][INFO ][logstash.runner          ] Logstash shut down.
[2023-01-12T15:40:08,548][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
	at org.jruby.RubyKernel.exit(org ссылка удалена ) ~[jruby.jar:?]
	at org.jruby.RubyKernel.exit(org ссылка удалена ) ~[jruby.jar:?]
	at usr.share.logstash.lib.bootstrap.environment.<main>( ссылка удалена ) ~[?:?]
[2023-01-12T15:40:21,546][INFO ][logstash.runner          ] Log4j configuration path used is:  ссылка удалена 
[2023-01-12T15:40:21,554][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"8.5.3", "jruby.version"=>"jruby 9.3.9.0 (2.6.8) 2022-10-24 537cd1f8bc OpenJDK 64-Bit Server VM 17.0.5+8 on 17.0.5+8 +indy +jit [x86_64-linux]"}
[2023-01-12T15:40:21,556][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-exports=jdk.compiler ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.base ссылка удалена , --add-opens=java.management ссылка удалена ]
[2023-01-12T15:40:22,607][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>" ссылка удалена "}
[2023-01-12T15:40:22,613][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
[2023-01-12T15:40:22,810][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2023-01-12T15:40:22,849][INFO ][logstash.runner          ] Logstash shut down.
[2023-01-12T15:40:22,854][FATAL][org.logstash.Logstash    ] Logstash stopped processing because of an error: (SystemExit) exit
org.jruby.exceptions.SystemExit: (SystemExit) exit
	at org.jruby.RubyKernel.exit(org ссылка удалена ) ~[jruby.jar:?]
	at org.jruby.RubyKernel.exit(org ссылка удалена ) ~[jruby.jar:?]
	at usr.share.logstash.lib.bootstrap.environment.<main>( ссылка удалена ) ~[?:?]

Это logstash-plain.log.

В логе эластика ничего не отображается.

Источник

0favorite

I am trying to execute this command: aymenstien@aymenstien-VPCEH2Q1E:/usr/share/logstash$ ./bin/logstash -f /home/aymenstien/Bureau/fb.conf

here is the config file:

input {
file {
path => «/home/aymenstien/Bureau/mydoc/*»
start_position => beginning
codec => json
sincedb_path => «/home/aymenstien/Bureau/mydoc/postj1.sincedb»
}
}
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => «http://localhost:9200»
index => «fbpost»
document_type => «post»
timeout => 30
workers => 1
}
}

and I am getting the result of execution below:

aymenstien@aymenstien-VPCEH2Q1E:/usr/share/logstash$ ./bin/logstash -f /home/aymenstien/Bureau/fb.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using —path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[FATAL] 2018-07-05 12:47:56.496 [main] runner — An unexpected error occurred! {:error=>#<ArgumentError: Path «/usr/share/logstash/data» must be a writable directory. It is not writable.>, :backtrace=>[«/usr/share/logstash/logstash-core/lib/logstash/settings.rb:448:in `validate'», «/usr/share/logstash/logstash-core/lib/logstash/settings.rb:230:in `validate_value'», «/usr/share/logstash/logstash-core/lib/logstash/settings.rb:141:in `block in validate_all'», «org/jruby/RubyHash.java:1343:in `each'», «/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in `validate_all'», «/usr/share/logstash/logstash-core/lib/logstash/runner.rb:279:in `execute'», «/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'», «/usr/share/logstash/logstash-core/lib/logstash/runner.rb:238:in `run'», «/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'», «/usr/share/logstash/lib/bootstrap/environment.rb:73:in `<main>'»]}
[ERROR] 2018-07-05 12:47:56.554 [main] Logstash — java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

Can anyone help me?

Источник

Problem

I have Logstash 6.3.2 from IBM Log Analysis 1.3.5.3. If I run multiple instances of Logstash 6.3.2, I get the following error:
 

[FATAL] 2020-01-02 10:10:10.732 [LogStash::Runner] runner - Logstash could not be started because there is already another instance using the configured data directory.  If you wish to run multiple instances, you must change the "path.data" setting.

[ERROR] 2020-01-02 10:10:10.745 [LogStash::Runner] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

Resolving The Problem

After you installed one Logstash remote installation, there are 2 variables needed for each instance. Modify the logstash-util.sh as follows:

1)  Add variable:
 
    logstash_data=
    logstash_conf=
  
For example, one instance can be:
 
    logstash_data="${LOGSTASH_HOME}/data1"
    logstash_conf="config/logstash-scala.conf"
  
2) Modify from:
 
    PIDTEMP=`ps ux | grep logstash | grep config/logstash-scala.conf | grep java | awk '{ print $2 }'`
  
To:

    PIDTEMP=`ps ux | grep logstash | grep ${logstash_conf} | grep java | awk '{ print $2 }'`
  
3) Add the --path.data ${logstash_data} variable as follows:
 
    nohup ${logstash_bin} --verbose -f ${logstash_conf}  --path.logs ${logstash_log} --path.settings ${logstash_path_settings} --path.data ${logstash_data} > /dev/null 2>&1 &
  
logstash_data and logstash_conf needs to be unique for every instance.

Document Location

Worldwide

[{«Business Unit»:{«code»:»BU053″,»label»:»Cloud & Data Platform»},»Product»:{«code»:»SSPFMY»,»label»:»IBM Operations Analytics — Log Analysis»},»ARM Category»:[{«code»:»a8m0z0000001gwtAAA»,»label»:»Log Analysis->Third Party Components->Logstash 6.3.2″}],»ARM Case Number»:»TS003587647″,»Platform»:[{«code»:»PF025″,»label»:»Platform Independent»}],»Version»:»1.3.5″,»Line of Business»:{«code»:»LOB45″,»label»:»Automation»}}]

Источник

Понравилась статья? Поделить с друзьями:

Читайте также:

  • Logstash json parse error original data now in message field
  • Logstash attempted to resurrect connection to dead es instance but got an error
  • Logs fml client latest log как исправить
  • Logrotate mysql error log
  • Logonui unknown hard error

    • 0 0 голоса
    Рейтинг статьи
    Подписаться
    Уведомить о
    guest

    0 комментариев
    Старые
    Новые Популярные
    Межтекстовые Отзывы
    Посмотреть все комментарии