fluentbit-operator日志：
Defaulted container "fluentbit-operator" out of: fluentbit-operator, setenv (init)
I0818 05:46:15.211951 1 request.go:668] Waited for 1.041511867s due to client-side throttling, not priority and fairness, request: GET:https://10.233.0.1:443/apis/certificates.k8s.io/v1?timeout=32s
2023-08-18T05:46:15.920Z INFO controller-runtime.metrics metrics server is starting to listen {"addr": ":8080"}
2023-08-18T05:46:15.925Z INFO setup starting manager
2023-08-18T05:46:16.228Z INFO controller-runtime.manager starting metrics server {"path": "/metrics"}
2023-08-18T05:46:16.228Z INFO controller-runtime.manager.controller.fluentbitconfig Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBitConfig", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.228Z INFO controller-runtime.manager.controller.fluentbitconfig Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBitConfig", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.228Z INFO controller-runtime.manager.controller.fluentbitconfig Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBitConfig", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.228Z INFO controller-runtime.manager.controller.fluentbitconfig Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBitConfig", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.228Z INFO controller-runtime.manager.controller.fluentbitconfig Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBitConfig", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.228Z INFO controller-runtime.manager.controller.fluentbitconfig Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBitConfig", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.228Z INFO controller-runtime.manager.controller.fluentbitconfig Starting Controller {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBitConfig"}
2023-08-18T05:46:16.229Z INFO controller-runtime.manager.controller.fluentbit Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBit", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.229Z INFO controller-runtime.manager.controller.fluentbit Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBit", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.229Z INFO controller-runtime.manager.controller.fluentbit Starting EventSource {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBit", "source": "kind source: /, Kind="}
2023-08-18T05:46:16.229Z INFO controller-runtime.manager.controller.fluentbit Starting Controller {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBit"}
2023-08-18T05:46:16.359Z INFO controller-runtime.manager.controller.fluentbitconfig Starting workers {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBitConfig", "worker count": 1}
2023-08-18T05:46:16.460Z INFO controller-runtime.manager.controller.fluentbit Starting workers {"reconciler group": "logging.kubesphere.io", "reconciler kind": "FluentBit", "worker count": 1}
opensearch-cluster-data-2日志：
Defaulted container "opensearch" out of: opensearch, fsgroup-volume (init)
Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755 **
**************************************************************************
OpenSearch Security Demo Installer
** Warning: Do not use on production or public reachable systems **
Basedir: /usr/share/opensearch
OpenSearch install type: rpm/deb on NAME="Amazon Linux"
OpenSearch config dir: /usr/share/opensearch/config
OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
OpenSearch bin dir: /usr/share/opensearch/bin
OpenSearch plugins dir: /usr/share/opensearch/plugins
OpenSearch lib dir: /usr/share/opensearch/lib
Detected OpenSearch Version: x-content-2.6.0
Detected OpenSearch Security Version: 2.6.0.0
tee: /usr/share/opensearch/config/opensearch.yml: Permission denied
Enabling OpenSearch Security Plugin
Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.6.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
WARNING: System::setSecurityManager will be removed in a future release
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.6.0.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
WARNING: System::setSecurityManager will be removed in a future release
[2023-08-18T06:13:24,576][INFO ][o.o.n.Node ] [opensearch-cluster-data-2] version[2.6.0], pid[46], build[tar/7203a5af21a8a009aece1474446b437a3c674db6/2023-02-24T18:57:04.388618985Z], OS[Linux/5.4.254-1.el7.elrepo.x86_64/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.6/17.0.6+10]
[2023-08-18T06:13:24,578][INFO ][o.o.n.Node ] [opensearch-cluster-data-2] JVM home [/usr/share/opensearch/jdk], using bundled JDK [true]
[2023-08-18T06:13:24,578][INFO ][o.o.n.Node ] [opensearch-cluster-data-2] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-14399892048983616024, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xmx1536M, -Xms1536M, -XX:MaxDirectMemorySize=805306368, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
[2023-08-18T06:13:25,869][WARN ][stderr ] [opensearch-cluster-data-2] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[2023-08-18T06:13:25,870][WARN ][stderr ] [opensearch-cluster-data-2] SLF4J: Defaulting to no-operation (NOP) logger implementation
[2023-08-18T06:13:25,870][WARN ][stderr ] [opensearch-cluster-data-2] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[2023-08-18T06:13:25,884][INFO ][o.o.s.s.t.SSLConfig ] [opensearch-cluster-data-2] SSL dual mode is disabled
[2023-08-18T06:13:25,885][INFO ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] OpenSearch Config path is /usr/share/opensearch/config
[2023-08-18T06:13:26,228][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-data-2] JVM supports TLSv1.3
[2023-08-18T06:13:26,230][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-data-2] Config directory is /usr/share/opensearch/config/, from there the key- and truststore files are resolved relatively
[2023-08-18T06:13:27,097][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-data-2] TLS Transport Client Provider : JDK
[2023-08-18T06:13:27,097][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-data-2] TLS Transport Server Provider : JDK
[2023-08-18T06:13:27,097][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-data-2] TLS HTTP Provider : JDK
[2023-08-18T06:13:27,098][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-data-2] Enabled TLS protocols for transport layer : [TLSv1.3, TLSv1.2]
[2023-08-18T06:13:27,098][INFO ][o.o.s.s.DefaultSecurityKeyStore] [opensearch-cluster-data-2] Enabled TLS protocols for HTTP layer : [TLSv1.3, TLSv1.2]
[2023-08-18T06:13:27,117][INFO ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] Clustername: opensearch-cluster
[2023-08-18T06:13:27,127][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] Directory /usr/share/opensearch/config has insecure file permissions (should be 0700)
[2023-08-18T06:13:27,127][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] File /usr/share/opensearch/config/opensearch.yml has insecure file permissions (should be 0600)
[2023-08-18T06:13:27,127][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] File /usr/share/opensearch/config/kirk.pem has insecure file permissions (should be 0600)
[2023-08-18T06:13:27,127][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] File /usr/share/opensearch/config/esnode.pem has insecure file permissions (should be 0600)
[2023-08-18T06:13:27,127][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] File /usr/share/opensearch/config/root-ca.pem has insecure file permissions (should be 0600)
[2023-08-18T06:13:27,128][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] File /usr/share/opensearch/config/esnode-key.pem has insecure file permissions (should be 0600)
[2023-08-18T06:13:27,128][WARN ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] File /usr/share/opensearch/config/kirk-key.pem has insecure file permissions (should be 0600)
[2023-08-18T06:13:27,777][INFO ][o.o.p.c.PluginSettings ] [opensearch-cluster-data-2] Config: metricsLocation: /dev/shm/performanceanalyzer/, metricsDeletionInterval: 1, httpsEnabled: false, cleanup-metrics-db-files: true, batch-metrics-retention-period-minutes: 7, rpc-port: 9650, webservice-port 9600
[2023-08-18T06:13:28,553][INFO ][o.o.i.r.ReindexPlugin ] [opensearch-cluster-data-2] ReindexPlugin reloadSPI called
[2023-08-18T06:13:28,554][INFO ][o.o.i.r.ReindexPlugin ] [opensearch-cluster-data-2] Unable to find any implementation for RemoteReindexExtension
[2023-08-18T06:13:28,617][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-data-2] Loaded scheduler extension: opendistro_anomaly_detector, index: .opendistro-anomaly-detector-jobs
[2023-08-18T06:13:28,661][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-data-2] Loaded scheduler extension: reports-scheduler, index: .opendistro-reports-definitions
[2023-08-18T06:13:28,663][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-data-2] Loaded scheduler extension: opendistro-index-management, index: .opendistro-ism-config
[2023-08-18T06:13:28,702][INFO ][o.o.j.JobSchedulerPlugin ] [opensearch-cluster-data-2] Loaded scheduler extension: observability, index: .opensearch-observability-job
[2023-08-18T06:13:28,711][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [aggs-matrix-stats]
[2023-08-18T06:13:28,711][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [analysis-common]
[2023-08-18T06:13:28,711][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [geo]
[2023-08-18T06:13:28,712][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [ingest-common]
[2023-08-18T06:13:28,712][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [ingest-geoip]
[2023-08-18T06:13:28,712][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [ingest-user-agent]
[2023-08-18T06:13:28,712][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [lang-expression]
[2023-08-18T06:13:28,712][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [lang-mustache]
[2023-08-18T06:13:28,712][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [lang-painless]
[2023-08-18T06:13:28,712][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [mapper-extras]
[2023-08-18T06:13:28,712][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [opensearch-dashboards]
[2023-08-18T06:13:28,713][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [parent-join]
[2023-08-18T06:13:28,713][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [percolator]
[2023-08-18T06:13:28,713][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [rank-eval]
[2023-08-18T06:13:28,713][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [reindex]
[2023-08-18T06:13:28,713][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [repository-url]
[2023-08-18T06:13:28,713][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [systemd]
[2023-08-18T06:13:28,713][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded module [transport-netty4]
[2023-08-18T06:13:28,714][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-alerting]
[2023-08-18T06:13:28,714][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-anomaly-detection]
[2023-08-18T06:13:28,714][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-asynchronous-search]
[2023-08-18T06:13:28,714][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-cross-cluster-replication]
[2023-08-18T06:13:28,715][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-geospatial]
[2023-08-18T06:13:28,715][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-index-management]
[2023-08-18T06:13:28,715][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-job-scheduler]
[2023-08-18T06:13:28,715][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-knn]
[2023-08-18T06:13:28,715][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-ml]
[2023-08-18T06:13:28,715][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-neural-search]
[2023-08-18T06:13:28,715][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-notifications]
[2023-08-18T06:13:28,716][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-notifications-core]
[2023-08-18T06:13:28,716][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-observability]
[2023-08-18T06:13:28,716][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-performance-analyzer]
[2023-08-18T06:13:28,716][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-reports-scheduler]
[2023-08-18T06:13:28,716][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-security]
[2023-08-18T06:13:28,716][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-security-analytics]
[2023-08-18T06:13:28,716][INFO ][o.o.p.PluginsService ] [opensearch-cluster-data-2] loaded plugin [opensearch-sql]
[2023-08-18T06:13:28,788][INFO ][o.o.s.OpenSearchSecurityPlugin] [opensearch-cluster-data-2] Disabled https compression by default to mitigate BREACH attacks. You can enable it by setting 'http.compression: true' in opensearch.yml
[2023-08-18T06:13:28,821][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [opensearch-cluster-data-2] uncaught exception in thread [main]
org.opensearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/opensearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:184) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-2.6.0.jar:2.6.0]
at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-2.6.0.jar:2.6.0]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103) ~[opensearch-2.6.0.jar:2.6.0]
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/opensearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:341) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.node.Node.<init>(Node.java:464) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.node.Node.<init>(Node.java:361) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-2.6.0.jar:2.6.0]
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180) ~[opensearch-2.6.0.jar:2.6.0]
... 6 more
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to obtain node locks, tried [[/usr/share/opensearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
at org.opensearch.env.NodeEnvironment.<init>(NodeEnvironment.java:341)
at org.opensearch.node.Node.<init>(Node.java:464)
at org.opensearch.node.Node.<init>(Node.java:361)
at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404)
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:180)
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:171)
at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
at org.opensearch.cli.Command.main(Command.java:101)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:137)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:103)
For complete error details, refer to the log at /usr/share/opensearch/logs/opensearch-cluster.log
