创建部署问题时,请参考下面模板,你提供的信息越多,越容易及时获得解答。如果未按模板创建问题,管理员有权关闭问题。
确保帖子格式清晰易读,用 markdown code block 语法格式化代码块。
你只花一分钟创建的问题,不能指望别人花上半个小时给你解答。
操作系统信息
例如:虚拟机,Centos7.54C/8G
Kubernetes版本信息
Client Version: version.Info{Major:“1”, Minor:“22”, GitVersion:“v1.22.12”, GitCommit:“b058e1760c79f46a834ba59bd7a3486ecf28237d”, GitTreeState:“clean”, BuildDate:“2022-07-13T14:59:18Z”, GoVersion:“go1.16.15”, Compiler:“gc”, Platform:“linux/amd64”}
Server Version: version.Info{Major:“1”, Minor:“22”, GitVersion:“v1.22.12”, GitCommit:“b058e1760c79f46a834ba59bd7a3486ecf28237d”, GitTreeState:“clean”, BuildDate:“2022-07-13T14:53:39Z”, GoVersion:“go1.16.15”, Compiler:“gc”, Platform:“linux/amd64”}
容器运行时
将 docker version
/ crictl version
/ nerdctl version
结果贴在下方
KubeSphere版本信息
3.4.1(kk 工具在线安装3.3.1 后升级3.4.1)
问题是什么
流水线跑不动了。我检查了devops上的日志,发现证书已过期。然后,我尝试更新 k8s 的证书(kubeadm 证书全部更新),但没有生效。然后,我尝试升级kubeshee,但仍然无法解决问题。我去开发者社区检查(https://ask.kubesphere.io/forum/d/23239-kubesphere-jing-xiang-gou-jian-qi-s2ifu-wu-zheng-shu-guo-qi-wen-ti)
修复了证书过期的问题,但在操作流水线时提示如下消息:
然后我检查了devops-apiserver的日志,不断提示以下内容:
然后我检查了 devops Jenkins 的日志,发现用户 wukai1 不存在:
但是 kubeshpere 中没有用户 wukai1,
相关日志输出
2024-04-07 14:40:13.390+0000 [id=143] WARNING h.i.i.InstallUncaughtExceptionHandler#handleException: Caught unhandled exception with ID 6d1c45c0-a747-44b8-b37a-0074f07ab25e
org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.
at org.acegisecurity.ldap.search.FilterBasedLdapUserSearch.searchForUser(FilterBasedLdapUserSearch.java:126)
at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:1320)
at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:1273)
at org.acegisecurity.userdetails.UserDetailsService.lambda$toSpring$1(UserDetailsService.java:52)
Caused: org.springframework.security.core.userdetails.UsernameNotFoundException
at org.acegisecurity.userdetails.UsernameNotFoundException.toSpring(UsernameNotFoundException.java:51)
at org.acegisecurity.userdetails.UsernameNotFoundException.toSpring(UsernameNotFoundException.java:34)
at org.acegisecurity.userdetails.UserDetailsService.lambda$toSpring$1(UserDetailsService.java:54)
at jenkins.security.ImpersonatingUserDetailsService2.loadUserByUsername(ImpersonatingUserDetailsService2.java:29)
at hudson.model.User.getUserDetailsForImpersonation2(User.java:406)
at hudson.model.User.getUserDetailsForImpersonation(User.java:429)
Caused: org.acegisecurity.userdetails.UsernameNotFoundException: org.springframework.security.core.userdetails.UsernameNotFoundException: org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.; nested exception is org.springframework.security.core.userdetails.UsernameNotFoundException: org.acegisecurity.userdetails.UsernameNotFoundException: User wukai1 not found in directory.
at org.acegisecurity.userdetails.UsernameNotFoundException.fromSpring(UsernameNotFoundException.java:58)
at org.acegisecurity.AuthenticationException.fromSpring(AuthenticationException.java:98)
at hudson.model.User.getUserDetailsForImpersonation(User.java:431)
at io.kubesphere.jenkins.devops.auth.KubesphereApiTokenAuthenticator.authenticate(KubesphereApiTokenAuthenticator.java:47)
Caused: javax.servlet.ServletException
at io.kubesphere.jenkins.devops.auth.KubesphereApiTokenAuthenticator.authenticate(KubesphereApiTokenAuthenticator.java:54)
at jenkins.security.BasicHeaderAuthenticator.authenticate2(BasicHeaderAuthenticator.java:43)
at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:83)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:62)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:109)
at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:51)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:85)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:39)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
通过控制台登录 Jenkins 并收到用户名和密码错误消息。devops Jenkins 响应日志如下:
2024-04-08 01:34:23.748+0000 [id=3836] 警告 o.a.p.l.a.BindAuthenticator2#handleBindException:绑定到 LDAP 失败:userDnuid=admin,ou=Users,dc=kubesphere,dc=io username=admin
javax.naming.AuthenticationException:[LDAP:错误代码 49 - 凭据无效]
at java.naming/com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3259)
at java.naming/com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3205)
at java.naming/com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2991)
at java.naming/com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2905)
at java.naming/com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:348)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxFromUrl(LdapCtxFactory.java:262)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(LdapCtxFactory.java:226)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(LdapCtxFactory.java:280)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(LdapCtxFactory.java:185)
at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(LdapCtxFactory.java:115)
at java.naming/javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:730)
at java.naming/javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:305)
at java.naming/javax.naming.InitialContext.init(InitialContext.java:236)
at java.naming/javax.naming.InitialContext.<init>(InitialContext.java:208)
at java.naming/javax.naming.directory.InitialDirContext.<init>(InitialDirContext.java:101)
at org.acegisecurity.ldap.DefaultInitialDirContextFactory.connect(DefaultInitialDirContextFactory.java:180)
at org.acegisecurity.ldap.DefaultInitialDirContextFactory.newInitialDirContext(DefaultInitialDirContextFactory.java:261)
at org.acegisecurity.ldap.LdapTemplate.execute(LdapTemplate.java:123)
at org.acegisecurity.ldap.LdapTemplate.retrieveEntry(LdapTemplate.java:165)
at org.acegisecurity.providers.ldap.authenticator.BindAuthenticator.bindWithDn(BindAuthenticator.java:87)
at org.acegisecurity.providers.ldap.authenticator.BindAuthenticator.authenticate(BindAuthenticator.java:72)
at org.acegisecurity.providers.ldap.authenticator.BindAuthenticator2.authenticate(BindAuthenticator2.java:49)
at org.acegisecurity.providers.ldap.LdapAuthenticationProvider.retrieveUser(LdapAuthenticationProvider.java:233)
at org.acegisecurity.providers.dao.AbstractUserDetailsAuthenticationProvider$1.retrieveUser(AbstractUserDetailsAuthenticationProvider.java:52)
at org.springframework.security.authentication.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:133)
at org.acegisecurity.providers.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:66)
at org.acegisecurity.providers.ProviderManager.doAuthentication(ProviderManager.java:200)
at org.acegisecurity.AbstractAuthenticationManager.authenticate(AbstractAuthenticationManager.java:47)
at hudson.security.LDAPSecurityRealm$LDAPAuthenticationManager.authenticate(LDAPSecurityRealm.java:1019)
at org.acegisecurity.AuthenticationManager.lambda$toSpring$1(AuthenticationManager.java:48)
at org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter.attemptAuthentication(UsernamePasswordAuthenticationFilter.java:85)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:222)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:212)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:97)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80)
at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:62)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:97)
at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:109)
at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:51)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:85)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:39)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:578)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
其他信息:
$$
Kubernetes:v1.22.12
kubesphere: 3.4.1(member:3.3.1)
devops-apiserver:ks-v3.4.1
ks-jenkins:v3.4.0-2.319.3-1
other:dex:v2.29.0(通过gitlab 登录)
$$