yudong
碰到类似的问题:
1,开始发现流水线 “未运行”,查看devops-jenkins日志:
023-08-21 03:07:25.422+0000 [id=20295] WARNING i.k.j.d.a.KubesphereApiTokenAuthenticator#authenticate: API token matched for user liding2 but the impersonation failed
org.acegisecurity.userdetails.UsernameNotFoundException: User liding2 not found in directory.
at org.acegisecurity.ldap.search.FilterBasedLdapUserSearch.searchForUser(FilterBasedLdapUserSearch.java:126)
at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:1320)
at hudson.security.LDAPSecurityRealm$LDAPUserDetailsService.loadUserByUsername(LDAPSecurityRealm.java:1273)
at org.acegisecurity.userdetails.UserDetailsService.lambda$toSpring$1(UserDetailsService.java:52)
Caused: org.springframework.security.core.userdetails.UsernameNotFoundException
at org.acegisecurity.userdetails.UsernameNotFoundException.toSpring(UsernameNotFoundException.java:51)
at org.acegisecurity.userdetails.UsernameNotFoundException.toSpring(UsernameNotFoundException.java:34)
at org.acegisecurity.userdetails.UserDetailsService.lambda$toSpring$1(UserDetailsService.java:54)
at jenkins.security.ImpersonatingUserDetailsService2.loadUserByUsername(ImpersonatingUserDetailsService2.java:29)
at hudson.model.User.getUserDetailsForImpersonation2(User.java:406)
at hudson.model.User.getUserDetailsForImpersonation(User.java:429)
Caused: org.acegisecurity.userdetails.UsernameNotFoundException: org.springframework.security.core.userdetails.UsernameNotFoundException: org.acegisecurity.userdetails.UsernameNotFoundException: User liding2 not found in directory.; nested exception is org.springframework.security.core.userdetails.UsernameNotFoundException: org.acegisecurity.userdetails.UsernameNotFoundException: User liding2 not found in directory.
2,按照你前面提的三个步骤操作了,流水线又成功运行了,一会儿后又发现流水线"未运行"
3,检查Jenkins的role-strategy/assign-roles
4,查看devops-controller日志
E0821 03:07:25.426300 1 pipelinerun_controller.go:205] pipelinerun-controller “msg”=“unable to run pipeline” “error”=“unexpected status code: 500” “Pipeline”=“elf” “PipelineRun”={“Namespace”:“tools6rwrl”,“Name”:“elf-gqmc7”} “namespace”=“tools6rwrl” “pipeline”=“elf”
E0821 03:07:25.426407 1 controller.go:326] “msg”=“Reconciler error” “error”=“unexpected status code: 500” “controller”=“pipelinerun” “controllerGroup”=“devops.kubesphere.io” “controllerKind”=“PipelineRun” “name”=“elf-gqmc7” “namespace”=“tools6rwrl” “pipelineRun”={“name”:“elf-gqmc7”,“namespace”:“tools6rwrl”} “reconcileID”=“6d87a5d2-3543-40d7-94d3-ec3d14fd0bef”
<nil>
<nil>
E0821 03:09:32.357775 1 devopscredential_controller.go:294] unexpected error type: not found resources, should be *restful.ServiceError
E0821 03:09:32.357826 1 devopscredential_controller.go:167] error syncing ‘cssgzndq/gitlab-token’: failed to remove devops credential finalizer due to bad communication with Jenkins, requeuingcould not reconcile devopsProject
E0821 03:09:32.357834 1 devopscredential_controller.go:168] error syncing ‘cssgzndq/gitlab-token’: failed to remove devops credential finalizer due to bad communication with Jenkins, requeuing
E0821 03:09:32.428121 1 devopscredential_controller.go:294] unexpected error type: not found resources, should be *restful.ServiceError
E0821 03:09:32.428175 1 devopscredential_controller.go:167] error syncing ‘cssgzndq/k8s-token’: failed to remove devops credential finalizer due to bad communication with Jenkins, requeuingcould not reconcile devopsProject
E0821 03:09:32.428183 1 devopscredential_controller.go:168] error syncing ‘cssgzndq/k8s-token’: failed to remove devops credential finalizer due to bad communication with Jenkins, requeuing
E0821 03:09:32.449208 1 devopscredential_controller.go:294] unexpected error type: not found resources, should be *restful.ServiceError
E0821 03:09:32.449252 1 devopscredential_controller.go:167] error syncing ‘uosjmdbk/gitlab-token’: failed to remove devops credential finalizer due to bad communication with Jenkins, requeuingcould not reconcile devopsProject
E0821 03:09:32.449260 1 devopscredential_controller.go:168] error syncing ‘uosjmdbk/gitlab-token’: failed to remove devops credential finalizer due to bad communication with Jenkins, requeuing
E0821 03:09:32.461410 1 devopscredential_controller.go:294] unexpected error type: not found resources, should be *restful.ServiceError
E0821 03:09:32.461456 1 devopscredential_controller.go:167] error syncing ‘uosjmdbk/k8s-token’: failed to remove devops credential finalizer due to bad communication with Jenkins, requeuingcould not reconcile devopsProject
E0821 03:09:32.461465 1 devopscredential_controller.go:168] error syncing ‘uosjmdbk/k8s-token’: failed to remove devops credential finalizer due to bad communication with Jenkins, requeuing
<nil>
<nil>
<nil>
<nil>
E0821 03:14:01.416187 1 pipelinerun_controller.go:205] pipelinerun-controller “msg”=“unable to run pipeline” “error”=“unexpected status code: 500” “Pipeline”=“css” “PipelineRun”={“Namespace”:“tools6rwrl”,“Name”:“css-vc4d8”} “namespace”=“tools6rwrl” “pipeline”=“css”
E0821 03:14:01.416736 1 controller.go:326] “msg”=“Reconciler error” “error”=“unexpected status code: 500” “controller”=“pipelinerun” “controllerGroup”=“devops.kubesphere.io” “controllerKind”=“PipelineRun” “name”=“css-vc4d8” “namespace”=“tools6rwrl” “pipelineRun”={“name”:“css-vc4d8”,“namespace”:“tools6rwrl”} “reconcileID”=“7917f670-219b-4738-b3fa-311dedcd39fe”
E0821 03:14:25.682510 1 pipelinerun_controller.go:205] pipelinerun-controller “msg”=“unable to run pipeline” “error”=“unexpected status code: 500” “Pipeline”=“elf” “PipelineRun”={“Namespace”:“tools6rwrl”,“Name”:“elf-rhgbf”} “namespace”=“tools6rwrl” “pipeline”=“elf”
E0821 03:14:25.682684 1 controller.go:326] “msg”=“Reconciler error” “error”=“unexpected status code: 500” “controller”=“pipelinerun” “controllerGroup”=“devops.kubesphere.io” “controllerKind”=“PipelineRun” “name”=“elf-rhgbf” “namespace”=“tools6rwrl” “pipelineRun”={“name”:“elf-rhgbf”,“namespace”:“tools6rwrl”} “reconcileID”=“1489e14b-bb48-4ca7-8472-2a6ca2ee2ad6”
5,对比devops-config,kubesphere-config,kubesphere-secret
6,疑惑的是kubesphere-devops-system/secrets/devops-jenkins中定义的jenkins-admin-password无法登录Jenkins,重启了Jenkins之后发现也登陆不了,最终用的是默认密码登录。