TransWikia.com

Jenkins - docker login doesn't seem to persist: docker pull won't work but docker-compose can pull without problems

DevOps Asked by Worp on August 22, 2021

I am setting up a new Jenkins as our old one is based on a deprecated base image.

Dockerfile is:

FROM jenkins/jenkins:lts-centos7

USER root

RUN yum -y install docker 
    && yum clean all

USER jenkins

When I docker exec ... into the container and log into our Sonatype Nexus to pull Docker images from it, the successful login does not seem to be heeded by Jenkins afterwards:

[root@909fb3a3d52c .docker]# docker login https://our.nexus.internal:<endpoint>
Username (jenkins): jenkins
Password:
Login Succeeded
[root@909fb3a3d52c .docker]# pwd
/root/.docker
[root@909fb3a3d52c .docker]# cat config.json
{
    "auths": {
        "https://our.nexus.internal:<endpoint>": {
            "auth": "<CORRECT base64 hash>"
        }
    }

Login has succeeded, the auth credentials are written to /root/.docker/config.json and when I base64-decode them they are correct "jenkins:".

However:

[root@909fb3a3d52c .docker]# docker pull https://our.nexus.internal:<endpoint>/myImage:myTag
Error response from daemon: Get https://our.nexus.internal:<endpoint>/v2/myImage/manifests/myTag: no basic auth credentials

This will also happen during execution of a Jenkins pipeline (obviously), where job log reports:

docker pull https://our.nexus.internal:<endpoint>/myImage:myTag

Error response from daemon: Get Error response from daemon: Get https://our.nexus.internal:<endpoint>/v2/myImage/manifests/myTag: no basic auth credentials

script returned exit code 1

I assume this to not be a question about nexus but about how the credentials from docker login are stored and used.

Sidenote: We have a different Jenkins currently operational which is based on jenkinsci/blueocean:1.22.0 which does NOT display this behaviour. It can docker login fine and docker pull fine.

edit1:

Running docker-compose does work while a manual docker pull... will not:

[root@3eeee032ff08 /]# docker login https://nexus:port
Username: jenkins
Password:
Login Succeeded

[root@3eeee032ff08 /]# docker pull nexus:port/company/image:myTag
Error response from daemon: Get https://nexus:port/v2/company/image/manifests/myTag: no basic auth credentials

[root@3eeee032ff08 /]# cat docker-compose.yml
version: '3.3'

services:
jenkins:
    user: root
    image: nexus:port/company/image:myTag

[root@3eeee032ff08 /]# docker-compose up

Creating network "default_default" with the default driver
Pulling jenkins (nexus:port/company/image:myTag)...
myTag: Pulling from company/image
f34b00c7da20: Pull complete
3f316190de62: Pull complete
480967924aca: Pull complete
619ac94672e0: Pull complete
cb1c6713a236: Pull complete
55766a17b1c6: Pull complete
8f6e881b2ef2: Pull complete
f118a721eee8: Pull complete
fae3c8e02b83: Pull complete
093ceb59408d: Pull complete
e3bd3cbf474d: Pull complete
41a6f4e5de6d: Pull complete
aa1aece4c754: Pull complete
fc897d8ca22f: Pull complete
Digest: sha256:3fd74169f648784fe960add9c0138e3802a91e82c9b5fdbd734faaeaaef696f9
Status: Downloaded newer image for nexus:port/company/image:myTag

edit2:

When not basing off of jenkins/jenkins:lts-centos7 but off of jenkins/jenkins:latest, which is a Debian base, then installing Docker on it using the official docs, login and pulling works:

worp@local$ docker run --rm -ti -u root -v /var/run/docker.sock:/var/run/docker.sock jenkins/jenkins:debian-docker bash

root@8aec8c792751:/# cat /etc/*-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

root@8aec8c792751:/# docker login https://nexus:port
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

root@8aec8c792751:/# cat /root/.docker/config.json
{
    "auths": {
        "nexus:port": {
            "auth": "<base64hash>"
        }
    },
    "HttpHeaders": {
        "User-Agent": "Docker-Client/19.03.12 (linux)"
    }
}

root@8aec8c792751:/# docker pull nexus:port/company/image:myTag
myTag: Pulling from company/image
Digest: sha256:3fd74169f648784fe960add9c0138e3802a91e82c9b5fdbd734faaeaaef696f9
Status: Image is up to date for nexus:port/company/image:myTag
nexus:port/company/image:myTag

This is getting weirder by the minute. I hope I am overlooking something extremely obvious so I feel very stupid afterwards.

2 Answers

I was able to produce a correct behaviour starting from centos:centos7 base image and installing Docker only (no Jenkins yet) via the steps provided in the official Docker docu: https://docs.docker.com/engine/install/centos/

Seen here:

[root@d8d441ae6a7a /]# history
    2  yum remove docker                   docker-client                   docker-client-latest                   docker-common                   docker-latest                   docker-latest-logrotate                   docker-logrotate                   docker-engine
    3  yum install -y yum-utils
    4  yum-config-manager     --add-repo     https://download.docker.com/linux/centos/docker-ce.repo
    5  yum install docker-ce docker-ce-cli containerd.io
    6  systemctl start docker
    7  docker ps
    8  docker run hello-world
    9  docker login https://nexus:port
10  docker login https://nexus:port
11  docker pull nexus:port/company/image:tag


# docker pull nexus:port/company/image:tag
...
Status: Downloaded newer image for nexus:port/company/image:tag

While I still have the faulty behaviour when I install docker via yum install -y docker. So it's pretty clear to assume that I'm not facing an issue with the Jenkins Docker images specifically but rather face centos problems with their Docker package.

I guess there is no need here for an answer. I will go on to talk to the centos people.

Cheers and thanks for your consideration everyone!

Correct answer by Worp on August 22, 2021

Based on your output, all of your docker commands are running as root when in general they should be running as the jenkins user. You can try switching the user to jenkins and running the same docker login and pull commands.

I've used the following in my Jenkins pipeline scripts to log into a Nexus 3 Docker repository (https://docs.cloudbees.com/docs/admin-resources/latest/plugins/docker-workflow):

docker.withRegistry('https://docker.mycorp.com/', 'docker-login') {
  git '…'
  docker.build('myapp').push('latest')
}

Answered by MSumulong on August 22, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP