Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

500 failed to verify data: digests do not match; Image fails to pull. #303

Open
will-ds opened this issue Apr 1, 2021 · 1 comment
Open

Comments

@will-ds
Copy link

will-ds commented Apr 1, 2021

Describe the bug
Pulling an image (whether it be the origin, agent or manually) returns the following in the origin logs (/var/log/kraken/kraken-origin/) after re-deploying kraken. Image fails to pull.

INFO	handler/handler.go:98	500 GET /namespace/nvidia/cuda/blobs/sha256:2806d80fe28f9a42dfe4a1b30aeaefb3f75436d3c7fbfc6dfe7e71e1e5d3840a failed to verify data: digests do not match

To Reproduce
Steps to reproduce the behavior:

  1. Spin up Kraken using the helm chart. Configure GCS as the backend.
  2. Push an image via the kraken proxy.
  3. Run a separate deployment that uses this image. Verify it gets pulled and runs correctly.
  4. Tear down both deployments.
  5. Re-deploy kraken using helm chart using same configuration as Step 1.
  6. Run a new deployment using the same yaml file as Step 3.

Check logs of kraken-origin to find the above error.

Expected behavior
Image should pull down to origin(s).

Screenshots
If applicable, add screenshots to help explain your problem.

Environments

  • Kubernetes on Google Cloud
  • Kraken backend is Google Cloud Storage
  • Running Kraken 0.1.4

Additional context
Add any other context about the problem here--environment configurations, logs, etc.

Build index backend configuration:

    - namespace: .*
      backend:
        gcs:
          username: kraken-service-account
          bucket: gke_image_cache
          root_directory: /tags/
          name_path: docker_tag

Oirigin backend configuration:

    - namespace: .*
      backend:
        gcs:
          username: kraken-service-account
          bucket: gke_image_cache
          root_directory: /blobs/
          name_path: identity
@ish-xyz
Copy link

ish-xyz commented Jun 10, 2021

@will-ds I have the same error, did you find any solution? Unfortunately, debug logs don't return any additional info.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants