support http(s) URLs for oci-archive/docker-archive images

[ritesh-harihar]

Fixes: #434

Issue

Summary

The podman driver lets you run an image straight from a tar archive using the oci-archive: and docker-archive: transports. Previously this only worked for archives on the local filesystem. This PR lets the archive also live behind an http:// or https:// URL, so a job can pull a one-off image from a web server, object store, or artifact registry.

Fix

Detect archive references that point at an http(s) URL before they reach the path-only parser, and handle them on a dedicated path:

• Recognized oci-archive: / docker-archive: + http(s):// .
• Downloaded the URL (bounded by the task's image_pull_timeout and cancelled on driver shutdown).
• Stream the response directly into podman's image-load endpoint — no temp file, no extra disk usage, no cleanup.
• Surfaced a clear, early error on a bad URL (e.g. a 404) instead of feeding garbage to podman.

All existing paths (registry pulls, local archives, short names) are untouched.

Example job config:

config {
  image = "oci-archive:https://artifacts.example.com/images/myapp.tar"
  # docker-archive works the same way:
  # image = "docker-archive:https://artifacts.example.com/images/myapp.tar"
}

Before/ After Flow

Step	Before	After
Reference detected	Sent to path-only parser	URL form intercepted first
URL parsing	Invalid image //host:port/...	recognized as a download target
Download	Never attempted	GET the URL (timeout-bound, cancellable)
Bad URL (e.g. 404)	confusing parse error	unexpected status 404 ..., fails early
Load into podman	n/a (never reached)	Response body streamed straight to load endpoint (no temp file)
Result	task never starts	image loaded, task runs

Testing

Details

After Fix:

-> job-http-archive.nomad.hcl

job "http-archive" {
  type = "batch"

  group "demo" {
    task "myhttp" {
      driver = "podman"

      config {
        image   = "oci-archive:http://127.0.0.1:9999/myhttp.tar"
        command = "sh"
        args    = ["-c", "echo loaded-from-oci-http-OK && sleep 20"]
      }

      resources {
        cpu    = 100
        memory = 64
      }
    }
  }
}

Result:

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-podman/test/e2e/oci-archive-http$ nomad alloc status cde66313
ID                  = cde66313-7a6c-6e4b-870c-9d39d71f5d3e
Eval ID             = efabd825
Name                = http-archive.demo[0]
Node ID             = 0ab78424
Node Name           = podman-dev
Job ID              = http-archive
Job Version         = 0
Client Status       = complete
Client Description  = All tasks have completed
Desired Status      = run
Desired Description = <none>
Created             = 37s ago
Modified            = 12s ago

Task "myhttp" is "dead"
Task Resources:
CPU        Memory         Disk     Addresses
0/100 MHz  44 KiB/64 MiB  300 MiB  

Task Events:
Started At     = 2026-06-24T07:10:47Z
Finished At    = 2026-06-24T07:11:09Z
Total Restarts = 0
Last Restart   = N/A

Recent Events:
Time                       Type        Description
2026-06-24T12:41:09+05:30  Terminated  Exit Code: 0
2026-06-24T12:40:47+05:30  Started     Task started by client
2026-06-24T12:40:47+05:30  Driver      Loading image http://127.0.0.1:9999/myhttp.tar
2026-06-24T12:40:47+05:30  Task Setup  Building Task Directory
2026-06-24T12:40:47+05:30  Received    Task received by client

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-podman/test/e2e/oci-archive-http$ nomad alloc logs cde66313
2026-06-24T12:40:47.966599307+05:30 stdout F loaded-from-oci-http-OK

-> job-http-archive-404.nomad.hcl

job "http-archive-404" {
  type = "batch"

  group "demo" {
    task "myhttp" {
      driver = "podman"

      config {
        image   = "oci-archive:http://127.0.0.1:9999/does-not-exist.tar"
        command = "sh"
        args    = ["-c", "echo should-not-run"]
      }

      resources {
        cpu    = 100
        memory = 64
      }
    }
  }
}

Result:

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-podman/test/e2e/oci-archive-http$ nomad alloc status b8814f10 
ID                  = b8814f10-868d-34af-01c4-8c3ea8d6c0e0
Eval ID             = 9f8ab5d6
Name                = http-archive-404.demo[0]
Node ID             = 17dc491f
Node Name           = podman-dev
Job ID              = http-archive-404
Job Version         = 0
Client Status       = failed
Client Description  = Failed tasks
Desired Status      = run
Desired Description = <none>
Created             = 38s ago
Modified            = 34s ago
Reschedule Attempts = 1/1

Task "myhttp" is "dead"
Task Resources:
CPU      Memory  Disk     Addresses
100 MHz  64 MiB  300 MiB  

Task Events:
Started At     = N/A
Finished At    = 2026-06-24T07:40:02Z
Total Restarts = 0
Last Restart   = N/A

Recent Events:
Time                       Type            Description
2026-06-24T13:10:02+05:30  Not Restarting  Error was unrecoverable
2026-06-24T13:10:02+05:30  Driver Failure  rpc error: code = Unknown desc = failed to create image: oci-archive:http://127.0.0.1:9999/does-not-exist.tar: failed to download image archive http://127.0.0.1:9999/does-not-exist.tar: unexpected status 404 File not found
2026-06-24T13:10:02+05:30  Driver          Loading image http://127.0.0.1:9999/does-not-exist.tar
2026-06-24T13:10:02+05:30  Task Setup      Building Task Directory
2026-06-24T13:10:02+05:30  Received        Task received by client

-> local-archive.nomad.hcl

job "local-archive" {
  type = "batch"

  group "demo" {
    task "myhttp" {
      driver = "podman"

      config {
        image   = "oci-archive:/tmp/web/myhttp.tar"
        command = "sh"
        args    = ["-c", "echo loaded-from-local-oci-OK && sleep 20"]
      }

      resources {
        cpu    = 100
        memory = 64
      }
    }
  }
}

Result:

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-podman/test/e2e/oci-archive-http$ nomad job run jobs/local-archive.nomad.hcl
==> 2026-06-24T13:14:25+05:30: Monitoring evaluation "6c04c944"
    2026-06-24T13:14:25+05:30: Evaluation triggered by job "local-archive"
    2026-06-24T13:14:26+05:30: Allocation "3b1d4f4c" created: node "17dc491f", group "demo"
    2026-06-24T13:14:26+05:30: Evaluation status changed: "pending" -> "complete"
==> 2026-06-24T13:14:26+05:30: Evaluation "6c04c944" finished with status "complete"
riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-podman/test/e2e/oci-archive-http$ nomad job status local-archive
ID            = local-archive
Name          = local-archive
Submit Date   = 2026-06-24T13:14:25+05:30
Type          = batch
Priority      = 50
Datacenters   = *
Namespace     = default
Node Pool     = default
Status        = running
Periodic      = false
Parameterized = false

Summary
Task Group  Queued  Starting  Running  Failed  Complete  Lost  Unknown
demo        0       0         1        0       0         0     0

Allocations
ID        Node ID   Task Group  Version  Desired  Status   Created  Modified
3b1d4f4c  17dc491f  demo        0        run      running  13s ago  12s ago
riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-podman/test/e2e/oci-archive-http$ nomad alloc logs 3b1d4f4c
2026-06-24T13:14:25.742780422+05:30 stdout F loaded-from-local-oci-OK

Before: tested for job http-archive

riteshharihar@podman-dev:/Users/riteshharihar/Desktop/Src/nomad-driver-podman/test/e2e/oci-archive-http$ nomad alloc status 6b627a41
ID                   = 6b627a41-0bd3-4472-81a8-2cb6a83a4dfa
Eval ID              = c8d5dfb5
Name                 = http-archive.demo[0]
Node ID              = 66b37253
Node Name            = podman-dev
Job ID               = http-archive
Job Version          = 0
Client Status        = failed
Client Description   = Failed tasks
Desired Status       = stop
Desired Description  = alloc was rescheduled because it failed
Created              = 25s ago
Modified             = 20s ago
Replacement Alloc ID = 8d60257f

Task "myhttp" is "dead"
Task Resources:
CPU      Memory  Disk     Addresses
100 MHz  64 MiB  300 MiB  

Task Events:
Started At     = N/A
Finished At    = 2026-06-24T07:57:21Z
Total Restarts = 0
Last Restart   = N/A

Recent Events:
Time                       Type            Description
2026-06-24T13:27:21+05:30  Not Restarting  Error was unrecoverable
2026-06-24T13:27:21+05:30  Driver Failure  rpc error: code = Unknown desc = failed to create image: oci-archive:http://127.0.0.1:9999/myhttp.tar: invalid image reference oci-archive:http://127.0.0.1:9999/myhttp.tar: Invalid image //127.0.0.1:9999/myhttp.tar
2026-06-24T13:27:21+05:30  Task Setup      Building Task Directory
2026-06-24T13:27:21+05:30  Received        Task received by client

• If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

[tehut]

I tested this locally and it worked fine for both docker-archive and oci-archive, I also got to see the error case a few times when the driver couldn't find or translate my image. But I did leave a first pass of notes on implementation and tests.

And just a reminder that it really helps me save time to have a step-by-step of whatever prep is required in the Testing section. It wasn't too much effort to look up the process of saving docker & podman archive or writing a little serve script. But the time to look up & troubleshoot these little things that we don't do very often can add up.

And any steps you've already done that I can re-use gives me more time with your code. Especially now that we're budgeting our AI usage

[ritesh-harihar]

And just a reminder that it really helps me save time to have a step-by-step of whatever prep is required in the Testing section. It wasn't too much effort to look up the process of saving docker & podman archive or writing a little serve script. But the time to look up & troubleshoot these little things that we don't do very often can add up.

And any steps you've already done that I can re-use gives me more time with your code. Especially now that we're budgeting our AI usage (though I tend to do my own googling).

Noted. I'll make this my default going forward, every PR with a Testing section will include copy-paste prep so there's nothing to look up or troubleshoot.

[tehut]

Noted. I'll make this my default going forward, every PR with a Testing section will include copy-paste prep so there's nothing to look up or troubleshoot.

Thanks, @ritesh-harihar! I really appreciate it and also appreciate the job specs and configs you've been including up until now. I hope I didn't phrase that in a way that suggested I didn't recognize that you've already been including more set up context in your PRs than we ordinarily would need when everyone is working on the same big repo where 99% of the set up is just built in to the makefile.

[tehut]

One small fix and I think we're there!

[tehut]

Looks good!