Skip to content

Podman containers do not reach certain hosts #27765

@ratzrattillo

Description

@ratzrattillo

Issue Description

Requests to some hosts from within a podman container seem to time out when not using --network=host.

This applies to e.g. https://acme-v02.api.letsencrypt.org/directory and several other hosts, but not to e.g. https://www.google.com

Steps to reproduce the issue

I have the problem that the following HTTP request in a container on rootless podman and rootful podman times out:

podman run --rm docker.io/curlimages/curl:latest curl --verbose https://acme-v02.api.letsencrypt.org/directory

When using the host network everything works fine:

podman run --rm --network=host docker.io/curlimages/curl:latest curl --verbose https://acme-v02.api.letsencrypt.org/directory

Describe the results you received

podman run --rm docker.io/curlimages/curl:latest curl --verbose https://acme-v02.api.letsencrypt.org/directory
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0   0     0   0     0     0     0  --:--:-- --:--:-- --:--:--     0* Host acme-v02.api.letsencrypt.org:443 was resolved.
* IPv6: 2606:4700:60:0:f53d:5624:85c7:3a2c
* IPv4: 172.65.32.248
*   Trying [2606:4700:60:0:f53d:5624:85c7:3a2c]:443...
* connect to 2606:4700:60:0:f53d:5624:85c7:3a2c port 443 from fe80::4e:f3ff:fee7:f70 port 43234 failed: Connection refused
*   Trying 172.65.32.248:443...
* ALPN: curl offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [1574 bytes data]
* SSL Trust Anchors:
*   CAfile: /cacert.pem
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [19 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2233 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [79 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*   subject: CN=acme-v02.api.letsencrypt.org
*   start date: Nov 25 16:47:03 2025 GMT
*   expire date: Feb 23 16:47:02 2026 GMT
*   issuer: C=US; O=Let's Encrypt; CN=E7
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
*   subjectAltName: "acme-v02.api.letsencrypt.org" matches cert's "acme-v02.api.letsencrypt.org"
* SSL certificate verified via OpenSSL.
* Established connection to acme-v02.api.letsencrypt.org (172.65.32.248 port 443) from 192.168.1.90 port 55126
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://acme-v02.api.letsencrypt.org/directory
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: acme-v02.api.letsencrypt.org]
* [HTTP/2] [1] [:path: /directory]
* [HTTP/2] [1] [user-agent: curl/8.17.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET /directory HTTP/2
> Host: acme-v02.api.letsencrypt.org
> User-Agent: curl/8.17.0
> Accept: */*
>
* Request completely sent off
  0     0   0     0   0     0     0     0  --:--:--  0:00:59 --:--:--     0} [5 bytes data]
* TLSv1.3 (OUT), TLS alert, decode error (562):
} [2 bytes data]
* OpenSSL SSL_read: OpenSSL/3.5.4: error:0A000126:SSL routines::unexpected eof while reading, errno 0
* Failed receiving HTTP2 data: 56(Failure when receiving data from the peer)
  0     0   0     0   0     0     0     0  --:--:--  0:01:00 --:--:--     0
* OpenSSL SSL_write: SSL_ERROR_SYSCALL, errno 0
* Connection #0 to host acme-v02.api.letsencrypt.org:443 left intact
curl: (56) OpenSSL SSL_read: OpenSSL/3.5.4: error:0A000126:SSL routines::unexpected eof while reading, errno 0

Describe the results you expected

podman run --rm docker.io/curlimages/curl:latest curl --verbose https://acme-v02.api.letsencrypt.org/directory
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0   0     0   0     0     0     0  --:--:-- --:--:-- --:--:--     0* Host acme-v02.api.letsencrypt.org:443 was resolved.
* IPv6: 2606:4700:60:0:f53d:5624:85c7:3a2c
* IPv4: 172.65.32.248
*   Trying [2606:4700:60:0:f53d:5624:85c7:3a2c]:443...
* Immediate connect fail for 2606:4700:60:0:f53d:5624:85c7:3a2c: Network unreachable
*   Trying 172.65.32.248:443...
* ALPN: curl offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [1574 bytes data]
* SSL Trust Anchors:
*   CAfile: /cacert.pem
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS change cipher, Change cipher spec (1):
{ [1 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [19 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2231 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [80 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / id-ecPublicKey
* ALPN: server accepted h2
* Server certificate:
*   subject: CN=acme-v02.api.letsencrypt.org
*   start date: Nov 26 16:22:17 2025 GMT
*   expire date: Feb 24 16:22:16 2026 GMT
*   issuer: C=US; O=Let's Encrypt; CN=E8
*   Certificate level 0: Public key type EC/prime256v1 (256/128 Bits/secBits), signed using ecdsa-with-SHA384
*   Certificate level 1: Public key type EC/secp384r1 (384/192 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
*   subjectAltName: "acme-v02.api.letsencrypt.org" matches cert's "acme-v02.api.letsencrypt.org"
* SSL certificate verified via OpenSSL.
* Established connection to acme-v02.api.letsencrypt.org (172.65.32.248 port 443) from 192.168.1.90 port 37380
  0     0   0     0   0     0     0     0  --:--:-- --:--:-- --:--:--     0* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://acme-v02.api.letsencrypt.org/directory
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: acme-v02.api.letsencrypt.org]
* [HTTP/2] [1] [:path: /directory]
* [HTTP/2] [1] [user-agent: curl/8.17.0]
* [HTTP/2] [1] [accept: */*]
} [5 bytes data]
> GET /directory HTTP/2
> Host: acme-v02.api.letsencrypt.org
> User-Agent: curl/8.17.0
> Accept: */*
>
* Request completely sent off
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [57 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [57 bytes data]
< HTTP/2 200
< server: nginx
< date: Sat, 13 Dec 2025 03:29:22 GMT
< content-type: application/json
< content-length: 1063
< cache-control: public, max-age=0, no-cache
< x-frame-options: DENY
< strict-transport-security: max-age=604800
<
{ {
  "VpWnck4lqh8": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "profiles": {
      "classic": "https://letsencrypt.org/docs/profiles#classic",
      "shortlived": "https://letsencrypt.org/docs/profiles#shortlived (not yet generally available)",
      "tlsclient": "https://letsencrypt.org/docs/profiles#tlsclient",
      "tlsserver": "https://letsencrypt.org/docs/profiles#tlsserver"
    },
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.6-August-18-2025.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "renewalInfo": "https://acme-v02.api.letsencrypt.org/acme/renewal-info",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}[1063 bytes data]
100  1063 100  1063   0     0  2409     0  --:--:-- --:--:-- --:--:--  2404
* Connection #0 to host acme-v02.api.letsencrypt.org:443 left intact

podman info output

host:
  arch: amd64
  buildahVersion: 1.42.2
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-1:2.1.13-1
    path: /usr/bin/conmon
    version: 'conmon version 2.1.13, commit: 82de887596ed8ee6d9b2ee85e4f167f307bb569b'
  cpuUtilization:
    idlePercent: 98.7
    systemPercent: 0.45
    userPercent: 0.85
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: arch
    version: unknown
  eventLogger: journald
  freeLocks: 2045
  hostname: wtrpro
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 961
      size: 1
    - container_id: 1
      host_id: 493216
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 962
      size: 1
    - container_id: 1
      host_id: 493216
      size: 65536
  kernel: 6.12.61-1-lts
  linkmode: dynamic
  logDriver: journald
  memFree: 29799337984
  memTotal: 33385009152
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.17.0-1
      path: /usr/lib/podman/aardvark-dns
      version: aardvark-dns 1.17.0
    package: netavark-1.17.1-1
    path: /usr/lib/podman/netavark
    version: netavark 1.17.1
  ociRuntime:
    name: crun
    package: crun-1.25.1-1
    path: /usr/bin/crun
    version: |-
      crun version 1.25.1
      commit: 156ae065d4a322d149c7307034f98d9637aa92a2
      rundir: /run/user/962/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-2025_12_10.d04c480-1
    version: |
      pasta 2025_12_10.d04c480
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/962/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 1h 44m 59.00s (Approximately 0.04 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries: {}
store:
  configFile: /var/lib/reverse-proxy/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/reverse-proxy/.local/share/containers/storage
  graphRootAllocated: 3816412684288
  graphRootUsed: 786524778496
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /run/user/962/containers
  transientStore: false
  volumePath: /var/lib/reverse-proxy/.local/share/containers/storage/volumes
version:
  APIVersion: 5.7.1
  Built: 1765447754
  BuiltTime: Thu Dec 11 11:09:14 2025
  GitCommit: f845d14e941889ba4c071f35233d09b29d363c75
  GoVersion: go1.25.5 X:nodwarf5
  Os: linux
  OsArch: linux/amd64
  Version: 5.7.1

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

Podman rootless user was created with:

user="xyz"
sudo useradd --system --add-subids-for-system --create-home --home-dir "/var/lib/${user}" ${user}
sudo loginctl enable-linger ${user}

A reset with podman system prune was already done - No change noticed.

Podman was also uninstalled with sudo pacman -Rns podman and reinstalled with sudo pacman -S podman.

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions