Image building fails

[kdambekalns]

Describe the bug

To Reproduce

With an empty directory, run ddev config, then add the addon, run ddev start

This is what I got:

$ mkdir test
$ cd test
$ ddev config
Creating a new DDEV project config in the current directory (/Users/karsten/test)
Once completed, your configuration will be written to /Users/karsten/test/.ddev/config.yaml

Project name (test):

The docroot is the directory from which your site is served.
This is a relative path from your project root at /Users/karsten/test

Leave docroot empty (hit <RETURN>) to use the location shown in parentheses.
Or specify a custom path if your index.php is in a different directory.
Or use '.' (a dot) to explicitly set it to the project root.

Docroot Location (project root):
Found a php codebase at /Users/karsten/test.
Project Type [backdrop, cakephp, craftcms, drupal, drupal6, drupal7, drupal8, drupal9, drupal10, drupal11, generic, laravel, magento, magento2, php, shopware6, silverstripe, symfony, typo3, wordpress] (php):
Configuration complete. You may now run 'ddev start'.

Now I install the add-on, my branch is from main and just allows to install on Apple Silicon, no other changes:

$ ddev add-on get https://github.com/kdambekalns/ddev-sqlsrv/tarball/task/support-apple-silicon

Executing pre-install actions:
👍 Checking DDEV version
To make this package work on arm64 machines, Rosetta is used.

👍 Check architecture type for incompatible arm64 type
👍 Check for existing Dockerfile.sqlsrv

Installing project-level components:
👍 docker-compose.sqlsrv.yaml
👍 web-build/pre.Dockerfile.sqlsrv
👍 config.sqlsrv.yaml
👍 commands/sqlsrv/drupal-regex
👍 commands/sqlsrv/sqlcmd

Executing post-install actions:

Installed DDEV add-on https://github.com/kdambekalns/ddev-sqlsrv/tarball/task/support-apple-silicon, use `ddev restart` to enable.
Installed sqlsrv: from https://github.com/kdambekalns/ddev-sqlsrv/tarball/task/support-apple-silicon

Ok, now I start:

$ ddev start
Network ddev_default created
Starting test...
 Container ddev-ssh-agent  Created
 Container ddev-ssh-agent  Started
ssh-agent container is running: If you want to add authentication to the ssh-agent container, run 'ddev auth ssh' to enable your keys.
Using custom web-build configuration: [
	/Users/karsten/test/.ddev/web-build/pre.Dockerfile.sqlsrv
]
Custom configuration is updated on restart.
If you don't see your custom configuration taking effect, run 'ddev restart'.
2019-CU12-ubuntu-20.04: Pulling from mssql/server
a31c7b29f4ad: Pull complete
a039ba299d1e: Pull complete
1d60c867fae0: Pull complete
b9278f283ade: Pull complete
120d64e87cdd: Pull complete
Digest: sha256:3a64da47fb2c8b4d730856b06930999a09bf7ed4eab2d540fae2c7063da7a4fd
Status: Downloaded newer image for mcr.microsoft.com/mssql/server:2019-CU12-ubuntu-20.04
mcr.microsoft.com/mssql/server:2019-CU12-ubuntu-20.04
The hostname test.ddev.site is not currently resolvable, trying to add it to the hosts file
DDEV needs to run with administrative privileges.
This is required to add unresolvable hostnames to the hosts file.
You may need to enter your password for sudo or allow escalation.
DDEV is about to issue the command:
  sudo --preserve-env=HOME /opt/homebrew/bin/ddev hostname test.ddev.site 127.0.0.1

Password:

Building project images......failed to solve: process "/bin/bash -c (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::=\"--force-confold\" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php$DDEV_PHP_VERSION-dev     unixodbc=$odbc_version     odbcinst=$odbc_version     odbcinst1debian2=$odbc_version     unixodbc-dev=$odbc_version     locales" did not complete successfully: exit code: 100

Failed to start test: docker-compose build failed: composeCmd failed to run 'COMPOSE_PROJECT_NAME=ddev-test docker-compose -f /Users/karsten/test/.ddev/.ddev-docker-compose-full.yaml --progress=plain build', action='[--progress=plain build]', err='exit status 1', stdout='#0 building with "default" instance using docker driver

#1 [web internal] load build definition from Dockerfile
#1 transferring dockerfile: 3.02kB done
#1 DONE 0.0s

#2 [web internal] load metadata for docker.io/ddev/ddev-webserver:v1.24.3
#2 DONE 0.0s

#3 [web internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [web  1/21] FROM docker.io/ddev/ddev-webserver:v1.24.3
#4 DONE 0.0s

#5 [web  4/21] RUN (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests apt-utils curl gnupg2 ca-certificates
#5 CACHED

#6 [web  6/21] RUN apt-key add <microsoft.asc
#6 CACHED

#7 [web  5/21] RUN curl -sSL -O https://packages.microsoft.com/keys/microsoft.asc
#7 CACHED

#8 [web  2/21] RUN getent group tty || groupadd tty
#8 CACHED

#9 [web  3/21] RUN (groupadd --gid 20 "karsten" || groupadd "karsten" || true) && (useradd -G tty -l -m -s "/bin/bash" --gid "karsten" --comment '' --uid 501 "karsten" || useradd -G tty -l -m -s "/bin/bash" --gid "karsten" --comment '' "karsten" || useradd  -G tty -l -m -s "/bin/bash" --gid "20" --comment '' "karsten" || useradd -G tty -l -m -s "/bin/bash" --comment '' karsten )
#9 CACHED

#10 [web  7/21] RUN curl -sSL -o /etc/apt/sources.list.d/mssql-release.list https://packages.microsoft.com/config/debian/11/prod.list
#10 CACHED

#11 [web  8/21] RUN (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php8.3-dev     unixodbc=2.3.7     odbcinst=2.3.7     odbcinst1debian2=2.3.7     unixodbc-dev=2.3.7     locales
#11 0.295 Hit:1 http://deb.debian.org/debian bookworm InRelease
#11 0.309 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#11 0.358 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#11 0.458 Hit:4 http://nginx.org/packages/debian bookworm InRelease
#11 0.474 Get:5 https://packages.sury.org/php bookworm InRelease [7551 B]
#11 0.518 Get:6 https://packages.microsoft.com/debian/11/prod bullseye InRelease [3649 B]
#11 0.530 Get:7 https://packages.sury.org/php bookworm/main arm64 Packages [252 kB]
#11 0.616 Hit:9 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg InRelease
#11 0.675 Get:10 https://dl.cloudsmith.io/public/platformsh/cli/deb/debian bookworm InRelease [3675 B]
#11 0.800 Get:11 https://packages.microsoft.com/debian/11/prod bullseye/main amd64 Packages [191 kB]
#11 0.837 Get:12 https://dl.cloudsmith.io/public/platformsh/upsun-cli/deb/debian bookworm InRelease [3681 B]
#11 0.924 Get:13 https://packages.microsoft.com/debian/11/prod bullseye/main all Packages [1444 B]
#11 0.979 Get:14 https://dl.cloudsmith.io/public/symfony/stable/deb/debian bookworm InRelease [4422 B]
#11 0.987 Get:15 https://packages.microsoft.com/debian/11/prod bullseye/main armhf Packages [37.1 kB]
#11 1.012 Get:16 https://packages.microsoft.com/debian/11/prod bullseye/main arm64 Packages [41.2 kB]
#11 1.015 Hit:8 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/debian bookworm InRelease
#11 1.048 Hit:17 http://packages.blackfire.io/debian any InRelease
#11 1.083 Fetched 649 kB in 1s (711 kB/s)
#11 1.083 Reading package lists...
#11 1.467 W: https://packages.microsoft.com/debian/11/prod/dists/bullseye/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
#11 1.476 Reading package lists...
#11 1.806 Building dependency tree...
#11 1.889 Reading state information...
#11 1.893 Package unixodbc-dev is not available, but is referred to by another package.
#11 1.893 This may mean that the package is missing, has been obsoleted, or
#11 1.893 is only available from another source
#11 1.893
#11 1.893 Package unixodbc is not available, but is referred to by another package.
#11 1.893 This may mean that the package is missing, has been obsoleted, or
#11 1.893 is only available from another source
#11 1.893 However the following packages replace it:
#11 1.893   odbcinst odbcinst:amd64 odbcinst1debian2 libodbc1 odbcinst1debian2:amd64
#11 1.893   libodbc1:amd64
#11 1.893
#11 1.893 Package odbcinst is not available, but is referred to by another package.
#11 1.893 This may mean that the package is missing, has been obsoleted, or
#11 1.893 is only available from another source
#11 1.893 However the following packages replace it:
#11 1.893   unixodbc-common
#11 1.893
#11 1.893 Package odbcinst1debian2 is not available, but is referred to by another package.
#11 1.893 This may mean that the package is missing, has been obsoleted, or
#11 1.893 is only available from another source
#11 1.893 However the following packages replace it:
#11 1.893   odbcinst1debian2:amd64 unixodbc-common libodbcinst2
#11 1.893
#11 1.894 E: Version '2.3.7' for 'unixodbc' was not found
#11 1.894 E: Version '2.3.7' for 'odbcinst' was not found
#11 1.894 E: Version '2.3.7' for 'odbcinst1debian2' was not found
#11 1.894 E: Version '2.3.7' for 'unixodbc-dev' was not found
#11 ERROR: process "/bin/bash -c (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::=\"--force-confold\" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php$DDEV_PHP_VERSION-dev     unixodbc=$odbc_version     odbcinst=$odbc_version     odbcinst1debian2=$odbc_version     unixodbc-dev=$odbc_version     locales" did not complete successfully: exit code: 100
------
 > [web  8/21] RUN (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php8.3-dev     unixodbc=2.3.7     odbcinst=2.3.7     odbcinst1debian2=2.3.7     unixodbc-dev=2.3.7     locales:
1.893 Package odbcinst1debian2 is not available, but is referred to by another package.
1.893 This may mean that the package is missing, has been obsoleted, or
1.893 is only available from another source
1.893 However the following packages replace it:
1.893   odbcinst1debian2:amd64 unixodbc-common libodbcinst2
1.893
1.894 E: Version '2.3.7' for 'unixodbc' was not found
1.894 E: Version '2.3.7' for 'odbcinst' was not found
1.894 E: Version '2.3.7' for 'odbcinst1debian2' was not found
1.894 E: Version '2.3.7' for 'unixodbc-dev' was not found
------
', stderr='failed to solve: process "/bin/bash -c (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::=\"--force-confold\" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php$DDEV_PHP_VERSION-dev     unixodbc=$odbc_version     odbcinst=$odbc_version     odbcinst1debian2=$odbc_version     unixodbc-dev=$odbc_version     locales" did not complete successfully: exit code: 100', output='#0 building with "default" instance using docker driver

#1 [web internal] load build definition from Dockerfile
#1 transferring dockerfile: 3.02kB done
#1 DONE 0.0s

#2 [web internal] load metadata for docker.io/ddev/ddev-webserver:v1.24.3
#2 DONE 0.0s

#3 [web internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [web  1/21] FROM docker.io/ddev/ddev-webserver:v1.24.3
#4 DONE 0.0s

#5 [web  4/21] RUN (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests apt-utils curl gnupg2 ca-certificates
#5 CACHED

#6 [web  6/21] RUN apt-key add <microsoft.asc
#6 CACHED

#7 [web  5/21] RUN curl -sSL -O https://packages.microsoft.com/keys/microsoft.asc
#7 CACHED

#8 [web  2/21] RUN getent group tty || groupadd tty
#8 CACHED

#9 [web  3/21] RUN (groupadd --gid 20 "karsten" || groupadd "karsten" || true) && (useradd -G tty -l -m -s "/bin/bash" --gid "karsten" --comment '' --uid 501 "karsten" || useradd -G tty -l -m -s "/bin/bash" --gid "karsten" --comment '' "karsten" || useradd  -G tty -l -m -s "/bin/bash" --gid "20" --comment '' "karsten" || useradd -G tty -l -m -s "/bin/bash" --comment '' karsten )
#9 CACHED

#10 [web  7/21] RUN curl -sSL -o /etc/apt/sources.list.d/mssql-release.list https://packages.microsoft.com/config/debian/11/prod.list
#10 CACHED

#11 [web  8/21] RUN (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php8.3-dev     unixodbc=2.3.7     odbcinst=2.3.7     odbcinst1debian2=2.3.7     unixodbc-dev=2.3.7     locales
#11 0.295 Hit:1 http://deb.debian.org/debian bookworm InRelease
#11 0.309 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
#11 0.358 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#11 0.458 Hit:4 http://nginx.org/packages/debian bookworm InRelease
#11 0.474 Get:5 https://packages.sury.org/php bookworm InRelease [7551 B]
#11 0.518 Get:6 https://packages.microsoft.com/debian/11/prod bullseye InRelease [3649 B]
#11 0.530 Get:7 https://packages.sury.org/php bookworm/main arm64 Packages [252 kB]
#11 0.616 Hit:9 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg InRelease
#11 0.675 Get:10 https://dl.cloudsmith.io/public/platformsh/cli/deb/debian bookworm InRelease [3675 B]
#11 0.800 Get:11 https://packages.microsoft.com/debian/11/prod bullseye/main amd64 Packages [191 kB]
#11 0.837 Get:12 https://dl.cloudsmith.io/public/platformsh/upsun-cli/deb/debian bookworm InRelease [3681 B]
#11 0.924 Get:13 https://packages.microsoft.com/debian/11/prod bullseye/main all Packages [1444 B]
#11 0.979 Get:14 https://dl.cloudsmith.io/public/symfony/stable/deb/debian bookworm InRelease [4422 B]
#11 0.987 Get:15 https://packages.microsoft.com/debian/11/prod bullseye/main armhf Packages [37.1 kB]
#11 1.012 Get:16 https://packages.microsoft.com/debian/11/prod bullseye/main arm64 Packages [41.2 kB]
#11 1.015 Hit:8 https://dlm.mariadb.com/repo/mariadb-server/10.11/repo/debian bookworm InRelease
#11 1.048 Hit:17 http://packages.blackfire.io/debian any InRelease
#11 1.083 Fetched 649 kB in 1s (711 kB/s)
#11 1.083 Reading package lists...
#11 1.467 W: https://packages.microsoft.com/debian/11/prod/dists/bullseye/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
#11 1.476 Reading package lists...
#11 1.806 Building dependency tree...
#11 1.889 Reading state information...
#11 1.893 Package unixodbc-dev is not available, but is referred to by another package.
#11 1.893 This may mean that the package is missing, has been obsoleted, or
#11 1.893 is only available from another source
#11 1.893
#11 1.893 Package unixodbc is not available, but is referred to by another package.
#11 1.893 This may mean that the package is missing, has been obsoleted, or
#11 1.893 is only available from another source
#11 1.893 However the following packages replace it:
#11 1.893   odbcinst odbcinst:amd64 odbcinst1debian2 libodbc1 odbcinst1debian2:amd64
#11 1.893   libodbc1:amd64
#11 1.893
#11 1.893 Package odbcinst is not available, but is referred to by another package.
#11 1.893 This may mean that the package is missing, has been obsoleted, or
#11 1.893 is only available from another source
#11 1.893 However the following packages replace it:
#11 1.893   unixodbc-common
#11 1.893
#11 1.893 Package odbcinst1debian2 is not available, but is referred to by another package.
#11 1.893 This may mean that the package is missing, has been obsoleted, or
#11 1.893 is only available from another source
#11 1.893 However the following packages replace it:
#11 1.893   odbcinst1debian2:amd64 unixodbc-common libodbcinst2
#11 1.893
#11 1.894 E: Version '2.3.7' for 'unixodbc' was not found
#11 1.894 E: Version '2.3.7' for 'odbcinst' was not found
#11 1.894 E: Version '2.3.7' for 'odbcinst1debian2' was not found
#11 1.894 E: Version '2.3.7' for 'unixodbc-dev' was not found
#11 ERROR: process "/bin/bash -c (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::=\"--force-confold\" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php$DDEV_PHP_VERSION-dev     unixodbc=$odbc_version     odbcinst=$odbc_version     odbcinst1debian2=$odbc_version     unixodbc-dev=$odbc_version     locales" did not complete successfully: exit code: 100
------
 > [web  8/21] RUN (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php8.3-dev     unixodbc=2.3.7     odbcinst=2.3.7     odbcinst1debian2=2.3.7     unixodbc-dev=2.3.7     locales:
1.893 Package odbcinst1debian2 is not available, but is referred to by another package.
1.893 This may mean that the package is missing, has been obsoleted, or
1.893 is only available from another source
1.893 However the following packages replace it:
1.893   odbcinst1debian2:amd64 unixodbc-common libodbcinst2
1.893
1.894 E: Version '2.3.7' for 'unixodbc' was not found
1.894 E: Version '2.3.7' for 'odbcinst' was not found
1.894 E: Version '2.3.7' for 'odbcinst1debian2' was not found
1.894 E: Version '2.3.7' for 'unixodbc-dev' was not found
------
', stderr='failed to solve: process "/bin/bash -c (apt-get update || true) && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::=\"--force-confold\" --no-install-recommends --no-install-suggests     build-essential     dialog     php-pear     php$DDEV_PHP_VERSION-dev     unixodbc=$odbc_version     odbcinst=$odbc_version     odbcinst1debian2=$odbc_version     unixodbc-dev=$odbc_version     locales" did not complete successfully: exit code: 100'

As you can see 2.3.7 is not a version available for:

• unixodbc
• odbcinst
• odbcinst1debian2
• unixodbc-dev

This is unchanged in my branch, so I assume something has changed for the PHP 8.3 image used as a base here?

Expected behavior
It should build and start without an error message. 😎

Desktop (please complete the following information):

• OS: macOS
• Hardware: Apple M1 Max
• Version 15.3.1
• OrbStack 1.10.2

[kdambekalns]

To be clear: The failing build instructions are in web-build/pre.Dockerfile.sqlsrv

[kdambekalns]

As per

ddev-sqlsrv/web-build/pre.Dockerfile.sqlsrv

Line 2 in 0a74aa6

ARG odbc_version=2.3.7

the version is 2.3.7. But for bullseye it should be 2.3.6 that exists and for bookworm it should be 2.3.11.

🤔

[stasadev]

You are looking at the wrong repo, the correct one is https://packages.microsoft.com/debian/11/prod/pool/main/u/unixodbc/

(I see there is a newer repo for bookworm https://packages.microsoft.com/debian/12, but I'm not an expert in mssql.)

Try

-ARG odbc_version=2.3.7
+ARG odbc_version=2.3.11

Or any other version that has amd64 and arm64 packages available.

[kdambekalns]

Oh my… but there it should have 2.3.7. then. 🙄 Anyway, with 2.3.11 I do get further, indeed. And then:

#12 [web  9/21] RUN ACCEPT_EULA=Y DEBIAN_FRONTEND=noninteractive apt-get install -y msodbcsql18 mssql-tools
#12 0.121 Reading package lists...
#12 0.465 Building dependency tree...
#12 0.587 Reading state information...
#12 0.593 E: Unable to locate package mssql-tools
#12 ERROR: process "/bin/bash -c ACCEPT_EULA=Y DEBIAN_FRONTEND=noninteractive apt-get install -y msodbcsql18 mssql-tools" did not complete successfully: exit code: 100
------
 > [web  9/21] RUN ACCEPT_EULA=Y DEBIAN_FRONTEND=noninteractive apt-get install -y msodbcsql18 mssql-tools:


0.593 E: Unable to locate package mssql-tools
------
', stderr='failed to solve: process "/bin/bash -c ACCEPT_EULA=Y DEBIAN_FRONTEND=noninteractive apt-get install -y msodbcsql18 mssql-tools" did not complete successfully: exit code: 100'

Even though https://packages.microsoft.com/debian/11/prod/pool/main/m/mssql-tools/ is a thing.

But with an added

-RUN ACCEPT_EULA=Y DEBIAN_FRONTEND=noninteractive apt-get install -y msodbcsql18 mssql-tools
+RUN ACCEPT_EULA=Y DEBIAN_FRONTEND=noninteractive apt-get install -y msodbcsql18 mssql-tools18

it starts. 🎉

[stasadev]

If this change works on amd64 as well (we'll see it in the tests), that's great news.

[kdambekalns]

Let's hope it does. 😇