User deleted after updating from v1.122.2 to v1.123.0

[Mehyar-ALS]

The bug

After updating from v1.122.2 to v1.123.0, the app signed out and when I tried to check Immich from the browser directly I was greeted with the initial greeting message from Immich as if my instance was just created. The admin user (which is my only user) was deleted. I tried and created the user again with the exact same information and logged in, but I can't see my photos. I have downgraded back to v1.122.2 but still didn't fix it.

The OS that Immich Server is running on

Ubuntu 22.04.5 LTS aarch64

Version of Immich Server

v1.123.0

Version of Immich Mobile App

V1.22.3 build.171

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

name: immich

services:
  immich-server:
    container_name: immich_server
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    user: ${PUID}:${GUID}
    env_file:
      - .env
    ports:
      - 2283:2283
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    user: ${PUID}:${GUID}
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    image: redis:6.2-alpine@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc
    restart: always

  database:
    container_name: immich_postgres
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
#UPLOAD_LOCATION=/mnt/hdd/immich
#UPLOAD_LOCATION=/mnt/truenas/mehyar/immich
UPLOAD_LOCATION=/mnt/truenas/immich

# run as user Mehyar
PUID=1000
GUID=1000

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION="v1.123.0"

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=xxxxxxx

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Update from v1.122.2 to v1.123.0

Relevant log output

No response

Additional information

No response

[Mehyar-ALS]

I noticed that the docker-compose.yml file has been updated.
In the new database:

volumes:
  - ${DB_DATA_LOCATION}:/var/lib/postgresql/data

Where in the old one:

volumes:
  - pgdata:/var/lib/postgresql/data

with:

volumes:
  pgdata:

How can I move my old DB from the docker volume pgdata to a new location where I can point to it in the .env file?

[bo0tzz]

What are the commands you ran to update? It seems like the pgdata volume may have gotten deleted.

The database volume change is from quite a few releases ago. You should always check the release notes for breaking changes before you update: https://github.com/immich-app/immich/discussions?discussions_q=label%3Achangelog%3Abreaking-change+sort%3Adate_created

[Mehyar-ALS]

I just run docker compose down then change the version in .env and run docker compose up -d again.

I do check for breaking changes before updating, that's why I do it manually. Not sure how I missed this one and in which update was it!

The other intreasting thing is how did the volume survive it and kept working for me with no problem all the way up to v1.122.2, which I updated to last week!

I do now! This time I ran docker compose down -v as it was the last one in my history.

I just ran docker volume ls | grep immich and the only volumes showing are:

local     immich_model-cache
local     immich_pgdata
local     immich_tsdata

Unfortuntly it looks like I lost my DB...

When I list the files in my Immich directory, I get this:

drwxr-xr-x - mehyar 18 Dec 03:00 backups
drwxr-x--- - mehyar 13 Sep 16:12 encoded-video
drwxr-x--- - mehyar 13 Sep 16:12 library
drwxr-xr-x - mehyar 13 Sep 16:12 profile
drwxr-x--- - mehyar 18 Dec 10:57 thumbs
drwxr-x--- - mehyar 18 Dec 10:57 upload

and inside backups:

.rw-r--r-- 185M mehyar  5 Dec 03:00 immich-db-backup-1733364000007.sql.gz
.rw-r--r-- 185M mehyar  6 Dec 03:00 immich-db-backup-1733450400005.sql.gz
.rw-r--r-- 185M mehyar  7 Dec 03:00 immich-db-backup-1733536800011.sql.gz
.rw-r--r-- 185M mehyar  8 Dec 03:00 immich-db-backup-1733623200011.sql.gz
.rw-r--r-- 185M mehyar  9 Dec 03:00 immich-db-backup-1733709600005.sql.gz
.rw-r--r-- 185M mehyar 10 Dec 03:00 immich-db-backup-1733796000008.sql.gz
.rw-r--r-- 185M mehyar 11 Dec 03:00 immich-db-backup-1733882400007.sql.gz
.rw-r--r-- 185M mehyar 12 Dec 03:00 immich-db-backup-1733968800007.sql.gz
.rw-r--r-- 185M mehyar 13 Dec 03:00 immich-db-backup-1734055200011.sql.gz
.rw-r--r-- 185M mehyar 14 Dec 03:00 immich-db-backup-1734141600010.sql.gz
.rw-r--r-- 185M mehyar 15 Dec 03:00 immich-db-backup-1734228000009.sql.gz
.rw-r--r-- 185M mehyar 16 Dec 03:00 immich-db-backup-1734314400014.sql.gz
.rw-r--r-- 185M mehyar 17 Dec 03:00 immich-db-backup-1734400800006.sql.gz
.rw-r--r-- 185M mehyar 18 Dec 03:00 immich-db-backup-1734487200006.sql.gz

Does this backup include the data from the deleted DB? Shall I try and restore it using this?

[bo0tzz]

I do now! This time I ran docker compose down -v as it was the last one in my history.

That's the issue, -v tells it to delete the volumes.

Yes, the backups dir has database dumps that you should be able to restore. That's a good moment to take the change to the DB_DATA_LOCATION setup as well.

[Mehyar-ALS]

I made the change to DB_DATA_LOCATION and restored the backup and it worked! I was able to recover my data.

Nothing feels better than a successful restore from a backup :)

Thanks for the support.