Skip to content

feat(audit): persist flash creation records #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
elibosley wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

feat(audit): persist flash creation records #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/USER_COMMANDS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,25 @@ Runtime uses GUI-first menu scripts:

At runtime, `/boot/install/menu.sh` is launched after persistence runtime overrides are applied.

## Persistent Flash Audit Logs

When persistence is mounted, flash creation writes compact JSON audit records to
the installer media under `/mnt/persist/logs/`.

- `flash-audit-<timestamp>.json`: timestamped audit record
- `flash-audit-latest.json`: copy of the latest audit record

Audit records include the flash action, status, selected Unraid ZIP filename and
SHA256, target device details, and runtime override SHA256s when override files
are present. They are written to the installer media persistence partition, not
to the target Unraid server or newly created flash device.

To avoid filling persistence, audit logs are pruned after writes. Defaults keep
at most 25 timestamped audit files and at most 1 MiB total timestamped audit log
data. Override with `INSTALLER_AUDIT_MAX_FILES` and
`INSTALLER_AUDIT_MAX_BYTES` if a partner workflow needs a different retention
window.
Comment on lines +186 to +203
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Document the internal-boot audit path too.

This section reads as flash-only, but the PR also persists create_internal_boot audit records into the same location. Update the wording so users know these files cover both installer actions, not just create_flash_boot.sh.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@docs/USER_COMMANDS.md` around lines 186 - 203, Update the "Persistent Flash
Audit Logs" section wording to state that audit records under /mnt/persist/logs/
include both flash creation and internal-boot creation actions (specifically
mention create_internal_boot as well as create_flash_boot.sh), so users know the
timestamped files (flash-audit-<timestamp>.json and flash-audit-latest.json)
cover installer actions for both flash and internal-boot; keep the existing
details about included fields, pruning defaults, and the
INSTALLER_AUDIT_MAX_FILES / INSTALLER_AUDIT_MAX_BYTES overrides but change any
language that implies "flash-only" to "installer actions (flash and
internal-boot)".


## Common Troubleshooting

1. Kernel/ZFS did not rebuild:
Expand Down
204 changes: 203 additions & 1 deletion scripts/create_flash_boot.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,11 @@ ui_view_log() {
RUN_LOG_FILE=""
STEP_COUNT=0
TOTAL_STEPS=8
AUDIT_STARTED=0
AUDIT_WRITTEN=0
AUDIT_STATUS="started"
AUDIT_ERROR=""
UEFI_ANSWER=""

init_run_log() {
RUN_LOG_FILE="$(mktemp /tmp/create-flash-boot.XXXXXX.log)"
Expand Down Expand Up @@ -238,6 +243,10 @@ log_msg() {

error_msg() {
local message="$*"
if (( AUDIT_STARTED == 1 )); then
AUDIT_STATUS="failure"
AUDIT_ERROR="$message"
fi
if [[ "$ui_backend" != "text" ]]; then
append_run_log "$message"
ui_msg "Flash Boot Error" "$message"
Expand Down Expand Up @@ -268,6 +277,186 @@ step_update() {
echo "==> [${STEP_COUNT}/${TOTAL_STEPS}] ${step_text}"
}

audit_json_escape() {
local value="${1:-}"
value="${value//\\/\\\\}"
value="${value//\"/\\\"}"
value="${value//$'\n'/\\n}"
value="${value//$'\r'/\\r}"
value="${value//$'\t'/\\t}"
printf '%s' "$value"
}

audit_json_string() {
printf '"%s"' "$(audit_json_escape "${1:-}")"
}

audit_file_sha256() {
local path="$1"
local digest=""

if [[ -f "$path" ]] && command -v sha256sum >/dev/null 2>&1; then
digest="$(sha256sum "$path" 2>/dev/null | awk '{print $1}' || true)"
fi

printf '%s' "${digest:-unavailable}"
}

audit_disk_value() {
local disk="$1" field="$2"
lsblk -dn -o "$field" "$disk" 2>/dev/null | head -n1 | sed 's/[[:space:]]*$//' || true
}

audit_blkid_value() {
local device="$1" field="$2"
if command -v blkid >/dev/null 2>&1; then
blkid -s "$field" -o value "$device" 2>/dev/null | head -n1 || true
fi
}

audit_write_disk_object() {
local out="$1" disk="$2" indent="$3"
local size_bytes model serial tran wwn

size_bytes="$(blockdev --getsize64 "$disk" 2>/dev/null || true)"
model="$(audit_disk_value "$disk" MODEL)"
serial="$(audit_disk_value "$disk" SERIAL)"
tran="$(audit_disk_value "$disk" TRAN)"
wwn="$(audit_disk_value "$disk" WWN)"

{
printf '%s{\n' "$indent"
printf '%s "path": %s,\n' "$indent" "$(audit_json_string "$disk")"
printf '%s "size_bytes": %s,\n' "$indent" "$(audit_json_string "$size_bytes")"
printf '%s "model": %s,\n' "$indent" "$(audit_json_string "$model")"
printf '%s "serial": %s,\n' "$indent" "$(audit_json_string "$serial")"
printf '%s "transport": %s,\n' "$indent" "$(audit_json_string "$tran")"
printf '%s "wwn": %s\n' "$indent" "$(audit_json_string "$wwn")"
printf '%s}' "$indent"
} >> "$out"
}

audit_write_runtime_overrides() {
local out="$1" indent="$2"
local override_root="${PERSISTENT_ROOT:-/mnt/persist}/runtime"
local file_name first=1 sha

printf '%s"runtime_overrides": [\n' "$indent" >> "$out"
if [[ -d "$override_root" ]]; then
for file_name in \
install-profile \
menu-backend \
menu.sh \
menu_gui_common.sh \
menu_gui_user.sh \
menu_gui.sh \
create_internal_boot.sh \
create_flash_boot.sh \
zip.sh; do
[[ -f "$override_root/$file_name" ]] || continue
sha="$(audit_file_sha256 "$override_root/$file_name")"
if (( first == 0 )); then
printf ',\n' >> "$out"
fi
first=0
printf '%s {"name": %s, "sha256": %s}' "$indent" "$(audit_json_string "$file_name")" "$(audit_json_string "$sha")" >> "$out"
done
fi
printf '\n%s]' "$indent" >> "$out"
}

audit_prune_logs() {
local dir="$1"
local max_files="${INSTALLER_AUDIT_MAX_FILES:-25}"
local max_bytes="${INSTALLER_AUDIT_MAX_BYTES:-1048576}"
local count total oldest audit_file audit_size

[[ "$max_files" =~ ^[0-9]+$ ]] || max_files=25
[[ "$max_bytes" =~ ^[0-9]+$ ]] || max_bytes=1048576
(( max_files >= 1 )) || max_files=1
(( max_bytes >= 4096 )) || max_bytes=4096

while true; do
count="$(find "$dir" -maxdepth 1 -type f -name 'flash-audit-[0-9]*.json' | wc -l | tr -d '[:space:]')"
total=0
while IFS= read -r audit_file; do
[[ -n "$audit_file" ]] || continue
audit_size="$(wc -c < "$audit_file" 2>/dev/null | tr -d '[:space:]' || true)"
[[ "$audit_size" =~ ^[0-9]+$ ]] || audit_size=0
total=$((total + audit_size))
done < <(find "$dir" -maxdepth 1 -type f -name 'flash-audit-[0-9]*.json' -print)
if (( count <= max_files && total <= max_bytes )); then
break
fi

oldest="$(find "$dir" -maxdepth 1 -type f -name 'flash-audit-[0-9]*.json' -print | sort | head -n1)"
[[ -n "$oldest" ]] || break
rm -f "$oldest" || break
done
}

write_flash_audit_record() {
local status="$1" error_message="${2:-}"
local audit_root audit_dir timestamp filename tmp final latest
local zip_sha target_uuid target_label target_partuuid

if [[ "${PERSIST_READY:-0}" != "1" || -z "${PERSISTENT_ROOT:-}" ]]; then
return 0
fi

audit_root="${PERSISTENT_ROOT}/logs"
audit_dir="$audit_root"
mkdir -p "$audit_dir" 2>/dev/null || return 0
[[ -w "$audit_dir" ]] || return 0

timestamp="$(date -u '+%Y-%m-%dT%H:%M:%SZ' 2>/dev/null || date)"
filename="flash-audit-$(date -u '+%Y%m%dT%H%M%SZ' 2>/dev/null || date '+%Y%m%dT%H%M%S').json"
final="$audit_dir/$filename"
tmp="$final.tmp"
latest="$audit_dir/flash-audit-latest.json"
Comment on lines +412 to +416
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Use a collision-resistant audit filename.

This name only has second-level precision, and scripts/create_internal_boot_user.sh writes to the same directory with the same pattern. Two installer actions finishing in the same second will clobber one another and silently drop an audit record.

💡 Suggested fix 
-    local audit_root audit_dir timestamp filename tmp final latest
+    local audit_root audit_dir timestamp tmp final latest
@@
-    filename="flash-audit-$(date -u '+%Y%m%dT%H%M%SZ' 2>/dev/null || date '+%Y%m%dT%H%M%S').json"
-    final="$audit_dir/$filename"
-    tmp="$final.tmp"
+    tmp="$(mktemp "$audit_dir/flash-audit-$(date -u '+%Y%m%dT%H%M%SZ' 2>/dev/null || date '+%Y%m%dT%H%M%S').XXXXXX.json.tmp")" || return 0
+    final="${tmp%.tmp}"
     latest="$audit_dir/flash-audit-latest.json"
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@scripts/create_flash_boot.sh` around lines 412 - 416, The audit filename
generation (variables timestamp and filename) uses only second-level precision
and can collide with other scripts writing the same pattern; change the filename
generation to be collision-resistant by appending a high-resolution timestamp or
unique token (e.g., include nanoseconds from date +%N, the process id $$, or a
generated UUID) or by creating a unique temporary name via mktemp and then
moving it into final; ensure you update tmp/final handling so the unique name is
preserved and latest still points to the correct file (adjust variables
filename, tmp, final, and latest accordingly).


zip_sha="$(audit_file_sha256 "${ZIP_FILE:-}")"
target_uuid="$(audit_blkid_value "${TARGET_PART1:-}" UUID)"
target_label="$(audit_blkid_value "${TARGET_PART1:-}" LABEL)"
target_partuuid="$(audit_blkid_value "${TARGET_PART1:-}" PARTUUID)"

{
printf '{\n'
printf ' "schema": "unraid-installer-flash-audit-v1",\n'
printf ' "timestamp": %s,\n' "$(audit_json_string "$timestamp")"
printf ' "action": "create_flash_boot",\n'
printf ' "status": %s,\n' "$(audit_json_string "$status")"
printf ' "error": %s,\n' "$(audit_json_string "$error_message")"
printf ' "unraid_zip": {\n'
printf ' "path": %s,\n' "$(audit_json_string "${ZIP_FILE:-}")"
printf ' "filename": %s,\n' "$(audit_json_string "$(basename "${ZIP_FILE:-}")")"
printf ' "sha256": %s\n' "$(audit_json_string "$zip_sha")"
printf ' },\n'
printf ' "target": {\n'
printf ' "disk": '
} > "$tmp"
audit_write_disk_object "$tmp" "${TARGET:-}" " "
{
printf ',\n'
printf ' "partition": {\n'
printf ' "path": %s,\n' "$(audit_json_string "${TARGET_PART1:-}")"
printf ' "label": %s,\n' "$(audit_json_string "$target_label")"
printf ' "uuid": %s,\n' "$(audit_json_string "$target_uuid")"
printf ' "partuuid": %s\n' "$(audit_json_string "$target_partuuid")"
printf ' }\n'
printf ' },\n'
printf ' "uefi_boot": %s,\n' "$(audit_json_string "${UEFI_ANSWER:-}")"
printf ' "make_bootable": %s,\n' "$(audit_json_string "${MAKE_BOOTABLE:-}")"
} >> "$tmp"
audit_write_runtime_overrides "$tmp" " "
printf '\n}\n' >> "$tmp"

mv -f "$tmp" "$final" 2>/dev/null || { rm -f "$tmp"; return 0; }
cp -f "$final" "$latest" 2>/dev/null || true
audit_prune_logs "$audit_dir"
log_msg "Persistent audit record: $final"
}

confirm() {
local prompt="$1" default="${2:-n}" ans hint

Expand Down Expand Up @@ -377,7 +566,7 @@ if [[ "${PERSIST_READY:-0}" != "1" ]]; then
fi

if compgen -G "${ZIP_DIR}/unRAIDServer-*-x86_64.zip" > /dev/null; then
ZIP_FILE="$(ls -1 "${ZIP_DIR}"/unRAIDServer-*-x86_64.zip | sort -V | tail -n1)"
ZIP_FILE="$(find "$ZIP_DIR" -maxdepth 1 -type f -name 'unRAIDServer-*-x86_64.zip' -print | sort -V | tail -n1)"
else
error_msg "ERROR: no unRAIDServer zip files found in ${ZIP_DIR}"
exit 1
Expand Down Expand Up @@ -460,12 +649,22 @@ else
[[ "$CONFIRM" != "YES" ]] && { echo "Aborted."; exit 1; }
fi

AUDIT_STARTED=1
mount_dir=""
cleanup() {
local rc=$?
if [[ -n "$mount_dir" && -d "$mount_dir" ]]; then
run_operation umount "$mount_dir" || true
run_operation rmdir "$mount_dir" || true
fi
if (( AUDIT_STARTED == 1 && AUDIT_WRITTEN == 0 )); then
if (( rc != 0 )); then
AUDIT_STATUS="failure"
[[ -n "$AUDIT_ERROR" ]] || AUDIT_ERROR="create_flash_boot.sh exited with status $rc"
fi
write_flash_audit_record "$AUDIT_STATUS" "$AUDIT_ERROR" || true
AUDIT_WRITTEN=1
fi
}
trap cleanup EXIT

Expand Down Expand Up @@ -532,6 +731,9 @@ run_operation umount "$mount_dir"
run_operation rmdir "$mount_dir"
mount_dir=""

AUDIT_STATUS="success"
write_flash_audit_record "$AUDIT_STATUS" ""
AUDIT_WRITTEN=1
status_msg "Flash boot image creation complete"
log_msg "Operation log: $RUN_LOG_FILE"

Expand Down
Loading