Volume backup notifications built links with route(), which uses APP_URL
in queue jobs and gave http://localhost links. Build them from base_url().
Closes#12285
GNU tar exits with 1 when a file changes while it is read. The archive
is still valid, so only exit codes above 1 fail the backup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mc release in coolify-helper 1.0.17 panics on `mc pipe --quiet`.
Drop the flag and keep the progress bar out of failure messages by
printing only real errors and the last progress frame.
Closes#11995
Add a /dashboard/canvas page that renders every team server with its
resources and their running or queued deployments, backed by a new
ServerCanvasController::dashboard endpoint and a per-server usage
endpoint for zone headers. The dashboard remembers the list/canvas
choice and each section (projects, servers) its own table/grid view.
- Show Deploying/Queued pills on canvas cards and a foldable recent
deployments panel; add shared CanvasControls component
- Allow backup downloads from the server host via the S3 download
storage (PrepareFileDownloadJob accepts a null container)
- Extract backup toggle/run actions into ManagesDatabaseBackup and
surface them in backup execution headers; merge S3 and retention
sections into the general backup settings
- Move deployments indicator to the top bar and add a main nav
resource menu with submenus; simplify breadcrumbs and settings
layouts
- Add shared variables overview canvas and scheduled task modals
- Add feature tests for dashboard canvas, view modes, navbar
submenus, backup S3 downloads and scheduled task modals
mc --resolve applies per invocation and is not stored in the alias, so
mc cp resolved the S3 host again inside the helper container. On hosts
with split DNS this failed with "no such host", and the upload did not
use the IP validated by SafeWebhookUrl.
Fixes#12129
Check enabled volume backup schedules hourly and notify teams when no
execution occurs within the configured number of days. Expose the alert
threshold in the UI and API, with alerts disabled by default.
Route volume backup results and warnings to the appropriate Telegram
backup threads.
- Hide proxy configuration contents from users without update permission.
- Select and validate backup S3 storage against the resource's team.
- Keep scheduled branch syncs green while a conflict PR remains open.
- Document external mount permissions and Sentinel traffic limits.
- Add regression coverage for authorization and backup storage selection.
Require deploy or root token access when creating managed mounts outside
resource directories or changing their file contents through the API.
Synchronize KeyDB and Dragonfly passwords with local and shared
REDIS_PASSWORD values while preserving secret manager resolution.
Allow validated host file and directory mounts outside resource directories
while protecting external paths from deletion.
Clean up cancelled deployment containers and clear tracked process IDs.
Preserve legacy KeyDB and Dragonfly password parsing, accept server backup
paths without extension restrictions, and allow Compose variable proxy ports.
Cloud production now runs one supervisor per queue (deployments, crons,
high, default, maintenance), so a busy queue cannot starve the others.
Add maintenance_queue() for slow Docker cleanups and a shared
horizon.worker_timeout config value. Self-hosted keeps the s6 supervisor.
The scheduler dispatched VolumeBackupRecoveryJob every minute for each
pending execution, and each job retried 20 times with a 60 second backoff.
A stuck execution kept about 20 failing jobs active at a time.
- Run each recovery job once; the scheduler is the only retry path.
- Make the job unique per execution so duplicates do not pile up.
- Dispatch recovery for an execution at most once every 5 minutes.
- Clear the pending flag and record the reason when recovery cannot
succeed (server, resource, S3 storage, or filename no longer exists),
so it does not fail forever and block new backups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The missed daily backups on Coolify Cloud came from Redis evicting queue keys
(allkeys-lfu on a full instance), not from the scan of all schedules. The
next_run_at design (452259aac, 5c5740533) did not prevent that, and it added
stored state that must be kept in sync on every frequency, timezone, and server
change. It was not released, so remove it:
- revert next_run_at, its model hooks, the "Next run" UI, and the diagnostics
changes; restore scheduled_job_states and the queue overlap lock
- a new migration removes the next_run_at columns and creates
scheduled_job_states again where the removed migration already ran
Keep the changes that fix real problems:
- run the manager every minute from the scheduler with `scheduled:dispatch`,
so a busy queue cannot delay it past the catch-up window
- load the server chain of scheduled backups with each chunk, which removes
about 7,000 queries per run on Coolify Cloud
- catch Throwable for each schedule, so one broken schedule does not stop the
others
- publish database and volume backups again when their queued job was not
started after 60 minutes (for example, lost in Redis); log lost tasks and
Docker cleanups as missed. A lost backup is not run again when a newer
occurrence of the same schedule exists.
The scheduled job manager checked every enabled schedule each minute and
evaluated its cron expression in PHP. On Coolify Cloud one run took more than
60 seconds, so runs overlapped and minutes were skipped.
Each schedule now stores its next due time:
- next_run_at on scheduled database backups, scheduled tasks, and volume
backups, and docker_cleanup_next_run_at on server settings (UTC, indexed)
- model hooks calculate it when a schedule is created or its frequency or
enabled flag changes; a server timezone change or a move to another server
calculates it again for the affected schedules
The `scheduled:dispatch` command runs the manager every minute from the
scheduler, not from a queue. It selects only due rows and claims each
occurrence with an atomic update of next_run_at, together with its delivery
record, so parallel runs on several nodes never dispatch it twice.
- database and volume backups run once when they are late; tasks and Docker
cleanups more than 10 minutes late are logged as missed
- a local time that occurs two times when the clocks go back runs once
- the migration continues each schedule after its last recorded occurrence and
drops scheduled_job_states
Show the next run, in the server timezone, on the database backup, scheduled
task, volume backup, and Docker cleanup pages, or "Invalid frequency" when the
frequency cannot be calculated. scheduled:diagnostics now lists overdue
schedules and open deliveries.
Remove workarounds that are no longer needed: the queue overlap lock and its
stale-lock repair, the catch-up window, and debug-only logs.
Remove tests that copy app logic into the test, grep PHP source as
text, only check literals, constants, class_exists or PHP built-ins,
duplicate other tests, or are always skipped. Keep source checks that
guard security and authorization rules.
Rename a duplicate Pest helper so the full suite loads, drop the
missing v5/Browser path, and remove the CLAUDE.md symlink in favor of
AGENTS.md. Correct AGENTS.md: Dusk is not installed and the upgrade
script is scripts/upgrade.sh.
- Delete Blade components, component classes, Livewire components and
views that nothing uses after the redesign.
- Delete tests that only compare Blade/CSS source text; keep the source
checks for authorization gates, stable wire:key values and escaping.
- Update stale tests to the current UI and behaviour.
- Update DESIGN.md to the current navigation and layout.
- Add a lesson: test behaviour, not source markup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Allow local and S3 archives to be deleted independently when removing a
volume backup schedule, and consolidate destructive-action layouts with a
shared danger-zone component.
Add queued DNS checks with polling, status indicators, and notifications for application and service domains. Improve volume backup target labels and names.
Add project icon storage and serving with local/S3 support, prevent deletion of compose-managed volumes, and optimize volume backups. Refine project and backup UI components with consistent styling and coverage.
Eager-load S3 on volume backup lists and add an S3 status column with
Configured/Unavailable/Not set badges. Surface Local vs S3 on the
persistent storages backup column. Move service resource domain links
to a globe action in the card footer, and update related tests and grid CSS.
Keep error pages full-viewport, lift the unsaved bar above the
mobile keyboard, make status/heading layouts stack cleanly on small
screens, route service DB backups to volume-backups, and tighten
volume backup grid/forms. Cover with feature tests.
Improve project resource UIs: sort domains by DNS failure, stop re-adding www pairs on refresh, lazy-load storage tabs with counts, tighten env-var tables, keep application tabs active across Livewire polls, unify database type labels, and update related CSS/JS and tests.
- Gate volume backup retention and S3 controls by update permission
- Preserve backup records when S3 deletion fails
- Share SFTP download streaming with consistent missing-file handling
- Handle schedule creation errors and link service database backups
Expose DELETE for application, database, and service storage backup
schedules (with OpenAPI docs), reject storage deletes while schedules
exist, skip retention cleanup when no limits are set, and remove S3
archives using the execution’s S3 storage.
Store s3_storage_id on scheduled volume backup executions so retention
and recovery use the S3 that received the upload. Extract
DeleteScheduledVolumeBackup for UI and resource deletion, and isolate
database backup retention failures so cleanup errors do not fail the job.
Andras BacsaiandClaude Opus 5.5
🏔️ Peak
