How GitLab Handles Stranded (Orphaned) Artifacts in Object Store Post Backup
Understanding the Scenario with Minio Clusters and gitaly-backup
When you have two minio clusters—one for production, another as a hot backup streaming via mc mirror, along with
periodic backups managed by gitaly-backup: What happens if new artifacts are added after the execution of your
gitaly-backup? Are these newly introduced items stranded (orphaned) within GitLab’s Object Store, or is there a
mechanism for pruning such instances? Can manual intervention be employed to trigger this process when needed?
The Role of Clean Up Task in Addressing Stranded Artifacts
To effectively clean up artifact storage after restoration with gitaly-backup, GitLab provides the following task:
- Gitlab::CleanUp Orphan Job – This rake command is designed to identify and remove orphan job files, essentially
stranded (orphan) items that exist without a corresponding active pipeline in your project’s CI/CD configuration:
sudo gitlab-rake "gitlab:cleanup:orphan_job_artifact_files" DRY_RUN=false # Remove orphan artifact files. - GitLab::CleanUp Project Uploads – This task is used to remove project uploads, which may also include any newer
items not covered by your backup:
sudo gitlab-rake "gitlab:cleanup:project_uploads" # Remove all uploaded files.
Running these tasks will help ensure that the Object Store reflects only what’s relevant and actively managed within
GitLab, thus maintaining an orderly environment post backup operations such as those performed by gitaly-backup.