Stage Cleanup Policy

The details of this policy grew out of bug 342972 and the associated discussions in the newsgroups.

Our goal with this policy is to minimize the need for new hardware acquisition while still maintaining those artifacts of the nightly and formal release processes that are essential to regression hunting and posterity.

Project leaders from all the projects currently hosted on stage.mozilla.org were involved in the policy process.

Any new projects will by default adopt the same policy, but can choose to modify the specific parameters concerning which artifacts are kept and for how long.

The Policy

1. Keep all official releases for all products online and available. There's no need to remove them.
2. Keep all en-US nightly builds for all products online and available. There's no need to remove them.
3. Delete nightly artifacts for all products that are not useful in regression hunting. Specifically, this means deleting installer files (linux and windows) and xpis older than 6 months. At the time of writing, that equates to December 2009 and earlier. Individual products can opt out of this cleanup with sufficient cause.
4. Automate the deletion of nightly MAR files older than one month. Only the most recent MAR files are required. This would be done across all products.
5. Delete builds from older candidates directories after official release. This will reclaim up to 13G per build attempt per release. This will be a manual process.
6. Automate the removal of unused nightly artifacts (installers and xpis) older than 6 months for all products.

Crontabs

Firefox

# Clean up symbols every night.
10 22 * * * /usr/local/bin/clean_symbols.py /mnt/breakpad_symbols/symbols_ffx

# Delete all installer and xpi files/dirs older than 6 months 
@weekly nice -n 19 find /home/ftp/pub/firefox/nightly -depth -mtime +180 \( -name '*.installer.*' -o -name '*xpi' \) -exec rm -rf '{}' \;

# Delete all MAR files older than 1 month that aren't in a candidates dir.
@weekly nice -n 19 find /home/ftp/pub/firefox/nightly -wholename '*-candidates' -prune -o -mtime +30 \( -name '*.mar' -o -name '*.tests.*' \) -exec rm -f '{}' \;

# Delete empty dirs
@weekly nice -n 19 find /home/ftp/pub/firefox/nightly -wholename '*-candidates/build*/contrib*' -prune -o -empty -type d -exec rmdir '{}' \;

Thunderbird

# Delete all msi, zip and xpi files/dirs older than 6 months
@weekly nice -n 19 find /home/ftp/pub/thunderbird/nightly -mtime +180 \( -name '*.zip' -o -name '*xpi' -o -name '*.msi'\) -exec rm -rf '{}' \;

# Delete all MAR files older than 1 month that aren't in a candidates dir.
# @weekly nice -n 19 find /home/ftp/pub/thunderbird/nightly -wholename '*-candidates' -prune -o -mtime +30 -name '*.mar' -exec rm -f '{}' \;

# Delete empty dirs
#@weekly nice -n 19 find /home/ftp/pub/thunderbird/nightly -wholename '*-candidates/build*/contrib*' -prune -o -empty -type d -exec rmdir '{}' \;

SeaMonkey

# Delete all installer and xpi files/dirs older than 6 months.
@weekly nice -n 19 find /home/ftp/pub/seamonkey/nightly -mtime +180 \( -name '*.installer.*' -o -name '*xpi' \) -exec rm -rf '{}' \;

# Delete all MAR files older than 1 month that aren't in a candidates dir.
@weekly nice -n 19 find /home/ftp/pub/seamonkey/nightly -wholename '*-candidates' -prune -o -mtime +30 -name '*.mar' -exec rm -f '{}' \;

# Delete empty dirs.
@weekly nice -n 19 find /home/ftp/pub/seamonkey/nightly -wholename '*-candidates/build*/contrib*' -prune -o -empty -type d -exec rmdir '{}' \;

Sunbird

# Delete all installer files older than 6 months.
@weekly nice -n 19 find /home/ftp/pub/calendar/sunbird/nightly -mtime +180 -name '*.installer.*' -type f -exec rm -f '{}' \;

# Delete empty dirs.
@weekly nice -n 19 find /home/ftp/pub/calendar/sunbird/nightly -wholename '*-candidates/build*/contrib*' -prune -o -empty -type d -exec rmdir '{}' \;