ReleaseEngineering/PuppetAgain/Data

Each tree in the puppet data store needs to be available at puppet:///$tree and http://repos/$tree, except the $repos tree, which is omitted from puppet:/// for efficiency.

Trees

The currently-defined trees are:

• repos - package repositories (construed broadly -- this includes firmware updates, for example)
• python - python stuff
• bmm - image files for Mozpool (from back when it was named BMM)

Each tree has a `private` subdirectory which is not publicly mirrored. Everything else is publicly mirrored! Do not place proprietary or secret documents outside of a private subdirectory!

Mirror

All of the public data for puppetagain is mirrored publicly on puppetagain.pub.build.mozilla.org. If you are running a puppet master outside of Mozilla, please set it up to rsync periodically from this mirror:

 rsync -a rsync://puppetagain.pub.build.mozilla.org/data /data

The mirror is also available through HTTP at http://puppetagain.pub.build.mozilla.org/data. This is primarily intended for manual download of particular files used to build releng systems, rather than as part of an automated system. It is not built to handle direct access by external puppet clients (e.g., via an HTTP proxy).

How to

(for the moco setup - other installs may have different procedures)

.. add a data file

First, consider the implications. If something will magically "discover" the files you're about to add, then you're making a production change and you should tread carefully. (This is not a problem for Python packages these days - we always pin versions for them.) In general, we want to avoid such magical discovery, but it happens sometimes anyway.

As root, login to the designated puppet master (currently releng-puppet2.srv.releng.scl3.mozilla.com) and add the file in the appropriate place under /data. In general, we try not to remove or overwrite files, but if you need to, do so. Run puppetmaster-fixperms to fix the permissions on those files (although this runs every half-hour anyway).

To accelerate the file-distribution process, run `puppetmaster-sync` on the other servers where you need the files. Note that this is run twice an hour on a crontask, so this step is not required if you've got time to spare. Whether you run it manually or from cron, a change-management email will be sent to indicate the changes you made.

How it works

Here's how we run this at moco:

All data is stored under /data/$tree on each master. A virtualhost makes all of /data available:

<VirtualHost *:80>
        DocumentRoot /data
</VirtualHost>

and a puppet fileserver.conf entry makes each tree available:

[python]
path /data/python
allow 0.0.0.0/0

The trees are synchronized, using rsync (see bug 769071).