Partnering:Projects:BYOB:PilotBuildQueueDetails

Overview

This is a summary of the moving parts involved in the PHP/MySQL based build queue in use for the pilot launch of BYOB.

Repack states

The workflow status of a repack is state machine, where each state describes what actions are currently available and to which states it may transition next. A history log of these state transitions is recorded in the repack data, along with comments.

state	description	next states
new	freshly created repack	edited; requested; deleted
edited	repack with changes	requested; deleted
requested	user has requested a build and review	cancelled; started; failed; pending
cancelled	user has cancelled a repack build and review request	edited; requested; deleted
started	build has started	failed; pending
failed	build failed for some reason	edited; requested; deleted
pending	build completed, review pending	cancelled; released; rejected
rejected	build rejected for public release	edited; requested; deleted
released	build approved for public release	reverted
reverted	public release of build taken down	edited; requested; deleted
deleted	repack to be deleted from system	--

Notes

• Repacks are locked as read-only (ie. editing via wizard is disabled) when in these states:
  • requested; started; pending; approved
• Repacks get copied on edit as an unreleased version when in these states:
  • released

Build queue tasks in BYOB

In reaction to certain state changes, build tasks get enqueued into a PHP/MySQL based offline work queue that gets exhausted at least every 5 minutes or so. The current build queue tasks are:

Perform build

• Enqueue triggered by repack transitioning to requested state.
• Process:
  • Fire up the repack build script
  • Move the built repack to private review hosting
• Result:
  • If successful, transition the repack to pending state.
  • If failed, transition the repack to failed state.

Release build

• Enqueue triggered by repack transitioning to released state.
• Process:
  • Move a built repack from private review to public release hosting
• Result:
  • In any case, repack remains in released state.
  • (No failure case currently implemented.)

Delete build

• Enqueue triggered by repack transitioning to deleted state.
• Process:
  • Delete a built repack from both private and public hosting
• Result:
  • In any case, the repack data has been deleted from the BYOB database.
  • (No failure case currently implemented.)

Build queue implementation details

The build queue consists of:

• a MySQL table listing tasks to be performed;
• a shell script that runs in a crontab or as an OS X launchd scheduled task;
  • every 5 minutes or so, it sets a lock to prevent concurrent runs and exhausts the queue
• a PHP controller that processes the queue, using a model library to perform the work.

The web app uses this build queue by:

1. Registering DeferredEvent callback handlers with unique IDs - these do the actual work
2. Firing Events that correspond with registered DeferredEvent handlers
3. Firing Events results in enqueued tasks, which the queue processor handles via calls to registered DeferredEvent callbacks.