Build/Database
Contents
Design Considerations
- Code must be suitable for submission to upstream
- We should store as much data about the builds in the database as possible
- The current exception to this is build logs, which are _not_ stored in the database.
- All active buildbot masters should report into the same database.
Source code
The source code currently resides at http://hg.mozilla.org/users/catlee_mozilla.com/bb_db.
model.py defines all the SQL tables as well as the mapping to python classes.
status.py implements the buildbot status plugin. It's added to the BuildmasterConfig like this:
import bbdb.status reload(bbdb.status) c['status'].append(bbdb.status.DBStatus(dburl))
dburl is a string describing how to connect to the database [1]
Schema
The SQL schema is close approximation of buildbot's internal data structures into SQL.
Querying the database
reporter.py contains some examples of querying the database. The basic process is:
- Call model.connect with the appropriate DB URL. A session-maker object is returned. Use this session-maker object to create a session.
- Use the session to query the various tables.
- Close the session when done
import model # Connect to database called 'buildbot' on localhost, using # 'buildbot', 'passw0rd' as the username, password session_maker = model.connect("mysql://buildbot:passw0rd@localhost/buildbot") session = session_maker() try: builds = session.query(model.Build).filter_by(result=0) for build in builds: # Do something with build print "Build", build.buildnumber, "started at", build.starttime finally: # Close the session. This cleans up cached objects, and frees the # connection to the database so it can be used by another session session.close()
The SQLAlchemy session is used to hold python representations of SQL rows in memory. It's kind of like a cache for these mapped objects.
Alternatively, raw SQL can also be run directly with the session:
result = session.execute("select * from builds where result=0 limit 10")
Hooks
The DBStatus plugin allows you to pass in a list of subscribers that get regular buildbot status notifications that are augmented with database information.
Recommended Reading
- SQLAlchemy docs: http://www.sqlalchemy.org/docs, especially the section on querying