Gaia/Build/Refactoring Plan/First half of 2014

Since we found build scripts still isn’t easy to modify on bug 964216 and had bug 897352 for copying all apps into build_stage/ directory, we should consider refactoring build system to get clearer build process. There are something I found and how can we do for refactoring it.

Plan

Here is a simple version of diagram for our refactoring plan:

basically I would like to follow the idea on bug 897352 to copy all files in app into build_stage/ directory, and refactoring all build scripts to process files in build_stage/ instead of in app source directory and extract webapp-shared.js from webapp-zip.js to handle which files need to be picking from shared directory, webapp-zip.js is only used to zip files into profile.

Let’s show all details in below diagram and explain what should them do for each module:

Copy rule 

each app has a copy.mk, if this file doesn’t exist we will use default way to copy files.

webapp-shared.js 

webapp-shared.js is used to pick files which is used in app from shared/ directory, this module will be extracted from webapp-zip.js

multilocale.js 

we load and use this module in webapp-zip.js for now, but it will be extracted to a xpcshell command and will be executed standalone.

preload-apps.js 

fetch preload apps from internet if VARIANT_PATH exists and create directory for each app in build_stage.

app-makefile 

each app should has their own makefile to generate and customize data, details are listed in next section.

webapp-optimize.js 

optimize javascript, l10n files, html here, this module will be modified to optimize in build_stage directory instead of app source directory.

webapp-zip.js 

we have a lot of features in this modules for now, so we plan to move picking files in shared directory feature to webapp-shared.js and picking l10n files feature to multilocale.js, webapp-zip.js will be only used to compress file into profile directory.

additional-extension.js, preferences.js & settings.js 

nothing change.

webapps-json.js 

extracted from webapp-manifest.js to generate webapps.json into profile directory.

app-makefile

(done, George) bug 968667 - create run-app-js-command  

create & export this macro to execute build script in app

bluetooth, clock, costcontrol, email, fl, fm, keyboard, music, ringtones, search, setringtone, video 

no change for stage 1.

(done, George) bug 968668 - browser 

need migrate some code from applications-data.js & webapp-manifests.js & webapp-optimize.js and it’s modified manifest so we need to build this app in build_stage.

(done, Yuren) bug 968669 - calendar

migrate some code from applications-data.js

(done, George) bug 968671 - camera

migrate some code form applications-data.js

(done, Yuren) bug 968675 - communications 

migrate some code from applications-data.js, webapp-manifests.js & webapp-zip.js, we need to build it in build_stage.

(done, George) bug 968680 - gallery 

migrate some code from applications-data.js

(done, Yuren) bug 968684 - homescreen 

migrate a lot of code from applications-data.js, webapp-zip

(done, Tim) bug 968688 - settings 

migrate some code from applications-data.js

(done, George) bug 968690 - sms 

migrate some code from applications-data.js

(done, George) bug 968697 - system 

migrate some code from applications-data.js, webapp-manifests.js & webapp-zip.js, we need to build it in build_stage

(done, Tim) bug 968702 - wallpaper 

migrate some code from webapp-zip.js and we need to build it in build_stage

(done, Tim) bug 968703 - wappush 

migrate some code from applications-data.js

Migration Plan

1. (done, Greg) bug 968657 - Add |make watch| to watch files change and execute |APP=<app> make|, because we should always get files from build_stage/ for DEBUG=1 httpd.js.
2. (done) Migrate build script for specific app from applications-data.js to app makefile.
3. (done) bug 897352 - Add copy rule to copy app into build_stage/
4. (done) bug 968661 - Extract new module webapp-shared.js from webapp-zip.js to copy files which are used in app to a specific directory
5. (done) bug 968666 - multilocale.js should be a standalone build script which can be executed by macro |run-js-command|
6. (done) bug 983573 - [Gaia] Refactoring webapp-manifest.js

we can do all action items 1, 2 in parallel and 4, 5 depend on 3.

Follow up bugs

• bug 972228 - Verify default file content in non-distribution test
• bug 969215 - execute |make| should only build specific apps we modified
• (done, George) bug 971437 - Unify GAIA_DIR/customization and GAIA_DIR/build/test/resource/distribution_test
• (done, Yuren) bug 972755 - use build_stage/preload-apps-data to store preload apps instead of GAIA_DISTRIBUTION_DIR/temp in variant.js
• (invalid, Yuren) bug 973786 - Remove l10n/shared path handler in httpds/bootstrap.js and point app directory to build_stage/$APP