User:Ishitva

From MozillaWiki
Jump to: navigation, search

Google Summer Of Code 2014

Project: Migrate from YUI2 to YUI3. Bug 453268
Assigned Mentor: Mr Byron Jones
Git Fork: https://github.com/ishitvagoel/bugzilla

Short description: The present stable release of Bugzilla uses YUI2 library for its user interface which is now deprecated and is no longer maintained . There is a compelling need to migrate to the latest version of YUI that is YUI3 as it is a more secure and an actively maintained framework . Therefore in this application , I propose to perform the task of complete migration of the Bugzilla codebase from YUI2 to YUI3 and subsequently carry on the task of refactoring the code.

Weekly Updates

Community Bonding Period:

The community bonding period was used for the experimentation purpose, the summary of the work done is as follows :

  • Discussed with my mentor the approach to be used for loading the YUI modules. We tried various approaches like using a separate JS file for creating a YUI instance with the required modules loaded and then including this file on all the Template files which used the YUI code or the code in the JS files. This approach didn't seem viable as then we would have had to load all the modules attached to the YUI instance on all the template pages. So we then finally decided to create a yui3 array on global/header.html.tmpl file, with “node” as the default module to be loaded and then subsequently use the push method to add modules to this array on the template pages to load the required modules during the runtime.To implement this we came up with the following changes to the codebase, as suggested by my mentor, global/header.html.tmpl changes. These changes are yet to be tested.
  • We tried to migrate the js/TUI.js file and came up with these changes: TUI.js changes A special thanks to Mr Clarence Leung (Yahoo) who provided valuable feedback during this work.
  • Learnt about using the Debugger Tool of Firefox : Debugger . As my mentor has suggested, we will be using this tool to test the changes which we will be making to the codebase. We will be using breakpoints to check whether the functions defined inside the JS files are getting hit or not and also to test for any other errors.

Week 1, 19th May-26th May 2014

  • Committed the changes to the js/attachment.js file to the git fork: js/attachment.js changes. The changes to this file also are yet to be tested.

NOTE: I will be having my final exams at college for which I will have to prepare full time, so I have taken permission from my mentor to take a preparatory break till 5th June 2014. After that all the pending work will be taken care off. All the remaining testing work will be done and after that only further migrations will be performed.

Week 4 , 6th June-15th June 2014

So finally my exams got over and I started with my work !

The work done during this phase is as follows:

  • The changes made to the file TUI.js were tried to be tested. While testing it we found out that the YUI3 modules were not getting loaded. The reason behind this was that the previous changes made to the header.html.tmpl file for creating a YUI3 loader were incomplete.When we include a module using the use() function of the YUI() constructor the module doesn’t work unless the the JS file in which its class is defined is loaded already. So what we were missing was the part which would pre-fetch the YUI3 JS files so required for the modules to work. We came up with these , Loader Fix, changes which made it possible for the script elements to be be created which pre-fetch the required JS files , while rendering the HTML.
  • Still this was not enough ! We again tried to test the TUI.js file on the page enter_bug.cgi page of Bugzilla and found out that even when all the required JS files were getting pre-fetched and the modules were expected to get loaded without any problem , as soon as we entered the enter_bug.cgi page we came across an error ” TypeError: Y.on is not a function”. This was completely unexpected ! We tried to fix the issue and spent a lot of time on this problem and finally headed towards #yui for some guidance.There :foxxtrot helped us identify what the problem was due to. The problem was due to the fact that we had not included all the dependencies for the modules we intended to use, because of which when the use() function tried to load the modules the asynchronous behaviour of the YUI loader was triggered such that unless the required dependencies would be loaded the rest of the code could proceed to be executed. This created a problem for us as Y.on() function was getting used even before the dependencies for the module(yui-base) it is defined in were loaded and hence the aforementioned error came up. So :foxxtrot suggested us to use a config object {bootstrap:false} for the YUI() constructor. We included this config object , and then using the YUI3 Configurator , calculated all the dependencies and included all the dependency modules along with the modules we wanted to use on the bug/create/create.html.tmpl file, through the yui3 array, and then tested the TUI.js file on enter_bug.cgi page again. This time Eureka ! it worked :) Although this method works , it has some shortcomings too, that the number of dependency modules to be loaded is way too large which makes the loader slow as the script element for each of the module has to be generated and also that we ourselves will have to calculate the dependencies. What we need to do is to automate the offline dependency calculation or maybe combine the modules into one module using a File combiner.So as :dobedobedoh (#yui) suggested , we tried to implement in our header.html.tmpl the YUI3 Loader, the gist of which can be seen here, YUI3 Loader Experiment .The experimentation on the Loader is still going on, but as of now to proceed with the migrations and testing we have decided to use the present working version of the Loader.
  • Facing all these challenges we somehow managed to progress ! We tested the TUI.js and the attachment.js files and the changes made to them worked fine. Then the migration of the files change-columns.js , comments.js, flag.js, util.js and global.js and the Voting Extension was done. The changes to these files will be pushed to the GitFork as soon as they are tested. Currently we are trying to migrate the bug.js and custom-search.js files.

So far so good !

Week 5 , 16th June-22nd June 2014

The work done during this phase is as follows:

  • The change-columns.js file was tested and the changes (link) were committed to the git fork.
  • The comments.js file was tested and the fixes to the errors were done. (link) While testing comments.js file, it was observed that on clicking on a tag while the comment was in collapsed state did not expand the comment. This was due to the fact that the changes in comments.js file had broken this functionality of the comment-tagging.js file, which can be fixed upon migrating this file.
  • The flag.js file was tested and the changes to it were committed. (link). The changes worked fine, but an error was observed, in which whenever we move to a new bug page from a current bug page , or create a new bug, the function Y.on() does not get recognized, but when we refresh the page it gets recognized. The reason for this might be a dependency not being loaded. This still needs to be fixed.

Week 6 , 22nd June- 29th June 2014

Continuing work from the previous week:

  • The global.js file was tested and the changes to it were committed. (link). YUI3 modules were added on the index.html.tmpl to enable the functioning of global.js file on index.cgi. The changes worked fine. It was observed that the set_language() function worked fine when changing the language from English to German, but failed while changing back from German to English. It was due to the fact that the DE templates (German) did not have the YUI3 modules and the required changes in them therefore YUI 3 functions were not getting recognized in the reverse process.
  • The untested changes to the util.js file (link) and the Voting Extension (link) were committed too.
  • After all this work we began with the migration of the custom-search.js file. Initially I had got stuck in it as I was getting confused deciding which DOM nodes need to be converted to YUI3 nodes. The confusion was due to the fact that at some places the DOM element was being passed as a parameter to a function and at other places the YUI3 node. Therefore I decided to convert all the uses of the DOM node to the YUI3 nodes. These are the initial untested changes that I have committed (link).

Week 7 , 30th June-7th July 2014

This week we decided to work on the migration of Bug.js file. The bug.js file defines a module "duptable" inside the "bugzilla" object inside the YAHOO global namespace. This module defines the functionality for creating a DataTable widget which retrieves the data by making JSON-RPC calls to the server. This datatable is used for displaying the possible duplicate bugs while creating new bugs. The bug.js file also defines a module "bugUserLastVisit" which gets created within the self executing function in which it is defined in. This responsibility of this module is to make JSON-RPC calls to the functions 'BugUserLastVisit.update' and 'BugUserLastVisit.get'.

To begin with the work:

  • I first read about what JSON-RPC is as I did not have much idea about it.
  • After these I read the documentation of YUI2 DataTable and DataSource and the corresponding documentation for YUI3 DataTable and DataSource. The implementation of DataTable and DataSource are very much different in YUI 2 and YUI 3 and many of the YUI 2 propertis do not exist in YUI 3, so even after reading the documentation I was unsure about how to proceed.After getting stuck here for quite some time I managed to make some initial changes (link) to the file to be able to discuss the further changes and approach for migration with my mentor :glob .

Week 8 , 8th July-13th July 2014

Continuing my work from Week 7:

  • I discussed about the approach to follow with my mentor :glob, who then suggested me to first experiment by making a new file and trying to make the JSON-RPC calls get working.So I tried to make a file through which I called the 'Bug.version' method using JSON-RPC. I could get the JSON-RPC calls work fine. Now I had to extend the approach I used here to the bug.js file. At this point my mentor :glob suggested me to take a look at how it is being done on BMO (link) for reference. I followed the approach used in BMO and came up with these changes. (link) The BMO work was really helpful. Thanks :glob !
  • After these changes I still wasn't sure about which functions to use in the callback for sendRequest method. So then I asked about it from :clarle and he suggested to use the DataTableDataSource plugin for the DataTable and the load() function in place of sendRequest() which by default used the callback which we needed here, so we did not need to define the callback functions.

These were the changes we came up with (link)

  • After this I tried to migrate the button widget being used in the formatter. Therefore I read the documentation of YUI 2 Button widget and YUI 3 Button Widget and came up with these changes (link). As it can be seen that I had used the 'innerHTML' property for the button, to prevent XSS, :glob suggested me to use the YUI 3 'text' property for displaying HTML escaped text.
  • The further testing and experimentation on bug.js file was begun. (link)

Week 9 , 14th July-20th July 2014

Continuing the work from previous week:

  • Now the data table had started working fine , but an error was coming , in which the data table message was not getting removed even after the data was being loaded. Finally, on discussing this problem on #yui , I came to know that this was because the yui3-skin-sam was not being used, the datatable uses the skin-sam to show and hide messages. So we had to enable the YUI 3 CSS files to be loaded. So I came up with the following changes (link).
  • After this a few more changes to bug.js file were made and the version in which the datatable works fine was committed.(link).
  • Upon further testing the bug.js file it was found out that inside the "bugUserLastVisit" modules I had missed to mention the "data" attribute of the Y.io() config. So I added these changes (link). A few more bugs were fixed.(link)
  • After this work the all the template files except the the report/report-table.html.tmpl and global/header.html.tmpl have been migrated. The changes to these files are untested and are kept in seperate files and will be committed later. The report-table.html.tmpl and header.html.tmpl files will be migrated later.

Week 10 , 21st July-27th July 2014

This week we decided to begin with the work on field.js file and comment-tagging.js file.

  • I began with reading the documentation for the YUI 2 Autocomplete and Calendar and the corresponding documentation for the YUI 3 version.
  • After reading through the documentation I came up with the following initial changes to the field.js file (link). The changes do not work fine yet.
  • The initial changes to the file comment-tagging.js were also made (link). Some changes are still to be done.

I have some doubts in these two files. I will proceed with these after taking guidance from my mentor.

Week 11 , 28th July-5th August 2014

In this week we continued our work on the field.js file and comment-tagging.js file.

  • The changes to the comment-tagging.js were made. The file was tested and the debugging was done.(link). While testing the file the problem of stringifying the form was coming. Earlier we were calling this "Y.io.stringify(form)" , which did not work. Upon discussing the problem on #yui it was found that the correct call was "Y.IO.stringify(form)".(link). Also, initially the Autocomplete widget was not working fine. It was making the JSON RPC calls but was unable to fetch the results. I reported this to my mentor and he found out that due to the recent changes in Bugzilla codebase, the paramaters which we were supplying to the "Bug.search_comment_tags" were coming out to be invalid, we were actually missing the "version" paramater. After my mentor :glob fixed this issue, we observed that only one suggestion was being shown in the autocomplete, again this was found to be an error in the bugzilla code, which my mentor fixed. After all this work on comment-tagging.js file the file worked fine and and all the JSON RPC calls were being made correctly (link) . Yippee !
  • Further debugging of the fields.js was done.(link). While testing the file, it was observed that calendar widget was working fine on enter_bug.cgi page but not on show_bug.cgi page. This is due to an error in the setFieldFromCalendar() function. On the show_bug.cgi page this function is used to update the deadline text box with the selected deadline for the bug. Right now the date which is being shown in the deadline text box is not correct , although errors are not coming related to the calendar widget in the console. A few minor errors still remain to be fixed.
  • The migration changes which had been made to the template files were also committed (link) . The changes to the report-table.html.tmpl and header.html.tmpl have also been started.

Week 12 , 6th August-15th August 2014

Continuing from the previous week:

  • So far we had tried to migrate the report-table.html.tmpl file taking insipiration from this example on using DataSource Local and made these changes (link). These changes used the XML schema plugin to convert an HTML table into a YUI3 datatable. These changes were not working fine.The datatable was not being created but the datasource was being created. Then I tried to just create a datatable without using the sort functionality and the formatters and the datatable was created but it did not replace the original HTML table. I discussed it my mentor and he advised me to change the approach that I had been using here. He suggested that we create a JSON of the data and then directly feed it into the YUI3 table. We no longer need to create an HTML table and then convert it into a YUI3 datatable. So I started reading the Templating Toolkit 2 documentation.The work on report-table.html.tmpl is still in progress.
  • We made some changes on the header.html.tmpl file as well. Initially I was stuck here identifying the approach to migrate the Firefox BFCACHE hacks on the header. I discussed it with my mentor and he informed me that the BFCACHE hacks were no longer required as the YUI team has fixed the issue in their code.So it was safe to remove them.I removed the hacks and tested that the problem which they addressed were no longer being encountered. The migration was performed on the file and after this I started the cleanup of YUI2 code from the file. Some parts of YUI2 could not be removed as on removing them an internal error on bugzilla pages come up. Work on this file is in progress too.
  • Since the scope of this project was found to be much larger than what could be completed within the duration of the Google Summer Of Code 2014 coding period ,the project will be continued post GSoC period as well.

Final Writeup

Finally the GSoC 2014 is nearing its completion. It had been a wonderful learning experience working on the migration of Bugzilla codebase from YUI 2 to YUI 3 while working under the guidance of my mentor Mr.Byron Jones. We faced a lot of challenges together and progressed :)

Following is the summary of the work we have done during the GSOC period:

  • Did some initial work on the loader to laod YUI3 modules, enough to be able to proceed with the migration of the files.
  • Learnt to use the Firefox Dev tools for debugging the Source files, inspecting the elements on the page and inspecting the POST requests.
  • Migrated all the Javascript files.
  • Migrated all the template files which used the YUI2 code and made changes to other templates wherever necessary. The work on report-table.html.tmpl and header.html.tmpl is still in progress.
  • Experimented on the Loader which calculates module dependencies itself.

No significant work comes without challenges, so the following are the challenges we faced:

  • YUI3 has been designed to be used within the sandboxes. The Bugzilla codebase had to be changed a lot to be able to use the sandboxes so we had to proceed like we used to do with YUI 2. This put up a difficult challenge in front of us. YUI3 modules have a lot of dependencies, so we had to use the Configurator to find out all the dependencies and mention all the modules inside the yui3 arrays on the templates. This made the loading of the modules a very slow process and when we missed out on a specific module it became very difficult to search for it.
  • While cleaning up th YUI2 code from the header.html.tmpl file not all code related to functioning of YUI2 code could be removed as they need further work before we can remove them.
  • Migration of the widgets was the most challenging part. YUI 2 and YUI 3 implement the Widgets quite differently.
  • Getting the JSON-RPC calls through the widgets working correctly was equally challenging.

The work to be done post GSoC period :

  • Completing the migration of report-table.html.tmpl file, using the JSON data approach.
  • Cleaning up of YUI2 code completely from the codebase by making appropriate changes.
  • Improving the Loader to make dependency calculation and module loading easier.
  • Conversion of HTML event attributes to Javascript event listeners.

Note of Thanks !

A heartfelt thank you to my mentor Mr. Byron Jones who helped me during my work. He always patiently clarified my doubts, even the silly ones :P He supported me in my work even when things had become a little challenging for me and was always very helpful. Thank you glob !
I would also like to thank the members of the YUI team who from time to time showed me the way when I had got stuck with the migrations or wanted to know the things which were not mentioned in the YUI3 documentation.

Thank you all :)


Retrieved from "https://wiki.mozilla.org/index.php?title=User:Ishitva&oldid=1006067"