Calendar:QA Chat:2006-07-20:IRCLog
From MozillaWiki
-->| Andreas (Andreas_Tr@F74AD796.4C00166E.33AABD5F.IP) has joined #calendar-qa
|<-- Andreas has left irc.mozilla.org (Quit: ChatZilla 0.9.61 [Mozilla rv:1.7.13/20060417])
-->| Andreas (Andreas_Tr@F74AD796.4C00166E.33AABD5F.IP) has joined #calendar-qa
<ctalbert> We will be getting under way in about 10 minutes...
-->| zach (zach@moz-46636307.office.mozilla.org) has joined #calendar-qa
<zach> morning...
<ctalbert> morning
<zach> ctalbert: the litmus issue is fixed
<ctalbert> Litmus started working!
<Andreas> hello
<ctalbert> Thanks zach
<ctalbert> hi andreas
<zach> there's a bit of a bug, which means that whenever you add a testgroup, I need
to do something manually in the db (adding subgroups is fine)
<ctalbert> Ouch.
<ctalbert> I think I'm through adding testgroups for a while. What about cloning
testgroups?
<zach> ctalbert: well you only have one testgroup
<zach> ctalbert: subgroups are unaffected
<ctalbert> I was going to flesh out my first testgroup a bit before adding more.
<ctalbert> That was one of the things I wanted to talk about today.
<zach> if you need to add more testgroups, just let me know once it's added and I
can add the needed entry to the db
<ctalbert> Ok thanks. I will.
<ctalbert> I guess I should get this started. Marcia should be joining us when she gets in
this morning.
<ctalbert> The tiny agenda is here:
http://wiki.mozilla.org/Calendar:Current_QA_Chat
=-= jminta has changed the topic to ``http://wiki.mozilla.org/Calendar:Current_QA_Chat''
-->| ssitter (chatzilla@F41C9EFD.65C15961.3E7AAD02.IP) has joined #calendar-qa
<ctalbert> my first attempt at a proposed structure for the test groups/subgroups
and testcases in litmus can be found here:
http://wiki.mozilla.org/Calendar:Test_Case_List#Litmus_Calendar_Product_Test_Case_Structure
<ctalbert> Ah! more people. Welcome!
<ssitter> hi, folks
<ctalbert> The first thing I wanted to get feedback on was a proposed
TestGroup/SubGroup/TestCase structure for Litmus.
<ctalbert> ssitter: did yo see the wiki link for that?
<ctalbert> s/yo/you
<ssitter> yes, some seconds ago
<ctalbert> I had intended to just point us all directly at litmust to take a look at it,
but there were some issues there. The wiki is a little
more complete anyway.
<ctalbert> Litmus has three groups for Prodcuts/TestGroups/SubGroups/TestCases
<ctalbert> The items in blue are the items that currently exist in Litmus
<ctalbert> For things that have very generalized testcases (like smoketests
and
localization) I made just one TestGroup for both.
<ctalbert> Does that seem sufficient?
-->| lilmatt (mwillis@moz-8BBD264E.twcny.res.rr.com) has joined #calendar-qa
<ctalbert> Hi Matt
<lilmatt> Hey. Sorry I was late.
-->| marcia (marcia@moz-46636307.office.mozilla.org) has joined #calendar-qa
-->| mvl (michiel@moz-2F035FEA.direct-adsl.nl) has joined #calendar-qa
<ctalbert> mvl, lilmatt, marcia, I was just talking about a proposal for structuring
calendar testcases in litmus here is a link:
<ctalbert> http://wiki.mozilla.org/Calendar:Test_Case_List#Litmus_Calendar_Product_Test_Case_Structure
<ctalbert> It's very simple
<ctalbert> No one has very many thoughts about it.
-->| marcia_imac (chatzilla@moz-A8CDE328.office.mozilla.org) has joined #calendar-qa
<ctalbert> One of my principal questions is can we just generalize the local vs. remote
calendar tests?
<marcia_imac> ctalbert: dmose is here looking over my shoulder
<ctalbert> marcia_imac: sweet. Hi dmose
<ctalbert> local vs. remote calendar: that means we'd have a provider specific suite of
testcases for each provider
<ctalbert> but in our basic functionality testgroup we would direct people to test on any
remote calendar
<ctalbert> does that sound ok?
<ctalbert> Or am I out of my mind?
* ctalbert thinks that this should be moved to 16:30UTC so that everyone in the west will
be awake :-)
<lilmatt> Well, as you add providers with diff functionality wouldn't you want to test
that?
<lilmatt> ie: WCAP vs. ICS vs. CalDAV
<marcia_imac> we are awake
<mvl> ctalbert: don't hope to get meaningful input for me, because i never looking
into this testcase/litmus thing
<mvl> and besided, i need to go shop for food
<marcia_imac> at least dmose and i are
<ctalbert> :-)
<zach> I'm awake :)
<ctalbert> Yes, those would be the provider specific testcases I was mentioned,
lilmatt
<lilmatt> ctalbert: I was late because I had fallen asleep, and I'm EST, so...
<zach> it would seem to me that the remote testcases should test each protocol...
<ctalbert> Even in the "basic" test group?
<ctalbert> My main concern was not duplicating too many tests.
* ctalbert realizes he forgot the "Provider" SubGroup completely.
<lilmatt> There's no printing test.
<ctalbert> Ah, that too.
* zach should go over by marcia and dmose
<ctalbert> So, it would seem I should add in testcases for each provider for the basic
functionality.
<ctalbert> For example, Create a WebDav calendar would be a testcase
<ctalbert> I like that since it will provide better tracking per feature.
<lilmatt> plus whatever spiffy things you can do with each provider.
<ctalbert> lilmatt: And those spiffy things will go in the Provider specific SubGroup
<lilmatt> right
<marcia_imac> ctalbert: you can look at some of the Thunderbird test cases to see how we
handled the features, esp IMAP and POP
<marcia_imac> ctalbert: ? from dmose, how is the Provider different from calendar?
<ctalbert> marcia_imac: thanks! I will.
<ctalbert> Hmmm....good question
<ctalbert> I guess it really isn't. It can't exist independently.
<ctalbert> Thank ya'll. This clears up things quite a bit.
<marcia_imac> ctalbert: besides printing, what other areas will you have for categories
in Litmus?
<ssitter> what about views?
<ctalbert> Did you see the wiki link? Calendars (Providers), Events, Tasks, Installation,
Navigation, Searching
<ctalbert> ssitter: I grouped Views into Navigation
<marcia_imac> ctalbert: No, i will have to check it out
<ctalbert> http://wiki.mozilla.org/Calendar:Test_Case_List#Litmus_Calendar_Product_Test_Case_Structure
<ssitter> drag'n'drop, display, setting view preferences, ...
<jminta> import/export
<ssitter> navigation is only one thing you can do in the views
<ctalbert> ssitter: good point.
<jminta> ctalbert: is there a plan for how these test groups will line up with either
bugzilla or our toronto feature list?
<ctalbert> jminta: I was thinking about that yesterday and thinking about that.
<ctalbert> I was thinking that many of the features from Toronto might be going under
the "Full Functional Test" bucket rather than the basic, and all I was doing
yesterday was trying to flesh out the basic ones
<marcia_imac> ctalbert: i was just mentioning to dmose that our FF and Tbird test cases
test basic functionality, but don't emphasize negative tests. I think
to have a good test suite you have to incorporate a bit of both
<ctalbert> I agree
<ssitter> maybe for mature sunbird: extension/theme installation/handling,
update handling, ..
<marcia_imac> i like to think that the BFTs are the most commonly used functions
<ctalbert> And the FFTs are everything else?
<ssitter> makes sense
<jminta> update handling applies to lightning too
<marcia_imac> FFTs for FF and Tbird take a deeper dive, getting into all the things
you can do in an area
<lilmatt> ...and eventually Sb
<ctalbert> This is very good feedback. I appreciate it.
<ssitter> jminta: lightning update is handled by thunderbird ;-)
<jminta> ssitter: we can still screw that up on the server side
<ssitter> except you want to test server side
<marcia_imac> ctalbert: I think starting with the BFTs is a good thing, and then refining
them over time. At least you have a base of tests to start with
<zach> what we've done with ff and tbird is to have the structure for the BFTs
and the FFTs be pretty much the same, but to have more testcases
in each FFT subgroup
<ctalbert> that's what I was noticing
<ctalbert> I think with this feedback I feel a bit more comfortable planning out a
complete proposal. Why don't I do that and submit it to m.d.a.calendar
and we'll let the discussion continue there?
<marcia_imac> We are also trying to write test plans for the upcoming FF 2.0 release.
I find that helps me to organize my thoughts and flesh out different
ideas
<marcia_imac> ctalbert: sounds like a plan to me
<mvl> ctalbert: i might be jumping in, and didn't follow the history of this,
but would it make sense to couple testcases to the toronto tasks and
features?
<ctalbert> mvl: definitely
<zach> yeah, even just a simple plan for each subgroup laying out the goals
of that subgroup is helpful to write to get started
* ctalbert nods
<ctalbert> While we have a bit more time, I'd like to ask marcia if she has any
words of wisdom regarding building a qa community around calendar
<mvl> ctalbert: isn't the features list already grouped? and can't you reuse
that grouping?
<ctalbert> mvl: yes, I'll look into that
<marcia_imac> ctalbert: our qa community building is still a work in progress. we have been
trying different things to see what works best
<ctalbert> What has seemed to work?
<marcia_imac> ctalbert: things like: blogging and holding regularly scheduled test
days. the challenge with that is covering all the time zones
<marcia_imac> the regularly sched test day seem to work well, but we get different
contributors all the time. we really need to try to build a base of regular
folks. and I would like to involve the comm more in test case writng, etc
<ssitter> how will the process for writing calendar test cases look like?
<ssitter> assume i read a bug and think this should be a testcase made of?
<ctalbert> The test case writing is an interesting thing.
<marcia_imac> ssitter: exactly. sometimes bugs provide good fodder for test cases
<ctalbert> ssitter: we could get you access to write a test case in litmus. But for
people in general, litmus is a bit too young for that at this point.
<marcia_imac> ctalbert: blogging and getting the word out is important to community
building.
<ctalbert> (but they're working on it)
<ctalbert> I understand. We could use the calendar blog we have for it. People
already know to look there.
<zach> exactly...I'm working on making adminship per-product, so that the calendar
product can designate its own admins
<ctalbert> zach: you're gonna hate this one...do you have a ball park guess for
when that will land?
<zach> ctalbert: haha
<zach> ctalbert: maybe late next week?
<ctalbert> wow.
<marcia_imac> ctalbert: do you post in the mozillazine forums at all announcing things?
<ctalbert> Not at the moment, no
<ctalbert> We should.
<zach> ctalbert: depends how long it takes me to untangle myself from perl module
dependency hell ;)
<marcia_imac> ctalbert: good set of folks there to harvest. We post to the QA blog,
MozillaZine forums and the mozilla newsgroups
<ctalbert> Ok. Thanks that's good to know. We can do the same and include our calendar
specific stuff that we've got going.
<marcia_imac> ctalbert: it seems to me that the forums are actually a better place
to try to get new recruits. especially for things related to Tbird
<ctalbert> marcia_imac: How "complete" were your testcases before you did the first
test day?
<marcia_imac> ctalbert: that sounds good.
<marcia_imac> ctalbert: how often do you guys blog? daily or weekly? i think frequent
blogging is helpful too. I know we do not do enough of it
<jminta> about weekly
<ssitter> (http://weblogs.mozillazine.org/calendar/)
<zach> ctalbert: even a small suite of tests is helpful
<ctalbert> I was thinking of having one when we got the first round of BFTs fleshed out.
<ctalbert> (for Sunbird)
<marcia_imac> ctalbert: they were already fairly complete, because we really wrote
most of them in the FF 1.0 days. I think waiting until you at least have
test cases for each section is a good idea
<zach> jminta has a good idea: hold a test writing day
<marcia_imac> what about also having a test day focused around one feature? we are doing
that for FF 2.0
<zach> people could write tests on the wiki or someplace and send them to ctalbert
for editing and insertion in litmus
<ctalbert> Kind of like around jminta's drag and drop tuff that just landed.
<ctalbert> zach: That's how I was thinking of doing it.
<marcia_imac> ctalbert: for example, with FF 2.0 appraching, we are zeroing on on RSS
and Session Restore, which we want to flesh out the problems early before
we ship
<marcia_imac> ctalbert: drag and drop stuff would make a great test day
<ctalbert> zach: In fact, if you scroll down that wiki page, that's exactly what it is.
<zach> ctalbert: awesome
<ssitter> i did a test day for that dnd landing, jminta will remember i'm sure ;-)
<marcia_imac> ctalbert: how you handle feedback for the test days is important too, we open
up the blog for comments too in addition to the test days
<ctalbert> I have one more thing before we wrap up.
<zach> ctalbert: a test writing day would be a good way to build up enough testcases
for test days
<marcia_imac> also rewarding folks on an ongoing basis is a good thing - maybe I can
help out with that
<ctalbert> marica_imac: good_idea. Thanks
<ctalbert> my other thing: If I use this timeslot again for a meeting or a test
day or whatever, would it be better for people to move it back 30mins?
<marcia_imac> jminta said there is a sunbird logo, maybe we could make a shirt
<ctalbert> 16:30 UTC for example
<ctalbert> Oh...that would be cool
<marcia_imac> ctalbert: that works better for me, dmose is mentioning that the sun
guys might need to weigh in there
<ssitter> yes, 16:00 is hard to make for me cause of my day job
<ctalbert> We have the sun guys who are at their day job at 16:00 and guys like
ssitter and mvl who could use a slightly later time
<ctalbert> I'll post that to the newsgroup as well
<ssitter> 16:30 utc would be better
<Andreas> I think it would be ok for Ulf and me (The Sun QA) to start the chat
30 minutes later.
<ctalbert> Awesome. Then I think we'll do that next week.
<ctalbert> Ok, I think we're out of time. I'll send out a complete test-plan proposal
to the m.d.a.calendar newsgroup
<ctalbert> Then I think we'll try to have a test-writing day to help get some of
these things written
<marcia_imac> ctalbert: sounds good to me. let me know if you have any other questions
via email or ping me on irc
<ctalbert> Thanks, I will.
<ctalbert> Thank you all for your help.
<marcia_imac> ctalbert: no problem
<ssitter> ctalbert: maybe you add definition what is/should be difference between
smoketest, basic test, full test
<ctalbert> I'll be posting this log on the calendar qa chat wiki
<ctalbert> ssitter: good idea.
<ctalbert> I will
<ctalbert> Ok. Meeting adjourned. Have a good day.
<ctalbert> zach: Thanks for your help with litmus!
<zach> ctalbert: certainly! coop says the bug will be fixed soon, so you'll
be able to add your own testgroups
<lilmatt> marcia_imac: Could I get some assistance from you in getting template
talkback master.ini files for us to make for Sunbird?
<lilmatt> I don't believe they're in public cvs
<ctalbert> brb
<marcia_imac> lilmatt: i think jay patel might be able to help better in that area
<ctalbert> all, I am going to have to step out for a while. I'll be back in about
an hour.
<lilmatt> Okay. His email to me had the following regarding configuring templates
for Sunbird : "I can help you with this if you know where the Talkback
files are."
=-= YOU are now known as ctalbert|away
<--| zach has left #calendar-qa
=-= lilmatt is now known as lilmatt-shower
<--| ssitter has left #calendar-qa
<--| jminta has left #calendar-qa
<-- Andreas has left irc.mozilla.org (Quit: ChatZilla 0.9.61 [Mozilla rv:1.7.13/20060417])
-= YOU are now known as ctalbert
--| mvl has left #calendar-qa (Leaving)
