Author: Jan Lahoda
Version: 1.1.0
Last update: 20.5.2004
Introduction: This test specification describes the tests used to test the tasklist module. Download test data.
Comment:
Purpose: This test suite tests the visual appearance of the components added by the tasklist module, like menuitems, toolbar buttons, tooltips, shortcuts, action.
Setup:
#
Test case
Description
1
Check Menu
1. Open the Window main menu
RESULT:
There should be menuitem TODOs in this menu.
2
Check Toolbar
1. Make sure the Window toolbar is visible. 2. Click on the TODOs icon in the Windows toolbar.
RESULT:
The TODOs window should be opened.
3
TODO Window
1. Open the TODO Window. 2. Switch between All Files and Current File (stop scanning in the All Files).
RESULT:
The window should look like on the following screenshot: .
4
Edit Filters Dialog Appearence
1. Open the TODO Window. 2. Open the Edit Filters dialog.
RESULT:
The window should look like on the following screenshot: .
Purpose: The purpose of this suite is to test the GUI and functionality of the filters.
Setup: Project TasklistTestData opened in the IDE.
#
Test case
Description
1
Set Filter I
Setup: Provided file "GenericTest.java" opened in the editor. The TODO Window opened in the Current File tab.
1. Open the Edit Filters dialog. 2. Create a filter Only P1 Bugs. 3. Assign condition: Task Contains "P1" to the newly created filter. 4. Close the dialog using "OK" button. 5. Select the Only P1 Bugs filter in the filters combo in the TODO Window.
RESULT:
Only one TODO (containing the text "P1") should left in the todo list.
2
(De)activate Filter
Setup: The setup is test case Set Filter I.
1. Select the No Filter filter. 2. Select the Only P1 Bugs filter. 3. Select the No Filter filter. 4. Select the Only P1 Bugs filter.
RESULT:
The list of TODOs should contain 4 todos if the "No Filter" filter is not deactive and 1 todo if the "Only P1 Bugs" filter is active.
3
View Switch I
Setup: The setup is test case Set Filter I.
1. Switch to the Selected Folder. 2. Switch back to the Current File.
RESULT:
The "Only P1 Bugs" filter should be activated, and 1 todo should be shown.
4
Set Filter II
Setup: Provided file GenericTest.java opened in the editor. The TODO window opened in the "Current File".
1. Open the Edit Filters dialog. 2. Create a new filter Only P1&P2 Bugs. 3. Assing conditionTask Contains "P1" or Task Contains "P2" to the newly created filter. 4. Close the Edit Filters dialog with the OK button. 5. Select the Suggestion Contains "P1" or Suggestion Contains "P2" filter in the filters combo.
RESULT:
Two TODOs (containing the text "P1" or "P2") should left in the todo list.
5
Edit Filters Dialog Basic
Setup: This test case starts from clean userdir. Open the Edit Filters dialog.
1. If the Default Filter filter is selected, deselect it using Ctrl-Click. 2. Add a few filters using the New button. 3. Remove all but one newly added filters using the Delete button. 4. Select the newly created filter. 5. Change its name. 6. Using More button, add a few rules. 7. Remove all the rules using the Fewer buttons. 8. Using the Delete button remove all filters.
Teardown: Close the dialog using the Cancel button.
RESULT:
If none filter is selected, whole right part should be disabled altogether with the Delete button. If a filter is selected, and there is no rule, the Fewer button should be disabled. The dialog should not throw any exception under any circumstaces.
This testcase verifies the basic correctness of the Edit Filters dialog.
6
Edit Filters Dialog Cancel
Setup: This test case starts from clean userdir. Current File tab selected in the TODO Window. The GenericTest file opened in the editor.
1. Open the Edit Filters dialog. 2. Using the New button create a new filter. 3. Cancel the dialog. 4. Open the Edit Filters dialog. 5. Using the Delete button remove the Default Filter. 6. Cancel the dialog. 7. Open the Edit Filters dialog. 8. Select the Default Filter and modify the rules. 9. Cancel the dialog. 10. Open the Edit Filters dialog. 11. Using the New button create a new filter. 12. Assign condition: Suggestion Contains "P1" to the newly created filter. 13. Press the Preview button. The list of TODOs should be filtered according to the given condition. 14. Cancel the dialog.
RESULT:
After canceling the dialog all changes made to the filters should be lost. After doing Preview, no only changes to the filters should be lost, but also the TODO list should not be filtered according to the preview, but according to the currently selected filter.
Verifies functionali of the Cancel button.
7
Edit Filters Dialog Add
Setup: This test case starts from clean userdir. Current File tab selected in the TODO Window. The "GenericTest" file opened in the editor.
1. Open the Edit Filters dialog. 2. Using the New button create a new filter named Only P1 Bugs with condition Task Contains "P1". 3. Close the dialog using the OK button.
RESULT:
The Only P1 Bugs filter should be created, selected in the filters combo and the TODO list should be filtered according to it.
8
Edit Filters Dialog Remove
Setup: The setup for this test case is the previous test case.
1. Open the Edit Filters dialog. 2. The filter selected in the list of filters should be Only P1 Bugs filter. 3. Using this fitler using the Delete button. 4. Close the dialog using the OK button.
RESULT:
The Only P1 Bugs filter should be removed, the Default Filter filter selected in the filters combo and the TODO list should be filtered according to it.
9
Edit Filters Dialog Remove Default
1. Open the Edit Filters dialog. 2. Using the Delete button remove all filters. 3. Close the dialog using the OK button.
RESULT:
The filters combo should contain only the No Filter filter.
Purpose: This test suite tests functionality specific to the Selected Folder part of the TODOs window.
Setup: Checkout sources for the OpenIDE and editor modules from the NetBeans CVS. Start IDE with a clean (required) userdir and open projects for the above modules.
#
Test case
Description
1
Start Scanning
1. Invoke the "TODOs" action (from main menu Window/TODOs). 2. Select the Selected Folder. 3. A folder selection dialog should appear. Choose the directory containing all OpenIDE sources. 4. Wait until it is scanned.
RESULT:
The scanning should stop after finding more than "300" (current default) of TODOs.
2
Caching I
Setup: The previous test case is the setup for this test case.
1. Switch to Current File and back to Selected Folder.
RESULT:
The selected folder should not be changed. No rescan should occur, the results should be restored from cache. The tasklist's status line should state this.
3
Rescan&Stop
Setup: The setup for this step is the previous test case. The scanning for TODOs should not be finished yet.
1. Locate the Rescan button and press it. 2. Wait until the Stop button next to the progress bar is shown. 3. Press the Stop button.
Teardown: Close the "TODOs" window.
RESULT:
The scanning should stop immediately. The todos scanned so far should not disappear.
4
Selected Folder Combo I
Setup: The previous steps are setup for this test case.
1. Press the combo button located next to the Selected Folder button. 2. A combo stating "Choose Folder..." and the current folder should be shown. 3. Select "Choose Folder...". 4. A folder selection dialog should appear. 5. Choose the editor's "libsrc" folder.
RESULT:
The selected folder should be scanned (wait for it).
5
Selected Folder Combo II
Setup: The previous test case is the setup.
1. Open the combo again. It should state "Choose Folder..." and the two already scanned folders. 2. Select the not currently chosen folder.
RESULT:
The scanning of the folder should start. Wait for it.
6
Selected Folder Combo III
1. Add two more folder to the Selected Folder combo ("openide/src" and "openide/execution". Each time wait until the scaning is done. 2. Scan anoter folder ("editor/src"). Wait until in finishes. 3. Open the Selected Folder combo.
RESULT:
There should be only the last 4 folders scanned.
Purpose: This test suite tests functionality specific to the Current File tab of the TODOs window.
Setup: Provided test project opened in the IDE, "GenericTest.java" and "OpenedFileA.java" files opened in the editor. The "GenericTest.java" should be the selected file initialy.
#
Test case
Description
1
Begining
1. Invoke Window/TODOs.
RESULT:
The TODO window should contain the list of TODOs in the "TODOTest.java" file.
2
Fixing a todo.
Setup: The setup for this step is the previous test case.
1. Double click on a todo (the appropriate line in the editor should be highlighted, and the focus should be in the editor). 2. Remove the todo tag (do not save the file).
RESULT:
The appropriate todo item should be removed from the list of todos.
3
Adding a new todo.
Setup: The setup for this step is the previous test case.
1. Go into the editor ("GenericTest.java"). 2. Add a todo item in the form of //TODO: test. Do not save the file.
RESULT:
The newly added todo item should be shown in the TODOs window.
4
Changing File in Editor
1. Select the "OpenedFileA.java" in the editor. 2. Select the "GenericTest.java" in the editor. 3. Select the "OpenedFileA.java" in the editor. 4. Select the "GenericTest.java" in the editor.
RESULT:
The content of the TODOs Window should be set according to the currently active file.
Purpose: The purpose of this suite is to test the Opened Files tab of the TODO Window.
Setup: Project TasklistTestData opened in the IDE.
#
Test case
Description
1
Opening TODO Window
Setup: This test case starts from clean userdir. In particular the TODO Window should not be opened.
1. Open test files "OpenedFile[A-D]" in the editor. 2. Open the TODO Window and select the Opened Files tab.
RESULT:
The TODOs from the opened files should be shown in the list of TODOs.
2
Fixing a todo II
Setup: The setup for this step is the previous test case.
1. Double click on a todo (the appropriate line in the editor should be highlighted, and the focus should be in the editor). 2. Remove the todo tag (do not save the file).
RESULT:
The appropriate todo item should be removed from the list of todos.
Althought this test case has its counterpart in the Current File part, this tests different functionality, with possibly different bugs (as proven in practice). See bug #43518.
3
Adding a new todo II.
Setup: The setup for this step is the previous test case.
1. Go into the editor ("OpenedFileA.java"). 2. Add a todo item in the form of //TODO: test. Do not save the file.
RESULT:
The newly added todo item should be shown in the TODOs window.
Althought this test case has its counterpart in the Current File part, this tests different functionality, with possibly different bugs (as proven in practice). See bug #43518.
4
Open Additional Files
Setup: The setup for this step is the previous test case.
1. Open file "OpenedFileE". 2. Open file "OpenedFileF".
RESULT:
The changes in the set of opened files should be (immeditally) reflected in the list of TODOs.
5
Close Files
Setup: The setup for this step is the previous test case.
1. Open file "OpenedFileA". 2. Open file "OpenedFileC".
RESULT:
The changes in the set of opened files should be (immeditally) reflected in the list of TODOs.
6
Change Selected File
Setup: The setup for this step is the previous test case.
1. Select "OpenedFileB.java" in the editor. 2. Select "OpenedFileD.java" in the editor. 3. Select "OpenedFileF.java" in the editor.
RESULT:
The content of the TODOs Window should not change while the selected file in the editor is changed.
Purpose: This test suite tests generic features of the TODO Window: double-click on the todo, go-to-source button, go-to-source action and F12 cycle.
Setup: Project TasklistTestData opened in the IDE.
#
Test case
Description
1
Go To Source I
Setup: File "GenericTest.java" opened in the editor. The TODO Window opened in the Current File tab.
1. Select the first TODO in the list. 2. Double click on it.
RESULT:
The editor should be activated, the caret should be at the corresponding line, the line should be highlighted and visible (the editor should scroll to it if necessary).
2
Go To Source II
Setup: Provided file GenericTest.java opened in the editor. The TODO window opened in the "Current File".
1. Select the first TODO in the list. 2. Click on the "Go To Source" icon.
RESULT:
The editor should be activated, the caret should be at the corresponding line, the line should be highlighted and visible (the editor should scroll to it if necessary).
3
Go To Source III
Setup: File "GenericTest.java" opened in the editor. The TODO Window opened in the Current File tab.
1. Select the first TODO in the list. 2. Open the context menu on this node (right click with the mouse) and invoke the Show Source action.
RESULT:
The editor should be activated, the caret should be at the corresponding line, the line should be highlighted and visible (the editor should scroll to it if necessary).
4
F12 cycle
Setup: Provided examples mounted and open provided "TODOTest.java" file.
1. Open the TODO Window. 2. Repeately press F12 to cycle.
RESULT:
For every and each TODO, the go to source action should be performed. After the last TODO, one more F12 should be pressed to start at the begining of the list again.
Purpose:
Setup: Project TasklistTestData opened in the IDE.
#
Test case
Description
1
Skip Tasks Outside Comments
Setup: Provided file "GenericTest.java" opened in the editor. The TODO Window opened in the Current File tab.
1. Open the options (Tools/Options). 2. Select node Editor/Source Tasks Settings. 3. Change the value of property "Skip Tasks Outside of Comments" to true. 4. Close the Options dialog.
RESULT:
The list of TODOs should be immediatelly changed not to show TODOs outside comments.
Concurrently, the flag does not recognize the line comment ('//') correctly, so todos
inside the line comment are ignored (add IZ number when reported). In the future, more file types than Java could be checked.
2
Task Tags I
Setup: Provided file "GenericTest.java" opened in the editor. The TODO Window opened in the Current File.
1. Open the options (Tools/Options). 2. Select node Editor/Source Tasks Settings. 3. In the property "Tasks Tags" add pattern SODO with normal priority. 4. Close the Options dialog.
RESULT:
A new "SODO" should appear on the list of TODOs.
3
Task Tags II
Setup: The setup required for this testcase is the previous testcase.
1. Open the options (Tools/Options). 2. Select node Editor/Source Tasks Settings. 3. In the property "Tasks Tags" remove pattern TODO. 4. Close the Options dialog.
RESULT:
All TODOs should be removed from the list of TODOs.
4
The TODOs limit I
Setup: OpenIDE module project opened in the IDE.
1. Open the options (Tools/Options). 2. Select node Editor/Source Tasks Settings. 3. Change the value of the property "List Limit" to "10". 4. Close the Options dialog. 5. Open the TODOs Window and choose Selected Folder, select folder containing the whole openide module.
RESULT:
The scaning should start and stop after finding a bit more that "10" TODOs.
5
The TODOs limit II
Setup: OpenIDE module project opened in the IDE.
1. Open the options (Tools/Options). 2. Select node Editor/Source Tasks Settings. 3. Change the value of the property "List Limit" to "100000". 4. Close the Options dialog. 5. Open the TODOs Window and choose Selected Folder, select folder containing the whole openide module.
RESULT:
The scaning should start and whole directory tree should be scanned and all TODOs found (unless there is more than "100000" of them).
Teardown:
Persistence is considered in the Persistence test suite.
Purpose:
Setup: Project TasklistTestData opened in the IDE.
#
Test case
Description
1
Settings Persistence
1. Open the options (Tools/Options). 2. Select node Editor/Source Tasks Settings. 3. Change the values of all properties "Skip Tasks Outside of Comments", "Tasks Tags" and "List Limit". 4. Close the Options dialog. 5. Restart the IDE. 6. Open the options (Tools/Options). 7. Select node Editor/Source Tasks Settings. 8. Check that all properties has the same values as before restarting. 9. Close the Options dialog. 10. Restart the IDE. 11. Open the options (Tools/Options). 12. Select node Editor/Source Tasks Settings. 13. Check that all properties has the same values as before restarting.
RESULT:
The properties should be always correctly set.
2
TODOs Window Persistence
1. Open the TODO Window. 2. Restart the IDE. 3. Close the TODO Window. 4. Restart the IDE.
RESULT:
After the IDE restart, the TODO Window should be opened/closed according the original state.
3
Filters Persistence
1. Using the Edit Filters create a few filters and remove the Default Filter. 2. Restart the IDE. 3. Verify that the filters were load correcly (that the previous changes are correctly reflected). 4. Restart the IDE. 5. Verify that the filters were load correcly (that the previous changes are correctly reflected).
RESULT:
The changes to the filters should be saved/loaded correctly.
Purpose:
Setup: The "List Limit" property set to very high value (at least "1000"). Open the Memory Toolbar.
#
Test case
Description
1
Many Java TODOs
Setup: Project TasklistTestData with Java sources (300x10 TODOs) opened in the IDE.
1. Start scaning for TODOs in the test2 directory
RESULT:
Track the memory consuption during the scanning process using the Memory toolbar. The scanning should be done in the reasonable time.
2
OpenIDE
1. Checkout the OpenIDE project from the NetBeans CVS. 2. Open the OpenIDE project. 3. Open the TODO Window and select the Selected Folder tab. 4. Select whole OpenIDE project and wait until it is scanned.
RESULT:
The scanning should be done in a reasonable time and should consume acceptable ammount of memory (verify by forcing the Garbage Collection for a few times).
3
Rescan
Setup: Project TasklistTestData with Java sources (300x10 TODOs) opened in the IDE.
1. Start scaning for TODOs in the test2 directory. 2. Force Garbage Collection at least 5 times and remeber the lowest value. 3. Invoke the rescan and wait until the rescan is done. Repeat 10 times. 4. Force Garbage Collection at least 5 times and remeber the lowest value.
RESULT:
The IDE should not consume much more memory after doing the rescan. If the difference between the second and first value is more than 2MB, suspect a memory leak.
ATTRIBUTES: leaks
4
Skip Outside Comments
Setup: Directory test2 with Java sources (300x10 TODOs) mouted in the IDE.
1. Open the options (Tools/Options). 2. Select node Editor/Source Tasks Settings. 3. Change the value of property "Skip Tasks Outside of Comments" to true. 4. Close the Options dialog. 5. Start scaning for TODOs in the test2 directory
RESULT:
Track the memory consuption during the scanning process using the Memory toolbar. The scanning should be done in the reasonable time.
5
Filters Enable/Disable
Setup: Project TasklistTestData with Java sources (300x10 TODOs) opened in the IDE.
1. Open the TODO Window and switch to Selected Folder tab, select the test2 directory. 2. Wait until the TODOs are scanned. 3. Force Garbage Collection at least 5 times. Remeber the lowest value. 4. Switch Filters from No Filter to Default Filter and back for 10 times. 5. Force Garbage Collection at least 5 times. Remeber the lowest value
RESULT:
Compare the two memory consumptions. If the second is imporantly higher (>2MB), suspect memory leak.
.
.
icon.