{{wiki.toc("/QA/Releases/Project_Test_Plan")}}
In addition to the people listed in the Project Definition table, n other people are available to help with verifications, device testing, and regression testing.
We plan to cover 100% of the testing effort as outlined below. If we start to run short on time, testing will be truncated in the Priority 3 regression tests.
The Songbird installer for Windows will be exercised on Tier 1 Windows platforms using the pre-defined regression tests in Litmus (full list of tests).
The following add-ons will be offered to users, in the following order, during the Songbird first run wizard:
The following update scenarios, using both partial and complete updates, will be tested on all supported platforms:
Text
Text
Text
Text
Text
The existing suite of regression tests will be split up into three priority tiers to ensure that the highest priority features are thoroughly tested. A list of the full set of regression tests, plus the execution status of those tests, is available in Litmus (http://litmus.songbirdnest.com/test_...t_run_id=xxxxx).
Regression areas fall into priority tiers (listed below), with Priority 1 being the highest priority and Priority 3 being the lowest. We intend to execute 100% of the regression tests; however, if we run low on time during the testing phase, we may choose to execute only minimal testing against Priority 3 areas. Priority 1 tests will be executed before Priority 2 tests during their scheduled week to ensure adequate time for bug fixing (see the xxxx Test Schedule for details).
Performance Testing will ensure that Songbird remains within 20% of the following baseline performance values on Windows:
| Test Task | Testcase ID | Test Setup/Description | Performance Metrics Goal | ||
| Application Startup | |||||
| First Run - Final Win setup wizard screen to --> first SB setup screen - w/ Network | TBD | Clean, full install | TBD | ||
| First Run - Final SB setup screen to first app window UI shown - w/ Network | TBD | No First Run Addons installed No files imported | TBD | ||
| First Run - Final Win setup wizard screen to --> first SB setup screen - w/o Network | TBD | Clean, full install | TBD | ||
| First Run - Final SB setup screen to first app window UI shown - w/o Network | TBD | No First Run Addons installed No files imported | TBD | ||
| Cold Startup - w/ Network | 3488 | No First Run Addons installed 10,000 local tracks in library | < 30s | ||
| Cold Startup - w/o Network | TBD | No First Run Addons installed 10,000 local tracks in library | < 30s | ||
| First Quit/Relaunch immediately after online install - w/ Network | TBD | No First Run Addons installed 10,000 local tracks in library | < 8s | ||
| First Quit/Relaunch immediately after online install - w/o Network | TBD | No First Run Addons installed 10,000 local tracks in library | < 8s | ||
| Warm Startup - Subsequent average relaunches w/ Network | 3489 | No First Run Addons installed 10,000 local tracks in library | < 8s | ||
| - Subsequent average relaunches w/o Network | TBD | No First Run Addons installed 10,000 local tracks in library | < 8s | ||
| Media Import | |||||
| Import 10,000 audio items | 3491 | Reference 10,000 local tracks, no feature extraction | 2m40s | ||
| Import XXX video items | TBD | Reference XX files | TBD | ||
| Device - MTP | |||||
| Mount time | 3494 | Time for the icon of a clean device to appear in the Service Pane | < 5s | ||
| Full Sync (1K/10K) Audio | 3492 | Transfer 1000 songs from library of 10,000 songs to clean device (MTP speed of ~4MB/s and tracks size of ~5MB) | 25m | ||
| Partial Sync Audio | 3496 | Transfer 20 songs from library of 10,000 songs to device with 1000 songs (MTP speed of ~4MB/s and tracks size of ~5MB) | < 30s | ||
| Device library display | 3519 | Display of 1K device library in list view | < 20s | ||
| Sync video | TBD | Reference XX files | TBD | ||
| Transcode video | TBD | Reference XX files | TBD | ||
| Device - MSC | |||||
| Mount time | 3491 | Time for an icon for the device to appear in the service pane; clean device | < 5s | ||
| Full Sync (1K/10K) Audio | 3493 | Transfer 1000 songs from library of 10,000 songs to clean device (MTP speed of ~4MB/s and tracks size of ~5MB) | 25m | ||
| Partial Sync Audio | 3495 | Transfer 20 songs from library of 10,000 songs to device with 1000 songs (MTP speed of ~4MB/s and tracks size of ~5MB) | < 30s | ||
| Device library display | 3520 | Display of 1K device library in list view | < 20s | ||
| Sync video | TBD | Reference XX files | TBD | ||
| Transcode video | TBD | Reference XX files | TBD | ||
| Usage | |||||
| Memory usage | 3521 | Application idle after a cold start | ~ 80MB | ||
| Memory usage | 3522 | Import 10,000 local tracks | Not over 120MB | ||
| Memory usage | 3523 | Transfer 1000 tracks to clean device | Not over 120MB | ||
| Memory usage | 3524 | Transfer 20 tracks to device with 1000 tracks | 80M -120MB | ||
| CPU usage during playback | 3525 | Audio track playback only | < 5% | ||
The following items will not be tested in this release:
| Images 0 | ||
|---|---|---|
| No images to display in the gallery. |