WORKING DRAFT
This document provides a high level overview of the test areas/and scenarios to be covered by QA for the Hendrix Songbird application release. Detailed test cases will be available via our test case management database at: http://litmus.songbirdnest.com/
QA will test with the following OS platforms for the Hendrix project. Tier 1 platforms will receive full new feature, regression, performance, stability and nest integration testing. Tier 2 will receive installation testing as well as quick looks of all Functional test types:
Updates must be verified as working within one week of code complete.
Installation testing scenarios will include:
First run Add-Ons for 1.1.0 will be the following, in this order:
The following foreign languages will be tested to ensure that they load/view correctly:
This section outlines the new features for the Hendrix 1.1.0 Songbird application and areas where new test cases will be written to execute throughout the product development.
Fetching Album Art Project Page
The new Album Art Fetching feature will give Songbird the ability to query multiple sources in an attempt to associate album artwork with every audio track in a user's library. Songbird will have three built-in sources of album art: track metadata, the local filesystem, and Last.fm. An add-on may also be created to allow fetching art from Amazon.
At a high level, testing will ensure that:
User Interface
A new, user-invoked dialog box will be added to Songbird. This dialog will display progress on an album by album detail, with the current fetcher being used to search for artwork. Progress will display as a progress bar and a cancel button is available to abort the search. Details needed.
songbird.metadata.artwork.enableWriting (default: true)
songbird.albumart.file.base_names (default: cover,folder)
songbird.albumart.file.extensions (default: jpg,jpeg,png,gif,bmp,tif,tiff)
songbird.albumart.maxsize (default: 16777216)
Video Playback Project Page
Full video support is currently planned for Isan, not Hendrix. In this release, known issues in video support will be fixed. Those issues are listed on the Video Playback project page (linked above). The test effort during Hendrix will be limited to verifying fixes for the bugs within the scope of the project.
Normalisation Project Page
Songbird will use volume normalization information, when available, to adjust playback volume. Normalization information may be available in the form of Replay Gain data and/or iTunes Sound Check metadata. In instances where both types of data are available, Replay Gain takes precedence.
Testing will ensure that volume normalization data, when present, is read and seems to be used properly. The testing effort will likely not extend beyond simple listening tests with files that we've modified to be either extremely loud or extremely quiet.
Replay Gain information can be stored in (at least) three different ways:
foobar2000 stores Replay Gain information using the id3v2.3 TXXX field.
replaygain_album_gain (lower numbers = softer playback)
replaygain_album_peak
replaygain_track_gain (lower numbers = softer playback)
replaygain_track_peak
See also: TXXX (user-defined text information frame, see section 4.2.6 at http://www.id3.org/id3v2.4.0-frames)
vorbisgain, flac 1.1.3, and probably other encoders/Replay Gain calculators store Replay Gain information for Ogg and FLAC files in Vorbis comments fields. The fields are the same as those for mp3s, except in all caps:
REPLAYGAIN_TRACK_GAIN
REPLAYGAIN_TRACK_PEAK
REPLAYGAIN_ALBUM_GAIN
REPLAYGAIN_ALBUM_PEAK
It appears that there are deprecated tags named RG_RADIO, RG_PEAK, and RG_AUDIOPHILE. These are probably not implemented and will not be tested.
The source code for vorbisgain defines NO_PEAK as -1.f and NO_GAIN as -10000.f. It appears that values greater than the NO_PEAK and NO_GAIN are valid peak and gain values.
All of these assumptions are based on the source code found here: http://sjeng.org/vorbisgain.html
Replay Gain information can also be stored within a LAME header inside an Xing header. The authoritative page seems to be this one: http://gabriel.mp3-tech.org/mp3infot...tml#replaygain.
Songbird will use iTunes Sound Check information, when available, to normalize playback volume. iTunes stores Sound Check data inside an id3v2.2 COMM (comment) tag using the tag iTunNorm.
Sorting Project Page
The overhaul of Songbird's sorting mechanism in Hendrix will rely primarily on Unicode Collation to create industry-standard sorted lists within Songbird's library (and similar media lists, such as Playlists and filtered library views). Collation algorithms will sort on a per-locale basis, so testing must cover not only sorted lists on machines in the US English locale, but also in the locales of our Tier 1 languages (Chinese, French, German, Italian, Japanese, and Spanish).
We will also be introducing special rules to Songbird's sorting functionality to be more intelligent about sorting punctuation, whitespace, and definite and indefinite articles.
In addition to the introduction of Unicode Collation, Songbird will begin omitting certain characters and words when sorting. Testing will ensure that the following characters/words, when appearing at the beginning of a name, do not determine the sort order of that particular item:
Items beginning with digits should sort numerically. The exception to this rule is negative numbers; items beginning with a negative number will sort as though the number is positive.
Secondary Sort rules determine how items in the library are sorted based upon which column is the Primary Sort column. See the Secondary Sort Column Rules section of the Hendrix Sorting feature document for a full list of secondary sorting behavior.
Code charts from Unicode should be used to test correct sorting. Note that we are using OS-specific implementations of the Unicode Collation algorithm, so sorting should be tested thoroughly on all Tier 1 platforms. The test effort should cover the following:
Unicode Collation Charts for English, French, German, Italian, and Spanish:
Basic Latin
Latin-1
Language-specific Collation Charts (locale-specific subsets of Latin and Latin-1):
English: http://developer.mimer.com/charts/english.htm
French: http://developer.mimer.com/charts/french.htm
German: http://developer.mimer.com/charts/german.htm
Italian: http://developer.mimer.com/charts/italian.htm
Spanish: http://developer.mimer.com/charts/spanish.htm
Unicode Collation Charts for Japanese:
Hiragana
Katakana
Unicode Collation Chart for Chinese and Japanese:
CJK Unified Ideographs
In addition to the Tier 1 Languages, the community has expressed gratitude for the fact that Songbird sorts Hebrew and Cyrillic properly. A spot-check should be performed on these languages to ensure we didn't regress.
Hebrew
Cyrillic
N.B.: The Swedish language will be a good way to quickly verify that locale-specific sorting is occurring, because the characters å, ä, and ö should be sorted at the end of the alphabet. See http://developer.mimer.com/charts/swedish.htm for a full Swedish collation chart.
Search: Songbird's library search functionality currently piggybacks off of the existing sorting data. We should expect search to be affected by the modification to the existing sorting data.
Database: Part of the upgrade from the 1.0 product will be a database migration that updates the existing sorting data. See bug 14271 for details.
Invoking Songbird from URLs Project Page
Content Type Detection Project Page
ACM mp3 Decoder (Bugzilla)
Mediacore Preferences Project Page
QA will conduct a full test pass of all of the following functional areas on all supported Tier 1 OS platforms to ensure the following legacy code does not regress with the Hendrix Songbird application release:
Locales
The following items are out of scope of this testing effort:
| File | Size | Date | Attached by | |||
|---|---|---|---|---|---|---|
| U0000.pdf Basic Latin | 159.78 kB | 13:40, 6 Jan 2009 | ali | Actions | ||
| U0080.pdf Latin-1 | 138.69 kB | 13:40, 6 Jan 2009 | ali | Actions | ||
| U0400.pdf Cyrillic | 171.09 kB | 13:57, 6 Jan 2009 | ali | Actions | ||
| U0590.pdf Hebrew | 77.96 kB | 13:57, 6 Jan 2009 | ali | Actions | ||
| U3040.pdf Hiragana | 105.71 kB | 13:40, 6 Jan 2009 | ali | Actions | ||
| U30A0.pdf Katakana | 96.39 kB | 13:54, 6 Jan 2009 | ali | Actions | ||
| U4E00.pdf CJK Unified Ideographs | 4.83 MB | 13:57, 6 Jan 2009 | ali | Actions | ||
| Images 0 | ||
|---|---|---|
| No images to display in the gallery. |
Restart w/ different languages in English OS.
Test import and sorting in native OSes for 2 non-English languages (German and Chinese) edited 16:03, 16 Jan 2009