Sorting

    Note: The only things that have been implemented thus far are in purple.

    Introduction

    The goal of this update is to improve the rules Songbird uses to sort metadata fields exposed in media list types.

    Currently, Songbird has minimal rules for handling sorting of the metadata associated with media that has been imported (causing users to grow frustrated with how their data appears.)

    Purpose of This Document

    The purpose of this document is to scope the work necessary to implement a basic set of metadata sorting features intended to satisfy a consumer media player audience.

    What this document covers:

    • A set of basic sorting rules the media list types should implement.
    • Specifically what columns these rules should be applied against.

    What this document doesn't cover:

    • Implementing locale specific sorting

     

    Improve Sorting by Introducing Additional Rules

    The following sort order rules are requested to be implemented in the following priority:

    Rule #1: Implement a General Sorting Order

    For columns supporting fields with strings of mixed types the following rules should be followed. Exceptions to columns will be documented in another section.

    First, strings should be sorted by Letters and Unicode characters.

    In order of the Latin alphabet A-Z and implementing the Unicode collation algorithm.  Order should continue to be case insensitive. 

    Note: this applies only to names, words or strings of text and doesn't include digits.

    Example:

    Un-collated Collated
    bad bad
    Bad Bad
    Bat bäd
    bat Bäd
    bäd bat
    Bäd Bat
    bät bät
    Bät Bät
    côté black-bird
    coté black-birds
    côte blackbird
    cote blackbirds
    black-bird cote
    blackbird coté
    black-birds côte
    blackbirds côté

    Second, strings should be sorted by misc 'other' characters

    For example, implement radical-and-stroke sorting for non-alphabetic writing systems.  Such as Chinese hanzi and Japanese kanj.

    Example:

    Un-collated Collated
    アウトキャスト 보통 빠르기/느리기
    보통 빠르기/느리기 アウトキャスト
    アルタネイティヴ・サン あさがお
    エル・スール(南) アルタネイティヴ・サン
    あさがお エル・スール(南)

    Third, sort by Digits.

    Example:

    Un-collated Collated
    10 0.720
    38 1
    0.720 5
    41 10
    5 38
    1 41

    Fourth, for null fields (or for fields that have been stripped to null) sort using field data in other columns.

    Example:

    Un-collated title:artist:track number Collated title:artist:track number
    + : Kazumasa Hashimoto : 2 !"£$%^&*(()) : Aphex Twin : 1
    !"£$%^&*(()) : Aphex Twin : 1 .......... : Caroline Martin : 8
    .......... : Caroline Martin : 8 + : Kazumasa Hashimoto : 2
    ? : Low : null ++ : Kazumasa Hashimoto : 6
    ? : My Morning Jacket : null +++ : Kazumasa Hashimoto : 8
    +++ : Kazumasa Hashimoto : 8 ? : Low : null
    ++ : Kazumasa Hashimoto : 8 ? : My Morning Jacket : null

    Rule #2: Ignore leading definite and indefinite articles

    Note: In English definite articles are "A ", "An ", and "The "
    Note: In addition we should ignore the localized equivalents for spanish, french, german, portuguese, and italian.

    Example: "The Rolling Stones" sorts as "Rolling Stones" "A Love Supreme" sorts as "Love Supreme"

    Rule #3: Ignore leading symbols and punctuation

    Note: Unless we tackle this differently, this rule implies that negative digits would be sorted minus the punctuation.

    Example: "¢50" should sorts as "50" " ' Aight, Dog " should sort as "Aight, Dog"

    Rule #4: Sort digits leading a string in numeric order

    Example: The songs "1 track", "2 track", "10 track" should sort as "1 track", "2 track", "10 track".

    Rule #5: Ignore leading white space

    Any number of spaces entered at the beginning of the title should be ignored when sorting.

    Stories 

    ID Title Story Testcase
    bug 14028 Implement General Sorting Rules

    Songbird should introduce a set of prioritized rules for sorting default columns. (Main Tracking bug)

    n/a
    bug 14029 Sort by letters and unicode characters Strings should be sorted by Letters and Unicode characters. 2247

     
    bug 14030 Handle radical-and-stroke sort Strings should be sorted by misc 'other' characters using radical-and-stroke sorting (for non-alphabetic writing systems) 2472
    bug 14031 Sort leading digits in numeric order Strings should be sorted by Digits, if any. 2473
    bug 14032 Sort null fields with other columns For null fields (or for fields that have been stripped to null) sort using field data in other columns. (Exact rules TBD and posted above) 2474
    bug 14033 Ignore leading definite and indefinite articles Ignore leading definite and indefinite articles for english, spanish, french, german, portuguese, italian (as described in bug 8996) 2476
    2507
    2509
    2511
    2513
    2515
    bug 14035 Ignore leading symbols, punctuation and whitespaces Leading symbols, punctuation and whitespaces should not have any impact on sorting. 2478
    2479
    2480
    bug 14272 Sort old databases with new sorting rules Old databases should be migrated so that the new sorting rules take effect on them too. 2481
    2482
    2483
    2484
    2485
    2486
    2487
    2488
    2508
    2510
    2512
    2514
    2516

    Engineering Tasks

    ID Cost Title Task Testcase
    bug 14027 3 Implement unicode collation for sort columns Turn current "sortable" column into "searchable", fix usage, implement new "sortable" column using unicode collation data.  
    bug 8996 3 Ignore articles based on dictionnary and position in string Implement a dictionnary of articles which should be ignored when found first or last in the string 2476 
    bug 14024 1 Ignore leading symbols, punctuation and whitespaces Ignore leading symbols, punctuation and whitespaces 2478
    2479
    2480
    bug 14025 3 Sort digits leading a string in numeric order Sort digits leading a string in numeric order 2473
    bug 14271 2 Migrate database Write db migration step for new sorting columns 2481
    2482
    2483
    2484
    2485
    2486
    2487
    2488
    2508
    2510
    2512
    2514
    2516
    bug 14725 2 Fix collate for osx Fix collation sort on osx, possibly move collate into sqlite custom collation (ADDED 05/01/2009) All testcases
    above
    WILL be executed on OS X 10.5.6

    Applying Sorting Rules

    After implementation of the above 5 rules/stories if I were to sort each of the above examples together by title (in ascending order, taking no other secondary rules into order) the following should be the results:

    bad
    Bad
    bäd
    Bäd
    bat
    Bat
    bät
    Bät
    black-bird
    black-birds
    blackbird
    blackbirds
    cote
    coté
    côte
    côté
    보통 빠르기/느리기
    アウトキャスト
    あさがお
    アルタネイティヴ・サン
    エル・スール(南)
    0.720
    1
    5
    10
    38
    41
    !"£$%^&*(()) : Aphex Twin : 1
    .......... : Caroline Martin : 8
    + : Kazumasa Hashimoto : 2
    ++ : Kazumasa Hashimoto : 6
    +++ : Kazumasa Hashimoto : 8
    ? : Low : null
    ? : My Morning Jacket : null

    Secondary Sort Column Rules

    Expected results in order of secondary sort priority

    Album

    Album Title => Disc Number => Track Number => Track Name
    (note: need to decide if we want to support selecting secondary sort orders ie by artists and by year on the column name)

    Artist

    Artist Name => Album Name => Disc Number => Track Number => Track Name

    Album Artist

    Album Artist Name => Album Name => Disc Number => Track Number => Track Name

    Beats Per Minute

    BPM => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Bit Rate

    BPM (no bitrate appears first since it effectively = 0) => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Comment

    Comment => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Composer

    Composer => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Date Added

    Date Added (since this is not editable by a user it should be a part of every single media item and thus there is no secondary sortable data set)

    Date Modified

    Date Modified (since this is not editable by a user it should be a part of every single media item and thus there is no secondary sortable data set. note: if we don't write date modified at every single import session then we need to fallback to date added)

    Disc Number

    Disc Number => Artist Name => Album Name => Track Number => Track Name

    File Location

            No secondary sort, un-necessary - sort alphabetically

    Genre

    Genre => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Last Played

    Last Played (most recently played appears first) => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Last Skipped

    Last Skipped (most recently played appears first) => Artist Name => Album Name => Disc Number => Track Number => Track Name


    Play Count

    Play Count (most recently played appears first) => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Rating

    Rating => Artist Name => Album Name => Disc Number => Track Number

    Sample Rate

    Sample Rate => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Size

    Size => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Skip Count

    Skip Count => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Time

    Duration => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Title

    Track Name => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Track Number

    Track Number => Artist Name => Album Name => Disc Number

    Year

    Year => Artist Name => Album Name => Disc Number => Track Number => Track Name

    Additional Sorting Rules for Metadata Editor Sorting Fields

    ID Cost Title Story Testcase
    bug 14169 2

    Sorting by a column should apply secondary sort rules where necessary

    When a column heading is sorted additional secondary sort rules should be applied. For instance sorting by Album would sort the tracks associated with it first by Album Title then Disc Number then Track Number then Track Name.  See above table.

     

    Known Risks

     

    Interdependencies


    Future

    Implement a mechanism to sort by alternate fields (UI and metadata fields). Implement support for sorting international indefinite/definite articles. Implement support for sorting downloads based on queue order.

    Tag page
    • No tags
    Viewing 11 of 11 comments: view all
    * Will need to do something with the search data. Search data currently comes from the sortable column
    * Sortable will become a binary weight
    * Filter lists will need changes, but should be minimal
    * Unit tests (generic text file for confirmation)
    * Only handle leading numbers (pad appropriately)
    Posted 17:58, 14 Nov 2008
    (not actually georges. the above was posted from JnB after the first meeting with Francis)
    Posted 17:58, 14 Nov 2008
    One of the story (bug 14032) says "For null fields (or for fields that have been stripped to null) sort using field data in other columns. (Exact rules TBD and posted above)", but isn't that covered by the second part of this document, which lists the secondary sorts for the default columns ? If a field is null, shouldn't it end up at the end of the list and then be sorted amongst all other null fields using the secondary columns ?
    Posted 14:01, 1 Dec 2008
    for the secondard sorting columns, can you please also make sure you include one for album artist such as:

    Album Artist => Album => Disc No => Track No.

    or similar.

    currently sorting on album artist has the track number all messed up, so is unusable.
    Posted 08:36, 4 Jan 2009
    Also for future: read, and use, the TSO* tags that may exist in songs to determine sort order. For example, John Lennon may have a TSOA tag of "Lennon, John" that sorts him within the L artists instead of the J artists.
    Posted 19:13, 5 Jan 2009
    One function that I have been reliant on in itunes, is when you start typing it goes to the Artist, album, etc. for whatever is selected. But I changed a hotkey to make space bar start/pause and so I can't find bands with multiple names. Is there a way you can make the spacebar start/pause but when you start typing (not in the search bar) it ignores this.
    Posted 11:06, 21 Jan 2009
    I think defining so many explicit sorting rules is an example of bad design style. Not only can the user have difficulty in remembering these orders, such complex rules give rise to unexpected, unintuitive behavior. As an example, I had to search online to figure out a way to sort tracks of an album in the natural order of first disc number and then track number.

    IMHO, a much better idea would be to implement stable sort where the user can quickly and easily obtain any order he desires using just few clicks.
    Posted 12:13, 17 Mar 2009
    Err, my previous comment was regarding " Secondary Sort Column Rules "
    Posted 12:16, 17 Mar 2009
    Is disc number sorting supposed to be implemented in version 1.2? It doesn't look like it using the DAAP client extension, despite it populating the discNumber property.
    Posted 17:16, 1 Jul 2009
    I think allowing user defined secondary sorting would greatly improve the users experience. A lot of people have their own way of organizing music that is not compatible with the current pre-defined secondary sorting rules. Allowing each user to control their secondary sorting is definitely one of the most important features in my opinion. I prefer my music to all be alphabetical Artist=>Track Name, that way when the "artist" column is selected, all of the music from one particular artist is automatically displayed alphabetically.
    Posted 09:18, 11 Jul 2009
    I don't think this thread has been updated in a while, but custom secondary sorting is a hugely valuable thing that you can do in foobar2000 that you can't currently do in songbird (to my knowledge)
    Posted 18:58, 19 Nov 2011
    Viewing 11 of 11 comments: view all
    You must login to post a comment.
    Powered by MindTouch Core
    Real Time Web Analytics