Creating a quality standard

[Goreld]

Hello everybody,

I think we should define a clear quality standard for scanned scores, with 3 or 4 grades (A, B, C...), based on objective criteria, which would be set (or confirmed) by the moderation team. The public notation system is underused and useless in my opinion. When I'm looking for a score, the information mentioned are generally very poor, concerning the quality of the file I'm downloading. I'm reduced to guess the visual quality, based on contributor, file size, etc... The other benefit or a clearly defined rating would be the possibility for a potential contributor to visualize the files that would gain to be reprocessed (without rescanning), as I sometimes do with my Photoshop scripts: converting to real B&W, cropping, splitting 2-pages scans, realigning, CCITT-4 or JBig2 compressing.

My starting proposition, with general guidelines:
A-grade: above 600dpi B&W (or above 300dpi grayscale), neat score almost without artifacts
B-grade: between 300 and 600dpi B&W (or between 150 to 300dpi grayscale), some artifacts and minor flaws tolerated
C-grade: below B-grade
Note: tests should be done to define dpi equivalence between B&W and grayscale quality on visual quality (x2 is a standard equivalence IMO).
I would add a "+" (A+, B+, C+) for a well optimized file : strict B&W, alignement ok, homogeneous margins (which I fix to 5% horizontally and 5% vertically), CCITT-4 or JBig2 compression.

I'm waiting for your thoughts and contributions about my idea.

[coulonnus]

We have already Quality rating in http://imslp.org/wiki/User:Peter

Note that some scores have been imperfectly printed or have poor music engraving standards. This should not influence your quality grade. And read viewtopic.php?t=7266

[Goreld]

I've already seen those posts. Am I missing something, I can't find where Peter's ratings are, even on his own contributions. The goal of using a rating is for public to see it! It really is not obvious for the casual user which file he should download first, in a given page. Does it never happen to you to download from imslp 3 files from a same score (sometime big, poorly optimized) to find out which one is correct enough? The download counter is a poor indicator, since more recent files could have a much higher quality but a lesser download counter. I'm repeating myself: the public quality rating is non-sense (or should only be complementary): nobody knows how to rate conveniently.
I'm aware about the poor quality of some printings. That's why I propose a grade quality standard with a "+" state. A score rated B+, even with original printing artifacts, would indicate (by the "+") that there is no room improving the file. With a score rated raw B, we immediately know that there is room for improving/optimizing the file to get the B+ rating. We also know which score should eventually be rescanned one day (to get from a B-rating to an A-rating). Those information could even be computed to have statistics, and lists of score by quality grade could be displayed. It would allow to have a big picture of where more contribution is needed to improve the existing database. It would dramatically enhance the flow of contributions and would make it much easier to find a score for the user.

By the way, I also propose a special rating, a "first choice"-rating for an unique file per page, which answer to this question: "I'm a casual user, I'm looking for this score, which one should I download first and have a good chance to be satisfied?" If a consensus emerge for an unique file, we should indicate this "first choice" rating. If a contributor posts a new and better version or an already existing piece, he may deserve this "first choice" rating, instead of being at the bottom of the page.

[Goreld]

By the way, I'm French too! So feel free to contact me by PM to discuss about it. I can't PM myself nor indicate my location on my profile, due to my newbie status (I guess).

[Notenschreiber]

I agree with Goreld, our rating system is totally useless. A public notation system will not work, our experience shows this very clearly, so we should stop this nonsense.

[cypressdome]

I would certainly applaud the elimination of our current file rating system but the rating system you propose would probably suffer from the same flaw as the current system--any user (or even limited to registered users) would have the ability to rate any file. To quote myself from a previous thread on this issue:

As for why users give low ratings to obviously high-quality scanned images well you can tell people until you are blue in the face to only rate an item based upon scan quality, or typeset quality, or performance quality and you will be right back to where you started: low ratings will given to high-quality scanned images/typesets/performances. And the reverse is true as well: high ratings for obviously low-quality images/typesets/performances. People will rate them based upon their like/dislike of the composer, the work in question, the edition, the scanner, the typesetter, the submitter, and on and on. Users and their friends will inflate the ratings of performances and new compositions.

What is to stop the composers who currently rate their files with 5 stars from rating their files as A+ under the proposed system? It would be impossible for the admins to police such a rating system. It seems to me that the only way a rating system could work would be for it to be a closed system--files could only be rated by users whose qualifications and judgement have been vetted. But those users (ones with a measurable record on the wiki) are likely the ones who spend their time scanning and uploading scores or performing other valuable work on the wiki. With over 400,000 files on the site and hundreds of files being added daily rating files would consume all of one's time on the wiki.

[Goreld]

To address those problems, there's only 2 things to do:
- remove the one-click rating system
- change the mean calculated rating to an unique consensus rating

Modifying the rating should only be accessible as a page modification (instead of a one-click), by any registered user. It would then be treated as a modification by moderators. Doing so, abuses would be sanctioned, a any modification in a wikipedia page. An unique notation would emerge by consensus, and an average user would be reluctant to change the notation after a moderator, or would be incited to argument his change! There's no need to notate or review all notations for all scores at one time, some could stay without rating or with an unappropriated rating for some time, it isn't a big deal. But the actual public mean-system is a non-sense, as we can see all day.

[Sallen112]

We should probably let Edward take down this system at some point and I have to agree with the admins and Users in this thread entirely about this. Some of my scans with the rating system were rated with low or no ratings at all (while they were high quality scans in the first place!)

We are going to need to find a new solution at some point to rating files that is a much more clearer and a less abused system with a way still to verify ratings more easily across the site (probably we should have a couple admins to deal with this position). Through much discussion, this could happen.

[Goreld]

Who's Edward? IMSLP's owner? Does he read our conversations?