Using Photoscore to convert IMSLP's PDF files to notation

[Starrmark]

I have made many attempts to convert IMSLP's PDF files of scanned music into SIbelius SIB files using Photoscore 6. The results have been mostly poor, especially with must of any complexity. Photoscore requires scans in 8-bit greyscale, not in 1-bit per pixelo B&W. In a discussion about the problems of using Photoscore on pdfs from the Petrucci Mujsic Library, Neurotron's tech support wrote:

"The format recommended by the Petrucci Music library is 600dpi monochrome (B&W, 1 bit per pixel), which is a shame. They do allow grey-scale submissions but don't encourage them."

Is there any particular reason why IMSLP recommends 1-bit B&W scans, and not 600 dpi 16->8 bit greyscale scans? By recommending 1-bit B&W scans, IMSLP is impeding users from converting its pdf files to musical notation on programs such as SIbelius and Finale, using music optical recognition programs, such as Photoscore and others.

MS

[Carolus]

The main reason is file size, which covers two aspects: 1) most of the world is still on dialup, so large files can be a real problem; 2) storage space, which starts to add up when you've got an ever-expanding archive (now at 41,000 files). Large file sizes also end up requiring more bandwidth.

How well does Photoscore 6 work with grayscales in the real world? One of the tech-support people for Sharp-Eye I talked with mentioned that a real problem music-OCR software has in dealing with hand-produced music is that the staves, for example, aren't perfectly evenly spaced (despite appearing OK to the naked eye). The same problem applies with positions of noteheads and other symbols, etc. Thus, the best result I was ever able to obtain about 2 years ago was about a 60% accuracy - for dealing with a very run-of-the-mill item like a Breitkopf vocal score for a Beethoven Mass. While better than nothing, the fix-up time alone probably exceeds the cost of having the whole thing put into Sibelius or Finale fresh.

[daphnis]

Even using their recommended settings, the chances of a mildly successful import using both the product you mentioned and at least 2 other major players I use is hand engraving, in that engraved scores, for the reasons Carolus mentioned et al, are wildly variant. In my experience, anything OCR'ed from such a score other than Haydn, Czerny, Mozart etc. meets with only the slimmest chances of success, and more often than not it is almost faster to input the music by hand (provided you're already a seasoned veteran with your notation package of choice). Keep in mind, also, that a scan, in whatever format, can only be, at maximum, as good as the source, so a 1940's reprint of a score originally published in 1880 that uses anything other than quarter notes and eighth notes is going to end in almost certain failure.

EDIT: I'll also add that producing scans in 8-bit greyscale (or 256 colors) would potentially dramatically reduce the overall quality of the scans since those thresholds will also pick up "salt and pepper" artifacts from aged and acidic paper (a common medium for many of our contributors, myself namely), pencil marks, and stains. It would also mean much more complex and time-consuming post-processing steps after the fact.

[horndude77]

"The format recommended by the Petrucci Music library is 600dpi monochrome (B&W, 1 bit per pixel), which is a shame. They do allow grey-scale submissions but don't encourage them."

For the quality and file size 600dpi b&w is hard to beat. For music (and text) grayscale really doesn't buy you that much at higher resolutions. In fact in some cases it can even look worse with out some smoothing or other cleaning.

OMR is a much harder problem than OCR (overlapping/connected glyphs, parallel voices, etc...) and people seem to be able to read 600dpi b&w sheets just fine. I'd say the software makers and the state of OMR research (which unfortunately isn't a particularly hot topic these days) are impeding the automatic conversion more than IMSLP's recommended image storage.

[Starrmark]

I found some of the responses interesting, but others begged the question.

I would be the last person in the world to defend the accuracy of OMR programs such as Photoscore. It is a very frustrating program to use. All of these programs make wildly extravagant claims about accuracy which do not reflect my actual use. However, one has to add: judging from the lastest upgrade of Photoscore (version 6), OMR is getting better. And, there can be no doubt these programs will continue to get better over time. There is no reason that someday, any musical notation that can be read by a human being -- including jagged notes, broken staff lines, notes partially covered by stains on the paper, etc. -- will be read corfrectly by OMR. These programs will access huge databases of musical rules and data that will correct many of the errors in recognition, much the same as OCR now corrects mistakes in spelling, grammar, names and even some facts. There are a lot of very smart people working in this field and they are not going to let up until OMR lives up to its possibilities. The essential point, it seems to me, is that IMSLP's 41,000+ scanned scores (and growing) will be around forever. I doubt many will be rescanned in 8-bit greyscale. When OCM eventually progresses, these scores, scanned at 1-bit B&W will be unusable for OMR. Scores in notation are an invaluable adjunct to scans of the original editions. If the scores are not in 8-bit greyscale, no one will be able to convert them to notation with OMR.

MS

[daphnis]

You're then implying that as technology of OMR advances its requirements remain static, and that's not correct. 1-bit scanned at 600dpi is very usable for OMR/OCR as I attempt it myself often, and furthermore, if Neuratron is making an official suggestion that their software works best in 256 colors then I know others that suggest 1-bit. The point being that as advances are made they will be made in areas of complexity and compatibility. And in actuality, the majority of visitors and users of this site do not view/download with the intention of strictly OMR'ing the scores.

[Starrmark]

Neuratron did not say their software works best with scans in 256 colors. It says Photoscore requires an image in 8-bit greyscale.

Nor did I suggest that that the majority of visitors and users of this site view/download with the intention of strictly OMR'ing the score. I don't see the relevance of the majority. 8-bit greyscale scans do not exclude the majority.

MS

[kalliwoda]

The real problem with OCR programs is that they require different optimization of a scan from each other and from the human eye, i.e. the musician who wants to use the music for performance.
I am only familiar with Smartscore (10.1), which ties into Finale in its "light" version. Smartscore works best with a resolution of not more than 300 dpi (monochrome, I believe), but scanned darker than pleasing to the human reader.

If you figure in, on top of this problem, that anything but the music notation is still almost impossible to convert correctly, I think IMSLP should stick to its current recommendations.

In my own experience, "Urtext" editions of very simple complexity like Gabrieli or Bach Triosonatas can be converted with less than one wrong note per page, but as soon as articulations, expressions etc are added as in most music from the late 18th century onwards, accuracy drops substantially (pretty much to the 60% daphnis mentions) - don't even think of cross-staff notes or X-toles other than 3's!
Still a very long way to go before OCR can be used to quickly produce parts from a romantic symphony score.

[w_m0zart]

In the Neuratron Manual from version 6, on page 56 they say:

The file saved will be 2 color black & white and be identical to the image you see on screen, minus stave markings and at 100% zoom. Please note that if you attempt to reopen this file in PhotoScore, it may complain about it only being in two colors – do not worry about this as PhotoScore has already processed the file to make it more suitable for reading music from.

The way I understand this, is that once Neuratron has processed a file containing grayscale scans into black & white, it is perfectly fine to save that image.
Opening these black & white images once they had been saved doesn't influence the reading results. (Of course, for now that applies only to Neuratron v6!). I see this as a the best way to both save music on my hard disc and still being able to edit certain music in the future if there would be a necessity for that. I think that most reading errors which appear still now and then can be solved by implementing better interpretation rules and not from better scanmaterial.

[Starrmark]

The resolution of scans on IMSLP is not a problem for Photoscore. If a pdf has been scanned let's say at 600 dpi, a resolution dialogue will pop up when that pdf is fed into Photoscore. The image will be converted to a lower, appropriate resolution in Photoscore. What Photoscore cannot do (nor can any program) in increase the resolution of a scan -- but that is not the issue here. The only relevant issue is bit depth.

Here is some additional information from Neurotron's tech support:

"It is important to realise, in the case of a PDF from a scan, or of a TIFF image, that all the decisions on resolution and colour depth have already been taken at the time of the scan, and frozen into the image file, and there is nothing that PhotoScore can do to recover information from a scan of inadequate quality. PhotoScore needs a grey-scale image (8 bits per pixel) and not black & white (1 bit per pixel), at a resolution appropriate to the staff size (see the manual). Looking around the web at well-known music library archive sites, I see that most music archive scans are offered at one-bit-per-pixel resolution, which is just not good enough for OMR (optical music recognition), even if it looks fine to the naked eye (Adobe Reader does a very good job at anti-aliasing its display so that the jaggies are not evident to the eye)...When reading PDFs of scanned images, the PhotoScore resolution dialog cannot be used to recover information which is just not present in the scan: the scan will already be a bit-map of fixed resolution. For successful use of PhotoScore on image files, the PhotoScore requirements on colour-depth (8-bit grey-scale) and resolution (pixels per inch) have to have been known and used at the time of originally scanning the image."

Also, it should be pointed out for what purpose the additional information in an 8-bit greyscale image is used in Photoscore. It is used to straighten each staff to line up precisely with a line of pixels. Neurotron's tech support states:

"Given an 8-bit-per-pixel input, PhotoScore can use the extra grey-level information to help straighten the image with a small rotation, without loss of much information, before the reduction to 1-bit. PhotoScore cannot straighten 1-bit per pixel input images. So, if your 1-bit-per-pixel images were already straightened before being reduced to 1-bit, so that a staff line runs along a line of pixels, then there is no need to worry about the PhotoScore input being only 1-bit per pixel."

Therefore, if pdf scans on IMSLP were perfectly aligned with a horizontal row of pixels, 1-bit B&W would be acceptable on Photoscore. But such perfect alignment while scanning is practally unattainable.

MS

[imslp]

I will just hazard that this is absolutely a problem with Photoscore and not IMSLP. What they are asking IMSLP to do is to conform to the standard best suited to their software, even after conceding that the current standard on IMSLP is perfectly readable (and in my opinion much better than grayscale, most certainly with regards to printing) to the naked eye. If it is as readable to the naked eye as grayscale, it is a lack of OMR technology that OMR cannot process it well, not a defect on the scanner's part.

What they have not considered on IMSLP's side (and they have no reason to from their perspective), is that monochrome is vastly smaller than grayscale at the same readability. The reason I strongly opposed grayscale scans was because some of them were taking up to 1-3MB/page JPG (lossy) compression, whereas similarly readable monochrome scans only take up 100-250KB/page (250KB w/ CCITT Group4 lossless compression for 600dpi A4 I believe, though admittedly I do not have much experience with this).

I know my recommendation will not be music to the ears of the guys at Photoscore, but objectively I believe that software should solve human problems, not the other way around as they argue (and understandably given the difficulty of fundamentally changing massive software like OMR). At some point they will need to address this issue I believe.

P.S. Antialiasing is not an argument, because if that is all that is necessary, Photoscore could just apply it regardless of whether Acrobat does so or not.

[daphnis]

Agreed.

[w_m0zart]

Please note, that Neuratron is not per se favouring grayscale images, but they only state, that it helps Neuratron's preprocessing features like cleaning up pages and leveling staves. Once pages have been cleaned up and staves leveled, the program uses black & white images for optical music recognition (omr).

The software which I wrote to batch convert raw scans to small pdf files will add some blur to black & white images before sending it to Neuratron to force it to level staves and to reduce jagged lines.