IMSLP Forums

jhellingman wrote:Project Gutenberg has its Distributed Proofreaders project, (http://www.pgdp.net/), which allows people to fix OCR problems, or transcribe hard to OCR pages on line, one-page-at-a-time. We have been experimenting with music, but this has mainly been limited to small excerpts found in various books. For example, I've dealt with music in Carl Lumholz' "Unknown Mexico", a very valuable resource on Mexican ethnography (see http://www.gutenberg.org/etext/16426, the author died in 1922), where music was transcribed to ABC, and added as newly set scores, as well as small midi files.

A similar system could be set up for music scores. At PGDP we can handle several thousand pages per day, with about 5.000 active volunteers. Music is considerably more labor intensive to transcribe, but there is a huge public for it as well, especially if you develop systems where people can initially play the notes. I know it would take quite some effort to build a kind of on-line lilypond system, but the end results would be very nice.

emeraldimp wrote:I've actually considered what this would take before, and I suspect that, with PGDP's current setup, it wouldn't require extraordinary changes to do a Lilypond section (I have done some small research in this area, although it's been quite some time). As a related example, I have seen Lilypond integrated with Mediawiki before.

I think it's workable if: 1) you can get enough volunteers, either from DP or here, to make steady progress; and 2) it's broken into manageable chunks (for example, it used to take me about 2 minutes to P1 a page (and not much more for P2), but it can take me several hours to transcribe a page of music, which would occasionally be needed, even with the best available music OCR software, which might or might not be available).

If you want to continue this discussion, maybe we should break it into a different thread?

jhellingman wrote:It is probably worth a different thread. The PGDP code is freely available, but will require considerable time and skills to set up and running. Integrating with Lilypond will be possible, but require people with skills in the area. The central idea of distributed proofreading is that such tasks are small enough to be done in a lost moment. -- 2 minutes is fine, 10 minutes ok, but if a single page takes of music about an hour, you'll probably need to split the work into smaller units.

Furthermore, since Lilypond requires more learning than just typing text, and is not WYSIWYG (But you get more), things will be slower again.

jhellingman wrote:It is probably worth a different thread. The PGDP code is freely available, but will require considerable time and skills to set up and running. Integrating with Lilypond will be possible, but require people with skills in the area.

I have set up Lilypond to work with Mediawiki before - once, several years ago - so I can probably drag that out of retirement and figure a way to hook it up with pgdp (shall we say lower case refers to the code, upper to the project?). I haven't, however, ever set up pgdp before. Additionally, PGDP has had a LOT of exposure - been slashdotted multiple times, had print articles, etc - so I wonder whether we could get sufficient attention for it to make it worthwhile.

The central idea of distributed proofreading is that such tasks are small enough to be done in a lost moment. -- 2 minutes is fine, 10 minutes ok, but if a single page takes of music about an hour, you'll probably need to split the work into smaller units.

I had been thinking about a single line at a time. Easy enough to proof quickly, and short enough to fix or re-enter if needed. But you have to have someone willing to chop images up and tag them correctly (or be a good enough programmer to do it automatically, which I am not).

Furthermore, since Lilypond requires more learning than just typing text, and is not WYSIWYG (But you get more), things will be slower again.

True, although a simple (java?) WYSIWYG could be included (eventually) for entering, with rendering performed by Lilypond.

I though about this in the past, and really support the idea.

emeraldimp wrote:I had been thinking about a single line at a time. Easy enough to proof quickly, and short enough to fix or re-enter if needed. But you have to have someone willing to chop images up and tag them correctly (or be a good enough programmer to do it automatically, which I am not).

Chopping the images doesn't have to be very difficult using the code of an existing OMR software (optical music recognition) as it can detect staves etc. See audiveris http://audiveris.dev.java.net/ , not very useful yet but looks promising.

emeraldimp wrote:True, although a simple (java?) WYSIWYG could be included (eventually) for entering, with rendering performed by Lilypond.

A browser-based score editor... nice idea though I doubt you'll be able to use lily as a backend (not browser-based, not java).
How about allowing users to download the OMR'd scores, edit them with external software, and upload back to the site?

itayperl wrote:Chopping the images doesn't have to be very difficult using the code of an existing OMR software (optical music recognition) as it can detect staves etc. See audiveris http://audiveris.dev.java.net/, not very useful yet but looks promising.

I wasn't aware of this one; I'll take a look at it!

A browser-based score editor... nice idea though I doubt you'll be able to use lily as a backend (not browser-based, not java).
How about allowing users to download the OMR'd scores, edit them with external software, and upload back to the site?

Well, I wasn't thinking of using lily as a backend to the score editor itself (my intention was a quick-and-dirty get-the-notes(etc)-right-and-send-the-result-to-the-server sort of thing; after the server gets the response, it can use lily to make something nicer).

Editing with external software would, of course, be possible, but it would somewhat defeat (it seems to be) the "I have a minute or two, I'm going to proof/typeset". Not that it couldn't or shouldn't be done, just that it's not quite the same thing.

Editing with external software would, of course, be possible, but it would somewhat defeat (it seems to be) the "I have a minute or two, I'm going to proof/typeset". Not that it couldn't or shouldn't be done, just that it's not quite the same thing.

Ha, I should've been clearer: I'm thinking of something automatic, e.g. a plugin for some existing score editor that will automatically fetch a scanned line and its OMR'd notes, let the user edit, and then upload back to the server, automatically. That way it won't hurt the two minutes concept, which is, I agree, important.

itayperl wrote:Ha, I should've been clearer: I'm thinking of something automatic, e.g. a plugin for some existing score editor that will automatically fetch a scanned line and its OMR'd notes, let the user edit, and then upload back to the server, automatically. That way it won't hurt the two minutes concept, which is, I agree, important.

Hmm, interesting concept! I don't know of any score editors that have a plugin structure robust enough to accommodate that (but I haven't looked at Sibelius' or Finale's in quite some time), although we could certainly integrate it into a free software editor.

I tried Audiveris (on both Linux and Windows), but couldn't get it to work properly, even on the demo. Dunno whether I just don't have everything installed correctly or what.

Project Gutenberg has its Distributed Proofreaders project, (http://www.pgdp.net/), which allows people to fix OCR problems, or transcribe hard to OCR pages on line, one-page-at-a-time. We have been experimenting with music, but this has mainly been limited to small excerpts found in various books. For example, I've dealt with music in Carl Lumholz' "Unknown Mexico", a very valuable resource on Mexican ethnography (see http://www.gutenberg.org/etext/16426, the author died in 1922), where music was transcribed to ABC, and added as newly set scores, as well as small midi files.

I could see this feature being incredibly useful for constructing online translations of all the critical commentary and prefaces connected with the Bach Gesellschaft Edition, complete with the numerous music examples as originally published.

I agree that this may be a very interesting idea for mainly-text books related to music.

The issue with OCRing actual scores is that music OCR is on the order of magnitudes harder than text OCR. A few years ago I did a little test with Finale (2003 I think), where I exported a simple score as a TIFF file, and then ran the OCR provided by Finale over the score. The results were... depressing at best. I hope music OCR has progressed since then, but I have my doubts about whether it may not just be easier to retypeset the score from scratch.

Whereas text OCR for English consists of mostly 26 possible letters and a few punctuation marks, there is almost no limit to the possibilities of music OCR. Imagine OCRing the Macrocosmos by Crumb. Ok, that may be a bit extreme, but I think it does show the possible variety in score types that a music OCR has to cope with. Not to mention that the number of non-note (text or symbol) marks is rather huge... and composers seem to take joy in inventing new ones on a daily basis (I'm not innocent myself)

imslp wrote:The issue with OCRing actual scores is that music OCR is on the order of magnitudes harder than text OCR. A few years ago I did a little test with Finale (2003 I think), where I exported a simple score as a TIFF file, and then ran the OCR provided by Finale over the score. The results were... depressing at best. I hope music OCR has progressed since then, but I have my doubts about whether it may not just be easier to retypeset the score from scratch.

True; one possibility (going in the 'harness the collective 2 minutes(es?) of the masses') would be to forgo the OCR completely and work on retypesetting; I think one could still use parts of the music OCR to automate breaking into staves.

Whereas text OCR for English consists of mostly 26 possible letters and a few punctuation marks, there is almost no limit to the possibilities of music OCR. Imagine OCRing the Macrocosmos by Crumb. Ok, that may be a bit extreme, but I think it does show the possible variety in score types that a music OCR has to cope with. Not to mention that the number of non-note (text or symbol) marks is rather huge... and composers seem to take joy in inventing new ones on a daily basis (I'm not innocent myself)

True, but quite a bit of (western) music is pretty standard, and that's largely what these programs focus on. How high does the accuracy need to be with standard symbols (assume a good scan of a clean imprint... I know, not exactly the norm...) for it to be useful? Especially considering that, even when retypesetting you're bound to make mistakes that need to be corrected.

Either way, if the commentary (etc) of the BGE isn't already done, then I agree PGDP is a great way to do it, though PG has pretty strict requirements of proof before accepting any works.

emeraldimp wrote:True, but quite a bit of (western) music is pretty standard, and that's largely what these programs focus on. How high does the accuracy need to be with standard symbols (assume a good scan of a clean imprint... I know, not exactly the norm...) for it to be useful? Especially considering that, even when retypesetting you're bound to make mistakes that need to be corrected.

Depends on who you ask Bach will tell you that they are not needed at all, whereas someone like Babbitt (Milton) will probably tell you that you cannot mis-position the thing by a pixel (that's an exaggeration of course, but you get what I mean).

Generally speaking, the classical music world is currently... rather particular about the exactness of the positioning and look of symbols/text. This is why scans are, perhaps unfortunately, almost indispensable for some pieces.

imslp wrote:Depends on who you ask Bach will tell you that they are not needed at all, whereas someone like Babbitt (Milton) will probably tell you that you cannot mis-position the thing by a pixel (that's an exaggeration of course, but you get what I mean).

Generally speaking, the classical music world is currently... rather particular about the exactness of the positioning and look of symbols/text. This is why scans are, perhaps unfortunately, almost indispensable for some pieces.

Well... I was referring more to the actual notes (pitch, accents, etc) than positioning & shape. I'd also presume that it's not as big a deal for most of the music that we'd be able to work on; even discounting Bach (who could keep such a project busy for a long time!), there must be thousands (millions?) of works for which such precision is overkill.

Understand, I'm not advocating replacing the scans or discontinuing scanning work; I think they're an important resource even if exact layout is not an issue, if for no other reason than having an authentic (minus the scanning) reference - plus, my thought was to use IMSLP as the source.

Actually, Mutopia has been doing something similar I believe I remember seeing several scores on Mutopia that has "IMSLP" as the source.

imslp wrote:Actually, Mutopia has been doing something similar I believe I remember seeing several scores on Mutopia that has "IMSLP" as the source.

True, and I highly support and encourage Mutopia, but it has (IIRC) two main differences:

First, they restrict themselves to pre-1923 AND life+70 (unless, of course, the copyright holder submits it).

Second, the workflow of Mutopia is a single 'typesetter' submitting a completed project, not a distributed group working on the same project. Again, similar to Project Gutenberg before DP.

Also, the project wouldn't have to use the Lilypond format as its storage/output. It could use something else - musicXml, for example - and translate into lilypond for rendering during the proofing, and provide musicXml + ly + whatever as the ultimate output.

...And I have more than one score on Mutopia myself (not many, though).

I'm actually currently testing the multi-file template. When IMSLP comes back online again, I see no reason why we cannot have both the scan and typeset They won't take up much space (layout-wise) in any case.

About being "distributed", this may present itself to be the most problematic element. Normally, I would suggest no parallization unless absolutely necessary, for performance reasons. This is because, as anyone who has written multi-threading applications will surely relate to, negotiation between different people takes a rather large toll on the efficiency of the project if the parallization is not done correctly (which, according to Moore's law, is always the case haha). The best case scenario is if one person (and only one person) is in charge of an independent chunk (a piece, or even a movement).

This is of course slightly different with text, as text is considerably more straightforward. On the other hand, connecting different pages of music is not a trivial task (and to make it look good!).

To prevent stalled projects, we can have some sort of transfer function which will transfer incomplete typesets to another person who will work on them.

In any case, I would very much welcome it if someone can explore this idea more. I myself would be unable to lead such a project, but I, of course, welcome anyone who is interested to do so, and I will support it.