IMSLP Forums

I've completed, after many uploads, the transfer of all files by Simone Stella and Fritz Brodersen.
I've tried to check if the automatic redirects from WIMA to IMSLP works with all files... but it doesn't!!
Some examples?
For Brodersen:

• certon pierre - Je ne l'ose dire] - file #1,2,4,5
• conseil jean - adiuva me domine] - all files
• Handel HWV314 2./3. and HWV 315 1.] complete score not linked
• and many others

It seems to me a random behaviour. It's surely not but....What is the motivation? What have I done wrong?

You have reached this message because the site ripping ban script has been triggered. Site ripping is forbidden; repeated offenders will be banned indefinitely ...

imslp wrote:Is the uploads not registering in wimaredirects.txt?

Choralia wrote:

imslp wrote:Is the uploads not registering in wimaredirects.txt?

I think this is the case. I looked for the "Cantus" file of the "Credo" of "Missa da pacem", as suggested by Reccmo. Specifically, I searched inside wimaredirects.txt for the WIMA filename "3_2_Credo_Cantus.pdf", with no success. So it seems that this upolad was not registered in wimaredirects.txt for some reason. I also searched for it inside the .htaccess file on the WIMA website, and obviously it was not present there either.
Max

You have reached this message because the site ripping ban script has been triggered. Site ripping is forbidden; repeated offenders will be banned indefinitely.

$infile = "wimaredirects.txt";
$outfile = "wimaredirect_errs.txt";

open (INFILE, "<$infile") or die "Can't open $infile\n";
open (OUTFILE, ">$outfile") or die "Can't open $outfile\n";

$i = 0;
while (<INFILE>) {
  $i++;
  chomp;
  ($skip,$wima,$imslp) = split;
  @args = ("curl","$imslp");
  system(@args) == 0
    or print "$wima\n$imslp\n$?\n\n";
  last if ($i > 10);
}

imslp wrote:Regarding entries missing from wimaredirects.txt, I have found a bug where, if the submitter makes a mistake during submission and the submission page shows an error on top, wimaredirects.txt will not be updated even if subsequent submission is successful. This bug is fixable but not easily, so I'm thinking of writing an alternate redirect function that can take entire WIMA urls and redirect them to the right place (if submitted; if not it bounces back to WIMA).

Regarding 404 errors, please provide me a sample (or few) of such URLs so that I can figure out what is wrong.

imslp wrote:Regarding the script, I would rather prefer to resolve the problem using other methods. The site ripping ban script is there precisely to prevent such bot access to IMSLP so that the server does not get overloaded for everyone else.

reccmo wrote:'Redirect /scores/haendel/H312/satz_1-02_Oboe-1.pdf http://imslp.org/wiki/Special:ReverseLookup/136565'

'Redirect /scores/c.raehs/Facsimiles/XM55/13.pdf http://imslp.org/wiki/Special:ReverseLookup/141574'

imslp wrote:You can now use http://imslp.org/index.php?title=Specia ... <urlstring>

<urlstring> is the same as the WIMA url in wimaredirects.txt (i.e. without http://icking-music-archive.org)

reccmo wrote:Max has helped me substantially with providing php code for handling the new generic IMSLP redirect on the WIMA server.

The main challenge was to prevent infinite Apache loops for WIMA URLS bounced back to the WIMA server. I believe it's working now and have replaced the WIMA redirects based on IMSLP's upload log by the new generic IMSLP redirect. On the WIMA server a simple .htaccess file in the score root folder and a php redirect handler in the document root folder are involved.

I encourage forum readers to test the redirects of WIMA files, transferred as well as not transferred. Please report any errors encountered.

File #1 Error: Error: Failure #47 on CURL for 'http://icking-music-archive.org/scores/ ... torale.pdf'

Error 47 for CURL is "too many redirects":

CURLE_TOO_MANY_REDIRECTS (47)

Too many redirects. When following redirects, libcurl hit the maximum amount. Set your limit with CURLOPT_MAXREDIRS.

I'm not sure that the problem is due to the downloading being managed by php "per se", as it is essentially transparent to the browser. I think that there might be two possible causes:

1) a recursion issue similar to the one that we solved yesterday, but seen from the IMSLP side rather than from the WIMA side;

2) the php downloading being fragmented into small chunks of 2048 bytes. If CURL accounts a redirection for each chunk, it is easy to hit the CURL limit.

Cause 1) should be possibly fixed by Feldmahler, so let's investigate case 2) first. I would suggest to change this instruction on Redirect.php:

$buffer = fread($fd, 2048);

using a much larger value than 2048, so that the file is not fragmented, or fragment in very few chunks. To the limit, one can use:

$buffer = fread($fd, $fsize);

That should always result in a single file chunk.

If the above modification solves the issue, we may conclude that cause 2) applies. If it doesn't, I guess we will have to investigate cause 1) with Feldmahler.

      header("Content-type: ".$content_type);
      header("Content-Disposition: inline; filename=\"".$path_parts["basename"]."\""); 
      header("Content-length: $fsize");
      header("Cache-control: private"); 
      while(!feof($fd)) {
        $buffer = fread($fd, 2048);
        echo $buffer;
      }			  
      fclose ($fd);