[fpc-devel] fphttpclient cannot download a file from w3.org

[Ondrej Pokorny]

Thank you Michael for the response. I'll check what headers INDY sends.

Ondrej


On 01.07.2019 14:57, Michael Van Canneyt wrote:
>
> I tested:
>
> If I run a strace, it just hangs on the read operation:
>
> connect(3, {sa_family=AF_INET, sin_port=htons(80), 
> sin_addr=inet_addr("128.30.52.100")}, 16) = -1 EINPROGRESS (Operation 
> now in progress)
> select(4, NULL, [3], NULL, {tv_sec=3, tv_usec=0}) = 1 (out [3], left 
> {tv_sec=2, tv_usec=884913})
> getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
> fcntl(3, F_GETFL)                       = 0x802 (flags O_RDWR|O_NONBLOCK)
> fcntl(3, F_SETFL, O_RDWR)               = 0
> sendto(3, "GET /TR/2002/REC-xmldsig-core-20"..., 112, 0, NULL, 0) = 112
> recvfrom(3,
>
> If I do the same with 'wget' it also hangs but in the end gets through..
>
> But if you add some headers, you get a proper error response.
>
> There is little I can do about that. I can only surmise Indy & wget 
> send different
> headers (E.G.user-agent) and the server reacts differently on that.
>
>
>
> Michael.
>
> On Mon, 1 Jul 2019, Ondrej Pokorny wrote:
>
>> Hello,
>>
>> I cannot download 
>> http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/xmldsig-core-schema.xsd 
>> with TFPHTTPClient. Delphi INDY downloads it just fine. Does anybody 
>> know what I am missing?
>>
>> Test code:
>>
>> program Project1;
>> uses Classes, SysUtils, fphttpclient;
>> procedure DownloadFile(const aURI: string; out outStream: TStream);
>> var
>>   xClient: TFPHTTPClient;
>> begin
>>   outStream := nil;
>>   xClient := TFPHTTPClient.Create(nil);
>>   try
>>     xClient.AllowRedirect := True;
>>     outStream := TMemoryStream.Create;
>>     xClient.Get(aURI, outStream);
>>     outStream.Position := 0;
>>   except
>>     on E: Exception do
>>     begin
>>       FreeAndNil(outStream);
>>       Writeln(E.ClassName);
>>       Writeln(E.Message);
>>     end;
>>   end;
>>   xClient.Free;
>> end;
>> var
>>   S: TStream;
>> begin
>> DownloadFile('http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/xmldsig-core-schema.xsd', 
>> S);
>>   S.Free;
>>   Readln;
>> end.
>>
>> Fails with exception:
>>
>> EHTTPClient
>> Error reading data from socket
>>
>> #0 fpc_raiseexception(0x145fb14, 0x0, 0xffffffff) at 
>> ..\inc\except.inc:158
>> #1 FILLBUFFER(0x145fb14) at fcl-web\src\base\fphttpclient.pp:731
>> #2 READSTRING(0x15e1c30, 0x0) at fcl-web\src\base\fphttpclient.pp:749
>> #3 READRESPONSEHEADERS(0x15e1c30) at 
>> fcl-web\src\base\fphttpclient.pp:854
>> #4 READRESPONSE(0x15e1c30, 0x15b5dc0, 0x145fdb8, 0, false) at 
>> fcl-web\src\base\fphttpclient.pp:1132
>> #5 DONORMALREQUEST(0x15e1c30, {PROTOCOL = 0x15b5e4c 'http', USERNAME 
>> = 0x0, PASSWORD = 0x0, HOST = 0x15b5ecc 'www.w3.org', PORT = 0, PATH 
>> = 0x1601c9c '/TR/2002/REC-xmldsig-core'..., DOCUMENT = 0x15bc58c 
>> 'xmldsig-core-schema.xsd', PARAMS = 0x0, BOOKMARK = 0x0, HASAUTHORITY 
>> = true}, 0x449d58 'GET', 0x15b5dc0, 0x145fdb8, 0, false, false) at 
>> fcl-web\src\base\fphttpclient.pp:1226
>> #6 DOMETHOD(0x15e1c30, 0x449d58 'GET', 0x443060 
>> 'http://www.w3.org/TR/2002'..., 0x15b5dc0, 0x145fdb8, 0) at 
>> fcl-web\src\base\fphttpclient.pp:1283
>> #7 HTTPMETHOD(0x15e1c30, 0x449d58 'GET', 0x443060 
>> 'http://www.w3.org/TR/2002'..., 0x15b5dc0, 0x145fdb8, 0) at 
>> fcl-web\src\base\fphttpclient.pp:1397
>> #8 GET(0x15e1c30, 0x443060 'http://www.w3.org/TR/2002'..., 0x15b5dc0) 
>> at fcl-web\src\base\fphttpclient.pp:1439
>> #9 DOWNLOADFILE(0x443060 'http://www.w3.org/TR/2002'..., 0x15b5dc0) 
>> at project1.lpr:14
>> #10 main at project1.lpr:30
>>
>> Ondrej
>>
>> _______________________________________________
>> fpc-devel maillist  -  fpc-devel at lists.freepascal.org
>> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
>>
>
> _______________________________________________
> fpc-devel maillist  -  fpc-devel at lists.freepascal.org
> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freepascal.org/pipermail/fpc-devel/attachments/20190701/d20c4442/attachment-0001.html>