SSH remote forwarding results in 404 Not Found

Question

On a remote server, remote1.myorg.io, I am unable to access a web resource, say, http://spaa.acm.org/2020/SPAA2020TutorialProgram.pdf. So, on the remote server, I'm using ssh to create a server on 127.0.0.0 and access the resource thru there, by forwarding the request back to my local machine (and have my local machine fetch the resource). This is what I'm doing on my local machine: ssh -R 6050:spaa.acm.org:80 remote1.myorg.io Then, on remote1.myorg.io, I do curl -v http://localhost:6050/2020/SPAA2020TutorialProgram.pdf However, I get * About to connect() to localhost port 6050 (#0) * Trying ::1... * Connected to localhost (::1) port 6050 (#0) > GET /2020/SPAA2020TutorialProgram.pdf HTTP/1.1 > User-Agent: curl/7.29.0 > Host: localhost:6050 > Accept: */* > < HTTP/1.1 404 Not Found < Connection: Keep-Alive < Content-Type: text/html < Transfer-Encoding: chunked < Date: Sun, 15 Nov 2020 00:41:45 GMT < Server: LiteSpeed < What am I doing wrong?

Gordon Davisson · Accepted Answer

The problem: The spaa.acm.org server, like many web servers, does something like name-based virtual hosting. That is, it inspects the Host header of incoming requests, and can serve different content depending on what hostname the client says it's trying to reach. If you look at the verbose output from curl, it includes this header:

> Host: localhost:6050

...but I'm pretty sure that server doesn't recognize "localhost" as a domain name it should be serving, so you get problems.

Solution 1: You can tell curl to fetch the normal URL (including the "spaa.acm.org" domain), but to proxy the connection via localhost:6050:

curl -v -x localhost:6050 http://spaa.acm.org/2020/SPAA2020TutorialProgram.pdf -O
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 6050 (#0)
> GET http://spaa.acm.org/2020/SPAA2020TutorialProgram.pdf HTTP/1.1
> Host: spaa.acm.org
> User-Agent: curl/7.54.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Content-Type: application/pdf
< Last-Modified: Sat, 11 Jul 2020 07:33:05 GMT
< Etag: "168eb-5f096b31-3718a48ea4a1f404;;;"
< Accept-Ranges: bytes
< Content-Length: 92395
< Date: Sun, 15 Nov 2020 01:24:27 GMT
< Server: LiteSpeed
<

(Note the Host: spaa.acm.org header.)

Solution 2: Explicitly override the Host header:

curl -v -H "Host: spaa.acm.org" http://localhost:6050/2020/SPAA2020TutorialProgram.pdf -O

I won't include the debug output on this one, suffice to say that it also works.

SSH remote forwarding results in 404 Not Found

One Answer

Add your own answers!

Ask a Question