Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hysteria2 masquerade options updated to let modern websites load correctly #2297

Open
wants to merge 36 commits into
base: dev-next
Choose a base branch
from
Open

hysteria2 masquerade options updated to let modern websites load correctly #2297

wants to merge 36 commits into from

Conversation

doggy
Copy link

@doggy doggy commented Nov 20, 2024

Removed r.Out.Host = r.In.Host to resolve an issue where the proxy was incorrectly using the client's host header instead of the target's host. This ensures the proxy request aligns with the intended masquerade behavior.

API Document

https://pkg.go.dev/net/http/httputil#ProxyRequest.SetURL

SetURL rewrites the outbound Host header to match the target's host.

Currently the code line r.Out.Host = r.In.Host is used for preserve the inbound request's Host header which is sing-box server's host name in our situation.

Sample Code

A go proxy sample here for occurring this bug:

go version
go version go1.23.3 darwin/arm64

cat proxy.go


package main

import (
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
)

func main() {
	masqueradeURL, _ := url.Parse("https://bing.com")
	masqueradeHandler := &httputil.ReverseProxy{
		Rewrite: func(r *httputil.ProxyRequest) {
			r.SetURL(masqueradeURL)
			r.Out.Host = r.In.Host
		},
		ErrorHandler: func(w http.ResponseWriter, r *http.Request, err error) {
			log.Printf("Proxy error: %v", err)
			w.WriteHeader(http.StatusBadGateway)
		},
	}

	http.Handle("/", masqueradeHandler)
	log.Println("Starting reverse proxy...")
	log.Fatal(http.ListenAndServe(":8080", nil))
}

The bad result:

➜  ~ curl -v http://localhost:8080
* Host localhost:8080 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8080...
* Connected to localhost (::1) port 8080
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 400 Bad Request
< Date: Wed, 20 Nov 2024 16:46:16 GMT
< X-Msedge-Ref: 0WRI+ZwAAAADVvLX68eHdSrZSJ6s7q3ALQlkzRURHRTA0MTAARWRnZQ==
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
< 
* Connection #0 to host localhost left intact
<h2>Our services aren't available right now</h2><p>We're working to restore all services as soon as possible. Please check back soon.</p>

After removed r.Out.Host = r.In.Host, it seems works properly.

➜  ~ curl -v http://localhost:8080
* Host localhost:8080 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8080...
* Connected to localhost (::1) port 8080
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 301 Moved Permanently
< Accept-Ch: Sec-CH-UA-Arch, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version, Sec-CH-UA-Full-Version-List, Sec-CH-UA-Mobile, Sec-CH-UA-Model, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version
< Cache-Control: private
< Content-Security-Policy: script-src https: 'strict-dynamic' 'report-sample' 'wasm-unsafe-eval' 'nonce-YdgXJAbl+5fxQkfvbJK4CuovCM+Hg+zVVIFsZe5oN8w='; base-uri 'self';
< Content-Type: text/html; charset=utf-8
< Date: Wed, 20 Nov 2024 16:46:43 GMT
< Location: https://www.bing.com:443/?toWww=1&redig=37A991C2208A4FEDBBF54E5BE47699F0
< Set-Cookie: MUID=3989612CA23B6FF82A2A7411A3776E11; domain=bing.com; expires=Mon, 15-Dec-2025 16:46:43 GMT; path=/; secure; SameSite=None
< Set-Cookie: MUIDB=3989612CA23B6FF82A2A7411A3776E11; expires=Mon, 15-Dec-2025 16:46:43 GMT; path=/; HttpOnly
< Set-Cookie: _EDGE_S=F=1&SID=1F12B189E4FB631B15E3A4B4E5B76278; domain=bing.com; path=/; HttpOnly
< Set-Cookie: _EDGE_V=1; domain=bing.com; expires=Mon, 15-Dec-2025 16:46:43 GMT; path=/; HttpOnly
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< Useragentreductionoptout: A7kgTC5xdZ2WIVGZEfb1hUoNuvjzOZX3VIV/BA6C18kQOOF50Q0D3oWoAm49k3BQImkujKILc7JmPysWk3CSjwUAAACMeyJvcmlnaW4iOiJodHRwczovL3d3dy5iaW5nLmNvbTo0NDMiLCJmZWF0dXJlIjoiU2VuZEZ1bGxVc2VyQWdlbnRBZnRlclJlZHVjdGlvbiIsImV4cGlyeSI6MTY4NDg4NjM5OSwiaXNTdWJkb21haW4iOnRydWUsImlzVGhpcmRQYXJ0eSI6dHJ1ZX0=
< Vary: Accept-Encoding
< X-Cache: CONFIG_NOCACHE
< X-Eventid: 673e12738d1d4715835655e99b215794
< X-Msedge-Ref: Ref A: 14012417CC364045A4F102B297BC103A Ref B: BY3EDGE0408 Ref C: 2024-11-20T16:46:43Z
< Transfer-Encoding: chunked
< 
<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="https://www.bing.com:443/?toWww=1&amp;redig=37A991C2208A4FEDBBF54E5BE47699F0">here</a>.</h2>
</body></html>
* Connection #0 to host localhost left intact

@nekohasekai nekohasekai force-pushed the dev-next branch 4 times, most recently from 9443c94 to 013a69c Compare November 21, 2024 10:29
@nekohasekai nekohasekai force-pushed the dev-next branch 3 times, most recently from 3c5c2b6 to fd806af Compare November 22, 2024 06:54
@nekohasekai
Copy link
Member

Maybe we need to add a new option for this, since this is the default behavior of hysteria2: https://github.com/apernet/hysteria/blob/15e31d48a09af0835773bd5c62cedbb3fc0e351d/app/cmd/server.go#L815

@nekohasekai nekohasekai force-pushed the dev-next branch 3 times, most recently from 940d78e to d3d7664 Compare November 23, 2024 05:31
@doggy doggy force-pushed the dev-next branch 2 times, most recently from 29fb3f8 to d5162dc Compare November 23, 2024 09:21
@doggy
Copy link
Author

doggy commented Nov 23, 2024

Maybe we need to add a new option for this, since this is the default behavior of hysteria2: https://github.com/apernet/hysteria/blob/15e31d48a09af0835773bd5c62cedbb3fc0e351d/app/cmd/server.go#L815

Thank you for mentioning this code.
It’s weird that the default settings for HTTPS masquerade do not work with the most popular websites like bing, yelp..

This PR updates the masquerade option style to include support for rewriteHost and making it more similar to Hysteria2.
It also retains support for the previous Sing-Box style for better understanding among existing users.

@doggy
Copy link
Author

doggy commented Nov 23, 2024

Docs and unit test updated also

@doggy doggy changed the title Fix hysteria2 masquerade bug: Ensure proxy request uses target host hysteria2 masquerade options updated for chosen proxy target host Nov 24, 2024
@nekohasekai nekohasekai force-pushed the dev-next branch 9 times, most recently from 9993502 to 3613e2a Compare November 24, 2024 11:51
@doggy
Copy link
Author

doggy commented Nov 29, 2024

It seems my pull request was refactored in this commit.

I’m glad to see that!

@nekohasekai nekohasekai force-pushed the dev-next branch 7 times, most recently from 1aa1333 to 2d4c4e2 Compare November 30, 2024 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@doggy @nekohasekai