Mitmproxy 11: Full HTTP/3 support

We are pleased to announce the release of mitmproxy 11, which introduces full support for HTTP/3 in both transparent and reverse proxy modes. We’re also making a bunch of DNS improvements that we discuss in this blog post.

Editorial note:

Hi! I’m Gaurav Jain, one of the students selected for this year’s Google Summer of Code program to work on mitmproxy. This summer I worked on improving several low-level networking components of mitmproxy, some of which included HTTP/3 and DNS. You can find my project report here.

HTTP/3

HTTP/3 now works “normally” for reverse proxies. Your mitmproxy instance listens to both TCP and UDP packets and processes any HTTP versions generated to it:

$ mitmproxy --mode reverse:https://http3.is

Our transparent proxy modes now all support HTTP/3:

$ mitmproxy --mode wireguard
$ mitmproxy --mode local
$ mitmproxy --mode transparent

We successfully tested HTTP/3 support with Firefox, Chrome, various cURL builds, and other clients to resolve compatibility issues. The only major limitation we’re aware of at this time is that Chrome doesn’t trust user-added certification authorities for QUIC. This means you’ll either need to provide a publicly trusted certificate (e.g. from Let’s Encrypt), launch Chrome with a command-line switch, or accept falling back to HTTP/2. Alternatively, Firefox doesn’t do these kinds of tricks. For more tips on troubleshooting HTTP/3 issues, see #7025.

Providing HTTP/3 support to mitmproxy is a major effort started in 2022 by Manuel Meitinger and Maximilian Hils. QUIC and HTTP/3 are making up an increasing share of network traffic in the wild, and we’re super excited that it’s now ready and enabled by default!

Improved DNS support

With the advent of DNS HTTPS records and new privacy enhancements such as Encrypted Client Hello (ECH), mitmproxy’s DNS functionality is becoming increasingly important. We would like to share several developments on this front:

Support for query types beyond A/AAAA

mitmproxy’s old DNS implementation getaddrinfo to resolve questions. Handy because everything is arranged by libc, but the getaddrinfo API only supports A/AAAA queries for IPv4 and IPv6 addresses. It does not allow us to answer queries for, for example, HTTPS records, which are used to signal HTTP/3 support.

To overcome this limitation, we redeployed our DNS support on top of Hickory DNS, a Rust-based DNS library. Using Hickory, we now obtain the default operating system name servers on Windows, Linux, and macOS and forward non-A/AAAA queries there. This behavior can also be adjusted with the new one dns_name_servers option:

$ mitmdump --mode dns --set dns_name_servers=8.8.8.8

Skip /etc/hosts

By switching to Hickory we now also have the option to ignore the system’s hosts file (/etc/hosts on Linux) with the new one dns_use_hosts_file option. We plan to move mitmproxy’s internal DNS resolution to Hickory as well, at which point this feature will become incredibly useful in enabling transparent same-machine redirection for specific domains. Currently, such a setup would cause mitmproxy to connect to itself recursively, because we always take the hosts file into account.

$ echo "192.0.2.1 mitmproxy.org" >> /etc/hosts

$ mitmdump --mode dns
$ dig @127.0.0.1 +short mitmproxy.org
192.0.2.1

$ mitmdump --mode dns --set dns_use_hosts_file=false
$ dig @127.0.0.1 +short mitmproxy.org
3.161.82.13

Support for DNS-over-TCP

DNS uses UDP by default, but can also use TCP to support records that do not fit into a single UDP packet. mitmproxy has previously gotten away with only supporting UDP, but now that we support arbitrary query types, message size and thus TCP support is more important. Long story short: DNS-over-TCP works with mitmproxy 11!

Encrypted Client Hello (ECH) keys are deleted

Unless a custom certificate is configured, mitmproxy uses the Server Name Indication (SNI) sent in the TLS ClientHello to construct a valid certificate. Conversely, if no SNI is present, we may not be able to generate a certificate that is trusted by the client.

Encrypted Client Hello (ECH) is an exciting new technology to increase privacy on the Internet. Basically, the client uses the new DNS HTTPS records to obtain an ECH key before establishing a connection, and then encrypts all the initial ClientHello handshake message with that key. If both DNS queries and handshake are encrypted, passive intermediaries cannot learn the target domain, only the target IP address (which is not critical for shared hosting and Content Delivery Networks). This is a major advance for privacy, but also breaks the way mitmproxy generates certificates. To resolve this, mitmproxy now removes ECH keys from HTTPS records. This way, the client has no keys to encrypt the initial handshake message with, and mitmproxy still learns the target domain and can construct an associated certificate.

Of course, ECH adds complexity for us and sometimes makes mitmproxy more difficult to use for our users. Nevertheless, we’re excited to see these privacy improvements rolling out to the rest of the Internet!

Acknowledgments

This work is supported by Google Summer of Code under the umbrella of the Honeynet Project and the NGI0 Entrust fund established by NLnet. Thanks to my mentor Maximilian Hils for the invaluable guidance and support.