QUIC restarts, slow problems: udpgrm to the rescue
2025-05-07
udpgrm is a lightweight daemon for graceful restarts of UDP servers. It leverages SO_REUSEPORT and eBPF to route new and existing flows to the correct server instance....
Multi-Path TCP: revolutionizing connectivity, one path at a time
2025-01-03
Multi-Path TCP (MPTCP) leverages multiple network interfaces, like Wi-Fi and cellular, to provide seamless mobility for more reliable connectivity. While promising, MPTCP is still in its early stages,...
The forecast is clear: clouds on e-paper, powered by the cloud
2024-12-31
Follow along as I build a custom weather display using Cloudflare Workers and a popular e-paper display....
Virtual networking 101: bridging the gap to understanding TAP
2023-10-06
Tap devices were historically used for VPN clients. Using them for virtual machines is essentially reversing their original purpose - from traffic sinks to traffic sources. In the article I explore the intricacies of tap devices, covering topics like offloads, segmentation, and multi-queue....
The day my ping took countermeasures
2023-07-11
Ping developers clearly put some thought into that. I wondered how far they went. Did they handle clock changes in both directions? Are the bad measurements excluded from the final statistics? How do they test the software?...
Cloudflare servers don't own IPs anymore – so how do they connect to the Internet?
2022-11-25
In this blog we'll discuss how we manage Cloudflare IP addresses used to retrieve the data from the Internet, how our egress network design has evolved, how we optimized it for best use of available IP space and introduce our soft-anycast technology...
When the window is not fully open, your TCP stack is doing more than you think
2022-07-26
In this blog post I'll share my journey deep into the Linux networking stack, trying to understand the memory and window management of the receiving side of a TCP connection...
How to stop running out of ephemeral ports and start to love long-lived connections
2022-02-02
Often programmers have assumptions that turn out, to their surprise, to be invalid. From my experience this happens a lot. Every API, technology or system can be abused beyond its limits and break in a miserable way...
Everything you ever wanted to know about UDP sockets but were afraid to ask, part 1
2021-11-25
Historically Cloudflare's core competency was operating an HTTP reverse proxy. We've spent significant effort optimizing traditional HTTP/1.1 and HTTP/2 servers running on top of TCP....
Branch predictor: How many "if"s are too many? Including x86 and M1 benchmarks!
2021-05-06
Is it ok to have if clauses that will basically never be run? Surely, there must be some performance cost to that......
Computing Euclidean distance on 144 dimensions
2020-12-18
Last year we deployed a CSAM image scanning tool. This is so cool! Image processing is always hard, and deploying a real image identification system at a Cloudflare scale is no small achievement! But we hit a problem - the matching algorithm was too slow for our needs....
Why is there a "V" in SIGSEGV Segmentation Fault?
2020-06-18
My program received a SIGSEGV signal and crashed with "Segmentation Fault" message. Where does the "V" come from? Did I read it wrong? Was there a "Segmentation *V*ault?"? Or did Linux authors make a mistake? Shouldn't the signal be named SIGSEGF? ...
Conntrack tales - one thousand and one flows
2020-04-06
We were wondering - can we just enable Linux "conntrack"? How does it actually work? I volunteered to help the team understand the dark corners of the Linux's "conntrack" stateful firewall subsystem....
When Bloom filters don't bloom
2020-03-02
Last month finally I had an opportunity to use Bloom filters. I became fascinated with the promise of this data structure, but I quickly realized it had some drawbacks. This blog post is the tale of my brief love affair with Bloom filters....
When TCP sockets refuse to die
2019-09-20
We noticed something weird - the TCP sockets which we thought should have been closed - were lingering around. We realized we don't really understand when TCP sockets are supposed to time out! We naively thought enabling TCP keepalives would be enough... but it isn't!...
A gentle introduction to Linux Kernel fuzzing
2019-07-10
For some time I’ve wanted to play with coverage-guided fuzzing. I decided to have a go at the Linux Kernel netlink machinery. It's a good target: it's an obscure part of kernel, and it's relatively easy to automatically craft valid messages....
Cloudflare architecture and how BPF eats the world
2019-05-18
Recently at I gave a short talk titled "Linux at Cloudflare". The talk ended up being mostly about BPF. It seems, no matter the question - BPF is the answer. Here is a transcript of a slightly adjusted version of that talk....
RFC8482 - Saying goodbye to ANY
2019-03-15
Ladies and gentlemen, I would like you to welcome the new shiny RFC8482, which effectively deprecates DNS ANY query type. DNS ANY was a "meta-query" - think about it as a similar thing to the common A, AAAA, MX or SRV query types, but unlike these it wasn't a real query type - it was special....
io_submit: The epoll alternative you've never heard about
2019-01-04
The Linux AIO is designed for, well, Asynchronous disk IO! They are not network sockets, but is it possible to use Linux AIO API with network sockets? The answer is a strong YES! ...