Abstract
Many developers entertain the notion that they could write a 'curl' clone in a
weekend. After all, *how hard could it be* to send a network request
and print the result?
In reality, they vastly oversimplify the intricacies involved.
My command-line tool lychee, an async Rust link checker, initially anticipated
as a weekend venture, has now extended into two years and still counting.
This journey unveils the deceptive complexity of what appears to be
a rudimentary task on the surface.
Throughout the developmental process, I encountered a plethora of challenges
that added layers of complexity. Some of these included handling rate limiting,
navigating around incorrect status codes, mastering the art of URL parsing,
dealing with timeouts, and grappling with client-side rendering using
JavaScript. We will go through each of these challenges step by step, and
explore the Weirdness of the Web.
Furthermore, we delve into the essential skills required in the development
process, exploring the often-overlooked aspects of funding open-source projects
and the relentless world of open-source maintenance. These insights shine a
light on the harsh realities and the hidden rewards of this crucial part of
software development.
The rabbit hole goes deep. How deep, you ask?
Join the talk to find out.
