Abstract
# We can’t live without Async, and that’s a problem
A journey through the origin of Async Rust, why it is necessary (and when it’s not) and the issues caused by the “over-async” phenomenon.
We will also cover why for library maintainers it’s hard to create sync/async public API for their crates and how (and whether) we can mitigate it.
Showcasing maybe-fut a library that with the power of advanced proc macro is able to allow developers to expose sync and async API reusing the same code.
Eventually, we’ll conclude with some thoughts on an async Rust std library and how the language could overcome code duplication when implementing both APIs.
## The async era begins
I’ve started working with Rust with version 1.36, where async wasn’t still a thing. It would have been released very soon though, in November 2019 with Rust 1.39.
When Async Rust was released I was mainly working with CLI tools in my spare time, such as termscp, which is now quite popular. At the time Async wasn’t something I was interested in and I thought it would have been just a craze, especially because from my point of view, multi-threading was just amazing.
## Why do we even need Async?
Nowadays most people will start a new Rust project with a tokio main and sometimes it seems that tokio has almost replaced the std library and multi-threading is obsolete.
But actually, there are particular reasons we have async and I’m quite sure many people don’t know them and the pros and cons of using async.
We all know that async is based on **Future**s, which is something like this
```rust
REDACTED CAUSE FILTERED BY THE FORM
```
and every async function can be rewritten like this
```rust
REDACTED CAUSE FILTERED BY THE FORM
```
and `sum` is equal to
```rust
REDACTED CAUSE FILTERED BY THE FORM
```
In this case async doesn’t make much sense though, because we’re not really polling anything, but there are of course cases where async is a big advantage, and are **mainly tasks that depend on external factors**, and we have mainly three:
* **I/O:** reading from and writing to the filesystem depends on the filesystem itself. Usually, this doesn't have significant delays, but it's still something we don't have control of and so it's good for async. We schedule an I/O operation to the OS and we poll for it to finish. Actually we should note that for example, Linux standard I/O is not async. There is uring, but it’s not used by default and it’s quite complex to be used.
* **Network**: any interaction with the network creates a lot of delays depending on many external resources. Every packet must across the network and reach the final destination, which will elaborate the packet and it will travel back until we receive it.
* **Time**: sometimes we ignore that, but time is async and can't be *resolved* immediately. If we need to wait 5 seconds, we depend on an *external system*, which is the Universe I guess, so async is also good for that.
### Why did async take over threads?
Threads are generally more performant than async, indeed applications which require extreme performance such as the Solana blockchain validator prefer threads over the async runtime, but somehow we generally opt for async.
Whether you should prefer async over threads depends on a simple question you should ask yourself: **is the amount of workers deterministic for my use case?**
For instance, the Solana validator has MANY different internal services which run on a multi-threaded architecture, but the amount of workers is constant and deterministic.
A good example of when we don’t have this deterministic amount of workers is an HTTP server. We may have 5 requests per minute or 5,000 requests per second, and each request must be handled by a worker. So it’s undeterministic.
Handling an HTTP server with threads would be overall very hard to do, because we must also take into account that managing requests will often require access to external resources which will add a delay overhead to each worker (such as access to the FS or to the database).
So eventually async provides a very simple way to handle an undeterministic amount of workers without having to care about I/O delays.
## My first ~~approach~~ conflict with Async Rust
In 2021 I was working on termscp and I needed an FTP client library, but the only one (rust-ftp) was unmaintained, so I decided to adopt it, initially with the temporary name of ftp4, and eventually with the definitive abandon of the rust-ftp project, suppaftp.
One of the first requests I received, after I migrated it from Rust 1.20, was to change the client to **async**.
There was just one problem though. The main use case for my FTP library at the time was termscp, which since it’s just a terminal application, didn’t and still doesn’t use async.
So on one hand I had flocks of people asking for async, and on the other side there was just me alone that I didn’t want async and I didn’t understand async. I still thought that multi-threading was fine.
Eventually suppaftp implemented both the async and sync client, like many other libraries used to do at the time.
The only issue with that is **interoperability**.
## Why Async/Sync interoperability is an issue
If you’ve never implemented a library with a type which implements both the async and sync versions you’re lucky because it consists of just copy-pasting the code and putting async everywhere by changing the types from the Std library to tokio or async-std or whatever.
In most of the cases, the code is 90% the same, but there are some compatibility issues.
Showcasing compatibility issues
* Send \+ Sync stuff
* Fn and closures
* Recursion
* …
### Why don’t you just Block on?
Some will say, okay just opt for async in wherever there is I/O and in case you don’t need async, just block on.
Indeed we could use things like futures::block\_on or implement our very simple example
*showcase a custom block\_on*
But again, it’s not always possible, because unfortunately there are actually some libraries which rely on the tokio runtime. For instance, the aws-sdk internally it uses the sync structures of tokio and without a tokio runtime it will just panic.
### Macros and other sorceries
There have been some experiments of creating macros that converted sync code to async code, but it’s of course very limited to simple cases and the reason is of course the compatibility issues we’ve seen before.
\*showcasing examples of this\*
### maybe-async?
Showcasing an experimental library which creates good interoperability between sync and async to create compatible code. This will be very experimental and a work in progress, but a really interesting case.
