3_2_macro

Step 3.2: Declarative and procedural macros

Estimated time: 1 day

Rust provides strong and convenient built-in capabilities for code generation in a form of macros.

The term macro refers to a family of features in Rust: declarative macros with macro_rules! and three kinds of procedural macros:

• Custom #[derive] macros that specify code added with the derive attribute used on structs and enums
• Attribute-like macros that define custom attributes usable on any item
• Function-like macros that look like function calls but operate on the tokens specified as their argument

Declarative macros

Declarative macros represent the most primitive form of macros in Rust. They are quite limited in their capabilities and their syntax (which represents a DSL-based match expression) may become quite cumbersome in complex cases.

They are called declarative, because macro implementation represents a declaration of code transforming rules (you're declaring how your code will be transformed):

macro_rules! vec {
    ( $( $x:expr ),* ) => {
        {
            let mut temp_vec = Vec::new();
            $(
                temp_vec.push($x);
            )*
            temp_vec
        }
    };
}

let v = vec![1, 2, 3];

The good part about declarative macros is that they are hygienic (and so, have much better IDEs support).

Code generation purpose is not the only one declarative macros are used for. Quite often they are used for building abstractions and APIs too, because they all to implement much more ergonomic features than regular functions do: named arguments, variadics, etc.

For better understanding declarative macros design, concepts, usage and features, read through the following articles:

• Rust Book: 19.6. Macros: Declarative Macros with macro_rules! for General Metaprogramming
• Rust By Example: 16. macro_rules!
• The Little Book of Rust Macros
• Rust Reference: 3.1. Macros By Example
• Aurorans Solis: macros_rule!

Procedural macros

Procedural macros represent much more powerful code generation tool. They are called procedural, because macro implementation represents a regular Rust code, which works directly with AST of transformed code (you're writing procedures which transform your code). Procedural macro requires a separate proc-macro = true crate to be implemented in.

Procedural macros are unhygienic, so implementing one you need to be careful to ensure that macro works in as many contexts as possible.

There are three kinds of procedural macros in Rust at the moment:

• proc_macro function-like macros, which usage looks like regular declarative macros usage, but they accept arbitrary tokens on input (while declarative ones don't), and are more powerful in general (can contain complex logic for generating simple code):

  #[proc_macro]
  pub fn make_answer(_: TokenStream) -> TokenStream {
      "fn answer() -> u32 { 42 }".parse().unwrap()
  }

  make_answer!();

• proc_macro_attribute attribute macros, which allow to create custom Rust attributes:

  #[proc_macro_attribute]
  pub fn route(attr: TokenStream, item: TokenStream) -> TokenStream {
     // code...
  }

  #[route(GET, "/")]
  fn index() {}

• proc_macro_derive derive macros, which allow to provide custom implementations for #[derive(Trait)] attribute:

  #[proc_macro_derive(AnswerFn)]
  pub fn derive_answer_fn(_: TokenStream) -> TokenStream {
      "impl Struct{ fn answer() -> u32 { 42 } }".parse().unwrap()
  }

  #[derive(AnswerFn)]
  struct Struct;

  Idiomatically, proc_macro_derive should be used for deriving trait implementations only. For arbitrary functions generation it's better to go with proc_macro_attribute.

Rust ecosystem has some well-know crates, which almost always are used for procedural macros' implementation:

• syn crate represents an implementation of Rust's AST.
• quote crate provides quasi-quoting, which allows to turn Rust syntax tree data structures into tokens of source code in an ergonomic and readable way.
• proc-macro2 crate provides unified proc_macro API across all Rust compiler versions and makes procedural macros unit-testable.

Nowadays, these are backbone for writing a procedural macro implementation. Even though, developers mostly tend ot omit using syn for trivial cases (not requiring much AST parsing), as it hits compilation times quite notably, or prefer to use simpler and less powerful AST parsing crates (like venial).

On top of them, more ecosystem crates may be used for having less boilerplate, better ergonomics and "batteries included". Most notable among them are:

• darling crate, making declarative attribute parsing more straight-forward and ergonomic.
• synstructure crate, providing helper types for matching against enum variants, and extracting bindings to each of the fields in the deriving struct or enum in a generic way.
• synthez crate, providing derive macros for parsing AST (yeah, derive macros for derive macros!) and other helpful "batteries" for daily routine of procedural macro writing.

For better understanding procedural macros design, concepts, usage and features, read through the following articles:

• Rust Book: 19.6. Macros: Procedural Macros for Generating Code from Attributes
• Rust Reference: 3.2. Procedural Macros
• Official syn crate docs
• Official venial crate docs
• Official quote crate docs
• Official proc-macro2 crate docs
• Nazmul Idris: Guide to Rust procedural macros
• Vitaly Bragilevsky: What Every Rust Developer Should Know About Macro Support in IDEs
• Arthur Cohen: Looking at Rust builtin derives

Task

Implement a btreemap! macro, which allows to create BTreeMap in an ergonomic and declarative way (similarly to vec!).

Provide two implementations: one via declarative macro and other one via procedural macro.

Questions

After completing everything above, you should be able to answer (and understand why) the following questions:

• What are macros? Which problem do they solve?
• Which benefits do declarative macros have in Rust comparing to procedural ones? Which downsides and limitations?
• Which kinds of procedural macros do exist in Rust?
• What are common crates for implementing procedural macros in Rust? What responsibilities does each one have? Which are mandatory, which are not?
• What are good practices for implementing procedural macros in Rust?