Path replacement support and add custom-options

Closes #51

docs/file_format/custom_options.md

@@ -0,0 +1,121 @@
+# Custom options
+
+The custom options are key-value pairs specified by the user at runtime that
+allow generating different linker scripts from the same input yaml file.
+
+This is somewhat similar to "conditional compilation" on compiled programing
+languages, kind of similar to what would be done in C with build-time defines
+handled by the C preprocessor or in Rust with its features.
+
+## Table of contents
+
+- [Custom options](#custom-options)
+  - [Table of contents](#table-of-contents)
+  - [How does this work?](#how-does-this-work)
+  - [Usage](#usage)
+    - [Path replacement](#path-replacement)
+      - [Example](#example)
+
+## How does this work?
+
+The user can provide any number of key-value pairs when using slinky (either via
+the CLI or the API) and use them to affect the generation of the linker script
+and other files generated by slinky. This pairs will be used by slinky to
+perform textual string replacement, affect which entries are emitted, etc.
+
+## Usage
+
+In the context of the CLI, the custom options are specified with the
+`--custom-options key=value` long flag or the `-c key=value` short flag.
+
+Multiple options can be set by passing the flag multiple times (i.e.
+`-c key1=value1 -c key2=value2 -c key3=value1 -c key2=value3`) or by passing
+multiple pairs separated by a comma (`,`) to the same flag (i.e.
+`-c key1=value1,key2=value2,key3=value1,key2=value3`)
+
+Keys are unique. If the same key is given multiple times to the CLI then the
+last key-value pair corresponding to this key is used. Values may be duplicated
+among different keys.
+
+Keys must be simple strings, containing only ASCII alphanumeric characters or
+underscore characters (`_`). Keys should not start with a numeric character.
+Keys should be composed of at least 1 character. This rules are similar to the
+ones imposed to C identifiers like variables. The following regular expression
+matches this restriction: `[a-zA-Z_][a-zA-Z0-9_]*`.
+
+If one or more custom option are passed that are not referenced by the input
+yaml file then they are simply ignored.
+
+The following subsections show how this custom options can affect the linker
+script generation.
+
+### Path replacement
+
+The path replacement feature allows to insert a custom key into a path in the
+input yaml and let slinky replace the given key with the passed custom value.
+This feature is specially useful when a single yaml file is wanted to be used
+among different versions of the same project by changing the build paths.
+
+slinky searches paths for strings contained within curly braces like `{key}`.
+The key and the braces are replaced with the corresponding value to the given
+key. Because of this, if a `{KEY}` is used on any path then a custom option pair
+containing the given `KEY` must be passed to slinky, otherwise it will refuse to
+generate the linker script and emit an error.
+
+A `{key}` used in path replacement can't cross each component boundary of the
+given path (each component being separated by `/`). In other words, something
+like `base_path: build/ga{exam/ple}me/` is not valid, but
+`base_path: build/{example}/game/`, `base_path: build/ga{example}me/`,
+`target_path: build/{example}/game.{version}.elf` are valid.
+
+#### Example
+
+Say we have a yaml file like the following:
+
+```yaml
+settings:
+  base_path: build/{version}
+  target_path: build/{version}/game.{version}.elf
+
+segments:
+  - name: header
+    files:
+      - { path: src/rom_header/rom_header.o }
+      - { path: "asm/{version}/data/ipl3.o" } # needs to be fenced in quotes
+      - { path: src/entry/entry.o }
+
+  - name: boot
+    files:
+      - { path: "src/boot/boot_main_{region}.o" } # needs to be fenced in quotes
+      - { path: src/boot/dmadata.o }
+      - { path: src/boot/util.o }
+```
+
+This yaml file requires two custom options to be passed, `version` and `region`,
+so if any of those are missing when invoking slinky then it will produce an
+error.
+
+Now we'll list what the each path expands to by using a few custom option
+combinations:
+
+- `--custom-option version=us -c region=ntsc`
+  - `target_path`: `build/us/game.us.elf`
+  - `header` segment:
+    - `build/us/src/rom_header/rom_header.o`
+    - `build/asm/us/data/ipl3.o`
+    - `build/us/src/entry/entry.o`
+  - `boot` segment:
+    - `build/us/src/boot/boot_main_ntsc.o`
+    - `build/us/src/boot/dmadata.o`
+    - `build/us/src/boot/util.o`
+
+- `--custom-option version=eu1.1,region=pal`
+  - `target_path`: `build/eu1.1/game.eu1.1.elf`
+  - `header` segment:
+    - `build/eu1.1/src/rom_header/rom_header.o`
+    - `build/asm/eu1.1/data/ipl3.o`
+    - `build/eu1.1/src/entry/entry.o`
+  - `boot` segment:
+    - `build/eu1.1/src/boot/boot_main_pal.o`
+    - `build/eu1.1/src/boot/dmadata.o`
+    - `build/eu1.1/src/boot/util.o`

slinky-cli/Cargo.toml

@@ -10,4 +10,5 @@ edition = "2021"
 
 [dependencies]
 clap = { version = "4.5.1", features = ["derive"] }
+regex = "1.10.5"
 slinky = { path = "../slinky", version = "0.1.0" }

slinky-cli/src/main.rs

@@ -1,9 +1,10 @@
 /* SPDX-FileCopyrightText: © 2024 decompals */
 /* SPDX-License-Identifier: MIT */
 
-use std::path::PathBuf;
+use std::{error::Error, path::PathBuf};
 
 use clap::Parser;
+use regex::Regex;
 
 // TODO: Add program description to cli
 
@@ -21,20 +22,50 @@ struct Cli {
     /// Requires both `partial_scripts_folder` and `partial_build_segments_folder` YAML settings to be set.
     #[arg(short, long, default_value_t = false)]
     partial_linking: bool,
+
+    #[arg(short = 'c', long, value_parser = parse_key_val::<String, String>, value_delimiter = ',')]
+    custom_options: Vec<(String, String)>,
+}
+
+// Taken from https://github.com/clap-rs/clap/blob/f5965e586292d31b2a2cbd83f19d145180471012/examples/typed-derive.rs#L48
+/// Parse a single key-value pair
+fn parse_key_val<T, U>(s: &str) -> Result<(T, U), Box<dyn Error + Send + Sync + 'static>>
+where
+    T: std::str::FromStr,
+    T::Err: Error + Send + Sync + 'static,
+    U: std::str::FromStr,
+    U::Err: Error + Send + Sync + 'static,
+{
+    let pos = s
+        .find('=')
+        .ok_or_else(|| format!("invalid KEY=value: no `=` found in `{s}`"))?;
+    Ok((s[..pos].parse()?, s[pos + 1..].parse()?))
 }
 
 fn main() {
     let cli = Cli::parse();
 
     // TODO: don't use expect?
-    let document = slinky::Document::read_file(&cli.input).expect("Error while parsing input file");
+    let mut document =
+        slinky::Document::read_file(&cli.input).expect("Error while parsing input file");
 
     // println!("settings {:#?}", document.settings);
 
+    let regex_identifier = Regex::new(r"[a-zA-Z_][a-zA-Z0-9_]*").unwrap();
+
+    document.custom_options.reserve(cli.custom_options.len());
+    for (key, value) in &cli.custom_options {
+        if !regex_identifier.is_match(key) {
+            // TODO: is there a better alternative than a plain panic?
+            panic!("Invalid key for custom option: '{}'", key);
+        }
+        document.custom_options.insert(key.into(), value.into());
+    }
+
     if cli.partial_linking {
         let mut writer = slinky::PartialLinkerWriter::new(&document);
 
-        writer.add_all_segments(&document.segments);
+        writer.add_all_segments(&document.segments).expect("");
 
         let output_path = cli
             .output
@@ -48,7 +79,7 @@ fn main() {
     } else {
         let mut writer = slinky::LinkerWriter::new(&document);
 
-        writer.add_all_segments(&document.segments);
+        writer.add_all_segments(&document.segments).expect("ah?");
 
         if let Some(output_path) = cli.output {
             writer

slinky/src/document.rs

@@ -1,7 +1,11 @@
 /* SPDX-FileCopyrightText: © 2024 decompals */
 /* SPDX-License-Identifier: MIT */
 
-use std::{fs, path::Path};
+use std::{
+    collections::HashMap,
+    fs,
+    path::{Path, PathBuf},
+};
 
 use serde::Deserialize;
 
@@ -17,6 +21,8 @@ pub struct Document {
     pub vram_classes: Vec<VramClass>,
 
     pub segments: Vec<Segment>,
+
+    pub custom_options: HashMap<String, String>,
 }
 
 impl Document {
@@ -41,6 +47,76 @@ impl Document {
 
         document_serial.unserialize()
     }
+
+    fn get_custom_option_value(
+        &self,
+        custom_option: &str,
+        original_path: &Path,
+    ) -> Result<&str, SlinkyError> {
+        match self.custom_options.get(custom_option) {
+            None => Err(SlinkyError::CustomOptionInPathNotProvided {
+                path: original_path.into(),
+                custom_option: custom_option.into(),
+            }),
+            Some(val) => Ok(val),
+        }
+    }
+
+    /// Replace all the `{key}` instances on the `path` argument with the corresponding value specified on the global `custom_options`.
+    ///
+    /// If the `key` is not present on the custom options then it returns an error.
+    pub(crate) fn escape_path(&self, path: &Path) -> Result<PathBuf, SlinkyError> {
+        let mut new_path = PathBuf::new();
+
+        for component in path.iter() {
+            // &OsStr is dumb so we convert each component into &str, hopefully the conversion isn't noticeable on runtime
+            if let Some(c) = component.to_str() {
+                if c.starts_with('{') && c.ends_with('}') {
+                    // left/{thingy}/right
+
+                    let custom_option = &c[1..c.len() - 1];
+
+                    new_path.push(self.get_custom_option_value(custom_option, path)?);
+                } else if !c.contains('{') || !c.contains('}') {
+                    // No replacement at all
+                    new_path.push(component);
+                } else {
+                    // There may be one or more replacements, so we need to look for all of them.
+
+                    let mut new_component = String::new();
+                    let mut within_replacment = false;
+                    let mut custom_option = String::new();
+
+                    for character in c.chars() {
+                        if within_replacment {
+                            if character == '}' {
+                                new_component +=
+                                    self.get_custom_option_value(&custom_option, path)?;
+
+                                within_replacment = false;
+                                custom_option.clear();
+                            } else {
+                                custom_option.push(character);
+                            }
+                        } else {
+                            // Haven't found a replacement yet, continue searching
+                            if character == '{' {
+                                within_replacment = true;
+                            } else {
+                                new_component.push(character);
+                            }
+                        }
+                    }
+
+                    new_path.push(new_component)
+                }
+            } else {
+                new_path.push(component);
+            }
+        }
+
+        Ok(new_path)
+    }
 }
 
 #[derive(Deserialize, PartialEq, Debug)]
@@ -87,6 +163,7 @@ impl DocumentSerial {
             settings,
             vram_classes,
             segments,
+            custom_options: HashMap::new(),
         })
     }
 }

slinky/src/error.rs

@@ -40,4 +40,10 @@ pub enum SlinkyError {
 
     #[error("At least one of the following options should be provided: {fields}")]
     MissingAnyOfOptionalFields { fields: String },
+
+    #[error("Path '{path}' referenced custom option {custom_option}, but it was not provided")]
+    CustomOptionInPathNotProvided {
+        path: PathBuf,
+        custom_option: String,
+    },
 }