Dynamic Modules with dylo

snug uses a dynamic module loading system called dylo (short for “dynamic loading”) to enable a flexible architecture with cleanly separated components.

Overview of dylo

dylo is a Rust framework that allows modules to be loaded dynamically at runtime, providing:

  1. Clean separation between interface and implementation
  2. The ability to update components independently
  3. A plugin architecture that avoids dependency bloat
  4. Faster iteration during development
  5. Efficient deployment with layer caching

Crate Structure

The snug workspace contains two types of crates for each module:

Implementation Crates (mod-*)
  • Named with the mod- prefix (e.g., mod-config, mod-media)
  • Contain the actual implementation code
  • Compiled as dynamic libraries (.so files)
  • These are the only crates you should manually edit

Client Crates
  • Named without the prefix (e.g., config, media)
  • Automatically generated from the corresponding mod-* crate
  • Provide the interface used by the main application
  • Never edit these manually - they’re regenerated when implementations change

Development Workflow

When modifying or creating modules:

  1. Edit the implementation crate (e.g., mod-config)
  2. Run dylo to generate the client interface
  3. Build and test your changes with just run serve

Module Building Process

Both development and production builds pre-compile all modules and disable dylo’s automatic building with DYLO_BUILD=0.

Development Build

The development build process is controlled by the run command in the Justfile. This process:

  1. Finds all workspace members with names containing mod-
  2. Runs the dylo tool to generate client interfaces
  3. Builds all modules together using Cargo
  4. Copies the compiled libraries to a runtime directory
  5. Sets DYLO_BUILD=0 to use pre-built modules

This approach leverages Cargo’s workspace-aware build system, which efficiently:

  • Deduplicates dependencies across crates
  • Uses build pipelining to overlap compilation steps
  • Shares compilation artifacts between modules

Production Build

The production build process is defined in the Earthfile and:

  1. Identifies all mod-* crates using cargo metadata
  2. Builds them together with the impl feature enabled
  3. Compresses debug sections for production use
  4. Critically: Copies each module as a separate container layer

This multi-layer approach provides significant deployment advantages:

  • When one module changes, only its layer needs to be rebuilt
  • Unchanged modules are reused from the container cache
  • Deployments complete much faster when most modules are unchanged

This optimizes the CI/CD pipeline by making incremental deployments extremely efficient.

Performance Trade-offs

The dynamic module system involves trade-offs:

  • Build time: Faster rebuilds during development (only changed modules are relinked)
  • Startup time: Slightly slower initial startup (runtime module loading)
  • Deploy time: Much faster deployments (layer-based caching)
  • Runtime overhead: Small performance cost due to dynamic dispatch

For snug, these trade-offs make sense because the benefits to development and deployment efficiency outweigh the minor runtime cost.

Technical Implementation

Module Discovery

dylo automatically processes any workspace member with the mod- prefix. The client crate name is derived by removing this prefix:

  • mod-config → config
  • mod-media → media

Creating a new module is as simple as adding a new mod- prefixed crate to the workspace.

The .dylo Directory

When you run dylo, it generates two key files in each client crate:

  • spec.rs: Contains the interface derived from the implementation
  • support.rs: Includes code for dynamically loading the module

The #[dylo::export] Attribute

The #[dylo::export] attribute marks types and functions for exposure in the client interface:

// In mod-config/src/lib.rs #[dylo::export] impl Mod for ModImpl { fn load_config(&self, site_path: &Utf8Path) -> Result<Config> { // Implementation... } }

The attribute automatically extracts the public API from the implementation. You never manually define interfaces - they’re generated from the implementation.

The impl Feature

The impl feature flag separates implementation details from the interface:

// In mod-config/src/lib.rs #[cfg(feature = "impl")] #[derive(Default)] struct ModImpl; #[dylo::export] impl Mod for ModImpl { // Implementation methods }

Code marked with #[cfg(feature = "impl")] is:

  • Included when building implementation crates
  • Stripped when generating client interfaces

This applies to both source code and dependencies in the Cargo manifest:

# In mod-config/Cargo.toml [dependencies] # Core dependencies available to both impl and interface dylo.workspace = true camino = { version = "1.1.7" } # Implementation-only dependencies marked as optional fs-err = { version = "2.11.0", optional = true } dotenvy = { version = "0.15.7", optional = true } figment = { version = "0.10.19", optional = true } [features] default = ["impl"] # The impl feature enables all implementation-only dependencies impl = [ "dep:dotenvy", "dep:figment", "dep:fs-err" ]

This pattern ensures that implementation dependencies are only included when building the actual module, not when using the client interface.

Interface Requirements

All exported interfaces must be in the top-level module (usually lib.rs), including:

  1. All impl blocks marked with #[dylo::export]
  2. All types used in the public interface
  3. All function signatures in the public API

Implementation details can be placed in submodules, but the interface must be defined at the root level.

Resources