> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/microsoft/onnxruntime-genai/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-LoRA Support

> Dynamically load and switch between multiple LoRA adapters at runtime

ONNX Runtime GenAI supports Multi-LoRA, allowing you to dynamically load, manage, and switch between multiple LoRA (Low-Rank Adaptation) adapters at runtime without reloading the base model.

## Overview

Multi-LoRA support enables:

* **Dynamic adapter loading**: Load adapters on-demand without restarting
* **Efficient memory usage**: Share the base model across multiple adapters
* **Adapter switching**: Change adapters between generations
* **Reference counting**: Automatically manage adapter lifecycle

## Use Cases

<CardGroup cols={2}>
  <Card title="Multi-Tenant Serving" icon="users">
    Serve different fine-tuned models to different users while sharing the base model
  </Card>

  <Card title="Task-Specific Adaptation" icon="list-check">
    Switch between adapters optimized for different tasks (summarization, translation, etc.)
  </Card>

  <Card title="A/B Testing" icon="flask">
    Test different adapter versions without infrastructure changes
  </Card>

  <Card title="Personalization" icon="user">
    Provide personalized model behavior per user or session
  </Card>
</CardGroup>

## Preparing LoRA Adapters

First, create your LoRA adapters using the Model Builder:

```bash theme={null}
python -m onnxruntime_genai.models.builder \
  -i path_to_base_model \
  -o path_to_output_folder \
  -p fp16 \
  -e cuda \
  -c cache_dir \
  --extra_options adapter_path=path_to_lora_weights
```

<Info>
  * Base model weights should be in `path_to_base_model`
  * LoRA adapter weights should be in `path_to_lora_weights`
  * The adapter must be compatible with the base model architecture
</Info>

See the [Model Builder guide](/guides/model-builder#lora-adapter-support) for more details.

## Using Multi-LoRA at Runtime

### Python Example

Here's a complete example showing how to use multiple LoRA adapters:

```python theme={null}
import onnxruntime_genai as og

# Load the base model
model = og.Model('path/to/base/model')
tokenizer = og.Tokenizer(model)

# Create the Adapters manager
adapters = og.Adapters.Create(model)

# Load multiple LoRA adapters
adapters.LoadAdapter(
    adapter_file_path='path/to/adapter1/adapter_weights.onnx',
    adapter_name='summarization'
)

adapters.LoadAdapter(
    adapter_file_path='path/to/adapter2/adapter_weights.onnx',
    adapter_name='translation'
)

adapters.LoadAdapter(
    adapter_file_path='path/to/adapter3/adapter_weights.onnx',
    adapter_name='coding'
)

# Set up generation parameters
params = og.GeneratorParams(model)
params.set_search_options(max_length=200)

# Create generator and set active adapter
generator = og.Generator(model, params)

# Use the summarization adapter
generator.SetActiveAdapter(adapters, 'summarization')

# Encode input and generate
prompt = "Summarize this article: ..."
input_tokens = tokenizer.encode(prompt)
generator.append_tokens(input_tokens)

while not generator.is_done():
    generator.generate_next_token()

summary = tokenizer.decode(generator.get_sequence(0))
print(f"Summary: {summary}")

# Switch to a different adapter for the next generation
generator2 = og.Generator(model, params)
generator2.SetActiveAdapter(adapters, 'translation')

# Generate with translation adapter
translation_prompt = "Translate to French: Hello, how are you?"
input_tokens = tokenizer.encode(translation_prompt)
generator2.append_tokens(input_tokens)

while not generator2.is_done():
    generator2.generate_next_token()

translation = tokenizer.decode(generator2.get_sequence(0))
print(f"Translation: {translation}")

# Unload an adapter when no longer needed
# Note: Will fail if the adapter is still in use
adapters.UnloadAdapter('summarization')
```

### C++ Example

```cpp theme={null}
#include "ort_genai.h"
#include <iostream>

int main() {
    // Load the base model
    auto model = OgaModel::Create("path/to/base/model");
    auto tokenizer = OgaTokenizer::Create(*model);
    
    // Create Adapters manager
    auto adapters = OgaAdapters::Create(*model);
    
    // Load LoRA adapters
    adapters->LoadAdapter(
        "path/to/adapter1/adapter_weights.onnx",
        "summarization"
    );
    
    adapters->LoadAdapter(
        "path/to/adapter2/adapter_weights.onnx",
        "translation"
    );
    
    // Create generator params
    auto params = OgaGeneratorParams::Create(*model);
    params->SetSearchOption("max_length", 200);
    
    // Create generator with specific adapter
    auto generator = OgaGenerator::Create(*model, *params);
    generator->SetActiveAdapter(*adapters, "summarization");
    
    // Encode and generate
    const char* prompt = "Summarize this article: ...";
    auto input_tokens = tokenizer->Encode(prompt);
    generator->AppendTokens(input_tokens);
    
    while (!generator->IsDone()) {
        generator->GenerateNextToken();
    }
    
    auto output_tokens = generator->GetSequence(0);
    auto output_text = tokenizer->Decode(output_tokens);
    std::cout << "Summary: " << output_text << std::endl;
    
    // Unload adapter
    adapters->UnloadAdapter("summarization");
    
    return 0;
}
```

### C# Example

```csharp theme={null}
using Microsoft.ML.OnnxRuntimeGenAI;

// Load the base model
using var model = new Model("path/to/base/model");
using var tokenizer = new Tokenizer(model);

// Create Adapters manager
using var adapters = new Adapters(model);

// Load LoRA adapters
adapters.LoadAdapter(
    adapterFilePath: "path/to/adapter1/adapter_weights.onnx",
    adapterName: "summarization"
);

adapters.LoadAdapter(
    adapterFilePath: "path/to/adapter2/adapter_weights.onnx",
    adapterName: "translation"
);

// Create generator params
using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 200);

// Create generator and set adapter
using var generator = new Generator(model, generatorParams);
generator.SetActiveAdapter(adapters, "summarization");

// Generate text
var prompt = "Summarize this article: ...";
var inputTokens = tokenizer.Encode(prompt);
generator.AppendTokens(inputTokens);

while (!generator.IsDone())
{
    generator.GenerateNextToken();
}

var summary = tokenizer.Decode(generator.GetSequence(0));
Console.WriteLine($"Summary: {summary}");

// Unload adapter
adapters.UnloadAdapter("summarization");
```

## API Reference

### Adapters Class

<ParamField path="Create" type="method">
  Creates an Adapters manager instance for the given model.

  **Parameters:**

  * `model`: The base model to manage adapters for

  **Returns:** Adapters instance
</ParamField>

<ParamField path="LoadAdapter" type="method">
  Loads a LoRA adapter from disk.

  **Parameters:**

  * `adapter_file_path`: Path to the adapter weights file
  * `adapter_name`: Unique identifier for this adapter

  **Throws:** Error if adapter name already exists
</ParamField>

<ParamField path="UnloadAdapter" type="method">
  Unloads a previously loaded adapter.

  **Parameters:**

  * `adapter_name`: Name of the adapter to unload

  **Throws:**

  * Error if adapter not found
  * Error if adapter is still in use (ref count > 0)
</ParamField>

### Generator Methods

<ParamField path="SetActiveAdapter" type="method">
  Sets the active LoRA adapter for this generator.

  **Parameters:**

  * `adapters`: The Adapters manager instance
  * `adapter_name`: Name of the adapter to activate

  **Throws:** Error if adapter not found
</ParamField>

## Best Practices

<AccordionGroup>
  <Accordion title="Adapter Lifecycle Management">
    * Load adapters at application startup for better performance
    * Unload adapters only when they're no longer needed across all sessions
    * The library uses reference counting to prevent unloading adapters that are in use
  </Accordion>

  <Accordion title="Naming Convention">
    Use descriptive, consistent names for your adapters:

    * `task-based`: "summarization", "translation", "code-generation"
    * `user-based`: "user\_123", "tenant\_abc"
    * `version-based`: "summarization\_v1", "summarization\_v2"
  </Accordion>

  <Accordion title="Memory Considerations">
    * Each adapter adds memory overhead (typically small compared to base model)
    * Monitor memory usage when loading many adapters
    * Consider lazy-loading adapters on-demand for large deployments
  </Accordion>

  <Accordion title="Adapter Compatibility">
    * Ensure adapters are created from the same base model
    * Use consistent precision (fp16, fp32) across base model and adapters
    * Verify adapter architecture matches the base model
  </Accordion>
</AccordionGroup>

## Performance Tips

<Steps>
  <Step title="Pre-load Common Adapters">
    Load frequently-used adapters at startup to avoid latency during inference.
  </Step>

  <Step title="Reuse Generator Instances">
    When possible, reuse generator instances and just switch adapters rather than creating new generators.
  </Step>

  <Step title="Batch Similar Requests">
    Group requests that use the same adapter together to minimize adapter switching overhead.
  </Step>

  <Step title="Monitor Reference Counts">
    Keep track of which adapters are in use to optimize when to load/unload them.
  </Step>
</Steps>

## Troubleshooting

<Warning>
  **"Adapter still in use" error when unloading:**

  This occurs when trying to unload an adapter that has active references. Ensure all generators using this adapter have completed or been destroyed.
</Warning>

<Warning>
  **"Adapter not found" error:**

  * Verify the adapter name is spelled correctly (case-sensitive)
  * Ensure the adapter was successfully loaded before attempting to use it
  * Check that the adapter hasn't been unloaded
</Warning>

<Warning>
  **Memory issues with many adapters:**

  * Limit the number of simultaneously loaded adapters
  * Implement an LRU cache to automatically unload least-used adapters
  * Monitor system memory and adapter usage patterns
</Warning>

## Next Steps

<CardGroup cols={2}>
  <Card title="Model Builder" icon="hammer" href="/guides/model-builder">
    Learn how to create LoRA adapters
  </Card>

  <Card title="Runtime Options" icon="sliders" href="/guides/runtime-options">
    Configure additional runtime settings
  </Card>

  <Card title="Python API" icon="python" href="/api-reference/python/adapters">
    Explore the Adapters API reference
  </Card>

  <Card title="Examples" icon="code" href="https://github.com/microsoft/onnxruntime-genai/tree/main/examples">
    View complete examples on GitHub
  </Card>
</CardGroup>