> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/microsoft/onnxruntime-genai/llms.txt
> Use this file to discover all available pages before exploring further.

# Generator

> Generate text sequences with ONNX Runtime GenAI models

The `Generator` class manages the text generation process. It handles the iterative generation of tokens and provides access to generated sequences.

## Constructor

### Generator(Model model, GeneratorParams generatorParams)

Creates a new generator instance.

<ParamField path="model" type="Model" required>
  The model to use for generation
</ParamField>

<ParamField path="generatorParams" type="GeneratorParams" required>
  Parameters controlling the generation process
</ParamField>

```csharp Generator.cs:13-16 theme={null}
using Microsoft.ML.OnnxRuntimeGenAI;

using Model model = new Model("/path/to/model");
using GeneratorParams generatorParams = new GeneratorParams(model);
using Generator generator = new Generator(model, generatorParams);
```

## Generation Control Methods

### GenerateNextToken()

Generates the next token in the sequence. This is the core method for the generation loop.

```csharp Generator.cs:58-61 theme={null}
while (!generator.IsDone())
{
    generator.GenerateNextToken();
}
```

### IsDone()

Checks if generation is complete (end-of-sequence token generated or max length reached).

**Returns:** `bool` - `true` if generation is complete, `false` otherwise

```csharp Generator.cs:18-21 theme={null}
if (generator.IsDone())
{
    Console.WriteLine("Generation complete");
}
```

### RewindTo(ulong newLength)

Rewinds the generator to a specific token length. Useful for chat scenarios where you want to keep the system prompt but remove the conversation history.

<ParamField path="newLength" type="ulong" required>
  The token length to rewind to
</ParamField>

```csharp Generator.cs:68-71 theme={null}
// Save system prompt length
ulong systemPromptLength = generator.TokenCount();

// ... generate response ...

// Rewind to system prompt for next turn
generator.RewindTo(systemPromptLength);
```

## Input Methods

### AppendTokens(ReadOnlySpan\<int> inputIDs)

Appends token IDs to the generator's input.

<ParamField path="inputIDs" type="ReadOnlySpan<int>" required>
  The token IDs to append
</ParamField>

```csharp Generator.cs:33-42 theme={null}
ReadOnlySpan<int> tokens = new int[] { 1, 2, 3, 4 };
generator.AppendTokens(tokens);
```

### AppendTokenSequences(Sequences sequences)

Appends encoded sequences to the generator's input.

<ParamField path="sequences" type="Sequences" required>
  The sequences to append (typically from `Tokenizer.Encode()`)
</ParamField>

```csharp Generator.cs:44-47 theme={null}
using Sequences sequences = tokenizer.Encode("Hello, world!");
generator.AppendTokenSequences(sequences);
```

### SetModelInput(string name, Tensor value)

Sets a specific model input tensor.

<ParamField path="name" type="string" required>
  The name of the input
</ParamField>

<ParamField path="value" type="Tensor" required>
  The tensor value to set
</ParamField>

```csharp Generator.cs:23-26 theme={null}
Tensor inputTensor = /* create tensor */;
generator.SetModelInput("custom_input", inputTensor);
```

### SetInputs(NamedTensors namedTensors)

Sets multiple model inputs at once.

<ParamField path="namedTensors" type="NamedTensors" required>
  Collection of named tensors to set as inputs
</ParamField>

```csharp Generator.cs:28-31 theme={null}
using NamedTensors inputs = processor.ProcessImagesAndAudios(prompt, images, audios);
generator.SetInputs(inputs);
```

## Output Methods

### GetNextTokens()

Returns the tokens generated in the last `GenerateNextToken()` call.

**Returns:** `ReadOnlySpan<int>` - The most recently generated tokens

```csharp Generator.cs:73-80 theme={null}
generator.GenerateNextToken();
ReadOnlySpan<int> newTokens = generator.GetNextTokens();
```

### GetSequence(ulong index)

Returns the complete token sequence for a specific sequence index.

<ParamField path="index" type="ulong" required>
  The sequence index (0 for single sequence generation)
</ParamField>

**Returns:** `ReadOnlySpan<int>` - The complete token sequence

```csharp Generator.cs:82-90 theme={null}
// Get the complete output sequence
var outputSequence = generator.GetSequence(0);
string outputText = tokenizer.Decode(outputSequence);
```

### TokenCount()

Returns the total number of tokens in the generator (including input and generated tokens).

**Returns:** `ulong` - The total token count

```csharp Generator.cs:53-56 theme={null}
ulong totalTokens = generator.TokenCount();
Console.WriteLine($"Total tokens: {totalTokens}");
```

## Tensor Access Methods

### GetInput(string inputName)

Retrieves an input tensor by name.

<ParamField path="inputName" type="string" required>
  The name of the input tensor
</ParamField>

**Returns:** `Tensor` - The input tensor

```csharp Generator.cs:98-104 theme={null}
using Tensor inputTensor = generator.GetInput("input_ids");
```

### GetOutput(string outputName)

Retrieves an output tensor by name.

<ParamField path="outputName" type="string" required>
  The name of the output tensor
</ParamField>

**Returns:** `Tensor` - The output tensor

```csharp Generator.cs:112-118 theme={null}
using Tensor outputTensor = generator.GetOutput("logits");
```

## Adapter Methods

### SetActiveAdapter(Adapters adapters, string adapterName)

Activates a previously loaded adapter (for LoRA/fine-tuned models).

<ParamField path="adapters" type="Adapters" required>
  The adapters container
</ParamField>

<ParamField path="adapterName" type="string" required>
  The name of the adapter to activate
</ParamField>

```csharp Generator.cs:126-131 theme={null}
using Adapters adapters = /* load adapters */;
generator.SetActiveAdapter(adapters, "my_adapter");
```

## Complete Generation Example

Here's a complete example of the generation loop:

```csharp examples/csharp/ModelChat/Program.cs:54-84 theme={null}
using Microsoft.ML.OnnxRuntimeGenAI;
using System.Diagnostics;

string modelPath = "/path/to/model";

// Load model
using Model model = new Model(modelPath);
using Tokenizer tokenizer = new Tokenizer(model);

// Prepare input
string prompt = "Once upon a time";
using var sequences = tokenizer.Encode(prompt);

// Create generator params
using GeneratorParams generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 200);
generatorParams.SetSearchOption("temperature", 0.7);

// Create generator and append input tokens
using Generator generator = new Generator(model, generatorParams);
generator.AppendTokenSequences(sequences);

// Run generation loop
var watch = Stopwatch.StartNew();
while (!generator.IsDone())
{
    generator.GenerateNextToken();
}
watch.Stop();

// Get output and decode
var outputSequence = generator.GetSequence(0);
string outputString = tokenizer.Decode(outputSequence);

// Display results
Console.WriteLine("Output:");
Console.WriteLine(outputString);

var totalTokens = (int)generator.TokenCount();
var tokensPerSecond = totalTokens / watch.Elapsed.TotalSeconds;
Console.WriteLine($"Tokens: {totalTokens}, Time: {watch.Elapsed.TotalSeconds:0.00}s, Tokens/sec: {tokensPerSecond:0.00}");
```

## Streaming Generation Example

```csharp examples/csharp/ModelChat/Program.cs:200-214 theme={null}
using Microsoft.ML.OnnxRuntimeGenAI;

using Model model = new Model("/path/to/model");
using Tokenizer tokenizer = new Tokenizer(model);
using TokenizerStream tokenizerStream = tokenizer.CreateStream();

// ... setup generator ...

Console.Write("Output: ");
var watch = Stopwatch.StartNew();
while (!generator.IsDone())
{
    generator.GenerateNextToken();
    // Decode and print each token immediately
    Console.Write(tokenizerStream.Decode(generator.GetNextTokens()[0]));
}
watch.Stop();
Console.WriteLine();

var totalTokens = (int)generator.TokenCount();
Console.WriteLine($"Tokens: {totalTokens}, Time: {watch.Elapsed.TotalSeconds:0.00}s");
```

## Chat with Rewind Example

```csharp examples/csharp/ModelChat/Program.cs:314-382 theme={null}
using Microsoft.ML.OnnxRuntimeGenAI;

using Model model = new Model("/path/to/model");
using Tokenizer tokenizer = new Tokenizer(model);
using TokenizerStream tokenizerStream = tokenizer.CreateStream();

// Create generator params
using GeneratorParams generatorParams = new GeneratorParams(model);
using Generator generator = new Generator(model, generatorParams);

// Encode system prompt
string systemPrompt = "You are a helpful AI assistant.";
var sequences = tokenizer.Encode(systemPrompt);
generator.AppendTokenSequences(sequences);
var systemPromptLength = (int)generator.TokenCount();

// Chat loop
while (true)
{
    // Get user input
    Console.Write("User: ");
    string userPrompt = Console.ReadLine();
    if (userPrompt == "quit()") break;

    // Encode and append user prompt
    sequences = tokenizer.Encode(userPrompt);
    generator.AppendTokenSequences(sequences);

    // Generate response
    Console.Write("Assistant: ");
    while (!generator.IsDone())
    {
        generator.GenerateNextToken();
        Console.Write(tokenizerStream.Decode(generator.GetNextTokens()[0]));
    }
    Console.WriteLine();

    // Rewind to system prompt for next turn
    generator.RewindTo((ulong)systemPromptLength);
}
```

## See Also

* [Model](/api/csharp/model) - Load models
* [Tokenizer](/api/csharp/tokenizer) - Encode and decode text
* [GeneratorParams](/api/csharp/params) - Configure generation parameters