> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/microsoft/onnxruntime-genai/llms.txt
> Use this file to discover all available pages before exploring further.

# OgaGeneratorParams

> Configure text generation parameters and search options

The `OgaGeneratorParams` class configures how text generation is performed. It provides methods to set search options like temperature, top-p, max length, and other parameters that control the generation process.

## Class Definition

```cpp theme={null}
struct OgaGeneratorParams : OgaAbstract {
  static std::unique_ptr<OgaGeneratorParams> Create(const OgaModel& model);
  
  void SetSearchOption(const char* name, double value);
  void SetSearchOptionBool(const char* name, bool value);
  void SetGuidance(const char* type, const char* data, bool enable_ff_tokens = false);
  
  double GetSearchNumber(const char* name) const;
  bool GetSearchBool(const char* name) const;
};
```

Defined in: `~/workspace/source/src/ort_genai.h:412`

## Methods

### Create()

Create generator parameters for a model.

```cpp theme={null}
static std::unique_ptr<OgaGeneratorParams> Create(const OgaModel& model)
```

<ParamField path="model" type="const OgaModel&" required>
  The model to create parameters for
</ParamField>

**Returns:** `std::unique_ptr<OgaGeneratorParams>` - A unique pointer to the created parameters

**Throws:** `std::runtime_error` if parameter creation fails

#### Example

```cpp theme={null}
auto model = OgaModel::Create("phi-2");
auto params = OgaGeneratorParams::Create(*model);
```

### SetSearchOption()

Set a numeric search option for generation.

```cpp theme={null}
void SetSearchOption(const char* name, double value)
```

<ParamField path="name" type="const char*" required>
  The name of the search option
</ParamField>

<ParamField path="value" type="double" required>
  The value to set
</ParamField>

**Throws:** `std::runtime_error` if the option name is invalid or value is out of range

#### Common Search Options

| Option                 | Type  | Description                                         | Default |
| ---------------------- | ----- | --------------------------------------------------- | ------- |
| `max_length`           | int   | Maximum total sequence length (prompt + generation) | 2048    |
| `min_length`           | int   | Minimum sequence length                             | 0       |
| `batch_size`           | int   | Number of sequences to generate                     | 1       |
| `temperature`          | float | Sampling temperature (higher = more random)         | 1.0     |
| `top_p`                | float | Nucleus sampling threshold                          | 1.0     |
| `top_k`                | int   | Top-K sampling threshold                            | 50      |
| `repetition_penalty`   | float | Penalty for repeated tokens                         | 1.0     |
| `length_penalty`       | float | Penalty for sequence length                         | 1.0     |
| `num_beams`            | int   | Number of beams for beam search                     | 1       |
| `num_return_sequences` | int   | Number of sequences to return                       | 1       |
| `diversity_penalty`    | float | Penalty for diverse beam search                     | 0.0     |

#### Example

```cpp theme={null}
auto params = OgaGeneratorParams::Create(*model);

// Set maximum generation length
params->SetSearchOption("max_length", 200);

// Set batch size
params->SetSearchOption("batch_size", 1);

// Configure sampling
params->SetSearchOption("temperature", 0.7);
params->SetSearchOption("top_p", 0.9);
params->SetSearchOption("top_k", 40);

// Set repetition penalty
params->SetSearchOption("repetition_penalty", 1.1);
```

From `~/workspace/source/src/ort_genai.h:30`

### SetSearchOptionBool()

Set a boolean search option.

```cpp theme={null}
void SetSearchOptionBool(const char* name, bool value)
```

<ParamField path="name" type="const char*" required>
  The name of the boolean search option
</ParamField>

<ParamField path="value" type="bool" required>
  The boolean value to set
</ParamField>

#### Common Boolean Options

| Option                      | Description                                   | Default |
| --------------------------- | --------------------------------------------- | ------- |
| `do_sample`                 | Enable sampling (vs greedy decoding)          | false   |
| `early_stopping`            | Stop beam search when enough candidates found | false   |
| `past_present_share_buffer` | Share KV cache buffers for memory efficiency  | true    |

#### Example

```cpp theme={null}
// Enable sampling
params->SetSearchOptionBool("do_sample", true);

// Enable early stopping for beam search
params->SetSearchOptionBool("early_stopping", true);
```

### SetGuidance()

Set guidance for constrained generation (e.g., JSON schema, function calling).

```cpp theme={null}
void SetGuidance(const char* type, const char* data, bool enable_ff_tokens = false)
```

<ParamField path="type" type="const char*" required>
  The guidance type (e.g., "json", "regex")
</ParamField>

<ParamField path="data" type="const char*" required>
  The guidance data (e.g., JSON schema, regex pattern)
</ParamField>

<ParamField path="enable_ff_tokens" type="bool">
  Enable fast-forward tokens optimization (default: false)
</ParamField>

**Throws:** `std::runtime_error` if guidance setup fails

#### Example

```cpp theme={null}
// JSON schema guidance for structured output
const char* json_schema = R"({
  "type": "object",
  "properties": {
    "name": {"type": "string"},
    "age": {"type": "number"}
  },
  "required": ["name", "age"]
})";

params->SetGuidance("json", json_schema);
```

From `~/workspace/source/examples/c/src/model_qa.cpp:99`

### GetSearchNumber()

Get the current value of a numeric search option.

```cpp theme={null}
double GetSearchNumber(const char* name) const
```

<ParamField path="name" type="const char*" required>
  The name of the search option
</ParamField>

**Returns:** `double` - The current value of the option

**Throws:** `std::runtime_error` if the option name is invalid

#### Example

```cpp theme={null}
double max_length = params->GetSearchNumber("max_length");
std::cout << "Max length: " << max_length << std::endl;
```

### GetSearchBool()

Get the current value of a boolean search option.

```cpp theme={null}
bool GetSearchBool(const char* name) const
```

<ParamField path="name" type="const char*" required>
  The name of the boolean search option
</ParamField>

**Returns:** `bool` - The current value of the option

**Throws:** `std::runtime_error` if the option name is invalid

#### Example

```cpp theme={null}
bool do_sample = params->GetSearchBool("do_sample");
std::cout << "Sampling enabled: " << (do_sample ? "yes" : "no") << std::endl;
```

## Generation Strategies

### Greedy Decoding (Default)

Simply select the most likely token at each step.

```cpp theme={null}
auto params = OgaGeneratorParams::Create(*model);
params->SetSearchOption("max_length", 100);
// No additional options needed - greedy is default
```

### Sampling

Randomly sample from the probability distribution.

```cpp theme={null}
auto params = OgaGeneratorParams::Create(*model);
params->SetSearchOptionBool("do_sample", true);
params->SetSearchOption("temperature", 0.8);
params->SetSearchOption("top_p", 0.95);
params->SetSearchOption("top_k", 50);
```

### Beam Search

Explore multiple hypotheses in parallel.

```cpp theme={null}
auto params = OgaGeneratorParams::Create(*model);
params->SetSearchOption("num_beams", 5);
params->SetSearchOption("num_return_sequences", 3);
params->SetSearchOptionBool("early_stopping", true);
```

### Diverse Beam Search

Generate diverse outputs using beam search.

```cpp theme={null}
auto params = OgaGeneratorParams::Create(*model);
params->SetSearchOption("num_beams", 5);
params->SetSearchOption("num_return_sequences", 3);
params->SetSearchOption("diversity_penalty", 1.0);
```

## Complete Examples

### Basic Configuration

From `~/workspace/source/src/ort_genai.h:29`:

```cpp theme={null}
#include "ort_genai.h"
#include <iostream>

int main() {
  OgaHandle handle;
  
  try {
    auto model = OgaModel::Create("phi-2");
    auto tokenizer = OgaTokenizer::Create(*model);
    
    // Create and configure generation parameters
    auto params = OgaGeneratorParams::Create(*model);
    params->SetSearchOption("max_length", 200);
    params->SetSearchOption("batch_size", 1);
    
    // Encode prompt
    auto sequences = OgaSequences::Create();
    tokenizer->Encode("A great recipe for Kung Pao chicken is ", *sequences);
    
    // Create generator with parameters
    auto generator = OgaGenerator::Create(*model, *params);
    generator->AppendTokenSequences(*sequences);
    
    // Generate...
    while (!generator->IsDone()) {
      generator->GenerateNextToken();
    }
    
    // Decode output
    auto output_sequence = generator->GetSequenceData(0);
    auto output_length = generator->GetSequenceCount(0);
    auto output_string = tokenizer->Decode(output_sequence, output_length);
    
    std::cout << "Output: " << output_string << std::endl;
    
  } catch (const std::exception& e) {
    std::cerr << "Error: " << e.what() << std::endl;
    return -1;
  }
  
  return 0;
}
```

### Advanced Sampling Configuration

```cpp theme={null}
// Configure advanced sampling with temperature, top-p, and penalties
auto params = OgaGeneratorParams::Create(*model);

// Basic length constraints
params->SetSearchOption("max_length", 512);
params->SetSearchOption("min_length", 50);

// Enable sampling
params->SetSearchOptionBool("do_sample", true);

// Sampling parameters
params->SetSearchOption("temperature", 0.7);  // Lower = more focused
params->SetSearchOption("top_p", 0.9);        // Nucleus sampling
params->SetSearchOption("top_k", 40);         // Limit to top 40 tokens

// Penalties to reduce repetition
params->SetSearchOption("repetition_penalty", 1.2);
params->SetSearchOption("length_penalty", 1.0);

// Create generator
auto generator = OgaGenerator::Create(*model, *params);
```

### Constrained Generation with JSON Schema

From `~/workspace/source/examples/c/src/model_qa.cpp:98`:

```cpp theme={null}
// Define JSON schema for structured output
const char* json_schema = R"({
  "type": "object",
  "properties": {
    "answer": {"type": "string"},
    "confidence": {"type": "number", "minimum": 0, "maximum": 1},
    "sources": {
      "type": "array",
      "items": {"type": "string"}
    }
  },
  "required": ["answer", "confidence"]
})";

// Configure parameters with guidance
auto params = OgaGeneratorParams::Create(*model);
params->SetSearchOption("max_length", 500);
params->SetGuidance("json", json_schema);

// Generate with schema constraints
auto generator = OgaGenerator::Create(*model, *params);
// ... generation code ...
```

### Multi-Sequence Beam Search

```cpp theme={null}
// Configure beam search to return multiple diverse outputs
auto params = OgaGeneratorParams::Create(*model);

params->SetSearchOption("max_length", 200);
params->SetSearchOption("num_beams", 5);              // Use 5 beams
params->SetSearchOption("num_return_sequences", 3);   // Return top 3
params->SetSearchOption("diversity_penalty", 0.5);    // Encourage diversity
params->SetSearchOptionBool("early_stopping", true);  // Stop when done

auto generator = OgaGenerator::Create(*model, *params);
generator->AppendTokenSequences(*sequences);

while (!generator->IsDone()) {
  generator->GenerateNextToken();
}

// Get all returned sequences
for (size_t i = 0; i < 3; ++i) {
  auto seq_data = generator->GetSequenceData(i);
  auto seq_length = generator->GetSequenceCount(i);
  auto text = tokenizer->Decode(seq_data, seq_length);
  std::cout << "Sequence " << i << ": " << text << "\n\n";
}
```

## Parameter Validation

The library validates parameters when creating the generator. Invalid values will throw exceptions:

```cpp theme={null}
try {
  auto params = OgaGeneratorParams::Create(*model);
  
  // This might throw if value is invalid
  params->SetSearchOption("temperature", -1.0);  // Temperature must be > 0
  
  auto generator = OgaGenerator::Create(*model, *params);
} catch (const std::runtime_error& e) {
  std::cerr << "Invalid parameter: " << e.what() << std::endl;
}
```

## See Also

* [OgaGenerator](/api/cpp/generator) - Use parameters for text generation
* [OgaModel](/api/cpp/model) - Create parameters from models
* [Search Options Reference](~/workspace/source/src/ort_genai.h:419) - Full list of available options