OgaModel

Load and manage ONNX Runtime GenAI models

The OgaModel class is the core interface for loading and managing generative AI models in ONNX Runtime GenAI. It provides methods to create model instances from configuration files or custom configurations.

Class Definition

struct OgaModel : OgaAbstract {
  static std::unique_ptr<OgaModel> Create(const char* config_path);
  static std::unique_ptr<OgaModel> Create(const char* config_path, const OgaRuntimeSettings& settings);
  static std::unique_ptr<OgaModel> Create(const OgaConfig& config);
  
  OgaString GetType() const;
  OgaString GetDeviceType() const;
};

Defined in: ~/workspace/source/src/ort_genai.h:227

Methods

Create()

Create a new model instance from a configuration path.

static std::unique_ptr<OgaModel> Create(const char* config_path)

config_path · const char* · required

Path to the model configuration directory containing the model files and genai_config.json

Returns: std::unique_ptr<OgaModel> - A unique pointer to the created model instance

Throws: std::runtime_error if the model cannot be loaded

Example

// Load a model from a configuration path
auto model = OgaModel::Create("phi-2");

Create() with Runtime Settings

Create a model with custom runtime settings.

static std::unique_ptr<OgaModel> Create(const char* config_path, const OgaRuntimeSettings& settings)

config_path · const char* · required

Path to the model configuration directory

settings · const OgaRuntimeSettings& · required

Runtime settings for the model

Returns: std::unique_ptr<OgaModel> - A unique pointer to the created model instance

Example

// Create runtime settings
auto settings = OgaRuntimeSettings::Create();
settings->SetHandle("custom_handle", handle_ptr);

// Load model with settings
auto model = OgaModel::Create("phi-2", *settings);

Create() from Config

Create a model from a custom configuration object.

static std::unique_ptr<OgaModel> Create(const OgaConfig& config)

config · const OgaConfig& · required

Custom configuration object for the model

Returns: std::unique_ptr<OgaModel> - A unique pointer to the created model instance

Example

// Create a custom config
auto config = OgaConfig::Create("phi-2");
config->AppendProvider("cuda");
config->SetProviderOption("cuda", "device_id", "0");

// Load model from config
auto model = OgaModel::Create(*config);

GetType()

Get the model type.

OgaString GetType() const

Returns: OgaString - The model type (e.g., “gpt2”, “llama”)

Example

auto model = OgaModel::Create("phi-2");
const char* model_type = model->GetType();
std::cout << "Model type: " << model_type << std::endl;

GetDeviceType()

Get the device type the model is running on.

OgaString GetDeviceType() const

Returns: OgaString - The device type (e.g., “cpu”, “cuda”, “dml”)

Example

auto model = OgaModel::Create("phi-2");
const char* device = model->GetDeviceType();
std::cout << "Running on: " << device << std::endl;

Complete Example

Here’s a complete example demonstrating model creation and basic usage:

#include "ort_genai.h"
#include <iostream>

int main() {
  // Responsible for cleaning up the library during shutdown
  OgaHandle handle;
  
  try {
    // Create model from configuration path
    auto model = OgaModel::Create("phi-2");
    
    // Get model information
    std::cout << "Model type: " << model->GetType() << std::endl;
    std::cout << "Device type: " << model->GetDeviceType() << std::endl;
    
    // Create tokenizer and generator from the model
    auto tokenizer = OgaTokenizer::Create(*model);
    auto params = OgaGeneratorParams::Create(*model);
    auto generator = OgaGenerator::Create(*model, *params);
    
    // Use the model for generation...
    
  } catch (const std::exception& e) {
    std::cerr << "Error: " << e.what() << std::endl;
    return -1;
  }
  
  return 0;
}

Resource Management

The OgaModel class uses RAII (Resource Acquisition Is Initialization) principles:

• Models are returned as std::unique_ptr, providing automatic memory management
• The model is automatically destroyed when the unique pointer goes out of scope
• No manual cleanup is required

See Also

• OgaTokenizer - Create tokenizers from models
• OgaGeneratorParams - Configure generation parameters
• OgaGenerator - Generate text using the model
• OgaConfig - Advanced model configuration