Model Functions

C API functions for model creation and configuration

The model functions provide the core capabilities for loading and configuring ONNX Runtime GenAI models.

Model Creation

OgaCreateModel · function

OgaResult* OGA_API_CALL OgaCreateModel(const char* config_path, OgaModel** out);

Creates a model from a configuration directory.

Parameters:

• config_path: Path to the model configuration directory (UTF-8 encoded)
• out: Pointer to store the created model

Returns: NULL on success, or OgaResult* containing error message on failure

Example:

OgaModel* model = NULL;
OgaResult* result = OgaCreateModel("/path/to/model", &model);
if (result != NULL) {
    fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
    OgaDestroyResult(result);
    return -1;
}

// Use the model...

OgaDestroyModel(model);

OgaCreateModelFromConfig · function

OgaResult* OGA_API_CALL OgaCreateModelFromConfig(const OgaConfig* config, OgaModel** out);

Creates a model from an OgaConfig object.

Parameters:

• config: Configuration object to use for the model
• out: Pointer to store the created model

Returns: NULL on success, or OgaResult* containing error message on failure

OgaCreateModelWithRuntimeSettings · function

OgaResult* OGA_API_CALL OgaCreateModelWithRuntimeSettings(
    const char* config_path,
    const OgaRuntimeSettings* settings,
    OgaModel** out
);

Creates a model with runtime settings and device configuration.

Parameters:

• config_path: Path to the model configuration directory (UTF-8 encoded)
• settings: Runtime settings to use for the model
• out: Pointer to store the created model

Returns: NULL on success, or OgaResult* containing error message on failure

OgaDestroyModel · function

void OGA_API_CALL OgaDestroyModel(OgaModel* model);

Destroys a model and frees its resources.

Parameters:

• model: The model to destroy

Model Properties

OgaModelGetType · function

OgaResult* OGA_API_CALL OgaModelGetType(const OgaModel* model, const char** out);

Returns the type of the model (e.g., “gpt2”, “llama”).

Parameters:

• model: The model to query
• out: Pointer to store the model type string

Returns: NULL on success, or OgaResult* containing error message on failure

Note: The returned string must be destroyed with OgaDestroyString()

Example:

const char* model_type = NULL;
OgaResult* result = OgaModelGetType(model, &model_type);
if (result == NULL) {
    printf("Model type: %s\n", model_type);
    OgaDestroyString(model_type);
}

OgaModelGetDeviceType · function

OgaResult* OGA_API_CALL OgaModelGetDeviceType(const OgaModel* model, const char** out);

Returns the device type of the model (e.g., “CPU”, “CUDA”, “DML”).

Parameters:

• model: The model to query
• out: Pointer to store the device type string

Returns: NULL on success, or OgaResult* containing error message on failure

Note: The returned string must be destroyed with OgaDestroyString()

Configuration Objects

OgaConfig

OgaCreateConfig · function

OgaResult* OGA_API_CALL OgaCreateConfig(const char* config_path, OgaConfig** out);

Creates a configuration object from a configuration directory.

Parameters:

• config_path: Path to the configuration directory (UTF-8 encoded)
• out: Pointer to store the created config

Returns: NULL on success, or OgaResult* containing error message on failure

OgaDestroyConfig · function

void OGA_API_CALL OgaDestroyConfig(OgaConfig* config);

Destroys a configuration object.

Parameters:

• config: The config to destroy

OgaConfigOverlay · function

OgaResult* OGA_API_CALL OgaConfigOverlay(OgaConfig* config, const char* json);

Overlays JSON configuration on top of the existing config.

Parameters:

• config: The config to modify
• json: JSON string to overlay

Returns: NULL on success, or OgaResult* containing error message on failure

Example:

const char* json_overlay = "{\"max_length\": 200}";
OgaResult* result = OgaConfigOverlay(config, json_overlay);
if (result != NULL) {
    fprintf(stderr, "Failed to apply overlay: %s\n", OgaResultGetError(result));
    OgaDestroyResult(result);
}

Provider Configuration

OgaConfigClearProviders · function

OgaResult* OGA_API_CALL OgaConfigClearProviders(OgaConfig* config);

Clears the list of execution providers in the configuration.

Parameters:

• config: The config to modify

Returns: NULL on success, or OgaResult* containing error message on failure

OgaConfigAppendProvider · function

OgaResult* OGA_API_CALL OgaConfigAppendProvider(OgaConfig* config, const char* provider);

Adds a provider to the end of the provider list. If the provider already exists, this function does nothing.

Parameters:

• config: The config to modify
• provider: The provider name to add (e.g., “CPU”, “CUDA”, “DML”)

Returns: NULL on success, or OgaResult* containing error message on failure

Example:

// Add CUDA provider
OgaResult* result = OgaConfigAppendProvider(config, "CUDA");
if (result != NULL) {
    fprintf(stderr, "Failed to add provider: %s\n", OgaResultGetError(result));
    OgaDestroyResult(result);
}

OgaConfigSetProviderOption · function

OgaResult* OGA_API_CALL OgaConfigSetProviderOption(
    OgaConfig* config,
    const char* provider,
    const char* key,
    const char* value
);

Sets a provider-specific option.

Parameters:

• config: The config to modify
• provider: The provider name
• key: The option key
• value: The option value

Returns: NULL on success, or OgaResult* containing error message on failure

Hardware Device Filtering

OgaConfigSetDecoderProviderOptionsHardwareDeviceType · function

OgaResult* OGA_API_CALL OgaConfigSetDecoderProviderOptionsHardwareDeviceType(
    OgaConfig* config,
    const char* provider,
    const char* hardware_device_type
);

Filters execution provider devices by hardware device type (e.g., “CPU”, “GPU”, “NPU”).

Parameters:

• config: The config to modify
• provider: The provider name
• hardware_device_type: The hardware device type

Returns: NULL on success, or OgaResult* containing error message on failure

OgaConfigSetDecoderProviderOptionsHardwareDeviceId · function

OgaResult* OGA_API_CALL OgaConfigSetDecoderProviderOptionsHardwareDeviceId(
    OgaConfig* config,
    const char* provider,
    uint32_t hardware_device_id
);

Filters execution provider devices by hardware device ID.

Parameters:

• config: The config to modify
• provider: The provider name
• hardware_device_id: The hardware device ID

Returns: NULL on success, or OgaResult* containing error message on failure

OgaConfigSetDecoderProviderOptionsHardwareVendorId · function

OgaResult* OGA_API_CALL OgaConfigSetDecoderProviderOptionsHardwareVendorId(
    OgaConfig* config,
    const char* provider,
    uint32_t hardware_vendor_id
);

Filters execution provider devices by hardware vendor ID.

Parameters:

• config: The config to modify
• provider: The provider name
• hardware_vendor_id: The hardware vendor ID

Returns: NULL on success, or OgaResult* containing error message on failure

OgaConfigClearDecoderProviderOptionsHardwareDeviceType · function

OgaResult* OGA_API_CALL OgaConfigClearDecoderProviderOptionsHardwareDeviceType(
    OgaConfig* config,
    const char* provider
);

Clears the hardware device type filter.

OgaConfigClearDecoderProviderOptionsHardwareDeviceId · function

OgaResult* OGA_API_CALL OgaConfigClearDecoderProviderOptionsHardwareDeviceId(
    OgaConfig* config,
    const char* provider
);

Clears the hardware device ID filter.

OgaConfigClearDecoderProviderOptionsHardwareVendorId · function

OgaResult* OGA_API_CALL OgaConfigClearDecoderProviderOptionsHardwareVendorId(
    OgaConfig* config,
    const char* provider
);

Clears the hardware vendor ID filter.

Model Data from Memory

OgaConfigAddModelData · function

OgaResult* OGA_API_CALL OgaConfigAddModelData(
    OgaConfig* config,
    const char* model_filename,
    const void* model_data,
    size_t model_data_length
);

Adds model data to load the model from memory instead of from disk.

Parameters:

• config: The config to modify
• model_filename: The name of the model file as defined in the config
• model_data: Pointer to the model data (must remain valid until model is created)
• model_data_length: Length of the model data in bytes

Returns: NULL on success, or OgaResult* containing error message on failure

Note: The model data must remain valid at least until the model is created. If using session.use_ort_model_bytes_directly, the data must remain valid until the model is destroyed.

OgaConfigRemoveModelData · function

OgaResult* OGA_API_CALL OgaConfigRemoveModelData(
    OgaConfig* config,
    const char* model_filename
);

Removes previously added model data.

Parameters:

• config: The config to modify
• model_filename: The name of the model file to remove

Returns: NULL on success, or OgaResult* containing error message on failure

Runtime Settings

OgaCreateRuntimeSettings · function

OgaResult* OGA_API_CALL OgaCreateRuntimeSettings(OgaRuntimeSettings** out);

Creates a runtime settings object.

Parameters:

• out: Pointer to store the created runtime settings

Returns: NULL on success, or OgaResult* containing error message on failure

OgaDestroyRuntimeSettings · function

void OGA_API_CALL OgaDestroyRuntimeSettings(OgaRuntimeSettings* settings);

Destroys a runtime settings object.

Parameters:

• settings: The runtime settings to destroy

OgaRuntimeSettingsSetHandle · function

OgaResult* OGA_API_CALL OgaRuntimeSettingsSetHandle(
    OgaRuntimeSettings* settings,
    const char* handle_name,
    void* handle
);

Sets a runtime handle (e.g., device context) for the runtime settings.

Parameters:

• settings: The runtime settings to modify
• handle_name: The name of the handle to set
• handle: Pointer to the handle value

Returns: NULL on success, or OgaResult* containing error message on failure

Complete Example

#include "ort_genai_c.h"
#include <stdio.h>

int main() {
    // Create configuration
    OgaConfig* config = NULL;
    OgaResult* result = OgaCreateConfig("/path/to/model", &config);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        return -1;
    }
    
    // Configure providers
    result = OgaConfigClearProviders(config);
    if (result == NULL) {
        result = OgaConfigAppendProvider(config, "CUDA");
    }
    if (result == NULL) {
        result = OgaConfigAppendProvider(config, "CPU");
    }
    if (result != NULL) {
        fprintf(stderr, "Error configuring providers: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroyConfig(config);
        return -1;
    }
    
    // Create model from config
    OgaModel* model = NULL;
    result = OgaCreateModelFromConfig(config, &model);
    if (result != NULL) {
        fprintf(stderr, "Error creating model: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroyConfig(config);
        return -1;
    }
    
    // Get model properties
    const char* model_type = NULL;
    const char* device_type = NULL;
    
    result = OgaModelGetType(model, &model_type);
    if (result == NULL) {
        printf("Model type: %s\n", model_type);
        OgaDestroyString(model_type);
    }
    
    result = OgaModelGetDeviceType(model, &device_type);
    if (result == NULL) {
        printf("Device type: %s\n", device_type);
        OgaDestroyString(device_type);
    }
    
    // Use the model for generation...
    
    // Cleanup
    OgaDestroyModel(model);
    OgaDestroyConfig(config);
    
    return 0;
}

See Also

C API Overview

Learn about memory management and error handling

Generator Functions

Generate text with your loaded model