> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/microsoft/onnxruntime-genai/llms.txt
> Use this file to discover all available pages before exploring further.

# Installation

> Install ONNX Runtime GenAI for Python, C#, or C++

# Installation

ONNX Runtime GenAI can be installed via package managers for Python and C#, or downloaded as binaries for C++. Choose the installation method that matches your development environment.

## System Requirements

Before installing, ensure your system meets these requirements:

<Tabs>
  <Tab title="General">
    * **Operating System**: Windows, Linux, or macOS
    * **Architecture**: x64, x86, or arm64
    * **Python**: 3.8 or later (for Python API)
    * **.NET**: .NET 8.0 or later (for C# API)
    * **C++ Compiler**: MSVC, GCC, or Clang (for C++ API)
  </Tab>

  <Tab title="GPU Acceleration">
    * **CUDA**: CUDA 11.8+ and cuDNN 8.x for NVIDIA GPUs
    * **DirectML**: Windows 10 version 1903+ for DirectX 12 GPUs
    * **TensorRT**: TensorRT 8.6+ for NVIDIA GPUs
    * **OpenVINO**: OpenVINO 2024.0+ for Intel hardware
  </Tab>
</Tabs>

## Python Installation

<Steps>
  <Step title="Install NumPy">
    NumPy is required for tensor operations:

    ```bash theme={null}
    pip install numpy
    ```
  </Step>

  <Step title="Install ONNX Runtime GenAI">
    Install the latest stable release:

    <CodeGroup>
      ```bash CPU theme={null}
      pip install onnxruntime-genai
      ```

      ```bash CUDA theme={null}
      pip install onnxruntime-genai-cuda
      ```

      ```bash DirectML (Windows) theme={null}
      pip install onnxruntime-genai-directml
      ```
    </CodeGroup>

    <Tip>
      Use `--pre` flag to install pre-release versions:

      ```bash theme={null}
      pip install --pre onnxruntime-genai
      ```
    </Tip>
  </Step>

  <Step title="Verify Installation">
    Verify the installation by checking the version:

    ```bash theme={null}
    pip list | grep onnxruntime-genai
    ```

    Or in Python:

    ```python theme={null}
    import onnxruntime_genai as og
    print("ONNX Runtime GenAI installed successfully")
    ```
  </Step>
</Steps>

### Nightly Builds (Python)

To install the latest nightly build with cutting-edge features:

```bash theme={null}
pip install --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ORT-Nightly/pypi/simple/ onnxruntime-genai
```

<Warning>
  Nightly builds are experimental and may be unstable. Use stable releases for production applications.
</Warning>

## C# Installation

<Steps>
  <Step title="Add NuGet Package">
    Add the ONNX Runtime GenAI package to your project:

    <CodeGroup>
      ```bash .NET CLI theme={null}
      dotnet add package Microsoft.ML.OnnxRuntimeGenAI
      ```

      ```xml Package Reference theme={null}
      <PackageReference Include="Microsoft.ML.OnnxRuntimeGenAI" Version="0.12.0" />
      ```

      ```bash NuGet PM Console theme={null}
      Install-Package Microsoft.ML.OnnxRuntimeGenAI
      ```
    </CodeGroup>
  </Step>

  <Step title="Choose Execution Provider (Optional)">
    For GPU acceleration, use the appropriate package:

    <Tabs>
      <Tab title="CUDA">
        ```xml theme={null}
        <PackageReference Include="Microsoft.ML.OnnxRuntimeGenAI.Cuda" Version="0.12.0" />
        ```
      </Tab>

      <Tab title="DirectML">
        ```xml theme={null}
        <PackageReference Include="Microsoft.ML.OnnxRuntimeGenAI.DirectML" Version="0.12.0" />
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Verify Installation">
    Add this to your C# code to verify:

    ```csharp theme={null}
    using Microsoft.ML.OnnxRuntimeGenAI;

    Console.WriteLine("ONNX Runtime GenAI installed successfully");
    ```
  </Step>
</Steps>

<Info>
  The C# package supports .NET 8.0, .NET Standard 2.0, and mobile platforms (Android, iOS, Mac Catalyst).
</Info>

## C++ Installation

<Steps>
  <Step title="Download Binaries">
    Download the pre-built binaries for your platform from the [GitHub Releases](https://github.com/microsoft/onnxruntime-genai/releases) page.

    Choose the appropriate package:

    * **Windows**: `onnxruntime-genai-win-x64-{version}.zip`
    * **Linux**: `onnxruntime-genai-linux-x64-{version}.tar.gz`
    * **macOS**: `onnxruntime-genai-osx-{arch}-{version}.tar.gz`
  </Step>

  <Step title="Extract and Configure">
    <Tabs>
      <Tab title="Windows">
        1. Extract the archive to your desired location
        2. Add the `bin` directory to your PATH
        3. Link against the library in your CMake or Visual Studio project:

        ```cmake theme={null}
        find_package(onnxruntime-genai REQUIRED)
        target_link_libraries(your_app onnxruntime-genai)
        ```
      </Tab>

      <Tab title="Linux">
        1. Extract the archive:

        ```bash theme={null}
        tar -xzf onnxruntime-genai-linux-x64-{version}.tar.gz
        ```

        2. Set library path:

        ```bash theme={null}
        export LD_LIBRARY_PATH=/path/to/onnxruntime-genai/lib:$LD_LIBRARY_PATH
        ```

        3. Link in your build system:

        ```cmake theme={null}
        find_package(onnxruntime-genai REQUIRED)
        target_link_libraries(your_app onnxruntime-genai)
        ```
      </Tab>

      <Tab title="macOS">
        1. Extract the archive:

        ```bash theme={null}
        tar -xzf onnxruntime-genai-osx-{arch}-{version}.tar.gz
        ```

        2. Set library path:

        ```bash theme={null}
        export DYLD_LIBRARY_PATH=/path/to/onnxruntime-genai/lib:$DYLD_LIBRARY_PATH
        ```

        3. Link in your build system:

        ```cmake theme={null}
        find_package(onnxruntime-genai REQUIRED)
        target_link_libraries(your_app onnxruntime-genai)
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Verify Installation">
    Create a simple test program:

    ```cpp theme={null}
    #include <onnxruntime_genai.h>
    #include <iostream>

    int main() {
        std::cout << "ONNX Runtime GenAI installed successfully" << std::endl;
        return 0;
    }
    ```
  </Step>
</Steps>

### Build from Source (C++)

For advanced users who need custom builds:

<CodeGroup>
  ```bash Python Wheel theme={null}
  python build.py
  ```

  ```bash Full Build theme={null}
  python build.py --build_dir build --parallel
  ```
</CodeGroup>

See the [build from source guide](https://onnxruntime.ai/docs/genai/howto/build-from-source.html) for detailed instructions.

## Platform-Specific Notes

<AccordionGroup>
  <Accordion title="Windows">
    * Visual C++ Redistributable 2019 or later is required
    * For DirectML, ensure Windows 10 version 1903 or later
    * CUDA builds require CUDA toolkit to be installed
  </Accordion>

  <Accordion title="Linux">
    * GLIBC 2.27 or later is required
    * For CUDA support, install CUDA 11.8+ and cuDNN
    * Ubuntu 20.04+ and CentOS 8+ are officially supported
  </Accordion>

  <Accordion title="macOS">
    * macOS 11.0 (Big Sur) or later
    * Both Intel (x64) and Apple Silicon (arm64) are supported
    * No GPU acceleration on macOS (CPU only)
  </Accordion>

  <Accordion title="Android">
    * Android API level 27 or higher
    * Available through .NET MAUI or build from source
    * QNN execution provider for Qualcomm devices
  </Accordion>
</AccordionGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Import Error (Python)">
    If you encounter import errors, ensure:

    1. NumPy is installed: `pip install numpy`
    2. You're using Python 3.8 or later
    3. The package matches your platform architecture (x64/arm64)

    ```bash theme={null}
    # Check Python version
    python --version

    # Reinstall with verbose output
    pip install --force-reinstall -v onnxruntime-genai
    ```
  </Accordion>

  <Accordion title="DLL/Shared Library Not Found">
    For C++ applications, ensure:

    * The library path is correctly set (LD\_LIBRARY\_PATH, PATH, or DYLD\_LIBRARY\_PATH)
    * All dependencies are installed (CUDA, cuDNN for GPU builds)
    * The architecture matches (x64 vs arm64)
  </Accordion>

  <Accordion title="Version Mismatch">
    If using examples from the repository, ensure they match your installed version:

    ```bash theme={null}
    # Get installed version
    pip list | grep onnxruntime-genai

    # Clone and checkout matching version
    git clone https://github.com/microsoft/onnxruntime-genai.git
    cd onnxruntime-genai
    git checkout v0.12.0  # Replace with your version
    ```
  </Accordion>
</AccordionGroup>

## Next Steps

<Card title="Quickstart Guide" icon="rocket" href="/quickstart">
  Now that you have ONNX Runtime GenAI installed, follow the quickstart guide to run your first model.
</Card>