Module inference

AIMX Inference Module

The inference module provides the core functionality for integrating AI inference capabilities into AIMX expressions and workflows. This module handles everything from prompt generation and provider communication to response parsing and validation.

Overview

The inference system enables AIMX expressions to trigger AI model calls through special syntax:

• $tool.function_name(args) - Inference calls to workflow nodes
• _variable_name - Assignment rules populated by AI inference
• UPPERCASE_IDENTIFIER - Modifier rules that construct prompts

The module is organized into several submodules that handle different aspects of the inference pipeline:

• item - Parse and represent inference data items (tasks, values, etc.)
• key - Parse inference keys and unique component identifiers
• prompt - Generate structured prompts for AI models
• provider - Configure and manage AI model providers
• request - Send inference requests to providers
• response - Parse and interpret AI model responses
• validate - Validate responses against workflow rules

Core Workflow

When an AIMX expression contains an inference call (e.g., $tool.summarize_email(text)), the system follows this pipeline:

1. Reference Resolution - The $tool prefix is resolved to a workflow node
2. Prompt Generation - Modifier rules from the workflow construct the prompt
3. Provider Selection - The appropriate AI model provider is selected
4. Request Construction - HTTP/REST requests are built for the provider API
5. Response Handling - Raw model responses are parsed and structured
6. Validation - Responses are validated against workflow rules
7. Assignment - Validated values populate assignment rules

Usage Examples

use aimx::inference::{Provider, Api, Model, Capability, send_request};
 
// Create a provider configuration for Ollama
let provider = Provider {
    api: Api::Ollama,
    url: "http://localhost:11434".to_string(),
    key: "".to_string(),  // No API key needed for local Ollama
    model: Model::Standard,
    capability: Capability::Standard,
    fast: "mistral:latest".to_string(),
    standard: "llama2:latest".to_string(),
    planning: "codellama:latest".to_string(),
    temperature: 0.7,
    max_tokens: 2048,
    connection_timeout_ms: 30000,
    request_timeout_ms: 120000,
};
 
// Send a request - this would typically be used internally by the inference system
// let response = send_request(&provider, "You are a helpful assistant", "Hello!");

Provider Support

The inference module supports multiple AI providers:

• OpenAI - GPT models via OpenAI API and compatible services like OpenRouter
• Ollama - Local models via Ollama API

Each provider can be configured with custom endpoints, API keys, and timeout settings.

Error Handling

The inference system provides comprehensive error handling:

• Provider Errors - Connection failures, rate limits, authentication issues
• Parsing Errors - Malformed responses, unexpected formats
• Validation Errors - Response validation failures against workflow rules
• Timeout Errors - Request timeouts and processing delays

All errors are returned as anyhow::Result types for easy integration with error handling systems.

Integration with AIMX Expressions

Inference calls are seamlessly integrated into AIMX expressions:

// Call an inference tool from an expression
summary: Text = $tool.summarize_email(email_body)

// Use inference results in calculations
sentiment_score: Number = $analyzer.get_sentiment(review) * 100

// Chain inference calls with other operations
classification: Text[] = $classifier.categorize(items).filter(cat => cat != "spam")

This integration allows AIMX workflows to leverage AI capabilities while maintaining the familiar expression syntax and evaluation semantics.

Re-exports

pub use item::Item;

pub use item::parse_inline_item;

pub use item::parse_item;

pub use key::Suffix;

pub use key::parse_key;

pub use key::parse_ucid;

pub use prompt::generate_prompt;

pub use provider::Api;

pub use provider::Model;

pub use provider::Capability;

pub use provider::Provider;

pub use request::send_request;

pub use response::Response;

pub use response::parse_response;

pub use validate::validate_responses;

Modules

item

Inference item parsing for Agentic Inference Markup (AIM) format.

key

Inference key parsing for Agentic Inference Markup (AIM) format.

prompt

Prompt generation for AI model inference

provider

Inference Provider Configuration

request

Inference request management for AI model providers

response

Inference response parsing and conversion

validate

Response validation for AIM (Agentic Inference Markup) workflows.