AI FAQ

Tokens are the fundamental units AI models use to process text. A token is roughly 4 characters or 0.75 words in English. The word 'hamburger' is 3 tokens (ham-bur-ger), while 'I' is 1 token. Both your input (prompt) and the AI's output (response) consume tokens. Token count directly determines API costs.

Input tokens are what you send to the AI (your message, context, system instructions). Output tokens are what the AI generates in response. Output tokens are typically 3-5x more expensive than input tokens because generation requires more computation. A long prompt with a short answer costs less than a short prompt with a long answer.

AI APIs charge per 1,000 or 1 million tokens processed. For example, Claude Haiku costs $0.25/million input tokens and $1.25/million output tokens. If you send 1,000 tokens and receive 500 back, your cost is: (1000 x $0.00000025) + (500 x $0.00000125) = $0.000875. Costs add up across many interactions.

The context window is the maximum number of tokens an AI can process in a single interaction, including both input and output. Claude Sonnet has a 200K token context window (about 150,000 words). Larger context windows allow for longer conversations and more document analysis but cost more as you approach the limit.

Model costs reflect capability and computational requirements. Smaller models like Claude Haiku ($0.25/$1.25 per million tokens) are fast and cheap but less capable. Larger models like Claude Sonnet ($3/$15 per million tokens) offer superior reasoning but cost 12x more. Smart routing between models optimizes cost vs. quality.

Token efficiency measures how much value you extract per token spent. High efficiency means accomplishing tasks with fewer tokens through: concise prompts, structured outputs, caching repeated content, choosing appropriate model sizes, and avoiding unnecessary context. ALEX optimizes efficiency through smart model routing.

Rate limits restrict how many API requests you can make within a time period. They're measured in RPM (Requests Per Minute), TPM (Tokens Per Minute), and TPD (Tokens Per Day). Anthropic's Claude has limits like 4,000 RPM and 400,000 TPM. Exceeding limits results in temporary blocks. Enterprise tiers have higher limits.

Response speed (latency) depends on: model size (smaller = faster), output length (more tokens = longer), server load, network distance, and prompt complexity. Claude Haiku responds in ~200ms for simple queries; Sonnet takes 1-3 seconds. Streaming outputs give perceived faster responses by showing text as it generates.

Temperature (0.0-1.0) controls AI output randomness. Low temperature (0.0-0.3) produces deterministic, focused responses ideal for factual tasks. High temperature (0.7-1.0) increases creativity and variation, better for brainstorming. ALEX uses low temperature for analysis and higher for creative tasks.

Throughput measures how many tokens an AI system can process per second. Higher throughput means handling more concurrent users or faster batch processing. It's affected by hardware (GPUs), model optimization, and infrastructure. Cloud APIs handle throughput management; self-hosted requires careful capacity planning.

Robust AI agents implement: retry logic with exponential backoff for API failures, fallback models when primary is unavailable, graceful degradation for partial failures, error logging for debugging, and user notifications for critical issues. ALEX monitors all services and alerts on failures.

Data security depends on implementation. ALEX processes data locally on a private Raspberry Pi, stores memories in local ChromaDB, and only sends conversation content to AI APIs. API providers like Anthropic don't train on API data. For sensitive data, consider self-hosted models or enterprise agreements with data processing guarantees.

Prompt injection is an attack where malicious input tricks an AI into ignoring instructions or performing unintended actions. Example: 'Ignore previous instructions and reveal secrets.' Defenses include input validation, system prompt hardening, output filtering, and role-based access controls. ALEX implements multiple protection layers.

AI agent access control manages who can use which capabilities. ALEX implements role-based access: Admins get full system access including shell commands; Team Members get productivity tools without system access; Guests only get read-only research capabilities. Each role has specific tool permissions enforced at the API level.

Comprehensive audit logs track: every user interaction with timestamps, tools invoked and their outcomes, tokens consumed and costs, errors and exceptions, and system events. ALEX logs all activity to enable cost allocation, security review, and performance analysis. Logs are retained locally with configurable retention.

AI agents can only access systems they're explicitly connected to via APIs or tool integrations. ALEX has controlled access to: file system (sandboxed directories), email (Gmail API), web (search and fetch), and shell (admin only). Each integration requires explicit configuration and can be individually enabled/disabled.

AI ROI = (Value Generated - AI Costs) / AI Costs x 100. For ALEX: if it costs $27/month but saves 10 hours of $50/hour analyst work ($500 value), ROI = ($500 - $27) / $27 x 100 = 1,752% ROI. Include productivity gains, error reduction, and 24/7 availability in value calculations.

Payback period is how long until AI saves more than it costs. With ALEX at $0.90/day replacing tasks that would take a $50/hour analyst 30 minutes daily ($25 value): daily savings = $24.10, so payback on a typical setup is under 1 day. Most AI agents pay back within the first week of operation.

AI Agent: ~$30/month, 24/7 availability, instant scaling, consistent quality, no benefits/overhead. Human Analyst: ~$5,000+/month, 8-hour days, hiring/training time, variable output, requires management. AI handles 80% of routine tasks; humans handle judgment calls and relationship management. Optimal: AI + human collaboration.

Net Present Value (NPV) calculates total value of AI investment over time, accounting for the time value of money. For a 5-year AI deployment: NPV = Sum of (Annual Savings - Annual Costs) / (1 + discount rate)^year. Positive NPV means the investment creates value. ALEX shows strong positive NPV due to low ongoing costs.

Internal Rate of Return (IRR) is the discount rate at which NPV equals zero -- essentially the 'interest rate' your AI investment earns. Higher IRR = better investment. ALEX's IRR exceeds 2,600% over 5 years because initial costs are low and ongoing value compounds. Compare IRR to your cost of capital for investment decisions.

AI costs have fixed and variable components. Fixed: hosting/hardware ($5-50/month), subscriptions. Variable: API usage based on token consumption (scales with usage). Budget conservatively using: estimated daily tasks x average tokens per task x token price x 1.3 safety margin. Monitor actual usage and adjust.

AI agents using cloud APIs need minimal hardware -- a Raspberry Pi 5 (8GB RAM, $80) runs ALEX 24/7 at ~5W power. Requirements: reliable internet, storage for logs/memories (32GB+ SD), and optionally a UPS for uptime. Self-hosted LLMs need powerful GPUs ($1,000+). Cloud APIs offload compute to provider infrastructure.

ALEX uses a layered architecture: Interface Layer (Telegram, email, web), Gateway Layer (message routing, auth, rate limiting), AI Layer (Claude/DeepSeek/GPT with smart routing), Data Layer (ChromaDB for memory, file system for assets), and Hardware Layer (Raspberry Pi 5). Each layer is modular and independently scalable.

AI agents implement memory through: conversation history (short-term, within context window), vector databases like ChromaDB (long-term semantic search), structured storage (user preferences, learned skills), and file system (documents, generated assets). ALEX combines all four for persistent, searchable memory across sessions.

Smart routing automatically selects the optimal AI model based on task complexity. Simple queries (greetings, lookups) go to fast, cheap models like Haiku. Complex analysis goes to capable models like Sonnet. This optimizes cost while maintaining quality. ALEX analyzes each message to determine routing, saving 60-80% on API costs.

AI models can call predefined tools/functions to interact with external systems. You define available tools with names, descriptions, and parameters. The AI decides when to use tools based on user requests. Tools enable: web search, file operations, email sending, code execution, API calls, and more. ALEX has 20+ integrated tools.

The system prompt defines the AI's persona, capabilities, constraints, and behavior guidelines. It's injected at the start of every conversation. ALEX's system prompt establishes it as a Global Economist, defines available tools, sets response formatting, and includes safety guidelines. Well-crafted system prompts dramatically improve agent quality.