Data Analysis with AI: Technical Terms

A Glossary of Key Concepts

Gábor Békés (CEU)

2026-01-19

Purpose

This glossary provides brief definitions of key technical terms used throughout the Data Analysis with AI course.

Use this as a reference when you encounter unfamiliar terminology.

Core Concepts

Token

The basic unit of text that LLMs process.

  • Roughly 4 characters or ¾ of a word in English
  • “Hello world” ≈ 2 tokens
  • Pricing and limits are measured in tokens

Context Window

The maximum amount of text (in tokens) an LLM can consider at once.

  • Includes: system prompt + your messages + AI responses + uploads
  • GPT-5.2: ~400k tokens; Gemini 3: ~1M tokens
  • Exceeding the limit causes the model to “forget” earlier content

Inference

The process of generating output from a trained model.

  • Input goes in → model processes → output comes out
  • Distinct from training (which creates the model)
  • What happens every time you send a message

Latency

The time delay between sending a request and receiving a response.

  • Measured in milliseconds or seconds
  • Affected by: model size, input length, server load
  • Trade-off: faster models are often less capable

Model Architecture

LLM (Large Language Model)

A neural network trained on massive text data to predict and generate language.

  • “Large” = billions of parameters
  • Learns patterns from training data
  • Examples: GPT-5, Claude 4.5, Gemini 3

Transformer

The neural network architecture underlying modern LLMs.

  • Introduced in 2017 (“Attention Is All You Need”)
  • Key innovation: self-attention mechanism
  • Enables parallel processing of sequences

Parameters

The learned values (weights) inside a neural network.

  • More parameters ≈ more capacity to learn patterns
  • GPT-4: ~1 trillion parameters (estimated)
  • Trade-off: larger models are slower and more expensive

Mixture of Experts (MoE)

  • The architecture behind Gemini 3, featuring over 1 trillion parameters.
  • Only a subset of these parameters (“experts”) is activated for any single query, which balances deep intelligence with faster response times.

Prompting & Context

System Prompt

Hidden instructions given to the AI before your conversation.

  • Defines persona, constraints, behavior
  • Set by platform or user (Custom Instructions, Projects)
  • The AI “sees” this before your first message

Context Engineering

The practice of curating all information the model receives to optimize performance.

  • Goes beyond prompt engineering
  • Includes: system prompt, memory, tools, retrieved documents
  • Key skill for building reliable AI applications

Learn more: Anthropic’s context engineering guide

RAG (Retrieval-Augmented Generation)

A technique that retrieves relevant documents and adds them to the prompt.

  • Helps ground responses in specific data
  • Reduces hallucinations
  • Powers many enterprise AI applications

Reasoning & Thinking

Chain-of-Thought (CoT)

A prompting technique where the model shows intermediate reasoning steps.

  • “Let’s think step by step”
  • Improves accuracy on complex tasks
  • Can be prompted or built into the model

Reasoning Model

A model specifically trained to “think” before responding.

  • Examples: OpenAI o3, o4-mini; Claude with extended thinking
  • Internal deliberation, then final answer
  • Better for math, logic, multi-step problems

Extended Thinking

A mode where the model explicitly reasons through problems.

  • Visible chain of thought (unlike o3)
  • Configurable thinking “budget”
  • Trade-off: higher latency and cost

Thinking Levels

Gemini 3’s approach to controlling reasoning depth.

  • LOW: Fast, minimal deliberation
  • MEDIUM: Balanced (default)
  • HIGH: Deep reasoning, higher latency
    • Uses parallel thinking and “self-correction” signatures to solve complex logic or math problems. %Gemini

Tools & Agents

Agent

An AI system that can take actions, not just generate text.

  • Uses tools (code execution, web search, file access)
  • Operates semi-autonomously
  • May involve multiple steps and decisions

Agentic Workflow

A process where AI acts across multiple steps with tool use.

  • Example: Search → Analyze → Write → Review
  • Less human intervention between steps
  • Requires careful design and guardrails

MCP (Model Context Protocol)

An open standard for connecting AI to external tools and data.

  • “USB-C for AI applications”
  • Developed by Anthropic, adopted broadly
  • Enables secure access to files, databases, APIs

Learn more: MCP documentation

Tool Use / Function Calling

The ability of an AI to invoke external functions or APIs.

  • Model outputs structured “tool call”
  • System executes the tool
  • Result fed back to model

Quality & Safety

Hallucination

When an AI generates plausible-sounding but false information.

  • Fabricated facts, fake references, incorrect code
  • Reduced but not eliminated in modern models
  • Mitigated by grounding, RAG, verification

Grounding

Connecting AI responses to verified sources of truth.

  • Web search, document retrieval, database queries
  • Reduces hallucinations
  • Enables citations

RLHF (Reinforcement Learning from Human Feedback)

A training technique where models learn from human preferences.

  • Humans rate model outputs
  • Model learns to produce preferred responses
  • Key to making models “helpful and harmless”

Constitutional AI

Anthropic’s approach to training models with explicit principles.

  • Model trained to follow a “constitution” of rules
  • Self-improvement through AI feedback
  • Alternative to pure RLHF

Platform Features

Skills (Claude)

Reusable, modular instruction packages in Claude.

  • Pre-defined workflows and behaviors
  • Shareable across conversations
  • Can be combined for complex tasks

Gems (Gemini)

Custom AI assistants in Gemini Advanced.

  • User-defined personas and instructions
  • Persistent across conversations
  • Shareable with team

Projects (Claude)

Workspaces with shared context across conversations.

  • Persistent system prompt
  • Uploaded knowledge files
  • All chats share the same context

Canvas / Artifacts

Interactive workspaces for editing AI-generated content.

  • ChatGPT Canvas, Claude Artifacts
  • Side-by-side editing
  • Good for code and documents

Performance & Efficiency

Temperature

A parameter controlling randomness in model outputs.

  • 0 = deterministic (same input → same output)
  • 1 = default, balanced creativity
  • Higher = more random/creative

Note: Gemini 3 and reasoning models work best at default (1.0)

Context Rot

Performance degradation as the context window fills up.

  • Model becomes less accurate over long conversations
  • Even within technical limits
  • Solution: start fresh, use memory tools

KV-Cache

A technical optimization that speeds up repeated inference.

  • Caches intermediate computations
  • Faster response for repeated prefixes
  • Why stable prompt prefixes matter for cost

Development Patterns

Vibe Coding

Describing desired behavior in natural language rather than writing syntax.

  • “Make this chart interactive with company colors”
  • AI translates intent to code
  • Requires human review and iteration

CLAUDE.md

A convention for providing project context to Claude Code.

  • Markdown file in project root
  • Contains: file descriptions, conventions, current task
  • Automatically read by Claude Code CLI

Prompt Chaining

Breaking complex tasks into sequential prompts.

  • Output of step N becomes input to step N+1
  • More reliable than single complex prompt
  • Enables debugging at each stage

Costs & Limits

Input/Output Tokens

Tokens are billed separately for input (prompt) and output (response).

  • Input: what you send (including context)
  • Output: what the model generates
  • Output tokens typically cost more

Rate Limiting

Restrictions on how many requests you can make.

  • Requests per minute (RPM)
  • Tokens per minute (TPM)
  • Prevents abuse and ensures fair access

Prompt Caching

Storing and reusing processed prompts to reduce cost.

  • Same prefix → cached, cheaper
  • Useful for repeated system prompts
  • Requires stable prompt structure

Summary

Quick Reference

Term One-liner
Token Basic text unit (~4 characters)
Context window Max tokens model can see at once
Inference Generating output from input
Agent AI that takes actions via tools
MCP Standard for AI tool connections
Hallucination AI-generated false information
RAG Retrieval + generation technique

Resources for Learning More

Date stamp

This version: 2026-01-19 (v0.1.0)

bekesg@ceu.edu