Pipeline
Also known as: AI pipeline, LLM pipeline, processing pipeline, prompt pipeline, workflow pipeline
Pipeline is an overloaded word from software engineering that AI builders use constantly. In basic usage it means any sequence of processing steps: data comes in one end, goes through transformations, and comes out the other. In AI the same idea applies: input goes through a retrieval step, then a summarization step, then a formatting step, then model generation, then output validation. Each step is a node in the pipeline.
The reason pipelines matter for AI builders is that most useful AI tasks can't be done in a single prompt. Summarizing 50 documents requires chunking, summarizing each chunk, then summarizing the summaries. Answering a complex question requires retrieving relevant information, then generating, then checking the output for accuracy. Pipelines let you break complex tasks into steps where each model call is focused on something manageable.
Common tools for building AI pipelines include LangChain, LangGraph, n8n, and many custom implementations. The complexity spectrum runs from a simple linear chain (step 1, step 2, step 3) to full agentic workflows where the model itself decides which steps to take. 'Pipeline' usually implies a more structured, predetermined sequence, while 'agent' implies more dynamic decision-making.