글

LLM Inference Metrics Explained: TTFT, TPOT, ITL and Goodput

-
LLM inference performance is not one speed number. TTFT describes the wait for the first output, TPOT and ITL describe how output continues, and E2EL describes the whole request. TPS and RPS measure processing capacity; SLO and goodput tell you whether that capacity meets a stated service goal. Start with the symptom: a slow start suggests checking TTFT; pauses after the start suggest checking output gaps; an overloaded service needs throughput and SLO checks together. These are directions for investigation, not proof of a root cause. Documentation checked October 18, 2026. All numerical examples and both diagrams are hypothetical. The short Python arithmetic was actually executed; no LLM, paid API, GPU or inference-engine benchmark was run. Contents · the eight essential terms TTFT — wait until the first output token. TPOT — average time per output token after the first. ITL — output-token gaps; check the tool’s aggregation. E2EL — time from request start to a defined endpoin...
댓글 쓰기
자세한 내용 보기

Build a DRAM Price-Change Calculator in Python

-
To combine consecutive DRAM price changes, multiply their growth factors instead of adding percentages. This Python calculator reads a CSV, starts a hypothetical index at 100, and creates a local HTML chart. The sample is invented; its output is not observed DRAM pricing, Samsung’s realized ASP, or a profit forecast. Quick diagnosis: if +20% followed by −20% gives you “back to zero,” check the denominator. The second change applies to the new level. Run the supplied CSV first and compare the output before using any permitted real dataset. No API key, package install, or paid service is needed. Executed October 18, 2026 on Windows 11 with Python 3.12.14 and 3.14.3. Standard library only. Other operating systems were not executed. The script reads your chosen CSV and exclusively creates a new chart file; it refuses to overwrite an existing output. Key terms, defined first QoQ percentage change: change from the immediately preceding quarter, not from a fixed base year. Growth facto...
댓글 쓰기
자세한 내용 보기

Memory Prices and Samsung’s Record Profit Estimate: What the Data Shows

-
Higher memory selling prices can lift Samsung’s profit, and the company identified rising prices as one contributor to its Q2 2026 memory results. Its Q3 guidance is a consolidated estimate—not a memory-profit breakdown. That supports a pricing tailwind, not an exact price-to-profit attribution. Evidence checked October 18, 2026, Korea time. Q3 figures below are preliminary guidance issued October 8; the detailed business discussion is explicitly Q2. No share-price move or investment return is inferred. The terms to know first ASP: average selling price; the average realized price across a defined set of sales. Changing the product mix can change it. Bit shipments: memory quantity sold, measured in bits—not the number of modules or the amount of revenue. Product mix: the proportions of different products sold, such as server memory and other memory products. Gross profit / operating profit: revenue after cost of sales / profit after operating expenses as well. Neither is automa...
댓글 쓰기
자세한 내용 보기

Read SOCAMM2 Specifications: Capacity, Bandwidth, and Module Counts

-
To read a SOCAMM2 specification, label every number with its unit, hardware scope, and qualification before calculating anything. Capacity is not bandwidth, a maximum is not installed memory, and dividing two differently defined capacity labels does not prove a module count. Quick check: if your module-count calculation produces a fraction, do not round it into a hardware claim. First check GB versus GiB, per-module versus per-CPU scope, and whether the number is a rounded maximum. If a field is missing, write “not determined” and request the exact configuration. Documentation checked October 17, 2026. Worked exercises below use explicitly declared assumptions; they are not a Vera bill of materials or a hardware benchmark. Key terms, defined first SOCAMM2: a memory-module format; do not confuse the physical module with the CPU or the entire server. Capacity: how much data can be held, expressed in bytes and a stated prefix. GB / GiB: decimal gigabyte / binary gibibyte. The ...
댓글 쓰기
자세한 내용 보기

SOCAMM2 Explained: What Vera’s Reported Memory Cut Really Means

-
SOCAMM2 is a compact, replaceable module that brings low-power LPDDR5X memory to the CPU side of an AI system. It is not the Rubin GPU’s HBM4. The reported Vera memory cut concerns capacity per configuration; that alone does not establish fewer module orders or falling total memory demand. Evidence checked October 17, 2026, Korea time. This is an explanation of dated public reports and product specifications—not a new NVIDIA order announcement, a hardware benchmark, or a stock-price forecast. Definitions before the headlines SOCAMM2: the second-generation small-outline compression-attached memory module format discussed here; a module is the physical assembly, not a new type of memory cell. LPDDR5X / LPDRAM: a low-power DRAM technology / the broader low-power DRAM category. These describe the memory, not the module’s shape. Vera: NVIDIA’s CPU, which runs host-side and standalone computing workloads. Rubin and HBM4: the GPU and its high-bandwidth memory in the Vera Rubin pla...
댓글 쓰기
자세한 내용 보기

Measure LLM Streaming TTFT and E2EL in C#: A Beginner’s Hands-On Guide

-
Start a monotonic timer just before sending the request, record the first non-empty generated-text event, and keep the last-text timestamp separate from the completion signal. This guide shows the distinction in a runnable C# streaming lab. What was actually tested? C# code over real loopback HTTP, using a simulated SSE generator. No actual LLM was called. No model, provider, GPU, token count or paid inference performance was measured. Environment checked: 2026-10-16 · Windows build 10.0.26200 x64 · .NET SDK 10.0.400 · .NET runtime 10.0.11. Other operating systems and .NET versions were not executed for this article. 1. Definitions first: what will we measure? Token: a model’s processing unit. It is not necessarily a word, character, HTTP chunk or SSE event. TTFT — Time to First Token: wait from a defined request start to the first generated output token at the measurement point. This text client uses the first non-empty generated-text event as a proxy , not a token-generation time...
댓글 쓰기
자세한 내용 보기