The idea: the slope of a curve is a limit of slopes of lines
A straight line has one slope: rise over run. A curve bends, so its steepness changes from point to point. To measure it at a point P = (x, f(x)), take a second point Q = (x+h, f(x+h)) on the curve and draw the line through both, the secant. Its slope is the difference quotient (f(x+h) − f(x)) / h. Now let h shrink: Q slides along the curve towards P and the secant turns into the tangent. The limit of the slopes is the derivative:
f′(x) = limh→0 (f(x+h) − f(x)) / h
The difference quotient and its error
Each Shrink h divides h by 2 and adds a row to the table on the canvas. For f(x) = x² at x = 1 the algebra is exact: ((1+h)² − 1) / h = (2h + h²) / h = 2 + h. So the slope is always 2 + h, and the error is exactly h. (Demo: x² at x = 1)
| h | f(1+h) | slope Δf / h | error |
|---|---|---|---|
| 1 | 4 | 3 | 1 |
| 1/2 | 2.25 | 2.5 | 0.5 |
| 1/4 | 1.5625 | 2.25 | 0.25 |
| 1/256 | 1.0078277… | 2.00390625 | 0.0039 |
For any smooth f the same holds approximately: the error of this forward difference is about ½·f″(x)·h, so halving h halves the error. sin x at 0 gives slopes 0.841, 0.959, 0.990 … towards cos 0 = 1. (Demo: sin x at 0)
The derivative is a new function
Nothing about the limit depends on the particular x. Doing it at every x gives a function f′, and Plot f′ draws it dot by dot: at each x the tangent's slope becomes the height of the bottom curve. x² gives the line 2x, sin x gives cos x. eˣ is special: its slope at every point equals its height, so the dots land on the same curve as f. That is the defining property of e. (Demo: eˣ is its own derivative)
When the limit does not exist
A limit must give the same answer whichever way h goes to 0. For f(x) = |x| at 0, the right quotient (|h| − 0) / h is +1 for every h > 0 and the left quotient (0 − |−h|) / h is −1. The two columns never get closer, so there is no single tangent and f′(0) does not exist. The curve has a corner. Its derivative jumps from −1 to +1 with a hole at 0. (Demo: |x| at 0)
Why h cannot be tiny on a computer
Mathematically smaller h is always better. In floating point it is not. A double carries about 16 significant digits (machine epsilon ε ≈ 2.2·10−16). When h is tiny, f(x+h) and f(x) agree in most of their digits, and the subtraction cancels them: catastrophic cancellation. What remains is rounding noise of size about ε·|f(x)|, which is then divided by h. So the total error is roughly
error ≈ ½·|f″|·h + ε·|f| / h
The first term falls with h, the second grows. The best h is near √ε ≈ 10−8, where about 8 digits are right. Below about 10−16, x + h rounds back to x, Δf is exactly 0 and the computed slope is 0. (Demo: h too small, eˣ at x = 1 with h divided by 10 each time)
Real numeric code therefore uses h ≈ √ε·|x|, or the central difference (f(x+h) − f(x−h)) / 2h, whose error is proportional to h² (best h near 10−5). See Floating Point (IEEE 754) for why doubles have about 16 digits.
Where derivatives are used
- Gradient Descent follows the derivative (the gradient) downhill to minimise a loss. Machine-learning libraries check their exact gradients against a central difference, a "gradient check".
- Newton's method uses the tangent line to jump to a root.
- Taylor series build a polynomial from the derivatives at one point.
- Riemann sums and integrals undo the derivative (the fundamental theorem of calculus).
Apply the same limit to the area under a curve and you get f back: see the Fundamental Theorem of Calculus. The chain rule shows how derivatives of composed functions multiply, and how a numeric slope checks them.
The Mean Value Theorem says some tangent on [a, b] has exactly the slope of the secant from a to b.
Zoom in on P instead of shrinking h and the curve itself turns into its tangent: see Local Linearity: Zoom In.
The limit itself, with ε–δ bands and the ways a limit can fail, is in Limits & Continuity.
The rules themselves (power, product, sin′ = cos) are drawn as growing squares, rectangles and circles on Derivative Rules as Pictures.
Once f′ is a function, Reading Derivative Graphs shows how its sign and f″ describe the shape of f.
The secant slope also drives a root finder: the secant method follows it to the axis, as Root Finding: Bisection vs Newton vs Secant shows.
What the page leaves out
The page uses the forward difference only (plus the backward difference for |x|); the central difference is described, not animated. The exact derivatives f′ come from the known formulas, not from symbolic or automatic differentiation. The ε-δ definition of a limit, the rules (product, chain), higher derivatives and functions of several variables are not covered.