The Essence of AI Models: Using Parameters to Fit High-Dimensional Functions Humans Can Hardly Understand
In today’s AI boom, we often hear terms like “large models,” “parameter count,” and “training.” But do you truly understand: the essence of AI models is using a bunch of parameters to fit a set of high-dimensional functions that humans themselves can hardly design or comprehend.
Sounds abstract? Don’t worry. Today, with less than 50 lines of Python code, we’ll train a “mini AI model” to fit a simple but non-linear function — giving you an intuitive feel for the core idea of “parameter fitting.”
Starting with a Simple Function
Suppose we have a function containing both periodic sine waves and a slowly growing quadratic term. A human can easily write its expression, but if you’re only given a bunch of x and y data points without the underlying formula, could you reconstruct the function?
That’s what machine learning solves.
Fitting It with a “Neural Network”
We build a minimal neural network with:
- 1 input (x)
- 1 hidden layer (10 neurons)
- 1 output (predicted y)
The model has no “prior knowledge.” It only knows: give me x, I use a bunch of adjustable parameters (weights and biases) to compute a y. By continuously adjusting these parameters, its output y gets closer to the real value.
The model has only 21 parameters total. It doesn’t “know” what sine or square is. It simply finds a set of numerical values in high-dimensional parameter space such that the output y best matches the data.
That’s the essence of AI models: using learnable parameters to implicitly encode a complex function’s mapping relationship.
From “Small Model” to “Large Model”
- GPT-4 has 1.76 trillion parameters — it fits the probability distribution of human language sequences
- Stable Diffusion fits the mapping from text descriptions to image pixels
- AlphaFold fits the function from amino acid sequences to 3D protein structures
None of them can be expressed with mathematical formulas, yet through parameterized functions + data-driven training, they accomplish tasks that humans can hardly achieve.
AI is not magic. It’s the art of high-dimensional fitting.
Comments