I want to preface this by saying this is not a call to change how cantrips and spells work; I presume that the current system exists for reasons related to mechanical balance. So please, no comments like "lol wut, do you want to make casters stronker?", or "I hate at-will cantrips!" (we're assuming that's just how things are for the purposes of this discussion). That's not the point, really. What I'm curious about is the narrative; D&D's spellcasting systems all evolved from an interpretation of what's going on in Jack Vance's Dying Earth novels (or, as I'm more familiar with, Roger Zelazny's Chronicles of Amber)- the Wizard takes time to "pre-cast" a spell in advance, to be released at a later time by leaving a few threads of magic unwoven (or something to that effect).
You can see this in play with Rituals, which can be used with no "cost" in spell slots, but take a lot of time to use. But Cantrips exist in this funny space, where they can do a lot for a very minimal cost, even if they eventually outperform low-level spells!
Most spellcasting classes can use Cantrips. Small bits of magic that can be used over and over again; mechanically, there's no real problem with this- it gives spellcasters something to do each turn, and lets them conserve spell slots.
However, something strange is definitely going on, because some of the Cantrips of higher level spellcasters get stronger over time, while the power of a spell slot doesn't change. Now, I'm just going to come out and say I don't think Cantrips are really infinite in universe; that's simply a game mechanic not wanting to actually place a limit on them. I'm just assuming that Cantrips can be used "an arbitrarily large amount of times" in universe.
But even that's a little strange. Consider: Fire Bolt is a ranged spell attack that deals 1d10 fire damage at level 1. At level 5 it becomes 2d10, at level 11 3d10, and level 17, 4d10.
Ice Knife is a 1st level ranged spell attack that deals 1d10 piercing damage, then explodes, causing the target and everyone within 5' to make a Dex save or take 2d6 cold. It never gets better than this, not even at levels 5, 11, and 17; instead, you'd have to use a higher level spell slot...but all that does is increase the secondary damage by 1d6. The initial d10 piercing never gets better.
What's going on, do you suppose, in-universe, that allows someone to wield a cantrip more efficiently and still maintain an arbitrary amount of uses per diem, but does not allow you to wield a spell more efficiently without using more and more of your allotment of magic energy? Even a Wizard who has attained Spell Mastery and could use Ice Knife at will like a cantrip, still only ever gains the minimum effect!
The only class that automatically scales spells is the Warlock, but even they are limited to X casts per day, as they must rest an hour to recover their slots.
So what is going on with Cantrips that causes them to function so differently from other spells? Why isn't there a dedicated Cantrip caster, who gets a large number of Cantrips and focuses on empowering them, not really caring about "spell slots" (the Warlock can be built in this fashion, but even they can mess around with leveled spells, not to mention Rituals)? You'd think there'd be some kind of Fighter or Rogue archetype called the "Cantrip Master" or something.
It is the way it is because things
used to be as you desire: lower-level spell slots scaled up with caster level.
The result was 3e/3.5e/PF1e's runaway overpowered spellcasters. Ironically, back then, cantrips
didn't scale at all
and were limited in quantity (until PF1e anyway.) They quickly became worthless for anything but color or RP because first-level spells were plentiful, doubly so with bonus spell slots from high ability score. Given how horribly, blatantly unbalanced this was, it was one of the few 3e-isms that 5e staunchly refused to adopt, and the game is better for it. But in the absence of scaling by spell level,
something has to scale, otherwise casters would (gasp! shock! horror!) actually
suck at something for a change, so they added scaling to cantrips instead. It's one of the few relatively openly gamist elements in 5e as a result.
I don't think they ever meant or intended there to be an explanation for this. It's just what it is, like how hit points just nebulously
are, or how having two different sources of benefit has no effect on how well you will do (Advantage doesn't stack at all.) It's just a thing.
If you want an explanation, however, the one that would make sense to me is that cantrips depend more on the power of the user than the power of the spell. Truly
slotted spells have a fixed value because they are precisely defined. Cantrips, being non-slot spells, depend far more on the mojo of the person casting them. A high-level, highly experienced Wizard has a bigger "pool" of magical energy to draw on, and the
size of the pool matters for cantrips, whereas it doesn't for slotted spells.
This would, in theory, mean that cantrips should get weaker once you've spent your higher-level spells, but maybe it's just a matter of being
capable of casting those high-level spells that makes the difference. And the levels you get better cantrips are pretty reasonable in this context: at 11th level, you get 6th level spells, which are much less available than anything lower (Wizard, Sorcerer, and Druid can all recover lower level spells, but not 6+.) At 17th, you get access to 9th level spells, the top end of magic ability. 5th is the only one that doesn't have a clear thematic justification, but it is generally seen as being the first "big boy spells" level, with things like
fireball, fly, haste, and
revivify all showing up with spells that full casters get at level 5 (meaning, 3rd level spells.) In that sense, one might say that spells come in tiers, 0th-2nd "lesser," 3rd-5th "greater," 6th-8th "grand," and 9th "greatest, with cantrips reflecting this growth in power.
Of course, that doesn't do crap to explain cantrips for half- or partial-casters like Eldritch Knights or Artificers. But I doubt anyone can do much better.