Computer storage is built on a hierarchy of increasing prices and performance, and the ensuing trade-offs.
For discussion purposes, the fastest storage is in processor registers, then processor cache, then main memory, then main storage, then archival storage. Processor registers are tiny hunks of storage and immensely expensive but also immensely fast. Cache is very fast, but slightly less immensely expensive. Main memory is fast, and expensive (more expensive lately, too), main storage varies and is much faster lately with somewhat expensive NVMe and SSD and immensely slower and much cheaper with HDDs, and archival storage relatively cheaper still. Again, trade-offs between price and performance abound throughout the system designs.
The operating system tries to move less- or unused stuff out of more expensive storage and into less-expensive storage, and tries to move more frequently used data from slower storage to faster, eventually traversing through the processor and its registers and caches. This includes virtual memory swapping; moving memory contents to storage, and back again.
Your current Mac is a middling processor with unspecified storage. If it’s HDD, that can usually be upgraded with an external SSD, but the I/O on a 2014 i5 is, well, still slow. And memory swapping to or from an HDD is very slow, and is less slow to an SSD on even a fairly slow I/O connection.
Oh, and GHz is best assumed to be marketing, not a useful measure of performance across different implementations of even the same processor, much less across different architectures. But in marketing, “big number good”, even if that doesn’t mean better performance. Or worse, a very expensive and very fast processor twiddling its proverbial thumbs wasting, err, waiting for main memory or main storage to transfer the data too or from the processor. Putting a fast processor with an HDD main storage means you want lots of main memory to cache that HDD data, and it’ll still be slow. Or it means you got spendy for an unbalanced and comparatively slow computer.
The 2014 iMac model used i5-4690 quad core, with base 3.5 GHz and boost 3.9 Ghz. And again, GHz are not a useful comparison outside of specific and related configurations.
I can’t find a Geekbench 6 results for the 2014 iMac 27” i5-4690.
The 2015 iMac 27” uses i5-4590 with base 3.3 GHz and boost 3.7 GHz and the score is 850 single and 1511 multicore.
The Mac mini M4 10-core score is 3941 single and 14938 multicore.
Slightly faster, yes.
Mac mini M4 ships with 16 GB minimally AFAIK, and (within the storage hierarchy mentioned above) immensely faster main memory and main storage. A straight comparison isn’t easy here either as the speeds and feeds have shifted, Ghz isn’t all that useful for comparison, and which is why something like Geekbench can potentially help. With the speedup in M4, swapping is much faster than it used to be — M4 main storage speeds aren’t that far off of DDR3 main memory speeds, within the storage hierarchy.
If you plan to keep this Mac mini M4 as long as that iMac, I’d go higher than default with memory, and with whatever main storage capacity meets your current storage usage trends, as (absent heroic efforts) neither can be expanded after purchase.
More pragmatically, it’s so much faster, you won’t notice even an unbalanced configuration until you push it far harder than you ever pushed that iMac i5.
Oh, and one more random note, if you do find yourself looking at a mid-range Mac mini, also price out the low-spec Mac Studio, as the mid- and upper-spec mini and the low-spec Studio can be closer than might be realized.