Embedded Memories The Next Generation

TITLE: Embedded Memories: The Next Generation CHANNEL: Asianometry DATE: 2026-06-21 ---TRANSCRIPT--- As chips speed up and get more capable, they  must also fetch more data and get it faster. Most of the time, that means going  off-chip to some external memory module.   It slows things down and uses energy. One alternative is to “embed” some memory  right alongside the logic circuits on the chip:   Embedded memories. For years, two types of embedded memories  dominated. But things are changing. In today’s   video, we take a look at those, plus some of the  next-generation memories coming down the pike.

Three Friends

Forty years ago in the 1980s, there were three big  categories of discrete, standalone memory chips: The SRAMs, DRAMs and Flash EEPROMs. But  as time and technology demands progressed,   these three changed like high  school friends after graduation. In the 1990s, the process nodes  used to make DRAM and Flash memories   greatly diverged from each other as well  as the nodes used for making logic chips. DRAMs transitioned from using flat  planar capacitors to vertical ones.   Today, the dominant DRAM nodes use  these tall and skinny capacitors   stacked on top of or below  their access transistors. And as for EEPROMs, they evolved into  the flash memories: NOR and NAND.   With planar NAND evolving yet again  into the lasagna-like 3D NAND. Such vertically stacked NANDs are some of  the most scalable in the semiconductor world.   I did a video about it a while ago!

SRAM: The Gold Standard Embedded Memory

As a standalone memory, SRAM hasn’t had  the same success as its two friends. Unlike DRAMs and EEPROMs however, SRAMs  can use only transistors to store bits,   which lets us make it alongside the rest of  the chip without needing any additional masks. In the late 1980s, CPU makers started embedding  SRAMs onto their chips as cache to store important   data. It remains very significant and the  single largest embedded memory market. However in recent years, SRAMs have found  themselves on the ropes. For one thing,   it is a thicc boy. The most widely used  SRAM cell design uses six transistors.   That is a lot compared to DRAM, which  famously is just 1 transistor and 1 capacitor. That’s a problem because transistors  aren’t getting much smaller nowadays.   Fabs have optimized SRAM to such an extent  that when they brag about their process nodes,   they use SRAM density to do so. One of the few hard numbers that TSMC has  publicly announced about their N2 process node   is how it can stuff more SRAM onto the die. With CPUs and other systems-on-chips getting more  advanced, you get situations where a surprisingly   significant portion of certain chips is just  embedded SRAM memory. Back in the mid-2000s,   some high-performance CPUs had as much as  70% of their whole dies being just SRAM.

eDRAM

So if SRAM is reaching its density limits, why  not embed something that can be far denser? That is why some have used  embedded DRAM, or eDRAM.   It is the same 1-transistor-1-capacitor  structure, just embedded on the die.   With that skinnier setup, we can stuff five to six  times more eDRAM than SRAM onto the same space. eDRAM also uses significantly  less power than SRAM.   Even if you still have to periodically  refresh them like with commodity DRAMs,   you use just a third of the power of SRAMs. Not to  mention the power saved from not going off-chip. There are also integration benefits. Since we  are less likely to get bad connections - bent   pins or other mechanical failure points,  etc - eDRAM tends to be more reliable.   Data transfers to and from  memory have better latency. So what are the downsides? Memory and logic  process nodes are nowadays very different.   So producing eDRAM adds maybe four to six masks to   the fabrication process - which exposes  your chip to yield risk and higher costs. The eDRAM market was once quite considerable -  used for items like the Xbox 360. However, its   momentum has sort of petered out in recent years 

• with fewer industry products being made with it. However, there seems to be plenty of   compute-in-memory research  done in academia with it.

eFlash

Also like SRAM, eDRAM is volatile. Once the  power goes out, everything is forgotten. Ideally, we want something nonvolatile. Something  that can hold its data when the power goes off.   So over time, vendors have embedded  Flash memories onto the chip: Embedded Flash or eFlash. eFlash  is a NOR type memory. With NOR,   we string together many special memory cells -  planar transistors equipped with a floating gate. Electrons are compelled into that  floating gate through an oxide,   raising the transistor’s threshold voltage. NOR arranges these cells in such a way   that we can access one at a time - random  access - at the cost of less density. Its younger cousin NAND on the other hand  networks its cells together in strings of   16 to 128 with each cell’s source connected to  its neighbor’s drain. It lets us pack cells very   closely together, but also means no random access.  We can only manipulate data in blocks or pages. That is why we aren’t getting  embedded NAND any time soon.   Programming and erasing eFlash’s NOR-style  networks require high voltages - something   like 9 to 18 volts - to compel the electrons  to go in and out of the floating gate. That is a problem because standard  logic transistors run at about 1 volt.   To protect these neighboring logic  transistors from getting fried,   you need deep isolation trenches  or some kind of hardening. NAND requires even higher voltages than NOR to  program or erase their long strings of cells,   which is too high for the die. Anyway. So eFlash being NOR means it cannot  achieve the same density as NAND has.   It also does not write as fast as SRAM  or DRAM. Its cells also suffer the same   endurance issues as discrete flash memories 

• breaking down over repeated write cycles. And same as with eDRAM, there is a fabrication  cost. It requires an additional 6-8 mask steps,   which is more than eDRAM.  You take on more yield risk. Today, eFlash is most often used  to store program code and data   for these small but vital chips called  microcontrollers or MCUs. These are   basically computers on a single chip, though  nowhere near as powerful as an Intel or AMD CPU. eFlash fit the bill for these systems because  it boots fast, is power-efficient, rewritable,   and can survive the rough conditions that cars  or other industrial devices often experience. The eFlash automotive MCU market is often cited  as the second largest overall embedded memory   market after SRAM. Though you can also find  them in edge AI and data center applications.

eFlash’s Scaling Wall

eFlash’s most serious issue, however, is scaling.   Largely speaking, 28-nanometers is the  end of the road scaling-wise for eFlash. Scaling down eFlash means making smaller  transistors and packing them closer together.   This becomes a serious issue at 28-nanometers,  the last planar transistor node. There, you have all the standard problems  of shrink - loss of control over the gate,   short channel effects, so on. Par for the course  and why the logic fabs switched to 3D FinFETs.   But then there are the flash  memory related issues too. The flash memory cells are now so physically  small that their floating gates contain about   a hundred or so electrons for a threshold of  1 volt. It takes fewer electrons leaking to   cause significant degradation. And with the tunnel  oxide layers so thin now, that is way more likely. These scaling problems are why the NAND makers  switched to 3D NAND. You loosen the floating   gates’ technical requirements by resetting  their sizes from 28-nanometers to 40-nanometers,   but then stack them vertically to  achieve massive storage numbers. 3D NAND is made in a parallel manner and  very cool (some cases, literally) but not   a valid technical pathway for eFlash. Nor is it  economically feasible for the chip designer to add   what can be up to ten additional masks to produce  a wholly different transistor type onto the chip. Without a valid successor to eFlash, OEMs  of MCUs and such products might move back   to discrete memories - perhaps using  advanced packaging to put them together.   So fabs and startups have suggested potential  successors: The next generation memories.

MRAM

There are a lot of next-generation embedded  memories out there. So let me cut it down   to the few such with serious backing by  major foundries like TSMC and Samsung.   First up are the Magnetoresistive RAMs or MRAMs. DRAMs and Flash memories encode  the bit by storing a charge.   The MRAM on the other hand does it by  manipulating electrical resistance levels. An MRAM cell is made up of two things: An access  transistor and the Magnetic Tunnel Junction,   or MTJ. The latter is where the magic happens. The MTJ is a tiny sandwich of  ferromagnets and non-magnetic layers.   The simplest MTJ has two ferromagnets and a  very thin insulator layer in between them.   The top ferromagnet is referred  to as the “free layer” … The ferromagnet on the bottom is called  the reference or fixed or pinned layer.   Both ferromagnets are usually made from  an iron alloy like cobalt-iron-boron.   And depending on the variant you  may have several pinned layers. As for the very thin - maybe about 1-2  nanometers thick - insulator layer,   that is most often made of magnesium oxide. The MTJ works by having an external current  orient the magnetization of the MTJ’s free layer   as compared to its reference or pinned layers. When the magnetic moments of both ferromagnet  layers are parallel, then the whole MTJ will   have low electrical resistance - so we  can easily run a current through it. And when the magnetic moments of the  two ferromagnets are not parallel   to each other - or anti-parallel - then the  electrical resistance gets significantly higher. So in a way, it is like twisting  a faucet open or closed.   We can map the two high or low  resistance states to a bit. How is that done? With MRAMs, we  send small currents into the MTJ   to try and discern its resistance state 

• comparing it against a middle-point   reference level to determine the final value.

STT-MRAM

The first MRAMs were introduced in the 1980s,   and are today known as conventional  or field-switched MRAMs. These older memories used magnetic  fields to write to the MTJ - i.e.   how we set the free layer. This  magnetic field was created by   running a current through a wire -  making this flip an indirect effect. Basically, that was how we wrote data  to the old ferrite core memories. For this mechanism to work, the magnetic  field has to be strong enough to flip the   MTJ’s magnetic state. The problem  was that as the MTJ got smaller,   it gets easier for that bit to  accidentally flip due to thermal noise. Hard Disk Drives suffer this problem too. They  have something called the Superparamagnetic Limit,   where the grains in a bit get so small  that thermal energy can flip them. Anyway. The response by engineers  has been to raise the “flip” limit,   but the downside of doing that is that we need  a more powerful magnetic field to switch it   when we actually need to do so. When the MTJ is  small, controlling that field is harder to do. In the 1990s, the field-switched MRAM was replaced  by a new variant known as Spin-Transfer Torque   MRAM, STT-MRAM. Instead of using a magnetic  field, we send a special current through the MTJ.   Such electrons flood into the  free layer and flip it directly.

STT-MRAM’s Pluses and Minuses

STT-MRAM is a very promising memory technology.  To start, it is non-volatile so the data stays   even after the power goes off. It does  not need to be continually refreshed. The area savings are also very significant. It is  basically just the MTJ and an access transistor -   a very DRAM-ish setup. At the 5-nanometer node,  we get like an 43% reduction as compared to SRAM. Despite being a non-volatile memory, you  can write to it in less than 10 nanoseconds,   which is DRAM-like speeds and far faster  than flash memory’s 20 to 100 microseconds.   And unlike flash memory, the  cells have very good endurance. The biggest challenge involves  things on the manufacturing side.   The technology is technically CMOS compatible,   but fabbing the 15-20 various metal and dielectric  stacks that make up the MTJ is challenging. In particular, the insulating oxide barrier  between the free and pinned layers needs to be   about 1-2 nanometers wide. Common issues often  happen during the etch or post-etch process,   where oxygen impinges into the insulator layer  to create an effect called “Bird’s Beaking”. There can also be issues with the bottom  electrode contact, which connects the MTJ   to the metal lines. Roughness in that  contact can make the MTJ’s layers rough too,   which causes the free layer to magnetically  “sync” with the reference layer. Thus   making it harder to discern the actual  saved resistance level of the MTJ. Beyond that, there are scaling  issues. At 3, 5-nanometer class nodes,   the STT-MRAM gets so small that we need a  decently strong current to properly write   to it and avoid thermal-induced bit  flips. But advanced node transistors   are so small and delicate that such  a current cannot be easily delivered.

ReRAM

The other major eFlash replacement  is Resistive RAM or ReRAM or RRAM. ReRAM is another non-volatile  memory that stores a bit using   either a high or low resistive  state. Yes, a lot like MRAM.   However, the way in which they go about  doing that is very different and kind of fun. There are a variety of ReRAM cells, but the most  commonly used one is the filament-based ReRAM.   It too is a sandwich of a metal oxide  insulator layer between two metal electrodes. The oxide might be of elements  like Hafnium, Tantalum or Titanium,   but research into more exotic  things like 2D materials is ongoing. The electrodes can be titanium,  platinum or something else. The ReRAM cell switches between high  and low resistive states by creating   (SET) or destroying (RESET) a  small conductive filament - maybe   as small as 10 by 10 nanometers 

• bridging the two electrodes. That little filament is essentially a wire  through the naturally insulating dielectric.   We set or reset the filament by applying a  voltage or current signal to the electrodes.   Very elegant, very simple concept. The concept reminds me of another memory  called Phase Change RAM (PCRAM) - which   uses heat to switch a chalcogenide glass  between an amorphous or crystalline phase.   In this case ReRAM does not  require the phase change.

ReRAM Pluses and Minuses

Producing embedded ReRAM requires fewer mask steps  than the other embedded non-volatile memories. It can scale down to advanced nodes fairly  well. It uses less energy, is non-volatile,   and reads/writes very quickly. On the other hand,  there are some variability and endurance issues. The SET and RESET processes appears to be  inherently random, leading to inconsistent   behaviors. Not what you want in a semiconductor  technology. And I do wonder how many times can   we make and break the filaments before  it starts to exhibit weird behaviors? Broadly speaking, the technology has commercial  potential. But it also feels like something that   pops its head up every so often - attempting  to ride the latest significant trend. Between 2005 and 2015, ReRAM gained serious  traction as a potential successor to 2D NAND   until the rise of 3D NAND closed the door on that.   There were proposals to do stacked  ReRAM, but those failed to compete. A few companies offer this technology today.   There is one called Weebit Nano from Israel  that shows up a lot in the literature. They   have been around for over ten years, licensing  ReRAM IP to customers for end user products. TSMC offers it as an option for  customers too. They have published   a few papers on the technology, though  far fewer than what they have on MRAM. It seems like they are positioning both  technologies as potential successors to   eFlash especially in the automotive MCU space.  There are others like the aforementioned Phase   Change RAM and ferroelectrics that might have a  shot, but I think MRAM and ReRAM are the leaders.

Conclusion

Despite both TSMC and Samsung positioning  STT-MRAM or ReRAM, eFlash remains quite resilient. Why? eFlash is a tried-and-true solution   in a space where reliability matters  more than raw performance. Most MCUs   are still made using trailing edge nodes like 65  nanometer node, though this is starting to change. And as we laid out throughout this video, the  next-generation contenders are not exactly free   of trade-offs. The semiconductor  industry is pretty conservative,   and they are not apt to try new  weird stuff until they have to. The current major hope for these embedded  Non-Volatile Memories technologies is AI.   Since embedded memories are so close to the logic,   there is some potential to evade the Von Neumann  Bottleneck - at least in the case of STT-MRAM. Another option is for doing AI inference on  the device at low power and fast latency,   maybe even using neuromorphic principles  to do so. This is more for ReRAM.   In both cases, the technologies seem  to have gotten ahead of the use case,   but we shall see if the end  user markets can get on-board.