Block RAM, DSP & Clocking

Chapter 5 showed that LUTs and flip-flops can build any digital circuit. But “can” is not “should”. A 16 × 16-bit multiplier built from LUTs needs a few hundred of them and several levels of logic; a memory of a thousand words would need hundreds of LUT-RAMs. Some functions are so common in real designs that it pays to build them directly in silicon, as fixed circuits placed at regular positions in the fabric. These are the hard blocks:

• block RAM (BRAM): dense on-chip memory, 140 blocks of 36 Kb on the XC7Z020;
• DSP slices (DSP48E1): multiply–accumulate units, 220 on the XC7Z020;
• clock management: global clock networks and frequency synthesisers (MMCM and PLL);
• on the Zynq, a whole processing system with two ARM cores, next door to the fabric.

A hard block is smaller, faster and uses less power than the same function in LUTs — typically by a factor of ten or more — but it only does what it was designed for. The synthesiser recognises suitable HDL patterns and maps them onto these blocks automatically (inference), as long as you write the patterns it expects. This chapter shows those patterns.

A 7-series block RAM stores 36 Kb (36,864 bits): 32 Kb of data plus 4 Kb of extra bits intended for parity. It can be used as one 36 Kb RAM (RAMB36E1) or split into two independent 18 Kb RAMs (RAMB18E1). The XC7Z020 has 140 of them — 280 halves — for a total of 4.9 Mb. That is small compared with the 512 MB of DDR3 on the board, but it is right next to your logic, and every block can be read and written every cycle.

The width and depth are configurable; the total stays the same. The extra “parity” bits are why the wide modes are 9, 18, 36 and 72 bits rather than 8, 16, 32 and 64 — you may use them as ordinary data.

Three properties matter most for a designer:

• True dual-port. Each block has two completely independent ports, A and B, each with its own address, data in, data out, write enable and even its own clock. Two parts of a design can use one memory at once. In simple dual-port (SDP) mode, one port only writes and the other only reads, which allows the 72-bit width.
• Synchronous read with one cycle of latency. The address is captured on a clock edge, and the data appears after that edge. You cannot get data “in the same cycle” as the address. An optional output register adds a second cycle but makes the output much faster to use.
• Initial contents from the bitstream. A block RAM can be preloaded with a table (coefficients, a sine wave, a font, a small program) at configuration, i.e. it can also serve as a ROM.

You rarely instantiate a RAMB36E1 by name. Instead you describe an array and access it in the way a block RAM works — write on the clock edge, read on the clock edge — and the synthesiser does the rest. The module below is a simple dual-port RAM of 1024 words × 18 bits = 18 Kb, exactly one RAMB18E1.

// Simple dual-port RAM: one write port, one read port, one clock.
// 1024 x 18 bits = 18 Kb, so Vivado maps it to ONE RAMB18E1
// (half of a 36 Kb block RAM). Read latency: 1 clock cycle.
module bram_sdp #(
    parameter AW = 10,                     // address width: 2**AW words
    parameter DW = 18                      // data width
) (
    input  wire          clk,
    input  wire          we,               // write enable
    input  wire [AW-1:0] waddr,
    input  wire [DW-1:0] din,
    input  wire [AW-1:0] raddr,
    output reg  [DW-1:0] dout = {DW{1'b0}}
);
    (* ram_style = "block" *)
    reg [DW-1:0] mem [0:(1 << AW) - 1];

    integer k;
    initial begin                          // contents loaded by the bitstream
        for (k = 0; k < (1 << AW); k = k + 1)
            mem[k] = {DW{1'b0}};
    end

    always @(posedge clk) begin
        if (we)
            mem[waddr] <= din;
        dout <= mem[raddr];                // synchronous read: data one cycle later
    end
endmodule

-- Simple dual-port RAM: one write port, one read port, one clock.
-- 1024 x 18 bits = 18 Kb, so Vivado maps it to ONE RAMB18E1
-- (half of a 36 Kb block RAM). Read latency: 1 clock cycle.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

entity bram_sdp is
    generic (
        AW : positive := 10;                 -- address width: 2**AW words
        DW : positive := 18                  -- data width
    );
    port (
        clk   : in  std_logic;
        we    : in  std_logic;               -- write enable
        waddr : in  std_logic_vector(AW - 1 downto 0);
        din   : in  std_logic_vector(DW - 1 downto 0);
        raddr : in  std_logic_vector(AW - 1 downto 0);
        dout  : out std_logic_vector(DW - 1 downto 0)
    );
end entity;

architecture rtl of bram_sdp is
    type ram_t is array (0 to 2**AW - 1) of std_logic_vector(DW - 1 downto 0);
    signal mem : ram_t := (others => (others => '0'));   -- loaded by the bitstream
    attribute ram_style : string;
    attribute ram_style of mem : signal is "block";
begin
    process (clk)
    begin
        if rising_edge(clk) then
            if we = '1' then
                mem(to_integer(unsigned(waddr))) <= din;
            end if;
            dout <= mem(to_integer(unsigned(raddr)));  -- synchronous read: data one cycle later
        end if;
    end process;
end architecture;

The ram_style = "block" attribute is optional — Vivado would choose block RAM for this size anyway — but it documents your intent and makes the tools warn if they cannot comply. Two details deserve attention:

• the read assignment dout <= mem[raddr] is inside the clocked block. That makes the read synchronous, which is what the BRAM supports;
• when the same address is written and read in the same cycle, this code returns the old contents, because the non-blocking/signal assignment reads mem before the edge updates it. Block RAMs support this “read-first” behaviour (as well as “write-first” and “no-change”); the testbench checks it explicitly.

`timescale 1ns / 1ps
module bram_sdp_tb;
    reg         clk = 1'b0, we = 1'b0;
    reg  [9:0]  waddr = 0, raddr = 0;
    reg  [17:0] din = 0;
    wire [17:0] dout;
    integer     n, errors;

    bram_sdp dut (.clk(clk), .we(we), .waddr(waddr), .din(din),
                  .raddr(raddr), .dout(dout));
    always #4 clk = ~clk;

    // the test pattern: word n holds (n * 37 + 5) mod 2**18
    function [17:0] pattern(input integer n);
        pattern = (n * 37 + 5) % 262144;
    endfunction

    initial begin
        errors = 0;
        // 1. fill all 1024 words
        for (n = 0; n < 1024; n = n + 1) begin
            @(negedge clk); we = 1'b1; waddr = n; din = pattern(n);
        end
        @(negedge clk); we = 1'b0;
        // 2. read them back: address at one edge, data after it
        for (n = 0; n < 1024; n = n + 1) begin
            @(negedge clk); raddr = n;
            @(negedge clk);
            if (dout !== pattern(n)) errors = errors + 1;
        end
        // 3. read and write the same address in the same cycle: old data comes out
        @(negedge clk); we = 1'b1; waddr = 10'd100; din = 18'h3FFFF; raddr = 10'd100;
        @(negedge clk); we = 1'b0;
        if (dout !== pattern(100)) errors = errors + 1;   // old value
        @(negedge clk);
        if (dout !== 18'h3FFFF) errors = errors + 1;      // new value, one cycle later
        if (errors == 0)
            $display("PASS: 1024 words written and read back, read-during-write returns old data");
        else
            $display("FAIL: %0d errors", errors);
        $finish;
    end
endmodule

library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

entity bram_sdp_tb is
end entity;

architecture sim of bram_sdp_tb is
    signal clk, we      : std_logic := '0';
    signal waddr, raddr : std_logic_vector(9 downto 0) := (others => '0');
    signal din, dout    : std_logic_vector(17 downto 0) := (others => '0');
    signal done         : boolean := false;

    -- the test pattern: word n holds (n * 37 + 5) mod 2**18
    function pattern(n : natural) return std_logic_vector is
    begin
        return std_logic_vector(to_unsigned((n * 37 + 5) mod 262144, 18));
    end function;
begin
    dut : entity work.bram_sdp
        port map (clk => clk, we => we, waddr => waddr, din => din,
                  raddr => raddr, dout => dout);
    clk <= not clk after 4 ns when not done;

    stimulus : process
        variable errors : natural := 0;
    begin
        -- 1. fill all 1024 words
        for n in 0 to 1023 loop
            wait until falling_edge(clk);
            we <= '1'; waddr <= std_logic_vector(to_unsigned(n, 10)); din <= pattern(n);
        end loop;
        wait until falling_edge(clk); we <= '0';
        -- 2. read them back: address at one edge, data after it
        for n in 0 to 1023 loop
            wait until falling_edge(clk); raddr <= std_logic_vector(to_unsigned(n, 10));
            wait until falling_edge(clk);
            if dout /= pattern(n) then errors := errors + 1; end if;
        end loop;
        -- 3. read and write the same address in the same cycle: old data comes out
        wait until falling_edge(clk);
        we <= '1'; waddr <= std_logic_vector(to_unsigned(100, 10));
        din <= (others => '1'); raddr <= std_logic_vector(to_unsigned(100, 10));
        wait until falling_edge(clk); we <= '0';
        if dout /= pattern(100) then errors := errors + 1; end if;      -- old value
        wait until falling_edge(clk);
        if dout /= "111111111111111111" then errors := errors + 1; end if;  -- new value
        if errors = 0 then
            report "PASS: 1024 words written and read back, read-during-write returns old data";
        else
            report "FAIL: " & integer'image(errors) & " errors" severity error;
        end if;
        done <= true;
        wait;
    end process;
end architecture;

Both versions print PASS: 1024 words written and read back, read-during-write returns old data. Chapter 17 builds FIFOs on top of this memory.

Signal processing, control loops, graphics and neural networks all boil down to the same operation repeated millions of times: multiply two numbers and add the result to a running sum, a multiply–accumulate(MAC). The DSP48E1 slice is a hard MAC unit. Its main parts, from input to output:

• a 25-bit pre-adder that can compute \(D \pm A\) before multiplying — useful in symmetric FIR filters, where two samples share one coefficient;
• a signed 25 × 18 multiplier (two's complement, Chapter 1), producing a 43-bit product;
• a 48-bit ALU / accumulator that adds, subtracts, or performs logic operations on the product and either its own previous output (accumulate) or another input;
• optional pipeline registers at every stage (A, B, D, AD, M and P registers), so the slice can run at several hundred MHz;
• dedicated cascade connections to the DSP slice above, so long chains (e.g. adder trees in filters) do not need general routing.

The 25 × 18 shape is not arbitrary: 18 bits matches the width of the block RAM ports (Section 2), so coefficients stored in BRAM feed the B input directly. Wider products are built by combining several slices; the 17-bit shift on the cascade path exists for exactly that purpose.

As with block RAM, the clean way to use a DSP slice is to write ordinary arithmetic in the shape the slice implements. The module below multiplies two signed 16-bit numbers and accumulates into 40 bits, with a register after the inputs, after the multiplier and after the adder. Those three registers correspond to the slice's A/B, M and P registers, so Vivado typically packs the whole datapath — multiplier, adder, registers and feedback — into a single DSP48E1, using no LUTs for the arithmetic (only the two small first registers stay in the fabric).

// Pipelined signed multiply-accumulate. 16 x 16 fits the DSP48E1's
// 25 x 18 multiplier and the 40-bit sum fits its 48-bit accumulator,
// so Vivado maps the multiplier, adder and all registers into ONE DSP48E1:
//   a_r, b_r -> AREG/BREG,  m_r -> MREG,  acc -> PREG.
// 'first' marks the first sample of a new sum. Latency: 3 cycles.
module mac16 (
    input  wire               clk,
    input  wire               first,     // 1 = start a new sum with this sample
    input  wire signed [15:0] a,
    input  wire signed [15:0] b,
    output reg  signed [39:0] acc = 40'sd0
);
    reg signed [15:0] a_r = 16'sd0, b_r = 16'sd0;
    reg signed [31:0] m_r = 32'sd0;
    reg               first_r = 1'b0, first_m = 1'b0;

    always @(posedge clk) begin
        // stage 1: input registers
        a_r     <= a;
        b_r     <= b;
        first_r <= first;
        // stage 2: product register
        m_r     <= a_r * b_r;
        first_m <= first_r;
        // stage 3: accumulator (restart or add)
        if (first_m)
            acc <= m_r;                  // sign-extended to 40 bits
        else
            acc <= acc + m_r;
    end
endmodule

-- Pipelined signed multiply-accumulate. 16 x 16 fits the DSP48E1's
-- 25 x 18 multiplier and the 40-bit sum fits its 48-bit accumulator,
-- so Vivado maps the multiplier, adder and all registers into ONE DSP48E1:
--   a_r, b_r -> AREG/BREG,  m_r -> MREG,  acc_i -> PREG.
-- 'first' marks the first sample of a new sum. Latency: 3 cycles.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

entity mac16 is
    port (
        clk   : in  std_logic;
        first : in  std_logic;                  -- 1 = start a new sum with this sample
        a     : in  signed(15 downto 0);
        b     : in  signed(15 downto 0);
        acc   : out signed(39 downto 0)
    );
end entity;

architecture rtl of mac16 is
    signal a_r, b_r          : signed(15 downto 0) := (others => '0');
    signal m_r               : signed(31 downto 0) := (others => '0');
    signal acc_i             : signed(39 downto 0) := (others => '0');
    signal first_r, first_m  : std_logic := '0';
begin
    process (clk)
    begin
        if rising_edge(clk) then
            -- stage 1: input registers
            a_r     <= a;
            b_r     <= b;
            first_r <= first;
            -- stage 2: product register
            m_r     <= a_r * b_r;
            first_m <= first_r;
            -- stage 3: accumulator (restart or add)
            if first_m = '1' then
                acc_i <= resize(m_r, 40);
            else
                acc_i <= acc_i + m_r;           -- m_r is sign-extended to 40 bits
            end if;
        end if;
    end process;

    acc <= acc_i;
end architecture;

The first input starts a new sum: instead of adding to the old value, the accumulator loads the product. Because the product reaches the accumulator two cycles after it entered, first is delayed through two registers alongside the data — a small example of keeping control signals aligned with a pipeline, a theme of Chapter 18. In the slice, this maps onto the ALU's ability to select either P or zero as the second addend.

`timescale 1ns / 1ps
// Two bursts: 200 worst-case products, then a fresh sum of 50 mixed terms.
module mac16_tb;
    reg                clk = 1'b0, first = 1'b0;
    reg  signed [15:0] a = 16'sd0, b = 16'sd0;
    wire signed [39:0] acc;
    reg  signed [39:0] expected;
    integer            n, errors;

    mac16 dut (.clk(clk), .first(first), .a(a), .b(b), .acc(acc));
    always #4 clk = ~clk;

    task feed(input integer len, input integer kind);
        integer k;
        begin
            expected = 0;
            for (k = 0; k < len; k = k + 1) begin
                @(negedge clk);
                first = (k == 0);
                if (kind == 0) begin a = -16'sd32768; b = -16'sd32768; end
                else begin a = k * 1311 - 32000; b = 700 - k * 29; end
                expected = expected + a * b;
            end
            @(negedge clk); first = 1'b0; a = 0; b = 0;   // then only zeros
            repeat (3) @(negedge clk);                     // pipeline latency
        end
    endtask

    initial begin
        errors = 0;
        feed(200, 0);
        $display("burst 1: acc = %0d (expected %0d)", acc, expected);
        if (acc !== expected) errors = errors + 1;
        feed(50, 1);
        $display("burst 2: acc = %0d (expected %0d)", acc, expected);
        if (acc !== expected) errors = errors + 1;
        if (errors == 0) $display("PASS: both sums correct");
        else             $display("FAIL: %0d errors", errors);
        $finish;
    end
endmodule

-- Two bursts: 200 worst-case products, then a fresh sum of 50 mixed terms.
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

entity mac16_tb is
end entity;

architecture sim of mac16_tb is
    signal clk, first : std_logic := '0';
    signal a, b       : signed(15 downto 0) := (others => '0');
    signal acc        : signed(39 downto 0);
    signal done       : boolean := false;

    -- decimal text of a wide signed value (VHDL integers are only 32 bits)
    function to_dec(x : signed) return string is
        variable v : signed(x'length - 1 downto 0) := x;
        variable d : integer;
    begin
        if v < 0 then
            return "-" & to_dec(-v);
        elsif v < 10 then
            return integer'image(to_integer(v));
        else
            d := to_integer(v mod 10);
            return to_dec(v / 10) & integer'image(d);
        end if;
    end function;
begin
    dut : entity work.mac16 port map (clk => clk, first => first, a => a, b => b, acc => acc);
    clk <= not clk after 4 ns when not done;

    stimulus : process
        variable expected : signed(39 downto 0);
        variable errors   : natural := 0;
        variable av, bv   : integer;

        procedure feed(len : natural; kind : natural) is
        begin
            expected := (others => '0');
            for k in 0 to len - 1 loop
                wait until falling_edge(clk);
                if k = 0 then first <= '1'; else first <= '0'; end if;
                if kind = 0 then av := -32768; bv := -32768;
                else av := k * 1311 - 32000; bv := 700 - k * 29; end if;
                a <= to_signed(av, 16);
                b <= to_signed(bv, 16);
                expected := expected + resize(to_signed(av, 16) * to_signed(bv, 16), 40);
            end loop;
            wait until falling_edge(clk);
            first <= '0'; a <= (others => '0'); b <= (others => '0');
            for i in 1 to 3 loop wait until falling_edge(clk); end loop;  -- pipeline latency
        end procedure;
    begin
        feed(200, 0);
        report "burst 1: acc = " & to_dec(acc) & " (expected " & to_dec(expected) & ")";
        if acc /= expected then errors := errors + 1; end if;
        feed(50, 1);
        report "burst 2: acc = " & to_dec(acc) & " (expected " & to_dec(expected) & ")";
        if acc /= expected then errors := errors + 1; end if;
        if errors = 0 then report "PASS: both sums correct";
        else report "FAIL: " & integer'image(errors) & " errors" severity error; end if;
        done <= true;
        wait;
    end process;
end architecture;

Both simulators print burst 1: acc = 214748364800, burst 2: acc = -395935575 and PASS: both sums correct. The first burst adds 200 copies of \((-32768)^2 = 2^{30}\), which needs 39 bits as a signed value: a check that nothing was truncated. The VHDL testbench contains a small to_dec function because VHDL's integer is only 32 bits wide and cannot hold such values.

A synchronous design assumes that the clock edge reaches every flip-flop at the same moment. On a chip several millimetres across, with over a hundred thousand flip-flops, that requires special wiring. The clock therefore never travels on the ordinary routing of Chapter 5. It enters through a clock-capable pin (on the PYNQ-Z2, the 125 MHz oscillator reaches pin H16), passes through a global clock buffer(BUFG), and is distributed by a balanced tree of dedicated low-skew wires to every clock region.

• The device has 32 BUFGs, so at most 32 global clocks; each clock region can receive up to 12 of them.
• A clock region is a horizontal band of the fabric (50 CLBs tall in the 7-series), with its own local distribution. Regional buffers (BUFR, BUFH) serve clocks needed in only one region.
• The difference in arrival time of one clock edge at two flip-flops is called skew. On the global network it is a small fraction of a nanosecond, and the timing analyser (Chapter 15) accounts for it exactly.

Vivado inserts the BUFG for you when a top-level port is used as a clock. What you must not do is create clocks in logic — dividing a clock with a counter, or gating it with an AND gate — because such a signal travels on general routing with large, unknown skew. Chapter 3 introduced the remedy, and Chapter 14 applies it on the board: one clock and clock enables. When you genuinely need a different frequency, the right tool is the clock synthesiser.

Each clock management tile (CMT) contains one MMCM (mixed-mode clock manager) and one PLL (phase-locked loop); the XC7Z020 has four CMTs. Both work on the same principle. A voltage-controlled oscillator (VCO) runs at a high frequency. Its output is divided by \(M\)and compared, in a phase-frequency detector (PFD), with the input clock divided by \(D\). The loop adjusts the VCO until the two match, which forces \(f_\text{VCO} = f_\text{in} M / D\). Output dividers \(O\) then produce up to seven (MMCM) or six (PLL) related clocks from the same VCO.

Why the VCO window? An oscillator only works well within a certain range, and the data sheet guarantees the jitter and lock behaviour only inside it. For a -1 speed-grade device such as the PYNQ-Z2's, the MMCM VCO range is about 600 to 1200 MHz (the PLL's is higher, about 800 to 1600 MHz); the phase detector input\(f_\text{in}/D\) also has limits. The MMCM's feedback multiplier and its first output divider can be fractional, in steps of 1/8, which makes awkward frequencies reachable. Vivado's Clocking Wizard does this search for you; the program below does it by hand so you can see what it involves.

Round numbers are easy: 100, 200, 50, 40 and 300 MHz come out exact with \(M=48, D=5\) (VCO 1200 MHz). The HDMI-style 74.25 and 148.5 MHz clocks are 421 ppm off with integer settings but exact with fractional ones (\(125 \times 37.125 / (5 \times 6.25) = 148.5\)). The program prefers the highest legal VCO frequency among equally accurate solutions, which generally gives lower output jitter. Remember that limits differ by device family and speed grade, so always confirm a setting with the Clocking Wizard, which knows the exact numbers.

MMCME2_ADV / MMCME2_BASE
  CLKIN1_PERIOD    = 8.000     # ns, i.e. 125 MHz
  DIVCLK_DIVIDE    = 5         # D
  CLKFBOUT_MULT_F  = 48.000    # M      -> VCO = 125 * 48 / 5 = 1200 MHz
  CLKOUT0_DIVIDE_F = 6.000     # O0     -> 200 MHz
  LOCKED           -> high once the loop has settled; hold logic in reset until then

The XC7Z020 adds one more, very large hard block: the processing system (PS). It is a complete computer built in hard silicon: two ARM Cortex-A9 cores (650 MHz on the PYNQ-Z2) with floating-point and NEON SIMD units, 32 KB L1 caches per core and a shared 512 KB L2 cache, 256 KB of on-chip RAM, a DDR3 controller (512 MB on the board), and peripherals: USB, Gigabit Ethernet, SD card, UART, I²C, SPI, GPIO and timers. The PS boots first and can configure the programmable logic (PL) at any time.

The PS and PL are joined by many dedicated signals, mainly AXI bus ports in both directions, up to four clocks generated by the PS for the PL, and interrupt lines from the PL to the processors. Part VII is devoted to that bridge: the ARM cores running Python in a Jupyter notebook will read and write registers in your circuits, and move data through block RAMs and DSP pipelines like those of this chapter.