max2771-raw-ethernet-recorder/README.md

# MAX2771 processor-free raw Ethernet recorder

This repository contains one protocol-compatible recorder implementation with
two FPGA board targets:

| Target | FPGA/tool | PHY | Hardware status |
|---|---|---|---|
| Digilent Genesys 2 | Kintex-7 `xc7k325tffg900-2`, Vivado | RTL8211E-VL | synthesis, implementation and simulation verified |
| Efinix Ti375 C529 kit | Titanium `Ti375C529`, Efinity | RTL8211FDI | built, programmed and tested over a real 1 Gb/s cable |

Both targets use exactly the same handwritten capture, FIFO, packetizer,
Ethernet MAC, CRC and MDIO controller RTL. They differ only where the board,
FPGA family or Ethernet PHY genuinely differs. There is no CPU, PS, RISC-V,
MicroBlaze, AXI, block design, generated Ethernet/FIFO IP, IP stack, ARP, IP,
UDP or TCP.

## What is generic and what is board-specific?

The portable modules live only in `rtl/common/`; they are not duplicated under
either target:

| Common module | Purpose |
|---|---|
| `max2771_capture.sv` | captures each 4-bit word, assigns its 64-bit index and counts loss |
| `async_fifo_gray.sv` | handwritten dual-clock FIFO with binary/Gray pointers and two-stage CDC |
| `gray_counter_sync.sv` | coherently transfers monotonically increasing counters |
| `pulse_sync.sv` | safe event transfer between unrelated clock domains |
| `reset_sync.sv` | asynchronous assertion, synchronous deassertion |
| `ethernet_packetizer.sv` | builds the 32-byte MX27 header and packs two samples per byte |
| `ethernet_tx_mac.sv` | emits preamble, SFD, Ethernet header, padding, FCS and IFG |
| `ethernet_crc32.sv` | reflected IEEE Ethernet CRC-32 calculation |
| `mdio_master.sv` | handwritten IEEE Clause-22 MDIO transaction engine |
| `phy_reset_ctrl.sv` | deterministic millisecond PHY reset timing |

Board-specific sources are deliberately separate:

- `targets/genesys2/rtl/` contains the Kintex-7 top-level, primitive MMCM,
  Xilinx ODDR RGMII output and RTL8211E PHY-status decoder.
- `targets/efinix_ti375_c529/rtl/` contains the Ti375 top-level, Efinity
  GPIO-DDIO mapping, RTL8211FDI PHY-status decoder and a separate hardware
  Ethernet self-test top-level.
- Each target owns its constraints, pin/periphery generation, build scripts,
  programming script, target tests and hardware notes.

Changing a common protocol/data-path module affects both targets. Changing a
file below one target affects only that board. The build scripts use explicit
source lists and never glob another target's RTL.

## Repository hierarchy

```text
data_recorder/
├── rtl/common/                    portable synthesizable RTL used by both boards
├── tb/common/                     portable self-checking protocol/FIFO/MAC tests
├── scripts/run_common_sim.sh      common Questa regression
├── targets/
│   ├── genesys2/
│   │   ├── rtl/                   Kintex-7 top, MMCM, ODDR, RTL8211E logic
│   │   ├── constraints/           verified Genesys 2 XDC
│   │   ├── scripts/               non-project Vivado build/program scripts
│   │   ├── docs/                  Genesys pin and timing assumptions
│   │   ├── Makefile
│   │   └── README.md
│   └── efinix_ti375_c529/
│       ├── rtl/                   Ti375 production and self-test tops
│       ├── interface/             Efinity Interface Designer Python API script
│       ├── constraints/           handwritten CDC/clock SDC additions
│       ├── scripts/               build, timing check, program and simulation
│       ├── tb/                    PHY and RGMII target tests
│       ├── docs/                  Efinity flow and hardware assumptions
│       ├── Makefile
│       └── README.md
├── host/                           Linux AF_PACKET/TPACKET_V3 receiver
├── docs/raw_ethernet_protocol.md   byte-exact board-independent wire protocol
├── docs/hardware_assumptions.md    cross-target assumptions/sign-off index
├── docs/timing_constraints.md      cross-target timing/CDC index
├── constraints/                    Genesys XDC compatibility entry point
├── Makefile                        convenient entry points for both targets
└── build/                          generated locally; ignored by Git
```

The root `scripts/build_vivado.tcl`, `scripts/program_fpga.tcl` and
`constraints/genesys2_max2771_recorder.xdc` paths are compatibility entry
points required by the original Genesys flow. They source the authoritative
files under `targets/genesys2/`; there is still only one implementation to
maintain.

## End-to-end architecture

```text
MAX2771 4-bit bus
      │ one word per selected RF clock edge
      ▼
input DDR/IO registers → sample index/loss tag → 16,384 × 69 async FIFO
      │                                             │
      │ RF clock domain                             │ 125 MHz domain
      └──────────────── Gray-pointer CDC ───────────┘
                                                    ▼
                                2936-sample packetizer
                                                    ▼
                          raw Ethernet MAC + CRC/FCS + IFG
                                                    ▼
                              GMII byte → RGMII DDR adapter
                                                    ▼
                         onboard 1000BASE-T PHY → Linux NIC
                                                    ▼
                         AF_PACKET/TPACKET_V3 → packed `.bin`
```

The MAX2771 cannot be backpressured. If the FIFO fills, the current sample is
not written over unread data; the drop counter increments and the first later
successful sample carries a discontinuity marker.

## Raw Ethernet protocol

- destination: `ff:ff:ff:ff:ff:ff`
- source: `02:27:71:00:00:01`
- EtherType: `0x88b5`
- standard payload: 1500 bytes
- custom header: 32 bytes, magic `MX27`, network byte order
- sample data: 1468 bytes = 2936 four-bit samples
- packing: earlier sample in `[7:4]`, later sample in `[3:0]`
- no VLAN and no jumbo frames

The FPGA emits preamble, SFD, FCS and 12-byte IFG. A normal Linux NIC removes
preamble/SFD/FCS before exposing the frame to `AF_PACKET`. The complete header,
flags and byte offsets are in [the raw protocol specification](docs/raw_ethernet_protocol.md).

## Prerequisites

Common host/simulation tools:

- Linux with GCC and GNU Make;
- root or `CAP_NET_RAW` for live capture;
- QuestaSim for the checked HDL regression (Icarus-compatible common RTL can
  also be integrated independently);
- a dedicated Gigabit Ethernet interface and storage faster than 16.368 MB/s
  for the maximum specified sample rate.

Genesys 2 additionally requires Vivado supporting Kintex-7. Efinix requires an
Efinity installation supporting Ti375; this workstation uses
`/tools/efinix/efinity/2026.1`.

## First checkout and common tests

From the repository root:

```bash
make host
make host-test
make sim
```

`make sim` tests CRC, complete frame bytes/FCS/IFG, randomized asynchronous
FIFO wrap/full/empty, sample reconstruction, packet sequence/index propagation,
overflow handling and reset behavior. Target-specific test commands are shown
below.

## Genesys 2: wiring, bitstream and programming

Detailed target documentation is in
[targets/genesys2/README.md](targets/genesys2/README.md).

MAX2771-to-JA wiring, all at 3.3 V LVCMOS:

| Signal | FPGA ball | Connector |
|---|---:|---:|
| RF clock | U27 | JA[0] |
| data 0 | U28 | JA[1] |
| data 1 | T26 | JA[2] |
| data 2 | T27 | JA[3] |
| data 3 | T22 | JA[4] |

Build from the repository root:

```bash
make genesys2-build
```

This runs Vivado in memory/non-project mode, synthesizes, places, routes,
checks timing/CDC/DRC and writes:

```text
build/genesys2/vivado/max2771_recorder.bit
build/genesys2/reports/*.rpt
```

Program volatile FPGA configuration over Digilent JTAG:

```bash
make genesys2-program
```

Or program a selected bitstream explicitly:

```bash
vivado -mode batch \
  -source targets/genesys2/scripts/program_fpga.tcl \
  -tclargs build/genesys2/vivado/max2771_recorder.bit
```

## Efinix Ti375 C529: wiring, bitstream and programming

Detailed target documentation is in
[targets/efinix_ti375_c529/README.md](targets/efinix_ti375_c529/README.md).

MAX2771-to-PMOD0/J13 wiring, all at `3.3_V_LVCMOS`:

| Signal | FPGA ball/resource | Connector |
|---|---|---:|
| RF clock | A12 / `GPIOR_95_CLK12` | J13.10 |
| data 0 | G16 / `GPIOR_89` | J13.2 |
| data 1 | F16 / `GPIOR_90` | J13.3 |
| data 2 | F17 / `GPIOR_91` | J13.4 |
| data 3 | G17 / `GPIOR_92` | J13.7 |

Set the tool location and build:

```bash
export EFINITY_HOME=/tools/efinix/efinity/2026.1
make efinix-build
```

The GUI-independent Efinity CLI flow generates temporary project/periphery
files, runs Interface Designer DRC, synthesis, P&R, timing/CDC checks and writes:

```text
targets/efinix_ti375_c529/build/outflow/ti375_max2771_recorder.bit
targets/efinix_ti375_c529/build/outflow/ti375_max2771_recorder.hex
```

The direct RF GCLK is constrained to 30.547 ns by default. A slower source,
including the hardware-tested ~16.368 MHz mode, works without rebuilding. For
a genuinely different maximum input frequency, rebuild with its period:

```bash
RF_CLOCK_PERIOD_NS=61.094 make efinix-build   # example 16.368 MHz constraint
```

Keeping the default 30.547 ns constraint while operating at 16.368 MHz is
conservative and recommended if later operation may return to 32.736 MHz.

Program the volatile JTAG configuration:

```bash
export EFINITY_HOME=/tools/efinix/efinity/2026.1
make efinix-program
```

The connected board was verified as JTAG ID `0x006a0a79`. Programming does not
write SPI boot flash.

### Efinix Ethernet hardware self-test

To test PHY/MDIO/RGMII/MAC/protocol without a MAX2771 clock:

```bash
export EFINITY_HOME=/tools/efinix/efinity/2026.1
make efinix-selftest-build
make efinix-selftest-program
```

This sends the repeating nibble sequence `0,1,...,15`. Capture it with the host
program and check that the file repeats `01 23 45 67 89 ab cd ef`. Restore the
production recorder afterwards:

```bash
make efinix-program
```

The real-board self-test received 35,446 consecutive frames and 104,069,456
samples in 3.034 seconds with zero frame gap, sample gap, FPGA drop, socket
drop or pattern mismatch.

## Linux receiver build

```bash
make -C host clean all
make -C host test
```

The compilation command is equivalent to:

```bash
gcc -O3 -std=c11 -Wall -Wextra -Wpedantic \
  -o host/max2771_raw_rx host/max2771_raw_rx.c
```

The receiver first requests `TPACKET_V3`/`PACKET_MMAP` and falls back to a
polling `recvfrom()` path when unavailable. Both paths validate source/filter,
EtherType, magic, version, header size, payload length, packet sequence, sample
index and FPGA-reported drop count.

Grant capability once to run without full root:

```bash
sudo setcap cap_net_raw,cap_net_admin=eip host/max2771_raw_rx
getcap host/max2771_raw_rx
```

Rebuilding the executable removes file capabilities; repeat `setcap` after a
clean rebuild. Alternatively run the receiver with `sudo`.

## Permanent Linux interface setup for raw Layer-2

No IP address is needed. NetworkManager's default DHCP profile may display an
“Activation failed” popup because an FPGA does not run DHCP. Replace the profile
for the dedicated NIC, shown here as `enp3s0`, with a persistent Layer-2-only
profile:

```bash
nmcli connection show
nmcli connection modify 'Wired connection 1' \
  connection.id 'FPGA Raw Ethernet' \
  connection.interface-name enp3s0 \
  connection.autoconnect yes \
  connection.autoconnect-retries 0 \
  ipv4.method disabled ipv4.never-default yes ipv4.ignore-auto-dns yes \
  ipv6.method disabled ipv6.never-default yes ipv6.ignore-auto-dns yes \
  connection.mdns no connection.llmnr no
nmcli connection up 'FPGA Raw Ethernet' ifname enp3s0
```

Verify the direct cable:

```bash
ip -br link show enp3s0
ethtool enp3s0
```

It must report `1000Mb/s`, full duplex and link detected. The FPGA deliberately
does not transmit at 100 Mb/s because 32.736 Msps × 4 bits already exceeds a
100 Mb/s link before Ethernet overhead.

Optional performance setup:

```bash
sudo ethtool -K enp3s0 gro off lro off
sudo ethtool --set-eee enp3s0 eee off
sudo sysctl -w net.core.rmem_max=268435456
```

Not every NIC supports every offload/EEE switch; `Operation not supported` is
not fatal. Firewall changes are normally irrelevant because these frames are
below IP. Prefer a direct cable or isolated switch because all FPGA frames are
broadcast.

## Starting a capture

Normal unlimited capture with zero-fill gap preservation:

```bash
sudo host/max2771_raw_rx \
  --interface enp3s0 \
  --output max2771_capture.bin \
  --source-mac 02:27:71:00:00:01 \
  --gap-mode zero-fill
```

Ten-second diagnostic capture:

```bash
sudo host/max2771_raw_rx \
  --interface enp3s0 \
  --output capture_10s.bin \
  --log capture_10s.log \
  --source-mac 02:27:71:00:00:01 \
  --gap-mode stop \
  --duration 10 \
  --verbose
```

Stop automatically at one GiB:

```bash
sudo host/max2771_raw_rx \
  --interface enp3s0 \
  --output capture_1GiB.bin \
  --max-bytes 1073741824 \
  --gap-mode zero-fill \
  --quiet
```

`SIGINT`/Ctrl-C and `SIGTERM` stop cleanly, flush the buffered output and write
the sidecar log.

### Receiver options

| Option | Meaning |
|---|---|
| `--interface IFACE` | required Linux interface |
| `--output FILE` | packed sample bytes; default `max2771_capture.bin` |
| `--log FILE` | text statistics log; default `OUTPUT.log` |
| `--source-mac MAC` | accept only the specified FPGA source |
| `--gap-mode stop` | terminate on the first discontinuity |
| `--gap-mode skip` | report gaps and concatenate received data |
| `--gap-mode zero-fill` | insert zero nibbles to preserve sample time; default |
| `--duration SECONDS` | wall-clock capture limit |
| `--max-bytes BYTES` | output-size limit |
| `--quiet` | final log only |
| `--verbose` | print every accepted frame |

Avoid `--verbose` during long high-rate captures because terminal output itself
can become a performance bottleneck.

## Output files

The `.bin` file contains only packed samples. It contains no Ethernet header,
MX27 header, preamble, SFD or FCS:

```text
byte 0 [7:4] = sample 0
byte 0 [3:0] = sample 1
byte 1 [7:4] = sample 2
byte 1 [3:0] = sample 3
...
```

To unpack in Python:

```python
data = open("max2771_capture.bin", "rb").read()
samples = [n for b in data for n in (b >> 4, b & 0x0f)]
```

The text log records start time, interface, observed source MAC, protocol
version, first/last sample indices, frame gaps, FPGA drops, host-observed sample
gaps, bytes/samples written, elapsed time, average rate and packet-socket drops.

## Throughput and FIFO behavior

At the maximum requested rate:

```text
32.736 Msps × 4 bits       = 130.944 Mb/s raw
32.736 Msps / 2936         = 11,149.864 frames/s
Ethernet wire consumption  ≈ 137.188 Mb/s = 13.719% of 1 Gb/s
sample-file write rate     = 16.368 MB/s ≈ 58.9 GB/hour
```

At the tested ~16.368 MHz RF clock, raw input is approximately 65.472 Mb/s and
the sample file grows at about 8.184 MB/s. The packet format and RTL do not
change with RF clock frequency; the host derives continuity from sample index,
not from a hard-coded rate.

The packetizer starts only with at least 2936 FIFO entries. Its conservative
three-cycle read handshake plus Ethernet transmission sustains about 35.47
Msps, providing margin at 32.736 Msps. The 16,384-entry FIFO cannot cover
multi-second PHY auto-negotiation. Capture therefore may report initial FPGA
drops if the MAX2771 clock starts before link-ready. Start RF after Ethernet
link if a loss-free sample-zero boundary is required.

## LED meanings

Genesys 2 uses eight LEDs; see its target README for the complete table. Efinix
has five verified LEDs:

| Efinix LED | Meaning |
|---|---|
| LED9 | Ethernet PLL/system ready |
| LED10 | qualified 1000/full link ready |
| LED11 | RF sample activity |
| LED12 | FIFO/frame activity |
| LED13 | overflow/sample loss latched |

## Timing and hardware sign-off

Both checked builds close internal setup and hold timing and report no
unexpected CDC warnings. A clean Vivado 2026.1 Genesys 2 build on 1 August 2026
reports +0.212 ns worst setup slack and +0.090 ns worst hold slack. The clean
Efinity 2026.1 A12 production build reports +0.636 ns Ethernet setup, +0.094 ns
Ethernet hold, +25.614 ns RF setup and +0.031 ns RF hold at the nominal
32.736 MHz constraint; its CDC report says `No Synchronizer warnings to
report`.

These numbers do not fabricate MAX2771 external timing. Final source-synchronous
input sign-off still requires the MAX2771 clock-to-data min/max values plus
cable/adapter skew. Each target's hardware/timing document identifies exactly
where measured constraints belong.

## Troubleshooting

- **Activation failed popup:** use the NetworkManager Layer-2 profile above;
  do not wait for DHCP.
- **No link:** check cable, PHY reset, board power and `ethtool`; only
  1000/full is accepted.
- **Link but no frames:** confirm RF clock/data are present and the link-ready
  LED is on. Use the Efinix internal self-test to isolate RF input from Ethernet.
- **Overflow LED:** capture started before link or transmission stopped. The
  drop counter and sample indices quantify the discontinuity.
- **Socket drops:** enlarge receive buffers, disable unsupported offloads where
  possible, use a dedicated NIC and verify storage speed.
- **Only a subset of nibble values:** RTL records pins literally. Check MAX2771
  output-mode configuration, signal wiring and logic levels before changing bit
  interpretation.
- **Timing failure after a change:** inspect the target timing/CDC reports; do
  not add broad false paths.
- **Efinity programmer finds no target:** verify `lsusb -d 0403:6011`, board
  power and the programming USB cable.

## Generated files and version control

Build products, tool work directories, reports, FPGA images, the compiled host
binary and capture `.bin` files are ignored by Git. A fresh checkout recreates
them with the commands in this README. Source-controlled files contain RTL,
constraints, testbenches, scripts, C source and documentation only.