OCUDU RAN Disaggregated FAPI Split

Split the gNB into two processes - odu_low (L1/PHY) and odu_high (L2/MAC) - and connect them through an xFAPI translator-bridge over an xSM (DPDK shared-memory) transport. The split lets L1 and L2 run in different DPDK domains on the same host, or on different servers connected by a dedicated DPDK-Ethernet link.

Highlights

• Process-level FAPI split - odu_high (L2/MAC, F1, scheduler) and odu_low (L1/PHY, OFH, radio) run as independent binaries; FAPI P5/P7 messages are serialised over an xSM ring buffer between them.
• xFAPI translator-bridge - a separate process owns the FAPI translation and message dispatch, decoupling L1 and L2 DPDK domains and enabling cross-host deployments without code changes on either side.
• Two supported topologies - single-host (all three processes on one server, one shared xsm_bridge memzone) and two-host xFAPI split (xFAPI-L1 and xFAPI-L2 connected over DPDK-Ethernet, each host has its own local memzone).
• xSM shared-memory transport - DPDK-backed SPSC ring with hugepage memzones, pair-indexed slot allocation, and a precompiled libxsm.so exposing a small C API.
• Configuration-driven activation - fapi_split_l1 (odu_low) and fapi_split_l2 (odu_high) YAML blocks control device names, pair indices, DPDK proc-type, and file-prefix. The same binary serves both monolithic and split deployments.
• FAPI stats recorder - optional in-memory ring buffer captures every FAPI message (P5 + P7) with direction and PDU summary, dumped as JSON at shutdown for offline analysis.

1. Prerequisites

1.1 Hardware

• x86-64 host with DPDK-capable NIC for the OFH front-haul (single-host topology) or two such hosts connected by a dedicated 25G link (two-host topology).
• For the two-host topology, an additional DPDK-Ethernet-capable NIC (Intel XXV710 / E810 or equivalent) bound to vfio-pci on each host.
• ≥ 2 GB of 1 GiB hugepages on each host (xSM memzones use 1 GiB pages by default).
• The DPDK-Ethernet link NIC (two-host only) and the OFH NIC must be on the same NUMA node as the upper-PHY / cell_rt worker cores.

1.2 Software

1.3 Kernel and hugepage setup

# Kernel boot parameters (then update-grub + reboot):
intel_iommu=on iommu=pt default_hugepagesz=1G hugepagesz=1G hugepages=4

# Mount hugetlbfs if not done by distro:
sudo mkdir -p /mnt/huge
sudo mount -t hugetlbfs -o pagesize=1G none /mnt/huge

# Confirm 1 GiB pages are available:
grep Huge /proc/meminfo

# Load VFIO modules (needed for any DPDK-bound NIC):
sudo modprobe vfio-pci

For the two-host topology, also bind the inter-host link NIC to vfio-pci:

sudo ip link set dev <ifname> mtu 9000          # MUST be jumbo before binding
sudo dpdk-devbind.py --bind=vfio-pci <NIC_PCI_BDF>

2. Architecture overview

2.1 Where the FAPI split sits in the stack

The FAPI boundary is the natural split point between the upper-PHY (L1) and MAC/scheduler (L2). On the monolithic path FAPI is a set of in-process C++ interfaces (p5_requests_gateway, p7_indications_notifier, etc.). The split mode replaces those interfaces with xSM-backed proxies that serialise every message and put it on a shared-memory ring. The receiver process deserialises and dispatches into the local FAPI consumer interfaces.

   L2 (odu_high)                              L1 (odu_low)
   ───────────────                             ───────────────
   MAC, scheduler           FAPI               PHY, OFH, RU
   F1AP / F1U          ◄══════ P5/P7 ══════►    cell_rt
        │                  xSM ring                    │
        │                                              │
   xsm_p5_requests_gateway        ─────►        fapi_xsm_dispatcher
   xsm_p7_requests_gateway        ─────►              │
   xsm_p7_indications_notifier    ◄─────        local PHY notifiers
   xsm_p7_slot_indication_notifier◄─────              │
   xsm_error_indication_notifier  ◄──────►            │

Upstream of FAPI (RLC, PDCP on L2 / OFDM mod-demod, OFH on L1) and downstream of FAPI (the application-level wiring around F1AP and the cell runtime) are unchanged. Only the four FAPI boundary interfaces become xSM proxies.

2.2 xSM transport layering

The xSM library (libxsm.so) sits between lib/ipc/xsm/xsm_context.{cpp,h} (C++ wrapper) and DPDK. The wrapper exposes a small RAII-ish API (open, wait_for_peer, alloc_buffer, put, get, close) and hides the underlying memzone management, slot pairs, and synchronisation primitives.

   ┌─────────────────────────────────────────────────────────────┐
   │ apps/du/fapi_xsm_transport, apps/du/fapi_xsm_proxy          │
   │ (FAPI boundary proxies + RX dispatcher)                     │
   ├─────────────────────────────────────────────────────────────┤
   │ lib/fapi/serialization/fapi_serializer_p5/p7_dl/p7_ul/p7_ind│
   │ (C++ <-> wire-format de/serialisers, one per FAPI message)  │
   ├─────────────────────────────────────────────────────────────┤
   │ lib/ipc/xsm/xsm_context (C++ RAII wrapper)                  │
   ├─────────────────────────────────────────────────────────────┤
   │ libxsm.so (imported shared library, C ABI)                  │
   ├─────────────────────────────────────────────────────────────┤
   │ DPDK (memzone, hugepage, semaphore, EAL)                    │
   └─────────────────────────────────────────────────────────────┘

The serialisation layer is wire-stable and shared by odu_high, odu_low, and xFAPI; all three link the same lib/fapi/serialization static library against their own xSM context to talk on the ring.

2.3 Single-host vs two-host topology

Both topologies use the same OCUDU binaries and the same odu_low_xfapi.yaml / odu_high_xfapi.yaml configs. The xFAPI config file and three knobs on the L2 side (xsm_pair_index, dpdk_proc_type, xsm_file_prefix) select between them.

Single-host

Single-host FAPI split. odu_low (PRIMARY · SLAVE) creates the shared xsm_bridge memzone with two slot pairs. xFAPI (SECONDARY · MASTER + SLAVE) attaches to both, and odu_high (SECONDARY · MASTER) attaches to pair 1. All three processes share a single DPDK domain via file-prefix=gnb0.

Two-host

Two-host xFAPI split. Each host has its own local xsm_bridge memzone with a single slot pair. The two xFAPI instances forward FAPI traffic over the DPDK-Ethernet wire link, transparently to odu_low and odu_high. The L1 host uses file-prefix=gnb0; the L2 host uses file-prefix=gnb0_l2.

3. Implementation summary

The split is implemented in three pieces that plug into the existing FAPI plumbing:

1. xSM proxy notifiers and gateways. On L2, the split-6 plugin (split6_plugin_xsm) replaces the dummy adaptor with one whose P5/P7 gateways serialise outgoing requests onto the local xSM ring. On L1, split6_o_du_low_plugin_xsm returns adaptors whose notifier getters point at xSM-backed senders, so any indication produced by the local PHY flows back over xSM to L2.

2. Wire-stable FAPI serialisation. lib/fapi/serialization/ contains one serialize / deserialize pair per FAPI message type (param, config, start, stop, error, slot ind, DL/UL TTI, UL DCI, TX data, RX data ind, CRC ind, UCI ind, SRS ind, RACH ind). They use a hand-written binary format with a 48-byte fapi_xsm_msg_header (msg_type, msg_len, time_stamp, scatter-gather pointers) followed by a flat payload. Strings, optionals, intervals, bitsets, and static_vector have first-class encoders. Deserialisation is hardened against torn reads (slot-point sanity guard, vector size clamp).

3. Single dispatcher thread per process. apps/du/fapi_xsm_transport.cpp owns one RX thread per process, pinned to a configured CPU at SCHED_FIFO priority. The thread does xsm_get in a busy-wait loop, deserialises the header to dispatch on msg_type, and calls into the local FAPI consumer. Outgoing path is lock-free per producer (the FAPI proxy on the calling thread serialises straight into an xSM buffer it allocated with xsm_alloc, then xsm_puts).

A few specifics worth knowing:

• Slot-indication deduplication. The xSM RX path sees slot indications strictly in order, but a torn buffer can yield a stale slot_count. The dispatcher tracks the last successfully dispatched count and skips duplicates without crashing.
• P5 async coroutines. mac_fapi_sector_fastpath_adaptor_impl drives P5 start/stop through a fifo_async_task_scheduler. Without that scheduler the async_task<bool> returned by the P5 operation controller would suspend immediately and never run; the queue depth is set to 4 to absorb a few in-flight start/stop wrappers per sector.
• Promiscuous OFH mode. Some RUs (notably the Liteon used in our deployments) transmit U-Plane from a different MAC than ru_mac_addr. A YAML toggle (enable_promiscuous: true) skips the Ethernet source-MAC equality check while keeping destination-MAC, VLAN, and ethertype checks.
• DPDK file-prefix coupling. On the L1 host, both odu_low (primary) and xFAPI-L1 (secondary) must share --file-prefix=gnb0. On the L2 host (two-host topology), xFAPI-L2 (primary) and odu_high (secondary) share --file-prefix=gnb0_l2. Cross-host the link NIC must also be listed in odu_low_xfapi.yaml’s hal.eal_args so the L1 secondary inherits visibility of it.

4. Build xFAPI from coranlabs

odu_low and odu_high link against libxsm.so and include xsm/xsm.h. Both artifacts come from the xFAPI source tree. Build xFAPI first.

4.1 Clone xFAPI

cd ~
git clone https://github.com/coranlabs/xfapi.git
cd xfapi

If your account does not have access to the repository, request it from the coRAN Labs maintainers. The main branch is what this release targets.

4.2 Build xFAPI

xFAPI ships a top-level build script that configures and compiles the project in one step. Build it with the ocudu_ocudu mode so the translator-bridge is wired for an OCUDU L1 ↔ OCUDU L2 pairing:

cd ~/xfapi
./build_xfapi.sh --mode=ocudu_ocudu

The script produces:

• bin/xfapi_main - the translator-bridge binary.
• src/ipc/xsm/xsm/libxsm.so - the xSM shared library that OCUDU also links against.
• src/ipc/xsm/xsm/xsm.h - the public xSM C header.

# Verify the artifacts:
ls -lh bin/xfapi_main
ls -lh src/ipc/xsm/xsm/libxsm.so
ls -lh src/ipc/xsm/xsm/xsm.h

If build_xfapi.sh fails on DPDK detection, set PKG_CONFIG_PATH to the DPDK install location (e.g. export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig) and re-run the script.

4.3 Copy the xSM artifacts into the OCUDU tree

OCUDU expects the precompiled xSM library under include/ocudu/xsm/. The CMake glue at include/ocudu/xsm/CMakeLists.txt creates an IMPORTED shared-library target pointing at ${CMAKE_CURRENT_SOURCE_DIR}/libxsm.so and uses the same directory as the include root for xsm/xsm.h.

cd ~/OCUDU          # the fapi_split checkout from Section 5.1

# 1) Copy the precompiled xSM shared library next to its CMakeLists.txt:
cp ~/xfapi/src/ipc/xsm/xsm/libxsm.so   include/ocudu/xsm/

# 2) Copy the public xSM C header into include/ocudu/xsm/xsm/:
mkdir -p include/ocudu/xsm/xsm
cp ~/xfapi/src/ipc/xsm/xsm/xsm.h       include/ocudu/xsm/xsm/

# 3) Confirm the layout:
ls include/ocudu/xsm/
# Expected:
#   CMakeLists.txt
#   libxsm.so
#   xsm/
#
ls include/ocudu/xsm/xsm/
# Expected:
#   xsm.h

Both files are deliberately not tracked in the OCUDU release repository - the source of truth for them is the xFAPI repo. Re-copy them whenever xFAPI is rebuilt against a new DPDK version or whenever the xSM API changes.

If either file is missing, the OCUDU CMake configure will succeed but the link step for ocudu_xsm_context will fail with:

ld: cannot find -lxsm

or the compile step will fail with:

fatal error: xsm/xsm.h: No such file or directory

5. Build OCUDU

5.1 Clone OCUDU and check out the FAPI split branch

cd ~
git clone https://github.com/ocudu-India/OCUDU.git
cd OCUDU
git checkout fapi_split

The fapi_split branch is the one that carries the xSM FAPI-split path. Copy the xSM artifacts into this tree as described in Section 4.3 before building.

5.2 Default build (FAPI split + DPDK)

cd ~/OCUDU
./build.sh

build.sh wraps the standard CMake configure + parallel make with the release defaults:

DU_SPLIT_TYPE  = SPLIT_7_2
ENABLE_DPDK    = True
ASSERT_LEVEL   = MINIMAL

ENABLE_XSM_FAPI_SPLIT defaults to ON in CMakeLists.txt, so the FAPI split path is built without any extra flag. The resulting binaries are:

build/apps/du_low/odu_low      # L1/PHY
build/apps/du/odu              # L2/MAC (also serves as the monolithic gNB)

5.3 Manual build (custom flags)

If you want to override build.sh:

cd ~/OCUDU
mkdir -p build && cd build

cmake -DDU_SPLIT_TYPE=SPLIT_7_2 \
      -DENABLE_DPDK=True \
      -DENABLE_XSM_FAPI_SPLIT=ON \
      -DASSERT_LEVEL=MINIMAL \
      ..

make -j$(nproc) odu odu_low

5.4 Verify the build picked up xSM

After build, confirm the imported xSM target was resolved:

nm -D include/ocudu/xsm/libxsm.so | grep -i xsm_open | head -3
ldd build/apps/du_low/odu_low | grep libxsm

The ldd line should resolve libxsm.so to the file under include/ocudu/xsm/. If it shows not found, set LD_LIBRARY_PATH to include that directory before launching:

export LD_LIBRARY_PATH=$PWD/include/ocudu/xsm:$LD_LIBRARY_PATH

6. Configuration

Two YAML configs cover both topologies. Both are committed under configs/.

6.1 Key knobs

fapi_split_l1 (odu_low):

fapi_split_l1:
  rx_cpu: 29              # isolated core for the xSM RX worker
  rx_priority: 85         # SCHED_FIFO priority (below OFH timing=90 and cell_rt=89)
  xsm_device_name: xsm_bridge
  dpdk_proc_type: primary
  xsm_pair_index: 0
  xsm_num_pairs: 2        # 1 standalone, 2 with xFAPI bridge

fapi_split_l2 (odu_high):

fapi_split_l2:
  rx_cpu: 14
  rx_priority: 85
  xsm_device_name: xsm_bridge
  xsm_pair_index: 0       # 0 on a two-host L2, 1 on single-host bridge
  xsm_file_prefix: gnb0_l2
  dpdk_proc_type: secondary

fapi_stats (both sides, optional):

fapi_stats:
  enabled: false                                 # flip to true to record
  output_path: ./logs/odu_low_fapi_stats.json    # auto-timestamped
  add_timestamp: true

6.2 Topology-specific settings

For the two-host topology the L1 host’s hal.eal_args must include the inter-host link NIC PCI BDF (-a 0000:51:00.0 in our reference setup). xFAPI-L1 runs as DPDK secondary; it can only see NICs the primary (odu_low) has probed.

7. Run

7.1 Single-host bridge

# Terminal 1 - start L1 (creates the shared xSM memzone)
./build/apps/du_low/odu_low -c configs/odu_low_xfapi.yaml

# Terminal 2 - start xFAPI bridge (attaches to both pairs)
~/xfapi/bin/xfapi_main --cfgfile ~/xfapi/conf/ocudu_ocudu_config.yaml

# Terminal 3 - start L2 (attaches as master on pair 1)
./build/apps/du/odu -c configs/odu_high_xfapi.yaml

Startup order is enforced by DPDK: secondary processes (xFAPI, odu_high) need the primary’s memzone to exist. If you start them in the wrong order, the secondary will fail with cannot find memzone "xsm_bridge" and exit.

7.2 Two-host xFAPI split

On the L1 host:

./build/apps/du_low/odu_low -c configs/odu_low_xfapi.yaml
~/xfapi/bin/xfapi_main --cfgfile ~/xfapi/conf/ocudu_ocudu_split_l1.yaml

On the L2 host:

~/xfapi/bin/xfapi_main --cfgfile ~/xfapi/conf/ocudu_ocudu_split_l2.yaml
./build/apps/du/odu -c configs/odu_high_xfapi.yaml

xFAPI-L1 will block at startup waiting for the wire link to come up; this is expected, the E810 PHY needs an active peer before the link is declared up. Start xFAPI-L2 in parallel on the L2 host.

7.3 Startup verification

On odu_low stdout you should see:

--== OCUDU DU low (xSM) (commit <sha>) ==--
EAL: Multi-process socket /var/run/dpdk/gnb0/mp_socket
...
[xSM] device="xsm_bridge" role=SLAVE dpdk_proc_type=primary - waiting for peer for up to 60s...
==== DU low (xSM) started ====

On xfapi_main stdout (single-host config):

[OCUDU_BRIDGE] DPDK EAL ready (file-prefix=gnb0, role=secondary).
[OCUDU_BRIDGE] split.role=L1 -> split-mode init.
[OCUDU_SPLIT] Memzone xSM handle open (device='xsm_bridge', pair=0).
[OCUDU_SPLIT] DPDK-Eth xSM handle open (port_id=0, ...).

On odu_high stdout:

[xSM] device="xsm_bridge" role=MASTER pair=1 dpdk_proc_type=secondary
[xSM] peer connected

Once all three processes are up, FAPI traffic flows as soon as the first cell starts; you can observe it via tcpdump-style hex dumps if fapi_stats.enabled: true was set, or by enabling the fapi_split_trace log (writes to ./logs/fapi_split_trace.log).

8. Metrics and observability

8.1 FAPI stats recorder

Enable fapi_stats in either or both YAMLs to dump every FAPI message into a JSON file at shutdown. The recorder is a fixed-size lock-free ring (100 000 entries by default) that captures direction, message type, SFN/slot, PDU summary, and a one-shot timestamp.

fapi_stats:
  enabled: true
  output_path: ./logs/odu_low_fapi_stats.json
  add_timestamp: true     # appends _YYYYMMDD_HHMMSS before .json

At process exit the file is written with one JSON object per recorded message. Useful for offline diffing against an external pcap or against the xFAPI-side dashboard.

8.2 Split trace log

For ad-hoc debugging, lib/support/fapi_split_trace.cpp provides a thread-safe append-mode trace that selected hot-path sites can write into without going through the normal logger. The output (./logs/fapi_split_trace.log) captures the moments where MAC hands off to FAPI and where the FAPI translator hands off to the PHY. It is silent by default; enable per-site via the source-level switches in the respective files.

8.3 Standard logger

The usual OCUDU log block applies; the FAPI split paths emit at info and warning levels. Production runs should keep all_level: warning because per-slot info-level lines from SCHED/MAC/PHY at FAPI rate saturate the logger queue and starve cell_rt.

9. Deployment checklist

• DPDK ≥ 22.11 installed on every host, ABI-compatible with the xFAPI build (Section 1.2).
• Kernel boot args set for IOMMU and 1 GiB hugepages, hugetlbfs mounted (Section 1.3).
• For two-host topology: inter-host link NIC bound to vfio-pci on both hosts, MTU 9000 set before binding (Section 1.3).
• xFAPI cloned from coranlabs/xfapi and built (Section 4.1, 4.2).
• OCUDU cloned from ocudu-India/OCUDU and checked out to the fapi_split branch (Section 5.1).
• libxsm.so and xsm/xsm.h copied from ~/xfapi/src/ipc/xsm/xsm/ into include/ocudu/xsm/ (Section 4.3).
• OCUDU built via ./build.sh (Section 5.2); ldd odu_low | grep libxsm resolves.
• YAML configs adjusted for the chosen topology (Section 6.2). For two-host: L1 hal.eal_args includes the inter-host link NIC PCI BDF.
• Startup order respected: odu_low first, then xFAPI, then odu_high (Section 7.1 / 7.2).
• On startup logs, confirm xSM peer connected appears on both odu_low and odu_high before expecting cell activity.
• If running the FAPI stats recorder for analysis, confirm output_path is writable and timestamping is enabled.

10. References

• coRAN Labs xFAPI repository: https://github.com/coranlabs/xfapi
• DPDK Multi-process support: https://doc.dpdk.org/guides-25.11/prog_guide/multi_proc_support.html
• DPDK Memzone API: https://doc.dpdk.org/guides-25.11/prog_guide/memory.html
• 3GPP TS 38.211 - Physical channels and modulation.
• 3GPP TS 38.213 - Physical layer procedures for control.
• SCF 222.10 - Small Cell Forum 5G FAPI: PHY API specification (the FAPI message reference).