TRIBE v2 × RTX 5090

1. What this test exercises

TRIBE v2 is a multimodal brain-response predictor, not a chat model and not a built-in virality scorer. For a video, the released demo pipeline extracts audio, transcribes speech into timed word events with WhisperX, computes visual features, audio features, and contextualized text features, then predicts one cortical activity map per second.

The official notebook also describes DINOv2 visual extraction. The active public cortical checkpoint config selects the text/audio/video feature path above. Treat DINOv2 as an optional follow-up until the first run confirms it is needed by the installed extractor stack.

2. Known RTX 5090 compatibility wrinkle

Reference: PyTorch 2.7 release notes.

3. NAS snapshot already prepared

Snapshot root

~/agents/models/facebook/tribev2 on the NAS-backed agents share

Weights directory

~/agents/models/facebook/tribev2/weights

Official source directory

~/agents/models/facebook/tribev2/source

Total snapshot size

680M

Checkpoint size

708,856,138 bytes

Checkpoint SHA-256

9c79ffff6b642b7b0c71d558c935fb3fa33f2788bfb509feead94fafbba2f321

Official source revision

34f52344e5ba96660fac877393e1954e399d3ef3

The rig may mount the NAS at a different location. The commands below deliberately use /path/to/mounted/agents as a placeholder rather than exposing NAS connection details.

4. Preflight checklist

Before touching Python, confirm the rig has a recent NVIDIA driver, Python 3.11, uv, Git, and ffmpeg. The commands assume Linux and a fresh work directory under ~/Developer.

nvidia-smi
python3.11 --version
uv --version
git --version
ffmpeg -version | head -n 2

# Optional Ubuntu package install if ffmpeg or git is missing:
sudo apt-get update
sudo apt-get install -y ffmpeg git

Create stable paths for this spike:

export NAS_AGENTS_ROOT="/path/to/mounted/agents"
export NAS_TRIBE_ROOT="$NAS_AGENTS_ROOT/models/facebook/tribev2"
export WORK_ROOT="$HOME/Developer/tribev2-smoke"
export TRIBE_MODEL_DIR="$WORK_ROOT/model"
export TRIBE_CACHE="$WORK_ROOT/feature-cache"
export HF_HOME="$WORK_ROOT/hf-cache"
export TRIBE_OUT="$WORK_ROOT/outputs"
export TRIBE_VIDEO="$WORK_ROOT/inputs/sintel_trailer-480p.mp4"

mkdir -p "$WORK_ROOT" "$TRIBE_MODEL_DIR" "$TRIBE_CACHE" "$HF_HOME" "$TRIBE_OUT" "$WORK_ROOT/inputs"

# Copy the official snapshot onto local NVMe for the smoke test.
rsync -a "$NAS_TRIBE_ROOT/source/" "$WORK_ROOT/tribev2/"
rsync -a "$NAS_TRIBE_ROOT/weights/" "$TRIBE_MODEL_DIR/"

# Verify the local checkpoint before running it.
printf '%s  %s\n' \
  '9c79ffff6b642b7b0c71d558c935fb3fa33f2788bfb509feead94fafbba2f321' \
  "$TRIBE_MODEL_DIR/best.ckpt" | sha256sum --check -

git -C "$WORK_ROOT/tribev2" rev-parse HEAD

Expected verification output:

/home/.../Developer/tribev2-smoke/model/best.ckpt: OK
34f52344e5ba96660fac877393e1954e399d3ef3

5. Create the Python environment

Work only inside the local source copy. Do not modify the NAS snapshot.

cd "$WORK_ROOT/tribev2"
uv venv --python 3.11 .venv
source .venv/bin/activate

python - <<'PY'
from pathlib import Path

path = Path("pyproject.toml")
text = path.read_text()
text = text.replace('"torch>=2.5.1,<2.7"', '"torch>=2.7,<2.8"')
text = text.replace('"torchvision>=0.20,<0.22"', '"torchvision>=0.22,<0.23"')
path.write_text(text)
print("Patched local PyTorch dependency bounds for Blackwell.")
PY

uv pip install \
  --index-url https://download.pytorch.org/whl/cu128 \
  torch==2.7.0 \
  torchvision==0.22.0

uv pip install -e ".[plotting]"

Verify that the environment sees the 5090:

python - <<'PY'
import torch
import torchvision

print("torch:", torch.__version__)
print("torchvision:", torchvision.__version__)
print("compiled CUDA:", torch.version.cuda)
print("CUDA available:", torch.cuda.is_available())
assert torch.cuda.is_available(), "PyTorch cannot see CUDA"
print("GPU:", torch.cuda.get_device_name(0))
print("capability:", torch.cuda.get_device_capability(0))
PY

Success means PyTorch reports CUDA availability and identifies the RTX 5090. Do not continue on a CPU fallback.

6. Authenticate for the gated Llama dependency

TRIBE v2 itself is public, but its language pathway requires gated meta-llama/Llama-3.2-3B. Request or accept access in the Hugging Face browser UI first, then log in locally with a read token.

hf auth login
hf auth whoami

# Cheap auth/access probe before the expensive run:
hf download meta-llama/Llama-3.2-3B config.json --cache-dir "$HF_HOME"

7. Download a safe public test clip

Use the same public Sintel trailer as Meta’s notebook for the first pass. A public clip avoids sending unreleased material through transcription services while the pipeline is still being characterized.

curl -fL \
  https://download.blender.org/durian/trailer/sintel_trailer-480p.mp4 \
  -o "$TRIBE_VIDEO"

ffprobe -v error \
  -show_entries format=duration \
  -of default=noprint_wrappers=1:nokey=1 \
  "$TRIBE_VIDEO"

If you want the very first pass to stay near 20 seconds, trim a short sample:

ffmpeg -y \
  -ss 10 -t 20 \
  -i "$TRIBE_VIDEO" \
  -c:v libx264 -c:a aac \
  "$WORK_ROOT/inputs/sintel-smoke-20s.mp4"

export TRIBE_VIDEO="$WORK_ROOT/inputs/sintel-smoke-20s.mp4"

8. Run the smoke test

Create a reusable script:

cat > "$WORK_ROOT/smoke_test.py" <<'PY'
import json
import os
import time
from pathlib import Path

import numpy as np
import torch
from tribev2.demo_utils import TribeModel

model_dir = Path(os.environ["TRIBE_MODEL_DIR"])
cache_dir = Path(os.environ["TRIBE_CACHE"])
video_path = Path(os.environ["TRIBE_VIDEO"])
out_dir = Path(os.environ["TRIBE_OUT"])
out_dir.mkdir(parents=True, exist_ok=True)

print("torch:", torch.__version__)
print("compiled CUDA:", torch.version.cuda)
print("CUDA available:", torch.cuda.is_available())
assert torch.cuda.is_available(), "Smoke test requires CUDA"
print("GPU:", torch.cuda.get_device_name(0))
print("capability:", torch.cuda.get_device_capability(0))
print("video:", video_path)

started = time.time()
model = TribeModel.from_pretrained(
    model_dir,
    cache_folder=cache_dir,
)
print("Loaded TRIBE model in %.1fs" % (time.time() - started))

events = model.get_events_dataframe(video_path=video_path)
print("Event counts:")
print(events["type"].value_counts().to_string())

preds, segments = model.predict(events=events)
elapsed = time.time() - started

assert preds.ndim == 2, preds.shape
assert preds.shape[0] > 0, preds.shape
assert preds.shape[1] == 20484, preds.shape
assert np.isfinite(preds).all(), "Predictions contain NaN or Inf"
assert len(segments) == preds.shape[0]

np.save(out_dir / "preds.npy", preds)
summary = {
    "video": str(video_path),
    "elapsed_seconds": elapsed,
    "prediction_shape": list(preds.shape),
    "segments": len(segments),
    "gpu": torch.cuda.get_device_name(0),
    "cuda_capability": list(torch.cuda.get_device_capability(0)),
    "torch": torch.__version__,
    "compiled_cuda": torch.version.cuda,
    "event_counts": {
        str(k): int(v) for k, v in events["type"].value_counts().items()
    },
}
(out_dir / "summary.json").write_text(json.dumps(summary, indent=2) + "\n")
print(json.dumps(summary, indent=2))
print("SMOKE TEST PASSED")
PY

cd "$WORK_ROOT/tribev2"
source .venv/bin/activate
python "$WORK_ROOT/smoke_test.py" 2>&1 | tee "$TRIBE_OUT/smoke-test.log"

In a second terminal, watch GPU memory during the run:

watch -n 1 nvidia-smi

9. Success criteria

10. Optional: render a brain-response image

Once the numeric smoke test passes, render the first few predicted seconds. This verifies the plotting dependencies and gives you a visual artifact for sanity checking.

cat > "$WORK_ROOT/render_preview.py" <<'PY'
import os
from pathlib import Path
import numpy as np
from tribev2.plotting import PlotBrain

out_dir = Path(os.environ["TRIBE_OUT"])
preds = np.load(out_dir / "preds.npy")
n = min(12, len(preds))
plotter = PlotBrain(mesh="fsaverage5")
fig = plotter.plot_timesteps(
    preds[:n],
    cmap="fire",
    norm_percentile=99,
    vmin=.6,
    alpha_cmap=(0, .2),
)
fig.savefig(out_dir / "brain-preview.png", dpi=180, bbox_inches="tight")
print(out_dir / "brain-preview.png")
PY

cd "$WORK_ROOT/tribev2"
source .venv/bin/activate
python "$WORK_ROOT/render_preview.py"

11. Expected downloads and storage

The NAS copy contains TRIBE v2 itself and the official source. The first rig run will fetch runtime dependencies into $HF_HOME. Keep those on local NVMe for fast iteration.

Selected upstream model artifacts total roughly 16.7 GiB before extracted features, package caches, and input media. Reserve 20–30 GiB of local NVMe space.

12. Failure triage

13. Privacy and safety notes

• Use the public Sintel clip for the first test. Do not start with unreleased campaigns or private content.
• The public video path transcribes speech with WhisperX. Characterize the installed transcription backend before feeding confidential media into it.
• The public text_path flow uses Google Text-to-Speech (gTTS) and therefore sends text to Google. Do not use it for private titles, scripts, or ad copy until it is replaced with a local adapter.
• Keep Hugging Face tokens local and out of logs.
• Store active caches on local NVMe. Copy only deliberate outputs back to the NAS after the smoke test passes.

14. After the smoke test passes

1. Archive summary.json, smoke-test.log, and brain-preview.png.
2. Measure cold-run versus cached-run wall-clock time on the same 20-second clip.
3. Try a tiny evaluation set of known strong and weak hooks.
4. Inspect temporal response patterns rather than inventing a single “viral score.”
5. If the signal looks useful, build a research-only batch worker that derives features and calibrates them against your own CTR, retention, conversion, and ad-performance data.
6. Before any commercial use, get explicit license clearance.

Primary references

• Meta release post
• Meta research publication
• Official source repository
• Official Hugging Face weights
• Official interactive demo
• PyTorch 2.7 Blackwell support notes