# ModernBERT sequence classification on Emily

```elixir
Mix.install(
  [
    {:emily, "~> 1.0"},
    {:bumblebee, "~> 0.7"},
    {:tokenizers, "~> 0.5"},
    {:nx, "~> 0.12"},
    {:kino, "~> 0.14"}
  ],
  config: [
    nx: [default_backend: Emily.Backend]
  ]
)
```

## Overview

This notebook runs a ModernBERT sequence-classification fine-tune
on `Emily.Backend`. ModernBERT is one of the three new model
families that landed with Bumblebee 0.7 — a long-context (8192
token) BERT successor with RoPE positional encoding, GeGLU
activations, and alternating local/global attention. It's the most
interesting of the three for Emily because it's the first
*encoder* with rotary embeddings to land in the conformance suite,
and it exercises both local and global attention paths in a single
forward.

The integration with Emily is the `Mix.install` config line above;
no further setup is required.

## Loading the model

```elixir
repo = "tasksource/ModernBERT-base-nli"

{:ok, model_info} = Bumblebee.load_model({:hf, repo})
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, repo})
```

`tasksource/ModernBERT-base-nli` is an NLI fine-tune (entailment /
neutral / contradiction across MNLI, ANLI, FEVER, …). Swap in any
ModernBERT-based `*ForSequenceClassification` checkpoint — the
Bumblebee 0.7 auto-detect resolves the architecture and the
serving pipeline is identical. The base checkpoint is ~600 MB on
first fetch.

## Building a classification serving

```elixir
serving =
  Bumblebee.Text.text_classification(model_info, tokenizer,
    top_k: 3,
    defn_options: [compiler: Emily.Compiler, native: true, native_fallback: :raise]
  )
```

`Emily.Compiler` pins the result backend to `Emily.Backend` and
caps partition concurrency at 1. For an encoder this size, a
single MLX command queue saturates the GPU on a single inference
job — use `Emily.Stream` if you need parallel inferences.
`native: true` lowers the whole forward through Emily's native Expr
compiler (one NIF replay per call rather than op-by-op dispatch), and
`native_fallback: :raise` fails loudly instead of silently degrading
to the evaluator. This encoder lowers fully native.

## Classifying a few examples

ModernBERT NLI fine-tunes expect the sentence-pair format
`<premise> [SEP] <hypothesis>`. Bumblebee's tokenizer accepts a
`{premise, hypothesis}` tuple and emits the right framing.

```elixir
examples = [
  {"Elixir runs on the BEAM virtual machine.", "Elixir runs on a virtual machine."},
  {"Elixir runs on the BEAM virtual machine.", "Elixir is a compiled language with no runtime."},
  {"Cats sleep most of the day.", "Cats are nocturnal predators."}
]

Nx.Serving.run(serving, examples)
```

Each result is a `%{predictions: [%{label: _, score: _} | _]}`
map. With this fine-tune the top label will be one of
`"entailment"`, `"neutral"`, or `"contradiction"`.

## Local-vs-global attention path

ModernBERT alternates `:sliding_attention` (local window, default
128 tokens) with `:full_attention` (global, every block sees every
other) across its 22 layers. Both paths run on `Emily.Backend`
through the standard scaled-dot-product attention kernel — the
local window is realised as an additive mask, so there's no
separate code path on the Emily side. Long-document inputs (~2k+
tokens) are where the saved compute on local layers actually
shows up in throughput.

## Telemetry

Under `native: true` the forward is a single NIF replay, so the
op-by-op `[:emily, :eval, :stop]` span never fires — there's no
per-op boundary to time. The native-compiler event to watch instead
is `[:emily, :compiler, :fallback]`: a tripwire that fires only if an
op can't lower and routes through the evaluator. Attach it, run the
forward, then read `Emily.Memory.stats/0` (which itself emits
`[:emily, :memory, :stats]`):

```elixir
:telemetry.attach(
  "modernbert-cls-fallback",
  [:emily, :compiler, :fallback],
  fn _event, %{count: count}, %{reason: reason}, _config ->
    IO.puts("native fallback (#{count}): #{reason}")
  end,
  nil
)

Emily.Memory.reset_peak()
Nx.Serving.run(serving, hd(examples))
%{active: active, peak: peak} = Emily.Memory.stats()

IO.puts("no fallback above => forward lowered fully native")

IO.puts(
  "MLX memory — active #{div(active, 1024 * 1024)} MiB, " <>
    "peak #{div(peak, 1024 * 1024)} MiB"
)
```

See `Emily.Telemetry` for the full event catalogue, including the
`[:emily, :fallback, *]` span that fires whenever an op routes
through `Nx.BinaryBackend`.