2025 was a big year for PDF accessibility. LaTeX and Typst both released support for PDF tagging and accessibility standards, just in time for new regulations in the EU (June 2025) and US (April 2026).

Quarto 1.9 brings this support to you as a Quarto user.

What PDF Standards Do

Currently LaTeX supports the newer UA-2 standard, and Typst supports the older UA-1 standard. Typst is likely to have UA-2 support later in 2026.

Both standards instruct the PDF renderer to provide screen readers:

• The semantic structure of the text (title, heading, paragraph, figure, etc)
• The natural reading order
• Spatial coordinates for highlighting and assistive navigation
• Required metadata such as title and language

How to enable a PDF Standard in Quarto

In Quarto 1.9, specify a PDF standard for your document or project with pdf-standard

format:
  pdf:
    pdf-standard: ua-2

format:
  typst:
    pdf-standard: ua-1

pdf-standard takes a single standard name or list of standard names. PDF version is used if provided in the list, but otherwise inferred from the standard.

If you specify a PDF standard, Quarto first instructs LaTeX or Typst to use the standard when producing the PDF, and then validates the output PDF against the standard using veraPDF, an open-source PDF validation tool. If veraPDF is not installed, you’ll get a warning but still receive a PDF – it just won’t be validated.

1

quarto install verapdf

When a document passes validation, you’ll see output like:

[verapdf]: Validating my-document.pdf against PDF/UA-2... PASSED

Creating accessible PDFs

Quarto’s Markdown-based workflow handles many accessibility requirements automatically:

• Document metadata (title, author, date, language) flows into the PDF’s built-in metadata fields.
• The semantic structure of Markdown satisfies PDF tagging requirements. For Typst this is always enabled; for LaTeX it is enabled when you specify a standard that requires it.
• Alt text for images is carried through to the PDF for screen readers.

But you do need to make sure your document has:

• A title in the YAML front matter.
• Alt text for every image, specified with fig-alt. See Figures for details.

See the LaTeX and Typst documentation for more details.

If your document fails validation

LaTeX does not perform validation during PDF generation, so if veraPDF validation fails, that’s a warning, and you still get a partially-accessible PDF as long as you use pdf-standard: ua-2.

Typst fails and does not produce a PDF if its built-in validation fails during PDF generation. However, in Typst all accessibility features are on by default, so you can generate a partially-accessible PDF by rendering without pdf-standard.

Current limitations

We ran our test suite – 188 LaTeX examples and 317 Typst examples – to find where Quarto PDFs do not yet pass UA-1 or UA-2, and where users will need to change their documents.

LaTeX

Margin content is the biggest structural blocker. If you use .column-margin divs, cap-location: margin, reference-location: margin, or citation-location: margin, the resulting PDF will not pass UA-2. The underlying sidenotes and marginnote LaTeX packages do not cooperate with PDF tagging .

(Margin content does work with Typst and passes UA-1 – see Typst Article Layout .)

There are smaller upstream issues in Pandoc, LaTeX, and LaTeX packages, documented here .

Typst

In our tests, Typst catches every UA-1 violation, and fails to generate the PDF. veraPDF did not detect any violation that Typst did not.

Typst also seems to do a very good job of generating UA-1 compliant output by default – almost all errors were due to missing titles or missing alt text.

However, we did discover that Typst books are not yet compliant. There is a structural problem with the Typst orange-book package and we’ll work with the maintainers to correct it.

Conclusion

Although Typst currently targets an the earlier UA-1 standard, today it seems to offer better PDF accessibility than LaTeX.

We expect PDF accessibility support to improve through the LaTeX ecosystem throughout 2026 as awareness of UA-2 and the new regulations spreads.

If you run into accessibility issues with PDF output, please search the Quarto discussions and open a new one with the accessibility label for any issues you discover.

Why a command-line interface for R?

A command-line interface (CLI) lets you run programs from a terminal, without opening an IDE or starting an interactive R session. This is useful when you want to:

• automate tasks via cron jobs, scheduled tasks, or CI/CD pipelines
• chain R scripts together with other tools in data pipelines
• let others run your R code without needing to know R
• package reusable tools that feel native to the terminal
• expose specific actions through a clean interface that LLM agents can invoke

There are several established packages for building CLIs in R, including argparse, optparse, and docopt, where you explicitly parse and handle command-line arguments in code. Rapp takes a different approach: it derives the CLI surface from the structure of your R script and injects values at runtime, so you never need to handle CLI arguments manually.

How Rapp works

At its core, Rapp is an alternative front-end to R: a drop-in replacement for Rscript that automatically turns common R expression patterns into command-line options, switches, positional arguments, and subcommands. You write normal R code and Rapp handles the CLI surface.

Rapp also uses special #| comments (similar to Quarto’s YAML-in-comments syntax) to add metadata such as help descriptions and short aliases.

A tiny example

Here’s a complete Rapp script (from the package examples), a coin flipper:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

#!/usr/bin/env Rapp
#| name: flip-coin
#| description: |
#|   Flip a coin.

#| description: Number of coin flips
#| short: 'n'
flips <- 1L

sep <- " "
wrap <- TRUE

seed <- NA_integer_
if (!is.na(seed)) {
  set.seed(seed)
}

cat(sample(c("heads", "tails"), flips, TRUE), sep = sep, fill = wrap)

Let’s break down how Rapp interprets this script:

The #| short: 'n' comment adds -n as a short alias for --flips. The #!/usr/bin/env Rapp line (called a “shebang”) lets you run the script directly on macOS and Linux without typing Rapp first.

Running the script

With Rapp installed and flip-coin available on your PATH (see Get started below), you can run the app from the terminal:

1
2
3
4
5

flip-coin -n 3
#> heads tails heads

flip-coin --seed 42 -n 5
#> tails heads tails tails heads

Auto-generated help

Rapp generates --help from your script (and --help-yaml if you want a machine-readable spec):

1

flip-coin --help

1
2
3
4
5
6
7
8
9

Usage: flip-coin [OPTIONS]

Flip a coin.

Options:
  -n, --flips <FLIPS>  Number of coin flips [default: 1] [type: integer]
  --sep <SEP>          [default: " "] [type: string]
  --wrap / --no-wrap   [default: true] Disable with `--no-wrap`.
  --seed <SEED>        [default: NA] [type: integer]

1
2
3
4
5
6

# Before 0.3.0: this positional was optional
name <- NULL

# In 0.3.0+: add this comment to keep it optional
#| required: false
name <- NULL

Highlights in 0.3.0

Rapp will be new to most readers, so rather than listing every change, here are the main ideas (and what’s improved in 0.3.0).

Options, switches, and repeatable flags from plain R

Rapp recognizes a small set of “declarative” patterns at the top level of your script:

• Scalar literals like flips <- 1L become options like --flips 10.
• Logical defaults like wrap <- TRUE become toggles like --wrap / --no-wrap.
• #| short: n adds a short alias like -n (new in 0.3.0).
• c() and list() defaults declare repeatable options (new in 0.3.0): callers can supply the same flag multiple times and values are appended.

Subcommands with switch()

Rapp can now turn a switch() block into subcommands (and you can nest switch() blocks for nested commands). Here’s a small sketch of a todo-style app:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#!/usr/bin/env Rapp
#| name: todo
#| description: Manage a simple todo list.

#| description: Path to the todo list file.
#| short: s
store <- ".todo.yml"

switch(
  command <- "",

  #| description: Display the todos
  list = {
    limit <- 30L
    # ...
  },

  #| description: Add a new todo
  add = {
    task <- NULL
    # ...
  }
)

Help is scoped to the command you’re asking about, so todo --help lists the commands, and todo list --help shows just the options/arguments for list (plus any parent/global options).

Installable launchers for package CLIs

A big part of sharing CLI tools is making them easy to run after installation. In 0.3.0, install_pkg_cli_apps() installs lightweight launchers for scripts in a package’s exec/ directory that use either #!/usr/bin/env Rapp or #!/usr/bin/env Rscript:

1

Rapp::install_pkg_cli_apps("mypackage")

(There’s also uninstall_pkg_cli_apps() to remove a package’s launchers.)

Get started

Here’s the quickest path to your first Rapp script:

1
2

# 1. Install the package
install.packages("Rapp")

1
2

# 2. Install the command-line launcher
Rapp::install_pkg_cli_apps("Rapp")

Then create a script (e.g., hello.R):

1
2
3
4
5
6

#!/usr/bin/env Rapp
#| name: hello
#| description: Say hello

name <- "world"
cat("Hello,", name, "\n")

And run it:

1
2

Rapp hello.R --name "R users"
#> Hello, R users

Learn more

To dig deeper into Rapp:

• browse examples in the package: system.file("examples", package = "Rapp")
• read the full documentation: https://github.com/r-lib/Rapp
• note that Rapp requires R ≥ 4.1.0

If you try Rapp, we’d love feedback! We especially want to hear about your experiences with edge cases in argument parsing, help output, and how commands should feel. Issues and ideas are welcome at https://github.com/r-lib/Rapp/issues .

With this release, it bridges the gap between your laptop and enterprise infrastructure – the same code you prototype locally now deploys to Posit Workbench or any cloud HTTP API, with a single function call.

You can install it from CRAN with:

1

install.packages("mirai")

The flagship feature for this release is the HTTP launcher for deploying daemons to cloud and enterprise platforms. This release also brings a C-level dispatcher for minimal task dispatch overhead, race_mirai() for process-as-completed patterns, synchronous mode for debugging, and daemon synchronization for remote deployments. You can see a full list of changes in the release notes .

How mirai works

If you’ve ever waited for a loop to finish fitting models, processing files, or calling APIs, mirai can help. Any task that’s repeated independently across items is a candidate for parallel execution.

The previous release post covered mirai’s design philosophy in detail. Here’s a brief overview for readers encountering mirai for the first time.

library(mirai)
# Set up 4 background processes
daemons(4)

# Send work -- non-blocking, returns immediately
m <- mirai({
  Sys.sleep(1)
  100 + 42
})
m
#> < mirai [] >

# Collect the result when ready
m[]
#> [1] 142

# Shut down
daemons(0)

That’s mirai in a nutshell: daemons() to set up workers, mirai() to send work, [] to collect results. Everything else builds on this.

In mirai’s hub architecture, the host session listens at a URL and daemons – background R processes that do the actual work – connect to it. You send tasks with mirai() , and the dispatcher routes them to available daemons in first-in, first-out (FIFO) order.

This design enables dynamic scaling: daemons can connect and disconnect at any time without disrupting the host. Add capacity when you need it, release it when you don’t.

A single compute profile can mix daemons launched by different methods, and you can run multiple profiles simultaneously to direct different tasks to different resources. The basic syntax for each deployment method:

Change one line and your local prototype runs on a Slurm cluster. Change it again and it runs on Posit Workbench. Your analysis code stays identical.

The async foundation for the modern R stack

mirai has become the convergence point for asynchronous and parallel computing across the R ecosystem.

It is the recommended async backend for Shiny – if you’re building production Shiny apps, you should be using mirai. It is the only async backend for the next-generation plumber2 – if you’re building APIs with plumber2, you’re already using mirai.

It is the parallel backend for purrr – if you use map(), mirai is how you make it parallel. Wrap your function in in_parallel() , set up daemons, and your map calls run across all of them:

1
2
3
4
5

library(purrr)
daemons(4)
models <- split(mtcars, mtcars$cyl) |>
  map(in_parallel(\(x) lm(mpg ~ wt + hp, data = x)))
daemons(0)

It powers targets – the pipeline orchestration tool for reproducible analysis. And most recently, ragnar – the Tidyverse package for retrieval-augmented generation (RAG) – adopted mirai for its parallel processing.

As an official alternative communications backend for R’s parallel package, mirai underpins workflows from interactive web applications to pipeline orchestration to AI-powered document processing.

Learn mirai, and you’ve learned the async primitive that powers the modern R stack. The same two concepts – daemons() to set up workers, mirai() to send work – are all you need to keep a Shiny app responsive or run async tasks in production.

HTTP launcher

This release extends the “deploy everywhere” principle with http_config() , a new remote launch configuration that deploys daemons via HTTP API calls – any platform with an HTTP API for launching jobs.

Posit Workbench

Many organizations use Posit Workbench to run research and data science at scale. mirai now integrates directly with it.¹ Call http_config() with no arguments and it auto-configures using the Workbench environment:

1

daemons(n = 4, url = host_url(), remote = http_config())

That’s it. Four daemons launch as Workbench jobs, connect back to your session, and you can start sending work to them.

Here’s what that looks like in practice: you’re developing a model in your Workbench session. Fitting it locally is slow. Add that line, and those fits fan out across four Workbench-managed compute jobs. When you’re done, daemons(0) releases them. No YAML, no job scripts, no leaving your R session – resource allocation, access control, and job lifecycle are all handled by the platform.

If you’ve been bitten by expired tokens in long-running sessions, http_config() is designed to prevent that. Under the hood, it stores functions rather than static values for credentials and endpoint URLs. These functions are called at the moment daemons actually launch, so session cookies and API tokens are always fresh – even if you created the configuration hours earlier.

See the mirai vignette for troubleshooting remote launches.

Custom APIs

The HTTP launcher works with any HTTP API, not just Workbench. Supply your own endpoint, authentication, and request body:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

daemons(
  n = 2,
  url = host_url(),
  remote = http_config(
    url = "https://api.example.com/launch",
    method = "POST",
    token = function() Sys.getenv("MY_API_KEY"),
    data = '{"command": "%s"}'
  )
)

The "%s" placeholder in data is where mirai inserts the daemon launch command at launch time. Each argument can be a plain value or a function – use functions for anything that changes between launches (tokens, cookies, dynamic URLs).

This opens up a wide range of deployment targets: Kubernetes job APIs, other cloud container services, or any internal job scheduler with an HTTP interface. If you can launch a process with an HTTP call, mirai can use it.

C-level dispatcher

The overhead of distributing your tasks is now negligible. In a mirai_map() over thousands of items, what you measure is the time of your actual computation, not the framework – per-task dispatch overhead is now in the tens of microseconds, where existing R parallelism solutions typically operate in the millisecond range.

Under the hood, the dispatcher – the process that sits between your session and the daemons, routing tasks to available workers – has been re-implemented entirely in C code within nanonext . This eliminates the R interpreter overhead that remained, while the dispatcher continues to be event-driven and consume zero CPU when idle.

This also removes the bottleneck when coordinating large numbers of daemons, which matters directly for the kind of scaled-out deployments that the HTTP launcher enables – dozens of Workbench jobs or cloud instances all connecting to a single dispatcher. The two features are designed to work together: deploy broadly, dispatch efficiently. mirai is built to scale from 2 cores on your laptop to 200 across a cluster, without the framework slowing you down.

race_mirai()

race_mirai() lets you process results as they arrive, rather than waiting for the slowest task. Suppose you’re fitting 10 models with different hyperparameters in parallel – some converge quickly, others take much longer. Without race_mirai() , you wait for the slowest fit to complete before seeing any results. With it, you can inspect or save each model the instant it finishes – updating a progress display, freeing memory, or deciding whether to continue the remaining fits at all.

race_mirai() returns the integer index of the first resolved mirai. This makes the “process as completed” pattern clean and efficient:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

daemons(4)

# Launch 10 model fits in parallel
fits <- lapply(param_grid, function(p) mirai(fit_model(data, p), data = data, p = p))

# Process each result as soon as it's ready
remaining <- fits
while (length(remaining) > 0) {
  idx <- race_mirai(remaining)
  cat("Finished model with params:", remaining[[idx]]$data$p, "\n")
  remaining <- remaining[-idx]
}

daemons(0)

Send off a batch of tasks, then process results in the order they finish – no polling, no wasted time waiting on the slowest one. If any mirai is already resolved when you call race_mirai() , it returns immediately. This pattern applies whenever tasks have variable completion times – parallel model fits, API calls, simulations, or any batch where you want to stream results as they land.

Synchronous mode

When tasks don’t behave as expected, you need a way to inspect them interactively.

Without synchronous mode, errors in a mirai return as miraiError objects – you can see that something went wrong, but you can’t step through the code to find out why. The task ran in a separate process, and by the time you see the error, that process has moved on.

daemons(sync = TRUE), introduced in 2.5.1, solves this. It runs everything in the current process – no background processes, no networking – just sequential execution. You can use browser() and other interactive debugging tools directly:

1
2
3
4
5
6
7
8

daemons(sync = TRUE)
mirai(
  {
    browser()
    mypkg::some_complex_function(x)
  },
  x = my_data
)

You can scope synchronous mode to a specific compute profile, isolating the problematic task for inspection while the rest of your pipeline keeps running in parallel.

Daemon synchronization with everywhere()

everywhere() runs setup operations on all daemons – loading packages, sourcing scripts, or preparing datasets – so they’re ready before you send work.

When launching remote daemons – via SSH, HPC schedulers, or the new HTTP launcher – there’s an inherent delay between requesting a daemon and that daemon being ready to accept work. The new .min argument ensures that setup has completed on at least that many daemons before returning:

1
2
3
4
5
6
7

daemons(n = 8, url = host_url(), remote = http_config())

# Wait until all 8 daemons are connected before continuing
everywhere(library(mypackage), .min = 8)

# Now send work once all daemons are ready
mp <- mirai_map(tasks, process)

This creates a synchronization point, ensuring your pipeline doesn’t start sending work before all daemons are ready. It’s especially useful for remote deployments where connection times are unpredictable.

Minor improvements and fixes

• miraiError objects now have conditionCall() and conditionMessage() methods, making them easier to use with R’s standard condition handling.
• The default exit behavior for daemons has been updated with a 200ms grace period before forceful termination, which allows OpenTelemetry disconnection events to be traced.
• OpenTelemetry span names and attributes have been revised to better follow semantic conventions.
• daemons() now properly validates that url is a character value where supplied.
• Fixed a bug where repeated mirai cancellation could sometimes cause a daemon to exit prematurely.

Try it now

1
2
3
4
5
6
7
8

install.packages("mirai")
library(mirai)

daemons(4)
system.time(mirai_map(1:4, \(x) Sys.sleep(1))[])
#>    user  system elapsed
#>   0.000   0.001   1.003
daemons(0)

Four one-second tasks, one second of wall time. If those were four model fits that each took a minute, you’d go from four minutes down to one – and if you needed more power, switching to Workbench or a Slurm cluster is a one-line change. Visit mirai.r-lib.org for the full documentation.

Acknowledgements

A big thank you to all the folks who helped make this release happen:

@agilly , @aimundo , @barnabasharris , @beevabeeva , @boshek , @eliocamp , @jan-swissre , @jeroenjanssens , @kentqin-cve , @mcol , @michaelmayer2 , @pmac0451 , @r2evans , @shikokuchuo , @t-kalinowski , @VincentGuyader , @wlandau , and @xwanner .

If you haven’t seen our previous post about the exciting new features in dplyr 1.2.0, you’ll want to go check that out first!

Here’s a before-and-after benchmark with if_else() :

# Using https://github.com/DavisVaughan/cross
cross::bench_versions(pkgs = c("tidyverse/dplyr@v1.1.4", "tidyverse/dplyr"), {
  library(dplyr)
  set.seed(123)

  condition <- sample(c(TRUE, FALSE, NA), size = 1e7, replace = TRUE)
  x <- sample(10, size = 1e7, replace = TRUE)
  y <- sample(10, size = 1e7, replace = TRUE)
  z <- sample(10, size = 1e7, replace = TRUE)

  bench::mark(if_else = if_else(condition, x, y, missing = z))
})

1
2
3
4
5

#> # A tibble: 2 × 6
#>   pkg                    expression      min   median `itr/sec` mem_alloc
#>   <chr>                  <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>
#> 1 tidyverse/dplyr@v1.1.4 if_else    248.25ms 249.25ms      4.02   381.6MB
#> 2 tidyverse/dplyr        if_else      7.27ms   7.51ms    132.      38.2MB

And with case_when() :

cross::bench_versions(pkgs = c("tidyverse/dplyr@v1.1.4", "tidyverse/dplyr"), {
  library(dplyr)
  set.seed(123)

  column <- sample(100, size = 1e7, replace = TRUE)

  x_condition <- column < 20
  y_condition <- column < 50
  z_condition <- column < 80

  x <- sample(10, size = 1e7, replace = TRUE)
  y <- sample(10, size = 1e7, replace = TRUE)
  z <- sample(10, size = 1e7, replace = TRUE)

  bench::mark(
    case_when = case_when(
      x_condition ~ x,
      y_condition ~ y,
      z_condition ~ z
    )
  )
})

1
2
3
4
5

#> # A tibble: 2 × 6
#>   pkg                    expression      min   median `itr/sec` mem_alloc
#>   <chr>                  <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>
#> 1 tidyverse/dplyr@v1.1.4 case_when   228.3ms  231.2ms      4.33   419.9MB
#> 2 tidyverse/dplyr        case_when    15.5ms   15.8ms     62.8     38.3MB

So a 33x speed improvement for if_else() , a 15x speed improvement for case_when() , and a 10x improvement in memory usage for both! In the rest of this post, we’ll explain how we’ve achieved these numbers.

library(dplyr)

Let’s talk memory

We’ll start with case_when() , because if_else() is actually just a small variant of that.

The most important place to start is with the memory usage. Memory usage and raw speed are often related, as allocating memory takes time. Let’s look at the memory usage of case_when() in dplyr 1.1.4:

set.seed(123)

column <- sample(100, size = 1e7, replace = TRUE)

x_condition <- column < 20
y_condition <- column < 50
z_condition <- column < 80

x <- sample(10, size = 1e7, replace = TRUE)
y <- sample(10, size = 1e7, replace = TRUE)
z <- sample(10, size = 1e7, replace = TRUE)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

profmem::profmem(
  threshold = 1000,
  case_when(
    x_condition ~ x,
    y_condition ~ y,
    z_condition ~ z
  )
)
#> Rprofmem memory profiling of:
#> case_when(x_condition ~ x, y_condition ~ y, z_condition ~ z)
#>
#> Memory allocations (>= 1000 bytes):
#>        what     bytes                                           calls
#> 1     alloc  40000048     case_when() -> vec_case_when() -> vec_rep()
#> 2     alloc  40000048                  case_when() -> vec_case_when()
#> 3     alloc  40000056       case_when() -> vec_case_when() -> which()
#> 4     alloc   7600664       case_when() -> vec_case_when() -> which()
#> 5     alloc  40000048                  case_when() -> vec_case_when()
#> 6     alloc  40000056       case_when() -> vec_case_when() -> which()
#> 7     alloc  12003312       case_when() -> vec_case_when() -> which()
#> 8     alloc  40000048                  case_when() -> vec_case_when()
#> 9     alloc  40000056       case_when() -> vec_case_when() -> which()
#> 10    alloc  11996112       case_when() -> vec_case_when() -> which()
#> 11    alloc  40000056       case_when() -> vec_case_when() -> which()
#> 12    alloc   8400112       case_when() -> vec_case_when() -> which()
#> 13    alloc   7600664   case_when() -> vec_case_when() -> vec_slice()
#> 14    alloc  12003312   case_when() -> vec_case_when() -> vec_slice()
#> 15    alloc  11996112   case_when() -> vec_case_when() -> vec_slice()
#> 16    alloc   8400112 case_when() -> vec_case_when() -> vec_recycle()
#> 17    alloc  40000048 case_when() -> vec_case_when() -> list_unchop()
#> total       440000864

That’s a lot of allocations! And it’s pretty hard to understand where they are coming from without a bit more explanation. For that, we’re actually going to “manually” implement an underpowered version of case_when() for this example.

Here’s a diagram of what we need to accomplish:

In bullets:

• x_condition selects the blue elements of x
• y_condition selects the red elements of y
• z_condition selects the green elements of z
• A default is built around the unused locations
• We combine all of the pieces into out

The trickiest part about case_when() is handling places where x_condition and y_condition overlap. In the image, even though both x and y are selected at location 5, only the value of x is retained since it is hit “first”. This forces us to have to modify y_condition to avoid already “used” locations.

An R implementation that computes these modified locations might look like:

n <- length(x_condition)

unused <- rep(TRUE, times = n) # 1

x_loc <- unused & x_condition # 2
x_loc <- which(x_loc) # 3,4
unused[x_loc] <- FALSE

y_loc <- unused & y_condition # 5
y_loc <- which(y_loc) # 6,7
unused[y_loc] <- FALSE

z_loc <- unused & z_condition # 8
z_loc <- which(z_loc) # 9,10
unused[z_loc] <- FALSE

Anything that is still unused falls through to the default:

default <- NA_integer_
default_loc <- which(unused) # 11,12

With x_loc, y_loc, z_loc, and default_loc in hand, we can build the output from the pieces:

out <- vector("integer", length = n) # 17

out[x_loc] <- x[x_loc] # 13
out[y_loc] <- y[y_loc] # 14
out[z_loc] <- z[z_loc] # 15

out[default_loc] <- rep(default, times = length(default_loc)) # 16

And sure enough, this is identical to case_when() :

identical(
  out,
  case_when(
    x_condition ~ x,
    y_condition ~ y,
    z_condition ~ z
  )
)
#> [1] TRUE

You might be wondering what all of the comments with numbers beside them mean. Those actually map 1:1 with the allocations that case_when() was emitting. In fact, we can now split up those allocations into their respective role:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

#> # Tracking `unused` locations
#> 1     alloc  40000048     case_when() -> vec_case_when() -> vec_rep()

#> # Computing `x_loc`, `y_loc`, and `z_loc`
#> 2     alloc  40000048                  case_when() -> vec_case_when()
#> 3     alloc  40000056       case_when() -> vec_case_when() -> which()
#> 4     alloc   7600664       case_when() -> vec_case_when() -> which()
#> 5     alloc  40000048                  case_when() -> vec_case_when()
#> 6     alloc  40000056       case_when() -> vec_case_when() -> which()
#> 7     alloc  12003312       case_when() -> vec_case_when() -> which()
#> 8     alloc  40000048                  case_when() -> vec_case_when()
#> 9     alloc  40000056       case_when() -> vec_case_when() -> which()
#> 10    alloc  11996112       case_when() -> vec_case_when() -> which()

#> # Computing `default_loc`
#> 11    alloc  40000056       case_when() -> vec_case_when() -> which()
#> 12    alloc   8400112       case_when() -> vec_case_when() -> which()

#> # Slicing `x`, `y`, and `z` to align with `x_loc`, `y_loc`, and `z_loc`
#> 13    alloc   7600664   case_when() -> vec_case_when() -> vec_slice()
#> 14    alloc  12003312   case_when() -> vec_case_when() -> vec_slice()
#> 15    alloc  11996112   case_when() -> vec_case_when() -> vec_slice()

#> # Recycling `default` of `NA` to align with `default_loc`
#> 16    alloc   8400112 case_when() -> vec_case_when() -> vec_recycle()

#> # Final output container, which we assign `x`, `y`, `z`, and `default` into
#> # at locations `x_loc`, `y_loc`, and `z_loc`
#> 17    alloc  40000048 case_when() -> vec_case_when() -> list_unchop()

We sought to remove every one of these allocations except for the last one, which is the final output container that is returned to the user. In other words, we were after this, which is the actual profmem result of this case_when() call in dplyr 1.2.0:

1
2
3
4
5
6
7

#> Rprofmem memory profiling of:
#> case_when(x_condition ~ x, y_condition ~ y, z_condition ~ z)
#>
#> Memory allocations (>= 1000 bytes):
#>        what    bytes                          calls
#> 1     alloc 40000048 case_when() -> vec_case_when()
#> total       40000048

Sliced assignment

To work towards this, let’s focus on what happens to x throughout this process:

We had a hypothesis that we could cut out the intermediate work here. Ideally, we’d take the logical LHS x_condition and the RHS x and map that straight into the output, with no extra allocations:

But this just wasn’t possible with the way that assignment typically works in R!

x <- c("a", "b", "c", "d", "e", "f", "g")
x_condition <- c(FALSE, TRUE, FALSE, FALSE, TRUE, FALSE, FALSE)
out <- vector("character", length = length(x_condition))

out[x_condition] <- x
#> Warning in out[x_condition] <- x: number of items to replace is not a multiple of replacement length

Instead, you must pre-slice x to a length that matches the locations that x_condition points to in out, i.e.:

out[x_condition] <- x[x_condition]

Now, in case_when() we don’t actually use [<- for assignment or [ for slicing. Instead, we use tools from vctrs , a low level package for building consistent tidyverse functions. In this case, we’d use vctrs::vec_assign() and vctrs::vec_slice():

out <- vctrs::vec_assign(out, x_condition, vctrs::vec_slice(x, x_condition))

But vec_assign() had the same problem!

To solve this, we’ve added a new boolean argument to vec_assign() called slice_value. You use it like this:

out <- vctrs::vec_assign(out, x_condition, x, slice_value = TRUE)

With slice_value = TRUE, vec_assign() assumes that both out and x are the same length and that x_condition applies to both of these. Internally, rather than materializing x[x_condition], we instead just loop over both out and x at the same time (at C level) and copy over values from x whenever x_condition is TRUE.

This is huge! It means that allocations 13-15 from above related to slicing x, y, and z all disappear.

Logical indices

You might have noticed that we’ve been using which() quite a bit in the above algorithm. This turns a logical vector of TRUE and FALSE into an integer vector of locations pointing to where the logical vector was TRUE:

x_condition <- c(TRUE, FALSE, TRUE, FALSE, FALSE, TRUE)
x_loc <- which(x_condition)
x_loc
#> [1] 1 3 6

We perform this conversion up front due to how the following works at C level:

out[x_condition] <- x[x_condition]

Both [ and [<- will convert a logical x_condition into the integer x_loc form before proceeding with the assignment, meaning that which() gets called twice if we don’t do it once up front. And vctrs is the same way! Both vec_assign() and vec_slice() here would convert x_condition to an integer vector.

vctrs::vec_assign(out, x_condition, vctrs::vec_slice(x, x_condition))

Now, with the previous optimization we’ve already seen that we can reduce this to:

vctrs::vec_assign(out, x_condition, x, slice_value = TRUE)

But vec_assign() still converts a logical x_condition to integer locations internally before doing the assignment. So now it doesn’t matter whether we do this conversion up front via which() or if we let vec_assign() do it, it still happens once per input. But we’d like to avoid it entirely!

The solution here wasn’t too magical, it just involved a good bit of grunt work. We’ve added a path in vec_assign()’s C code that can handle logical indices like x_condition directly, rather than forcing them to be converted to integer locations first.

But this is a huge win, because it means that allocations 1-10, which were all related to which() , can now be removed. vec_assign() will just handle that optimally for us without any extra allocations.

The nice part about an optimization like this is that any other existing code that is using vec_assign() with a logical index will also benefit from this without having to change a thing!

default handling

The remaining allocations are 11-12 and 16, which all have to do with the implied default. Allocations 11-12 were about figuring out where to put default, and allocation 16 was about recycling a typed size 1 default to the right size before assigning it into out.

As it turns out, we don’t need any of this!

In vctrs, when we initialize any output container, we use vec_init():

vctrs::vec_init(integer(), n = 5)
#> [1] NA NA NA NA NA

vctrs::vec_init(tibble(x = integer(), y = character()), n = 5)
#> # A tibble: 5 × 2
#>       x y    
#>   <int> <chr>
#> 1    NA NA   
#> 2    NA NA   
#> 3    NA NA   
#> 4    NA NA   
#> 5    NA NA

This already has the implied default assigned to every location. We then overwrite this with x, y, and z at the appropriate locations, but anything left untouched by those is still set to the default, so we’re done!

For cases where the user supplies their own default, things are slightly more complicated. We actually do have to compute a default_loc implied from x_condition, y_condition, and z_condition, but internally we do so using a C vector of bool (even more efficient than R’s logical vector type), so the memory footprint is as small as it can be.

The “first wins” conundrum

One thing we’ve skipped over is the “first wins” behavior of case_when() mentioned earlier. Now that we’ve removed x_loc, y_loc, and z_loc, which is where that was being handled, how do we keep this behavior without slowing things down?

To be explicit, we are talking about this feature of case_when() where only the first hit is kept when you have overlapping logical indices:

x <- c("x1", "x2", "x3")
y <- c("y1", "y2", "y3")
z <- c("z1", "z2", "z3")

x_condition <- c(TRUE, FALSE, TRUE)
y_condition <- c(TRUE, TRUE, FALSE)
z_condition <- c(FALSE, TRUE, TRUE)

case_when(
  x_condition ~ x,
  y_condition ~ y,
  z_condition ~ z
)
#> [1] "x1" "y2" "x3"

A naive approach doesn’t work, as you end up with “last wins” behavior:

out <- vctrs::vec_init(character(), n = 3)

out <- vctrs::vec_assign(out, x_condition, x, slice_value = TRUE)
out <- vctrs::vec_assign(out, y_condition, y, slice_value = TRUE)
out <- vctrs::vec_assign(out, z_condition, z, slice_value = TRUE)

# This is wrong!
out
#> [1] "y1" "z2" "z3"

identical(
  out,
  case_when(
    x_condition ~ x,
    y_condition ~ y,
    z_condition ~ z
  )
)
#> [1] FALSE

Instead, case_when() just iterates in reverse, assigning z, then y, then x:

out <- vctrs::vec_init(character(), n = 3)

out <- vctrs::vec_assign(out, z_condition, z, slice_value = TRUE)
out <- vctrs::vec_assign(out, y_condition, y, slice_value = TRUE)
out <- vctrs::vec_assign(out, x_condition, x, slice_value = TRUE)

identical(
  out,
  case_when(
    x_condition ~ x,
    y_condition ~ y,
    z_condition ~ z
  )
)
#> [1] TRUE

This diagram demonstrates how that works:

Optimizing speed?

Now that we’ve optimized the memory usage of case_when() , you might be wondering if we did anything else to specifically optimize its speed. Not really! We have moved everything from R to C, but focusing our efforts on reducing memory also resulted in some pretty performant code, and there wasn’t much left to optimize after that.

if_else()

if_else() can actually be written as a form of case_when() :

if_else(condition, true, false, missing)

case_when(
  condition ~ true,
  !condition ~ false,
  is.na(condition) ~ missing
)

In our actual C implementation of if_else() , for simple types like integer, character, or numeric vectors we have an extremely fast path that’s even more optimized than this, but for anything with a class we pretty much use this exact case_when() approach.

For package developers

If you’re a package developer, you’ll be happy to know that vctrs itself now exposes low dependency versions of if_else() and case_when() , here’s the full family:

• vec_if_else()
• vec_case_when()
• vec_replace_when()
• vec_recode_values()
• vec_replace_values()

dplyr::if_else() and friends are now just very thin wrappers over these. Feel free to use the vctrs versions in your package if you need the consistency of the tidyverse without the heavy-ish dependency of dplyr.

At the deepest level, list_combine()

At the deepest level of all of this is one final new vctrs function, list_combine(). This is a flexible way to combine multiple vectors together at locations specified by indices.

list_combine() powers all of vec_case_when(), vec_replace_when(), vec_recode_values(), vec_replace_values(), vec_if_else(), and even vec_c(), the tidyverse version of c() .

set.seed(123)

column <- sample(100, size = 1e7, replace = TRUE)

x_condition <- column < 20
y_condition <- column < 50
z_condition <- column < 80

x <- sample(10, size = 1e7, replace = TRUE)
y <- sample(10, size = 1e7, replace = TRUE)
z <- sample(10, size = 1e7, replace = TRUE)

out <- vctrs::list_combine(
  x = list(x, y, z),

  # `indices` are allowed to be logical and aren't forced to integer
  indices = list(x_condition, y_condition, z_condition),
  size = length(x_condition),

  # When there are overlaps, take the "first"
  multiple = "first",

  # Same as `slice_value` from `vec_assign()`
  slice_x = TRUE
)

identical(
  out,
  case_when(
    x_condition ~ x,
    y_condition ~ y,
    z_condition ~ z
  )
)
#> [1] TRUE

When we introduced nanonext last year, we showed how it connects R directly to Python, Go, Rust, and other languages through NNG’s messaging protocols. We hinted at its web capabilities – but that was just the beginning.

R already has excellent web infrastructure. Shiny and plumber2 are the go-to tools for building interactive applications and REST APIs in R. They are both powered by httpuv . nanonext adds a complementary option at the httpuv level of the stack – a low-level streaming HTTP and WebSocket server built on NNG, giving developers fine-grained control over connections, streaming, and static file serving over TLS. nanonext is for when you need lower-level control – custom protocols, infrastructure endpoints, or embedding a server alongside an existing Shiny or plumber2 application.

You can install it from CRAN with:

1

install.packages("nanonext")

You can see a full list of changes in the release notes .

Streaming HTTP/WebSocket server

The flagship feature of this release is http_server() , a streaming HTTP and WebSocket server with full TLS support. Built on NNG’s HTTP server architecture, it brings the same performance that powers nanonext’s messaging layer to web serving.

One server, one port – HTTP endpoints, WebSocket connections, and streaming all coexist. Static files bypass R entirely, served natively by NNG. WebSocket and streaming connections run callbacks on R’s main thread via the later package. Mbed TLS is built in for HTTPS/WSS, and there’s no need to run separate processes or bind additional ports.

Because it shares the same event loop that Shiny uses, http_server() can run alongside a Shiny app in the same R process. You could spin up a nanonext server to handle health checks, serve static assets, or stream real-time events – while Shiny or plumber2 handles the application logic. They’re designed to work together.

As part of our investment in expanding what’s possible with R, we’re already using nanonext at Posit to explore new real-time capabilities, and we’re excited to see what the community builds with it.

Basic HTTP server

Where frameworks like plumber2 give you a full-featured API layer with routing, serialization, and documentation out of the box, http_server() gives you direct control over requests and responses – the kind of direct access you’d reach for when building custom infrastructure or embedding a server inside a larger system.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

library(nanonext)

server <- http_server(
  url = "http://127.0.0.1:8080",
  handlers = list(
    handler("/", function(req) {
      list(status = 200L, body = "Hello from nanonext!")
    }),
    handler("/api/data", function(req) {
      list(
        status = 200L,
        headers = c("Content-Type" = "application/json"),
        body = '{"value": 42}'
      )
    }, method = "GET")
  )
)
server$start()

Handlers receive a request and return a response list. You can freely mix handler types in a single server:

Specifying port 0 in the URL lets the operating system assign an available port. The actual port is reflected in server$url after $start(), so you can set up test servers without worrying about port conflicts.

Static file serving

Static handlers bypass R entirely – NNG serves content directly and efficiently:

1
2
3
4
5
6
7

handler_directory("/static", "www/assets")  # serve a folder
handler_file("/favicon.ico", "favicon.ico") # serve a single file
handler_inline(
  "/robots.txt",
  "User-agent: *\nDisallow:",
  content_type = "text/plain"
) # serve in-memory content

For example, you can serve a rendered Quarto website with a single handler:

1
2
3
4
5
6
7

server <- http_server(
  url = "http://127.0.0.1:0",
  handlers = handler_directory("/", "_site")
)
server$start()
server$url
# Browse to the URL to see your Quarto site

WebSocket server

WebSockets provide full bidirectional communication – the server can push messages to the client, and the client can send messages back. WebSocket and HTTP handlers share the same server and port, so a browser can load a page over HTTP and open a WebSocket to the same origin – no cross-origin configuration needed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

server <- http_server(
  url = "http://127.0.0.1:8080",
  handlers = list(
    handler("/", function(req) list(status = 200L, body = "<html>...</html>")),
    handler_ws("/ws",
      on_message = function(ws, data) ws$send(data),
      on_open = function(ws) cat("connected:", ws$id, "\n"),
      on_close = function(ws) cat("disconnected:", ws$id, "\n")
    )
  )
)

This makes it easy to build lightweight real-time services – monitoring endpoints or live-updating feeds that push results to the browser as they arrive.

HTTP streaming and Server-Sent Events

When you only need to push data in one direction – server to client – streaming is a lighter-weight alternative to WebSockets. It works over plain HTTP, so any client that speaks HTTP can consume the stream without needing a WebSocket library. handler_stream() enables chunked transfer encoding for streaming responses:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

conns <- list()

handler_stream("/events",
  on_request = function(conn, req) {
    conn$set_header("Content-Type", "text/event-stream")
    conn$set_header("Cache-Control", "no-cache")
    conns[[as.character(conn$id)]] <<- conn
    conn$send(format_sse(data = "connected", id = "1"))
  },
  on_close = function(conn) {
    conns[[as.character(conn$id)]] <<- NULL
  }
)

The format_sse() helper formats messages per the SSE specification. On the browser side, updates arrive automatically as they happen – no page refreshes or repeated requests needed. Streaming also supports NDJSON and custom formats – useful for streaming model training progress, sensor readings, monitoring endpoints, or pipeline notifications.

TLS/SSL support

For HTTPS, pass a TLS configuration. nanonext bundles Mbed TLS, so there’s nothing extra to install:

1
2
3
4
5
6

cert <- write_cert(cn = "127.0.0.1")
server <- http_server(
  url = "https://127.0.0.1:0",
  handlers = handler("/", function(req) list(status = 200L, body = "Secure!")),
  tls = tls_config(server = cert$server)
)

Full response headers for HTTP client

ncurl() now accepts response = TRUE to return all response headers:

resp <- ncurl("https://postman-echo.com/get", response = TRUE)
resp$headers |> names()
#>  [1] "Date"                          "Content-Type"                 
#>  [3] "Content-Length"                "Connection"                   
#>  [5] "CF-RAY"                        "etag"                         
#>  [7] "vary"                          "Set-Cookie"                   
#>  [9] "x-envoy-upstream-service-time" "cf-cache-status"              
#> [11] "Server"

Previously you could only request specific headers by name. Now you can retrieve the complete set – useful for inspecting rate limits, caching directives, and other metadata from REST APIs.

Async HTTP with Shiny

If your Shiny app calls a REST API, a slow or unresponsive endpoint will block the R process and freeze the app for all users, not just the one who triggered the request. ncurl_aio() avoids this – it performs the HTTP call on a background thread and returns a promise, so the R process stays free to serve other sessions. It works anywhere that accepts a promise, including Shiny’s ExtendedTask:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

library(shiny)
library(bslib)
library(nanonext)

ui <- page_fluid(
  p("The time is ", textOutput("current_time", inline = TRUE)),
  hr(),
  input_task_button("btn", "Fetch data"),
  verbatimTextOutput("result")
)

server <- function(input, output, session) {
  output$current_time <- renderText({
    invalidateLater(1000)
    format(Sys.time(), "%H:%M:%S %p")
  })

  task <- ExtendedTask$new(
    function() ncurl_aio("https://postman-echo.com/get", response = TRUE)
  ) |> bind_task_button("btn")

  observeEvent(input$btn, task$invoke())
  output$result <- renderPrint(task$result()$headers)
}

shinyApp(ui, server)

New documentation

The package documentation has been reorganized into focused, self-contained guides:

Whether you need a quick API cheatsheet or a deep dive into WebSocket chat servers, the new vignettes are designed to get you up and running fast.

Bug fixes and improvements

A new race_aio() function returns the index of the first resolved async operation in a list – useful when waiting on multiple concurrent operations and you want to act on whichever completes first.

This release also fixes two critical issues – one affecting TLS operations in fresh sessions with newer system versions of Mbed TLS, another when custom serialization hooks threw errors. Error handling is now more graceful throughout, with closed streams returning error values instead of throwing. Under the hood, serialization, streaming, and async sends are all faster, and the bundled Mbed TLS is updated to 3.6.5 LTS. Building from source no longer requires xz.

Looking ahead

nanonext gives R a new building block for web infrastructure – one that complements httpuv. We see it as part of a broader investment in making R a first-class platform for real-time, connected applications. If you want to dig deeper, visit the package website or explore the source on GitHub .

Acknowledgements

A big thank you to everyone who contributed to this release:

@jeroenjanssens and @shikokuchuo .

• filter_out() , the missing complement to filter() , and accompanying when_any() and when_all() helpers.

• recode_values() , replace_values() , and replace_when() , three new functions that join case_when() to create a cohesive family of powerful tools for recoding and replacing.

Both of these sets of features are backed by successful tidyups, the tidyverse’s community facing proposal process (filtering , recoding ). We really enjoyed having the community weigh in on these features!

You can install dplyr 1.2.0 from CRAN with:

install.packages("dplyr")

You can see a full list of changes in the release notes .

library(dplyr)

Expanding the filter() family

filter() has been a core dplyr verb since the very beginning, but over the years we’ve isolated a few key issues with it:

• The name filter() is ambiguous, are you keeping or dropping rows? i.e., are you filtering for rows or filtering out rows?

• filter() is optimized for the case of keeping rows, but you are just as likely to try and use it for dropping rows. Using filter() to drop rows quickly forces you to confront complex boolean logic and explicitly handle missing values, which is difficult to teach, error prone to write, and hard to understand when you come back to it in the future.

• filter() combines comma separated conditions with & because this covers the majority of the cases. But if you’d like to combine conditions with |, then you have to introduce parentheses around your conditions and combine them into one large condition separated by |, reducing readability.

In the next few sections, we’ll motivate these issues and discuss how some new features in dplyr can simplify things dramatically!

Filtering…out!

Take a look at this patients data. Our task with this data is:

Filter out rows where the patient is deceased and the year was before 2012.

patients <- tibble(
  name = c("Anne", "Mark", "Sarah", "Davis", "Max", "Derek", "Tina"),
  deceased = c(FALSE, TRUE, NA, TRUE, NA, FALSE, TRUE),
  date = c(2005, 2010, NA, 2020, 2010, NA, NA)
)

patients
#> # A tibble: 7 × 3
#>   name  deceased  date
#>   <chr> <lgl>    <dbl>
#> 1 Anne  FALSE     2005
#> 2 Mark  TRUE      2010
#> 3 Sarah NA          NA
#> 4 Davis TRUE      2020
#> 5 Max   NA        2010
#> 6 Derek FALSE       NA
#> 7 Tina  TRUE        NA

With filter() , you’d probably start by translating “patient is deceased and the year was before 2012” into deceased & date < 2012, and then inverting that with !(<expression>) to drop rows:

patients |>
  filter(!(deceased & date < 2012))
#> # A tibble: 3 × 3
#>   name  deceased  date
#>   <chr> <lgl>    <dbl>
#> 1 Anne  FALSE     2005
#> 2 Davis TRUE      2020
#> 3 Derek FALSE       NA

That seems to have worked, let’s use an anti_join() to check with rows have been dropped from patients:

# These rows were dropped
anti_join(
  patients,
  patients |> filter(!(deceased & date < 2012)),
  join_by(name, deceased, date)
)
#> # A tibble: 4 × 3
#>   name  deceased  date
#>   <chr> <lgl>    <dbl>
#> 1 Mark  TRUE      2010
#> 2 Sarah NA          NA
#> 3 Max   NA        2010
#> 4 Tina  TRUE        NA

This is subtly wrong! We only wanted to drop rows where we know that the patient was deceased before 2012. If a missing value is present, we don’t want to drop that row because we aren’t sure about the condition. In this case, we were hoping to only drop Mark! It seems like filter() has unexpectedly dropped more rows than we expected.

Here’s what a technically correct filter() call might look like:

patients |>
  filter(
    !((deceased & !is.na(deceased)) &
      (date < 2012 & !is.na(date)))
  )
#> # A tibble: 6 × 3
#>   name  deceased  date
#>   <chr> <lgl>    <dbl>
#> 1 Anne  FALSE     2005
#> 2 Sarah NA          NA
#> 3 Davis TRUE      2020
#> 4 Max   NA        2010
#> 5 Derek FALSE       NA
#> 6 Tina  TRUE        NA

That’s horrible! You’ll likely look back on this in a year wondering what you were even trying to do here.

This phenomenon is rather confusing, but is due to the fact that filter() is designed around the idea that you’re going to tell it which rows to keep. With that design in mind, dropping NAs makes sense, i.e. if you don’t know that you want to keep that row (because an NA is ambiguous), then you probably don’t want to keep it.

This works well until you try to use filter() as a way to filter out rows, at which point this behavior works against you. At this point, most people (reasonably!) reach for & !is.na() and you end up with the mess from above.

We took a close look at many examples like this one, and eventually realized that the core issue is:

• filter() is designed around supplying which rows to keep
• We are missing a verb designed around supplying which rows to drop

filter_out() fills that gap:

patients |>
  filter_out(deceased, date < 2012)
#> # A tibble: 6 × 3
#>   name  deceased  date
#>   <chr> <lgl>    <dbl>
#> 1 Anne  FALSE     2005
#> 2 Sarah NA          NA
#> 3 Davis TRUE      2020
#> 4 Max   NA        2010
#> 5 Derek FALSE       NA
#> 6 Tina  TRUE        NA

Just like with filter() , filter_out() treats NA values as FALSE. The difference is that filter_out() expects that you are going to tell it which rows to drop (rather than which rows to keep), so the default behavior of treating NA like FALSE works with you rather than against you. It’s also much easier to understand when you look back on it a year from now!

In general, our advice is that if you find yourself using “negative” operators like != or ! or reaching for the !is.na() pattern to manually handle missing values, try reaching for filter_out() instead.

Personally, I’ve always been pretty jealous of Stata here because they had both keep if and drop if , allowing them to write drop if deceased & date < 2012. In my first job, I translated a bunch of Stata code over to R and still remember being frustrated by NA handling every time I had to translate a drop if to a filter() . With filter_out() , it feels like I can finally let go of a long term grudge I’ve held over the past 6 years 🙂.

Combining with OR rather than AND

So far, we’ve talked a lot about dropping rows, but dplyr 1.2.0 also has a new feature to help with keeping rows using conditions combined with | - when_any() .

Our goal here is:

Filter for rows where “US” and “CA” have a score between 200-300, or rows where “PR” and “RU” have a score between 100-200.

countries <- tibble(
  name = c("US", "CA", "PR", "RU", "US", NA, "CA", "PR"),
  score = c(200, 100, 150, NA, 50, 100, 300, 250)
)

countries
#> # A tibble: 8 × 2
#>   name  score
#>   <chr> <dbl>
#> 1 US      200
#> 2 CA      100
#> 3 PR      150
#> 4 RU       NA
#> 5 US       50
#> 6 NA      100
#> 7 CA      300
#> 8 PR      250

Here’s a filter() solution, note how we lose the ability to specify comma separated conditions, and in the process we’ve introduced 3 operators, &, |, and (), decreasing readability and increasing the mental gymnastics required to understand it:

countries |>
  filter(
    (name %in% c("US", "CA") & between(score, 200, 300)) |
      (name %in% c("PR", "RU") & between(score, 100, 200))
  )
#> # A tibble: 3 × 2
#>   name  score
#>   <chr> <dbl>
#> 1 US      200
#> 2 PR      150
#> 3 CA      300

With when_any() , you specify comma separated conditions like you’re used to, but they get combined with | rather than &. This allows us to reduce the amount of operators introduced down to just &, and it remains very readable:

countries |>
  filter(when_any(
    name %in% c("US", "CA") & between(score, 200, 300),
    name %in% c("PR", "RU") & between(score, 100, 200)
  ))
#> # A tibble: 3 × 2
#>   name  score
#>   <chr> <dbl>
#> 1 US      200
#> 2 PR      150
#> 3 CA      300

when_any() and its counterpart when_all() aren’t restricted to filter() . They are normal vector functions that can be used anywhere. And if you’re a package author, you might be interested in vctrs::vec_pany() and vctrs::vec_pall() , the underlying low dependency functions that power the dplyr variants.

Reaching recoding nirvana

Over the years, we’ve experimented with various ways of recoding columns and replacing values within them, including:

• plyr::mapvalues()
• plyr::revalue()
• dplyr::recode()
• dplyr::case_when()

Despite all of our improvements, it’s felt like there have always been holes in our solutions. Most recently, this came to the forefront in a post about recoding using a lookup table , which is almost impossible to do with case_when() , and had people resorting to confusing solutions using the superseded dplyr::recode() combined with !!! to splice in a lookup table.

After seeing this, we took a step back and were finally able to isolate the issues with our current solutions. The result of our analysis is three new functions that join case_when() to form a powerful recoding and replacing family.

It’ll be helpful to define exactly what we mean by recoding vs replacing:

• Recoding a column creates an entirely new column using values from an existing column. The new column may have a different type from the original column.

• Replacing values within a column partially updates an existing column with new values. The result has the same type as the original column.

The family of functions can be summarized by the following table:

We’ve written a new vignette that expands on all of these from first principles, and in the next few sections we’ll look at some examples.

recode_values()

The goal of the post from above was to recode a numeric column of Likert scale scores into their string counterparts.

likert <- tibble(
  score = c(1, 2, 3, 4, 5, 2, 3, 1, 4)
)

We could certainly try case_when() :

likert |>
  mutate(
    category = case_when(
      score == 1 ~ "Strongly disagree",
      score == 2 ~ "Disagree",
      score == 3 ~ "Neutral",
      score == 4 ~ "Agree",
      score == 5 ~ "Strongly agree"
    )
  )
#> # A tibble: 9 × 2
#>   score category         
#>   <dbl> <chr>            
#> 1     1 Strongly disagree
#> 2     2 Disagree         
#> 3     3 Neutral          
#> 4     4 Agree            
#> 5     5 Strongly agree   
#> 6     2 Disagree         
#> 7     3 Neutral          
#> 8     1 Strongly disagree
#> 9     4 Agree

But score == is repeated so many times! When you find yourself using == in this way, recognize that what you’re really doing is matching on the values of a single column. In cases like these, you’ll want to switch to recode_values() . Rather than taking logical vectors, recode_values() takes values on the left-hand side to match against a single input that you’ll provide as the first argument.

likert |>
  mutate(
    category = score |>
      recode_values(
        1 ~ "Strongly disagree",
        2 ~ "Disagree",
        3 ~ "Neutral",
        4 ~ "Agree",
        5 ~ "Strongly agree"
      )
  )
#> # A tibble: 9 × 2
#>   score category         
#>   <dbl> <chr>            
#> 1     1 Strongly disagree
#> 2     2 Disagree         
#> 3     3 Neutral          
#> 4     4 Agree            
#> 5     5 Strongly agree   
#> 6     2 Disagree         
#> 7     3 Neutral          
#> 8     1 Strongly disagree
#> 9     4 Agree

This removes all of the repetition, allowing you to focus on the mapping. And it should feel pretty familiar! This is the same formula interface of case_when() .

If you squint, the mapping should look roughly like a lookup table between the numeric value and the Likert encoding. One of the novel features of recode_values() is that it has an alternate interface that allows you to make this lookup table more explicit. Using a tribble() , we can extract out the lookup table into its own standalone data frame.

lookup <- tribble(
  ~from , ~to                 ,
      1 , "Strongly disagree" ,
      2 , "Disagree"          ,
      3 , "Neutral"           ,
      4 , "Agree"             ,
      5 , "Strongly agree"    ,
)

We can then utilize the alternative from and to arguments of recode_values() rather than supplying formulas to specify how the values should be recoded:

likert |>
  mutate(category = recode_values(score, from = lookup$from, to = lookup$to))
#> # A tibble: 9 × 2
#>   score category         
#>   <dbl> <chr>            
#> 1     1 Strongly disagree
#> 2     2 Disagree         
#> 3     3 Neutral          
#> 4     4 Agree            
#> 5     5 Strongly agree   
#> 6     2 Disagree         
#> 7     3 Neutral          
#> 8     1 Strongly disagree
#> 9     4 Agree

Lifting the lookup table out to the top of the file is particularly nice when you have a long pipe chain. The details of the mapping get some room to breathe, and in the pipe chain you can focus on the actual data manipulations.

It’s also very common for your lookup table to exist in a CSV file that you have to read in separately. In that case, you can replace the tribble() call with:

lookup <- readr::read_csv("lookup.csv")

Then everything else works the same.

Unmatched cases

If you are confident that you’ve captured every case during the recoding process, you can now supply unmatched = "error" as an alternative to default. recode_values() will error if that assertion doesn’t hold. This is great for defensive programming!

# Notice the `0` that we don't have a mapping for!
likert <- tibble(
  score = c(0, 1, 2, 2, 4, 5, 2, 3, 1, 4)
)

likert |>
  mutate(
    score = score |>
      recode_values(
        from = lookup$from,
        to = lookup$to,
        unmatched = "error"
      )
  )
#> Error in `mutate()`:
#> ℹ In argument: `score = recode_values(score, from = lookup$from, to = lookup$to, unmatched = "error")`.
#> Caused by error in `recode_values()`:
#> ! Each location must be matched.
#> ✖ Location 1 is unmatched.

Note that missing values must be explicitly handled when setting unmatched = "error", even if that’s just setting NA ~ NA, otherwise they will trigger the unmatched error. This forces you to explicitly opt in to expecting missing values.

Similar to recode_values() , case_when() has also gained the .unmatched argument.

replace_values()

Out of all of the new things introduced in dplyr 1.2.0, I think I’m most excited about replace_values() .

While recode_values() is great for creating an entirely new column (possibly with a new type), if you just need to replace a few rows of an existing column, then replace_values() is the best tool for the job!

Imagine we’d like to collapse some, but not all, of these school names into common buckets:

schools <- tibble(
  name = c(
    "UNC",
    "Chapel Hill",
    NA,
    "Duke",
    "Duke University",
    "UNC",
    "NC State",
    "ECU"
  )
)

We could use case_when() or recode_values() :

schools |>
  mutate(
    name = case_when(
      name %in% c("UNC", "Chapel Hill") ~ "UNC Chapel Hill",
      name %in% c("Duke", "Duke University") ~ "Duke",
      .default = name
    )
  )
#> # A tibble: 8 × 1
#>   name           
#>   <chr>          
#> 1 UNC Chapel Hill
#> 2 UNC Chapel Hill
#> 3 NA             
#> 4 Duke           
#> 5 Duke           
#> 6 UNC Chapel Hill
#> 7 NC State       
#> 8 ECU

schools |>
  mutate(
    name = recode_values(
      name,
      c("UNC", "Chapel Hill") ~ "UNC Chapel Hill",
      c("Duke", "Duke University") ~ "Duke",
      default = name
    )
  )
#> # A tibble: 8 × 1
#>   name           
#>   <chr>          
#> 1 UNC Chapel Hill
#> 2 UNC Chapel Hill
#> 3 NA             
#> 4 Duke           
#> 5 Duke           
#> 6 UNC Chapel Hill
#> 7 NC State       
#> 8 ECU

But this “partial update” operation is so common that it really deserves its own name that doesn’t require you to specify default and is type stable on the input. For that, we have replace_values() :

schools |>
  mutate(
    name = name |>
      replace_values(
        c("UNC", "Chapel Hill") ~ "UNC Chapel Hill",
        c("Duke", "Duke University") ~ "Duke"
      )
  )
#> # A tibble: 8 × 1
#>   name           
#>   <chr>          
#> 1 UNC Chapel Hill
#> 2 UNC Chapel Hill
#> 3 NA             
#> 4 Duke           
#> 5 Duke           
#> 6 UNC Chapel Hill
#> 7 NC State       
#> 8 ECU

Notice how pipe friendly replace_values() is! The first input is your “primary” input, and you can expect the output to have the same type and size as that input.

Like recode_values() , replace_values() has an alternative from and to API that works well with lookup tables and allows you to move your mapping out of the pipe chain:

lookup <- tribble(
  ~from             , ~to               ,
  "UNC"             , "UNC Chapel Hill" ,
  "Chapel Hill"     , "UNC Chapel Hill" ,
  "Duke"            , "Duke"            ,
  "Duke University" , "Duke"            ,
)

schools |>
  mutate(name = replace_values(name, from = lookup$from, to = lookup$to))
#> # A tibble: 8 × 1
#>   name           
#>   <chr>          
#> 1 UNC Chapel Hill
#> 2 UNC Chapel Hill
#> 3 NA             
#> 4 Duke           
#> 5 Duke           
#> 6 UNC Chapel Hill
#> 7 NC State       
#> 8 ECU

An extremely neat feature of the from and to API is that they also take lists of vectors that describe the mapping, which has been designed to work elegantly with the fact that tribble() can create list columns, allowing you to further collapse this lookup table:

# Condensed lookup table with a `many:1` mapping per row
lookup <- tribble(
  ~from                        , ~to               ,
  c("UNC", "Chapel Hill")      , "UNC Chapel Hill" ,
  c("Duke", "Duke University") , "Duke"            ,
)

# Note that `from` is a list column
lookup
#> # A tibble: 2 × 2
#>   from      to             
#>   <list>    <chr>          
#> 1 <chr [2]> UNC Chapel Hill
#> 2 <chr [2]> Duke

lookup$from
#> [[1]]
#> [1] "UNC"         "Chapel Hill"
#> 
#> [[2]]
#> [1] "Duke"            "Duke University"

# Works the same as before
schools |>
  mutate(name = replace_values(name, from = lookup$from, to = lookup$to))
#> # A tibble: 8 × 1
#>   name           
#>   <chr>          
#> 1 UNC Chapel Hill
#> 2 UNC Chapel Hill
#> 3 NA             
#> 4 Duke           
#> 5 Duke           
#> 6 UNC Chapel Hill
#> 7 NC State       
#> 8 ECU

The formula interface of replace_values() is a bit of a Swiss Army knife for all manner of scenarios where you might have previously reached for dplyr::coalesce() , dplyr::na_if() , or tidyr::replace_na() :

state <- c("NC", "NY", "CA", NA, "NY", "Unknown", NA)

# Replace missing values with a constant
replace_values(state, NA ~ "Unknown")
#> [1] "NC"      "NY"      "CA"      "Unknown" "NY"      "Unknown" "Unknown"

# Replace missing values with the corresponding value from another column
region <- c("South", "North", "West", "East", "North", "Unknown", "West")
replace_values(state, NA ~ region)
#> [1] "NC"      "NY"      "CA"      "East"    "NY"      "Unknown" "West"

# Replace problematic values with a missing value
replace_values(state, "Unknown" ~ NA)
#> [1] "NC" "NY" "CA" NA   "NY" NA   NA

# Standardize multiple issues at once
replace_values(state, c(NA, "Unknown") ~ "<missing>")
#> [1] "NC"        "NY"        "CA"        "<missing>" "NY"        "<missing>"
#> [7] "<missing>"

We also think it better expresses intent than if_else() or case_when() when performing a partial update:

# - Type stable on `x`
# - Intent of "partially updating" `state` is clear
# - Pipe friendly
state |> replace_values(NA ~ "Unknown")

# Historically this has been "the way" to do a partial update,
# but it's odd that the "primary" input is at the end!
if_else(is.na(state), "Unknown", state)
case_when(is.na(state) ~ "Unknown", .default = state)

If you’re a package author, you’ll probably also be interested in vctrs::vec_recode_values() and vctrs::vec_replace_values() , which are low dependency functions that power the dplyr variants.

What about case_match()?

We’ve soft-deprecated case_match() in favor of recode_values() , which is a drop in replacement.

case_match() was an incremental step towards this recoding family, but:

• It has a pretty confusing name compared with recode_values() .
• It lacked a way to work with lookup tables, like from and to.
• It lacks a replacement variant, like replace_values() .

Rather than keeping case_match() around indefinitely, we’ve decided to initiate the process of its removal since it was only introduced in dplyr 1.1.0.

Deprecations

dplyr 1.2.0 advances the lifecycle stage of many deprecated functions. These deprecations have been in the works for many years now, due to our slow and very deliberate deprecation process via the lifecycle package . We’ll cover the highlights, and you can find the full list here .

For any packages that we broke via these deprecations, we provided a pull request (or at least an issue, for complex cases) and some advance warning. We semi-automated some of this process using Claude Code, which you can read about here .

• All underscored verbs have moved from deprecated to defunct, such as mutate_() and arrange_() . These have been deprecated since dplyr 0.7.0 back in 2017 (yes, 2017!!). Use the non-underscored versions, see vignette("programming") for details.

• mutate_each() and summarise_each() have moved from deprecated to defunct. These were also deprecated in dplyr 0.7.0. Use mutate() and summarise() with across() instead.

• Returning more or less than 1 row per group in summarise() has moved from deprecated to defunct. This was deprecated in dplyr 1.1.0 in 2023 after we realized that this was an unsafe feature for summarise() , which you should always expect to return exactly 1 row per group. reframe() is a drop in replacement when you need this.

• In case_when() , supplying all size 1 LHS inputs along with a size >1 RHS input is now soft-deprecated. This is an improper usage of case_when() that should instead be a series of if statements, like:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

  # Scalars! code <- 1L flavor <- "vanilla" # Improper usage: case_when( code == 1L && flavor == "chocolate" ~ x, code == 1L && flavor == "vanilla" ~ y, code == 2L && flavor == "vanilla" ~ z, .default = default ) # Recommended: if (code == 1L && flavor == "chocolate") { x } else if (code == 1L && flavor == "vanilla") { y } else if (code == 2L && flavor == "vanilla") { z } else { default }

  The recycling behavior that allows this style of case_when() to work is unsafe, and can result in silent bugs that we’d like to guard against with an error in the future. See this issue for context.

• The dplyr.legacy_locale global option is soft-deprecated. If you used this to affect the ordering of arrange() , use arrange(.locale =) instead. If you used this to affect the ordering of group_by() |> summarise(), follow up with an additional call to arrange(.locale =) instead.

• if_else() no longer allows condition to be a logical array. It must be a logical vector with no dim attribute.

• We’ve removed a number of previously defunct functions, shrinking the footprint of dplyr’s API:

  • id()
  • failwith()
  • select_vars() and select_vars_()
  • rename_vars() and rename_vars_()
  • select_var()
  • current_vars()
  • bench_tbls(), compare_tbls(), compare_tbls2(), eval_tbls(), and eval_tbls2()
  • location() and changes()

Acknowledgements

We’d like to thank all 177 contributors who help in someway, whether it was filing issues or contributing code and documentation: @abalter , @abichat , @adupaix , @AlexBainton , @alexmcsw , @AltfunsMA , @AmeliaMN , @antdurrant , @AnthonyEbert , @apalacio9502 , @apeterson91 , @arnaudgallou , @awpsoras , @bakaburg1 , @barnabasharris , @BHII-KSC , @bholtemeyer , @billdenney , @bounlu , @brendensm , @bridroberts1 , @brookslogan , @catalamarti , @cboettig , @cbrnr , @ccani007 , @charliejhadley , @ChrisHIV , @ChristianRohde , @cobac , @conig , @const-ae , @Copilot , @d-morrison , @DanChaltiel , @daniel-simeone , @DanielBraddock , @david-romano , @davidrsch , @davidss101 , @DavisVaughan , @dcaud , @deschen1 , @DesiQuintans , @devster31 , @dkutner , @dmuenz , @ds-jim , @eitsupi , @EmilHvitfeldt , @etiennebacher , @eutwt , @EvertonTLima , @ferreirafm , @gaborcsardi , @GabryS3 , @Gastonia02 , @GBarnsley , @gevro , @ggrothendieck , @GischD , @gks281263 , @gracehartley , @graphdr , @hadley , @heliconone , @Hzanib , @ilovemane , @ja-ortiz-uniandes , @jack-davison , @james-kilgour , @JamesHWade , @jaymicro , @JBrandenburg02 , @jc-usda , @jennybc , @jeroenjanssens , @jestover , @jl5000 , @jmbarbone , @john-b-edwards , @jordanmross , @joshua-theisen , @jrwinget , @juliaapolonio , @jxu , @KaiAragaki , @kiki830621 , @KittJonathan , @kleinerChemiker , @kletts , @krlmlr , @ks8997 , @kylebutts , @larsentom , @latot , @lboller-pwbm , @lionel- , @Longfei2 , @lschneiderbauer , @LukasTang , @lukebandy , @maciekbanas , @maelle , @marcuslehr , @Mark-AP , @markwestcott34 , @maskegger , @matiasandina , @matthewjnield , @mbcann01 , @Meghansaha , @metanoid , @MichaelChirico , @MikeJohnPage , @MilesMcBain , @mine-cetinkaya-rundel , @MohsenSoltanifar , @moodymudskipper , @Moohan , @mp8 , @mpsturbo , @mr-c , @muschellij2 , @musvaage , @Mzhuk7 , @nalimilan , @nathanhaigh , @nirguk , @nmercadeb , @olivermagnanimous , @olivroy , @orgadish , @pangchaoran , @paschatz , @prubin73 , @PStaus , @psychelzh , @py9mrg , @Raesu , @randyzwitch , @Raoul-Kima , @ReedMerrill , @RodDalBen , @RodrigoZepeda , @rossholmberg , @RoyalTS , @ryandward , @sbanville-delfi , @ScientiaFelis , @shirdekel , @slager , @sschooler , @steffen-stell , @szimmer , @TheClownBongo , @thomasjwood , @TimTaylor , @tlyons253 , @tomalrussell , @tomwagstaff-opml , @torfason , @Tyrrx , @Unaimend , @VisruthSK , @vorpalvorpal , @walkerjameschris , @wbvguo , @wbzyl , @wkumler , @yaboody , @yjunechoe , @ynsec37 , @ywhcuhk , @ZHBHSMILE , @zhjx19 , and @ZIBOWANGKANGYU .

We’re happy to announce that ragnar 0.3.0 is now available on CRAN. ragnar is a tidy, transparent toolkit for building trustworthy retrieval-augmented generation (RAG) workflows: ingest documents, build a store, retrieve relevant chunks, and inspect exactly what’s being fed to a model.

If you’re new to ragnar, the quickest way to get oriented is the Getting Started vignette . If you’ve already built a store with ragnar 0.2, this release focuses on making it easier to scale ingestion, use more embedding providers, and connect your store to the tools you already use.

You can install ragnar from CRAN with:

install.packages("ragnar")

This post covers the biggest user-facing changes in ragnar 0.3.0. For a complete list of changes, see the NEWS .

library(ragnar)

A quick refresher

If you’re already familiar with ragnar, feel free to skip this section.

ragnar helps you build retrieval-augmented generation (RAG) workflows by turning your trusted documents into a local store that you can query with both vector search (embeddings) and keyword search (BM25).

At the “front door”, read_as_markdown() can ingest web pages, PDFs, Office documents, images (via OCR), archives, and even YouTube URLs (via transcripts), so you can usually start from the same sources you’d use for manual research.

At a high level, a typical ragnar workflow has three parts:

1. Build a store:
   • Collect document sources (URLs or files) and convert them to Markdown with read_as_markdown() .
   • Split documents into chunks with markdown_chunk() (optionally adding context).
   • Embed and store chunks in a DuckDB-backed RagnarStore.
2. Query and inspect the store:
   • Retrieve chunks directly with ragnar_retrieve() . It returns a tibble with scores, source information, and the chunk text (including columns like origin, cosine_distance, bm25, context, and text), so you can inspect exactly what will be passed downstream.
   • Use the Store Inspector or Embedding Atlas (ragnar_store_inspect() and ragnar_store_atlas() ) to understand what’s working, then iterate and go back to step 1 as needed.
3. Connect the store to tools:
   • Register a retrieval tool with an ellmer chat so an agent can search the store on demand.
   • Serve retrieval over MCP so external tools and agents can query the store directly.
   • Write your own loop using ragnar_retrieve() or lower-level helpers like ragnar_retrieve_vss() and ragnar_retrieve_bm25() .

What’s new

This release focuses on four big improvements:

• Faster ingestion for large corpora with ragnar_store_ingest() .
• Better retrieval with multi-query support and better de-duplication and de-overlapping of results.
• New embedding providers: Azure OpenAI and Snowflake.
• New integrations and tooling: serve a store over MCP, plus improved inspection with the Store Inspector and embedding atlas.

In the sections below, we’ll walk through each change in more detail.

Faster ingestion with ragnar_store_ingest()

Ingestion is usually the slowest part of building a knowledge store. ragnar_store_ingest() parallelizes the document preparation step with mirai , and then writes prepared chunks to the store in the main process. It’s designed to make it easy to ingest hundreds (or thousands) of pages without hand-rolling your own parallel pipeline.

Only preparation (reading, chunking, and optionally embedding) is parallelized; store writes still happen in the main process.

store <- ragnar_store_create(
  "docs.ragnar.duckdb",
  embed = \(x) ragnar::embed_openai(x, model = "text-embedding-3-small")
)

paths <- ragnar_find_links("https://quarto.org/sitemap.xml")

ragnar_store_ingest(store, paths, n_workers = 4, prepare = \(path) {
  path |> read_as_markdown() |> markdown_chunk()
})

Better retrieval: multiple queries and fewer duplicates

Retrieval is where ragnar tries to be pragmatic: we run both semantic search (embeddings) and keyword search (BM25) because they fail in different ways. This release makes it easier to do that intentionally.

• ragnar_retrieve() now accepts a vector of queries, so you can pass one query tuned for semantic search and one tuned for keywords.
• ragnar_register_tool_retrieve() uses a new default tool name prefix: search_{store@name} (instead of rag_retrieve_from_{store@name}).
• When registered with ellmer, ragnar’s retrieval tool continues to avoid returning previously returned chunks, enabling deeper searches via repeated tool calls.
• BM25 result ordering was corrected to sort by descending score.
• Duplicate rows from ragnar_retrieve() when running multiple queries were removed.

ragnar_retrieve(
  store,
  c(
    "How do I subset a data frame with a logical vector?",
    "subset dataframe logical vector"
  ),
  top_k = 10
)

New embedding providers: Azure OpenAI and Snowflake

ragnar’s embedding helpers continue to expand so you can use the infrastructure you already have:

• embed_azure_openai() supports embeddings from Azure AI Foundry.
• embed_snowflake() supports embeddings via the Snowflake Cortex Embedding API.

These integrate the same way as the other providers: you choose an embed function when creating a store, and ragnar uses it during insertion and retrieval.

Better document reading (including YouTube transcripts)

read_as_markdown() is now more robust across common inputs, so you get higher-quality documents without having to hand-fix edge cases.

• Substantial improvements to HTML-to-Markdown conversion, including correct handling of nested code blocks, plus a range of other robustness fixes driven by real-world failure cases.
• read_as_markdown() once again fetches YouTube transcripts and now supports a youtube_transcript_formatter so you can include timestamps or links in the transcript output.
• Reading plain text with non-ASCII content was fixed.
• read_as_markdown() gained an origin argument to control what gets recorded on returned documents.

Together, these changes make ingestion more reliable, which helps improve retrieval quality downstream.

New integrations: serve a store over MCP

mcp_serve_store() lets you expose a RagnarStore as an MCP tool. This is particularly useful if you already have a local store and want an MCP-enabled client (like Codex CLI or Claude Code) to query it directly.

For example, with Codex CLI you can add something like this to ~/.codex/config.toml:

1
2
3
4
5
6

[mcp_servers.my_store]
command = "Rscript"
args = [
  "-e",
  "ragnar::mcp_serve_store('docs.ragnar.duckdb', top_k=10)"
]

This runs a long-lived R process that exposes retrieval over MCP.

New ways to inspect a store

ragnar now has more tools to help you understand what your store contains and why retrieval is (or isn’t) working:

• The Store Inspector received a number of usability improvements (keyboard shortcuts, improved preview, better metadata display, and general bug fixes).
• ragnar_store_atlas() integrates with the Embedding Atlas project to visualize your embedding space (via reticulate).

The Store Inspector makes it easy to iterate on retrieval: try a query, compare vector search and BM25, and inspect the underlying chunks and metadata that were returned. The screenshots below show a store built from the Quarto documentation.

If you’re not sure whether a store “looks right”, ragnar_store_atlas() gives you a high-level view of how your documents cluster in embedding space. It’s a useful way to spot outliers, see which areas of the space match a query, and explore how clusters relate back to your sources.

Get started

Install ragnar with install.packages("ragnar"), then work through the Getting Started vignette . For details on individual functions, see the function reference . For the full changelog, see NEWS .

ragnar is designed to help you build trustworthy RAG workflows by making it easy to inspect what gets retrieved and what ultimately gets sent to your model. If you try ragnar 0.3.0, we’d love to hear what you’re using it for in GitHub Discussions .

Acknowledgements

Thanks to everyone who contributed to ragnar 0.3.0 through code, issues, testing, and feedback: @agricolamz , @AlekFisher , @bianchenhao , @brooklynbagel , @bshashikadze , @christophscheuch , @cstubben , @dfalbel , @eschillerstrom-usfws , @grantmcdermott , @howardbaik , @jeroenjanssens , @jhbrut , @JosiahParry , @jpmarindiaz , @luisDVA , @mattwarkentin , @Rednose22 , @shikokuchuo , @smach , @SokolovAnatoliy , @t-kalinowski , @thisisnic , and @vrognas .

Let’s say you developed a dashboard to display women’s international soccer matches. You are proud of what you have built and you eagerly show it to a colleague:

Colleague: “Amazing! Can you show me the soccer matches for the FIFA World Cup only?”

You: “Of course, let me filter it down for you and select the FIFA World Cup tournaments”

Colleague: “Interesting, can you show me all the matches in which The Netherlands have played?”

You: “Eh… Well, I could, but I just have to include a country filter in my dashboard then!”

Right. How are we supposed to filter down to a specific country if there is no input for it? And what about getting summary statistics for countries or players? By now it becomes painfully clear that our soccer dashboard has its limits.

This is exactly the moment when querychat  becomes interesting. It is a multilingual package that allows you to chat with your data using natural language queries. No more clicking, no more limited filters, just you and your questions. And in this article, you’re going to learn everything about it!

To bring querychat to life, we will keep returning to two examples:

• The classic diamonds dataset. After all, diamonds are a girl’s best friend, and a data scientist’s too! The familiar dataset offers a mix of variables such as cut, colour, clarity and price, which makes it ideal for all sorts of natural language questions. You might wonder about average prices for particular cuts, or you want to compare colours, look at how clarity affects value, or explore simple patterns in the data. In other words, it is a perfect playground for testing how well natural language queries behave on structured data.
• SheScores, the soccer dashboard that you were so proud of earlier. This app originates from the shiny::conf(2024) workshop “Shiny 101: The Modular App Blueprint” , although it has been tweaked to make it a bit more interesting and updated with matches through to November 2025.

• Python
• R

Both datasets set the stage nicely, so let’s roll the ball and see how querychat plays. We’re talking about soccer after all!

Full code available on GitHub

Instead of copy-pasting the content of this blog into your favourite IDE, you can also pull the project from GitHub and follow along. All the code is available in both Python and R.

Short on time?

Jump straight to the SheScores app with querychat or visit the querychat website

Hello, querychat

In short, querychat makes it easy to query data using natural language. It offers a drop-in component for Shiny, a console interface, and other programmatic building-blocks. You ask questions, querychat translates it to a SQL query, executes it, and returns the results. The results are available as a reactive data frame, which makes it easy to display or further process the data.

querychat would solve the problem we encountered earlier. We can ask it any question we can imagine without constantly adding filters or other analysis. No country filter? No problem. And yes, that sounds as cool as it is!

So, what do we need?

querychat is powered by a Large Language Model (LLM), so you need access to a model. You first need to register at an LLM provider that provides those models. You can choose any model you like, with two little “restrictions”: chatlas (Python) or ellmer (R) supports it (which shouldn’t be hard, because all the major models are) and the model has the ability to do tool calls.

Recommended models

In this blog we’ll use Claude Sonnet 4.5 from Anthropic. Other good choices would be GPT-4.1 (the current default for querychat) and Google Gemini 3.0 (as of November 2025).

Once you’ve made your choice and registered, you can get an API key. You need this key to authenticate with the LLM provider. One important note: never, ever hardcode the key directly into your script. You’ll be amazed how many keys are publicly available on GitHub repos. Don’t be that developer. As always with secrets, store it as an environment variable. Just note that the exact name of the key depends on the provider. For example, Anthropic expects ANTHROPIC_API_KEY=yourkey, while OpenAI uses OPENAI_API_KEY=yourkey.

1

ANTHROPIC_API_KEY=yourkey

1
2

from dotenv import load_dotenv
load_dotenv()

1

ANTHROPIC_API_KEY=yourkey

Of course we can’t use querychat without installing it, so that’s the next step:

1

pip install querychat

1

uv add querychat

1

import querychat

1

install.packages("querychat")

1

pak::pak("posit-dev/querychat/pkg-r")

1

library(querychat)

Wouldn’t it be great if you can use querychat straight away without much code? Just to see what it’s all about? Luckily you can with the “quick launch” Shiny app! You can simply call app() which spins up an app with querychat chat interface. Let’s try it out for our diamonds dataset:

1
2
3
4
5
6

from seaborn import load_dataset
from querychat import QueryChat

diamonds = load_dataset("diamonds")
qc = QueryChat(diamonds, "diamonds", client="anthropic/claude-sonnet-4-5")
app = qc.app()

1

shiny run --reload diamonds-app.py 

1
2
3
4

library(ellmer)
library(ggplot2)

querychat_app(diamonds, client = "claude/claude-sonnet-4-5")

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

library(ellmer)
library(ggplot2)

qc <- QueryChat$new(
  diamonds,
  "diamonds",
  client = "claude/claude-sonnet-4-5"
)

qc$app()

The result: a Shiny app that allows users to interact with a data source using natural language queries.

• Python
• R

Custom branding

Do you notice the nice green touches and custom font in this demo app? That’s because the project we’ll be using in this article uses brand.yml : a simple, portable YAML file that codifies brand guidelines into a format that can be used by Quarto, Python and R. And in this case, it works beautifully for Shiny. Curious to see what such a _brand.yml file looks like? You can check it out here .

You can ask the diamonds dataset some surprisingly rich questions, and querychat handles them with ease. A simple place to begin is something like “show the 10 most expensive diamonds”. It produces straightforward SQL and updates the table in the app instantly.

1

SELECT * FROM diamonds ORDER BY price DESC LIMIT 10

The question may be simple, but it already highlights the convenience of querychat. Without it, users would need to sort the table manually or rely on a picker, slider or some other input that filters the data for this very specific request.

Things get more interesting when we introduce a calculation. Asking “can you show the 20 biggest diamonds, based on volume?” still results in simple SQL, but the output now includes an extra column, volume, which appears neatly in the app.

1
2
3
4
5
6

-- Calculate volume (x * y * z) and sort by largest volume
SELECT *, 
       x * y * z AS volume
FROM diamonds 
ORDER BY volume DESC 
LIMIT 20

From there, we can try some grouping and window functions. “Within each cut, what is the most expensive diamond?” works perfectly, showing the grouped results along with the necessary window function behind the scenes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

-- Find the most expensive diamond for each cut type
SELECT 
    cut,
    MAX(price) AS max_price,
    -- Get the details of the diamond with the max price for each cut
    ARG_MAX(carat, price) AS carat,
    ARG_MAX(color, price) AS color,
    ARG_MAX(clarity, price) AS clarity,
    ARG_MAX(depth, price) AS depth,
    ARG_MAX("table", price) AS table_pct
FROM diamonds
GROUP BY cut
ORDER BY max_price DESC

1
2
3
4
5
6

        cut max_price carat color clarity depth table_pct
1   Premium     18823  2.29     I     VS2  60.8        60
2 Very Good     18818  2.00     G     SI1  63.5        56
3     Ideal     18806  1.51     G      IF  61.7        55
4      Good     18788  2.80     G     SI2  63.8        58
5      Fair     18574  2.01     G     SI1  70.6        64

Now, for a grand finale, we can throw in a more analytical question: “are larger physical dimensions always associated with higher price?” This one generates a slightly more complex SQL query, but it also comes with a clear and helpful explanation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24