Python on Posit Open Source

Recreating Septa Transit Timetables in Python

Michael Chow — Thu, 12 Mar 2026 00:00:00 +0000

Recently, Rich and I were poking around transit data, and we were struck by the amount of structuring that goes into transit timetables.

For example, consider this weekend rail schedule table from SEPTA, Philadelphia’s transit agency.

Notice these big pieces:

The vertical text on the left indicating trains are traveling “TO CENTER CITY”.
The blue header, and spanner columns (“Services” and “Train Number”) grouping related columns.
The striped background for easier reading. Also the black background indicating stations in Center City (the urban core).

Tables like this often have to be created in tools like Illustrator, and updated by hand. At the same time, when agencies automate table creation, they often sacrifice a lot of the assistive features and helpful affordances of the table.

We set out to recreate this table in Great Tables (and by we I mean 99% Rich). In this post, I’ll walk quickly through how we recreated it, and share some other examples of transit timetables in the wild. For the theory behind why tables like this are useful, see The Design Philosophy of Great Tables .

The final result

Here’s a look at our quick version in Great Tables. In this post we’ll walk through quickly how we created it, but wanted to treat you to the final result up front! Note that the table is fully in HTML for accessibility.

Code

PDF Accessibility and Standards

Gordon Woodhull — Thu, 05 Mar 2026 00:00:00 +0000

Pre-release Feature

This feature is new in the upcoming Quarto 1.9 release. To use the feature now, you’ll need to download and install the Quarto pre-release.

2025 was a big year for PDF accessibility. LaTeX and Typst both released support for PDF tagging and accessibility standards, just in time for new regulations in the EU (June 2025) and US (April 2026).

Quarto 1.9 brings this support to you as a Quarto user.

What PDF Standards Do

Currently LaTeX supports the newer UA-2 standard, and Typst supports the older UA-1 standard. Typst is likely to have UA-2 support later in 2026.

Both standards instruct the PDF renderer to provide screen readers:

The semantic structure of the text (title, heading, paragraph, figure, etc)
The natural reading order
Spatial coordinates for highlighting and assistive navigation
Required metadata such as title and language

How to enable a PDF Standard in Quarto

In Quarto 1.9, specify a PDF standard for your document or project with pdf-standard

PDF (LaTeX)

format:
  pdf:
    pdf-standard: ua-2

Typst

format:
  typst:
    pdf-standard: ua-1

pdf-standard takes a single standard name or list of standard names. PDF version is used if provided in the list, but otherwise inferred from the standard.

If you specify a PDF standard, Quarto first instructs LaTeX or Typst to use the standard when producing the PDF, and then validates the output PDF against the standard using veraPDF, an open-source PDF validation tool. If veraPDF is not installed, you’ll get a warning but still receive a PDF – it just won’t be validated.

Installing veraPDF

To install veraPDF, you’ll first need Java, then run:

Terminal

1

quarto install verapdf

When a document passes validation, you’ll see output like:

[verapdf]: Validating my-document.pdf against PDF/UA-2... PASSED

Creating accessible PDFs

Quarto’s Markdown-based workflow handles many accessibility requirements automatically:

Document metadata (title, author, date, language) flows into the PDF’s built-in metadata fields.
The semantic structure of Markdown satisfies PDF tagging requirements. For Typst this is always enabled; for LaTeX it is enabled when you specify a standard that requires it.
Alt text for images is carried through to the PDF for screen readers.

But you do need to make sure your document has:

A title in the YAML front matter.
Alt text for every image, specified with fig-alt. See Figures for details.

See the LaTeX and Typst documentation for more details.

If your document fails validation

LaTeX does not perform validation during PDF generation, so if veraPDF validation fails, that’s a warning, and you still get a partially-accessible PDF as long as you use pdf-standard: ua-2.

Typst fails and does not produce a PDF if its built-in validation fails during PDF generation. However, in Typst all accessibility features are on by default, so you can generate a partially-accessible PDF by rendering without pdf-standard.

Current limitations

We ran our test suite – 188 LaTeX examples and 317 Typst examples – to find where Quarto PDFs do not yet pass UA-1 or UA-2, and where users will need to change their documents.

LaTeX

Margin content is the biggest structural blocker. If you use .column-margin divs, cap-location: margin, reference-location: margin, or citation-location: margin, the resulting PDF will not pass UA-2. The underlying sidenotes and marginnote LaTeX packages do not cooperate with PDF tagging .

(Margin content does work with Typst and passes UA-1 – see Typst Article Layout .)

There are smaller upstream issues in Pandoc, LaTeX, and LaTeX packages, documented here .

Typst

In our tests, Typst catches every UA-1 violation, and fails to generate the PDF. veraPDF did not detect any violation that Typst did not.

Typst also seems to do a very good job of generating UA-1 compliant output by default – almost all errors were due to missing titles or missing alt text.

However, we did discover that Typst books are not yet compliant. There is a structural problem with the Typst orange-book package and we’ll work with the maintainers to correct it.

Conclusion

Although Typst currently targets an the earlier UA-1 standard, today it seems to offer better PDF accessibility than LaTeX.

We expect PDF accessibility support to improve through the LaTeX ecosystem throughout 2026 as awareness of UA-2 and the new regulations spreads.

If you run into accessibility issues with PDF output, please search the Quarto discussions and open a new one with the accessibility label for any issues you discover.

Where Questions Become Queries: Meet querychat

Veerle Eeftink - Van Leemput — Thu, 22 Jan 2026 00:00:00 +0000

You love data. And you love building dashboards with it, especially with your favourite tool Shiny. But even with a bullet-proof design, dozens of user stories, feedback loops, and adjustments, you also know that there are always questions that your dashboard leaves unanswered.

Let’s say you developed a dashboard to display women’s international soccer matches. You are proud of what you have built and you eagerly show it to a colleague:

Colleague: “Amazing! Can you show me the soccer matches for the FIFA World Cup only?”

You: “Of course, let me filter it down for you and select the FIFA World Cup tournaments”

Colleague: “Interesting, can you show me all the matches in which The Netherlands have played?”

You: “Eh… Well, I could, but I just have to include a country filter in my dashboard then!”

Right. How are we supposed to filter down to a specific country if there is no input for it? And what about getting summary statistics for countries or players? By now it becomes painfully clear that our soccer dashboard has its limits.

This is exactly the moment when querychat becomes interesting. It is a multilingual package that allows you to chat with your data using natural language queries. No more clicking, no more limited filters, just you and your questions. And in this article, you’re going to learn everything about it!

To bring querychat to life, we will keep returning to two examples:

The classic diamonds dataset. After all, diamonds are a girl’s best friend, and a data scientist’s too! The familiar dataset offers a mix of variables such as cut, colour, clarity and price, which makes it ideal for all sorts of natural language questions. You might wonder about average prices for particular cuts, or you want to compare colours, look at how clarity affects value, or explore simple patterns in the data. In other words, it is a perfect playground for testing how well natural language queries behave on structured data.
SheScores, the soccer dashboard that you were so proud of earlier. This app originates from the shiny::conf(2024) workshop “Shiny 101: The Modular App Blueprint” , although it has been tweaked to make it a bit more interesting and updated with matches through to November 2025.

Python
R

Both datasets set the stage nicely, so let’s roll the ball and see how querychat plays. We’re talking about soccer after all!

Full code available on GitHub

Instead of copy-pasting the content of this blog into your favourite IDE, you can also pull the project from GitHub and follow along. All the code is available in both Python and R.

Short on time?

Jump straight to the SheScores app with querychat or visit the querychat website

Hello, querychat

In short, querychat makes it easy to query data using natural language. It offers a drop-in component for Shiny, a console interface, and other programmatic building-blocks. You ask questions, querychat translates it to a SQL query, executes it, and returns the results. The results are available as a reactive data frame, which makes it easy to display or further process the data.

querychat would solve the problem we encountered earlier. We can ask it any question we can imagine without constantly adding filters or other analysis. No country filter? No problem. And yes, that sounds as cool as it is!

So, what do we need?

querychat is powered by a Large Language Model (LLM), so you need access to a model. You first need to register at an LLM provider that provides those models. You can choose any model you like, with two little “restrictions”: chatlas (Python) or ellmer (R) supports it (which shouldn’t be hard, because all the major models are) and the model has the ability to do tool calls.

Recommended models

In this blog we’ll use Claude Sonnet 4.5 from Anthropic. Other good choices would be GPT-4.1 (the current default for querychat) and Google Gemini 3.0 (as of November 2025).

Once you’ve made your choice and registered, you can get an API key. You need this key to authenticate with the LLM provider. One important note: never, ever hardcode the key directly into your script. You’ll be amazed how many keys are publicly available on GitHub repos. Don’t be that developer. As always with secrets, store it as an environment variable. Just note that the exact name of the key depends on the provider. For example, Anthropic expects ANTHROPIC_API_KEY=yourkey, while OpenAI uses OPENAI_API_KEY=yourkey.

Python
R

In Python, the recommended approach is to create a .env file in your project folder:

1

ANTHROPIC_API_KEY=yourkey

It’s recommended to use the dotenv package to load the .env file into your environment:

1
2


from dotenv import load_dotenv
load_dotenv()

To keep the demo code concise we’ll omit these lines from subsequent code examples.

In R, environment variables are typically stored in a .Renviron file. You can create this file in your project root or in your home directory (~/.Renviron). Or, if you want to make it yourself really easy: you can also open/edit the relevant file with usethis::edit_r_environ().

1

ANTHROPIC_API_KEY=yourkey

Of course we can’t use querychat without installing it, so that’s the next step:

Python
R

For Python, querychat is available on PyPI, so you can install it easily with pip:

1

pip install querychat

Or, if you’re using uv, add it like so:

1

uv add querychat

Once installed, import it like this:

1

import querychat

You can get querychat from CRAN using:

1

install.packages("querychat")

Alternatively, if you want the latest development version, you can install querychat from GitHub using:

1

pak::pak("posit-dev/querychat/pkg-r")

Once installed, load the package as usual:

1

library(querychat)

Wouldn’t it be great if you can use querychat straight away without much code? Just to see what it’s all about? Luckily you can with the “quick launch” Shiny app! You can simply call app() which spins up an app with querychat chat interface. Let’s try it out for our diamonds dataset:

Python
R

1
2
3
4
5
6


from seaborn import load_dataset
from querychat import QueryChat

diamonds = load_dataset("diamonds")
qc = QueryChat(diamonds, "diamonds", client="anthropic/claude-sonnet-4-5")
app = qc.app()

qc = QueryChat(...) creates an instance of the QueryChat class. You pass in the dataset, give it a name and specify the model client (powered by chatlas ). qc.app() returns the web app that lets you explore the diamonds data using natural language questions.

Want to try this with a different provider and/or model? No problem, just change the client argument accordingly. For example, to use GPT-4.1 from OpenAI, you would write: client="openai/gpt-4.1". You can learn more about the different options in the querychat documentation .

To run this app, you need to save the code above in a file (and call it diamonds-app.py for example) and run it like so:

1

shiny run --reload diamonds-app.py

1
2
3
4


library(ellmer)
library(ggplot2)

querychat_app(diamonds, client = "claude/claude-sonnet-4-5")

Or, alternatively, you could write:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


library(ellmer)
library(ggplot2)

qc <- QueryChat$new(
  diamonds,
  "diamonds",
  client = "claude/claude-sonnet-4-5"
)

qc$app()

Both result in the same outcome, the first one is just a simplified version.

QueryChat$new() creates the R6 object, taking the dataset, a table name and the model client (which will be passed to ellmer::chat()). Calling qc$app() then launches the Shiny app so you can query the diamonds dataset in plain English.

Want to change the provider and/or model? No problem, just change the client argument accordingly. For example, to use GPT-4.1 from OpenAI, you would write: client = "openai/gpt-4.1". You can learn more about the different options in the querychat documentation .

The result: a Shiny app that allows users to interact with a data source using natural language queries.

Python
R

Custom branding

Do you notice the nice green touches and custom font in this demo app? That’s because the project we’ll be using in this article uses brand.yml : a simple, portable YAML file that codifies brand guidelines into a format that can be used by Quarto, Python and R. And in this case, it works beautifully for Shiny. Curious to see what such a _brand.yml file looks like? You can check it out here .

You can ask the diamonds dataset some surprisingly rich questions, and querychat handles them with ease. A simple place to begin is something like “show the 10 most expensive diamonds”. It produces straightforward SQL and updates the table in the app instantly.

1

SELECT * FROM diamonds ORDER BY price DESC LIMIT 10

The question may be simple, but it already highlights the convenience of querychat. Without it, users would need to sort the table manually or rely on a picker, slider or some other input that filters the data for this very specific request.

Things get more interesting when we introduce a calculation. Asking “can you show the 20 biggest diamonds, based on volume?” still results in simple SQL, but the output now includes an extra column, volume, which appears neatly in the app.

1
2
3
4
5
6


-- Calculate volume (x * y * z) and sort by largest volume
SELECT *, 
       x * y * z AS volume
FROM diamonds 
ORDER BY volume DESC 
LIMIT 20

From there, we can try some grouping and window functions. “Within each cut, what is the most expensive diamond?” works perfectly, showing the grouped results along with the necessary window function behind the scenes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


-- Find the most expensive diamond for each cut type
SELECT 
    cut,
    MAX(price) AS max_price,
    -- Get the details of the diamond with the max price for each cut
    ARG_MAX(carat, price) AS carat,
    ARG_MAX(color, price) AS color,
    ARG_MAX(clarity, price) AS clarity,
    ARG_MAX(depth, price) AS depth,
    ARG_MAX("table", price) AS table_pct
FROM diamonds
GROUP BY cut
ORDER BY max_price DESC

1
2
3
4
5
6


        cut max_price carat color clarity depth table_pct
1   Premium     18823  2.29     I     VS2  60.8        60
2 Very Good     18818  2.00     G     SI1  63.5        56
3     Ideal     18806  1.51     G      IF  61.7        55
4      Good     18788  2.80     G     SI2  63.8        58
5      Fair     18574  2.01     G     SI1  70.6        64

Now, for a grand finale, we can throw in a more analytical question: “are larger physical dimensions always associated with higher price?” This one generates a slightly more complex SQL query, but it also comes with a clear and helpful explanation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


-- Calculate correlation between physical dimensions and price
-- Also look for outliers where small diamonds cost more than large ones
WITH dimension_stats AS (
  SELECT 
    carat,
    x, y, z,
    -- Calculate volume as a proxy for overall size
    x * y * z AS volume,
    price,
    -- Rank by volume and price separately
    RANK() OVER (ORDER BY x * y * z DESC) AS volume_rank,
    RANK() OVER (ORDER BY price DESC) AS price_rank
  FROM diamonds
  WHERE x > 0 AND y > 0 AND z > 0  -- Exclude malformed records
)
SELECT 
  CORR(x, price) AS x_price_correlation,
  CORR(y, price) AS y_price_correlation,
  CORR(z, price) AS z_price_correlation,
  CORR(volume, price) AS volume_price_correlation,
  -- Count cases where rank differences are significant (outliers)
  COUNT(CASE WHEN volume_rank - price_rank > 1000 THEN 1 END) AS large_cheap_diamonds,
  COUNT(CASE WHEN price_rank - volume_rank > 1000 THEN 1 END) AS small_expensive_diamonds
FROM dimension_stats

1
2
3
4
5
6


x_price_correlation        : 0.8872314
y_price_correlation        : 0.8678642
z_price_correlation        : 0.8682064
volume_price_correlation   : 0.9042547
large_cheap_diamonds       : 21082
small_expensive_diamonds   : 21065

The conclusion?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


Answer: No, larger physical dimensions are NOT always associated with higher price

While there's a strong positive correlation (~0.90 for overall volume), quality factors can override size.

Correlation Analysis
Volume ↔ Price: 0.90 correlation
X dimension ↔ Price: 0.89 correlation
Y dimension ↔ Price: 0.87 correlation
Z dimension ↔ Price: 0.87 correlation
This means physical size explains about 80-90% of price variation, but not all of it.

The Key Finding
I found over 21,000 cases where small diamonds are more expensive than much larger ones, and vice versa. This happens because quality factors (cut, color, clarity) matter significantly.

Concrete Example
Here's a smaller diamond that costs more per unit volume than typical large diamonds:

Small, premium diamond: 74mm³ volume, 1.41 carat, $9,752 (Ideal cut, H color, VS1 clarity)
This costs $131 per mm³ of volume
Compare this to larger diamonds with lower quality that sell for much less per unit volume due to poor cut, color, or clarity grades.

To encourage further exploration, querychat presents suggestions such as comparing how cut quality affects the price to carat ratio, finding diamonds where clarity has the biggest impact on price, or checking the price difference between the best and worst colour grades for similar sized stones. Yeah, that’s right, you don’t even have to come up with questions yourself.

Some questions result in a filtered table, others result in an explanation with results in the chat window. querychat figures out, based on your question, whether you want an answer straight away, or want to inspect the filtered data yourself. Pretty cool that this only took a few lines of code.

LLMs can make mistakes

Note that it’s still an LLM that generates these queries. LLMs can make mistakes. The nice thing about querychat though, is that you can inspect the SQL query yourself.

Most issues fall into two categories:

Query errors: the SQL may fail to run or may not fully reflect what you intended. When it fails, the model will often try again. In this case, giving more context about the data can help.

Result errors: even when the query is correct, the model may misunderstand or oversimplify the results, especially if that result is large or complex. The result might be that key insights are missed or misinterpreted.

Why this matters: reliability, transparency, reproducibility

What makes the “quick launch” app so powerful is that it is far more than a chat window sitting on top of a dataset. Think back to the questions we explored earlier. We filtered, sorted, computed new columns, grouped data and used window functions. We also looked at analytical relationships without writing a single line of code. And that is only the beginning. If you want to go further, you can hunt for anomalies, create categories, build benchmarks or explore almost any analysis you can imagine. The key is that you never have to think about how to do it. You just ask.

And yes, you could ask all those questions in a typical LLM chat tool. But querychat is different. You are not relying on the model to invent answers or reason about the data internally. Instead, every single question is translated into SQL, executed on the actual dataset and returned exactly as the data dictates. And crucially, the SQL is always shown, so you can see precisely what is being run.

This brings four important benefits:

Reliability: the LLM does not analyse or transform the raw data itself. It only generates SQL text. querychat handles the execution of that SQL via tool calling so all results come from the real data engine, not from the model’s internal guesswork.
Transparency: every query reveals the full SQL statement. Nothing is hidden, nothing is adjusted, and you always know how the answer was produced.
Reproducibility: since every SQL query is visible, analyses can be reused, shared, and audited.
Safety: querychat’s tools are designed with read-only actions in mind, meaning the LLM is essentially unable to perform destructive actions. However, to fully guarantee no destructive actions on your production database, make sure querychat’s database permissions are read-only!

Python
R

How it works: tool calling

If you read The Shiny Side of LLMs blog series, you already know a bit about tool calling. In that series we explored how LLMs can call external tools instead of trying to do everything themselves, and querychat is a very practical example of this idea in action.

Tool calling is essentially a bridge between an LLM and your Python or R session. The model does not execute code. Instead, it requests your Python or R session execute a certain function with certain inputs (e.g., a SQL statement). Once Python or R performs the execution, the result is then passed back to the model for interpretation.

So how does tool calling help us here? Well, LLMs have their strengths and weaknesses. They are not great at counting things, creating data summaries or doing basic calculations. But they are excellent at taking natural language and turning it into structured code. SQL that is. This SQL is then executed through a tool call: a function that executes the (read only) SQL. In both Python and R this means the LLM can express your question as a request to call a function with precise arguments, and the host language performs the real work. This makes it all reliable, reproducible, and safe (read only SQL).

Given that, generally speaking, LLMs are very good at writing SQL, it makes perfect sense to ask one to translate your natural language questions into SQL queries. In order to generate SQL that can be executed, the LLM does need to know something about your data: which columns are there, what do they mean, and what type are they? This schema information is shared with the model, but not the raw data. With this information, it produces an SQL query as a tool call. Now, to run SQL you need a database engine. querychat’s weapon of choice: DuckDB . Basically, our diamonds dataset gets turned into a DuckDB database, and generated SQL queries are executed on this database. Then the results are passed back to the LLM so it can say some interesting things about it.

To summarise:

prompt → SQL query → tool call → execute SQL query → return results¹

Tool calling is worth emphasising because it gives us a controlled and predictable interface between LLMs and real code execution. Instead of writing and maintaining your own custom tools, you can turn to querychat. It already provides the functions needed to turn natural language into reliable SQL that Python or R can execute with confidence.

Customising `querychat`: from chat to toolkit

Alright, enough talking. You now know what querychat can do, and how it does it (high-level). You might even have brilliant ideas for your next app… In that case it would be nice to know how to build your own app with querychat. The Diamonds “quick launch” app from earlier, that you run with qc.app() (Python) or qc$app() (R), consists of a handful of methods that you can find in querychat, and we’re going to use them directly.

The main component is the QueryChat object, which has different arguments and methods.

QueryChat object

You call QueryChat to initialise a QueryChat object (often called qc), like so:

Python
R

1

qc = QueryChat(...)

1

qc <- QueryChat$new(...)

You can pass QueryChat several arguments:

data_source and table_name

These are the two most important arguments: they specify your data source and the name of your table that can be used for the SQL queries. The data source can be your data frame, a tibble, a table or any other Python or R data object, and the table name is usually the variable name of your data frame. In our example our data_source was diamonds, which we also stored in a variable called diamonds.
- Python
- R
1

qc = QueryChat(diamonds, "diamonds")
Generally, in R, the table name isn’t required as it can be inferred from the variable name. However, it is required when you use a database connection, which we’ll use later.
1 2 3 4

qc <- QueryChat$new( diamonds, "diamonds" )
You’re not limited to data objects: you can also pass a database connection to data_source. We’ll come back to that later.
client

We used the client argument before: we use it to tell querychat that we want to use Claude Sonnet 4.5 (or any other model). This gets us back at the starting point of our Diamonds “quick launch” app.
- Python
- R
1

qc = QueryChat(diamonds, "diamonds", client="anthropic/claude-sonnet-4-5")
Alternatively, you can set the client in options the QUERYCHAT_CLIENT environment variable.
1 2 3 4 5

qc <- QueryChat$new( diamonds, "diamonds", client = "claude/claude-sonnet-4-5" )
Alternatively, you can set the client in options with options(querychat.client = "claude/claude-sonnet-4-5").
id

This is an optional argument, and if it’s not given it’s derived from the table_name. When to use it? If you want to work with multiple QueryChat instances , for example.

greeting

A nice greeting message to display to your users. It’s the first thing your users see, so you better make it good! If not provided, one is generated at the start. While this one looks fine on first sight, it’s rather slow and wasteful (it costs extra tokens because it’s generated every single time). Also, because it’s generated on the fly, it’s far from consistent. Earlier, when we ran the “quick launch” app, you already might have noticed that it generated a warning message:

Python
R

1

Warning: No greeting provided; the LLM will be invoked at conversation start to generate one. For faster startup, lower cost, and determinism, please save a greeting and pass it to init(). You can also use `querychat.greeting()` to help generate a greeting.

1
2
3
4


Warning message:
No greeting provided; the LLM will be invoked at conversation start to generate one.
• For faster startup, lower cost, and determinism, please save a greeting and pass it to QueryChat$new().
ℹ You can generate a greeting with $generate_greeting(). 

So yes, we need a greeting! You can add your own greeting by providing a string in Markdown format.

Some inspiration on what you can put in there: basic instructions, suggestions for filtering, sorting or analysing the data, addressing data privacy concerns, or letting people know where they can get support if something goes wrong.

And if you don’t feel like writing your own greeting, or if you feel uninspired, you can let querychat handle it! Simply use generate_greeting():

Python
R

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


qc = QueryChat(diamonds, "diamonds", client="anthropic/claude-sonnet-4-5")

# Generate a greeting with help from the LLM
greeting_text = qc.generate_greeting()

# Save it
with open("diamonds_greeting.md", "w") as f:
    f.write(greeting_text)

# Then use the saved greeting in your app
qc = QueryChat(diamonds, "diamonds", client="anthropic/claude-sonnet-4-5", greeting=Path("diamonds_greeting.md"))

Which give us this nice greeting:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


# Welcome! 👋

I'm here to help you explore and understand your diamonds dataset. 
I can filter and sort the data, answer questions with SQL queries, 
and help you discover insights.

Here are some ideas to get started:

## Explore the data
* Show me the most expensive diamonds
* What's the average price of diamonds by cut quality?
* How many diamonds are in each clarity category?

## Filter and analyze
* Show only ideal cut diamonds over 2 carats
* Filter to diamonds with the best color grades (D, E, F)
* Which cut has the highest average price per carat?

What would you like to explore first?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


qc <- QueryChat$new(
  diamonds,
  "diamonds",
  client = "claude/claude-sonnet-4-5"
)

# Generate a greeting with help from the LLM
greeting_text <- qc$generate_greeting()

# Save it
writeLines(greeting_text, "diamonds_greeting.md")

# Then use the saved greeting in your app
qc <- QueryChat$new(
  diamonds,
  "diamonds",
  client = "claude/claude-sonnet-4-5",
  greeting = "diamonds_greeting.md"
)

Which give us this nice greeting:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


# Welcome to the Diamond Dashboard! 💎

I'm here to help you explore and analyze this dataset of diamond characteristics and prices. 
I can filter and sort the data, answer questions, and help you discover interesting patterns.

Here are some ideas to get started:

**Explore the Data**
  * What's the average price of diamonds in this dataset?
  * How many diamonds are there in each clarity category?
  * Which diamond has the highest price?

**Filter and Sort**
  * Show me only Ideal cut diamonds
  * Filter to diamonds over 2 carats and sort by price
  * Show me the most expensive diamonds with VS1 clarity

What would you like to explore?

You can see that the generated greeting contains a span HTML tag: …. If you make your own greeting, you can use this tag to automatically populate the chatbox when it’s being clicked.

data_description

querychat automatically helps the LLM by providing things like the column names and datatypes of your data (the schema information), but results can be even more accurate when you provide additional context in the data description. There’s no specific format needed, and you can add whatever information you like. To give some inspiration, this is what we could say about the diamonds dataset:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


# Diamonds Dataset Description

A structured dataset describing physical and quality attributes of individual diamonds,
commonly used to model or predict price.

## Fields

- carat (float) — Diamond weight  
- cut (category) — Cut quality: Fair, Good, Very Good, Premium, Ideal  
- color (category) — Color grade from D (best) to J (worst)  
- clarity (category) — Clarity grades: I1, SI2, SI1, VS2, VS1, VVS2, VVS1, IF  
- depth (float) — Total depth percentage  
- table (float) — Table width percentage  
- price (int) — Price in USD  
- x (float) — Length in mm  
- y (float) — Width in mm  
- z (float) — Depth in mm

We can save this in a Markdown file and pass it on to querychat:

Python
R

1
2
3
4
5
6
7


qc = QueryChat(
    diamonds,
    "diamonds",
    client="anthropic/claude-sonnet-4-5",
    greeting=Path("diamonds_greeting.md"),
    data_description=Path("diamonds_data_description.md")
)

1
2
3
4
5
6
7


qc <- QueryChat$new(
  diamonds,
  "diamonds",
  client = "claude/claude-sonnet-4-5",
  greeting = "diamonds_greeting.md",
  data_description = "diamonds_data_description.md"
)

extra_instructions

For further tweaking the LLMs behaviour you can use extra_instructions. You can go nuts here: make it talk like a pirate, use an emoji in every sentence, or use an annoying amount of diamond-related phrases. You can also use this section for more practical guidance like notes on preferred spelling, tone, or handling of sensitive terms. For example:

1
2
3
4
5


- Assume the user doesn't know much about diamonds: 
  keep explanations simple and accessible.
- When describing diamond attributes, default to plain English. 
  If a term is highly technical, include a short clarification.
- Maintain consistent spelling in British English.

Python
R

1
2
3
4
5
6
7
8


qc = QueryChat(
    diamonds,
    "diamonds",
    client="anthropic/claude-sonnet-4-5",
    greeting=Path("diamonds_greeting.md"),
    data_description=Path("diamonds_data_description.md"),
    extra_instructions=Path("diamonds_extra_instructions.md")
)

1
2
3
4
5
6
7
8


qc <- QueryChat$new(
  diamonds,
  "diamonds",
  client = "claude/claude-sonnet-4-5",
  greeting = "diamonds_greeting.md",
  data_description = "diamonds_data_description.md",
  extra_instructions = "diamonds_extra_instructions.md"
)

categorical_threshold

This threshold applies to text columns, and sets the maximum number of unique values to consider it as a categorical variable. The default is 20.
prompt_template

The prompt_template is a more advanced parameter to provide a custom prompt template. If you don’t provide it, querychat will use the built-in prompt, which we’ll inspect a little bit closer later.

Besides arguments, you can also call methods on the QueryChat object. One of them is cleanup(), which releases any resources (e.g. database connections) associated with the data source. You should call this when you are done using the QueryChat object to avoid resource leaks:

Python
R

1

qc.cleanup()

1

qc$cleanup()

That’s… A lot! And all you need to chat safely with your data. As you’ve seen in our earlier examples, you don’t need a lot to get started (data_source and table_name are enough, and in R you can even omit the table_name). But knowing the possibilities makes it easier to customise querychat to your liking.

Beyond chat: bespoke interfaces

Now you know everything there is to know about the QueryChat object. You know how to add a greeting, additional context, and your favourite LLM. However, it’s time to dream bigger and time to get building! Because chatting with your data safely is one thing, but if you truly want to amaze your users you can build an entire dashboard around it. Plots, maps, tables, and value boxes that all update based on the user’s questions. Your own bespoke interface. Before we dive into that, let’s first take a step back and see if we can reconstruct the “quick launch” app.

You need two things if you want to build a Shiny app with querychat:

The UI component (the chat window)
A server method that deals with the results

For the UI component, there are two choices: sidebar() or ui(). The difference? ui creates a basic chat interface, while sidebar wraps the chat interface in a (bslib, for the R lovers) sidebar component designed to be used as the sidebar argument to page_sidebar.

If we want to do something with the results that get returned by querychat, we need to make use of the server() method. The server method returns:

sql: a reactive that returns the current SQL query. And, if you want to run your own queries, you can also call the $sql() method on the QueryChat object to run queries.
title: a reactive that returns the current title.
df: a reactive that returns the data frame, filtered and sorted by the current SQL query.

Let’s take a look at a minimal example that rebuilds the “quick launch” app:

Python
R

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65


from shiny import App, render, ui
from seaborn import load_dataset
from querychat import QueryChat
from pathlib import Path

# ===============================
# Setup
# ===============================
# 1. Initialize QueryChat with custom files
diamonds = load_dataset("diamonds")
diamonds_greeting = Path(__file__).parent / "diamonds_greeting.md"
diamonds_data_description = Path(__file__).parent / "diamonds_data_description.md"
diamonds_extra_instructions = Path(__file__).parent / "diamonds_extra_instructions.md"

qc = QueryChat(
    diamonds,
    "diamonds",
    client="anthropic/claude-sonnet-4-5",
    greeting=diamonds_greeting,
    data_description=diamonds_data_description,
    extra_instructions=diamonds_extra_instructions,
)

# ===============================
# UI
# ===============================
app_ui = ui.page_sidebar(
    # 2. QueryChat sidebar UI component
    qc.sidebar(),
    ui.card(
        ui.card_header("SQL Query"),
        ui.output_text_verbatim("sql_output"),
        fill=False,
    ),
    ui.card(
        ui.card_header(ui.output_text("title")),
        ui.output_data_frame("data_table"),
        fill=True,
    ),
    fillable=True,
    theme=ui.Theme.from_brand(__file__),
)

# ===============================
# Server
# ===============================
def server(input, output, session):
    # 3. QueryChat server component
    vals = qc.server()

    # 4. Use the filtered/sorted data frame reactively
    @render.data_frame
    def data_table():
        return vals.df()

    @render.text
    def title():
        return vals.title() or "Diamonds"

    # 5. Display the generated SQL query
    @render.text
    def sql_output():
        return vals.sql() or "SELECT * FROM diamonds;"

app = App(app_ui, server)

To keep things simple, we opted for a simple verbatim text output, but we also could’ve chosen for this combination, which is from the shinychat package:

1

ui.output_ui("sql_output")

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


@render.ui
def sql_output():
    sql_value = vals.sql() or f"SELECT * FROM {table_name}"
    sql_code = f"```sql\n{sql_value}\n```"
    return output_markdown_stream(
        "sql_code",
        content=sql_code,
        auto_scroll=False,
        width="100%",
    )

This actually happens in the source code for the quick launch app. It would give us the nice “copy to clipboard” feature and nice formatting. Another alternative would be the native markdown stream component in Shiny .

brand.yml

If you want to make use of brand.yml, you need to add a theme argument: theme=ui.Theme.from_brand(**file**). Make sure you have installed the latest version of shiny with the theme extra! You can simply add it with: uv add "shiny[theme]" (if using uv), or pip install "shiny[theme]"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67


library(shiny)
library(bslib)
library(DT)
library(querychat)
library(ellmer)
library(ggplot2)

# ===============================
# Setup
# ===============================
# 1. Initialize QueryChat with custom files
qc <- QueryChat$new(
  diamonds,
  "diamonds",
  client = "claude/claude-sonnet-4-5",
  greeting = "diamonds_greeting.md",
  data_description = "diamonds_data_description.md",
  extra_instructions = "diamonds_extra_instructions.md"
)

# ===============================
# UI
# ===============================
ui <- page_sidebar(
  title = "Diamonds Explorer",
  # 2. QueryChat sidebar UI component
  sidebar = qc$sidebar(),
  card(
    card_header("SQL Query"),
    verbatimTextOutput("sql_query")
  ),
  card(
    card_header(textOutput("title")),
    DT::DTOutput("data_table")
  )
)

# ===============================
# Server
# ===============================
server <- function(input, output, session) {
  # 3. QueryChat server component
  vals <- qc$server()

  # 3. Display generated SQL query
  output$sql_query <- renderText({
    if (is.null(vals$sql())) {
      return("SELECT * FROM diamonds;")
    }
    vals$sql()
  })

  # 4. Display data table based on user query
  output$data_table <- DT::renderDT({
    vals$df()
  })

  # 5. Dynamic title based on user query
  output$title <- renderText({
    if (is.null(vals$title())) {
      return("Diamonds Data")
    }
    vals$title()
  })
}

shinyApp(ui, server)

Looks pretty similar to the quick launch app, right?! So that’s how it was build. Note that there a few aesthetic differences though. The quick launch app has a few extra sparks here and there, and our app makes use of custom theming with brand.yml.

So far in our diamonds adventure we have only looked at a simple table, but we can extent this idea much further and build an entire dashboard around it: value boxes, graphs, tables, maps, you name it! This is also what sidebot does, and this template is available to get you started quickly. A nice touch is the inclusion of the ✨ icon, which sends a screenshot of the visuals to the LLM for an explanation. How cool is that!

Adding querychat to your existing Shiny app

The idea of sidebot is certainly interesting: why build a dashboard with all kind of filters when you can just add a chat window with access to a smart LLM. You ask it questions, querychat returns some SQL and reactive filtered data, and you make sure you update the entire dashboard. Unlimited filter possibilities. And it doesn’t have to be complicated to achieve that.

To demonstrate how easy it is, we are going to use an existing dashboard (SheScores), that currently has a number of filters in it: a slider for the year(s), a dropdown for the continent where the matches took place, the tournaments that took place on those continents, and a switch that filters the data to include only data with known scorers, or not.

So what does SheScores look like behind the scenes? We’re not going into the nitty gritty details of the SheScores dashboard, and we don’t have to if we want to add querychat to it. The most important bit of logic is stored in a reactive that contains the filtered data. It reacts to changes in any of the inputs (year, continent, tournament, scorer only or not).

The reactive, filtered_data(), forms the basis for all the elements in the dashboard: the value boxes, the map, the graph, and the table.

Python
R

Tip

See GitHub for the full source code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86


# ===============================
# Setup
# ===============================
results_with_scorers = pd.read_csv(
    Path(__file__).parent.parent / "data/results_with_scorers.csv"
)

results_with_scorers["date"] = pd.to_datetime(results_with_scorers["date"])

results_with_scorers = results_with_scorers[
    (results_with_scorers["tournament"] != "Friendly")
    & (results_with_scorers["date"] >= "2000-01-01")
]

# ===============================
# UI
# ===============================
app_ui = ui.page_sidebar(
    ui.sidebar(
        ui.h4("Filters"),
        ui.input_slider(
            "year_filter",
            "Select year range:",
            min=int(results_with_scorers["date"].dt.year.min()),
            max=int(results_with_scorers["date"].dt.year.max()),
            value=[
                int(results_with_scorers["date"].dt.year.min()),
                int(results_with_scorers["date"].dt.year.max()),
            ],
            sep="",
        ),
        ui.input_selectize(
            "continent_filter",
            "Select continents:",
            choices=sorted(
                results_with_scorers["continent"].dropna().unique().tolist()
            ),
            selected="Europe",
            multiple=True,
        ),
        ui.input_select(
            "tournament_filter",
            "Select tournaments:",
            choices=[],
            selected=[],
            multiple=True,
        ),
        ui.input_switch(
            "scorer_only", "Show matches with scorer data only", value=False
        ),
        width="30%",
    ),
    # Other UI content
    # ...
    title="She Scores ⚽️: Women's International Soccer Matches",
    fillable=False,
    theme=ui.Theme.from_brand(__file__),
)

# ===============================
# Server
# ===============================
def server(input, output, session):
    # Reactive filtered data based on inputs
    @reactive.calc
    def filtered_data():
        req(len(input.continent_filter()) > 0)
        req(len(input.tournament_filter()) > 0)

        data = results_with_scorers.copy()
        data = data[
            (data["date"].dt.year >= input.year_filter()[0])
            & (data["date"].dt.year <= input.year_filter()[1])
            & (data["continent"].isin(input.continent_filter()))
            & (data["tournament"].isin(input.tournament_filter()))
        ]

        if input.scorer_only():
            data = data[data["scorer"].notna()]

        return data
    
    # Other server logic
    # ...
    
app = App(app_ui, server)

Tip

Check out the full source code on GitHub .

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98


# ===============================
# Setup
# ===============================
results_with_scorers <- read.csv("../data/results_with_scorers.csv") |>
  filter(tournament != "Friendly", date >= "2000-01-01")

# Other setup
# ...

# ===============================
# UI
# ===============================
ui <- page_sidebar(
  fillable = FALSE,
  title = "She Scores ⚽️: Women's International Soccer Matches",
  sidebar = sidebar(
    title = "Filters",
    width = "30%",
    # Year filter
    sliderInput(
      inputId = "year_filter",
      label = "Select year range:",
      min = year(min(as.Date(results_with_scorers$date))),
      max = year(max(as.Date(results_with_scorers$date))),
      value = c(
        year(min(as.Date(results_with_scorers$date))),
        year(max(as.Date(results_with_scorers$date)))
      ),
      sep = ""
    ),
    # Continent filter (dropdown)
    pickerInput(
      inputId = "continent_filter",
      label = "Select continents:",
      choices = sort(unique(results_with_scorers$continent)),
      selected = sort(unique(results_with_scorers$continent)),
      options = pickerOptions(
        actionsBox = TRUE,
        selectedTextFormat = "count > 1",
        countSelectedText = "{0} continents selected"
      ),
      multiple = TRUE
    ),
    # Tournament filter (dropdown)
    pickerInput(
      inputId = "tournament_filter",
      label = "Select tournaments:",
      choices = NULL,
      selected = NULL,
      options = pickerOptions(
        actionsBox = TRUE,
        liveSearch = TRUE,
        liveSearchPlaceholder = "Search for a tournament",
        selectedTextFormat = "count > 1",
        countSelectedText = "{0} tournaments selected"
      ),
      multiple = TRUE
    ),
    # Switch to show data with scorers only
    input_switch(
      id = "scorer_only",
      label = "Show matches with scorer data only",
      value = FALSE
    ),
  )
  # Other UI content
  # ...
)

# ===============================
# Server
# ===============================
server <- function(input, output, session) {
  # Reactive filtered data based on inputs
  filtered_data <- reactive({
    req(length(input$continent_filter) > 0)
    req(length(input$tournament_filter) > 0)

    results_with_scorers |>
      mutate(date = as.Date(date)) |>
      filter(
        year(date) >= input$year_filter[1],
        year(date) <= input$year_filter[2],
        continent %in% input$continent_filter,
        tournament %in% input$tournament_filter,
        if (input$scorer_only) {
          !is.na(scorer)
        } else {
          TRUE
        }
      )
  })

  # Other server logic
  # ...
}

shinyApp(ui, server)

Now we want to get rid of all those filters. We want a chat window instead. What do we need to change in order to use querychat? Spoiler alert: not much.

Of course we need to initialise our QueryChat object. And since we’re not talking about diamonds, we need to make sure to provide a proper soccer-themed greeting, a data description, and extra instructions:

shescores_greeting.md:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


# Welcome to SheScores! ⚽

I'm here to help you explore international women's soccer match data. 
You can ask me to filter and sort the dashboard, answer questions about the data, 
or provide insights about teams, players, tournaments, and more.

## Here are some ideas to get started:

**Explore match data:**
  * Show me the highest-scoring matches in World Cup tournaments
  * Which teams have played the most matches against each other?
  * Filter to matches from the 2025 UEFA Euro
  * Which team has the best win rate in the Canada vs United States rivalry?
  
**Analyze player performance:**
  * Who are the top scorers in World Cup history?
  * Which players have scored the most penalty goals?
  * Which matches had the most own goals?

What would you like to explore?

shescores_data_description.md:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


# Dataset description
This dataset contains international women’s football match results. 
It includes match metadata (date, location, teams), outcomes (scores), 
plus optional event-level information such as individual scorers. 
Not all friendly matches are represented; 
major tournaments are mostly complete.

# Column descriptions
- date (string, YYYY-MM-DD): The calendar date on which the match was played.
- home_team (string): Name of the home team.
- date (string, YYYY-MM-DD): The calendar date on which the match was played.
- home_team (string): Name of the home team.
- away_team (string): Name of the away team.
- home_score (integer): Goals scored by the home team at full time (extra time included, - penalty shoot-outs excluded).
- away_score (integer): Goals scored by the away team at full time (extra time included, - penalty shoot-outs excluded).
- tournament (string): Name of the competition or event.
- city (string): City or administrative area where the match was played.
- country (string): Country where the match was played.
- neutral (boolean): Indicates whether the match took place at a neutral venue.
- team (string, optional): Team associated with a recorded scoring event.
- scorer (string, optional): Player who scored the goal.
- minute (integer, optional): Match minute in which the goal occurred.
- own_goal (boolean, optional): Indicates whether the goal was an own goal.
- penalty (boolean, optional): Indicates whether the goal was scored from a penalty kick.
- country_flag_home (string): Emoji or symbol representing the home country.
- country_flag_away (string): Emoji or symbol representing the away country.
- continent (string): Continent associated with the home country.
- country_code (string): Country code associated with the home team (e.g., ISO-like).
- latitude (float): Latitude of the match location.
- longitude (float): Longitude of the match location.
- match_id (string): Unique identifier for the match, typically based on date and team names.

shescores_extra_instrucions.md:

1
2
3
4
5
6


- Maintain consistent spelling in British English.
- Don't add any extra columns to the dataset. You may use them internally 
  for calculations, but the final output should only include the original 
  columns with the original column names.
- Soccer terminology should be used throughout the analysis 
  (e.g., "goal" instead of "point").

Now, adding querychat into the mix is as simple as replacing our inputs in the sidebar with the querychat sidebar component (sidebar()), and our reactive with the results of server().

Python
R

Tip

See GitHub for the full source code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54


# ===============================
# Setup
# ===============================
results_with_scorers = pd.read_csv(
    Path(__file__).parent.parent / "data/results_with_scorers.csv"
)

results_with_scorers["date"] = pd.to_datetime(results_with_scorers["date"])

results_with_scorers = results_with_scorers[
    (results_with_scorers["tournament"] != "Friendly")
    & (results_with_scorers["date"] >= "2000-01-01")
]

# 1. Initialize QueryChat with custom files
shescores_greeting = Path(__file__).parent / "shescores_greeting.md"
shescores_data_description = Path(__file__).parent / "shescores_data_description.md"
shescores_extra_instructions = Path(__file__).parent / "shescores_extra_instructions.md"

qc = QueryChat(
    results_with_scorers,
    "results_with_scorers",
    client="anthropic/claude-sonnet-4-5",
    greeting=shescores_greeting,
    data_description=shescores_data_description,
    extra_instructions=shescores_extra_instructions,
)

# Other setup
# ...

# ===============================
# UI
# ===============================
app_ui = ui.page_sidebar(
    qc.sidebar(),
    # Other UI components
    # ...
    title="She Scores ⚽️: Women's International Soccer Matches",
    fillable=False,
    theme=ui.Theme.from_brand(__file__),
)

# ===============================
# Server
# ===============================
def server(input, output, session):
    # Reactive filtered data based on query
    filtered_data = qc.server()
    
    # Other server logic
    # ...
    
app = App(app_ui, server)

Tip

Check out the full code on GitHub .

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43


# ===============================
# Setup
# ===============================
results_with_scorers <- read.csv("../data/results_with_scorers.csv") |>
  filter(tournament != "Friendly", date >= "2000-01-01")

# 1. Initialize QueryChat with custom files
qc <- QueryChat$new(
  results_with_scorers,
  "results_with_scorers",
  client = "claude/claude-sonnet-4-5",
  greeting = "shescores_greeting.md",
  data_description = "shescores_data_description.md",
  extra_instructions = "shescores_extra_instructions.md"
)

# Other setup
# ...

# ===============================
# UI
# ===============================
ui <- page_sidebar(
  fillable = FALSE,
  title = "She Scores ⚽️: Women's International Soccer Matches",
  sidebar = qc$sidebar(),
  # Other UI components
  # ...
)

# ===============================
# Server
# ===============================
server <- function(input, output, session) {
  # Reactive filtered data based on query
  filtered_data <- qc$server()

  # Other server logic
  # ...
  
}

shinyApp(ui, server)

It results in a lot less code and logic too. Win-win. Thanks querychat !

Note

While we don’t have a reset button in the app, querychat knows very well what to do when you ask it to reset the dashboard. In this case, it will display the unfiltered data, just like we started when we launched the app.

Database options

So far we’ve only worked with simple datasets: the diamonds dataset that ships with a package, and our soccer data loaded from a .csv. But here’s how it works under the hood: even in those examples, you weren’t really querying a data frame directly. querychat hands everything off to DuckDB, which becomes the engine that executes all generated SQL. And DuckDB does so quickly and efficiently. Your data frame or .csv is effectively registered inside DuckDB, and every answer comes from real SQL running on that engine.

But what if you don’t want to work with in-memory tables at all? What if you already have a database you want to query directly? Maybe a DuckDB file, a SQLite database, Postgres, or even BigQuery? That’s exactly what the data_source argument is for. Earlier we used it with plain data frames, but it also accepts database connections. In Python, that means any SQLAlchemy-supported database ; in R, anything that DBI can handle. querychat will inspect the schema of whatever you connect, and from that moment on the workflow is identical as before, only now you’re interacting with your own database.

Let’s take a look at how to set up querychat with another backend (SQLite) using the data_source argument.

Python
R

For demonstration purposes, we’ll create a SQLite database from the SheScores data (results_with_scorers.csv).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


from pathlib import Path
from sqlalchemy import create_engine

# From results_with_scorers.csv, create a SQLite database named shescores.db
df_path = Path(__file__).parent.parent / "data/results_with_scorers.csv"

df = pd.read_csv(df_path)

# Create the SQLite database and store the DataFrame in it
# Save database in top-level /data directory
df.to_sql(
    "results_with_scorers",
    con=create_engine(
        "sqlite:///" + str(Path(__file__).parent.parent / "data/shescores.db")
    ),
    if_exists="replace",
    index=False,
)

We can then use this database in our QueryChat instance like so:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


from pathlib import Path
from sqlalchemy import create_engine
from querychat import QueryChat
from dotenv import load_dotenv

load_dotenv()

# Custom files for SheScores
shescores_greeting = Path(__file__).parent / "shescores_greeting.md"
shescores_data_description = Path(__file__).parent / "shescores_data_description.md"
shescores_extra_instructions = Path(__file__).parent / "shescores_extra_instructions.md"

# Now create a QueryChat instance to interact with the database
db_path = Path(__file__).parent.parent / "data/shescores.db"
engine = create_engine(f"sqlite:///{db_path}")

qc = QueryChat(
    engine,
    "results_with_scorers",
    client="anthropic/claude-sonnet-4-5",
    greeting=shescores_greeting,
    data_description=shescores_data_description,
    extra_instructions=shescores_extra_instructions,
)

app = qc.app()

You can also create a DuckDB database from a CSV file or a pandas DataFrame, which is definitely nice for larger datasets. For more examples you can check out the package documentation on data sources .

Even if you have a database that isn’t supported by SQLAlchemy or isn’t suited for DuckDB, you can still let querychat access it. In that case, you need to implement the DataSource interface/protocol.

For demonstration purposes, we’ll create a SQLite database from the SheScores data (results_with_scorers.csv). To create a new SQLite database, you simply supply the filename to dbConnect() . And with dbWriteTable(), you can easily copy an R dataframe into that newly generated SQLite database:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


library(DBI)

# From results_with_scorers.csv, create a SQLite database named shescores.db
results_with_scorers <- read.csv("data/results_with_scorers.csv")

# Create a connection to a new SQLite database
# Save database in top-level /data directory
conn <- dbConnect(RSQLite::SQLite(), "data/shescores.db")

# Write the data frame to a table named results_with_scorers
dbWriteTable(
  conn,
  "results_with_scorers",
  results_with_scorers,
  overwrite = TRUE
)

dbDisconnect(conn)

If you have a SQLite database, connecting to it works in the same manner:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


library(querychat)
library(DBI)

# Create a connection to a SQLite database
conn <- dbConnect(RSQLite::SQLite(), "data/shescores.db")

# Write the data frame to a table named results_with_scorers
dbWriteTable(
  conn,
  "results_with_scorers",
  results_with_scorers,
  overwrite = TRUE
)

# Now create a QueryChat instance to interact with the database
qc <- QueryChat$new(
  conn,
  "results_with_scorers",
  client = "claude/claude-sonnet-4-5",
  greeting = "shescores_greeting.md",
  data_description = "shescores_data_description.md",
  extra_instructions = "shescores_extra_instructions.md"
)

qc$app()

Looking for more examples? Check out these database setup examples for querychat .

querychat knows how to deal with databases, and it has some convenient features for it too, especially when things go wrong: it validates whether tables actually exist and handles any issues gracefully (without cryptic error messages).

One thing to keep in mind when you move from in-memory data to real databases, especially inside Shiny apps, is proper connection management. Whenever your app opens a database connection, it also needs to close it. In Python that usually means calling engine.dispose() when the app shuts down. In R you would use dbDisconnect(conn), or rely on a connection pool. SQLAlchemy already provides pooling on the Python side, but in R you’ll want the pool package to handle this in a nice manner.

For the curious: how does querychat know what to do?

You’ve seen what querychat can do, and you know a bit how it works conceptually. But behind all those concepts is of course some real code. So, for the curious amongst us, here’s a little peek into the querychat code!

To talk with an LLM you need a good prompt: prompt design is crucial for a good outcome. A prompt contains context and instructions that an LLM will use to come up with its answer. querychat has a set of instructions for the LLM too, the system prompt, which is stored in a Markdown file (prompt.md).

Python
R

You can check out the prompt.md file here , or you can simply print it out:

1

print(qc.system_prompt)

You can check out the prompt.md file here , or you can simply print it out:

1

print(qc$system_prompt())

So, what’s in this prompt? Let’s highlight a few bits:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


You have access to a {{db_type}} SQL database with the following schema:


{{schema}}


{{#data_description}}
Here is additional information about the data:


{{data_description}}

{{/data_description}}

For security reasons, you may only query this specific table.

1
2
3
4
5


{{#extra_instructions}}
## Additional Instructions

{{extra_instructions}}
{{/extra_instructions}}

The prompt is a Mustache template. It’s a fill-in-the-blanks template: the {name} parts get replaced with real values, and the {#something} ... {{/something}} blocks only appear if that “something” actually exists. When you call QueryChat with corresponding arguments, everything gets filled in.

We talked about tool calling earlier, and there was a little note that said that there’s not just one tool. There are multiple, for different tasks. You can see that back clearly in the prompt, where we instruct the LLM to call a certain tool (e.g. querychat_update_dashboard) when it receives a request:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


You can handle three types of requests:

### 1. Filtering and Sorting Data

...

- Call `querychat_update_dashboard` with the query and a descriptive title

...

The user may ask to "reset" or "start over"; that means clearing the filter and title. Do this by calling querychat_reset_dashboard().

### 2. Answering Questions About Data

...

- Use the `querychat_query` tool to run SQL queries

...

### 3. Providing Suggestions for Next Steps

...

There are three tools in querychat:

querychat_query: used whenever the user asks a question that requires data analysis, aggregation, or calculations.
querychat_update_dashboard: used whenever the user requests filtering, sorting, or data manipulation on the dashboard with questions like “Show me…” or “Which records have…”. Basically any request that involves showing a subset of the data or reordering it.
querychat_reset_dashboard: if the user asks to reset the dashboard

All the tools are written as chatlas or ellmer tools. As a user, you don’t have to worry about this though. The LLM makes sure to use the rights tools, which will make sure the SQL gets executed and the data gets filtered accordingly. But hey, this section was for the curious amongst us!

Safety, control, and confidence

At some point, everyone asks the same question: is this safe? And it’s a fair one. Luckily, querychat is designed entirely around control. The LLM never executes anything itself, never touches your data(base) and never sees raw data. Its only job is to propose read-only SQL.

Remember the moment we asked it to drop a table? It refused. Not because it’s polite, but because it’s instructed to do so. Combine that with an underlying database (the built-in DuckDB temporary database or your own) that only provides read-only access, and your data will always be left untouched.

It’s not a black box either: every generated query can be logged, inspected or audited at any time. In Shiny v1.12.0 this becomes even easier thanks to built in OpenTelemetry support via otel. If you’re curious about what that looks like in practice, you can read more in this article .

The safety, control, and (hopefully) the confidence you’ve gained by now, make it also suitable for enterprise and regulated environments. If you need to use private or managed LLMs, you’re covered: Azure, AWS Bedrock and Google Vertex AI all provide versions of popular models that support tool calling and can work with querychat.

Other querychat apps in the wild

It’s always nice to see what others have done with querychat. So here are few sources of inspiration:

Do you like trail running? This Race Stats dashboard is for you!
Is the American football league more your thing? This Shiny for Python app shows you a lot of stats.
Joe Cheng and Garrick Aden-Buie hosted a workshop at posit::conf(2025) called “Programming with LLMs” that also contains some examples .
And one that we mentioned before: sidebot , a dashboard analysing restaurant tipping, which is a template you can use very easily.

Whether you’re playing with a small example dataset or building something much bigger, querychat can be the companion in your app that you’re users will love. Build a whole dashboard around the chatbot, or add a touch of LLM magic for those extra side questions. With all this knowledge under your belt, you can build it all!

While it seems like there is only one tool call, there’s not. In querychat there are different tools for, surprise, different tasks. For the curious there’s a deep dive into querychat’s source code later in this article. ↩︎

Shiny Talks from posit::conf(2025)

Tue, 02 Dec 2025 00:00:00 +0000

Videos from posit::conf(2025) are now available. To make it easier to explore how people are building with Shiny across R and Python, we’ve put together a curated playlist of the Shiny-focused talks from the conference. These sessions highlight practical apps, new tooling, design patterns, AI workflows, and real-world deployments built with Shiny.

Shiny Talks Playlist:

Talks included in the playlist, broken up into a few categories for easier browsing, are as follows:

Shiny + AI & Intelligent Automation

Speakers	Title	Link
Winston Chang (Shiny Team)	Web applications with Shiny and React (and AI)
Karan Gathani (Shiny Team)	Old Apps, New Tricks: How AI can write Automated Tests for Shiny
Regis A. James	AskRADS: An AI Recommendation Agent for Maximizing Shiny Development

Shiny Architecture, Performance & Data

Speakers	Title	Link
Alex Chisholm	Keeping Data Alive: Persistent Storage Options for Shiny
Melissa Albino Hegeman	Get your ducks in a row… faster Shiny apps with DuckDB
Barret Schloerke (Shiny Team)	Observability at scale: Monitoring Shiny Applications

Shiny UX, UI Design & Accessibility

Speakers	Title	Link
Cameron Race; Sarah Wong-Brown	shinyGovStyle – accessible government design in Shiny
Casey Aguilar-Gervase; Maya Gans	Design of Everyday Shiny Apps
Jeremy Winget, PhD	Death by Dropdown? Engineer Insightful Shiny Apps
Kim Schouten	Modular, layout-as-code approach for customizable Shiny dashboards

Shiny Extensions, Packages & Developer Tooling

Speakers	Title	Link
Eric Nantz	shinystate: Launching collaboration and session state
James Wade	shinyEventLogger — logging events in Shiny apps

Applied Shiny Case Studies

Speakers	Title	Link
Marcus Beck	Shiny for ecological data workflows
Hugo Fitipaldi	Building a Real-Time COVID-19 Surveillance System with Shiny

Shiny Workshops from posit::conf(2025)

Full workshop materials are now available, including slides, exercises, and code for all Shiny workshops.

LLMs + Shiny Workshop

Joe Cheng & Garrick Aden-Buie

LLM Course Materials

Shiny for Python Workshop

Daniel Chen

Shiny for R Materials

Shiny for R Workshop

Colin Rundel

Shiny for Python Materials

Closing

We hope you enjoy this roundup of Shiny talks from posit::conf(2025). The Shiny community continues to explore new patterns—AI-assisted development, real-time analytics, performance tooling, design systems, and more. We hope to see you next year at posit::conf—and maybe even see your Shiny work up on stage.

posit::conf(2025) Quarto talks

Andrew Holz — Mon, 24 Nov 2025 00:00:00 +0000

The posit::conf(2025) session videos are now live! We’ve created a curated playlist highlighting all the talks that showcase Quarto—how it is evolving, how people are using it, and how they’re building on top of it.

Talks included in the playlist, broken up into a few categories for easier browsing, are as follows:

Quarto Extensions & Advanced Features

Speakers	Title	Thumbnail
Carlos Scheidegger (Quarto Team)	What we’re doing to make Quarto fast(er)
Christophe Dervieux (Quarto Team)	Beyond the Basics: Expanding Quarto’s Capabilities
Garrick Aden-Buie	Theming Made Easy: Introducing brand.yml
Gordon Woodhull (Quarto Team)	Brand YML and Dark Mode in Quarto
JooYoung Seo	maidr: Empowering Accessible, Multimodal Data Science with Quarto

Workflow Automation & Reporting

Speakers	Title	Thumbnail
Becca Krouse	Instant Impact: Developing {docorator} to Simplify Document Production
John Paul Helveston	surveydown: A Markdown-Based Platform for Interactive Surveys
Keaton Wilson	Using Quarto to Improve Formatting and Automation

Teaching & Education

Speakers	Title	Thumbnail
Claus Wilke	Teaching data visualization with R entirely in Quarto
Mine Çetinkaya-Rundel (Quarto Team)	Leveraging LLMs for student feedback in introductory data science courses
Ted Laderas	Empowering Learners with WebR, Pyodide, and Quarto

Business, Collaboration & Publishing

Speakers	Title	Thumbnail
Andrew Heiss and Gabe Osterhout	Election Night Reporting Using R & Quarto
Bill Pikounis	Quarto for Business Collaboration and Technical Documentation in Word docx format
Timothy Keyes	Trust, but Verify: Lessons from Deploying LLMs

Tip

Workshop materials now available:
The Quarto team has published full materials from the two workshops at posit::conf 2025: “Branded Websites, Presentations, Dashboards, and PDFs with Quarto” and “Extending Quarto”.
You can access the workshop websites, exercise source code, and full slide decks under a CC BY-SA 4.0 license from the Quarto blog .

We hope you enjoyed this look back at the Quarto sessions from posit::conf(2025). Every year the community brings new ideas, new tools, and new ways of working — and we’d love to see your voice added to the mix. We hope to see you next year, and maybe even see you up on stage sharing your own work at posit::conf!

Tool Calling UI in shinychat

Garrick Aden-Buie — Thu, 20 Nov 2025 00:00:00 +0000

We’re jazzed to announce that shinychat now includes rich UI for tool calls! shinychat makes it easy to build LLM-powered chat interfaces in Shiny apps, and with tool calling UI, your users can see which tools are being executed and their outcomes. This feature is available in shinychat for R (v0.3.0) and shinychat for Python (v0.2.0 or later).

R
Python

1

install.packages("shinychat")

1

pip install shinychat

This release brings tool call displays that work with ellmer (R) and chatlas (Python). When the LLM calls a tool, shinychat automatically displays the request and result in a collapsible card interface.

In this post we’ll cover the new Tool calling UI features, how to set them up in your apps, and ways to customize the display. We’ll also highlight some chat bookmarking support and other improvements in shinychat for R v0.3.0 . As always, you can find the full list of changes in the R release notes and Python release notes .

Tool calling UI

Tool calling lets you extend an LLM’s capabilities by giving it access to functions you define. When you provide a tool to the LLM, you’re telling it “here’s a function you can call if you need it.” The key thing to understand is that the tool runs on your machine (or wherever your Shiny app is running) — the LLM doesn’t directly run the tool itself. Instead, it asks you to run the function and return the result.

Both ellmer and chatlas make it easy to define tools and register them with your chat client¹, and they also handle the back-and-forth of tool calls by receiving requests from the LLM, executing the tool, and sending the results back. This means you can focus on what you do best: writing code to solve problems.

Any problem you can solve with a function can become a tool for an LLM! You can give the LLM access to live data, APIs, databases, or any other resources your app can reach.

btw: A complete toolkit for R

If you’re working in R, btw is a complete toolkit to help LLMs work better with R. Whether you’re copy-pasting to ChatGPT, chatting with an AI assistant in your IDE, or building LLM-powered apps with shinychat, btw makes it easy to give LLMs the context they need.

And, most importantly, btw provides a full suite of tools for gathering context from R sessions, including tools to: read help pages and vignettes, describe data frames, search for packages on CRAN, read web pages, and more.

Learn more at posit-dev.github.io/btw !

When the LLM decides to call a tool, shinychat displays the request and result in the chat interface. Users can see which tools are being invoked, what arguments are being passed, and what data is being returned. The tool display is designed to be customizable, so shinychat developers can customize the appearance and display of tool calls to best serve their users.

Basic tool display

Let’s start by creating a simple weather forecasting tool that fetches a weather data (in the United States) for a given latitude and longitude.

R
Python

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


library(shinychat)
library(ellmer)
library(weathR)

get_weather_forecast <- tool(
  function(lat, lon) {
    point_tomorrow(lat, lon, short = FALSE)
  },
  name = "get_weather_forecast",
  description = "Get the weather forecast for a location.",
  arguments = list(
    lat = type_number("Latitude"),
    lon = type_number("Longitude")
  )
)

# Register the tool with your chat client
chat <- ellmer::chat("openai/gpt-4.1-nano")
chat$register_tool(get_weather_forecast)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


from chatlas import ChatOpenAI
import requests

def get_weather_forecast(lat: float, lon: float) -> dict:
    """Get the weather forecast for a location."""
    lat_lng = f"latitude={lat}&longitude={lon}"
    url = f"https://api.open-meteo.com/v1/forecast?{lat_lng}¤t=temperature_2m,wind_speed_10m"
    response = requests.get(url)
    return response.json()["current"]

# Register the tool with your chat client
chat = ChatOpenAI(model="gpt-4.1-nano")
chat.register_tool(get_weather_forecast)

With this tool registered, when you ask a weather-related question, the LLM might decide to call the get_weather_forecast() tool to get the latest weather.

In a chat conversation in your R console with ellmer, this might look like the following.

1
2
3
4
5
6
7
8
9


chat$chat("Will I need an umbrella for my walk to the T?")
#> ◯ [tool call] get_weather_forecast(lat = 42.3515, lon = -71.0552)
#> ● #> [{"time":"2025-11-20 16:00:00 EST","temp":42,"dewpoint":0,"humidity":67,"p_rain":1,"wi…
#>
#> Based on the weather forecast, there is a chance of rain around 4 to 5 PM,
#> with mostly cloudy to partly sunny skies. It seems there might be some rain
#> during this time, so carrying an umbrella could be a good idea if you plan
#> to go out around that time. Otherwise, the weather looks relatively clear
#> in the evening.

Notice that I didn’t provide many context clues, but the model correctly guessed that I’m walking to the MBTA in Boston, MA and picked the latitude and longitude for Boston’s South Station .

In shinychat, when the LLM calls the tool, shinychat automatically displays the tool request in a collapsed card:

OpenTelemetry error: there is no package called 'otelsdk'

Expanding the card shows the arguments passed to the tool. When the tool completes, shinychat replaces the request with a card containing the result:

If the tool throws an error, the error is captured and the error message is shown to the LLM. Typically this happens when the model makes a mistake in calling the tool and often the error message is instructive.

shinychat updates the card to show the error message:

Setting up streaming

To enable tool UI in your apps, you need to ensure that tool requests and results are streamed to shinychat:

R
Python

You don’t need to do anything if you’re using chat_app() or the chat module via chat_mod_ui() and chat_mod_server(); tool UI is enabled automatically.

If you’re using chat_ui() with chat_append(), set stream = "content" when calling $stream_async():

1
2
3
4
5
6
7
8
9


server <- function(input, output, session) {
  client <- ellmer::chat("openai/gpt-4.1-nano")
  client$register_tool(get_weather_forecast)

  observeEvent(input$chat_user_input, {
    stream <- client$stream_async(input$chat_user_input, stream = "content")
    chat_append("chat", stream)
  })
}

In Python with Shiny Express, use content="all" when calling stream_async():

app.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


from chatlas import ChatOpenAI
from shiny.express import ui
from shinychat.express import Chat

client = ChatOpenAI(model="gpt-4.1-nano")
client.register_tool(get_weather_forecast)

chat = Chat(id="chat")
chat.ui()

@chat.on_user_submit
async def handle_user_input(user_input: str):
    response = await client.stream_async(user_input, content="all")
    await chat.append_message_stream(response)

For Shiny Core mode:

app.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


from chatlas import ChatOpenAI
from shiny import App, ui
from shinychat import Chat

client = ChatOpenAI(model="gpt-4.1-nano")
client.register_tool(get_weather_forecast)

app_ui = ui.page_fluid(
    Chat(id="chat").ui()
)

def server(input, output, session):
    chat = Chat(id="chat")

    @chat.on_user_submit
    async def handle_user_input(user_input: str):
        response = await client.stream_async(user_input, content="all")
        await chat.append_message_stream(response)

app = App(app_ui, server)

Customizing tool title and icon

You can enhance the visual presentation of tool requests and results by adding custom titles and icons to your tools. This helps users quickly identify which tools are being called.

R
Python

Use tool_annotations() to add a title and icon:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


get_weather_forecast <- tool(
  function(lat, lon) {
    point_tomorrow(lat, lon, short = FALSE)
  },
  name = "get_weather_forecast",
  description = "Get the weather forecast for a location.",
  arguments = list(
    lat = type_number("Latitude"),
    lon = type_number("Longitude")
  ),
  annotations = tool_annotations(
    title = "Weather Forecast",
    icon = bsicons::bs_icon("cloud-sun")
  )
)

With chatlas, you can customize the tool display in two ways:

Use the ._display attribute to customize the tool display:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


import faicons

def get_weather_forecast(lat: float, lon: float) -> dict:
    """Get the weather forecast for a location."""
    # ... implementation ...

get_weather_forecast._display = {
    "title": "Weather Forecast",
    "icon": faicons.icon_svg("cloud-sun")
}

This approach sets the title and icon for all calls to this tool, so it’s ideal for predefined tools or tools that are bundled in a Python module or package.

Set the tool annotations at registration time:

1
2
3
4
5
6
7


chat.register_tool(
    get_weather_forecast,
    annotations={
        "title": "Weather Forecast",
        "icon": faicons.icon_svg("cloud-sun")
    }
)

This approach allows you to customize the display for a specific chat client or application without modifying the tool function itself.

Now the tool card shows your custom title and icon:

" show-request value="[ { "time": "2026-04-02 18:00:00 EDT", "temp": 38, "dewpoint": 1.1111, "humidity": 85, "p_rain": 13, "wind_speed": 12, "wind_dir": "NE", "skies": "Patchy Fog", "geometry": { "type": "Point", "coordinates": [-71.0589, 42.3601] } }, { "time": "2026-04-02 19:00:00 EDT", "temp": 38, "dewpoint": 1.1111, "humidity": 85, "p_rain": 12, "wind_speed": 12, "wind_dir": "NE", "skies": "Patchy Fog", "geometry": { "type": "Point", "coordinates": [-71.0589, 42.3601] } }, { "time": "2026-04-02 20:00:00 EDT", "temp": 38, "dewpoint": 1.1111, "humidity": 85, "p_rain": 10, "wind_speed": 10, "wind_dir": "NE", "skies": "Patchy Fog", "geometry": { "type": "Point", "coordinates": [-71.0589, 42.3601] } }, { "time": "2026-04-02 21:00:00 EDT", "temp": 38, "dewpoint": 1.1111, "humidity": 85, "p_rain": 10, "wind_speed": 9, "wind_dir": "NE", "skies": "Patchy Fog", "geometry": { "type": "Point", "coordinates": [-71.0589, 42.3601] } }, { "time": "2026-04-02 22:00:00 EDT", "temp": 38, "dewpoint": 0.5556, "humidity": 82, "p_rain": 11, "wind_speed": 8, "wind_dir": "E", "skies": "Cloudy", "geometry": { "type": "Point", "coordinates": [-71.0589, 42.3601] } }, { "time": "2026-04-02 23:00:00 EDT", "temp": 38, "dewpoint": 0.5556, "humidity": 82, "p_rain": 11, "wind_speed": 7, "wind_dir": "E", "skies": "Cloudy", "geometry": { "type": "Point", "coordinates": [-71.0589, 42.3601] } } ]" value-type="code">

Custom display content

By default, shinychat shows the raw tool result value as a code block. But often you’ll want to present data to users in a more polished format—like a formatted table or a summary.

You can customize the display by returning alternative content:

R
Python

Return a ContentToolResult with extra$display containing alternative content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


get_weather_forecast <- tool(
  function(lat, lon, location_name) {
    forecast_data <- point_tomorrow(lat, lon, short = FALSE)
    forecast_table <- gt::as_raw_html(gt::gt(forecast_data))

    ContentToolResult(
      forecast_data,  # This is what the LLM sees
      extra = list(
        display = list(
          html = forecast_table,  # This is what users see
          title = paste("Weather Forecast for", location_name)
        )
      )
    )
  },
  name = "get_weather_forecast",
  description = "Get the weather forecast for a location.",
  arguments = list(
    lat = type_number("Latitude"),
    lon = type_number("Longitude"),
    location_name = type_string("Name of the location")
  ),
  annotations = tool_annotations(
    title = "Weather Forecast",
    icon = bsicons::bs_icon("cloud-sun")
  )
)

Return a ToolResult with display options:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


from chatlas import ToolResult
import pandas as pd

def get_weather_forecast(lat: float, lon: float, location_name: str):
    """Get the weather forecast for a location."""
    # Get forecast data
    data = fetch_weather_data(lat, lon)

    # Create a DataFrame for the LLM
    forecast_df = pd.DataFrame(data)

    # Create HTML table for users
    forecast_table = forecast_df.to_html(index=False)

    return ToolResult(
        value=forecast_df.to_dict(),  # LLM sees this
        display={
            "html": forecast_table,  # Users see this
            "title": f"Weather Forecast for {location_name}"
        }
    )

The display options support three content types (in order of preference):

html: HTML content from packages like {gt}, {reactable}, or {htmlwidgets} (R), or Pandas/HTML strings (Python)
markdown: Markdown text that’s automatically rendered
text: Plain text without code formatting

Here’s what a formatted table looks like in the tool result:

time	temp	dewpoint	humidity	p_rain	wind_speed	wind_dir	skies	geometry
2026-04-02 18:00:00 EDT	38	1.666667	89	13	14	NE	Patchy Fog	c(-71.0552, 42.3515)
2026-04-02 19:00:00 EDT	38	1.666667	89	12	13	NE	Patchy Fog	c(-71.0552, 42.3515)
2026-04-02 20:00:00 EDT	38	1.666667	89	10	13	NE	Patchy Fog	c(-71.0552, 42.3515)
2026-04-02 21:00:00 EDT	38	1.666667	89	10	13	E	Patchy Fog	c(-71.0552, 42.3515)
2026-04-02 22:00:00 EDT	38	1.111111	85	11	12	E	Patchy Fog	c(-71.0552, 42.3515)
2026-04-02 23:00:00 EDT	38	1.111111	85	11	10	E	Patchy Fog	c(-71.0552, 42.3515)

" value-type="html">

Additional display options

You can control how tool results are presented using additional display options:

show_request = FALSE: Hide the tool call details when they’re obvious from the display
open = TRUE: Expand the result panel by default (useful for rich content like maps or charts)
title and icon: Override the tool’s default title and icon for this specific result

Another helpful feature is to include an _intent argument in your tool definition. When present in the tool arguments, shinychat shows the _intent value in the tool card header, helping users understand why the LLM is calling the tool.

R
Python

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


tool_with_intent <- tool(
  function(`_intent`) {
    runif(1)
  },
  name = "random_number",
  description = "Generate a random number.",
  arguments = list(
    `_intent` = type_string(
      "Explain why you're generating this number"
    )
  )
)

1
2
3
4
5
6
7
8


def random_number(_intent: str) -> float:
    """Generate a random number.

    Args:
        _intent: Explain why you're generating this number
    """
    import random
    return random.random()

Notice that the tool function itself doesn’t actually use the _intent argument, but its presence allows shinychat to give the user additional context about the tool call.

Bookmarking support

When a Shiny app reloads, the app returns to its initial state, unless the URL includes bookmarked state .² Automatically updating the URL to include a bookmark of the chat state is a great way to help users return to their work if they accidentally refresh the page or unexpectedly lose their connection.

Both shinychat for R and Python provide helper functions that make it easy to restore conversations with bookmarks. This means users can refresh the page or share a URL and pick up right where they left off.

R
Python

In R, the chat_restore() function restores the message history from the bookmark when the app starts up and ensures that the chat client state is automatically bookmarked on user input and assistant responses.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


library(shiny)
library(shinychat)

ui <- function(request) {
  page_fillable(
    chat_ui("chat")
  )
}

server <- function(input, output, session) {
  chat_client <- ellmer::chat_openai(model = "gpt-4o-mini")

  # Automatically save chat state on user input and responses
  chat_restore("chat", chat_client)

  observeEvent(input$chat_user_input, {
    stream <- chat_client$stream_async(input$chat_user_input)
    chat_append("chat", stream)
  })
}

# Enable URL-based bookmarking
shinyApp(ui, server, enableBookmarking = "url")

enableBookmarking = "url" stores the chat state in encoded data in the query string of the app’s URL. Because browsers have native limitations on the size of a URL, you should use enableBookmarking = "server" to store state server-side without URL size limitations for chatbots expected to have large conversation histories.

And if you’re using chat_app() for quick prototypes, bookmarking is already enabled automatically.

In Python, the .enable_bookmarking() method handles the where, when, and how of bookmarking chat state.

Express mode

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


from chatlas import ChatOllama
from shiny.express import ui

chat_client = ChatOllama(model="llama3.2")

chat = ui.Chat(id="chat")
chat.ui(messages=["Welcome!"])

chat.enable_bookmarking(
    chat_client,
    bookmark_store="url", # or "server"
    bookmark_on="response", # or None
)

Core mode

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


from chatlas import ChatOllama
from shiny import ui, App

app_ui = ui.page_fixed(
    ui.chat_ui(id="chat", messages=["Welcome!"])
)

def server(input):
    chat_client = ChatOllama(model="llama3.2")
    chat = ui.Chat(id="chat")

    chat.enable_bookmarking(
        chat_client,
        bookmark_on="response", # or None
    )

app = App(app_ui, server, bookmark_store="url")

Configuration options

The .enable_bookmarking() method handles three aspects of bookmarking:

Where (bookmark_store)
- "url": Store the state in the URL.
- "server": Store the state on the server. Consider this over "url" if you want to support a large amount of state, or have other bookmark state that can’t be serialized to JSON.
When (bookmark_on)
- "response": Triggers a bookmark when an "assistant" response is appended.
- None: Don’t trigger a bookmark automatically. This assumes you’ll be triggering bookmarks through other means (e.g., a button).
How is handled automatically by registering the relevant on_bookmark and on_restore callbacks.

When .enable_bookmarking() triggers a bookmark for you, it’ll also update the URL query string to include the bookmark state. This way, when the user unexpectedly loses connection, they can load the current URL to restore the chat state, or go back to the original URL to start over.

Other improvements in shinychat for R

Beyond tool calling UI and bookmarking support, shinychat for R v0.3.0 includes several other enhancements.

Better programmatic control

chat_mod_server() now returns a set of reactive values and functions for controlling the chat interface:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


server <- function(input, output, session) {
  chat <- chat_mod_server("chat", ellmer::chat_openai())

  # React to user input
  observe({
    req(chat$last_input())
    print(paste("User said:", chat$last_input()))
  })

  # React to assistant responses
  observe({
    req(chat$last_turn())
    print("Assistant completed response")
  })

  # Programmatically control the chat
  observeEvent(input$suggest_question, {
    chat$update_user_input(
      value = "What's the weather like today?",
      submit = TRUE  # Automatically submit
    )
  })

  observeEvent(input$reset, {
    chat$clear()  # Clear history and UI
  })
}

The returned list includes:

last_input and last_turn reactives for monitoring chat state
update_user_input() for programmatically setting or submitting user input—great for suggested prompts or guided conversations
append() for adding messages to the chat UI
clear() for resetting the chat, with options to control how the client history is handled
client for direct access to the ellmer chat client

There’s also a standalone update_chat_user_input() function if you’re using chat_ui() directly, which supports updating the placeholder text and moving focus to the input.

Custom assistant icons

You can now customize the icon shown next to assistant messages to better match your application’s branding or to distinguish between different assistants:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


library(bsicons)

# Set a custom icon for a specific response
chat_append(
  "chat",
  "Here's some helpful information!",
  icon = bs_icon("lightbulb")
)

# Or set a default icon for all assistant messages
chat_ui("chat", icon_assistant = bs_icon("robot"))

This is especially useful when building multi-agent applications where different assistants might have different personalities or roles.

Safer external links

External links in chat messages now open in a new tab with a confirmation dialog. This prevents users from accidentally navigating away from the chat session and losing their conversation. This is particularly helpful when LLMs include links in their responses, for example when shinychat is used in combination with Retrieval Augmented Generation via ragnar .

Learn more

The tool calling UI opens up exciting possibilities for building transparent, user-friendly AI applications. Whether you’re fetching data, running calculations, or integrating with external services, users can now see exactly what’s happening.

To dive deeper:

Read the tool calling UI article for comprehensive examples in R
Explore tool calling with ellmer (R) or chatlas (Python)

Acknowledgements

A huge thank you to everyone who contributed to this release with bug reports, feature requests, and code contributions:

@bianchenhao , @cboettig , @chendaniely , @cpsievert , @DavZim , @DeepanshKhurana , @DivadNojnarg , @gadenbuie , @iainwallacebms , @janlimbeck , @jcheng5 , @jimrothstein , @karangattu , @ManuelSpinola , @MohoWu , @nissinbo , @noamanemobidata , @parmsam , @PaulC91 , @rkennedy01 , @schloerke , @selesnow , @simonpcouch , @skaltman , @stefanlinner , @t-kalinowski , @thendrix-trlm , @wch , @wlandau , and @Yousuf28 .

See the ellmer tool calling documentation for R and the chatlas tool calling documentation for Python for more details on defining and registering tools. ↩︎
This can be especially frustrating behavior since hosted apps, by default, will close an idle session after a certain (configurable ) amount of time. ↩︎

Python Open-Source Developer

Max Kuhn — Wed, 12 Nov 2025 00:00:00 +0000

We are hiring a Python open-source developer with a specialization in data analysis and modeling tools. Since deep learning models already have extensive support in Python, our focus is on the analysis and modeling of tabular data. Our primary goal is to develop packages that enhance the existing capabilities of frameworks such as scikit-learn. Note that the current position doesn’t involve building or publishing models created here; this is a pure package developer role.

We think that the tidymodels philosophy can enhance modeling in Python, and we believe that learning more about modeling in Python will provide us with ideas on how to improve tidymodels. To clarify, we are not creating a tidymodels Python package; however, some of its APIs may be beneficial to Python users. Two examples are recipes and broom’s tidy and augment verbs.

There are numerous ways that individuals can contribute to improving the Python ecosystem. We have a lot of ideas, but we also want to know what our new hire believes is most important. A sample of potential projects that have been on our mind:

Grid search for model tuning has a fairly bad reputation for being inefficient. However, space-filling designs (SFDs) are excellent tools for making small tuning grids that methodically cover the entire parameter space. Users would benefit from having an integrated tool that can create grids from their pipelines using optimal SFDs.
There are occasions where we might decline to produce a prediction, perhaps due to our prediction data being extrapolations from the training set. Applicability Domain methods measure extrapolation and can help determine where the model’s predictions are likely to be poor. An API that can take a model object and the training set could generate a score that informs users when their predictions are ~~hallucinated~~ unlikely to be accurate.
Python deserves an original (Python-only) implementation of the popular glmnet model . There are Python libraries that wrap the original Fortran code; however, this approach isn’t easily supported, nor is it able to facilitate the numerous extensions of this particular model (e.g., group-wise penalties , MCP penalties, etc). This would be a substantial and in-depth project.

We have about a dozen other project ideas.

Open-source developers at Posit often have a broader role than just writing and testing code. We are often the folks defining what is deemed important and prioritizing our work. Additionally, we frequently undertake tasks that are typically considered part of developer relations, such as reaching out to the community, creating additional technical content, speaking at conferences, and teaching workshops. If you are a driven and independent developer, you might be interested in this position. You’ll have a lot of agency here and in an environment that encourages quality. Our developers tend to feel a personal stake in our work and want it to be as good as it can be. We’re a completely remote team, flexible in terms of how and when we work, and we do a good job of minimizing administrative overhead for engineers.

To learn more about the position and to apply, visit the Careers page .

posit::conf(2025) Quarto workshop materials

Charlotte Wickham — Mon, 27 Oct 2025 00:00:00 +0000

At posit::conf(2025), we were thrilled to offer two comprehensive Quarto workshops designed to help users at different stages of their journey with Quarto. Whether you were looking to create beautifully branded outputs or extend Quarto’s functionality with custom solutions, these workshops provided hands-on learning experiences with expert instructors.

All materials from both workshops are available online. You can access the full workshop websites, source code, and exercises to learn at your own pace or adapt them for your own teaching; they are both released with a CC BY-SA 4.0 license.

Branded Websites, Presentations, Dashboards, and PDFs with Quarto

Workshop website Source Exercises

Led by Isabella Velásquez , Posit, PBC and Sara Altman , Posit, PBC.

Designed for data scientists, analysts, and content creators, this immersive session will teach you how to craft cohesive reports and presentations while refining your workflow with Quarto’s latest features.

You will learn how to create dynamic websites, professional PDF documents, engaging presentations, and interactive dashboards using Quarto. This workshop highlights Quarto’s powerful theming capabilities, including the new support for brand.yml, which ensures that your work maintains a professional and cohesive style across all formats.

By the end of the session, you’ll be equipped to:

Build and deploy Quarto websites.

Generate professional presentations and PDF reports.

Create interactive dashboards for data visualization and reporting.

Use brand.yml to define and apply consistent theming across all outputs.

Whether you’re looking to enhance your personal projects or streamline organizational outputs, this workshop will equip you with the tools to create polished, professional results.

Extending Quarto

Workshop website Source Exercises

Led by Mine Çetinkaya-Rundel , Posit, PBC + Duke University and Charlotte Wickham Posit, PBC.

In this workshop, we will dive deep into ways of customizing your Quarto outputs with tooling beyond built-in features. This workshop is designed for data scientists, analysts, and technical writers looking to extend Quarto’s capabilities to suit their unique workflows better.

Participants will learn how to create custom extensions, including new formats, templates, and filters, to enhance their document production process. Through hands-on exercises and real-world examples, you’ll gain practical skills in:

Developing and integrating custom formats to support diverse outputs while reducing repetition across projects.

Substituting Quarto’s templates with your own to customize formats beyond the built-in options.

Implementing filters to automate and streamline content transformation.

By the end of the workshop, you will be able to leverage Quarto’s extensibility to create powerful, tailored solutions for your documentation needs. Whether you have just worked on a few Quarto projects or are an everyday user, this workshop will equip you with the tools and knowledge to take your document workflows to the next level.

Quarto Wizard 1.0.0: Democratising Quarto Extension Management

Mickaël CANOUIL, _Ph.D._ — Mon, 20 Oct 2025 00:00:00 +0000

Community Contribution

The Quarto Wizard extension and listing directory website are built and maintained by Mickaël CANOUIL, Ph.D. .

In this post, he explains what it is and how it can help you manage Quarto extensions directly from Positron or Visual Studio Code.

I’m absolutely thrilled to announce Quarto Wizard 1.0.0, a groundbreaking extension for Visual Studio Code and Positron that transforms how you interact with the Quarto ecosystem. If you’ve ever found yourself wrestling with command-line extension management or struggling to discover the perfect template for your project, this tool is about to become your new best friend.

Install it today from the VS Code marketplace or Open VSX Registry :

Via VS Code or Positron Extensions view:
- Search for “Quarto Wizard”.
- Click “Install”.

Via the command line:
- Visual Studio Code
- Positron
Terminal
code --install-extension mcanouil.quarto-wizard
Tip

Be sure to execute the command Shell Command: Install ‘code’ command in PATH from Visual Studio Code’s Command Palette (Cmd-Shift-P (mac), Ctrl-Shift-P (windows), Ctrl-Shift-P (linux)) if you haven’t done so already.
Terminal
positron --install-extension mcanouil.quarto-wizard
Tip

Be sure to execute the command Shell Command: Install ‘positron’ command in PATH from Positron’s Command Palette (Cmd-Shift-P (mac), Ctrl-Shift-P (windows), Ctrl-Shift-P (linux)) if you haven’t done so already.

Quarto has revolutionised scientific and technical publishing by enabling reproducible documents that seamlessly blend code, narrative text, and visualisation. However, one persistent friction point has been managing the rich and ever-growing ecosystem of extensions and templates—until now.

Quarto Wizard: Your GUI for Quarto extensions

I designed Quarto Wizard to address a fundamental challenge I’ve observed in the community: whilst Quarto’s command-line interface is powerful, many users prefer visual interfaces for discovering, installing, and managing extensions.

Seamless IDE integration

Quarto Wizard integrates beautifully with both the VS Code and Positron ecosystems, appearing as a dedicated icon in the Activity Bar alongside your other development tools. This provides instant access to extension management without disrupting your coding flow, whether you’re in Microsoft’s VS Code or Posit’s new Positron IDE.

The solution is multi-modal installation: you can now install extensions through multiple pathways that suit your workflow: from the command line, through the web directory, or via the Quarto Wizard GUI in your IDE.

Open the Command Palette (Cmd-Shift-P (mac), Ctrl-Shift-P (windows), Ctrl-Shift-P (linux)).
Type Quarto Wizard: Install Extensions and select it.
Browse the list of available Quarto extensions.
Select the Quarto extension(s) you want to install.
Answer the prompts to confirm the installation.

Intelligent extension management

The “Recently Installed Extensions” feature helps track your workflow and easily reproduce project setups across different environments. This is invaluable for researchers collaborating across multiple machines or teaching workshops where consistent setups are essential, regardless of whether team members use VS Code or Positron.

What makes this particularly powerful is that Quarto Wizard tracks which extensions were installed through its interface by adding source metadata to the _extensions.yml file, enabling seamless updates and removals.¹ This source tracking transforms extension maintenance from manual archaeology into an effortless workflow. The extension maintains detailed metadata about installed extensions, enabling batch operations and dependency tracking. The Explorer View provides a comprehensive overview of all installed extensions with visual indicators for updates and management options.

Template workflow simplified

Beyond extension management, I’ve designed Quarto Wizard to ease the process of discovering and using document templates. Once you’ve selected a template, Quarto Wizard lets you customise and save the document. The file is not created until you confirm, allowing you to adjust the filename and location.

Open the Command Palette (Cmd-Shift-P (mac), Ctrl-Shift-P (windows), Ctrl-Shift-P (linux)).
Type Quarto Wizard: Use Template and select it.
Browse the list of available Quarto templates.
Select the Quarto template(s) you want to use.
Answer the prompts to confirm the selection.

Powered by a comprehensive extension directory

A curated catalogue of 250+ extensions

At the heart of Quarto Wizard lies the Quarto Extensions directory (m.canouil.dev/quarto-extensions/) , a comprehensive listing I maintain that catalogues extensions from across the entire Quarto ecosystem.

To date, it includes over 250 extensions contributed by the community, covering a vast array of functionalities from citation management to interactive visualisations. This directory powers Quarto Wizard’s discovery features, providing rich metadata about each extension including descriptions, licensing, version tags, and GitHub stars. The directory is continuously updated through GitHub’s API, ensuring you always have access to the latest extensions from the community.

One-click installation from the web

What’s particularly exciting is that you can install extensions or use templates directly from the website itself. Each extension listed at m.canouil.dev/quarto-extensions/ includes multiple installation options: traditional command-line via terminal, or one-click installation through Quarto Wizard in VS Code, Positron, or VSCodium. Simply browse the directory, find the extension you need, and choose your preferred installation method. The website generates the appropriate commands or launches your IDE directly. This flexibility means teams with mixed technical backgrounds can all access the same powerful extensions.

For example, you might want to add the Iconify extension to access over 200,000 open source vector icons in your documents, or the Animate extension to bring your presentations to life with CSS animations. Perhaps the Spotlight extension for Reveal.js catches your eye for creating dramatic Reveal.js presentations that highlight your mouse position. All of these extensions (and hundreds more) are just a click away.

Template discovery made easy

Additionally, the Quarto Extensions directory excels at template discovery and deployment which is enhanced with powerful filtering options: you can sort by recently updated, filter by popularity, browse by categories (i.e., Shortcodes, Filters, Formats, Projects, Reveal.js Plugins), or search for specific functionality. Each extension clearly indicates whether it’s a template with “Use” buttons alongside “Install” options.

Whether you’re crafting an academic paper using journal-specific formats, creating professional invoices with my Invoice extension template, or building stunning presentations with themed templates like my Reveal.js Coeos extension, browsing available templates becomes as simple as scrolling through a curated gallery.

This directory creates a seamless experience: instead of manually searching GitHub repositories or memorising command-line syntax, you can browse hundreds of extensions with detailed information at your fingertips. This transforms extension discovery from a treasure hunt into a curated shopping experience.

Addressing real workflow friction

I designed Quarto Wizard and the extension directory at m.canouil.dev/quarto-extensions to directly tackle several persistent Quarto pain points I’ve encountered:

Discovery challenges: Finding relevant extensions in the growing ecosystem becomes intuitive through the visual browser interface powered by the comprehensive extensions directory.
Command-line intimidation: Users who prefer graphical interfaces no longer need to memorise terminal commands.
Document setup complexity: Template-based document initialisation eliminates manual YAML configuration.
Extension maintenance: Updates, removals, and dependency management become point-and-click operations rather than command-line archaeology.
Source tracking: Quarto Wizard automatically adds source metadata to installed extensions, enabling future updates and proper version management.
Stable installations: Quarto Wizard installs extensions from GitHub releases/tags by default instead of the potentially unstable default branch, ensuring more reliable installations and more replicable environments.

Perfect for diverse use cases

The extension shines across multiple scenarios:

Academic researchers can quickly install citation management tools, bibliography extensions like Multibib or Section Bibliographies , and journal-specific formatting, whether they’re using VS Code or Positron for their analysis work.
Data scientists gain easy access to computational extensions like WebR , visualisation tools, and interactive notebook capabilities. This is particularly powerful in Positron, which is designed specifically for data science workflows.
Technical writers can browse and install extensions for enhanced typography with for example the Highlight Text extension for multi-format text highlighting, code highlighting, and advanced formatting options of their Reveal.js presentations with Reveal.js Editable extension in their preferred IDE.
Workshop instructors can ensure all participants have consistent extension setups through guided installation processes, regardless of whether attendees prefer VS Code or Positron.

Future-ready architecture

I’ve built Quarto Wizard on robust foundations that ensure long-term reliability. The extension integrates with GitHub’s API through the Quarto Extensions directory for real-time metadata, includes attestation verification for security, and maintains full compatibility with both VS Code and Positron environments.

The modular architecture allows for future enhancements whilst maintaining backwards compatibility. As the Quarto ecosystem continues expanding and as Positron evolves alongside VS Code, Quarto Wizard will support new extension types and project management workflows in both environments.

Getting started today

Begin your Quarto Wizard journey by installing the extension from the VS Code marketplace, the Open VSX Registry, or directly through your IDE’s Extensions view. Once installed, the Quarto Wizard icon appears in your Activity Bar, providing immediate access to extension management and project tools.

You have multiple paths to explore Quarto extensions:

Through Quarto Wizard: Click the Quarto Wizard icon in your IDE’s Activity Bar and browse the integrated catalogue.
Via the web directory: Visit m.canouil.dev/quarto-extensions where you can browse all extensions and install them directly from the website. Each extension offers installation buttons for Quarto Wizard, VS Code, Positron, VSCodium, or traditional terminal commands.
Traditional command-line: Use the familiar quarto add commands if you prefer.

Try installing your first extension: perhaps Iconify extension for comprehensive icon support or GitHub for seamless GitHub linking. Whether you click “Install” on the website, use Quarto Wizard’s interface, or type commands in the terminal, the choice is yours. The difference in experience compared to traditional command-line installation is immediately apparent, especially when browsing the visual catalogue with its filtering options, popularity indicators, and rich metadata.

I believe Quarto Wizard represents a significant step forward in making Quarto’s powerful publishing capabilities accessible to users regardless of their comfort level with command-line tools or their choice of IDE. By providing intuitive visual interfaces for complex operations, Quarto Wizard democratises access to the rich Quarto ecosystem whilst maintaining the flexibility and power that makes Quarto exceptional.

The future of reproducible publishing is here, and it’s more accessible than ever in whichever modern development environment you prefer.

Quarto CLI does not natively track installation sources as of version 1.8.24 (quarto-dev/quarto-cli#11468 ). ↩︎

Quarto 1.8

Charlotte Wickham — Mon, 13 Oct 2025 00:00:00 +0000

Quarto 1.8 is available! You can get the current release from the download page .

Quarto 1.8 improves support for light and dark brand colors and logos, brand extensions for sharing brands across Quarto projects, HTML accessibility checks powered by Axe-core, and access to more information about execution context from your code cells. You can read about these improvements and some other highlights below. You can find all the changes in this version in the Release Notes .

Dark and light colors and logos in brand

You can now specify light and dark versions of any colors or logo in a brand specification:

_brand.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


color:
  foreground:
    light: "#333333"
    dark: "#EEEEEE"
  background:
    light: "#EEEEEE"
    dark: "#333333"
logos:
  medium:
    light: logo.png
    dark: logo-white.png         

This works in _brand.yml files as well as brand specified directly in document metadata. You can also present in dark mode by specifying brand-mode: dark in your format: revealjs presentations.

Read more in the updated Guide > Brand :

Brand extensions

Share brand definitions and assets across Quarto projects with a brand extension.

Get started with:

Terminal

quarto create extension brand

Read more in Extensions > Brand , and keep an eye out for other ways to reuse and share your brand in future releases.

Accessibility checks for HTML

You can add accessibility checks using the Axe-core engine to HTML documents (format: html, revealjs and dashboard) with the new axe option.

For example, you can get a summary of violations right in your document preview:

Read about your options in HTML Accessibility Checks

We know accessability is a big concern for many of our users, and more improvements will be coming in future releases.

Accessing execution information

Quarto sets the QUARTO_EXECUTE_INFO environment variable, which allows you to access information about execution context from code cells.

Read the JSON file located at QUARTO_EXECUTE_INFO and access properties such as document-path, format, metadata and more:

R
Python
Julia

1
2
3


library(jsonlite)
execute_info <- read_json(Sys.getenv("QUARTO_EXECUTE_INFO"))
execute_info$`document-path`

1
2
3
4
5
6


import json
import os

with open(os.getenv("QUARTO_EXECUTE_INFO")) as f:
    execute_info = json.load(f)
execute_info["document-path"]

1
2
3
4


using JSON

execute_info = JSON.parsefile(ENV["QUARTO_EXECUTE_INFO"])
execute_info["document-path"]

Other Highlights

Access metadata and variables in filters and shortcodes: Use the new quarto.variables.get() and quarto.metadata.get() APIs.
The default LaTeX engine is now lualatex.

Dependency updates:

mermaidjs updated to 11.6.0.
Bootstrap icons updated to v1.13.1
QuartoNotebookRunner in julia engine updated to 0.17.3

Acknowledgements

We’d like to say a huge thank you to everyone who contributed to this release by opening issues and pull requests:

Aariq , AndreasThinks , ArthurData , Blake-Madden , ColinFay , DCEW , DanStuder , Data-Wise , EllaKaye , EmilHvitfeldt , FrankwaP , GabrielCoffee9 , GeorgRamer , Gewerd-Strauss , GuillaumeDehaene , HarunCelikOtto , HayesJohnD , Joao-O-Santos , MateusMolina , MichaelHatherly , PeteArm , Selbosh , SergeCroise , SrShelo , VisruthSK , Vistales , abhiaagarwal , aborruso , adamblake , adamiturabi , alastairrushworth , albertomercurio , alecloudenback , alex-r-bigelow , allefeld , alyst , andrewheiss , andrewpbray , austin-hoover , batpigandme , bauerj , benkeks , benz0li , bkowshik , blackerby , boshek , brandonmontez , bryce-carson , carschandler , christopherkenny , cl-roberts , cmadland , co1emi11er2 , coatless , cpcloud , daxkellie , dixslyf , dkapitan , econmaett , edavidaja , edvinsyk , ethanwhite , fermarsan , fredguth , fuhrmanator , gadenbuie , georgestagg , ghisvail , ghost , github-actions[bot] , glin , gregswinehart , gwbrck , halleysfifthinc , hansfn , hchulkim , holtzy , htbunn , hturner , hugetim , hutch3232 , iagopinal , ihrke , jameslairdsmith , jdfoote , jeremy9959 , jfy133 , jkrumbiegel , jmgirard , jonpeake , jvcarli , jxpeng98 , kandolfp , kapsner , kathsherratt , kazuyanagimoto , kevinah95 , kippandrew , koldle , lachlansimpson , lbm364dl , leovuong , lostmygithubaccount , lu-kas , lukmanaj , lwjohnst86 , maelle , mahmudstat , masud90 , melaniewalsh , mfisher87 , mipmip , mpr1255 , multimeric , musvaage , mvuorre , nathanj3 , nessan , nichtich , odysseu , ofkoru , olivroy , oyvindbso , pagiraud , parmsam , peter-gy , pm-gusmano , produnis , rabyj , raffaem , randyzwitch , rben01 , rossbowen , rundel , ryanzomorrodi , ryjohnson09 , s2t2 , salim-b , samcarter , serialc , sgelzenleuchter , skriptum , spaette , stragu , sun123zxy , sverrirarnors , tecosaur , temospena , thatchermo , topepo , tylere , winniehell , wklimowicz , yogabonito , youcc , yves-amevoin , yyzeng .

The lightbulb emoji in the listing and social card image for this post comes from OpenMoji– the open-source emoji and icon project. License: CC BY-SA 4.0

Build Your First LLM App with Shiny for Python or R

Veerle Eeftink - Van Leemput — Mon, 15 Sep 2025 00:00:00 +0000

You’ve made it to third part of “The Shiny Side of LLMs” series, where we turn everything we’ve learned into something real and interactive! Our weapon of choice: Shiny, of course.

In the first part, What LLMs Actually Do (and What They Don’t) , we explored what LLMs actually do. We covered how they generate text, what they’re good (and bad) at, and we covered most of the jargon that gets thrown around. Then in part two, Talking to LLMs: From Prompt to Response , we got practical. You learned how to structure prompts, send them to a model via an API using chatlas or ellmer , and handle the responses in your code. Now it’s time to wrap that logic in an interface your users can love and can actually interact with!

In this post, we’ll cover how to:

Structure a basic Shiny app (in both Python and R) with a user interface (UI) and reactive server logic
Connect user input to an LLM call and show the result in the UI
Set up a chat interface
Keep your app responsive while doing lengthy tasks or calculations (yes, a quick guide to async programming!)
Add UI polish like pretty looking value boxes, loading indicators, and error messages
Bonus: deploy your app with a single click to Posit Connect (Cloud) or shinyapps.io

By the end of this part, you’ll have your first real, working, LLM-powered app. And most importantly: the knowledge to build many more.

What are we going to do in this last part?

This part of “The Shiny Side of LLMs” series will build an app called “DeckCheck”: a genius app that gets rid of lengthy unfocused presentations, like a perfect “Presentation Rehearsal Buddy”. The goal: let users upload their Quarto presentation and provide them with feedback on how to make it better.

No time for the walkthrough?

Want to dive straight into the full app? Head over to the end result .

Why Shiny?

We knew pretty early on that “DeckCheck” was going to be an app. Why? Because an app is just easier. People can click buttons, type things, and get results without ever touching code (which is great, because “copy-paste this into your terminal” tends to scare off half the audience).

But ok, why Shiny? You’re reading this on the Shiny blog, so yes, it seems pretty obvious we love it. Still, we have solid reasons:

You can go from idea to something you can actually click in almost no time. Shiny is great for prototyping. You can quickly experiment, gather feedback, and gradually build an app out to a production-ready state.
Shiny is reactive by design, which is a fancy way of saying: it keeps track of what depends on what, and only updates what actually needs to change. You don’t need to worry about the logic, which means you can spend more time building your app instead of dealing with state management.
This reactive engine also makes Shiny efficient: outputs only re-render when the stuff they depend on changes.
If your LLM workflow is in Python, there’s Shiny for Python. If it’s in R, there’s Shiny for R. No language wars or conversations about “X is better than Y”. Just pick whatever you’re comfortable with.
You can even drop in your own HTML, CSS, and JavaScript to really make it your own.

Alright, enough with the sales pitch. Shiny it is!

How Shiny Assistant assists you

Fun fact: this article contains side-by-side examples in both Python and R. To showcase how Shiny Assistant can support you in either language, it was used to generate some of the conversions. That’s a neat way to highlight how an LLM can help you get started with Shiny! The true “Shiny Side of LLMs”. Of course result were not always 100% spot on, but luckily there was still a human in the loop.

Optimising conversations for Shiny

Getting your API key

Remember you need to grab an API key for your chosen LLM provider . You need this key to authenticate. Store this key as an environment variable. For example, to use Claude from Anthropic, ANTHROPIC_API_KEY=yourkey needs to be in .Renviron or .env file.

Going from a script-like workflow to an app requires a different way of thinking. We simply have other expectations from a web app compared to just a regular Python or R script. We want things to be interactive, and ideally we want to have the result instantly. If we click on something, we expect something to happen, fast. Ever encountered a web page that stayed blank for just 5 seconds? How long did that feel? Like 10 minutes? Or didn’t you even stick out the 5 seconds? Yes, you are impatient! You need to see something is happening, and get some visual feedback.

LLMs take your impatience into account, and that’s why most models stream tokens progressively. Instead of making you wait forever for a big wall of text, they start showing you the answer bit by bit, almost like they’re thinking out loud. This streaming feels faster, keeps you engaged, and makes the whole experience way less frustrating. chatlas and ellmer use this streaming capability too and they print the result on the console as soon as words come in. But… a Shiny app doesn’t have a console! So what to do?

Where the chat() method does not return any results until the entire response is received and only prints the streaming results to the console, the stream() method can process the response as it arrives. It’s perfect for something like a Shiny chat window. You simply replace the chat() method with the stream() method, which returns something called a “generator”. A generator is a function that can pause (yield) and resume later, remembering where it left off. That’s handy because it lets your code:

Process text as it arrives (aka, by chunk) instead of waiting for the whole thing.
Pause between chunks without blocking other things.
Keep its place so it can pick up right where it left off when the next chunk arrives.

Python
R

In the chatlas documentation you can read more about streams .

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


from chatlas import ChatAnthropic

chat = ChatAnthropic(
    model="claude-sonnet-4-20250514",
    system_prompt="You are a presentation coach for data scientists. You give constructive, focused, and practical feedback on titles, structure, and storytelling.",
)

# Set model parameters (optional)
chat.set_model_params(
    temperature=0.8,  # default is 1
)

stream = chat.stream(
    """I'm working on a presentation with the title: 'The Shiny Side of LLMs'.
        Please evaluate the clarity, tone, and relevance of this title for the intended audience.
        For context, this is a 10-minute lightning talk at posit::conf(2025).
        The audience is Python and R users who are curious about AI and large language models,
        but not all of them have a deep technical background.
        
        The talk uses Shiny as a way to explore and demo LLMs in practice.
        
        Return your answer as a JSON array of objects, where each object has the following keys:
        
        - 'aspect': one of 'clarity', 'tone', or 'relevance'
        - 'feedback': your concise assessment
        - 'suggestion': an optional improvement if applicable"""
)

for chunk in stream:
    print(chunk)

The object that gets returned by the stream() method is a generator .

In the ellmer documentation you can read more about streaming .

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


library(ellmer)

chat <- chat_anthropic(
  model = "claude-sonnet-4-20250514",
  system_prompt = "You are a presentation coach for data scientists.
  You give constructive, focused, and practical feedback on titles, structure, and storytelling.",
  params = params(
    temperature = 0.8 # default is 1
  )
)

stream <- chat$stream(
  "I'm working on a presentation with the title: 'The Shiny Side of LLMs'.
Please evaluate the clarity, tone, and relevance of this title for the intended audience.
For context, this is a 10-minute lightning talk at posit::conf(2025).
The audience is Python and R users who are curious about AI and large language models,
but not all of them have a deep technical background.

The talk uses Shiny as a way to explore and demo LLMs in practice.

Return your answer as a JSON array of objects, where each object has the following keys:

- 'aspect': one of 'clarity', 'tone', or 'relevance'
- 'feedback': your concise assessment
- 'suggestion': an optional improvement if applicable"
)

coro::loop(
  for (chunk in stream) {
    cat(chunk)
  }
)

The object that gets returned by the stream() method is a coro generator .

Streaming is great for things like chatbots, live transcription, or anything where seeing text appear in real time feels natural. It’s perfect for conversations, or when the answer is long and you don’t want to keep users staring at a blank screen.

In apps like “DeckCheck”, where we’re analysing a Quarto presentation behind the scenes and then showing a finished result, streaming doesn’t really add much. Users expect a clear, polished answer all at once. Not a JSON, but ready-to-go value boxes, graphs and tables. So in this case, a smooth loading indicator or progress bar will probably satisfy impatient users and we can stick to our chat() method. Don’t forget the stream() method though: we’ll use that a little bit later in a small demo chatbot.

Whether we’re streaming or not, users simply have to wait for an LLM response and you can keep them entertained while they’re doing that. But another important thing related to a long waiting time is when your app is used by multiple users at the same time. They might ask things from the model at the same time too, meaning that you’re dealing with concurrent chat sessions. If you would just use chat() or stream(), the Shiny app (or technically speaking: the Python or R session running the Shiny app) will be blocked for other users for the duration of each response. And that’s not cool. The more users are concurrently using your app, the longer the queue gets, and the longer users have to wait. This is synchronous behaviour. We rather deal asynchronously (aka async) with a model’s responses. It means that we can receive responses at the same time, in parallel. To use async chat, we need to call chat_async() / stream_async() instead of chat() / stream(). The _async variants take the same arguments but returns a coroutine object, (aka a placeholder for something that will come) instead of the actual response.

So ideally, asking a question to an LLM would look something like this in our Shiny application:

Python
R

Note for Positron/Jupyter users

Positron and Jupyter already run their own event loop, so asyncio.run(main()) will fail with a runtime error. Instead of wrapping in asyncio.run, you can just do: await main() at the top level.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


import asyncio
from chatlas import ChatAnthropic

chat = ChatAnthropic(
    model="claude-sonnet-4-20250514",
    system_prompt="You are a presentation coach for data scientists. You give constructive, focused, and practical feedback on titles, structure, and storytelling.",
)

async def main():
    response = await chat.chat_async(
        """I'm working on a presentation with the title: 'The Shiny Side of LLMs'.
        Please evaluate the clarity, tone, and relevance of this title for the intended audience.
        For context, this is a 10-minute lightning talk at posit::conf(2025).
        The audience is Python and R users who are curious about AI and large language models,
        but not all of them have a deep technical background.
        
        The talk uses Shiny as a way to explore and demo LLMs in practice.
        
        Return your answer as a JSON array of objects, where each object has the following keys:
        
        - 'aspect': one of 'clarity', 'tone', or 'relevance'
        - 'feedback': your concise assessment
        - 'suggestion': an optional improvement if applicable"""
    )
    print(response)

asyncio.run(main())

Show output

Talking to LLMs: From Prompt to Response

Veerle Eeftink - Van Leemput — Fri, 05 Sep 2025 00:00:00 +0000

Welcome back to “The Shiny Side of LLMs”: a blog series for Python and R users who want to build real, useful LLM-powered apps without getting buried in jargon or deep learning theory.

In Part 1: What LLMs Actually Do (and What They Don’t) , we looked at how large language models generate responses, why they sometimes seem so smart (and other times so confidently wrong), and what kinds of tasks they’re actually good at. Now it’s time to get hands-on!

In this part, we’ll focus on what it means to talk to an LLM. We’ll cover crafting prompts to parsing responses, and everything in between. You’ll learn:

How to call an LLM via chatlas (Python) and ellmer (R)
The basics of prompt design: roles, instructions, system messages, temperature, and more
What goes in and what comes out, and how your own data fits into that loop
How to work with model outputs: parsing, structuring, and getting predictable results
How to give LLMs extra power with tools
Costs involved when talking to LLMs

By the end of this part, you’ll be ready to go from “I asked ChatGPT a thing” to “I can programmatically talk to an LLM by sending custom prompts and doing something useful with the results”.

No time for the walkthrough?

Jump straight to the full workflow .

The app: DeckCheck

In this series we’re working towards a genius app that gets rid of lengthy unfocused presentations, like a perfect “Presentation Rehearsal Buddy”. No app without a snappy name, so hello “DeckCheck”. The goal: let users upload their Quarto presentation and provide them with feedback on how to make it better.

So how would we go about that? To give meaningful feedback on a presentation, we need to ask the LLM the right questions. For example:

Does the presentation have a clear structure?
Are the key messages easy to spot?
Does the tone match the intended audience?
Are the slides too wordy, too vague, or just right?
Is there a good balance between words, visuals, and code?

If you’ve used ChatGPT or another LLM in a chat interface, the process feels straightforward: you ask any of the above questions, and the model replies. But when you’re building an app, you’re not just having a conversation. You want to have some form of control. After all, you’re designing a system. You need to construct prompts carefully, handle responses programmatically, and ensure the whole exchange is stable enough to support user-facing features. In our case, that means feeding slide content into the LLM and asking targeted questions about structure, clarity, and tone. And we’re not just hoping for a helpful paragraph with some witty comments. For our app we need output that’s predictable, parseable, and ready to plug into our app logic. Eventually we want to be able to extract suggestions, metrics, scores: things that we also need to retrieve programmatically.

So in this part, it’s not just about asking questions, it’s about:

How to design prompts that get reliable, structured answers
How to “control” model behaviour (as far as you can, at least)
How to send slide content (or any other data) to an LLM
And how to process the model’s output into something our future app can actually use

Let’s build!

What can you expect from this second part?

This part focusses on programmatically talking to an LLM. We leave the Shiny app building to the third and last part of “The Shiny Side of LLMs” series. The code we end up with in this part serves as a basis for the back-end of our app in this next part.

What do you need?

Ah, is it even a tutorial if there isn’t a “getting started” step? But there’s one obvious thing we need: access to an LLM. To get that, you’ll need to sign up for a (developer) account with a provider like OpenAI or Anthropic. And with that comes something that feels like a huge barrier: setting up payment details. I feel you when you might be getting anxious about an invoice with a lot of zeros, but really, don’t worry too much about that. Almost all platforms offer free trial credits, and you can usually set usage limits or buy fixed prepaid credits (Anthropic makes this especially easy). So that invoice-nightmare won’t come alive easily. Still a little bit worried? Google’s model Gemini offers a generous free tier.

Once you’ve registered, you can get an API key. You need this key to authenticate with the LLM provider. Never, ever hardcode the key directly into your script. You’ll be amazed how many keys are publicly available on GitHub repos. Don’t be that developer. As always with secrets, store it as an environment variable. Just note that the exact name of the key depends on the provider. For example, Anthropic expects ANTHROPIC_API_KEY=yourkey, while OpenAI uses OPENAI_API_KEY=yourkey.

To keep things simple, we’ll go with Anthropic for the examples in this blog series, but feel free to choose any provider you want. Don’t worry about lock-in either: you’ll see that switching providers and models is super easy.

In Python, the recommended approach is to create a .env file in your project folder:

1

ANTHROPIC_API_KEY=yourkey

It’s recommended to use the dotenv package to load the .env file into your environment:

1
2
3


from dotenv import load_dotenv

load_dotenv()

1

ANTHROPIC_API_KEY=yourkey

Just be sure to add .env and .Renviron to your .gitignore file so you don’t accidentally publish your keys.

Installing `chatlas` or `ellmer`

Working with different LLM providers can be a pain, but luckily there is the perfect solution: chatlas for Python and ellmer for R. Hello simple and consistent interface! From Anthropic’s Claude to (Azure) OpenAI and Google Gemini: it’s all possible. And bonus: chatlas and ellmer both have the same interface, making it easy to translate your LLM projects between Python and R!

Python
R

For Python, chatlas is available on PyPI, so you can install it easily with pip:

1

pip install chatlas

Or, if you’re using uv, add it like so:

1

uv add chatlas

Once installed, import it like this:

1

import chatlas

Depending on the LLM provider you will choose, you might need to install additional packages (e.g. anthropic for, surprise, models from Anthropic). For every provider, the relevant function reference page in the chatlas docs will tell you what’s necessary together with credential acquisition tips (e.g. ChatAnthropic ).

You can install ellmer from CRAN using:

1

install.packages("ellmer")

Once installed, load the package as usual:

1

library(ellmer)

Switching between Python and R

Did you know you can very easily switch between Python and R in Positron ? Just select the interpreter you want (you can even choose between Python and R installs), and you’re good to go!

Your first conversation

Time to chat! Whether you’re using Python or R, the workflow is the same. First, you need to create a chat object. You can give it any name you like, but chat seems the most obvious choice here. This chat object is created with a specific function for your chosen provider, where you (optionally) can specify the model. While the model argument isn’t required, it is strongly recommended you do specify it. After all, we’re looking for consistent behaviour in our Shiny app, and a change in default model choice isn’t going to give us that. In our case we will go for claude-sonnet-4-20250514 from Anthropic , which is the latest (reasonably priced) model at the time of writing.

Python
R

1
2
3
4
5
6


from dotenv import load_dotenv
from chatlas import ChatAnthropic

load_dotenv()  # Loads key from the .env file

chat = ChatAnthropic(model="claude-sonnet-4-20250514")

1
2
3
4
5


library(ellmer)

chat <- chat_anthropic(
  model = "claude-sonnet-4-20250514"
)

In ellmer 0.3.0 you can also use chat() using a string like chat("anthropic") or chat("openai/gpt-4.1-nano").

Not sure what models are available? You can use models_*(), e.g. models_anthropic(), to get a list of available models.

API keys and environment variables

There’s no need to explicitly provide your API key to chatlas or ellmer. If you specify it with the correct name in your environment file (.env or .Renviron), chatlas or ellmer will automatically find it. There’s an api_key argument, but generally you won’t use this. For Python, it’s recommended to use the dotenv package to load the .env file into your environment.

You can call the chat method on the chat object. When you use the chat method:

You see the response appear in your console, piece by piece, like it’s being typed out. That’s the streaming part.
The full message is also (invisibly) saved as a character vector, so you can use it in your code later.

For simplicity, let’s start with our very basic first prompt: “I’m working on a presentation with the title: ‘The Shiny Side of LLMs’. What’s your feedback just based on that title?”

Python
R

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


from dotenv import load_dotenv
from chatlas import ChatAnthropic

load_dotenv()  # Loads key from the .env file

chat = ChatAnthropic(model="claude-sonnet-4-20250514")

chat.chat(
  "I'm working on a presentation with the title: 'The Shiny Side of LLMs'. 
  What's your feedback just based on that title?"
)

Show output

mall 0.2.0

Edgar Ruiz — Tue, 19 Aug 2025 00:00:00 +0000

mall uses Large Language Models (LLM) to run Natural Language Processing (NLP) operations against your data. This package is available for both R, and Python. Version 0.2.0 has been released to CRAN and PyPi respectively.

In R, you can install the latest version with:

1

install.packages("mall")

In Python, with:

1

pip install mlverse-mall

This release expands the number of LLM providers you can use with mall. Also, in Python it introduces the option to run the NLP operations over string vectors, and in R, it enables support for ‘parallelized’ requests.

It is also very exciting to announce a brand new cheatsheet for this package. It is available in print (PDF) and HTML format!

More LLM providers

The biggest highlight of this release is the the ability to use external LLM providers such as OpenAI , Gemini and Anthropic . Instead of writing integration for each provider one by one, mall uses specialized integration packages to act as intermediates.

In R, mall uses the ellmer package to integrate with a variety of LLM providers . To access the new feature, first create a chat connection, and then pass that connection to llm_use(). Here is an example of connecting and using OpenAI:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


install.packages("ellmer")

library(mall)
library(ellmer)

chat <- chat_openai()
#> Using model = "gpt-4.1".

llm_use(chat, .cache = "_my_cache")
#> 
#> ── mall session object 
#> Backend: ellmerLLM session: model:gpt-4.1R session: cache_folder:_my_cache

In Python, mall uses chatlas as the integration point with the LLM. chatlas also integrates with several LLM providers . To use, first instantiate a chatlas chat connection class, and then pass that to the Polars data frame via the .llm.use() function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


pip install chatlas

import mall
from chatlas import ChatOpenAI

chat = ChatOpenAI()

data = mall.MallData
reviews = data.reviews

reviews.llm.use(chat)
#> {'backend': 'chatlas', 'chat': 
#> , '_cache': '_mall_cache'}

Connecting mall to external LLM providers introduces a consideration of cost. Most providers charge for the use of their API, so there is a potential that a large table, with long texts, could be an expensive operation.

Parallel requests (R only)

A new feature introduced in ellmer 0.3.0 enables the access to submit multiple prompts in parallel, rather than in sequence. This makes it faster, and potentially cheaper, to process a table. If the provider supports this feature, ellmer is able to leverage it via the parallel_chat() function. Gemini and OpenAI support the feature.

In the new release of mall, the integration with ellmer has been specially written to take advantage of parallel chat. The internals have been re-written to submit the NLP-specific instructions as a system message in order reduce the size of each prompt. Additionally, the cache system has also been re-tooled to support batched requests.

NLP operations without a table

Since its initial version, mall has provided the ability for R users to perform the NLP operations over a string vector, in other words, without needing a table. Starting with the new release, mall also provides this same functionality in its Python version.

mall can process vectors contained in a list object. To use, initialize a new LLMVec class object with either an Ollama model, or a chatlas Chat object, and then access the same NLP functions as the Polars extension.

1
2
3
4
5
6
7


# Initialize a Chat object
from chatlas import ChatOllama
chat = ChatOllama(model = "llama3.2")

# Pass it to a new LLMVec
from mall import LLMVec
llm = LLMVec(chat)    

Access the functions via the new LLMVec object, and pass the text to be processed.

1
2
3
4
5


llm.sentiment(["I am happy", "I am sad"])
#> ['positive', 'negative']

llm.translate(["Este es el mejor dia!"], "english")
#> ['This is the best day!']

For more information visit the reference page: LLMVec

New cheatsheet

The brand new official cheatsheet is now available from Posit: Natural Language processing using LLMs in R/Python . Its mean feature is that one side of the page is dedicated to the R version, and the other side of the page to the Python version.

An web page version is also availabe in the official cheatsheet site here . It takes advantage of the tab feature that lets you select between R and Python explanations and examples.

What LLMs Actually Do (and What They Don’t)

Veerle Eeftink - Van Leemput — Thu, 31 Jul 2025 00:00:00 +0000

Welcome to the Shiny Side of LLMs!

Large language models (LLMs) are everywhere, and we’re way beyond denying their existence. New use cases pop up every single day, from autocompleting code to customer support bots, summarisers, and tutors. And everybody feels the urge to jump in. Tools like ChatGPT, Claude, and GitHub Copilot have made us all comfortable using AI. But when it comes to building something that is tailored, interactive, and actually useful for others, it all starts to feel… muddy. Buzzwords start flying: embedding, encoding, RAG, vector databases, streaming tokens… and suddenly you’re wondering if this is still for you.

Spoiler alert: it is, and it’s all more accessible than it seems. Yes, the underlying technology is complex, but that doesn’t mean it’s out of reach. For the first time, you have easy access to something that mimics human reasoning. As a programmer, that means you can describe a problem in plain language and get useful output: code, explanations, ideas, or even full analyses. And you don’t need to train your own models or master deep learning theory to use it. You just need to understand what’s happening under the hood well enough to start building. In this blog series, we’ll demystify all the buzzwords and walk you through:

What LLMs really do, conceptually and technically, but explained super simply
How to interact with them while staying in your comfort zone (Python or R)
How to turn those interactions into real, usable tools using Shiny (again: Python or R)
The magic of mixing in your own data such as documents, notes, and anything else you use

This isn’t just about building a chatbot. It’s about understanding how things work and how you can use an LLM to build your own tools. A document summariser, a smart search assistant, or a decision helper for your team: the choice is yours. We’ll go step by step. From “zero” (I’ve only ever typed things into ChatGPT) to “hero” (I’ve built my own LLM-powered app and I understand how it works).

No need to know anything about model hosting, server clusters, or fancy acronyms. Just you, some Python or R, and a fun idea!

What can you expect from this first part?

This is Part 1 of “The Shiny Side of LLMs” series: What LLMs Actually Do (and What They Don’t). It is intended to give you some insight into how LLMs work under the hood.

You don’t need to know all this to start building things. But having a basic understanding of the mechanics can help you reason more clearly about what LLMs are (and aren’t) good at and avoid getting lost in the hype.

The “Large”, the “Language”, and the “Model”

TL;DR

When we talk about a “Large Language Model”, we’re really talking about a system trained on huge amounts of text, designed to predict and generate human-like language, powered by a model that recognises patterns at scale.

First things first, what is a Large Language Model? A good first step is breaking down the term “Large Language Model”. Let’s do it backwards, starting with the last word: “Model”. In this context, a model is a statistical system that learns patterns from data and uses them to make predictions. Yes, that’s right: plain old statistics.

When we’re talking about a “Language Model” we make predictions about a language, consisting of sentences and its words. It all boils down to the likelihood of a next word, or sequence of words. A language model is trained to understand the kinds of words that tend to appear together, how sentences flow, and how meaning is built from context. Early models could only guess a single word at a time and struggled with anything longer or more complex, but modern ones can generate entire paragraphs or documents that stay on topic and sound surprisingly coherent. Almost like a human.

Bigger is better. The “Large” in “Large Language Model” refers to both the amount of data the model is trained on and the number of parameters it has (the internal settings the model adjusts as it learns). The “larger” a model is, the more patterns it can recognise and the more fluent its outputs tend to be.

So what does that mean when you interact with one? When you ask an LLM something, you prompt it to do something (that’s why the input or text you type is also called a prompt). That something is to predict the most likely sequence of words based on the sequence you have given.

LLMs and AI

You didn’t see much mention of Artificial Intelligence (AI) yet, and that’s on purpose. While LLMs are a specific kind of AI, they’re just one part of a much broader field. But since LLMs are the most visible (and frankly, the most hyped) examples of AI right now, people often use the terms interchangeably. To keep things focussed, we’ll only use the term LLM here.

How LLMs learn

We can talk about LLMs like they’re the newest thing, but the foundation is still as “ancient” as statistics. It’s about understanding data: making inferences, finding patterns, estimating (un)certainty. Machine Learning (ML) is statistics, and ML models learn patterns from data. Think about algorithms like decision trees, support vector machines, and linear regression that all rely on the same idea: learning from examples.

LLMs learn through a technique called deep learning, which is a subfield of ML. At the core of deep learning are neural networks: mathematical structures loosely inspired by biological neurons within the brain. You can think of them as layers of tiny decision-makers, where each layer takes in numbers, transforms them, and passes them on to the next.

Each connection between layers has a weight, aka a number that determines how strongly one “neuron” influences another. During training, the model adjusts these weights millions (or billions) of times to reduce the gap between its predictions and the actual data. This process is called back propagation, and it’s basically a giant optimisation loop: the model makes a guess, sees how wrong it was, tweaks the weights, and tries again. Over time, it gets surprisingly good at spotting patterns in language.

The bigger the model, the more opportunities it has to learn how to make good guesses. That’s why the “Large” in “Large Language Model” is so important. In the last few years, model size and model capability has exploded. This is due to hardware improvements (something has to crunch all that data), better training techniques and something called transformers. Ah, jargon.

A transformer is a kind of model architecture. “Model architecture” sounds difficult, but in simple terms this is a blueprint for how a model processes and learns from data. It was introduced by Vaswani et al. (2017) . Transformers are part of the neural network family, and they’re designed to handle huge amounts of data quickly and efficiently, which makes them perfect for crunching lots of text.

What made transformers such a game-changer is a clever idea called attention. Instead of reading text word by word like older models, transformers look at the whole sentence (or paragraph) at once and figure out which words matter most to each other. That’s how large language models can generate surprisingly accurate, context-aware text.

They also made it possible to train on much bigger chunks of data and pick up more subtle patterns in language. This is combined with a technique called self-supervised learning, where the model learns to predict missing words in sentences. If you show it “I lifted some ___” it learns that “weights” is more likely than “horses”. Because the data itself provides the training signals, we don’t need humans to label anything. That makes training scalable. It doesn’t mean it’s totally label-free, though. The “labels” (like missing words) just come naturally from the text.

The combo of smart architecture and massive training data is what unlocked the big leap in quality of today’s LLMs and made models like ChatGPT possible. Fun fact: the “T” in GPT stands for Transformer. Now you know why that is!

From statistics to transformers, to models we can chat with and help us to get things done faster. How’s that for the “ancient” statistical techniques?!

It all sounds too good to be true: a model that learns on its own, getting “smarter” with every chunk of input you provide. And it indeed is, because with this self-supervised learning come challenges too. As more online text is generated by LLMs, there’s a risk the model ends up learning from itself. It’s like copying a copy: input gets blurry. And in the end LLM are just models that predict something based on the data it has been given. If that data is garbage, so is the outcome. If we don’t want to end up with boring, soulless, similar, and even incorrect content, we still have to write content ourselves.

Still, one thing is clear: what makes LLMs so powerful is the amount of data they learn from. And that leads us straight into the magic of scale.

The magic of scale

As LLMs are trained on more and more data, something cool happens: they begin to show emergent abilities (Wei et al., 2022 ). These are skills the model wasn’t directly trained for, yet still manages to do. Nobody explicitly programmed an LLM to summarise text, translate between languages, or generate code. But once the model is large enough and has seen enough examples, these abilities just… emerge. Magic!

This is different from what we call generative abilities. “Generative” simply means the model can generate new text. All LLMs are generative and can produce new text. You give them a prompt, and they predict what comes next. That’s their basic job and it’s expected behaviour. Emergent abilities were not in the original job description and the model just suddenly gets good at tasks nobody told it to do. It’s surprising behaviour.

For example, you can ask it to convert a sentence into pirate-speak, even if it was never explicitly trained to do that. It uses general language patterns it has learned to give it a try, which is known as zero-shot learning. And if the model struggles to understand what you want, you can help it by giving a few examples in your prompt: few-shot learning. If you want a list reformatted in a certain way, you can provide some examples and let the LLM finish the rest.

All of this is possible not because the model is “intelligent” in a human sense, but because it has seen so much text and learned such general patterns that it can apply them in all kinds of situations. This ability to generalise, or doing more with less instruction, is one of the most powerful outcomes of large-scale learning.

What LLMs aren’t

So you might think LLMs are pretty smart. After all, they can write essays, answer questions, and even debug your Python and R code. It’s the shiny side of LLMs. But it isn’t all rainbows and unicorns: LLMs aren’t smart and they aren’t thinking. LLMs don’t reason like humans. They don’t understand what they’re saying. They don’t “know” facts in the way we do. What they do is predict the next word in a sentence, based on patterns they’ve seen in massive amounts of text.

That means an LLM can be confidently wrong. They might “hallucinate” an answer that sounds plausible but is completely made up. The code it generates might look solid, but when you actually run it you notice that it contains packages that don’t exist or variable names that were never defined. They might reflect or even amplify biases in their training data. They might sound accurate, but miss the mark entirely. And crucially, you don’t really control them. You can’t force an LLM to always give a safe or correct answer. What you can do is guide it with better prompts, build tools around it that check its output, and design systems that use the model responsibly.

So while LLMs are great at generalising across tasks, that power doesn’t come from intelligence. It comes from scale. And that is important to remember when building with LLMs.

Let’s talk about context

As mentioned earlier, LLMs can understand and generate surprisingly accurate, context-aware text. And that context deserves some attention (literally: remember transformers!). It’s the context that makes sure an LLM doesn’t blurt out random sentences (or, ok, only occasionally do). But what does “context” actually mean for a Large Language Model?

It all starts with tokens, which are tiny chunks of text. A token can be a word, a part of a word, or even just characters or punctuation. Every time you type something into an LLM, your input is split into these tokens. Depending on the model, an LLM can “see” a certain number of tokens at once. This is called the context window. Think of it like a sliding window over a long scroll of text. The model can only see what fits inside that window. If your input is too long and tokens fall outside of that window: too bad. An LLM might trims the beginning or the end and hopes you didn’t put anything important there. The exact behaviour varies by provider and how the system around the model is designed. Some providers implement strategies like summarising or condensing previous conversation history, while others simply drop older messages to make room for new ones. And in some cases, the system might reject the request outright rather than attempt to process an incomplete prompt.

But wait… We previously learned LLMs were trained on all of language right? Yes. But at generation time, it doesn’t remember everything it’s ever seen. It only has this input, right now, and the tokens inside the window. That’s its entire world. There’s no long-term memory (unless you’ve built that in, we’ll get to that later). So if you want the model to give you a good answer, what you say really matters. And that’s not the only thing: it also matters how you say it.

Imagine these two prompts:

1. “Create a Shiny for Python app that plots a histogram of random numbers”
2. “Plot a histogram of random numbers. Use Shiny for Python.”

Same words. Different order. Potentially different behaviour.

Language models are super sensitive to the flow of a sentence. If you put instructions at the start, the model is more likely to follow them. Bury them at the end, and it might treat it more like a side-note. Clarity is an LLMs best friend.

So context isn’t just about what’s in the window. It’s also about where words are and how they’re phrased. Order and surrounding words matter. In the model’s brain, each token looks around (thanks to that attention trick you met earlier) and asks, “who’s standing next to me, and what story are we telling?” Those neighbouring words:

Disambiguate meaning: words on their own only say so much. “Bass” is a fish next to “river”, and a guitar next to “band.”
Signal tone: a single word can flip the sentiment. For example, “nice” after “not”. On its own, the word “nice” would be positive.
Set expectations: since a model is trained on sequences of tokens actually written by humans, some sequences are familiar. So after “Once upon a” the model is primed for “time,” not “ship”.

Because the model predicts the next word by weighing everything it can see in the window, every surrounding word tweaks the odds. Change one word, and the whole probability landscape shifts. This becomes clear with these different prompts:

1. “Write a short horror story about a clown”
2. “Write a funny short horror story about a clown”

Can you already sense why “prompt engineering” became so hot?

Now back to the memory thing. Out of the box, the model doesn’t “remember” what you said three chats ago. It doesn’t even remember what you said two messages ago, unless you send the whole chat history in the current prompt. So how big a prompt can get, determines how much an LLM can remember. This brings us back to the context window. The maximum number of tokens they can handle at once matter. It could be 2000 or 128000 tokens. That all depends on the model. If your conversation gets longer than the model can hold, the oldest messages get chopped off. That’s why in very long chats, you might notice an LLM starts “forgetting” early facts. These facts are not in the context window anymore, so the model can’t “see” them anymore either.

It is possible for models to have access to persistent memory across conversations, but in that case facts and knowledge have to be explicitly stored somewhere. This memory is separate from the context window. Whether and how that memory gets updated depends on the broader system designed by the LLM provider. This system, the surrounding infrastructure and logic around the model, determines what’s worth saving. Only if something seems important enough the memory will be updated.

From tokens to embedding to encoding

Your prompt is divided into tokens, those tiny chunks of text. But computers don’t “understand” words like we do. Computers work with numbers. That’s where something fancy called “embedding” comes in: each token is turned into numbers that represents its meaning. And these numbers are not a plain “4982” or “2974”, but rather a pattern of numbers. You can compare it with an unique barcode. To “understand” your prompt, the model looks at all these number patterns together and uses the earlier mentioned attention to figure out how they relate to each other and what the relationship between the words is. This process is called encoding. A well-known model that does this is BERT, which was introduced by Google in 2018 (Devlin et al., 2019 ).

Because of word embeddings and encoding, a prompt like “Create a Shiny for Python app that plots a histogram” is not just a set of random words. The model knows that “Shiny” and “Python” relate to things like “web app” or “dashboard” because their embeddings are close together. It gives the model directions on the tech stack to use. It also sees that “plot” and “histogram” are both in the prompt, so it understands you want a chart. And the word “create” signals that you’re asking for code, not just an explanation. All these relationships, links and signals are taken into account when generating the output token by token.

So far, we only talked about generating language, like plain text or some code. But what about images? The latest models can take images as input and they spit out a polished (perhaps fake-looking) image when being asked too. These multimodal models (like GPT-4o or Claude 3) combine both vision and language. How? By treating images like language!

Images and multimodal models

When talking about images, we’re taking a small side-step in our LLM journey. Image tasks are typically handled by specialised models (like diffusion models for image generation) or by multimodal LLMs (like GPT-4o or Gemini) that can process both text and images. So when we talk about “LLMs” generating or understanding images, we’re really referring to LLMs with extra capabilities.

Just like text is broken down into tokens, images are also converted into a form the model can understand. Instead of words, an image is split into patches (tiny chunks of pixels). You can think of this as “tokenizing an image.” And just like tokens, patches are turned into numbers that capture the meaning of those pixels, which brings us back to the fancy word: “embedding”. These image embeddings are then processed by the model just like text embeddings. That’s how a model can look at an image and describe it or answer questions about it.

Generating new images is also not a problem: in that case embeddings are translated to a visual representation (pixels) and a full image is being built from it. This is exactly what well-known generative vision models like DALL·E, Midjourney, or Stable Diffusion do: they’re trained to turn a sequence of text tokens into a sequence of image tokens, gradually “painting” a picture that matches the prompt. If we ask Stable Diffusion for “A weightlifting Dalmatian holding a barbell with red plates”, it has no problem doing so. It’s all about processing the embeddings together and figure out how every word relates to each other. In this case, “weightlifting” modifies what the “Dalmatian” should look like, and “holding” links “Dalmatian” and “barbell”.

Note that depending on the model you’re using, you might need to provide more context in your prompt by saying something like “Generate an image that shows…”. This is especially important for multimodal models, like GPT-4o or Gemini, which can generate text, images, audio, and more. A prompt like “A weightlifting Dalmatian holding a barbell with red plates” might seem clear to you and me, but to a multimodal model, it could mean several things: perhaps you want an image, or a description, or a funny meme. This is different for a model like Stable Diffusion, a text-to-image model, which is trained specifically for image generation. Naturally it will assume you want an image when you give it a prompt.

More jargon: RAG and vector databases

LLMs can only generate such “human-like” language because they are good at embeddings, encodings, and understanding context. Context is everything. This context comes from the “knowledge” an LLM has, or from you. On their own, LLMs can only use data that has been used during training to generate output. LLMs only “know” facts that happened during their training process (and not after), and proprietary information is off limits. That doesn’t mean you can’t let an LLM use new facts, or private company info. You can add it to the context, taking the context window into account. So what do you do when you want to add so much information that it doesn’t fit that window? Hello, RAG!

It’s probably not the first time you encountered the term “RAG”. RAG stands for Retrieval-Augmented Generation (Lewis et al., 2020 ). It means that instead of relying only on what the language model was trained on, it first retrieves relevant information (like from a document or database) and then generates a response using both your prompt and that retrieved info. This helps the model give more accurate and up-to-date answers.

“Retrieving information” sounds simple when you just have one document, but imagine you have a big pile of documents. That’s a bit more complicated. To make retrieval easier, RAG systems usually break documents into smaller chunks, like paragraphs or even sentences. Each chunk gets turned into a vector and stored in a vector database. When you ask a question, the system searches for the chunks whose vectors are closest in meaning to your query and uses those to be added to the context. It’s all about making the most of the context window.

So many models

We’re not talking about one big giant LLM in this blog series. It’s plural: LLMs. That’s because there are a lot of different LLMs out there. All these models are trained a little differently. That’s important, because how a model is trained (what and how many data it saw, what tasks it practiced, what architecture it uses) shapes what it’s good at. Some models are trained mostly on text (think books, articles, web pages), while others are trained more heavily on code (like GitHub repos). That’s why some models might be better suited as your coding-companion, and others may serve you well as your ghostwriter.

There are many ways to compare LLMs, but here are few things you might want to consider:

Context window: larger windows help with long documents or multi-turn conversations.
Training data: what kind of data the model was exposed to (e.g. text, code, math problems, dialogue, or some mix).
Instruction tuning: some models are trained to follow prompts more precisely, often using human feedback (this has a very lengthy name: Reinforcement Learning from Human Feedback, aka RLHF).
Output quality: ah, there it is! Probably the the most important thing. How good are the model’s answers?! This depends on everything above, plus things like formatting ability, reasoning steps, or hallucination rates.
Cost and speed: we’ll come back when actually talking to LLMs, but not all models are priced equally, and speed can vary a lot.

Since output quality is probably one of your top priorities, you might wonder how you can quickly get a sense of LLM performance. You can try out different models, sure. But as a data-savvy person you’re probably keen to turn towards a more objective method. Luckily, there are tools like llm-stats.com , Vellum’s leaderboard , and LM Arena that help you with a comparison. The latter is based on subjective human ratings of output quality across tasks.

Looking at these leaderboards you see all the popular models like GPT (OpenAI), Claude (Anthropic), Gemini (Google), Llama (Meta), DeepSeek, Mistral and Nemotron (NVIDIA). And all of these models have different versions too! It might spin your head. Each model has strengths and weaknesses. Some are better with long input, some are cheaper, some are open source… But honestly, don’t stress too much about picking the “perfect” one. Once you start integrating an LLM into your Shiny app, it’s easy to change models later if you need to. At this stage, it’s more important to just get started.

When it comes to personal experience and “word on the street” from Python and R developers: with Claude you’re generally good when it comes to writing code (e.g. as a copilot) and summarising text. If you’re looking for a model with a large context window, Gemini might be what you would be after. And your friendly neighbourhood LLM could potentially be found in GPT models (like GPT-4o) which generally have a good balance between reasoning, code, and creative tasks.

Building with LLMs

LLMs can be applied in a wide range of natural language tasks. They’re perfect for building chatbots or virtual assistants. They can generate content, which can be blog posts, emails, or code snippets. They can handle machine translation across languages, summarise long reports into key points, and perform sentiment analysis to extract opinions or emotional tone from text. They even can do data analysis for you.

It’s worth recognising that the examples above can have multiple use cases: you can use LLMs to increase your own productivity, or you can move from using an LLM to building with one. As a copilot, an LLM can help you write code, draft content, summarise documents, or generate ideas, which (hopefully) makes you faster and more efficient in your day-to-day work. But when you start building tools for others, the focus shifts. Now it’s not just about what the model can do, but how you wrap it in a helpful interface. You need to think about the user’s problem, how they interact with your tool, and how to guide their input to get meaningful results. That’s a different kind of challenge. And this is exactly where something like Shiny shines. Whether you’re using Python or R, Shiny makes it easy to build interactive, user-friendly applications that connect to an LLM behind the scenes. You can quickly prototype ideas, test interfaces, and deliver real value. It’s the perfect bridge between your data science skills and creating practical tools powered by LLMs. The only thing you need is a bit of creativity, some coding skills, and a foundational understanding of LLMs. The latter we tackled already!

So what are we going to build in this blog series? Imagine you need to present something to your colleagues in the development team, you’re a little nervous and would like to have some feedback on the presentation first. Since you’re a data scientist who loves reproducible slides, you’ve chosen to build a Quarto presentation . Wouldn’t it be wonderful if you could upload your slides in an app and get instant feedback? Like the perfect “Presentation Rehearsal Buddy” that analyses your slides and gives you tailored suggestions? And wouldn’t it be great if you can help other data scientists with the same problem? You can even prevent your colleagues from giving too lengthly and unfocussed presentations. Also, it’s great for polishing your talks at conferences, like posit::conf(2025) . Now that’s a cool app!

And it’s the kind of thing you can actually start building now you understand what LLMs can and can’t do. But wait… I can feel your creativity flowing, the app idea growing on you, and I understand you might be dreaming bigger. Maybe a model trained on your own (sensitive) company data to really tailor presentations to your work? Amazing! You probably heard it’s not the best idea to send sensitive data to just any public API, so your mind jumps to a local LLM: one that runs on your own machine(s), just for you. A private model that knows everything about your org… That’s tempting! And while it sounds wonderful, it’s probably not the best place to start.

Running your own LLM comes with lots of complexity: infrastructure, cost, fine-tuning, and security. It’s a bit like buying a race car before learning to drive, then trying to build the track and engine from scratch. A better place to begin? Use the best pre-trained models that are already available.

And remember the magic of scale: LLMs show their real power only when trained on massive amounts of data and compute. That’s where those emergent abilities come from. Why reinvent the wheel when you can just take it for a spin? There’s a reason why the best performing models can’t be run on a laptop!

Responsible use and regulations

Ready, set, … disclaimers. Damn. This is not an ethics class. We’re here to code. But we can’t get around some disclaimers, and if you’re using an LLM, you should take some things into account when it comes to responsible use:

Perhaps this goes without saying, but you can’t bluntly copy and paste data or (sensitive) information into any LLM. That means your own data, and somebody else’s data. Generated or uploaded input (text, data, prompts, images, etc.) could (and will) be used for other purposes, such as the training of AI models. If you don’t want things to be out in the open, you shouldn’t input it unless you’re really sure that data will not be re-used.
You can’t trust an LLM completely either. You must always take a critical approach to using the output produced and be aware of the limitations, such as bias, hallucinations and inaccuracies.
You’re accountable for the integrity of the content generated by or with the support of an LLM. There’s also an important difference between using an LLM for your own tasks and building tools on top of LLMs that others will rely on. In the first case, you’re the human in the loop, judging what to trust and what to ignore. But when others interact with your app, you’re responsible for the entire experience: how the model is prompted, how outputs are presented, and whether users are warned of uncertainty. In any case, if you use an LLM to make decisions or build an app that others use to act upon, you can’t blame the LLM for giving false directions.

Many organisations know this too, and it makes them wary of using LLM-based tools. Especially in regulated environments, there’s a natural hesitation to adopt these tools without strong safeguards in place. If you want your organisation to be open to the idea, you need to think about clear data handling policies, usage boundaries, a transparent explanation of how the model is prompted, what kind of outputs are produced, and where the limitations lie. The more control and clarity you can offer, the more trust you’ll build.

If your organisation already trusts a cloud vendor like AWS, Azure, or Google Cloud with your data, then using an LLM hosted by that same vendor often fits within existing security and compliance frameworks, which might make it easier to justify adoption.

It’s also good to be aware of applicable regulations and guidelines. Know what you can and can’t use an LLM for when building your application. For example, in Europe, there’s the AI Act (which is the first-ever legal framework on AI). Generally speaking, legislation is not that fast, but since AI capabilities are quickly expanding it’s only a matter of time before rules and guidelines do catch up. Stay informed!

And with those less fun but necessary warnings we wrap up this introduction to LLMs. It’s not a magic all-mighty black box anymore: you now know exactly what LLMs are and what they are not.

Coming up: talking to LLMs

Our app for polished Quarto presentations is just around the corner, but before we can build it, we need to learn how to actually talk to an LLM. Programmatically that is. After all, the magic only happens once you know how to send the right prompts, pass along the right content, and make sense of what comes back. In our case, that means: extracting a presentation’s content, feeding it to the model with smart prompts, and formatting the output into something useful. Think about value boxes, improvement tips, or even edits to your Quarto file directly. That’s where we’re headed. But first, the next step in our app-building journey: talking to an LLM.

References

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv. https://doi.org/10.48550/arXiv.1810.04805

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval‑augmented generation for knowledge‑intensive NLP tasks. arXiv. https://doi.org/10.48550/arXiv.2005.11401

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. arXiv. https://doi.org/10.48550/arXiv.1706.03762

Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., & Fedus, W. (2022). Emergent Abilities of Large Language Models (Version 2). arXiv. https://doi.org/10.48550/arXiv.2206.07682

From One Notebook to Many Reports: Parameterized reports with the `jupyter` engine

Charlotte Wickham — Thu, 24 Jul 2025 00:00:00 +0000

Based on a talk at SciPy 2025

This post is based on the talk “From One Notebook to Many Reports: Automating with Quarto” delivered at SciPy 2025 by Charlotte Wickham. You can find the slides at cwickham.github.io/one-notebook-many-reports and example code at github.com/cwickham/one-notebook-many-reports .

The Problem: Repetitive Reporting

Would you rather read a generic “Climate summary” or a “Climate summary for exactly where you live”? Reports that are personalized to a specific situation increase engagement and connection. But producing many customized reports manually is tedious and error-prone.

Quarto solves this with parameterized reports—you create a single document template, then render it multiple times with different parameter values to generate customized outputs automatically.

A great example is the customized soil health reports from Washington Soil Health Initiative’s State of the Soils Assessment , presented at posit::conf(2023) by Jadey Ryan (watch on YouTube ). Jadey demonstrated this approach using R and plain text Quarto files (.qmd).

This post shows you how to apply the same principles using Python: we’ll walk through converting a Jupyter notebook (.ipynb) into a parameterized report, then automating the generation of multiple customized outputs. Then I’ll give you some tips for making your reports look polished.

The Solution: Parameterized Reports

Start with a notebook

As an example, let’s start with a Jupyter notebook analyzing climate data for Corvallis, Oregon.

You can see the full notebook, corvallis.ipynb, on GitHub , but here are the key pieces:

The code cells import some data for all of Oregon, and filter it to just rows relevant for Corvallis, then produce a summary sentence and a plot.
The document options specify echo: false so no code appears in the final output, and format: typst so the output is a PDF produced via Typst , a modern alternative to LaTeX.

This single notebook can be rendered with Quarto:

Terminal

1

quarto render corvallis.ipynb

The result is a PDF file, corvallis.pdf, a simple report with the title “Corvallis” and a single sentence summary of the climate data, along with a plot highlighting the mean temperature for this year against the last 30 years.

corvallis.pdf

Now, imagine we want to create this report for the 50 largest cities in Oregon. Here’s the steps we’ll take:

Turn hardcoded values into variables
Declare those variables parameters
Render the notebook with different parameter values
Automate rendering with many parameter values

1. Turn hardcoded values into variables

We want a report for each city. We’ll start by creating a variable, city, which we’ll designate a parameter in our next step. In a new code cell at the top of our notebook, we define the variable:

code

1

city = "Corvallis"

Then anywhere we previously hardcoded "Corvallis" in the notebook, we replace it with this variable.

The first occurrence is in the title of the document. Originally, we had a markdown cell defining a level 1 heading:

markdown

1

# Corvallis

We replace it with a code cell that uses an f-string to produce markdown for a level 1 heading based on the city variable:

code

1

Markdown(f"# {city}")

In the filtering step the replacement is straightforward, we just change the string to the variable:

Before:

code

tmean = tmean_oregon.filter(
    pl.col("city") == "Corvallis",
)

After:

code

tmean = tmean_oregon.filter(
    pl.col("city") == city,
)

Finally, the plot code (using plotnine ), sets the title of the plot to include the city name:

code

1
2
3


...
+ labs(title = "Corvallis, OR", ...)
...

We can also use an f-string here to include the city variable:

code

1
2
3


...
+ labs(title = f"{city}, OR", ...)
...

Now, we should be able to test our changes by explicitly setting the city variable to something other than “Corvallis” and re-running the cells. Since our report is no longer specific to Corvallis, we can rename it climate.ipynb.

2. Declare those variables parameters

Now we have a variable that represents the parameter, we need to let Quarto know it’s a parameter. Quarto’s parameterized reports are implemented using Papermill , and inherit Papermill’s approach: tag the cell defining the parameter with parameters.

In Jupyter, you can add this tag through the cell toolbar:

You can see the updated notebook, now a parameterized notebook, on GitHub: climate.ipynb .

3. Render with different parameter values

If we render climate.ipynb, it will still produce the same report for Corvallis, because we haven’t changed the parameter value:

Terminal

1

quarto render climate.ipynb

But we can now pass parameter values to Quarto with the -P flag:

Terminal

1
2
3
4
5


# Generate report for Portland
quarto render climate.ipynb -P city:Portland --output-file portland.pdf

# Generate report for Eugene  
quarto render climate.ipynb -P city:Eugene --output-file eugene.pdf

We’ve also added --output-file to ensure each report gets its own filename.

4. Automate rendering with many parameter values

To generate all 50 reports, we need to run quarto render 50 times, each time with a different city as the parameter value. You could automate this in many ways, but let’s use a Python script. For example, you might have a dataset of cities and their corresponding output filenames:

gen-reports.py

1
2
3
4


cities = pl.DataFrame({
    "city": ["Portland", "Cottage Grove", "St. Helens", "Eugene"],
    "output_file": ["portland.pdf", "cottage_grove.pdf", "st_helens.pdf", "eugene.pdf"]
})

I’ve generated a small example above, but in reality you would likely read cities in from a file. Then you could iterate over the rows of this dataset, rendering the notebook for each city:

gen-reports.py

1
2
3
4
5
6
7


from quarto import render
for row in cities.iter_rows(named=True):
    render(
        "climate.ipynb",
        execute_params={"city": row["city"]},
        output_file=row["output_file"],
    )

Run this script once, and you’ll get all 50 custom reports!

You can find the complete working example on GitHub: cwickham/one-notebook-many-reports/03-many-reports .

Pretty Reports: Brand and Typst

The steps above to produce parameterized reports apply to any output format supported by Quarto. However, if you are targeting typst you can take advantage of additional features to create beautiful PDF reports.

Brand.yml

Quarto supports brand.yml a way to specify colors, fonts, and logos:

_brand.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


color:
  palette:
    forest-green: "#2d5a3d"     
    charcoal-grey: "#555555"    
  foreground: charcoal-grey    
  primary: forest-green       
typography:
  fonts:
    - family: Open Sans
      source: google
  base:
    family: Open Sans
logo:
  medium: logo.png

Quarto will detect the _brand.yml file and apply the colors, fonts and logo to your report. Colors and fonts in your figures will need to be customized in your code, but that is made much easier with the brand-yml Python package which imports your values from _brand.yml.

You can see a full example of using _brand.yml with climate.ipynb at cwickham/one-notebook-many-reports/04-branded-reports , and learn more about Quarto’s support for brand in the Brand guide .

Typst

Learning a little bit of Typst syntax can take your reports from basic to beautiful. You can include raw Typst syntax in your notebooks, or wrap elements in Typst functions using the typst-function Quarto extension . As an example, you could add a header with the city name and a map of the location:

corvallis.pdf

You can see the source for this example at cwickham/one-notebook-many-reports/05-pretty-reports .

`jupyter` vs `knitr`

The steps for creating a parameterized report above are specific to documents that use the jupyter engine. With a Jupyter notebook (.ipynb), or a plain text Quarto document (.qmd) with only Python code cells, Quarto will default to the jupyter engine. As described above, the jupyter engine uses cell tags to identify parameters.

If you are working in a .ipynb file, your IDE will likely provide a way to add these tags through the cell toolbar. If you are working in a .qmd file, you can add tags as a code cell option:

1
2
3
4


```{python}
#| tags: [parameters]
city = "Corvallis"
```

With the jupyter engine, parameters can then be accessed directly as variables, e.g. city, in later code cells.

If you are working in a Quarto document (.qmd) with R code cells, Quarto will default to the knitr engine. With the knitr engine, you set parameters in the document header under params:

1
2
3
4


---
params:
  city: "Corvallis"
---

In knitr, parameters are accessed as elements of params, e.g. params$city.

You can read more about setting and using parameters in Guide > Computations > Parameters .

Wrapping Up

Parameterized reports turn one notebook into many customized outputs. You’ve seen the process of going from a notebook with a hardcoded value to a parameterized report that can be rendered with different values. You can then automate the rendering in any way you choose to generate dozens of reports at once.

Great Tables `v0.18.0`: Easy Column Spanners and More!

Rich Iannone — Fri, 18 Jul 2025 00:00:00 +0000

The development of Great Tables continues! We’re excited to announce the release of v0.18.0, which brings several powerful new features. These features make it even easier to create beautiful, informative tables. The key additions in this release include new methods (and a tweak to an existing one):

~~.GT.tab_spanner_delim(): quick spanner creation
~~.GT.fmt_tf(): easy boolean value formatting
~~.GT.cols_label_rotate(): enables column label rotation
~~.GT.fmt_datetime(): added format_str= parameter for extra customization

Let’s explore each of these interesting new features!

Quick spanner creation with `tab_spanner_delim()`

Working with data that has hierarchical column names can be tedious when manually creating spanners. The new ~~.GT.tab_spanner_delim() method automates this process by intelligently splitting column names based on a delimiter and creating the appropriate spanner structure.

Here’s a practical example using the towny dataset, which contains population data for a collection of municipalities across multiple census years. Let’s start by looking at the most populated cities and examining their column structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


from great_tables import GT
from great_tables.data import towny
import polars as pl
import polars.selectors as cs

# Create a smaller version of the `towny` dataset
towny_mini = (
    pl.from_pandas(towny)
    .filter(pl.col("csd_type") == "city")
    .sort("population_2021", descending=True)
    .select(
        "name",
        cs.starts_with("population_"),
        cs.starts_with("density_")
    )
    .head(5)
)

# Let's look at the column names
print(towny_mini.columns)

['name', 'population_1996', 'population_2001', 'population_2006', 'population_2011', 'population_2016', 'population_2021', 'density_1996', 'density_2001', 'density_2006', 'density_2011', 'density_2016', 'density_2021']

Notice how the column names have a clear hierarchical structure with underscores as delimiters. Let’s now create a table that takes advantage of this structure:

1
2
3
4
5
6
7
8


(
    GT(towny_mini, rowname_col="name")
    .tab_spanner_delim(delim="_")
    .fmt_integer(columns=cs.contains("population"))
    .fmt_number(columns=cs.contains("density"), decimals=1)
    .tab_header(title="Population and Density Trends from Census Data")
    .opt_align_table_header(align="left")
)

Population and Density Trends from Census Data
	population						density
	1996	2001	2006	2011	2016	2021	1996	2001	2006	2011	2016	2021
Toronto	2,385,421	2,481,494	2,503,281	2,615,060	2,731,571	2,794,356	3,779.8	3,932.0	3,966.5	4,143.6	4,328.3	4,427.8
Ottawa	721,136	774,072	812,129	883,391	934,243	1,017,449	258.6	277.6	291.3	316.8	335.1	364.9
Mississauga	544,382	612,925	668,599	713,443	721,599	717,961	1,859.6	2,093.8	2,283.9	2,437.1	2,465.0	2,452.6
Brampton	268,251	325,428	433,806	523,906	593,638	656,480	1,008.9	1,223.9	1,631.5	1,970.4	2,232.7	2,469.0
Hamilton	467,799	490,268	504,559	519,949	536,917	569,353	418.3	438.4	451.2	464.9	480.1	509.1

The ~~.GT.tab_spanner_delim() method recognizes the underscore delimiter and creates a hierarchical structure: "population" and "density" become top-level spanners, with the years (1996, 2001, 2021) as the final column labels. This creates a clean, organized appearance that clearly groups related metrics together. And, this one method can be used instead of a combination of ~~.GT.cols_label() and ~~.GT.tab_spanner() (which requires a separate invocation per spanner added).

Beautiful boolean formatting with `fmt_tf()`

Boolean data is common in analytical tables, but raw True/False values can look unprofessional. The new ~~.GT.fmt_tf() method provides elegant ways to display boolean data using symbols, words, or custom formatting.

Here’s a simple example showing different tf_style= options:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


from great_tables import GT
import polars as pl

# Create a simple DF with boolean data
bool_df = pl.DataFrame({
    "feature": ["Premium Sound", "Leather Seats", "Sunroof", "Navigation"],
    "model_a": [True, False, True, True],
    "model_b": [True, True, False, True],
    "model_c": [False, True, True, False]
})

(
    GT(bool_df, rowname_col="feature")
    .fmt_tf(tf_style="check-mark", colors=["green", "red"])
    .tab_header(title="Car Features Comparison", subtitle="Using check-mark style")
)

Car Features Comparison
Using check-mark style
	model_a	model_b	model_c
Premium Sound	✔	✔	✘
Leather Seats	✘	✔	✔
Sunroof	✔	✘	✔
Navigation	✔	✔	✘

You can also use different symbols and colors for a more distinctive look:

1
2
3
4
5


(
    GT(bool_df, rowname_col="feature")
    .fmt_tf(tf_style="circles", colors=["#4CAF50", "#F44336"])
    .tab_header(title="Car Features Comparison", subtitle="Using circles style")
)

Car Features Comparison
Using circles style
	model_a	model_b	model_c
Premium Sound	●	●	⭘
Leather Seats	⭘	●	●
Sunroof	●	⭘	●
Navigation	●	●	⭘

The ~~.GT.fmt_tf() method transforms boolean values into visually appealing symbols that make it easy to quickly scan and compare data across rows and columns.

Rotating column labels with `cols_label_rotate()`

When dealing with many columns or long column names, horizontal space becomes precious. The ~~.GT.cols_label_rotate() method solves this by rotating column labels vertically, allowing for more compact table layouts.

Here’s an example where we use the gtcars dataset to create a table which communicates a feature matrix:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49


from great_tables import GT, style, loc
from great_tables.data import gtcars
import polars as pl
import polars.selectors as cs

# Manipulate dataset to create a feature comparison table
gtcars_mini = (
    pl.from_pandas(gtcars)
    .filter(pl.col("year") == 2017)
    .filter(pl.col("ctry_origin").is_in(["Germany", "Italy", "United Kingdom"]))
    .with_columns([
        (pl.col("hp") > 500).alias("High Power"),
        (pl.col("mpg_h") > 25).alias("Fuel Efficient"),
        (pl.col("drivetrain") == "awd").alias("All Wheel Drive"),
        (pl.col("msrp") > 100000).alias("Premium Price"),
        (pl.col("trsmn").str.contains("manual")).alias("Manual Transmission")
    ])
    .select([
        "mfr", "model", "trim",
        "High Power",
        "Fuel Efficient",
        "All Wheel Drive",
        "Premium Price",
        "Manual Transmission"
    ])
    .head(10)
)

(
    GT(gtcars_mini)
    .fmt_tf(
        columns=cs.by_dtype(pl.Boolean),
        tf_style="check-mark",
        colors=["#2E8B57", "#DC143C"]
    )
    .cols_label_rotate(
        columns=cs.by_dtype(pl.Boolean),
        dir="sideways-lr"
    )
    .tab_header(
        title="European Luxury Cars Feature Matrix",
        subtitle="2017 Models with Performance & Luxury Features"
    )
    .opt_stylize(style=1)
    .tab_style(
        style=style.text(size="11px"),
        locations=loc.body()
    )
)

European Luxury Cars Feature Matrix
2017 Models with Performance & Luxury Features
mfr	model	trim	High Power	Fuel Efficient	All Wheel Drive	Premium Price	Manual Transmission
Ferrari	GTC4Lusso	Base Coupe	✔	✘	✔	✔	✘
Aston Martin	DB11	Base Coupe	✔	✘	✘	✔	✘
Lotus	Evora	2+2 Coupe	✘	✘	✘	✘	✘
Porsche	718 Boxster	Base Convertible	✘	✔	✘	✘	✘
Porsche	718 Cayman	Base Coupe	✘	✔	✘	✘	✘

This example demonstrates how both the ~~.GT.fmt_tf() and ~~.GT.cols_label_rotate() methods can work well together. The boolean columns use checkmarks (✓/✗) with custom colors=, while the rotated labels save horizontal space in this dense feature matrix. The combination allows you to put more information into a compact and still readable format.

Enhanced datetime formatting with `fmt_datetime()`

The ~~.GT.fmt_datetime() method now supports custom format strings through the new format_str= parameter, giving you complete control over how datetime values appear in your tables.

Here’s an example using the included gibraltar weather dataset:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


from great_tables import GT
from great_tables.data import gibraltar
import polars as pl

# Prepare the meteorological data
gibraltar_mini = (
    pl.from_pandas(gibraltar)
    .with_columns(
        [
            pl.concat_str([pl.col("date"), pl.lit(" "), pl.col("time")])
            .str.strptime(pl.Datetime, format="%Y-%m-%d %H:%M")
            .alias("datetime")
        ]
    )
    .filter(pl.col("datetime").dt.hour().is_in([6, 12, 18]))
    .select(["datetime", "temp", "humidity", "condition"])
    .sort("datetime")
    .head(10)
)

(
    GT(gibraltar_mini)
    .fmt_datetime(
        columns="datetime",
        format_str="%b %d %Y (%a) - %I:%M %p",
    )
    .fmt_number(columns="temp", decimals=1, pattern="{x}°C")
    .fmt_percent(columns="humidity", scale_values=False, decimals=0)
    .cols_label(
        datetime="Time",
        temp="Temperature",
        humidity="Humidity",
        condition="Conditions",
    )
    .tab_header(
        title="Gibraltar Temperature and Humidity Conditions",
        subtitle="Morning, Noon, and Evening Readings"
    )
    .opt_stylize(style=1, color="cyan")
)

Gibraltar Temperature and Humidity Conditions
Morning, Noon, and Evening Readings
Time	Temperature	Humidity	Conditions
May 01 2023 (Mon) - 06:50 AM	17.2°C	1%	Fair
May 01 2023 (Mon) - 12:20 PM	22.2°C	1%	Fair
May 01 2023 (Mon) - 12:50 PM	22.2°C	1%	Fair
May 01 2023 (Mon) - 06:20 PM	20.0°C	1%	Fair
May 01 2023 (Mon) - 06:50 PM	20.0°C	1%	Fair
May 02 2023 (Tue) - 06:50 AM	17.8°C	1%	Mostly Cloudy
May 02 2023 (Tue) - 12:20 PM	18.9°C	1%	Mostly Cloudy
May 02 2023 (Tue) - 12:50 PM	20.0°C	1%	Mostly Cloudy
May 02 2023 (Tue) - 06:20 PM	22.2°C	1%	Fair
May 02 2023 (Tue) - 06:50 PM	22.2°C	1%	Fair

The custom datetime formatting string in format_str="%b %d %Y (%a) - %I:%M %p" creates a readable datetime format that’s perfect for weather reporting, showing the day of week, month, day, year, and the time in 12-hour format.

Acknowledgements and what’s next

We’re grateful to all the contributors who made this release possible. These new features represent significant improvements for creating space-efficient tables while also maximizing visual appeal.

The combination of these features lets you now create complex, professional tables with hierarchical column structures, boolean indicators, space-saving labels, and nicely formatted datetime displays.

We’re always happy to get feedback and hear about how you’re using Great Tables:

Keep building those beautiful tables!

Posit at SciPy 2025

Shiny Team — Mon, 14 Jul 2025 00:00:00 +0000

The Shiny Team was at SciPy 2025! If you missed us at SciPy 2025 here’s what we were up to.

We just got back from a week at SciPy 2025 and had a great time meeting and talking with folks at the conference. If you didn’t get a chance to say hi to us, you can also find us online on GitHub or Discord . You can also learn more about Posit at posit.co .

If you missed a tutorial or talk we have all the information below. You’ll have to wait a bit for the SciPy folks to get all the tutorials and talks uploaded to Youtube .

Talks by Posit Folks

Adding Plots to Great Tables

Jules Walzer-Goldfeld — Thu, 03 Jul 2025 00:00:00 +0000

While working on gt-extras , I’ve been exploring how to add small plots to Great Tables. These can go by many names, like spark lines, nanoplots, and so on. In this post, I’ll look at three approaches I tried: adding plots with plotnine , svg.py , or adding HTML directly. In the first two cases, the plots are SVGs, while the latter entails a collection of composed HTML div elements.

Here are the pieces I’ll cover:

svg.py: creating your own tiny chart directly for a row.
direct HTML: adding HTML divs directly.
plotnine: adding a full, stripped-down chart to a row.

In the end, it’s often simplest to use svg.py, since you can create basic charts with minimal overhead. Building elements with HTML has even less overhead, but it is also slightly less user-friendly. At the other end of the spectrum, as your charts become more complex, using existing packages like the more exhaustive plotnine is a good alternative.

Animal	Legs	Plot
Ostrich	2	2
Spider	8	8
Lion	4	4

Here is the final result:

Code

Great Tables + marimo = Interactive Tables

Rich Iannone — Tue, 24 Jun 2025 00:00:00 +0000

Jerry Wu’s recent post demonstrates how Great Tables can be made reactive in a marimo notebook. The post showcases a really nice integration where marimo’s reactive widgets can be embedded directly into Great Tables using the html() helper function. This effectively creates an interactive table that updates in real-time whenever users interact with the controls.

Interactive Table Demo

Here’s a screencast (based on the aforementioned post ) showing Jerry’s GT table in marimo. It demonstrates how the embedded marimo widgets create reactive effects, allowing you to directly modify the table in real-time.

The marimo reactive widgets can be embedded into a GT table through the html() helper function. In Jerry’s example, the table responds to widget changes in the opt_stylize() method call, where the .value attributes for all of the widgets are passed in as arguments.

1
2
3
4
5


gt.opt_stylize(
    style=style_widget.value,
    color=color_widget.value,
    add_row_striping=row_striping_widget.value,
)

marimo Showcases the Integration

The marimo team must have seen that post and liked what they saw! They recently released a video showcasing how marimo widgets work with Great Tables to create reactive tables.

For more details on marimo’s widget system, check out their UI elements documentation and reactivity guide .

marimo + Quarto Integration

For those interested in embedding marimo notebooks directly in Quarto documents, the marimo team has also created a quarto-marimo plugin . This extension allows you to run marimo code blocks natively within Quarto documents, making it easy to create interactive content that combines the best of both tools. It’s a very exciting development since we use Quarto extensively (our docs and this very blog post were authored through Quarto!). And naturally there’s a video for this as well.

Wrapping Up

We’re excited by how the marimo team is committed to creating seamless integrations with popular tools like Great Tables and Quarto. We’re also excited to see what new features and integrations they’ll develop next!

Be sure to check out the marimo YouTube channel as they consistently produce high-quality videos that demonstrate the power of reactive notebooks in action.

Data Validation Libraries for Polars (2025 Edition)

Rich Iannone — Wed, 04 Jun 2025 00:00:00 +0000

Data validation is a very important part of any data pipeline. And with Polars gaining popularity as a superfast and feature-packed DataFrame library, developers need validation tools that work seamlessly with it. But here’s the thing: not all validation libraries are created equal, and choosing the wrong one can lead to frustration, technical debt, or validation gaps that could bite you later.

In this survey (conducted halfway through 2025) we’ll explore five Python validation libraries that support Polars DataFrames, each bringing distinct strengths to different validation challenges.

Note

Great Expectations, while being one of the most established data validation frameworks in the Python ecosystem, is not included in this survey as it doesn’t yet offer native Polars support. See this issue and this discussion for the inside baseball.

Recommendations

Here are the unique strengths for each library:

Library	⭐	Best Features
Pandera	3,838	Statistical testing, schema-centric validation, mypy integration
Patito	468	Pydantic integration, model-based validation, row-level objects
Pointblank	173	Interactive reports, threshold management, stakeholder communication
Validoopsie	63	Built-in logging, composable validation, impact levels, lightweight Great Expectations alternative
Dataframely	319	Collection validation, advanced type safety, failure analysis

Based on these strengths, here are my recommendations for which libraries to use according to use case:

Use Case	Best Libraries	Description
Type-safe pipelines	Pandera, Dataframely, Patito	Static type checking and compile-time validation
Stakeholder reporting	Pointblank	Sharing validation results with non-technical teams
Row-level object modeling	Patito	Converting DataFrame rows to Python objects with business logic
Statistical validation	Pandera	Testing data distributions and statistical properties
Data quality improvement	Pointblank, Validoopsie	Gradual quality improvement with threshold tracking

Setup

We are going to run through examples with Pandera, Patito, Pointblank, Validoopsie, and Dataframely, using this Polars DataFrame as our test case:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


import polars as pl

# Standard dataset for all validation examples
user_data = pl.DataFrame({
    "user_id": [1, 2, 3, 4, 5],
    "age": [25, 30, 22, 45, 95],  # <- includes a very high age
    "email": [
        "user1@example.com", "user2@example.com", "invalid-email",  # <- has an invalid email
        "user4@example.com", "user5@example.com"
    ],
    "score": [85.5, 92.0, 78.3, 88.7, 95.2]
})

We’ll try to run the same data validation across the surveyed libraries, so we’ll check:

schema validation (correct column types)
user_id values greater than 0
age values between 18 and 80 (inclusive)
email strings matching a basic email regex pattern
score values between 0 and 100 (inclusive)

Now let’s dive into each library, starting with the statistically-focused Pandera.

1. Pandera: Schema-First Validation with Statistical Checks

Pandera is a statistical data validation toolkit designed to provide a flexible and expressive API for performing data validation on dataframe-like objects. The library centers on schema-centric validation, where you define the expected structure and constraints of your data upfront. You can enable both runtime validation and static type checking integration. Pandera added Polars support in version 0.19.0 (early 2024).

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


import pandera.polars as pa

# Define schema using our standard dataset
schema = pa.DataFrameSchema({
    "user_id": pa.Column(pl.Int64, checks=pa.Check.gt(0)),
    "age": pa.Column(pl.Int64, checks=[pa.Check.ge(18), pa.Check.le(80)]),
    "email": pa.Column(pl.Utf8, checks=pa.Check.str_matches(r"^[^@]+@[^@]+\.[^@]+$")),
    "score": pa.Column(pl.Float64, checks=pa.Check.in_range(0, 100))
})

# Validate the schema
try:
    validated_data = schema.validate(user_data)
    print("Validation successful!")
except pa.errors.SchemaError as e:
    print(f"Validation failed: {e}")

Validation failed: Column 'age' failed validator number 1:  failure case examples: [{'age': 95}]

This example demonstrates Pandera’s declarative approach, where you define what your data should look like rather than writing imperative validation logic. The schema acts as both documentation and as a validation contract. Notice how multiple checks can be applied to a single column (here, the age column receives two checks), and the validation either succeeds completely or provides error information about what failed.

Comparisons

Both Pandera and Patito use declarative, schema-centric approaches, but differ in their design philosophies:

Pandera uses a dictionary-like schema structure with Column objects for defining validation rules
Patito uses Pydantic model classes with familiar Field syntax for validation constraints
Pandera focuses heavily on statistical validation capabilities like hypothesis testing
Patito emphasizes integration with existing Pydantic workflows and object modeling
a key behavioral difference: Patito reports all validation errors in a single pass, while Pandera stops at the first failure

The choice between them often comes down to whether you prefer Pandera’s statistical focus or Patito’s Pydantic integration.

Unlike Pointblank’s step-by-step validation reporting, Pandera validates the entire schema at once. Compared to Patito’s model-based approach, Pandera focuses more on statistical validation capabilities. Unlike Validoopsie’s and Pointblank’s method chaining style, Pandera uses a more declarative, schema-centric approach.

Unique Strengths and When to Use

Here are some of stand-out features that Pandera has:

type-safe schema definitions with mypy integration
statistical hypothesis testing for data distributions: perform t-tests, chi-square tests, and custom statistical tests directly in your validation schema
excellent integration with Pandas, Polars, and Arrow support
declarative schema syntax that serves as documentation
built-in support for data coercion and transformation

This statistical validation capability goes beyond basic type and range checking to test actual data relationships and distributional assumptions. For example, you can validate that the mean height of group "M" is significantly greater than group "F" using a two-sample t-test, or test whether a column follows a normal distribution. This makes Pandera uniquely powerful for data science workflows where the statistical properties of your data are as important as individual data points meeting basic constraints.

Data practitioners should choose Pandera when building type-safe data pipelines where schema validation is critical, especially in data science workflows that require statistical validation. It’s ideal for users that value static type checking, need to validate statistical properties of their data, or want schemas that double as documentation.

Pandera also excels in environments where data contracts between teams are important and where the statistical properties of data matter as much as basic type checking.

2. Patito: Pydantic-Style Data Models for DataFrames

Patito brings Pydantic’s well-received model-based validation approach to DataFrame validation, creating a bridge between Pydantic-style data validation and DataFrame processing. The library’s primary goal is to provide a familiar, Pydantic-style interface for defining and validating DataFrame schemas, making it particularly appealing to developers already using Pydantic in their applications.

Patito launched with Polars support from the beginning (in late 2022). Native Polars integration is touted as one of its core features, reflecting the growing adoption of Polars in the Python ecosystem.

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


import patito as pt
from typing import Annotated

class UserModel(pt.Model):
    user_id: int = pt.Field(gt=0)
    age: Annotated[int, pt.Field(ge=18, le=80)]
    email: str = pt.Field(pattern=r"^[^@]+@[^@]+\.[^@]+$")
    score: float = pt.Field(ge=0.0, le=100.0)

# Validate using the model
try:
    UserModel.validate(user_data)
    print("Validation successful!")
except pt.exceptions.DataFrameValidationError as e:
    print(f"Validation failed: {e}")

Validation failed: 2 validation errors for UserModel
age
  1 row with out of bound values. (type=value_error.rowvalue)
email
  1 row with out of bound values. (type=value_error.rowvalue)

This example showcases Patito’s model-centric approach where validation rules are embedded in class definitions. The use of Python’s type hints and Pydantic’s Field syntax makes the validation rules self-documenting. Notably, Patito reports all validation errors at once, providing a fairly comprehensive view of data quality issues, whereas other libraries (e.g., Pandera) stop at the first failure.

Column Validation Approaches: Pandera vs Patito

Pandera offers a much more extensive and flexible system for column validation compared to Patito’s field-based approach. While Patito provides a solid set of built-in field constraints (like gt, le, regex, unique, etc.) that cover common validation scenarios, Pandera’s Check system is designed for both simple and highly sophisticated validation logic.

The key architectural difference seems to lie in extensibility and complexity. Pandera’s Check objects accept arbitrary functions, allowing you to write custom validation logic that can be as simple as lambda s: s > 0 or as complex as statistical hypothesis tests using scipy. You can create vectorized checks that operate on entire Series objects for performance, element-wise checks for atomic validation, and even grouped checks that validate subsets of data based on other columns. Patito’s Field constraints, while clean and declarative, are more limited to the predefined validation types that Pydantic and Patito provide.

Pandera also supports advanced validation patterns that Patito doesn’t directly offer, such as wide-form data checks (validating relationships across multiple columns), grouped validation (where checks are applied to subsets of data based on grouping columns), and the ability to raise warnings instead of errors for non-critical validation failures. While Patito does support custom constraints through Polars expressions via the constraints parameter, this requires knowledge of Polars expression syntax and, depending on where you’re coming from, could be less intuitive than Pandera’s function-based approach.

For most common validation scenarios, Patito’s field-based validation is simpler and more readable, especially for teams already familiar with Pydantic. However, for complex data validation requirements, statistical validation, or when you need maximum flexibility in defining validation logic, Pandera’s Check system provides significantly more power and extensibility.

Unique Strengths and When to Use

Pydantic-style model definitions with familiar syntax for Pydantic users
rich type system integration with Python’s typing system
model inheritance and composition for complex data structures
seamless integration with existing Pydantic-based applications
row-level object modeling for converting DataFrame rows to Python objects with methods
mock data generation for testing with .examples() method

People should choose Patito when they’re already using Pydantic in their applications and want consistent validation patterns across data processing and application logic. It’s great when you need to validate DataFrames and then work with individual rows as rich Python objects with embedded business logic and methods (e.g., a Product row that has a .url property or .calculate_discount() method). Patito is also good when you need to generate realistic test data and want object-oriented interfaces for their data models.

3. Pointblank: Comprehensive Validation with Beautiful Reports

Pointblank is a comprehensive data validation framework designed to make data quality assessment both thorough and accessible to stakeholders. Originally inspired by the R package of the same name, Pointblank’s primary mission is to provide validation workflows that generate beautiful, interactive reports that can be shared with both technical and non-technical team members.

Pointblank launched with Polars support as a core feature from its initial Python release in late 2024, built on top of the Narwhals and Ibis compatibility layers to provide consistent DataFrame operations across multiple backends including Polars, Pandas, and database connections.

Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


import pointblank as pb

schema = pb.Schema(
    columns=[("user_id", "Int64"), ("age", "Int64"), ("email", "String"), ("score", "Float64")]
)

validation = (
    pb.Validate(data=user_data, label="An example.", tbl_name="users", thresholds=(0.1, 0.2, 0.3))
    .col_vals_gt(columns="user_id", value=0)
    .col_vals_between(columns="age", left=18, right=80)
    .col_vals_regex(columns="email", pattern=r"^[^@]+@[^@]+\.[^@]+$")
    .col_vals_between(columns="score", left=0, right=100)
    .col_schema_match(schema=schema)
    .interrogate()
)

validation

		STEP	COLUMNS	VALUES	EVAL	UNITS	PASS	FAIL	W	E	C	EXT
Pointblank Validation
An example. PolarsusersWARNING0.1ERROR0.2CRITICAL0.3
#4CA64C	1	col_vals_gt()	user_id	0	✓	5	5 1.00	0 0.00	○	○	○	—
#EBBC14	2	col_vals_between()	age	[18, 80]	✓	5	4 0.80	1 0.20	●	●	○
#EBBC14	3	col_vals_regex()	email	^[^@]+@[^@]+\.[^@]+$	✓	5	4 0.80	1 0.20	●	●	○
#4CA64C	4	col_vals_between()	score	[0, 100]	✓	5	5 1.00	0 0.00	○	○	○	—
#4CA64C	5	col_schema_match()	—	SCHEMA	✓	1	1 1.00	0 0.00	○	○	○	—
2026-04-02 19:07:38 UTC< 1 s2026-04-02 19:07:38 UTC
Notes Step 5 (schema_check) ✓ Schema validation passed. Schema Comparison

Python on Posit Open Source

Recreating Septa Transit Timetables in Python

The final result

PDF Accessibility and Standards

What PDF Standards Do

How to enable a PDF Standard in Quarto

Creating accessible PDFs

If your document fails validation

Current limitations

LaTeX

Typst

Conclusion

Where Questions Become Queries: Meet querychat

Hello, querychat

Why this matters: reliability, transparency, reproducibility

How it works: tool calling

Customising querychat: from chat to toolkit

QueryChat object

Beyond chat: bespoke interfaces

Adding querychat to your existing Shiny app

Database options

For the curious: how does querychat know what to do?

Safety, control, and confidence

Other querychat apps in the wild

Shiny Talks from posit::conf(2025)

Shiny Talks Playlist:

Shiny + AI & Intelligent Automation

Shiny Architecture, Performance & Data

Shiny UX, UI Design & Accessibility

Shiny Extensions, Packages & Developer Tooling

Applied Shiny Case Studies

Shiny Workshops from posit::conf(2025)

LLMs + Shiny Workshop

Shiny for Python Workshop

Shiny for R Workshop

Closing

posit::conf(2025) Quarto talks

Quarto Extensions & Advanced Features

Workflow Automation & Reporting

Teaching & Education

Business, Collaboration & Publishing

Tool Calling UI in shinychat

Tool calling UI

Basic tool display

Setting up streaming

Customizing tool title and icon

Custom display content

Additional display options

Bookmarking support

Express mode

Core mode

Configuration options

Other improvements in shinychat for R

Better programmatic control

Custom assistant icons

Safer external links

Learn more

Acknowledgements

Python Open-Source Developer

posit::conf(2025) Quarto workshop materials

Branded Websites, Presentations, Dashboards, and PDFs with Quarto

Extending Quarto

Quarto Wizard 1.0.0: Democratising Quarto Extension Management

Quarto Wizard: Your GUI for Quarto extensions

Seamless IDE integration

Intelligent extension management

Template workflow simplified

Powered by a comprehensive extension directory

A curated catalogue of 250+ extensions

One-click installation from the web

Template discovery made easy

Addressing real workflow friction

Perfect for diverse use cases

Future-ready architecture

Getting started today

Quarto 1.8

Dark and light colors and logos in brand

Brand extensions

Accessibility checks for HTML

Accessing execution information

Customising `querychat`: from chat to toolkit

Installing `chatlas` or `ellmer`

`jupyter` vs `knitr`

Quick spanner creation with `tab_spanner_delim()`

Beautiful boolean formatting with `fmt_tf()`

Rotating column labels with `cols_label_rotate()`

Enhanced datetime formatting with `fmt_datetime()`