How do I set up a listing with ipynb files?

[minnerbe]

Description

I'm trying to set up a custom listing with a bunch of notebooks. The notebooks are stored as py:percent files, but because execution is expensive (and the notebooks with final state are interesting to me on their own) I've converted them to .ipynb files using

quarto render scripts/notebook.py -t ipynb

I've also set up a custom listing that should take all the .ipynb files as content (specified by a _quarto.yaml and an index.qmd to orchestrate everything), but wasn't able to get the result I want: If I do quarto render . I get WARN: File scripts/notebook.ipynb in the listing 'listing' contains no metadata.. The file shows up in the listing, but seems broken (not clickable).

I was suspecting that I either messed up the listing orchestration, or the notebook metadata. However, the following works:

• If I replace scripts/*.ipynb with scripts/*.py in the render and contents attributes of the orchestration files, the listing works as expected (but execution is triggered), suggesting that the listing orchestration is ok.
• If I render a single notebook with quarto render scripts/notebook.ipynb, the output also looks as expected, suggesting that the notebook metadata is ok.

I've reduced the problem to three ~10 line files and I'm attaching the .ipynb for convenience (click here):

├── _quarto.yml
├── index.qmd
└── scripts
    ├── notebook.ipynb
    └── notebook.py

I already checked other discussions here about missing metadata in .ipynb files, but the solutions there didn't seem to apply to my situation. I'm happy for any pointers about what I might be doing wrong. Thank you!

(Quarto 1.7.31 on MacOS 15.4.1)

[mcanouil]

Your Jupyter Notebooks are not valid for Quarto as stated by the error message: they do not have any metadata.

quarto render does not convert. It renders. There are discussion output using ipynb as a target format and keeping the metadata.
Here is one of them:

• format: ipynb but keep yaml header #11644

quarto convert can convert (keeping everything including metadata) from Quarto plain text document to/from Jupyter Notebooks (json).

The listing feature works the same way for qmd/ipynb as long as they have the proper metadata such as title, author, date, etc. that the listing requires.
So you need to ensure the field/metadata requirements from your custom listing (or default) are met by your ipynb.

[minnerbe]

Thanks for the quick reply! It seems that I've been having a misconception about what quarto render does as opposed to quarto convert. I understand now that quarto render is not what I should use to get a notebook with cell outputs.

The part that I'm still not understanding is this: Suppose I already have a notebook that contains a yaml header in the first markdown cell (see below):

• if I do quarto render scripts/notebook.ipynb, the resulting html looks as expected and the metadata is recognized correctly (title, author, category, ...);
• if I do quarto render . with the minimal listing setup in my example, the metadata is not recognized and no html output is generated for the notebook.

I think my problem is similar to #9987, but the solution there has been to add metadata, which I believe I now did. Thanks for your help!

The notebook (all cells except the first one are omitted):

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ed7826ec",
   "metadata": {},
   "source": [
    "---\n",
    "title: the metadata title\n",
    "author: some author\n",
    "categories:\n",
    "  - a category\n",
    "format: html\n",
    "jupyter: python3\n",
    "---"
   ]
  },
 <...>
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

Listing setup:

# _quarto.yaml
project:
  type: website
  render:
    - "index.qmd"
    - "scripts/notebook.ipynb"
    
website:
  title: "My Website"

format:
  html:
    theme:
      light: flatly
      dark: darkly
    toc: true
    
# index.qmd
---
title: "Rendering a list of notebooks"
listing:
  contents: "scripts/*.ipynb"
  type: default
  categories: true
  fields: [image, title, description, categories, filename, file-modified]
  sort-ui: [title, file-modified]
  filter-ui: true
---

[mcanouil]

I don't know what you are doing exactly, but let's take a simple example.

quarto create project blog issue12758 12758 --no-open --no-prompt
cd issue12758
quarto convert posts/welcome/index.qmd
rm posts/welcome/index.qmd
quarto convert posts/post-with-code/index.qmd
rm posts/post-with-code/index.qmd
quarto render

.
├── _quarto.yml
├── about.qmd
├── index.qmd
├── posts
│   ├── _metadata.yml
│   ├── post-with-code
│   │   ├── image.jpg
│   │   └── index.ipynb
│   └── welcome
│       ├── index.ipynb
│       └── thumbnail.jpg
├── profile.jpg
└── styles.css

4 directories, 10 files

This leads to a website with a listing showing the Jupyter Notebooks.

[minnerbe]

Thanks for working with me on this so patiently! You can see that I'm not super familiar with all of quarto's moving parts by my struggle to formulate a truly minimal problem.

Your example works great when used with .qmd files. I tried substituting posts/post-with-code/index.qmd with the following py:percent file:

# %% [markdown]
# ---
# title: "Post With Code"
# author: "Harlow Malloc"
# date: "2025-05-28"
# categories: [news, code, analysis]
# image: "image.jpg"
# format: html
# jupyter: python3
# ---

# %% [markdown]
# This is a post with executable code.

# %%
print("hello world")

quarto create project blog issue12758 12758 --no-open --no-prompt
cd issue12758
cat <contents above> > posts/post-with-code/index.py
rm posts/post-with-code/index.qmd
quarto render

This worked great and produced the same output as the .qmd-based version.

However, any attempt to convert the .py file to a .ipynb file fails to preserve the metadata:

• quarto convert index.py most likely interprets the file as .qmd, which produces a renderable but ill-formatted notebook.
• jupytext --to ipynb index.py && jupyter nbconvert --to notebook --execute --inplace index.ipynb leads to the error reported in my original post.

If I understand #588 correctly, it seems that quarto is able to render py:percent notebooks, but not convert them yet. I would appreciate any hint for going from py:percent to ipynb (with any number of intermediate representations) while preserving quarto metadata and thus making the resulting ipynb render just as the py:percent would!

[mcanouil]

quarto convert only do Quarto to Jupyter Notebook and the other way around as documented (quarto convert --help) and yes the issue is about extending the command to scripts:

• Quarto convert for script files (R, py) #588

[minnerbe]

Thank you, I appreciate your reply. While it's unfortunate that quarto can't convert py:percent files (yet), I now have an answer to my question in the sense that I know that my original workflow using py:percent files is not possible and I will look for alternatives.