KeyError: 'kernel_dirs' - while running quarto as singularity

[bhagesh-h]

Bug description

I am trying to run quarto via singularity, and received KeyError: 'kernel_dirs' when I run a quarto check and the same while running quarto render.

please find complete error below

Steps to reproduce

_metadata.yaml content

format:
  html:
    toc-title: |
      ![](images/logo.png)

      Contents
    css: css/styles.css
logo: images/logo.png
csl: csl/Diabetologia.csl
bibliography: references/ref.bib

report.qmd content

---
title: ' '
date: today
format:
  html:
    toc: true
    toc-location: left
    code-fold: true
    smooth-scroll: true
    self-contained: true
    number-sections: true
    anchor-sections: true
    embed-resources: true
jupyter: python3
filters:
   - /ext/quarto/_extensions/quarto-ext/include-code-files/include-code-files.lua
execute:
  freeze: false
  echo: false
  cache: false
---

```{python}
#| tags: [parameters]

version="Placeholder version"
project_ID="Placeholder project_ID"
pipeline="Placeholder pipeline name"
read_merging_metrics="Placeholder read_merging_metrics"
sequence_cleaning_metrics="Placeholder sequence_cleaning_metrics"
    ```
::: callout-note
## Project Details

Project ID: **`{python} project_id`**\
Date of Processing: **{{< meta date >}}**\
Pipeline: **`{python} pipeline`**\
Version: **`{python} version`**
:::

## References {.unnumbered}

::: {#refs}
:::

##  {.unnumbered .unlisted}

###### SOME TEXT {.unnumbered .unlisted .center}

Singularity command

singularity shell --bind /mnt/,/mnt/temp:/run/user/ quarto_v1_5_56.sif

quarto commands

export XDG_RUNTIME_DIR='/mnt/result/'
export QUARTO_PRINT_STACK=true

quarto check

quarto render /mnt/result/report.qmd \
--to html --output-dir quarto \
--metadata-file /mnt/result/_metadata.yaml \
--metadata title="Test Report" \
-P pipeline="Test Pipeline" \
-P project_id=12345 \
-P version=v1.1.1 \
-P sequence_cleaning_metrics=/mnt/result/sequence_cleaning_metrics.tsv \
-P read_merging_metrics=/mnt/result/read_merging_metrics.tsv      

Expected behavior

No response

Actual behavior

No response

Your environment

• Base Image OS for singularity Ubuntu:22.04
• Quarto v1.5.56
• Python 3.11

Quarto check output

Error:

Quarto 1.5.56

[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.2.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK

[✓] Checking versions of quarto dependencies......OK

[✓] Checking Quarto installation......OK
      Version: 1.5.56
      Path: /opt/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Tex:  (not detected)

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.2 (Conda)
      Path: /opt/conda/bin/python
      Jupyter: 5.7.2
      Kernels: python3

(\) Checking Jupyter engine render....2024-08-12 13:18:21,047 - traitlets - ERROR - /opt/conda/lib/python3.12/lib-dynload/_sqlite3.cpython-312-x86_64-linux-gnu.so: undefined symbol: sqlite3_deserialize
Traceback (most recent call last):
  File "/opt/conda/lib/python3.12/site-packages/traitlets/traitlets.py", line 632, in get
    value = obj._trait_values[self.name]
            ~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'kernel_dirs'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/manager.py", line 87, in wrapper
    out = await method(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/manager.py", line 435, in _async_start_kernel
    kernel_cmd, kw = await self._async_pre_start_kernel(**kw)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/manager.py", line 397, in _async_pre_start_kernel
    self.kernel_spec,
    ^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/manager.py", line 195, in kernel_spec
    self._kernel_spec = self.kernel_spec_manager.get_kernel_spec(self.kernel_name)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/kernelspec.py", line 282, in get_kernel_spec
    resource_dir = self._find_spec_directory(kernel_name.lower())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/kernelspec.py", line 256, in _find_spec_directory
    for kernel_dir in [kd for kd in self.kernel_dirs if os.path.isdir(kd)]:
                                    ^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/traitlets/traitlets.py", line 687, in __get__
    return t.cast(G, self.get(obj, cls))  # the G should encode the Optional
                     ^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/traitlets/traitlets.py", line 635, in get
    default = obj.trait_defaults(self.name)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/traitlets/traitlets.py", line 1897, in trait_defaults
    return t.cast(Sentinel, self._get_trait_default_generator(names[0])(self))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/kernelspec.py", line 195, in _kernel_dirs_default
    from IPython.paths import get_ipython_dir
  File "/opt/conda/lib/python3.12/site-packages/IPython/__init__.py", line 55, in <module>
    from .terminal.embed import embed
  File "/opt/conda/lib/python3.12/site-packages/IPython/terminal/embed.py", line 15, in <module>
    from IPython.core.interactiveshell import DummyMod, InteractiveShell
  File "/opt/conda/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 110, in <module>
    from IPython.core.history import HistoryManager
  File "/opt/conda/lib/python3.12/site-packages/IPython/core/history.py", line 10, in <module>
    import sqlite3
  File "/opt/conda/lib/python3.12/sqlite3/__init__.py", line 57, in <module>
    from sqlite3.dbapi2 import *
  File "/opt/conda/lib/python3.12/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: /opt/conda/lib/python3.12/lib-dynload/_sqlite3.cpython-312-x86_64-linux-gnu.so: undefined symbol: sqlite3_deserialize

/opt/conda/lib/python3.12/lib-dynload/_sqlite3.cpython-312-x86_64-linux-gnu.so: undefined symbol: sqlite3_deserialize

[✓] Checking Jupyter engine render....OK

ERROR: Error
    at renderFiles (file:///opt/quarto/bin/quarto.js:78081:29)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async render (file:///opt/quarto/bin/quarto.js:82929:21)
    at async checkJupyterRender (file:///opt/quarto/bin/quarto.js:101297:21)
    at async file:///opt/quarto/bin/quarto.js:101261:21
    at async withSpinner (file:///opt/quarto/bin/quarto.js:76250:16)
    at async checkJupyterInstallation (file:///opt/quarto/bin/quarto.js:101257:17)
    at async check (file:///opt/quarto/bin/quarto.js:101072:13)
    at async Command.actionHandler (file:///opt/quarto/bin/quarto.js:101389:5)
    at async Command.execute (file:///opt/quarto/bin/quarto.js:8017:13)

[mcanouil]

Thanks for the report

Thanks for report. Unfortunately, as is, we can't use it.

Note that we are asking a reproducible example because without it's very unlikely we'll be able to guess what is wrong.
For instance, you did not provide the content of the Quarto document. You shared the quarto check output for Quarto 1.5.56 but the sif image displays quarto_v1_4_553.sif.
Also, when trying to debug, you should try a small and simple document which is not the case based on the Quarto command you shared (see examples below).

Could you share a small self-contained "working" (reproducible) example to work with, i.e., a complete Quarto document or a Git repository? The goal is to make it as easy as possible for us to recreate your problem so that we can fix it: please help us help you! Thanks.

---

You can share a self-contained "working" (reproducible) Quarto document using the following syntax, i.e., using more backticks than you have in your document (usually four ````).
See https://quarto.org/bug-reports.html#small-is-beautiful-aim-for-a-single-document-with-10-lines.

If you have multiple files (and if it is absolutely required to have multiple files), please share as a Git repository.

R	Python
````qmd --- title: "Reproducible Quarto Document" format: html engine: knitr --- This is a reproducible Quarto document. {{< lipsum 1 >}} ```{r} x <- c(1, 2, 3, 4, 5) y <- c(1, 4, 9, 16, 25) plot(x, y) ``` {#fig-placeholder} {{< lipsum 1 >}} The end after @fig-placeholder. ````	````qmd --- title: "Reproducible Quarto Document" format: html engine: jupyter --- This is a reproducible Quarto document. {{< lipsum 1 >}} ```{python} import matplotlib.pyplot as plt x = [1, 2, 3, 4, 5] y = [1, 4, 9, 16, 25] plt.plot(x, y) plt.show() ``` {#fig-placeholder} {{< lipsum 1 >}} The end after @fig-placeholder. ````

Additionally and if not already given, please share the output of quarto check within a code blocks (i.e., using three backticks ```txt), see https://quarto.org/bug-reports.html#check.

[mcanouil]

For reference and information:

• Running quarto in singularity #2559

[bhagesh-h]

thank you @mcanouil , I have updated the query above with examples and corrected singularity version.

[cscheid]

Unfortunately, we can't really help you here, since we're not singularity users nor do we have the time to learn it.