Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pandoc failure during local rendering #1911

Closed
henrif75 opened this issue Mar 10, 2024 · 22 comments
Closed

Pandoc failure during local rendering #1911

henrif75 opened this issue Mar 10, 2024 · 22 comments
Labels
bug Something isn't working documentation Improvements or additions to documentation

Comments

@henrif75
Copy link
Collaborator

Following the build procedures listed in the README.md page results in pandoc failure:

mdbook serve
2024-03-10 09:56:02 [INFO] (mdbook::book): Book building has started
2024-03-10 09:56:02 [INFO] (mdbook::book): Running the exerciser backend
2024-03-10 09:56:02 [INFO] (mdbook::renderer): Invoking the "exerciser" renderer
2024-03-10 09:56:03 [INFO] (mdbook::book): Running the html backend
2024-03-10 09:56:04 [INFO] (mdbook::book): Running the pandoc backend
2024-03-10 09:56:04 [INFO] (mdbook::renderer): Invoking the "pandoc" renderer
[2024-03-10T16:56:07Z INFO  mdbook_pandoc] Processing redirects in [output.html.redirect]
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: why-rust/an-example-in-c.html => ../hello-world/example.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/why-rust/../hello-world/example.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-2/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-2/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-3/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-3/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: testing/mockall.html => ../android/testing/mockall.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/testing/../android/testing/mockall.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: traits/iterator.html => ../iterators/iterators.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/traits/../iterators/iterators.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-1/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-1/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: basic-syntax/static-and-const.html => ../unsafe-rust/static-and-const.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/basic-syntax/../unsafe-rust/static-and-const.html: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link 'comprehensive-rust.pdf' in chapter 'Welcome to Comprehensive Rust 🦀': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/comprehensive-rust.pdf: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'Compass': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'RTC Driver': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-10T16:56:07Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../borrowing/interior-mutability' in chapter 'Mutex': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/concurrency/shared_state/../../borrowing/interior-mutability: No such file or directory (os error 2)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/d907026615712cd13ce7e4e5023eb47711d8122f.svg: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/5180f1968cb395680acc12835345595cdedd9550.svgz: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
[WARNING] Could not convert image /private/var/folders/nk/38dlc4z527143grzs9b12cnw0034vl/T/tex2pdf.-6bf89262faae817a/aedd0ef191cf2e1f37d81b92916a869810082d7e.svgz: check that rsvg-convert is in path.
  rsvg-convert: createProcess: posix_spawnp: does not exist (No such file or directory)
lualatex not found. Please select a different --pdf-engine or install lualatex
pandoc exited unsuccessfully
2024-03-10 09:56:10 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-03-10 09:56:10 [ERROR] (mdbook::utils): Error: Rendering failed
2024-03-10 09:56:10 [ERROR] (mdbook::utils): 	Caused By: The "pandoc" renderer failed
@henrif75 henrif75 added the bug Something isn't working label Mar 10, 2024
@djmitche
Copy link
Collaborator

It looks like the issue is that rsvg-convert is not installed? I suspect installing that will fix the immediate issue, and perhaps an README update or the like is required to help the next person who encounters this.

I wonder if pandoc just has too many dependencies to expect folks to build locally. Maybe we should be suggesting building the PDF using a docker invocation of some sort?

@henrif75
Copy link
Collaborator Author

henrif75 commented Mar 11, 2024

I had to install librsvg and lualatex on Mac, but I'm getting another error now:

2024-03-11 14:27:54 [INFO] (mdbook::book): Book building has started
2024-03-11 14:27:55 [INFO] (mdbook::book): Running the exerciser backend
2024-03-11 14:27:55 [INFO] (mdbook::renderer): Invoking the "exerciser" renderer
2024-03-11 14:27:55 [INFO] (mdbook::book): Running the html backend
2024-03-11 14:27:56 [INFO] (mdbook::book): Running the pandoc backend
2024-03-11 14:27:56 [INFO] (mdbook::renderer): Invoking the "pandoc" renderer
[2024-03-11T21:27:57Z INFO  mdbook_pandoc] Processing redirects in [output.html.redirect]
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-1/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-1/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-3/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-3/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: why-rust/an-example-in-c.html => ../hello-world/example.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/why-rust/../hello-world/example.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: testing/mockall.html => ../android/testing/mockall.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/testing/../android/testing/mockall.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: basic-syntax/static-and-const.html => ../unsafe-rust/static-and-const.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/basic-syntax/../unsafe-rust/static-and-const.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: traits/iterator.html => ../iterators/iterators.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/traits/../iterators/iterators.html: No such file or directory (os error 2)
[2024-03-11T21:27:57Z WARN  mdbook_pandoc::preprocess] Failed to resolve redirect: exercises/day-2/soluções-tarde.html => solutions-afternoon.html: Unable to normalize redirect destination: Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/book/pandoc/pdf/src/exercises/day-2/solutions-afternoon.html: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link 'comprehensive-rust.pdf' in chapter 'Welcome to Comprehensive Rust 🦀': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/comprehensive-rust.pdf: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'Compass': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../comprehensive-rust-exercises.zip' in chapter 'RTC Driver': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/exercises/bare-metal/../../comprehensive-rust-exercises.zip: No such file or directory (os error 2)
[2024-03-11T21:28:00Z WARN  mdbook_pandoc::preprocess] Unable to normalize link '../../borrowing/interior-mutability' in chapter 'Mutex': Unable to canonicalize path: /Users/henrif/Documents/comprehensive-rust/src/concurrency/shared_state/../../borrowing/interior-mutability: No such file or directory (os error 2)
	Error producing PDF.
!  ==> Fatal error occurred, no output PDF file produced!

pandoc exited unsuccessfully
2024-03-11 14:28:27 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-03-11 14:28:27 [ERROR] (mdbook::utils): Error: Rendering failed
2024-03-11 14:28:27 [ERROR] (mdbook::utils): 	Caused By: The "pandoc" renderer failed

It worked once, then no more.
Could we have some sort of config to turn off the PDF generation?

@henrif75 henrif75 added documentation Improvements or additions to documentation bug Something isn't working and removed bug Something isn't working labels Mar 11, 2024
@mgeisler
Copy link
Collaborator

@max-heller, do you have experience with building the course on Mac?

@henrif75, you should be able to turn this off by removing the mdbook-pandoc binary you have installed. I get

2024-03-24 21:04:55 [WARN] (mdbook::renderer): The command `mdbook-pandoc` for backend `pandoc` was not found, but was marked as optional.

on my machine because I haven't installed it here.

@max-heller
Copy link
Collaborator

max-heller commented Apr 6, 2024

@max-heller, do you have experience with building the course on Mac?

Yes, I primarily develop on a Mac.

@henrif75 could you try enabling verbose logging for Pandoc in book.toml and see what it spits out?

[output.pandoc.profile.pdf]
output-file = "comprehensive-rust.pdf"
pdf-engine = "lualatex"
verbosity = "INFO" # ADD THIS LINE

@mgeisler mgeisler changed the title Pandoc failure during Pandoc failure during local rendering Apr 7, 2024
@henrif75
Copy link
Collaborator Author

henrif75 commented Apr 8, 2024

Here is the output:
mdbook.log.gz

@max-heller
Copy link
Collaborator

Here is the output:
mdbook.log.gz

Do you have Noto fonts installed? The end of the log suggests missing fonts might be the fatal error:

  luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: Font "NotoSansMonoCJKSC" not found.
  luaotfload | resolve : sequence of 3 lookups yielded nothing appropriate....texmf-dist/tex/luatex/luaotfload/luaotfload-fallback.lua:50: attempt to inde
  x a nil value (local 'f').
  <to be read again> 
  relax 
  l.154 \begin{document}
                      
   515 words of node memory still in use:
     5 hlist, 1 vlist, 2 rule, 1 dir, 4 kern, 1 glyph, 7 attribute, 60 glue_spec,
   7 attribute_list, 3 if_stack, 1 write, 1 user_defined, 1 pdf_colorstack nodes
     avail lists: 1:1,2:10,3:3,4:1,5:7,7:2,9:4
  !  ==> Fatal error occurred, no output PDF file produced!
  Transcript written on input.log.
Error producing PDF.
!  ==> Fatal error occurred, no output PDF file produced!

pandoc exited unsuccessfully
2024-04-07 17:51:17 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-04-07 17:51:17 [ERROR] (mdbook::utils): Error: Rendering failed
2024-04-07 17:51:17 [ERROR] (mdbook::utils): 	Caused By: The "pandoc" renderer failed

I have the following installed through homebrew, although I'm not sure they're all needed:

$ brew list --casks | grep noto
font-noto-color-emoji
font-noto-emoji
font-noto-sans
font-noto-sans-cjk
font-noto-sans-math
font-noto-sans-symbols
font-noto-serif
font-noto-serif-cjk
font-noto-serif-cjk-tc

@henrif75
Copy link
Collaborator Author

henrif75 commented Apr 9, 2024

It worked when I installed the fonts, but I couldn't find a casket so I manually installed them.
Would you have the brew command to fetch these fonts? I'll update the documentation.

@max-heller
Copy link
Collaborator

It worked when I installed the fonts, but I couldn't find a casket so I manually installed them. Would you have the brew command to fetch these fonts? I'll update the documentation.

brew tap homebrew/cask-fonts (see here)

@kashopi
Copy link

kashopi commented May 27, 2024

In my Silicon laptop I fixed it by using "tectonic" as a PDF renderer (changing it in the book.toml file) and installing the Noto fonts

@max-heller
Copy link
Collaborator

In my Silicon laptop I fixed it by using "tectonic" as a PDF renderer (changing it in the book.toml file) and installing the Noto fonts

tectonic simplifies a lot of dependency wrangling, but If I'm remembering correctly it doesn't support all of the font shenanigans necessary to render multilingual books, so unfortunately comprehensive rust probably can't switch over to it.

google/mdbook-i18n-helpers#200 is trying to figure out a way to skip running optional = true renderers when they aren't needed, which will prevent failures like these

@letesdev
Copy link

letesdev commented Jun 4, 2024

Hi, I hope you can help me 😄

I am trying to build the book in order to contribute with translating to spanish, but I am facing some errors... I am using a Windows 10 machine and I followed your README.md, CONTRIBUTING.md and TRANSLATIONS.md guides, but still face some errors.

Here are all my steps :

  1. I installed Rust by following the instructions on https://rustup.rs/.
  2. I cloned this repository
  3. I installed the necessary tools (mdbook, mdbook-svgbob, mdbook-i18n-helpers, mdbook-exerciser, and mdbook-course) using cargo.
  4. I installed pandoc from Pandoc
  5. I installed mdbook-pandoc using cargo, which is not mentioned in the README.md guide.
  6. I installed the Noto Fonts from Google Fonts.
  7. I installed MikTex to get lualatex from MiKTeX.

Now I am able to build the english version of book with mdbook build, but I can't build the spanish version with MDBOOK_BOOK__LANGUAGE=es mdbook build -d book/es. Here's the output:
mdbook_build_es_output1.txt

pandoc: C:\Users\s023793\letesdev\comprehensive-rust\book\es\pandoc\pdf\comprehensive-rust.pdf: withBinaryFile: permission denied (Permission denied)
pandoc exited unsuccessfully
2024-06-04 14:50:33 [ERROR] (mdbook::renderer): Renderer exited with non-zero return code.
2024-06-04 14:50:33 [ERROR] (mdbook::utils): Error: Rendering failed
2024-06-04 14:50:33 [ERROR] (mdbook::utils):    Caused By: The "pandoc" renderer failed

@max-heller
Copy link
Collaborator

5. I installed mdbook-pandoc using cargo, which is not mentioned in the README.md guide.

mdbook-pandoc is marked as optional, so if you uninstall it you should be able to build the book without it. max-heller/mdbook-pandoc#93 will soon provide a way to disable mdbook-pandoc outside of CI to avoid having to install its dependencies locally.

pandoc: C:\Users\s023793\letesdev\comprehensive-rust\book\es\pandoc\pdf\comprehensive-rust.pdf: withBinaryFile: permission denied (Permission denied)
pandoc exited unsuccessfully

This is strange, are there restrictive permissions on any of the parent directories of this path?

@letesdev
Copy link

letesdev commented Jun 5, 2024

Hi max-heller, thanks for the answer 😄

  • Are you sure about mdbook-pandoc being optional? Maybe it is possible to build the book without it once max-heller/mdbook-pandoc#93 is implemented, but here's an output of mdbook build before installing it (mdbook_build_output.txt):

    2024-06-04 11:35:24 [WARN] (mdbook::renderer): The command `mdbook-pandoc` for backend `pandoc` was not found, but was marked as optional.
  • Regarding the issue while building the spanish version: no, there aren't any restrictive permissions on any of the parent directories... you can actually tell by the build of the english version, right ? If I am able to build the english version, shouldn't I be able to build the spanish one ?

@max-heller
Copy link
Collaborator

2024-06-04 11:35:24 [WARN] (mdbook::renderer): The command `mdbook-pandoc` for backend `pandoc` was not found, but was marked as optional.

This is a warning, not an error, and shouldn't impact the rest of the build.

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

@letesdev
Copy link

letesdev commented Jun 6, 2024

This is a warning, not an error, and shouldn't impact the rest of the build.

Well, as you can see in the output file (having stderr and stout outputs), the build stops there without any other message... Let me know if you need more details 😄

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

Great, thank you so much! 😄

@max-heller
Copy link
Collaborator

Well, as you can see in the output file (having stderr and stout outputs), the build stops there without any other message... Let me know if you need more details 😄

At that point, the build has completed successfully. mdbook-pandoc is the last renderer that gets run, so if it's skipped, nothing else is expected. You can verify this by e.g. commenting all of the pandoc configuration in book.toml.

@max-heller
Copy link
Collaborator

If I am able to build the english version, shouldn't I be able to build the spanish one ?

You should be able to, yes. I'll look into it some more

Great, thank you so much! 😄

Seems like the error can occur for a variety of reasons (see e.g. here). Do you happen to have the Spanish PDF open while building? An antivirus that might be blocking Pandoc? A network drive/OneDrive setup? Have you tried wiping out the book directory and trying again?

@letesdev
Copy link

I am finally able to build the english and the spanish book! Thank you so much @max-heller !
I tried commenting all of the pandoc configuration in the book.toml file as you mentioned, and, just in case, I wiped out the book/ directory and closed OneDrive...

However, I found many of the paragraphs in english in the spanish version are already translated... for instance, after formatting the po/es.po file with dprint fmt po/es.po command, I am able to modify the paragraph in the index.html (I added several 's'
to 'Basándonos' just for showing) but not the list :

image

Do you have a clue in this, by the way ?

@max-heller
Copy link
Collaborator

I am finally able to build the english and the spanish book! Thank you so much @max-heller ! I tried commenting all of the pandoc configuration in the book.toml file as you mentioned, and, just in case, I wiped out the book/ directory and closed OneDrive...

Happy to hear it! Just curious, if you reenable the pandoc configuration in book.toml, do you get the same error as before?

However, I found many of the paragraphs in english in the spanish version are already translated... for instance, after formatting the po/es.po file with dprint fmt po/es.po command, I am able to modify the paragraph in the index.html (I added several 's'
to 'Basándonos' just for showing) but not the list :
Do you have a clue in this, by the way ?

I'm not familiar with the translation infrastructure, this seems like a good question for a separate issue to keep this one on topic

@letesdev
Copy link

Just curious, if you reenable the pandoc configuration in book.toml, do you get the same error as before?
I tried several builds:

@max-heller
Copy link
Collaborator

Thanks for testing! This is a specific issue that should be easy to fix, I've opened max-heller/mdbook-pandoc#96 to track it

mgeisler pushed a commit that referenced this issue Jul 18, 2024
)

Implements the approach summarized in
google/mdbook-i18n-helpers#200 (comment)
to disable running `mdbook-pandoc` even if it is available to avoid
dependency-related issues during local rendering.

Should address #1911
since `mdbook-pandoc` will no longer run locally.
@mgeisler
Copy link
Collaborator

I think we can close this now": We no longer run mdbook-pandoc by default and so people should not see local errors unless they opt-in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

6 participants