Approaches for Supporting Paragraph Attributes in Pandoc AST

[Valgard]

Approaches for Supporting Paragraph Attributes in Pandoc AST

Problem Statement

Currently, Pandoc's AST supports attributes for various block elements (like Header, CodeBlock, and Div), but not for paragraphs. This inconsistency leads to information loss when converting between formats that support paragraph attributes (such as HTML with classes, IDs, or custom attributes on <p> tags).

This issue is part of a broader architectural consideration regarding attribute handling in Pandoc. Related discussions include:

• [Issue #684: "Permit adding attributes to all Markdown elements"](Permit adding attributes to all Markdown elements #684) which proposed a general solution for all elements
• [Discussion #9324: "From HTML to Markdown. How to get rid of HTML attributes and raw HTML code, but preserve comments"](From HTML to Markdown. How to get rid of HTML attributes and raw HTML code, but preserve comments #9324) which deals with the opposite problem of selectively removing attributes
• [Discussion #9526: "pandoc vs djot document model / footnotes"](pandoc vs djot document model / footnotes #9526) which compares the architectural approaches in Pandoc and Djot

Current Workarounds and Limitations

Currently, to preserve attributes on paragraphs, users must wrap them in Div elements. This creates:

1. Additional nesting that complicates document structure
2. Semantically inaccurate representations (attributes intended for paragraphs are applied to containing divs)
3. Inconsistent handling between paragraphs and other block elements

Potential Solutions

I see three main approaches to addressing this issue, each with different trade-offs:

1. Direct Attribute Extension

-- Current
Para :: [Inline] -> Block

-- Modified
Para :: Attr -> [Inline] -> Block

Pros:

• Minimal changes to AST structure
• Consistent with how other block elements handle attributes
• Relatively straightforward implementation

Cons:

• Doesn't address the broader inconsistency in attribute handling across the AST
• Still requires updating all readers, writers, and pattern matching throughout the codebase
• Doesn't follow the more elegant approach used in Djot.AST

2. Node-based Architecture (like Djot)

data Node a = Node Pos Attr a
type Blocks = [Node Block]

data Block
  = Para [Inline]
  | CodeBlock Text
  | Header Int [Inline]
  ...

Pros:

• Clean separation between content and metadata
• Consistent attribute handling for all element types
• More flexible and extensible architecture
• Aligns with approach used in Djot (as seen in [Discussion #9526](pandoc vs djot document model / footnotes #9526) and [Djot's AST implementation](https://github.com/jgm/djoths/blob/main/src/Djot/AST.hs#L95-L293))

Cons:

• Major breaking change affecting pandoc-types, pandoc, and dependent packages
• Requires rewriting filters and tools
• Significant migration effort for the ecosystem

3. Intermediate Solutions

There might be intermediate solutions that provide paragraph attribute support without massive architectural changes:

a) Wrapper Types with Backward Compatibility:

data ParaWithAttr = ParaWithAttr Attr [Inline]

-- Conversion functions to maintain backward compatibility
paraToParaWithAttr :: Para -> ParaWithAttr
paraWithAttrToPara :: ParaWithAttr -> Para

b) Extension Framework:
Creating an extension mechanism that allows attaching metadata to existing AST nodes without changing their structure.

Questions for Discussion

1. Which approach would be most acceptable to maintainers and the community?
2. Is the benefit of paragraph attributes worth the disruption of a major architectural change?
3. Are there other approaches I haven't considered that might provide a better balance of benefits vs. migration costs?
4. Would a phased approach be possible? (e.g., start with solution Including 'amssymb' in the LaTeX template #1, then migrate to Cannot omit libraries from install #2 in a future major release)

I'm willing to work on implementing whichever approach is deemed most appropriate, but wanted to gather thoughts before proceeding with development work.

Related Issues and Discussions

• [Issue #684: Permit adding attributes to all Markdown elements](Permit adding attributes to all Markdown elements #684)
• [Discussion #9324: From HTML to Markdown. How to get rid of HTML attributes and raw HTML code, but preserve comments](From HTML to Markdown. How to get rid of HTML attributes and raw HTML code, but preserve comments #9324)
• [Discussion #9526: pandoc vs djot document model / footnotes](pandoc vs djot document model / footnotes #9526)