PDF-1.5 Image insertion - Pandoc should use correct /MediaBox dimensions/ratio when exporting to .docx, or output warning

[sladen]

For PDF-1.5 files with compressed trailer and compressed object streams (/ObjStm); all other PDF software including qpdf and pdfinfo use the correct page dimensions read from /MediaBox:

$ pdfinfo 'x.pdf' | grep pts
Page size:       596 x 842 pts (A4)

However, when using Pandoc and outputting to .docx:

$ echo '![Some PDF Image](x.pdf)' > x.md &&
   pandoc x.md -o x.docx

the image dimensions are incorrectly detected, and the image is inserted into the .docx with incorrect aspect ratio and size.

Manually walking through the PDF objects using qpdf shows a compressed Cross-reference (xref) and compressed object streams (\ObjStm) in use. A compressed object stream has multiple objects in a single /Flate compression stream:

$ for object in trailer 18 40 1 2 ; do qpdf --show-object=$object x.pdf -- ; done
<< /Filter /FlateDecode /Info 39 0 R /Length 155 /Root 40 0 R /Size 43 /Type /XRef /W [ 1 2 2 ] >>
% Object is stream.  Dictionary:
<< /Filter /FlateDecode /First 17 /Length 41 0 R /N 3 /Type /ObjStm >>
<< /Pages 1 0 R /Type /Catalog >>
<< /Count 2 /Kids [ 2 0 R 12 0 R ] /Type /Pages >>
<< /Contents 4 0 R /Group << /CS /DeviceRGB /I true /S /Transparency /Type /Group >> /MediaBox [ 0 0 596 842 ] /Parent 1 0 R /Resources 3 0 R /Type /Page >>

The /Page object with its /MediaBox[left bottom right top] is there, but only obtainable by (fully) walking/parsing/unpacking the PDF file.

These PDF-1.5 files in question are not particularly exotic, they are PDF-1.5 files printed from Firefox, via Cairo:

obj 39 0
<<
  /CreationDate (D:20240611224431+02'00)
  /Creator (Mozilla Firefox 126.0)
  /Producer (cairo 1.17.4 \(https://cairographics.org\))
>>

(See previous issue "Wrong image size for pdf images in docx" #4322, newly opened here per request of @jgm )

[jgm]

Well that explains it. Here is how we currently extract the PDF size (in T.P.ImageSize):

pdfSize :: ByteString -> Maybe ImageSize
pdfSize img =
  case A.parseOnly pPdfSize img of
    Left _   -> Nothing
    Right sz -> Just sz

pPdfSize :: A.Parser ImageSize
pPdfSize = do
  A.skipWhile (/='/')
  A.char8 '/'
  (do A.string "MediaBox"
      A.skipSpace
      A.char8 '['
      A.skipSpace
      [x1,y1,x2,y2] <- A.count 4 $ do
        A.skipSpace
        raw <- A.many1 $ A.satisfy (\c -> isDigit c || c == '.')
        case safeRead $ T.pack raw of
          Just (r :: Double) -> return $ floor r
          Nothing            -> mzero
      A.skipSpace
      A.char8 ']'
      return $ ImageSize{
              pxX  = x2 - x1
            , pxY  = y2 - y1
            , dpiX = 72
            , dpiY = 72 }
   ) <|> pPdfSize

That doesn't work if the size info is in a compressed object stream.

We could consider depending on a PDF parsing library, if there is an appropriate one on Hackage. (Though this was a nice dependency-free solution.)

There is https://hackage.haskell.org/package/pdf-toolbox-document but it adds quite a bit to our dependency tree.

[sladen]

nb. to self, code snippet above is found in:

• https://github.com/jgm/pandoc/blob/main/src/Text/Pandoc/ImageSize.hs#L296

@jgm: is it necessary to understand the ImageSize? For .docx the PDF always first gets rasterised to a bitmap by (some external process?). So what about first allowing the rasterisation/rendering to happen, and then fetching the dimensions (most importantly, the ratio) from that returned bitmap?

[sladen]

Since there is probably already a dependency on imagemagick for scaling/conversion, one could theoretically use IM's identify -ping mechanism:

$ identify -ping -format '%w %h\n' x.jpg
2604 3812

Unfortunately, using that on a PDF fails:

$ identify -ping -format '%w %h\n' x.pdf
identify-im6.q16: attempt to perform an operation not allowed by the \
security policy `PDF' @ error/constitute.c/IsCoderAuthorized/426.

Likewise trying to fetch /MediaBox (pdf:HiResBoundingBox) directly:

identify -format "%[pdf:HiResBoundingBox]" x.pdf 
identify-im6.q16: attempt to perform an operation not allowed by the \
security policy `PDF' @ error/constitute.c/IsCoderAuthorized/426

More hunting, this was added because of 2018 vulnerability in GhostScript:

• https://www.kb.cert.org/vuls/id/332928/

Patching the security policy in /etc/ImageMagick-6/policy.xml

--- /etc/ImageMagick-6/policy.xml~	2023-03-30 17:45:39.000000000 +0200
+++ /etc/ImageMagick-6/policy.xml	2025-06-11 11:31:06.725751421 +0200
@@ -97 +97 @@
-  <policy domain="coder" rights="none" pattern="PDF" />
+  <policy domain="coder" rights="read" pattern="PDF" />

Allows success (note, this PDF has two pages), but these are very slightly scaled compared to the actual /MediaBox values returned under Page geometry:

$ identify -ping -format '%w %h\n' pdfs/x.pdf 
612 828
612 828
$ identify x.pdf
x.pdf[0] PDF 612x828 612x828+0+0 16-bit sRGB 2707B 0.000u 0:00.000
x.pdf[1] PDF 612x828 612x828+0+0 16-bit sRGB 2538B 0.000u 0:00.000
$ identify -verbose pdfs/x.pdf | grep 'Page geometry:'
  Page geometry: 596x842+0+0
  Page geometry: 596x842+0+0

[jgm]

For .docx the PDF always first gets rasterised to a bitmap by (some external process?).

No, it is not. Docx can handle pdf.

And requiring a patched ghostscript is not a viable option.

[sladen]

(Imagemagick identify -ping bug reported at: ImageMagick/ImageMagick6#373 - identify -ping always seems to be returning hardcoded dimensions of an US 8.5x11.5 piece of paper, regardless of the input PDF)

[jgm]

Here is what would be added to the dependency tree if we used pdf-toolbox:

 - byteable-0.1.1 (lib) (requires download & build)
 - cryptonite-0.30 (lib) (requires download & build)
 - entropy-0.4.1.11 (lib:entropy) (requires download & build)
 - zlib-bindings-0.1.1.5 (lib) (requires download & build)
 - securemem-0.1.10 (lib) (requires download & build)
 - cryptohash-0.11.9 (lib) (requires download & build)
 - crypto-api-0.13.3 (lib:crypto-api) (requires download & build)
 - io-streams-1.5.2.2 (lib) (requires download & build)
 - crypto-cipher-types-0.0.9 (lib) (requires download & build)
 - cipher-aes-0.2.11 (lib) (requires download & build)
 - cipher-rc4-0.1.4 (lib) (requires download & build)
 - pdf-toolbox-core-0.1.3 (lib) (requires download & build)
 - pdf-toolbox-content-0.1.2 (lib) (requires download & build)
 - pdf-toolbox-document-0.1.4 (lib) (requires download & build)

Quite a heavy addition.

[jgm]

Can you upload one of the problematic PDFs?

[sladen]

empty.txt.pdf

An Empty PDF-1.7 created with:

$ cat > empty.txt
$ firefox empty.txt
Ctrl-p
[Save]

[jgm]

It's a bit hard to tell from that, because there's no image to look at in the resulting docx.

[jgm]

I tried adding some code to deflate streams as we go through the PDF, but in this PDF it is only deflating the first two streams; on the third I get an error "compressed data stream format error (invalid distance too far back)." Not sure why.

[sladen]

In the /XRef object, the array /W needs evaluating to make use of the decoded binary data:

<< /Type /XRef
   /Length 58
   /Filter /FlateDecode
   /Size 14
   /W [1 2 2]
  …
>>
stream

This gives the field widths of the table (1 byte, 2 bytes, 2 bytes). Source: PDF Specification (2008), pages 48‒50:

• https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/PDF32000_2008.pdf

[sladen]

If this is a different file, the /XRef stream may also have been pre-processed with one of the PNG /Predictor functions (eg. delta encoded against the previous line). Those are noted in the PDF Specification, pages 28‒29.

[jgm]

It's the same file you sent. Sorry, I don't know what to do with this information. The error I get is in decompressing the stream. Does the W parameter tell us to modify the bytes in the stream somehow before we decompress them?