# DOM API: Modifying the HTML Document Before Conversion

With the v42 layout engine, PD4ML gives Java code access to the HTML document it has parsed: the DOM API. Between `readHTML()` and the output (`writePDF()`, `writeRTF()`, `writeDOCX()`, `renderAsImages()`), the document can be read, queried with CSS selectors and changed, and the output is laid out as if the HTML had been written that way. A second part of the API, `layout()`, lays the document out into pages without writing it and tells where each element ended up, so code can check a layout and adjust the document before the real output.

No external library is involved: the API is a thin layer over PD4ML's own document tree, in the package `com.pd4ml.dom`.

{% hint style="version" %}
The DOM API and `layout()` are available from **PD4ML v4.2.0**, with the v42 layout engine only: call `useHtmlRenderer(Constants.LayoutEngine.v42)` before `readHTML()`. With the v40 engine, `getDocument()` and `layout()` throw an `IllegalStateException`, as they do when called before `readHTML()`.
{% endhint %}

## Typical uses

* **Filling a template.** Put values into an HTML template by id or selector instead of building the HTML as a string.
* **Cleaning up third-party HTML.** Remove navigation, cookie banners or tracking pixels; add missing `alt` texts; fix links.
* **Adding content.** Insert a cover note, a generated table, or a style sheet with `@page` rules from code.
* **Checking the layout.** Find out how many pages a document takes, whether a block is split across pages, or where an element is on its page, and react before writing.

## Getting the document

```java
import com.pd4ml.Constants;
import com.pd4ml.PD4ML;
import com.pd4ml.dom.Document;
import com.pd4ml.dom.Element;

PD4ML pd4ml = new PD4ML();
pd4ml.useHtmlRenderer(Constants.LayoutEngine.v42);
pd4ml.readHTML(new URL("https://example.com/invoice.html"));

Document doc = pd4ml.getDocument();

doc.getElementById("total").setTextContent("1 250,00 EUR");
for (Element img : doc.querySelectorAll("img:not([alt])")) {
    img.setAttribute("alt", "");
}

pd4ml.writePDF(new FileOutputStream("invoice.pdf"));
```

`getDocument()` returns the same tree on every call: changes made through one `Document` object are seen through any other, and by every later output.

The objects of the API are views of the tree's nodes. Two objects for the same node are `equals()` and have the same `hashCode()`, but are not necessarily the same object: compare nodes with `equals()`, not `==`. They can be used as keys of a `HashMap` or members of a `HashSet`.

## The node types

All classes are in `com.pd4ml.dom` and extend `Node`:

| Class | Is | `getNodeName()` |
|---|---|---|
| `Document` | the document itself, the root of the tree | `#document` |
| `DocumentType` | the `<!DOCTYPE>`: `getName()`, `getPublicId()`, `getSystemId()` | `#doctype` |
| `Element` | an element | its tag name, lower case (`div`, `td`, `svg`) |
| `Text` | a run of text: `getData()`, `setData()`, `isWhitespace()` | `#text` |
| `Comment` | an HTML comment: `getData()`, `setData()` | `#comment` |

`Document` also describes the document as a whole:

| Method | Returns |
|---|---|
| `getDocumentElement()` | the root element, `<html>` |
| `getHead()`, `getBody()` | the `<head>` and `<body>` elements |
| `getDoctype()` | the `DocumentType`, or null when the source has none |
| `isQuirksMode()` | whether the document is laid out in quirks mode (no doctype, or an old one) |
| `getBaseURI()` | the base URI relative links are resolved against |
| `getTitle()` | the text of `<title>` |

## Navigating the tree

Every node has:

| Method | Returns |
|---|---|
| `getParentNode()`, `getParentElement()` | the parent node; the parent if it is an element |
| `getOwnerDocument()` | the document the node belongs to |
| `getChildNodes()`, `hasChildNodes()` | the children (all node types), in document order |
| `getFirstChild()`, `getLastChild()` | the first and last child |
| `getNextSibling()`, `getPreviousSibling()` | the neighbours |
| `contains(other)` | whether `other` is this node or inside it |
| `getTextContent()` | the text of the node and everything in it |

An element also has element-only navigation, which skips text and comments:

| Method | Returns |
|---|---|
| `getChildren()` | the child elements |
| `getFirstElementChild()`, `getLastElementChild()` | the first and last child element |
| `getNextElementSibling()`, `getPreviousElementSibling()` | the neighbouring elements |

Lists returned by the API are snapshots: they do not change when the document changes afterwards. It is therefore safe to remove or move the nodes of a list while going through it.

## Querying

| Method | Returns |
|---|---|
| `getElementById(id)` | the first element with the id, or null (on `Document`) |
| `getElementsByTagName(name)`, `getElementsByClassName(names)` | the elements with the tag name (`"*"` for all); the elements that have all the space-separated classes - in document order |
| `querySelector(selectors)`, `querySelectorAll(selectors)` | the first element matching a CSS selector list, or null; all elements matching it, in document order |
| `matches(selectors)`, `closest(selectors)` | whether the element matches; the element itself or its nearest ancestor that matches, or null (on `Element`) |

The searches work on the `Document` and on an `Element`; on an element, they look only inside it.

The selectors are those of PD4ML's CSS engine, so anything that works in a style sheet works here: type, class, id and attribute selectors (`[href^="http"]`, `[data-x~="y"]`), the combinators (descendant, `>`, `+`, `~`), structural pseudo-classes (`:root`, `:first-child`, `:nth-child(2n+1)`, `:nth-last-child()`, `:nth-of-type()`, `:last-of-type`, `:only-child`, `:empty`), `:not()`, `:is()`, `:where()`, `:has()`, `:lang()`, `:dir()`, `:checked`, `:any-link`, and selector lists (`h1, h2, h3`). An invalid selector throws an `IllegalArgumentException`.

```java
// every price cell in the last column
List<Element> prices = doc.querySelectorAll("table.prices td:last-child");

// the table row a cell is in
Element row = cell.closest("tr");

// sections without a heading
for (Element s : doc.querySelectorAll("section:not(:has(h1, h2, h3))")) {
    s.addClass("untitled");
}
```

## Reading elements

| Method | Returns |
|---|---|
| `getTagName()` | the tag name, lower case |
| `getAttribute(name)`, `hasAttribute(name)` | an attribute's value (null if absent); whether it is there |
| `getAttributes()` | all attributes, in source order |
| `getId()`, `getClassName()` | the `id` and `class` attributes |
| `getClassList()`, `hasClass(name)` | the classes; whether one is set |
| `getTextContent()` | the text inside |
| `getInnerHTML()`, `getOuterHTML()` | the HTML of its content; of the element itself |

Attribute names are case-insensitive, as in HTML. Values are returned as they are in the document, with entities decoded; relative URLs are not resolved (`doc.getBaseURI()` is the base to resolve them against).

## Changing the document

### Creating and inserting nodes

```java
Element note = doc.createElement("p");
note.setClassName("note");
note.setTextContent("Prices include VAT.");
doc.getBody().insertBefore(note, doc.querySelector("table.prices"));

Element footer = doc.createElement("div");
footer.appendChild(doc.createTextNode("Generated " + LocalDate.now()));
footer.appendChild(doc.createComment("generated by the invoice service"));
doc.getBody().appendChild(footer);
```

| Method | Does |
|---|---|
| `doc.createElement(tag)`, `doc.createTextNode(text)`, `doc.createComment(text)` | a new node, not yet in the tree |
| `appendChild(node)` | adds the node as the last child |
| `insertBefore(node, ref)` | inserts before the child `ref` (`ref` null: at the end) |
| `replaceChild(replacement, old)` | replaces the child `old`, and returns it detached |
| `removeChild(child)`, `remove()` | detaches a child; detaches the node itself |
| `cloneNode(deep)` | a copy, not yet in the tree; with `deep`, its content too |

The rules are those of the browser DOM:

* A node that is already in the tree is **moved** when it is inserted somewhere else. A node from another PD4ML document is moved into this one.
* A node cannot be inserted into itself or into one of its descendants, a document cannot be inserted at all, and text, comments and the doctype cannot have children. These throw an `IllegalArgumentException`, as does a `ref` or `old` that is not a child of the node.
* A detached node can be kept and inserted again later.

### Text

`setTextContent(text)` replaces everything inside an element with one text node. The text is taken as text, not HTML: `<` stays a `<` character. On a `Text` or `Comment` node it sets the node's data.

### Attributes and classes

| Method | Does |
|---|---|
| `setAttribute(name, value)`, `removeAttribute(name)` | sets or removes an attribute |
| `setId(id)`, `setClassName(names)` | sets `id` / `class` |
| `addClass(name)`, `removeClass(name)` | adds or removes one class |
| `toggleClass(name)` | adds the class if it is not there, removes it otherwise; returns whether it is there now |

The document is read again when it is laid out, so a changed `style` attribute, class, `colspan`, `width` or `src` takes effect in the next output exactly as if the source had had it:

```java
Element totals = doc.getElementById("totals");
totals.setAttribute("style", "page-break-inside: avoid; border-top: 2px solid black");
doc.querySelector("img.logo").setAttribute("src", "https://example.com/logo-print.png");
```

Images added or changed are loaded when the document is laid out.

### Inserting HTML

```java
doc.querySelector("tbody#items").setInnerHTML(
        "<tr><td>Consulting</td><td>8 h</td></tr>"
      + "<tr><td>Travel</td><td>1</td></tr>");

doc.getElementById("legacy-banner").setOuterHTML("<p class='notice'>See the attached terms.</p>");
```

* `setInnerHTML(html)` replaces the element's content with the HTML, parsed the way a browser parses `innerHTML`: as the content of an element of that tag. Table rows can be set into a `<tbody>` or `<tr>`, options into a `<select>`, list items into a `<ul>`.
* For `<script>`, `<style>`, `<textarea>` and `<title>`, the string is set as text, as is. This is the way to replace a style sheet.
* `setOuterHTML(html)` replaces the element itself with the nodes the HTML parses into, and detaches the element. It throws an `IllegalStateException` if the element has no parent element.

### Adding style sheets

A `<style>` element added from code applies like one in the source, `@page` rules included (see [CSS @page Rules](css-page-rules-page-size-margins-named-pages-and-margin-boxes)):

```java
Element style = doc.createElement("style");
style.setTextContent(
        "@page { size: A4 landscape; margin: 15mm;"
      + "  @bottom-center { content: 'Page ' counter(page) ' of ' counter(pages) } }"
      + ".draft { display: none }");
doc.getHead().appendChild(style);
```

A `<link rel="stylesheet" href="...">` added to `<head>` is loaded when the document is laid out.

## Writing HTML back out

| Method | Returns |
|---|---|
| `doc.toHtml()` | the whole document, doctype included |
| `element.getOuterHTML()`, `node.toHtml()` | the HTML of the node itself |
| `element.getInnerHTML()` | the HTML of its content |

Void elements (`<br>`, `<img>`, ...) are written without an end tag, script and style content as is, text and attribute values escaped. This is useful to log or archive the document as it was converted:

```java
Files.write(Paths.get("invoice-as-converted.html"), doc.toHtml().getBytes(StandardCharsets.UTF_8));
```

## Inline SVG and custom tags

An inline `<svg>` and a custom tag (MathML `<math>`, `<qrcode>`, and tags registered with `addCustomTagHandler()`) are drawn from their markup, not from child nodes. In the DOM they have no child nodes: their content is read and changed as HTML with `getInnerHTML()`/`setInnerHTML()`, or the whole element is replaced with `setOuterHTML()`. Their attributes are ordinary attributes:

```java
Element logo = doc.querySelector("svg#logo");
logo.setInnerHTML("<circle cx='50' cy='50' r='40' fill='teal'/>");
logo.setAttribute("width", "120");
```

Inserting a node into such an element throws an `IllegalArgumentException`.

## Reading the layout

`pd4ml.layout()` lays the document out into pages as `writePDF()` would, without writing it, and returns a `Layout`: where each element is, on which page, and its computed style.

```java
import com.pd4ml.dom.Layout;
import com.pd4ml.dom.LayoutBox;

Layout layout = pd4ml.layout();

System.out.println(layout.getPageCount() + " pages");
for (LayoutBox box : layout.getBoxes(doc.getElementById("signature"))) {
    System.out.println(box);   // page 2: 56.69, 640.10, 481.89 x 85.00 pt
}
```

| Method | Returns |
|---|---|
| `getPageCount()` | the number of pages |
| `getPageWidth(page)`, `getPageHeight(page)` | a page's size, in points (page 1 is the first) |
| `getPageNumber(page)` | the number printed on the page (`$[page]`); `setInitialPageNumber()` and page counter resets can make it differ from the page's position |
| `getBoxes(element)` | the element's boxes, in page order; empty if it is not displayed (`display: none`, in `<head>`, ...) |
| `getPages(element)` | the pages the element is on, in order |
| `getComputedStyle(element, property)` | the computed value of a CSS property (`color`, `font-size`, `display`, `width`, ...) as CSS text; null if the element is not displayed |
| `elementAt(page, x, y)` | the innermost element at a point of a page; null in the margins or where there is nothing |

A `LayoutBox` has `getPage()`, `getX()`, `getY()`, `getWidth()` and `getHeight()`. Positions are in PDF points (1/72 inch) from the **top left** corner of the page, not from the bottom left as PDF coordinates usually are. A box is the element's border box (content, padding and border). When the element continues on the next page, it has a box on each page for the part there; an inline element broken over several lines has a box per line.

The layout uses the page size, margins, `@page` rules, `setHtmlWidth()` and the other settings in effect, so a `writePDF()` with the same settings puts the elements at the same places.

{% hint style="warning" %}
A `Layout` is a snapshot of the document as it was when `layout()` was called: after changing the document, call `layout()` again. The content of `pd4ml:page.header`/`pd4ml:page.footer`, of `@page` margin boxes, footnotes and endnotes is not part of it. With `generateMulticolumn()`, a page of the layout is a column; on a page turned by `<pd4ml:page.break pageFormat="rotate">`, boxes are given as on the page before turning.
{% endhint %}

### Example: keep a block on one page

```java
Layout layout = pd4ml.layout();
Element totals = doc.getElementById("totals");
if (layout.getPages(totals).size() > 1) {
    // split over two pages: start it on a new one
    totals.setAttribute("style", "page-break-before: always");
}
pd4ml.writePDF(out);
```

### Example: fit a document on one page

```java
int htmlWidth = 800;
pd4ml.setHtmlWidth(htmlWidth);
while (pd4ml.layout().getPageCount() > 1 && htmlWidth < 1600) {
    htmlWidth += 100;              // a wider layout is scaled down more
    pd4ml.setHtmlWidth(htmlWidth);
}
pd4ml.writePDF(out);
```

### Example: where to put a signature field

```java
LayoutBox box = pd4ml.layout().getBoxes(doc.getElementById("sign-here")).get(0);
// box.getPage(), box.getX(), box.getY(), box.getWidth(), box.getHeight():
// where a PDF tool should place the signature field after writing
```

### Example: read computed styles

```java
Layout layout = pd4ml.layout();
for (Element h : doc.querySelectorAll("h1, h2")) {
    System.out.println(h.getTextContent() + ": " + layout.getComputedStyle(h, "font-size")
            + " on page " + layout.getPages(h));
}
```

## Timing

Make the changes after `readHTML()` and before the output:

* `$[name]` placeholders of `setDynamicData()` are already filled in and `pd4ml:include` is expanded when `readHTML()` returns, so the DOM shows the document with them.
* The document title (`getDocumentTitle()`, `$[title]`) is taken from the document by `readHTML()`, so changing `<title>` afterwards does not change it. Use `setDocumentTitle()`.
* Settings such as `setPageSize()`, `setHtmlWidth()` and `setPageHeader()` can still be changed between `readHTML()` and the output; `layout()` and the outputs use the settings in effect when they are called.
* Writing a document, and `layout()`, lays it out and moves footnotes and endnotes into place in the tree. Later changes still apply to later outputs, but `pd4ml:footnote`/`pd4ml:endnote` elements added after the first layout are not numbered.
* One `PD4ML` object can write several outputs from one `readHTML()`, with changes in between: for example a PDF, then the same document with some elements removed as DOCX.

## Complete example: an invoice from a template

```java
PD4ML pd4ml = new PD4ML();
pd4ml.useHtmlRenderer(Constants.LayoutEngine.v42);
pd4ml.readHTML(new File("templates/invoice.html").toURI().toURL());
Document doc = pd4ml.getDocument();

doc.getElementById("number").setTextContent(invoice.getNumber());
doc.getElementById("customer").setTextContent(invoice.getCustomerName());

Element rows = doc.querySelector("table.items tbody");
rows.setInnerHTML("");                                  // drop the sample rows
for (Item item : invoice.getItems()) {
    Element tr = doc.createElement("tr");
    for (String value : new String[] { item.getName(), item.getQuantity(), item.getAmount() }) {
        Element td = doc.createElement("td");
        td.setTextContent(value);
        tr.appendChild(td);
    }
    rows.appendChild(tr);
}
doc.getElementById("total").setTextContent(invoice.getTotal());

if (!invoice.isDraft()) {
    for (Element e : doc.querySelectorAll(".draft-only")) {
        e.remove();
    }
}

// the payment details must not be split over two pages
Element payment = doc.getElementById("payment");
if (pd4ml.layout().getPages(payment).size() > 1) {
    payment.setAttribute("style", "page-break-before: always");
}

pd4ml.writePDF(new FileOutputStream("invoice-" + invoice.getNumber() + ".pdf"));
```

## See also

* [CSS @page Rules: Page Size, Margins, Named Pages and Margin Boxes](css-page-rules-page-size-margins-named-pages-and-margin-boxes)
* [PD4ML API Reference (Javadoc)](https://pd4ml.com/javadoc/) -- the `com.pd4ml.dom` package, `PD4ML.getDocument()` and `PD4ML.layout()`.
* [PD4ML v42 -- HTML5 Element Support Matrix](../html5-element-support-matrix) -- what the v42 layout engine renders.
