To create nice pdf for coworkers
The problem#
I am using org-roam to take notes, mostly about science. Following somekind of Zettelkasten Philosophy, the notes remains small (“atomic”) and are linked together. This is great for studying a topic and building connections
Yet, I (more often I though) got requested about a particular subject, and I want to export all the notes related to this subject. Exporting a single file from org-mode to \LaTeX and PDF is easy, but how would you export all the notes that I have on matrix algebra, or on convex optimizations algorithms, with references, and possibly in a standalone format.
The data at hand#
Notes are plain text .org file, with links to other notes.1
For example, a note on matrices could look like this:
:PROPERTIES:
:ID: c7e0fdff-6e4e-450c-bdba-3e4d9752421e
:END:
#+title: Matrix
#+filetags: :index:
Matrices can represent a lot of stuff in math, such as linear transformations between two Vector Spaces, or a system of linear equations.
* Particular matrices
** [[id:396d3cda-705b-4db7-b102-67f93a9de334][Toepliz Matrix]]
** [[id:007392a7-a6e4-461c-ba38-04c03bb4bf9b][Unitary Matrix]]
** [[id:3ac210fc-2eee-4e81-9ea7-55ae2c3be567][Orthogonal Matrix]]
* [[id:65e34230-6df5-447e-80f7-b54e507ff042][Matrix Norm]]
* TODO Matrix Decompositions
** [[id:530253a4-81a0-48a3-8678-97aecdd81123][Singular Value Decomposition]]
* Matrix Approximations
Most of the time you want to do a [[id:530253a4-81a0-48a3-8678-97aecdd81123][Singular Value Decomposition]], and compress the matrix using the first few singular values.
* References
https://en.m.wikipedia.org/wiki/Matrix_(mathematics)
A few remarks:
- links are either part of heading, and basically represent a subsection of the note topic, or related directly in text, and present multiple time in text.
- we only want to treat notes links, not images or web URL
What we want#
- Expand all “heading notes” with potential recursion
- Have a “Appendix” section, containing all the notes that are present in text.
- Avoid Inserting a note multiple times
- Have a nice exported PDF, with no dead links
Some bonuses: don’t export the notes that have a headline/file-tags like :noexport: or :empty:.
Oh and btw, this is the first time that I write elisp by not simply copy pasting random chunk of someonelse’s config.
Org-Roam uses the term node to describe a piece of text which is either a standalone file or a heading with an ID property, and a title (either the headline’s text, or the #+title: content). This can also be refered as a note. I will use the two terms equivalently.
The solution#
As exporting a single .org file is easy, let’s build one. The main idea is:
- Create a new empty note,
- Fill it with a root note
- Find all the headline links, and insert them in place (and do so recursively).
The note file linked into the headline, has some header content (at least a property block with ID and a #+title:, maybe some #+options:, a #+filetags: etc). Apart from the tags and the properties, not much can be passed upstream. The root notes will impose its configuration to the new exported and expanded file.
- Add the “related notes” that are only refered in text.
- Export everything to a PDF.
New headings will have a property drawer with org-roam id, so to avoid duplicated notes conflict, the new expanded note should NOT be saved in your roam directory.
First lets gives us some configurations variables, and load the required packages.
(require 'org)
(require 'org-element)
(require 'org-roam)
(require 'ox)
(require 'cl-lib)
(defvar ox-roam-max-depth 4
"Depth of recursive export for a note.")
(defvar ox-roam-appendix-name "Appendix"
"Name of the appendix section for related notes.")
(defvar ox-roam-excluded-tags '("noexport" "empty")
"List of tags to exclude from export.")
Surprisingly there was not a dedicated regex for ID already available in emacs, so lets build our own (based on org-link-any-re)
(defvar ox-roam--linkid-re "\\[\\[id:\\([[:alnum:][:punct:]]+\\)\\]\\[\\(.*\\)\\]\\]"
"Regular expression matching a link to a note with a custom id.")
Implementation#
(defvar ox-roam--visited-nodes-list nil
"List of visited nodes. Don't use this variable directly.")
(defun ox-roam-ignore-node-p (node)
"Return t if the node should be ignored."
(or (member node ox-roam--visited-nodes-list)
(cl-intersection (org-roam-node-tags node) ox-roam-excluded-tags :test #'string=)))
(define-error 'ox-roam-empty-node-error "The node is empty.")
(defun ox-roam-demote (tree depth)
"Demote all headers in the tree by depth level."
(org-element-map tree '(headline)
(lambda (el) (org-element-put-property el :level (+ depth (org-element-property :level el))))
)
tree)
(defun ox-roam-node-as-subtree (node depth)
"Get the NODE as a subtree of org-element, indent to DEPTH,
and return the org-element tree, expanded again."
(with-temp-buffer
(insert
(org-roam-with-temp-buffer (org-roam-node-file node)
(if (> (org-roam-node-level node) 0)
(progn
(goto-char (org-roam-node-point node))
(org-narrow-to-element)
(re-search-forward org-property-drawer-re nil t 1))
(progn
(goto-char (point-min))
(re-search-forward "#\\+.*?\n[^#]" nil t 1)))
(if (>= (match-end 0) (point-max))(signal 'ox-roam-empty-node-error '(node)))
(buffer-substring-no-properties (match-end 0) (point-max))))
(let ((tree (org-element-parse-buffer))
(new-depth (1+ depth)))
(ox-roam-demote tree new-depth)
(ox-roam-expand-tree tree new-depth)
tree)))
(defun ox-roam-expand-tree (tree depth)
"Expand the TREE until DEPTH reaches ox-roam-max-depth."
(if (>= depth ox-roam-max-depth)
(org-element-map tree 'headline
(lambda (headline)
(when-let* ((raw-title (org-element-property :raw-value headline))
(headline-loc (org-element-property :begin headline))
(headline-level (org-element-property :level headline))
(headline-tag (or (org-element-property :tags headline) '('notags)))
(linked-headline (when (string-match ox-roam-linkid-re raw-title)
(list :ID (match-string 1 raw-title)
:title (match-string 2 raw-title))))
(child-node (org-roam-node-from-id (plist-get linked-headline :ID))))
(unless (or (ox-roam-ignore-node-p child-node)
(cl-intersection (org-roam-node-tags child-node) ox-roam-excluded-tags))
(cl-pushnew child-node ox-roam--visited-nodes-list)
(org-element-adopt-elements headline (ox-roam-node-as-subtree child-node 1))))))
tree)
)
(defun ox-roam--expand-headlines ()
"Expand all the headlines that are valid org roam links in the current buffer."
(let ((tree (org-element-parse-buffer)))
(org-element-map tree 'headline
(lambda (headline)
(when-let* ((raw-title (org-element-property :raw-value headline))
(headline-loc (org-element-property :begin headline))
(headline-level (org-element-property :level headline))
(headline-tag (or (org-element-property :tags headline) '('notags)))
(linked-headline (when (string-match ox-roam-linkid-re raw-title)
(list :ID (match-string 1 raw-title)
:title (match-string 2 raw-title))))
(child-node (org-roam-node-from-id (plist-get linked-headline :ID))))
;; Reformat the headline with title and id properties.
(unless (or (member child-node ox-roam--visited-nodes-list)
(cl-intersection (org-roam-node-tags child-node) ox-roam-excluded-tags))
(cl-pushnew child-node ox-roam--visited-nodes-list)
(org-element-put-property headline :title (plist-get linked-headline :title))
(org-element-adopt-elements headline (format ":PROPERTIES:\n:ID: %s\n:END:\n"
(plist-get linked-headline :ID)))
(org-element-adopt-elements headline (ox-roam-node-as-subtree child-node (1+ headline-level)))))))
(erase-buffer)
(insert (org-element-interpret-data tree))))
(defun ox-roam-include-links ()
"Include all the links to org-roam notes at the end of buffer."
(let ((unique-links '()))
(org-element-map (org-element-parse-buffer) 'link
(lambda (link)
(when-let* ((link-id (org-element-property :path link))
(link-node (org-roam-node-from-id link-id))
(link-node-file (org-roam-node-file link-node))
(link-node-title (org-roam-node-title link-node)))
(unless (or (member link-id unique-links) (ox-roam-ignore-node-p link-node))
(save-excursion
(push link-id unique-links)
(condition-case nil
(progn
(goto-char (point-max))
(insert (format "\n** %s\n" link-node-title))
(insert (format ":PROPERTIES:\n:ID: %s\n:END:\n" link-id))
(insert (org-element-interpret-data (ox-roam-node-as-subtree link-node 2)))
)
(ox-roam-empty-node-error
(set-mark (point))
(line-move -4)
(delete-region (mark) (point)))
(message "Node %s is empty" link-node-title)
))
))))))
Now its time to give a command to use this
(defun ox-roam-export (&optional arg)
(interactive "P")
(let* ((node (org-roam-node-at-point))
(dest (pcase arg
(`nil (expand-file-name (concat (file-name-as-directory org-directory) "exports/" (org-roam-node-slug node) "_extended.org")))
(`(4) (expand-file-name (read-file-name "Export to: ")))
((pred stringp) (expand-file-name arg)))
))
(with-temp-file dest
(make-local-variable 'ox-roam--visited-nodes-list)
(erase-buffer)
(insert "# this file is generated by ox-roam-export.el\n")
;; expands all the headlines that are valid org roam links.
(insert-file-contents (org-roam-node-file node))
(ox-roam--expand-headlines)
;; Add the Content of related nodes
(goto-char (point-max))
(insert (concat "\n*" ox-roam-appendix-name "\n"))
;; Cleanup: Remove the print_bibliography from all the imported notes,
;; and create a single one at the end.
(goto-char (point-min))
(while (re-search-forward "#\\+print_bibliography:.*?\n" nil t) (replace-match "" nil t))
(goto-char (point-max))
(insert "\n#+print_bibliography:\n"))
(message "Exported to %s" dest)))
TODO Adding it to org exporter#
(org-export-define-derived-backend 'org-roam 'org
:menu-entry '(?r "Export Note as standalone"
((?F "To org-buffer" (lambda a s v b) (org-roam-export-to-org t))
(?f "To org-file" (lambda a s v b) (org-roam-export-to-org))
(?p "To pdf" (lambda a s v b) (org-roam-export-to-pdf))
)))
TODO Wrapping things up#
Conclusion#
This was my first major attempt to write elisp code, that was not some blind copy-pasting from someonelse’s config. I am pretty happy with the result, and I hope it will be useful to someone else. I am sure there is a lot of room for improvement, so feel free to reach to me and suggest changes.
Here we are only considering forward links, Exporting backward links would also be interesting, it is left as an exercise to the reader ↩︎