↓Skip to main content
  1. Posts/

LaTeX Equation OCR in Emacs

·2 mins
Emacs Orgmode LaTeX
Pierre-Antoine Comby
Author
Pierre-Antoine Comby
PhD Student on Deep Learning for fMRI Imaging
Google Scholar
Table of Contents

Or how to type less, and read more.

If typing LaTeX Equation can be painfull, copying them from existing material is even more painfull. But this painfull problem has been attacked by many, and the marvel of Image-to-Text Deep Learning Models can be used to extract LaTeX code from images.

Here is a quick demo:

NB: The preview of the equation inside emacs is the joy of the new org-mode equation preview.

Pix2Tex
#

A while back I was using mathpix . You take a screenshot of the equation, and it returns the LaTeX code. It was great, and I was happy.

But they put their API behind a paywall, and a long desert crossing began. But hope was not lost and I found two alternatives, running locally:

  • Pix2Text: A simple OCR that can be used to extract text from images, and can be used to extract LaTeX equations.
  • RapidLatexOCR: Which improves on LaTeX-OCR for inference.

Both tools provide a command line interface, taking an image file as input, and returning the LaTeX code. I settled on RapidLatexOCR, as it was faster.

Emacs Integration
#

Taking the screenshot is easy with gnome-screenshot, and we can use the following script to extract the LaTeX code from the image. And this work everywhere, not only in Emacs 😉.

#!/bin/bash
# ~/bin/latex_ocr.sh
# Take a screenshot and extract the LaTeX code from it
IMG_FILE="$HOME/.cache/tmp_latex_img.png"
gnome-screenshot -a -f $IMG_FILE && rapid_latex_ocr $IMG_FILE | head -n 1

For the emacs part, we can use the following function to insert the LaTeX code at point, and surround it with the appropriate delimiters.

(defvar latex-ocr-cmd "~/bin/latex_ocr.sh"
  "Command to run to extract LaTeX code from a screenshot.")

(defun pac/latex-ocr-insert (pref suff)
  "Add the LaTeX code of the equation in the buffer at point, surrounded by PREF and SUFF."
  (interactive)
  ;; Get the LaTeX code from the screenshot, trim it,
  (let ((raw_output (string-trim (shell-command-to-string latex-ocr-cmd))))
    ;; Proceed only if we have a non-empty output
    (unless (string-empty-p raw_output)
      ;; Remove region if active
      (when (region-active-p) (delete-region (region-beginning) (region-end)))
        ;; Insert the LaTeX code
      (if (texmathp)
        (insert raw_output) ;; without delimiters
      (insert (concat pref raw_output suff))
    ))))

Adding the following keybinding to your config will allow you to insert LaTeX code from a screenshot with a single key press.

;; For doom emacs
(map! :leader
      :prefix "i"
       "m" (cmd! (pac/latex-ocr-insert "\\(" " \\)"))
       "M" (cmd! (pac/latex-ocr-insert "\\[\n" "\n\\]"))
       )

Conclusion
#

This was a good example of UNIX Philosophy (one tool for the job) embracing Emacs extensibility. And overall pretty easy to set up.