AWS API in Org Mode with Clojure

The Cognitect AWS Client library is a great tool for exploring AWS APIs because the original API documentation is built in.

You can explore the API documentation from the REPL, as Clojure data, but I wanted to try rendering it as Emacs Org Mode, which I use for my personal note-taking.

Dependencies

I’ll need the client library and a few APIs. I’ll use S3 and Athena as examples. 

;;; deps.edn
{:deps {org.clojure/clojure         {:mvn/version "1.11.1"}
        com.cognitect.aws/api       {:mvn/version "0.8.652"}
        com.cognitect.aws/endpoints {:mvn/version "1.1.12.415"}
        com.cognitect.aws/s3        {:mvn/version "825.2.1250.0"}
        com.cognitect.aws/athena    {:mvn/version "825.2.1283.0"}}}

Namespace Setup

(ns com.lambdasierra.aws-org
  (:require [clojure.java.io :as io]
            [clojure.java.shell :refer [sh]]
            [clojure.pprint :refer [pprint]]
            [cognitect.aws.client.api :as aws]))

Converting HTML to Org

The documentation strings in the AWS API are written in HTML. Pandoc can convert them to org-mode syntax. 

(defn pandoc-html-to-org
  "Calls pandoc executable to convert a string of HTML to org-mode."
  [html]
  (let [{:keys [exit out] :as result}
        (sh "pandoc" "--from=html" "--to=org" :in html)]
    (if (zero? exit)
      out
      (throw (ex-info "Pandoc error" result)))))

Printing Org Syntax

Org Mode uses an outline-style syntax, similar to Markdown, in which heading levels are indicated by the number of * characters at the start of the line.

I often use a dynamic Var to keep track of the “depth” in a nested structure. Maybe there’s a better way to do it, but this pattern is familiar and easy to use. It’s one of the few places I find dynamic Vars to be entirely appropriate. 

(def ^:dynamic *heading-level*
  "Current outline heading level in Org Mode output."
  1)

(defmacro with-heading-inc
  "Evaluates body with the Org Mode heading level increased by one."
  [& body]
  `(binding [*heading-level* (inc *heading-level*)]
    ~@body))

(defn print-heading
  "Prints an Org Mode outline heading at the current level."
  [title]
  (dotimes [i *heading-level*]
    (print "*"))
  (printf " %s%n%n" title))

For lists I don’t need anything fancy, just simple bullets: 

(defn print-list
  "Prints a collection of strings as an Org Mode unordered list."
  [items]
  (doseq [item items]
    (printf "- %s%n" item))
  (printf "%n"))

AWS API Documentation as Org

Now the actual output. I’m not going to pretend this is elegant code. It’s just imperative printing, printf even. But it gets the job done.

Pandoc has an abstract syntax tree that can be represented as JSON, so if I wanted to be more structural about this I might try that. 

(defn print-operation
  "Prints an API operation description as Org Mode."
  [{:keys [documentation documentationUrl request required response]
    op-name :name}]
  (print-heading op-name)
  (when documentationUrl
    (printf "API Documentation: [[%s][%s]]%n%n" documentationUrl op-name))
  (when documentation
    (with-heading-inc
      (print-heading "Documentation")
      (print (pandoc-html-to-org documentation))
      (printf "%n%n")))
  (when request
    (with-heading-inc
      (print-heading "Request")
      (printf "#+name %s-request%n" op-name)
      (printf "#+begin_src clojure :eval no%n")
      (pprint request)
      (printf "#+end_src%n%n"))
    (when (seq required)
      (printf "Required:%n%n")
      (print-list required)))
  (when response
    (with-heading-inc
      (print-heading "Response")
      (printf "#+name %s-response%n" op-name)
      (printf "#+begin_src clojure :eval no%n")
      (pprint response)
      (printf "#+end_src%n%n"))))

Finally, to wrap it all up, I just need to get an instance of the API client for a particular service and print the documentation for all of its operations. 

(defn print-api-docs
  "Creates an AWS API client and prints its documentation."
  [api]
  (let [ops (aws/ops (aws/client {:api api}))]
    (print-heading (name api))
    (with-heading-inc
      (doseq [op-key (sort (keys ops))]
        (print-operation (get ops op-key))))))

Demo

Finally, some output: 

(defn demo []
  (with-open [out (io/writer "s3-athena.org")]
    (binding [*out* out]
      (printf "#+TITLE: AWS API Documentation: S3 and Athena%n%n")
      (print-api-docs :s3)
      (print-api-docs :athena))))

The results in Org Mode: s3-athena.org

The results aren’t perfect. Pandoc splits links across lines when word-wrapping, which sometimes breaks Org Mode’s parser. There’s some extraneous whitespace here and there. But it’s not a bad start for an afternoon’s tinkering.

The full source code from this post is available as aws_org.clj.