In this post I’d like to show some handy way to improve your process of documentating your project. Since we all know documentation is a must you might have wondered how to handle that without any big efforts. In fact it would be great if you could write your code along with the documentation and extract it later on for publishing.

Meet sphinx

It is a tool that you’ll love! Being more technical:

It was originally created for the new Python documentation, and it has excellent facilities for the documentation of Python projects, but C/C++ is already supported as well, and it is planned to add special support for other languages as well.

I have found this tool reading some documentation on readthedocs.org. I was very impressed how easy you could document your code and build a documentation format upon on your needs:

Quickstart

Install tools

First of create a new virtualenv environment:

1
2
3
4
5
6

$ mkdir doku
$ virtualenv doku
Using base prefix '/usr'
New python executable in doku/bin/python3
Also creating executable in doku/bin/python
Installing setuptools, pip...done.

Then activate it:

1
2
3
4
5
6
7
8

$ source doku/bin/activate
➜  tmp  source doku/bin/activate
(doku)➜  tmp  cd doku
(doku)➜  doku  ls -l
total 12
drwxr-xr-x 2 victor users 4096 May  4 13:37 bin
drwxr-xr-x 2 victor users 4096 May  4 13:37 include
drwxr-xr-x 3 victor users 4096 May  4 13:37 lib

Now you’ll have to install sphinx using pip:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

(doku) pip install sphinx
Downloading/unpacking sphinx
  Downloading Sphinx-1.2.2-py33-none-any.whl (1.1MB): 1.1MB downloaded
Downloading/unpacking docutils>=0.10 (from sphinx)
  Downloading docutils-0.11.tar.gz (1.6MB): 1.6MB downloaded
  Running setup.py (path:/home/victor/tmp/doku/build/docutils/setup.py) egg_info for package docutils

warning: no files found matching 'MANIFEST'
warning: no files found matching '*' under directory 'extras'
warning: no previously-included files matching '.cvsignore' found under directory '*'
warning: no previously-included files matching '*.pyc' found under directory '*'
warning: no previously-included files matching '*~' found under directory '*'
warning: no previously-included files matching '.DS_Store' found under directory '*'
Downloading/unpacking Jinja2>=2.3 (from sphinx)
  Downloading Jinja2-2.7.2.tar.gz (378kB): 378kB downloaded
  Running setup.py (path:/home/victor/tmp/doku/build/Jinja2/setup.py) egg_info for package Jinja2

warning: no files found matching '*' under directory 'custom_fixers'
warning: no previously-included files matching '*' found under directory 'docs/_build'
warning: no previously-included files matching '*.pyc' found under directory 'jinja2'
warning: no previously-included files matching '*.pyc' found under directory 'docs'
warning: no previously-included files matching '*.pyo' found under directory 'jinja2'
warning: no previously-included files matching '*.pyo' found under directory 'docs'
Downloading/unpacking Pygments>=1.2 (from sphinx)
  Downloading Pygments-1.6.tar.gz (1.4MB): 1.4MB downloaded
  Running setup.py (path:/home/victor/tmp/doku/build/Pygments/setup.py) egg_info for package Pygments

Downloading/unpacking markupsafe (from Jinja2>=2.3->sphinx)
  Downloading MarkupSafe-0.21.tar.gz
  Running setup.py (path:/home/victor/tmp/doku/build/markupsafe/setup.py) egg_info for package markupsafe

Installing collected packages: sphinx, docutils, Jinja2, Pygments, markupsafe
  Running setup.py install for docutils
Skipping implicit fixer: buffer
Skipping implicit fixer: idioms
Skipping implicit fixer: set_literal
Skipping implicit fixer: ws_comma
copy/convert test suite
changing mode of build/scripts-3.4/rst2html.py from 644 to 755
changing mode of build/scripts-3.4/rst2s5.py from 644 to 755
changing mode of build/scripts-3.4/rst2latex.py from 644 to 755
changing mode of build/scripts-3.4/rst2xetex.py from 644 to 755
changing mode of build/scripts-3.4/rst2man.py from 644 to 755
changing mode of build/scripts-3.4/rst2xml.py from 644 to 755
changing mode of build/scripts-3.4/rst2pseudoxml.py from 644 to 755
changing mode of build/scripts-3.4/rstpep2html.py from 644 to 755
changing mode of build/scripts-3.4/rst2odt.py from 644 to 755
changing mode of build/scripts-3.4/rst2odt_prepstyles.py from 644 to 755

warning: no files found matching 'MANIFEST'
warning: no files found matching '*' under directory 'extras'
warning: no previously-included files matching '.cvsignore' found under directory '*'
warning: no previously-included files matching '*.pyc' found under directory '*'
warning: no previously-included files matching '*~' found under directory '*'
warning: no previously-included files matching '.DS_Store' found under directory '*'
changing mode of /home/victor/tmp/doku/bin/rstpep2html.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2xml.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2pseudoxml.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2html.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2odt_prepstyles.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2latex.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2man.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2odt.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2xetex.py to 755
changing mode of /home/victor/tmp/doku/bin/rst2s5.py to 755
  Running setup.py install for Jinja2

warning: no files found matching '*' under directory 'custom_fixers'
warning: no previously-included files matching '*' found under directory 'docs/_build'
warning: no previously-included files matching '*.pyc' found under directory 'jinja2'
warning: no previously-included files matching '*.pyc' found under directory 'docs'
warning: no previously-included files matching '*.pyo' found under directory 'jinja2'
warning: no previously-included files matching '*.pyo' found under directory 'docs'
  Running setup.py install for Pygments
Skipping implicit fixer: buffer
Skipping implicit fixer: idioms
Skipping implicit fixer: set_literal
Skipping implicit fixer: ws_comma

Installing pygmentize script to /home/victor/tmp/doku/bin
  Running setup.py install for markupsafe

building 'markupsafe._speedups' extension
gcc -pthread -Wno-unused-result -Werror=declaration-after-statement -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector --param=ssp-buffer-size=4 -fPIC -I/usr/include/python3.4m -c markupsafe/_speedups.c -o build/temp.linux-x86_64-3.4/markupsafe/_speedups.o
gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro build/temp.linux-x86_64-3.4/markupsafe/_speedups.o -L/usr/lib -lpython3.4m -o build/lib.linux-x86_64-3.4/markupsafe/_speedups.cpython-34m.so
Successfully installed sphinx docutils Jinja2 Pygments markupsafe
Cleaning up...
(doku) doku

Now you should be ready to go.

Create new documentation dir structure

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

(doku)➜  doku  mkdir my_documentation
(doku)➜  doku  sphinx-quickstart
Welcome to the Sphinx 1.2.2 quickstart utility.

Please enter values for the following settings (just press Enter to
accept a default value, if one is given in brackets).

Enter the root path for documentation.
> Root path for the documentation [.]: my_documentation

You have two options for placing the build directory for Sphinx output.
Either, you use a directory "_build" within the root path, or you separate
"source" and "build" directories within the root path.
> Separate source and build directories (y/n) [n]: n

Inside the root directory, two more directories will be created; "_templates"
for custom HTML templates and "_static" for custom stylesheets and other static
files. You can enter another prefix (such as ".") to replace the underscore.
> Name prefix for templates and static dir [_]:

The project name will occur in several places in the built documentation.
> Project name: My Documentation
> Author name(s): Victor

Sphinx has the notion of a "version" and a "release" for the
software. Each version can have multiple releases. For example, for
Python the version is something like 2.5 or 3.0, while the release is
something like 2.5.1 or 3.0a1.  If you don't need this dual structure,
just set both to the same value.
> Project version: 0.1
> Project release [0.1]: 0.1.0

The file name suffix for source files. Commonly, this is either ".txt"
or ".rst".  Only files with this suffix are considered documents.
> Source file suffix [.rst]:

One document is special in that it is considered the top node of the
"contents tree", that is, it is the root of the hierarchical structure
of the documents. Normally, this is "index", but if your "index"
document is a custom template, you can also set this to another filename.
> Name of your master document (without suffix) [index]:

Sphinx can also add configuration for epub output:
> Do you want to use the epub builder (y/n) [n]: n

Please indicate if you want to use one of the following Sphinx extensions:
> autodoc: automatically insert docstrings from modules (y/n) [n]: y
> doctest: automatically test code snippets in doctest blocks (y/n) [n]: n
> intersphinx: link between Sphinx documentation of different projects (y/n) [n]: n
> todo: write "todo" entries that can be shown or hidden on build (y/n) [n]: n
> coverage: checks for documentation coverage (y/n) [n]: n
> pngmath: include math, rendered as PNG images (y/n) [n]: n
> mathjax: include math, rendered in the browser by MathJax (y/n) [n]: n
> ifconfig: conditional inclusion of content based on config values (y/n) [n]: n
> viewcode: include links to the source code of documented Python objects (y/n) [n]: n

A Makefile and a Windows command file can be generated for you so that you
only have to run e.g. `make html' instead of invoking sphinx-build
directly.
> Create Makefile? (y/n) [y]:
> Create Windows command file? (y/n) [y]: n

Creating file my_documentation/conf.py.
Creating file my_documentation/index.rst.
Creating file my_documentation/Makefile.

Finished: An initial directory structure has been created.

You should now populate your master file my_documentation/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
   make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.

Create some builds

Now you can call make to build your desired documentation format:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

(doku)➜  my_documentation  make
Please use `make <target>' where <target> is one of
  html       to make standalone HTML files
  dirhtml    to make HTML files named index.html in directories
  singlehtml to make a single large HTML file
  pickle     to make pickle files
  json       to make JSON files
  htmlhelp   to make HTML files and a HTML help project
  qthelp     to make HTML files and a qthelp project
  devhelp    to make HTML files and a Devhelp project
  epub       to make an epub
  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter
  latexpdf   to make LaTeX files and run them through pdflatex
  latexpdfja to make LaTeX files and run them through platex/dvipdfmx
  text       to make text files
  man        to make manual pages
  texinfo    to make Texinfo files
  info       to make Texinfo files and run them through makeinfo
  gettext    to make PO message catalogs
  changes    to make an overview of all changed/added/deprecated items
  xml        to make Docutils-native XML files
  pseudoxml  to make pseudoxml-XML files for display purposes
  linkcheck  to check all external links for integrity
  doctest    to run all doctests embedded in the documentation (if enabled)

Let’s try with html:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

(doku)➜  my_documentation  make html
sphinx-build -b html -d _build/doctrees   . _build/html
Making output directory...
Running Sphinx v1.2.2
loading pickled environment... failed: [Errno 2] No such file or directory: '/home/victor/tmp/doku/my_documentation/_build/doctrees/environment.pickle'
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
writing additional files... genindex search
copying static files... done
copying extra files... done
dumping search index... done
dumping object inventory... done
build succeeded.

Build finished. The HTML pages are in _build/html.

How about PDF?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

(doku)➜  my_documentation  make latexpdf
sphinx-build -b latex -d _build/doctrees   . _build/latex
Making output directory...
Running Sphinx v1.2.2
loading pickled environment... done
building [latex]: all documents
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
processing MyDocumentation.tex... index
resolving references...
writing... done
copying TeX support files...
done
build succeeded.
Running LaTeX files through pdflatex...
make -C _build/latex all-pdf
make[1]: Entering directory '/home/victor/tmp/doku/my_documentation/_build/latex'
...
Transcript written on MyDocumentation.log.
make[1]: Leaving directory '/home/victor/tmp/doku/my_documentation/_build/latex'
pdflatex finished; the PDF files are in _build/latex.

You can find you builds inside the _build folder:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

(doku)➜  my_documentation  tree _build
_build
├── doctrees
│   ├── environment.pickle
│   └── index.doctree
├── html
│   ├── genindex.html
│   ├── index.html
│   ├── objects.inv
│   ├── search.html
│   ├── searchindex.js
│   ├── _sources
│   │   └── index.txt
│   └── _static
│       ├── ajax-loader.gif
│       ├── basic.css
│       ├── comment-bright.png
│       ├── comment-close.png
│       ├── comment.png
│       ├── default.css
│       ├── doctools.js
│       ├── down.png
│       ├── down-pressed.png
│       ├── file.png
│       ├── jquery.js
│       ├── minus.png
│       ├── plus.png
│       ├── pygments.css
│       ├── searchtools.js
│       ├── sidebar.js
│       ├── underscore.js
│       ├── up.png
│       ├── up-pressed.png
│       └── websupport.js
└── latex
    ├── fncychap.sty
    ├── Makefile
    ├── MyDocumentation.aux
    ├── MyDocumentation.idx
    ├── MyDocumentation.ilg
    ├── MyDocumentation.ind
    ├── MyDocumentation.log
    ├── MyDocumentation.out
    ├── MyDocumentation.pdf
    ├── MyDocumentation.tex
    ├── MyDocumentation.toc
    ├── python.ist
    ├── sphinxhowto.cls
    ├── sphinxmanual.cls
    ├── sphinx.sty
    └── tabulary.sty

You can easily customize the output of your PDF (when dealing with LaTeX). Just have a look at Pedros tips how to do that. However if you don’t want to use LaTeX to generate your PDF, you could also have a look at rst2pdf which will convert your RST file into some PDF. Attention: rst2pdf is not available for Python 3.x!

Themes

Although I like to use PDFs when generating my documentation, HTML file are also fine. Make sure you have a look at how themes are used. There are some basic themes provided within sphinx itself, but you can also create your own ones. Also search for themes: GitHub is your friend!

Conclusion

You can use sphinx to generate all kind of documentation. Examples: