Add support for plantuml/graphviz diagrams via the plantuml plugin

Readme updated with documentation about the plugin
Add an architecture page in the wiki to show an example

README.md

@@ -17,6 +17,11 @@ Install python stuff
     source bin/activate
     pip install pelican pelican-youtube markdown beautifulsoup4
 
+Install system dependencies for plantuml plugin (plantuml and GraphViz utilities) :
+
+ * Install plantuml : use your package manager or http://plantuml.com/starting
+ * Install GraphViz : use your package manager or http://www.graphviz.org/Download..php
+
 Generate the site
 
     pelican
@@ -38,3 +43,9 @@ You may want to change the production parameters, like the domain name: just edi
 For example if you want to host the site at `https://my.website.org`, set:
 
     SITEURL = u'https://my.website.org'
+
+## Plantuml plugin documentation
+
+ * Plantuml plugin documentation : https://github.com/Scheirle/pelican-plugins/tree/master/plantuml
+ * Plantuml documentation: http://plantuml.com
+ * Plantuml support DOT language of GraphViz: http://www.graphviz.org/Gallery.php

content/pages/wiki/duniter.md

@@ -17,3 +17,7 @@ Page concernant spécifiquement le logiciel Duniter.
 
 * [Duniter CLI](./commands)
 * [Development tutorial (in french)](https://github.com/duniter/duniter/blob/master/doc/contribute-french.md)
+
+## Concepts
+
+* [Architecture](./architecture)

content/pages/wiki/duniter/architecture.md

+Title: Architecture
+Order: 10
+Date: 2017-10-09
+Slug: architecture
+Authors: vit
+
+## Clients network architecture
+
+Here is a glimpse of the architecture of Duniter between one server and clients.
+
+::uml:: format="svg" alt="clients architecture"
+
+@startuml
+
+title Clients network architecture
+
+node "Duniter" {
+
+    interface "HTTP" as HTTP_duniter
+
+    HTTP_duniter - [Basic Merkle Api]
+}
+
+database "Elastic Search" as ES {
+
+    interface "HTTP" as HTTP_ES
+
+    frame "Duniter4j Plugin" {
+
+        [Sync] -- HTTP_duniter
+    }
+
+    [Members Profiles Index]
+    [Companies Registry Index]
+    [Market Place Index]
+}
+
+package "Duniter Desktop" {
+
+    interface "client BMA" as client_duniter_desktop
+
+    client_duniter_desktop --- HTTP_duniter
+}
+
+package "Remuniter" {
+
+    interface "client BMA" as client_remuniter
+
+    client_remuniter --- HTTP_duniter
+}
+
+
+package "Cesium" {
+
+    interface "client BMA" as client_bma
+
+    frame "ES Data Store Plugin" {
+        interface "client ES" as client_es
+
+        [Members Profiles]
+        [Companies Registry]
+        [Market Place]
+        client_es --- HTTP_ES
+    }
+
+    client_bma -- HTTP_duniter
+
+}
+
+package "Duniter Android App" {
+
+    frame "Duniter4j API" {
+        interface "client BMA" as client_android_bma
+
+        client_android_bma --- HTTP_duniter
+    }
+}
+
+package "Sakia" {
+
+    frame "Duniter Python API" {
+        interface "client BMA" as client_sakia
+
+        client_sakia --- HTTP_duniter
+    }
+}
+
+@enduml
+
+::end-uml::

pelican-plugins/plantuml/Readme.rst

+================================================
+PlantUML plugin for Pelican rst and md documents
+================================================
+
+This plugin allows you to define UML diagrams directly into rst or md documents using the great PlantUML_ tool.
+
+It gets the content of ``uml`` directive, passes it to the external program PlantUML_ and then links the generated image to the document.
+
+.. contents::
+
+Installation
+============
+
+You need to install PlantUML_ (see the site for details) and Graphviz_ 2.26.3 or later. The plugin expects a program ``plantuml`` in the classpath. If not installed by your package manager, you can create a shell script and place it somewhere in the classpath. For example, save te following into ``/usr/local/bin/plantuml`` (supposing PlantUML_ installed into ``/opt/plantuml``):
+
+.. code-block:: bash
+
+    #!/bin/bash
+    java -jar /opt/plantuml/plantuml.jar ${@}
+
+For Gentoo_ there is an ebuild at http://gpo.zugaina.org/dev-util/plantuml/RDep: you can download the ebuild and the ``files`` subfolder or you can add the ``zugaina`` repository with _layman (reccomended).
+
+Usage
+=====
+
+Add ``plantuml`` to plugin list in ``pelicanconf.py``. For example:
+
+.. code-block:: ptyhon
+
+    PLUGINS = [ "sitemap", "plantuml" ]
+
+One loaded the plugin register also the Pyhton-Markdown_ extension. 
+
+RST usage
+---------
+Use the ``uml`` directive to start UML diagram description. It is not necessary to enclose diagram body between ``@startuml`` and ``@enduml`` directives: they are added automatically  before calling ``plantuml``.
+
+In addition to ``class`` and ``alt`` options common to all images, you can use the ``format`` option to select what kind of image must be produced. At the moment only ``png`` and ``svg`` are supported; the default is ``png``.
+
+Please note that the ``format`` option in not recognized by the ``plantuml`` extension of ``rst2pdf`` utility (call it with ``-e plantuml.py``) so if you use it you can get errors from that program.
+
+MD usage
+--------
+For use with the Pyhton-Markdown_ syntax, the UML block must be enclose with ``::uml::``:
+
+.. code-block:: markdown
+
+    ::uml:: [format=...] [classes=...] [alt=...]
+       PlantUML script
+    ::end-uml::
+
+Please keep a blank line before ``::uml::`` and after ``::end-uml::`` to be sure that the UML code will be correctly recognized. See Examples_ for more details.
+
+With MD syntax options must be specified in the same line as the opening ``:uml::``, with the order ``format``, ``classes`` anmd ``alt``. The general syntax for option is
+
+.. code-block:: text
+
+    option="value"
+
+Option can be enclosed with single or double quotes, as you like. Options defaults are the same as for the rst plugin.
+
+For pandoc_reader plugin users
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The plugin ``pandoc_reader`` sends the Markdown body to the pandoc_ tool which has it's own Markdown parser, written in Haskell_ language: Python_ plugins manipulating Markdown posts (such this) are not used because the entire body id passed to pandoc_ without any iteraction by Pelican_.
+
+For those who are using the ``pandoc_reader`` plugin and wants to include PlantUML_ diagrams, use the ``pandoc-plantuml`` script (only *nix, sorry): it is a wrapper for filtering the code blocks parsed by pandoc_ before
+writing out the converted file. It is an adaption of the great work by Kurt Bonne for his `pandoc-plantuml-filter <https://github.com/kbonne/pandoc-plantuml-filter.git>`_.
+
+To use it, copy the ``pandoc-plantuml`` file in a subdirectory of your pelican project (for example `pandoc_extensions`) and make sure it is executable (``chmod +x pandoc-plantuml``).
+
+In the ``pelicanconf.py`` configure the needed plugins:
+
+.. code-block:: python
+
+    PLUGINS = ['pandoc_reader'] // Yes, plantuml plugin non necessary
+    PANDOC_ARGS = ['--filter=pandoc_extensions/pandoc-plantuml']
+
+In Markdown posts use the following syntax to include PlantUML_ diagrams:
+
+.. code-block:: markdown
+
+    ```plantuml
+    @startuml
+      Alice -> Bob: Authentication Request
+      Bob --> Alice: Authentication Response
+
+      Alice -> Bob: Another authentication Request
+      Alice <-- Bob: another authentication Response
+    @enduml
+    ```
+
+Rendered images will bu put in the output/images folder.
+
+**NOTE:** ``pandoc-plantuml`` is broken from pandoc 1.16 cause an API change in pandoc ``Image`` function. I'm working on a fix but in the meanwhile use a version of pandoc prior to ``1.16`` .
+
+Debugging
+---------
+The plugin can produce debugging informations to help to locate errors. To enable debugging execute ``pelican`` in debug mode:
+
+ .. code-block:: console
+
+     make DEBUG=1 html
+
+  
+Examples
+========
+
+Sequence diagram (from PlantUML_ site):
+
+.. code-block:: rst
+
+  .. uml::
+    :alt: Sample sequence diagram
+
+    participant User
+
+    User -> A: DoWork
+    activate A #FFBBBB
+
+    A -> A: Internal call
+    activate A #DarkSalmon
+
+    A -> B: << createRequest >>
+    activate B
+
+    B --> A: RequestCreated
+    deactivate B
+    deactivate A
+    A -> User: Done
+    deactivate A
+
+Output:
+
+.. image:: http://plantuml.sourceforge.net/imgp/sequence_022.png
+   :alt: Sample sequence diagram
+
+
+Same diagram with Pyhton-Markdown_ syntax:
+
+.. code-block:: markdown
+
+    ::uml:: format="png" alt="Sample sequence diagram"
+      participant User
+
+      User -> A: DoWork
+      activate A #FFBBBB
+
+      A -> A: Internal call
+      activate A #DarkSalmon
+
+      A -> B: << createRequest >>
+      activate B
+
+      B --> A: RequestCreated
+      deactivate B
+      deactivate A
+      A -> User: Done
+      deactivate A
+    ::end-uml::
+
+Another example from PlantUML_ site (activity diagram):
+
+.. code-block:: rst
+
+  .. uml::
+
+    start
+    :ClickServlet.handleRequest();
+    :new page;
+    if (Page.onSecurityCheck) then (true)
+      :Page.onInit();
+      if (isForward?) then (no)
+	:Process controls;
+	if (continue processing?) then (no)
+	  stop
+	endif
+	
+	if (isPost?) then (yes)
+	  :Page.onPost();
+	else (no)
+	  :Page.onGet();
+	endif
+	:Page.onRender();
+      endif
+    else (false)
+    endif
+
+    if (do redirect?) then (yes)
+      :redirect process;
+    else
+      if (do forward?) then (yes)
+	:Forward request;
+      else (no)
+	:Render page template;
+      endif
+    endif
+
+    stop
+
+Generated image:
+
+.. image:: http://plantuml.sourceforge.net/imgp/activity2_009.png
+   :alt: Sample activity diagram
+
+
+
+.. _PlantUML: http://plantuml.sourceforge.net
+.. _Sabayon: http://www.sabayon.org
+.. _Gentoo: http://www.gentoo.org
+.. _layman: http://wiki.gentoo.org/wiki/Layman
+.. _Graphviz: http://www.graphviz.org
+.. _Pyhton-Markdown: http://pythonhosted.org/Markdown
+.. _pandoc: http://johnmacfarlane.net/pandoc
+.. _Haskell: http://www.haskell.org/haskellwiki/Haskell
+.. _Python:: http://www.python.org
+.. _Pelican: http://docs.getpelican.com/en

pelican-plugins/plantuml/__init__.py

+from .plantuml_rst import *

pelican-plugins/plantuml/generateUmlDiagram.py

+#!/usr/bin/env python
+import logging
+import os
+import tempfile
+from zlib import adler32
+from subprocess import Popen, PIPE
+from pelican import logger
+
+
+def generate_uml_image(path, plantuml_code, imgformat):
+    tf = tempfile.NamedTemporaryFile(delete=False)
+    tf.write('@startuml\n'.encode('utf8'))
+    tf.write(plantuml_code.encode('utf8'))
+    tf.write('\n@enduml'.encode('utf8'))
+    tf.flush()
+
+    logger.debug("[plantuml] Temporary PlantUML source at "+(tf.name))
+
+    if imgformat == 'png':
+        imgext = ".png"
+        outopt = "-tpng"
+    elif imgformat == 'svg':
+        imgext = ".svg"
+        outopt = "-tsvg"
+    else:
+        logger.error("Bad uml image format '"+imgformat+"', using png")
+        imgext = ".png"
+        outopt = "-tpng"
+
+    # make a name
+    name = tf.name+imgext
+    # build cmd line
+    cmdline = ['plantuml', '-o', path, outopt, tf.name]
+
+    try:
+        logger.debug("[plantuml] About to execute "+" ".join(cmdline))
+        p = Popen(cmdline, stdout=PIPE, stderr=PIPE)
+        out, err = p.communicate()
+    except Exception as exc:
+        raise Exception('Failed to run plantuml: %s' % exc)
+    else:
+        if p.returncode == 0:
+            # diagram was correctly generated, we can remove the temporary file (if not debugging)
+            if not logger.isEnabledFor(logging.DEBUG):
+                os.remove(tf.name)
+            # renaming output image using an hash code, just to not pollute
+            # output directory with a growing number of images
+            name = os.path.join(path, os.path.basename(name))
+            newname = os.path.join(path, "%08x" % (adler32(plantuml_code.encode()) & 0xffffffff))+imgext
+
+            if os.path.exists(newname):
+                os.remove(newname)
+
+            os.rename(name, newname)
+            return 'images/' + os.path.basename(newname)
+        else:
+            # the temporary file is still available as aid understanding errors
+            raise RuntimeError('Error calling plantuml: %s' % err)

pelican-plugins/plantuml/pandoc-plantuml

+#!/usr/bin/runhaskell
+{-|
+
+  Script adapted from pandoc-plantuml-filter by Kurt Bonne
+  Original source at https://github.com/kbonne/pandoc-plantuml-filter.git
+
+  This script is meant to be run by pandoc executed by the pandoc_reader pelican plugin.
+  I've changed output paths to be compatibile with pelican output structure.
+
+  If using the pandoc_reader pelican plugin with this script, the plantuml plugin is not necessary.
+
+  Installation:
+  -------------
+    This script requires Haskell, but if you are using pandoc, it's already installed :-)
+    Copy this file in your pelican project, in the same directory of pelicanconf.py, and make sure it is executable.
+
+    In the pelicanconf.py configure the need plugins:
+
+      PLUGINS = ['pandoc_reader']
+      PANDOC_ARGS = ['--filter=pandoc-plantuml']
+
+    If this script will be putted in a different location, adapt the PANDOC_ARGS value.
+
+  Usage:
+  ------
+    In Markdown posts use the following syntax to include PlantUML diagrams:
+
+    ```plantuml
+    @startuml
+       Alice -> Bob: Authentication Request
+       Bob --> Alice: Authentication Response
+
+       Alice -> Bob: Another authentication Request
+       Alice <-- Bob: another authentication Response
+    @enduml
+    ```
+
+    Rendered images will bu put in the output/images folder.
+-}
+
+import Text.Pandoc.JSON
+import Data.ByteString.Lazy (hGetContents, hPut)
+import Data.ByteString.Lazy.UTF8 (fromString)
+import Data.Digest.Pure.SHA (sha1, showDigest)
+import System.IO (hClose, hPutStr, IOMode(..), openBinaryFile, hPutStrLn, stderr)
+import System.Process
+import System.Directory
+
+processBlocks :: Block -> IO Block
+processBlocks b =
+  case b of
+    CodeBlock (_ , ["plantuml"], _) content -> plantUMLToImg content
+    _ -> return b
+
+plantUMLToImg :: String -> IO Block
+plantUMLToImg content =  do
+  path <- renderImage content
+  
+  --hPutStrLn stderr "dopo renderImage"
+
+  return $ Para [Image [] (path, "")]
+
+renderImage :: String -> IO String
+renderImage content = do
+  createDirectoryIfMissing (True) "output/images"
+  let path = uniqueName content ++ ".png"
+  (Just hIn, Just hOut, _, _) <-
+    createProcess (proc "plantuml" ["-pipe", "-tepg"]){ std_in = CreatePipe,
+                                                        std_out = CreatePipe }
+  hPutStr hIn content
+  hClose hIn
+
+  let outPath = "output/images/" ++ path
+  hFile <- openBinaryFile outPath WriteMode
+  img <- hGetContents hOut
+  hPut hFile img
+
+  hClose hFile
+  hClose hOut
+
+  let imgPath = "images/" ++ path
+
+  return imgPath
+
+uniqueName :: String -> String
+uniqueName = showDigest . sha1 . fromString
+
+main :: IO ()
+main = toJSONFilter processBlocks

pelican-plugins/plantuml/plantuml_md.py

+#!/usr/bin/env python
+"""
+   PlantUML_ Extension for Python-Markdown_
+   ========================================
+
+   Syntax:
+
+       ::uml:: [format="png|svg"] [classes="class1 class2 ..."] [alt="text for alt"]
+          PlantUML script diagram
+       ::end-uml::
+
+   Example:
+
+       ::uml:: format="png" classes="uml myDiagram" alt="My super diagram"
+          Goofy ->  MickeyMouse: calls
+          Goofy <-- MickeyMouse: responds
+       ::end-uml::
+
+   Options are optional, but if present must be specified in the order format, classes, alt.
+   The option value may be enclosed in single or double quotes.
+
+.. _Python-Markdown: http://pythonhosted.org/Markdown/
+.. _PlantUML: http://plantuml.sourceforge.net/
+"""
+import os
+import re
+import markdown
+from markdown.util import etree
+from .generateUmlDiagram import generate_uml_image
+
+# For details see https://pythonhosted.org/Markdown/extensions/api.html#blockparser
+class PlantUMLBlockProcessor(markdown.blockprocessors.BlockProcessor):
+    # Regular expression inspired by the codehilite Markdown plugin
+    RE = re.compile(r'''::uml::
+                        \s*(format=(?P<quot>"|')(?P<format>\w+)(?P=quot))?
+                        \s*(classes=(?P<quot1>"|')(?P<classes>[\w\s]+)(?P=quot1))?
+                        \s*(alt=(?P<quot2>"|')(?P<alt>[\w\s"']+)(?P=quot2))?
+                    ''', re.VERBOSE)
+    # Regular expression for identify end of UML script
+    RE_END = re.compile(r'::end-uml::\s*$')
+
+    def test(self, parent, block):
+        return self.RE.search(block)
+
+    def run(self, parent, blocks):
+        block = blocks.pop(0)
+        text = block
+
+        # Parse configuration params
+        m = self.RE.search(block)
+        format  = m.group('format')  if m.group('format')  else self.config['format']
+        classes = m.group('classes') if m.group('classes') else self.config['classes']
+        alt     = m.group('alt')     if m.group('alt')     else self.config['alt']
+
+        # Read blocks until end marker found
+        while blocks and not self.RE_END.search(block):
+            block = blocks.pop(0)
+            text = text + '\n' + block
+        else:
+            if not blocks:
+                raise RuntimeError("[plantuml] UML block not closed, text is:\n"+text)
+
+        # Remove block header and footer
+        text = re.sub(self.RE, "", re.sub(self.RE_END, "", text))
+
+        path = os.path.abspath(os.path.join('output', 'images'))
+        if not os.path.exists(path):
+            os.makedirs(path)
+
+        # Generate image from PlantUML script
+        imageurl = self.config['siteurl']+'/'+generate_uml_image(path, text, format)
+        # Create image tag and append to the document
+        etree.SubElement(parent, "img", src=imageurl, alt=alt, attrib={'class':classes})
+
+
+# For details see https://pythonhosted.org/Markdown/extensions/api.html#extendmarkdown
+class PlantUMLMarkdownExtension(markdown.Extension):
+    # For details see https://pythonhosted.org/Markdown/extensions/api.html#configsettings
+    def __init__(self, *args, **kwargs):
+        self.config = {
+            'classes': ["uml","Space separated list of classes for the generated image. Default uml."],
+            'alt'    : ["uml diagram", "Text to show when image is not available."],
+            'format' : ["png", "Format of image to generate (png or svg). Default png."],
+            'siteurl': ["", "URL of document, used as a prefix for the image diagram."]
+        }
+
+        super(PlantUMLMarkdownExtension, self).__init__(*args, **kwargs)
+
+    def extendMarkdown(self, md, md_globals):
+        blockprocessor = PlantUMLBlockProcessor(md.parser)
+        blockprocessor.config = self.getConfigs()
+        md.parser.blockprocessors.add('plantuml', blockprocessor, '>code')
+
+def makeExtension(**kwargs):
+    return PlantUMLMarkdownExtension(**kwargs)

pelican-plugins/plantuml/plantuml_rst.py

+#!/usr/bin/env python
+"""Custom reST_ directive for plantuml_ integration.
+   Adapted from ditaa_rst plugin.
+
+.. _reST: http://docutils.sourceforge.net/rst.html
+.. _plantuml: http://plantuml.sourceforge.net/
+"""
+
+import sys
+import os
+
+from docutils.nodes import image, literal_block
+from docutils.parsers.rst import Directive, directives
+from pelican import signals, logger
+
+from .generateUmlDiagram import generate_uml_image
+
+
+global_siteurl = "" # URL of the site, filled on plugin initialization
+
+
+class PlantUML_rst(Directive):
+    """ reST directive for PlantUML """
+    required_arguments = 0
+    optional_arguments = 0
+    has_content = True
+
+    global global_siteurl
+
+    option_spec = {
+        'class' : directives.class_option,
+        'alt'   : directives.unchanged,
+        'format': directives.unchanged,
+    }
+
+    def run(self):
+        path = os.path.abspath(os.path.join('output', 'images'))
+        if not os.path.exists(path):
+            os.makedirs(path)
+
+        nodes = []
+        body = '\n'.join(self.content)
+
+        try:
+            url = global_siteurl+'/'+generate_uml_image(path, body, "png")
+        except Exception as exc:
+            error = self.state_machine.reporter.error(
+                'Failed to run plantuml: %s' % exc,
+                literal_block(self.block_text, self.block_text),
+                line=self.lineno)
+            nodes.append(error)
+        else:
+            alt = self.options.get('alt', 'uml diagram')
+            classes = self.options.pop('class', ['uml'])
+            imgnode = image(uri=url, classes=classes, alt=alt)
+            nodes.append(imgnode)
+
+        return nodes
+
+def pelican_init(pelicanobj):
+
+    global global_siteurl
+    global_siteurl = pelicanobj.settings['SITEURL']
+
+    """ Prepare configurations for the MD plugin """
+    try:
+        import markdown
+        from .plantuml_md import PlantUMLMarkdownExtension
+    except:
+        # Markdown not available
+        logger.debug("[plantuml] Markdown support not available")
+        return
+
+    # Register the Markdown plugin
+    config = { 'siteurl': pelicanobj.settings['SITEURL'] }
+
+    try:
+        if 'MD_EXTENSIONS' in pelicanobj.settings.keys(): # pre pelican 3.7.0
+            pelicanobj.settings['MD_EXTENSIONS'].append(PlantUMLMarkdownExtension(config))
+        elif 'MARKDOWN' in pelicanobj.settings.keys() and \
+             not ('extension_configs' in pelicanobj.settings['MARKDOWN']['extension_configs']):  # from pelican 3.7.0
+            pelicanobj.settings['MARKDOWN']['extension_configs']['plantuml.plantuml_md'] = config # Fix bug default settings
+    except:
+        logger.error("[plantuml] Unable to configure plantuml markdown extension")
+
+
+def register():
+    """Plugin registration."""
+    signals.initialized.connect(pelican_init)
+    directives.register_directive('uml', PlantUML_rst)

pelicanconf.py

@@ -24,14 +24,15 @@ SITELOGO_SIZE = 36
 DEFAULT_LANG = u'fr'
 
 PLUGIN_PATHS = ['pelican-plugins/']
-PLUGINS = ['i18n_subsites', 'tipue_search', 'pelican-page-hierarchy']
+PLUGINS = ['i18n_subsites', 'tipue_search', 'pelican-page-hierarchy', 'plantuml']
 
 MARKDOWN = {
   'extension_configs': {
     'markdown.extensions.codehilite': { 'css_class': 'highlight' },
     'markdown.extensions.fenced_code': {},
     'markdown.extensions.extra': {},
-    'markdown.extensions.toc': {}
+    'markdown.extensions.toc': {},
+    #'plantuml.plantuml_md': {'siteurl': '/en'} DOES NOT WORK, NEED A PATCH ON THE PLUGIN PLANTUML, USE SITEURL BY DEFAULT
   }
 }
 

publishconf.py

@@ -24,14 +24,15 @@ SITELOGO_SIZE = 36
 DEFAULT_LANG = u'fr'
 
 PLUGIN_PATHS = ['pelican-plugins/']
-PLUGINS = ['i18n_subsites', 'tipue_search', 'pelican-page-hierarchy']
+PLUGINS = ['i18n_subsites', 'tipue_search', 'pelican-page-hierarchy', 'plantuml']
 
 MARKDOWN = {
   'extension_configs': {
     'markdown.extensions.codehilite': { 'css_class': 'highlight' },
     'markdown.extensions.fenced_code': {},
     'markdown.extensions.extra': {},
-    'markdown.extensions.toc': {}
+    'markdown.extensions.toc': {},
+    #'plantuml.plantuml_md': {'siteurl': '/en'} DOES NOT WORK, NEED A PATCH ON THE PLUGIN PLANTUML, USE SITEURL BY DEFAULT
   }
 }