How to draw straight lines on your diagrams or “Diagrams as Code”
Have you ever wondered why it is so hard to make the line straight when creating your architecture diagrams or why services in your AWS diagram are not in the same stylistics? Well, you are not the first one asking these questions and the solution could be “Diagrams as Code”
Concept
The concept of writing diagrams in code is not new. You could have seen PlantUML before, although it is not clearly “diagrams as code”, more like diagrams as text solution. But the main idea stays the same — instead of dragging blocks and drawing lines between them, like we do in draw.io or lucidchart, we write some text and get the rendered diagrams as a result.
Notwithstanding “Diagrams as Text” solutions’ flexibility in creating nearly all kinds of UML diagrams, you still need to have some skill in writing all these text commands or in some cases deploying the server to render the result. And then I found diagrams — a solution that seems to fit my desire in drawing architecture diagrams by writing code (not text commands).
Why do I need it?
If you have not decided yet whether you need “Diagrams as Code” or not, Diagrams are shouting about the following on their main page:
draw the cloud system architecture in Python code …
prototyping without any design tools …
track the architecture diagram changes in any version control system …
if it does not convince you — here is one more thing I thought about when I first saw Diagrams.
dynamical diagrams
I want to test whether it is possible to “draw” diagrams dynamically, without knowing the specific elements of my system — at least “Diagrams as Code” sounds like a solution that should allow me to do such things.
Overview
Installation
To install diagrams you should have Graphviz — additional software to render diagrams. Then install diagrams with pip:
$ pip install diagramsI am going to test diagrams with Python, but you can check out go-diagrams, if you are familiar with Go.
I personally recommend trying diagrams in the Jupyter notebooks, as they support inline diagrams, which could be really useful when experimenting.
Basics
Diagrams provide few core concepts:
- Diagrams
- Nodes
- Clusters
- Edges
Nodes are the main component of your diagrams. They could be created inside the clusters or diagrams and could be connected with edges. Nodes could also be grouped.
Diagrams and clusters could be created simply by using python context managers. The cluster should have at least one node inside to be displayed. With that being said, let’s try creating our first diagram.
We have used Diagram class, the main container in Diagrams, to create a diagram, and embedded two clusters. The bottom level cluster, named “Database cluster”, has a Database node inside it. The results look like the following:
Pretty easy, right? With just 4 lines of code, we have created a pretty diagram containing one node. Let's try creating one more node and with a connection to the existing Database node.
Note that we have assigned our nodes to the variables, created FastAPI node outside of the Database cluster, and connected two nodes using Python bitwise shift (<<) operator. The result looks like expected.
Okay, but what if we want to create 4 instances of Fast API and connect them all to our database. Should we create four nodes and use bitwise operators four times? Nope, just create a Python array!
So diagrams allow creating elements dynamically. What if we want to write something on the edge, or, say, make it another color? No problem, use with Edge class you can customize the style of your edge.
Oh, yeah, undirected edges could also be created. Just use minus operator (-).
The last thing I want to show is creating custom nodes. Custom nodes could be created easily with the diagrams — all you need is to think about the desired title and find an icon for your node. So I thought, why couldn’t doge use our application?
As you can see creating custom nodes is pretty easy and we have created pretty decent diagrams with just several lines of code.
Practical use-case
As you may have seen, the concept of creating diagrams in code is pretty simple and straightforward, but I wanted to test it in a real-life situation. As we could create elements of our diagram dynamically, why couldn’t we create a diagram for existing stack written in some of the IaC tools?
So I thought, that it would be nice to find out if it’s possible to create a diagram based on AWS stack defined in Pulumi. I have defined a simple stack consisting of:
- 2 AWS Lambdas
- SQS triggering the first lambda
- AWS DynamoDB Stream triggering the second lambda
Lambdas were located in the different subnets in the same VPC.
Unfortunately, it is not possible to use Pulumi defined resources in Python code (at least I didn’t find a way to do so), because every program that uses Pulumi should be executed using Pulumi CLI and Pulumi resource are being created asynchronously, which makes it impossible to know whether an attribute was defined or not. So I have just exported Pulumi stack to .json file and then parsed the resources to created diagrams.
The code looks like so
As you can see, a lot of calling functions inside other functions were needed, because diagrams requires the nodes to be created inside context managers. Creating connections here was a little bit tricky: firstly, the SQS and DynamoDB Streams resources should be created outside of VPC, and to do so, we have to somehow pass the created lambda nodes to the outside. Using a global array seemed like the easiest way to do it. Secondly, extracting all the resource names from the Pulumi stack is not so pretty :P
Notice, that we haven’t defined any of the resources explicitly. Everything is created based on Pulumi stack .json resources. As a result, a pretty little diagram was created:
Pretty impressive, right? With just 40 lines of code, we were able to draw a diagram of a stack, defined in an IaC tool. One more time: we haven’t defined any of the resources explicitly. For me, such an opportunity sounds like a good thing — diagrams allowed us to look at the stack, without knowing how things work there, how stuff related, and where resources are located. Furthermore, if the stack will be changed — the diagram will also be changed.
Summary
So, diagrams as code is a pretty interesting concept and diagrams is a great way of creating diagrams in Python. And though it is hard to imagine Solution Architects to immediately switch from traditional tools like draw.io or lucidchart to DaC, I see the future of such tools in autogenerating diagrams based on Stacks created in Cloud.
References
Source code of Pulumi stack based diagram is available in my repository
