{
"cells": [
{
"cell_type": "raw",
"metadata": {
"raw_mimetype": "text/restructuredtext"
},
"source": [
".. _neural-networks-case-study:\n",
"\n",
"Generating Neural Network Diagrams\n",
"==================================\n",
"\n",
"The following explores how Toyplot's graph visualization can be used to generate high-quality diagrams of neural networks."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Network Data\n",
"\n",
"First, we will define the edges (weights) in our network, by explicitly listing the source and target for each edge:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy\n",
"import toyplot\n",
"\n",
"numpy.random.seed(1234)\n",
"\n",
"edges = numpy.array([\n",
" [\"x0\", \"a0\"],\n",
" [\"x0\", \"a1\"],\n",
" [\"x0\", \"a2\"],\n",
" [\"x0\", \"a3\"],\n",
" [\"x1\", \"a0\"],\n",
" [\"x1\", \"a1\"],\n",
" [\"x1\", \"a2\"],\n",
" [\"x1\", \"a3\"],\n",
" [\"x2\", \"a0\"],\n",
" [\"x2\", \"a1\"],\n",
" [\"x2\", \"a2\"],\n",
" [\"x2\", \"a3\"],\n",
" [\"a0\", \"y0\"],\n",
" [\"a0\", \"y1\"],\n",
" [\"a1\", \"y0\"],\n",
" [\"a1\", \"y1\"],\n",
" [\"a2\", \"y0\"],\n",
" [\"a2\", \"y1\"],\n",
" [\"a3\", \"y0\"],\n",
" [\"a3\", \"y1\"],\n",
"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Network Layout\n",
"\n",
"As a straw-man, we can quickly render a graph using just the edge data:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"canvas, axes, mark = toyplot.graph(edges)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Clearly, this needs work - Toyplot's default force-directed layout algorithm obscures the fact that our neural network is organized in layers. What we want is to put all of the `x` nodes in the first (input) layer, all of the `a` nodes in a second (hidden) layer, and all of the `y` nodes in the last (output) layer. Since Toyplot doesn't have a graph layout algorithm that can do that for us, we'll have to compute the vertex coordinates ourselves:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"vertex_ids = numpy.unique(edges)\n",
"\n",
"layer_map = {\"x\": 0, \"a\": -1, \"y\": -2}\n",
"offset_map = {\"x\": 0.5, \"a\": 0, \"y\": 1}\n",
"vcoordinates = []\n",
"for vertex_id in vertex_ids:\n",
" layer = vertex_id[0]\n",
" column = int(vertex_id[1:])\n",
" x = column + offset_map[layer]\n",
" y = layer_map[layer]\n",
" vcoordinates.append((x, y))\n",
"vcoordinates = numpy.array(vcoordinates)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we can see what the graph looks like with our explicitly defined coordinates:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"canvas, axes, mark = toyplot.graph(edges, vcoordinates=vcoordinates)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Vertex and Edge Styles\n",
"\n",
"With the graph layout looking better, we can begin to work on the appearance of the vertices and edges:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"canvas, axes, mark = toyplot.graph(\n",
" edges,\n",
" ecolor=\"black\",\n",
" tmarker=\">\",\n",
" vcolor=\"white\",\n",
" vcoordinates=vcoordinates,\n",
" vmarker=\"o\",\n",
" vsize=50,\n",
" vstyle={\"stroke\":\"black\"},\n",
" width=500,\n",
" height=500,\n",
")\n",
"\n",
"# So we can control the aspect ratio of the figure using the canvas width & height\n",
"axes.aspect=None\n",
"\n",
"# Prevent large vertex markers from falling outside the canvas\n",
"axes.padding=50"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In many cases, we might not want to see the vertex labels:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"vertex_values = numpy.random.uniform(size=len(vertex_ids))\n",
"vertex_labels = [\"%.2f\" % value for value in vertex_values]\n",
"\n",
"canvas, axes, mark = toyplot.graph(\n",
" edges,\n",
" ecolor=\"black\",\n",
" tmarker=\">\",\n",
" vcolor=\"white\",\n",
" vcoordinates=vcoordinates,\n",
" vlabel=vertex_labels,\n",
" vmarker=\"o\",\n",
" vsize=50,\n",
" vstyle={\"stroke\":\"black\"},\n",
" width=500,\n",
" height=500,\n",
")\n",
"axes.aspect=None\n",
"axes.padding=50"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Edge Weights\n",
"\n",
"We might also want to display the network edge weights. Edge *middle* markers are a good choice to do this:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"edge_weights = numpy.random.uniform(size=len(edges))\n",
"mstyle = {\"fill\": \"white\"}\n",
"lstyle = {\"font-size\": \"12px\"}\n",
"mmarkers = [toyplot.marker.create(shape=\"s\", label=\"%.1f\" % weight, size=30, mstyle=mstyle, lstyle=lstyle) for weight in edge_weights]\n",
"\n",
"canvas, axes, mark = toyplot.graph(\n",
" edges,\n",
" ecolor=\"black\",\n",
" mmarker=mmarkers,\n",
" tmarker=\">\",\n",
" vcolor=\"white\",\n",
" vcoordinates=vcoordinates,\n",
" vlabel=vertex_labels,\n",
" vmarker=\"o\",\n",
" vsize=50,\n",
" vstyle={\"stroke\":\"black\"},\n",
" width=500,\n",
" height=500,\n",
")\n",
"axes.aspect=None\n",
"axes.padding=50"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the middle markers are aligned with the edges, making the weight values difficult to read. To fix this, we can set an explicit orientation for the middle markers:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"mmarkers = [toyplot.marker.create(angle=0, shape=\"s\", label=\"%.1f\" % weight, size=30, mstyle=mstyle, lstyle=lstyle) for weight in edge_weights]\n",
"\n",
"canvas, axes, mark = toyplot.graph(\n",
" edges,\n",
" ecolor=\"black\",\n",
" mmarker=mmarkers,\n",
" tmarker=\">\",\n",
" vcolor=\"white\",\n",
" vcoordinates=vcoordinates,\n",
" vlabel=vertex_labels,\n",
" vmarker=\"o\",\n",
" vsize=50,\n",
" vstyle={\"stroke\":\"black\"},\n",
" width=500,\n",
" height=500,\n",
")\n",
"axes.aspect=None\n",
"axes.padding=50"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now however, many of the middle markers overlap, making it appear as if there are fewer weights than edges. One way to address this is to randomly reposition the markers so that they rarely overlap:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"mposition = numpy.random.uniform(0.1, 0.8, len(edges))\n",
"\n",
"canvas, axes, mark = toyplot.graph(\n",
" edges,\n",
" ecolor=\"black\",\n",
" mmarker=mmarkers,\n",
" mposition=mposition,\n",
" tmarker=\">\",\n",
" vcolor=\"white\",\n",
" vcoordinates=vcoordinates,\n",
" vlabel=vertex_labels,\n",
" vmarker=\"o\",\n",
" vsize=50,\n",
" vstyle={\"stroke\":\"black\"},\n",
" width=500,\n",
" height=500,\n",
")\n",
"axes.aspect=None\n",
"axes.padding=50"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the `mposition` argument is a value between zero and one that positions each middle marker anywhere along its edge from beginning to end, respectively."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Layer-Only Diagrams\n",
"\n",
"Once a network reaches a certain level of complexity, it is typical to only diagram the layers in the network, instead of all the activation units. Here, we define per-layer data for a simple layer-only diagram:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"layers = [\n",
" \"conv1 3×3 convolutional\",\n",
" \"pool1 max pooling\",\n",
" \"fc_1 4096 dense\",\n",
" \"fc_2 1000 dense softmax\",\n",
"]\n",
"\n",
"vertex_ids = numpy.arange(len(layers))\n",
"\n",
"edges = numpy.column_stack((\n",
" vertex_ids[:-1],\n",
" vertex_ids[1:],\n",
"))\n",
"\n",
"vcoordinates = numpy.column_stack((\n",
" numpy.zeros_like(layers, dtype=\"float\"),\n",
" numpy.arange(0, -len(layers), -1),\n",
" ))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" In this case, it's useful to use Toyplot's special rectangular markers for the graph nodes:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"