_images/toyplot.png

Generating Neural Network Diagrams

The following explores how Toyplot’s graph visualization can be used to generate high-quality diagrams of neural networks.

Network Data

First, we will define the edges (weights) in our network, by explicitly listing the source and target for each edge:

[1]:
import numpy
import toyplot

numpy.random.seed(1234)

edges = numpy.array([
    ["x0", "a0"],
    ["x0", "a1"],
    ["x0", "a2"],
    ["x0", "a3"],
    ["x1", "a0"],
    ["x1", "a1"],
    ["x1", "a2"],
    ["x1", "a3"],
    ["x2", "a0"],
    ["x2", "a1"],
    ["x2", "a2"],
    ["x2", "a3"],
    ["a0", "y0"],
    ["a0", "y1"],
    ["a1", "y0"],
    ["a1", "y1"],
    ["a2", "y0"],
    ["a2", "y1"],
    ["a3", "y0"],
    ["a3", "y1"],
])

Network Layout

As a straw-man, we can quickly render a graph using just the edge data:

[2]:
canvas, axes, mark = toyplot.graph(edges)
a0a1a2a3x0x1x2y0y1

Clearly, this needs work - Toyplot’s default force-directed layout algorithm obscures the fact that our neural network is organized in layers. What we want is to put all of the x nodes in the first (input) layer, all of the a nodes in a second (hidden) layer, and all of the y nodes in the last (output) layer. Since Toyplot doesn’t have a graph layout algorithm that can do that for us, we’ll have to compute the vertex coordinates ourselves:

[3]:
vertex_ids = numpy.unique(edges)

layer_map = {"x": 0, "a": -1, "y": -2}
offset_map = {"x": 0.5, "a": 0, "y": 1}
vcoordinates = []
for vertex_id in vertex_ids:
    layer = vertex_id[0]
    column = int(vertex_id[1:])
    x = column + offset_map[layer]
    y = layer_map[layer]
    vcoordinates.append((x, y))
vcoordinates = numpy.array(vcoordinates)

Now, we can see what the graph looks like with our explicitly defined coordinates:

[4]:
canvas, axes, mark = toyplot.graph(edges, vcoordinates=vcoordinates)
a0a1a2a3x0x1x2y0y1

Vertex and Edge Styles

With the graph layout looking better, we can begin to work on the appearance of the vertices and edges:

[5]:
canvas, axes, mark = toyplot.graph(
    edges,
    ecolor="black",
    tmarker=">",
    vcolor="white",
    vcoordinates=vcoordinates,
    vmarker="o",
    vsize=50,
    vstyle={"stroke":"black"},
    width=500,
    height=500,
)

# So we can control the aspect ratio of the figure using the canvas width & height
axes.aspect=None

# Prevent large vertex markers from falling outside the canvas
axes.padding=50
a0a1a2a3x0x1x2y0y1

In many cases, we might not want to see the vertex labels:

[6]:
canvas, axes, mark = toyplot.graph(
    edges,
    ecolor="black",
    tmarker=">",
    vcolor="white",
    vcoordinates=vcoordinates,
    vlshow=False,
    vmarker="o",
    vsize=50,
    vstyle={"stroke":"black"},
    width=500,
    height=500,
)
axes.aspect=None
axes.padding=50

Or we might want to substitute our own, explicit vertex labels, to illustrate the values of individual activation units during network evaluation:

[7]:
vertex_values = numpy.random.uniform(size=len(vertex_ids))
vertex_labels = ["%.2f" % value for value in vertex_values]

canvas, axes, mark = toyplot.graph(
    edges,
    ecolor="black",
    tmarker=">",
    vcolor="white",
    vcoordinates=vcoordinates,
    vlabel=vertex_labels,
    vmarker="o",
    vsize=50,
    vstyle={"stroke":"black"},
    width=500,
    height=500,
)
axes.aspect=None
axes.padding=50
0.190.620.440.790.780.270.280.800.96

Edge Weights

We might also want to display the network edge weights. Edge middle markers are a good choice to do this:

[8]:
edge_weights = numpy.random.uniform(size=len(edges))
mstyle = {"fill": "white"}
lstyle = {"font-size": "12px"}
mmarkers = [toyplot.marker.create(shape="s", label="%.1f" % weight, size=30, mstyle=mstyle, lstyle=lstyle) for weight in edge_weights]

canvas, axes, mark = toyplot.graph(
    edges,
    ecolor="black",
    mmarker=mmarkers,
    tmarker=">",
    vcolor="white",
    vcoordinates=vcoordinates,
    vlabel=vertex_labels,
    vmarker="o",
    vsize=50,
    vstyle={"stroke":"black"},
    width=500,
    height=500,
)
axes.aspect=None
axes.padding=50
0.90.40.50.70.70.40.60.50.00.80.90.40.60.10.40.90.70.40.80.30.190.620.440.790.780.270.280.800.96

Note that the middle markers are aligned with the edges, making the weight values difficult to read. To fix this, we can set an explicit orientation for the middle markers:

[9]:
mmarkers = [toyplot.marker.create(angle=0, shape="s", label="%.1f" % weight, size=30, mstyle=mstyle, lstyle=lstyle) for weight in edge_weights]

canvas, axes, mark = toyplot.graph(
    edges,
    ecolor="black",
    mmarker=mmarkers,
    tmarker=">",
    vcolor="white",
    vcoordinates=vcoordinates,
    vlabel=vertex_labels,
    vmarker="o",
    vsize=50,
    vstyle={"stroke":"black"},
    width=500,
    height=500,
)
axes.aspect=None
axes.padding=50
0.90.40.50.70.70.40.60.50.00.80.90.40.60.10.40.90.70.40.80.30.190.620.440.790.780.270.280.800.96

Now however, many of the middle markers overlap, making it appear as if there are fewer weights than edges. One way to address this is to randomly reposition the markers so that they rarely overlap:

[10]:
mposition = numpy.random.uniform(0.1, 0.8, len(edges))

canvas, axes, mark = toyplot.graph(
    edges,
    ecolor="black",
    mmarker=mmarkers,
    mposition=mposition,
    tmarker=">",
    vcolor="white",
    vcoordinates=vcoordinates,
    vlabel=vertex_labels,
    vmarker="o",
    vsize=50,
    vstyle={"stroke":"black"},
    width=500,
    height=500,
)
axes.aspect=None
axes.padding=50
0.90.40.50.70.70.40.60.50.00.80.90.40.60.10.40.90.70.40.80.30.190.620.440.790.780.270.280.800.96

Note that the mposition argument is a value between zero and one that positions each middle marker anywhere along its edge from beginning to end, respectively.

Layer-Only Diagrams

Once a network reaches a certain level of complexity, it is typical to only diagram the layers in the network, instead of all the activation units. Here, we define per-layer data for a simple layer-only diagram:

[11]:
layers = [
    "<b>conv1</b><br/>3&#215;3 convolutional",
    "<b>pool1</b><br/>max pooling",
    "<b>fc_1</b><br/>4096 dense",
    "<b>fc_2</b><br/>1000 dense softmax",
]

vertex_ids = numpy.arange(len(layers))

edges = numpy.column_stack((
    vertex_ids[:-1],
    vertex_ids[1:],
))

vcoordinates = numpy.column_stack((
    numpy.zeros_like(layers, dtype="float"),
    numpy.arange(0, -len(layers), -1),
    ))

In this case, it’s useful to use Toyplot’s special rectangular markers for the graph nodes:

[12]:
canvas, axes, mark = toyplot.graph(
    edges,
    ecolor="black",
    tmarker=">",
    vcoordinates=vcoordinates,
    vlabel=layers,
    vmarker=toyplot.marker.create("r3x1", lstyle={"font-size":"12px"}, size=50),
    vstyle={"stroke":"black", "fill":"white"},
    width=200,
    height=400,
)
axes.padding = 50
conv13×3 convolutionalpool1max poolingfc_14096 densefc_21000 dense softmax