Lesson Learned: Managing Large Numbers of Plots, with an Example in Python

Problem Statement:

A project generates hundreds, thousands, or more graphs over its life. These graphs are copied and pasted into e-mail, power point slides, etc. The plots become divorced from any of the documents they were originally distributed with. Invariably, at some point in the project, a plot is brought back and the question is what were the assumptions used to generate this graph. With only the graph available, it can be difficult or impossible to answer this question.

To complicate matters, the plots are generated using legacy codes and modifying all of the existing code base is a detailed endeavor.

How can this situation be improved?

Discussion:

There are two problems here. First, a given graph is not traceable to its origin. This can be remedied in one of many ways.  If the source data is well controlled and can be described using a short phase, then adding that phrase somewhere on the chart is helpful. If the source data is constantly changing or requires too much information to describe with a short phrase, then something else is needed. A hash of the input data can help identify and verify the source data set used to generate the graph. A universally unique ID (UUID) can be used to give a graph a unique name. If the source data, assumptions, etc are stored using that same UUID, then when a graph if brought back for review, all of the necessary parts can be found.

The second problem is handling legacy code. There are at least three choices.

• The first is probably the easiest. A function to add hash and uuid could be created and inserted at the appropriate location in each of the major pieces of plotting code. This is problematic because there are several interfaces and actions in the scripts which could make this work poorly. Also, every plotting routine would need to be modified.
• A second choice is to wrap the plotting routines into function, then pass this function and its data to a wrapper function which would add a hash and uuid as the last thing done by the plotting routine. If the plotting functions already exist, then this can be done without changing any of the plotting code.   
• A third choice is to create a decorator which wraps plotting routines, adding the hash and uuid. This has the same issues as using a wrapper call and requires changes to the plotting routines source. However, the changes consist of an import statement and application of the decorator at the correct location.

For this problem, the decorator solution is used. The code that following implements a decorator that creates a hash of the plot data and a UUID which are added to the right side of the plot.  This way, no matter where the plot goes, there is a high likelihood that its pedigree can be preserved.

'''
Script to demonstrate the use of decorators to add a unique identifier to
   a plot. The identifier incudes a hash of input data, to help see if 
   version of a plot really have different data, and a UUID to uniquely 
   identify this plot independent of when or where it was generated.
'''

__author__  = 'Ed Tate'
__email__   = 'edtate<AT>gmail-dot-com'
__website__ = 'exnumerus.blogspot.com'
__license__ = 'Creative Commons Attribute By - http://creativecommons.org/licenses/by/3.0/us/'

from matplotlib.pylab import *
import random
import md5
import uuid

def identifiable_plot(fn):
    def newfun(*args,**kwargs):
        # do before decorated function call
        fn(*args,**kwargs)
        # do after decorated function call
        # create the tag string from a hash of the data and a 
        #    universially unique ID
        x = args[0]
        y = args[1]
        xy = zip(x,y)
        m = md5.new()
        m.update(str(xy))
        this_uuid = str(uuid.uuid1())
        this_tag = 'hash=' + m.hexdigest() + ',' + 'UUID=' + this_uuid
        # write the tag to the figure for future reference
        figtext(1,0.5,this_tag ,rotation='vertical',
                horizontalalignment='right',
                verticalalignment='center',
                size = 'x-small',
                )
        return this_uuid

    return newfun

###############################
    
@identifiable_plot
def my_plot(x,y):
    plot(x,y,'o')
    grid(True)
    
###############################
    
x = [random.random() for i in range(100)]
y = [random.random() for i in range(100)]
    
plot_uuid = my_plot(x,y)
savefig(plot_uuid+'.png')

show()

 

---

Test Configuration:

• win7
• PythonXY 2.7.2.1

---

References:

• http://www.siafoo.net/article/68
• http://wiki.python.org/moin/PythonDecoratorLibrary
• http://avinashv.net/2008/04/python-decorators-syntactic-sugar/
• http://stackoverflow.com/questions/739654/understanding-python-decorators

This work is licensed under a Creative Commons Attribution By license.

Using HTML to View Large Sets of Plots - An Example in Python

Red_PlotBlue_PlotGreen_Plot Circle_PlotSquare_PlotDiamond_Plot

This example doesn't work because of Blogger limitations. However if you run the example you will be able to select graphs in the generated HTML page.

Problem Statement

You have a program which generates lots of similar plots that end users would like to compare and explore. The end users may not be able to install any code. You can't setup a web server to nagivate the data set. You can not install any new programs on their window's desktop. How to you provide a solution?

Discussion

You can assume that any modern computer at least has a copy of Firefox, Safari, or Explorer. Since there browers support javascript (except in the worst case security settings), you can build a very lightweight data viewer using a few simple methods. The most important design decisions when generating the plots is to name the plots so they are easy to recreate from selections an end user might make.

Example

The following snippet of Python code generates 9 graphs that have random numbers plotted on two axes using different colors and markers. There are three choices of colors and three choices of markers. After generating the plots and saving them, the script creates an HTML file which simplies the navigation of the images. A user can open the HTML page and select the graph by changing the form selections at the top of the page.

There are a couple of key concepts that help make this work:

• The plot names can be created from selections using javascript. For example, in this example there are three colors and three different markers. All of the plot file names are formed by concatenating the color and marker description to form a plot name.
• When a user changes their choice of color or marker a javascript function rebuilds the plot file name and causes the browser to reload the image by change the img source.
• The python script uses templates to set up the bulk of the HTML page, then substitutes in specific options for the user after the plots have been generated. 

import pylab as plt
from random import random
from string import Template

colors  = {'Red_Plot':'r',
           'Blue_Plot':'b',
           'Green_Plot':'g',
           }
markers = {'Circle_Plot':'o',
           'Square_Plot':'s',
           'Diamond_Plot':'d',
           } 

plt.figure()
for c_key in colors.keys():
    for m_key in markers.keys():
        plt.clf()
        plot_name = c_key + ',' + m_key + '.png'
        x = [random() for i in range(0,100)]
        y = [random() for i in range(0,100)]
        color = colors[c_key]
        marker = markers[m_key]
        plt.plot(x,y,color+marker,markersize=15)
        plt.savefig(plot_name)
        
HTML_template = Template('''<head>
   <script language="JavaScript"><!--
      function sel_plot() {
         // only do this if the brower supports images
         if(document.images) {
            // get plot color name
            var e=document.getElementById("color_name");
            var c_name = e.options[e.selectedIndex].text;
            // get plot marker name
            var e=document.getElementById("marker_name");
            var m_name = e.options[e.selectedIndex].text; // create the filename
            // build the filename from these selections
            var plot_filename = c_name + "," + m_name + ".png";
            // cause the correct plot to be loaded
            document["plot"].src = plot_filename;
            }
         }
      // select the plot to display initially after loading document
      window.onload = function() { sel_plot() };
      // silence errors
      window.onerr = null;
   </script>
</head>
<body>
   <center>
      <form name="Plot Select Form" id="plot_select_form">
         <select id="color_name" size="3"
            onchange="sel_plot()">
            $color_options
         </select>
         <select id="marker_name" size="3"
            onchange="sel_plot()">
            $marker_options
         </select>
      </form>
      <img name="plot" src="dummy.png" height="200"
         width="500">
   </center>
</body>
''')
        

# build the option strings
def build_options(opt_dict):
    s = ''
    for i,key in enumerate(opt_dict.keys()):
        if i==1:
            s += '<option selected>'
        else:
            s += '<option>'
        s += key
        s += '</option>\n'
    return s    

color_options = build_options(colors)
marker_options = build_options(markers)

# build the HTML page        
HTML = HTML_template.substitute(color_options=color_options,
                     marker_options=marker_options)
    

# write the HTML page
f = open('example.html','wb')
f.write(HTML)
f.close()

---

Test Configuration

• PythonXY 2.7.2.1
• IE 9

---

This work is licensed under a Creative Commons Attribution By license.

First Draft: A Script to Display Multicolor Voronoi Diagrams in Matplotlib

Code Sample

'''
This module generates 2D voronoi diagrams from a list of points.
The vornoi cells can be colored by supplying a value associated with 
   each node.
'''

__author__ = 'Ed Tate'
__email__  = 'edtate<at>gmail-dot-com'
__website__ = 'exnumerus.blogspot.com'
__license__ = 'Creative Commons Attribute By - http://creativecommons.org/licenses/by/3.0/us/'''

from matplotlib.pyplot import cm

def voronoi2D(xpt,ypt,cpt=None,cmap=None):
    '''
    This function returns a list of line segments which describe the voronoi
        cells formed by the points in zip(xpt,ypt).
    
    If cpt is provided, it identifies which cells should be returned.
        The boundary of the cell about (xpt[i],ypt[i]) is returned 
            if cpt[i]<=threshold.
            
    This function requires qvoronoi.exe in the working directory. 
    The working directory must have permissions for read and write access.
    This function will leave 2 files in the working directory:
        data.txt
        results.txt
    This function will overwrite these files if they already exist.
    '''
    
    if cpt is None:
        # assign a value to cpt for later use
        cpt = [0 for x in xpt]
        
    if cmap is None:
        cmap = cm.gray
    
    # write the data file
    pts_filename = 'data.txt'
    pts_F = open(pts_filename,'w')
    pts_F.write('2 # this is a 2-D input set\n')
    pts_F.write('%i # number of points\n' % len(xpt))
    for i,(x,y) in enumerate(zip(xpt,ypt)):
        pts_F.write('%f %f # data point %i\n' % (x,y,i))
    pts_F.close()

    # trigger the shell command
    import subprocess
    p = subprocess.Popen('qvoronoi TI data.txt TO results.txt p FN Fv QJ', shell=True)
    p.wait()

    # open the results file and parse results
    results = open('results.txt','r')

    # get 'p' results - the vertices of the voronoi diagram
    data = results.readline()
    voronoi_x_list = []
    voronoi_y_list = []
    data = results.readline()
    for i in range(0,int(data)):
        data = results.readline()
        xx,yy,dummy = data.split(' ')    
        voronoi_x_list.append(float(xx))
        voronoi_y_list.append(float(yy))
        
    # get 'FN' results - the voronoi edges
    data = results.readline()
    voronoi_idx_list = []
    for i in range(0,int(data)):
        data = results.readline()
        this_list = data.split(' ')[:-1]
        for j in range(len(this_list)):
            this_list[j]=int(this_list[j])-1
        voronoi_idx_list.append(this_list[1:])
        
    # get 'FV' results - pairs of points which define a voronoi edge
    # combine these results to build a complete representation of the 
    data = results.readline()
    voronoi_dict = {}
    for i in range(0,int(data)):
        data = results.readline().split(' ')

        pair_idx_1 = int(data[1])
        pair_idx_2 = int(data[2])

        vertex_idx_1 = int(data[3])-1
        vertex_idx_2 = int(data[4])-1

        try:
            voronoi_dict[pair_idx_1].append({ 'edge_vertices':[vertex_idx_1,vertex_idx_2],
                                      'neighbor': pair_idx_2 })
        except KeyError:
            voronoi_dict[pair_idx_1] = [{ 'edge_vertices':[vertex_idx_1,vertex_idx_2],
                                      'neighbor': pair_idx_2 } ]

        try:
            voronoi_dict[pair_idx_2].append({ 'edge_vertices':[vertex_idx_1,vertex_idx_2],
                                      'neighbor': pair_idx_1 })
        except KeyError:
            voronoi_dict[pair_idx_2] = [{ 'edge_vertices':[vertex_idx_1,vertex_idx_2],
                                      'neighbor': pair_idx_1 } ]    

                    
    #################
    # generate a collection of voronoi cells
    result_list = []
    for point_idx in voronoi_dict.keys():
        # determine cell color
        this_color = cmap(cpt[point_idx])
        
        # display this cell, so add the data to the edge list
        e_list = []
        for edge in voronoi_dict[point_idx]:
            p1_idx = edge['edge_vertices'][0]
            p2_idx = edge['edge_vertices'][1]
            e_list.append((p1_idx,p2_idx))
        
        # put the vertices points in order so they
        #   walk around the voronoi cells
        p_list = [p1_idx]
        while True:
            p=p_list[-1]
            for e in e_list:
                if p==e[0]:
                    next_p = e[1]
                    break
                elif p==e[1]:
                    next_p = e[0]
                    break
            p_list.append(next_p)
            e_list.remove(e)
            if p_list[0]==p_list[-1]:
                # the cell is closed
                break
            
        # build point list
        this_x_list = []
        this_y_list = []
        if all([p>=0 for p in p_list]):
            for p in p_list:
                if p>=0:
                    this_x_list.append(voronoi_x_list[p])
                    this_y_list.append(voronoi_y_list[p])     
                    
        result_list.append([this_x_list,this_y_list,this_color])
        
    return result_list

if __name__=='__main__':
    
    import random
    
    xpt = [random.random()-0.5 for i in range(0,100)]
    ypt = [random.random()-0.5 for i in range(0,100)]
    cpt = [int(random.random()*255) for i in range(0,100)]

    import matplotlib.pyplot as pp
    
    result_list = voronoi2D(xpt,ypt,cpt,cmap=pp.cm.copper)
    
    pp.figure()
    for item in result_list:
        x_list = item[0]
        y_list = item[1]
        this_color = item[2]
        pp.fill(x_list,y_list,color=this_color,edgecolor='none')
    
   
    pp.axis([-0.5,0.5,-0.5,0.5])

    pp.show()
    
    
    

This work is licensed under a Creative Commons Attribution By license.

Finding the shortest path between two points: An example of the A* algorithm in Python

Introduction:

One common challenge is finding the shortest or least expensive path between two points. This simple problem can represent many different engineering and information processing problems. One of the most familiar versions of this problem is finding the shortest or fastest way to travel through between two points on a map.A famous variant of this problem is the traveling salesman problem. To set up this kind of problem, a directed graph needs to be created. A directed graph defines the connection between nodes. It also defines the cost to travel between nodes.

Given an origin and a destination in the graph, there are many ways to find the shortest path (or least cost) path between the origin and destination. One way which is usually intractable is to do a brute force search. However, of the tractable ways to solve the problem, the A* algorithm can be fast and efficient if a few conditions are met: an approximate estimate of the cost to reach the destination from every point in the graph is available and this approximation is always equal to or less than the optimal cost to reach the destination. 

---

An Example Problem:

A shortest path problem has the goal of finding a path through a graph which costs the least. Consider the simple graph shown below. Each node is represented by a red circle. The ways to travel between the nodes (the edges, arcs, arrows, etc) are shown by arrows between the nodes. each node has a name which is called out. To make the graph interesting, there is no way to travel in one step between the pair of nodes in 002 and 003 and the other pair of 013 and 014. To keep the problem simple, the cost to travel on every edge is equal to the Euclidean distance along the edge.

The Astar algorithm can find a route from an origin to a destination node. For this example, the goal is to find a minimum cost route from node 001 to node 015. The graph is stored as a dictionary which has an entry for each node. This entry has a dictionary which defined the x and y coordinates of the node along with a list of nodes which can be reached. Furthermore, the approximate cost from each node to the destination is defined as the Euclidean distance from a given node to the destination. This distance satisfies the requirements for a good approximation in this model because it will always be equal or less than the best distance that can be achieved following the edges.

This implementation of the Astar algorithm is almost identical to the algorithm outlined in Wikipedia.

Once the Astar algorithm solves for the shortest path, the shortest path is visualized by laying it on top of the graph.

One key result can easily be seen in this example. By looking carefully at the graph, it should be obvious that there are other routes which can travel between the origin and destination with  the same distance or cost. For some problems, there is not a single shortest path, there are potentially many paths which have the same cost. This algorithm will generate only one. There may be other routes which are the shortest path.

---

Code Example:

Download

def Astar(start,goal,dist_between,neighbor_nodes,heuristic_estimate_of_dist):
    closedset = set([])             # The set of nodes already evaluated.     
    openset   = set([start])    # The set of tentative nodes to be evaluated.
                                    # starts off containing initial node
    came_from = set([])             # The map of navigated nodes.
    
    g_score={start:0}             # Distance from start along optimal path.
    h_score={start:heuristic_estimate_of_dist(start,goal)}
    f_score={start:h_score[start]}  #Estimated total distance from start to goal through y.
    came_from={start:None}
    
    while len(openset)>0:   # open set is not empty
         #x := the node in openset having the lowest f_score[] value
        bestX = None
        for x in openset:
            if bestX==None:
                bestX = x
            elif f_score[x] < f_score[bestX]:
                bestX = x
        x = bestX
        if x == goal:
            return reconstruct_path(came_from,came_from[goal])
        openset.discard(x)
        closedset.add(x)
        
        neighbor_list = neighbor_nodes(x)
        for y in neighbor_nodes(x):
            if y in closedset:
                continue
            tentative_g_score = g_score[x] + dist_between(x,y)

            if y not in openset:
                openset.add(y)
                tentative_is_better = True
            elif tentative_g_score <  g_score[y]:
                tentative_is_better = True
            else:
                tentative_is_better = False
            if tentative_is_better == True:
                 came_from[y] = x
                 g_score[y] = tentative_g_score
                 h_score[y] = heuristic_estimate_of_dist(y, goal)
                 f_score[y] = g_score[y] + h_score[y]
    return None
 
def reconstruct_path(came_from, current_node):
     if not came_from[current_node] == None:
        p = reconstruct_path(came_from, came_from[current_node])
        return (p + [current_node])
     else:
        return [current_node]
    
    
    
#####################################

if __name__=='__main__':

    import pylab as p


    graph = {
        '001':dict(x=1, y=0,nextList=['002','011']),
#        '002':dict(x=2, y=0,nextList=['001','003','012']),
#        '003':dict(x=3, y=0,nextList=['002','004','013']),
        '002':dict(x=2, y=0,nextList=['001','012']),
        '003':dict(x=3, y=0,nextList=['004','013']),
        '004':dict(x=4, y=0,nextList=['003','005','014']),
        '005':dict(x=5, y=0,nextList=['004','015']),
        '011':dict(x=1, y=1,nextList=['012','001']),
        '012':dict(x=2, y=1,nextList=['011','013','002']),
#        '013':dict(x=3, y=1,nextList=['012','014','003']),
#        '014':dict(x=4, y=1,nextList=['013','015','004']),
        '013':dict(x=3, y=1,nextList=['012','003']),
        '014':dict(x=4, y=1,nextList=['015','004']),
        '015':dict(x=5, y=1,nextList=['014','005']),
        }
        
    def neighbor_nodes(x):
        return graph[x]['nextList']

    def distance_between(x,y):
        _x1 = graph[x]['x']
        _x2 = graph[y]['x']
        _y1 = graph[x]['y']
        _y2 = graph[y]['y']
        
        return ((_x1-_x2)**2+(_y1-_y2)**2)**(0.5)
        

    def drawArrow(xFrom,xTo,yFrom,yTo):
        length = ((xFrom-xTo)**2+(yFrom-yTo)**2)**(0.5)
        head_len = 0.1
        head_width = 0.05
        dx = ((length-head_len)/length)*(xTo-xFrom)
        dy = ((length-head_len)/length)*(yTo-yFrom)
        p.arrow(xFrom,yFrom,dx,dy,
                head_width=head_width,head_length=head_len)

    def plotGraph(graph,ax):
        first = True
        for origin in graph.keys():
            for dest in graph[origin]['nextList']:
                xFrom = graph[origin]['x']
                xTo   = graph[dest]['x']
                yFrom = graph[origin]['y']
                yTo   = graph[dest]['y']
                drawArrow(xFrom,xTo,yFrom,yTo)
                if first:
                   minX = xFrom
                   maxX = xFrom
                   minY = yFrom
                   maxY = yFrom
                   first = False
                else:
                   minX = min(minX,xFrom)
                   maxX = max(maxX,xFrom)
                   minY = min(minY,yFrom)
                   maxY = max(maxY,yFrom)
            p.plot([xFrom],[yFrom],'or')
            ax.annotate(origin, xy=(xFrom,yFrom),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                size=20,
                #bbox=dict(boxstyle="round", fc="0.8"),
                arrowprops=dict(arrowstyle="fancy",
                                fc="0.6", ec="none",
                #                patchB=el,
                                connectionstyle="angle3,angleA=0,angleB=-90"),
                )

        #p.axis([minX-0.25,maxX+0.25,minY-0.25,maxY+0.25])

    def plotRoute(route,graph):
        for idx,point in enumerate(route):
            if idx < len(route)-1:
                nextPoint = route[idx+1]
                xFrom = graph[point]['x']
                xTo   = graph[nextPoint]['x']
                yFrom = graph[point]['y']
                yTo   = graph[nextPoint]['y']
                p.plot([xFrom,xTo],[yFrom,yTo],'-g',lw=10,alpha=0.5,solid_capstyle='round')


    fig = p.figure()
    ax = fig.add_subplot(111, autoscale_on=False, xlim=(0,5.5), ylim=(-0.5,1.75))

    route = Astar('001','015',distance_between,neighbor_nodes,distance_between) 
    route.append('015')
    print route
    plotRoute(route,graph)
    plotGraph(graph,ax)
    print 'Done'
    p.show()

---

References:

• Directed Graph
  
• A* algorithm
  
• Shortest Path Problems

Beta release: How to render OpenStreetMaps using Python and Matplotlib

Introduction:

Open Street Maps is an collaboratively developed and open source map database. The entire OSM database or specific portions can be downloaded. The database includes information on roads, political boundaries, natural features, rail lines, trails, and much more. The website can generate output files with sufficient information to create custom and detailed surface street maps for many areas of the world. With Python and Matplotlib the XML data in an OpenStreetMaps OSM file can be rendered as a custom map can be rendered to meet specific needs. Below is an example of a map which can be generated from and OSM data file.

This is a small snippet of code which makes it relatively easy to see how to use the raw data from Open Street Maps. For a complete rendering examples see the Open Street Maps wiki and the pyrender source code.

Key Concepts:

An OSM file is generated by selecting an area of interest at www.OpenStreetMap.org and selecting the export tab. Under the export tab, there is an option to export the map to 'OpenStreetMap XML Data'. If this is selected, then an XML file which contains all of the data for the selected region is generated. This XML file includes road, railways, trails, and land features among other things. The XML file has a a few header elements which define the XML standard, the data source, and the bounds on the data set. After that, the data file consists of a long list of nodes in the map. Each node describes one point on the surface of the earth. After the nodes, the ways are defined. A way consists of a list of nodes and tags. The nodes are are order and describe something like a road or a lake. The tag provide sufficient data to understand what the way represents. After the ways, come relations. Relations combine other elements and provide tags which area associated with those elements.

To render roads, only nodes and ways need to be considered.

When plotting lots of elements, Matplotlib can be sped up significantly (3X or more) by turning off autoscaling.

Since the ways are not ordered in the OSM file, z-ordering in matplotlib provides a nice way to control what has the most prominence.

So that the roads which are wider than 1 pixel have smooth transitions, the solid_capstyle and the solid_endstyle are set to ‘round’.

To simplify conversion of the XML data in the OSM file into an object which behaves like a dictionary, a script from ActiveState was used.

The Code:

download source code

download example OSM file

#############################################################
## from http://code.activestate.com/recipes/534109-xml-to-python-data-structure/

import re
import xml.sax.handler

def xml2obj(src):
    """
    A simple function to converts XML data into native Python object.
    """

    non_id_char = re.compile('[^_0-9a-zA-Z]')
    def _name_mangle(name):
        return non_id_char.sub('_', name)

    class DataNode(object):
        def __init__(self):
            self._attrs = {}    # XML attributes and child elements
            self.data = None    # child text data
        def __len__(self):
            # treat single element as a list of 1
            return 1
        def __getitem__(self, key):
            if isinstance(key, basestring):
                return self._attrs.get(key,None)
            else:
                return [self][key]
        def __contains__(self, name):
            return self._attrs.has_key(name)
        def __nonzero__(self):
            return bool(self._attrs or self.data)
        def __getattr__(self, name):
            if name.startswith('__'):
                # need to do this for Python special methods???
                raise AttributeError(name)
            return self._attrs.get(name,None)
        def _add_xml_attr(self, name, value):
            if name in self._attrs:
                # multiple attribute of the same name are represented by a list
                children = self._attrs[name]
                if not isinstance(children, list):
                    children = [children]
                    self._attrs[name] = children
                children.append(value)
            else:
                self._attrs[name] = value
        def __str__(self):
            return self.data or ''
        def __repr__(self):
            items = sorted(self._attrs.items())
            if self.data:
                items.append(('data', self.data))
            return u'{%s}' % ', '.join([u'%s:%s' % (k,repr(v)) for k,v in items])

    class TreeBuilder(xml.sax.handler.ContentHandler):
        def __init__(self):
            self.stack = []
            self.root = DataNode()
            self.current = self.root
            self.text_parts = []
        def startElement(self, name, attrs):
            self.stack.append((self.current, self.text_parts))
            self.current = DataNode()
            self.text_parts = []
            # xml attributes --> python attributes
            for k, v in attrs.items():
                self.current._add_xml_attr(_name_mangle(k), v)
        def endElement(self, name):
            text = ''.join(self.text_parts).strip()
            if text:
                self.current.data = text
            if self.current._attrs:
                obj = self.current
            else:
                # a text only node is simply represented by the string
                obj = text or ''
            self.current, self.text_parts = self.stack.pop()
            self.current._add_xml_attr(_name_mangle(name), obj)
        def characters(self, content):
            self.text_parts.append(content)

    builder = TreeBuilder()
    if isinstance(src,basestring):
        xml.sax.parseString(src, builder)
    else:
        xml.sax.parse(src, builder)
    return builder.root._attrs.values()[0]



#############################################################



def render(myMap):



    # make dictionary of node IDs
    nodes = {}
    for node in myMap['node']:
        nodes[node['id']] = node
        
    ways = {}
    for way in myMap['way']:
        ways[way['id']]=way



    import pylab as p


    renderingRules = {
        'primary': dict(
                linestyle       = '-',
                linewidth       = 6,
                color           ='#ee82ee', 
                zorder          = -1,
                ),
        'primary_link': dict(
                linestyle       = '-',
                linewidth       = 6,
                color           = '#da70d6',
                zorder          = -1,            
                ),
        'secondary': dict(
                linestyle       = '-',
                linewidth       = 6,
                color           = '#d8bfd8',
                zorder          = -2,            
                ),
        'secondary_link': dict(
                linestyle       = '-',
                linewidth       = 6,
                color           = '#d8bfd8',
                zorder          = -2,            
                ),
        'tertiary': dict(
                linestyle       = '-',
                linewidth       = 4,
                color           = (0.0,0.0,0.7),
                zorder          = -3,            
                ),
        'tertiary_link': dict(
                linestyle       = '-',
                linewidth       = 4,
                color           = (0.0,0.0,0.7),
                zorder          = -3,            
                ),
        'residential': dict(
                linestyle       = '-',
                linewidth       = 1,
                color           = (0.1,0.1,0.1),
                zorder          = -99,            
                ),            
        'unclassified': dict(
                linestyle       = ':',
                linewidth       = 1,
                color           = (0.5,0.5,0.5),
                zorder          = -1,            
                ),
        'default': dict(
                linestyle       = '-',
                linewidth       = 3,
                color           = 'b',
                zorder          = -1,            
                ),
        }
                        

    # get bounds from OSM data            
    minX = float(myMap['bounds']['minlon'])
    maxX = float(myMap['bounds']['maxlon'])
    minY = float(myMap['bounds']['minlat'])
    maxY = float(myMap['bounds']['maxlat'])



    fig = p.figure()
    
    # by setting limits before hand, plotting is about 3 times faster
    ax = fig.add_subplot(111,autoscale_on=False,xlim=(minX,maxX),ylim=(minY,maxY))
    
    for idx,nodeID in enumerate(ways.keys()):
        wayTags         = ways[nodeID]['tag']
        if not wayTags==None:
            hwyTypeList  = [d['v'] for d in wayTags if d['k']=='highway']
            if len(hwyTypeList)>0:
                    wayType = hwyTypeList[0]  
            else:
                    wayType = None
        else:
            wayType = None
        try:
            if wayType in ['primary','primary_link',
                            'unclassified',
                            'secondary','secondary_link',
                            'tertiary','tertiary_link',
                            'residential',
                            'trunk','trunk_link',
                            'motorway','motorway_link',
                            ]:
                oldX = None
                oldY = None
                
                if wayType in renderingRules.keys():
                    thisRendering = renderingRules[wayType]
                else:
                    thisRendering = renderingRules['default']
                    
                for nCnt,nID in enumerate(ways[nodeID]['nd']):
                    y = float(nodes[nID['ref']]['lat'])
                    x = float(nodes[nID['ref']]['lon'])
                    if oldX == None:
                        pass
                    else:
                        p.plot([oldX,x],[oldY,y],
                            marker          = '',
                            linestyle       = thisRendering['linestyle'],
                            linewidth       = thisRendering['linewidth'],
                            color           = thisRendering['color'],
                            solid_capstyle  = 'round',
                            solid_joinstyle = 'round',
                            zorder          = thisRendering['zorder'],
                            )
                    oldX = x
                    oldY = y
                       
        except KeyError:
            pass
          
    print'Done Plotting'
    p.show()




########################################

src = file('map.osm')
myMap = xml2obj(src)
render(myMap)

Testing Configuration:

• Windows 7
•  PythonXY 2.5.6.1
• OpenStreetMap example data , mirrored example data,  Map data © OpenStreetMap contributors, CC-BY-SA

Other Useful Links and references:

• Open Street Maps
• ActiveState XML code example
• The Open Street Maps python based render

---

Questions Answered:

• How to visualize an Open Street Map from data
• How to plot a map
• How to draw an Open Street Map from the OSM file

How to get Panda3D to work with PythonXY

This note applies to Panda3d ver 1.7.0 and PythonXY ver 2.6.2 on a windows system.

1. Install PythonXY.
2. Install Panda3D, however do NOT set it as the default python interpreter.
3. Add a ‘pth’ file to the python site-packages directory. I created a file named panda.pth and saved it at “C:\Python26\Lib\site-packages”. The file had the following contents:

C:/Panda3D-1.7.0
C:/Panda3D-1.7.0/bin

To check that the installation works, use an example from the Panda3D manual.

from direct.showbase.ShowBase import ShowBase
class MyApp(ShowBase):
	def __init__(self):
		ShowBase.__init__(self)
app = MyApp()
app.run()