advertisement
from the Automation List department...
Graphics Question
Human-Machine Interface and SCADA. topic
Posted by curt wuollet on 28 May, 2010 - 4:20 am
Hi All

Not that I need another project, but...
A long, long, time ago I wrote a program in Turbo Pascal on DOS that read four thermistors attached to a game card and plotted their temperature in strip chart recorder fashion on screen with the chart grads, etc. added as the chart scrolled. It also output the chart to a dot matrix printer attached for a permanent record.

I recently had to resurrect that program, blow the dust out of an old Zenith XT compatible computer and an epson FX286 printer and set the thing up.

I now have someone that wants that capability on a modern laptop. I explained that I don't do Windows and that was fine, so I set out to write the chart recorder part in C. This program was fairly easy to write in ancient TP with a Graphic library they had as an add on. The screen was basically just a big array of dots and you just plotted away. Amazingly enough, I haven't been able to find a simple way to do it under Linux.

There are console libs used for games, but to be really modern, it should be in a window. Since this is very much like a trending graph and there are some HMI folks here, I thought I could perhaps save some grief and solicit suggestions on what to use to do this in a reasonable amount of time. I would prefer C but I'm open to Python or other Linux solutions. I just need to define a canvas and be able to write one bit column at a time and scroll.

Regards
cww


Posted by Paul Breneman on 28 May, 2010 - 9:18 am
I suggest you check out FreePascal which is very compatible with TurboPascal and also works great on Linux. FreePascal is mature and being used for a lot of *real* projects. Let me know if I can be of help.

See http://www.TurboControl.com for more information about FreePascal and for some example projects.


Posted by William Sturm on 28 May, 2010 - 11:29 am
Have you looked at SVGALib? It creates non X-Windows graphics in Linux "IF" you have a compatible video card. It is similar in capabilities to the old Borland BGI graphics. DirectFB may also be an option.

Bill Sturm


Posted by pvbrowser on 29 May, 2010 - 2:43 am
In order to output graphics on modern computers you must use a library you can't write to the graphics card directly.

On Windows the low level interface is GDI (graphics device interface).
On Linux/Unix it is the XLIB.

On top of these libs you have higher level libraries. I would suggest to use Qt as the higher level library because it works on practically all modern OS.
http://qt.nokia.com/
(Qt is the basis for KDE for example but also for Google Earth ...)

Since you do not want to start completely from scratch you should use some sort of framework.

We provide a framework for HMI/SCADA that is based on Qt and also works on all modern OS.
See:
http://pvbrowser.org

This framework is easy to use and does not necessitate too much programming skills.
Graphics like you use can be done very easy.
Once you have it on screen you can print everything or insert it into office programs for example.

Please also have a look on:
http://pvbrowser.org/pvbrowser/doc/pvb.en.pdf


Posted by Steinhoff on 30 May, 2010 - 4:10 am
> In order to output graphics on modern computers you must use a library you can't write to the graphics card directly.

On Windows the low level interface is GDI (graphics device interface).
> On Linux/Unix it is the XLIB. <

The XLIB is really the lowest level of X but it is not easy to use and not very fast. The best choice for Linux is the framebuffer:
http://www.tldp.org/HOWTO/Framebuffer-HOWTO.html

The libsdl is able to use the framebuffer (http://www.libsdl.org ... for 2D apps use
http://www.ferzkopp.net/Software/SDL_gfx-2.0

For 3D apps is available a subset of OpenGL at:
http://bellard.org/TinyGL and SDL based:
http://www.kyb.tuebingen.mpg.de/bu/people/gf/software

> On top of these libs you have higher level libraries. I would suggest to use Qt as the higher level library because it works on practically all modern OS.
http://qt.nokia.com/
(Qt is the basis for KDE for example but also for Google Earth ...)

Since you do not want to start completely from scratch you should use some sort of framework. <

If you want a complete solution ... go with the ROOT system:
http://root.cern.ch
Some screenshots:
http://root.cern.ch/drupal/category/image-galleries/data-analysis-v isualization

Regards,
Armin Steinhoff
http://www.steinhoff-automation.com


Posted by curt wuollet on 29 May, 2010 - 2:54 am
Yeah, the only thing is that SVGALib is deprecated to be replaced by? But it is much like the old direct screen memory stuff. I've been pondering picking a graphics method for a while. It's just trying to pick the right one so as not to add to my immense store of non-reusable knowledge:^).

That's an occupational hazard for old programmers. Right now, nobody around here cares how many languages you can program in unless they belong to the evil empire. That's why I do maintenance work.

Regards
cww


Posted by Ken E on 1 June, 2010 - 8:28 am
I find GTK with Mono to be a very good tool for graphics and you can get going fast. I wouldn't mess with direct framebuffer stuff since X windows is well supported. C# is a good transition from C language as they have maintained a lot of the same syntax. A lot of the GTK libraries make more sense when the language supports OOP IMHO, the only trouble is that a lot of the documentations is thin for the more obscure GTK calls so you end up looking at the C API documentation and trying to figure out what the method name is that corresponds to the C function, etc.

KEJR


Posted by curt wuollet on 1 June, 2010 - 11:27 pm
Yes, there are many paths to do GUIs, these days. and I was looking at Glade and the Gnome side of things and the Qt stuff as well, as I kinda like KDE. I don't need Mono as running Windows stuff is pretty close to zero priority and I have no exposure to .NETness. Nor, do I want any. As I look at it, Python transcends the "whose GUI" and aims at any. I can write for any Linux desktop. I'm sure there are some things I need to do that won't be served by an interpreted scripting language, but I've been fairly impressed by way this high level language is put together as far as code reuse, as the critical bits of the libraries seem to be well crafted and I'm not sure I could do better working at a lower level. And it seems well suited for small one off projects. A a great way for a non-gui programmer to add the almost requisite GUI these days without burning a lot of hours. And the whole OO thing makes a lot more sense in this context.

Regards,
cww


Posted by Ken E on 2 June, 2010 - 11:20 am
Curt,

Mono is just like Java or python in a lot of ways.. It's a language that has a runtime component that happens to run on multiple OS's. It was crafted by some really smart people at Microsoft, and is embraced by a lot of folks in the open source community because of the mono effort.

Like I said before, I happen to like it because I know C well and the syntax is nearly identical. The aspects of it that are Object oriented are very easy to understand (much more than C++ in my opinion). So whether you like Microsoft or .NET isn't really a big deal because Mono is an open source project. I guess what I'm saying is that I would not dismiss the power and simplicity of the C# language because of the Microsoft component because you can take it or leave it.

KEJR


Posted by M Griffin on 3 June, 2010 - 1:00 am
In reply to Ken E:

Mono is a clone of MS Dotnet. It's not in itself a language however. It's a VM (a cross between an interpreter and a compiler), runtime, and set of libraries. The main language used with both Mono and Dotnet is C#. DotNet belongs to Microsoft, while Mono belongs to Novell. I'll try to keep the rest of this brief, but here's the history of it in a nutshell.

In the 1990s Microsoft licensed Java from Sun because they liked what they saw in it. They then added features to integrate their version more tightly into their own OS instead of using the portable libraries (they liked Java, but they didn't like the idea of portable applications). Sun eventually sued them over license violations and that business relationship came to an end.

Microsoft then hired a designer from Borland to design a language and system for them that was similar to Java, but with enough differences to avoid ending up in court with Sun again. The result was C# and Dotnet.

Someone from a small company called Ximian then in turn cloned C# and Dotnet. Ximian were bought by Novell (who wanted their server management software). Some of the Ximian management stayed with Novell and they managed to persuade them to continue development of Mono.

Mono hasn't been a big success. They've had some interest from people doing games development for games consoles (their licensing terms are much more reasonable than Microsoft's). The anticipated market for the use of C# on Unix and Linux servers never materialized however, because because C# and Mono didn't offer anything to those developers that they didn't already have from better established and more widely accepted development tools. Sun became more reasonable about outside participation in Java development and it came to look like Ximian (and Novell) had backed the wrong horse.

Dotnet was never designed to be cross platform. Mono on the other hand is portable, but that is only because it has additional platform independent libraries that Dotnet doesn't. A non-trivial C# (or VB) program written for Dotnet usually requires at least some re-writing to port it to Mono (this is according to Novell's customer statistics).

If you wrote a program specifically for Mono with their portable libraries and then used Mono on all your target platforms (Linux, MS Windows, Mac OS/X), you might have a portable program. You could get the same result with C++ and Qt however,

If you want to use C# and Mono because you already know the language, that's fine, go ahead. If you happen to like it, that's fine too. However, you should understand that outside of the MS Windows development world there just isn't much interest in it. Existing applications aren't going to get rewritten, while new ones tend to use more recently developed languages.


Posted by Ken E on 3 June, 2010 - 9:13 am
M. Griffin, Curt,

I understand your apprehension about C# from a standpoint of the Free Software. I do appreciate the comment as I did some web research and I was unaware of all of the "potential" threats from Microsoft. It seems the FSF has gone back and forth on Mono regarding its "Free" status, but lately it seems that as long as you are using the core language constructs as defined in the language standard you are OK. One of the gray areas is with ASP.net, ADO.net, and windows forms, which are Microsoft provided extensions to the language spec. I personally don't use any of those (I use GTK because I like it and it is well supported on multiple platforms). Seriously now, who would use ASP.net on linux? Windows Forms is probably a lot more of a gray area, but I would hope developers of open source software would use GTK and not windows forms.

For my machine projects I can compile a simple HMI, test it on windows, FTP the same binary to the target Linux box and it just runs (Provided Mono is installed on the linux box and either .NET or mono on the windows box). Granted, the same can most likely be done with Python and/or Java. One of the reasons I initially looked into C# is that Delta Tau is recommending to use Mono for the HMI development on their new Power PMAC motion controller(it is not limited to this, of course, as it is debian linux with Xenomai). I think they figure most people will be comfortable with using Visual Studio and can easily build an HMI, but you can use Mono and a Makefile just as well (we may go this way so as not to chase releases of Visual Studio...). I had looked into Python, but to learn a new language/syntax and switch from C programming to Python programming for the HMI inside one project didn't make for a seamless environment (We need to program in C for non HMI parts of the controller due to the RTOS). C# is a lot closer to C if you don't heavily get into OOP aspects of C#, which for HMI scripting and placing buttons it isn't that necessary. It makes a big difference going back and forth from one file to another and not having to worry if you need a semicolon or not.

One of the things I like about C# is that the syntax doesn't try to be "simpler" than traditional C language syntax by adding vague keywords. I am forever annoyed with the BASIC and Pascal type syntax that tries to make things easier for people who can't program in C. I program in a lot of "automation" languages related to proprietary controllers and hardware (Motion controllers, robot controllers, etc). It seems each one has its own syntax and I am constantly cursing them for not using C syntax since they all seem to use one of five different types of BASIC syntax. One such example is the if statement. I constantly need to check the code to see if I'm supposed to use endif, end_if, and in one case "fi" (old Compumotor project I had to deal with). The same can be said for Select-case, while, etc. Some languages don't support "elseif" at all, which is annoying. I also curse IEC structured text because its syntax is ugly and arbitrarily different. So to see that C# is very similar to C and cut out the unnecessary crap of C++ (but left most of the power) was enlightening. Admittedly Python and Java might be similar in this regard, except they chose radically different syntax (Python more so than Java). I would never use C# for anything needing to reside close to hardware, or for RTOS, obviously, but for GUI style programs it is appropriate.

Bottom line, use C++, C#, Java, Python, or C, but for the love of all things good use a modern graphical abstraction layer such as GTK, or KDE.

~KEJR

In that sense it makes sense to me, as a language.


Posted by curt wuollet on 3 June, 2010 - 6:25 pm
I think C and Python will cover my needs unless I end up back in IT. Python for adding graphics to say, machine vision apps and C for controls and working with hardware. Up until this point I have done OK using the bounty of Linux apps to do my display work using pipes, etc. I don't really want to be a heads down application programmer. And the whole web space thing just doesn't appeal to me. Since most of the jobs posted want Windows or web skills, I'm probably not going back into IT. But, I do need a paying job soon. I'm using the time to do things that would be hard if I was working. Like learning Python.

Regards,
cww


Posted by Steinhoff on 4 June, 2010 - 3:28 am
> I think C and Python will cover my needs unless I end up back in IT. Python for adding graphics to say, machine vision apps and C for controls and working with hardware. <

To get access to hardware with Python:
http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/

Armin Steinhoff


Posted by M Griffin on 4 June, 2010 - 3:31 pm
> I think C and Python will cover my needs unless I end up back in IT. Python for adding graphics to say, machine vision apps and C for controls and working with hardware. <

As well as Pyrex, there's also SWIG and ctypes. Ctypes seems to be the popular way of doing things these days and it's a standard library module. However, the different methods have advantages in different applications, so there's no one best way of doing it. Ctypes is usually easier, but a proper wrapper has less overhead which may matter in some applications.


Posted by M Griffin on 3 June, 2010 - 10:04 pm
In reply to Ken E: Just to be clear on this, I said if you want to use Mono, go right ahead, I don't have a problem with that. I was just correcting the bit where you said Mono came from Microsoft. Mono and Dotnet are two different things from two different companies. We're just debating minor points here.

As for what the FSF has said about Mono, they've been very clear and consistent on it. They've said that it's Free Software and they think it's a good idea that it exists so that people like you can use it for your applications.

They have also said however that they don't think it should be used for key infrastructure systems in Linux because of the uncertain legality of some of the components. The ECMA spec covered only the core language and libraries whereas much of the useful functionality is in libraries and features which were not covered. Furthermore, the patent license from Microsoft was written in a way which allows it to be withdrawn at a future date. Getting the language spec rubber stamped by ECMA was intended as a PR gesture by Microsoft. I don't think they expected anyone to actually make use of it.

As for your question "seriously now, who would use ASP.net on linux?" I think you've unwittingly hit the nail right on the head with that statement. Novell's (Ximian's) business plan for Mono was based on persuading Microsoft's customers to switch from MS Windows Server with IIS, ASP Dotnet, and MS SQL Server to Novell ELS, Apache, Mono, and Postgres or MySQL. The same goes for Winforms in conventional clients.

The target market was vertical applications (hotel, restaurant management, retail POS, etc.). Novell would offer software vendors lower costs on support contracts and greater security and reliability, better remote management, etc., and the software vendors would get the products into customer sites. Is anyone doing this? Well, Novell has a list of customers (software vendors) who ported their products over and are selling them to their customers. Do they have enough to make a financial success of it? That's a good question. What is clear is that if it really does become successful Microsoft will be very unhappy about the effect it has on their bottom line and will likely be looking for ways to do something about it.

I looked into Mono in detail a couple of years ago when I was deciding what to use in a software project. I looked at library support, documentation, packaging and distribution, cross platform capabilities, "community" support (how easily can I find an answer in Google), and various other factors. I compared a lot of different languages, and what I ended up with was Python.

A few months ago you were discussing with us what you should use in your application. At the time I believe I said that your vendor's (Delta Tau) support for Mono in their product was an argument in favour of using it there. I would still make that recommendation today.

However, for what Curt Wuollet is trying to do, I think that Python is a better choice. You can both be right and make different decisions.

As for language syntax, I really don't get too excited about it. I can use C, Python, and Javascript all together on the same day and they're all fine when used for the right things. I'm more concerned about getting practical results.


Posted by Steinhoff on 29 May, 2010 - 3:26 am
Hi all,

AFAIK ... DirectFB is a more or less dead project.

An active and good supported graphics project is SDL (Simple Direct
Layer) or libsdl (http://www.libsdl.org)

--Armin



Posted by curt wuollet on 29 May, 2010 - 11:10 pm
Yes, that is supposed to be the replacement for SVGALib. When I last looked at it, it was in a dormant state and there was really nothing I could see that actually suggested how to use it. That was quite a while ago. That was quite a while ago, I'll have to take another look. I've been away from C programming for a while, messing with some goofy relay based stuff:^) I'm not sure which path my future is in. Probably the one that offers to pay me first.

Regards,
cww


Posted by curt wuollet on 28 May, 2010 - 4:52 pm
I did download FreePascal and have pored through the docs a bit. I haven't yet rigged a computer with ethernet and a 5.25" floppy drive or USB or some other way to get the program to a modern box.

I should, I suppose, try to just compile it and see what happens. I would like to do a rewrite as Pascal isn't as popular as it one was and I could do more with a C version, but the F route _might_ be faster. Some of the blessed simplicity of TP seems to have been lost in the move to portability. I do vaguely recall doing a rewrite to MIX C on DOS at one time but it would crash and I never did find out why even with their nifty visual debugger. I suspect that would be less portable than the TP version.

Regards
cww


Posted by M Griffin on 28 May, 2010 - 6:47 pm
Here's a simple example in Python that just draws a line graph using random numbers for "y". This uses the Tkinter graphics toolkit, although Gtk, wXwidgets or Qt would be fairly similar.


#!/usr/bin/python
# A simple example that draws a line graph.
import Tkinter
import random

# This creates the window.
root = Tkinter.Tk()
root.title('A simple graph.')

# This creates the "canvas" object to draw on.
canvas = Tkinter.Canvas(root, width=500, height=250, bg = 'white')

canvas.pack()

# This makes a "quit" button.
Tkinter.Button(root, text='Quit', command=root.quit).pack()


# This loop draws a line using sequential values for "x"
# and random values for "y".
# Note: In case the web page eats the formatting, the next 3 lines after the
# "for" are supposed to be indented.
lasty = random.uniform(50, 200)
for x in xrange(0, 500, 50):
nexty = random.uniform(50, 200)
canvas.create_line(x, lasty, x + 50, nexty)
lasty = nexty

# This is the main event loop.
root.mainloop()


This is cross platform and will work just about anywhere that Python will run. Tkinter comes in a standard Python install, so that is the easiest to install. If you want to use Gtk, wXwidgets, or Qt on MS Windows, you will have to install those packages separately, along with the corresponding Python bindings.

You can do this in 'C' but it takes a lot more lines of code to define things and set up call backs.

What I haven't shown is continuous updating. That would be done by using a timer to do a call back on a regular basis. There are timers built into the GUI event handler that are designed for this, so it's actually pretty easy.

A typical program of the type you are talking about involves having the GUI event loop (which happens inside "mainloop()" in this example) handling a timer and calling a designated function. That function does all the updating (reading the I/O, redrawing the screen, etc.) and then exiting, which hands things back to the event loop. Mouse clicks and keyboard presses all have their own handlers, but they are all dispatched by the same event handler.

If you want a more detailed example, let me know. The sort of thing you are talking about is fairly straightforward. I haven't got an example for printing, but that is similar to drawing on the screen except it's handled by a different API.


Posted by curt wuollet on 29 May, 2010 - 2:34 am
Thanks Michael

Just the sort of example I was looking for. I'd messed with Tcl before but not low level graphics. That gives me something to check out. Plotting seems straightforward. I'll check out the Tkinter stuff. If they don't have a dot plot I can use a line length of 1. Next trick would be to scroll the canvas left when you get to the end to keep going. But this looks doable. I've wanted a vehicle to learn Python anyway. It is "new and hot" enough to justify the effort.

Regards
cww


Posted by M Griffin on 29 May, 2010 - 6:21 pm
Here's a scrolling graph. There's not much more to it than the original


#!/usr/bin/python
# A simple example that draws a scrolling line graph.
import Tkinter
import random


# We create a class to make it easier to keep static data between calls.
class Graph:

def __init__(self):
# Initialise the first Y point.
self._YPoints = [125]
# The X points are all pre-defined.
self._XPoints = range(0, 500, 50)


def UpdateGraph(self):
# Create a new Y point.
self._YPoints.append(random.uniform(50, 200))
# This bit of magic combines the X and Y points in one list.
GraphPoints = list(sum(zip(self._XPoints, self._YPoints), ()))
# Delete everything which is already in the canvas.
canvas.delete(Tkinter.ALL)
# Draw a new line.
canvas.create_line(GraphPoints, fill='red', width=5)
# Shift the Y buffer left one if it is full.
if len(self._YPoints) > 10:
self._YPoints.pop(0)
# Call the update function again later.
root.after(250, self.UpdateGraph)

# This creates the graph object.
SimpleGraph = Graph()

# This creates the window.
root = Tkinter.Tk()
root.title('A simple graph.')

# This creates the "canvas" object to draw on.
canvas = Tkinter.Canvas(root, width=500, height=250, bg = 'white')

canvas.pack()

# This makes a "quit" button.
Tkinter.Button(root, text='Quit', command=root.quit).pack()

# Start the graph running.
SimpleGraph.UpdateGraph()


# This is the main event loop.
root.mainloop()


The main differences to note here are that to scroll it we delete the previous line and then draw a new one. The other thing to note is the call using "after" at the end of "UpdateGraph" This sets up a call back to "UpdateGraph" after the specified interval (250 milliseconds in this example). For your application you would probably use one call back which would drive everything, including the reading the I/O and updating the graph with the data. In that case, you would put the call to "after" somewhere else more convenient.

If you don't understand the GraphPoints = list(sum(zip(self._XPoints, self._YPoints), ())) bit, don't worry about it. It's just a short cut that I took to keep the example program short. It just combines the X and Y lists into a single list with the X and Y interleaved with each other. You can do the exact same thing with a for loop if you are more comfortable with that. Python has a lot of features which allow you to work on an entire list (array) at once using a single instruction instead of using an explicit for loop. I could have used a deque (FIFO buffer) to automatically manage the length of YPoints (instead of testing it with an if statement), but I didn't want to have to explain that bit.

I also just took a short cut to delete just the whole graph every time. If I had axes and grids drawn on there, I might want to be a bit more selective about what I delete.

As for a dot plot, there is a parameter called "stipple" which uses a bit map graphic as a fill instead of using a colour. 'gray25' seems to be a example that gets shipped with it, but you are supposed to create your own. I've never used it however, so I don't know much about it. Using different colours for the lines has always been satisfactory for me.

For this example, CPU usage on a fairly slow computer is less than 1%. You don't have to worry to much about efficiency just to do something like that.

If you can, try to get your hands on a copy of "Python and Tkinter Programming" by John E. Grayson. It is the best GUI programming book that I have seen for any tool kit or language. Even if you decide to use a different tool kit later (e.g. wXWidgets), a lot of the principles are the same, so learning Tkinter would not be a waste of effort.


Posted by curt wuollet on 30 May, 2010 - 12:00 pm
Thanks Michael

I'll pore through it, last night I was looking at pygames which offers scrolling and blitting. Hardly needed since this will be seconds per frame rather than fps, but it seems there are many ways to accomplish this. I could have a graph paper image, copy it to two surfaces and once the trace reaches the end of the first one, start scrolling it off and the other one on, plotting the traces as I go then flipping to the screen. I just need to pay my dues. I'm not very OO so, it'll take some getting used to. From what I've seen, Python is real and around to stay and worth learning. I'm afraid at the moment, I have to rely on the community for info and tutorials, adding to my library of aging programming books is for better times. The really great thing about OSS is that I can keep on learning. I'd like to be studying Contrologix and the new gen PACs but there's no way to do that on pocket change. I doubt the automation world will mind if I wander back into general computing.

Regards,
cww


Posted by Steinhoff on 30 May, 2010 - 3:22 pm
> I'll pore through it, last night I was looking at pygames which offers scrolling and blitting. <

BTW ... pygames is SDL powered.

> Hardly needed since this will be seconds per frame rather than fps <

Yes, that's necessary for animated graphics.

Regards
--Armin




Posted by M Griffin on 30 May, 2010 - 5:20 pm
Here it is again with 50 points, and with grid and axis labels. This could obviously be made into a library which automatically calculates the axes, grid positions, and labels, but I have hard coded all the parameters here to keep the example short and simple.

You will notice that I keep track of the line using the self._PrevLine variable so I can delete just the line and leave the other graphics alone. I also made a few other minor changes (e.g. used a deque) to keep the example short.

CPU usage for this on a slow computer is about 1% for the program, and overall the CPU is 97% idle. Forget about framebuffer graphics or anything like that for this sort of application. You don't need it. This is not high performance graphics. For this sort of project you just want to get the program finished as quickly as possible with the minimum effort and number of lines of code. I'm not aware of anything else that even comes close to Python for those types of projects.

This will also be cross-platform. I have written Python programs on Linux that run as is on MS Windows without any making any special effort other than to use the normal Python libraries (e.g. let the libraries handle path separators and line endings). That includes communicating with IO through Ethernet and serial connections.

The only differences that I've found are platform limitations (things that MS Windows simply doesn't have, or poorer performance due to OS design limitations). You need to test for those, but generally it is a lot easier to develop Python programs on Linux and then copy them to MS Windows if that is what the user wants to use.

How are you going to interface with the hardware? If that involves a C library, Python has lots of options there as well.

If you have any more questions, let me know and I will be happy to answer them.


#!/usr/bin/python
# A simple example that draws a scrolling line graph.
import Tkinter
import random
import collections


# We create a class to make it easier to keep static data between calls.
class Graph:

def __init__(self):
# Initialise the first Y point. This is a FIFO buffer that
# automatically manages the buffer length for us.
self._YPoints = collections.deque([125], maxlen=45)
# The X points are all pre-defined.
self._XPoints = range(40, 490, 10)
# Draw the axes.
canvas.create_line(40, 225, 490, 225, fill='blue', width=2)
canvas.create_line(40, 10, 40, 225, fill='blue', width=2)
# Label the axes.
for x in range(40, 490, 50):
canvas.create_text(x, 235, text=(x - 40))
for y in range(10, 225, 25):
canvas.create_text(15, y, text=(225 - y))

# Draw the grid.
for x in range(90, 490, 50):
canvas.create_line(x, 10, x, 225, fill='grey', width=1)
for y in range(10, 225, 25):
canvas.create_line(40, y, 490, y, fill='grey', width=1)

# We draw a small line to initialise the line delete operation
# so we don't have to check each time if a line is already there.
self._PrevLine = canvas.create_line(0,0,0,0)

def UpdateGraph(self):
# Create a new Y point.
self._YPoints.append(random.uniform(50, 200))
# This bit of magic combines the X and Y points in one list.
GraphPoints = list(sum(zip(self._XPoints, self._YPoints), ()))
# Delete the previous line.
canvas.delete(self._PrevLine)
# Draw a new line.
self._PrevLine = canvas.create_line(GraphPoints, fill='red', width=5)
# Shift the Y buffer left one it is full.
# Call the update function again later.
root.after(250, self.UpdateGraph)


# This creates the window.
root = Tkinter.Tk()
root.title('A simple graph.')

# This creates the "canvas" object to draw on.
canvas = Tkinter.Canvas(root, width=550, height=250, bg = 'white')
canvas.pack()

# This makes a "quit" button.
Tkinter.Button(root, text='Quit', command=root.quit).pack()

# This creates the graph object.
SimpleGraph = Graph()

# Start the graph running.
SimpleGraph.UpdateGraph()


# This is the main event loop.
root.mainloop()


Posted by curt wuollet on 31 May, 2010 - 1:10 am
I am embarrassed by your kindness and expenditure of time just to help out and most impressed with your command of the language, which to me at the moment looks rather overwhelming and large.

We must have a version difference, the first ran but the second bailed with:

File "grapher.py", line 13, in __init__
self._YPoints = collections.deque([125],maxlen=45)
TypeError: deque() does not take keyword arguments

My debugging ability is not good at the moment, but perhaps the deque has been extended recently?

You're right though, it may take a book to get the big picture and background.

Regards,
cww

Python -V gives:
Python 2.5.2


Posted by Steinhoff on 31 May, 2010 - 4:24 pm
> My debugging ability is not good at the moment, <

Eric could help you: http://eric-ide.python-projects.org/index.html

Regards,
Armin Steinhoff


Posted by M Griffin on 31 May, 2010 - 5:10 pm
I don't mind making the explanation and I imagine that other people are interested as well. Go ahead and ask questions, this is a subject that I like to talk about.

The maxlen parameter in deque is a 2.6 feature. If you're using 2.5 you will get an error. Version 2.6 has been out for a while (2.7 is about to come out and the 3.x series is actually the newest), but I'm guessing that you are using something like Debian Stable or Centos (enterprise server distros tend to be a bit behind the times). This however is an issue that I have to deal with regularly with my MBLogic project. New features like this make things more convenient for me, but I get complaints from users if the software won't run on Debian Stable (which is using 2.5 until the new Debian release comes out). I get more downloads for the MS Windows version, but a lot of the people who contact me directly about actual applications they are working on seem to deploy on Debian.

The deque was something that I added in the last version to save a couple of lines. You just have to go back to the method shown in the previous ones to fix the problem:

1) Remove the line with "import collections"

2) Replace the following:
# Initialise the first Y point. This is a FIFO buffer that
# automatically manages the buffer length for us.
self._YPoints = collections.deque([125], maxlen=45)

with:

# Initialise the first Y point.
self._YPoints = [125]

3) Restore the following back into "UpdateGraph":


# Shift the Y buffer left one it is full.
if len(self._YPoints) > 45:
self._YPoints.pop(0)


This check should probably go right after the "append" operation, rather than at the end of the function like I originally had it.

Make sure you use proper indentation. In Python, indentation is syntactically significant, as in a messy program is also an incorrect program. Use consistent indentation (either spaces or tabs) all they way through. Don't mix indents with tabs or you'll have headaches. On the other hand, you'll never have to worry about forgetting a curly brace (which is the number one error I have in Javascript).

Most editors understand Python syntax and will highlight and auto-indent (or whatever they do) for it, but you may have to enable that feature if it's turned off.

Here it is with the changes. I will include a line by line explanation just after the listing.


#!/usr/bin/python
# A simple example that draws a scrolling line graph.
import Tkinter
import random


# We create a class to make it easier to keep static data between calls.
class Graph:

def __init__(self):
# Initialise the first Y point.
self._YPoints = [125]
# The X points are all pre-defined.
self._XPoints = range(40, 490, 10)

# Draw the axes.
canvas.create_line(40, 225, 490, 225, fill='blue', width=2)
canvas.create_line(40, 10, 40, 225, fill='blue', width=2)
# Label the axes.
for x in range(40, 490, 50):
canvas.create_text(x, 235, text=(x - 40))
for y in range(10, 225, 25):
canvas.create_text(15, y, text=(225 - y))

# Draw the grid.
for x in range(90, 490, 50):
canvas.create_line(x, 10, x, 225, fill='grey', width=1)
for y in range(10, 225, 25):
canvas.create_line(40, y, 490, y, fill='grey', width=1)

# We draw a small line to initialise the line delete operation
# so we don't have to check each time if a line is already there.
self._PrevLine = canvas.create_line(0,0,0,0)

def UpdateGraph(self):
# Create a new Y point.
self._YPoints.append(random.uniform(50, 200))
# Shift the Y buffer left one it is full.
if len(self._YPoints) > 45:
self._YPoints.pop(0)

# This bit of magic combines the X and Y points in one list.
GraphPoints = list(sum(zip(self._XPoints, self._YPoints), ()))

# Delete the previous line.
canvas.delete(self._PrevLine)
# Draw a new line.
self._PrevLine = canvas.create_line(GraphPoints, fill='red', width=5)

# Call the update function again later.
root.after(250, self.UpdateGraph)


# This creates the window.
root = Tkinter.Tk()
root.title('A simple graph.')

# This creates the "canvas" object to draw on.
canvas = Tkinter.Canvas(root, width=550, height=250, bg = 'white')
canvas.pack()

# This makes a "quit" button.
Tkinter.Button(root, text='Quit', command=root.quit).pack()

# This creates the graph object.
SimpleGraph = Graph()

# Start the graph running.
SimpleGraph.UpdateGraph()


# This is the main event loop.
root.mainloop()


Here's a line by line explanation of what I did and why:

import Tkinter
import random

This imports the libraries we need. Tkinter is the graphics library, and random is just to generate the random numbers I used to fill the graph.

class Graph:

This creates a class (the prototype for an object). You don't have to use object oriented programming with Python, but it makes accessing the associated data a lot easier than trying to use global variables.

def __init__(self):

This is the declaration for the initialisation function (constructor). That exact name __init__ belongs to the language and is automatically called when the object is created. The "self" is a reserved name which allows the object to reference itself. You have to include it as the first parameter in declarations, but you don't have to include it explicitly in the actual function calls. "Self" exists because there are ways of doing special things without it, but I've never tried those.

self._YPoints = [125]
self._XPoints = range(40, 490, 10)

Notice the use of "self" again when referencing things inside the class. This is how Python knows you are using the class variables, rather than locals (local to the function). The underscore in these names is not significant. It is just a convention to show these are intended to be private variables.

The "self._YPoints = [125]" creates a list one element long with the first element being the number 125. Just [] would have created an empty list. A Python list is sort of a cross between an array and a linked list. It's one of the basic types, and it gets used a lot.

self._XPoints will form the X coordinates for out graph. We only need to create this once (since they never change), and just use it over and over again.

"range(40, 490, 10)" outputs a list starting at 40, ending at 490, and incrementing in steps of 10. The first and last parameters are option, so for example "range(10)" would just output the list of numbers from 0 to 9.

Range is used a lot in for loops where instead of saying "for (in i=0; i < 10; i++)" you would just say "for i in range(10):"

canvas.create_line(40, 225, 490, 225, fill='blue', width=2)

"canvas" was created in the "main" level using the statement "canvas = Tkinter.Canvas(root, width=550, height=250, bg = 'white')". I really should have explicitly imported it into the example as a parameter in the __init__ (or better still created it inside the Graph class), but I was trying to keep the example simple. create_line accepts a series of points as the X and Y coordinates for lines. The more pairs of points you provide, the more line segments you get. fill='blue' illustrates calling optional parameters by name. There are a lot more optional parameters which can be used to control the characteristics of the line.


for x in range(40, 490, 50):
canvas.create_text(x, 235, text=(x - 40))

This is just labelling one of the graph axes. create_text obviously creates text. The first two parameters give the coordinates (0,0 is the upper left of the canvas). text=(x - 40) is the text we want to print (text is an optional parameter). In this case we are printing the x coordinate (minus 40).


for x in range(90, 490, 50):
canvas.create_line(x, 10, x, 225, fill='grey', width=1)

This is drawing grid lines. I won't explain it as it should be obvious at this point.

self._PrevLine = canvas.create_line(0,0,0,0)

I drew a line which gets thrown away as soon as we start drawing. This way I don't have to check if a line exists before I delete it.


def UpdateGraph(self):

This is the function declaration. A function declaration can include parameters, but we don't have any in this case (except for "self").


self._YPoints.append(random.uniform(50, 200))

self._YPoints is a list, a list is an object, and among the properties it has is "append". Here we are appending a random number (between 50 and 200) to the end of the list. Normally you would be appending your readings.


# Shift the Y buffer left one it is full.
if len(self._YPoints) > 45:
self._YPoints.pop(0)


Here we're checking the length of self._YPoints to see if we need to start trimming off old points. If so, we "pop" a point from position 0 in the list.


GraphPoints = list(sum(zip(self._XPoints, self._YPoints), ()))

I won't explain the details of this, other than to say that "zip" combines two lists (our X and Y points) into a list of pairs of numbers, sum "flattens" the pairs, and "list" turns the result back into a list. The use of "sum" this way is a bit of a strange trick, since technically it is supposed to be used to add numbers together (sum a list). However, it also seems to be able to flatten lists as well. I could have done all this with a "for" loop, but this wasn't the point of the example, and I wanted to keep things short.


canvas.delete(self._PrevLine)
self._PrevLine = canvas.create_line(GraphPoints, fill='red', width=5)

Delete the old line, and draw a new one. We keep a reference to the new line so we can delete it later. This is why we created the "dummy" line in the initialisation - we could delete the line without having to check if we have created one yet.

root.after(250, self.UpdateGraph)

We have a window named "root" (it could be named anything). Windows have a method called "after" which schedules a call to a function after a set period of time. Pretty much every GUI toolkit that I am familiar with has something similar. In this case we are calling UpdateGraph again (note the use of "self"). In your own program you may not call it here, but you would have a call to "after" somewhere in your program to schedule the next cycle. That would wake up your program and have it do everything - read the data, log it (if necessary), update the graph, etc.

The call to "after" schedules the next wake-up and then exits. The program then returns to the main loop (we'll get to that later).


root = Tkinter.Tk()
root.title('A simple graph.')

At this point we're back in the outermost level of the program (the equivalent to a 'main' in C). Here, we've created a window and called it "root". We then add a title to the title bar of the window. Tkinter is the name of the module we imported, and somewhere inside it has a class called "Tk". Calling Tkinter.Tk() creates an object. Note the use of "()". That causes the "__init__" function in the class to execute. Without that we're just getting a reference (effectively a pointer) to the class, not creating an initialised object.


canvas = Tkinter.Canvas(root, width=550, height=250, bg = 'white')
canvas.pack()

Here we're creating a canvas object (which we just called "canvas"). A canvas is a GUI widget that you can draw on using drawing properties. All the toolkits that I am familiar with have something similar. The "pack" tells the canvas to fit itself into the window. You're going to want to read up on this but it is a common GUI concept. If you've used something like Glade however, the concepts will become familiar even if you didn't realise what was going on at the time. You have different options for packing widgets into containers, and packing containers into other containers, so that the GUI will lay itself out automatically into the best fit. You can position things using absolute coordinates, but that is not the normal way to do things. This isn't special to Tkinter. Gtk+, wxWidgets, or just about anything else will work the same way.

If anyone else who is reading this is trying to relate these concepts to how you do things in something like VB, the concepts may not translate directly. The native MS Windows GUI system is rather primative compared to what is used just about anywhere else. Here you deal with the GUI at an abstract level and use containers and packing options to say what you want to achieve and let the computer figure out how to actually position things for you (e.g. I want to arrange these buttons horizontally across the bottom of the window).


Tkinter.Button(root, text='Quit', command=root.quit).pack()

This adds the "quit" button. A "quit" method is built into Tkinter (to let us exit the program), but we could have used a different function to do something else (e.g. command=SimpleGraph.PrintGraph if we had a "PrintGraph" function in our graph object.)


SimpleGraph = Graph()

We create our graph object by calling our Graph function (note the use of the "()" - this is important). If this was a general purpose graph library we could have passed parameters into it to specify the size, colour, axis lables, etc. For example we could write a library class that looked like this: MoreComplicatedGraph = Graph2(5, 10, 'Time', 'Temperature'). In our example above however, we didn't use any parameters.


SimpleGraph.UpdateGraph()

We make a first call to UpdateGraph to start the cycle running. After this, the call to "after" in the function will schedule subsequent calls.


root.mainloop()

This has to be at the *end* of our start up. It doesn't exit "mainloop" until our program exits. Instead, mainloop dispatches calls to whatever functions we've scheduled to handle mouse clicks, keyboard input, and timed delays (including "after"). Again, this is a common GUI concept (the event dispatcher) and other toolkits will have something similar.




Posted by curt wuollet on 1 June, 2010 - 4:32 am
Hi Michael

> I don't mind making the explanation and I imagine that other people are interested as well. Go ahead and ask questions, this is a subject that I like to talk about. <

Yes, I kinda sensed that you are a pythonista.

> but a lot of the people who contact me directly about actual applications they are working on seem to deploy on Debian. <

My guess was pretty good then. I am using a fairly old version, Fedora 10, this is my "stable" machine and it wiil probably stay with F10 until I have good reason to change. I will see if they have back ported the newer Python. If a yum update doesn't do it. I'll probably upgrade another machine to F12 and move programming efforts to it. Or, I suppose I can see if I have the deps and compile from source.

> The deque was something that I added in the last version to save a couple of lines. You just have to go back to the method shown in the previous ones to fix the problem:

---- snip ---

> Most editors understand Python syntax and will highlight and auto-indent (or whatever they do) for it, but you may have to enable that feature if it's turned off.

> Here it is with the changes. I will include a line by line explanation just after the listing. <

Yes, the style enforcement seems a bit strange and it conflicts with my ingrained editing methods like searching for braces to jump around, but I'll eventually get used to it. I even considered learning a new IDE or editor, but OO is enough change for now. And some do bad things when you do VI commands out of habit.

That's it for tonight, the sun will be coming up soon.

Regards,
cww


Posted by M Griffin on 1 June, 2010 - 6:42 pm
Fedora 13 came out at the beginning of last week. If you (or the user) prefer a Red Hat style distro for deployment but don't like Fedora's rapid turn over, then Centos is probably a good choice. However, that's an "enterprise" server distro, so you're back to dealing with older versions again. I had someone contact me recently about wanting a change to allow Python 2.4 support for Centos (in that case it's just a matter of porting some date calculation code over).

As for Python versions, the language is in transition between the 2.x series and the 3.x series. The 3.x series gets rid of a number of deprecated features (there is a systematic development and deprecation cycle), rearranges some libraries, and makes a few changes which are not completely backwards compatible (e.g. strings are unicode by default now). However, the majority of people are still using the 2.x series because existing releases of "stable" enterprise distros are still using older versions, and also not all third party libraries have been ported yet. Newer distros are starting to package parallel installs of 3.x, so the 3.x transition will probably start to pick up steam now. Version 2.6 (and the upcoming 2.7) is designed as a "bridge", and 2.x support will continue for as long as there is demand for it.

There are multiple implementations of the Python language (at least half a dozen that I know of), including several from major software companies. The long and the short of this however is to stick with what is shipped with your distro, or what whatever default version is shipped with the platform you intend to deploy on.

As for vi, I believe that has a Python syntax template (or extension, or whatever - I don't happen to use vi). Most programming editors do. Very few people use IDEs or special editors with Python however. Most debugging is done via tracebacks (the error message you posted earlier) or just inserting print statements. There is a debugger, but there is seldom need to use it. I find debugging with print statements quite satisfactory, because I can very easily insert some some conditional code to provide more useful debugging information. Doing that with a conventional debugger (any debugger) usually involves jumping through a few hoops.

Turn around time on a edit-test cycle is instantaneous, so the work process with Python tends to be different from that used with many other languages. You work incrementally by writing a small amount of code, testing it, adding some more, testing it, etc. If there is something you aren't sure of, you can check it interactively in a console window. The result of that is you don't often get into a situation where you need long in-depth debugging sessions. If there is a problem, it's usually something you've done in the past few minutes.

You do need to learn to read trace-backs (error messages) however. They give you the module name and line number where the error was which gives you a good starting point for finding the root cause. That's just a matter of experience however.


Posted by Steinhoff on 31 May, 2010 - 4:57 am
> Here it is again with 50 points, and with grid and axis labels. This could obviously be made into a library which automatically calculates the axes, grid positions, and labels, but I have hard coded all the parameters here to keep the example short and simple.

> You will notice that I keep track of the line using the self._PrevLine variable so I can delete just the line and leave the other graphics alone. I also made a few other minor changes (e.g. used a deque) to keep the example short.

> CPU usage for this on a slow computer is about 1% for the program, and overall the CPU is 97% idle. Forget about framebuffer graphics or anything like that for this sort of application. You don't need it. <

You need it if you want to develop efficient animated graphics. SDL and e.g. SDL-gfx are offering low level graphic widgets... you can also build your own low level widgets which allows you to optimize the blitting actions down to single pixels.

> This is not high performance graphics. For this sort of project you just want to get the program finished as quickly as possible with the minimum effort and number of lines of code. I'm not aware of anything else that even comes close to Python for those types of projects. <

Yes, I like Python very much. PyQt, PyGtk are also an interesting options for standard GUIs. But for object oriented graphical frameworks go with ROOT and CINT. BTW ... the integration of a C/C++ library into CINT (C/C++ interpreter, http://root.cern.ch) is a matter of few text lines!

(http://root.cern.ch/phpBB3/viewtopic.php?f=5&amp;t=10314 scroll down for links)

> This will also be cross-platform. I have written Python programs on Linux that run as is on MS Windows without any making any special effort other than to use the normal Python libraries (e.g. let the libraries handle path separators and line endings). That includes communicating with IO through Ethernet and serial connections. <

Ethernet Powerlink, EtherCAT, Modbus-TCP and EthernetIP are available by open source projects ... and they are compiling on Linux "out of the box"

> The only differences that I've found are platform limitations (things that MS Windows simply doesn't have, or poorer performance due to OS design limitations). You need to test for those, but generally it is a lot easier to develop Python programs on Linux and then copy them to MS Windows if that is what the user wants to use.

> How are you going to interface with the hardware? If that involves a C library, Python has lots of options there as well. <

> If you have any more questions, let me know and I will be happy to answer them.

> [ clip]

> # This is the main event loop.
> root.mainloop()

The main loop is the weak point. It should allow conditional blitting...

Regards
--Armin


Posted by M Griffin on 31 May, 2010 - 5:50 pm
In reply to Armin Steinhoff: Displaying a scrolling strip chart doesn't require efficient animated graphics. Your points are quite valid, but my own point was that a scrolling strip chart isn't very demanding. I do the same thing by the way in a web browser using SVG and Javascript, and CPU usage is also very low.

You said that "Ethernet Powerlink, EtherCAT, Modbus-TCP and EthernetIP are available by open source projects"

Modbus-TCP is, without question open. With EtherCAT the specs were initially open, but then the EtherCAT people changed their minds when and sent their lawyers after a university in either Belgium or The Netherlands (I forget which) and force them to close down their open source EtherCAT project. The EtherCAT people were demanding registration, licensing and certification (which the university wasn't willing to to pay). If that has changed back once again, then I would appreciate hearing about any information you might have on it.

For Ethernet Powerlink, I wouldn't mind any up to date information.

With EthernetIP so far as I know they're still proprietary. You can get source, but it's still under a proprietary license. They were calling it a "modified BSD license", but the "modification" turned out to be to toss out all the BSD clauses and substitute their own.

They actually had two different licenses in the package. The first was BSD-like. The second (which you had to unpack everything to find) was their traditional proprietary license (the usual purchase a proprietary license, registration, certification, and approval).

The hosting service would see the first license, which lets the project qualify for free hosting (Free Software projects get free hosting because that helps promote the hosting company's commercial project hosting business). Under the second (hidden) license however, they should be paying for hosting under normal commercial terms (the hosting company isn't running a charity for big businesses). I'm not sure of the motivations behind having a hidden license, but it looks a bit unusual to me.

If you know of a version of EthernetIP that is available under genuinely open terms, I would be very interested in hearing about it.


Posted by Steinhoff on 1 June, 2010 - 4:00 am
> Displaying a scrolling strip chart doesn't require efficient animated graphics. Your points are quite valid, but my own point was that a scrolling strip chart isn't very demanding. I do the same thing by the way in a web browser using SVG and Javascript, and CPU usage is also very low. <

OK ... I did a test on a LINUX machine. When you scroll more smoothly the CPU load is much higher ... but not so dramatically :)

> You said that "Ethernet Powerlink, EtherCAT, Modbus-TCP and EthernetIP are available by open source projects"

> Modbus-TCP is, without question open. With EtherCAT the specs were initially open, <

The specs of the standard IEC 61784 are open and will never be closed.

"EtherCAT" is a proprietary name of a product based on the fieldbus standard IEC 61784-2. You can't use the name "EtherCAT" without the permission of its owner ... IMHO, that's normal.

> but then the EtherCAT people changed their minds when and sent their lawyers after a university in either Belgium or The Netherlands (I forget which) and force them to close down their open source EtherCAT project. The EtherCAT people were demanding registration, licensing and certification (which the university wasn't willing to to pay). If that has changed back once again, then I would appreciate hearing about any information you might have on it. <

No, no this are simply fairy tales ... you can always use the source of this project without registration and certification:

http://www.etherlab.org/en/ethercat/index.php

But don't use the name "EtherCAT" !!

> For Ethernet Powerlink, I wouldn't mind any up to date information. <

Here are the links:

http://sourceforge.net/projects/openpowerlink
opensource Powerlink stack for LINUX, Windows und FPGAs

http://sourceforge.net/projects/openconf
open source configuration program for Powerlink

It's completely open !

---- snip ----
> If you know of a version of EthernetIP that is available under genuinely open terms, I would be very interested in hearing about it. <

Here is the link to the Ethernet/IP sources:

http://sourceforge.net/projects/opener (partial implemention) ... but the "openness" of that licensing is indeed very strange :)

Best Regards,
Armin Steinhoff


Posted by M Griffin on 1 June, 2010 - 7:48 pm
In reply to Armin Steinhoff: Thanks for the links. I just had a look at these, and the terms for Powerlink look quite reasonable.

As for Ethercat, the problem wasn't "fairy tales", I was referring to an announcement in an actual press release a couple of years ago. I think it was the University of Leuven (but I'm not 100% sure it was them). I can't find that original press release (I had no reason to keep it), but I just did a search and found a statement from an organisation affiliated with them which alludes to a compromise having been reached with Beckhoff which allows the code to be publicly released again.

"The concerns that open source projects might dilute conformance with the standard were met by careful additions to the GNU public license: code developed in this project has to remain compatible with the EtherCAT specs and a valid EtherCAT implementation."

I didn't see these terms in the actual source code release package however, so the status is a bit unclear. Their web site states:

"The license above concerns the source code only. Using the EtherCAT technology and brand is only permitted in compliance with the industrial property and similar rights of Beckhoff Automation GmbH."

The trademark policy is quite reasonable and not unusual. The bit about "using the ... technology ... in compliance with the industrial property and similar rights of Beckhoff Automation GmbH" would have me a bit concerned however. That seems to imply that Beckhoff could claim a license is required to use their "technology" (this is a vague term that many proprietary software companies use in their license contracts - an ODVA contract uses similar wording).

If I was going to use that software, I would want that cleared up. I don't have access to any EtherCAT hardware at this time however, so it isn't something I could pursue at the moment anyway.

As for the EthernetIP software, the license states:

"the software does not, by itself, convey any right to make, have made, use, import, offer to sell, sell, lease, market, or otherwise distribute or dispose of any products that implement this software, which products might be covered by valid patents or copyrights of ODVA, Inc., its members or other licensors"

That is a proprietary software license. Nothing has changed with Ethernet/IP. Being able to get source code does not by itself make something "open source".

On a related topic, I was contacted recently by someone who is working on a DBus interface for industrial protocols (DBus is an interprocess communications bus for Linux and other unix-like operating systems). His objective is to provide an open means of interfacing protocol drivers to any software that wants to use them. It is somewhat analogous to OPC, but it is intended to be completely open and to be a bit more focused on protocols and have less of the "everything plus the kitchen sink" approach that OPC has turned into.

He is in the early development stages now, and working on a proof of concept demonstration. He contacted me to see if I would be interested in adding support for it in my software (which I will, if it works as planned).

If you are interested, I could tell you how to contact him. He's in Switzerland, and works in the process control field.


Posted by curt wuollet on 1 June, 2010 - 11:50 pm
Please keep us informed on the DBus thing. Linux for Automation really needs something like that.

Regards,
cww


Posted by Steinhoff on 2 June, 2010 - 3:59 am
> "The concerns that open source projects might dilute conformance with the standard were met by careful additions to the GNU public license: code developed in this project has to remain compatible with the EtherCAT specs and a valid EtherCAT implementation." <

Again ... this is only the case if you want to use the proprietary name "EtherCAT" . You can always do an implementation based on the IEC standard ... and call it EtherRing or something else. Sources are available at:

http://developer.berlios.de/projects/ethercatmaster ( have problems to reach the homepage of this project)

http://developer.berlios.de/projects/soem

> I didn't see these terms in the actual source code release package however, so the status is a bit unclear. Their web site states:

> "The license above concerns the source code only. Using the EtherCAT technology and brand is only permitted in compliance with the industrial property and similar rights of Beckhoff Automation GmbH."

> The trademark policy is quite reasonable and not unusual. The bit about "using the ... technology ... in compliance with the industrial property and similar rights of Beckhoff Automation GmbH" would have me a bit concerned however. That seems to imply that Beckhoff could claim a license is required to use their "technology" (this is a vague term that many proprietary software companies use in their license contracts - an ODVA contract uses similar wording).

> If I was going to use that software, I would want that cleared up. I don't have access to any EtherCAT hardware at this time however, so it isn't something I could pursue at the moment anyway. <

To buy a Realtek 8169 based Ethernet board should not be a big issue ...

> On a related topic, I was contacted recently by someone who is working on a DBus interface for industrial protocols (DBus is an interprocess communications bus for Linux and other unix-like operating systems). His objective is to provide an open means of interfacing protocol drivers to any software that wants to use them. It is somewhat analogous to OPC, but it is intended to be completely open and to be a bit more focused on protocols and have less of the "everything plus the kitchen sink" approach that OPC has turned into. <

Are you aware about an open implementation of OPC for Python ? ->
http://openopc.sourceforge.net/

> He is in the early development stages now, and working on a proof of concept demonstration. He contacted me to see if I would be interested in adding support for it in my software (which I will, if it works as planned).

> If you are interested, I could tell you how to contact him. He's in Switzerland, and works in the process control field. <

There is a lot of infos about DBus on the net:
http://freedesktop.org/wiki/Software/dbus
AFAIK ... DBUs was initiated by the KDE project.

Best Regards,
Armin Steinhoff


Posted by M Griffin on 2 June, 2010 - 10:20 pm
In reply to Armin Steinhoff:

>To buy a Realtek 8169 based Ethernet
>board should not be a big issue ...

The issue would be buying a selection of field I/O to test with. The software that I've seen just gives you a client (master) only. I don't have anything against EtherCAT itself, I just don't have any particular incentive to put the resources into pursuing it further at this time.

>Are you aware about an open implementation of OPC for Python ? ->
>http://openopc.sourceforge.net/

I'm aware of that, but it's just a client binding for OPC. I have a program called HMIServer (web based HMI) which I modified last year to make it easier for a user to use OpenOPC with it. It doesn't replace the OPC server; you still need one of those. The "multi-platform support" they talk about just means that you can proxy to an OPC server from another computer by using Pyro's RPC instead of DCOM.

>AFAIK ... DBUs was initiated by the KDE project.

As a trivial point, KDE used DCop. DBus was developed mainly by Red Hat who did copy a lot of ideas from DCop. DBus was then adopted by Gnome, and I believe that KDE then adopted it at some point in version 4. It's now considered to be a common infrastructure project and is also being ported to other platforms.

Here's the web page where the the guy is working the specs for what he is called "DBus for process control" (DBPC). According to what I have heard from him, he has done some tests and is working on some proof of concept code to test how well the idea will work.

http://freedesktop.org/wiki/Specifications/DBPC

I don't see a contact link there, but if anyone wants to get a hold of him I have his e-mail address (I can't post it here however, for obvious reasons).


Posted by Steinhoff on 3 June, 2010 - 9:07 am
[clip ...]

> I'm aware of that, but it's just a client binding for OPC. I have a program called HMIServer (web based HMI) which I modified last year to make it easier for a user to use OpenOPC with it. It doesn't replace the OPC server; you still need one of those. The "multi-platform support" they talk about just means that you can proxy to an OPC server from another computer by using Pyro's RPC instead of DCOM. <

I found at OpenOPC the link to an OPC XML-DA based implementation: http://pyopc.sourceforge.net

Any comment about that project?

Armin Steinhoff


Posted by M Griffin on 3 June, 2010 - 11:47 pm
Armin Steinhoff:

> I found at OpenOPC the link to an OPC XML-DA based implementation:
> http://pyopc.sourceforge.net

>Any comment about that project? <

It's a toolkit for building OPC XML-DA servers or clients. I don't know how well it works, as I've never used it. It uses the Twisted communications framework (which I also use), the Zolera SOAP libraries, and some additional code of its own.

You could use it to write a client interface to access an OPC XML-DA server. You could also write your own OPC XML-DA server. Again though, I don't know how well it actually works. It had one release several years ago. If I recall correctly, I believe it was a project by a university student who wrote it as part of his studies.

There's not much to say about it beyond that. So far as I know, there aren't a lot OPC XML-DA servers or clients around, so I'm not sure how useful it would be.

The difference between OpenOPC and PyOPC is that OpenOPC provides a client interface (and an optional proxy) to the more common DCOM based OPC, while PyOPC works with the less common OPC XML-DA and also lets you write servers as well as clients. OPC XML-DA is however if anything slower than the regular DCOM based OPC, so it's probably not a solution for general IO applications.

If I was going to use PyOPC for anything, I would be prepared to spend some time testing and debugging it to make sure it worked acceptably with the specific end point it needed to connect to. That's not a comment on the author of PyOPC. It has to do with the use of SOAP in OPC. I've worked with SOAP in the past (including the Zolera libraries), and cross vendor compatibility with it was very poor. The SOAP protocol is very verbose and poorly specified, so when something doesn't work it's hard to say where the problem is.

I don't know if that answered your questions on this. If not, I would be happy to say more.


Posted by curt wuollet on 3 June, 2010 - 2:36 am
Given their consistent history of embrace, extend, and destroy, I'm too old to invest time in any technology they have any measurable degree of control over. If they were really interested at all in open anything they would adopt existing languages and protocols instead of shunning anything "not invented here". Indeed, I'm pretty sure that's what the whole .NET thing is about, derailing Java and other popular modern languages that are cross platform. That's not to say it isn't good stuff, especially from their point of view. I don't know because the further away from their stuff I stay the easier things become. Witness what they've done to pervert ODF.

Regards,
cww


Posted by curt wuollet on 3 June, 2010 - 2:52 am
Hi Michael, all

Trimmed for brevity

Ok

Here's how I hacked up Michael's elegant code to work like a chart recorder. With the loops, I couldn't get scrolling to work, specifically saving and deleting the last set of vertical grid lines so the grid scrolls with the lines when you hit the end. Please be kind as I just started doing this and I'm sure I violated some of the feel and intent of Python. But it runs and except that I seem to recall physical CRs moving the paper the other way, it looks right. Now I need to add times that move with the grid and I'm sure there is a more efficient way to do the grid. Especially if I want to add ticks on the grid.

***************************************************

# simple example that draws a scrolling line graph.
import Tkinter
import random

# We create a class to make it easier to keep static data between calls.
class Graph:

def __init__(self):
# Initialise the first Y point.
self._YPoints = [451]
self.offset = 50
# The X points are all pre-defined.
self._XPoints = range(40, 490, 1)
# Draw the axes.
canvas.create_line(40, 225, 490, 225, fill='blue', width=2)
canvas.create_line(40, 10, 40, 225, fill='blue', width=2)
# Label the axes.
#for x in range(40, 490, 50):
# canvas.create_text(x, 235, text=(x - 40))
for y in range(10, 225, 25):
canvas.create_text(15, y, text=(225 - y))
# Draw the grid.
# I'm doing all this stuff up here so that I can
# pass the lines to delete them when scrolling
self._Vertline1 = canvas.create_line(90, 10,90 , 225, fill='green', width=1)
self._Vertline2 = canvas.create_line(140, 10,140 , 225, fill='green', width=1)
self._Vertline3 = canvas.create_line(190, 10,190 , 225, fill='green', width=1)
self._Vertline4 = canvas.create_line(240, 10,240 , 225, fill='green', width=1)
self._Vertline5 = canvas.create_line(290, 10,290 , 225, fill='green', width=1)
self._Vertline6 = canvas.create_line(340, 10,340 , 225, fill='green', width=1)
self._Vertline7 = canvas.create_line(390, 10,390 , 225, fill='green', width=1)
self._Vertline8 = canvas.create_line(440, 10,440 , 225, fill='green', width=1)
self._Vertline9 = canvas.create_line(490, 10,490 , 225, fill='green', width=1)

for y in range(10, 225, 25):
canvas.create_line(40, y, 490, y, fill='grey', width=1)

# We draw a small line to initialise the line delete operation
# so we don't have to check each time if a line is already there.
self._PrevLine = canvas.create_line(0,0,0,0)

def UpdateGraph(self):
# Create a new Y point.
self._YPoints.append(random.uniform(50, 200))
# Shift the Y buffer left one it is full.
if len(self._YPoints) > 450:
self._YPoints.pop(0)
self.offset -=1
# This bit of magic combines the X and Y points in one list.
GraphPoints = list(sum(zip(self._XPoints, self._YPoints), ()))
canvas.delete(self._Vertline1)
self._Vertline1 = canvas.create_line(self.offset+40, 10,self.offset+40 , 225,
fill='green', width=1)
canvas.delete(self._Vertline2)
self._Vertline2 = canvas.create_line(self.offset+90,
10,self.offset+90 , 225, fill='green', width=1)
canvas.delete(self._Vertline3)
self._Vertline3 = canvas.create_line(self.offset+140,
10,self.offset+140 , 225, fill='green', width=1)
canvas.delete(self._Vertline4)
self._Vertline4 = canvas.create_line(self.offset+190,
10,self.offset+190 , 225, fill='green', width=1)
canvas.delete(self._Vertline5)
self._Vertline5 = canvas.create_line(self.offset+240,
10,self.offset+240 , 225, fill='green', width=1)
canvas.delete(self._Vertline6)
self._Vertline6 = canvas.create_line(self.offset+290,
10,self.offset+290 , 225, fill='green', width=1)
canvas.delete(self._Vertline7)
self._Vertline7 = canvas.create_line(self.offset+340,
10,self.offset+340 , 225, fill='green', width=1)
canvas.delete(self._Vertline8)
self._Vertline8 = canvas.create_line(self.offset+390,
10,self.offset+390 , 225, fill='green', width=1)
canvas.delete(self._Vertline9)
self._Vertline9 = canvas.create_line(self.offset+440,
10,self.offset+440 , 225, fill='green', width=1)


if self.offset == 0:
self.offset = 50
# Delete the previous line.
canvas.delete(self._PrevLine)
# Draw a new line.
self._PrevLine = canvas.create_line(GraphPoints, fill='red', width=2)
# Call the update function again later.
root.after(25, self.UpdateGraph)


# This creates the window.
root = Tkinter.Tk()
root.title('A simple graph.')

# This creates the "canvas" object to draw on.
canvas = Tkinter.Canvas(root, width=550, height=250, bg = 'white')
canvas.pack()

# This makes a "quit" button.
Tkinter.Button(root, text='Quit', command=root.quit).pack()

# This creates the graph object.
SimpleGraph = Graph()

# Start the graph running.
SimpleGraph.UpdateGraph()


# This is the main event loop.
root.mainloop()

***********************************************

Still a fairly small program.

Regards
cww


Posted by M Griffin on 4 June, 2010 - 2:49 am
Here it is with the grid shifting to the left. For the items that I want to shift, I gave them a 'tag'. I can then use canvas.move to move all the items with that tag by a specified x and y amount.

For the vertical lines, I move them one to the left until they have moved one full grid increment. At that point, I move them back to the right and start over again. The same thing happens with the text, except I delete the text and re-label it.

I've added two more functions - one to shift the grid, and one to label the X axis.

A possible optimisation technique for the graph might be to draw the data trace in alternating blocks and moving both blocks to the left (while appending to the right most block). The leftmost block would slide *under* a rectangle to the left of the graph, and the block would get deleted once it had passed completely off the chart. That avoids redrawing the entire line for each point. I haven't tried this because I thought from the application description you were going to be using a lot less data at a much slower speed. I can explain that idea further if you wish.

I've also added a new feature - printing. It prints the graph to a postscript file. There are a lot of different possible configuration options for this, but I've just left it as the defaults.

There are three other points to make here. One is that you can include a #!/usr/bin/python at the top of the file and set the executable bit, and you can run it directly (e.g. "./graph.py"). My examples had that, but your version was missing it. That feature doesn't work on MS Windows of course, as the OS doesn't have that capability.

Another point is that you will see that I've added comments just below a couple of the function names which use triple quotes (""") to start and end the block of comments. A triple quote denotes a block of text. It is a common convention to use this feature for class and function level comments. A lot of software can pull these comments out of the program and display them in documentation.

The final point is that don't mix indentation types. Either use all tabs or all spaces. Having some lines as tabs and some as spaces is asking for trouble since indentation matters. I like to use tabs but either method is acceptable provided you are consistent.



#!/usr/bin/python
# simple example that draws a scrolling line graph.
import Tkinter
import random

# We create a class to make it easier to keep static data between calls.
class Graph:


def Xlabel(self, textoffset):
""" Create the x axis labels for a graph.
Parameters: textoffset (integer) = An offst to add to the
label values.
"""
# Label the axes.
for x in range(90, 540, 50):
canvas.create_text(x, 235, text=(x + textoffset - 40), tag='xlabel')

def ShiftGrid(self):
"""This shifts the grid one increment to the left each time it
is called. When the grid has moved one full grid increment, it
is shifted back and relabeled.
"""
# Move the vertical grid lines and labels to the left.
canvas.move('grid', -1, 0)
canvas.move('xlabel', -1, 0)
self._GridPos -= 1
# When the grid has moved one full grid increment, move it
# back to the start position.
if self._GridPos <= 0:
self._GridPos = 50
canvas.move('grid', 50, 0)
# Delete the old labels and draw new ones.
canvas.delete('xlabel')
self._TextOffset += 50
self.Xlabel(self._TextOffset)



def __init__(self):
# Initialise the first Y point.
self._YPoints = [125]

# The X points are all pre-defined.
self._XPoints = range(40, 490, 1)
# Draw the axes.
canvas.create_line(40, 225, 490, 225, fill='blue', width=2)
canvas.create_line(40, 10, 40, 225, fill='blue', width=2)

# Label the axes.
self._TextOffset = 0
self.Xlabel(self._TextOffset)
for y in range(10, 225, 25):
canvas.create_text(15, y, text=(225 - y))

# Draw the grid.
for x in range(90, 540, 50):
canvas.create_line(x, 10, x, 225, fill='green', width=1, tag=('grid'))
self._GridPos = 50


for y in range(10, 225, 25):
canvas.create_line(40, y, 490, y, fill='grey', width=1)

# We draw a small line to initialise the line delete operation
# so we don't have to check each time if a line is already there.
self._PrevLine = canvas.create_line(0,0,0,0)

def UpdateGraph(self):
# Create a new Y point.
self._YPoints.append(random.uniform(50, 200))
# Shift the Y buffer left one it is full.
# Also, shift the grid to the left.
if len(self._YPoints) > 450:
self._YPoints.pop(0)
self.ShiftGrid()

# This bit of magic combines the X and Y points in one list.
GraphPoints = list(sum(zip(self._XPoints, self._YPoints), ()))

# Delete the previous line.
canvas.delete(self._PrevLine)
# Draw a new line.
self._PrevLine = canvas.create_line(GraphPoints, fill='red', width=2)

# Call the update function again later.
root.after(25, self.UpdateGraph)


def PrintGraph(self):
"""Print the graph to a file.
"""
f = open('graphout.ps', 'w+b')
output = canvas.postscript()
f.write(output)
f.close()


# This creates the window.
root = Tkinter.Tk()
root.title('A simple graph.')

# This creates the "canvas" object to draw on.
canvas = Tkinter.Canvas(root, width=550, height=250, bg = 'white')
canvas.pack()

# This creates the graph object.
SimpleGraph = Graph()

# This makes a "quit" button.
Tkinter.Button(root, text='Quit', command=root.quit).pack(side=Tkinter.LEFT)
Tkinter.Button(root, text='Print', command=SimpleGraph.PrintGraph).pack(side=Tkinter.RIGHT)

# Start the graph running.
SimpleGraph.UpdateGraph()


# This is the main event loop.
root.mainloop()


Posted by curt wuollet on 5 June, 2010 - 12:11 am
I figured that horrible hacking might drive you to the keyboard:^) I'll have to study the tags bit. Actually we are thinking along the same lines, but I can see drawing and redrawing all the lines and text with three more channels might be a lot of code and the starting static and getting dynamic when you reach the scrolling point makes things messy. The way I did it would require each label to be an attribute so you can use canvas.remove

I've been looking at wall clock times on the chart divisions and that is quite a head scratcher too. The scoping and immutable variables has been a change for sure. I guess they are saying anything that belongs to a class instance should be an attribute, to use the vernacular. It's been a fairly long time since I hacked till the sun came up:^)

The approach that has come to mind is well beyond my python skills at the moment, but that's a good way to learn for me. My thought is to have two sheets of "graph paper" with the markings built in (loaded from a bitmap) each would be one canvas length. Show one and draw the variable on it and the labels as they are reached. Shift modes and scroll sheet 1 off the canvas as you scroll sheet 2 on, labeling and drawing the variable as you go. Rinse and repeat. That way you would only be drawing the variable and labels and moving the labels and you can simply reuse the two sheets of graph paper. I didn't see in my perusal of the Canvas widget how to do this, but pygame looked to have the stuff. Does that make any sense or am I thinking like an old C hack?

Regards,
cww


Posted by M Griffin on 5 June, 2010 - 2:55 pm
The grid works fairly well with the present scheme, which is to shift the lines and text, and then move the grid back and re-draw the next once per division (every 50 points). The grid numbers are meaningless at the moment, but that's just a scaling issue.

Your idea of "two sheets of graph paper" is what I was trying to describe as "alternative blocks". This would be used for just the data traces, as the grid seems to work fine with the existing system.

We would have two data traces which we'll call 'a' and 'b'. We start by drawing 'a', and add one line segment at a time (similar to the first example) rather than drawing the entire line with one list. We would also apply a tag of 'a' to each line segment.

When the chart was full, We start moving it to the left (using the tag), but now add new lines to 'b'. When 'a' is completely off the chart, we can delete it. Then we stop adding to 'b' and start adding to 'a' at the right of the chart again.

With this method we don't erase and re-draw the entire chart each time we add a point. We only draw one segment at a time, and only delete a plot once for every screen full.

The only thing really new required here is to have a rectangle at the left side of the chart which the line would pass under in order to hide the line as it passes off the chart. Alternatively, it might be possible to have a canvas on top of (or inside) a canvas so that the decorations are on one canvas, and the line plot on the other.

It will by the way be easier with this scheme if the plot starts from the right side of the chart and flows left rather than filling from the left and only then start scrolling. I've seen charts use both methods, and the current method is easier if you delete and re-draw the entire line each time. Starting from the right however is probably easier if we are moving the lines however, as the new lines always get drawn in the same place (the two X coordinates never change).

I'll do a new example of this and post it.


Posted by M Griffin on 5 June, 2010 - 10:11 pm
Here's a version with some additional changes. This one adds individual line segments to the right side of the chart. It labels plots in alternating blocks. When a block of points has completely scrolled off the screen, they are all erased. There is a rectangle drawn on the left side of the chart, and the plot lines go *under* this rectangle. That's why you don't see them when they go off the left side of the plot area.

I made a few other changes. I converted the numeric literals to variables and then calculated the necessary derived values so that I could change one number and have everything automatically reflected elsewhere. I'm not 100% sure that I have all the equations correct however.

I also added three more plot lines, since you said you were using more than one.

I also found the graph to almost unreadable at the density that you originally used. There are simply too many points on the screen. I spread the points out further (2 pixels apart instead of 1), slowed the chart down (50 msec update instead of 25), made the chart bigger, and made the plot lines narrower. You might need to change that back if those numbers dont' suit your application. Even with these values I find the chart to be too dense for my tastes.

I also added a new class (object) to simulate collecting data. This is a more reasonable place to put the timed call-back (rather than in the plot code). It calls the plot code to update data. I put the data in a list (array) because that was more flexible if plots need to be added or removed. The graph object still needs to be initialised with the update interval however, so that it can calculate the correct X axis labels.

Another thing that has changed is that the canvas is now local to the graph object instead of being global. This is also a bit more realistic.

A new feature is that it is using tag_raise and tag_lower. These "raise" or "lower" objects in the graphics stack relative to each other. This is used to for example make the graph plots go under the rectangle which acts as a mask on the left side of the chart.

There are by the way lots of off the shelf plotting modules for Python, but they are normally intended for scientific use where you collect data and then display it. Most of these are just Python bindings for the same plotting libraries that everyone else uses.

There is also one called Blt which is part of Pmw (an additional widget library for Tkinter). It supposedly is a scrolling strip chart widget which should do the job exactly. However, I've never been able to get it to work (not even with their examples), so I can't recommend it.

If you're looking for numeric processing (ffts, matrix math, digital filters, etc.) there are libraries called Numpy and Scipy which are very widely used. They are mostly Python bindings to standard Fortran and C numerical libraries that everyone uses. Some of the expensive commercial libraries that you can buy are simply these same open source libraries bundled up and re-packaged.

#!/usr/bin/python
# simple example that draws a scrolling line graph.
import Tkinter
import random

############################################################
class Graph:
"""We create a class to make it easier to keep static data between calls
"""


###################################################
def __init__(self, window, timescale):
"""window = Reference to window the canvas is to be placed in.
timescale = Update interval in milliseconds.
"""

# X offset for graph origin.
self._GXOffset = 40
# Width of graph itself.
self._GXWidth = 750
# Padding on right of graph.
self._GXPad = 25
# Height of graph.
self._GYHeight = 400
# Padding for bottom of graph.
self._YPad = 50
# Padding for top of graph.
self._YTopPad = 10
# X increment for grid.
self._GridXInc = 50
# Y increment for grid.
self._GridYInc = 25
# X axis resolution.
self._XResolution = 2
# Time increment for each update.
self._TimeInc = (self._GridXInc * timescale * 1.0) / (self._XResolution * 1.0) / 1000.0


# This creates the "canvas" object to draw on.
self._GrCanvas = Tkinter.Canvas(window, width=self._GXWidth + self._GXOffset + self._GXPad,
height=self._GYHeight + self._YPad + self._YTopPad, bg = 'white')
self._GrCanvas.pack()

# Place a rectangle to act as a mask for the graph.
self._GrCanvas.create_rectangle(0, 0, self._GXOffset, self._GYHeight + self._YTopPad,
outline='', fill='white', tag='mask')

# Draw the axes.
# X axis.
self._GrCanvas.create_line(self._GXOffset, self._GYHeight + self._YTopPad,
self._GXWidth + self._GXOffset,
self._GYHeight + self._YTopPad, fill='blue', width=2)
# Y axis.
self._GrCanvas.create_line(self._GXOffset, self._YTopPad,
self._GXOffset, self._GYHeight + self._YTopPad, fill='blue', width=2)

# Label the axes.
self._TextOffset = 0
self.Xlabel(self._TextOffset)
for y in range(0, self._GYHeight + self._GridYInc, self._GridYInc):
self._GrCanvas.create_text(self._GXOffset - 25, y + self._YTopPad,
text=(self._GYHeight - y), tag=('ylabels'))

# Draw the grid. Vertical lines.
for x in range(self._GridXInc, self._GXWidth + self._GridXInc, self._GridXInc):
self._GrCanvas.create_line(x + self._GXOffset, self._YTopPad, x + self._GXOffset,
self._GYHeight + self._YTopPad, fill='green', width=1, tag=('vgrid'))

# Tracks when the grid needs to shift.
self._GridPos = self._GridXInc


# Horizontal lines.
for y in range(0, self._GYHeight, self._GridYInc):
self._GrCanvas.create_line(self._GXOffset, y + self._YTopPad,
self._GXOffset + self._GXWidth, y + self._YTopPad, fill='grey', width=1, tag=('hgrid'))


# Position of current plot position.
self._PlotPos = self._GXWidth
# The current and previous plot tags.
self._CurrentTag = 'plota'
self._PreviousTag = 'plotb'

# The previous point we plotted.
self._PreviousYPoints = [(self._GYHeight / 2) + self._YTopPad] * 4
# The two X positions for each graph line segment that we draw.
self._XPoint1 = self._GXOffset + self._GXWidth - self._XResolution
self._XPoint2 = self._GXOffset + self._GXWidth

# Arrange the stacking order for the data.
self._GrCanvas.tag_raise('ylabels', 'mask')


###################################################
def UpdateGraph(self, datapoints):
"""datapoint = The new point to plot.
"""
# Shift the plot to the left.
self._ShiftPlot()
# Shift the grid to the left.
self.ShiftGrid()

# Append the new point to the graph.
self._GrCanvas.create_line(self._XPoint1, self._PreviousYPoints[0], self._XPoint2, datapoints[0],
fill='red', width=1, tag=self._CurrentTag)

# Append the new point to the graph.
self._GrCanvas.create_line(self._XPoint1, self._PreviousYPoints[1], self._XPoint2, datapoints[1],
fill='blue', width=1, tag=self._CurrentTag)

# Append the new point to the graph.
self._GrCanvas.create_line(self._XPoint1, self._PreviousYPoints[2], self._XPoint2, datapoints[2],
fill='yellow', width=1, tag=self._CurrentTag)

# Append the new point to the graph.
self._GrCanvas.create_line(self._XPoint1, self._PreviousYPoints[3], self._XPoint2, datapoints[3],
fill='violet', width=1, tag=self._CurrentTag)


# Make sure the data is below the mask rectangle.
self._GrCanvas.tag_lower(self._CurrentTag, 'mask')

# Save the point so we can plot it next time.
self._PreviousYPoints = datapoints


###################################################
def _ShiftPlot(self):
"""Shift the graph plot left one increment.
"""

# Shift both plots over one.
self._GrCanvas.move(self._CurrentTag, -self._XResolution, 0)
self._GrCanvas.move(self._PreviousTag, -self._XResolution, 0)
self._PlotPos -= self._XResolution

# Has the current plot block filled the screen entirely?
if (self._PlotPos < 0):
self._PlotPos = self._GXWidth

# Delete the *previous* plot block.
self._GrCanvas.delete(self._PreviousTag)

# Toggle the block selection
if (self._CurrentTag == 'plota'):
self._CurrentTag = 'plotb'
self._PreviousTag = 'plota'
else:
self._CurrentTag = 'plota'
self._PreviousTag = 'plotb'




###################################################
def Xlabel(self, textoffset):
""" Create the x axis labels for a graph.
Parameters: textoffset (integer) = An offst to add to the
label values.
"""
# Label the axes.
for x in range(0, self._GXWidth):
self._GrCanvas.create_text((x * self._GridXInc) + self._GXOffset + self._GridXInc,
self._GYHeight + self._YTopPad + 25,
text=((x + textoffset) * self._TimeInc), tag='xlabel')

###################################################
def ShiftGrid(self):
"""This shifts the grid one increment to the left each time it
is called. When the grid has moved one full grid increment, it
is shifted back and relabeled.
"""
# Move the vertical grid lines and labels to the left.
self._GrCanvas.move('vgrid', -self._XResolution, 0)
self._GrCanvas.move('xlabel', -self._XResolution, 0)
self._GridPos -= self._XResolution
# When the grid has moved one full grid increment, move it
# back to the start position.
if self._GridPos <= 0:
self._GridPos = self._GridXInc
self._GrCanvas.move('vgrid', self._GridXInc, 0)
# Delete the old labels and draw new ones.
self._GrCanvas.delete('xlabel')
self._TextOffset += 1
self.Xlabel(self._TextOffset)



###################################################
def PrintGraph(self):
"""Print the graph to a file.
"""
f = open('graphout.ps', 'w+b')
output = self._GrCanvas.postscript()
f.write(output)
f.close()



############################################################
class DataCapture:
""" Data could be read and processed here. This is a bit more realistic
than putting all the logic in the graph routines.
"""
###################################################
def __init__(self, twindow, interval):
"""Parameters: twindow = A reference to the window we want to use to attach
the call-back timer to.
interval (integer) = The desired update interval in milli-seconds.
"""
self._TWindow = twindow
self._UpdateInterval = interval


###################################################
def CaptureData(self):
"""Start capturing the data.
"""
# Capture some simulated data.
readings = []
readings.append(random.random() * 100 + 20)
readings.append(random.random() * 100 + 120)
readings.append(random.random() * 100 + 200)
readings.append(random.random() * 100 + 310)

# Update the graph with the new data.
SimpleGraph.UpdateGraph(readings)

# Call the update function again later.
self._TWindow.after(self._UpdateInterval, self.CaptureData)


# This creates the window.
root = Tkinter.Tk()
root.title('A simple graph.')

GraphFrame = Tkinter.Frame(root)

# Scan interval in milliseconds.
ScanInterval = 50

# This creates the graph object.
SimpleGraph = Graph(GraphFrame, ScanInterval)

GraphFrame.pack()

# This makes a "quit" button.
Tkinter.Button(root, text='Quit', command=root.quit).pack(side=Tkinter.LEFT)
Tkinter.Button(root, text='Print', command=SimpleGraph.PrintGraph).pack(side=Tkinter.RIGHT)

# This initialises the data capture routine.
DataGet = DataCapture(root, ScanInterval)
# Start getting data.
DataGet.CaptureData()


# This is the main event loop.
root.mainloop()


Posted by Curt Wuollet on 6 June, 2010 - 5:55 pm
Yes, the performance is really good on this one. I may switch back. I'm sure some of it is the small window. I'm doing a 1000x700 page for hi res hardcopy. Actually I will probably mess around with both and I'm getting accustomed to python in any case.

For the printing, the early TP version did line by line on a dot matrix printer. For today's page printers a directory of screens is probably more appropriate so they can be reviewed in any viewer and the ones of interest printed. Not exact CR emulation, but cheaper:^) I should look into a cheap DAQ card, the modern game ports drift quite a bit. In the XT days they were really surprisingly accurate for this app.

Regards
cww


Posted by M Griffin on 7 June, 2010 - 4:08 am
Here's some handy documentation links:

The official Python documentation. This is for the version 2.6. For other versions see the links at the upper left.
http://docs.python.org/index.html

This is a direct link to the library reference. I keep this link handy:
http://docs.python.org/library/index.html

Links to other Tkinter information:
http://wiki.python.org/moin/TkInter

This is a (incomplete) Tkinter reference:
http://effbot.org/tkinterbook/

Here's another:
http://infohost.nmt.edu/tcc/help/pubs/tkinter/

This is a summary of Tkinter:
http://www.astro.washington.edu/users/rowen/TkinterSummary.html

This is a link to the publisher of the book that I mentioned earlier (Python and Tkinter Programming)
http://www.manning.com/grayson/

Even if you don't want the book, it has two sample chapters as PDF files, and all the source code examples as a ZIP file. One of the sample chapters you can download happens to be "Graphs and charts", and one of the examples happens to be how to do strip charts. The strip chart example is oriented towards weather data and uses the "delete and redraw" approach that we tried previously (it actually does it in a different way than I did however). The example is not directly usable in your application, but the chapter is worth reading (as I said, it's an excellent book on GUI programming).

As for printing, the example that I posted earlier prints to a postcript file. I didn't set any options however. You can set scale factors and rotation. There are two printing options which I don't see listed in the references that I linked above, and these are "pageheight" and "pagewidth". These are X and Y values with specify the size to which the finished plot should be scaled on the page. Since all the parameters are optional, you would specify them by name. E.g. output = self._GrCanvas.postscript(rotate=True)

If you are interested in other SDL libraries there is also one called Pyglet. I'm not familiar with it, so I don't know how it differs from Pygame. It's often called "equivalent to Pygame" however.


Posted by curt wuollet on 7 June, 2010 - 1:53 pm
Thanks Michael

I did find out how to blit text onto the pages. I haven't found any documentation which is curious, but I can copy the examples. It seems there is considerable detail that isn't documented, but passed from hack to hack.

Here's a snippet

# Display some text
font = pygame.font.Font(None, 16)

ttext = font.render("+100", 1, (10, 10, 10))
textpos = ttext.get_rect()
#textpos.centerx = pp[1].get_rect().centerx
#textpos.centery = pp[1].get_rect().centery
textpos.centerx = 20
textpos.centery = 95
pp[1].blit(ttext, textpos)

So what type is ttext? rect?, Font? Lard? (it came from rendering :^)) Haven't seen the docs on the centerx and centery methods. No matter, I've figured out what is going on, but it would be good to know what other methods there might be.

The wife knows I'm programming and therefore oblivious, so I'll have to mow the yard to pretend I'm not obsessed.

Regards
cww


Posted by M Griffin on 7 June, 2010 - 4:37 pm
According to the documentation, "The font module allows for rendering TrueType fonts into a new Surface object." The documentation on "Surface" has a "get_rect" method, along with 3 or 4 dozen others. I don't know if all of those would be meaningful for a font, but it looks like you can do just about anything with a font that you can with any other graphic - the font is just rendered as a bunch of pixels after all.

The pygame documentation is at "http://www.pygame.org/docs/"

The Debian package also installs the docs on your PC when you install python-pygame. Your RPM may have done the same.


Posted by curt wuollet on 8 June, 2010 - 2:15 am
I finally came across a comment on the docs that mentioned the same problem with finding all the stuff you can do.

python
>>>> import pygame
>>>> dir(pygame.Rect)

lists a whole bunch of stuff not necessarily mentioned in the docs. Probably someplace but I haven't found it, yet.

Regards
cww


Posted by curt wuollet on 5 June, 2010 - 10:20 pm
Here's what I cooked up with pygame today.

I'll have to send the .pngs to anyone who wants them. I don't think the list supports them.

> import pygame, sys,os,math
> from pygame.locals import *
>
> pygame.init()
>
> window = pygame.display.set_mode((700,1000 ))
> pygame.display.set_caption('Chart Recorder')
> background = pygame.image.load("gpb.png")
> cleanp = pygame.image.load("gp.png")
> paper1 = pygame.Surface((700,1000),0,background)
> paper2 = pygame.Surface((700,1000),0,background)
> pp = [paper1,paper2]
> pp[0] = pygame.Surface.copy(cleanp)
> pp[1] = pygame.Surface.copy(cleanp)
>
> screen = pygame.display.get_surface()
>
> screen.blit(background,(0,0))
> screen.blit(paper1, (0,0))
> pygame.display.flip()
> z = 150
> degree = 0
> while 1 :
> for x in range(700):
> screen.blit(background,(0,0))
> screen.blit(pp[0],(-x,0))
> screen.blit(pp[1],(700-x,0))
> z = (math.sin(math.radians(degree)) *50) + 200
> pygame.draw.line(pp[1],(255,0,0),(x -1,z),(x,z),1)
> pygame.draw.circle(screen,(255,0,0),(700,z),5,0)
> pygame.display.flip()
> degree += 1
> if degree == 360:
> degree = 0
> pp.reverse()
> pp[1] = pygame.Surface.copy(cleanp)
> def input(events):
> for event in events:
> if event.type == QUIT:
> sys.exit(0)
> else:
> print event
>
> while True:
> input(pygame.event.get())

It works and I don't think it will get any slower with the rest of the channels implemented as the heavy lifting is in the blits and flips.

Regards
cww


Posted by M Griffin on 6 June, 2010 - 1:25 pm
curt wuollet said: "I'll have to send the .pngs to anyone who wants them. I don't think the list supports them."

Post them on some place like Tinypic, Imageshack, or some other image hosting service. That's what those places are meant for.

I don't have the image so I haven't run it, but I think you need a time.sleep() in it somewhere to control the pace (unless Pygame has its own equivalent). Also, I assume you want to put the drawing loop into a function and call one iteration at a time from the event input handler.


Posted by curt wuollet on 6 June, 2010 - 5:16 pm
Tes,

I am running it "wide open" so I can see changes in a reasonable amount of time. It seems no one library has all the features to make this easy. Pygame doesn't seem to have any obvious means to write the system time on as labels, or indeed any facility for easily dropping text on the sheets. The big feature for this approach is that the sheets can be any graphic. I created the pngs in Inkscape and could do even log or semi log or even store a chart form for each scale with the markings embedded, which I may do. But I still need to do the time axis and for HVAC that should be wall clock time. Pygame does, logically, have means to read the game port for the poor man's thermistor DAQ. I'm not sure whether the math is up to the parallel linear equations in three variables to linearize thermistors on the fly, but I'm not sure I am anymore either :^). A lookup may be in order. One very interesting thing is that it's faster in X than SDL with software drawing which is faster than SDL with HARDWARESURFACE drawing. Very strange. I'll mail you the pngs.

Regards,
cww


Posted by Victor Rocha on 5 June, 2010 - 4:36 pm
Hello,

There are a number of open-source solutions, "ready-made" for graphics, so you don't need to hack much around codes. I could mention PHPlot, GNUplot, JPgraph, Jfreechart and much others.

But in my opinion you could handle this task with a "ready-made" Scada or HMI open-source package (like PVbrowser which was already mentioned here).

I'm personally involved in one of these open-source SCADA's which is ScadaBR. Much of ScadaBR is based on Mango M2M by Serotonin software.

With Mango, you can much simply add your own datasources (modbus, sql, http and others supported), create a graphical View and add a timeseries Chart.

The chart will automatically scroll with grids, axis, legends etc. In ScadaBR we are even adding OPC support, based on OpenScada's Utgart libraries.

On the other hand, my personal choice if I were to roll out a quick solution on my own, would be picking an open-source chart library (for example phplot is much easy to use), then just "redraw" it in a couple of seconds.

Ah by the way, maybe you could check this page, www.digitemp.com --- digitemp reads 1-wire Dallas temperature sensors, but the project's author has added a very nice utility that plots the graphs using RRDtool. Worth taking a look. :-)

best regards
Victor


Posted by curt wuollet on 10 June, 2010 - 1:55 am
Well, the guy called and asked how it's going, so I sent a demo. It does what it needs to do as far as presentation is concerned. The project is running into massive cost overruns though. Because his target PC, like my Dell has neglected to include a game port, I was forced to order a PCI sound card with a game port, that being about the only way to get a game port anymore. That cost $5.65 delivered. And I couldn't find my stock of thermistors so I had to spend another $6 and change to buy 40 1% 10k thermistors. So the R&D expenditure is getting close to $12. But the thermistors should be good for 10 systems.

I didn't post the program this time, I will if anyone is following along. Right now, it does the job, but it's not OO and it's not structured. And it's not apparent right at the moment what syntax I need to use for the joystick stuff. I need to study these topics while I wait for the hardware to arrive from the PRC or Hong Kong or wherever. And I'm looking at alternative A/D since the game port seems to have been deprecated. Dataq has some $25 serial units that should work fine but they're over on the dark side and only support Windows. I can use it and RE the protocol, but I will see if I can find more Open minded folks to deal with first. So it's off to the books for a while. Eventually I might generalize it and with Michael's OK for anything I copied, I'll release it GPL.

Regards,
cww


Posted by Ken Emmons Jr. on 10 June, 2010 - 8:30 am
I'd also stay away from the gameport. A lot of guys are using a USB to RS-232 chip in their projects, so you can get into modern era and even laptop computer territory this way (Check out the chips they use on the Arduino project, I think that is how they do USB...) Drivers shouldn't be an issue as those guys support linux.

You should be able to get a RS-232 A/D going with a AVR or PIC for under $25 with a protoboard, etc... I'm sure someone has some kit boards out there for cheap on this front. Of course if you go your own way and don't have a programmer for the AVR/PIC you'll have to dish out $25-$50 for that. If you make a bunch you will need to have some boards made though.

Having said that about finding a project out there for the RS-232 A/D kit, I have been surprised that some of these "Open" projects are charging huge sums of money for their kits and PCBs. I know that is how they make money, but some of these guys are really trying to gouge you on price. I looked into a project to do a cheap digital readout on my machine tools (using those inexpensive chinese caliper scales) and the guy wants something like $50 for just the PCB (it was a small PCB), and I know in even moderate volume those things are only worth $8 or so. By the time I'd be done buying all the components and spending time I'd rather spend the money on glass scales on fleabay and use a real encoder interface with some precision (and maybe some accuracy)! Maybe I should spark my own project, but I have too many at the moment!

Sorry for the digression... :o)

~KEJR


Posted by curt wuollet on 16 June, 2010 - 2:51 am
Well after checking out the Dataq offerings fairly thoroughly, I do have to apologize to them because they do have a Linux SDK. It seems to me to be an extremely complex piece of code for what it does, which isn't much, and it really doesn't do what I want to do as the gadget controls the timing. And the bits are pretty well scrambled. I can't really say I want to support it. So, while waiting for the game port and thermistors, I've investigated the uPs that have A/D built in and require minimal external hardware. Since these are actually small standalone systems, there are a legion of trade offs. I finally came up with one that should do.

http://www.nkcelectronics.com/freeduino-serial-v20-board-kit-arduin o-diecimila-compatib20.html

It has supporting software available for Linux and that other stuff. It's inexpensive, and has 6 10 bit A/D channels. It's a serial (RS232) version so it can be a ways away from the PC, it uses a real 232 chip so it should be more reliable than the kludges and has enough of a following where it should be around for a while. Folks that are so inclined can get a USB version, but I hate dongles and it's a lot easier to make a 232 cable on site if needed, cheaper too. And USB connectors aren't remarkably robust. The software is all OSS and I'm checking on the board, but since the Arduino designs are are CC share and share alike I don't think there are any IP issues. The kit is $17 US., so the whole package can be had for probably $10 a channel allowing for cabling, etc. I'm not real keen on the IDE and language, but it is simple and the whole ball of wax exists already. A no calibration version could be built, but that would require a quad Op Amp and LM35s or 36s as they are millivolt output and the A/Ds are +5V fs. I will have to come up with a protocol, but that will be Open as well. Something like: Send start character, look for ack, receive 8 bytes, ack. The other side will be similar.

Regards,
cww


Posted by M Griffin on 10 June, 2010 - 1:31 pm
Any of the code examples that I have posted for this topic can be used as you see fit without credit or attribution.

As for the Dataq serial module, they have a GPL Linux developer's kit. Just look for their Linux software web page. The download link is in a posting on their web forum, which is the same way they seem to do any of their other no-charge software (perhaps their e-commerce partner wants a commission on downloads through the web store).

You might want to just look at the source code (C++) and figure out the serial protocol. That would let you just integrate the communications directly in the Python program rather than compiling their source code and then calling that. I'm assuming that you can just do a simple poll/response cycle. I would suggest asking some questions on their Linux developer's forum.


Posted by curt wuollet on 10 June, 2010 - 4:23 pm
Thank you Michael,
I have a boiler plate C serial reader that I can use, given the protocol. I've used it for a dozen such things. For $25 they might just blast readings at the host. But, if I go that way, I will probably at least try the Python facilities to make synchronization easy.

And in reply to Ken?, yes, I have thought of using Arduino or the kool Arm6 clone that has 7 A/D channels, but a packaged unit for $25 is more attractive in the short term. Right now, to roll the demo into a product, I need to reorganize and structure it and make it event driven as the clear way to get accurate timing is to set a timer to put events on the queue so the time to update the screen is not a factor, running in parallel as it were. Hopefully, crazy mousing, etc. won't swamp it and affect the timing too much. I don't have to do the generalization as what is wanted is a dedicated box that you start up and it does it's thing, 4 channels, 60 to 80 degrees F, time: minutes per division to hours per division, TBD. I'm sticking with the bitmap paper as the hardcopy is impressive. I'll send the latest hack and screens. You need not burn any time on it as I am already much in your debt.

One humorous note is that it's somewhat slower than TP2 on DOS on a 5 MHz 8088. But it does look nicer :^). If you compiled to a com file 99% less code would be running.

Regards
cww


Posted by curt wuollet on 24 June, 2010 - 1:15 am
Just by way of mentioning how this resolved: I ordered a game card, but it hasn't come yet. So I investigated uPs with analog and serial built in. Ordered a Freeduino serial version 2 for the princely sum of $16. It came, I assembled it, and I have it running providing input to the recorder. Here's the highly complex program in the Arduino Sketch language which is basically a thin veneer over C.

// Program to use Freeduino as a 4 channel analog remote
// unit for the chart recorder. Mostly copied straight from
// the examples

int firstSensor = 0; // first analog sensor
int secondSensor = 0; // second analog sensor
int thirdSensor = 0; // third analog sensor
int fourthSensor = 0; // fourth analog sensor
int inByte = 0; // incoming serial byte

void setup()
{
// start serial port at 9600 bps:
Serial.begin(9600);

}

void loop()
{
// if we get a valid byte, read analog ins:
if (Serial.available() > 0) {
// get incoming byte:
inByte = Serial.read();
// read first analog input
firstSensor = analogRead(0);
// delay 10ms to let the ADC recover:
delay(10);
// read second analog input
secondSensor = analogRead(1);
// delay 10ms to let the ADC recover:
delay(10);
// read third analog input
thirdSensor = analogRead(2);
// delay 10ms to let the ADC recover:
delay(10);
// read fourth analog input
fourthSensor = analogRead(3);
// send sensor values:
Serial.println(firstSensor);
Serial.println(secondSensor);
Serial.println(thirdSensor);
Serial.println(fourthSensor);
}
}

While I'm not crazy about the idea of inventing a new "vanity" language rather than simply using the standards, I do have to admit that it is a very simple program, and it does the job. It lacks error checking, etc. etc. But $16 and 20 lines of code to add 6 10 bit analog channels to a PC, (I'm only using 4) is a pretty good deal. I wrote pyserial code on the other end to grab the readings. I had to stick to the most basic routine as the pyserial code didn't work as expected when I got fancy. That is, I'm sure it works, but will require study to do what I want. I wrote code in C to find the Hart Steinhart coefficients for the thermistor and generate a lookup table but I believe I'll try to write the R to T equation into the recorder rather than do lookup. All I need now for a fully functional version are the 10k 1% series resistors for the thermistors. Still need someplace to put all this stuff up to share.

Regards
cww


Posted by Ken E on 24 June, 2010 - 7:44 am
Curt,

I agree with your distaste of what you call "vanity languages", its how I feel about IEC structured text. I also thought the "sketch" and "wiring" veneer to be laughable, but it does work out well. Really the Arduino setup is C at the core with a more high level library structure. I might give this board you bought a try as I have a AVR development kit, but I hardly ever use it because I don't have any deployment boards and have to stop and think about how to use the environment every time I sit down with it...

KEJR


Posted by curt wuollet on 24 June, 2010 - 5:41 pm
Well, I'm warming up to sketch as it's a thin veneer and not very different from simply having a bunch of tried and true routines in a library. If you look at wiring though, it's pages and pages of code for a simple demo. I think python is a great deal higher level.

Regards
cww


Posted by M Griffin on 24 June, 2010 - 10:21 am
If you want to create a look-up table in Python, use a dictionary. Dictionaries are one of the fundamental data structures in Python and are ideal for this type of thing.


Posted by curt wuollet on 24 June, 2010 - 5:52 pm
Well, it depends on how fast the math is in python. The thing is getting pretty big already importing pygame, pyserial, math, time, etc.
And I'm still struggling with the whole OO thing. And I managed to get the Freeduino in some type of a loop where it ignores the serial port. Might be bricked, but it's only $5 or so to replace the chip. Kinda stalls the flow. I was looking at using the internal pull ups as the source resistor for the thermistors which would be convenient, but must be verbotten.

Regards
cww


Posted by curt wuollet on 5 July, 2010 - 3:24 am
Well, The Chart Recorder Project is winding down. I replaced the ATmega328 chip in the Freeduino and that came back to life. It works so well that I am going to forget the game port version, although I did eventually get the sound/gameport card and it works. I will resurrect that if anyone is interested in really low cost. The four individual graph version is undergoing testing with the requester. I rehashed the program to make a 900 X 700 pixel version that fits on the screen of my ancient T23 Thinkpad. It plots all four with different "ink" on the same "graph paper". That way it would be trivial to expand to all 6 analog inputs. The thinkpad was running Fedora 6, so the new purpose provided a reason to upgrade, which was an ordeal. Well. the upgrade went without a hitch, even the wireless works out of the box. But, all the upgrade options wanted to remake the partitions so, I spent a day backing everything up first. I had a lot of stuff on the TP. I have a question for the python fans: Since Python is interpreted, is it faster to code inline or make say, 4 function calls? There are a lot of statements I repeat 4 times that cry out to be a function, but I've left them that way for clarity. I still don't have a place to put the files up, so if anybody wants the code and bitmaps before I find one, you can help me work out what needs to be done to get it running on a "virgin" Linux install. It was easy on the new Fedora 13 install on the Thinkpad, but pyserial needed hacks. I set the Freeduino up with ordinary phone connectors and boxes so the packaging and cabling come from any hardware or big box store. Total outlay for 6 channels of analog added to the PC of your choice should be less than $30 if you shop well.

Regards
cww


Posted by M Griffin on 5 July, 2010 - 12:02 pm
Not making a function call is always faster than making a function call. Whether or not it's worth the cost depends on how much work is being done.

If you want an Python program to run fast, you don't write it like a C program. A lot of things that you would do with a loop in C can be done with a single instruction in Python. You look at processing lists using list comprehensions or map (list comprehensions are usually faster). You look at what is in the built-in functions or the standard library. Etc. Of course, better algorithms usually trump all other methods.

The key of course is to benchmark it and see if you can find any measurable difference. There's a profiler in the standard library, but for a lot of simple algorithms you can just put it in a loop and measure it with time.time() statements.

If you want to post that part of the program here, I can have a look at it. Optimization is a good subject and no doubt other people would be interested in that aspect of the discussion.


Posted by curt wuollet on 5 July, 2010 - 1:32 pm
Sure I can post it.
The program runs as is on my desktop. To run on my old T23 laptop it wants a readline before the 900 iteration loop to prime things. It returns nothing and times out, but after that everything works. Go figure. I will send the bitmaps for it oob.

****************************************************************

import pygame, sys,os,math,time,serial
from pygame.locals import *

pygame.init()

pygame.display.set_mode((900,700))
# Initialize joysticks
#class Stick:


# def _init_(self):
# pygame.joystick.init()
# self.joystickAxes = {}
# self.joystickHats = {}

# Thermistor Coefficients
C1 = .001030
C2 = .000239
C3 = .000000157

pygame.display.set_caption('Chart Recorder')
screen = pygame.display.get_surface()
bg = pygame.image.load("4in1b.png")
background = bg.convert()
cleanp = pygame.image.load("4in1.png")
paper1 = pygame.Surface((900,700),0,screen)
paper2 = pygame.Surface((900,700),0,screen)
pp = [paper1,paper2]
pp[0] = pygame.Surface.copy(cleanp).convert()
pp[1] = pygame.Surface.copy(cleanp).convert()
scale = pygame.image.load("4in1scale.png")
# Display some text
font = pygame.font.Font(None, 16)

screen.blit(background,(0,0))
screen.blit(paper1, (0,0))
pygame.display.flip()
oz1 = 350
oz2 = 350
oz3 = 350
oz4 = 350
# Setup event queque for timer and quit
pygame.event.set_allowed(None)
pygame.event.set_allowed([14,QUIT])
pygame.event.clear()
# Adjust this time, 96,000 = 24hrs per screen, 4000 = 1hr/screen, Seconds/.9
# Will set with a button etc., eventually.
pygame.time.set_timer(14,500)
step = 5.000/1024
ser = serial.Serial(port=0,baudrate=9600, timeout=20)
# Needed to "prime" laptop. No idea why.
# ser.readline()
while 1:
pp[1] = pygame.Surface.copy(cleanp)
ttext = font.render(time.strftime("%c",time.localtime()), 1, (10, 10, 10))
textpos = ttext.get_rect()
textpos.centerx = pp[1].get_rect().centerx
textpos.centery = 10
pp[1].blit(ttext, textpos)

btext = font.render(time.strftime("%H:%M:%S",time.localtime()), 1, (10, 10, 10))
textpos = btext.get_rect()
textpos.centerx = 25
textpos.centery = 40
pp[1].blit(btext, textpos)
scalepos = scale.get_rect()
scalepos.centerx = 8
scalepos.centery = 349
pp[1].blit(scale,scalepos)
scalepos.centerx = 468
scalepos.centery = 349
pp[1].blit(scale,scalepos)
for x in range(900):
pygame.event.wait()
print ser.write('A')
line1 = ser.readline()
line2 = ser.readline()
line3 = ser.readline()
line4 = ser.readline()
v1 = int(line1) * step
v2 = int(line2) * step
v3 = int(line3) * step
v4 = int(line4) * step
print v1
print v2
print v3
print v4
r1 = v1/((5.00 - v1)/10000)
r2 = v2/((5.00 - v2)/10000)
r3 = v3/((5.00 - v3)/10000)
r4 = v4/((5.00 - v4)/10000)
print r1
print r2
print r3
print r4
# last term is a calibration offset for the thermistor spread

T1 = ((math.pow((C1 + C2 * math.log(r1) + C3 * math.pow(math.log(r1),3)), - 1) - 273.15) * 1.8) + 32 -3
T2 = ((math.pow((C1 + C2 * math.log(r2) + C3 * math.pow(math.log(r2),3)), - 1) - 273.15) * 1.8) + 32 -3
T3 = ((math.pow((C1 + C2 * math.log(r3) + C3 * math.pow(math.log(r3),3)), - 1) - 273.15) * 1.8) + 32
T4 = ((math.pow((C1 + C2 * math.log(r4) + C3 * math.pow(math.log(r4),3)), - 1) - 273.15) * 1.8) + 32 + .18
print T1
print T2
print T3
print T4
screen.blit(background,(0,0))
screen.blit(pp[0],(-x,0))
screen.blit(pp[1],(900-x,0))
z1 = int(-((T1 - 70.0) * 4.0) + 270)
z2 = int(-((T2 - 70.0) * 4.0) + 270)
z3 = int(-((T3 - 70.0) * 4.0) + 270)
z4 = int(-((T4 - 70.0) * 4.0) + 270)
pygame.draw.line(pp[1],(255,0,0),(x -1,oz1),(x,z1),1)
pygame.draw.line(pp[1],(0,255,0),(x -1,oz2),(x,z2),1)
pygame.draw.line(pp[1],(0,0,255),(x -1,oz3),(x,z3),1)
pygame.draw.line(pp[1],(0,0,0),(x -1,oz4),(x,z4),1)
pygame.draw.circle(screen,(255,0,0),(900,int(z1)),5,0)
pygame.draw.circle(screen,(255,0,0),(900,int(z2)),5,0)
pygame.draw.circle(screen,(255,0,0),(900,int(z3)),5,0)
pygame.draw.circle(screen,(255,0,0),(900,int(z4)),5,0)
oz1 = z1
oz2 = z2
oz3 = z3
oz4 = z4
pygame.display.flip()
#if event.type == QUIT:
# sys.exit(0)
ct = time.strftime("%H%M%S",time.localtime())
pygame.image.save(screen,ct)
pp.reverse()

*****************************************************
Efficiency isn't too big a concern with an event interval of 96 seconds, but at .5 seconds it uses some CPU. Even with the inline coding, it's still remarkably compact for what it does. Could get some more speed by converting all bitmaps, but weird things happen.

Regards
cww


Posted by M Griffin on 6 July, 2010 - 11:46 am
I will show the program re-written several different ways. I'll just do the calculations, as I don't have the hardware to run the entire program with live readings. I have added in a dummy serial data class to simulate the readings. I don't know what the data format is (I assume it's just an integer represented as an ASCII string).

First, here's the common section, with various constants and the dummy serial port class.


#################################################

#!/usr/bin/python

import math, time, operator

class serial():
"""This is a dummy class to simulate the serial port.
"""
def __init__(self):
pass
def readline(self):
return '350'
# This substitutes for the serial port.
ser = serial()

# Thermistor Coefficients
C1 = .001030
C2 = .000239
C3 = .000000157

step = 5.000/1024

# This is the number of iterations to run the test.
iterations = 100000

###############################################


Now, I'll show the original code with the graphics calls left out.


##############################################

# In-line instructions.
starttime = time.clock()


for x in range(iterations):

line1 = ser.readline()
line2 = ser.readline()
line3 = ser.readline()
line4 = ser.readline()

v1 = int(line1) * step
v2 = int(line2) * step
v3 = int(line3) * step
v4 = int(line4) * step


r1 = v1/((5.00 - v1)/10000)
r2 = v2/((5.00 - v2)/10000)
r3 = v3/((5.00 - v3)/10000)
r4 = v4/((5.00 - v4)/10000)

# last term is a calibration offset for the thermistor spread

T1 = ((math.pow((C1 + C2 * math.log(r1) + C3 * math.pow(math.log(r1),3)), - 1) - 273.15) * 1.8) + 32 -3
T2 = ((math.pow((C1 + C2 * math.log(r2) + C3 * math.pow(math.log(r2),3)), - 1) - 273.15) * 1.8) + 32 -3
T3 = ((math.pow((C1 + C2 * math.log(r3) + C3 * math.pow(math.log(r3),3)), - 1) - 273.15) * 1.8) + 32
T4 = ((math.pow((C1 + C2 * math.log(r4) + C3 * math.pow(math.log(r4),3)), - 1) - 273.15) * 1.8) + 32 + .18


z1 = int(-((T1 - 70.0) * 4.0) + 270)
z2 = int(-((T2 - 70.0) * 4.0) + 270)
z3 = int(-((T3 - 70.0) * 4.0) + 270)
z4 = int(-((T4 - 70.0) * 4.0) + 270)


oz1 = z1
oz2 = z2
oz3 = z3
oz4 = z4


print 'Elapsed time for in-line is ', time.clock() - starttime

print z1, z2, z3, z4



###########################################

Now here's a version using map. Map is a built in function that operates on an entire "list" of data. It takes one or more lists as inputs, applies the specified logic to each one, and outputs a list as a result. A list is like an array (except the list elements don't have to be all of the same type).

You will notice the use of the "lambda" key word. That is a way of specifying a simple anonymous function. That is, it's like creating a function in place that doesn't have a name. For example:

vdata = map(lambda x: int(x) * step, linedata)

This takes the data in the "linedata" list (which has 4 elements), and feeds it one at a time to "lambda x: int(x) * step". This is equivalent to doing something like this:


def calcv(x, step):
return int(x) * step


You will also notice that I imported the "operator" module. This provides math operators as functions. That lets me do this:


Tdata = map(operator.add, T1data, caloffsets)


That takes 2 lists, passes one pair of numbers at a time to "add", which then adds them together.

Lambda can handle multiple lists at once as well (e.g. "lambda x,y: x+y"), so I could have combined this with the previous operation. However, I wanted to make this more directly comparable with the next example.


##########################################



# Using map.
starttime = time.clock()

# last term is a calibration offset for the thermistor spread
caloffsets = [32 -3, 32 -3, 32, 32 + .18]
numreadings = len(caloffsets)

for x in range(iterations):

linedata = map(lambda x: ser.readline(), range(numreadings))

vdata = map(lambda x: int(x) * step, linedata)
rdata = map(lambda x: x/((5.00 - x)/10000.0), vdata)

T1data = map(lambda x: ((math.pow((C1 + C2 * math.log(x) + C3 * math.pow(math.log(x),3)), - 1) - 273.15) * 1.8), rdata)
Tdata = map(operator.add, T1data, caloffsets)

zdata = map(lambda x: int(-((x - 70.0) * 4.0) + 270), Tdata)

prevdata = zdata


print 'Elapsed time for mapped version is ', time.clock() - starttime

print zdata



############################################

Here's a version using list comprehensions. A list comprehension is like map, but uses a different syntax. The syntax is based on mathematical set notation. It is more flexible than map in some ways, but it won't handle two lists without a lot of faffing about, so I used map for one step.

############################################



# Using list comprehensions.
starttime = time.clock()
# last term is a calibration offset for the thermistor spread
caloffsets = [32 -3, 32 -3, 32, 32 + .18]
numreadings = len(caloffsets)

for x in range(iterations):

linedata = [ser.readline() for x in range(numreadings)]

vdata = [int(x) * step for x in linedata]
rdata = [x/((5.00 - x)/10000.0) for x in vdata]

T1data = [((math.pow((C1 + C2 * math.log(x) + C3 * math.pow(math.log(x),3)), - 1) - 273.15) * 1.8) for x in rdata]
Tdata = map(operator.add, T1data, caloffsets)

zdata = [int(-((x - 70.0) * 4.0) + 270) for x in Tdata]

prevdata = zdata


print 'Elapsed time for list comprehension version is ', time.clock() - starttime

print zdata


##############################################

Here's a version using list comprehensions, but with the equations altered a bit to avoid redundant operations.

##############################################



# Using list comprehensions and optimising calculations.
starttime = time.clock()
# last term is a calibration offset for the thermistor spread
caloffsets = [32 -3, 32 -3, 32, 32 + .18]
numreadings = len(caloffsets)

for x in range(iterations):

linedata = [ser.readline() for x in range(numreadings)]

vdata = [float(x) * step for x in linedata]
rdata = [x/((5.00 - x)/10000.0) for x in vdata]

rdatalog = map(math.log, rdata)
T1data = [((math.pow((C1 + C2 * x + C3 * math.pow(x, 3.0)), - 1.0) - 273.15) * 1.8) for x in rdatalog]
Tdata = map(operator.add, T1data, caloffsets)

zdata = [int(-((x - 70.0) * 4.0) + 270) for x in Tdata]

prevdata = zdata


print 'Elapsed time for optimised list comprehension version is ', time.clock() - starttime

print zdata



############################################

The alternate methods are a lot briefer, but what about speed? I ran these in a loop (see the "iterations" variable near the beginning) and measured the time for each using time.clock(). Here are some typical results (the numbers will vary between consecutive runs).

In-line: 3.63 seconds.
map: 4.3 seconds.
list comprehensions: 3.77 seconds.
optimised: 3.21 seconds.

Map was 18% slower.
List comprehension was 4% slower
The optimised version was 12% faster.

For the optimised version I did 2 things. One was in the first equation it converted the value to a float instead of an int. I'm not sure if that is correct for the application, but it avoids unnecessary type conversions later. I also changed several constants from integers to floats for the same reason.

I also split up the main equation so that the "log" function is calculated separately. This avoids calculating it twice.

List comprehensions are usually faster than map, especially if you are using lambda with map. A lambda is like making a function call (it has to set up a stack frame), so there is extra overhead. However, it isn't that much slower here because so much of the real work is in the math equations themselves.

When you use map with a built-in function or with a library like "operator" however, then map can be very fast. So, there's no simple rule of thumb for this.

The final results are in a list called "zdata" (I tried to keep the names close to your originals). If you want to access individual elements of zdata, you can index it as zdata[0], zdata[1], zdata[2], etc.

The end result of this is you are expressing each equation only once, and it is probably much more compact than using function calls (because is processes all 4 calls in one line).

If you want to take more readings, the calculations automatically accommodate them.

I have put the serial port read operation in a map or list comprehension. That looks a bit "odd" but it will probably work. If you want 4 separate calls you can do this and still have the data in a list:


# Create an empty list.
linedata = []
# Add the readings.
linedata.append(ser.readline())
linedata.append(ser.readline())
linedata.append(ser.readline())
linedata.append(ser.readline())


You can also do this:


linedata = [0, 0, 0, 0]
linedata[0] = ser.readline()
linedata[1] = ser.readline()
linedata[2] = ser.readline()
linedata[3] = ser.readline()




Posted by curt wuollet on 15 July, 2010 - 3:18 am
Hi Michael

I looked at these features with great interest, because I believe you code best when you capture the spirit of the language you are using. That's why I prefer the old pre ansi C (K&R) style and I'm trying to catch the spirit of Python. These constructs do look a little bit forced for this program, but would be more useful for a larger, more complex dataset. I did do the float types and math deconstruction and generally cleaned things up. The employment situation had me tense the other night and I couldn't sleep so I threw myself into a more challenging project. I now have a virtual circular chart recorder which stretched my memory of trigonometry almost to the edge. After a couple of approaches that were foiled by the lossy rotate transform, it works pretty good, needing only to copy over the serial code to be functional. Mechanical Cir. chart recorders are an anachronistic hack that has had surprising staying power mostly for limit auditing and the like. It too is surprisingly small and I can post it if there is interest. Coding and problem solving till the sun comes up do let you sleep the sleep of the exhausted. It's amazing what you can do with a page of Python, but it was a slow page and will need comments for anyone to see that there is method to the madness. Issues with blitting square pegs onto round holes and the like :^)

Regards
cww


Posted by M Griffin on 15 July, 2010 - 11:55 am
I had a look at that again, and the "math.pow(x, 3.0)" should have been "math.pow(x, 3)" like in the original. I should have been paying more attention when adding decimal points. Also, the thermistor offsets should have been real numbers as well in order to avoid a type conversion.

If I was going to do this however, I would probably calculate a look-up table on start up and put it in a dictionary (hash table). That would be about as fast as you can get.

If you have an application that needs to do a lot of number crunching (this example is pretty simple by comparison), then the normal solution is to use "numpy" or "scipy" (the two are closely related). That's a third party FOSS Python library that has everything - vector math, FFTs, digital filters, matrix math, etc. It will also take complex equations and compile them to run in the library outside of Python. The libraries themselves are written in C and Fortran.

Numpy/Scipy are a wrapper around the same standard scientific and engineering math libraries used by almost everyone. Some of the very expensive proprietary math packages use these exact same open source libraries as the foundation for all their calculations.

For this application, you don't need Numpy/Scipy. If you were writing a production test system that had to do this sort of math however, then that would be the solution. I would definitely pick Python over something like Labview in that type of application, because the language flexibility and overall library support are so much better.


Posted by curt wuollet on 17 July, 2010 - 2:54 am
One of the things about the modular nature of Python is that there are numerous dependencies and the transforms at least, may already use numpy. I seem to recall it being loaded as such when I installed python with yum. Besides the friend that wanted the thing in the first place, I have had use for such a thing in winter when I'm heating with the corn stove and wood stove and kerosene heaters to balance temps and monitor unheated spaces to keep pipes from freezing, etc. I have an old Compaq Armada M700 notebook that I got free at a company sponsored hazardous waste free disposal day. Someone was going to pitch it in with all the tvs and old monitors and I diverted it for just such an occasion. It runs Fedora 13 well so now all my mobile computers are up to date. It can be a semi-permanent temp recorder. So another question came to mind. I wonder if I can get someone who has Python set up on Windows to test if the program will run there as well. Not an imperative, but interesting to know. Since the Arduino stuff works on Windows, if the Python part does as well, I made something that works cross platform with zero effort. The only catch is that to really test it, you would need an Arduino as the serial part is one of the bigger questions. It serves my purposes now, but if it would be of use to others, I might be able to spend the effort to make it release grade SW with selectable times, scales, etc. While I wait for the funding to proceed with the PLC project, I could put it up on the Sourceforge site before I lose interest and move on.

Regards
cww


Posted by M Griffin on 17 July, 2010 - 1:10 pm
If you're using Fedora, then I think that Python should have come already installed as part of the base system. I'm not sure how they do their packages (I'm more familiar with debs), but if it pulled in numpy then they have some additional package that pulls in extras that they thought you might want. If you installed something with yum, then it wasn't the base Python install because yum is a Python program (so it already had to be there for yum to run). I won't swear that Pygame doesn't use numpy, but I don't know what for (unless it uses it for some sort of complex game physics calculations).

All of the stuff that I posted was from the base language or standard libraries. That is 100% cross platform. Pyserial is cross platform, and I have used it to write and test software on Linux and then deploy it without any changes on MS Windows in the factory. The only "problem" that I had is that serial communications on MS Windows (using the standard Microsoft drivers) has lower performance than serial communications on Linux. That's not a practical problem however unless you need to push the serial port to the absolute limit. I don't know about Pygame, but I suspect that won't be a problem either.

Python is very good at being cross platform. Anything platform specific is usually very clearly marked as such. Python even handles things like using the correct line endings when it writes text files. You have to watch out for things like the Windows file system is not case sensitive, but that's not something that is happening outside the language itself. You're probably pretty safe publishing the program as is and saying it will probably work on MS Windows.

To be completely sure however, you have to test a complete system (including an Arduino). The biggest problems that I've had with MS Windows is that the serial and network I/O systems are just not as robust as the ones in Linux. I once had to re-design a program from the ground up because the networking system in MS Windows couldn't handle the load. I didn't find that out until final testing when I could put the complete system under full load for an extended period of time.

I've also seen MS Windows (Server 2008 and Windows 7) simply fall over and die under extended periods of heavy load. First it will kill off the application, and then the Windows subsystems themselves would one by one go to 100% CPU usage and then crash themselves until nothing was left except a blank screen (grey, not blue however). That's not something you will find however except by actual testing using your specific conditions, and those are "normal" Windows problems (not something specific to your application).

Despite the above "war stories", I don't think you'll have a problem in your application.

Your use of this site is subject to the terms and conditions set forth under Legal Notices and the Privacy Policy. Please read those terms and conditions carefully. Subject to the rights expressly reserved to others under Legal Notices, the content of this site and the compilation thereof is © 1999-2014 Nerds in Control, LLC. All rights reserved.

Users of this site are benefiting from open source technologies, including PHP, MySQL and Apache. Be happy.


Fortune
Don't hit a man when he's down -- kick him; it's easier.
Advertise here
advertisements
View free setup and multi-vendor EtherCAT demo videos online
Time to incorporate data handling, web HMI and motion in one system!
Servo, stepping motor control, analog & web HMI in one system!
Servo, steppers, analog, digital & web HMI - Fully Integrated!
Control.com is the largest Automation community on the web. Learn how to advertise here now...
our advertisers
Help keep our servers running...
Patronize our advertisers!
Visit our Post Archive