History of timedata.

timedata was originally an offshoot of the echomesh project.

I have long been interested in controlling lights - and in order to do that, you need to be able to do a lot of color arithmetic very fast - and this intersects my interest in fast DSP.

Python, my go-to language for rapid development with hardware, simply does not provide anywhere near enough raw speed to do even quotidian tasks like fades or scrolls at a reasonable framerate, particularly on small platforms like the Raspberry Pi.

Part of the echomesh library had fast color arithmetic in C++ and Cython - but it was all hooked up with the Juce library and many other bells and whistles.

Moreover, since I'd first written this, I had learned a very great deal about modern C++, and had thought somewhat more about color - so I was itching to put my idea into practice.

timedata's goals.

The project's goals were four-fold: to be able to represent many disparate color models; to provide services in C++ and Python idiomatically; to cover Linux, OS/X and Windows; and performance.

On performance.

Writing for performance from the start can be a bad idea because it results in a lot of micro-optimizations of dubious value which obscure the code. In this case I decided to try to hit two large goals that I found aesthetically appealing - "no memory management when rendering" and "no cost overheads for color types" - and one "guiding principle", that being "locality of reference".

"No memory management when rendering."

Very broadly speaking, if you are trying to control in real time a system like audio, video or lighting with your program, there are two phases - set up, where you perform somewhat costly one-time operations, and rendering, which is called repeatedly at a very high speed and thus must be aggressively optimized.

So "no memory management when rendering" means that it must be possible to perform an entire rendering phase and never create a new object (in Python) or call new or delete (in C++) - that is, as long as you are willing to do a little set up in advance of course.

"No runtime cost overhead for color types."

In C++-world, the key step to achieving this is that lists of colors are represented generically by long contiguous blocks of floats or integers in memory, while a color is a zero-overhead abstraction that has no extra cost over "three numbers red, green, blue".

In C++, there's a lot of information associated with a color - not just its model but also the range of numbers expected and even their type - but that information is stored in the type at compilation and costs nothing at run-time.

Interoperating between colors and colorlists of the same type costs nothing - interoperating with disparate models or ranges has an unavoidable conversion cost but kept as cheap as possible.

To achieve this as best as possible in Python took considerably more work, and required a template system to automatically generate a large number of Cython files from small parts.

Where is it today?

To cut a long story short, we hit the targets described above. We did in fact create the generic C++ system fairly neatly, and a templating system for Cython code that isn't too too horrible and lets us automatically crank out as many as 36 color variants corresponding to 18 generic C++ classes.

Adding a new color model is a matter of a couple of pages of code specific to that model and very little fooling around at all. Other models don't have to know about the new model and existing code pays no cost for new color models or new numeric ranges.

In Python, we have a wide range of useful list-like, color-like and permutational operations on each Color and ColorList type. Because we're automatically generating these operations, we can offer each in three flavors:

* functional, which returns a new value: color_list.add(0.1)
* into, which changes the existing value: color_list.add_into(0.1)
* to, which performs the operation and stores the values elsewhere: color_list.add_to(0.1, result)

This is very convenient in practice, because you can do convenient experimentation with the functional form, and then switch to the into or onto forms to avoid memory management when rendering.

It works on three platforms and we're working today to interface it with a groundbreaking new LED platform from hardware designers ManiacalLabs.