Lessons learnt DMX over IP.. Part(1) there is standards / there is no standards

[mrpackethead]

Somebody asked me a question the other day that prompted me to think about some of the issues that i've had in getting DMX over IP to run. While the lessons learned come from larger commercial installations, i think they are still well worth considering in any display. You can "get away" with things when they are small, but those decisions might come back to haunt you at a later date. I'm thinking I'm going to write a few things about what i've learned over the next few weeks...


Theres two main protocols that are used for transporting DMX over an IP network; Art-net ( and that comes in three versions, v1, v2, and most recently v3 ) and Streaming ACN otherwise known as E1.31. At a most basic level both of them take a "universe" of channels ( thats a block of 512 channels ) wrap them up in a data packet and transmit them onto a IP network. The specifications for the 'standards' are published.. The art-net standard is free, the e1.31 standard you need to pay for.

So the first thing i've learned is that while there is standards, theres not that many devices / systems that really adhere to the 'standards' to the letter and or fully. This isn't just in DIY style hardware and software, this also in commercial systems as well ( and some of them cost > $100k !! ). I must add that the controllers that i've developed and sell also are not 100% compliant with the standards, but we are working on it.

The lack of adhering strictly to standards is done for a bunch of reasons. Sometimes its clearly just bad design, but more often its a compromise that has been made to make something work and it was a deliberate choice.

Lesson #1. Don't assume because your media server ( vixen, lsp, Grandma, madrix or whatever ) which says it runs a 'standard' will interconnect and run flawlessly on a network with your end devices and controllers, when they also say they will use a 'standard'. Either test it all out yourself, or take the easy path and learn from someone else's experience. This testing needs to be at the scale you intend to run it at.

My lesson 1a is to try to design into the stuff i'm doing 100% compliance to the standards. Though thats really tough.

I've love to hear your stores about when stuff hasn't worked ( or worked ) the way you expected.. I hope that we can all learn some good stuff.

[CaptKirk]

"There are lies, damned lies, and then there are 'standards'!"

[dirknerkle]

I know precious little about DMX in general and even less about Art-Net or E1.31 so I'm not one to spout off here too much. But I know that designing a product requires a bit of a balance between performance and cost.

In empirical situations, 100% compliance with a published standard is most certainly an admirable goal, but in today's world of practical economics, it's quite a temptation to shave a bit of cost here and there with a part that's perhaps not the best quality, or one that has a narrower operating temperature window, or sometimes we find that leaving a part out altogether (even though it's suggested that it be there) seems to make little or no difference in the product's performance.

For example, perhaps the addition of a decoupling cap doesn't seem to make the chip work better or worse. The technician would say "Put it in there, it'll make it more stable" while the accounting department says "if it doesn't seem to hamper performance, take it out and save the money on both the board and the part" while the marketing guy says "the customer won't notice -- they'll replace this stuff in a year anyway with the next generation model."

Cases in point: do your lights blink better with 18 gauge than the 16 gauge extension cords? Does your controller work better with diagnostic LEDs or without them? Do you use the taller 5mm terminal blocks on your controllers or the shorter ones, and do you notice a performance difference between the two? Do the strings of mini incan lights you buy at $1.99 work better than the ones you buy at $1.79?

My point is that while there are standards in everything we do, we likely tend to modify our application of those standards by practical means, lest we be guilty of overengineering.

In my case, I had some Ren-W's that I used one winter that used the MAX232N chip, of which I had bought several dozen at dirt-cheap prices. They worked fine until the temperature dropped below zero, which was quite a bit below the lowest operating temperature for the N part. Switching to MAX232IN chips (good down to -40C/-40F) was an instant solution. Lesson learned: always check the operating temperature for designing gear that may be used in extreme conditions.

[jstjohnz]

A couple of comments. Sometimes portions of the standard may not make sense for a particular application. An example that comes to mind is arbitrating bwtween 2 or more simultaneous senders. The default is supposed to be "HTP" or Highest Takes Precedence. In other words if 2 senders are sending different dim values to the same channel, you go with the highest value. While that may make sense for individual single-channel light fixtures. it doesn't for pixels. If one sender is trying to light a pixel RED while another is trying to light the same pixel GREEN, and you use HTP you wind up with a YELLOW pixel.

There are other examples, for example in multicast E1.31, the universe numper appeares twice in every packet, in the header and in the packet itself. The standard says that if the 2 values differ, use the one in the packet. Why? If the 2 values differ, something's screwed up, and whichever way you go you've got a 50% chance of being right.

Also ETC has added their 'CC' extensions to the SACN standard that allow a sender to take control of a portion of a universe. Well, there's a lot of overhead involved in implementing that feature because there's no theoretical limit to the number of possible simultaneous senders.

Anyway, the good news is, for the most part, if you're dealing with a single sender, which is the case for most of the installations we deal with here, everything generally talks together pretty well.

[Zeph]

The hard part in a well established ecosystem of connecting products, is being interoperable with other devices which bend or break the standards. This comes up in nearly every sphere of technology I've dealt with in any depth. Sometimes a large part of the complexity of a long-established piece of code is all the workarounds they put into it over the years to compensate for the quirks they've run into (and pray they documented them!).

The lighting world appears to have loose standards, for better or worse. Formal certification, or even attending plugfests, isn't generally required from what I can tell. So the kind of sensible voluntary attempts to adhere to standards as often as possible described above seems like a good balance. Some features just aren't needed so they get simplified; but it's good to avoid pointless deviations and to have a really good reason for departing from the norms.

[mrpackethead]

The lighting world appears to have loose standards, for better or worse.

And that really summarizes why i've made Lesson #1, "don't assume its going work". I' was really hoping to hear some storys about when it didn't work ( due to standards 'complacence' issues ) and how you worked around it.

[Henedce]

In my line of work I have multiple sayings about assumptions or to assume something.
to ASSUME will make an ASS out of U and ME
Assumption is the Mother of all F$#K ups
I drum this in to all new employees. I am yet to see a major screw up that does not contain the words " I ASSUMED"

[mrpackethead]

Ok, this is from a real world actual problem, but i'll word it as a question to see if i can get you to think about it; I'll answer questions in a yes/no/does'nt matter fashion and i'll let you know when someones delivered the answer. You need to tell me the solution and what the problem is and why the solution fixed it.

We have a display that has ~120 universes, delivering to 16 controllers (8 universes per controller). The media server is sending multicast E1.31 ( as per standard ).. The receiverers were listening, but appear to be completely 'lagged' and somewhat unresponsive. We did some trouble shooting and just started sending 1, then 2 and then 4 universes of data and everything worked perfectly.. As we added more and more universes back on, things started to go weird once we had about 28-30 universes

You can assume that the media server was more than powerful enough to send all the universes out without issue, and that the network was not congested, and was dedicated.

[Skunberg]

HEY......Are we trouble shooting your system for you! LOL ;-)


Had to do it. Sorry.

Brian

[mrpackethead]

HEY......Are we trouble shooting your system for you! LOL ;-)
Brian

Yes. Yes you are in fact trouble shooting it. The good news is that back in November we actually did sort it out, but there was some good learning in it, that i hope i can share.