Lessons learnt DMX over IP.. Part(1) there is standards / there is no standards

[dirknerkle]

I've reread all the posts to here and from my understanding, all I can say is that it's pretty obvious that the cam rakes were all out of proportion, and the two-tone shifter axle was out of phase with the global intake scoop, thereby convoluting the remote trimuster bisquit applicator when the voltpile overly magnified the transglormer.



(Is it obvious that all of this is wayyyyyy over my head???)

[jstjohnz]

I would tend to agree with natf. If the controller's design is such that the processor has to examine every multicast packet on the network, then there's going to be a saturation point where the total number of packets exceeds the processor's ability to inspect them. Beyond that point packets are going to get randomly dropped.

If that is the problem then the possible solutions are to segment the network in such a way that no controller sees more packets than it can process, or use gmp-aware switches so that only subscribed-to packets are sent to each port, or go to unicast.

But it brings up a point that one specification that's rarely mentioned is how many universes can be present on the LAN before a controller chokes.

[raplass]

I'm thinking I'm going to write a few things about what i've learned over the next few weeks...

WOW! Talk about foresight, I have enough trouble learning from past mistakes, much less future ones...

And just in case anyone doesn't know the classic engineering terminology Dirk is referring to:
http://www.youtube.com/watch?v=RXJKdh1KZ0w

[mrpackethead]

wow, a few brain cells used here.. We are getting close to understanding the problem.

My guess would be your packet rate (not the bandwidth) was way too high and the switches choked - further guessing streaming multicast made this much worse.

The switches were more than able to handle all the packets without dropping or mangling any of them.

edit - or... more likely since it IS multicast, every controller would receive every packet and process them and they couldn't handle it. If you switched to unicast it would help but not all controllers support that. I know Ed's j1sys stuff supports it but it's not commonly used for E1.31...

That is correct, every controller saw every packet. Swtiching to unicasted E1.31 was not an option as the media server did not support unicasted E1.31. The controllers do however support it, though thats a side issue. The issues occurring certainly seemed to be in proportion to the total volume of traffic


Possibly the multicast implementations were using some sort of hashing scheme to pre-filter the incoming data, perhaps one or other of the hash tables was overflowing (or getting too many hash collisions), resulting in greatly increased processing time.


Deep. However multicast hash tables were not implemented at this time.


I would tend to agree with natf. If the controller's design is such that the processor has to examine every multicast packet on the network, then there's going to be a saturation point where the total number of packets exceeds the processor's ability to inspect them. Beyond that point packets are going to get randomly dropped.

Ok, we have partial winner ( only partial ).. The controller is seeing every packet, and its having to discard many of them, and it does get to a point ( it turns out it about 45 universes ) where it can't process any more, and stuff starts going to bad. However the controller was never intended to receive this many universes!


If that is the problem then the possible solutions are to
segment the network in such a way that no controller sees more packets than it can process,

While thats just possible, it would require some pretty complex setup in terms of source routing multicast, and PIM.. Not something i'd attempt. Or you could run split controllers.


or use gmp-aware switches so that only subscribed-to packets are sent to each port,


This would have been the ideal solution. However, the controllers didn't support IGMP. ( note: a software update (4.0.0 scheduled for June ) ) will include a IGMP client ). [/i]


or go to unicast.

[i] yes, going unicast was the only option left but unfortunately the media server did'nt support E1.31 in unicast.. However it did support Art-net in unicast. Forutnatly the controllers support art-net but it was a software re-image.


So, art-net won the day in this battle. but in my next lesson, ( which is about 'scaling' ) we'll see that the unicast thing only goes so far...

[P. Short]

So the controllers effectively had to do just as much work as if all the packets were broadcast. Not very sensible, but this doesn't seem have much to do with whether or not the controller follow the standards.

[mrpackethead]

So the controllers effectively had to do just as much work as if all the packets were broadcast. Not very sensible, but this doesn't seem have much to do with whether or not the controller follow the standards.

Yes, the controllers did have to discard all the packets that they didn't need. Not ideal. But the controller was *not* doing something that it should have according to the standard. It should have been replying to IGMP querys. And the switch should have been snooping on those and building multicast forwarding tables based on that...

So, both claimed to be E1.31 talking.. which was right, but in this case the controllers had to do more than just talk E1.31

[michaelc]

Darn, My money was on the Flux Capacitor not being completely charged up. I was going to suggest adding more garbage into the intakes!

[jstjohnz]

Yes, the controllers did have to discard all the packets that they didn't need. Not ideal. But the controller was *not* doing something that it should have according to the standard. It should have been replying to IGMP querys. And the switch should have been snooping on those and building multicast forwarding tables based on that...

So, both claimed to be E1.31 talking.. which was right, but in this case the controllers had to do more than just talk E1.31

That's assuming the switches handle gmp, most cheap ones don't.

[mrpackethead]

Yes, some low cost switches won't snoop the igmp ( internet group management protocol ) conversations..

So, i can hear most of you saying, hey why not just use unicast......

[jstjohnz]

Unicast gets about one sentence of discussion in the e1.31 standard. I personally like multicast a lot as it's generally plug and play. As long as the universe numbers match, everything (generally) talks.

The issue that you were seeing though can be a factor, and it's something that's generally not spec'd. Every E1.31 controller will tell you how many universes it can control, but generally will not tell you if there is a limitation as to how many universes of data can be present on the LAN. For 99% of users it won't be an issue.

When I was doing some initial testing with the WIZ5100 chip, I found that I could do 6 universes of E1.31 by running the chip in raw mode and letting the cpu handle the packet inspection. But instead I decided to go with one universe per hardware socket. This limited me to 4 universes, but since the packet inspection is all done in hardware rather than having to be handled by the CPU, the presence of large numbers of packets on the LAN doesn't cause a problem. The WIZ5200 should be good for 8 universes. Everything is a tradeoff.