Import Xlights to Vixen
- Thread starter TomsLEDs
- Start date
dirknerkle
Supporting Member
Since the FSEQ format is common to both Vixen and Xlights, wouldn't there be a way to import from that common format?
AT one time we looked at this Dirknerkle. The fact that the FSEQ format does not contain any information as to what prop it is sending data to (just a pile of output sent enmass), it became very clear that it would not work in its raw FSEQ form. It is going to take someone to write a conversion routine from the XSQ format (xLights) before it is compiled/rendered in coordination of the XML files that contain the setup information to the appropriate forms used in Vixen.
Another item that was brought up is the methods that both Vixen and xLights use to "do things". Some very important methodology of operation between the two that is not compatible. (Neither one is wrong, just different.)
As you can see, there was no need to continue with this. FYI - This is also why we had to make a choice between xLights and Vixen in our display. Loved both of the sequencers, each with their own pluses, but for keeping things simple here we chose xLights.
Now - if things have changed "behind the scenes", I would love to participate in making a conversion routine. At least I know how to make coffee and write things down. (Beware, I don't drink coffee.)
Another item that was brought up is the methods that both Vixen and xLights use to "do things". Some very important methodology of operation between the two that is not compatible. (Neither one is wrong, just different.)
As you can see, there was no need to continue with this. FYI - This is also why we had to make a choice between xLights and Vixen in our display. Loved both of the sequencers, each with their own pluses, but for keeping things simple here we chose xLights.
Now - if things have changed "behind the scenes", I would love to participate in making a conversion routine. At least I know how to make coffee and write things down. (Beware, I don't drink coffee.)
dirknerkle
Supporting Member
The data in an FSEQ file must follow some kind of order, otherwise it wouldn't work. I understand that there's patching going on which is what (I think) either links the data to the controller in live-time or creates some sort of pseudo-channel-order. In a way, it seems that the patching becomes a sort of "translator." Hmmmm....
I coded all sorts of interfaces to various other accounting, payment processing and point-of-sale systems as part of my work with my software company, and each was different. Usually, to decode the data I had to load multiple configuration files into memory (or create temporary databases onto which I'd blast a bunch of queries) and all the interfaces resulted in a transaction based record which finally had all the information I needed to write a workable record into our accounting system. Some of the interfaces were quite complex and I was told multiple times that making an interface for their product was just too complicated to be feasible (Hah... a challenge!) and sometimes it required iterations of reads to glean and clean the source data but in the end, the resulting interfaces were efficient and error-free -- as long as their source data was properly formatted.
I'm sitting here listening to Taylor Swift's Eras Tour video with Mrs. Dirk (our Saturday movie night) and wondering if I should see if I try to remember all the screwy little tricks I used when I wrote those interfaces and give this a go.... but..... I dunno.... I like retirement.... a lot.....
I coded all sorts of interfaces to various other accounting, payment processing and point-of-sale systems as part of my work with my software company, and each was different. Usually, to decode the data I had to load multiple configuration files into memory (or create temporary databases onto which I'd blast a bunch of queries) and all the interfaces resulted in a transaction based record which finally had all the information I needed to write a workable record into our accounting system. Some of the interfaces were quite complex and I was told multiple times that making an interface for their product was just too complicated to be feasible (Hah... a challenge!) and sometimes it required iterations of reads to glean and clean the source data but in the end, the resulting interfaces were efficient and error-free -- as long as their source data was properly formatted.
I'm sitting here listening to Taylor Swift's Eras Tour video with Mrs. Dirk (our Saturday movie night) and wondering if I should see if I try to remember all the screwy little tricks I used when I wrote those interfaces and give this a go.... but..... I dunno.... I like retirement.... a lot.....
Hockeyhunter1956
Supporting Member
AAh yes .......... retirement ............ it is good. But then again ....... doesn't your brain need a good challenge every once in a while to keep you young? I know mine does. What is lost if you would be unsuccessful at this venture?
dirknerkle
Supporting Member
What is lost? Time. None of the men on my side of the family have lived past 81. That could mean that I've got only 6 years left. There's a lot of other stuff I want to do. 
Dang it Dave. Now you have me looking at this again. Here is what I'm working with at this time... (at least this is what appears to me at this time)
(For most of us, the following is already known. I just put it down here so that we are both on the same page. (hopefully)).
The .fseq file is a pile of data that is sent out in a serial stream. To the best of my knowledge (looking through the output) there is no designation as to which prop gets which, just a pile that is sent out and the controller receives, putting the data on the ouputs per the predefined (and setup) channels. (e.g., Output channel 1 gets bits 1 through 126, Output channel 2 gets bits 127 through 219, etc.) The controller only knows what goes out on which output due to pre-defined setup. The controller doesn't even know what is on what channel (other than holding a human-readable note given during setup). It just takes the bits and sprays them down one output, then the next, then the next... and then starts again with the next stream. No actions, such as dim or brighten, is in the stream as the stream only contains a single frame of data. With each frame piling on after the other.
The last time that the data was distinguishable as to what prop is what, is during the rendering phase. At that time the list of props for a particular controller is known so that the renderer can grab the information needed to send to the controller, gathered and put in order, and then the result is recorded as the .fseq. (note - I would love to take a class on how the .fseq file is built, on both sides. I'm also retired, so I might have the time...)
I believe, but have not verified yet, that the only way to make a conversion from xLights to Vixen, we will need to gather the information from the sequencer, just like the rendering process does, and build the new files. A complete knowledge of the file structures in xLights and in Vixen will need to be known. Along with the file structure will be the "how to" of applying all the settings available for each prop, the timings, and anything else that goes into making those lights blink. The equivalent function in the "other" sequencer will also need to be known.
It is understood that there are quirks on both sides of this equation. The biggest that I can see is the constantly changing of functions (upgrades/updates) on both sides. This throws a big monkey wrench into keeping this working.
I have not had my breakfast yet, so the above may be askew. Leave it to say that the best of the best for each of the programs need to get together and pass information back and forth and come to a common ground. As they are all volunteers, this would be asking even more from a great group of folks. I, for one, don't see this happening.
BTW - I would like to thank all the programmers, contributors, and additional thinkers that have put out both products. I've shown my sequencing to other "retired IT folks" and they are in awe. The standard question is, "this costs HOW MUCH?" or "this can't be FREE! How much did it really cost?" I tend to agree with them. Unbelievable programs - at any price.
(For most of us, the following is already known. I just put it down here so that we are both on the same page. (hopefully)).
The .fseq file is a pile of data that is sent out in a serial stream. To the best of my knowledge (looking through the output) there is no designation as to which prop gets which, just a pile that is sent out and the controller receives, putting the data on the ouputs per the predefined (and setup) channels. (e.g., Output channel 1 gets bits 1 through 126, Output channel 2 gets bits 127 through 219, etc.) The controller only knows what goes out on which output due to pre-defined setup. The controller doesn't even know what is on what channel (other than holding a human-readable note given during setup). It just takes the bits and sprays them down one output, then the next, then the next... and then starts again with the next stream. No actions, such as dim or brighten, is in the stream as the stream only contains a single frame of data. With each frame piling on after the other.
The last time that the data was distinguishable as to what prop is what, is during the rendering phase. At that time the list of props for a particular controller is known so that the renderer can grab the information needed to send to the controller, gathered and put in order, and then the result is recorded as the .fseq. (note - I would love to take a class on how the .fseq file is built, on both sides. I'm also retired, so I might have the time...)
I believe, but have not verified yet, that the only way to make a conversion from xLights to Vixen, we will need to gather the information from the sequencer, just like the rendering process does, and build the new files. A complete knowledge of the file structures in xLights and in Vixen will need to be known. Along with the file structure will be the "how to" of applying all the settings available for each prop, the timings, and anything else that goes into making those lights blink. The equivalent function in the "other" sequencer will also need to be known.
It is understood that there are quirks on both sides of this equation. The biggest that I can see is the constantly changing of functions (upgrades/updates) on both sides. This throws a big monkey wrench into keeping this working.
I have not had my breakfast yet, so the above may be askew. Leave it to say that the best of the best for each of the programs need to get together and pass information back and forth and come to a common ground. As they are all volunteers, this would be asking even more from a great group of folks. I, for one, don't see this happening.
BTW - I would like to thank all the programmers, contributors, and additional thinkers that have put out both products. I've shown my sequencing to other "retired IT folks" and they are in awe. The standard question is, "this costs HOW MUCH?" or "this can't be FREE! How much did it really cost?" I tend to agree with them. Unbelievable programs - at any price.
TomsLEDs
Supporting Member
Bragging rights is pretty much all the programmers who volunteer so much time and effort to keep Vixen updated and running receive. I thank them from the bottom of my heart! That said, I think there may be a group of us who would be willing to pay for a program (at a modest price) if it would be able to do that. That also raises concerns that now there would be expectations for the program to maintain a certain degree of reliability and function and become a job and not a hobby for the programmers. It's always a 2 edged sword.
dirknerkle
Supporting Member
My reticence in tackling this project is essentially time, but it's also because software almost always requires after-the-release support. I'm not really in a position to do either anymore. The latter is also one of the mitigating reasons why I closed the DIGWDF Store in 2018. But I'll offer a few thoughts, based on my experience in creating software interfaces when it was thought that interfacing was impossible. I'll use interfacing to a POS (point-of-sale) system as an example; this would be taking the POS system's sales output and porting it into an accounting system -- a "one-direction" interface. (Most POS providers would allow reading their data, but not writing back to it.) I'll offer these thoughts because some folks usually think of a data converter as a one-step process. It's usually not.
Discovery of the order of creating data is important. Having access to all the data files of both the source system and receiving system is important. Knowing what the pieces of data are, how they're formatted, and what their limits are is important.
The interface begins with the creation of an empty data table which will hold all the elements of data which are important. In our integrated accounting system, we needed a transaction# (ticket#), transaction date and time (ticket open or close time or both), customer#, employee#, pos station#, base sale $ (cost of an item before tax or service charge), whether it was taxable and at what rate, whether it applied to service charges and at what rate, sale total, how it was tendered (how it was paid for, including multiple tender types, cash, check, charge, visa, mastercard, etc.). In some cases, there may be multiple employees who service the same customer, so there was also a need to identify which items were sold by each employee. Many times, these pieces of data were spread across multiple data files, usually linked together by a transaction#/ticket#. Sometimes the data was provided as summary totals only and it was necessary to reverse-engineer the POS system to figure it out. One of these interfaces could easily cost $10K to develop for a customer; the benefit is the elimination of an employee to manually punch data into the accounting system.
Discovery of the order of creating data is important. Having access to all the data files of both the source system and receiving system is important. Knowing what the pieces of data are, how they're formatted, and what their limits are is important.
The interface begins with the creation of an empty data table which will hold all the elements of data which are important. In our integrated accounting system, we needed a transaction# (ticket#), transaction date and time (ticket open or close time or both), customer#, employee#, pos station#, base sale $ (cost of an item before tax or service charge), whether it was taxable and at what rate, whether it applied to service charges and at what rate, sale total, how it was tendered (how it was paid for, including multiple tender types, cash, check, charge, visa, mastercard, etc.). In some cases, there may be multiple employees who service the same customer, so there was also a need to identify which items were sold by each employee. Many times, these pieces of data were spread across multiple data files, usually linked together by a transaction#/ticket#. Sometimes the data was provided as summary totals only and it was necessary to reverse-engineer the POS system to figure it out. One of these interfaces could easily cost $10K to develop for a customer; the benefit is the elimination of an employee to manually punch data into the accounting system.
dirknerkle
Supporting Member
(continued)
One of these interfaces could easily cost $10K to develop for a customer; the customer's benefit is the elimination of one or more employees' salaries to manually punch data into the accounting system, not to mention keying errors.
The first pass through the data was to read the transaction#'s/ticket#'s for the chosen date (sales are tracked daily) and write these to the empty table. Then one-by-one, the process would begin to gather each of the other pieces of data, usually using multiple passes through one or more source databases, often making multiple queries or sometimes, creating yet another empty table for each transaction. It seems like this would be a slow process, and sometimes it might require 20-30 passes through the POS system's data to finally gather all the requisite information for that ticket#. When it was finally gathered, the process continued with the next transaction/ticket#. When all tickets had been processed, the data was then transferred from the temporary table to the accounting system for further processing that the accounting system required.
The point is that it was always possible to make it work, although sometimes it took some heroics to get it done. Sometimes I had to reconstruct my own working copy of the POS system's complete inventory and server/employee records or even read the button layout information for the POS system's screens as I reverse-engineered the POS system. And because screens could change daily, this became yet another set of passes through the data. But it was always possible, as long as the POS provider would provide us with information about the structure of the data we were looking at.
This sounds to me what might be necessary to decode the FSEQ file. There obviously is some order to the FSEQ (or other output file, whatever it's called) and what's required is to understand what's in it. It's NOT just a bunch of data. It's in a specific order and it was created by the sequencing software in a specific order so that it could be read and displayed in a specific order by FPP. What's necessary is to keep working backwards into that process to glean the underlying data, and then equate that to the underlying data in the other software package. It's likely that some sort of intermediary temporary holding database will be required, but I believe it's possible to do.
But sorry, I'm just not the one to do it.
One of these interfaces could easily cost $10K to develop for a customer; the customer's benefit is the elimination of one or more employees' salaries to manually punch data into the accounting system, not to mention keying errors.
The first pass through the data was to read the transaction#'s/ticket#'s for the chosen date (sales are tracked daily) and write these to the empty table. Then one-by-one, the process would begin to gather each of the other pieces of data, usually using multiple passes through one or more source databases, often making multiple queries or sometimes, creating yet another empty table for each transaction. It seems like this would be a slow process, and sometimes it might require 20-30 passes through the POS system's data to finally gather all the requisite information for that ticket#. When it was finally gathered, the process continued with the next transaction/ticket#. When all tickets had been processed, the data was then transferred from the temporary table to the accounting system for further processing that the accounting system required.
The point is that it was always possible to make it work, although sometimes it took some heroics to get it done. Sometimes I had to reconstruct my own working copy of the POS system's complete inventory and server/employee records or even read the button layout information for the POS system's screens as I reverse-engineered the POS system. And because screens could change daily, this became yet another set of passes through the data. But it was always possible, as long as the POS provider would provide us with information about the structure of the data we were looking at.
This sounds to me what might be necessary to decode the FSEQ file. There obviously is some order to the FSEQ (or other output file, whatever it's called) and what's required is to understand what's in it. It's NOT just a bunch of data. It's in a specific order and it was created by the sequencing software in a specific order so that it could be read and displayed in a specific order by FPP. What's necessary is to keep working backwards into that process to glean the underlying data, and then equate that to the underlying data in the other software package. It's likely that some sort of intermediary temporary holding database will be required, but I believe it's possible to do.
But sorry, I'm just not the one to do it.
Coming out of both the banking IT world and medical IT, I agree with you about 99.9% of this.
To reiterate: The thing I'm seeing is that the .fseq appears to be an "intermediate" file, containing only the data stream created by the sequencer. The stream is sent to the controller, which is already setup with the information as how many bits go to the first output, then the second, and so on. The player uses a different source to say, "this many bits go to this controller, this many to the next, and so on). The controller setup information is either sent via the programming software (sequencer in our vocabulary), or is manually setup by the user (what has become "the old days" for me, but still exists for some).
It is my belief that the real possibility is back at and before the data stream (.fseq) is created.
(I need to relook at the way that Vixen works. )
I do believe that there must be a way to create a transfer program to share sequences between the two of them. There are great sequences on both sides of the fence. Since we have gone to a "sharing/purchase" world from everyone doing their own sequencing (yes - for you new folks, we used to have to do it all ourselves) this seems to me to be prudent, and most desirable.
And please, if there are any programmers of the base creation level for either program out there, speak up. All of my observations are just that, from the outside looking in. I would love to see a video/class on the creation of the data stream by someone who really knows.
To reiterate: The thing I'm seeing is that the .fseq appears to be an "intermediate" file, containing only the data stream created by the sequencer. The stream is sent to the controller, which is already setup with the information as how many bits go to the first output, then the second, and so on. The player uses a different source to say, "this many bits go to this controller, this many to the next, and so on). The controller setup information is either sent via the programming software (sequencer in our vocabulary), or is manually setup by the user (what has become "the old days" for me, but still exists for some).
It is my belief that the real possibility is back at and before the data stream (.fseq) is created.
(I need to relook at the way that Vixen works. )
I do believe that there must be a way to create a transfer program to share sequences between the two of them. There are great sequences on both sides of the fence. Since we have gone to a "sharing/purchase" world from everyone doing their own sequencing (yes - for you new folks, we used to have to do it all ourselves) this seems to me to be prudent, and most desirable.
And please, if there are any programmers of the base creation level for either program out there, speak up. All of my observations are just that, from the outside looking in. I would love to see a video/class on the creation of the data stream by someone who really knows.
dirknerkle
Supporting Member
Agreed, especially as Vixen3, XLights and FPP are still undergoing changes in their own featuresets, which usually causes changes in their own database structures. To make an interface feasible, the source platforms have to be virtually locked in stone. And we all know that as soon as anything new shows up in this hobby, there are gobs of people exercising their glommers to get it.
Hockeyhunter1956
Supporting Member
Having reviewed all of the comments so far, it again really makes me appreciate all the work the the developers and programmers put into our software for Christmas lighting.
Agreed.Having reviewed all of the comments so far, it again really makes me appreciate all the work the the developers and programmers put into our software for Christmas lighting.
I skimmed a lot of the thread here, but here are some comments based on the high-level ideas I saw.
Mapping the props themselves between the two platforms would not be incredibly difficult. Both systems are modeling the same physical item, so all the relevant aspects of it need to be represented. Today you can import the custom xLights model types (xModel) into Vixen and create a Vixen prop from the similar parts. This could be expanded to include more of the standard types that are equivalent to the standard Vixen types with a reasonable amount of effort. The core ways we create and model a prop are different, but at the very basic you need to know where the lights are in some coordinate system. (Conversely you could create code that could export the Vixen prop to xModel format).
Many of the effects that both sequencers employ are similar in their basics, but each one has their own nuances and settings. Many of the Vixen effects had their roots from the same place as the xLights ones. We have each incorporated ideas from each other over the years. Some of this could be mapped but would never really yield exact results since it is a much larger difference than just trying to model the same physical item. The rendering and layering is significantly different. Although some similar end results can be achieved the two approaches are not equivelent.
As far as FSEQ, the output for a given channel / controller is the same. FPP can play those files just the same as long as the channel mapping is the same. I.E if you export a FSEQ file from xLights and Vixen that have the same channel mapping, they both can be played on the same FPP device with the only difference being the content created. The formats / mapping are the same given you modeled the display with the same controller layout in each. xLights can create model maps to map the channels in the FSEQ to a prop. This is seen in the testing and overlay parts of FPP where you can target specific models. That is just mapping channels in the FSEQ output setup to the prop. This same thing could be built into Vixen. (It's been on my todo list). But in essence this should result in again similar channel blocks to prop mappings from the two sequencers since again assuming the FSEQ from each is channel mapped the same.
I am not sure if this fully addresses all the questions, but hopefully provides some useful insight.
Jeff
Mapping the props themselves between the two platforms would not be incredibly difficult. Both systems are modeling the same physical item, so all the relevant aspects of it need to be represented. Today you can import the custom xLights model types (xModel) into Vixen and create a Vixen prop from the similar parts. This could be expanded to include more of the standard types that are equivalent to the standard Vixen types with a reasonable amount of effort. The core ways we create and model a prop are different, but at the very basic you need to know where the lights are in some coordinate system. (Conversely you could create code that could export the Vixen prop to xModel format).
Many of the effects that both sequencers employ are similar in their basics, but each one has their own nuances and settings. Many of the Vixen effects had their roots from the same place as the xLights ones. We have each incorporated ideas from each other over the years. Some of this could be mapped but would never really yield exact results since it is a much larger difference than just trying to model the same physical item. The rendering and layering is significantly different. Although some similar end results can be achieved the two approaches are not equivelent.
As far as FSEQ, the output for a given channel / controller is the same. FPP can play those files just the same as long as the channel mapping is the same. I.E if you export a FSEQ file from xLights and Vixen that have the same channel mapping, they both can be played on the same FPP device with the only difference being the content created. The formats / mapping are the same given you modeled the display with the same controller layout in each. xLights can create model maps to map the channels in the FSEQ to a prop. This is seen in the testing and overlay parts of FPP where you can target specific models. That is just mapping channels in the FSEQ output setup to the prop. This same thing could be built into Vixen. (It's been on my todo list). But in essence this should result in again similar channel blocks to prop mappings from the two sequencers since again assuming the FSEQ from each is channel mapped the same.
I am not sure if this fully addresses all the questions, but hopefully provides some useful insight.
Jeff
Thanks Jeff.
Your statement confirms (to me) that the conversion of actual sequences from xLights to Vixen, or Vixen to xLights will be a bit of a problem ("The rendering and layering is significantly different.")
In so many ways, this saddens me as there are so many good sequences on both sides of the fence that cannot jump that fence without just rewriting them. (Trust me, I spent a LOT of time doing just that when we moved from the mix of Vixen and xLights to 100% xLights.)
Your statement confirms (to me) that the conversion of actual sequences from xLights to Vixen, or Vixen to xLights will be a bit of a problem ("The rendering and layering is significantly different.")
In so many ways, this saddens me as there are so many good sequences on both sides of the fence that cannot jump that fence without just rewriting them. (Trust me, I spent a LOT of time doing just that when we moved from the mix of Vixen and xLights to 100% xLights.)