PDA

View Full Version : Help Getting Started with Module for Papagayo Imports



tmarshal61
05-06-2014, 12:03 PM
Hi,

I am interested in trying my hand at creating a module for importing Papagayo files into Vixen. I am tired of doing it by hand.

I am a bit rusty at c# and development in general and I could use a little guidance.

I downloaded the vixen source from GitHub and opened it in Visual Studio 2013 Professional. When I try to open some forms and modules, I get warnings complaining about libraries that were compiled with a different processor. (like AMD64) I am running Windows 7 x64 using an intel i7 processor. When I try to rebuild, I get similar warnings.

Should I ignore these warnings? Any tips on getting an optimal development environment up and running?

Also, What is the basic procedure for programmatically inserting a simple a simple "Set Level" effect on a group in the timeline and refreshing the display to show it?

Thanks,

Tom

ctmal
05-06-2014, 03:02 PM
It's a bit outdated but maybe this will help?

https://www.youtube.com/watch?v=tHFk0vhvEoM&list=PL4BCC7CCA3FE53C1E

kwoodard
05-06-2014, 07:56 PM
I'm really interested in seeing a Papagayo module for Vixen. I have C# experience so I might be able to help out some. I haven't done a Vixen module, but I'll try to get setup to help out. I've started watching some of the development videos that ctmal pointed out and it looks very well designed.

Sent from my SCH-I545 using Tapatalk

tmarshal61
05-06-2014, 07:58 PM
It's a bit outdated but maybe this will help?

https://www.youtube.com/watch?v=tHFk0vhvEoM&list=PL4BCC7CCA3FE53C1E

Thank You, That is going to be a tremendous help. Based the URL you provided, I was able to find the following post you created:

http://doityourselfchristmas.com/forums/showthread.php?18369-Vixen-3-Output-Module-Programming-Tutorial
http://doityourselfchristmas.com/forums/showthread.php?19267-Vixen-3-0-Effect-Module-Programming-tutorial


I have some homework to do.

At the time these tutorials were created, C# Studio 2010 Express and .Net 4.0 were being used. Is .Net 4.0 still being used and would there be any issues using Visual Studio 2013 Professional?

Thanks

tmarshal61
05-06-2014, 08:08 PM
I'm really interested in seeing a Papagayo module for Vixen. I have C# experience so I might be able to help out some. I haven't done a Vixen module, but I'll try to get setup to help out. I've started watching some of the development videos that ctmal pointed out and it looks very well designed.

Sent from my SCH-I545 using Tapatalk

That would be great. I pretty much have the Papagayo part figured out. I created a proof-of-concept in Excel that imports a Papagayo File and computes Vixen friendly starting and ending times for all the Papagayo Phonemes. It was pretty easy to figure out.

The hard part for me will be figuring out how to integrate this with Vixen. I appreciate any help I can get. I have dabbled a little but, I am not a programmer.

I am thinking this would be a good example of an Vixen App Module. I am not going to be home where I can work on it for a few days but that should give me time to go through the tutorials.

kwoodard
05-06-2014, 08:17 PM
I'll be away from home a few days also. In the mean time, I'll get my development environment setup and try to work through some of the videos. If you don't mind sharing your spreadsheet, I can spend some time thinking about the module design. This sounds like a cool project and would fill in a big need in Vixen in my opinion.

Sent from my SCH-I545 using Tapatalk

kwoodard
05-06-2014, 08:21 PM
Maybe ctmal can chime in here and point towards the proper module type. My gut tells me this might be an effects module, but that's without watching all the tutorials yet.

Sent from my SCH-I545 using Tapatalk

tmarshal61
05-06-2014, 08:36 PM
Maybe ctmal can chime in here and point towards the proper module type. My gut tells me this might be an effects module, but that's without watching all the tutorials yet.

Sent from my SCH-I545 using Tapatalk

It could work as an effects module. It might also be implemented as a plugin app accessed from the "tools" menu. I have not read enough of the tutorials to know for sure yet which way it should go but, I think an effects module would work. In fact, it might even be better. We will certainly be creating effects, and lots of them. They would be Set Level effects applied to element groups on the timeline. I am thinking of a Wizard type interface where you select the Papagayo file, and it parses through the file and displays the results in a grid where you can map the Vixen Groups that make up the mouth shapes to the Papagayo Phonemes, specify the color and intensity for each voice and, click a button, and each effect element is added to the timeline on associated with the appropriate mouth shape group and voila, you are done.

ctmal
05-07-2014, 07:55 AM
Maybe ctmal can chime in here and point towards the proper module type. My gut tells me this might be an effects module, but that's without watching all the tutorials yet.

Sent from my SCH-I545 using Tapatalk

Unfortunately I have been away from playing with Vixen 3 for quite awhile(although I hope to get back to it soon) so I'm not sure I'm the best to answer the question. Hopefully someone who is active on the team will chime in.
I'm not sure an effect is what you are looking for though. The module your creating will be creating effects.

jeffu231
05-08-2014, 09:20 AM
This can be done a few different ways. Today we import beats from Audacity into the mark manager. This could probably be done in a similar way with a toolbar menu in the main editor to import and process the Papagayo file. Basically you will be looping over your segments defined in your import file and creating a effect on the proper element of the correct time and length. By using set levels you will could be generating a fair amount of effects. If you are willing to go further, you can look at the mouth as a group of elements and use the pulse effect at the group level. Within that effect you can make intents for the smaller parts of the mouth parts and make a longer single effect which is much more efficient. My thoughts where to do this at the word level. So each word would be a single effect. I actually wanted to do it directly in the editor by creating a word effect that could be manipulated similar to how it is done in Papagayo. When the elements and group is created in the setup, there could be properties to define the proper mouth parts so the effect could place the parts properly.

I am not sure about others, but I can't seem to find a reliable version of Papagayo and always end up getting frustrated with it crashing or loosing data on longer songs.

I am willing to help here as I am very familiar with the parts of Vixen you will need to work on.

ebrady
05-08-2014, 02:00 PM
I have been working on adding Phoneme and Papagayo support for a couple of months now in Vixen3. My approach has been to allow Phoneme Data to be added both natively and as a linked Papagayo file. I ran into a significant coding issue that prevented previews from working correctly and had to rework a significant amount of code and had hoped to have it completed by now.

I am about a week away from having a working (but unpolished) implementation of native Phoneme support, with Papagayo .pgo support following hopefully by the end of may. I plan to check in to my personal branch soon ... I figured I would get a working implementation out there to use a starting point and then let the community evolve it from there...

Attached is a peek, more to follow, I plan to discuss some at Light Up Ohio next week also.... :)

2399323994

kwoodard
05-08-2014, 02:12 PM
Awesome, this looks very promising. I look foreward to giving it a shot. When you are ready for some beta testers, let us know!!

Keith

Sent from my SCH-I545 using Tapatalk

tmarshal61
05-08-2014, 11:23 PM
This can be done a few different ways. Today we import beats from Audacity into the mark manager. This could probably be done in a similar way with a toolbar menu in the main editor to import and process the Papagayo file. Basically you will be looping over your segments defined in your import file and creating a effect on the proper element of the correct time and length. By using set levels you will could be generating a fair amount of effects. If you are willing to go further, you can look at the mouth as a group of elements and use the pulse effect at the group level. Within that effect you can make intents for the smaller parts of the mouth parts and make a longer single effect which is much more efficient. My thoughts where to do this at the word level. So each word would be a single effect. I actually wanted to do it directly in the editor by creating a word effect that could be manipulated similar to how it is done in Papagayo. When the elements and group is created in the setup, there could be properties to define the proper mouth parts so the effect could place the parts properly.

I am not sure about others, but I can't seem to find a reliable version of Papagayo and always end up getting frustrated with it crashing or loosing data on longer songs.

I am willing to help here as I am very familiar with the parts of Vixen you will need to work on.

Thanks for the info. I think I understand your suggestion of using the Pulse effect to emulate multiple Set Level effects. It would be more efficient but it woulod be a lot easier the visually tweak the starting and ending points for a given phoneme if individual set level effects were used.

I agree that Papagayo is very unstable, particularly with larger files. It would be awesome if a similar interface were built right into Vixen. I have looked into it, and creating a program like Papagayo looks quite simple. The hardest part is getting a dictionary mapping of words to Phonemes there is an extended version available at lostmarble.com (http://www.lostmarble.com/downloads/extended_dictionary.zip)

So, How does one go about creating Something like the Mark Maker in Vixen? I really like that idea. Creating a "Collection" of "Set Level" effects, similar to the Mark Collections, one could set up each voice in its own collection then you could move/nudge the entire collection together or turn on/off individual collections representing the individual voices.

Creating new modules looks like something I might be able to figure out but, I would have no idea how to go about integrating something like the Mark Maker into Vixen.

tmarshal61
05-09-2014, 12:29 AM
I have been working on adding Phoneme and Papagayo support for a couple of months now in Vixen3. My approach has been to allow Phoneme Data to be added both natively and as a linked Papagayo file. I ran into a significant coding issue that prevented previews from working correctly and had to rework a significant amount of code and had hoped to have it completed by now.

I am about a week away from having a working (but unpolished) implementation of native Phoneme support, with Papagayo .pgo support following hopefully by the end of may. I plan to check in to my personal branch soon ... I figured I would get a working implementation out there to use a starting point and then let the community evolve it from there...

Attached is a peek, more to follow, I plan to discuss some at Light Up Ohio next week also....

I posted that last comment before I saw your post. This looks awesome. Anyway you could do something like the Mark Collections I was describing in my post to jeffu231?

Does, or will your implementation support multiple voices with separate animations? If so, maybe you will find these useful as some alternate mouth shapes. I plan on making several more sets but this is the only one I have at the moment.





https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/AI.jpg
https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/E.jpg
https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/etc.jpg
https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/FV.jpg
https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/L.jpg


AI
E
etc
FV
L


https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/MBP.jpg
https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/O.jpg
https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/rest.jpg
https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/U.jpg
https://dl.dropboxusercontent.com/u/75341078/Photos/Mouth%20Shapes/Pumpkin%20Faces/WQ.jpg


MBP
O
rest
U
WQ



If there is anything I can do to help, let me know.

ebrady
05-09-2014, 07:03 AM
Edited:


This can be done a few different ways. Today we import beats from Audacity into the mark manager. This could probably be done in a similar way with a toolbar menu in the main editor to import and process the Papagayo file. Basically you will be looping over your segments defined in your import file and creating a effect on the proper element of the correct time and length. By using set levels you will could be generating a fair amount of effects. If you are willing to go further, you can look at the mouth as a group of elements and use the pulse effect at the group level. Within that effect you can make intents for the smaller parts of the mouth parts and make a longer single effect which is much more efficient. My thoughts where to do this at the word level. So each word would be a single effect. I actually wanted to do it directly in the editor by creating a word effect that could be manipulated similar to how it is done in Papagayo. When the elements and group is created in the setup, there could be properties to define the proper mouth parts so the effect could place the parts properly.

I am not sure about others, but I can't seem to find a reliable version of Papagayo and always end up getting frustrated with it crashing or loosing data on longer songs.

I am willing to help here as I am very familiar with the parts of Vixen you will need to work on.

This is the direction I am heading with Papagayo support. I am thinking about adding 3 methods -

Method #1 (Planned) - Convert Papagayo file to Native Vixen Phonemes - The tools menu would contain an item to "import" a Papagayo file. On the import, a set of Phoneme Elements (This is new element type) would be created based upon the current cursor position, or sent to the clipboard so that they can be pasted into the sequence. Future editing would occur in Vixen3, and Papagayo would no longer be needed. (Yea!)
24004

Method #2 (Planned) - Add the ability to link a Papagayo file as a single sequence element. Once linked, the single element would represent the Papagayo file that would be interpreted at runtime. This method is for anyone who prefers to leave the Papagayo file alone and to perform all editing in the Papagayo Editor.
24005
Method #3 - (Investigating) - The Tools menu would contain an item to convert lyric words into a collection of Phonemes that would then inserted into the Vixen Sequence and then edited using the Vixen editor. This method would allow the Vixen editor to replace Papagayo completely (Super Yea!). I think this is feasible with the aid of a public Phoneme library such as that published by CMU.

ebrady
05-09-2014, 07:10 AM
I posted that last comment before I saw your post. This looks awesome. Anyway you could do something like the Mark Collections I was describing in my post to jeffu231?

Does, or will your implementation support multiple voices with separate animations? If so, maybe you will find these useful as some alternate mouth shapes. I plan on making several more sets but this is the only one I have at the moment.

If there is anything I can do to help, let me know.

I hadn't thought of Mark collections, but it is a good idea. It would also be a good way to free the world from the perils of Papagayo! :) I think this is something worth exploring.

While not in this version, I am planning (or anyone else could do it for that matter) to add a feature that will allow the user to provide their own animation picture. The animations you see in the pictures ares stored natively in the execution The animations you see in pictures are the "Default" animations that are embedded natively into he exe as a resource.

jeffu231
05-09-2014, 08:45 AM
I think you are heading down the right path with #1 and #3. #2 will be interesting. Your screen shots are very much in line with some of my thoughts. As such, it sounds like you have a good start. If you have not done so, consider making properties to place on the elements when they are setup to define them as parts of a mouth. You can then use that in your effects to place the proper movements. Look at the discrete colors for some ideas. Once you get it to the point where you want some input let me know and I can grab your branch and give it a spin.

Jeff


Edited:



This is the direction I am heading with Papagayo support. I am thinking about adding 3 methods -

Method #1 (Planned) - Convert Papagayo file to Native Vixen Phonemes - The tools menu would contain an item to "import" a Papagayo file. On the import, a set of Phoneme Elements (This is new element type) would be created based upon the current cursor position, or sent to the clipboard so that they can be pasted into the sequence. Future editing would occur in Vixen3, and Papagayo would no longer be needed. (Yea!)
24004

Method #2 (Planned) - Add the ability to link a Papagayo file as a single sequence element. Once linked, the single element would represent the Papagayo file that would be interpreted at runtime. This method is for anyone who prefers to leave the Papagayo file alone and to perform all editing in the Papagayo Editor.
24005
Method #3 - (Investigating) - The Tools menu would contain an item to convert lyric words into a collection of Phonemes that would then inserted into the Vixen Sequence and then edited using the Vixen editor. This method would allow the Vixen editor to replace Papagayo completely (Super Yea!). I think this is feasible with the aid of a public Phoneme library such as that published by CMU.

tmarshal61
05-09-2014, 12:17 PM
Edited:

Method #3 - (Investigating) - The Tools menu would contain an item to convert lyric words into a collection of Phonemes that would then inserted into the Vixen Sequence and then edited using the Vixen editor. This method would allow the Vixen editor to replace Papagayo completely (Super Yea!). I think this is feasible with the aid of a public Phoneme library such as that published by CMU.

I like this idea a lot. I was looking into something similar, and I also came across the public CMU Word to Phoneme Dictionary. Since their phoneme set is based on speech sounds rather than mouth shapes, it is much larger than the the phoneme set used by Papagayo, (39 vs 10). There is a Papagayo extended dictionary set available at lostmarble.com but, it only contains 84k words whereas the CMU dictionary has 133k, so, the CMU library sounds like the better choice. To simplify things though, you might think about mapping the CMU phoneme set to the Papagayo set. It should be a one to many relationship.

Roterhead
05-10-2014, 01:32 AM
I am watching this thread with great interest and am fascinated by the thoughts and goals expressed so far.

Has any thought been given to the mapping of pixels into the mouth shapes? I can see certain pixels being part of multiple mouth shapes and even colors hence the question.

Will be interesting to see how we will be able to play with color and the mouth shapes. For example, a Halloween mouth may have teeth that might show as yellow, while the mouth parts are orange. The eye outlines could be orange, red or yellow during parts of the song as well with perhaps an actual center portion (eg iris) being yet a different color.

Thank you all for this effort, I am not thinking of building props for Halloween using this as a possible way to bring to life!

ebrady
05-10-2014, 09:11 AM
I am watching this thread with great interest and am fascinated by the thoughts and goals expressed so far.

Has any thought been given to the mapping of pixels into the mouth shapes? I can see certain pixels being part of multiple mouth shapes and even colors hence the question.

Will be interesting to see how we will be able to play with color and the mouth shapes. For example, a Halloween mouth may have teeth that might show as yellow, while the mouth parts are orange. The eye outlines could be orange, red or yellow during parts of the song as well with perhaps an actual center portion (eg iris) being yet a different color.

Thank you all for this effort, I am not thinking of building props for Halloween using this as a possible way to bring to life!

Thanks, yes the mappings have been considered.

For each mouth movement, the user will specify how the mouth movement translates to a given element of a particular name. Multiple mappings can be defined in the System.

24008 24009

As pixels are a consideration here, I also added the ability to independently set a static color, for now, on each element. Ultimately, this functionality could also be used to drive mechanical , servo based, animations as well. As such, it would make sense to remove the color column, and instead add a column that would allow the user to specify one or more existing functions.

i.e. -> Set Level, Pulse, Set Custom Value, Alternating,

Roterhead
05-10-2014, 06:50 PM
I see, I could have a mouth element called Left Tooth, and make that yellow. Then combine that into the OpenMouth name.

And If I wanted multiple colors that will need to wait. Ok, just having something like this to start would be Great.

The add-on you mention would be really kewl, with the ability to control servos to make certain mouth movements!

ebrady
05-11-2014, 11:22 PM
I see, I could have a mouth element called Left Tooth, and make that yellow. Then combine that into the OpenMouth name.

And If I wanted multiple colors that will need to wait. Ok, just having something like this to start would be Great.

The add-on you mention would be really kewl, with the ability to control servos to make certain mouth movements!

Yep, that sums it up!

--- New stuff ---
I have finished implementing the ability to import a Papagayo file. Using the attached pictures as a reference, the user will select a Papagayo pgo file which will get imported to the clipboard as a series of phonemes.

24011240122401324014

This week I will get the source code cleaned up and checked in for anyone who wants to jump in on the development effort. Time permitting I will try to put a beta build together and quick tutorial also.


Ed

Roterhead
05-11-2014, 11:51 PM
Very nice, looking forward to seeing this!

kevr
05-12-2014, 08:15 PM
Fantastic!! I can't wait to have play with this.

Kev

ebrady
05-29-2014, 10:55 PM
I have decided to abandon Method #2 and decided to move forward with Method #3. I added code to utilize the latest version of the CMU Phoneme Dictionary and have native text conversion support working. This also integrates with the Mark manager to allow this process to be automated as much as possible using audacity.

Here are some pictures. I have one more code modification left and I will work on getting a development build together for everyone to throw darts at, hopefully by the beginning of next week.

241172411824119





Edited:



This is the direction I am heading with Papagayo support. I am thinking about adding 3 methods -

Method #1 (Planned) - Convert Papagayo file to Native Vixen Phonemes - The tools menu would contain an item to "import" a Papagayo file. On the import, a set of Phoneme Elements (This is new element type) would be created based upon the current cursor position, or sent to the clipboard so that they can be pasted into the sequence. Future editing would occur in Vixen3, and Papagayo would no longer be needed. (Yea!)
24004

Method #2 (Planned) - Add the ability to link a Papagayo file as a single sequence element. Once linked, the single element would represent the Papagayo file that would be interpreted at runtime. This method is for anyone who prefers to leave the Papagayo file alone and to perform all editing in the Papagayo Editor.
24005
Method #3 - (Investigating) - The Tools menu would contain an item to convert lyric words into a collection of Phonemes that would then inserted into the Vixen Sequence and then edited using the Vixen editor. This method would allow the Vixen editor to replace Papagayo completely (Super Yea!). I think this is feasible with the aid of a public Phoneme library such as that published by CMU.