Page 1 of 5 123 ... LastLast
Results 1 to 10 of 46

Thread: Singing Pumpkin Lip-Sync

  1. #1
    Join Date
    Jan 2008
    Location
    Canton, Ohio
    Posts
    39
    Post Thanks / Like

    Default Singing Pumpkin Lip-Sync

    Well this year I wanted one of those 6' singing pumpkins on the house. I spent a bunch of time researching how to do this, and I really didn't find everything I was looking for in one place.

    Now you can find it here, because I'm going to save you the pain.

    First, you need to build a scaffolding for the pumpkin. I made mine out of five 10' pieces of 1/2" PVC piping, eight 1/2" T connectors, four 1/2" + connectors, and four 1/2" elbows. I also purchased (at Lowes), a roll of plastic hardware cloth (24" x 25' x 1/2" squares). I also used garden twist ties on a spool (for mounting the cloth on the frame, but zip ties would work). Then I purchased some spray paint to darken it all up.

    Cut all of the PVC pipe to 23" (you will have one extra 23" piece). Build a tic-tac-toe shape with a frame around it. It should be 6' square. Unroll the hardware cloth in three strips and mount it to the frame with the ties, enough to support the weight of the light strands you will be putting onto the frame for the face.

    Paint the frame a suitably dark flat color to hide it. so it doesn't reflect when the lights come on.

    Mount either rope light, or mini lights to the mesh cloth to make the pumpkin face. I made triangular eyes. The lower part of the eyes are one channel. The upper two edges of the triangles for the eyes are on a second channel. This allows the pumpkin to "blink" approximately every 4-5 seconds for 300 ms.

    The "mouth" has several shapes, and a channel for each. An upper lip, bottom lower lip, lower lip (just slightly above the bottom), a middle lip, an O mouth, and a smaller o mouth with some corners showing (this is the U mouth described later).

    The pumpkin can be controlled with eight channels. So how to animate it? Here's what I found that was interesting. The various facial poses shown on this site were my guide --

    www.garycmartin.com/mouth_shapes.html

    Using the various lip shapes, I mimic the facial features shown by Gary. I started off by taking my lyrics and picking a letter like "O" and marking all of them, then using Vixen to animate the "O" mouth. Big mistake, and really slow progress....

    So I read a very interesting web page:

    www.keithlango.com/tutorials/old/lipSync.htm

    which explains the principles of lip sync animation. Less is more. Ok, but I still am a geeky guy, and I wanted something to take text, and convert it to phonemes (the mouth shapes). I love open source, so I wanted something free.

    What I found was a cool program that takes .wav files of audio, and typed in lyrics, and helps you layout the phonemes. It's called Papagayo, and it's available here:

    www.lostmarble.com/papagayo/index.shtml

    I took my .mp3 files and used audacity to chop them down to verses, and convert them to .wav files. Each .wav verse is loaded into Papagayo (trust me it's easier to work with smaller sections at a time!) and animated.

    Before using Papagayo, I used a Vixen add-in called Tap-Tempo to determine the timing of the song I was sequencing. For example, I chose "Thriller", and I used a timing of 64, giving 16 events per second. The 16 number is used to set the FPS rate in Papagayo.

    Now, loading the first verse into Papagayo, I immediately set the frames per second (FPS) to 16, then paste in the lyrics of the first verse. Use the tutorials to learn how to manipulate the sentences, then the words, then the phonemes. Use the software to get the mouth animations to look just right. After you've done that, you are ready to "Export Voice" or just use the grid display in Papagayo to map your phonemes into Vixen.

    The mouth shapes in Papagayo are; AI, E, etc, FV, L, MBP, O, rest, U, WQ. In Vixen, on my pumpkin face, I mapped this to; AI = Upper Lip + Bottom Lower Lip, E = Upper Lip + Lower Lip, etc = Upper Lip + Middle Lip, FV = Middle Lip, L = Upper Lip + Bottom Lower Lip (same as AI), MBP = Upper Lip, O = O mouth, rest = nothing or Middle Lip -- depending on how it looked, U = U mouth, and WQ = U mouth.

    Now, pick the first word (phoneme) and find the corresponding audio portion in vixen. In my case, Papagayo time index 18 = 0:58.752 in Vixen. At this location in Vixen, I used the AI phoneme to turn on the upper and bottom lips (that's my mapping). Let's say that the next Papagayo transition phoneme is at time index 22, and it's "etc". This means that the AI phoneme runs from time index 18 to time index 21, or 4 frames. In Vixen, the upper and bottom lower channels are on from 0:58.752 through 0:58.944.

    Rinse, repeat for the entire verse, then the entire song. Errors in timing will creep in, so you may need to re-align the locations of the transitions in the sequence, but the mouth shapes should be correct for the lyrics.

    What we really need is an add-in that takes the export from Papagayo, asks for the channel mappings from the mouth shapes to the appropriate Vixen channels, and the time index in Vixen for time index 1 in Papagayo. Presto, chango, a tool for quickly (relatively speaking ;) doing lip-sync.

    I hope this helps someone else, and I welcome any comments, or even better -- a faster way of doing this.

    Andy

  2. #2
    Join Date
    Oct 2008
    Location
    San Jose, CA
    Posts
    10,283
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    Wow - great work putting this all together! I keep saying every year I'm going to go big with Halloween as I do with Christmas but never quite get there... Add this stuff to the wireframes sold by Dougp and you're set!
    [COLOR=#000080][B][I]Brian[/I][/B][/COLOR]

    [COLOR=#ff0000]Christmas in San Jose! - [URL="http://www.christmasinsj.com"]WEB[/URL] - [URL="https://www.facebook.com/ChristmasInSanJose"]FB[/URL] - [URL="https://www.youtube.com/playlist?list=PL1W78s7liEQEE0ed7WSyLF7B6j3lBX43w"]VIDEOS[/URL]
    [/COLOR][COLOR=#800080]Halloween in San Jose! - [URL="https://www.facebook.com/pages/Halloween-in-San-Jose/356280784428581?ref=tn_tnmn"]FB[/URL]
    [SIZE=1]2015 Halloween Show - Planning now - hopefully some house projection...
    2015 Christmas Show - 5x E681-12, 1x 6804, Ren48LSD, 3x RenSS16, 1x Falcon16v2 (w/expander), 24x90 WS2811 pixel MT (James MT Strips), 12x DIYC Floods, SuperPixelStar, 3x Pixel Arches, PixaBulb House outline
    [/SIZE]
    [/COLOR][SIZE=3][COLOR=#008000][I]Ignorance is Temporary [/I][/COLOR][COLOR=#800080][I]- [/I][/COLOR][COLOR=#ff0000][I]Stupidity is Forever[/I][/COLOR][COLOR=#800080][I]...[/I][/COLOR][/SIZE][I][COLOR=#ff0000]
    [/COLOR][/I]

  3. #3
    Join Date
    Jan 2008
    Location
    Canton, Ohio
    Posts
    39
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    Wow, hadn't seen Doug's stuff prior to this. I wish I'd known before building my pumpkin. In a few days, I'll add screen shots and pictures, if it will help.

    Andy

  4. #4
    Join Date
    Dec 2008
    Location
    Ft. Mitchell, KY (Cincinnati Area)
    Posts
    3,923
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    Quote Originally Posted by andyb View Post
    Wow, hadn't seen Doug's stuff prior to this. I wish I'd known before building my pumpkin. In a few days, I'll add screen shots and pictures, if it will help.

    Andy
    Don't kick yourself. Doug just posted his stuff. I'm very excited about his stuff too. Your info makes it evern better.
    [COLOR="#A52A2A"][FONT=Arial Black][SIZE=3][B]Dark Yard again for 2017 [/B][/SIZE][/FONT][/COLOR]
    [COLOR="#008000"][SIZE=2] Still searching for more Time and Money[/SIZE][/COLOR]
    [COLOR="BLUE"][URL="https://www.facebook.com/KentuckyChristmas"]Display Facebook Page[/URL][/COLOR]

  5. #5
    Join Date
    Dec 2009
    Posts
    392
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    Definitely add pictures. I'm sure many of us would be interested in seeing them.

  6. #6
    Join Date
    Nov 2007
    Location
    Blasdell NY (Hamburg)
    Posts
    64
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    andyb - any updates. I would love to see pics!

  7. #7
    Join Date
    Jan 2010
    Location
    Omaha, NE
    Posts
    2,025
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    If you are having trouble sequencing the mouths, I have one set to Thriller that I can sell you. I did not program it, but it's good and I have to pay the individual that did it.

    It only uses 3 mouths. Send me a PM if you want it.
    Doug

    Having Fun with Wire Frames and Lights.

    www.wireframedlites.com
    Custom Hand Made In USA

    FaceBook:
    http://www.facebook.com/wireframedlites

    Phillips Family Display
    http://www.facebook.com/phillipsfamilydisplay

  8. #8
    Join Date
    Oct 2008
    Location
    San Jose, CA
    Posts
    10,283
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    I've been playing a bit with Papagayo from andyb's suggestion. I can definitely see the advantage into cutting the song up into bits. It is no fun at all to move the phrases around after cutting-n-pasting in the lyrics to an entire song. I wish that was one thing they would fix but it doesn't look this is really actively developed anymore. It's open source but my programming skills aren't even close to being able to make changes.

    Anyway - I looked closely at my pumpkin and laid out where the various mouth breaks are so I would know how to run the lights. As this is a relatively small pumpkin, some of the mouth sections are only a few inches long. I bought some rope light for this with 18" sections so it looks like I'll have lots of black tape to hide parts of the sections. If I get it done, I'll post some pics.

    I also laid things out in Visio and exported it to a PDF so you can match the mouths from Papagayo to the pumpkin. It's not exact but it will probably look just fine (or should I say - close enough...
    Attached Files Attached Files
    [COLOR=#000080][B][I]Brian[/I][/B][/COLOR]

    [COLOR=#ff0000]Christmas in San Jose! - [URL="http://www.christmasinsj.com"]WEB[/URL] - [URL="https://www.facebook.com/ChristmasInSanJose"]FB[/URL] - [URL="https://www.youtube.com/playlist?list=PL1W78s7liEQEE0ed7WSyLF7B6j3lBX43w"]VIDEOS[/URL]
    [/COLOR][COLOR=#800080]Halloween in San Jose! - [URL="https://www.facebook.com/pages/Halloween-in-San-Jose/356280784428581?ref=tn_tnmn"]FB[/URL]
    [SIZE=1]2015 Halloween Show - Planning now - hopefully some house projection...
    2015 Christmas Show - 5x E681-12, 1x 6804, Ren48LSD, 3x RenSS16, 1x Falcon16v2 (w/expander), 24x90 WS2811 pixel MT (James MT Strips), 12x DIYC Floods, SuperPixelStar, 3x Pixel Arches, PixaBulb House outline
    [/SIZE]
    [/COLOR][SIZE=3][COLOR=#008000][I]Ignorance is Temporary [/I][/COLOR][COLOR=#800080][I]- [/I][/COLOR][COLOR=#ff0000][I]Stupidity is Forever[/I][/COLOR][COLOR=#800080][I]...[/I][/COLOR][/SIZE][I][COLOR=#ff0000]
    [/COLOR][/I]

  9. #9
    Join Date
    Jan 2010
    Location
    Omaha, NE
    Posts
    2,025
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    Look's great. If you all see something that I need to do in making these to make it easier for you to put lights on them, please let me know. Send an email to info@wireframedlites.com

    Doug

    Quote Originally Posted by budude View Post
    I've been playing a bit with Papagayo from andyb's suggestion. I can definitely see the advantage into cutting the song up into bits. It is no fun at all to move the phrases around after cutting-n-pasting in the lyrics to an entire song. I wish that was one thing they would fix but it doesn't look this is really actively developed anymore. It's open source but my programming skills aren't even close to being able to make changes.

    Anyway - I looked closely at my pumpkin and laid out where the various mouth breaks are so I would know how to run the lights. As this is a relatively small pumpkin, some of the mouth sections are only a few inches long. I bought some rope light for this with 18" sections so it looks like I'll have lots of black tape to hide parts of the sections. If I get it done, I'll post some pics.

    I also laid things out in Visio and exported it to a PDF so you can match the mouths from Papagayo to the pumpkin. It's not exact but it will probably look just fine (or should I say - close enough...
    Doug

    Having Fun with Wire Frames and Lights.

    www.wireframedlites.com
    Custom Hand Made In USA

    FaceBook:
    http://www.facebook.com/wireframedlites

    Phillips Family Display
    http://www.facebook.com/phillipsfamilydisplay

  10. #10
    Join Date
    May 2007
    Location
    Spencer Iowa
    Posts
    1,236
    Post Thanks / Like

    Default Re: Singing Pumpkin Lip-Sync

    Quote Originally Posted by budude View Post
    Anyway - I looked closely at my pumpkin and laid out where the various mouth breaks are so I would know how to run the lights. As this is a relatively small pumpkin, some of the mouth sections are only a few inches long. I bought some rope light for this with 18" sections so it looks like I'll have lots of black tape to hide parts of the sections. If I get it done, I'll post some pics.
    never used it, but 12v rope light usually lists a 3 or 4 inch cut.

    Joel
    Lets get serious, I have over 10,000 channels on my back, just where do you want to start counting.

Page 1 of 5 123 ... LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •