PDA

View Full Version : Face animation mixup?



dkulp
03-16-2013, 09:37 AM
I'm trying to use the HLS face animation stuff, but it looks like it's mixing up some channels. Specifically, it looks like it's mixing up the "O" and maybe FV channels. If I use words with hard M and B sounds ("My", "Monster", "Lab", "Slab") where I'd expect to have the Fv, I'm getting O. Examples:

My: O, Ai
Lab: mpb, Ai, O

It makes the lib sync look horrible wrong. Is something wrong?

JHinkle
03-16-2013, 12:56 PM
Please provide some words and the codes that you think it should be.

I will use it to check my assignments.

Thanks.

Joe

dkulp
03-16-2013, 01:58 PM
Well, any word that starts with an "M". My, Monster, Mash, etc... or words that have a B like Lab, began, slab.

HLS seems to be mapping the M and B sounds to "O". Papagayo is mapping them to MBP.

Lab is an interesting one. HLS is mapping to "MBP", "AI", "O". Papagayo is mapping it to "L", "AI", "MBP" which certainly seems to make more sense. L + A + B

Basically, I uses the Monster Mash lyrics for the first verse or so. (and for my face mapping, I'm using a direct map for all 10 channels. mbp -> mbp, o -> o, etc....)

JHinkle
03-16-2013, 03:08 PM
Dan:

You have far more knowledge in this area than myself.

The following are the assignments I made based on my research.

Please comment of what you think needs to be changed.

Joe

Map_SPVISEMES2Faces[Phoneme_aa] = HVS_AI_A;
Map_SPVISEMES2Faces[Phoneme_ae] = HVS_AI_A;
Map_SPVISEMES2Faces[Phoneme_ah] = HVS_AI_A;
Map_SPVISEMES2Faces[Phoneme_ao] = HVS_O_G;
Map_SPVISEMES2Faces[Phoneme_aw] = HVS_O_G;
Map_SPVISEMES2Faces[Phoneme_ax] = HVS_O_G; // ??? O or AI
Map_SPVISEMES2Faces[Phoneme_ay] = HVS_AI_A;
Map_SPVISEMES2Faces[Phoneme_b] = HVS_MBP_F;
Map_SPVISEMES2Faces[Phoneme_ch] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_d] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_dh] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_eh] = HVS_E_C;
Map_SPVISEMES2Faces[Phoneme_er] = HVS_E_C;
Map_SPVISEMES2Faces[Phoneme_ey] = HVS_E_C;
Map_SPVISEMES2Faces[Phoneme_f] = HVS_FV_D;
Map_SPVISEMES2Faces[Phoneme_g] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_h] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_ih] = HVS_AI_A;
Map_SPVISEMES2Faces[Phoneme_iy] = HVS_E_C;
Map_SPVISEMES2Faces[Phoneme_jh] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_k] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_l] = HVS_L_E;
Map_SPVISEMES2Faces[Phoneme_m] = HVS_MBP_F;
Map_SPVISEMES2Faces[Phoneme_n] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_ng] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_ow] = HVS_O_G;
Map_SPVISEMES2Faces[Phoneme_oy] = HVS_WQ_I;
Map_SPVISEMES2Faces[Phoneme_p] = HVS_MBP_F;
Map_SPVISEMES2Faces[Phoneme_r] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_s] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_sh] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_t] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_th] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_uh] = HVS_U_H;
Map_SPVISEMES2Faces[Phoneme_uw] = HVS_U_H;
Map_SPVISEMES2Faces[Phoneme_v] = HVS_FV_D;
Map_SPVISEMES2Faces[Phoneme_w] = HVS_WQ_I;
Map_SPVISEMES2Faces[Phoneme_y] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_z] = HVS_ETC_B;
Map_SPVISEMES2Faces[Phoneme_zh] = HVS_ETC_B;

dkulp
03-16-2013, 04:19 PM
That all looks right. LAB would be HVS_L_E + HVS_AI_A + HVS_MBP_F which is exactly what I want. Thus, that part looks fine. Is it something then occurring between the Create Vocal Sheets and the Place Animations that may be messing things up?

In my HLS_Dictionary.HLSdic, for LAB, I see:

LAB 31 11 17

Those numbers obviously map to the positions somehow, but not what the correct values would be.

JHinkle
03-16-2013, 08:03 PM
I map the 25 Phonemes into 10 mouth positions.

Your job is to map those 10 to your particular mouth --- most wire based faces don't use all ten

That assignment is done via "Assign Mouth Positions to Face Positions" --- see below.

Your face positions then drive channels.

Check your assignments ... see if the expected face position is what you have mapped.

Joe

18607

dkulp
03-16-2013, 08:10 PM
As I mentioned above, in the Assign Mouth Positions to Face Positions", I have them all mapped directly 1:1. Thus, all "10" positions should be available for mapping.

When I then go to the Place Animations dialog, it has the incorrect mouth positions next to the words.

18608
18609

JHinkle
03-16-2013, 08:28 PM
I'll look into it - Thanks.

Now I understand what you were saying earlier.

I will use LAB as my test.

Joe

dkulp
03-16-2013, 08:35 PM
Note sure if this helps, but this is what Papagayo generates for that phrase:


I 0 4 1
0 AI
was 5 19 3
5 WQ
10 AI
15 etc
working 20 44 5
20 WQ
25 E
30 etc
35 AI
40 etc
in 45 54 2
45 AI
50 etc
the 55 64 2
55 etc
60 AI
lab 65 79 3
65 L
70 AI
75 MBP
late 80 94 3
80 L
85 E
90 etc
one 95 109 3
95 WQ
100 AI
105 etc
night 110 124 3
110 etc
115 AI
120 etc

JHinkle
03-16-2013, 09:47 PM
I had a translation matrix error.

I believe I have corrected it --- my testing of "LAB" and "working" produces the expected results.

The processing is complex in places - shift between zero based arrays and one based arrays.

Please continue ... using 13E or later and let me know is all is OK.

Thanks for your input.

Joe

dkulp
03-18-2013, 01:13 PM
Joe,

The fixes definitely seem to help a lot. Major thanks. Things look a lot better now.

However, I'd like a "feature request" of the ability to edit a word mapping. There are a couple of cases where the way HLS maps a word to movements doesn't look quite right. For example, the word "Suddenly" is mapped to "etc, AI, etc, O, etc, L, E". The "O" in the middle looks slightly out of place when run. I'd like to be able to edit that to an AI. (BTW: papagayo maps that O to an AI instead). I understand that mapping text to mouth positions is a very complex thing to do and likely isn't 100% correct no matter what you do. I'd love to be able to double click on the "Positions" cell in the Place Animations dialog to edit it.

JHinkle
03-18-2013, 09:50 PM
Sounds like a great idea.

Let me ponder the best way to implement it.

18661 18662

I typed in the word Suddenly as shown in the left pic - generating the the list of Phonemes indicated by the arrow.

There's like 50+ different Phonemes I generate.

I pass them through the translation matrix shown several posts up to get to the "Working Ten".

So is the O wrong from my mapping or just that I pick O and papagayo maps differently.

I do maintain a dictionary of words and their translated values ... so if you change the info in "Position" as requested - then that change would be saved in my dictionary and would always be what YOU wanted in the future.

I'll look into allowing you to modifying the complete map --- so you can add/remove/modify as you wish.

Joe