Results 1 to 7 of 7

Thread: Automatically generate timestamps for music / lyrics

  1. #1
    Join Date
    Jan 2017
    Posts
    46
    Post Thanks / Like

    Default Automatically generate timestamps for music / lyrics

    Hello everyone this is my first post so please be kind...

    I'm new to the scene and I have been playing around with xLights and I really admire the work people have done with sequencing / lipsyncing, I just dont have the patience!

    Anyway I thought I would look into ways of automating or simplifying some of the work and I came up with the following script that uses google voice recognition to find phrases in a music file.

    There are some limitations, google does not return time stamps of where the text is in comparison with the audio (there are other libraries that do, but I could not get them to work)
    so i have to chunk the audio into segments, but hopefully this is enough to approximate the locations of the phrases to reduce the amount of work needed to be done. This of course causes issues with some mis-detected words when they are cut in half (see 25-30 and 30-35 below), again probably doesnt matter much for faces/lipsyncing

    Still a WIP and any suggestions will be welcome.

    To use please install:
    Code:
    pip install SpeechRecognition
    Output as follows:
    Code:
    File is 242 secs long
    From 0 to 5 => 
    From 5 to 10 => 
    From 10 to 15 => 
    From 15 to 20 => 
    From 20 to 25 => glows white on the mountain tonight not a footprint to
    From 25 to 30 => violation
    From 30 to 35 => McQueen
    From 35 to 40 => is howling like this swirling storm inside
    From 40 to 45 => heaven knows I
    From 45 to 50 => 
    From 50 to 55 => let them see be the good girl you always have to pee
    From 55 to 60 => seal don't feel don't let them know
    From 60 to 65 => well now they know
    From 65 to 70 => Let It Go Let It Go


    Code as follows:
    Code:
    import speech_recognition as sr
    from os import path
    
    AUDIO_FILE = path.join("/Users/home/Documents/Xlights/Let It Go.wav")
    
    def getAudioLength(recognizer, file_name):
        """docstring for getAudioLength in secs"""
        with sr.AudioFile(file_name) as source:
            audio = r.record(source)
        
        # length = no. rawsamples / (sample_size * sample_rate)
        return len(audio.frame_data)/(audio.sample_rate*audio.sample_width)
    
    def getAudioSegment(recognizer, file_name, offset, duration):
        """docstring for getAudioSegment"""
        with sr.AudioFile(file_name) as source:
            audio = recognizer.record(source,offset=offset,duration=duration)
        
        return audio
    
    def recognize(recognizer, audio):
        """docstring for recognize"""
        # recognize speech using Google Speech Recognition
        try:
            return r.recognize_google(audio)
        except sr.UnknownValueError:
            return ""
        except sr.RequestError as e:
            print("Could not request results from Google Speech Recognition service; {0}".format(e))
    
     
    # use the audio file as the audio source
    r = sr.Recognizer()
    
    
    lenght_s = getAudioLength(r,AUDIO_FILE)
    print("File is " + str(lenght_s) + " secs long")
    
    
    step_dur=5
    for offset in range(0,lenght_s,step_dur):
        audio = getAudioSegment(r,AUDIO_FILE,offset,step_dur)
        print("From " + str(offset) + " to " + str(offset+step_dur) + " => " + recognize(r,audio))

  2. Likes djulien liked this post
  3. #2
    Join Date
    Jan 2008
    Location
    Fountain Valley, CA (Orange County, So. Cal)
    Posts
    2,144
    Post Thanks / Like

    Default Re: Automatically generate timestamps for music / lyrics

    Great info. Thanks for sharing!

    Are there any dependencies on which version of Python is required?

    don
    Click for display details >>
    web site: http://www.eShepherdsOfLight.com or http://www.facebook.com/eShepherdsOfLight
    technical articles: http://downloads.eshepherdsoflight.com

  4. #3
    Join Date
    Jan 2017
    Posts
    46
    Post Thanks / Like

    Default Re: Automatically generate timestamps for music / lyrics

    I dont think its dependant on python version just in case you have issues i'm using Python 2.7.10

    I actually found a better (but non-free) way of doing this, when I get a chance make something useful I'll post another reply.

  5. #4
    Join Date
    Jan 2017
    Posts
    46
    Post Thanks / Like

    Default Re: Automatically generate timestamps for music / lyrics

    Just found this project. Will try it out tomorrow and post back with results

    https://github.com/readbeyond/aeneas


    Sent from my iPhone using Tapatalk

  6. #5
    Join Date
    Oct 2014
    Location
    Sauk City, WI USA
    Posts
    557
    Post Thanks / Like

    Default Re: Automatically generate timestamps for music / lyrics

    This is cool first step in an idea I had from when I started. I too am inpatient but I don't have the knowledge to figure this out. Thanks for working on this.

  7. #6
    Join Date
    Nov 2014
    Location
    New Jersey
    Posts
    84
    Post Thanks / Like

    Default Re: Automatically generate timestamps for music / lyrics

    This would save so much time.

  8. #7
    Join Date
    Feb 2009
    Location
    Plymouth, MN
    Posts
    8,752
    Post Thanks / Like

    Default Re: Automatically generate timestamps for music / lyrics

    It would have to be done for every version of a song, because the timing is dependent on the overall duration of the song, how much of a leader or introduction the song has, whether the song was recorded the same -- no two recordings of a song may be the same or use the same timing. For example, a conductor's understanding of what "allegro" means for a tempo is quite arbitrary and subject to personal interpretation.

    Same thing, by the way, with pitch. Yes, there are absolute pitch definitions but sometimes a recording may be 1/4 tone under or over pitch depending on how it sounds to the producer's ear. Or a piece may be re-scored and transposed to a different key signature than the original.

    That's the beauty of music -- it's open to so many varied interpretations. It's not exact!

    Nonetheless, this is an interesting way to analyze a song. Nice work!
    http://digwdf.com

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •