I'm actually mid process of making an "AI first" sequencer right now.
https://www.youtube.com/watch?v=JNXxdOwjwLc
Upload song > Music is analyzed and broken into section (Includes BPM, Tempo, Theme of music, etc) > Lightforge gives a custom description of your channels/groups, and the song which has been analyzed, to upload to an LLM of your choice > LLM provides a JSON file back to upload back into LightForge which contains your entire sequence.
I've been using Opus 5.5 for the songs in the demo video, and it does a great job at understanding the channel groups. Any version of Chat has been lack luster and only focused on maybe two elements at a time.
The songs in the video, from finding the link on youtube, to clicking play within LightForge takes roughly 2.5 minutes. I have not done any manually tweaks to the sequence.
Is there AI that can do timing tracks for singers singing gobbledygook? All the AI I've seen, you have to tell it the language. My singers don't have a language. They're singing gibberish.
OK so I went and did a thing...
I had Claude Code generate a sequence from scratch. I gave it an empty LOR file with a Beat Wizard track and tapped lyrics. I gave it the audio file, and a text "map" of the song ("starts quiet with piano until 0:xx, vocals come in..." etc.
It came up with an overal plan, and we sequenced it section by section.
There was a lot of back and forth, with me telling it to change things and in total we went through 28 revisions. Probably about 3-4 hours of my time.
I have not done much hand editing yet as I wanted to show what a strictly generated sequence might look like. There are some things I might still change if I end up using this one.
Let me know what you think. Note the song starts very quiet but the energy level hits a surprising level partway through so stick with it.
https://www.youtube.com/watch?v=E0Lr_yTzKq8
Nice! It’s definitely getting close, I would talk to it about smoothing out the effects in the slow sections of the song. I noticed specifically the mini trees doing the repeated fill upward effect and snapping off after each one. When I was doing this experiment, I noticed a tendency of the AI where it would just take a somewhat clunky effect and repeat it over and over. To fix it I started by telling the AI about the specific element where this was happening and just telling it to be more creative with that element in that section. Once it comes up with something more interesting then you can give it more feedback to refine it. Other than that I would tell it to check the timing on some of the big hits of the song to make sure everything lines up.
