Shimojima in the audio jungle part 2

Hey guys,

Continuing some BTS. If you missed Part 1, you can find it here.

1.  JUN 2023 - JUL 2023: SOUND EDITING TESTING
   
   I've worked as a video editor for a long time — specifically if you count my amateur high school days on Adobe Premiere 6.5, before it became Adobe Premiere Pro and took over the market after a detour to Final Cut Pro 7.
   
   When you’re editing video, you’re editing the audio that goes with it. You’re often adding the music and sound design. You’re just not necessarily an expert at it, or the best organizer, because when there’s a robust team, your temporary sound work will be passed off to a pro sound engineer to be augmented, supplemented, replaced, and mixed better.
   
   To this day, I’m surprised how many of us video editors are guilty of making an incoherent mush of the soundscape, blasting too many layers. I get it. We’re working fast, and to be honest, the creatives, agencies, and clients — like the average person — mostly just want the overall impact; they care less about getting just the right layers assembled with technical nuance.
   
   This time would be different. With this Project being entirely audio, audio editing would be the opportunity to craft the story. I knew I’d have to do it myself. And use more carefully chosen sounds — not temp ones, but ones meant to be final — to get a closer approximation of the final story immediately in front of me to react to.
   
   This was a practical choice too. With a limited budget, I didn’t want to do a rough edit of the full show, only to make a sound professional waste time rebuilding much of it.
   

2. You’re not going to do your edit in Adobe Premiere or DaVinci Resolve. You’re going to pick your DAW. Because I have Adobe Creative Cloud already, I started Episode 1 in Adobe Audition.
   
   There are some functionalities I like — easy for newbies to use. There are templates that apply EQ, compression, and reverb to different degrees, to quickly emulate different distances. This proved very helpful.
   
   But after 2 weeks of learning my way through and putting the episode together with nice effects, I realized my process was lacking. I wasn’t making use of buses and sends, and other complex organizational things. I wanted to do this properly. I wanted this project to feel like a movie, and specifically an intricate one… with audio montages… with fluctuating different spatial reverbs. From working with mixers over the years, I had the sense that Pro Tools is the industry standard for working on film. I’d likely be hiring a pro to complete the mix, and it’d be likely that pro would be using Pro Tools. In order to best collaborate and pass the session back and forth, I really needed to be using it too.
   
   I got the monthly subscription, and I spent a week translating Ep 1 from old session to new session. If you’re trying to slow down a clip by a lot, Audition gets it cleaner. There are other things that Pro Tools doesn’t build in, forcing you to get additional plug-ins… but so it goes.
   

3. JUL 2023 - JAN 2024: FULL-ON EDITING
   
   You feel naked at first.
   
   Without footage to guide what you’re doing, you’re kind of lost.
   How long does this pause hold for?
   When should the next line of dialogue come in?
   Should I pull a few words from a faster take?
   How long should this character walk?
   How long should the scene be?
   
   You can manipulate all of this independently, because you don’t have to match it to picture. You can control it down to microscopic dimensions. Video moves by frames. But audio is more fluid, adjustable to such a small degree that sometimes you play back your change and it doesn’t sound any different.
   
   Eventually, you realize it’s freeing. But you have to close your eyes, or look away from the screen, so that you can imagine the visuals in your mind, and use that as your guide.
   
   And you have to learn a bunch of new keyboard shortcuts.
   

4. I had an existing “editor’s” library of sound fx gathered, but that was a fraction of what I needed.
   
   Soundsnap became my friend.
   
   As did rips of YouTube uploads, which though somewhat compressed in quality, can sometimes get you closer to the sound you have in mind.
   
   Since I now had a good mic, I also recorded some sounds.
   
   Later, I was introduced to Soundly. One cool feature is you can highlight portions of a clip you need and insert it directly into Pro Tools. But I preferred instead to copy everything into organized folders, so I’d have a record for later use. Not the best naming structure you see there, but good enough. Time management is king. You really can take years on a project, and that would suck. Get through the important stuff. Finish the edit.

   
   None of these are perfectly extensive libraries. You have to look all over. I recommend these custom-made collections.
   

5. MAY - JUL 2023: MUSIC RESEARCH
   
   Around the time I started looking into AI voices, I was also looking into AI music. The big issue with the generative sites available was that they weren’t creating high-fi human sounding material.
   
   I did find Evoke, which functions instead like many traditional music libraries, is cheap (and had a free trial period), and gives pretty good quality tracks, along with instrument stems. The site and search is slow though, which they’ll hopefully address.
   
   And because their library is relatively small and my track needs were vast and eclectic, I still scoured from traditional libraries. PremiumBeat. Epidemic. Soundstripe. MusicBed. Artlist. Marmoset. Some of these offer a monthly subscription, so to save, I subscribed at the end of production, working with temp watermarked versions until that point.
   

6.  AUG 2023 - FEB 2024: ORIGINAL MUSIC
   
   I also needed a cultish humming to be a recurring motif throughout the series. I had an idea what the theme would be, so I needed someone to take that and come up with a bunch of effected variations and extensions with sound design and synth.
   
   Felix Lindsell-Hales, based in Cardiff, was able to work loosely with me over several months and even enlisted a choir for a few moments. And he created a few additional cues where my music library picks weren’t cutting it. He also leads the band The Year of the Dog.
   

7. I thought we might also need to compose sonic mnemonics to represent characters and locations. My fear was that without visuals, it’d be harder to communicate recurring locations or get characters lodged into the heads of casual listeners.
   
   What I found is that in most instances, changes in background sounds were enough to designate scenes. And our main character voices were all distinct enough on their own. At other times, the Narrator filled us in on any info we really needed. I did, however, create sound cues for a few recurring actions and beats. This was more to communicate pervasiveness — and for my sense of humor — rather than clarity.
   

8. NOV 2023 - FEB 2024: MIX
   
   I would’ve loved to have mixed this with great dynamic range. Even with Dolby Atmos. Maybe we’ll come back to that, like if we can get funding for a theatrical screening or in-person exhibit. Could be cool. Everyone sitting in total darkness together while the audio movie plays.
   
   But as it stands, people will play this while commuting, or working out. The audience that needs Dolby Atmos is even more niche than this audience already is. Not enough benefit outweighing the cost.
   
   Charlie Braham was able to put time to this mix. He’s also in the UK, and though long-form projects usually benefit from in-person working sessions, we did everything through emails (except for spotting sessions over Google Meets). A mixer is going to EQ the elements and work with the levels and reverbs and effects and supplement some sounds and balance it all — I was relieved to have a pro with real technical knowhow in this.

   
   One thing we had to pay attention to:
   making sure the dialogue felt embedded in the space. A lot of audio fiction gets away with a more “literary” sound booth sound, but I wanted this to seem like it was recorded like a film, or in some cases like we’re eavesdropping on a real imperfect moment. It wasn’t just about adding proper reverb; we took away more of the low-end in the voices and raised the ambience more than is typical.
   
   Charlie started on this while I was halfway through editing. I’d pass him new episodes as soon as I got through them.

9. FEB 2024: AI VISUALS
   
   In an ideal world, I would’ve jumped into this sooner along with all marketing aspects. In an ideal world, we would’ve had a team of people jumping on it sooner, so as not to get sidetracked from the actual production.
   
   Not that it takes up that much time, compared to the traditional visual route. But it does still take a lot of time, and you can get really sucked into it, trying prompts over and over, because the damn AI still won’t understand you the way a person can.
    

   But it seems like every week there’s a new update. A year ago, it didn’t look up to snuff, and then half a year ago — with Chris Raleigh repeatedly bringing it to my attention — the possibilities began to grow. He was making things like this. He’s eager to find the next frontier.
   
   We looked at what Dave Clark’s doing: getting good press for showcasing the potential in commercials, trailers, and films. He just met with Ridley Scott.
   
   Now Sora is looming. And Tyler Perry halted his planned studio because of it.
   
   The main thing you should know is: AI will be increasingly used, whether we like it or not. It just means your tasks and toolsets must shift. In many cases, we’ll be getting things done faster. We’ll be the creative lead of our personal studio.
   

10. I churned out some project art. Each one of these might’ve taken days and lots of money to be staged and photographed or painted or digitally-drawn, rather than the several hours it took me to direct Midjourney to do it. (Logo was done the old-fashioned way, as Midjourney couldn’t handle the specifics well.)
    

11. The trailer was next, which I’ll share next time.
    Though you could have AI write and structure this for you, it may not be exactly what you want. If you have a specific idea, then you really should script it yourself. Not to be a traditionalist, but because thinking through these things yourself helps you better know your own story. You’re probably already doing so in your head. Just get it down on a good ol’ word doc or spreadsheet. List and organize the visuals that will tease the story.
    
    Then, start prompting for those images in Midjourney. You have to understand the clear strengths and limitations. With the basic available tools, you’re not going to easily create a dialogue exchange, or action shots, or multiple specific angles of a scene. You’ll mostly rely on tableaus, or specific moments distilled to whatever point you want to make.
    
    Raleigh helped put these images in motion with Runway and Stable Diffusion. Things move in surreal ways. Think of all these as features, not bugs.

    
    I started wondering: in addition to this being a trailer for the audio drama, what if it could also tease the possibilities for the feature film? Hollywood is already demanding such AI-generated pitch videos, but I mean actually incorporating AI visuals into the film in a noticeable way, to get more interesting styles, and to save on VFX costs or even shoot days. Again, Sora will make a big step here. I don’t think Dave Clark’s short was an effective example of this, or that interesting in its own right, but it points to possibilities. Think of how some live actions films suddenly cut to a graphics shot or sequence, to call attention to those moments, or to communicate a different perspective or heightened awareness.
    
    Also, this thing is great for what it is.
    
    Some people want to make entire character-driven fiction films with just AI. That’s still a long way off. And to be honest, if it does happen, just think of it like another modality, like animation. It doesn’t mean we’ll want to stop seeing real people performing — by which I don’t just mean Cruise jumping off a cliff, but also people expressing themselves.
    

12. MAR 2024: AND SO NOW WHAT?
    
    Well, very soon, there’s going to be nothing more I can do aside from release this. There’s a little anxiety with any creative output, but that’s compounded here by not having my feet on firm ground. I just don’t know.
    
    What is the path for an audio project? With a film — though so much is more and more futile — at least there’s a charted road of festivals and distribution. There’s a game plan that’s been talked about ad nauseam everywhere.
    
    For a podcast, what are those festivals that are actually recognizable or beneficial? And how do you compete against a field that is still mostly associated with talk shows, and ones that are ongoing, allowing an audience to build over time?
    
    Will this audience feel that my 3.5 hours are not as satiating as the many many hours of some series? Or my 15-25 min episodes not as satiating as the 35-45 min episodes of others? Everyone loves chastising a filmmaker for not cutting their material, but in this medium, did I perhaps not go verbose or dense enough?
    
    There are podcast networks that have subscriber bases. There are production companies that have an existing audience and the right connects. There are reviewers — surprisingly few compared to film. Overall, the people in this world have been easier to reach than the film world… but not by much.
    
    And the bigger question is, are these forces that important? Does their name recognition really make much of a difference?
    
    And wait, wasn’t the whole point of doing a podcast to be independent of all that?
    
    Is it better to just directly hire your own publicist to seed it out there?
    
    If you do, will it just mean people glance at the media coverage and not actually click forward to the show?
    
    And if they do start to listen, will their attentions perk? Did you consider that they’ll likely be doing other things simultaneously, perhaps with their vision, which is a very formidable sense to compete with? Should you have factored this in and labored over every passage more, pacing the narrative slower, in case those listeners miss something? Or does that risk boredom?
    
    This format may be at even more of a disadvantage, because it can never truly be “big.” We sit to watch the epic Dune: Part Two, knowing that we’ll see the money on screen (and yes, it truly succeeded at that, and on many levels). With audio, we can’t tell how much money was put in, because top quality is achievable by the masses. And that makes it amount to a relative “no big deal.”
    
    Doesn’t that mean that the audience for this will never be as big? Making it more of an acquired taste? If so, as someone that likes swinging big, that’s difficult to reconcile.
    
    I have few good answers to any of this.
    
    What’s scarier is that few people do.
    
    For my feature film Alberto and the Concrete Jungle, I wrote a digital nomad adventurer realizing he will always be one. I see where I partly got that sentiment from.
    
    I am resigned — or in a different light, grateful — to play in the wilderness. This is the lesson I’ve learned over and over for as far back as I can remember, and which is key to my identity. Others will have different key lessons, so don’t take this as advice. But if you’re not careful, if you put yourself in a position where you must rely on others — meaning, you have no plan B to set it in motion yourself — you risk waiting indefinitely. And there are many better things to do than that. Don’t circle endlessly eating your own tail. Instead, whenever the island is lonely and you wish it weren’t so, just focus instead on empowering yourself.
    
    Part 2 ends here. The release comes soon.

Feel free to share any of this with anyone you know who'd like it -- you can send the links -- or forward the email so they have the chance to subscribe.

If you’re seeing my scrapbook for the first time, you can subscribe here to keep getting it.

Chris