Recording Fail

I was very excited about my recording set up from my first research trip up to the Chicago area. Turns out, it was an epic fail. Having a “room recording” microphone like the one I described in my last post is fine for just capturing reference audio, but the quality of the recording is NOT good enough to be used for automated transcription.

As I was messing around with the tools and software for this stage of the research, I did some quick math. It took me about 2 hours to transcribe about 30 minutes of audio. If I have 50 interviews by the time I’m done, and each interview is 90 minutes, that’s 75 hours of interviews. As each hour of interview takes about four hours to transcribe (even using a USB transcription foot pedal and typing around 85 wpm). So that’s about 300 hours of work just transcribing the interviews. Let’s say I can somehow do this for 3 hours each day before my fingers rebel and my brain fries (and knowing I have other responsibilities, too); that puts me at 100 days of work. I am still a pastor and weekends are dedicated to my family (Saturdays) and the Church (Sundays). So I’m working on this 5 days per week. That puts me JUST transcribing… not doing ANYTHING else on my dissertation… for… 25 weeks. That’s 6 months of transcribing, and that’s if I never miss a day for a funeral or a crisis counseling appointment or to invest in a sermon that’s not quite coming together, or responding to a world-wide pandemic as the coronavirus spreads through community after community.

So should I pay for transcription? Not if the estimates I see online are any indication. The estimates say that professional transcribers can work at double my speed (1 hour of audio = 2 hours to transcribe). The median rate of pay for transcription services is about $16-25 hour. So for 75 hours of audio, or 150 hours of transcription work, I’d be looking at $2,400 to $3,750. And that’s if all the interviews come in at 90 minutes. If I go over, it’s even more. I do not have thousands of dollars to pay a transcriptionist.

Cue the majestic, cinematic swell: Enter Amazon Web Services. I can upload an mp3 file to S3, and use their AI-based transcription service. The cost? For 90 minutes of transcribed audio (drumroll, please): $2.16. That’s it. This means the total cost for 50 interviews, at 90 minutes each (thus 75 hours of interview) is $122.40. Now THAT I can handle. I’ll easily pay a hundred bucks to save me six months of work or a couple of thousand dollars. But does it work?

Here’s the hitch. AWS transcription works phenomenally well… if you have a HIGH QUALITY audio source file. The audio from my Blue multi-capsule USB mic was decidedly low quality. The mic sat too far away from my subject; when the subject’s voice dropped, AWS couldn’t transcribe it. My recording yielded an accuracy of less than 40 %. It takes more time to fix a 40% accuracy transcription than it does to just transcribe it the old fashioned way with my foot pedal.

So I tried an experiment. I “re-recorded” the interview. I played the audio on my iPhone through headphones, so I could just repeat everything I heard back into my MacBook Pro through the Blue mic… but this time, the mic was right in front of my face and I took care to speak with clear diction. I did a 5 minute test file, and uploaded it to my AWS S3 storage. From there, I submitted it to the AWS transcription algorithms. And what came back was something close to 98% accuracy — and where it wasn’t accurate, I could easily tell what the correct words should have been.

So I re-recorded an entire 2 hour interview with the Lead Pastor at this first church I visited in Chicago. Yes, this was a time-heavy investment. But I sped up the audio on my iPhone, so I got the whole interview re-recorded in just over 2 hours. Then I sent it to Amazon, and then I tidied up the transcript that AWS provided. Yes, it still took about 30 minutes to clean up the transcript and to format it. But that’s nowhere near the kind of brain-numbing finger-killing work that transcribing it myself would be.

So the key to everything?

High quality recordings. From the outset. Whatever it takes to get amazingly clear audio recordings of the interviews. It’s worth significant investment in equipment if it will save me thousands of dollars of transcription fees and hundreds of days of work. Stay tuned. I’m off to research some new toys…

Ready, Set, Go (again)!

So I have received mostly official positive responses from our Canadian seminary friends! Regent College (Vancouver, BC) has agreed to promote the study on their alumni website. I will be interested to see if this garners any traffic, but I’m grateful that they’re willing to participate. Ambrose Seminary has agreed to the study, so now we’re working on the mechanics of reaching their alumni in an effective way. And Tyndale has now agreed to join the study as well! I’m so grateful for the trust and partnership of these schools. I think I’ve lost a few connections along the way, and I’m sad about that. But I’m excited to move forward with the schools that have agreed to participate.

So MailChimp once again becomes my best friend. The Google Forms are ready to receive responses. The backend databases are prepped to capture the data. And SPSS is hungry for more rows as we approach statistical significance! Let’s go, Canada!

O Canada! (update)

I have now been in direct contact with people at McMaster Divinity School, Ambrose Seminary, and Regent College that have actually expressed interest in participating in this study! It looks like my timing isn’t ideal, as many of them are either on vacation or sabbatical across these summer months, but I have been encouraged none-the-less by the willingness to even have a conversation about what participation might look like. The only school on my list I haven’t heard back from yet is Tyndale Seminary… I was enrolled at Tyndale and ready to start my seminary journey back in the day before receiving a job offer to teach high school science in Ottawa. I chose instead to teach for a few years to pay off some student loans (and to get married!), and ended up a few years later at Gordon-Conwell Theological Seminary. But I’ve always felt a connection to Tyndale. I reached out to a few more people from there today, so we’ll see if anything comes of those attempts.

But if everything goes forward according to the master plan (which, of course, it ALWAYS does, right?) I will indeed have some Canadian representation in the data!

Way to Go, Truett Alumni!

So I tend to choose terrible times to send my survey invitation emails. I sent out the invitation to Northern Seminary alumni right before Thanksgiving weekend. And, having learned nothing from that experience, I sent out the survey invitation to Truett alumni on the July 4 weekend. I’m amazing.

And yet…

Truett alumni are crushing it. It hasn’t even been 24 hours, and 81 people have clicked through to the survey, and 37 people have already completed the entire survey through to its end! It’s kind of fun for me to log in and check on how many respondents I’ve got… kind of like a telethon or something, watching the big number on the screen go up and up.

Not even 24 hours since the survey invitation was sent, creeping up on 40 respondents. Keep going!

So just a quick word of thanks to all those who have taken the time this weekend, with all the festivities and everything else going on, to support research on preaching. I honestly appreciate your time and your willingness to be a part of this study. Thanks so much!

O Canada!

So I’ve spent this week remembering that I’m Canadian. My interruption of studies from LST officially ended on July 1, 2019, which is the date commemorating the signing of the British North America Act, making Canada an independent dominion and establishing it as a country of its own. And here I am, a Canadian studying at a British theological school. The partnership continues!

So I spent the day reaching out to some Canadian seminaries in the hopes of broadening my data set beyond just American schools. Some of them emailed back right away… as in, the President of the school emailed me back within 10 minutes. I was humbled and impressed!

Regardless, I am hopeful that this research project will gain some traction up in the True North Strong And Free. So as I’m writing this post on July 4th, Independence Day here in the United States, I am grateful for a country that has welcomed me and allowed me to invest here as a pastor and a resident. And I am also grateful for my Canadian roots, especially as I spent the day trying to invite Canadian representation into my dataset!

Gordon-Conwell Alumni Are Amazing.

So we're one week into collecting data.  The first population I've reached out to is the Gordon-Conwell Alumni community.  You are all amazing.  According to Mailchimp, the average "Open" rate for emails sent to religion-based mailing lists is 30.9%.  Gordon-Conwell alumni just blew that away at 55.3% just opening the email to see what it's about.  But beyond just opening an email, getting someone to actually click on a link in that email is even harder.  The industry average, again in the category for religion, is a mere 3.2%.  Gordon-Conwell alumni have clicked through (either to the PreachingTeams.com website or to the survey itself) at a rate of 17.6 percent... after just one week.  That's more than 5.5x the industry average!

What can we take away from these numbers?  Nothing, really, with any level of certainty or credibility.  But if I were to speculate?  I'd say that Gordon-Conwell alumni love the church, care deeply about the preaching of God's Word, and are interested in research that furthers our understanding of preaching and improves our practice of preaching.

As of today, a full 10% of people who received my invitation have clicked through AND filled out the survey and submitted it.  I am so thankful for the responses I have received so far!  Thank you for taking the time to participate in this research.  You've encouraged this researcher and fired me up to keep this project moving forward!

Tim.

It's Alive!

As of Wednesday, July 11, 2018, this research project is live!  The first survey has been sent to over 650 Gordon-Conwell alumni, and within the first hour we are up to 20 responses!  If you're one of those first 20 people, thank you!  Thanks for caring enough about preaching and the local church to weigh in on this topic!  

A second request will go out in two weeks, followed by a final request in August before I shut down phase 1 of the research project.  But I'm already excited to see how many people have been collaborating in some form or another in the preaching ministries of their various churches.

Thanks again for the encouraging initial response!