Intensive listening/reading software

[neverending]

(This is a speculative thread about software which I don't think already exists, so there are no links!)

I'm currently using the listening-reading method for most of my learning, with most of my time spent on the listening. To be more specific, I've been intensively listening to stories in normal-speed Chinese along with a transcript. Now, for this intensive listening I use the desktop application Audacity, as it is super-useful to zoom in and see the waveform - any phrase I miss, I can easily pick out visually and loop, and then after looping, easily continue on with the rest of the track. It's about 100x more useful/faster than an ordinary media player's slider, or a "go back 5/10 seconds" option. I usually have the transcript on paper, so I don't need to keep switching between apps on my computer.

However, there are two problems I've found with this method:

1. It's impossible to do intensive listening effectively without my desktop/laptop. I haven't been able to find any mobile media player near as useful as audacity for setting up loops, skipping to different places, etc... They're all aimed at playing music, really
   
2. It can be a right pain to find the right place in the transcript when I need to look up a character - I don't look at the transcript as I'm listening, but only to confirm when I think I've misheard something/encountered a new word, so for longer texts it's hard to find where I am in the audio
   

So, it occurred to me that for maximum efficiency of my learning methods, I would really want two things:

• A mobile app that allows navigation of a sound file as easily as audacity (preferably by showing the waveform)
  
• An app, desktop or mobile, that shows the transcript on-screen, automatically aligning it with the audio as it is spoken. (This would use an artificial intelligence algorithm similar to those used in speech recognition. According to my brief literature review, it's actually possible to do, and it is an easier problem to solve than full speech recognition.). Indeed, such an app could also allow navigation of the audio file *via* the transcript - you would select the sentences/phrases you want to loop from the transcript, and then automatically hear the associated audio.
  

So, my question is, has anyone heard of either such app? If you knew where to find such apps, would you find them useful for your Chinese studies at all? Are there any other pie-in-the-sky ideas that people have for their ideal language learning app? I'm contemplating writing an app along the lines of the above ideas (especially the first one, as it has the best work/reward ratio) but I'd like to see if anyone else would be interested in this sort of thing.

[imron]

I'd toyed with the idea of doing something similar to option 2, but hadn't looked at it too deeply. Do you have any links to relevant literature? (did a quick google search and found a few things, but would be interested to see if you had anything particular that was relevant).

[neverending]

@imron: do you have a university/research org Internet connection? If so, google scholar turns up some interesting reads: http://scholar.googl...1,5&as_subj=eng . I can also pick out some more pertinent papers tomorrow.

In any case, if I were to implement this it'd be no small project, which is why an iphone waveform viewer would probably be my first project.

Edit: scratch all that. I found a toolkit called SailAlign that apparently already does all the heavy lifting to align text/speech. So, presuming that toolkit works as intended, the only work required would be to design a decent GUI around it. Perhaps worth a look.

[imron]

do you have a university/research org Internet connection?

Unfortunately not. A regular Google search turned up some interesting papers though.

The first one would definitely be easier to do. Are you a developer?

[imron]

The SailAlign website appears to be completely blank when I visit it (under 2 different browsers).

[scoff]

Here is a working link to the author's page on SailAlign:

http://sipi.usc.edu/~nkatsam/nassos/sail_align.html

[laurenth]

http://fr.androidzoo...order_ozsp.html

This program for Android is supposed to have the following features:

- pitch bend

- speed bend

- reverse

- loop

- MP3 export

- WAV export

- SoundCloud Export

- waveform edition and generation

I haven't tried it, though and the comments left by people who have aren't very impressive.

[laurenth]

Or maybe this:

http://fr.androidzoo...ayer_btijh.html

"

WaveLoop is an audio player that teaches the user foreign languages through pre-recorded sound files.

WaveLoop allows you to listen to mp3/wav files containing study material from exams like the TOEIC LC, TOEFL LC, JLPT, JPT, etc.

Features

1. Automatically extract sentences from audio files.

2. Intutively explore your audio with waveform scroll.

3. Freely move forward and backwards between sentences.

4. Repeat sentences and sections when needed.

5. Control the speed of audio playback.

6. Bookmark sentences and make notes."

Looks promising, might be worth a download...

Edit again: I've downloaded and briefly tried this one:

http://fr.androidzoom.com/android_applications/music_and_audio/tapemachine-lite-recorder_gjrj.html

It seems to do at least the audio part of what you are looking for (waveform, loop). You'd still need to switch app or to use paper to read the transcript, though.

Edited April 13, 2012 at 08:11 AM by laurenth

[neverending]

@laurenth: thanks! Apparently my google-fu skills were lacking earlier, but if one searches for clip editor or sound recorder, it seems that there are lots of waveform viewers that can be used for this. None are quite as handy as audacity, but they're better than the default media players.

@scoff: thanks for the link - that's useful

@imron: I've been a professional developer in the past, but I do AI research nowadays, so I've been looking for a spare-time project to refresh my coding skills. If I can get my hands on SailAlign, and can get a Chinese-trained ASR engine to plug into it, I'll have a bash at doing a desktop app for this, considering there are already a few waveform viewers for mobile platforms.

[c_redman]

Regarding #2, there may be two possibilities matching what you're looking for. The screenshots for each page should indicate well whether it would help you.

Transcriber

Transana (can also do videos)

As for matching the text to the audio, you need to do that part manually.