andreas.niedermair.name

How can I record audio within a Silverlight 4 application?

Attention/Danger/Hazard

With Silverlight 5 some new functionality has been added. Currently I am asking for some more information @ StackOverflow to maybe improve my solution.

My use-case: Record audio from a phone-call
Technology to use: Silverlight 4 for client, WCF/C# 3.5 for server
Hardware: ACS Recording Jack

I will not post the full code here, rather a description which I've replied to a guy who asked me for some details, as I've posted @ StackOverflow

Converting PCM to WAV can be done with pure C# – example
The problem here is that you have to record a complete PCM-stream, because the total length is included in the WAV-header:
bwOutput.Write((uint)rawData.Length);

Yeti is not a pure “managed code“-solution – rather it’s an interface which uses P/Invoke (see lame64 implentation and lame86 implentation for more details).

I am recording an audio-stream in silverlight – pure PCM. Then I’ve downloaded g711audio and did a port to a silverlight assembly (which is quite simple: just create a new silverlight-assembly-project and add all the files as linked-items to the new solution) – actually I’ve uploaded my solution to github (just compile it and you can use it on the server and client – maybe you need to do another port to a windows-phone-sepcific assembly)
With G711 I can compress my PCM-stream realtime to 50%. After recording is finished, I use a WCF-Service which takes a byte-array to upload the compressed PCM-Stream (not really the best solution here, because I cannot use streaming here as I need additional information – such as identifier, fileName, ...), then decompress it on the server, use NAudio to convert the PCM-stream to a mp3-compatible wav-stream (I record in following format: Mono, 8.000 Samples per Second and 16 Bits per Sample – which gives me a compressed datarate of about 64kbits – but for mp3-conversion I need at least Mono, 16.000 Samples per Second and 16 Bits per Sample – do not try to use 8 Bits per Sample, because the result is very bad).
Then I use yeti to convert the wav-file to a mp3-file.

Another solution, which I’ve evaluated as not really practicable in my scenario:
Whenever OnSamples() is hit in the audioSink, you could upload the (compressed) chunk with a WCF-service to the server – important: use an iterator-variable (aka i++) here, to identify the order of the chunks. When recording is completed, you could invoke a CompleteSession on the server, which merges all the chunks and then does the stuff from above.

Critical here: As I was not able (based on technology aspects) to convert to mp3 on the client and therefore my scenario depends on a server. So I’ve assigned quite a bunch of tasks to the server, which would be horrifying to the client (based on performance aspects).

If you need more information on how to use System.IO.Stream with WCF, there's a msdn-entry available.
For our german-speaking friends we have a nice blog-entry from matze

Here is how I do the resampling and conversion:

using System;
using System.IO;
using NAudio.Wave;
using Yeti.Lame;
using Yeti.MMedia.Mp3;

internal static class AudioUtils
{
	private const uint Mp3SampleRate = 24;
	private const int Mp3Rate = 16000;
	private const int Mp3Bits = 16;
	private const int BufferSize = 4096;

	internal static bool ResampleWavFile(string wavFullPath, string resampledWavFullPath)
	{
		FileStream sourceFileStream;
		// open wav-file here (wavFullPath)

		using (sourceFileStream)
		{
			using (var sourceWaveFileReader = new WaveFileReader(sourceFileStream))
			{
				var sourceWaveFormat = sourceWaveFileReader.WaveFormat;
				var targetWaveFormat = new WaveFormat(Mp3Rate, Mp3Bits, sourceWaveFormat.Channels);

				if (sourceWaveFormat == targetWaveFormat)
				{
					return false;
				}

				using (var sourceWaveFormatConversionStream = new WaveFormatConversionStream(targetWaveFormat, sourceWaveFileReader))
				{
					using (var sourceBlockAlignReductionStream = new BlockAlignReductionStream(sourceWaveFormatConversionStream))
					{
						FileStream targetFileStream;
						// create wav-file here (resampledWavFullPath)

						using (targetFileStream)
						{
							using (var targetWaveFileWriter = new WaveFileWriter(targetFileStream, targetWaveFormat))
							{
								var buffer = new byte[BufferSize];
								int read;
								while ((read = sourceBlockAlignReductionStream.Read(buffer, 0, buffer.Length)) > 0)
								{
									targetWaveFileWriter.Write(buffer, 0, read);
								}
							}
						}
					}
				}
			}
		}

		return true;
	}

	internal static bool TryConvertWavFileToMp3File(string wavFullPath, string mp3FullPath)
	{
		FileStream wavReadFileStream;
		// open wav-file here (wavFullPath - actually resampledWavFullPath from above)

		using (wavReadFileStream)
		{
			var wavWaveStream = new WaveLib.WaveStream(wavReadFileStream);
			var waveFormat = wavWaveStream.Format;
			var beConfig = new BE_CONFIG(waveFormat, Mp3SampleRate);
			var mp3WriterConfig = new Mp3WriterConfig(waveFormat, beConfig);

			FileStream mp3WriteFileStream;
			// open mp3-file here (mp3FullPath)

			using (mp3WriteFileStream)
			{
				Mp3Writer mp3Writer;
				try
				{
					mp3Writer = new Mp3Writer(mp3WriteFileStream, mp3WriterConfig);
				}
				catch (Exception ex)
				{
					// TODO log the exception
					return false;
				}
				try
				{
					var buffer = new byte[mp3Writer.OptimalBufferSize];
					int read;
					while ((read = wavReadFileStream.Read(buffer, 0, buffer.Length)) > 0)
					{
						mp3Writer.Write(buffer, 0, read);
					}
				}
				catch (Exception ex)
				{
					// TODO log th exception
				}
				mp3Writer.Close();
			}
		}

		return true;
	}
}

Actually you do not need to write files for everything - you could also work with System.IO.MemoryStream (which would be faster, as you do not have any significant latency in your I/O). The reason why I prefer files is that I need a lot more debug-information in this stage - and actually disk space ain't cost-a-lot ...