Friday, 23 November 2012

Enabling NAudio for Windows 8 Store Apps–First Steps

One of my goals for NAudio 1.7 is to have a version available for Windows Store apps. Obviously there are a lot of classes in NAudio that simply won’t work with Windows Store apps, but I have been pleasantly surprised to discover that the bulk of the WASAPI and Media Foundation APIs are allowed. ACM, and all the rest of the old MME functions (waveIn.., waveOut…) are obviously not available. I’m not entirely sure what the status of DirectX Media Objects is (DMOs), but I suspect they are not available.

The first step was simply to create a Windows Store class library and see how much of the existing code I could move across. Here’s some notes on classes that I couldn’t move across

  • WaveFormatCustomMarshaller - not supported because there is no support for System.Runtime.InteropServices.ICustomMarshaller. This is a bit of a shame, but not a huge loss.
  • ProgressLog and FileAssociations in the Utils folder probably should have been kicked out of the NAudio DLL a long time ago. I’ll mark them as Obsolete
  • Some of the DMO interfaces were marked with System.Security.SuppressUnmanagedCodeSecurity. I can’t remember why I needed to do this. It may be irrelevant if Windows Store apps can’t use DMO. I’ve simply allowed the code to compile by hiding this attribute with #if !NETFX_CORE
  • One really annoying thing is that the Guid constructor has subtly changed, meaning that you can’t pass in unsigned int and shorts. It means that I had to put unchecked casts to short or int on lots of them
  • One apparent oversight is that COMException no longer has an error code property. I guess it might be available in the exception data dictionary. It was only needed for DMO so again it may not matter
  • The ApplicationException class has gone away, so I’ve replaced all instances of it with more appropriate exception types (usually InvalidDataException or ArgumentException)
  • The fact that there is no more GetSafeHandle on wait handles means that I will need to rework the WASAPI code to use CreateEventEx.
  • I’ve not bothered to bring across the Cakewalk drum map or sfz support. Both can probably be obsoleted from NAudio.
  • The AcmMp3FrameDecompressor is not supported, and I suspect that Media Foundation will become the main way to decode MP3s (with the other option being fully managed decoders for which I have a working prototype – watch this space)
  • Encoding.ASCIIEncoding is no longer present. Quite a bit of my code uses it, and I’ve switched to UTF8 for now even though it it is not strictly correct. I’ll probably have to make my own byte encoding utility for legacy file formats. Also Encoding.GetString has lost the overload that takes one parameter.
  • I had some very old code still using ArrayList removing it had some knock-on effects throughout the SoundFont classes (which I suspect very few people actually use).
  • WaveFileChunkReader will have to wait until RiffChunk gets rewritten to not depend on mmioToFourCC
  • Everything in the GUI namespace is WindowsForms and won’t come across
  • The Midi namespace I have left out for now. The classes for the events should move across, and the file reader writer will need reworking for Windows 8 file APIs. I don’t think windows store apps have any support for actual MIDI devices unfortunately.
  • The old Mixer API is not supported at all in Win 8. The WASAPI APIs will give some control over stream volumes.
  • ASIO – I’m assuming ASIO is not supported at all in Windows Store apps
  • The Compression folder has all the ACM stuff. None of this is supported in Windows Store apps.
  • The MmeInterop folder also doesn’t contain anything that is supported in Windows Store apps.
  • SampleProviders - all came across successfully. These are going to be a very important part of NAudio moving forwards
  • MediaFoundation (a new namespace), has come across successfully, and should allow converting MP3, AAC, and WMA to WAV in Windows Store apps. It will also be very useful for regular Windows apps on Vista and above. Expect more features to be added in this area in the near future..
  • WaveInputs – not much of this folder could be ported
    • WasapiCapture - needs rework to not use Thread or WaitHandle. Also I think the way you specify what device to use has changed in Windows Store apps
    • WasapiLoopbackCapture – I don’t know if Windows Store apps are going to support loopback capture, but I will try to see what is possible
    • I may revisit the IWaveIn interface, which I have never really been happy with, and come up with an IRecorder interface in the future,to make it easier to get at the samples as they are recorded (rather than just getting a byte array)
  • WaveOutputs:
    • WasapiOut – should work in Windows Store, but because it uses Thread and EventWaitHandle it needs some reworking
    • AsioOut, WaveOut, WaveOutEvent, DirectSoundOut  - not supported. For Windows Store apps, it will either be WasapiOut or possibly a new output device depending on what I find in the Windows RT API reference.
    • AiffFileWriter, CueWaveFileWriter , WaveFileWriter – all the classes that can write audio files need to be reworked as you can’t use FileStreams in Windows Store. I need to find a good approach to this that doesn’t require the Windows Store and regular .NET code to completely diverge. Suggestions welcome.
  • WaveProviders – mostly came across with a few exceptions:
    • MixingWaveProvider32 - used unsafe code, MixingSampleProvider should be preferred anyway
    • WaveRecorder – relies on WaveFileWriter which needs rework
  • WaveStream – lots of classes in this folder will need reworking for
    • WaveFileReader, AiffFileReader, AudioFileReader, CueWaveFileReader  all need to support Windows Store file APIs
    • Mp3FileReader – may be less important now we have MediaFoundationReader, but it still can be useful to have a frame by frame decode, so I’ll see if I can make a new IMp3FrameDecompressor that works in Windows Store apps.
    • RiffChunk – to be reworked
    • WaveInBuffer, WaveOutBuffer are no longer applicable (and should really be moved into the MmeInterop folder)
    • Wave32To16Stream – contains unsafe code, should be obsoleted anyway
    • WaveMixerStream32 – contains unsafe code, also should be obsoleted

So as you can see, there is plenty of work still to be done. There are a few additional tasks once I’ve got everything I wanted moved across.

  • I want to investigate all the new Media APIs (e.g transcoding) and see if NAudio can offer any value-add to using these APIs
  • Make a Windows Store demo app to show off and test what can be done. Would also like to test on a Surface device if possible (not sure if I’ll run into endian issues on ARM devices – anyone know?).
  • Update the nuget package to contain a Windows Store binary

6 comments:

Ananth Balasubramaniam said...

I love NAudio and have used it for desktop apps before but want to do so for Windows Store apps - and this is great news.

My current need is quite specific, how can I decode MP3s using NAudio on Windows 8? How much is missing to be able to that at this time?

Thanks!

Mark H said...

The MediaFoundationReader class will be able to do this, it may be enough for you for now. I've not ported the WaveFileWriter yet though. Hopefully I'll get round to a demo app soon showing how to do this.

Jason Olson said...

This is fantastic Mark! Any place we can go to grab early bits if we want to test it out ourselves (I'm writing a synthesizer and want to see if I can avoid introducing a C++ component just for my WASAPI audio renderer).

Mark H said...

hi Jason, currently I'm not publishing any builds so you'd need to build from source over at CodePlex. Unfortunately I've not done WASAPI yet, although I don't think there will be too much to do given that I've already done the bulk of the interop. Annoyingly there seems to be a problem with my VS2012 install which means I can't make Windows Store apps at the moment, so there will be a bit of a delay until I work out how to fix that.

Jasmine wilson said...

I am glad to see this beneficial piece of writing.
Critical information loss situation is often an Inconvenience for the individual.
But for the simplicity of users, experts have designed a a solution for data damage issues.
If a data removal scenario has taken place, person must know which tool or application to apply to be able to handle this case of data damage.
pst file size

Christoph said...

If I remember correctly Encoding.GetEncoding("utf-7") is a way to get ASCII encoding in the Windows Runtime.