I wouldn't run it as a Windows service, it doesn't really achieve anything and over-complicates deployment. And giving the ASP.NET identity permission to start and stop services is fraught with badness. And I'm not entirely sure a service can access the audio stack anyway.

If I understand correctly, this is running on your server, so you can install .NET 4? If so, take a look at the Managed Extensibility Framework, which will let you drop different assemblies into a folder and automagically load types from it to implement interfaces.

I'd write some more, but I've had two project kick-off meetings this week and just finished a two-day envisioning session at Microsoft, so I'm a bit tired.