[Incomplete/Help wanted/Blocked] Initial work on Wasm/Browser #137
Conversation
This does not work! As of this commit the only thing that works is launching the program. Much of this is not properly tested, but hey, at least it's someting! Includes opus.a for testing purposes (Idk if it works). Includes simpleserv python script to serve on port 8000 (Also for testing purposes; Serves the output of dotnet publish).
This commit makes use of the emscripten openal api! Currently I'm doing some weird stuff with DllImport's to make it work. (Idk if there is a better solution) Also removed opus.a because that was what was causing compilation issues the whole time! Which means that it's back to the drawing board for it.
Idk if I was supposed to do this in the rebase, but I fixed the broken code by once again copying the Linux code and modifying it as needed... Which actually just boils down to added the DllImport workaround and that's it. In fact it we could figure out an actual solution to it I guess we could maybe do VoiceCraft.Client.Linux.*? Also I think I did NativeStorageService correctly maybe. Idk about async stuff though.
|
If we want to continue working on this, we will need to implement a WebRTC communication protocol within the web server that will then forward it to the native VoiceCraft Server. This is a massive amount of work to do as we will need to separate the low level socket IO protocol from the shared project in order to achieve this... |
Yikes, I didn't realize that Wasm had such restrictions on UDP and TCP connections, this is definitely over my head. I unfortunately don't know how much more I can help on this, so I guess I'll leave it up to you whether you want to close this, or to keep this open for people to find it easier to reference (Idk what people do in this kind of situation). |
|
After some research. I found out web browsers allow WebSockets. The server already has a websocket implementation with Z85 raw bytes > string codec and back again for MCWSS so we can just build a protocol on top of raw WebSockets without WebRTC. |
|
We still need to separate socket IO into abstract classes but this does significantly make it easier |
|
Socket.io also uses websockets tho... You could just use only the socket io server and do everything from there... |
|
The socket io control I'm talking about is lower than websockets which is UDP which is what VoiceCraft uses as it's more performant than TCP which is the underlying layer for websockets which is also the underlying layer for Socket.IO. Plus I already have control over raw websockets so it's easy to do. |
This does not work!
Update: Also blocked by the .NET emsdk. It uses Binaryen
v117, we requirev121or higher which adds thebulk-memory-optandcall-indirect-overlongfeatures! We need this because opus and speexdsp won't compile otherwise.As of this commit the only thing that works is launching the program, but hey, at least it's something!
Hoping this is at least somewhat useful to someone and not a useless mess.
While I'm stopping working on this because of my lack of knowledge on audio, if anyone wants to help continue this I'd be happy to help in any way that I can!Update: I have figured out how to use the emscripten OpenAL api! The problem is that I'm doing a weird hack to make it work, but I feel like that there should be another way of doing this that doesn't involve throwing a bunch of dummy DllImport's all over the place.
Things that are left to do (Of my knowledge) are:
Audio inputAudio outputMost of the code was copy-pasted from Android and Linux and modified.
A lot of this is probably not formatted correctly with a lot of code commented out.
Includes
simpleservpython script to serve on port 8000 (For testing purposes becausedotnet rundoesn't work for me for some reason; Serves the output ofdotnet publish).