8000 Missing LayerNormalization & MultiHeadAttention Keras Layers · Issue #797 · SciSharp/TensorFlow.NET · GitHub

Missing LayerNormalization & MultiHeadAttention Keras Layers #797

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

lqdev opened this issue Apr 13, 2021 · 2 comments

Assignees

lqdev commented

@Oceania2018 can you assign this issue to me. I'll try and take a stab at it.

I was trying to see whether implementing the Keras Transformer recommendation sample is feasible in .NET. However, I believe two of the layers, LayerNormalization and MultiHeadAttention are missing from the current implementation.

Oceania2018 assigned lqdev

GeorgeS2019 commented

•

@lqdev In your approach to MultiHeadAttention, perhaps you could share your feedback to this thread of discussion.

Perhaps we may find more people interested on this topics for .NET

lindadamama commented

missing layerNormalization
However, Torchsharp's APIS are complete
https://github.com/tensorflow/tensorflow/tree/v2.7.0/tensorflow/python/keras/layers/normalization
https://github.com/dotnet/TorchSharp/tree/main/src/TorchSharp/NN/Normalization

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

0