Feedback about NLP From Scratch: Generating Names with a Character-Level RNN

There is the following issue on this page: https://docs.pytorch.org/tutorials/intermediate/char_rnn_generation_tutorial.html

I might be missing something, but in the code on this tutorial, there is no nonlinearity in the recurring part of the RNN, degrading all of the internal updates to a single linear function. 