What are the two activation functions used inside the LSTM cell? What happens to the input after it gets multiplied with the weight matrix?
Answer
The two activation functions are simgoid and tanh. The input, after multiplication with the weights matrix, gets added to the hidden state from the earlier time step (the hidden state also gets multiplied with a weight matrix first). The result then gets separated into four different pieces which follow different paths (play different roles) inside the cell.