Simple gated unit (`SGUCell`) and Deep simple gated unit (`DSGUCell`)

[MartinuzziFrancesco]

Paper

https://proceedings.mlr.press/v63/gao30.pdf

Equations

$$\begin{align} x_g &= W_{xh} x_t + b_g \\\ z_g &= \sigma_1\!\left(W_{zzh}\left(x_g \cdot h(t-1)\right)\right) \\\ z_{\text{out}} &= \sigma_2\!\left(z_g \cdot h(t-1)\right) \\\ z_t &= \sigma_3\!\left(W_{xz} x_t + b_z + W_{hz} h(t-1)\right) \\\ h(t) &= \left(1 - z_t\right) h(t-1) + z_t \cdot z_{\text{out}} \end{align}$$ $$\begin{align} x_g &= W_{xh} x_t + b_g \\\ z_g &= \sigma_1\!\left(W_{zxh}\left(x_g \cdot h(t-1)\right)\right) \\\ z_{\text{out}} &= \sigma_2\!\left(W_{go}\left(z_g \cdot h(t-1)\right)\right) \\\ z_t &= \sigma_3\!\left(W_{xz} x_t + b_z + W_{hz} h(t-1)\right) \\\ h(t) &= \left(1 - z_t\right) h(t-1) + z_t \cdot z_{\text{out}} \end{align}$$

Official implementation

none