How to add PReLU activation?

[cooltopics]

I need to add PReLU activation which has a learn-able parameter? What would be the best way to extend this library in this respect?
thanks.

[cooltopics]

Similarly, I find difficulty to even add a leakyReLU layer.

[yixuan]

In this case you need to define an ordinary layer that computes the derivative of the parameter, which is not an easy task since derivatives are coded manually.

[Hykni]

A vanilla leakyReLU activation would be very simple to add, copy the code from Activation/ReLU.h and replace the activation&gradient for Z<0...
i.e.
A.array() = Z.array().cwiseMax(Scalar(0));
to
A.array() = (Z.array() > Scalar(0)).select(Z, Z.array() * Scalar(p));
and
G.array() = (A.array() > Scalar(0)).select(F, Scalar(0));
to
G.array() = (A.array() > Scalar(0)).select(F, Scalar(p));
where p would be your leaky gradient for Z<0. Not really sure if the first replacement is valid eigen code but you should understand the concept.