A Trick for Backpropagation of Linear Transformations

74 pointsposted 5 months ago
by tripplyons
(tripplyons.com)

6 Comments

hansvm

5 months ago

That's quite a neat trick. You do need to be careful if the same matrix appears more than once in your einsum call though.

brandonb

5 months ago

Very nicely explained. Your article on rendering fractals was also well-done!

greatgib

5 months ago

Nice and easy article for a normally complicated topic.

fzimmermann89

5 months ago

..and for complex valued tensors, you need to conjugate.

tripplyons

5 months ago

I've only accounted for real numbers. I'm not sure how to cleanly account for conjugates when some of the einsums would need them and others wouldn't.

For example, a matrix product would need a complex conjugate, but a Hadamard product wouldn't.

If there is an elegant way to extend this to complex numbers, let me know!