Ok just to update you. I soldered the rf module on the incorrect way. Prize idiot that I am.
This is the source of all the bad behavior. When I removed it the board is doing exactly the same thing, so I assume this is what happens if you choose telem and ppm and you have no radio module attached.