Tying Languages Together with YarnBernard Duggan
schedule 1 year agoSold Out!
Everyone loves to start a project with a clean slate, but the reality that very often we end up being asked to maintain and extend existing systems that were written by people who hadn't realised that Erlang is the correct solution for every problem we're ever presented with. Our company's VoIP softswitch was no exception.
Having realised the mistake of the previous designers, the clear solution was to rewrite everything in Erlang. This was both self-evidently sensible and, for some reason, also utterly unacceptable to management who couldn't understand why we thought there was any problem with the current system.
So instead we decided that we'd just write all the new parts in Erlang, and rewrite other parts in a piecemeal manner as opportunities arose.
Of course, it turns out that that's easier said than done. The C++ code was written with the (initially well-founded) assumption that it would all be talking to other C++ code in a homogeneous environment, from the compilers all the way down to the hardware. This allowed the developers to take a number of “clever” shortcuts, all of which made plugging in the new Erlang parts several orders of magnitude more difficult than it would have been if some language-agnostic IPC system had been used.
The solution we developed, named Yarn, is a code generator implemented in Erlang using erlydtl and yecc. It allows us to specify, in a single canonical location, data schemas and structures that are used across multiple languages. All the declaration and marshalling code is then created automatically as part of the build. After success with Erlang and C++, it has now extended to include generation of Python, Java and SQL.
This talk will examine the technical problems that lead us down the road to Yarn, the details of its implementation, and the many “gotchas” we hit in the process of building it.