Version 1.1
Vortrag: Compiling for fun and profit!
Or how I learned to stop worrying and love the compiler
This is the story of how we solved our performance and reliability issues while giving our users’ workflow a speed boost and saving their sanity by generating code automatically from human-readable specifications. Internally, we provide a configurable stream data processing tool to normalize loglines into a common structure from multiple sources. We developed a DSL and DSL-to-python compiler to let users express their needs without requiring them to have coding skills, but just their domain knowledge.
At RadarServices, we deal with real-time log processing from disparate sources from various customers, and we need to process them in a normalized form for analysis. Due to the differences in data format this can be quite a challenge. Originally, we made use of handwritten normalizers which, of course, was time consuming and inefficient. Making use of our knowledge, we implemented from scratch a system that would translate our custom, easily understandable DSL to python bytecode, which allowed our analysts to greatly increase their productivity. To do this, we used the python AST standard library, Ply (lex/yacc), and our knowledge of compilers.
Info
Tag:
03.05.2019
Anfang:
11:30
Dauer:
00:30
Raum:
F0.01
Track:
PyDays 1
Sprache:
en
Links:
Dateien
Feedback
Uns interessiert deine Meinung! Wie fandest du diese Veranstaltung?
Concurrent Events
Referenten
 |
Dario Meloni |