TriPython October 2016 Meeting: Engineering a Robust(ish) Data Pipeline with Luigi and AWS Elastic Map Reduce
Note that this meeting is on the third Thursday of the month for this month only due to a conflict with the closing parties at the All Things Open conference. With great data comes great responsibility. In this talk, we'll look at how to build a data reporting pipeline that's reliable, scalable, and maintainable. This will involve Luigi, a simple but powerful tool for managing interdependent batch processes. We'll also look at using the Datapipeline and EMR services in AWS. This talk is by Aaron Knight. Aaron is a full-stack engineer at Voxy, an education startup that creates English lessons that are personalized to a learner's interests and goals. He is also the organizer of the weekly Triangle Hacker Hours meetup. Extemporaneous "lightning talks" of 5-10 minute duration are also welcome and don't need to be pre-announced. Lightning talks are for you to "show and tell" something you've learned about Python recently, no matter how small. We all use Python, therefore, we are always learning something new about Python that we can tell others. Plenty of free parking is available in the RENCI parking deck. The meeting will be followed by our usual after-meeting at a nearby tavern for food and beverage. Come join us for a fun and informative evening.
For more information, visit
- Renaissance Computing Institute (RENCI), Biltmore Conference Room, 5th Floor, 100 Europa Drive, Suite 590, Chapel Hill
- Chris Calloway
- Contact Email
- Contact Phone
- (919) 599-3530