SF Python Presentation Night

On Sept 13, join SF Python and learn more about how Cisco moved from TcL to Python, how caching works, and how Yahoo tackles scaling. Our generous sponsor Yelp will also provide pizza and beer for the evening.

Talk #1: A journey from TCL to Python by Sheng Wang and Varun Krishnan (20 mins)

BRIEF DESCRIPTION

This talk explains about how Network Automation in Cisco has evolved from Tcl to Python, what were the major technical challenges involved before the transition and how switching to Python has helped address many of those challenges.

DETAILED ABSTRACT

Network Test automation is becoming an increasingly critical component in network product certification amongst large Tier-1 SPs and Web/DC customers. Tcl was the primary automation language in the industry and tons of script have been written based on Tcl. However, due to the nature of non-OOP and lack of extensive libraries, engineers always found they were repeating themselves. This talk explains about how Network Automation in Cisco has evolved from Tcl to Python, what were the major technical challenges involved before the transition and how switching to Python has helped address many of those challenges. The talk will include a brief introduction to CafyKit, a Python-based automation infrastructure library created in-house at Cisco, which aims at helping users develop platform-independent scripts and emphasizes the Python principle of "DRY". Thought processes behind building the library would also be shared with the audience to shed some light on how to go about designing better libraries. It will also talk about how some python standard libraries like telnetlib, pexpect, paramiko, multiprocessing and multi-threading were used in the process along with the pros and cons of these libraries and the technical choices which have been made during the process.

BIO

Varun Krishnan is a Software developer at Cisco where he works on scaled systems. Currently, he is working on a python based automation framework for testing distributed routing platforms. His interests include C and Python programming, embedded development and listening to obscure podcasts

Sheng Wang is also a Software developer at Cisco.

Talk 2: All Things Cached by Grant Jenks (20 mins)

BRIEF DESCRIPTION

Caching is critical to the performance of web services and scientific systems. Today's computing puts a premium on memory while leaving gigabytes of disk space unused. Ideally our cache systems would utilize both memory and disk to balance speed and space. Together we'll survey cache features, backends, and frameworks and learn about DiskCache, a fast Django-compatible disk backed cache.

DETAILED ABSTRACT

"Caching is often the best way to improve the performance of services and systems. But caching is also complex and hard, full of features and tradeoffs. There is a lot to consider when selecting a cache: eviction policies, object serialization, maintaining a separate process or server, synchronization support, platform support and more. Together we will survey cache features, backends, and frameworks in Python. We will then look at DiskCache, a fast Django-compatible disk backed cache, and see how it fits in the landscape of features and backends. DiskCache supports a number of use cases beyond the typical get, set, and delete operations. Examples of accelerated serving of static files, cross-process communication and synchronization, analytics counters, and a persistent mapping will be explained. Benchmarks between DiskCache, Memcached, and Redis will also be shown. Finally we will see what makes DiskCache fast and understand the tradeoffs to be made between memory, disk and the network. Supporting the cache metadata in DiskCache is SQLite and with the right settings, we can make it faster than Memcached."

BIO

Grant Jenks is a founding partner and consultant in online businesses. His current roles range from building CRM-type solutions to mathematical modeling of markets and bidding. Previously he worked as a C/C++/C# compiler developer and found it lay directly at the intersection of his interests in algorithms and performance. He enjoys the vibrant community around Python and looks for opportunities to contribute.

Talk #3: Honey, I scaled up Python by Varun Varma (30-35 mins)

BRIEF DESCRIPTION

This talk will take you through the requirements, problems, design decisions, implementation details and lessons learned while building a planetary scale network telemetry system at Yahoo. You’ll see all the joys and wonders of using Python for building a scalable, distributed system and all the mistakes (and their solutions too!) we made along the way.

DETAILED ABSTRACT

At the end of 2015, Yahoo started to look for networking telemetry and monitoring system to replace their existing set of tools. Our team set out features and operability criterion and, sure enough, did not find anything that met the requirements at ‘Yahoo Scale’. We decided to do greenfield development for this on our own. This talk takes you through the journey of building this system and deploying it to production, the scaling issues we faced at each layer and how we powered through them. We will talk about the core framework we built, the platform that was put on top of that and the finally the applications built on top of the platform. We will consider what all framework elements a distributed system needs (e.g. configuration and logging management, distributed locking, leader election, work queue management, caching, federation, instrumentation) and how we mapped these to the tech stack choices we made. You’ll see how we used Celery, Yapsy and Django on top of Zookeeper, Kafka, Redis and MySQL to build this, the mistakes we made, the mistakes we thought we had made and how we solved performance and scaling problems.

BIO

Varun Varma is a Principal Engineer at Yahoo leading the design and development of the Network Telemetry Platform. Over the course of his 19-year career, he has worked on everything from embedded systems in start-ups to ad tech systems at web scale.

Agenda:

6:00p - Check-in and mingle, with food provided by our generous sponsor Yelp!

7:05p - Welcome

7:10p - Announcements and main talk

7:30p - Doors Close

9:00p - More mingling

9:30p - Hard Stop

SF Python is run by volunteers aiming to foster the Python community in the Bay Area. Please consider making a donation to SF Python and saying a big thank you to Yelp for providing pizza, beer, and the venue for this Wednesday's meetup.

Yelp sees 89 million mobile users and 79 million desktop users every month. Keeping everything running smoothly requires the best and brightest in the industry. Their engineers come from diverse technical backgrounds and value digital craftsmanship, open-source, and creative problem-solving. They write tests, review code, and push multiple times a day. Come out and talk to them.