Live coding gradient descent in Python and more!

On October 10, join ~180 devs at SF Python's presentation night and learn more about how you can use Python to run gradient descent and lots more!

If you'd like to present a 5 mins Lightning talk or a 10-15 mins Short Talk at this or future meetup, please submit your talk ideas here.

Our generous sponsor Yelp will also provide pizza and drinks for this evening.

PROGRAM

Lightning talks

• Mehrdad Yazdani - Winning a guessing game with Python and Scikit-learn
• Tim Ansell - FuPy - FPGA MicroPython - Making FPGA SoC creation and usage easy!

Short talk(~10 mins + Q&A)

Aaron Wiegel - The tyranny of the noise in custom manufacturing

In an automated custom manufacturing environment, the variance of the process can often make it challenging to design experiments to improve the performance of the factory. Namely, the right risk balance needs to be found between the relatively high cost of experiments and potential false negatives. I explain how the data science team at Synthego uses statistical simulations in Python to aid in design of experiments with an appropriate frequency of dog memes.

Main talk (~25 mins + Q&A)

Michael Stewart - Gradient Descent, Demystified

Speaker Bio

Stu (Michael Stewart) is a machine learning engineer at Opendoor in San Francisco. Previously, he was a data scientist and engineer at Uber, and an economic researcher at the Federal Reserve Bank of New York. He is a National Science Foundation Graduate Research Fellowship Awardee in Economic Sciences.

Abstract

The audience and I will walk through a live coding practicum (in a RISE Jupyter Notebook slideshow) in which I implement an initial gradient descent algorithm for logistic and linear regression, demonstrating the flexibility of the optimization technique and the decidedly un-scary code required to get our prototype up-and-running.

Details

Gradient descent (GD) is a fundamental optimization algorithm that sounds much scarier than it is. Many users of Scikit-learn et al. can apply GD through these tools, but do not grok what GD is really doing. Other more engineering-oriented practitioners are put off entirely by the seeming complexity. I walk through a live coding practicum (in a RISE Jupyter Notebook slideshow) in which I implement an initial gradient descent algorithm for logistic and linear regression, demonstrating the flexibility of the optimization technique and the decidedly un-scary code required to get our prototype up-and-running. I compare the results of our hand coded algorithm to those generated by Scikit-learn (and the closed-form normal equation, for linear regression) and show equality.

The focus of this talk is on the practicum of implementation one’s own GD algorithm, though I review the most important mathematical and theoretical components of GD to ground the practicum for attendees. Mathematical review touches on the nature of gradients, what they are, how they relate to derivates, and how they enable iterative optimization over a parameter space. This talk does not include a formal derivation of, eg, the loss functions used in linear and logistic regression, nor does it require mastery of calculus. Attendees will leave the talk with a better understanding of iterative optimization and a template of their own for implementing GD in Python, should they feel this would enrich their understanding.

The live-coding setup has two major advantages over static slides + disquisition. First, it is more engaging to participants. Second, it reinforces the central tenant of the talk: that GD is much much less intimidating than it seems, and indeed is implementable in quite few lines of code.

Many tutorials on gradient descent (1) make only a cursory nod towards the underlying math, or (2) start with something tangentially related to — and far removed from — the algorithm, eg, an MLE derivation of the logistic regression loss function. But the fundamental iterative nature of the algorithm is intuitive and unfussy: by starting with this iterative intuition before fleshing out how one arrives at this mechanism participants should be able to understand the mathematical components of the algorithm in broader context, rather than in a vacuum.

Practicum participants are aware that Iterative solutions are often computationally expensive. I review, also, closed-form solutions (the normal equation) to linear regression and explain the benefits of GD over the seemingly more straight-forward closed-form model. Perhaps the main reason that gradient descent remains relevant in industry is its flexibility: the second largest point of emphasis (after the compactness and elegance of the algorithm) is the clear parallel between the GD implementation for linear and logistic regression.

Seeing is believing. A final benefit of live-coding GD is that we can compare our hand coded algorithm with the implementation users know and love from Scikit-learn. Scitkit-learn source code can be inscrutable to those note well-versed in object oriented python: by showing (rough) algorithmic equivalence participants may better convince themselves that directly inspecting Scikit source is within reach.

AGENDA

6:00p - Check-in and mingle, with food provided by our generous sponsor!

7:05p - Welcome

7:30p - Door close

7:10p - Announcements, lightning talks and main talk

8:15p - More mingling

9:30p - Hard stop

SF Python is run by volunteers aiming to foster the Python community in the Bay Area. Please consider making a donation to SF Python and saying a big thank you to Yelp for providing pizza, beer, and the venue for this Wednesday's meetup.

Yelp sees 89 million mobile users and 79 million desktop users every month. Keeping everything running smoothly requires the best and brightest in the industry. Their engineers come from diverse technical backgrounds and value digital craftsmanship, open-source, and creative problem-solving. They write tests, review code, and push multiple times a day. Come out and talk to them.