How does a code completion tool work? What is an Abstract Syntax Tree, and how is it created in Python? How does an AST help you write programs and projects that inspect and modify your Python code? This week on the show, Meredydd Luff, co-founder of Anvil, shares his PyCon talk, “Building a Python Code Completer.”

Meredydd talks about his experience building a code completion engine for the Anvil platform. The conversation leads us to discuss how Python parses the code that you type. We examine tokenization, abstract syntax trees, and how parsing has changed in Python.

We cover additional projects that you can explore once you have a tool that inspects the Python code you’re writing. Join us as we dive into multiple rabbit holes of research and exploration.

Course Spotlight: Python Assignment Expressions and Using the Walrus Operator

In this course, you’ll learn about assignment expressions and the walrus operator. The biggest change in Python 3.8 was the inclusion of the := operator, which you can use to assign variables in the middle of expressions. You’ll see several examples of how to take advantage of this feature.

Topics:

  • 00:00:00 – Introduction
  • 00:02:11 – Building a Python Code Completer - Talk
  • 00:08:48 – How is recursion being used?
  • 00:10:28 – Abstract Syntax Trees, Parsers, and Tokens
  • 00:14:58 – Sponsor: CData Software
  • 00:15:39 – Is parsing into tokens a separate step?
  • 00:21:45 – What makes the new PEG parser unique?
  • 00:25:32 – Using the AST doesn’t have to be scary.
  • 00:27:33 – More on the PEG parser
  • 00:29:54 – Keeping scope and holding completions
  • 00:39:43 – Video Course Spotlight
  • 00:41:15 – Was this your first expirement in creating a code completer?
  • 00:46:46 – How can adding type hints assist a completer?
  • 00:50:53 – Projects you can build using AST
  • 00:53:02 – Linter project
  • 00:54:15 – Automatic code formatter
  • 00:54:59 – Code testing and the pytest project
  • 00:56:22 – Security tools
  • 00:57:11 – Friendlier tracebacks
  • 00:59:57 – Code completion vs GitHub Copilot
  • 01:04:14 – Code completion in a text editor vs IDE
  • 01:08:53 – What are you excited about in the world of Python?
  • 01:09:54 – What do you want to learn next?
  • 01:11:20 – How can people follow your work?
  • 01:12:31 – Thanks and goodbye

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas