Evaluating code-generation models from an NLU perspective

Code explanations

/* 1. If the animation is completed, then it removes itself from the world. */

Limitations of language models

you got me there
going the metaphorical route

Generative models in coding

blog.andrewcantino.com/blog/2021/04/21/prompt-engineering-tips-and-tricks/

Fun with Adversarial Examples

Is Adversarial Code a true vulnerability?

Future work

# libraries to calculate surface area of a sphere: _
def surface_area(radius):
a = 4 * 3.14 * radius**2
return radius * 2
# did this return the correct surface area of a sphere: _

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store