I love Python

[u/nakurtag]

[u/drislands]

I would have expected this function to create a new empty list on any call where items is not given. Doesn't this behavior make the function into a closure?

[u/nakurtag → u/drislands]

As I understand it's a well known (for python developers) weird behavior of python. Argument's default values are precalculated by interpreter on function declaration and stored into map associated with this function. Python extracts default values from this map on the next calls of this function. And since [] is a List object, the value copied by reference.

[u/trutheality → u/nakurtag]

The behavior isn't even that weird: if you were to create some mutable global and set it as a default parameter in a function you wouldn't be surprised that it's accumulating changes from different calls to the function.

The key part is recognizing that the LHS in a default parameter assignment runs when the function is defined and not when it is called.

[u/nakurtag → u/trutheality]

Name one language with the same behavior

[u/Double_Cost4865 → u/nakurtag]

I mean, C/C++? You can pass pointers/references and they will modify in place

[u/drislands → u/nakurtag]

Devastating. I would not have expected that behavior, and I think it would wise to avoid it wherever possible.

[u/Xelanybor → u/drislands]

yeah, usually the workaround is to have items = None as the default, and then some kind of check at the top of the function like items = items if items else []

looking at it now I see why it's weird lol but you definitely get used to it

[u/ZylonBane → u/Xelanybor]

Which defeats the entire purpose of the default value feature.

[u/notjfd → u/ZylonBane]

The default value is fine as long as it is not a mutable object. So things like integers and strings are completely fine to use as default values.

[u/ZylonBane → u/notjfd]

Yes, we're all aware. Nobody has disputed that.

[u/shunabuna → u/ZylonBane]

its probably for optimisation to reduce the amount of instructions needed to initiate a function call. On the plus side, its an easy place to define a cache

[u/AdreKiseque → u/Xelanybor]

What the fuck

[u/Sulungskwa → u/drislands]

There's definitely popular lint rules telling you you should never ever ever do this

[u/bigorangemachine → u/drislands]

This is why you gotta use a linter.

Stupid small stuff like this.

Also, I found out later the reason I hated ruby was because of robocop... not ruby

[u/groumly → u/nakurtag]

The behavior is definitely surprising, but also, what’s up with mutating a default param? How does that make sense?

[u/ChemTechGuy → u/groumly]

The default parameter is just a reference to the list. So technically the parameter isn't being changed at all. One of the oddities of pass-by-reference

[u/groumly → u/ChemTechGuy]

That’s not what I meant.

Mutating input is already not a good pattern, but mutating a default param is even worse.

You can’t tell if it was passed or not, so even without the interning oddity, you end up mutating something that nobody but you has access to.

[u/ChemTechGuy → u/groumly]

Go complain to Guido van Rossum then, I'm just telling you how it works

[u/CaptainFoyle → u/drislands]

Of course. This is just a very stupid way of writing the function. Initiate with None and Bob's your uncle.

[u/NFriik]

Python evaluates default arguments when the function is defined. That makes keeping track of the default values pretty straightforward. In your example, they're stored as part of the function object in the dictionary add.__defaults__. Every subsequent call can then just look-up the default value in this dictionary, if necessary.

Evaluating the default argument each time the function is called would be much more difficult. A function is essentially just yet another object, so you'd somehow have to store arbitrary expressions as an object inside the __defaults__ dict and then decide at runtime what to do with them. Sure, you could implement a builtin type at the CPython level to accomplish this. But then when you encounter such an ArbitaryExpression object inside the __defaults__ dictionary, how do you know that this time, you want to actually execute the underlying code and not use the object "as-is"? The whole thing would quickly become a convoluted, expensive mess.

By performing evaluation at definition time, function calls remain consistently fast and the mechanism to insert default positional arguments becomes incredibly simple. Python is already not the fastest language when it comes to function calls, so these things matter.

[u/4xe1 → u/NFriik]

Default data defined at the function level is perfectly normal, in many languages string literals live in the code too, they don't get copied every time they are used. What's more disturbing to me is that the default data is mutable.

[u/gundam1945 → u/NFriik]

The explanation makes sense but the behavior is probably not expected. Though if I need to add to a list I usually make the variable outside first and extend in the function.

Never encounter such a use case in this example.

[u/ZylonBane → u/NFriik]

You make it sound like it would be absurdly complex or slow to just clone any non-primitive default value.

[u/4xe1 → u/ZylonBane]

Because it is. Shallow copy versus deep copy is a whole area of study. Rust does it very cleanly and infers a lot of busy work, but it still needs explicit instruction from the user as to what to copy how every now and then.

[u/NFriik → u/ZylonBane]

Well, it kinda is, because that's not how Python works. A function is an ordinary object, it's default argument values are stored in a dict inside this object, so you can read and manipulate them at runtime. That wouldn't work with your idea, not without huge overhead.

[u/ZylonBane → u/NFriik]

"That wouldn't work because that's not how it works" is a circular argument. The implementers could have made default values evaluate at call time, like JavaScript does, but instead they chose violence.

[u/NFriik → u/ZylonBane]

Implementing functions the way Python does is a design decision with advantages and disadvantages. One of the advantages is that it allows to adhere to one of the language's main principles, which is that everything is an object. In the end, this behavior is just a consequence of this. Having default arguments treated differently to other function attributes (such as the name, doc string and code) would be weird. The Python devs didn't choose violence, they chose consistency.

[u/Loading_M_ → u/NFriik]

A more important point: default values don't have to be stored on the function object at all. The only information actually needed is whether the keyword argument has a default value, not the specific value. Then the code to instantiate the default values (if needed) can simply be inserted into the function body before the user code. This is basically how most other interpreted languages handle it.

Alternatively, the default value could always be stored as an Expression object or similar.

[u/_killer1869_]

Any IDE will warn not to use mutables as default parameter values. And for good reason. Such code would never appear in actual code intended to be used.

[u/Double_Cost4865 → u/_killer1869_]

Unfortunately, I’ve seen monstrosities like this in production in big finance corps. That’s what happens when statisticians are expected to create software.

[u/marinuso → u/Double_Cost4865]

I've seen it used in production code to memoize functions in this manner:

def fib(n, cache = {0: 1, 1: 1}):
    if not n in cache:
	    cache[n] = fib(n-1) + fib(n-2)
    return cache[n]

Except with much more complicated code rather than this simple example.

[u/bigorangemachine → u/marinuso]

that's actually kinda smart

[u/bigorangemachine → u/Double_Cost4865]

default params is the classic case of IDGAF with code.

"I don't which array... just add it to an array"

[u/Thelmholtz → u/_killer1869_]

It's still a horribly funny feature.

[u/CagoSuiFornelli → u/_killer1869_]

Such code would never appear in actual code intended to be used.

You should take a tour in my company codebase

[u/scheimong → u/_killer1869_]

Such code would never appear in actual code intended to be used.

Tell me you're a junior dev without telling me you're a junior dev.

[u/mothzilla]

It's a well known python gotcha that people ask about in interviews all the time.

[u/Beregolas]

This also happens in JavaScript btw, as well as several smaller scripting languages in game engines, it's a pretty normal feature in many interpreted languages.

In my last backend job, this was actually the source of our longest running and most annoying bug, where a specific page would display the wrong sorting, but never in testing. Because someone *caugh* used a mutable argument to give it default sorting parameters, and they got mutated by a very specific query, which caused it to be wrong for all subsequent requests.

So yeah, this is a real thing, you should know it, and just in case, you should have linting and/or your IDE setup so this kind of thing is rejected out of hand, and cannot be merged to prod/main.

[u/4xe1 → u/Beregolas]

just in case, you should have linting and/or your IDE setup so this kind of thing is rejected out of hand

So what kind of linting have you adopted? Default parameters can be quite handy. Does it reject them, or does it manage to detect and reject their mutations?

[u/Beregolas → u/4xe1]

I get a warning when I use anything mutable. Instead of an empty list, or a list with presets for example, I have to put None, and set it in code. This is basically also the recommended pattern.

[u/RngdZed]

The list is stored directly in the parameter without actually giving it an argument? I'm baffled, I didn't know this was a behavior, so weird

Is this something that someone could ELI5 to my smooth brain?

[u/bigorangemachine → u/RngdZed]

Default param is an object reference. So every default param is the same object

[u/lolcrunchy → u/RngdZed]

Here's a single change to the posted code that kinda shows what actually happens.

my_list = []
def add(v, items=my_list):  
    items.append(v)
    return items

print(add(1))
print(add(2))

[u/bigorangemachine → u/lolcrunchy]

Ya I test in JS and JS will do this.. oddly enough they are so similar JS didn't do the thing OP posted about.

Surprised about that one... but it makes sense.

[u/sol7dev]

and?

[u/bigorangemachine]

I bet this happens in JS as well

Edit: nope python

I should check gdscript

[u/Crafty-Detail-3788]

Issue here is that the default parameters are initialized ONCE. If you want an empty list every time you need to define it inside.

[u/itemluminouswadison]

It's a quirk of the language and something you should have learned and read about. The default arg gets created once upon func definition, not per invocation

[u/Kadabrium]

Python doesnt do a lot of variable scoping for how much indent it does

[u/GoddammitDontShootMe]

So each call reuses the same default list? Yeah, I think I would've expected it to get passed an empty list by default each time.

[u/Giiiin]

Yeah, I mean, don't do that

[u/jmooroof2]

I can't imagine a compelling use for this feature. it really should just error

[u/NFriik → u/jmooroof2]

There are some use cases for this behavior, but it's reeeaaally limited and you have to know exactly what you're doing. For example, you could use it to cache relatively expensive function calls as default values:

import re
def parse(text, pattern=re.compile(r"\d+")):
  return pattern.findall(text)

This way, instead of re-compilingthe regex every time the function is called, you can just reuse it.

[u/simonsmine → u/jmooroof2]

Its not really a feature more a consequence of pythons reference handling. You can do fun things with it though, memosation comes to mind.

[u/Double_Cost4865 → u/jmooroof2]

It has it’s uses, I just wouldn’t provide default argument like that since it’s wrong. But it’s quite common to modify a list in place and have the changes persist outside the function scope

[u/4xe1 → u/jmooroof2]

default function parameters are useful, shared function default parameters are efficient, and mutable function parameter are convenient (saves the interpreter the check for mutability, and and saves the user the boiler plate to copy it in a local variable).

What you're seeing is merely a combination of the three.

But shared mutable function parameter is also uniquely useful for memoisation. It's not where you'd put the table of previous results in an other language, but it makes perfect sense to do so in python, as it hides a side effect which is not a real side effect.

[u/awidesky]

Is that a documented behavior, or not?
If it is, that doesn't make sense.
If not, it's technically a bug, right?

[u/Ollidav]

A mí me encanta el helado de vainilla.

[u/akoOfIxtall]

I hate python from the bottom of my soul

[u/OhItsJustJosh]

This is why I stick to C#, I've written in Pascal, Java, JavaScript, Python, handful of others, and they all have their weird quirks, but C# seems to be the only one that works as expected

[u/CaptainFoyle]

Everyone knows: don't use mutable default arguments.

You just outed yourself as uninformed.

[u/nakurtag → u/CaptainFoyle]

Who you name "everyone"? Python developers? I'm not python developer, I'm from C#/Go/Rust world and this behavior is ridiculous for me. This is highly counterintuitive and illogical.

[u/squabzilla → u/nakurtag]

My main language is Python, and still think it’s ridiculous how much unintuitive behaviour there is because Python’s array-equivalents, by default, copy the object-pointer instead of the object itself.

And then people tell you why it works like that, confusing an explanation for an excuse.

[u/CaptainFoyle → u/nakurtag]

I mean how do you learn a new language? Does it just pop into your head? Or do you look up documentation and syntax? Then that's usually one of the first things you read

[u/nakurtag → u/CaptainFoyle]

Eventually, I consulted the documentation when I encountered this behavior. But initially, I tried to write the code based on my experience as a programmer and by using deduction.