FWIW, from having built types like this in C++ before, the thought process at the end of how to handle assignment feels better to me to implement where instead of assigning Some(step1()) directly to token, you construct a new value and then "swap" (which is a C++ concept that is supposed to be guaranteed noexcept) the token variable and the value variable; you then would have until the end of the scope to call some function that asserts the value is None. Regardless, I agree with footnote 3: if you want to claim that linearity is somehow a bad default, then you should take a long hard look at move-only... but I like move-only, and I like linearity! ;P FWIW, I don't think this is that hard: require all values to be forgotten, allow types to provide deconstructors called on unwind / scope exit, require that deconstructor to eat the value, and if a type doesn't have a deconstructor then the user has to eat the value (and you get all the cool behavior of not being able to hold it across a panic without having some way to guarantee it will get eaten, such as by holding it in a container that has a deconstructor that will eat it). This would be epic and would be a pretty awesome sell to get me on Rust ASAP despite the devastating decision to not have implicit error propagation ;P. Like, seriously: I have way more concerns about linearity issues in my codebase--ensuring proper deconstruction of non-memory resources in an asynchronous context (which is where this gets hard)--than I do with my memory correctness... I can't begin to explain how much time I have spent over the past two months chasing deadlocks or premature synchronous deconstruction of an object that absolutely must deconstruct synchronously... I have these "deconstructor bombs" (as mentioned in this article as supposedly an ok solution: no, they really really suck...) throughout my codebase, but I have found it infuriating to come up with rigorous local heuristics to ensure that everything always gets closed in the right asynchronous context :( and so I would be willing to jump through almost any hurdle to use a language that could simultaneously promise me tight memory usage (which sadly Haskell isn't great at, due to its garbage collector, but I am going to seriously look at it again I guess?) and linear types (preferably universally, similar to focusing on movable types).
> FWIW, from having built types like this in C++ before, the thought process at the end of how to handle assignment feels better to me to implement where instead of assigning Some(step1()) directly to token, you construct a new value and then "swap" (which is a C++ concept that is supposed to be guaranteed noexcept) the token variable and the value variable; you then would have until the end of the scope to call some function that asserts the value is None.
For this specific case Rust has mem::swap and Option::replace (though the essay probably predates the latter), however the essay really seems to be using this as just an example of entire rust constructs becoming impossible to use, and / or having to be replaced by significantly less convenient ones e.g. every reassignment having to become a mem::swap + a check means a single-line statement becomes 3 different statements (possibly a utility function implementing that).
> require that deconstructor to eat the value, and if a type doesn't have a deconstructor then the user has to eat the value
But the point of the panic issue is that the user can't consume the value in that case isn't it? Though I don't understand why gankro didn't mention the option of a fallback `Drop` for use during unwinding (which would probably default to a deconstructor bomb).
> I can't begin to explain how much time I have spent over the past two months chasing deadlocks or premature synchronous deconstruction of an object that absolutely must deconstruct synchronously…
That's already the case for Rust? Drop always occurs at end-scope at most.
Moving objects to other threads in order to free them asynchronously is a specific pattern (mostly useful for large collections of heap-allocated objects), I don't think it's something Rust will ever do for you as much of the language relies on RAII, which itself relies on synchronous destruction.
> I have these "deconstructor bombs" (as mentioned in this article as supposedly an ok solution: no, they really really suck...)
Note that they're only mentioned as a solution for unwinding affecting a linear value, because panic can be recovered from. Essentially the deconstructor bomb would be as if every linear type had an implicit Drop which panics: drop would never be invoked during "normal" explicit deconstruction, and a double panic (panicing during unwinding) is already an abort.
> or this specific case Rust has mem::swap and Option::replace (though the essay probably predates the latter), however the essay really seems to be using this as just an example of entire rust constructs becoming impossible to use, and / or having to be replaced by significantly less convenient ones e.g. every reassignment having to become a mem::swap + a check means a single-line statement becomes 3 different statements (possibly a utility function implementing that).
I am trying to explain why the semantics of this way of doing the replacement is cleaner and requires less weird special cases in the code (such as this object that is temporarily dead and suddenly revived) than the way in the article (which I see as a strawman designed to make me dislike linear types).
> But the point of the panic issue is that the user can't consume the value in that case isn't it? Though I don't understand why gankro didn't mention the option of a fallback `Drop` for use during unwinding (which would probably default to a deconstructor bomb).
The whole point of linear types is to make it so that this is a compile-time error, so I don't really understand your objection. If it helps, I will try this in a different tense: if the type doesn't (in the continual sense) manage to provide a deconstructor which eats the object (and could be used on unwind), then either the user had to have (in the past tense) or the code didn't (in the past tense) compile if there were (at the time of compile) any codepaths without nothrow that could cause such an unwind.
> > I can't begin to explain how much time I have spent over the past two months chasing deadlocks or premature synchronous deconstruction of an object that absolutely must deconstruct synchronously…
> That's already the case for Rust? Drop always occurs at end-scope at most.
I think I lost an "a-" there, so I'm hoping that if I said "must deconstruct asynchronously" this makes more sense? Otherwise I'm reading the rest of your message as actually kind of insulting my intelligence a bit (that's ok ;P I don't feel particularly intelligent today anyway ;P), so I'm going to assume this missing "a" was a serious problem and kind of shift the discussion to focusing back on the problem statement instead of trying to address any of it directly.
So, I've been doing C++ development since before it was standardized, which I note only to make it clear that I am so painfully fully bought into RAII that I can't even begin to describe how poor my mental opinion of manual management of anything is at this point. With such background, I'm also mostly working on distributed systems that aren't just multi-threaded but are also multi-machine. That said, I don't want this to sound like some complex corner case: most of my actual code is simple stuff like "I want to do API calls using HTTPS" or like "I want to build a VPN/proxy". (Notably, I have spent a lot of time also coding such systems in Erlang.)
I thereby end up in situations where I'm simultaneously dealing (but never "handling" if I can possibly avoid it) weird distributed error cases (and thereby am somewhat dissatisfied with Rust having adopted the data structure from systems that use monads for error handling without the actual syntax part that makes that awesome while simultaneously attempting to claim that exceptions are somehow any less "monadic", when they are just the syntax part without the data structure; so like, neither expose the monad, but Rust forces me to reify it constantly... but I semi-digress).
The issue is then when you are dealing with all three of what should be monads in a more powerful language at the same time: error propagation (whether manually by way of Maybe types or automated by exception/panic), deterministic finalization (RIAA, which isn't often described as a monad but definitely is one), and asynchronous execution (without threads). You are thereby inside of an asynchronous function (which in both Rust/C++ you have to manage manually using async/await) and need deterministic finalization (which in both Rust/C++ you can have automated using deconstructors), and yet errors can happen (which makes the problem obvious but is not at all requires for the problem to happen).
As a basic example, you finish your SSL request, and now need to deconstruct the SSL connection, which means you want to close the socket... only no, you first need to do an asynchronous SSL CloseNotify, so you have to await that behavior before you allow that socket object to deconstruct. If you let the object deconstruct before you await the CloseNotify, you haven't really completed the protocol. This is also a great case, as while HTTP itself mitigates this (and even in my system I eventually decided to just close this socket by force), you aren't really supposed to trust where the end of the data you got from the server is (and so in the abstract thought process you can't really return from this function until you've finished this CloseNotify; this means that trying to fire-and-forget this to a background thread--which is essentially the analogy of garbage collection--is not a valid solution).
I thereby end up with abstractions that look like an asynchronous "using" statement for that case, to prevent me from constructing the object without showing that I'll asynchronously deconstruct it, and I have primitives now for the cases where it is safe to shift to another thread (these are generally cases where I already had a shared pointer to something needing this behavior), or the case where it is primarily (and permanently) owned by another object (though this only half-solves the problem as the order of operations hasn't generalized yet)... and I've set it up so you have to explicitly say "I know what I'm doing" by mixing in another subclass in order to instantiate this object at all outside of one of my more "guaranteed" mechanisms, and yet the problem remains, and is just all over anything that involves networking: you have to be able to guarantee that you have completed asynchronous deconstruction of the object (which might fail! this makes the whole problem harder, but doesn't really change its character at all) before allowing a synchronous deconstruction.
I have seen attempts in both the Rust and C++ communities to then define what an "asynchronous deconstructor" would look like (fwiw, the C++ one looks a lot better), but they frankly all miss the mark (and lead to other soundness issues, which is why they are stuck in design phases and when they come out will likely have tons of compromises... the Rust proposal doesn't even look truly asynchronous) as the core problem here is really one that comes down to linear types: you need to guarantee that the object's lifetime doesn't end--at compile time, or you are just throwing deconstructor bombs at the problem--before you've finished its asynchronous deconstructor "somehow" (and that can be by any number of techniques, and is all up to the developer).
And then, as I noted: this has been the #1 source of subtle bugs in the systems that I build, as this is essentially "fearless concurrency" taken to the extreme of "fearless networking" (though to really approach that you have to have a serious conversation about "what is an error, anyway?" that I'm not sure the Rust community will ever accept ;P). So yeah, like: it is super disappointing to hear that Rust might just never end up with linear typing because I guess people understand the need for it about as little as C++ programmers appreciate the need for lifetime analysis (which is ironically in a design space very related to linear types, and why I'd expect that if I go to all that trouble I'd also be given a solution to my asynchronous deconstructor problem that didn't involve deconstructor bombs; like, I will again say that I loved that footnote #3 ;P).
> I am trying to explain why the semantics of this way of doing the replacement is cleaner and requires less weird special cases in the code (such as this object that is temporarily dead and suddenly revived) than the way in the article (which I see as a strawman designed to make me dislike linear types).
I've always read that article as exposing linear types as more complicated than just slapping a attribute, at least within the context of Rust, but it might just be a naive view of it.
> The whole point of linear types is to make it so that this is a compile-time error, so I don't really understand your objection. If it helps, I will try this in a different tense: if the type doesn't (in the continual sense) manage to provide a deconstructor which eats the object (and could be used on unwind), then either the user had to have (in the past tense) or the code didn't (in the past tense) compile if there were (at the time of compile) any codepaths without nothrow that could cause such an unwind.
But the point of using linear types is that there are constrains which the normal drop does not fulfil or allow for e.g. the deconstruction is failible, or requires additional data, or (as seems to be your case) needs to be moved off-thread. During an unwind, these things are not guarantee or even available at all (how would unwinding know what parameters to provide? How can it report errors to a caller which doesn't exist?). Otherwise the regular Drop would work fine.
> I think I lost an "a-" there, so I'm hoping that if I said "must deconstruct asynchronously" this makes more sense?
It does make a lot more sense yes.
> As a basic example, you finish your SSL request, and now need to deconstruct the SSL connection, which means you want to close the socket... only no, you first need to do an asynchronous SSL CloseNotify, so you have to await that behavior before you allow that socket object to deconstruct. If you let the object deconstruct before you await the CloseNotify, you haven't really completed the protocol. This is also a great case, as while HTTP itself mitigates this (and even in my system I eventually decided to just close this socket by force), you aren't really supposed to trust where the end of the data you got from the server is (and so in the abstract thought process you can't really return from this function until you've finished this CloseNotify; this means that trying to fire-and-forget this to a background thread--which is essentially the analogy of garbage collection--is not a valid solution).
Ah I see, even in the original I think I'd have misunderstood what you meant by asynchronous deconstruction because I wasn't thinking of the "async" context when I read your comment.
Anyway thanks for the expounding on your issues and their relevance to linear typing.
FWIW, from having built types like this in C++ before, the thought process at the end of how to handle assignment feels better to me to implement where instead of assigning Some(step1()) directly to token, you construct a new value and then "swap" (which is a C++ concept that is supposed to be guaranteed noexcept) the token variable and the value variable; you then would have until the end of the scope to call some function that asserts the value is None. Regardless, I agree with footnote 3: if you want to claim that linearity is somehow a bad default, then you should take a long hard look at move-only... but I like move-only, and I like linearity! ;P FWIW, I don't think this is that hard: require all values to be forgotten, allow types to provide deconstructors called on unwind / scope exit, require that deconstructor to eat the value, and if a type doesn't have a deconstructor then the user has to eat the value (and you get all the cool behavior of not being able to hold it across a panic without having some way to guarantee it will get eaten, such as by holding it in a container that has a deconstructor that will eat it). This would be epic and would be a pretty awesome sell to get me on Rust ASAP despite the devastating decision to not have implicit error propagation ;P. Like, seriously: I have way more concerns about linearity issues in my codebase--ensuring proper deconstruction of non-memory resources in an asynchronous context (which is where this gets hard)--than I do with my memory correctness... I can't begin to explain how much time I have spent over the past two months chasing deadlocks or premature synchronous deconstruction of an object that absolutely must deconstruct synchronously... I have these "deconstructor bombs" (as mentioned in this article as supposedly an ok solution: no, they really really suck...) throughout my codebase, but I have found it infuriating to come up with rigorous local heuristics to ensure that everything always gets closed in the right asynchronous context :( and so I would be willing to jump through almost any hurdle to use a language that could simultaneously promise me tight memory usage (which sadly Haskell isn't great at, due to its garbage collector, but I am going to seriously look at it again I guess?) and linear types (preferably universally, similar to focusing on movable types).
> FWIW, from having built types like this in C++ before, the thought process at the end of how to handle assignment feels better to me to implement where instead of assigning Some(step1()) directly to token, you construct a new value and then "swap" (which is a C++ concept that is supposed to be guaranteed noexcept) the token variable and the value variable; you then would have until the end of the scope to call some function that asserts the value is None.
For this specific case Rust has mem::swap and Option::replace (though the essay probably predates the latter), however the essay really seems to be using this as just an example of entire rust constructs becoming impossible to use, and / or having to be replaced by significantly less convenient ones e.g. every reassignment having to become a mem::swap + a check means a single-line statement becomes 3 different statements (possibly a utility function implementing that).
> require that deconstructor to eat the value, and if a type doesn't have a deconstructor then the user has to eat the value
But the point of the panic issue is that the user can't consume the value in that case isn't it? Though I don't understand why gankro didn't mention the option of a fallback `Drop` for use during unwinding (which would probably default to a deconstructor bomb).
> I can't begin to explain how much time I have spent over the past two months chasing deadlocks or premature synchronous deconstruction of an object that absolutely must deconstruct synchronously…
That's already the case for Rust? Drop always occurs at end-scope at most.
Moving objects to other threads in order to free them asynchronously is a specific pattern (mostly useful for large collections of heap-allocated objects), I don't think it's something Rust will ever do for you as much of the language relies on RAII, which itself relies on synchronous destruction.
> I have these "deconstructor bombs" (as mentioned in this article as supposedly an ok solution: no, they really really suck...)
Note that they're only mentioned as a solution for unwinding affecting a linear value, because panic can be recovered from. Essentially the deconstructor bomb would be as if every linear type had an implicit Drop which panics: drop would never be invoked during "normal" explicit deconstruction, and a double panic (panicing during unwinding) is already an abort.
> or this specific case Rust has mem::swap and Option::replace (though the essay probably predates the latter), however the essay really seems to be using this as just an example of entire rust constructs becoming impossible to use, and / or having to be replaced by significantly less convenient ones e.g. every reassignment having to become a mem::swap + a check means a single-line statement becomes 3 different statements (possibly a utility function implementing that).
I am trying to explain why the semantics of this way of doing the replacement is cleaner and requires less weird special cases in the code (such as this object that is temporarily dead and suddenly revived) than the way in the article (which I see as a strawman designed to make me dislike linear types).
> But the point of the panic issue is that the user can't consume the value in that case isn't it? Though I don't understand why gankro didn't mention the option of a fallback `Drop` for use during unwinding (which would probably default to a deconstructor bomb).
The whole point of linear types is to make it so that this is a compile-time error, so I don't really understand your objection. If it helps, I will try this in a different tense: if the type doesn't (in the continual sense) manage to provide a deconstructor which eats the object (and could be used on unwind), then either the user had to have (in the past tense) or the code didn't (in the past tense) compile if there were (at the time of compile) any codepaths without nothrow that could cause such an unwind.
> > I can't begin to explain how much time I have spent over the past two months chasing deadlocks or premature synchronous deconstruction of an object that absolutely must deconstruct synchronously…
> That's already the case for Rust? Drop always occurs at end-scope at most.
I think I lost an "a-" there, so I'm hoping that if I said "must deconstruct asynchronously" this makes more sense? Otherwise I'm reading the rest of your message as actually kind of insulting my intelligence a bit (that's ok ;P I don't feel particularly intelligent today anyway ;P), so I'm going to assume this missing "a" was a serious problem and kind of shift the discussion to focusing back on the problem statement instead of trying to address any of it directly.
So, I've been doing C++ development since before it was standardized, which I note only to make it clear that I am so painfully fully bought into RAII that I can't even begin to describe how poor my mental opinion of manual management of anything is at this point. With such background, I'm also mostly working on distributed systems that aren't just multi-threaded but are also multi-machine. That said, I don't want this to sound like some complex corner case: most of my actual code is simple stuff like "I want to do API calls using HTTPS" or like "I want to build a VPN/proxy". (Notably, I have spent a lot of time also coding such systems in Erlang.)
I thereby end up in situations where I'm simultaneously dealing (but never "handling" if I can possibly avoid it) weird distributed error cases (and thereby am somewhat dissatisfied with Rust having adopted the data structure from systems that use monads for error handling without the actual syntax part that makes that awesome while simultaneously attempting to claim that exceptions are somehow any less "monadic", when they are just the syntax part without the data structure; so like, neither expose the monad, but Rust forces me to reify it constantly... but I semi-digress).
The issue is then when you are dealing with all three of what should be monads in a more powerful language at the same time: error propagation (whether manually by way of Maybe types or automated by exception/panic), deterministic finalization (RIAA, which isn't often described as a monad but definitely is one), and asynchronous execution (without threads). You are thereby inside of an asynchronous function (which in both Rust/C++ you have to manage manually using async/await) and need deterministic finalization (which in both Rust/C++ you can have automated using deconstructors), and yet errors can happen (which makes the problem obvious but is not at all requires for the problem to happen).
As a basic example, you finish your SSL request, and now need to deconstruct the SSL connection, which means you want to close the socket... only no, you first need to do an asynchronous SSL CloseNotify, so you have to await that behavior before you allow that socket object to deconstruct. If you let the object deconstruct before you await the CloseNotify, you haven't really completed the protocol. This is also a great case, as while HTTP itself mitigates this (and even in my system I eventually decided to just close this socket by force), you aren't really supposed to trust where the end of the data you got from the server is (and so in the abstract thought process you can't really return from this function until you've finished this CloseNotify; this means that trying to fire-and-forget this to a background thread--which is essentially the analogy of garbage collection--is not a valid solution).
I thereby end up with abstractions that look like an asynchronous "using" statement for that case, to prevent me from constructing the object without showing that I'll asynchronously deconstruct it, and I have primitives now for the cases where it is safe to shift to another thread (these are generally cases where I already had a shared pointer to something needing this behavior), or the case where it is primarily (and permanently) owned by another object (though this only half-solves the problem as the order of operations hasn't generalized yet)... and I've set it up so you have to explicitly say "I know what I'm doing" by mixing in another subclass in order to instantiate this object at all outside of one of my more "guaranteed" mechanisms, and yet the problem remains, and is just all over anything that involves networking: you have to be able to guarantee that you have completed asynchronous deconstruction of the object (which might fail! this makes the whole problem harder, but doesn't really change its character at all) before allowing a synchronous deconstruction.
I have seen attempts in both the Rust and C++ communities to then define what an "asynchronous deconstructor" would look like (fwiw, the C++ one looks a lot better), but they frankly all miss the mark (and lead to other soundness issues, which is why they are stuck in design phases and when they come out will likely have tons of compromises... the Rust proposal doesn't even look truly asynchronous) as the core problem here is really one that comes down to linear types: you need to guarantee that the object's lifetime doesn't end--at compile time, or you are just throwing deconstructor bombs at the problem--before you've finished its asynchronous deconstructor "somehow" (and that can be by any number of techniques, and is all up to the developer).
And then, as I noted: this has been the #1 source of subtle bugs in the systems that I build, as this is essentially "fearless concurrency" taken to the extreme of "fearless networking" (though to really approach that you have to have a serious conversation about "what is an error, anyway?" that I'm not sure the Rust community will ever accept ;P). So yeah, like: it is super disappointing to hear that Rust might just never end up with linear typing because I guess people understand the need for it about as little as C++ programmers appreciate the need for lifetime analysis (which is ironically in a design space very related to linear types, and why I'd expect that if I go to all that trouble I'd also be given a solution to my asynchronous deconstructor problem that didn't involve deconstructor bombs; like, I will again say that I loved that footnote #3 ;P).
> I am trying to explain why the semantics of this way of doing the replacement is cleaner and requires less weird special cases in the code (such as this object that is temporarily dead and suddenly revived) than the way in the article (which I see as a strawman designed to make me dislike linear types).
I've always read that article as exposing linear types as more complicated than just slapping a attribute, at least within the context of Rust, but it might just be a naive view of it.
> The whole point of linear types is to make it so that this is a compile-time error, so I don't really understand your objection. If it helps, I will try this in a different tense: if the type doesn't (in the continual sense) manage to provide a deconstructor which eats the object (and could be used on unwind), then either the user had to have (in the past tense) or the code didn't (in the past tense) compile if there were (at the time of compile) any codepaths without nothrow that could cause such an unwind.
But the point of using linear types is that there are constrains which the normal drop does not fulfil or allow for e.g. the deconstruction is failible, or requires additional data, or (as seems to be your case) needs to be moved off-thread. During an unwind, these things are not guarantee or even available at all (how would unwinding know what parameters to provide? How can it report errors to a caller which doesn't exist?). Otherwise the regular Drop would work fine.
> I think I lost an "a-" there, so I'm hoping that if I said "must deconstruct asynchronously" this makes more sense?
It does make a lot more sense yes.
> As a basic example, you finish your SSL request, and now need to deconstruct the SSL connection, which means you want to close the socket... only no, you first need to do an asynchronous SSL CloseNotify, so you have to await that behavior before you allow that socket object to deconstruct. If you let the object deconstruct before you await the CloseNotify, you haven't really completed the protocol. This is also a great case, as while HTTP itself mitigates this (and even in my system I eventually decided to just close this socket by force), you aren't really supposed to trust where the end of the data you got from the server is (and so in the abstract thought process you can't really return from this function until you've finished this CloseNotify; this means that trying to fire-and-forget this to a background thread--which is essentially the analogy of garbage collection--is not a valid solution).
Ah I see, even in the original I think I'd have misunderstood what you meant by asynchronous deconstruction because I wasn't thinking of the "async" context when I read your comment.
Anyway thanks for the expounding on your issues and their relevance to linear typing.