[dlc-dev] Breaking Changes Discussion

[Chris Stewart]

Replying to Nadav but meant for Thibaut, not sure why i don't have the
reply all option on his email (I can only reply to him, but i want the
entire list to see this)

>Maybe the mistake here is to merge specs into master that have not been
implemented by multiple parties yet? I think that's for example the rules
for BOLTs that don't become accepted before being supported by two
implementations. For example, I didn't notice anything wrong with the type
prefixes in the serialization while reviewing the specs, but only when
starting implementing it (which is why it got merged).

I think this is a good policy to enact. If someone is feeling ambitious it
might be worthwhile to try and list out things that are already merged into
the spec, and then who has implemented those.

For things that have no implementor, we should probably remove them from
the spec, and I guess we can discuss what to do with things in the spec
that have 1 team that has implemented. This probably needs to be reviewed
on a case by case basis. Anything that has 2+ teams implemented, we should
keep in the spec IMO.

-Chris

On Tue, Sep 28, 2021 at 9:57 PM Nadav Kohen <nadavk25 at gmail.com> wrote:

> Hi all,
>
> Here are my current thoughts on these questions, would be happy to
> continue this discussion though as I am open to changing my positions.
>
> 1) I agree that the solution here seems to be having a way for nodes to
> detect and broadcast the version they are running. I believe this is easily
> solvable by adding a `protocol_version` field as Thibaut has already
> proposed (and perhaps an additional error message detailing that you have
> an incompatible version). That said, adding such a field is itself a
> breaking change and so I would like to note that in my mind, the
> responsibility on existing deployed nodes of dealing with this problem lies
> with the implementations themselves.
>
> 3) I agree that our goal should be general DLC-ecosystem compatibility not
> only between different implementation's nodes but also between different
> backends and front-ends and other such components that should be
> compatible. Since this is the goal of having a specification, my view on 2
> is the following:
>
> 2) We should not stop making breaking changes to the DLC specification. As
> it currently stands, no implementation is fully specification compliant and
> no implementations are currently cross-compatible (beyond small components
> e.g. bitcoin-s oracle and atomic-finance client, but no two client-side
> implementations are compatible in any real sense). As such, there is
> currently no holistic gain (specifically the goals mentioned above by Chris
> and by me for 3) to having the specification as is, and more specifically I
> do not believe that any implementation currently has plans of supporting
> the entire specification as it currently exists, without any breaking
> changes.
>
> My view is that what has happened is that around February or March of this
> year (2021), many implementations had a change in priorities from trying to
> build out the spec and implementations approaching compatibility with each
> other to building stand-alone self-compatible implementations that were
> user-facing, user-friendly and usable by a larger audience. I don't think
> that this change in priorities is a negative thing, but I do think it
> happened and as a result breaking changes to the specification and to
> implementations essentially stopped happening altogether initiating an
> informal but functional freeze on some aspects of DLC development. I don't
> believe that individual implementations should be discouraged from pursuing
> stable versions of working (non-compliant) DLC code, quite the opposite,
> but I do want to argue that there is no real reason why this approach
> should hinder specification development.
>
> I don't see why the DLC specification should remain unchanged and
> unfinished due to resistance towards breaking changes if the
> implementations that are trying to be stable are already specification
> in-compatible and that, generally speaking, there aren't plans for
> cross-stable-implementation compatibility under any notion of "the current
> spec version". I see no reason why the specification can't continue to have
> non-backwards-compatible improvements that these stable implementations
> simply don't adopt until some future version of their software is released.
>
> Essentially my point is that there is no reason to view breaking changes
> to the spec as a burden on implementations since we don't *yet* live in a
> world of cross-implementation compatibility where breaking changes become
> important. Until we come close to this vision of the future, breaking
> changes should continue in the DLC specification until it reaches a stable
> *specification* version, that is, a specification version which is
> intended for implementations to view as stable and which is intended for
> use in cross-implementation compatibility. The current specification is not
> such a specification version and as such building on top of unfinished
> specifications comes with some annoyances such as having the responsibility
> of dealing with breaking changes lying with the implementation rather than
> the specification, especially in the current setting where implementations
> that are being released as user-facing aren't even current-specification
> compliant.
>
> 4) As has been stated at length above, I think our path forward to
> cross-compatibility does not come from immediately trying to be compatible
> with our implementations current versions' (which would be paradoxical
> since we all have different, non-compatible implementations), but rather by
> continuing to improve and finish up v0 of the DLC specification (which will
> involve multiple breaking changes) and once this version is reached it then
> becomes reasonable to attempt cross-implementation compatibility according
> to that finalized specification version and that specification version
> should be resistant to backwards-incompatible changes. On a more pragmatic
> note, once this  version is completed, I think test vectors for each
> specification document should be prioritized in order to facilitate
> cross-implementation compatibility and correctness.
>
> Looking forward to hearing what others have to say :)
>
> Best,
> Nadav
>
> On Sun, Sep 19, 2021 at 2:37 PM Chris Stewart <chris at suredbits.com> wrote:
>
>>
>>    1. The DLC specification and improvements to it are not done and it
>>    appears that (partial, non-compliant) implementations have become
>>    user-facing, what mechanisms are available to us that are the least painful
>>    for these applications that will allow DLC specification work to continue?
>>
>> I think with breaking changes, I think it needs to be easy for old
>> implementations to "fail fast and loud". Whatever the change is, it needs
>> to be easy for existing _deployed_ nodes to understand that there has been
>> a new feature into the dlc spec and their node software cannot understand
>> that upgrade unless they update. Perhaps this means you want to
>> automatically disconnect from your peer.
>>
>> 2. Are there explicit benefits to be gained by freezing development or
>> else tagging a version? It is currently unclear (at least to me) what falls
>> in the realm of specification-level concerns and what is
>> implementation/application-specific concerns.
>>
>> I don't know.
>>
>> 3. Explicitly, what are the purported/desired benefits from having a
>> unified specification for DLC development and how do these affect the above
>> questions?
>>
>> To allow for dlc wallets to collaboratively build DLCs between each
>> other. This means they need to speak the same negotiation protocol, and
>> agree on the onchain parameters for both funding transactions for the DLC
>> and the CETs. Similar to how C-lightning and LND can open channels between
>> each other, we want to open DLCs between Atomic Finance and Suredbits, or
>> Crypto Garage and Atomic Finance.
>>
>> 4. What is our path forward towards cross-implementation compatibility?
>>
>> I think this is a good question for implementations that are less
>> developed. AFAIK, bitcoin-s has tried to implement the specifications as
>> they get merged into master for the dlcspecs repo.
>>
>> There is a lot of work to be done outside of the specification regardless
>> of changing the spec, such as implementing a transport layer. We have
>> another email on the mailing list talking about changing the messaging
>> protocol, which introduces work for people that have implemented the
>> messaging spec, and doesn't really reduce the work (they still have to
>> implement something!) for people that haven't implemented the messaging
>> spec.
>>
>> In my mind, interoperability would mean that two implementations can open
>> a DLC with each other the same way two lightning implementations can open a
>> channel between each other. I'm not sure if any proposed changes on the
>> specification are going to help this, it's a matter of applying elbow
>> grease to get it done.
>>
>> The smallest integration test that could accomplish this is opening an
>> enumerated outcome DLC between two peers on the network.
>>
>> -Chris
>>
>> On Sat, Sep 18, 2021 at 11:53 AM Nadav Kohen via dlc-dev <
>> dlc-dev at mailmanlists.org> wrote:
>>
>>> Hi all,
>>>
>>> So at the last spec meeting we began a general discussion about making
>>> breaking changes to the DLC specs and it didn't really reach a conclusion
>>> so I'd like to continue that conversation here.
>>>
>>> Here is my attempt to summarize the state of things:
>>>
>>>    - There has functionally been a freeze on breaking changes to the
>>>    dlcspecs repo for many months now.
>>>    - It appears that this is largely due to wanting to be cautious
>>>    about making breaking changes to existing, user-facing applications.
>>>    - On the other hand, the specs are not done and furthermore, none of
>>>    the existing applications are (fully) spec-compliant.
>>>       - Most implementations currently agree on enumerated outcomes and
>>>       oracle messages, but little else.
>>>    - There are a good number of breaking changes in the pipeline
>>>    including (but probably not limited to)
>>>       - P2P message serialization
>>>       - Oracle messages and key infrastructure
>>>       - Event order specification
>>>       - Taproot
>>>    - Roughly, the stances I've heard so far is that
>>>       - Specification changes should continue and it should be on
>>>       implementations to decide how spec compliant (and to which spec version) to
>>>       be compliant with the goal that someday everyone is compliant to the same
>>>       thing (noting no one is currently at that state anyway)
>>>       - Specification changes shouldn't be made very often and there
>>>       should be some mechanism to group a bunch of changes together (e.g. all
>>>       four of the above as one protocol update with a new version)
>>>
>>>
>>> I won't put my personal opinions in this discussion starter email (I
>>> will likely do this later in the thread), but here are the questions I
>>> think need to be discussed:
>>>
>>>
>>>    1. The DLC specification and improvements to it are not done and it
>>>    appears that (partial, non-compliant) implementations have become
>>>    user-facing, what mechanisms are available to us that are the least painful
>>>    for these applications that will allow DLC specification work to continue?
>>>    2. Are there explicit benefits to be gained by freezing development
>>>    or else tagging a version? It is currently unclear (at least to me) what
>>>    falls in the realm of specification-level concerns and what is
>>>    implementation/application-specific concerns.
>>>    3. Explicitly, what are the purported/desired benefits from having a
>>>    unified specification for DLC development and how do these affect the above
>>>    questions?
>>>    4. What is our path forward towards cross-implementation
>>>    compatibility?
>>>
>>>
>>> There are likely other considerations I'm missing here so please feel
>>> free to respond on this thread with other things you think are important.
>>> Also people should feel free to respond to any number (including just one)
>>> of these questions.
>>>
>>> Best,
>>> Nadav
>>>
>>> dlc-dev mailing list
>>> dlc-dev at mailmanlists.org
>>> https://mailmanlists.org/mailman/listinfo/dlc-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailmanlists.org/pipermail/dlc-dev/attachments/20211005/39534b17/attachment.htm>