[dlc-dev] Message Serialization Changes

Thibaut Le Guilly thibaut at cryptogarage.co.jp
Wed Oct 6 04:41:56 CEST 2021


So I gave a deeper look at bincode specs (probably should have done that
before sending my previous mail...)

The main difference I could spot with our/LN specifications is that similar
to us they have two types of encodings for (unsigned) integers, fixed sized
and variable size. Unfortunately the specs seem to hint that only one can
be used at a time, while we mix them. Furthermore their VarintEncoding
differs from the BigSize in the LN specs. I don't know how others will feel
but having two different versions of variable integers doesn't sound great
to me...


On Wed, Oct 6, 2021 at 11:14 AM Lloyd Fournier <lloyd.fourn at gmail.com>
wrote:

> On Wed, 6 Oct 2021 at 12:51, Thibaut Le Guilly <thibaut at cryptogarage.co.jp>
> wrote:
>
>> Hi Lloyd,
>>
>> Personally I really like the idea! Though we still want to be compatible
>> with LN TLVs so I think we would need to still adhere to the LN specs for
>> external messages.
>>
>
> Does this mean just wrap the DLC message in a lightning compat TLV? Yeah
> that's what I was thinking.
>
>>
>>  > It would allow you to simplify the specs by removing all the language
>> around your own conception of types and just use the rust language as the
>> standard.
>>
>> This part I'm not sure. I think the specs should be language agnostic so
>> it'd be probably better to properly specify everything even if we're just
>> repeating the bin-code specs.
>>
>
> Sure. You could go so far as to make the specs rust types but that's not
> necessary. For me, just because the spec happens to be valid rust code does
> not make it less language agnostic. The actual rust code you'd write while
> implementing it might still be different. I guess the concern is that it
> might make people feel the spec is designed to be implemented only in rust
> if it looks too rustish.
>
>>
>> On Wed, Oct 6, 2021 at 10:38 AM Lloyd Fournier <lloyd.fourn at gmail.com>
>> wrote:
>>
>>> Hi Thibaut,
>>>
>>> Looking at https://github.com/discreetlogcontracts/dlcspecs/pull/163 it
>>> looks like what you are designing is almost the same as bincode. Bincode is
>>> a binary serialization format popular in rust since the beginning of the
>>> language and in use by google/mozilla and many others.
>>>
>>> There is a specification in progress in a branch on the repo here:
>>> https://github.com/bincode-org/bincode/blob/feature/deserde/docs/spec.md
>>> It was not a specification driven projec from the beggining but the
>>> implementation has been stable and in use for a long time and it looks like
>>> it's going that way.
>>>
>>> What do you think about just using bincode to specify DLCs? It would
>>> allow you to simplify the specs by removing all the language around your
>>> own conception of types and just use the rust language as the standard.
>>> Your type definitions are already very rust like. For example instead of
>>> this markdown spec [1] it'd be something like:
>>>
>>> ```
>>> enum ContractDescriptor {
>>>     Enumerated { outcomes: Vec<(String, u64)> }
>>>     Numeric { num_digits: u16, payout_function: PayoutFunction }
>>> }
>>>
>>> struct PayoutFunction {
>>>  ...
>>> }
>>> ```
>>>
>>> Putting #[derive(serde::Serialize, serde::Deserialize)] and using
>>> bincode would give you almost exactly the same output as you have specified.
>>> The only small difference  is that bincode starts at 0 whereas you start
>>> at 1 for "sibling" types and also an option is just a "sibling" type (enum).
>>> Leveraging a simplified rust like language as the specification language
>>> and bincode as the serialization spec seems like it would save a lot of
>>> work.
>>> Even if I was implementing it in another language I think it'd be easier
>>> to work from that.
>>>
>>> Cheers,
>>>
>>> LL
>>>
>>> P.S. I hope I am not delaying the decision here. The suggestion is
>>> designed to make things easier. If not please ignore!
>>>
>>> [1]
>>> https://github.com/discreetlogcontracts/dlcspecs/blob/0103cc55c7d5aa508326fd150744457ed6d06e68/Messaging.md#the-contract_descriptor-type
>>>
>>> On Mon, 20 Sept 2021 at 21:56, Thibaut Le Guilly <
>>> thibaut at cryptogarage.co.jp> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> > Things that Tibo doesn't like about the current Messaging.md
>>>> <https://github.com/discreetlogcontracts/dlcspecs/blob/master/Messaging.md>
>>>> >
>>>> > - Inefficient encoding at the byte level
>>>>
>>>> What I mainly don't like about the current format, even without
>>>> thinking about the inefficiency, is that we have hard coded values that
>>>> need to be kept around in implementations for no good reason (in addition
>>>> to length prefixes).
>>>> Also we are supposed to ask to reserve the type numbers we are using
>>>> from the custom LN message space (at some point?), but are we really going
>>>> to ask for like 20 types when in practice we actually need maximum 4? Now
>>>> assuming we ask only for 4, we end up having future messages with type
>>>> numbers completely apart from the first ones (e.g. see
>>>> https://github.com/discreetlogcontracts/dlcspecs/pull/170#discussion_r676235523
>>>> ).
>>>> But to be clear, my main motivation in reviewing the serialization is
>>>> getting rid of ~15 constants that are of no use, and a maintenance burden.
>>>>
>>>> Apart from that, the proposals also give us:
>>>> * The increased in efficiency you mentioned
>>>> * TLV streams at the end of messages for extensibility
>>>> * Protocol versioning (if we want that, personally I think we do)
>>>>
>>>> Then I must apologize but I didn't really get the rest of your message
>>>> about variance of subtypes. In most aspects I think the proposed changes
>>>> give a functionally equivalent format compared to the previous one. I think
>>>> there was maybe the discussion about whether we wanted optional fields or
>>>> not but I don't feel that's what you're referring to so I would need a bit
>>>> more explanation to answer you.
>>>>
>>>> > I think it would be useful for Tibo to write out his current
>>>> implementation concerns, as we have discussed it on calls, but there hasn't
>>>> been a writeup on either #163/#171 indicating the problems with the current
>>>> specification.
>>>>
>>>> Good point updated the description of #163 (
>>>> https://github.com/discreetlogcontracts/dlcspecs/pull/163#issue-621730888
>>>> )
>>>>
>>>> Tibo
>>>>
>>>> On Mon, Sep 20, 2021 at 3:18 AM Chris Stewart <chris at suredbits.com>
>>>> wrote:
>>>>
>>>>> So when reading through both these PRs, here is my high level
>>>>> takeaway. I'm going to abstain from commenting on specific proposals in
>>>>> this email until I am sure I understand the intention of Tibo with his
>>>>> proposals.
>>>>>
>>>>> Here is my understanding of Tibo's concerns, and I apologize in
>>>>> advance if I mischaracterize his view points.
>>>>>
>>>>> Things that Tibo doesn't like about the current Messaging.md
>>>>> <https://github.com/discreetlogcontracts/dlcspecs/blob/master/Messaging.md>
>>>>>
>>>>> - Inefficient encoding at the byte level
>>>>> - Nesting of TLV types
>>>>>
>>>>> Things Tibo does like about the the current Messaging.md format
>>>>>
>>>>> - Allows for variance of subtypes (I'm assuming this as in #163/#171
>>>>> we still have variance of nested types, albeit encoded slightly
>>>>> differently).
>>>>>
>>>>> From taking a quick glance at the Lightning message spec
>>>>> <https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#channel-establishment>,
>>>>> it appears they have no variance in their message types. We have discussed
>>>>> this as a desirable feature on dev calls. How many messages would we have
>>>>> if we wanted to take this approach to this messaging spec?
>>>>>
>>>>> It seems like this could blow up in complexity rather fast, as some of
>>>>> deepest types (such as contract descriptor
>>>>> <https://github.com/discreetlogcontracts/dlcspecs/blob/master/Messaging.md#the-contract_descriptor-type>)
>>>>> have variance associated with it (enumerated contracts, numeric contracts)
>>>>> and then are used in higher level types such as contract_info
>>>>> <https://github.com/discreetlogcontracts/dlcspecs/blob/master/Messaging.md#the-contract_info-type>.
>>>>> My understanding is we would lose a lot of flexibility, especially in the
>>>>> context of disjoint union contracts where we allow arbitrary enumerated and
>>>>> numeric contract descriptors to be combined. Please correct me if this is
>>>>> wrong.
>>>>>
>>>>> ### Things I like about the new proposal
>>>>>
>>>>> Things that I do like about the sibling concept is it gives type
>>>>> locality. For instance, if you look at Tibo's proposed contract info
>>>>> type
>>>>> <https://github.com/discreetlogcontracts/dlcspecs/pull/163/files#diff-b8a9c2888fe05c198758fbe393798cd0802b578f7eb07dbf82cc2c90616f97c3R139>you
>>>>> can see there now is a clear relationship via the sibling type between
>>>>> `single_contract_info` and `multi_contract_info`, where as in the current
>>>>> messaging spec, AFAIK, there isn't a relationship between the two tlv types
>>>>> we allocated for these messages which are 55342 and 55344. This seems very
>>>>> similar to algebraic data types to me.
>>>>>
>>>>> I think it would be useful for Tibo to write out his current
>>>>> implementation concerns, as we have discussed it on calls, but there hasn't
>>>>> been a writeup on either #163/#171 indicating the problems with the current
>>>>> specification.
>>>>>
>>>>> -Chris
>>>>>
>>>>>
>>>>> On Sat, Sep 18, 2021 at 11:57 AM Nadav Kohen via dlc-dev <
>>>>> dlc-dev at mailmanlists.org> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> There are currently two open proposals for simplifying and improving
>>>>>> the current P2P message serialization protocol:
>>>>>>
>>>>>> https://github.com/discreetlogcontracts/dlcspecs/pull/163
>>>>>> https://github.com/discreetlogcontracts/dlcspecs/pull/171
>>>>>>
>>>>>> Which of these proposals do we prefer? They are apparently both very
>>>>>> close in terms of actually implementing but have structural differences
>>>>>> (e.g. one is closer, but still not in agreement, with the LN TLV model
>>>>>> whereas the other makes slightly bigger structural improvements).
>>>>>>
>>>>>> Thibaut (and I) would love to hear your thoughts.
>>>>>>
>>>>>> Best,
>>>>>> Nadav
>>>>>>
>>>>>> dlc-dev mailing list
>>>>>> dlc-dev at mailmanlists.org
>>>>>> https://mailmanlists.org/mailman/listinfo/dlc-dev
>>>>>>
>>>>>
>>>>> dlc-dev mailing list
>>>>> dlc-dev at mailmanlists.org
>>>>> https://mailmanlists.org/mailman/listinfo/dlc-dev
>>>>>
>>>>
>>>> dlc-dev mailing list
>>>> dlc-dev at mailmanlists.org
>>>> https://mailmanlists.org/mailman/listinfo/dlc-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailmanlists.org/pipermail/dlc-dev/attachments/20211006/4c7eaa6b/attachment-0001.htm>


More information about the dlc-dev mailing list