Please review - New version of bit order/MIL-STD-2045 document
I posted this to our documents. http://redmine.ogf.org/dmsf_files/13300 This version is quite different. This version proposes that there are actually two different variations on the least-significant-bit first format. One is where the bits are actually reversed (logical 0xE2 becomes 0x47 when unparsed to a file) The other is where the bits are simply interpreted as LSB first, but the bytes are stored 'normally'. (0xE2 is in the file, but the 'first' 3 bits of it are 010, not 111.) Turns out it is this latter form that MIL-STD-2045 and related standards need, and that the Daffodil code implements today. I hope this version clarifies some things. You'll see there is a need to come up with useful names for these concepts when you read the document. ...mikeb Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php>
Mike We have reviewed the document. So MIL-STD-2045 does not in fact reverse the physical bit order on the wire. As you point out in your comment, you don't know of a format that does. Therefore we do not need three enums for dfdl:bitOrder for DFDL 1.0. We should just add the enums that we need, like we propose for dfdl:byteOrder. However we are still having trouble relating your description of Number Model to the example quoted from MIL-STD-2045. Let me give an example of what we understand the MIL-STD-2045 example to be implying, and you can say whether we have the correct understanding. Assume big-endian byte order and twos complement binary for integers. You'll need to look at this in rich text to see the colours. Logical data: Struct length=16 bits FieldA type=int, length=3 bits, value=3 FieldB type=int, length=7 bits, value=9 FieldC type=int, length=4 bits, value=5 FieldD type=int, length=2 bits, value=2 MSBF (how current DFDL parser would expect bits): 01100010 01010110 AAABBBBB BBCCCCDD M L M L LSBF (how MIL-STD-2045 would expect bits): 01001011 10010100 BBBBBAAA DDCCCCBB M L M L If that is correct, then we find this a good way to illustrate the concept. How analogous is this bit ordering stuff to bi-directional text? I ask because we have properties textBidiOrder 'implicit' & 'visual', and textBidiOrientation 'LTR' & 'RTL'. If there is a good analogy then we could use the same terminology. If it is not then we should not. Though it should be pointed out that we have deferred action 241 so this area is not fully understood. Regards Steve Hanson Architect, IBM DFDL Co-Chair, OGF DFDL Working Group IBM SWG, Hursley, UK smh@uk.ibm.com tel:+44-1962-815848 From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Date: 17/07/2014 23:29 Subject: [DFDL-WG] Please review - New version of bit order/MIL-STD-2045 document Sent by: dfdl-wg-bounces@ogf.org I posted this to our documents. http://redmine.ogf.org/dmsf_files/13300 This version is quite different. This version proposes that there are actually two different variations on the least-significant-bit first format. One is where the bits are actually reversed (logical 0xE2 becomes 0x47 when unparsed to a file) The other is where the bits are simply interpreted as LSB first, but the bytes are stored 'normally'. (0xE2 is in the file, but the 'first' 3 bits of it are 010, not 111.) Turns out it is this latter form that MIL-STD-2045 and related standards need, and that the Daffodil code implements today. I hope this version clarifies some things. You'll see there is a need to come up with useful names for these concepts when you read the document. ...mikeb Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Thanks much for giving this quick attention. I think your example is right. That's what those tables in the 2045 spec are conveying. I will incorporate your example next spin, colors and all. Your idea about the bidi terminology is interesting. I'll have to mull it over. I did circulate this doc also to someone familiar with 2045 and link16 over at Lockheed-Martin, for feedback as well. I'd like to see if he agrees that the truly-reversed bits isn't needed before we decide not to have that capability. It's quite possible that both are needed. But I agree we should put in what we know is needed, and not speculative additional variations unless we have some use case. ...mikeb Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policy <http://www.ogf.org/About/abt_policies.php> On Fri, Jul 18, 2014 at 9:55 AM, Steve Hanson <smh@uk.ibm.com> wrote:
Mike
We have reviewed the document.
So MIL-STD-2045 does not in fact reverse the physical bit order on the wire. As you point out in your comment, you don't know of a format that does. Therefore we do not need three enums for dfdl:bitOrder for DFDL 1.0. We should just add the enums that we need, like we propose for dfdl:byteOrder.
However we are still having trouble relating your description of Number Model to the example quoted from MIL-STD-2045. Let me give an example of what we understand the MIL-STD-2045 example to be implying, and you can say whether we have the correct understanding. Assume big-endian byte order and twos complement binary for integers. You'll need to look at this in rich text to see the colours.
*Logical data:* Struct length=16 bits FieldA type=int, length=3 bits, value=3 FieldB type=int, length=7 bits, value=9 FieldC type=int, length=4 bits, value=5 FieldD type=int, length=2 bits, value=2
*MSBF (how current DFDL parser would expect bits):* *011**00010 01**0101**10* *AAA**BBBBB BB**CCCC**DD* * M L M L *
*LSBF (how MIL-STD-2045 would expect bits):* *01001**011 **10**0101**00* *BBBBB**AAA** DD**CCCC**BB* * M L M L *
If that is correct, then we find this a good way to illustrate the concept.
How analogous is this bit ordering stuff to bi-directional text? I ask because we have properties textBidiOrder 'implicit' & 'visual', and textBidiOrientation 'LTR' & 'RTL'. If there is a good analogy then we could use the same terminology. If it is not then we should not. Though it should be pointed out that we have deferred action 241 so this area is not fully understood.
Regards
Steve Hanson Architect, *IBM DFDL* Co-Chair, *OGF DFDL Working Group* <http://www.ogf.org/dfdl/> IBM SWG, Hursley, UK *smh@uk.ibm.com* <smh@uk.ibm.com> tel:+44-1962-815848
From: Mike Beckerle <mbeckerle.dfdl@gmail.com> To: "dfdl-wg@ogf.org" <dfdl-wg@ogf.org>, Date: 17/07/2014 23:29 Subject: [DFDL-WG] Please review - New version of bit order/MIL-STD-2045 document Sent by: dfdl-wg-bounces@ogf.org ------------------------------
I posted this to our documents. *http://redmine.ogf.org/dmsf_files/13300* <http://redmine.ogf.org/dmsf_files/13300>
This version is quite different. This version proposes that there are actually two different variations on the least-significant-bit first format.
One is where the bits are actually reversed (logical 0xE2 becomes 0x47 when unparsed to a file)
The other is where the bits are simply interpreted as LSB first, but the bytes are stored 'normally'. (0xE2 is in the file, but the 'first' 3 bits of it are 010, not 111.)
Turns out it is this latter form that MIL-STD-2045 and related standards need, and that the Daffodil code implements today.
I hope this version clarifies some things. You'll see there is a need to come up with useful names for these concepts when you read the document.
...mikeb
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | *www.tresys.com* <http://www.tresys.com/> Please note: Contributions to the DFDL Workgroup's email discussions are subject to the *OGF Intellectual Property Policy* <http://www.ogf.org/About/abt_policies.php> -- dfdl-wg mailing list dfdl-wg@ogf.org https://www.ogf.org/mailman/listinfo/dfdl-wg
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
participants (2)
-
Mike Beckerle -
Steve Hanson