lengthUnits bits not allowed for complex type - Unnecessary restriction on length units?
In section 12.3.7.3 we have this sentence: "The dfdl:lengthUnits may be 'bytes' or 'characters' and it is a schema definition error otherwise. " Does anyone recall why we have this? I have data formats which are bit oriented and there are complex types which would naturally not be a multiple of 8 bytes long, e.g., 1 bit field, 3 bit field, 10 bit field, 6 bit field = 20 bits. I can't think of any reason for this restriction other than to explain how fillByte is used to fill in unused bits. But I think we can say that any unused bits are filled in with bits from the fillByte, and we don't have to be specific about which bits from the fillByte. Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policyhttp://www.ogf.org/About/abt_policies.php
I know on the call I agreed that complex types with length bits wouldn't be
needed, but I rechecked the format I am trying to implement, which is
MIL-STD-2045-47001D (which is public) and I found this (emphasis mine):
5.6.42 Group Size field.
This field shall be a 12-bit binary number indicating the size,* in
bits,*of the Future Use Group in which this field is
contained. A value of "0" should not be used for this field. If the parent
group is specified present then this child
field is mandatory.
Turns out that these things are all just binary blobs in this spec
document. There is no sub-structure provided as these are "Future Use", but
it's always possible they will get specified or already are and I am just
unaware of the document which gives the format of some of these. There are
a flock of related standards to this one. Most are, unfortunately, not
publicly available.
I am at a bit of a loss how to model these bit-length fields. HexBinary
type allows length only in bytes to be compatible with XSD hexBinary which
has a string-of-hex representation.
I suppose I could use an xs:nonNegativeInteger with up to 4096 bits as the
blob. Is there any other viable option? (array of bit is not viable)
My concrete suggestion: this data format is the same one that motivates the
proposed dfdl:bitOrder property. I suggest that dfdl:bitOrder, along with
any other issues needed to implement this standard, all be addressed at
once when I've had time to complete an initial implementation in the
daffodil code base.
Lifting the restriction on these lengthUnits bits for complex types may
well be required.
...mikeb
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology |
www.tresys.com
Please note: Contributions to the DFDL Workgroup's email discussions are
subject to the OGF Intellectual Property
Policyhttp://www.ogf.org/About/abt_policies.php
On Tue, Mar 25, 2014 at 9:01 AM, Mike Beckerle
In section 12.3.7.3 we have this sentence:
"The dfdl:lengthUnits may be 'bytes' or 'characters' and it is a schema definition error otherwise. "
Does anyone recall why we have this?
I have data formats which are bit oriented and there are complex types which would naturally not be a multiple of 8 bytes long, e.g., 1 bit field, 3 bit field, 10 bit field, 6 bit field = 20 bits.
I can't think of any reason for this restriction other than to explain how fillByte is used to fill in unused bits. But I think we can say that any unused bits are filled in with bits from the fillByte, and we don't have to be specific about which bits from the fillByte.
Mike Beckerle | OGF DFDL Workgroup Co-Chair | Tresys Technology | www.tresys.com Please note: Contributions to the DFDL Workgroup's email discussions are subject to the OGF Intellectual Property Policyhttp://www.ogf.org/About/abt_policies.php
Mike,
I think the expectation was that such fields would be modelled as unsigned
integers. The issue then becomes one of max size.
From your MIL spec quote below, it looks like this type is dfdl:lengthKind
'prefixed', with dfdl:prefixLengthIncludesPrefix 'yes' ?
Regards
Steve Hanson
Architect, IBM DFDL
Co-Chair, OGF DFDL Working Group
IBM SWG, Hursley, UK
smh@uk.ibm.com
tel:+44-1962-815848
From: Mike Beckerle
participants (2)
-
Mike Beckerle -
Steve Hanson