- dfdl-wg - lists.ogf.org

binary xml document from w3c
by mike.beckerle＠ascentialsoftware.com 08 Apr '05

08 Apr '05

http://www.w3.org/TR/2005/NOTE-xbc-characterization-20050331/ <http://www.w3.org/TR/2005/NOTE-xbc-characterization-20050331/> This is dated 31 March 2005. This is just FYI. Since I just ran across this. ...mikeb

1 0

tagged data examples
by mike.beckerle＠ascentialsoftware.com 08 Apr '05

08 Apr '05

DFDLers, Suman and I were discussing a particular data format problem that I undertook to solve in DFDL. I thought it would be good to bring the discussion to the whole group. The problem is tagged data fields. Many computer messaging formats use things like this for some or all of their data fields. Here are two records containing a first and last name in a tagged format: fname:Tim!LName:Stewart; LASTNAME:Smith!firstName:Tom; Notes: the tags have varying forms, i.e., fname, firstname firstName, FIRSTNAME, FNAME, all are accepted as the tag for the first name field. The definition here is that it is case insensitive and either fname or firstname forms. Similar for lastname. Also the tagged fields can appear in any order, and are optional. Here's my test file showing how the XML comes out: (I've attached these as files also in case the email system hammers them.) This is testTaggedData1.xml <?xml version="1.0" encoding="iso-8859-1"?>  <dfdlTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://dataformat.org/testCase" xmlns:tc="http://dataformat.org/testCase" xsi:schemaLocation="http://dataformat.org/testCase ../../xsd/testCase.xsd http://dataformat.org/tests testTaggedData1.dfdl.xsd"> <inputTest>  <data kind="text">fname:Tim!LName:Stewart; LASTNAME:Smith!firstName:Tom; </data> <dfdlSchema file="testTaggedData1.dfdl.xsd"/> <tc:xmlResult xmlns="http://dataformat.org/tests"> <myData> <custInfo> <firstName>Tim</firstName> <lastName>Stewart</lastName> </custInfo> <custInfo> <firstName>Tom</firstName> <lastName>Smith</lastName> </custInfo> </myData> </tc:xmlResult> </inputTest> </dfdlTest> Now, the DFDL itself There are 3 variants here. I'll start with the simplest one. You get a very simple DFDL for this if you assume you can have (a) a way to specify the values of initiators, terminators, and separators as regular expressions (b) support for xsd:all groups. This is testTaggedData3.dfdl.xsd (ok, this one has long lines, so the email system is sure to hammer it, so I won't inline it here.) I think this particular example is pretty straightforward. However, I have two other example DFDL schemas for this which make fewer assumptions. testTaggedData2.dfdl.xsd still allows one to specify regular expressions for the initiator rep property, but does not allow use of xsd:all. Which is a construct I *was* trying to avoid because, well, it's complicated and feels non-primitive. I think you'll agree the complexity goes up significantly. testTaggedData1.dfdl.xsd eliminates specifying the initiator at all, and specifies the tags by way of an additional field hidden in a hidden-layer which has value constrained by an XSD pattern facet to match a specific regular expression. It also does not use xsd:all. My summary from going through this exercise: We need both xsd:all support, and regular expressions for initiators, and all delimiters. I'm happy that one can express these things without needing these constructs, but tagged representations are too commonplace for this much complex construction to be required. The complex constructions I used in testTaggedData1 and testTaggedData2 would only be needed if the tags were complex formatted entities the format of which couldn't be handled by a regular expression. Note that if the tags are actually not case insensitive, but are really fixed strings, then there is no need for the regular expression capability. I'm not sure where we should draw the line here. I'm comfortable with xsd:all support, and plain strings as delimiters or with regexps as delimiters. ...mikeb

2 1

F2F planning?
by Myers, James D 05 Apr '05

05 Apr '05

Any progress on deciding on the date/time/place for a f2f meeting? As I mentioned, if it's Toronto, Tara and I will need a minimum of 30 days notice to be able to attend, which means we'd have to file paperwork by Friday to meet the earliest suggested dates (week of May 9). Thanks, Jim

1 0

New document
by mike.beckerle＠ascentialsoftware.com 31 Mar '05

31 Mar '05

https://forge.gridforum.org/projects/dfdl-wg/document/ggf-dfdl-proposed-2005 -004/en/1 That's the URL to the new document on Grid Forge. I plan to make this document describe the ASCL DFDL Prototype language as we fill it out. Right now there's one important discussion in it that I'd like people to look at and comment on, which is of scoping of DFDL annotations.

1 0

Today's call
by Steve Hanson 31 Mar '05

31 Mar '05

Due to the extra hour time difference this week, I may have trouble joining the call today. If anyone has an agenda, please post here. One thing I wanted to bring up - when and where of the next f-2-f. Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848

1 0

Fw: [dfdl-wg] Annotation complexity
by Steve Hanson 17 Mar '05

17 Mar '05

Some thoughts on this from one of the IBMers on the XML schema WG. Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 02/12/2004 10:05 ----- Sandy Gao/Toronto/IBM@I BMCA To Steve Hanson/UK/IBM@IBMGB 30/11/2004 14:58 cc Suman Kalia/Toronto/IBM@IBMCA Subject Re: Fw: [dfdl-wg] Annotation complexity(Document link: Steve Hanson) > One more thought in this area. XML Schema itself does not have the notion of inheriting properties from a parent object. I think by "parent object" you mean the containing component, instead of something like the "base type". I don't think schema has a general principle on this. The only case I can think of where things are inherited (or defaulted using values from parent) is for blockDefault, finalDefault, elementFormDefault and attributeFormDefault attributes on the <schema> element. As for your specific example, I don't know whether the Schema WG has considered it (my involvement in schema doesn't go that far back), but my intuition here is that I don't think that's something schema would want to do. I think defaults are for the common cases, but it's not that common for all elements inside a complex type to share the same type. Don't know whether that answered your question. Let me know if there's anything more I can do. Cheers, Sandy Gao Software Developer, IBM Canada (1-905) 413-3255 sandygao(a)ca.ibm.com ----- Forwarded by Steve Hanson/UK/IBM on 30/11/2004 11:39 ----- Steve Hanson/UK/IBM To 23/11/2004 09:38 Sandy Gao/Toronto/IBM cc Suman Kalia/Toronto/IBM@IBMCA Subject Fw: [dfdl-wg] Annotation complexity Hi Sandy There is a Global Grid Forum (GGF) initiative underway called Data Format Description Language (DFDL) which aims to model non-XML data using XML schema for the logical structure and xsd:appinfo annotations for the physical 'bitstream' representation information. IBM is involved in this (myself and Suman Kalia) because the IBM message broker product (WVBI-MB) already does exactly this in its 'message model', so we are using our experience in this area to help influence this initiative. I'd like to ask your opinion on something quite specific. See the mail directly below. Did the XML Schema WG think about the possibility of providing an inheritance or defaulting capability as in my example? And if so, what were the precise reasons for excluding it? I'm asking because several folk in the DFDL WG are keen on having this as a way of making a DFDL annotated schema more compact. If you are interested in finding out more, please visit http://forge.gridforum.org/projects/dfdl-wg/ Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 23/11/2004 09:30 ----- Steve Hanson/UK/IBM To 23/11/2004 09:29 dfdl-wg(a)gridforum.org cc Subject Fw: [dfdl-wg] Annotation complexity One more thought in this area. XML Schema itself does not have the notion of inheriting properties from a parent object. For example, if I have a complex type that contains only integer elements, there is no facility to factor out the xsd:type attribute from all the elements and instead set it as an attribute of the complex type, thereby making the schema more compact. Was this a deliberate decision? I'll see if I can find out. Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 ----- Forwarded by Steve Hanson/UK/IBM on 23/11/2004 09:27 ----- Steve Hanson/UK/IBM To 22/11/2004 11:24 dfdl-wg(a)gridforum.org cc Subject RE: [dfdl-wg] Annotation complexity (Document link: Steve Hanson) > Another approach would be to allow annotation locations to be > specified using XPath. Since XPath can specify both specific > locations and large groups of locations this could be quite > powerful. We used to have a similar concept in the IBM broker message model, we called it 'element qualifier'. A top level global element could carry a reference (XPATH) to an element it ultimately contained, for the purposes of overriding certain properties, including default value. We found that: a) Their use made the model much harder to understand b) The code we had to write was considerably more complex c) No customer ever admitted to using the capability We withdrew the facility with no deprecation warning in V5 (June 2003) and so far nobody has shouted. Was this the kind of thing you are proposing? Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848 "Myers, James D" <jim.myers(a)pnl.go v> To Sent by: dfdl-wg(a)gridforum.org owner-dfdl-wg@ggf cc .org Subject RE: [dfdl-wg] Annotation complexity 19/11/2004 21:03 > > Myers, James D wrote: > > I guess I'm not sure how restricting annotations to elements solves > > things. > > Sorry, I am finding it more difficult to be precise than I > thought with > this. What I mean here is _leaf_ elements/attributes - which > I think can > be defined as elements with simple type descriptions. I think if you > stick to these its unambiguous...no? I agree, I think - if anotations are only on simple types. That would eliminate the element hierarchy, though - does it eliminate the type hierarchy - simple element in base type has an annotation that gets inherited by the same simple element in a derived type. I think this would be unambiguous, but perhaps hard to find (surprise the global dfdl default for endianess doesn't apply because some subtype three levels back had an annotation) > > I think Mike has examples of annotations that he would like higher in > the tree but with this extreme position there would be no inheritance. Except through type derivation...? > > Two other thoughts occur: > - I think definitions applied to a data source are > orthogonal to this > scoping issue. We still need to understand precedence but that seems > relatively easy to resolve. Definitions on sources: I know we've talked about this (!!! - the whole outside-in or inside out representations...), but I think the only mechanism for this in the examples is to assign a source to an element and annotate that element with, for example, endianness. For some types of sources, e.g. other layers, that have an internal structure, any annotation of the source itself would essentially become an inheritable annotation on the elements within it. So - we could try to define such a mechanism to apply to sources, but I'm not sure it would be applicable to all without recreating some of the inheritance issues. > > - Another approach would be to allow annotation locations to be > specified using XPath. Since XPath can specify both specific > locations > and large groups of locations this could be quite powerful. Hmmm. Yes, as long as xpaths can only be defined from a single root, i.e. not from within type definitions relative to the type (not talking about layers here). Otherwise we don't get rid of inheritance/precedence issues. At a minimum, this approach might help collect all of the defaults together at the top level for readability, but it might separate defaults from reusable types. Jim

1 0

Call today
by mike.beckerle＠ascentialsoftware.com 17 Mar '05

17 Mar '05

I will be on the call today. I can give a status report on the prototype we're doing here. ...mikeb

1 0

Interesting news: IBM Acquires Ascential
by mike.beckerle＠ascentialsoftware.com 14 Mar '05

14 Mar '05

Press Release IBM to Acquire Ascential Software Extends Leadership in Helping Customers Integrate, Structure and Manage Information ARMONK, NY and WESTBORO, Mass.-March 14, 2005-IBM (NYSE: IBM) and Ascential Software Corporation (Nasdaq: ASCL) today announced the two companies have entered into a definitive agreement for IBM to acquire the equity of Ascential Software, a publicly held company based in Westboro, Mass., in an all cash transaction at a price of approximately $1.1 billion or $18.50 per share. The acquisition is subject to Ascential Software shareholder and regulatory approvals and other customary closing conditions, and is expected to close in the second quarter of 2005. Ascential Software is a leading provider of enterprise data integration software. Customers worldwide use Ascential's data integration software to build enterprise data warehouses, power business intelligence systems, consolidate enterprise applications, create and manage master repositories of critical business information, and enable on demand data access. Ascential Software's integrated portfolio helps companies solve their most complex and demanding data integration challenges to streamline operations, improve customer support, expedite business transactions, and make better strategic decisions. The acquisition complements and strengthens IBM's fast-growing information integration business, a key part of the company's information management efforts and a unit that produced triple-digit growth in 2004. Ascential Software also grew rapidly in 2004, with a reported 46 percent total revenue increase to $271.9 million. "Information integration is an important enabler of an on demand business strategy and customers are increasing their investments in software that allows them to rapidly analyze, consolidate and extract value from their business data," said Steve Mills, senior vice president and group executive, IBM Software Group. "The acquisition of Ascential Software expands IBM's open information integration platform and strengthens our ability to help customers create an environment that delivers the data they need, in the right form, to the right location, and at the right time. That's the mark of an on demand business." The acquisition announced today addresses a key customer challenge -- applying technology to respond much more quickly to changing market conditions. For example, some of the largest retailers use Ascential's software to gather, standardize and structure sales information from multiple channels (for example, Internet, catalogs, storefronts), providing a single view of vendors and customers from disparate systems, and make rapid inventory and pricing adjustments in response to changing market demands. Enabling customers to react and respond with speed to rapidly emerging business opportunities is an example of IBM's on demand business strategy. Ascential Software's ability to quickly gather, move and enhance the quality of large amounts of data complements IBM's WebSphere Information Integrator product portfolio. WebSphere Information Integrator software enables customers to centrally manage and access data that is stored across a variety of structured and unstructured sources, from IBM and non-IBM vendors, in real-time. This combination will help customers solve numerous integration problems. For example, a company trying to consolidate data from multiple ERP systems into a single system could leverage WebSphere Information Integrator to access various mainframe or distributed sources for profiling and assessment, and then use Ascential Software's data migration and transformation capabilities to integrate the data. IBM is acquiring Ascential Software to extend the industry's broadest and deepest offering for standards-based business integration across heterogeneous environments. With more than 3,000 customers and partners across all major industries, Ascential Software is widely viewed as a leader in enterprise data integration. IDC, a global market intelligence and advisory firm in the information technology and telecommunications industries, projects that worldwide data integration spending will increase from $9.3 billion in 2003 to $13.6 billion in 2008. IDC also reports that the top 3 business issues most organizations say they need to be addressed through integration are to respond faster to changing business needs, improve operational productivity of employees, and provide better service to customers.1 "The integration market is changing and evolving as customers demonstrate a preference for broad solutions that meet enterprise integration requirements," said Peter Gyenes, chairman and chief executive officer, Ascential Software. "IBM shares our understanding that data integration is a strategic component of information and process integration and offered us this opportunity to combine and form a stronger competitor in the large and growing integration market. At the same time, this transaction offers Ascential Software shareholders solid appreciation over recent trading prices of our stock, without the execution and business risk." Once the acquisition is completed, IBM will: * Establish Ascential Software's operations as a business unit within IBM's Information Management software division led by General Manager Janet Perna. * Incorporate Ascential Software technology and solutions into IBM's Information Management and Software Group offerings. * Market and sell Ascential Software products through IBM's and Ascential Software's worldwide sales channels and Business Partners. * Build upon the companies' existing partnership, which includes more than 550 joint customers worldwide. Ascential Software already integrates with IBM WebSphere Business Integration software as part of a service-oriented architecture (SOA), providing complex data movement and transformation within a process step, thereby helping customers speed time-to-market and increase their ROI. This acquisition, combined with IBM's middleware portfolio, strengthens IBM's leadership in key on demand initiatives such as business intelligence, business performance management, business transformation, multi-channel commerce, RFID, merger and acquisition consolidation, master data management and regulatory compliance. The Ascential Software technology also furthers IBM's commitment to open computing through its ability to support both IBM and non-IBM data sources, including those of Oracle, Microsoft and SAP. About IBM IBM is the world's largest information technology company, with 80 years of leadership in helping businesses innovate. Drawing on resources from across IBM and key Business Partners, IBM offers a wide range of services, solutions and technologies that enable customers, large and small, to take full advantage of the new era of e-business. For more information about IBM, visit <http://www.ibm.com/> www.ibm.com. About Ascential Software Corporation Ascential Software Corporation (Nasdaq: ASCL) is the leader in enterprise data integration. Customers and partners worldwide use the Ascential Software Enterprise Integration Suite to confidently transform data into accurate, reliable and complete business information to improve operational performance and decision-making across every critical business dimension. Our comprehensive end-to-end solutions provide on demand data integration complemented by our professional services, industry expertise, and methodologies. Ascential Software is headquartered in Westboro, Mass., and has more than 3,000 customers and partners globally across such industries as financial services and banking, insurance, healthcare, retail, manufacturing, consumer packaged goods, telecommunications and government. For more information call 1-800-966-9875 (or 1-508-366-3888 from outside the U.S. or Canada) or visit the Ascential Software website at www.ascential.com. NOTE TO INVESTMENT COMMUNITY: Financial analysts can find more information regarding this acquisition at: www.ibm.com/investor. Ascential Software management will host a conference call at 8:00 AM (EST) today to discuss this important announcement. The conference call will be broadcast live through a link on the Investor Relations page on the Ascential Software web site at www.ascential.com/investors <http://www.ascential.com/investors> . For those who cannot attend the live broadcast, a replay will be available on the web site at www.ascential.com/investors <http://www.ascential.com/investors> or by calling (800) 945-1634 beginning about two hours after the call ends. No passcode is required for the replay. 1 IDC Worldwide Data Integration Spending 2004-2008 Forecast, July 2004 (IDC #31546), Steve McClure IBM, DB2, and IBM are trademarks of International Business Machines Corporation in the United States, other countries, or both. Other company, product or service names may be trademarks or service marks of others. ADDITIONAL INFORMATION ABOUT THE MERGER AND WHERE TO FIND IT In connection with the proposed merger and required stockholder approval, Ascential Software Corporation (the "Company") will file with the SEC a preliminary proxy statement and a definitive proxy statement that will contain important information about the Merger. Investors and security holders of the Company are urged to read the proxy statement, and any other relevant materials filed by the Company because they contain, or will contain, important information about the Company and the Merger. All documents filed by the Company with the SEC, may be obtained for free at the SEC's website at www.sec.gov. In addition, the documents filed with the SEC by the Company may be obtained free of charge by directing such request to: David Roy, Vice President of Investor Relations, (800) 966-9875 or from the Company's website at www.ascential.com. The Company and its executive officers and directors may be deemed to be participants in the solicitation of proxies from the stockholders of the Company in favor of the acquisition. Information about the executive officers and directors of the Company and their ownership of the Company's common stock is set forth in the proxy statement for the Company's 2004 Annual Meeting of Shareholders, which was filed with the SEC on April 27, 2004. Investors and security holders may obtain more detailed information regarding the direct and indirect interests of the Company and its respective executive officers and directors in the acquisition by reading the preliminary and final proxy statements regarding the acquisition, which will be filed with the SEC. Safe Harbor Statement under the Private Securities Litigation Reform Act of 1995: This release contains forward-looking statements based on current expectations or beliefs, as well as a number of assumptions about future events, and these statements are subject to important factors and uncertainties that could cause actual results to differ materially from those described in the forward-looking statements. The forward-looking statements in this release address a variety of subjects including, for example, the functionality, characteristics, quality and performance capabilities of Ascential Software's products and technology; results achievable and benefits attainable through deployment of Ascential Software's products and provision of services; the ability of Ascential Software's products to help companies streamline operations, improve customer support, expedite business transactions, and make better tactical and strategic decisions; and improvements in operational performance and decision-making. The following factors, among others, could cause actual results to differ materially from those described in these forward-looking statements: the ability of the company to expand its market share; growth rates for the enterprise data integration software market; general business conditions in the software industry, the technology sector, and in the domestic and international economies; rapid technological change in the markets served by the company; dependence on international operations; global and geopolitical instability; compliance costs associated with regulations to which the company is subject, including the Sarbanes-Oxley Act of 2002, and the impact of such regulations; and difficulties that the company may experience integrating technologies, operations and personnel of completed or future acquisitions. For a detailed discussion of these and other cautionary statements, please refer to the filings made by Ascential Software with the Securities and Exchange Commission, including, without limitation, the most recent Quarterly Report on Form 10-Q. Ascential Software disclaims any intent or obligation to update any forward-looking statements made herein to reflect any change in Ascential Software's expectations with regard thereto or any change in events, conditions, or circumstances on which such statements are based. Ascential Software is not responsible for statements attributed to third parties within this release or in any materials referenced herein. Partnerships described herein shall be deemed to be strategic alliances only, and shall not imply the power of either partner to bind the other partner or to incur obligations on behalf of the other partner.

1 0

Some things that need discussion
by Steve Hanson 11 Mar '05

11 Mar '05

Here is a list of things that are important to resolve for DFDL but which I don't recall seeing discussed or in a spec. 1) Rules for application of default values. XML Schema has rules for default value application when handling XML instance documents (they are different for elements and attributes). These rules could/should apply in some non-XML circumstances but are not applicable to others. I think we need to agree the rules that apply to different non-XML circumstances on both input and output. For example, - Fixed length data (eg, COBOL) - here the data must be present on input, default values could be added if missing on output - When a separator is present - this changes things as a double separator can occur - but does this indicate missing or empty? Customers use both semnatics, and Schema rules distinguish the two cases. - When an initiator is present - here missing is observably different from empty, so we could probably use Schema rules here. I have a draft proposal I am working on for this which I could share. 2) Properties - on object or on object inclusion. I didn't see anywhere in Mike's properties spec that said a property occurred on an element/attribute per se, or on its use in a structure. Eg, offset is clearly something that is only applicable to a local element or element reference, you would not put it on a global element. But some other properties are perhaps not so clear cut. 3) Mike's properties spec does not impose any restrictions on the use of different ways of identifying an element in the bitstream. Examples: - Optionality - are eg COBOL fixed length elements allowed to be optional - if so how can you identify one is missing. Here the IBM model mandates that maxOccurs must appear. - Unordered content - what should a DFDL parser do when faced with an xsd:all group - unless an initiator is present, should xsd:all be treated as an xsd: sequence when parsing? 4) Wildcards and 'self-defining' content - what are the rules that apply here? While this might seem unimportant if your starting point is a fixed file, in the messaging world this is frequently encountered - eg HL7 or X12 users will agree their own private extensions to the standard and add extra data, we must be able to model and parse/write this. 5) Truncation/omission rules when separators are involved - we have some extra options in the IBM model and several parsing rules, which we find necessary to cope with our user's CSV style messages. I should note at this point that the IBM model has the concept of 'separation type' which is a property of a group. It stipulates that all members of that group follow a certain pattern - examples are 'fixed length', 'separated', 'tagged and separated', 'use a regular expression'. We have found this a convenient way to define rules for default value, unordered content, wildcard and open content, etc, processing. These 'separation types' can be considered as specializations of a general case where the members of a group do not all follow a pattern. Clearly we need to define rules for the general case, but I think we should also consider whether such specializations are a useful addition to the DFDL model. I would observe that the majority of customers element groups to indeed follow one pattern or another (eg, COBOL - fixed length, CSV - separated). Apologies if any of these have been discussed already, eg at the f-2-f, or prior to GGF12, and I have missed it. If it is more convenient we could start up a series of discussion documents on the forum rather than the usual e-mail chain. I was certainly having trouble tracking the various mails about multi-dimensional arrays. Regards, Steve Steve Hanson WebSphere Business Integration Brokers, IBM Hursley, England Internet: smh(a)uk.ibm.com Phone (+44)/(0) 1962-815848

1 0

dfdl.info
by mike.beckerle＠ascentialsoftware.com 10 Mar '05

10 Mar '05

Ascential has parked the dfdl.info domain as well for our future use. ...mikeb

1 0