Issues with the Audit Guidelines Document GFD 169
Hi Yoshio, hi EUGridPMA list, hi CAOPS-WG, while working with the Audit Guidelines Document (GFD 169) I came across some surprising issues: The PDF offered from http://www.ogf.org/documents/GFD.169.pdf dated from 19.04.2010 differs from the latest .doc version available from https://forge.gridforum.org/sf/go/doc4858 which is called version 10 dated from 20.01.2010. Both documents self-claim that they are each version 1.0. Aside some minor differences like release dates, table of contents, etc the PDF is missing a numbering of an audit case. The section numbering in the PDF is different from the one in the word doc. But immediately after section heading "3.1.2. CA System" in the PDF the case number (7) for "The CA computer where the signing of the certificates..." is missing. Inserting the number (7) here will introduce an off-by-one error for current numbers (7) to (48) being (8) to (49) after the correction. Case (49) in the current(!) PDF is actually redundant to case (50)i. and needs to be deleted. The requirement quoted in case (49) is no longer included in the IGTF-AP-Classic v4.3 and v4.2 document. Instead it became part of case (50)i. which is to be found in section 6 of the IGTF-AP-Classic document. This latter bug is also found in the .doc(!) version from 19.01.2010 except that the case numbering here is different again. Case (50) is the redundant requirement to be deleted so that cases (51) to (56) are off-by-one which need to be renumbered to (50) to (55) once the redundant case is deleted. Be aware that the Auditing Template document (audit check-list) available from https://www.eugridpma.org/guidelines/classic does not match its audit case numbers to any of the above (PDF & .doc) GFD 169 document's case numbers. That indeed got me so confused that I started to look into these issues. How can we go about getting GFD 169 fixed? I did not see any bug reporting mechanism on the OGF site.... Thanks Reimer -- Dipl.-Inform. Reimer Karlsen-Masur (PKI Team), Phone +49 40 808077-615 DFN-CERT Services GmbH, https://www.dfn-cert.de, Phone +49 40 808077-580 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Sachsenstr. 5, 20097 Hamburg/Germany, CEO: Dr. Klaus-Peter Kossakowski
Hi Reimer,
Thanks for the information and sorry for the confusion.
Since I'm exhausted to write a project proposal, I'll check the
documents early next week and discuss in the CAOPs session on
Thursday.
Thanks again and best regards,
--
Yoshio Tanaka (yoshio.tanaka@aist.go.jp)
http://ninf.apgrid.org/
http://www.apgridpma.org/
From: "Reimer Karlsen-Masur, DFN-CERT"
Hi Yoshio, hi EUGridPMA list, hi CAOPS-WG,
while working with the Audit Guidelines Document (GFD 169) I came across some surprising issues:
The PDF offered from http://www.ogf.org/documents/GFD.169.pdf dated from 19.04.2010 differs from the latest .doc version available from https://forge.gridforum.org/sf/go/doc4858 which is called version 10 dated from 20.01.2010. Both documents self-claim that they are each version 1.0.
Aside some minor differences like release dates, table of contents, etc the PDF is missing a numbering of an audit case. The section numbering in the PDF is different from the one in the word doc. But immediately after section heading "3.1.2. CA System" in the PDF the case number (7) for "The CA computer where the signing of the certificates..." is missing. Inserting the number (7) here will introduce an off-by-one error for current numbers (7) to (48) being (8) to (49) after the correction.
Case (49) in the current(!) PDF is actually redundant to case (50)i. and needs to be deleted. The requirement quoted in case (49) is no longer included in the IGTF-AP-Classic v4.3 and v4.2 document. Instead it became part of case (50)i. which is to be found in section 6 of the IGTF-AP-Classic document.
This latter bug is also found in the .doc(!) version from 19.01.2010 except that the case numbering here is different again. Case (50) is the redundant requirement to be deleted so that cases (51) to (56) are off-by-one which need to be renumbered to (50) to (55) once the redundant case is deleted.
Be aware that the Auditing Template document (audit check-list) available from https://www.eugridpma.org/guidelines/classic does not match its audit case numbers to any of the above (PDF & .doc) GFD 169 document's case numbers.
That indeed got me so confused that I started to look into these issues.
How can we go about getting GFD 169 fixed? I did not see any bug reporting mechanism on the OGF site....
Thanks
Reimer -- Dipl.-Inform. Reimer Karlsen-Masur (PKI Team), Phone +49 40 808077-615
DFN-CERT Services GmbH, https://www.dfn-cert.de, Phone +49 40 808077-580 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Sachsenstr. 5, 20097 Hamburg/Germany, CEO: Dr. Klaus-Peter Kossakowski
Hi, Ar 21.10.10 13:45, scríobh Reimer Karlsen-Masur, DFN-CERT:
while working with the Audit Guidelines Document (GFD 169) I came across some surprising issues: [...] How can we go about getting GFD 169 fixed? I did not see any bug reporting mechanism on the OGF site....
I want to echo Reimer's comments. I recently used the GFD.169 (PDF) guidelines and the auditing spreadsheet and found discrepancies such as those mentioned by him. I haven't made time to write up the problems, but will endeavour to do so as part of the Grid-Ireland CA self-audit report (due soon...) In my mind, it would be some help if we could restructure our policies into the RFC 3647 template, where that makes sense. Kind regards, David -- Ánra Taighde - Scoil na hEolaíochta Ríomhaireachta ⁊ na Staitisticí, Coláiste na Tríonóide, Baile Átha Cliath 2 Research Fellow - School of Computer Science & Statistics, Trinity College, Dublin 2 Guthán / Telephone: +353 1 896 1720
Hi Reimer and all, I have checked the guidelines documents (GFD.169.pdf and .doc version 10 which is available on GridForge), and .doc files in my note PC. I understand that this inconsistency is definitely due to by my careless mistakes. 1. .doc version on Grid Forge (AuditGuidelines-Jan20_2010.doc) is not the latest version. This file does not include table of contents. I have the newer version (dated April 13) which includes the table of contents. But I did not upload this .doc file on the GridForge. This is my first mistake. 2. Unfortunately, when I inserted the table of contents, I made a mistake for numbering auditing items. I unintentionally deleted the number for item (7). But I converted the .doc file to PDF and submitted the PDF file for OGF Editor for publication as GFD.169. This is my second mistake and the reason of the problem which Reimer pointed out as below:
PDF is missing a numbering of an audit case. The section numbering in the PDF is different from the one in the word doc. But immediately after section heading "3.1.2. CA System" in the PDF the case number (7) for "The CA computer where the signing of the certificates..." is missing. Inserting the
I have fixed the two problems (missing item (7) and redundancy of
items (50) and (51).
The .doc file of the revised version 1.1 is uploaded on the GridForge.
PDF version is attached in this email. It would be appreciated if you
check the document so that we can confirm the document is ok before
asking OGF Editors to replace GFD.169.
Thanks,
--
Yoshio Tanaka (yoshio.tanaka@aist.go.jp)
http://ninf.apgrid.org/
http://www.apgridpma.org/
From: "Reimer Karlsen-Masur, DFN-CERT"
Hi Yoshio, hi EUGridPMA list, hi CAOPS-WG,
while working with the Audit Guidelines Document (GFD 169) I came across some surprising issues:
The PDF offered from http://www.ogf.org/documents/GFD.169.pdf dated from 19.04.2010 differs from the latest .doc version available from https://forge.gridforum.org/sf/go/doc4858 which is called version 10 dated from 20.01.2010. Both documents self-claim that they are each version 1.0.
Aside some minor differences like release dates, table of contents, etc the PDF is missing a numbering of an audit case. The section numbering in the PDF is different from the one in the word doc. But immediately after section heading "3.1.2. CA System" in the PDF the case number (7) for "The CA computer where the signing of the certificates..." is missing. Inserting the number (7) here will introduce an off-by-one error for current numbers (7) to (48) being (8) to (49) after the correction.
Case (49) in the current(!) PDF is actually redundant to case (50)i. and needs to be deleted. The requirement quoted in case (49) is no longer included in the IGTF-AP-Classic v4.3 and v4.2 document. Instead it became part of case (50)i. which is to be found in section 6 of the IGTF-AP-Classic document.
This latter bug is also found in the .doc(!) version from 19.01.2010 except that the case numbering here is different again. Case (50) is the redundant requirement to be deleted so that cases (51) to (56) are off-by-one which need to be renumbered to (50) to (55) once the redundant case is deleted.
Be aware that the Auditing Template document (audit check-list) available from https://www.eugridpma.org/guidelines/classic does not match its audit case numbers to any of the above (PDF & .doc) GFD 169 document's case numbers.
That indeed got me so confused that I started to look into these issues.
How can we go about getting GFD 169 fixed? I did not see any bug reporting mechanism on the OGF site....
Thanks
Reimer -- Dipl.-Inform. Reimer Karlsen-Masur (PKI Team), Phone +49 40 808077-615
DFN-CERT Services GmbH, https://www.dfn-cert.de, Phone +49 40 808077-580 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Sachsenstr. 5, 20097 Hamburg/Germany, CEO: Dr. Klaus-Peter Kossakowski
Hi Yoshio, Yoshio Tanaka wrote on 27.10.2010 14:09:
I have fixed the two problems (missing item (7) and redundancy of items (50) and (51). The .doc file of the revised version 1.1 is uploaded on the GridForge.
PDF version is attached in this email. It would be appreciated if you check the document so that we can confirm the document is ok before asking OGF Editors to replace GFD.169.
cool, many thanks, I will check the PDF later this week. Question to David O'Callaghan: Do you have any additional immediate obvious bug fix requests regarding GFD.169 that you wish to resolve now? Or are your issues more with the audit spreadsheet available from the eugridpma website? -- Beste Gruesse / Kind Regards Reimer Karlsen-Masur DFN-PKI FAQ: https://www.pki.dfn.de/faqpki -- 18. DFN Workshop "Sicherheit in vernetzten Systemen" am 15./16. Februar 2011 im Grand Hotel Elysee in Hamburg Call-for-Papers: https://www.dfn-cert.de/veranstaltungen/workshop.html -- Dipl.-Inform. Reimer Karlsen-Masur (PKI Team), Phone +49 40 808077-615 DFN-CERT Services GmbH, https://www.dfn-cert.de, Phone +49 40 808077-580 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Sachsenstr. 5, 20097 Hamburg/Germany, CEO: Dr. Klaus-Peter Kossakowski
Hi, Ar 27.10.10 13:20, scríobh Reimer Karlsen-Masur, DFN-CERT:
cool, many thanks, I will check the PDF later this week.
Question to David O'Callaghan: Do you have any additional immediate obvious bug fix requests regarding GFD.169 that you wish to resolve now? Or are your issues more with the audit spreadsheet available from the eugridpma website?
The only one that springs to mind is: Section 3.2.1 (5) An RA must validate the association of the certificate signing request. I don't understand the requirement (as someone familiar with PKI and as a native English speaker!), and the audit guidelines document does not explain, but just repeats it as a question "How does an RA validate the association of the certificate signing request?" I think the audit point should clarify the meaning of "the association": * Does it mean the association between subscriber's identity and the CSR? * Does it mean the association between the identity vetting performed by the RA and the CSR? * Does it mean the association between the private key and the public key in the CSR? * (or, less likely) Does it mean the subscriber's organization? This requirement comes from section 3.1 of the Classic AP v4.3, so perhaps my comment should be directed at that document. Beyond that, I would need to spend some time to look at the updated document and my notes from preparing for my EU Grid PMA Self Audit. Kind regards, David -- Ánra Taighde - Scoil na hEolaíochta Ríomhaireachta ⁊ na Staitisticí, Coláiste na Tríonóide, Baile Átha Cliath 2 Research Fellow - School of Computer Science & Statistics, Trinity College, Dublin 2 Guthán / Telephone: +353 1 896 1720
Hi, David O'Callaghan wrote on 27.10.2010 14:44:
Hi,
Ar 27.10.10 13:20, scríobh Reimer Karlsen-Masur, DFN-CERT:
cool, many thanks, I will check the PDF later this week.
Question to David O'Callaghan: Do you have any additional immediate obvious bug fix requests regarding GFD.169 that you wish to resolve now? Or are your issues more with the audit spreadsheet available from the eugridpma website?
The only one that springs to mind is:
Section 3.2.1 (5) An RA must validate the association of the certificate signing request.
I don't understand the requirement (as someone familiar with PKI and as a native English speaker!), and the audit guidelines document does not explain, but just repeats it as a question "How does an RA validate the association of the certificate signing request?"
I think the audit point should clarify the meaning of "the association":
* Does it mean the association between subscriber's identity and the CSR? * Does it mean the association between the identity vetting performed by the RA and the CSR? * Does it mean the association between the private key and the public key in the CSR? * (or, less likely) Does it mean the subscriber's organization?
This requirement comes from section 3.1 of the Classic AP v4.3, so perhaps my comment should be directed at that document.
since this is a quote from the IGTF-AP-Classic, I don't see this to be fixed in GFD.169 now. We should enhance the hint on how to check this requirement in a real life CA. I guess this needs to be addressed in the next edition of GFD.169, not in a "bug fix" release. And yes, if some clarification is needed on the semantics of this requirement, IGTF-AP-Classic should be enhanced in this respect as well. At Yoshio: Actually the RA section 3.2.1 in GFD.169 includes audit cases RA (5) and (6) as well as section 3.2.2 contains audit cases RA (5) and (6). The cases are different though, resulting in 12 RA audit cases all together when fixed. Other than that I see some general issues that CAOPS should consider with the next edition of GFD.169, not as a bug fix, see below.
Beyond that, I would need to spend some time to look at the updated document and my notes from preparing for my EU Grid PMA Self Audit.
Generally - and I am re-iterating on this idea - I find GFD.169 is too tightly bound to IGTF-AP-Classic v4.1. The check list is specific to IGTF-AP-Classic v4.1, even the general text (in section 2.6) is referencing IGTF-AP-Classic v4.1. And the current IGTF-AP-Classic stands at version 4.3. I suggest in the next edition of GFD.169 to split the check list out into a separate appendix or into a separate document. This way the audit guidelines are applicable to all IGTF-APs. Also since the IGTF/Grid-PMAs are requiring and promoting the self audits, the Grid-PMAs, ie. the AP owners/editors, not CAOPS, should think about releasing a matching audit check list with each new approved version of their owned APs. Also each audit case should reference the AP section it was taken from. That way the version mismatch between check list and actual AP and the type of AP and resulting confusions that we are observing now should be a thing of the past. Thanks Reimer -- 18. DFN Workshop "Sicherheit in vernetzten Systemen" am 15./16. Februar 2011 im Grand Hotel Elysee in Hamburg Call-for-Papers: https://www.dfn-cert.de/veranstaltungen/workshop.html -- Dipl.-Inform. Reimer Karlsen-Masur (PKI Team), Phone +49 40 808077-615 DFN-CERT Services GmbH, https://www.dfn-cert.de, Phone +49 40 808077-580 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Sachsenstr. 5, 20097 Hamburg/Germany, CEO: Dr. Klaus-Peter Kossakowski
Hi David and Reimer,
Thank you very much for valuable comments.
We will discuss this at the CAOPs in this afternoon.
Best Regards,
--
Yoshio Tanaka (yoshio.tanaka@aist.go.jp)
http://ninf.apgrid.org/
http://www.apgridpma.org/
From: "Reimer Karlsen-Masur, DFN-CERT"
Hi,
David O'Callaghan wrote on 27.10.2010 14:44:
Hi,
Ar 27.10.10 13:20, scríobh Reimer Karlsen-Masur, DFN-CERT:
cool, many thanks, I will check the PDF later this week.
Question to David O'Callaghan: Do you have any additional immediate obvious bug fix requests regarding GFD.169 that you wish to resolve now? Or are your issues more with the audit spreadsheet available from the eugridpma website?
The only one that springs to mind is:
Section 3.2.1 (5) An RA must validate the association of the certificate signing request.
I don't understand the requirement (as someone familiar with PKI and as a native English speaker!), and the audit guidelines document does not explain, but just repeats it as a question "How does an RA validate the association of the certificate signing request?"
I think the audit point should clarify the meaning of "the association":
* Does it mean the association between subscriber's identity and the CSR? * Does it mean the association between the identity vetting performed by the RA and the CSR? * Does it mean the association between the private key and the public key in the CSR? * (or, less likely) Does it mean the subscriber's organization?
This requirement comes from section 3.1 of the Classic AP v4.3, so perhaps my comment should be directed at that document.
since this is a quote from the IGTF-AP-Classic, I don't see this to be fixed in GFD.169 now. We should enhance the hint on how to check this requirement in a real life CA. I guess this needs to be addressed in the next edition of GFD.169, not in a "bug fix" release. And yes, if some clarification is needed on the semantics of this requirement, IGTF-AP-Classic should be enhanced in this respect as well.
At Yoshio: Actually the RA section 3.2.1 in GFD.169 includes audit cases RA (5) and (6) as well as section 3.2.2 contains audit cases RA (5) and (6). The cases are different though, resulting in 12 RA audit cases all together when fixed. Other than that I see some general issues that CAOPS should consider with the next edition of GFD.169, not as a bug fix, see below.
Beyond that, I would need to spend some time to look at the updated document and my notes from preparing for my EU Grid PMA Self Audit.
Generally - and I am re-iterating on this idea - I find GFD.169 is too tightly bound to IGTF-AP-Classic v4.1. The check list is specific to IGTF-AP-Classic v4.1, even the general text (in section 2.6) is referencing IGTF-AP-Classic v4.1. And the current IGTF-AP-Classic stands at version 4.3. I suggest in the next edition of GFD.169 to split the check list out into a separate appendix or into a separate document. This way the audit guidelines are applicable to all IGTF-APs.
Also since the IGTF/Grid-PMAs are requiring and promoting the self audits, the Grid-PMAs, ie. the AP owners/editors, not CAOPS, should think about releasing a matching audit check list with each new approved version of their owned APs. Also each audit case should reference the AP section it was taken from. That way the version mismatch between check list and actual AP and the type of AP and resulting confusions that we are observing now should be a thing of the past.
Thanks
Reimer -- 18. DFN Workshop "Sicherheit in vernetzten Systemen" am 15./16. Februar 2011 im Grand Hotel Elysee in Hamburg
Call-for-Papers: https://www.dfn-cert.de/veranstaltungen/workshop.html -- Dipl.-Inform. Reimer Karlsen-Masur (PKI Team), Phone +49 40 808077-615
DFN-CERT Services GmbH, https://www.dfn-cert.de, Phone +49 40 808077-580 Sitz / Register: Hamburg, AG Hamburg, HRB 88805, Ust-IdNr.: DE 232129737 Sachsenstr. 5, 20097 Hamburg/Germany, CEO: Dr. Klaus-Peter Kossakowski
participants (3)
-
David O'Callaghan -
Reimer Karlsen-Masur, DFN-CERT -
Yoshio Tanaka